Senior Infrastructure Engineer – Managed Inference

We are seeking a Staff-level or higher Infrastructure Engineer with deep, hands-on experience operating production LLM inference infrastructure. This role focuses on scaling highly available Kubernetes environments, optimizing inference serving systems, and ensuring reliable performance across GPU and accelerator platforms.

What You Will Do

Design, operate, and scale highly available Kubernetes clusters across AWS, GCP, and specialized GPU cloud providers to support inference workloads exceeding 1,000 tokens per second. Troubleshoot production infrastructure issues directly, working with technical teams and customers to identify and resolve complex challenges.

Why It Might Be a Fit

The ideal candidate is a highly technical individual contributor who takes direct ownership of production infrastructure, troubleshoots complex systems, and works closely with engineering teams and customers to solve challenging infrastructure problems.

Requirements

• 5+ years of infrastructure engineering experience, currently operating at Staff Engineer level or above in a hands-on individual contributor role.
• Direct, day-to-day experience with production LLM inference infrastructure, including building, deploying, operating, or troubleshooting inference serving systems.
• Extensive experience operating production Kubernetes at scale across multiple cloud providers, particularly AWS and GCP.
• Hands-on expertise debugging NVIDIA GPU and accelerator systems, including drivers, CUDA, NCCL, node health, and network-related issues.
• Practical experience with at least one inference serving framework, such as vLLM, SGLang, TensorRT-LLM, or NVIDIA Dynamo.
• Experience with high-performance networking technologies such as InfiniBand, RDMA/EFA, or MPI.
• Strong background in observability and reliability engineering for latency-sensitive production services, including metrics, alerting, and automated failover.
• Ability to independently diagnose and resolve complex production infrastructure issues.
• Strong communication and problem-solving skills, including the ability to collaborate directly with customers and technical stakeholders.

Benefits

• Base Salary: 350,000–450,000
• Competitive equity package
• Open to eligible visa transfers, including H-1B and OPT