Senior Inference Optimization ML Engineer

The Senior Inference Optimization ML Engineer will own cost-efficient inference end-to-end, making Rhoda's foundation models faster and cheaper to run on a wide range of hardware. They will work closely with the research, robotics, and infrastructure teams to achieve this goal.

What You Will Do

The engineer will own cost-efficient inference end-to-end, make Rhoda's foundation models faster and cheaper to run, measured in latency, throughput, and cost per inference. They will port and optimize models for non-NVIDIA and lower-cost hardware, own on-robot inference, evaluate hardware rigorously, build hardware-portable optimizations, and apply model compression for constrained hardware.

Why It Might Be a Fit

The ideal candidate will have 2+ years of experience in inference optimization, ML systems, ML compilers, or a closely related field, deep hands-on experience with PyTorch, and a strong understanding of compute, memory bandwidth, memory capacity, and interconnect bottlenecks.

Requirements

• 2+ years of experience in inference optimization, ML systems, ML compilers, or a closely related field
• Deep hands-on experience with PyTorch
• Strong understanding of compute, memory bandwidth, memory capacity, and interconnect bottlenecks
• Experience with model optimization techniques: quantization (INT8/FP8/FP4, AWQ), distillation, pruning, and compilation (torch.compile, XLA, TensorRT, IREE/MLIR, TVM, or vendor compilers)
• Familiarity with inference serving frameworks and their support for different hardware (e.g. vLLM on ROCm/TPU, SGLang, JetStream, Triton Inference Server)
• Excellent debugging and measurement skills
• A high-ownership, pragmatic mindset