Senior Machine Learning Engineer, Quantized Inference

We are now looking for a Senior Machine Learning Engineer for Quantized Inference! NVIDIA is seeking machine learning engineers to accelerate the discovery and deployment of efficient inference recipes for LLMs. A recipe defines which operators are transformed into low-precision or sparsified variants unlocking throughput and latency gains without regressing accuracy nor verbosity. Recipes may incorporate techniques such as rotations, block scaling to attenuate outlier impact, or improved calibration data drawn from SFT/RL pipelines.

Pushing the frontier of inference efficiency requires a holistic view of the workload. The candidate will navigate the full design space: identifying which layers are sensitive to quantization relative to their inference cost, diagnosing why specific recipes fail, and adapting training techniques such as quantization-aware distillation or targeted fine-tuning to recover accuracy where needed. Our team develops quantized and sparse recipes that ship and run at scale across NVIDIA's LLM product portfolio. Our recipes directly determine the cost and latency of serving models to millions of users. We collaborate with inference framework teams (vLLM, TRT-LLM) to ensure recipes translate into real throughput gains, and with post-training teams to source calibration data and co-design quantization-aware training curricula.

What you'll be doing:

• Prototype state-of-the-art quantization and sparsity recipes applied to LLM workloads

• Design and execute post-training quantization or quantization-aware distillation experiments: prepare SFT/RL calibration datasets, manage checkpoint-level eval sweeps, and iterate on recipes based on results

• Run accuracy and verbosity evaluations of quantized/sparsified LLM workloads at cluster scale

• Develop data analysis tooling and visualizations for numerics debugging

• Participate in code reviews and incorporate feedback

• Contribute improvements upstream to open-source inference and optimization libraries; publish findings at ML conferences where appropriate

What we need to see:

• Proficient in Python and PyTorch

• Experience with quantization, sparsity, or other model compression techniques

• Ability to design and run rigorous experiments: controlled ablations, statistical significance, reproducibility

• Familiarity with LLM evaluation methodology (benchmarks, human-preference proxies, verbosity metrics)

• MS/PhD in Computer Science, Computer Engineering, Machine Learning, or equivalent experience.

• 3+ years of experience in an applied ML role

• Demonstrated ability to move fast with ambiguous requirements, with strong written and verbal communication

Ways to stand out from the crowd:

• Published work or production experience in post-training quantization or quantization-aware training

• Experience with SFT, RLHF/DPO, or distillation pipelines

• Familiarity with inference serving frameworks (vLLM, TRT-LLM, SGLang)

• Track record of debugging numerical issues in mixed-precision training or inference

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until March 1, 2026.

This posting is for an existing vacancy. 

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.