Senior LLMOps Engineer

Who We Are

Healthcare needs a better rhythm: one that keeps care continuous and deeply human. Heidi is building an AI Care Partner that works alongside clinicians to make that possible.

We’re a team of doctors, engineers, designers, researchers, and creatives building tools that help clinicians stay focused on what matters most: their patients.

In just 18 months, Heidi has given back more than 18 million hours to healthcare professionals — supporting 73 million patient visits in 116 countries. Today, more than two million patient visits each week are powered by Heidi worldwide.

Backed by nearly $100 million in funding, we’re growing in the US, UK, Canada, and Europe, partnering with leading health systems including the NHS, Beth Israel Lahey Health, and Monash Health.

The Role

Working closely with our Engineering Manager, you’ll be a Senior LLMOps Engineer on the Model Platform team. You are a technical leader responsible for building and scaling the infrastructure that powers our entire model lifecycle.

Your mission is to build a robust, scalable, and reliable platform for deploying and managing our LLMs. You will lead the design and implementation of our LLMOps strategy, ensuring our AI engineers can move models from development to production seamlessly and efficiently.

You will combine your deep infrastructure knowledge with MLOps principles to solve the critical challenges of serving models at scale.

What you’ll do:

• Lead LLMOps Platform Development: Lead the architecture, design, and implementation of our end-to-end LLMOps platform, from data ingestion and model training pipelines to production deployment and monitoring.

• Automate the LLM Lifecycle: Build and maintain robust CI/CD/CT (Continuous Integration/Continuous Delivery/Continuous Training) pipelines to automate the testing, validation, and deployment of large language models.

• Ensure Scalable and Reliable Deployment: Engineer highly available and scalable model serving solutions using modern infrastructure like Kubernetes, ensuring low latency and high throughput for our production services.

• Partner with AI and Engineering Teams: Collaborate closely with AI research and engineering teams to understand their needs, streamline workflows, and create the tooling that accelerates their development cycles.

• Establish MLOps Best Practices: Champion and implement best practices for model versioning, experiment tracking, monitoring, and governance across the organization.

• Mentor and Guide: Mentor mid-level and junior engineers, sharing your deep expertise in infrastructure, automation, and operational excellence to foster a culture of reliability and scalability.

What we will look for:

• You’ve a proven track record of designing, building, and maintaining MLOps or LLMOps infrastructure in a production environment.

• You’ve previous hands-on experience building scalable, cloud-native infrastructure and platforms.

• You’ve deployed and managed large-scale machine learning models in a production environment, with a deep understanding of the associated challenges.

• You are considered an expert in Python, cloud platforms (AWS, GCP, or Azure), containerization (Docker, Kubernetes), and Infrastructure as Code (e.g., Terraform, CloudFormation).

• You have a deep and practical understanding of the entire machine learning lifecycle and the specific operational challenges of large language models.

• You have the ability to translate complex engineering and research requirements into concrete, robust, and automated platform solutions.

• A Bachelor's or Master's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Bonus:

• Experience with advanced model serving and optimization techniques (e.g., quantization, distillation, multi-model serving).

• Experience with specialized MLOps frameworks like MLflow, Kubeflow, or Weights & Biases.

• Contributions to open-source MLOps or infrastructure-related projects.

The way we work

1. Build to Last

We design for safety and reliability so clinicians, patients, and our teams can trust what we build every day.

2. Own Your Practice

Ideas rise on merit, not title, and everyone shares responsibility for the standards we set together.

3. Move Fast, Stay Steady

We move quickly but never at the cost of trust. Progress only matters if people can depend on what we make.

4. Make Others Better

Honest feedback, steady support, and shared growth keep our teams improving together.

Why you will flourish with us

• Flexible hybrid working environment, with 3 days in the office.

• A generous personal development budget of $500 per annum

• Learn from some of the best engineers and creatives, joining a diverse team

• Become an owner, with shares (equity) in the company, if Heidi wins, we all win

• The rare chance to create a global impact as you immerse yourself in one of Australia’s leading healthtech startups

• If you have an impact quickly, the opportunity to fast track your startup career!

Heidi is dedicated to creating an equitable, inclusive, and supportive work environment that brings people together from diverse backgrounds, experiences, and perspectives. Our strength is in our differences.

We're proud to be an equal opportunity employer and welcome all applicants as we're committed to promoting a culture of opportunity for all.