Head of Quality, AI Business

Head of Quality, AI Business

Team: Client Services

Prolific

Prolific isn’t just enabling AI innovation – we’re redefining it. While foundational AI technologies are becoming commoditized, Prolific’s human data infrastructure provides the high-quality, diverse data required to train the next generation of AI models. Through our platform, we empower researchers and companies to access a global, ethically curated participant base, ensuring cutting-edge AI research and training grounded in inclusivity and precision.

The Role

As Head of Quality, AI Data, you will own the systems that turn messy human and model signals into trustworthy, scalable AI data. Our customers’ models are only as good as the data and evaluations we deliver—and as we operate across regions, languages, and time zones, quality cannot be an afterthought or a local optimisation.
This role sets the global quality bar. You will define standards, enforce rigor, and build quality systems that scale across distributed teams without slowing delivery. You’ll act as the senior quality leader internally and externally, shaping how customers understand, trust, and use Prolific’s data.

What You’ll Be Doing

• Set the global quality strategy across annotation, human feedback, model evaluation, red-teaming, and multilingual data.

• Define and enforce quality standards, acceptance criteria, and playbooks across project and pipeline types.

• Own the quality measurement system, including sampling strategies, agreement metrics, defect taxonomies, and drift detection.

• Lead a globally distributed quality organisation and design operating rhythms that work asynchronously across regions.

• Ensure quality is embedded from project kickoff through delivery, including audits, remediation loops, and root-cause analysis.

• Act as the senior quality partner for customers, clearly explaining methodology, results, risks, and improvement plans.

• Partner with Product, Engineering, and Operations to embed quality into tooling, workflows, and automation.

• Hire, develop, and lead quality managers and senior individual contributors, building a culture of rigor and accountability.

What You’ll Bring to the Role

• 8–12+ years of experience in quality, data operations, trust and safety, or ML evaluation, with 3+ years leading managers or senior leads.

• Proven experience running quality programs across globally distributed teams and multiple regions.

• Strong statistical intuition (sampling, agreement, bias/variance) and comfort working with quality data.

• Fluency with SQL and dashboards; working knowledge of Python or R for analysis or automation.

• Deep understanding of human-in-the-loop systems and how data quality impacts model behaviour.

• Strong judgment balancing speed, cost, and quality, and the ability to make trade-offs explicit.

• Clear, confident communication with both engineers and executives.

Even Better if you have:

• Experience with multilingual or localisation-style QA.

• Background designing taxonomies or evaluation frameworks tied to model performance.

• Exposure to regulated or high-scrutiny environments where auditability matters.

Why Prolific is a great place to work

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breath and the best of humanity.

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research.

Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture. At Prolific, our compensation packages for eligible roles include base salary, equity, and benefits. Many roles also include the opportunity to earn a cash variable element, such as a bonus or commission. Each job posting shows a salary range that reflects the minimum and maximum target for new hires, based on the role’s location as well as your skills, experience, and relevant education or training. Your recruiter will also be happy to share the specific salary range for your preferred location during the hiring process.

Links to More Information

Website

Youtube

Privacy Statement

By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.