AI QA Engineer (L3)

Role Title: AI QA Engineer (L3)

Reporting to: Engineering Manager (QA & Delivery Chapter)

Salary: £55,000-£65,000

Location: While our offices are in Bristol and London, we welcome remote applicants based in the UK. Attendance at our Bristol office will be required once a month or as needed.

Introduction to Bunch

At Bunch, we believe managing your home bills should take minutes — not months of chasing. So we’ve built the smartest way to handle shared household costs, starting with a single fixed monthly payment that covers everything from energy to broadband to water. No more awkward money chats. No more missed due dates. No more stress.

We exist to give people back their time, their headspace, and a bit of sanity. Because in a world that’s always on, admin shouldn’t be another full-time job.

We’re a fast-growing team with a rebellious streak. We challenge the way things have always been done and ask: how can this be smarter, easier, better for the customer?

We’re looking for people who are bold, human, and obsessed with making things simpler. People who thrive in fast-paced environments, love solving real-world problems, and care deeply about the impact their work has.

Sound like you? Join the Bunch crew.

Position Summary

We’re looking for a capable and forward-thinking AI QA Engineer (L3) to own automation quality strategy across both traditional and AI-powered features at Bunch. This role combines deep automation engineering skills with growing specialism in testing intelligent workflows, prompt-based systems, and LLM-driven functionality.

As an L3 Owner, you will lead on coverage strategy, drive regression health, design frameworks, and contribute to the evolution of how we validate outputs from probabilistic systems. You'll help shape how we evaluate accuracy, safety, and usefulness in AI-powered features, while also raising the bar for stability and efficiency across the whole test suite.

Role Accountabilities

Own and maintain high-confidence automation coverage across multiple products and flows

Design and evolve reusable automation patterns for UI and API-level test cases

Review testability of upcoming features and raise architectural gaps early

Collaborate on LLM evaluation plans including hallucination tolerance, factuality, relevance, and edge case capture

Build test harnesses or tooling to validate prompt consistency and AI-agent flows

Drive regression stability efforts and reduce flaky or redundant tests

Own documentation of quality principles, test approach, and platform guardrails

Mentor L2 contributors or support engineers on writing maintainable tests

Contribute to shared QA frameworks, CI/CD workflows, and observability tools for test outcomes

Bunch Values

General Key Behaviours of Bunch

DELIVERY: We get stuck in, overcome setbacks, and hold ourselves accountable

GOOD VIBES: We aim to be a positive and friendly place, where people want to be.

ONE TEAM: We work collaboratively across the organisation in the interests of our customers and wider outcomes.

GROWTH: We strive to grow ourselves, and the Company.

Specific Key Behaviours for this Role

DELIVERY: Acts as the quality owner for the systems they touch

GOOD VIBES: Collaborates across frontend, backend and product to advocate for robust testability

ONE TEAM: Applies critical thinking to both functional and AI-specific validation

Knowledge, Skills and Experience

Essential

Education: Degree in Computer Science, Software Engineering, or a related discipline; or equivalent hands-on experience in a QA or automation-focused role.

Experience: 4+ years in QA, test automation, or related engineering roles, with experience owning and delivering high-confidence automated coverage in CI/CD environments.

Key Skills: Advanced test automation skills in modern frameworks (e.g. Cypress, Playwright, TestCafe). Strong knowledge of REST APIs, async test patterns, mocking/stubbing strategies. Proven debugging and test flake isolation skills. Comfortable integrating automated tests into GitHub Actions or other CI pipelines. Clear communicator and strong collaborator across product and engineering

Desirable

Education: Certifications in Test Automation, ISTQB Advanced, or AI/ML Fundamentals

Experience: Built or contributed to LLM eval tooling or test harnesses for AI workflows; evaluated AI-driven product features using both scripted and exploratory techniques

Key Skills: Experience writing custom LLM eval scripts or prompt scenario libraries. Familiarity with OpenAI, Anthropic or LangChain agents. Comfortable scripting in JS/TS or Python to support test tooling. Strong grasp of observability tools, test coverage reports, and quality metrics

What we offer: 

While our offices are in Bristol and London, we welcome remote applicants based in the UK. Attendance at our Bristol office will be required once a month or as needed.

A salary of £55,000-£60,000.

Enjoy 25 days of holiday.

You get an extra day off for your birthday!

Dedicated learning & development budget.

Supported onboarding and training to fully engage you with your new team.

Celebrate our collective success with quarterly social events.

Access the Vitality Health package to prioritise your well-being.

Enjoy exclusive Vitality rewards, including free weekly coffees, discounted gym memberships, and a wide range of other great discounts!

Enhanced Maternity, Adoption, and Paternity Pay - we understand the importance of family, and we're committed to supporting you.

Enhanced sick pay with two weeks paid leave.

5 discretionary compassionate leave days (which would also cover e.g. a parent having to take a day off to care for a sick child).

Cycle to Work Scheme - enjoy the perks of a healthier commute and save on your journey.

3 hours ‘breakout block’ available to take each month to catch up on life's admin.

A dog friendly office.

A great culture.