Senior AI Quality Assurance Engineer
Bottomlinetechnologies
Posted: February 17, 2026
Interested in this position?
Create a free account to apply with AI-powered matching
Quick Summary
A Senior AI Quality Assurance Engineer is responsible for ensuring the accuracy and reliability of AI models, with a focus on delivering impactful results for customers.
Required Skills
Job Description
Why Choose Bottomline?
Are you ready to transform the way businesses pay and get paid? Bottomline is a global leader in business payments and cash management, with over 35 years of experience and moving more than $16 trillion in payments annually. We're looking for passionate individuals to join our team and help drive impactful results for our customers. If you're dedicated to delighting customers and promoting growth and innovation - we want you on our team!
Senior AI Quality Assurance Engineer
Role Overview
An AI Quality Assurance Engineer ensures the accuracy, reliability, and compliance of AI systems and models before deployment. This role is critical for validating AI-driven applications, detecting defects, and safeguarding against risks such as bias, hallucinations, and non-deterministic behavior. The position involves designing robust test strategies for machine learning models and collaborating with data scientists, ML engineers, and product teams to maintain high-quality standards.
Key Responsibilities
· Testing & Validation
o Develop and execute comprehensive test plans for AI models and systems.
o Perform manual and automated tests to validate AI functionality under various scenarios.
o Conduct bias detection, edge-case testing, and stress testing for AI models.
· Automation & Tools
o Design and implement automated testing frameworks for AI workflows.
o Utilize AI-powered QA tools and self-healing test scripts to improve efficiency.
· Compliance & Governance
o Ensure AI systems adhere to ethical guidelines, regulatory standards, and internal policies.
o Maintain documentation such as Generative AI Bill of Materials (BoM) for transparency and audits.
· Collaboration
o Work closely with ML engineers, data scientists, and product managers to resolve defects.
o Participate in code reviews and contribute to continuous improvement initiatives.
· Monitoring & Reporting
o Analyze test results, identify defects, and track quality metrics.
o Provide risk assessments and recommendations for AI deployments.
Required Skills & Qualifications
· Education: Bachelor’s in Computer Science, Software Engineering, or related field.
· Experience:
o 3–7 years in QA, with at least 2 years in AI/ML testing.
o Familiarity with non-deterministic testing and AI-specific QA challenges.
· Technical Expertise:
o Proficiency in Python and testing frameworks (e.g., PyTest, Selenium).
o Knowledge of ML frameworks (TensorFlow, PyTorch) and MLOps tools.
o Experience with CI/CD pipelines and cloud platforms (AWS, Azure, GCP).
· Soft Skills:
o Strong analytical and problem-solving abilities.
o Excellent communication and collaboration skills.
Preferred Skills
· Understanding of ethical AI principles and bias testing.
· Familiarity with low-code/no-code automation tools and AI-driven QA platforms.
· Knowledge of regulatory compliance for AI systems.
#LI-DNI
#LifeAtBottomline
We welcome talent at all career stages and are dedicated to understanding and supporting additional needs. We're proud to be an equal opportunity employer, committed to creating an inclusive and open environment for everyone.