Gen AI Tester

Gen AI Tester

Job Title: Generative AI Tester
Experience: 3+ years / 6+ years

Job Description:
We are seeking a detail-oriented Generative AI Tester to validate the quality, reliability, and ethical outputs of AI systems powered by large language models (LLMs). This role focuses on testing prompt responses, evaluating edge cases, and ensuring AI outputs align with business and compliance standards.

Key Responsibilities:

• Design and execute test cases for Gen AI features and conversational workflows.
• Validate outputs of LLMs for accuracy, relevance, coherence, and safety.
• Conduct prompt testing, fine-tuning evaluation, and edge-case scenario validation.
• Collaborate with AI engineers, product managers, and QA teams to improve test coverage and feedback loops.
• Document defects, issues, and improvement areas using tools like JIRA or similar.
• Evaluate AI systems against ethical and fairness benchmarks.

Requirements:

• 2+ years of experience in QA/testing, with exposure to AI/ML or NLP systems.
• Familiarity with LLMs (e.g., GPT, Claude, or open-source alternatives).
• Strong analytical and critical thinking skills for non-deterministic system testing.
• Experience with test management tools and basic scripting (Python preferred).
• Good understanding of prompt engineering and model evaluation principles.

Preferred Qualifications:

• Experience in testing conversational AI, chatbots, or virtual assistants.
• Understanding of responsible AI practices, including bias and hallucination detection.
• Exposure to RAG (Retrieval-Augmented Generation) pipelines and vector databases.