Generalist Evaluator Expert

This role is for one of our clients

Compensation: $35-$40 per hour

We are seeking detail-oriented writing professionals to contribute to a high-impact AI research initiative in collaboration with a leading research lab. In this role, you will develop high-quality prompt–golden answer pairs used to train and evaluate advanced language models.

This is a short-term, flexible opportunity ideal for individuals with strong academic foundations and exceptional clarity in written communication. The role is well-suited for professionals who enjoy translating complex ideas into structured, precise, and easy-to-understand content.

Requirements:
Key Responsibilities

• Design and Optimize Prompts: Develop detailed, constraint-rich prompts with clear instructions and multiple requirements
• Define Evaluation Standards: Establish expectations for high-quality responses in general consumer contexts and create comprehensive grading rubrics
• Model Testing and Assessment: Execute prompts using AI systems and evaluate outputs against defined standards
• Benchmarking & Quality Assurance: Collaborate in QA processes to ensure prompt tasks and rubrics meet high standards of rigor, clarity, and consistency before inclusion in benchmarking workflows
• Maintain structured documentation and adhere to project guidelines

Minimum Qualifications

• Bachelor’s degree (BS or BA) from a reputable institution (completed or in progress)
• Strong writing, analytical, and critical thinking skills
• Ability to work independently and meet structured deadlines
• Meaningful familiarity with ChatGPT or similar AI tools for personal, academic, or professional use
• Must be based in the United States or Canada

Preferred Qualifications

• Experience in teaching, curriculum design, academic research, or structured evaluation
• Experience developing grading rubrics or assessment frameworks

Project Details

• Start: Immediate
• Duration: Approximately 2 months
• Commitment: Minimum 20 hours per week
• Fully remote with flexible scheduling
• Structured project environment with defined goals, workflows, and tools

Application & Onboarding Process

• Complete a short AI-led interview (approximately 15 minutes)
• Complete a 45-minute written assessment focused on rubric development
• Selected candidates will receive project onboarding instructions

Contract & Payment Terms

• Engagement will be structured as an independent contractor agreement
• Work can be completed remotely on your own schedule
• Projects may be extended, shortened, or concluded early based on performance and evolving project needs
• Assignments will not require access to confidential or proprietary information from any employer, client, or institution
• Payments are processed weekly via Stripe or Wise based on services rendered
• Visa sponsorship is not available; H1-B and STEM OPT candidates cannot be supported at this time