Pictor | Arabic (Gulf) AI Evaluation Specialist

Overview
We are looking for Arabic (Gulf) AI Evaluation Specialists to support the testing and evaluation of an Arabic language model. In this role, you will be instrumental in refining and evaluating large language models (LLMs). You'll design prompts, evaluate the responses based on the functionality, accuracy, and safety of cutting-edge AI systems, and generate the best possible answer for the target audience. Your expertise will help us build smarter, more reliable, and more helpful technology. 🤖

Project Details
Location: Remote-Egypt
Language: Native level fluency in Gulf Arabic
Project Duration: 3 months
Pay Rate: $10 USD/Hour
Schedule: 40 hours a week. 8 hours per day Mon-Fri
Start Date: February 2nd

Key Responsibilities
-Design scenario-based and edge-case prompts to test AI behavior, including trick and incomplete-information cases.
- Develop evaluation rubrics to assess AI responses across instruction-following, factuality, tone, safety, refusals, and helpfulness.
- Perform side-by-side evaluations of AI outputs and score them on a 1–5 scale using defined criteria.
- Create high-quality source documents (articles, transcripts, reports) as the single source of truth for testing.
- Write accurate and well-structured Golden Responses that correctly follow instructions and handle ambiguity.

Qualifications
- Bachelor's degree or equivalent experience in Linguistics, Computational Linguistics, Communications, Technical Writing, or a related analytical field.
- B2 or superior level of English.
- Native fluency in Modern Standard Arabic in Gulf dialect.
-Strong understanding of the distinction between Fusha and ‘Ammiyya
- Proven experience in a role involving AI data annotation, content quality review, search quality rating, or prompt engineering.
- Ability to work independently and manage workflows effectively in a remote environment.

Nice to Have
- Multilingual proficiency in one or more Arabic dialects.
- Strong attention to detail and critical thinking to identify hallucinations and bias
- Familiarity with data annotation platforms and model evaluation tools.
- Experience in prompt engineering, AI evaluation, linguistic QA, or translation is a plus
- Cultural familiarity with regional norms and high-context communication styles, particularly in the GCC region.

Note: Please do not use VPNs or IP-masking tools during the recruitment process — our security system requires accurate regional verification.

Why Join Welo Data?
✨ Limitless Flexibility 
Project-based opportunities that fit your availability. Choose when and how much you want to contribute—fully remote, with complete autonomy. 
🌱 Limitless Growth 
Optional access to AI and Large Language Model workshops designed specifically for professionals like you. No coding required—just your expertise. 
🌍 Limitless Support 
Be part of a global contributor community with responsive guidance and support. 
💡 Real Impact 
Apply your expertise in the Legal field to influence the AI systems shaping the future of your industry—while collaborating with data professionals and expanding your skills.

How to Apply?
Apply now by answering a few quick questions to join our database and become part of our growing community.

About Welo Data
Welo Data, part of Welocalize, is a global AI data company with 500,000+ contributors delivering high-quality, ethical data to train the world’s most advanced AI systems. We’re building smarter, more human AI with a diverse community in 100+ countries.
At Welo Data, Limitless AI. Limitless You. isn’t just a slogan—it’s our promise. We build smarter AI through the power of human contribution, offering limitless opportunities for our global community to grow, contribute, and work on their terms.