AI Evaluation Specialist – Remote Contractor

About the role

We are looking for an AI Evaluation Specialist to help train next‑generation AI systems. Working remotely as a contractor, you will provide high‑quality, real‑world input that shapes how models learn, reason, and perform.

Key responsibilities

• Review and critically assess AI‑generated outputs for quality, clarity, usability, and overall user experience.
• Identify inconsistencies, weaknesses, and improvement opportunities across diverse content types and visual experiences.
• Apply structured evaluation guidelines and deliver nuanced feedback to inform model improvements.
• Collaborate with cross‑functional teams to define and refine quality standards, scoring criteria, and review processes.
• Maintain high accuracy and consistency across evaluations, ensuring reliable and actionable results.
• Contribute to the continuous enhancement of evaluation workflows and best practices.
• Support the development of training data that enhances AI system performance and reliability.

Required profile

• Exceptional reading comprehension and acute attention to detail.
• Strong observational skills with excellent aesthetic or editorial judgment.
• Ability to think critically and deliver consistent, high‑quality evaluations in ambiguous scenarios.
• Excellent written and verbal communication skills.
• Comfort working across various content types and evaluation guidelines.
• Strong sense of ownership, reliability, and commitment to delivering high‑quality work.
• Open‑minded approach to learning and quickly adapting to new workflows.

Required skills

• Background in design, UX/UI, creative direction, editing, or content strategy (preferred).
• Experience reviewing creative, visual, or AI‑generated content (preferred).
• Familiarity with annotation, QA, moderation, or evaluation workflows, and working with AI tools or LLMs (preferred).