AI prompt data creation
Create original prompts and reference responses for a defined evaluation goal.
Atlas category
AI Training & Human Evaluation
Human work used to create, evaluate or improve AI systems.
How value is exchanged
- What is supplied
- Judgment, labeled data, testing or domain expertise.
- Why a buyer pays
- Model developers need human evaluation that automation cannot reliably supply.
- Payment models
- Hourly project, Per task, Per accepted unit
- Prerequisites
- Instruction accuracy; Possible domain credentials; Qualification
Costs, risks and progression
Startup-cost considerations: Usually computer and internet access; qualification time may be unpaid.
- Project volatility
- Confidentiality
- Rejection
- Disturbing safety material
Potential progression: Develop specialist expertise or evaluation operations.
Canonical earning methods
AI response evaluation
Compare and score model outputs against a rubric.
AI safety red teaming
Test model behavior against defined safety scenarios and document failures.
Coding model evaluation
Review generated code for correctness, security and instruction compliance.
Data labeling
Annotate or classify data using project rules.
Domain-expert AI review
Review AI work using demonstrated professional expertise.
Multilingual AI evaluation
Assess model outputs for meaning, fluency and cultural fit in a known language.
Associated sourced platforms
Clickworker
Register, activate the account, complete profile and qualification steps, and select available tasks in the workplace or app.
CrowdGen by Appen
Create a CrowdGen account, complete a profile, apply to available projects, and complete project-specific qualification and onboarding steps.
OneForma
Create a profile with MFA, browse projects, complete application forms, and take project certifications where required.
OpenTrain
A network and job-discovery platform for AI trainers and data labelers.
Outlier
Apply for an expert area, verify identity and credentials, complete onboarding and assessments, and receive eligible project work if available.
TaskVerse
A task platform whose public examples include annotation, speech recording, tagging and transcription.
TELUS Digital AI Community
Select a current AI Community opening, submit the requested application information, and complete any role-specific assessment, identity verification and onboarding.