Role Overview The Generalist Reviewer is responsible for evaluating task submissions used in research and large language model (LLM) training. This role focuses on quality assurance, guideline fidelity, and structured feedback, ensuring that submitted work meets defined standards for accuracy, clarity, safety, and compliance. Generalist Reviewers apply consistent judgment across a wide range of task types and domains, relying on documented rubrics rather than personal preference. This role is ideal for professionals with experience in quality review, auditing, evaluation, compliance, instructional assessment, or editorial QA.
Key Responsibilities
- Review task submissions against detailed written guidelines, rubrics, and policy requirements
Assess work for: - Accuracy and completeness - Adherence to task instructions - Compliance with safety, privacy, and content standards - - Internal consistency and logical reasoning
- Identify errors, omissions, and guideline deviations with precision
- Provide clear, actionable, and professional written feedback to contributors
- Apply evaluation standards consistently across high volumes of work
- Flag potential compliance, safety, or integrity issues promptly
- Document review decisions in required systems with appropriate rationale
- Maintain confidentiality and data handling standards at all times
Required Qualifications
- Bachelor’s degree or equivalent professional experience
3+ years of experience in one or more of the following: - Quality assurance or quality control - Content review or editorial evaluation - Compliance, audit, or policy enforcement - - Instructional design review or assessment scoring
- Demonstrated ability to follow complex written guidelines precisely
- Strong written communication skills with a professional, neutral tone
- High attention to detail and consistency
- Ability to make objective decisions under ambiguity
Preferred Qualifications
- Experience reviewing work for research, training data, or evaluation programs
- Familiarity with rubric-based scoring or structured evaluation frameworks
- Background in trust & safety, content moderation, or regulated industries
- Experience working with confidential or sensitive information
Key Competencies
- Judgment and consistency
- Written feedback quality
- Compliance awareness
- Time management and reliability
- Professional discretion
Expectations
- Strict adherence to confidentiality and data protection requirements
- Consistent application of guidelines across all reviews
- Willingness to update evaluation approach as standards evolve
- Professional, unbiased, and respectful communication at all times
Logistics
- Pay range: $80/hour
- Employment type: Contract, flexible hours (part-time and full-time options).
- Location requirements: Remote (U.S. and Canada only)
