San Francisco, CA · On-site · Full-time
Compensation: $180,000–$220,000 + competitive equity
An early-stage (post–Series A) company building the training data and evaluation infrastructure that frontier AI labs use to improve their models — designing high-signal datasets and running rigorous evaluations that go beyond static benchmarks. A small team where individual contributors have direct impact on how the next generation of models learns. The company has raised $30M (~$300M valuation), with a founding team drawn from Jane Street, Citadel, Google, Goldman, and Stanford AI Lab.
Founded 2025 · 11–50 people · Industry: Consumer Tech
As a SWE (Environments), you'll design the datasets and evaluation rubrics that directly influence how frontier models learn — going from hypothesis to live experiment quickly, with output feeding directly into model training runs at scale.
What you'll be doing
-
Design data slices and explore data shapes that expose meaningful model failure modes across domains like finance, code, and enterprise workflows
-
Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines
-
Model annotator behavior and run experiments to improve different model capabilities
-
Develop quantitative frameworks for measuring dataset quality, diversity, and downstream impact on model alignment and capability
-
Create and manage both real-world and synthetic data pipelines
-
Partner with lab research teams to translate their training objectives into concrete data and evaluation specifications
Tech stack: Not specified
-
1–4 years of software engineering experience with strong technical depth
-
Design targeted data slices that surface model failure modes across high-stakes domains (finance, code generation, enterprise workflows)
-
Build and iterate on evaluation rubrics and reward signals powering RLHF and RLVR training pipelines
-
Develop quantitative frameworks to measure dataset quality, diversity, and downstream impact on model alignment and capability
-
Own end-to-end real world and synthetic data pipelines, from scoping with research teams to production-ready evaluation specs
-
Run annotator modeling experiments to improve model capabilities across task types
-
Experience at RL environment companies
-
Background in AI safety or benchmarking organizations like METR or Artificial Analysis
-
Genuine obsession with how data structure, selection, and quality drive model behavior
-
Ability to design lightweight experiments and move fast
-
Former founders or early engineers at early stage startups
-
Demonstrated ability to work hard, learn fast, and care deeply about details
-
Pure research profile with limited engineering output, this is a SWE role, shipping matters
-
Looking for standard product engineering work — the real scope is data pipelines, reward modeling, and eval infra
-
Outsized total cash: base plus substantial profit share, plus competitive equity
-
Direct impact on frontier AI model development, working with the world's leading AI labs
-
High ownership on a small, early team — scope, build, and ship end to end
-
Location: San Francisco, CA
-
Work policy: On-site
-
Compensation: $180,000–$220,000 + equity
-
Visa sponsorship: O-1, OPT
-
Employment type: Full-time
