Top 17 Tech Analyst/Designer Jobs in San Francisco, CA

Assessment Designer & Learning Analyst
Mercor —San Francisco, CA
- $120,000 - $160,000 a year
- Full-time
- Writing skills
- Mid-level
- Office
13d
Senior Human Factors Researcher
Oura —San Francisco, CA
- $147,900 - $203,000 a year
- Hybrid work
- Office
- Master's degree
- Analysis skills
- Paid holidays
- Health insurance
Senior Product Designer (AI Builder)
Babylist —Emeryville, CA
- $178,035 - $213,642 a year
- Remote
- 1 year
- Senior level
- Communication skills
- Paid parental leave
- Health insurance
Quick Apply
Product Manager (Evergreen)
Blue State —Oakland, CA
- $91,000 - $93,000 a year
- Hybrid work
- 5 years
- Mid-level
- Office
- Commuter assistance
- Paid parental leave
Quick Apply
4107: Virtual Production Studio Associate
Logic —San Francisco, CA
- $24.42 - $72.31 an hour
- Weekends as needed
- Microsoft Excel
- Customer service
- Mid-level
- Paid holidays
- Disability insurance
Contract Senior Strategist, Content & Campaigns (Evergreen)
Blue State —Oakland, CA
- $65 - $85 an hour
- Contract
- Remote
- Office
- 3 years
- Senior level
Quick Apply
Founding Data Scientist
Assort Health —San Francisco, CA
- $190,000 - $220,000 a year
- Full-time
- Hybrid work
- HIPAA
- SQL
- Senior level
- Commuter assistance
- Health insurance
Lead Product Manager
Cleo Ai —San Francisco, CA
- Office
- Product management
- Senior level
- Health insurance
- Dental insurance
Implementation Engineer
Recidiviz —Oakland, CA
- $97,000 a year
- Full-time
- Remote
- Mid-level
- SQL
- Bachelor's degree
- Paid parental leave
- Paid holidays
Quick Apply
Senior Software Engineer, Ads Measurement & Targeting
Discord —San Francisco, CA
- $196,000 - $245,000 a year
- Full-time
- Analysis skills
- Mentoring
- APIs
Engineering Manager, Rider Verticals
Lyft —San Francisco, CA
- $176,000 - $220,000 a year
- On call
- Hybrid work
- Computer science
- Computer Science
- Bachelor of Science
- Commuter assistance
- Paid parental leave
6d
Senior Gameplay Analytics Engineer (Unreal Engine)
2K —Novato, CA
- $108,900 - $161,160 a year
- Full-time
- 5 years
- Office
- SQL
- Commuter assistance
- Paid parental leave
Quick Apply
Senior Full-Stack Software Engineer, Quests Experience
Discord —San Francisco Bay Area, CA
- $196,000 - $220,500 a year
- Full-time
- Analysis skills
- Mentoring
- APIs
- Relocation assistance

More jobs to discover

Find similar jobs that you might be interested in.

Senior Human Factors Researcher
Oura —San Francisco, CA
Work closely with industrial and product designers, fellow human factors researchers, electrical and mechanical engineers, and program management so that form…
$147,900 - $203,000 a year
UX Researcher, YouTube Trust and Safety
YouTube —San Bruno, CA
You will explore user behaviors and motivations by conducting primary research, including: field studies, interviews, diary studies, ethnography, surveys,…
$132,000 - $190,000 a year
12d

Show more similar jobs

Explore jobs in cities nearby

Check out jobs in cities near your current location, Columbus

Integration Architect L2(CONTRACT)
Wipro Limited —Columbus, OH
Iii. Define and understand current state integration solutions and identify improvements, options & tradeoffs to define target state solutions.
$100,000 - $185,000 a year
3d

I want to receive the latest job alert for Tech Analyst/Designer in San Francisco, CA

By signing in to your account, you agree to SimplyHired's Terms of Service and consent to our Cookie and Privacy Policy.

Explore jobs in more locations

Tech Analyst/Designer jobs near San Francisco, CA

Engineer jobs in Oakland, CA

Senior Software Engineer jobs in San Francisco Bay Area, CA

Production Associate jobs in San Francisco, CA

Lead Product Manager jobs in San Francisco, CA

Senior Software Engineer jobs in San Francisco, CA

Designer jobs in San Francisco, CA

Data Engineer jobs in San Francisco, CA

Product Manager jobs in Oakland, CA

Cloud Engineer Worldwide jobs in Remote

Dft Engineer jobs in Remote

Protection & Controls Engineer jobs in Houston, TX

Systems Engineer jobs in Virginia

Electrical Engineer jobs in Sacramento, CA

Mechanical Engineer jobs in Appleton, WI

Mechanical Engineer jobs in San Luis Obispo, CA

Entry Level Software jobs in Colorado

Senior Software Engineer jobs in Fort Lee, NJ

Senior Software Engineer jobs in Durham, NC

Senior Software Engineer jobs in Westford, MA

Senior Software Engineer jobs in Wisconsin

Assessment Designer & Learning Analyst

Mercor -
San Francisco, CA

Apply Now

Job Details

Full-time
$120,000 - $160,000 a year
13 days ago

Qualifications

Data preprocessing
Quantitative applied research
Master's degree
Regression analysis
Computer skills

Full Job Description

About Mercor

Mercor's mission is to organize human intelligence to power the AI economy. We partner with leading AI labs and enterprises to provide the human intelligence essential to AI development. Our vast talent network trains frontier AI models in the same way teachers teach students: by sharing knowledge, experience, and context that can't be captured in code alone. Today, more than 30,000 experts in our network collectively earn over $3 million a day.

Mercor is creating a new category of work where expertise powers AI advancement. Achieving this requires an ambitious, fast-paced and deeply committed team. You’ll work alongside researchers, operators, and AI companies at the forefront of shaping the systems that are redefining society. Mercor is a profitable Series C company valued at $10 billion. We work in-person five days a week in our San Francisco, NYC, or London offices.

We're looking for an Assessment Designer & Learning Analyst who can build rigorous measurement systems and use data to understand what actually drives expert performance.

This is not an instructional design role. You won't be building courses or writing training materials. You will be designing the assessments and certification frameworks that measure whether our talent experts and internal teams are genuinely skilled — and then doing the analytical work to understand what those assessments reveal, what predicts expert effectiveness, and how our programs should evolve based on evidence. You will be working closely with the Learning & Development team to understand the relationship between materials and assessments, and making recommendations to the team based on your analysis.

If you've come from an ed school background, taught in a high-accountability environment, and completed quantitative projects or theses, and are energized by the measurement and data side of education — this role is for you.

What You'll Do

Assessment Design

Design and continuously improve assessments and certification frameworks that validly and reliably measure expert readiness for specific project types
Build assessments and measurements of skills that are consistent, interpretable, and actually predictive of on-the-job performance — not just checklists.
Develop item banks, scoring guides, and inter-rater reliability protocols for evaluating complex human judgment tasks.
Run validity studies: do our assessments measure what we think they measure?

Learning Analytics & Impact Analysis

Analyze the relationship between instructional materials, assessments, and expert performance — identifying what's working and what isn't and make recommendations accordingly.
Analyze assessment data at the item level — difficulty, discrimination, reliability — and iterate based on findings.
Investigate the relationship between assessment performance and real-world expert effectiveness: who performs well on our assessments, and does that predict quality outcomes?
Build reports and dashboards that surface actionable insights to program and operations teams.
Design and analyze quasi-experimental, quantitative and qualitative (mixed methods) studies to understand what interventions actually move the needle on expert quality.

Ongoing Measurement & Improvement

Track certification and assessment outcomes over time and flag when programs need revision
Partner with learning designers and project teams to translate your findings into program improvements
Bring a continuous improvement mindset — ship, measure, learn, iterate

What We're Looking For

Education

Master's degree in Learning Sciences, Educational Psychology, Educational Measurement, Psychometrics, or a closely related field — required
Coursework in quantitative research methods, psychometrics, and educational statistics — required
Familiarity with classical test theory (CTT) and ideally item response theory (IRT)

Quantitative Skills — Required This role requires genuine comfort with numbers. We're looking for someone who can do the following and show their work:

Item-level analysis: difficulty index, discrimination index, inter-rater reliability (Cohen's kappa, Krippendorff's alpha, ICC)
Assess and report on assessment validity and reliability — and know what to do when results look off
Analyze relationships between variables: correlation, regression, and basic predictive modeling
Work fluently in Excel or Google Sheets for data cleaning and summaries
Use Python, STATA or R for deeper analysis (basic proficiency expected; we'll grow this with you)
Translate quantitative findings into plain-language recommendations for non-technical stakeholders

We will ask you to demonstrate this. Finalists will complete a short take-home exercise involving a real assessment dataset — you'll analyze item performance, identify problems, and recommend improvements.

Experience

1–2 years of experience in assessment design, educational research, learning analytics, or a related role
Teaching or similar experience in a high-accountability environment (Teach For America, urban education, or similar) is a strong plus; people who've lived with assessment data in the classroom understand it differently
Experience designing assessments with a clear theory of what you're measuring — not just writing questions
A portfolio or work samples showing both assessment design and quantitative analysis — we want to see how you think

Skills

Deep understanding of measurement: validity, reliability, and what makes an assessment actually good
Ability to move between data and meaning — you can run the analysis and explain what it means for the program
Strong writing — you can communicate complex findings clearly to non-technical audiences
Systems thinker — you see how individual assessments connect to broader operational quality and expert performance
Comfortable with ambiguity and rapid iteration — this is a fast-moving environment and you'll need to ship and improve continuously

Nice to Have

Experience with item response theory (IRT) or latent variable modeling
Familiarity with data annotation, labeling, or AI evaluation workflows
Experience in tech, AI/ML, or data operations environments
Background in competency-based or mastery learning frameworks
Experience building and analyzing assessments

Why This Role

The quality of AI systems depends on the quality of the humans who train them. Your job is to measure that quality rigorously, understand what drives it, and help Mercor build smarter systems for developing expert performance. It's a rare opportunity to apply serious measurement science at a company operating at the frontier of AI development — where the stakes for getting it right are unusually high.

Compensation Range: $120K - $160K

Apply Now

Refine Your Search

Tech Analyst/Designer jobs in San Francisco, CA

Assessment Designer & Learning Analyst

Senior Human Factors Researcher

Senior Product Designer (AI Builder)

Product Manager (Evergreen)

4107: Virtual Production Studio Associate

Contract Senior Strategist, Content & Campaigns (Evergreen)

Founding Data Scientist

Lead Product Manager

Implementation Engineer

Senior Software Engineer, Ads Measurement & Targeting

Engineering Manager, Rider Verticals

Senior Gameplay Analytics Engineer (Unreal Engine)

Senior Full-Stack Software Engineer, Quests Experience

More jobs to discover

Explore jobs in cities nearby

I want to receive the latest job alert for Tech Analyst/Designer in San Francisco, CA

Explore jobs in more locations

About Mercor

What You'll Do

What We're Looking For

Nice to Have

Why This Role

Job Seeker Tools

Employer Tools

Browse

Stay Connected