Mission Summary:
The Systems Readiness and Performance team is the crucial bridge between software development and real-world deployment. We are responsible for driving system design, for verifying and validating the autonomy stack, and for defining, measuring, and validating system performance targets. We work closely with stakeholders in autonomy, infrastructure, and operations to build the definitive safety case for the commercial launch of our fully driverless IONIQ 5 robotaxis in Las Vegas.
Rigorous behavioral and system performance evaluation is critical to scaling our service and achieving Motional's long-term goals. We are seeking a Senior Data Scientist to lead initiatives that improve evaluation and testing methodologies, measure the quality and trustworthiness of our evaluation portfolio, and partner with engineering teams to monitor and strengthen the health of the evaluation ecosystem. You will help ensure Motional's performance evaluation is efficient, scientifically rigorous, and aligned with our growth priorities.
In this role, you will lead development of evaluation methodologies and metrics that assess the quality and business relevance of solutions spanning on-road and off-board data. You will influence the evaluation signals software engineers rely on to validate that changes to the autonomy stack deliver intended improvements, conduct deep-dive analyses to understand bottlenecks in current methodologies, and prototype improvements in metrics, sampling strategy, and statistical inference. You will develop deep expertise in how evaluation signals inform launch and release decisions, weigh trade-offs across the evaluation portfolio, and provide actionable insights for designing launch criteria.
If you are a rigorous, collaborative data scientist with a passion for improving how autonomous systems are measured and validated at scale, we encourage you to apply.
What You'll Be Doing:
- Lead the development of evaluation frameworks for the autonomous system, connecting technical problems to rigorous, data-driven approaches for measuring and validating performance.
- Collaborate closely with Functional Safety and Systems Engineering teams to ensure evaluation metrics map effectively to automotive safety standards (e.g., SOTIF, ISO 21448) and launch readiness decisions.
- Ensure evaluation metrics are reliable enough to inform safety cases and launch readiness decisions.
- Monitor the reliability of evaluation metrics and incoming performance data over time, including detecting drift, inconsistencies, and degradation in metric definitions, to ensure the evaluation ecosystem remains accurate and trustworthy.
- Drive our approach to performance analysis using data-backed statistical methods for simulation and on-road data.
- Develop new statistical analysis methods to analyze AV performance data and lead by example in applying them to real problems.
- Partner with triage operators and simulation engineers to turn raw disengagements and identified edge cases into procedural or generative scenarios, feeding them back into the simulation catalog to strengthen test coverage.
- Use fleet and evaluation data to identify edge cases in an automated manner and coverage gaps, and partner with engineering to feed novel scenarios back into the simulation catalog and strengthen test coverage.
- Build confidence in the evaluation framework through data-driven insights and clear communication of findings to technical leaders and stakeholders.
- Establish correlation between on-road and simulation data to improve how we interpret and act on evaluation results.
- Make sense of large datasets to drive insights, solve ambiguous performance questions, and communicate results effectively across teams and upward to leadership.
- Establish a self-service model for developers to understand the impact of their changes.
- Develop new metrics, interpret trends, and investigate anomalies in simulation and on-road data.
- Collaborate with developers to drive action based on these results.
- Serve as an advisor and influence collaborators across multiple teams, promote data-aware decision making, and establish best practices around the use of data.
- Mentor and collaborate with fellow engineers and foster a positive, collaborative work environment.
- Introduce the use of ML methods for performance evaluation where they add rigor and scale.
What You Bring:
- 5+ years of industry experience solving complex problems with large datasets, with a track record of framing ambiguous questions into rigorous, data-driven analyses.
- Bachelor's or higher degree in Computer Science, Computer Engineering, Data Science, Robotics, Physics, Mathematics, or a related quantitative field. Master's or PhD preferred.
- Strong problem-solving skills: ability to break down complex performance and evaluation challenges, think logically, and remove bias from how problems are defined and assessed.
- Strong Python and SQL skills, with demonstrated experience using data analysis libraries to work with large, complex datasets.
- Experience applying advanced statistical and ML methods to drive insights from large and complex data sets.
- Demonstrated experience with statistical analysis, hypothesis testing, causal analysis and data analysis.
- Demonstrated ability to work independently with minimal guidance and drive projects from problem definition through to actionable results.
- Proven communication and interpersonal skills, with the ability to explain technical findings clearly to engineering partners and leadership.
- Eager to learn new statistical and ML techniques and demonstrated willingness to teach
Bonus Points:
- Experience with adversarial scenario generation and closed-loop simulation environments.
- Experience in autonomous driving or robotics, specifically evaluating sub-systems like Perception, Prediction, or Motion Planning.
- Familiarity with data pipelines and distributed compute (e.g., AWS) to seamlessly collaborate with our Data Engineering and MLOps partners.
- Expertise in Machine Learning and Deep Learning.
- Expertise in modern sequence modeling (e.g., Transformers applied to time-series or trajectory data) and probabilistic ML / uncertainty quantification for distinguishing rare-but-safe behavior from out-of-distribution failures.
- Familiarity with automotive safety standards like ISO 26262 or ISO 21448 (SOTIF).
We encourage a hybrid schedule with in-office time at one of our locations in Boston, Pittsburgh, or Las Vegas to support collaboration, or this role can be fully remote.
Motional is a driverless technology company making autonomous vehicles a safe, reliable, and accessible reality. We're driven by something more.
Our journey is always people first.
We aren't just developing driverless cars; we're creating safer roadways, more equitable transportation options, and making our communities better places to live, work, and connect. Our team is made up of engineers, researchers, innovators, dreamers and doers, who are creating a technology with the potential to transform the way we move.
Higher purpose, greater impact.
We're creating first-of-its-kind technology that will transform transportation. To do so successfully, we must design for everyone in our cities and on our roads. We believe in building a great place to work through a progressive, global culture that is diverse, inclusive, and ensures people feel valued at every level of the organization. Diversity helps us to see the world differently; it's not only good for our business, it's the right thing to do.
Scale up, not starting up.
Our team is behind some of the industry's largest leaps forward, including the first fully-autonomous cross-country drive in the U.S, the launch of the world's first robotaxi pilot, and operation of the world's longest-standing public robotaxi fleet. We're driven to scale; we're moving towards commercialization of our technology, and we need team members who are ready to embrace change and challenges.
Formed as a joint venture between Hyundai Motor Group and Aptiv, Motional is fundamentally changing how people move through their lives. Headquartered in Boston, Motional has operations in the U.S and Asia. For more information, visit www.Motional.com and follow us on Twitter, LinkedIn, Instagram and YouTube.
Motional AD Inc. is an EOE. We celebrate diversity and are committed to creating an inclusive environment for all employees. To comply with Federal Law, we participate in E-Verify. All newly-hired employees are queried through this electronic system established by the DHS and the SSA to verify their identity and employment eligibility.
