Robert Half, one of FORTUNE’s World’s Most Admired Companies and a Fortune 100 Best Companies to Work For is hiring for a Data Engineer III to join the ATI Data Science Innovation department.
-
Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming)
-
Define technical approach for data engineering initiatives, mentor less-senior engineers, and set standards for code quality through leadership and code reviews
-
Design and build data foundations that enable AI/ML capabilities — feature stores, embedding pipelines, vector search indexes, and model training datasets
-
Align data engineering solutions with business strategy, including support for Agentic AI workloads
-
Own health, scalability, and modernization of data infrastructure with Databricks as the strategic platform — including workload migration, compute optimization, and Unity Catalog adoption
-
Optimize pipeline performance (Delta Lake table layouts, clustering, Z-ordering) and establish monitoring/alerting best practices with clear SLAs
-
Build data infrastructure supporting Agentic AI systems — real-time data access layers, context retrieval pipelines, and agent-accessible data services
-
Collaborate cross-functionally with DevOps, Platform Engineering, and MLOps roles to integrate data solutions into the broader technology environment and shared AI infratstructure – Mlflow registries, feature stores, and agent orchestration layers
-
Provide consultation to Senior Leadership on complex projects and drive continuous improvement initiatives
-
Champion data governance at all layers for data, models, and AI assets
-
Implement data quality strategies (master data management, validation rules, Delta Live Tables expectations) to ensure trust in enterprise data
-
Serve as liaison across data engineering, AI engineering, and business teams; promote data literacy and stewardship
-
Bachelor's in Computer Science, Engineering, or related field (Master's preferred)
-
5+ years with Python and SQL in data engineering for big data ML/analytics workloads
-
5+ years designing, building, and troubleshooting scalable ETL/ELT pipelines for business-critical production systems
-
3+ years with cloud data services (AWS), container orchestration (Docker, Kubernetes), and IaC (Terraform, CloudFormation)
-
3+ years architecting ML workflows and data platforms with CI/CD, automated testing, and distributed processing (Spark)
-
3+ years collaborating cross-functionally with Data Science, MLOps, Platform Engineering, and DevOps teams
-
3+ years implementing data quality testing and optimizing SQL/Python for cost/performance in the cloud
-
Understanding of the full Data Science SDLC, and experience mentoring engineers
-
2+ years hands-on with Databricks (Delta Lake, Unity Catalog, Databricks SQL)
-
Experience with MLflow experiment tracking and model registry workflows
-
Experience designing pipelines that serve AI/ML inference — real-time feature engineering, embedding generation, and context retrieval for LLM-based systems
-
Understanding of how data engineering supports Agentic AI: agent-accessible data services, low-latency retrieval, and pipelines enabling autonomous multi-step workflows
-
Familiarity with Databricks Mosaic AI, Vector Search, and/or Feature Store
-
FinOps awareness — compute cluster optimization, cost attribution by workload
-
Familiarity with Salesforce/Heroku data infrastructures
-
Experience with data virtualization (e.g., Dremio)
-
Understanding of Platform Engineering concepts and internal developer platforms
-
Experience migrating from legacy data warehouse/lake to unified lakehouse architecture
-
Familiarity with Odaseva data security and management
The typical annual salary range for this position is shown below and is negotiable depending upon experience and location. The position is eligible for a discretionary annual bonus.
$104,000.00 - $153,000.00
We offer exceptional earning potential and a competitive benefits package, including group health insurance benefits (medical, vision, dental), FSA and HSA healthcare accounts, life and accident insurance, adoption and fertility assistance, paid parental leave of up to 6 weeks, and short/long term disability. Robert Half provides paid time off for vacation, personal needs, and sick time. The amount of Choice Time Off (CTO) our people receive varies based on their years of service and is pro-rated based on the hours worked per week. A new hire earns up to 17 days of CTO per calendar year. Our people also receive up to 11 paid holidays per calendar year. We also offer the opportunity to contribute to our company 401(k) savings and investment plan or deferred compensation plan (if eligible), with an employer match of 100% on the first 3% of your contributions for eligible employees. Learn more at https://roberthalfbenefits.com.
Robert Half Inc. is an Equal Opportunity Employer. M/F/Disability/Veteran
As part of Robert Half’s Corporate Services facility employment process, any offer of employment is contingent upon successful completion of a background check.
Our recruiters use their expertise and may utilize AI to help with their evaluation of candidates.
Robert Half is committed to being an equal employment employer offering opportunities to all job seekers, including individuals with disabilities. If you believe you need a reasonable accommodation in order to search for a job opening or to apply for a position, please contact us by sending an email to [email protected] or call 1.855.744.6947 for assistance.
In your email please include the following:
The specific accommodation requested to complete the employment application.
The location(s) (city, state) to which you would like to apply.
For positions located in San Francisco, CA: Robert Half will consider qualified applicants with criminal histories in a manner consistent with the requirements of the San Francisco Fair Chance Ordinance.
For positions located in Los Angeles County, CA: Robert Half will consider for employment qualified applicants with arrest or conviction records in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.
