Role Overview
We are seeking a Senior Platform Engineer to design, secure, and optimize our enterprise data infrastructure. You will own the foundational architecture of our Databricks Lakehouse, ensuring data teams can build efficiently within a secure, high-performance, and cost-controlled ecosystem.
Key Responsibilities
-
Infrastructure Management: Architecture, deployment, and daily support of enterprise Databricks Workspaces.
-
Data Governance: Full implementation of Unity Catalog for data lineage and access control.
-
Guardrails & Governance: Designing global cluster policies to balance user agility with cost.
-
Performance Tuning: Optimizing complex Spark configurations, storage layouts, and query speeds.
-
Innovation Delivery: Productionizing cutting-edge capabilities like Databricks Genie and Lakehouse Apps.
Required Technical Skills
-
Data Platform: Advanced Databricks administration (Workspaces, Unity Catalog, Delta Lake).
-
Cloud Infrastructure: Strong experience with AWS, Azure, or GCP via Terraform (IaC).
-
Compute Optimization: Deep knowledge of Apache Spark internals and Spark UI tuning.
-
Security & Compliance: Implementing Role-Based Access Control (RBAC) and data masking.
-
CI/CD Pipelines: Building automated deployment tracks using Git, GitHub Actions, or GitLab CI.
Soft Skills & Qualifications
-
Experience: 8+ years in Platform, DevOps, or Data Infrastructure Engineering.
-
Collaboration: Ability to bridge the gap between Data Science teams and IT security.
-
Problem Solving: Strong root-cause analysis skills for complex, distributed computing issues.
