Job Title: Databricks Platform Architect – AWS (Onsite)
Location: Dallas, TX
Duration: 6.7 Months
Interview Type: Virtual / In-Person
Job Description:
We are seeking an experienced Databricks Platform Architect with strong AWS expertise to architect, implement, secure, and manage enterprise-grade Databricks Lakehouse platforms. The resource will be responsible for platform architecture, cloud infrastructure, networking, security, data governance, environment management, automation, observability, performance optimization, and AI/GenAI enablement.
The ideal candidate will have deep hands-on experience with Databricks on AWS, including AWS S3, Delta Lake, Unity Catalog, Databricks Workspaces, Spark/PySpark, Terraform, CI/CD, and cloud security. The role requires strong architectural leadership and the ability to design scalable, highly available, secure, and cost-optimized data platforms supporting enterprise analytics, streaming, machine learning, and GenAI workloads.
Key Responsibilities:
- Architect and manage secure, scalable, and highly available Databricks Lakehouse platforms on AWS.
- Design enterprise data platforms using AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces.
- Define AWS networking architecture, including VPCs, PrivateLink, IAM roles, security groups, network controls, and encryption standards.
- Design secure connectivity between Databricks, AWS services, enterprise applications, and data sources.
- Establish and manage multi-environment architectures across Development, Test, UAT, and Production.
- Implement automated infrastructure provisioning using Terraform and Infrastructure as Code (IaC).
- Establish platform configuration, deployment, and environment management standards.
- Implement enterprise data governance, data lineage, metadata management, access controls, and compliance using Unity Catalog.
- Design high-performance data ingestion and processing architectures for batch, streaming, and real-time analytics workloads.
- Support Apache Spark and PySpark-based data processing architectures.
- Design and enable AI and GenAI capabilities using Mosaic AI, Vector Search, Model Serving, AI/BI Dashboards, and Genie Spaces.
- Support ML lifecycle management using MLflow.
- Define CI/CD frameworks using GitHub, Jenkins, and related DevOps tools.
- Implement monitoring, logging, alerting, and observability using CloudWatch, Databricks Workflows, and other platform capabilities.
- Design and support Databricks Workflows for automated data processing and orchestration.
- Integrate Kafka, Airflow, Kubernetes, and other enterprise data platform technologies where required.
- Optimize Databricks platform performance, reliability, scalability, and availability.
- Implement FinOps and workload optimization practices to control cloud and platform costs.
- Establish platform security, compliance, and operational best practices.
- Provide architectural leadership for Lakehouse modernization and enterprise data platform adoption.
- Define best practices for advanced analytics, machine learning, AI governance, and GenAI workloads.
- Evaluate platform architecture, identify risks and gaps, and recommend scalable solutions.
- Provide technical guidance, mentoring, and knowledge transfer to project and platform teams.
- Create architecture documentation, process flows, technical standards, and implementation guidance.
- Provide hands-on technical configuration and development support when required.
Required Qualifications:
- Strong experience architecting and managing Databricks Lakehouse platforms on AWS.
- Extensive experience with Databricks platform architecture and administration.
- Strong AWS cloud architecture experience.
- Hands-on experience with AWS S3, Delta Lake, Unity Catalog, and Databricks Workspaces.
- Strong knowledge of AWS networking, including VPC, PrivateLink, IAM, security groups, and encryption.
- Strong experience with Terraform and Infrastructure as Code (IaC).
- Experience designing and managing Dev, Test, UAT, and Production environments.
- Strong experience implementing enterprise data governance using Unity Catalog.
- Experience with data lineage, metadata management, access controls, and compliance.
- Strong Spark/PySpark experience.
- Experience designing batch, streaming, and real-time data processing architectures.
- Experience with CI/CD, GitHub, Jenkins, and DevOps practices.
- Experience with monitoring, logging, and observability frameworks.
- Strong experience with Databricks Workflows.
- Experience optimizing Databricks performance, scalability, reliability, and cloud costs.
- Strong understanding of Lakehouse architecture and modernization.
- Experience supporting AI, ML, and GenAI workloads on Databricks.
- Strong architectural leadership, communication, and stakeholder management skills.
Preferred Qualifications:
- Databricks Certified Data Engineer certification.
- Databricks Platform Administrator certification.
- AWS Certified Solutions Architect or AWS Certified DevOps Engineer certification.
- Terraform certification.
- Experience with Mosaic AI.
- Experience with Vector Search and Model Serving.
- Experience with AI/BI Dashboards and Genie Spaces.
- Experience with MLflow.
- Experience with Kafka.
- Experience with Apache Airflow.
- Experience with Kubernetes.
- Experience with FinOps and cloud cost optimization.
- Experience leading enterprise Lakehouse modernization programs.
Technical Skills:
- Databricks
- Databricks Lakehouse
- Databricks Workspaces
- AWS
- AWS S3
- Delta Lake
- Unity Catalog
- Apache Spark
- PySpark
- Terraform
- Infrastructure as Code (IaC)
- AWS VPC
- AWS PrivateLink
- AWS IAM
- Security Groups
- Encryption
- Data Governance
- Data Lineage
- Metadata Management
- CI/CD
- GitHub
- Jenkins
- AWS CloudWatch
- Databricks Workflows
- Mosaic AI
- Vector Search
- Model Serving
- AI/BI Dashboards
- Genie Spaces
- MLflow
- Kafka
- Airflow
- Kubernetes
- FinOps
- Cloud Cost Optimization
- Lakehouse Modernization
- AI Governance
- GenAI
- Advanced Analytics
Deliverables:
- Develop and maintain Databricks and AWS platform architecture documentation.
- Create process flows and technical architecture diagrams.
- Define platform standards, security controls, and deployment patterns.
- Provide best-practice and industry-specific technical solutions.
- Recommend alternative and out-of-the-box architecture solutions.
- Provide technical thought leadership for Lakehouse modernization.
- Provide hands-on platform configuration and development support.
- Support data conversion and data migration activities where applicable.
- Mentor project and platform team members.
- Provide knowledge transfer and technical guidance.
- Participate as a primary, co-author, or contributing author for project deliverables.
- Support enterprise data platform adoption and implementation activities.
- Perform other related duties as assigned.
Position Requirements:
- Onsite position in Dallas, TX.
- Local candidates only.
- Resource is expected to work onsite a minimum of 3 days per week.
- Strong Databricks on AWS platform architecture experience is mandatory.
- Hands-on AWS and Databricks experience is required.
- Strong Terraform, Unity Catalog, AWS networking, and CI/CD experience is required.
- Experience with enterprise data governance and Lakehouse architecture is required.
Skill Matrix:
- Databricks Platform Architecture: Required – Expert
- Databricks on AWS: Required – Expert
- AWS Architecture: Required – Expert
- AWS S3: Required
- Delta Lake: Required
- Unity Catalog: Required – Expert
- Databricks Workspaces: Required
- AWS VPC: Required
- AWS PrivateLink: Required
- AWS IAM / Security Groups: Required
- AWS Encryption / Security: Required
- Terraform: Required – Expert
- Infrastructure as Code: Required
- Multi-Environment Architecture: Required
- Data Governance: Required
- Data Lineage / Metadata Management: Required
- Apache Spark / PySpark: Required – Expert
- Batch / Streaming / Real-Time Processing: Required
- CI/CD: Required
- GitHub: Required
- Jenkins: Required
- CloudWatch: Required
- Databricks Workflows: Required
- Mosaic AI: Preferred
- Vector Search: Preferred
- Model Serving: Preferred
- AI/BI Dashboards: Preferred
- Genie Spaces: Preferred
- MLflow: Preferred
- Kafka: Preferred
- Airflow: Preferred
- Kubernetes: Preferred
- FinOps / Cost Optimization: Required
- Lakehouse Modernization: Required
- AI / GenAI Architecture: Preferred
- AI Governance: Preferred
- Platform Performance Optimization: Required
- High Availability / Disaster Recovery: Required
- Technical Architecture & Leadership: Required
- Knowledge Transfer / Mentoring: Required
- Databricks Certification: Preferred
- AWS Certification: Preferred
- Terraform Certification: Preferred
About GSK Solutions Inc:
GSK Solutions Inc. is a leading IT services company specializing in consulting solutions and staff augmentation. We deliver high-quality, on-time, and cost-effective solutions, consistently exceeding client expectations with superior execution.
#GSKIT
Pay: $70.00 - $75.00 per hour
Expected hours: 40.0 per week
Work Location: In person
