Job Description: Responsibilities:
- Oversee and drive the delivery of end-to-end projects focused on analytical applications and data solutions .
- Provide technical leadership and mentorship to data engineering teams, guiding them on complex challenges and best practices.
- Collaborate with business and technical stakeholders to gather requirements, design solutions, and manage project deliverables.
- Utilize Generative AI frameworks like OpenAI, Hugging Face, or Google Vertex AI to design, deploy, and integrate cutting-edge AI/ML solutions.
- Manage cross-functional collaboration with Business Data Analysts (BDAs), ensuring alignment between data engineering and business goals.
- Take ownership of release planning , change management , and stakeholder engagement to ensure project success.
Ensure adherence to Agile practices, facilitating sprint planning, retrospectives, and delivery timelines.
-
Technical Expertise:
- Framework and Foundational Services : Expertise in building modular, scalable, and reusable frameworks for data integration, data quality validation, and pipeline orchestration.
- Generative AI Integration : Hands-on experience with LLM (Large Language Models), Agentic AI, and integration with APIs from platforms like OpenAI, Hugging Face, and AWS AI services.
- Data Engineering: Strong background in Snowflake, SQL, DBT, and PySpark, with advanced knowledge of query optimization, CDC (Change Data Capture), and data transformation.
- Cloud Platforms: Extensive experience with AWS (Glue, EMR, S3, Lambda, Redshift), Azure Data Factory, and cloud-native architectures.
- Application Development: Expertise in developing event-driven and microservices-based architectures using Java, Python, and modern frameworks.
- CI/CD & Automation : Proficient in building CI/CD pipelines with tools like Jenkins, GitLab, and Terraform for infrastructure-as-code automation.
- Big Data & Analytics : Proven experience in managing Big Data platforms and implementing solutions for large-scale data ingestion, storage, and processing.
- Security & Compliance : Implementation of security best practices, including OWASP guidelines, RBAC, and vulnerability detection/remediation.
Must Have:
- Proven experience in building scalable data engineering frameworks and foundational services to support enterprise-wide analytics initiatives.
- Hands-on expertise in Snowflake, Spark, and AWS Glue, with knowledge of event-driven architectures and streaming data solutions.
- Strong leadership experience in managing teams and delivering complex projects in Agile environments with leading team size of more than 20 members
- Proficiency in integrating Generative AI solutions into existing platforms and workflows.
- Excellent communication and stakeholder management skills to collaborate across diverse teams.
Responsibilities: Responsibilities:
- Oversee and drive the delivery of end-to-end projects focused on analytical applications and data solutions .
- Provide technical leadership and mentorship to data engineering teams, guiding them on complex challenges and best practices.
- Collaborate with business and technical stakeholders to gather requirements, design solutions, and manage project deliverables.
- Utilize Generative AI frameworks like OpenAI, Hugging Face, or Google Vertex AI to design, deploy, and integrate cutting-edge AI/ML solutions.
- Manage cross-functional collaboration with Business Data Analysts (BDAs), ensuring alignment between data engineering and business goals.
- Take ownership of release planning , change management , and stakeholder engagement to ensure project success.
Ensure adherence to Agile practices, facilitating sprint planning, retrospectives, and delivery timelines.
-
Technical Expertise:
- Framework and Foundational Services : Expertise in building modular, scalable, and reusable frameworks for data integration, data quality validation, and pipeline orchestration.
- Generative AI Integration : Hands-on experience with LLM (Large Language Models), Agentic AI, and integration with APIs from platforms like OpenAI, Hugging Face, and AWS AI services.
- Data Engineering: Strong background in Snowflake, SQL, DBT, and PySpark, with advanced knowledge of query optimization, CDC (Change Data Capture), and data transformation.
- Cloud Platforms: Extensive experience with AWS (Glue, EMR, S3, Lambda, Redshift), Azure Data Factory, and cloud-native architectures.
- Application Development: Expertise in developing event-driven and microservices-based architectures using Java, Python, and modern frameworks.
- CI/CD & Automation : Proficient in building CI/CD pipelines with tools like Jenkins, GitLab, and Terraform for infrastructure-as-code automation.
- Big Data & Analytics : Proven experience in managing Big Data platforms and implementing solutions for large-scale data ingestion, storage, and processing.
- Security & Compliance : Implementation of security best practices, including OWASP guidelines, RBAC, and vulnerability detection/remediation.
Must Have:
- Proven experience in building scalable data engineering frameworks and foundational services to support enterprise-wide analytics initiatives.
- Hands-on expertise in Snowflake, Spark, and AWS Glue, with knowledge of event-driven architectures and streaming data solutions.
- Strong leadership experience in managing teams and delivering complex projects in Agile environments with leading team size of more than 20 members
- Proficiency in integrating Generative AI solutions into existing platforms and workflows.
- Excellent communication and stakeholder management skills to collaborate across diverse teams.
Qualifications: Bachelors in engineering, computer science or related feild
