Job Overview
We are seeking an experienced Ab Initio ETL Lead to join our data engineering team. In this pivotal role, you will oversee the design, development, and implementation of complex Extract, Transform, Load (ETL) processes using Ab Initio software. Your expertise will ensure efficient data integration across diverse systems, supporting enterprise-wide data warehousing and analytics initiatives. The ideal candidate will possess a strong background in data management, software development, and cloud-based big data systems, contributing to our mission of delivering reliable and scalable data solutions.
Duties
- Lead the end-to-end development and maintenance of ETL pipelines utilizing Ab Initio tools, ensuring high performance and scalability.
- Collaborate with business analysts and data architects to translate business requirements into technical specifications.
- Design and optimize data models, including dimensional modeling and data warehousing structures, to support reporting and analytics needs.
- Manage integration of diverse data sources such as SQL databases, Hadoop ecosystems, cloud platforms like AWS and Azure Data Lake, and linked data frameworks.
- Develop and implement best practices for data quality, validation, and governance within ETL workflows.
- Oversee the deployment of ETL processes in cloud environments, ensuring compliance with security standards and performance benchmarks.
- Utilize programming languages such as Java and Python for custom scripting, automation, and API integrations including RESTful APIs.
- Support agile development cycles by participating in sprint planning, code reviews, and continuous improvement initiatives.
- Conduct analysis to troubleshoot issues within ETL pipelines and implement effective solutions promptly.
- Mentor junior team members on ETL development techniques, tools like Talend or Informatica, and big data systems such as Hadoop or Spark.
Skills
- Extensive experience with Ab Initio ETL toolset including Graphs, Metadata Hub, Data Profiler, and Co>Operating System.
- Strong proficiency in SQL databases such as Microsoft SQL Server, Oracle, and cloud-based query management systems.
- Hands-on knowledge of big data technologies including Hadoop ecosystem components (HDFS, Hive), Spark, and related frameworks.
- Familiarity with cloud platforms like AWS (Amazon Web Services), Azure Data Lake, and public cloud environments.
- Expertise in data modeling techniques including dimensional modeling for data warehousing design.
- Proficiency in programming languages such as Java, Python, Bash (Unix shell), VBA scripting for automation tasks.
- Experience with software development methodologies like Agile to facilitate iterative delivery.
- Knowledge of business intelligence tools such as Looker for visualization and reporting purposes.
- Understanding of RESTful API integration for seamless data exchange between systems.
- Strong analysis skills for troubleshooting complex data pipeline issues and optimizing performance.
- Ability to design scalable data management solutions that meet enterprise standards for security and compliance.
This role offers an exciting opportunity to lead innovative data projects within a dynamic environment. We welcome candidates who are passionate about leveraging cutting-edge technologies to transform raw data into actionable insights.
Pay: $95,368.09 - $114,851.89 per year
Work Location: In person
