Job Summary:
We are seeking a highly skilled and detail-oriented Software Development Engineer in Test (SDET) to join our team with a focus on data reconciliation , complex SQL , and ETL testing . The ideal candidate will have a unique blend of skills in software development , quality assurance , and data engineering . You will be responsible for ensuring data accuracy, validating ETL processes, and building automation frameworks for data-related testing processes. This is a key role to ensure the integrity and quality of our data infrastructure and analytics platforms.
Key Responsibilities:
- Complex SQL Queries :
- Write and optimize complex SQL queries to validate data pipelines, transformations, and reporting accuracy.
- Perform root cause analysis of data discrepancies and provide insights into data quality issues.
- Develop automated SQL scripts to streamline data validation and reconciliation processes.
- ETL Testing :
- Validate ETL processes by designing comprehensive test strategies to ensure data accuracy and completeness.
- Test and verify ETL pipelines from various data sources (structured and unstructured) into data warehouses, ensuring data conforms to business rules and logic.
- Implement automated ETL testing frameworks and continuously improve them to optimize the data validation process.
- Data Reconciliation :
- Develop, implement, and automate test scripts to validate data integrity between source and target systems.
- Identify and resolve data inconsistencies, mismatches, and errors in large datasets.
Design and execute reconciliation processes to ensure accurate data migration and transformation.
-
Nice to have:
- Experience with HiveQL for Hive Database
- Experience with Python as a scripting language
