Job Description: Role: AWS Data Engineer
We are looking for a Data Engineer with strong skills in Python and PySpark to design and build data solutions for a Fortune 500 client. The role focuses on building data pipelines and integrations on AWS as part of an enterprise data lake platform.
Key Responsibilities
- Build and test data processing applications using PySpark and Python
- Develop data pipelines using AWS Glue ETL or EMR
- Create AWS Lambda functions using Python (pandas, json, requests, awswrangler)
- Work with data from:
- Relational databases (Oracle)
- NoSQL databases (DynamoDB, MongoDB)
- File systems (S3, HDFS)
- Implement event-driven pipelines using Kafka
- Develop REST APIs using FastAPI or Flask
- Implement basic authentication (OAuth2 / JWT) for APIs
- Build workflow orchestration pipelines using Step Functions and EventBridge
- Work with big data file formats such as Parquet, Avro, ORC, JSON
- Optimize Spark jobs using standard techniques (partitioning, joins, etc.)
- Use Glue Crawlers to catalog datasets
- Monitor and troubleshoot jobs using CloudWatch
- Support deployment using Docker containers
Required Skills
- Hands-on experience with Python and PySpark
- Experience with AWS services:
- S3, Glue, Lambda, EMR, Step Functions, EventBridge, Athena
- Experience with Kafka integration
- Strong SQL skills (writing complex queries)
- Experience working with data file formats (Parquet, Avro, ORC, JSON, XML)
- Experience using Python libraries (pandas, requests, boto3)
- Experience building REST APIs (FastAPI or Flask)
Experience
- 4+ years of experience in Data Engineering or related field
- Bachelor’s degree in Computer Science or related field (or equivalent)
Base Compensation Range: $130,000 – $150,000
The posted range is the hiring range for this role — a subset of the broader range available to employees over time — and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process.
