Big Data Engineer w/ElasticSearch Experience (TS/SCI)
Location: Ashburn, VA w/telework available
We are looking for a highly skilled Data Engineer w/ElasticSearch experience to join our team. The ideal candidate will be responsible for designing, implementing, and maintaining large-scale data processing systems. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure supports business needs and enables advanced analytics and data-driven decision-making.
Ninja Analytics is an information technology company that provides analytical and software development services to companies with Anti-Money Laundering (AML), fraud detection, trade surveillance, and systems integration, documentation, and governance needs. We provide an exciting, challenging, and rewarding work experience. We select people based on ability, attitude, character, skill, and training without discrimination regarding age, race, color, marital status, national origin, political belief, religious beliefs, or a disability that does not prohibit performance of essential job functions. Ninja employees enjoy competitive wages, paid time off, paid holidays, as well as health and welfare benefits. Our work environment is comfortable, orderly, safe, and partially remote (2 days per week on-site). Ninja Analytics treats all employees with respect, courtesy and consideration.
Key Responsibilities:
Data Architecture and Design:
- Design and develop scalable, robust, and high-performance data pipelines and data storage solutions.
- Architect and implement data models and schemas optimized for both performance and scalability.
Data Ingestion and Processing:
- Develop and maintain ETL (Extract, Transform, Load) processes to ingest data from various sources, including structured and unstructured data.
- Implement data transformation and cleaning procedures to ensure data quality and consistency.
Big Data Technologies:
- Optimize data processing workflows to ensure efficient resource utilization and minimize latency.
- Utilize big data technologies such as Hadoop, Databricks, Spark SQL, Kafka, and Hive to process and analyze large datasets.
Data Integration and Management:
- Integrate data from multiple sources, including databases, APIs, and third-party data providers.
- Manage and maintain data lakes and data warehouses, ensuring data is organized and accessible for analysis.
Collaboration and Communication:
- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
- Provide technical guidance and support to team members and other departments as needed.
Continuous Improvement:
- Stay updated with the latest industry trends and best practices in big data engineering.
- Continuously improve data processing techniques and tools to enhance the overall efficiency and effectiveness of the data engineering team.
- Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- Proven experience as a Big Data Engineer or in a similar role, with a strong focus on big data technologies and data processing.
- Minimum 9 to 12 years of experience, depending upon education
- Preferred: Certification in big data or cloud technologies.
Required Technical Skills:
- Proficiency in programming languages such as Java, Scala, or Python.
- Strong experience with big data technologies and frameworks (e.g., Hadoop, Databricks, Spark SQL, Kafka, Hive).
- Knowledge of database systems, both SQL (e.g., PostgreSQL, MySQL, Oracle) and NoSQL (e.g., Cassandra, MongoDB).
- Implement and maintain Elasticsearch clusters, ensuring data availability, reliability, and consistency across distributed environments.
- Integrate Elasticsearch with other big data tools and frameworks, such as Apache Kafka, Hadoop, and Spark, to streamline data processing pipelines.
- Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services.
- Knowledge of data warehousing solutions like Amazon Redshift, Google BigQuery, or Snowflake.
Required Soft Skills:
- Excellent analytical and problem-solving skills.
- Strong verbal and written communication skills.
- Ability to work effectively both independently and as part of a team.
- Strong organizational skills and the ability to manage multiple tasks and priorities.
Security Clearance:
Selected applicants must be a current US Citizen and must have an active Top-Secret security clearance with recent background investigation (within the last five years), or current enrollment in continuous evaluation.
