A Boar’s Head Data Engineer II is a key analytical resource that is part of the Enterprise Applications team and is driven to identify, define, develop, and deliver innovative solutions that are in alignment with the organization's goals. As a Data Engineer II, you will enhance operations for analytics, reporting, applications, and data science by gathering and processing raw data at scale. Your focus will be on maintaining scalable, reliable, consistent, and repeatable systems that efficiently provide data for company-wide analytics and operations. A Data Engineer II must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. The primary focus of the Data Engineer II role is centered on the development and support of the Boar’s Head Analytics Data Warehouse and associated technologies. Areas of focus and responsibilities include: • Create and maintain optimal data pipelines using current Boar’s Head streaming technologies (Kafka, Flink and Clickhouse) • Assisting with the designing, building, and maintaining of the infrastructure to support data storage, processing, and retrieval. • Assemble large, complex data sets that meet functional / non-functional business requirements. • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Assist business analytics teams with analytics tools (Tableau) that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. • Work with stakeholders (Business and IT teams) to assist with data-related technical issues and support the data quality and infrastructure needs of the organization. • Support Business analytics teams by ensuring the availability of data and tools necessary to support organizational data analytics projects. • Work with data and analytics experts to strive for greater functionality in our data systems. Essential Duties and Responsibilities
General Duties
- Understands how to use technology to competitive advantage and to solve business issues – focusing on data streaming and analytics concepts and business-level functionality.
- Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Develops and maintains a thorough understanding of business operations, identifying opportunities for operational efficiency through automation and effective utilization of reporting & and analytics tools critical for all business areas (e.g., Marketing, Sales, Distribution, Manufacturing, Finance, Human Resources).
- Assists in technical delivery management and relationship management in identifying options for potential solutions and assessing them for both technical and business suitability.
- Partners with knowledge management leads within Boar’s Head to ensure and maintain consistency on Standard Operating Procedures and formal Work Instructions that outline use of Data Governance and Analytics applications.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Supports roll outs of new and expanding applications/processes/projects.
Specific Duties
- Create and maintain optimal data pipelines using current Boar’s Head streaming technologies (Kafka, Flink and Clickhouse)
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
- Adept at outlining the current state and designing conceptual/future state visual process level diagramming and documentation (in Visio and other more advanced visual modeling tools) that help articulate options to reduce costs, improve efficiencies, and effectiveness, and streamline execution while keeping quality norms in alignment.
- Identifies, documents, and communicates current state of business process, partnering with Technical Delivery Managers, Data Engineers, and Developers to recommend future state options.
- Provides process documentation (to include interaction points with other systems and/or applications).
- Fosters understanding of software functions and limitations, usage scenarios, outlines error handling (system/human interaction points) and expected /necessary quantitative measures for success (e.g., expected performance targets, specific percentages for quality improvements)
- Develops and executes thorough data validation unit and functional test plans to ensure data accuracy and integrity throughout data streams to the data warehouse.
- Partners with QA teams in end-to-end systems testing and coordination of user acceptance testing strategy development and validation.
Project Management Support
- Assists Technical Delivery Manager(s) in business strategy analysis and enhancement demand management and assists Project Manager (PM) in refining project plans/objectives for items responsible for delivering and executing according to best practice methodologies and IT process standards.
- Assesses project requirements and works with PMs, resource managers, and others delegated as PMs to assess level of effort estimates.
- Partners with the project manager to define change requests for missed requirements and scope change, helping maintain the right focus on defined project constraints (schedule, resources, and scope/quality).
- Experience with Atlassian project management tools (JIRA, Confluence) is a significant plus
Communication
- Demonstrates strong oral communication skills, including the ability to effectively interact with business partners and internal team members.
- Demonstrates strong written communication skills, including the ability to produce clear and concise technical documentation.
- Communicates technical concepts and issues to non-technical people.
- Interacts with all levels of the company with patience, courtesy, diplomacy, and professionalism
Education and Experience
- A Bachelor's degree in computer science, software engineering, information technology, or a related field.
- A minimum of 4 years of experience in a Data Engineer (or equivalent) role to be considered; to include experience with data streaming, data wrangling, data modeling, and/or data analytics.
- Technology and Desired Skills –
- Experience and/or knowledge of Data Streaming technologies
- Apache Kafka (or equivalent)
- Apache Flink (or equivalent)
- Clickhouse Columnar Databases (plus)
- Experience with a variety of coding languages such as SQL, Python and/or R. Java Programming and SQL experience is necessary.
- Experience with data integrity and validation techniques
- Experience with Oracle EBS is a plus
- Knowledge of Tableau (or equivalent) analytics tool is a plus
- Ability to articulate business and technology needs/constraints to IT members and end users alike.
- Self-starter, motivated, and having a proactive and strategic mindset are a must.
- Strong written and verbal communication skills
- Strong collaboration and coalition-building skills
- Hands-on experience and proficiency with the full Microsoft Office Suite, Microsoft Visio, and SmartDraw is a plus.
Physical Demands & Work Environment
- This position requires working in an office environment.
- Occasional travel may be required.
- This is primarily a first shift position but adjustments to shift start/end times may be required depending on future production support schedules.
- Availability for on-call during non-business hours is required.
- After-hours and weekend work is intermittently required for system troubleshooting, maintenance, and upgrades.
