Senior Data Engineer

Numentica LLC - Palo Alto, CA

Full-timeEstimated: $120,000 - $160,000 a year
Skills

We are looking for a motivated Data Engineer / Data Scientists to contribute towards the success of our Data and Analytics Technology initiatives. This person will be experienced in different machine learning, deep learning and AI algorithms and work on data mining and statistical modelling for predictive and prescriptive analytics and generating actionable insights. The right candidate will play a deep dive hands-on critical development role in the digital transformation and in shaping how we acquire diverse set of data, design, develop, deploy and interpret models and outcomes.

Responsibilities

  • Implement techniques to acquire, discover and explore data for analytics
  • Identify use cases and implement statistical modelling, machine learning, deep learning and AI capabilities to create data-driven user experiences
  • POC / POV and prototyping data and analytics solutions and derive viability
  • Ensure data quality, integrity, security and completeness throughout the data lifecycle
  • Define data validation strategies, pre-processing and feature engineering strategies
  • Test/train and fine tune models and deployment of models in production
  • Work with business stakeholders to understand requirements and business use cases and translating data into metrics, KPI, and solutions
  • Develop deep understanding of the data sources, implement data standards, maintain data quality and master data management
  • Design and develop data services and API
  • Mentor, coach, train data engineers and analysts

Requirements

  • Strong Machine learning, deep learning experience especially in banking and financial sector with hands-on development using technology stack including Python, R, Spark and Hadoop
  • Experienced in analyzing large quantities of data in cloud platform like AWS or Azure eco-system. (Azure preferred)
  • Exposure and deep understanding of supervised and unsupervised machine learning algorithms, logistic regression, random forest, SVM, KNN and other analysis.
  • Solid understanding of python libraries for machine learning such as scikit-learn, pandas, numpy and deep learning frameworks such as tenserflow, keras etc…
  • Strong experience and exposure in classification, clustering, segmentation, targeted recommendations and managing customer behavior data
  • Handling large internal and external data including social media and other 3rd party datasets to derive valuable insights
  • NoSQL Databases and Big data technologies including Hadoop, MongoDB, and Azure CosmosDB
  • Experience with API / RESTful data services
  • Worked on real-time data capture, processing and storing using technologies like Azure Event Hubs and Streaming Analytics
  • Experience working with different data storage options including AWS S3, Azure BLOB storage etc.
  • Understanding of different data formats including Parquet, Avro, CSV, ORC etc.
  • Prior experience with MPP databases and maintain large amount of data processing
  • Experience with Azure Data Factory and Azure Data Catalog is a big plus but not mandatory
  • Experience with Microsoft/Azure ETL solutions and business Intelligence technologies is a big plus but not mandatory
  • Past working experience on a fast paced and agile environment
  • Ability to review architecture design and provide direction on system capabilities
  • Perform ongoing monitoring, automation and refinement of data engineering solutions
  • Experience in leading high visibility projects that interacts with multiple business lines
  • Experience working with an on-shore / off-shore model
  • Build and meet project timelines and manage delivery commitments with proper communication to management
  • Collaborate and communicate with key business lines, technology partners, vendors and architects

Qualifications

  • MS degree with 8+ years of relevant experience in Computer Science/ Statistical Modelling & Analysis/Operations Research
  • Willingness to learn new technologies and thrive in an extremely fast paced environment
  • Team player and easy to work with.

Job Type: Full-time