SQL Database Administrator

PulsePoint - New York, NY (30+ days ago)3.8

PulsePoint Data Engineering team plays a key role in our technology company that’s experiencing exponential growth. Our data pipeline processes over 50 billion impressions a day ( > 20TB of data, 220 TB uncompressed). This data is used to generate reports, update budgets, and drive our optimization engines. We do all this while running against extremely tight SLAs and provide stats and reports as close to real-time as possible.

The most exciting part about working at PulsePoint is the enormous potential for personal and professional growth. We are always seeking new and better tools to help us meet challenges such as adopting proven open-source technologies to make our data infrastructure more nimble, scalable and robust. Some of the cutting-edge technologies we have recently implemented are Kafka, Spark Streaming, Docker and Mesos.

What you'll be doing:
  • Support SQL Server Database and added relational and non-relational databases: MySQL, PostgreSQL, Vertica
  • Collaborate with engineering, TechOps, Data Science and Analytics to define requirements, conduct effective planning, and architect the design of optimal database solutions
  • Minimize the occurrence of downtime and implement Disaster Recovery Procedures
  • Troubleshoot database related application performance issues and perform routine and proactive maintenance activities
  • Design and implement effective monitoring solutions that promote proactive discovery of database related issues before they arise
  • Perform query and schema analysis/review for efficient SQL processing and performance tuning
  • Mentor and coach engineering & analytics teams in SQL query and performance best practices
Team Responsibilities:
  • Installation, upkeep, maintenance and monitoring of Kafka, Hadoop, Vertica, RDBMS
  • Ingest, validate and process internal & third party data
  • Create, maintain and monitor data flows in Hive, SQL and Vertica for consistency, accuracy and lag time
  • Maintain and enhance framework for jobs(primarily aggregate jobs in Hive)
  • Create different consumers for data in Kafka such as flafka for Hadoop, flume for Vertica and Spark Streaming for near time aggregation
  • Train Developers/Analysts on tools to pull data
  • Tool evaluation/selection/implementation
  • Backups/Retention/High Availability/Capacity Planning
  • Disaster Recovery- We have all our core data services in another Data Center for complete business continuity
  • Review/Approval - DDL for database, Hive Framework jobs and Spark Streaming to make sure they meet our standards
  • 24*7 On call rotation for Production support
Required Skills:
  • 4+ years of experience configuring and maintaining preferably SQL databases with exposure to MySQL/PostgreSQL/Vertica databases in a Linux-based production environment with expertise in the areas of replication, backup/restore, clustering, DB mirroring, log shipping and performance tuning
  • Responsible for SQL database performance monitoring, storage optimization, SQL tuning and configuration management
  • Expertise or openness to learn Linux administration and scripting languages (e.g. Python, Bash) and a passion for automation and DevOps, leveraging tools like Ansible, Puppet, Docker.
  • Willingness to participate in 24x7 on-call rotation for production support
Technologies we Use:
  • Chronos - for job scheduling
  • Docker - Packaged container images with all dependencies
  • Graphite/Beacon - for monitoring data flows
  • Hive - SQL data warehouse layer for data in HDFS
  • Impala- faster SQL layer on top of Hive
  • Kafka- distributed commit log storage
  • Marathon – cluster wide unit for Docker Containers
  • Mesos - Distributed cluster resource manager
  • Spark Streaming - Near time aggregation
  • SQL Server - Reliable OLTP RDBMS
  • Sqoop - Import/Export data to RDBMS
  • MySQL – Less critical databases
  • Vertica - fast parallel data warehouse
What you’ll get:
  • Sane work hours (with flexible scheduling)
  • Competitive Salary & 401K Plan Match
  • Generous paid vacation (we consider your birthday a holiday)
  • Sabbatical at 5 years of employment
  • Health & Wellness Fairs
  • The opportunity to partake in our Office Fitness Shape-Up Program
  • Professional training and development budget
  • Annual Company Retreat
  • Complimentary membership to local programs like NYC CitiBike
  • Corporate Discount to New York Sports Club (NYSC)
  • Free team lunches twice a month
  • Team happy hours and beer-o-clock Fridays
  • Awesome snacks: drink bar, coffee bar, ice cream bar, candy bar & fruit bar
  • The opportunity to join our Company Sports Teams
  • Indoor dart wars, PAC-MAN, Basketball and Ping-Pong Tournaments