Senior Data Engineer

Job Description
About Epsilon PeopleCloud Discovery
Epsilon PeopleCloud Discovery empowers users with a deep understanding of millions of people—on a marketing platform that unites audience insights, segmentation, creative analysis, media planning and digital activation. It includes a consistent, transparent, best-in-class approach to data, analytics solutions, partnerships and technology via a closely joined network of engineers, technology experts, product designers, analysts and data scientists all empowering marketing and digital business transformation.
Role:
A Senior Data Engineer will use his or her hands-on expertise in Spark/Hadoop, Scala programming, Linux scripting, AWS, DevOps and API technologies to expand and optimize architecture, data flows, infrastructure, APIs and data pipelines to support the Epsilon PeopleCloud Discovery Platform. Successful candidates will take ownership of developing, maintaining, debugging large-scale real-time data workflows and enforcing best practices across various coding projects. He or she will work across functional teams and collaborate very closely with the VP of Data Engineering to plan, develop and deploy features. Additionally, the right candidate will be passionate about learning new things, curious about handling large data sets and be proactive about finding, communicating, and troubleshooting any issues that may come up in daily data processing.
Core Responsibilities:
Write and review code for new features. This is a Senior role, and the candidate will be expected to be involved and own several projects.
Collaborate with the VP of Engineering to plan for new features and architect data solutions.
Ensure data quality by implementing best practices.
Ensure code quality by developing robust test cases.
Think automation first!
Find areas of optimization and implement changes to create more efficient workflows.
Monitor and troubleshoot issues as they come up.
Provide feedback on areas of improvement
SKILLS:
Specialization in data engineering development using very large data sets (10s of TBs).
Expertise in data warehousing, data streaming, relational databases, search databases and in-memory databases.
Strong understanding of cloud computing, distributed computing, and scalability principles.
Strong knowledge of relational databases.
Strong knowledge of data structures and algorithms.
Understanding of NoSQL or columnar databases.
Outstanding communication skills and the ability to work well within agile environment.
Qualifications
Advanced Degree in computer science or electrical engineering
4+ years hands-on data engineering software
Must have development expertise using Scala and Spark.
Strong understanding of cloud services in AWS or Azure
Expertise with relational databases using SQL and Postgres.
Experience using software automation tools such as Jenkins, Ansible, and Git.
Experience working with containers in Docker and Kubernetes.
Experience in Test Driven Development Practices.
Experience working with agile scrum practices.
Knowledge of real time streaming and/or search databases.
Nice to have experience working with Python.