Principal Developer - Data Integration Engineer

Job Description
ESSENTIAL DUTIES AND RESPONSIBILITIES:
Develop, test, and deploy data integration pipeline jobs using chosen cloud based native tool according to Western Digital design principles.
Troubleshoot existing pipeline jobs based on BASH, Python, Java and modify jobs for performance improvements.
Develop new pipelines using Java/Spark/Scala on AWS EMR platform and schedule using AirFlow
Design data tables for faster load and optimal query performance
Debug SQL statements and stored procedures in Postgres and Redshift databases.
Use git-based source control system bit bucket and continuous integration tools like Jenkins and deployment using Ansible.