Senior Software Engineer, Data Platform

About SmartNews
SmartNews is a global leader in redefining information and news discovery, committed to providing users with accurate, timely information and supporting quality journalism. We combine the power of our unique machine-learning technology with the expertise of our first-rate editorial team to bring news that matters to millions of users from our over 3,000 global publishing partners.
Founded in 2012 in Tokyo, Japan, SmartNews has offices in Tokyo, San Francisco, Palo Alto, New York, and Singapore.
The Team
The Data Platform Team is responsible for building systems that will power the next 5 years of SmartNews data analytics. They develop and maintain common data platforms, ETL pipelines and data analytics infrastructure that can operate with low latency and high reliability while maintaining 99.99% uptime or above.
The Role
As a Data Platform Engineer, you will play a pivotal role in ensuring that data within the SmartNews environment is highly available, with minimal latency and correctly transferred. You will be a key architect in building and maintaining systems that will power the data use at SmartNews for the next five years. Your expertise will contribute to the foundation that supports SmartNews' ambitious growth goals.
Your Reponsibilities
Be the team SME expert in Data pipelines
Develop and realize the overall data strategy for the company
Develop and manage ETL/ELT pipelines and infrastructure of analytics databases
Cooperate with the product team, data science team to design data features to enable product improvement and timely business KPI measurements.
Be involved in data technology and security to be able to help design the next generation of data platform infrastructure
Determine the importance of each dataset and take appropriate measures to ensure data reliability and quality according to priority
Improve the reliability and speed of queries for ever-increasing volumes of data
Implement new types of data workflows (e.G. automatic retraining of machine learning models for predicting user churn)
Build dashboards to visualize key metrics
Requirements
Minimum requirements
At least 2 years of team lead experience
Able to prioritize and work on data issues ranging from overall data strategy to detailed data modification
Experience with data warehousing and business intelligence systems
5 years of programming experience (understand multiple languages)
Experience with Hive, Airflow, Scylla, Redis, Elastic search
Experience in development related to Spark, Hadoop, Flink, Clickhouse, Presto/Trino
Experience with data processing related to an app or website.
Experience with Modern Machine Learning technologies
Have architected/implemented a data warehouse
Experience designing and creating data tables
BS degree or above in related fields
Experience with AWS (EC2, Kinesis, EMR, S3, etc.)
Eligible to work in the US
Nice to have experiences/skills
Experience in dashboard development with modern tools, such as Tableau or Superset
Experience with infrastructure or data workflows for machine learning
Knowledge of real-time streaming data systems
Strong interest in new big data tools and technologies
Business-level Japanese
Benefits
Compensation and Benefits
Compensation range:
$175,000 to $230,000 base salary, plus bonus and equity
401(k) with employer matching
Up to 100% medical, dental, and vision insurance for you and your family
A benefits plan with monthly allowances
Complimentary lunch, snacks, and beverages in the office