Job Description:
RESPONSIBILITIES:
- Build the first iterations of high quality and sustainable data pipelines and ETL processes to extract data from a variety of APIs and relational databases and ingest into AWS services.
- Efficiently draft complex SQL queries and data models to aggregate and transform data for reporting and analytics teams.
- Monitor existing solutions and work proactively to rapidly resolve errors and identify future problems before they occur.
- Own and keep the system design and operations documents up to date.
- Consult with a variety of stakeholders to gather new project requirements and transform these into well-defined tasks and targets.
MUST HAVE REQUIREMENTS:
- 3 years of practical experience in data / analytics with at least 1 year working in an engineering / BI role.
- At least 1 year of practical experience working on data pipelines or analytics projects with Python and SQL.
PREFERRED REQUIREMNETS:
- 5 years of practical experience in data / analytics with at least 2 years working in an engineering / BI role.
- Experience with NoSQL databases (DynamoDB, MongoDB, Elasticsearch, Redis, Neo4j, etc.)
- Strong knowledge and practical experience working with at least 3 of the following AWS services: S3, EMR, ECS/EC2, Lambda, Glue, Athena, Kinesis/Spark Streaming, Step Functions, CloudWatch, DynamoDB.
- Strong experience working with data processing and ETL systems such as Oozie, Airflow, Azkaban, Luigi, SSIS.
- Experience developing solutions inside a Hadoop stack using tools such as Hive, Spark, Storm, Kafka, Ambari, Hue, etc.
- Ability to work with large volumes of both raw and processed data in a variety of formats (JSON, CSV, Parquet, ORC, etc.).
- Ability to work in a Linux/Unix environment (predominantly via EMR & AWS CLI / HDFS).
- Experience with DevOps tools (Jenkins, GitHub, Ansible, Docker, Kubernetes).
Interested parties please send your application to our recruiter via email directly and quote the job reference number if any.