About the role

Data Engineer/Sr Data Engineer/Lead Data Engineer

Data Engineers will be responsible for the design, development, testing, maintenance, and support data assets including; Azure Data Lake and data warehouse development, modeling, package creation, SQL script creation, stored procedure development, integration services support among other responsibilities.

Candidate have at least 4-6 years hands-on Azure experience as a Data Engineer, must be an expert in SQL and have extensive expertise building data pipelines. Candidate will be accountable for meeting deliverable commitments including schedule and quality compliance. This Candidate must have skills to plan and schedule own work activities, coordinate activities with other cross-functional team members to meet project goals.

Basic understanding of:

Scheduling and workflow management & working experience in either ADF, Informatica, Airflow or Similar
Enterprise Data Modelling and Semantic Modelling & working experience in ERwin, ER/Studio, PowerDesigner or Similar
Logical/Physical model on Big Data sets or modern data warehouse & working experience in ERwin, ER/Studio, PowerDesigner or Similar
Agile Process (Scrum cadences, Roles, deliverables) & basic understanding in either Azure DevOps, JIRA or Similar
Architecture and data modelling for Data Lake on cloud & working experience in Amazon WebServices (AWS), Microsoft Azure, Google Cloud Platform (GCP)
Basic understanding of Build and Release management & working experience in Azure DevOps, AWS CodeCommitt or Similar

Strong In:

Writing code in programming language & working experience in Python, PySpark, Scala or Similar
Big Data Framework & working experience in Spark or Hadoop or Hive (incl. derivatives like pySpark (prefered), SparkScala or SparkSQL) or Similar
Data warehouse working experience of concepts and development using SQL on single (SQL Server, Oracle or Similar) and parallel platforms (Azure SQL Data Warehouse or Snowflake)
Code Management & working experience in GIT Hub, Azure DevOps or Similar
End to End Architecture and ETL processes & working experience in ETL Tool or Similar
Reading Data Formats & working experience in JSON, XML or Similar
Data integration processes (batch & real time) using tools & working experience in either Informatica PowerCenter and/or Cloud, Microsoft SSIS, MuleSoft, DataStage, Sqoop or Similar
Writing requirement, functional & technical documentation & working experience in Integration design document, architecture documentation, data testing plans or Similar
SQL queries & working experience in SQL code or Stored Procedures or Functions or Views or Similar
Batch Ingestion & working experience in Data Factory and Azure Databricks.
Storage & working experience in Azure Data Lake Storage GEN1/GEN2
Orchestration & working experience in either Data Factory or HDInsight or Data Pipeline or Cloud composer or Similar
Database & working experience in any of the database like MS SQL, Oracle or Similar
Analytical Problem Solving skills & working experience in resolving complex problems or Similar
Communication (read & write in English), Collaboration & Presentation skills & working experience as team player or Similar

Good to have:

Stream Processing & working experience in either Databricks Streaming, Azure Stream Analytics or HD Insight or Kinesis Data Analytics or Similar
Analytical Warehouse & working experience in either SQL Data Warehouse or Amazon Athena or AWS Redshift or Big Query or Similar
Real-Time Store & working experience in either Azure Cosmos DB or Amazon Dynamo-DB or Cloud Bigdata or Similar