Data Engineer-BigData

Roles & Responsibilities:

·          Strong hands-on experience on Big Data technologies and frameworks.

·          Responsible for overseeing and managing the Azure Data engsg services.

·          Strong programming skills in Java with experience in software development, version control and CI/CD practices.

·          Knowledge of containerization, orchestration and Kubernetes would be an added advantage.

·          Certification on Azure Data Engineering (DP-203) would be a plus.

·          Handhold teams in Design, development and optimize data pipelines and ETL processes using Spark to extract, transform, load large volumes of structured and unstructured data from diverse sources.

·          Strong skills in writing sql queries for data extraction and manipulation.

·          Implement data security and compliance measures to protect sensitive information and ensure regulatory compliance.

·          Participate in On-call support for Azure related incidents.

·          Excellent communication and collaboration skills, with a strong ability to work in a team-oriented environment.

·          Collaborate with cross functional teams to troubleshoot data related issues and optimized data processes.

 

 

 

Preferred Skillset :

Cloud Technologies       : Azure

Programming                 :  Java

Azure Services               :  HDInsight, Azure Datalake Storage, Apache Spark, Azure Synapse Analytics,  Azure Kubernetes, Azure Blob Storage, Azure Keyvault, Private Endpoints, Service Principal, SEP, Azure Event hub, Apache Kafka.

Orchestration tool         : Apache Airflow

Database                         : Azure SQL / PostgreSQL

Version Control              : Git

Scripting                         : Powershell / Shell Scripting

Soft Skills: 

Mentor the team of engineers to increase the team's efficiency.
Excellent team player with high learning agility.
Excellent communication, influencing, problem-solving, interpersonal and time management skills.