Tech & Digital-Sr Data Engineer

 

Job Title : (Job Name) – Data Engineer
 

Job Purpose
• Azure Data Engineer & Power BI Developer with experience in Big Data Tools and Technologies and Spark Streaming.
•
Skilled in designing and implementing scalable data pipelines, working with cloud-based data solutions, and insights through Power BI reports and dashboards
•
Work with Azure Data Factory (ADF) and Databricks to design, develop, and maintain data pipelines and data platforms.
•
Build operational and automated dashboards , reports with help of Power BI and other reporting tools
•
Responsible for day-to-day tasks related to data engineering, data modeling, ETL (Extract Transform Load), data warehousing, and data analytics.
Job Responsibilities(JR) : 6 – 8 Areas
Actionable (4-6)
•
Designing and implementing data storage solutions on Azure
•
Building and maintaining data pipelines for data integration and processing
•
Optimizing data processing performance through tuning and monitoring
•
Developing and maintaining data models and schemas
•
Ensuring data security and privacy prerequisites are followed
•
Designing and developing BI reports and dashboards
•
Collect data from various sources (specifically Kafka and CDC), transform into a practical format, and load into the appropriate data storage solution on Azure using tools such as Azure Data Factory and Azure Databricks.
•
Develop and maintain data models and schemas optimized for specific use cases.
•
Ensure that the data stored and processed on Azure conforms with applicable data security and privacy requirements
•
Develop operational and functional reports
•
Build automated reports and dashboards with the help of Power BI and other reporting tools
•
Understand business requirements to set functional specifications for reporting applications
•
Be experienced in tools and systems on MS SQL Server BI Stack, including PowerBI
Classification - Internal
Classification - Internal
•
Be able to quickly shape data into reporting and analytics solutions
Educational Qualifications (examples listed below)
Key Skills(examples listed below)
Bachelors degree in Computer Science, IT or related area
•
5 - 8 years of experience in design and development – particularly in databases and reporting ( relational and otherwise)
• Data Engineer with Pyspark, Databricks skillset 4+ years of experience Data Engineering with min 3+ years hands on experience in Pyspark for data transformations
•
Proficiency in reporting tools and systems on MS SQL Server BI Stack, including SSRS and TSQL, Power Query, MDX, PowerBI
•
Proficiency in Azure Data Factory (ADF) , Azure Data Lake , Databricks and Delta Lake
•
Extensive knowledge on big data concepts like Hive, and spark framework.
•
Should be able to write complex SQL queries. Sound understanding of DWH concepts.
•
Strong hands-on experience on Python and/or Scala.