About the role

Azure Data Engineer

Job Summary:

We are seeking an Azure Data Engineer with strong, hands-on Microsoft Fabric experience to build and operate a scalable middle-layer data and integration platform. This role focuses on ingesting data from diverse sources, implementing transformation and business rules, and delivering curated, consumption-ready datasets/models for analytics and downstream applications.

Key Responsibilities:

Implement robust data solutions on the fabrics platform, including One Lake architecture for efficient data storage and processing.
Design, develop, and maintain scalable ingestion and transformation pipelines using Microsoft Fabric, ADF/Synapse Pipelines, PySpark, Python, and SQL.
Develop API-driven and scheduled workflows for data ingestion from external APIs, supporting multiple daily runs to curated layers.
Integrate data from cloud and on-prem sources including databases, third-party systems, files, and REST/SOAP APIs with robust error handling.
Build curated layers and consumption-ready models, applying data modeling best practices aligned with reporting and analytics needs.
Develop and optimize complex SQL queries, stored procedures, and datasets for efficient reporting, partnering with analytics teams to meet performance SLAs.
Tune Spark jobs, ADF data flows, and SQL workloads to improve reliability and reduce runtime/cost while ensuring data consistency.

Requirements:

Proven experience with Microsoft Fabric and data engineering principles.
Strong proficiency in data pipeline design using PySpark, Python, and SQL (T-SQL).
Experience with data integration from both cloud and on-prem sources, including APIs and databases.
Demonstrated ability to implement data modeling and transformation logic for reporting/analytics.
Familiarity with automated data quality checks and validation frameworks.
Knowledge of security protocols such as Azure AD, RBAC, and secure connectivity.
Experience with CI/CD practices, specifically utilizing Azure DevOps/Git for deployment automation.

Preferred Qualifications:

Experience with Azure Databricks and associated data governance tools like Azure Purview.
Familiarity with Terraform and infrastructure as code (IaC).
Understanding of performance tuning, cost optimization techniques in cloud environments.
Experience in Agile/Scrum methodologies and collaborative team environments.