About the role

Job Description – Lead Data Engineer (Azure, Databricks)

Job Title: Lead Data Engineer
Experience: 8–12+ Years
Location: Delhi NCR, Pune, Bengaluru, Hyderabad
Employment Type: Full-Time

Job Summary

We are seeking an experienced Lead Data Engineer to design, build, and lead the development of scalable cloud-based data platforms and enterprise data pipelines. The ideal candidate will have deep expertise in Azure , Azure Databricks , Azure Data Lake , Azure Data Factory (ADF) , SQL , Python , and ETL/ELT pipeline development . This role requires technical leadership, architecture design, mentoring team members, and collaborating with cross-functional stakeholders to deliver high-quality data solutions.

Key Responsibilities

Lead the design, development, and implementation of scalable data engineering solutions on Microsoft Azure.
Architect and build robust ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks, and Azure Data Lake Storage (ADLS).
Develop high-performance data transformation workflows using Python, SQL, and PySpark.
Design and optimize enterprise data models, data lakes, and modern data platforms.
Ensure data quality, governance, security, and performance across the data ecosystem.
Optimize SQL queries and data processing jobs for large-scale datasets.
Collaborate with business stakeholders, data architects, analysts, and engineering teams to define data requirements.
Lead code reviews, establish engineering best practices, and mentor junior and mid-level data engineers.
Implement CI/CD pipelines and DevOps best practices for data engineering projects.
Monitor production data pipelines, troubleshoot issues, and ensure high availability and reliability.
Drive continuous improvements in data architecture, automation, and operational efficiency.

Required Skills

8+ years of experience in Data Engineering with at least 2+ years in a technical lead role.
Strong hands-on expertise in Microsoft Azure services.
Extensive experience with Azure Databricks and PySpark .
Strong experience with Azure Data Factory (ADF) .
Expertise in Azure Data Lake Storage (ADLS) .
Advanced proficiency in SQL with query optimization and performance tuning.
Strong programming skills in Python .
Proven experience designing and maintaining scalable ETL/ELT pipelines .
Experience with data warehousing, data modeling, and big data processing.
Knowledge of Delta Lake, Lakehouse architecture, and modern cloud data platforms.
Experience with Git, Azure DevOps, CI/CD, and Agile methodologies.

Preferred Skills

Experience with Azure Synapse Analytics or Microsoft Fabric.
Exposure to Spark optimization and distributed data processing.
Knowledge of streaming technologies such as Kafka or Event Hubs.
Experience with Power BI or other BI/reporting tools.
Azure Data Engineer Associate certification is a plus.