JOB DESCRIPTION
Senior Azure Databricks Data Engineer
9-15 Years of IT Experience
We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
Key Responsibilities ● · Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services. ● · Build and optimize ETL/ELT workflows for large-scale structured and unstructured data. ● · Develop data models and implement data quality, validation, and governance frameworks. ● · Integrate data from multiple sources into a unified Lakehouse architecture. ● · Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency. ● · Implement security controls, access management, and data governance using Unity Catalog. ● · Collaborate with business, analytics, and AI/ML teams to deliver trusted data products. ● · Monitor, troubleshoot, and resolve data pipeline issues. ● · Support CI/CD, DevOps, and infrastructure automation practices. ● · Maintain technical documentation and best practices.
