About the role

Databricks Architect (10-15 Years)

Work Mode: Hybrid

Mandatory: At least 2 Databricks certifications

About the Role
We are looking for an experienced Databricks Engineer to design, develop, and optimize scalable data platforms and lakehouse solutions on Databricks. The ideal candidate will have strong expertise in PySpark, Delta Lake, Unity Catalog, Databricks Workflows, cloud data platforms, and modern data engineering practices.

Key Responsibilities
● Design, build, and maintain enterprise-scale Databricks platforms and data pipelines.
● Develop batch and real-time data processing solutions using PySpark, Spark SQL, Structured Streaming, and Delta Lake.
● Implement and manage Delta Lake, Delta Live Tables (DLT), Medallion Architecture, and Unity Catalog.
● Build scalable data ingestion frameworks leveraging Auto Loader, Kafka, Event Hubs, and cloud-native services.
● Optimize Spark workloads, cluster performance, and cloud costs through performance tuning and best practices.

● Develop and maintain CI/CD pipelines and Infrastructure-as-Code (Terraform/Databricks Asset Bundles).
● Implement data governance, security controls, metadata management, and data lineage solutions.
● Support ML and AI initiatives using MLflow, Feature Store, RAG, Vector Search, and Databricks Mosaic AI.
● Collaborate with cross-functional teams to deliver high-quality, production-grade data engineering solutions.

Required Skills
● Strong hands-on experience with Databricks, PySpark, Spark SQL, and Delta Lake.
● Expertise in Unity Catalog, Databricks Workflows, Jobs, Clusters, and Repos.
● Experience designing and supporting Lakehouse Architectures.
● Advanced SQL and data modeling skills.
● Experience with cloud platforms such as Azure, AWS, or GCP.
● Knowledge of CI/CD, DevOps, Terraform, and DataOps practices.
● Strong understanding of data governance, security, and performance optimization.
● 2 DATABRICKS CERTIFICATION IS A MUST HAVE

Nice to Have
● Experience with MLflow, MLOps, Feature Store, and Generative AI use cases.
● Databricks certifications.
● Exposure to Azure Data Factory, Kafka, Event Hubs, Fivetran, dbt, or Airbyte.
Education
Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.