About the role

Data Engineer

Job Summary:

As a Data Engineer specializing in Databricks and Delta Lake, you will play a critical role in designing and implementing robust data pipelines that enable the organization to maximize the value of its data. Your expertise will drive data-driven decision-making across the company by ensuring reliable and high-quality data is readily available to stakeholders.

Key Responsibilities:

Design and develop scalable data pipelines using Databricks and Delta Lake to support data ingestion, transformation, and analysis.
Collaborate with data scientists and analysts to understand data requirements and ensure data availability for analytics and reporting.
Implement data quality checks and monitoring to maintain the integrity and reliability of data sets.
Optimize data storage and retrieval processes to enhance performance and reduce costs in cloud environments.
Document data architecture, data models, and ETL processes for team knowledge sharing and compliance.
Troubleshoot and resolve data-related issues and performance bottlenecks to ensure seamless operations.
Stay current on emerging technologies and best practices in data engineering, Databricks, and Delta Lake, and recommend improvements.

Requirements:

Proven experience as a Data Engineer or similar role with a strong focus on Databricks and Delta Lake.
Solid understanding of data modeling, ETL processes, and data warehousing concepts.
Proficiency in programming languages such as Python or Scala for data processing.
Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) for data storage and processing.
Knowledge of SQL and data query languages to access and manipulate data effectively.
Excellent problem-solving skills and attention to detail.
Strong communication skills and the ability to work collaboratively in a team environment.

Preferred Qualifications:

Experience with Apache Spark and big data technologies.
Familiarity with machine learning frameworks and libraries.
Knowledge of data governance and security practices.
Prior experience with real-time data processing and streaming technologies.
Relevant certifications in data engineering or cloud platforms.

Benefits:

Competitive salary and performance-based bonuses.
Comprehensive health, dental, and vision insurance plans.
Flexible work hours and remote work options to support work-life balance.
Access to ongoing professional development and training programs.
Generous paid time off and holiday policies.
Retirement savings plan with company matching contributions.
Inclusive workplace culture that values diversity and encourages innovation.

Data Engineer Databricks