About the role

Senior Data Engineer

Job Summary:

As a Senior Data Engineer, you will play a critical role in designing, implementing, and maintaining scalable data pipelines that transform large volumes of data into actionable insights. Your expertise will impact the organization's data architecture and governance, enabling seamless collaboration across teams to derive value from data.

Key Responsibilities:

Lead the design, implementation, and maintenance of scalable data pipelines for processing and transforming data from various sources using Databricks, Python, and PySpark.
Architect efficient and scalable data solutions using modern architecture designs.
Design and optimize data models and schemas to ensure efficient data storage, retrieval, and analysis.
Develop, optimize, and automate ETL workflows to extract, transform, and load data into data warehouses or lakes.
Utilize big data technologies like Spark, Kafka, and Flink for distributed data processing and analytics.
Deploy and manage data solutions on cloud platforms such as AWS, Azure, or GCP, leveraging native services for data processing.
Implement data quality and governance measures while monitoring and troubleshooting data pipelines for optimal performance.

Requirements:

Proven experience as a Senior Data Engineer or similar role, with hands-on expertise in building and optimizing data pipelines and architectures.
Strong problem-solving and analytical skills to diagnose and resolve complex data-related challenges.
Excellent understanding of data engineering principles and practices.
Solid communication and collaboration skills to work effectively with cross-functional teams and non-technical stakeholders.
Proficiency in programming languages such as Python, Java, Scala, or SQL for data manipulation.
Familiarity with data governance frameworks and practices.
Understanding of machine learning workflows and data pipeline support.

Preferred Qualifications:

Experience with modern data architectures, particularly lakehouse models.
Background in software engineering or related fields.
Experience using CI/CD pipelines, version control systems like Git, and containerization tools such as Docker.
Knowledge of ETL tools like Apache Airflow, Azure Data Factory, Informatica, or Talend.
Familiarity with Apache Spark Streaming, Kafka, or real-time data streaming technologies.

Benefits:

Competitive salary with performance-based bonuses.
Comprehensive health, dental, and vision insurance packages.
Flexible work hours and remote working options.
Generous paid time off and holiday leave policies.
Continuous learning and development opportunities, including training and certifications.
Retirement savings plan with company match.
Innovative and inclusive company culture that values diversity and employee input.

Sr. Data Engineer