About the role

Senior Data Engineer

Job Summary:

At SOTI, we are dedicated to empowering our employees with opportunities to learn, innovate, and make a significant impact. As a Senior Data Engineer, you will design and maintain robust ETL/ELT pipelines, driving data excellence within our organization and delivering critical insights to stakeholders.

Key Responsibilities:

Design, build, and maintain production-grade ETL/ELT pipelines using SQL, Python, PySpark, and Spark SQL.
Own end-to-end data pipelines from ingestion through to analytics-ready datasets for various stakeholders.
Maintain and enhance existing data integrations to ensure high performance, stability, and scalability.
Troubleshoot complex pipeline failures and serve as a key escalation point for production support issues.
Collaborate with Lead Engineers and Data Architects on cloud migration initiatives, specifically migrating on-premise systems to the cloud.
Assess legacy data pipelines and integrations, contributing to migration sequencing and risk mitigation strategies.
Implement data validation, monitoring, logging, and alerting systems to ensure the integrity and trustworthiness of data products.

Requirements:

7+ years of experience in Data Engineering, Analytics Engineering, or a relevant technical role.
Strong hands-on experience with SQL, Python, PySpark, and Spark SQL in distributed data environments.
Proven expertise in building and supporting enterprise-scale ETL/ELT pipelines within cloud platforms.
Strong experience with Azure-based data platforms, including Azure Synapse Analytics and Microsoft Fabric.
Solid understanding of cloud-native data architectures and best practices in analytics engineering.
Experience with Git-based source control, CI/CD pipelines, and automated testing for data solutions.
Demonstrated problem-solving skills with a history of supporting production data systems effectively.

Preferred Qualifications:

Hands-on experience with Microsoft Fabric Lakehouse and Spark workloads.
Familiarity with Azure Data Factory, Fabric Pipelines, or other orchestration tools.
Experience with data governance, lineage, and certification processes in cloud platforms.
Relevant certifications such as Microsoft Certified: Azure Data Engineer Associate or Microsoft Fabric Analytics Engineer are highly regarded.

Benefits:

Exciting opportunities for professional development and continuous learning.
Access to cutting-edge technologies and innovative projects.
Flexible work arrangements to promote work-life balance.
Comprehensive health and wellness benefits for employees and their families.
Collaborative and inclusive company culture that values diverse perspectives.
Engaging team activities and social events to foster community.
Competitive compensation and performance-based incentives.

If you want to bring your ideas to life, apply at SOTI today. Please note that SOTI does not accept unsolicited resumes from recruiters or employment agencies. In the absence of a signed Services Agreement with an agency/recruiter, SOTI will not consider or agree to payment of any referral compensation or recruiter fee.