About the role

Data Engineering Lead

Department

IT – Digital Manufacturing

Role

Data Engineering Lead

Reporting to

Head of Digital Manufacturing

Location

Tirupati

Job Summary

Amara Raja Batteries is seeking an innovative and detail-oriented Data Engineering Lead to design, develop, and optimize data infrastructure across business domains. This role involves managing data pipelines, integrating multimedia data into a structured medallion data lake-house. The ideal candidate will collaborate with cross-functional teams to ensure reliable and scalable data architecture on Azure platform to support data scientists, analysts and Ai engineers.

About the Company

Key Responsibilities

Key Accountabilities Areas
- Key Activities
  - Data Pipeline Ownership
    - Design, build, and maintain scalable data pipelines to integrate data from multiple systems.
    - Enable efficient data flow for real-time monitoring and predictive analytics aligned with business objectives.
  - Collaboration with Data Teams
    - Partner with data analysts and data scientists to provide clean, structured, and accessible data for advanced analytics.
    - Collaborate to build transformation Silver pipelines and semantic Gold layer for business analytics.
  - IoT and Manufacturing Data Integration
    - Integrate IoT-generated data with manufacturing systems to enable real-time insights and decision-making.
    - Drive IT-OT convergence to enhance connectivity and visibility across digital and physical manufacturing environments.
  - Data Infrastructure Management
    - Optimize data storage and processing systems to support AI and digital use cases.
    - Develop scalable architectures to handle large volumes of high frequency transactional and slow moving enterprise data.
  - ETL & Workflow Automation
    - Develop and automate ETL (Extract, Transform, Load) processes to ensure accurate and timely data integration.
  - Data Quality & Compliance
    - Implement frameworks to maintain data accuracy, integrity, and security in compliance with ISO 9001 and other industry standards.
    - Ensure adherence to data governance.

Required skills

Data Engineering Tools
- Proficiency in tools like SQL, Python, or ETL frameworks for managing and processing data.
- Azure tools like Databricks, Azure Data Factory and Fabric.
IoT Data Integration
- Understanding of IoT protocols (e.g., MQTT, OPC-UA) and ability to process IoT data for real-time insights.
Database Systems
- Expertise in relational databases (MySQL, PostgreSQL) and knowledge of non-relational databases as needed.
ETL Development
- Proven ability to design, automate, and optimize ETL pipelines for manufacturing data.
Cloud technologies
- Broad expertise in at least one Cloud platform, preferably Azure.
- Working knowledge of automation and deployment tools.
Data Governance & Security
- Knowledge of data governance principles and compliance with industry regulations like ISO 9001 and GDPR.

Nice to Have

Industry 4.0 Familiarity
- Understanding of smart manufacturing concepts, including predictive maintenance and digital twins.
Technical Proficiency
- Good understanding of DevOps, CI/CD pipelines, orchestration, and containerization tools like Docker and Kubernetes.

Qualifications

Work Experience
- 5-8 years of experience in data engineering, with a strong focus on Azure platform.
- Proven track record of integrating data from MES, ERP, CRM and manufacturing systems.
Educational Background
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
- Coursework or specialization in data architecture, analytics in Azure platform (preferred).
Preferred Industry Experience
- Manufacturing, energy, or related industries with exposure to Industry 4.0.
Technical Expertise
- Expertise in developing and optimizing data pipelines, ETL processes, and managing large-scale data systems in Azure (preferred).
- Familiarity with tools and platforms like SQL, Python, Databricks, Airflow, and IoT protocols (e.g., MQTT, OPC-UA).