Hadoop Data Engineer

Hadoop Data Engineer

Experience: 8–12 Years
Location: Bangalore
Role Type: Full-Time

Role Overview

We are looking for an experienced Hadoop Data Engineer with strong expertise in building and maintaining scalable data processing solutions. The ideal candidate should have hands-on experience with Python, PySpark, Hadoop, and SQL , along with a strong understanding of data engineering principles.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark, Python, and Hadoop .
• Develop efficient data processing and transformation workflows for large-scale datasets.
• Write complex and optimized SQL queries for data extraction, transformation, and analysis.
• Work with distributed data processing and storage technologies within the Hadoop ecosystem.
• Perform data cleansing, validation, transformation, and integration across multiple data sources.
• Optimize Spark jobs, data pipelines, and SQL queries for performance and scalability.
• Troubleshoot data pipeline issues and ensure reliability and data quality.
• Collaborate with data architects, analysts, and other engineering teams to deliver robust data solutions.
• Follow best practices for coding, testing, documentation, and production deployment.

Required Skills

• 8–12 years of overall experience in Data Engineering.
• Strong hands-on experience with Python .
• Strong expertise in PySpark / Apache Spark .
• Hands-on experience with Hadoop ecosystem and distributed data processing.
• Strong SQL development and query optimization skills.
• Good understanding of ETL/ELT concepts and data pipeline development.
• Experience working with large-scale datasets and distributed computing environments.
• Strong analytical, troubleshooting, and problem-solving skills.

Preferred

• Experience with cloud-based data platforms and modern data engineering tools.
• Exposure to CI/CD, version control, and production data pipeline deployment.
• Experience working in Agile development environments.

Location

Bangalore