Loading open roles
Vrinda Global
GCP Data Engineer
Vrinda Global · posted 1 month ago
Pune, Bengaluru +1 more
About the role
GCP Data Engineer
Experience: 8+ Years
Location: Bengaluru/Pune/Hyderabad
Position Summary
We are seeking a highly skilled GCP Data Engineer to design, develop, and maintain scalable data platforms and pipelines on Google Cloud Platform (GCP). The ideal candidate should possess strong expertise in Big Data technologies, Data Warehousing, ETL/ELT development, Cloud Infrastructure, and modern DevOps practices. This role requires close collaboration with business stakeholders, analysts, and engineering teams to build robust data solutions that support business insights and strategic decision-making.
Key Responsibilities
Design, develop, and optimize scalable data pipelines and data processing frameworks on GCP.
Build and maintain ETL/ELT workflows for data ingestion, transformation, and delivery across multiple platforms.
Develop and manage enterprise-scale data warehouses and data marts.
Design efficient and scalable data models for both OLTP and OLAP environments.
Implement batch and real-time data processing solutions.
Ensure data quality, governance, reliability, security, and performance across the data ecosystem.
Collaborate with cross-functional teams to understand business requirements and deliver data-driven solutions.
Implement monitoring, logging, and performance optimization strategies for production data workflows.
Support CI/CD implementation and DevOps best practices within data engineering projects.
Continuously evaluate emerging GCP services and industry trends to improve data platform capabilities.
Required Qualifications
1. Core Data Engineering Skills
Big Data & GCP Technologies
Hands-on experience with the following GCP services:
BigQuery – Data warehousing, analytics, and SQL-based reporting.
Dataproc – Running and managing Spark/Hadoop-based workloads.
Apache Airflow (Cloud Composer) – Pipeline orchestration and workflow automation.
Dataflow – Batch and stream data processing (good understanding required).
Pub/Sub – Real-time messaging and event ingestion (good understanding required).
Data Modeling
Strong understanding of designing scalable and optimized data models for OLTP and OLAP systems.
Experience with dimensional modeling, data warehousing concepts, and data marts.
ETL/ELT Development
Expertise in building automated, scalable, and reliable ETL/ELT pipelines.
Experience using Python, Scala, or Cloud Data Fusion for data integration and transformation.
2. Programming & Scripting
Strong proficiency in Python and SQL.
Exposure to Java and/or Scala is desirable.
Experience working with APIs and GCP SDKs to develop custom data engineering solutions.
3. Cloud Infrastructure
Strong understanding of core GCP services, including:
Cloud Storage
Compute Engine
Cloud Functions
Experience with containerization and deployment technologies.
Exposure to Google Kubernetes Engine (GKE) is preferred.
4. DevOps & CI/CD
Experience implementing CI/CD pipelines using:
Cloud Build
GitHub Actions
Similar DevOps tools
Hands-on experience with monitoring and observability tools such as:
Cloud Monitoring
Cloud Logging
Understanding of production support, deployment automation, and infrastructure best practices.
Preferred Skills
Experience with Spark, Hadoop, and distributed processing frameworks.
Knowledge of real-time data processing architectures.
Familiarity with Data Governance, Data Quality, and Security best practices.
Strong analytical, problem-solving, and communication skills.
Experience working in Agile development environments.
Must-Have Skills
GCP, BigQuery, Dataproc, Airflow, Python, SQL, ETL/ELT, Data Modeling, Data Warehousing, Cloud Storage, CI/CD
Experience: 8+ Years
Location: Bengaluru/Pune/Hyderabad
Position Summary
We are seeking a highly skilled GCP Data Engineer to design, develop, and maintain scalable data platforms and pipelines on Google Cloud Platform (GCP). The ideal candidate should possess strong expertise in Big Data technologies, Data Warehousing, ETL/ELT development, Cloud Infrastructure, and modern DevOps practices. This role requires close collaboration with business stakeholders, analysts, and engineering teams to build robust data solutions that support business insights and strategic decision-making.
Key Responsibilities
Design, develop, and optimize scalable data pipelines and data processing frameworks on GCP.
Build and maintain ETL/ELT workflows for data ingestion, transformation, and delivery across multiple platforms.
Develop and manage enterprise-scale data warehouses and data marts.
Design efficient and scalable data models for both OLTP and OLAP environments.
Implement batch and real-time data processing solutions.
Ensure data quality, governance, reliability, security, and performance across the data ecosystem.
Collaborate with cross-functional teams to understand business requirements and deliver data-driven solutions.
Implement monitoring, logging, and performance optimization strategies for production data workflows.
Support CI/CD implementation and DevOps best practices within data engineering projects.
Continuously evaluate emerging GCP services and industry trends to improve data platform capabilities.
Required Qualifications
1. Core Data Engineering Skills
Big Data & GCP Technologies
Hands-on experience with the following GCP services:
BigQuery – Data warehousing, analytics, and SQL-based reporting.
Dataproc – Running and managing Spark/Hadoop-based workloads.
Apache Airflow (Cloud Composer) – Pipeline orchestration and workflow automation.
Dataflow – Batch and stream data processing (good understanding required).
Pub/Sub – Real-time messaging and event ingestion (good understanding required).
Data Modeling
Strong understanding of designing scalable and optimized data models for OLTP and OLAP systems.
Experience with dimensional modeling, data warehousing concepts, and data marts.
ETL/ELT Development
Expertise in building automated, scalable, and reliable ETL/ELT pipelines.
Experience using Python, Scala, or Cloud Data Fusion for data integration and transformation.
2. Programming & Scripting
Strong proficiency in Python and SQL.
Exposure to Java and/or Scala is desirable.
Experience working with APIs and GCP SDKs to develop custom data engineering solutions.
3. Cloud Infrastructure
Strong understanding of core GCP services, including:
Cloud Storage
Compute Engine
Cloud Functions
Experience with containerization and deployment technologies.
Exposure to Google Kubernetes Engine (GKE) is preferred.
4. DevOps & CI/CD
Experience implementing CI/CD pipelines using:
Cloud Build
GitHub Actions
Similar DevOps tools
Hands-on experience with monitoring and observability tools such as:
Cloud Monitoring
Cloud Logging
Understanding of production support, deployment automation, and infrastructure best practices.
Preferred Skills
Experience with Spark, Hadoop, and distributed processing frameworks.
Knowledge of real-time data processing architectures.
Familiarity with Data Governance, Data Quality, and Security best practices.
Strong analytical, problem-solving, and communication skills.
Experience working in Agile development environments.
Must-Have Skills
GCP, BigQuery, Dataproc, Airflow, Python, SQL, ETL/ELT, Data Modeling, Data Warehousing, Cloud Storage, CI/CD