GCP Data Engineer

GreenTree Advisory Services Pvt. Ltd. · posted 5 months ago

Gurugramna

About the role

Core Data Engineering Skills
- Big Data Technologies: Proficiency in using GCP's big data tools like:
  - BigQuery: For data warehousing and SQL analytics.
  - Dataproc: For running Spark and Hadoop clusters.
  - Airflow: For pipeline orchestration.
  - Dataflow: For stream and batch data processing (High level idea).
  - Pub/Sub: For real-time messaging and event ingestion (High level idea).
- Data Modeling: Experience/Knowledge on designing scalable, efficient data models for OLAP and OLTP use cases.
- ETL/ELT Pipelines: Expertise in building automated, scalable, and reliable pipelines using custom Python/Scala solutions or Cloud Data Fusion.
Programming and Scripting
- Strong coding skills in Python, SQL, and optionally Java/Scala.
- Familiarity with APIs and SDKs for GCP services to build custom data solutions.
Cloud Infrastructure
- Understanding of GCP services such as Cloud Storage, Compute Engine, and Cloud Functions.
- Familiarity with Kubernetes (GKE) and containerization for deploying data pipelines (Optional but good to have).
DevOps and CI/CD
- Experience setting up CI/CD pipelines using Cloud Build, GitHub Actions, or other tools.
- Monitoring and logging tools like Cloud Monitoring and Cloud Logging for production workflows.

Innovation and Problem-Solving
- Ability to think creatively and design innovative solutions for complex data challenges.
- Experience in prototyping and experimenting with cutting-edge GCP tools or third-party integrations.
- Strong analytical mindset to transform raw data into actionable insights.
Collaboration
- Teamwork: Ability to collaborate effectively with data analysts and business stakeholders.
- Communication: Strong verbal and written communication skills to explain technical concepts to non-technical audiences.
Adaptability and Continuous Learning
- Open to exploring new GCP features and rapidly adapting to changes in cloud technology.