GCP Data Engineer

• GCP Data Engineer (4+ years)
• Chennai, Tamil Nadu, India

 

Job Description

Skills : Big Data; CI/CD; ETL - Big Data / Data Warehousing; GCP; Python; SQL; Java. Spark/pyspark mandatory

JD:
Senior hands-onGCP Data Engineer who can work independently across a variety of GCP Big Data services.Strong experience working with Google Cloud Platform (GCP) data services, including BigQuery and BigtableProficiency in Python programming (Java experience is a plus)Strong hands-on experience with Cloud Composer (Airflow) for workflow orchestrationHands-on experience with Dataproc for large-scale data processingGood to have experience in building messaging and event-driven data pipelines using Pub/SubExperience with CI/CD pipelines (GitHub Actions)Marketing Platforms team within US Consumer and Digital Technology division is responsible for enabling creation, execution and tracking of Marketing Campaigns across all Amex products and channels."Flywheel" initiative aims to modernize our technology stack to improve Marketer"s operational efficiency and bring campaigns to market quicker, creating more dynamic, personalized customer experience.An individual in this role will work on brining marketing data into Unified Marketing Data Layer on our GCP BigData platform to facilitate reporting and analytics of marketing campaign performance.

 

• Only immediate joiners to be shared.
• Exp – 5-8 Yrs relevant to GCP ( Do not share profiles with 8+ yrs exp)
• Candidate location - Chennai
• Clear BGV and educational checks
• Candidates should be highly stable with min 2 – 3 yrs with every employer
• No major career gaps

 

GCP Data Engineer – MLSE – 5- 8 years’ experience

Chennai / Bangalore

Job Description

Total - 3 Positions:
2 Positions: Big Data; GCP; PySpark; Python; SQL
1 Position  : Big Data; GCP; ETL - Big Data / Data Warehousing; Git (GitHub, GitLab, BitBucket, SVN); PySpark; Python; SQL

Position Required : Google Cloud - Data Engineer. We are looking for energetic, high-performing and highly skilled data engineers to help shape our technology and product roadmap. You will be part of the fast-paced, entrepreneurial Global Campaign Tracking (GCT) team under Enterprise Personalization Portfolio focused on delivering the next generation of global marketing capabilities. The team is responsible for marketing campaign tracking of new accounts acquisition and bounty payments and leverages large scale data engineering technologies, such as such as SQL, PySpark, GCP, Big Query, Data Proc, Adobe Analytics, Google Analytics, Hive, Kafka & Java. Focus : Designs, develops, solves problems, debugs, evaluates, modifies, deploys, and documents software and systems that meet the needs of customer-facing applications, business applications, and/or internal end user applications.

Responsibilities· Develop and maintain large scale data processing pipeline using PySpark Data Proc, Big Query and SQL.· Use Big Query and Data proc to migrate existing Hadoop/Spark/Hive workloads to Google Cloud.· Proficient in Big Query to carry out batch and interactive data analysis.· Function as member of an agile team by contributing to software builds through consistent development practices (tools, common components, and documentation)· Develops and tests software, including ongoing refactoring of code, and drives continuous improvement in code structure and quality· Enable the deployment, support, and monitoring of software across test, integration, and production environments

Minimum Qualifications:This high energy Engineer must have:A Bachelor’s degree in computer science, computer engineering, other technical discipline, or equivalent work experience6 – 9 years of software development experience. Hands-on expertise with application design, software development, and automated testing. Strong programming knowledge in SQL, PySpark, Data Proc, Big QueryHands-on experience in Big Data technologies (Spark, Hive)Understanding and experience with UNIX / Shell / Perl / Python scriptingDatabase query optimization and indexingWeb services design and implementation using REST / SOAP and Java is a plusExperience collaborating with the business to drive requirements/Agile story analysis.Experience with design and coding across one or more platforms and languages as appropriateBonus skills:Machine learning/data miningObject-oriented design and codingAdobe Marketing Campaign products