Specialist Software Engineer – Big Data Production Support

We are seeking an experienced Senior Production Support Engineer with strong expertise in Spark, SCALA, SNOWFLAKE,  Unix, DevOps tools, and distributed computing environments. The ideal candidate will have hands-on experience supporting Big Data ecosystems and cloud-based platforms, ensuring the stability, performance, and reliability of critical business applications.

 

Responsibilities :

• Provide end-to-end production support for large-scale data processing and analytics applications running on distributed systems.
• Monitor, troubleshoot, and resolve issues in real-time across Unix-based and Big Data environments.
• Support and maintain data pipelines and workflows involving Spark, Scala, Hive, Hue, and HDFS.
• Collaborate with development, DevOps, and infrastructure teams to ensure high system availability and incident prevention.
• Automate operational tasks using DevOps tools (e.g., Jenkins, Git, Ansible, or similar).
• Work closely with identity management teams and have a good understanding of IDM and related tools.
• Ensure adherence to change management and incident management processes.
• Support data integrations and transformations involving Datastage and Snowflake.
• Participate in on-call rotation, root cause analysis, and continuous improvement initiatives
• Profile Required: ACE can help you write this job description (go/ACE)

Operating Systems:  

• Unix/Linux (strong command-line and shell scripting skills)
• Big Data Ecosystem: Spark, Scala, Hive, Hue, HDFS, Cloud Era (Cloudera)
• DevOps Tools: Jenkins, Git, Ansible, or similar CI/CD and automation tools
• Data Integration & Warehousing: Datastage, Snowflake
• Identity Management (IDM): Understanding of user access, authentication, and security-related processes
• Cloud and Distributed Computing: Hands-on experience with distributed data platforms and performance tuning
• Experience working in a 24x7 support model with on-call rotation
• Exposure to performance monitoring and alerting tools (e.g., Grafana, Splunk, or Datadog)
• Ability to automate repetitive support tasks using scripting or DevOps pipelines
• Familiarity with ITIL processes (Incident, Problem, and Change Management)6+ years of experience in production support, system operations, or data engineering environments
• Proven experience supporting Big Data and cloud-based systems in enterprise settings
• Strong analytical, troubleshooting, and problem-solving skills
• Excellent communication and collaboration abilities