C9 : Big Data Engineer - Python & Spark\Scala

The Data/Information Mgt Analyst is a trainee professional role. Requires a good knowledge of the range of processes, procedures and systems to be used in carrying out assigned tasks and a basic understanding of the underlying concepts and principles upon which the job is based. Good understanding of how the team interacts with others in accomplishing the objectives of the area. Makes evaluative judgements based on the analysis of factual information. They are expected to resolve problems by identifying and selecting solutions through the application of acquired technical experience and will be guided by precedents. Must be able to exchange information in a concise way as well as be sensitive to audience diversity. Limited but direct impact on the business through the quality of the tasks/services provided. Impact of the job holder is restricted to own job.

Qualifications:

• Master’s / Engineering Degree with 2 years of experience in Big Data systems, Hive, Hadoop, Spark (Python/ scala) and cloud-based data management technologies
• Hands-on experience in Unix Scripting, Python and Scala programing along with strong experience in SQL.
• Comfortable working with completed unstructured, undocumented code and turning it around into best-in-class code redesigning costly compute and data processes and aligning to best development standards
• Experienced in working with large and multiple datasets, data warehouses and ability to pull data using relevant programs and coding.
• Well versed with necessary data preprocessing and application engineering skills
• At least 3 years of experience designing software systems with intense computational needs across real time and batch process .
• Experience and understanding of Supervised, unsupervised machine learning techniques
• Exposure to data ingestion, ETL tools such as Talend, modeling tools, Performance Management tooling such as Pepper data, Cloudera stack will be a plus
• Knowledge of data management, data governance, data security and regulatory practices
• Ability to identify, clearly articulate and solve complex business problems and present them to the management in a structured and simpler form
• Should have experience of working in onsite, offsite delivery model
• Experience working with large and multiple datasets, data warehouses and ability to pull data using relevant programs and coding.
• Previous related experience preferred
• High attention to detail

Education:

• Bachelors/University degree or equivalent experience

  
  This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.