Data Engineering Consultant

Primary Responsibilities:

• Lead the design, development, and implementation of enterprise-grade data and AI solutions, including the use of Large Language Models (LLMs) to automate data ingestion, transformation, and real-time streaming workflows
• Act as a senior technical consultant, partnering with business stakeholders to translate ambiguous or undefined requirements into clear data strategies, architectures, and implementation plans
• Implement and integrate agentic frameworks to enable autonomous decision-making workflows
• Architect, build, and optimize scalable data pipelines and intelligent workflows using Python, PySpark, SQL, and cloud-native technologies
• Develop, refine, and govern prompt engineering strategies and agentic AI frameworks to support autonomous and decision-oriented data workflows
• Create and maintain detailed data specifications, mappings, and transformation logic to standardize disparate healthcare and insurance datasets into common, governed structures
• Perform in-depth data analysis to identify data quality, logic, and mapping issues; determine root causes and drive resolution through structured, consultative problem-solving
• Provide subject-matter expertise in healthcare and claims data, supporting issue documentation, stakeholder reviews, and iterative resolution within an agile delivery framework
• Ensure all solutions align with healthcare data standards, regulatory requirements, data governance policies, and Responsible AI principles
• Contribute to the evaluation, testing, validation, and production deployment of data and AI solutions
• Mentor junior engineers and analysts, promoting best practices, technical rigor, and a culture of knowledge sharing and continuous improvement
• Operate with a high degree of autonomy and accountability, delivering solutions aligned with business priorities, timelines, and quality expectations
• Communicate complex technical and analytical concepts clearly and effectively to both technical and non-technical audiences, including senior leadership
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Qualifications - External

Required Qualifications:

• Undergraduate degree or equivalent experience
• Hands-on experience in building AI Agents or Agentic AI Systems integrating with Data Engineering Pipelines 
• Hands-on experience in data engineering and backend automation
• Experience with ETL tools, workflow orchestration (Databrick Workflow, Lakeflow Spark Declarative Pipelines), and data warehousing (Snowflake)
• Demonstrated familiarity with CI/CD pipelines, GitHub Actions, and containerization tools (Docker)
• Proven solid expertise in Python, PySpark, T-SQL, SQL, Snowflake and Databrick technologies
• Proficiency in cloud technologies: Azure (Databricks, ML Studio), AWS, GCP
• Proven ability to work independently and mentor junior team members
• Proven excellent problem-solving skills and adaptability in a fast-paced environment

Preferred Qualifications:

• Relevant certifications in Generative AI, Agentic AI, cloud platforms, or data engineering
• Experience deploying AI/ML solutions in regulated or healthcare environments
• Knowledge of AI governance, compliance standards, and Responsible AI practices
• Background or familiarity in data science, machine learning, or deep learning