Data Engineer Architect

Internal Job Description – Data Platform Solution Architect (Azure Synapse & Databricks)

Role Overview

We are looking for an experienced Data Platform Solution Architect to join an existing enterprise data platform team. The role requires the ability to rapidly assimilate a complex, partially implemented platform — understanding what has been built, what remains, and what needs to evolve — and to translate that into a clear, actionable architecture that guides the team through current delivery and the next stage of platform transformation. This person will work closely with senior engineers and stakeholders, providing the architectural vision and technical direction needed to complete ongoing work and design the path forward.

Key Responsibilities

• Rapidly assess and develop a deep understanding of the current platform state across all layers and components, working alongside the existing engineering team
• Define and own the target-state architecture, bridging the gap between what is currently implemented and what needs to be completed or evolved
• Design the architectural roadmap for the next stage of platform transformation, ensuring continuity with current investments while enabling future scalability
• Translate business and data requirements into architectural decisions, reference designs, and technical standards
• Provide hands-on architectural guidance to engineering teams on a day-to-day basis — not limited to design documents but actively engaged in delivery
• Identify architectural risks, dependencies, and trade-offs across the platform and recommend mitigation strategies
• Define integration patterns, data flow designs, and governance standards across sourcing ingestion, transformation, and consumption layers
• Establish and enforce architecture principles, design patterns, and platform standards across the team
• Collaborate with data governance, business, and technology stakeholders to align platform direction with enterprise requirements
• Support migration planning by defining target architectures, transition states, and validation strategies for workload migrations across platforms
• Drive decisions on platform tooling, infrastructure, and service selection in alignment with cloud and enterprise architecture standards

Core Technical Skills

Architecture & Design

• Proven experience designing enterprise-scale data platform architectures end-to-end — from source ingestion through to consumption
• Deep understanding of medallion / lakehouse architecture patterns (Raw / Harmonized / Conformed / Consumption)
• Strong knowledge of data modelling approaches: relational, dimensional, and lakehouse-oriented
• Experience designing data governance architectures: lineage, data quality frameworks, audit and control patterns, SCD versioning, CDC
• Ability to design for both batch and real-time data processing at scale
• Experience architecting parallel-run and phased migration strategies across platforms
• Azure Data Platform Strong hands-on knowledge of:
  • Azure Synapse Analytics (Pipelines, Spark Pool, Dedicated SQL Pool)
  • Azure Data Lake Storage Gen2
  • Delta Lake on Azure (Synapse Lakehouse)
  • Azure Analysis Services
  • Oracle Golden Gate or equivalent real-time replication patterns
  • Power BI and semantic layer design
  • Databricks & Lakehouse
  • Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred)
  • Strong understanding of Databricks platform capabilities: Delta Lake, Spark-based transformation, workflow orchestration, governance
  • Experience designing migration architectures from legacy data warehouse or Synapse environments to Databricks Lakehouse
  • Ability to architect governance and control frameworks natively within Databricks

Cloud & Infrastructure

• Strong Azure cloud architecture skills — security, networking, scalability, and cost optimization
• Experience with Infrastructure as Code (Terraform preferred)
• CI/CD and DevOps practices for data platform delivery (Azure DevOps / GitHub Actions)
• Containerization and compute management (Docker, Kubernetes awareness)

Data Engineering Depth

• Sufficient hands-on engineering background to engage credibly with senior engineers on implementation decisions
• Strong Python and SQL skills — able to review, challenge, and contribute to code where necessary
• Experience with ETL/ELT frameworks, pipeline design patterns, and performance optimization at scale

Nice to Have

• Experience with Unity Catalog and Azure Purview for enterprise data governance
• Exposure to real-time and streaming architectures (Event Hub / Kafka / Kinesis)
• Familiarity with MLOps and GenAI platform integration patterns
• Experience with monitoring and observability frameworks (e.g., Dynatrace)
• Background in financial services or other regulated industries

Experience & Profile

• 12+ years of experience in Data Engineering and/or Data Architecture, with at least 4–5 years in a solution or platform architect role
• Proven experience joining mid-flight programmes — able to quickly understand existing implementations and make sound architectural decisions without starting from a blank slate
• Track record of delivering architecture across complex, multi-layer data platforms in enterprise environments
• Strong ability to balance pragmatism with rigour — delivering architecture that works for the team today while keeping the longer-term direction sound
• Excellent communication and stakeholder management skills — able to present technical architecture clearly to both engineering teams and non-technical stakeholders
• Collaborative by nature — this is a working architect role embedded with the team, not a remote advisory position
• Experience working in regulated or enterprise-scale environments (financial services a plus)

Internal