Role purpose
We are seeking an exceptional Lead Data Engineer who combines deep engineering credibility with outstanding people leadership and stakeholder management. You will lead multi-disciplinary data engineering teams to design, build and operate secure, resilient and scalable data products and cloud-native platforms, while translating business priorities into clear engineering outcomes. This role is accountable not only for technical quality, but also for team performance, delivery confidence, colleague development and trusted relationships across Technology, Data, Risk and business leadership.
What you’ll be doing
- Lead and inspire engineering teams. Lead, coach and develop Data Engineers; set clear objectives, create psychological safety, give high-quality feedback, manage performance and build succession and capability plans.
- Own delivery and outcomes. Take end-to-end accountability for data engineering delivery across multiple initiatives, balancing customer value, quality, security, resilience, cost and time-to-market.
- Shape engineering strategy. Define technical direction, reference patterns and engineering standards for modern data platforms; challenge complexity and drive simplification, reuse and automation.
- Build trusted stakeholder partnerships. Influence senior business and technology stakeholders, Data Architects, Product Owners, Data Scientists, Risk, Security and Operations; turn ambiguous needs into prioritised, achievable roadmaps.
- Deliver scalable data products. Design and oversee batch and real-time pipelines, data models, data marts and analytics-ready datasets supporting BI, AI/ML and operational use cases.
- Modernise legacy estates. Lead migration from traditional warehouses and ETL estates to cloud-native, lakehouse and data-product architectures with safe transition and measurable benefits.
- Embed data governance by design. Ensure data quality, lineage, metadata, access controls, privacy, retention and regulatory requirements are engineered into platforms and delivery practices.
- Run reliable services. Set SLOs/operational measures, lead incident and problem improvement, eliminate recurring failure through automation, and ensure observability, capacity and disaster-recovery readiness.
- Raise engineering excellence. Champion code reviews, automated testing, CI/CD, Infrastructure as Code, secure software practices and pragmatic architecture governance.
- Manage dependencies and risk. Surface delivery, technology and people risks early; negotiate trade-offs, resolve cross-team blockers and communicate decisions with clarity and evidence.
Leadership & stakeholder expectations
- Operate as a visible engineering leader: set direction, create alignment and empower teams rather than becoming the delivery bottleneck.
- Communicate complex data and engineering decisions in business language, including options, risks, cost, customer impact and recommended action.
- Build strong partnerships across organisational boundaries and handle constructive challenge, conflicting priorities and escalations calmly.
- Develop an inclusive, high-performing team with clear role expectations, mentoring, coaching, knowledge-sharing and purposeful hiring.
- Use delivery and operational metrics to drive continuous improvement, demonstrating measurable improvements in reliability, flow, quality and cost.
What we’re looking for
- Leadership & people management: Proven experience leading data engineering teams in a large, complex organisation; strong coaching, delegation, performance management, workforce planning and talent-development capability.
- Stakeholder leadership: Demonstrable ability to influence senior stakeholders, facilitate difficult technical/business decisions and build alignment across product, architecture, security, risk, operations and delivery.
- Large-scale data engineering: Extensive hands-on background designing, building and supporting distributed data systems and large-scale processing in production, with strong software-engineering fundamentals.
- Cloud data platforms: Strong experience with GCP preferred, including BigQuery, Dataflow/Apache Beam, Cloud Storage, Pub/Sub and Cloud Composer; equivalent Azure/AWS depth is valuable where accompanied by strong GCP adaptability.
- Data processing & streaming: Expertise in batch and real-time pipelines, Spark/PySpark and Kafka or equivalent event-streaming technologies; strong SQL and Python, with Java/Scala/Go advantageous.
- Data architecture: Strong knowledge of warehouses, lakes/lakehouses, operational data stores, dimensional modelling and modern data-product/data-mesh principles.
- Engineering practices: CI/CD, Git-based workflows, automated testing, Terraform/IaC, containers and Kubernetes; experience improving developer experience and engineering controls.
- Non-functional excellence: Ability to design for security, privacy, resilience, recoverability, performance, scalability, observability and cloud cost efficiency.
- Regulated environments: Experience delivering data capabilities in banking, financial services or another highly regulated environment, with strong awareness of governance, audit and operational risk.
Core technology landscape
Languages & processing
SQL, Python, PySpark, Spark; Java / Scala / Go
GCP & cloud data
BigQuery, Dataflow, Cloud Storage, Pub/Sub, Composer, Spanner; Azure/AWS exposure
Streaming & orchestration
Kafka, Apache Beam, Airflow / Composer
Data platforms & modelling
Data warehouse, data lake/lakehouse, Data Vault, star/snowflake models, data products
DevOps & platform
Git, Jenkins/GitHub pipelines, Terraform, Docker, Kubernetes, observability tooling
Desirable
- Google Cloud Professional Data Engineer, Professional Cloud Architect or equivalent cloud certification.
- Databricks/Delta Lake, dbt, MDM, Data Mesh and AI/ML or GenAI data-platform experience.
- Track record leading major data transformation, cloud migration or platform modernisation programmes across multiple teams.
How success will be measured
- Predictable delivery of high-value data products with clear stakeholder confidence and transparent risk management.
- Improved platform reliability, data quality, security, automation, deployment frequency and cost efficiency.
- A stronger engineering organisation: capable successors, improved team health, reduced key-person dependency and demonstrable skills growth.
- Pragmatic architectural simplification and increased reuse of standard patterns and shared platform capabilities.
