Staff Data Engineer

🔒 Confidential Employer
Posted 16 October 2026
LOCATION
London
TYPE
Contract
LEVEL
Mid-Senior level
CATEGORY
Technology
This employer holds a UK Home Office sponsor license — sponsorship for this specific role is at the employer’s discretion

SKILLS

PySpark AWS Glue Apache Spark Apache Airflow Python Data Testing AI-assisted development Cloud-native Data Engineering

FULL DESCRIPTION

Staff Data Engineer

London, UK | Contract | Hybrid | £750-900 per day

Job Description

We're supporting a global, technology-led organisation undertaking a large-scale cloud data transformation programme. This isn't a traditional Data Engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns and embedding engineering excellence across its Data function. As a Staff-level engineer, you'll combine deep hands-on technical expertise with organisation-wide technical influence, helping shape how modern Data Engineering is delivered across multiple engineering teams. This is an opportunity to work on enterprise-scale data movement, cloud-native architectures and AI-assisted software engineering within a highly collaborative engineering culture.

Responsibilities

  • Defining technical standards and engineering patterns for a large-scale Data Modernisation programme
  • Re-engineering complex legacy data pipelines into production-grade PySpark
  • Building scalable, reusable data movement frameworks across multiple engineering teams
  • Driving best practice around AWS Glue, Apache Spark and Apache Airflow
  • Designing high-quality testing frameworks covering performance, data quality and reliability
  • Leveraging Agentic AI development tools to improve engineering productivity while maintaining code quality
  • Coaching and influencing experienced engineers through technical leadership rather than line management
  • Working closely with enterprise architecture and platform teams to improve how data is governed, managed and consumed

Requirements

  • Staff, Principal or Lead Data Engineering roles
  • Python and expert-level PySpark
  • AWS Glue
  • Apache Spark
  • Apache Airflow
  • Cloud-native Data Engineering architectures
  • Large-scale data pipeline modernisation
  • Building reusable engineering patterns within enterprise environments
  • Data testing, validation and observability
  • AI-assisted software development using modern coding assistants
  • Working within regulated or highly governed organisations

Contract Details

  • £750-900 per day (inclusive)
  • Initial 6-month contract with significant programme runway
  • Inside IR35
  • London (3 days per week on-site)
Sign up free — access 45,000+ UK sponsor-licensed jobs