Lead Data Engineer

🔒 Confidential Employer
Posted 11 October 2026
LOCATION
London
TYPE
Full-time
LEVEL
Mid-Senior level
SALARY
£105,000 / year
CATEGORY
Data & Analytics
This employer holds a UK Home Office sponsor license — sponsorship for this specific role is at the employer’s discretion

SKILLS

Databricks Spark SQL Python Airflow Terraform CI/CD Data Pipelines

FULL DESCRIPTION

Lead Data Engineer

[Employer hidden — sign up to reveal] is seeking a Lead Data Engineer to lead the design, development and continuous improvement of a modern cloud-based data platform in London (hybrid). The role involves mentoring a team, building scalable data pipelines, and working with analytics, AI and business teams.

Salary: £95,000 - £105,000 per year

Location: London, Eastleigh (twice a month in office)

Contract Type: Permanent

Main Responsibilities

  • Lead the development and improvement of the data platform and engineering standards.
  • Mentor and support a team of data engineers to help them grow and succeed.
  • Design, build and maintain scalable data pipelines and data products.
  • Work with analytics, AI and business teams to solve data challenges.
  • Improve platform reliability, performance, security and cost efficiency.
  • Drive best practice in testing, automation, CI/CD and Infrastructure as Code.
  • Support the delivery of trusted data for reporting, modelling and AI use cases.
  • Help shape data strategy and assess new technologies and tools.
  • Promote continuous improvement and knowledge sharing across teams.
  • Ensure data governance and compliance requirements are built into solutions.

Skills and Experience You Need

  • Strong experience in Data Engineering and cloud-based data platforms.
  • Hands-on knowledge of Databricks, Spark, SQL and Python.
  • Understanding of Airflow, Terraform and modern engineering practices.
  • Experience building large-scale batch and real-time data pipelines.
  • Strong problem-solving and decision-making skills.
  • Ability to explain technical concepts in a clear and simple way.
  • Collaborative approach and enjoyment of working across teams.
  • Passion for coaching, mentoring and helping others develop.
  • Focus on quality, customer outcomes and continuous improvement.
  • Interest in AI and emerging technologies.

Benefits at [Employer hidden — sign up to reveal]

We offer flexible working, minimum 35 days holiday, health and wealth support, and many other benefits. For full details, visit the [Employer hidden — sign up to reveal] careers page.

About [Employer hidden — sign up to reveal]

We are one of the largest car and home insurers in the UK. Our People help [Employer hidden — sign up to reveal] to be a thriving, creative and innovative place to work. We are a Disability Confident Leader and signatory of the Race at Work Charter and Women in Finance Charter. For more information, visit [Employer hidden — sign up to reveal] Everyone.

Recruitment Scam Alert: [Employer hidden — sign up to reveal] will never ask for money. Legitimate applications only through our official channels. Report scams to [Employer hidden — sign up to reveal] with subject FRAUD.

Apply by clicking the Apply button on the [Employer hidden — sign up to reveal] careers site.

Sign up free — access 45,000+ UK sponsor-licensed jobs