Senior Data Engineer

What’s the role?

At [Employer hidden], we build software that enables us and our customers to physically trade and manage a significant portfolio of grid scale batteries and renewable energy assets which facilitate the clean energy transition, and data is our driving force. As a Senior Data Engineer, you will be instrumental in designing, building, and managing the infrastructure that enables efficient processing and analysis of our large and diverse datasets, ranging from power market data to asset telemetry. We are looking for an engineer with experience not just writing ETLs but delivering data products, as well as processing data outside walled-garden platforms. As part of a small data team working with a significant client-base, you will have a hands-on role driving data-driven decisions across [Employer hidden] and the growing green energy market, transforming the electricity grid in the UK and beyond.

Our primary tech stack & toolchain includes:

• Data Services: Python
• Data Lake: S3 / Athena
• ETL Pipelines: Prefect
• Messaging and Event Streaming: Pulsar
• Databases: PostgreSQL and Cassandra
• Infrastructure: AWS, Terraform, Docker
• Version Control and CI/CD: GitLab

As part of the Data team, you will work in an agile, cross-functional team, collaborating closely with your teammates and stakeholders across the business. Like all high-performing teams, you will have a high degree of autonomy to solve interesting problems and contribute to key products that will make a real impact on the energy transition.

• Build modern data services: Take charge of planning, designing, and developing data engineering projects, working with big data frameworks and integrating with a broader microservice stack.
• Enable industry-leading asset optimisation: Build the infrastructure that empowers our team to implement performant data science methods that will best optimise energy assets on the electricity grid.
• ETL process design and implementation: Create efficient Extract, Transform, Load (ETL) processes to streamline data integration and transformation.
• DevOps adoption: Embrace a DevOps culture by implementing Terraform and infrastructure as code (IaC) for our AWS data estate.
• Ensure data quality and consistency: Maintain high standards of data integrity across multiple data sources.
• Engage in cross-team collaboration: Work closely with data scientists, software engineers, stakeholders, etc. to deliver solutions that optimise energy storage assets.
• Mentorship and team guidance: Define and promote best practices for data infrastructure development and tooling.
• Mentor the wider data team on effective ways of working.

Who are you?

Here at [Employer hidden], we appreciate every single employee and the values they bring to the company so you might be completely different to what we expected. That being said, here’s a few things we’d love you to tick the boxes on;

• You’re degree level educated in computer science or a related discipline OR able to demonstrate equivalent workplace experience.
• You have extensive experience delivering projects focused on data collection, storage systems and data pipelines.
• You have advanced Python skills with a focus on writing clean, maintainable, and production-ready code.
• You’re experienced with the end-to-end development and deployment of data-focused software services.
• You have an in-depth understanding of ETL technologies and best practices.
• You’ve worked with SQL and NoSQL databases and big data frameworks like Spark, Pulsar, etc.
• You can demonstrate ability to integrate with external APIs for both data collection and providing data feeds to customers and third parties.
• You have hands-on experience with data storage, processing and analytics in cloud platforms (e.g., AWS, Google Cloud, Azure).
• You have an understanding of MLOps best practices and the tools required to maintain a suite of machine learning models.

You're an organised and adaptable team player capable of leading and mentoring colleagues.You're able to communicate with stakeholders to identify needs and evaluate alternative technical solutions and strategies.

You have a strong technical capacity for evaluating and expressing the effort and value associated with technical development decisions.

So what’s in it for you?

Hopefully the first thing is you love what we do and want to be part of the mission but let’s be honest…that’s probably not why you’re getting up to work everyday.

We know that rewarding our employees is the best way to make them feel valued. This is a journey we’d love you to take with us and as we grow we look forward to finding more ways to reward and thank you for your commitment to our mission. We currently offer the following benefits with more to come in future;

• 25 days holidays in addition to public holidays, carry over flex and festive office closure (a pretty decent annual leave allowance even if we do say so ourselves!)
• A genuine approach to flexible working - the vast majority of our employees spend at least a day or 2 each week in the office but we’re open to making this work for everyone, their personal circumstances and their workloads
• Enhanced parental leave offerings with 6 months of full salary for the primary caregiver and 1 month for the secondary caregiver
• A yearly salary review for all employees
• We are proud to support the continuous professional growth of our employees by providing a personal annual learning and development budget and fully covering the costs of professional membership fees and subscriptions relevant to your role
• Access to MindTools on us
• Bike to Work Scheme (because it wouldn’t be very on brand if we didn’t encourage you to think about how your travel affects the environment)
• Octopus EV Scheme (see above!)
• A commitment to your wellbeing, with access to our Employee Assistance Programme and complimentary eye tests in partnership with Specsavers
• Internal working groups to focus on the things that matter to you like our “Women of [Employer hidden]”, “Diversity, Equity and Inclusion” and “Parents of [Employer hidden]” groups
• As part of our DEI efforts, we are currently in partnership with MotherBoard which enable us to be committed to improving the working lives of parents at [Employer hidden]
• Regular opportunities to blow off some steam engage with our team, including quarterly socials, the big Festive Gathering and the Summer Party. We’re also working on our wellness initiatives to support your Physical, social, mental and financial wellbeing.