Azure Data Engineer

Azure Data Engineer

Salary: £40,500 - £45,500

Job Type: Full-time

Shift and Schedule: Remote

Location: iSoft in Birmingham

Experience Level: Mid to Senior

Skills: Pyspark/Scala, SQL, ETL, Data Modeling

Tools: Azure Synapse, Azure Data Bricks, ADF, ADLS, Azure DevOps, Azure SQL Database, Azure Blob Storage, Azure Stream Analytics, Fabric, Power BI, SSIS

Requirements: Strong data engineer skills

Job Description:

We are looking for a Azure Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and maintaining business intelligence solutions. You will work closely with stakeholders to understand their reporting needs and translate them into effective data visualizations and dashboards.

Duties and Responsibilities:

• Designed and developed end-to-end ETL pipelines in Azure Data Factory (ADF), orchestrating data workflows to ingest, transform, and load data from diverse sources.
• Develop and automate complex ETL pipelines using Azure Data Factory (ADF).
• Manage and configure Azure Data Lake Storage (ADLS) for structured/unstructured data.
• Develop data transformation jobs using PySpark in Azure Databricks.
• Utilize Azure Synapse for big data and analytical workloads.
• Integrated Azure Databricks and Azure Synapse Analytics for advanced data processing.
• Developed Synapse notebooks using PySpark for complex data transformations.
• Set up CI/CD pipelines in Azure DevOps for continuous integration and deployment.
• Built and managed Power BI dashboards integrating with Azure Synapse Analytics and Azure SQL Database.
• Created data models using DAX for advanced analytics and reporting.
• Automated scheduled data refreshes in Power BI.
• Monitored and optimized Synapse queries and Spark jobs using Azure Monitor and Log Analytics.
• Ensured data governance and compliance with RBAC, encryption, and Azure Key Vault.
• Mentored junior data engineers on Azure DevOps best practices and Power BI reporting techniques.