Data Engineer
Department: Digital & Product Department
Salary Details: £60k-70k depending on experience
Location: Chiswick Park (Hybrid)
JOB DESCRIPTION
An exciting opportunity for an experienced Data Engineer to join the Engineering team working within the Digital & Product Department.
The successful candidate will be responsible for creating the pipelines that transform data in a scalable and repeatable way. You will need to produce efficient code that automates the ingestion and cleansing of data and apply the transformations required to achieve the target data format.
You will work with the Engineering and Data & Business Analytics teams to understand the source and target data requirements and use this knowledge to develop data integration solutions to achieve the necessary migrations.
The Data Engineer will support our software developers, database architects and data analysts on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. You must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products.
DUTIES & RESPONSIBILITIES
- Understand, build and develop ETL and data integration solutions using a wide array of technologies and data sources
- Analyse and organise raw data
- Combine raw information from different sources
- Explore ways to enhance data quality and reliability
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, etc.
- Work with cloud-based infrastructure (Azure) for hosting data solutions and applications
- Collaborate with architects, data analysts and data scientists to help meet the business goals
SKILLS REQUIRED
- Previous experience as a data engineer
- Proven experience in development and maintenance of ETL/ELT processes within a medallion architecture
- Strong experience with SQL and relational databases
- Good knowledge of the Azure data engineering stack, including Azure Synapse and Azure Data Lake
- Programming experience in Python, PySpark and T-SQL
- Previous experience with Azure DevOps and understanding of CI/CD is desirable
- Analytical skills related to working with structured and unstructured datasets
- Excellent written and verbal communication skills
- Experience supporting and working with cross-functional teams in a dynamic environment