Senior Data Engineer
Location: Remote, Oxford (Once per Month)
Salary: £60,000 - £85,000 + Benefits
Overview:
We are looking for a Senior Data Engineer to play a key role in evolving our data infrastructure, ensuring enhanced reliability and scalability. Reporting to the Data & Analytics Director, you will be integral in driving forward our data capabilities, as we move towards a larger data mesh set-up managed from a central hub of excellence. Over the past three years, we have built a lakehouse architecture with automated reporting and self-service capabilities. With an expanding scope, we now need a Senior Data Engineer to mature and scale our solution efficiently, focusing on DataOps/DevOps, data ingestion, and processing within Databricks.
Key Responsibilities:
- Enhance and stabilize our data infrastructure for increased reliability.
- Manage data ingestion pipelines (currently using ADF and Python).
- Ensure high-quality raw datasets to enable Data Analysts in creating reliable data models.
- Deploy and manage data-related tools on Kubernetes, including Airflow, Superset, and RStudio Connect.
- Support Data Analytics workflows by managing DBT, implementing deployment rules, and handling DevOps processes.
Skills & Experience:
- Proven Data Engineering experience, particularly with Azure, Databricks, and Delta Lake.
- Expertise in Kubernetes, Infrastructure as Code, and Terraform.
- Strong knowledge of Azure DevOps.
- Preference for simplicity and transparency in solutions, avoiding unnecessary complexity.
- Innovative and proactive approach, driving forward best practices and delivering effective solutions.
What We Offer:
- Enhanced holiday package to support work-life balance.
- Flexible hours and hybrid working options.
- Access to our modern head office in Oxford, Oxfordshire-a great environment for collaboration and teamwork.
- Comprehensive benefits package, including private medical insurance, competitive pension, cycle-to-work scheme, volunteering days, death-in-service insurance, and sick pay.
- Social events to foster team connection and maintain a fun, engaging workplace culture.
- Career development support through a detailed performance management system.
In Technology Group Ltd is acting as an Employment Agency in relation to this vacancy.