A rare opportunity for a highly skilled senior Azure data engineer to join a dynamic data engineering team at a national professional services firm at their Northeast Head Office.
The ideal candidate will have extensive experience in designing, implementing, and managing data solutions on Microsoft Azure. You will be responsible for developing and maintaining data architecture, migrating data acquired through M&A, ensuring data quality, and enabling advanced analytics capabilities to support business decision-making.
Reporting to the Group Head of Infrastructure & Security, you will oversee the performance and maintenance of the Azure Data Warehouse. Additionally, you will manage the regular data migration of acquired firms' back-office system data into a proprietary SQL database.
This role requires deep expertise with Azure data services, a strong understanding of data engineering best practices, and a passion for optimising data systems for performance and scalability.
Overview
As an experienced infrastructure data engineer, you will be 100% hands-on and will be responsible for the following:
- Ownership of the data migration from M&A into a proprietary SQL database
- Design and develop robust, scalable, and efficient data architectures using Azure data services.
- Collaborate with the business analysts, IT Ops/Dev, finance and other stakeholders to understand data requirements and translate them into technical solutions.
- Ensure through continuous maintenance and improvements that the data architecture meets the needs for data ingestion, storage, processing, and analysis.
- Ensure data security, data quality, and compliance with data governance policies.
- Ensure understanding and compliance with data governance and data quality principles
- Design robust data models that enhance data accessibility and facilitate deeper analysis.
- End to end ownership of the Data Engineering Lifecycle
- Build and maintain ETL/ELT processes to ingest data from various sources
- Implement and manage data integration solutions using Azure Data Factory, Azure Databricks, Azure Data Lake, and Azure Synapse Analytics.
- Monitor and troubleshoot data pipeline issues to ensure data availability and accuracy.
- Implement data transformation, cleansing, and aggregation processes to prepare data for analysis.
- Develop scripts and automation using PowerShell, Python, or other relevant languages.
- Design, implement, and manage databases and data models using Azure SQL Database, MySQL Database, Cosmos DB, and other Azure database services.
- Optimise database performance and ensure data integrity and security.
- Migrate existing on-premises databases to the Azure Cloud.
- Assist with monitoring and optimising the performance, reliability/availability, and costs efficiency of the Azure data solutions infrastructure and services.
- Document data processes, system designs, and architecture to ensure maintainability and scalability.
- Stay up to date with the latest industry trends and technologies to drive continuous improvement.
- Provide technical guidance and mentorship to other team members.
- Building and maintaining relationships with other IT teams.
- Contributing to continuous improvement and innovation initiatives.
- Involvement in solution design work.
- Contributing to product team delivery of brownfield and greenfield change initiatives.
- Taking ownership of delivery of work assigned.
Ideal Candidate
- Passionate about data and have a genuine interest and understand its power to drive business decisions. You are committed to ensuring data quality, accuracy, and availability.
- You have excellent analytical and problem-solving skills. You enjoy tackling complex data challenges and finding innovative solutions.
- You work well in a team environment and communicate effectively with both technical and non-technical stakeholders. You are open to feedback and value the perspectives of others.
- Enjoys autonomy and has the ability to work independently and is willing to take ownership of initiatives from inception to completion.
- You have a keen eye for detail and ensure that data is accurate, consistent, and reliable. You understand the importance of data governance and compliance.
- Strong communication and collaboration skills, with the ability to work effectively in a team environment.
- You stay current with the latest industry trends and technologies. You are eager to learn and continuously improve your skills and knowledge.
- You take initiative and are highly self-motivated. You can manage your workload efficiently and meet deadlines without compromising on quality.
- You thrive in a fast-paced, dynamic environment. You are flexible and can adapt to changing priorities and requirements.
- You are always looking for ways to optimise and improve data processes and systems. You embrace new ideas and technologies to enhance data engineering practices.
Skills & Qualifications
Required:
- 5+ Years of experience in database administration/data engineering, with a focus on Microsoft Azure.
- Microsoft Azure experience with a focus on: SQL, Azure Data Factory, Azure Data Bricks, Python/Pyspark, Azure Synapse Analytics, and MySQL on Azure.
- Microsoft Azure Certifications such as Azure Database Administrator, Data Engineer Associate.
- Expertise in SQL programming, including designing and optimising complex queries and databases.
- Strong understanding of data warehousing concepts and ETL processes.
- Experience with data modelling, data warehousing, building data pipelines and data integration techniques.
- Data warehouse development experience using Azure Synapse Analytics.
Desirable:
- Microsoft Azure Certifications such as Fabric Analytics Engineer Associate.
- Familiarity with data visualization tools like Power BI or Tableau.