Data Engineer Ref. 3715

Job Description

Salary: £66,332. This includes a £3,000 non-consolidated market rate supplement. There is also an opportunity to qualify for additional allowances and a skills-related payment under the Government Digital and Data Profession Capability Framework (formerly the Digital Data and Technology (DDaT) Profession Capability Framework.)

Flexible Working: We offer a strong work-life balance, supporting full-time, part-time, and flexible hours. You’ll be predominantly office-based, with working from home limited and subject to business need.

About us

MI5 keeps the country safe from serious threats like terrorism and attempts by states to harm the UK, its people and way of life. We carry out investigations by obtaining, analysing, and assessing intelligence, and then work with a range of partners including MI6 and GCHQ to disrupt these threats. Through our protective security arm, we provide advice and guidance to government, businesses, and other organisations about how to keep themselves safe. A role in MI5 means you’ll do unique and challenging work in a supportive and encouraging environment, making a real difference to UK national security.

The role

Technology sits at the heart of everything we do at MI5. We're looking for people with hands on experience with data engineering: at its core designing and managing data pipelines and storage, be it in the extraction, transformation, integration or loading of multiple sources of data, to take on core roles within a multi-disciplinary product team. As a data engineer, you will work alongside other engineering disciplines, product owners, business analysts, delivery managers and data scientists, to build and run secure data applications and data products.

You’ll use and promote Agile technical practises and methodologies to deliver digital products that are core to MI5’s operation, making use of modern standards and approaches whilst using cloud technologies, as well as on-premises platforms and long-established technologies and frameworks.

You’ll gain experience in a fast-paced environment as part of a forward-thinking team and strive to develop elegant digital solutions. Managing tasks, you’ll be responsible for the design and implementation of moderate-to-complex data flows, to connect operational systems, data for analytics, or business intelligence (BI) systems.

Ensuring our customer’s needs are represented at every stage, you’ll support the products owned by the team, working with users to deliver new features, identify and fix defects and develop automated tests to maintain the assurance of our products.

You’ll frame problems and set priorities for yourself and your team, taking ownership of large problems, breaking them down and working with the team to deliver new features throughout the engineering lifecycle. As a Data Engineer, you’ll focus on creating professional and comprehensive digital experiences and share a collective responsibility in keeping the country safe, by staying one step ahead in an evolving digital world.

You’ll help set the standard for what good looks like, sharing digital and innovative ways of working with the team and across the skills community. Importantly, you’ll have a passion for sharing knowledge, whether it’s a customer or colleague - you’ll help them to understand what great engineering looks like. You’ll also participate in our Communities of Practice, and encourage cross-organisation initiatives to help build our community of engineers, whilst mentoring and developing junior members of the team.

Our tech stacks vary between products using a range of databases including SQL and No-SQL and commercial and custom-built technologies. We embrace a DevSecOps (Development, Security and Operations) mindset, allowing our teams to automate, run and support what we build in a secure manner.

Whether it’s building a platform to enable the analysis of data systems to track the progress of our investigations, ingesting new complex partner data, developing robust and scalable capabilities for our Data Science teams, delivering compliance services to ensure we are operating legally, or tooling to improve our developer experience, the nature of our work presents unique and fascinating engineering challenges unique to MI5.

About you

You’ll be able to communicate information clearly and effectively, influencing technical and non-technical colleagues. You’ll be able to define and shape problems while working with a wide range of stakeholders to meet customer needs. You’ll look to build leadership qualities, supporting others to upskill and build their career in an inclusive environment, while continuously looking for opportunities to develop and learn new engineering practises and approaches.

You’ll be able to demonstrate a proficiency in developing data solutions and data integration designs. Crucially, you’ll be comfortable learning and using a broad range of technologies whilst utilising modern approaches, such as the use of continuous integration and delivery (CICD), and use of cloud services. You’ll have an awareness of good design patterns and how to implement them appropriately with security in mind, and demonstrate proficiency in working with an agile mindset.

You’ll have experience delivering moderate-to-complex data flows as part of a development team in collaboration with others. You’ll be confident using data processing and integration technologies for example any of: AWS Glue, Fivetran, Airbyte, Apache NiFi, SAS DI Studio, Apache Spark or other equivalents.

You can implement, deliver, and translate across common data models, for example across unstructured data, document formats, tabular and relational models, or formats to support geo-enabled applications or graph database structures as well as knowledge using their equivalent differing data file formats for example JSON, XML, CSV, Parquet, Avro, etc. You can also understand how to manage metadata across your development teams systems and beyond, with a practical knowledge of tracking data provenance and data lineage.

You’ll work to agreed and recognised standards to build solutions using various traditional or big data languages such as: Python, Java, SQL, Scala, PL/SQL (Oracle), SAS Macro Language, etc. whilst using various data tools for example Jupyter, Hue (Hive/Impala), Kibana (Elastic Search), Sagemaker (AWS), SAS for example.

Training and development

We’re committed to developing our people and recognise this will look different for everyone. We use the Government Digital and Data Profession Capability Framework (formerly the Digital Data and Technology (DDaT) Profession Capability Framework) as a compass to provide guidance and tools to support you, and which recognises and rewards specialist expertise. Find out more about the framework here: Government Digital and Data Profession Capability Framework.

There’ll be opportunities to gain qualifications and pursue specialist pathways, as well as undertaking tailored training, coaching, and mentoring. Individuals can also attend hackathons and have up to 20% innovation and personal development time.

As you develop, grow, and maintain your technical skills, you can progress through the Government Digital and Data pay points. You’ll need to regularly demonstrate and accredit your skills based on the framework. If you don’t maintain the required skills, you’ll be given the opportunity and support to rectify this, as you could revert to a lower level of capability and pay.

Rewards and benefits

You’ll receive a starting salary of £66,332 dependent on skills, experience, and location plus other benefits including:

• 25 Days Annual Leave automatically rising to 30 days after 5 years’ service, and an additional 10.5 days public and privilege holidays.
• opportunities to be recognised through our employee performance scheme
• dedicated development budget
• interest-free season ticket loan
• excellent pension scheme
• cycle to work scheme
• facilities such as a gym, restaurant, and on-site coffee bars (at some locations)
• paid parental and adoption leave.

Equal opportunities

At MI5 diversity and inclusion is critical to our mission. To protect the UK, we need a truly diverse workforce that reflects the society we serve. This includes diversity in every sense of the word: those with different backgrounds, ages, ethnicities, gender identities, sexual orientations, ways of thinking and those with disabilities or neurodivergent conditions. We therefore welcome and encourage applications from everyone, including those from groups that are under-represented in our workforce such as women, those from an ethnic minority background, people with disabilities and those from low socio-economic backgrounds.

Find out more about our culture, working environment and diversity on our website.

We’re Disability Confident

MI5 is proud to have achieved Leader status within the Department for Work and Pensions' Disability Confident scheme. This is aimed at encouraging employers to think differently about disability and take action to improve how they recruit, retain and develop disabled people. As a Disability Confident Leader, we aim to ensure that a fair and proportionate number of disabled applicants who best meet the essential minimum criteria for this position, will be offered an interview, if it is practical for us to do so. (This is known as the Offer of an Interview.) To secure an interview for this role, the minimum criteria (in order of application process) are:

• you will be required to reach the minimum pass mark for the Organisational Situational Judgement Test which looks at your ability to problem solve. If you meet this criteria, you will be directed to complete an application form
• able to demonstrate your ability in developing data solution (assessed at application form sift)
• can demonstrate an understanding of the principles of modern standards approaches such as continuous integration and delivery, test driven development and cloud services (assessed at application form sift)
• able to demonstrate taking on a more senior role within a team. Provide technical direction and the ability to guide and support others with regards to developing data products and solutions (assessed at application form sift)

There is a wide range of extra support available throughout the recruitment process to enable you to do your best, see our website for information on reasonable adjustments we can offer.

What to expect

Our recruitment process is fair, transparent, and based on merit. Here is a brief overview of each stage, in order:

1. You’ll be invited to upload your CV and complete an online application form.
2. Online Situational Judgement Test (SJT) rating the appropriateness of your response to a series of short scenarios. The SJT tests criteria important for all roles in our organisation.
3. If you pass, you’ll be invited to an interview where you will be asked questions relating to your motivation and our core competencies.
4. If you pass, you’ll be invited to a technical skill-based interview based on the Government Digital and Data Capability Profession framework for the role.
5. If successful you will receive a conditional offer of employment, subject to vetting.

Please note, you must successfully pass each stage of the process to progress to the next. Your application may take around 6 - 9 months to process including vetting, so we advise you to continue any current employment until you have received your final job offer.

Before you apply

To work at MI5, you need to be a British citizen or hold dual British nationality. You can read our full eligibility criteria here.

This role requires the highest security clearance, known as Developed Vetting (DV). It’s something everyone in the UK Intelligence Community undertakes. You can find out more about the vetting process here.

Please note we have a strict drugs policy, so once you start your application, you can’t take any recreational drugs and you’ll need to declare your previous drug usage at the relevant stage.

Before you apply, we advise you to consider setting up a separate email address for your contact with us, to ensure your personal and application correspondence remain separate.
Try to avoid having identifying features in your email address, such as your first and/or surname and date of birth. This is good practise and will help you to manage your application with us more discretely.

The role is based in Central London, so you’ll need to live within a commutable distance. Please consider any financial implications and practicalities before submitting an application. An interest-free loan via our benevolent fund is available to assist with relocating into privately rented accommodation to take up the offer of employment.

We offer reasonable reimbursement of travel costs for candidates attending in-person appointments during the recruitment and vetting process. Full details will be provided with your interview or assessment invitation.

Reimbursement is discretionary and will only be made in line with the Candidate Expenses Policy, as amended from time to time. Candidates must book their own travel, use the most economical option, and provide original hardcopy receipts for reimbursement.

Please note, you should only launch your application from within the UK. If you are based overseas, you should wait until you visit the UK to launch an application. Applying from outside the UK will impact on our ability to progress your application. You should not discuss your application, other than with your partner or a close family member.

Right to Withdraw Statement:

Please be aware that we withhold the right to bring forward the closing date for this role from the original closing date once a certain number of applications have been received. Please be mindful of this and submit your application at your earliest convenience to avoid disappointment.