Senior Spark Developer

Job Description

Recruiter: Lloyd Hughes

Career Grade: D

Internal Closing Date: 1st May 2026

Due to the sensitive nature of this role, you may be required to undergo DV (Developed Vetting) level Security Clearance (https://www.gov.uk/government/publications/united-kingdom-security-vetting-clearance-levels/national-security-vetting-clearance-levels). An allowance of £5k per annum may be payable monthly while you hold this DV and continue to work in a role that requires that level of security clearance. The terms of this allowance will be made available if you are successful in being recruited into this role.

Security isn’t always the first thing that comes to mind when you think of BT, but when it comes to keeping everyone safely connected, We Are The Protectors. We deal with thousands of cyber-attacks every day, so that millions of people can safely go about their daily lives and run their businesses. We deliver vital work at scale, with real breadth and impact. We connect for good.
You'll be joining a specialist security team that is a trusted partner to governments worldwide, protecting critical national infrastructure and committed to the safety and security of our nation and global communities. Our mission focused work is innovative, inspiring and technologically challenging in a way that makes every day different and stimulating. We provide the opportunity to work on rare projects, with exciting tools and brilliant people. Everyone has access to unparalleled professional and personal development opportunities and your contribution is always valued.

About this role

• This role sits within the Cyber Processing & Data Science engineering team, supporting the development of data processing capabilities that underpin critical cyber security services. You will contribute to the design and delivery of highquality Sparkbased solutions that are reliable, scalable and maintainable.
• As a Senior Spark Developer, you will be a hands-on engineer with strong knowledge of Apache Spark and its surrounding ecosystem. You will demonstrate technical leadership through the quality of your work, applying engineering best practices, improving how solutions are built and operated, and supporting others through collaboration and knowledge sharing. The role values curiosity, innovation and a clear focus on delivering reliable outcomes for customers.

This role requires 5 days a week onsite. Mon - Fri 37.5 hours per week. DV Security Clearance is required. Must have lived in the UK for 10+ years to obtain clearance.

What you’ll be doing

• Build highquality Spark solutions that customers can rely on
• Design, develop and maintain welltested Spark applications and data pipelines. Ensure solutions are robust, performant and maintainable by applying agreed engineering standards and best practices.
• Support continuous improvement through technical leadership
• Lead development tasks by contributing to design discussions, reviewing code and sharing practical knowledge. Challenge inefficient ways of working, introduce improvements where appropriate, and support other engineers by demonstrating good practice and explaining technical concepts clearly.

Essential Experience

• Strong hands-on experience developing Apache Spark applications using Scala
• Good working knowledge of the Spark ecosystem, including Spark SQL and DataFrames
• Experience building and supporting data pipelines and ETL processes
• Solid understanding of software engineering best practice, including testing, code quality and version control
• Experience working with containerised platforms such as Docker and Kubernetes or OpenShift

Desirable Experience

• Experience with Kafka for data ingestion
• Experience with workflow or orchestration tools such as Airflow or NiFi
• Familiarity with cloud object storage (e.g. Azure Blob Storage or S3)
• Experience using monitoring and observability tools such as Prometheus, Grafana or Spark UI
• Working knowledge of Linux environments and basic scripting (e.g. Bash or Python)

Our Package

Tailored benefits make a real difference. That’s why we offer a comprehensive range to support your growth, wellbeing, and everyday life. You can design the package to suit you and your lifestyle. Your core benefits include:

• 10% on target annual bonus
• Access to an online private GP 24/7 for you and your immediate family
• Market-leading paid carers leave with up to 2 weeks off
• Equalised maternity, paternity, and adoption leave – 18 weeks’ full pay and 8 weeks’ half pay
• Discounted EE and BT products, including mobile and broadband
• Market leading Pension scheme – 5% from you and 10% from us
• Holiday purchase scheme
• You can select additional benefits, including healthcare, dental, gym memberships and more when you’re ready.

With over 180 years of heritage, BT Business has brought together our best people and capabilities into a B2B powerhouse serving 1.2 million business customers in the UK

We’re a leader for secure connectivity and collaboration platforms for businesses of all shapes and sizes, from big household names and government departments, right through to sole traders and new start-ups. But it’s not just the technology that matters, it’s what it can do to help them build stronger, smarter, more secure businesses.

We are Customer-First Challengers, who are Committed, Clear and Connected. We value diversity and inclusion and believe in making a positive impact. We connect for good by championing digital inclusion and equipping people, businesses, and communities with digital skills to thrive.

As a member of our team, you will be part of an organisation that celebrates difference, fosters innovation and provides you with opportunities to be your best. With millions of businesses relying on us daily, joining BT means you can be part of a diverse and multi-skilled team that makes a significant impact to society.