Platform Engineering Professional - Data Engineer

Job Description

Why BT
We’ve always been an organisation with a purpose; to use the power of communication to make a better world. You can trace this back to our beginning as pioneers of the world’s firs telecommunications company. At our heart we’re a technology company with research and innovation in our bones and a desire to be personal, simple, and brilliant for our customers - those are the values we live by whilst also creating an inclusive working environment where people from all backgrounds can succeed.Our pursuit of progress over the past 180 years has established BT as a strong, successful brand, with huge scale capable of achieving great things. From assisting emergency services, hospitals, banks and keeping economies around the world online, safe and secure, to delivering large scale technology infrastructure like the creation of BT Sport.
Today in this fast-changing, always on, digital world our purpose remains true. Yet the market conditions, regulations and competition we face are tougher than ever before. So, if you have the drive, optimism and resilience to help propel us forward we’ll offer unrivalled personal development, a wealth of opportunities to learn, experience new things and pursue new careers. If that’s you and what you’re looking for, we’d love you to be part of our future.

Platform Engineering Professional - Data Engineer

Why this job matters
Data is central to our business, and this role helps strengthen the engineering excellence behind our data products. As a Platform Engineer, you’ll contribute to building the judicious data platform for Networks using modern cloud technologies across Kubernetes On‑Prem, AWS, GCP, and Cloudera. You’ll join the Data & AI Enablement Team to create, build, and assist high‑quality engineering solutions sourced from extensive data across a multi‑cloud environment. Our team operates on agile values—transparency, transparency, and continuous improvement—fostering strong stakeholder relationships and providing opportunities to learn, innovate, and grow.

What you’ll be doing
Create, develop, and maintain end‑to‑end data pipelines using Ab Initio (GDE, PDL, Conduct>IT, Express>IT).
Build scalable batch and streaming workloads using Apache Spark (PySpark/Scala).
Develop and orchestrate real-time/near‑real‑time data ingestion flows using Apache NiFi.
Optimize ETL/ELT pipelines for performance and efficiency.
Perform tuning and bottleneck analysis for Spark, NiFi, and Ab Initio.
Deliver demos and knowledge sharing to stakeholders.
Enhance automated testing and deployments through CI/CD.
Continuously learn through internal and external training.

Skills you’ll need to succeed
Experience building Big Data solutions using Hadoop, GCP, or AWS.
Strong skills in Scala, Java, Python, Spark, Terraform, Kafka, Kinesis, BigQuery, Dataflow, SQL.
Solid understanding of Infrastructure as Code.
Proficiency with Kubernetes and CI/CD practices.
Familiarity with Confluence, JIRA, and version control (Git/SVN).
Ability to convey technical concepts effectively.
Curiosity and enthusiasm for learning new technologies.

Experience we expect
Big Data solution development using technologies such as Scala, Java, Python, Spark, Terraform, Kafka, Kinesis (and Analytics), BigQuery, Dataflow, BigTable, SQL.
Strong Ab Initio expertise (GDE, PDL, Conduct>IT, Express>IT, EME), including parallel processing and component optimization.
Advanced proficiency in Apache Spark (PySpark/Scala, SQL, Streaming, tuning).
Hands‑on experience with Apache NiFi (flow create, tuning, error handling, security).
Practical knowledge of CI/CD strategies.
Experience with Atlassian tools and version control (Git/SVN).
Ability to present technical issues and solutions clearly.
Keen interest in exploring and applying new technologies.

BT is part of BT Group, along with EE, Openreach, and Plusnet.

Millions of people rely on us every day to help them live their lives, power their businesses, and keep their public services running. We connect friends to family, clients to colleagues, people to possibilities. We keep the wheels of business spinning, and the emergency services responding.

We value diversity and celebrate difference. ‘We embed diversity and inclusion into everything that we do. It’s fundamental to our purpose: we connect for good.’

We all stick to the same values: Personal, Simple, and Brilliant. From day one, you’ll get stuck in to tough challenges, pitch in with ideas, make things happen. But you won’t be alone: we’ll be there with help and assist, learning and development.

This is your chance to make a real difference to the world: to be part of the digital transformation of countless lives and businesses. Grab it.

A FEW POINTS TO NOTE:

Although these roles are listed as full-time, if you’re a job share partnership, work reduced hours, or any other way of working flexibly, please still get in touch.

We will also offer reasonable adjustments for the selection process if required, so please do not hesitate to inform us.

DON'T MEET EVERY SINGLE REQUIREMENT?

Studies have shown that women and people who are disabled, LGBTQ+, neurodiverse or from ethnic minority backgrounds are less likely to apply for jobs unless they meet every single qualification and criteria. We're committed to building a diverse, inclusive, and authentic workplace where everyone can be their best, so if you're excited about this role but your past experience doesn't align perfectly with every requirement on the Job Description, please apply anyway - you may just be the right candidate for this or other roles in our wider team.