Data Integration Engineering Specialist

Job Description

Recruiter: Sowmya Reddy

Hiring Manager: Nischita Shetty

Career Grade: D

Internal Closing Date: 31st December, 2025

Why this job matters

BT is in the midst of a major transformation – with data and AI at the heart. We’re simplifying our products, transforming our customer journeys, modernizing our IT and networks and decommissioning legacy platforms.

As part of this ambition, it is essential that we take advantage of the latest technological advances in AI to deliver for the business while ensuring we cohere to security and privacy concerns. This job is vital in delivering on our ambition by automating processes and freeing resources to focus on value add activity.

The Principal AI Enterprise Architect will lead major pivotal initiatives and architectural governance in the AI domain, ensuring alignment with BT’s enterprise architecture and business goals.

What you’ll be doing

Architecture & Design

• Design and implement Kafka-based streaming solutions for high availability and fault tolerance.
• Define topic structure, partitioning, replication, retention, and compaction policies.
• Architect stream processing pipelines using Apache Flink for real-time analytics.
• Implement disaster recovery, geo-replication, and security (TLS, SASL, RBAC, ACLs).
• Hands on experience in setting up confluent kafka on On-prem and cloud platforms
• Optimize cloud resources for cost efficiency and performance.
• Implement Dynatrace for end-to-end monitoring of Kafka clusters and streaming jobs.
• Configure dashboards, alerts, and performance metrics for proactive issue detection.
• Build producers, consumers, and connectors using Kafka Connect.
• Integrate with data lakes, databases, and APIs across hybrid environments.
• Build POC's for different solutions.

Collaboration

• Work with DevOps teams for infrastructure automation (Terraform, Ansible).
• Mentor teams on Kafka best practice and streaming architecture.

The skills you’ll need to succeed

• Core Technologies: Confluent/Apache Kafka, Kafka Streams, Kafka Connect, ksqlDB, Schema Registry, Ab initio,
• Stream Processing: Apache Flink, Spark Streaming (optional)
• Cloud Platforms: AWS (MSK, Lambda, S3), GCP (Pub/Sub, Dataflow, BigQuery).
• Monitoring Tools: Dynatrace, Prometheus, Grafana.
• Containerization: Kubernetes, Docker, Helm
• Programming: Java, Scala, Python.
• Infrastructure: Terraform, Ansible.
• Security: TLS, SASL, RBAC, ACLs.
• Data Integration & Messaging: Ab Initio (Continuous Process Integration), RabbitMQ, IBM MQ, Ab Initio Queue
• Preferred: Confluent certifications, experience with CDC patterns and event sourcing.
• Soft Skills:
- Strong problem-solving and communication skills
- Ability to lead architecture discussions and mentor teams
- Experience in cross-functional collaboration

Our leadership standards

Looking in:
Leading inclusively and Safely
I inspire and build trust through self-awareness, honesty and integrity.
Owning outcomes
I take the right decisions that benefit the broader organisation.

Looking out:
Delivering for the customer
I execute brilliantly on clear priorities that add value to our customers and the wider business.
Commercially savvy
I demonstrate strong commercial focus, bringing an external perspective to decision-making.

Looking to the future:
Growth mindset
I experiment and identify opportunities for growth for both myself and the organisation.
Building for the future
I build diverse future-ready teams where all individuals can be at their best.

About us

BT Group was the world’s first telco and our heritage in the sector is unrivalled. As home to several of the UK’s most recognised and cherished brands – BT, EE, Openreach and Plusnet, we have always played a critical role in creating the future, and we have reached an inflection point in the transformation of our business.

Over the next two years, we will complete the UK’s largest and most successful digital infrastructure project – connecting more than 25 million premises to full fibre broadband. Together with our heavy investment in 5G, we play a central role in revolutionising how people connect with each other.

While we are through the most capital-intensive phase of our fibre investment, meaning we can reward our shareholders for their commitment and patience, we are absolutely focused on how we organise ourselves in the best way to serve our customers in the years to come. This includes radical simplification of systems, structures, and processes on a huge scale. Together with our application of AI and technology, we are on a path to creating the UK’s best telco, reimagining the customer experience and relationship with one of this country’s biggest infrastructure companies.

Change on the scale we will all experience in the coming years is unprecedented. BT Group is committed to being the driving force behind improving connectivity for millions and there has never been a more exciting time to join a company and leadership team with the skills, experience, creativity, and passion to take this company into a new era.

A FEW POINTS TO NOTE:

Although these roles are listed as full-time, if you’re a job share partnership, work reduced hours, or any other way of working flexibly, please still get in touch.

We will also offer reasonable adjustments for the selection process if required, so please do not hesitate to inform us.

DON'T MEET EVERY SINGLE REQUIREMENT?

Studies have shown that women and people who are disabled, LGBTQ+, neurodiverse or from ethnic minority backgrounds are less likely to apply for jobs unless they meet every single qualification and criteria. We're committed to building a diverse, inclusive, and authentic workplace where everyone can be their best, so if you're excited about this role but your past experience doesn't align perfectly with every requirement on the Job Description, please apply anyway - you may just be the right candidate for this or other roles in our wider team.

Please note that as per the IJP guidelines for India if you have an active PIP/Disciplinary sanctions then you cannot apply for a role internally for the duration of that sanction. In circumstances when you are in a disciplinary process but the outcome hasn’t been reached or there is an active appeal then you can apply, however, the Offer could be withdrawn if, subsequently, you’re issued with a sanction or your appeal is unsuccessful.
It is the responsibility of an applicant to not apply to IJP in case if they are on an active PIP/ Disciplinary case/ Warnings or sanction. If such an applicant is found to have applied for a role, then, strong disciplinary action will be taken against that applicant.