Mid to Senior Data Engineer (Databricks)

Job Description

We help the world run better
At SAP, we keep it simple: you bring your best to us, and we'll bring out the best in you. We're builders touching over 20 industries and 80% of global commerce, and we need your unique talents to help shape what's next. The work is challenging – but it matters. You'll find a place where you can be yourself, prioritize your wellbeing, and truly belong. What's in it for you? Constant learning, skill growth, great benefits, and a team that wants you to grow and succeed.

The development center of SAP in Bulgaria plays a key role in the defining and developing of the SAP Business Technology Platform. With its more than 2000 professionals, SAP Labs Bulgaria also has strong contributions toward life-cycle management, user interface & user experience across the broader portfolio of SAP products and business applications. Most recently, SAP Labs Bulgaria expanded in the topics of artificial intelligence and cybersecurity. For its 25-year history, the company has established itself as a preferred employer in the IT sector in Bulgaria.

What you’ll build

We are looking for a Data Engineer to design, implement, and maintain scalable data transformation pipelines in Azure Databricks, working with data stored in Azure Data Lake Storage Gen2 (ADLS). The role focuses on transforming crawled data (ingested by another team) into curated, structured datasets and ensuring that these are delivered to downstream systems such as Coveo and OpenSearch for search and discovery.

Key Responsibilities

• Develop, optimize, and maintain data transformation pipelines in Databricks (PySpark).
• Work with data stored in ADLS Gen2 (primarily Delta/Parquet format).
• Implement and maintain data quality checks (schema validation, deduplication, enrichment, tagging).
• Ensure data is prepared and delivered correctly to Coveo and OpenSearch via Databricks integrations.
• Collaborate with QA to support testing feedback.
• Monitor and troubleshoot data pipelines for performance, scalability, and reliability.
• Document transformations and operational procedures.

What you bring

• Hands-on experience with Databricks (PySpark, Delta Lake).
• Knowledge of Azure Data Lake Storage Gen2 (structure, permissions, data formats like Parquet/Delta).
• Ability to implement ETL data pipelines at scale.
• Performance tuning for large-scale datasets.
• Proficiency in Python; experience with CI/CD for data pipelines is a plus.
• Familiarity with cloud environments (Azure preferred).

Nice to Have

• Knowledge of Coveo APIs and/or OpenSearch ingestion patterns.
• Experience with data governance and cataloging in Azure ecosystem (Unity Catalog)

Where you belong

CPIT Product, Data and Technology organization has the purpose to “Deliver and Run the Intelligent Enterprise”. CPIT is SAP’s technology backbone and most compelling reference customer, and as such, we influence SAP product development and strategy, with our customer experience at heart. Our organization has made great progress towards infusing SAP’s business processes with AI where high-quality data products are fueling the adoption of AI on enterprise level, making sure that we deliver the right answers to the right people. The Enterprise AI CoE is now developing the next generation of Intelligent Search Architecture to provision SAP internal/external unstructured data into the CPIT DataLake with a common metadata model, enabling various AI, application and analytics use cases

Bring out your best
SAP innovations help more than four hundred thousand customers worldwide work together more efficiently and use business insight more effectively. Originally known for leadership in enterprise resource planning (ERP) software, SAP has evolved to become a market leader in end-to-end business application software and related services for database, analytics, intelligent technologies, and experience management. As a cloud company with two hundred million users and more than one hundred thousand employees worldwide, we are purpose-driven and future-focused, with a highly collaborative team ethic and commitment to personal development. Whether connecting global industries, people, or platforms, we help ensure every challenge gets the solution it deserves. At SAP, you can bring out your best.

We win with inclusion
SAP’s culture of inclusion, focus on health and well-being, and flexible working models help ensure that everyone – regardless of background – feels included and can run at their best. At SAP, we believe we are made stronger by the unique capabilities and qualities that each person brings to our company, and we invest in our employees to inspire confidence and help everyone realize their full potential. We ultimately believe in unleashing all talent and creating a better world.

SAP is committed to the values of Equal Employment Opportunity and provides accessibility accommodations to applicants with physical and/or mental disabilities. If you are interested in applying for employment with SAP and are in need of accommodation or special assistance to navigate our website or to complete your application, please send an e-mail with your request to Recruiting Operations Team: Careers@sap.com.

For SAP employees: Only permanent roles are eligible for the SAP Employee Referral Program, according to the eligibility rules set in the SAP Referral Policy. Specific conditions may apply for roles in Vocational Training.

Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, ethnicity, gender (including pregnancy, childbirth, et al), sexual orientation, gender identity or expression, protected veteran status, or disability, in compliance with applicable federal, state, and local legal requirements.

Successful candidates might be required to undergo a background verification with an external vendor.

AI Usage in the Recruitment Process

For information on the responsible use of AI in our recruitment process, please refer to our Guidelines for Ethical Usage of AI in the Recruiting Process.

Please note that any violation of these guidelines may result in disqualification from the hiring process.

Requisition ID: 436081 | Work Area: Software-Design and Development | Expected Travel: 0 - 10% | Career Status: Professional | Employment Type: Regular Full Time | Additional Locations: #LI-Hybrid