Senior Data Engineer

Job Description

🌳 Infogrid began in 2018 on a mission to make buildings more sustainable, buildings account for 39% of global carbon emissions and we’re creating real solutions to impact this!
🚀We are still early in our journey but have already achieved a lot; Infogrid now operates across the globe with over 300 people helping to drive our mission forward.
The role
As a Senior Data Engineer at Infogrid, you will build real-time streaming data pipelines to create the richest and most extensive model of building health and energy usage in the world.
A passion for agility is critical for success: a focus on outcomes and a willingness to pivot and change to meet the next challenge is vital - you and the team will look to provide tooling that enables the company to scale with stability, predictability and evolve-ability.
Our tools and technology will need to scale too - and you will play a pivotal role in evolving our technical architecture, working with peers from across our engineering organisation. While growing our data pipelines by several orders of magnitude, our challenges ahead include implementing governance, security and ensuring data is reliable and accurate.
We currently work in AWS with tools like dbt, Snowflake, Dagster, Kafka, Python, but we’d like you to help grow our tech stack as we go. A healthy dose of pragmatism, an openness to change and a focus on outcomes is key!

What will you be doing? Glad you asked!

• Build, support and manage our data ingestion capability
• Work with the team to define requirements and rapidly iterate to build great products
• Manage data pipelines in a live production environment - taking ownership of bugs and errors and ensuring they are fixed quickly and with minimal negative impact
• Champion best practice in data modeling and usage. Understand the domain and the business and ensure our models are the right fit.
• Act as a domain expert, demonstrate the art of the possible and help stakeholders and peers build the best possible product.
• Build tools to govern retention, control access and ensure compliance with relevant regulations.

In order to do the above, what will I need to bring with me?

• Experience working with large-scale data systems at a detailed level.
• Good breadth of knowledge across data tech: SQL, relational query behaviour & optimisation, column-oriented DBs, NoSQL DBs, pipeline definition & orchestration (ETL as a special case), schemas & serialisation, events/messages/streaming.
• Comfortable with Python, pandas and notebook usage.
• Expert user of SQL, and of some pipeline tech.
• Familiarity with at least one major BI frontend tool (Tableau, Looker, Power BI) or with a notebook-based dashboard tool (Hex, Deepnote), ideally both.
• Experience with managing the path from large scale datasets to analytics or data apps, including data modelling, caching, and scalability to billions of rows, TBs of content.
• Familiarity with modern data and service architectures, and ability to guide architectural decision making.
• Experience with a modern public cloud, ideally AWS, and an understanding of the principles behind public infrastructure
• Familiarity with modern data tech, for example: column-oriented DBs, orchestration & monitoring of pipelines, caching and optimisation strategies.

Come save the world with us!
❤️ Feel good about your work: Infogrid is actively helping big companies reduce their environmental footprint. We also offset everyone’s carbon footprint.🤸 We’re flexible: We encourage remote working and offered flexible working hours before COVID and we’ll keep that flexibility afterwards.📚 Learn and get promoted: Join us at the start of our journey - we’re growing quickly and believe in promoting from within. You can learn and grow your career here.🌟 Improve yourself: We give everyone a day per month to focus on personal development.🧑🏽‍🤝‍🧑🏽 Make friends: Everyone here is lovely.
And we'll do everything we can to support you during your application. If you need us to make any adjustments to your recruitment process, speak to our recruitment team who will be happy to support you.