Ready for Takeoff with Data Mesh

Welcome to our data journey!

With this series of data blog posts, we'll show you how we put data at the core of what we do. You'll learn why we decided to set up data mesh right from the start and how we built up our modern data stack.

^[1]

Good old greenfield

When we started our journey at the beginning of 2022, we saw a lot of departments at Volocopter growing fast, with rising demand to put their data into action. However, we were lacking common processes, standards, and the tools needed to scale data use cases across the company. That's when we decided to put our heads together and seize this unique opportunity to build a data platform that could support us as we bring urban air mobility to life.

As data engineers, we get extremely excited whenever we can start from the greenfield. But in this case, the challenge was not only to build the data platform, we also had to find an organizational setup to support our various use cases across different domains.

In a nutshell, while we stood there looking at the beauty of the good old greenfield, we identified a couple of major challenges:

• The obvious one: As exciting as starting on a greenfield might sound, designing and implementing everything from scratch also comes with a lot of work. Architecture workshops, technology decisions, defining standards, infrastructure deployment, data modeling ... you name it.
• Not only did we need to solve the technical challenges, we also had to set up an organizational structure that would allow us to simultaneously scale for multiple use cases across multiple domains. Only then could we support all the growing departments as they turned their data into action.

To face these challenges, we adopted a strategic approach that allowed us to ramp up multiple data initiatives at once. But we also wanted to leverage synergies across the different domains.

Data Mesh as guiding principle

Many of us already had experience with team setups, with a centralized data team handling various requests from different domains. Additionally, these teams are typically responsible for all cloud infrastructure used for setting analytical use cases. Nevertheless, we didn't see how this could be a great choice for us here at Volocopter. That's mainly because we assumed that a centralized team would have had a less-than-enjoyable working experience.

Instead, they would almost instantly become a bottleneck. Requirements from all departments would have rained down on them at the same time while they'd be trying to gather, order, and prioritize their tasks. What's more, Additionally, it would have required a constant mental shift from one unique domain challenge to the next. And, they would have been busy building the data platform to support all of those use cases from the ground up.

Our strategy ultimately needed to include an approach for scaling the team. Luckily, we didn't have to search for very long to find a framework that addresses those challenges. In Data Mesh we found the guidance we needed to become autonomous teams. Teams that apply a product mindset to their data challenges, leverage synergies via a centralized data platform across domains, and compile a governance framework that provides guidance and safety when working with data.

Taking on challenges as a community

At Volocopter, we took autonomous team responsibilities one step further. While many setups still feature a centralized platform team that is required to build and maintain the data platform, we focused on a self-acting community. We call this data community a community of practice (CoP), and it includes individuals from every domain team who come together regularly to find solutions to every team's challenges. At the same time, every domain team contributes a new feature to the data platform, tries out new technologies, contributes to our tech radar, or publishes new guidelines on how to work with different data workloads. All of this is supported by the Volocopter DevOps team, which brings its expertise to the table to create scalable, reliable, and maintainable cloud infrastructure.

Stay tuned for future articles, where we'll be showing you how this data strategy supports specific data products that help us to bring urban air mobility to life. We'll also be sharing insights into how we adapted the four principles of data mesh domain ownership, data as a product, self-serve data platform, and federated computational governance. Moreover, you will get a glimpse at our architecture, learn how different teams scale within our data mesh setup, and find out about some of our learnings along the way.

See you for the next leg of our blog journey!