Defining the Data Fabric
Understanding the value of the "Meta" Data Architecture
I just attended a data conference and realized that the concept of a data fabric is not yet well understood. Furthermore there are recurring discussions about this topic on LinkedIn like recently by Piotr Czarnas or Bill Inmon’s Part I and Part II of “Data Fabric and Reality”.
As I also just talked about at TDWI Munich I want to share some insights based on the presentation.
When we watch for a definition we especially can find descriptions by the leading tech market research companies:
Fig. 1: What is the Data Fabric?
As a result for myself I found a kind of working definition for Data Fabric - “Connected services to (virtually) connect and intelligently provision of heterogeneous and distributed data”.
Today nothing is easier than to discuss your thoughts with the world. Therefore I posted my initial thoughts about what a Data Fabric is to LinkedIn, getting a lot of different perspectives on the topic:
Fig. 2: Perception of “Data Fabric” from experts on LinkedIn
Data Fabric & Data Mesh - better together?
Compared to Data Mesh, Data Fabric seems to be a very unclear concept in the market, driven by market research, software vendors and field experience. But even if vendors like SAP or Microsoft play with the term “Fabric” in their solutions, it make things even more unclear. I would consider both as a kind of “super” or “meta” data architectures with which you have an approach to handle decentralized data landscapes for common goals. The alternative would be to have a bunch of individually managed operational data stores, data lakes, data warehouses and data lakehouses with a very fragmented version of truth and high barriers to really make use of your companies data.
You will find many explanations about Data Mesh in the web. Here I put together the both approaches, what makes them unique, and the common goals they share:
Fig. 3: Data Fabric & Data Mesh - alternatively or jointly?
While Data Fabric is rather putting a central solution on a decentral landscape to make data access easier, Data Mesh is a federated approach where domains (business departments) work together based on common standards, a federated governance process and loosely coupled data products instead of a monolithic architecture.
Both approaches have similar goals like breaking down data silos, scale into the organisation and increase data quality in complex decentralized organisations. They represent just different ways how to do it.
What a Data Fabric is - NOT!
After all, possibly it makes sense to formulate what NOT is a Data Fabric.
If you have to copy all your data in a central data repository, it is fine, but not a Data Fabric
If you use virtualization to make access to different separated data repository, it is fine, but not (yet) a Data Fabric
Just if you name a product or solution something with “Fabric”, it is fine, but not automatically a Data Fabric
If you extend your central data platform with virtualization to integrate a new data source, it is fine, but not a Data Fabric.
I also discussed this perspective on LinkedIn and invite everyone to share additional experiences and views.
A final word about business value. Often when discussing concepts like Data Fabric or Data Mesh I hear, it is not worth to search for a definiton of or discuss Data Fabric (or other data architecture paradigms), because it is more important to care about creating business value. Sure it is. Understanding a concept means to be able to make use of it, which means creating value from data and finally business value.
This is the first part of a blog series about data fabric. Stay tuned to dive deeper in Data Fabric capabilities, relevant vendors and more!