Data Contract - Q&A

Lessons from discussions about what a Data Contract is

A Data Contract is a formal agreement between a data producer and a data consumer.

Fig. 1. - How a Data Contract works (adapted from Data Contract Specification¹)

One of my most successful posts² this year is one about some take aways about Data Contracts from a data conference. As a result I got not only > 119.000 impressions and around 1.000 engagements. I also had around 80 commentaries within a week. Here are what I learned from the discussions:

1. Does a Data Contract need to be in YAML format?

   —> No. Other formats as JSON, PACT or CUE³ where discussed. But finally it seems that all current implementations use YAML. Finally it has to be machine readable to enaple automation.

2. Why is a Data Contract not a contract?

   —> Based on this post⁴, a Data Contract is not a contract in a legal sense. It is an offer if both partys (data provider and data consumer) sign them. Data Mesh Manager name this then a "data usage agreement”.

3. Is a Data Contract an API?

   —> Technically a Data Contract is an API specification of a data set. It has it's orgin in API contracts or interface contracts.

4. Is a Data Contract something new or was already there under another name?

   —> Many orgins like Design by Contract or Pact⁵ can be referenced. But Data Contract goes beyond. According to Andrew Jones⁶ “The primary influence for data contracts was APIs, and interfaces more generally.“

5. Must a Data Contract be provider-oriented?

   —> Yes. Consumers define requrements, the producer has the ability to meet those requirements and put a contract around the delivery.

6. What is a Data Sharing Agreement (DSA) and how does it correspond to Data Contract?

   —> Data sharing agreements are an extension of data contracts. The agreements outline data usage, privacy, and purpose, including any limitations. Data sharing agreements are interface independent and offer insights into what data is used for a particular purpose.⁷ While a Data Contract supports the technical side, DSA supports the legal side and can be connected to a Data Contract.

This can be subject to extend or change over time. Feel free to contribute typical questions you hear in the comments.

1. https://datacontract.com/

2. Insights & discussions about Data Contract - https://www.linkedin.com/posts/peter-baumann-7b945438_barc-datacontract-dataproduct-activity-7250792370038804480-l9mI

3. https://cuelang.org/

4. Post describing why a Data Contract is not a contract - https://www.linkedin.com/posts/datamesh-manager_what-is-a-data-contract-activity-7134798065495859202-aSJ7/

5. https://docs.pact.io/

6. Interview with Andrew Jones, orginator of “Data Contract” - https://agiledata.io/podcast/data-contracts-with-andrew-jones/

7. Microsoft documentation about Data Contract and Data Shareing Agreements - https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/cloud-scale-analytics/architectures/data-contracts