Conway's Law for Data Strategy
Is Conway always right? Be water my friend!
In the following I will dive into Conway's Law and the meaning for Data & Analytics. If you use it right, it can have an incredible effect making your data strategy succesful.
This article, similar to Ashby's Law for Data Strategy, and Amara’s Law for Data Strategy shouldn’t be seen as the one and only truth. It is rather a mind game about how you can make patterns work for you.
Conway’s Law became very popular in Data & Analytics in the last years as we learned more about going away from fully centralized data architectures. Especially Data Mesh pushed it. It is cited as follows:
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
Often when we speak or hear about, there is a concept based on this called Inverse Conway Maneuver:
The 'Inverse Conway Maneuver' recommends evolving your team and organizational structure to promote your desired architecture. Ideally your technology architecture will display isomorphism with your business architecture.1
Picture: Generated by ChatGPT
But what does this mean for todays data teams?
Let’s take a super simplified example and imagine you have a central 10 people data team as part of the IT department. There are a team with 5 people for the backend and a team with 5 people for the frontend.
Typically you have a strong communication within each team. You will build a strong integrated (monolithic) data integration architecture aka Data Warehouse ans support a single BI solution.
The communication from business is based on a ticket system for support, changes and project (e. g. Jira) where requirements are gathered, priorizised and finally implemented by the team if ressources are available or external partners.
Let’s assume Dev and Ops is separated and within every team of 5 there are 2 people for support and changes and 3 people for building projects. The 2 Teams are completely responsible for everything on there side End-to-End.
Now you discover Conway’s Law and after thinking about you understand that your current organisation is limited in self-service for the users as the systems are complex and monolithic and build for experts. Furthermore it is technology-centered what optimized on costs, but not on value. Third the team has no domain expertise and implementing solutions need a lot of documentation and project-specific communication. Forth your teams have tasks which need business responsibility like to care about data quality problems or handle authorizations and data usage.
A solution on all of this are Data Products. They support higher autonomy, self-service and business responsibility. Furthermore the complexity of the single Data Product is limited. What you need is the right Data Plaform which separates technical tasks from creating value with your business. Then everything is nice. Is it? You also learned about the Inverse Conway Maneuver. You get the idea to deliver better, your team has to adapt to the technology to create a better socia-technical approach.
You change your team-structure to flexible end-to-end (DevOps, Front & Backend) teams. Let’s assume you always have 2-people teams (1 frontend, 1 backend) working strongly together with 1-3 business domains (like logistics, sales, production, …). So you can serve for a maximum of 12 business domains. If you need more, you will go external, if you have larger projects you get flexible with external experts or temporarily switching of teams between business domains.
Excurse - going external means having a certain maturity in your organisation like clear guidelines, processes, governance and responsibilities. If not you will build technical debt very easy.
In my calculation 2 people/1 team is left. You will probably need a Data Platform Team which cares for overall responsibilities and central tasks you can not give into a domain responsibility.
So now you enabled self-service, increased business-orientation, and what is maybe most important you have changed the communication structure in a more business-aligned way.
This sounds easy so far, but it isn’t. To really understand communication structures in a company is not easy. They are often not formal. This means even if you change the team/organisation it will probably fail. Furthermore switching teams without really switching your architecture or trying to build it on your old one will probably fail. There are 1.000 ways to do what I described and sometimes we talk about Data Mesh to give it a name. But we have to understand it is about what fits to your organisation and creates the highest value, not about a name or the ‘one way’ to do something.
Tell my what do you think about applying Inverse Conway Maneuver and if you ever experienced such a change. Does it work for you?
Inverse Conway Maneuver explained in Thoughworks Technology Radar -w https://www.thoughtworks.com/radar/techniques/inverse-conway-maneuver