Amara's Law for Data Strategy
Or why Data & Analytics benefits from a long-term view
In the following I will dive into Amara's Law and the meaning for Data & Analytics. If you use it right, it can have an incredible effect making your data strategy succesful.
This article, similar to Ashby's Law for Data Strategy, shouldn’t be seen as the one and only truth. It is rather a mind game about how you can make patterns work for you.
If you are not familiar with, here it is:
We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
- Robert Charles Amara, 2006
Picture: Generated by ChatGPT
This Law is well known in tech in the visual form of the Gartner Hypecycle.
I think it is well understood, that there are hypes in the data world about technologies like GenAI or concepts like Data Mesh and Big Data before, where everyone feels - this is or was a true hype. Why we often don’t think about the consequences (like burning cash, running in the wrong direction, binding resources, …) can have several reasons, e.g.:
FOMO - Fear of Missing Out - your competitor is doing it, can’t be wrong for us…
Uncertaincy about the future - no one has a crystal ball, so why not…
It is exactly what we waited for… really?
You need budget and this hype came at the right time…
This means, timing is very important. The data & analytics stack never sleeps. Today, we are already working more with the vendor's roadmap than with what is really there, and we are driven by digitalization and dynamic markets. It's as simple as following the latest trends:
Data Organization - everyone wants the Spotify Modell, Team Topologies or just more self-service
Consumer/Analysts - GenAI for the hope of a better user experience/satisfaction and yes, we need this 173th visual element in Power BI
Data Engineers - Data lakehouses are the new hot sh.., Headless Data Architecture will solve all the problems of my growing data stack and shift to the cloud will make us more flexible
Data Governance - Shift Left with data contracts and buy this data catalog-thing which need to have a knowledge graph and acitve metadata support, whatever this means - because my users ask for transparency
DataOps - Establish data observability and switch to Data-as-Code will let us sleep at night (finally!)
Don't get me wrong. New features, technologies and concepts are here because it solved someones problem really well. And I love new technology, who not? But it is as it is, new things are not very mature even if marketing tells you otherwise. In the beginning, there are often very few, specific use cases (the promise is on the roadmap), it is not stable and can change or even vanishing very fast.
This is where Amara's Law comes into play.
The first thing and prerequisite to make use of new technology is to understand yourself. You are not Spotify (or Amazon, Google, Apple, …)! I assume that you have a limited budget and limited resources, your company is not in a complete transformation and probably your consumers just need something what works better on the long run than Excel. So choose wisely and understand what fits to you.
Do your homework, care about data governance as far as necessary (no, I don’t just speak about authorizations), reduce your technical debt and understand that not everything can be solved through technology. Processes, organization and collaboration can change your game. The best tool used by the wrong organization or with wrong processes will not create any benefit.
If you have solved 1+2 and need progress (really or maybe just to show that you are up to date), then make use of this new technology, this new feature or this new tool. Evaluate as necessary and try it out, but don't go all in at the beginning. If you proceed slowly and consistently, you will be faster than 95% of your peers in the long run.
Yes, that's hard. Success has to be earned. But even in a VUCA world, working with a long horizon and make use of the hype on your terms, has a big impact compared to constantly chasing quick wins. Because bad timing with new technologies is even harder and often not sustainable. But never forget to celebrate your successes when the time comes and build a pipeline for your next innovation. If you don’t do it, someone else will. After all, it will be worth it, but timing is crucial. Because we tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run…
So if you are indeed not Spotify, Amazon, Google, Netflix, Uber, Airbnb or Apple, what do you think about Amara's Law? Are you riding the wave as it comes, trying out every trend and chasing quick wins? Or do you see enough time and trust for your technology decisions?