Why we should stop ranking ecosystems
Last week a report came out which contained a lot of research on the entrepreneurial ecosystems of the Netherlands. The report seems to acknowledge the nature of ecosystems: They are complex, nonlinear and hard to manage. Yet, the summary of this report contains a list of all ecosystems, with a number one and a number forty. Curious as I am, I wanted to know how (and why!) they went from analysing complex, non linear systems to the most linear of things: a list. Summarizing all this research to a “winner” and therefore also “losers”.
In one of my previous articles I mentioned that startups (or the economy in general) are best served with ecosystem support (opposed to programs, or coaches, or shared office buildings). This insight is not new, but it seems that people still have a hard time grasping what an ecosystem is, so I’ll refresh quickly:
Ecosystems have mostly open boundaries; they don’t stop at municipality borders for example. Almost always there are subsystems or different layers within ecosystems, in this case often clustered around incubators or universities. Ecosystems are dynamic, they evolve continuously, but not in a linear way. Often there is a substantial delay between actions and effects, and this makes it hard to identify cause and effect in ecosystems.
All this makes it hard to model an ecosystem. One way of modelling ecosystems is by looking at the actors, factors and their connections at a certain moment in time. This is the approach of most of the “ranking reports” out there as well: You count the actors, you count the factors, if possible you count the “interconnections” and the one who scores the highest overall wins.
While the actor/factor model is a solid model to describe an ecosystem, you could argue if it was also meant to be the basis of a ranking. The value of an ecosystem is not so much in the number of actors, but in their interactions, and in the factors that are present. When saying factors, what I mean is: is there talent? is there an entrepreneurial culture? Is there financial capital? Intellectual capital? And here it gets tricky. You can count startups, you can count founders, employees, you can even count VCs, active banks, active accelerator programs etc, but how do you count talent? Culture? Knowledge? Infrastructure? And even harder: how do you count or measure how the actors interact with the factors? There might be talent or infrastructure present, but it might go unused, or it’s used in another region!
The answer here is proxies. You list the actors and factors you want to measure, and you try to find proxies for every actor or factor you are unable to count. For instance: if you wanted to count talent in the region, you take the % of higher educated people of the total amount of inhabitants of the region. If you want to count entrepreneurial culture you take the # of founded companies in the region. If you want to count leadership, you count the # of innovation projects in the region. You can already see from my examples that not every proxy is a good proxy. Take a look here (page 6) and decide for yourself.
There are several issues with this ranking methodology in combination with an actor/factor model:
1. Ecosystems are not bound by geographic regions. You measure the % of highly educated people, but you don’t know who works in the region, and who doesn’t.
2. The value of an ecosystem is in the interaction between the actors and factors. You might measure a high # of innovation projects, but they might be between the same actors all the time. You measure the distance to the highway, or the railway to say something about infrastructure, but you don’t count the travellers.
3. Proxies are just proxies. In the best case several different proxies give you a pretty good idea of the factor or actor in the region. In the worst case the data is not there, the numbers are not to be trusted or the proxy is just wrong.
4. There is only room for a finite number of actors, factors and interactions in the ranking model. How do you scope? What is important and what’s not? In this case whole ecosystems are brought back to 10 “elements”, indicated by 17 indicators. Every model is always an extreme simplification of the system.
Now, all of this isn’t so bad right? Every model has its limitations, every model is a simplification of the reality, and you’re right: the model is not the problem. The problem is we use the model as a method to rank the system. The ranking implies there is a gravity, a linearity, a similarity, a certainty to the numbers, which just isn’t there. It implies the ecosystems are the same. But, when you compare ecosystems, you are comparing forest, with sea, with farmland, with the dunes. Of course, you can count the number of trees, the number of insects and the number of fishes, but does it make sense to rank them based on those numbers? If you’re a fish you’re probably best of in the sea and you don’t really care about all the interactions going on in the forest. The same goes for a biotech startup, who doesn’t care about the financial ecosystem in Amsterdam, or for the financial startup who doesn’t care about the factors in the Westland.
Ecosystems are tough, complex, hard to understand systems and every model out there (actors & factors, network, agent based, logical) is a god-send gift to help us understand these systems. Models can show us the internal dynamics, highlight best practices, show us valuable nodes in the system or models can try to predict certain behaviour and they are very valuable in that sense. What ecosystem models should not be is the input for a (geographical) ranking system.
Let’s keep modelling ecosystems but let’s start sharing the outcomes of these models not in rankings, but in “use cases” or “best practices”. Even without rankings you can show regions which need to improve, and regions which keep developing in the right direction. Let them interact, let them share war stories, let them share best practices, instead of listing the regions and calling winners and losers. I’m sure the first approach will create a national ecosystem of a higher value than the latter approach.
