Last week BuzzData published our first map, Open Data Hubs Worldwide. It’s a satisfying exercise, putting as much open-data activity as we can find in one place!

But eventually we want this map to evolve into something more informative. Questions that continue to plague us: 

Which open data communities are thriving?

Which ones are flailing or stalled?

What are reliable indicators of either state?

The CTIC Foundation in Asturias, Spain published their own map of open-data initiatives, rating them using Tim Berners-Lee’s five-star system for linked data. This is a great start — after all, common sense tells us the accessibility and interoperability of available data has to be a key limiting factor of community engagement and interest. But how do we know if data’s being put to good use? And who’s using it best?

In its Open Data Manual, the U.K.-based Open Knowledge Foundation predicts that open data can/should have the following positive effects:

•    Improved transparency and democratic control
•    Improved participation
•    Self-empowerment
•    Improved or new private products and services
•    Innovation
•    Improved efficiency of government services
•    Improved effectiveness of government services
•    Impact measurement of policies
•    New knowledge from combined data sources and patterns in large data volumes

In light of other experts’ opinions on the impact of open data (for a diverse, one-stop shop of great reads, we suggest the first issue of Google’s Think Quarterly), this list rings pretty true. So let’s start there.

On a whim we whittled the OKFN list down to what we thought to be its most quantifiable elements:

•    Transparency and democratic control
•    Participation
•    Self-empowerment (subjective, hard to quantify)
•    Improved or new knowledge, private products and services
•    Innovation (notoriously hard-to-quantify concept. The above measure is enough)
•    Improved efficiency of government services
•    Improved effectiveness of government services
•    Impact measurement of policies (getting gov’t to quantify this would take too long)
•    New knowledge from combined data sources and patterns in large data volumes (threw this in with fourth item)

We then tried to parse each of the above items into a yes-or-no question and a measurable quantity:

i.    Participation
Question:    Are people using the available data?
Measure:     Number of people

ii.   Democratic control and government transparency
Question:    Are people asking for/getting the data they want?
Measure:     Number of requests made, number of requests met

iii.  Improved or new products, services, knowledge?

Question:   Is the data being used to create/discover things?
Measure:    Number of new products, services, definitive discoveries

iv.  Improved efficiency of services
Question:    Are services cheaper because of open data?
Measure:     Savings per open-data (or open-data app) user, whether   consumer/company/government department

v.   Improved effectiveness of services
Question:    Are services better because of open data?
Measure:     Number of people using a given service, number of people satisfied with a service

Of course, all of these measures would have to be normalized to a respective community’s size/age, etc.

Now we’re getting somewhere! Granted, this is still a bit of a thought exercise, but this list is starting to actually look like a kind of survey we can apply to different cities to make meaningful comparisons.

Got more suggestions? Send them our way!