Open Data - why it matters

I had the privilege of attending a White House event on international development and open data on December 10, 2012. This packed three hour event was filled with great ideas and enthusiasm for what could be done with open data and why it is important.

When you think how the world has changed in the last five years, think of two items:

• Geographical information (aka google maps, GPS, foursquare and other location based applications)
• Weather predictions and reporting.

What do these two areas have in common? Open Source Data powers them.

Brave new data world

We are in a world of big data, which is allowing us real time access to changes and information in unprecedented ways. The commercial sector is using this data to tailor their products and services to clients.

This leads naturally to the question of what else could the public sector do if we had more access to this sort of data? What could we do in terms of getting real time feedback on policies and programs? What if we could accurately predict crises or emerging changes that need responses? 

Gotta be free!

The solution is not just data - it is that data is open and available and free for local adaptation and emergent usage. Let's look at the internet - no one could have predicted the vast array and diversity of internet based applications and businesses that developed. The internet (especially the web) is a fertile environment that allows for local entrepreneurs, communities and individuals take what is available and repackage, repurpose, and innovate. Open data is another resource, like the internet, that can promote that fertile environment.

What do I gotta do?

So how do we support open data? First of all, when we capture data, ask the question about whether that data should be shared - maybe not always publicly, but certainly with other parts of your organization, clients, vendors or partners.

Secondly, the format of that data is important - the more "developer friendly" you can make your datasets, the more likely it will be shared. Data processing (i.e. clean up, reformatting, migrating into a machine readable format) is expensive and time consuming. The higher quality your data, and the more it adheres to common standards within your industry, the more easily and useful it will be to be shared. More information about format questions can be found in my blog post Data Data Data! Also International Aid Transparency Initiative has good guidelines for international development assistance data standards.

Thirdly, look around for existing data sets that may be published and integrate them into your tools for richer applications. I posted an informal list of international development data sets (started by the USAID IDEA office).  For example, Sonjara uses the National Center for Education Statistics school data set to prepopulate school databases for clients who work with American primary schools. Instead of asking our users to type in the name of the school and address from memory,  we offer a "persistent search" feature that matches the first few letters you type in with schools that match that name. Once you select the school, the address and other key information is populated. This leads to much higher quality of data that we capture because we are removing a common point of error for our users.

Fourth, combining data sets allows levels of analysis that were not possible before. We pulled in the World Bank climate and precipitation data into a nutrition Food Frequency survey tool at the point in the workflow when the survey manager needs to decide the time of the year to hold the survey; the weather data gives the manager information on times during the year when food is more likely to be scarce or plentiful, elements that will influence the outcome of the survey. We also pulled in Google Maps to allow survey managers to geocode their communities before the survey questionnaires were even finalized, so that the final results can be layered with other geocoded data. The final results that measure levels of nutritional deficiency in specific communities can be combined with health surveys showing disease outbreaks or market prices of nutritious foods, allowing project implementers a more nuanced view of what is happening in those communities.

Data can change the world!

There are tons of examples of how open data allows humans to see the full context and complexity of the world around them. By contributing to open data, we make these new views real.  Visit Fakoli.org to learn more about Sonjara's Rapid Application Framework's Open Data Tools

« Back to Sonjara Blog