NY Innovates – Jim Hendler on The Big Problem in Open Data

During the recent NY Innovates conference in Albany, New York State’s Open Data Advisor, Dr. Jim Hendler, shared a story underscoring what he feels is the major problem with open data initiatives: getting data to users want it (and would understand it).

Some time ago, as Dr. Hendler tells the story, he created a data mashup of two sources: the number of books in each state’s libraries, and the number of people residing in each state. This mashup divided the former by the latter, calculating the number of library books for each person in that state. After publishing the mashup and its figures, Hendler received a brusque email from someone who found his mashup lacking. According to Hendler, the emailer told him that he was (obviously!) using the wrong data set, which only counted books available in libraries which were owned by the state, and not books available in all libraries in that state.

It was interesting to hear an expert on open data talk about his own misstep with open data, but the lesson to take away here is that open data movements need to be mindful of making their data easier for real people to use. Hendler analogized the situation to cotton farmers and people who want to buy shirts. There’s an entire process which happens between “cotton is harvested” and “shirt is purchased,” and open data’s analogous process, according to Hendler, still remains to be commoditized and perfected.