One of the fun things about actually going to a football or basketball game is that you can watch what is happening away from the ball. What we know about government — and what government knows about itself — is often like watching a football game on TV when the camera stays very narrowly focused on only the ball, and the quarterback. There’s still a lot of action, but we really don’t know what is going on. Likewise, we see the governor a great deal, we hear about new initiatives, but really, we don’t have a good picture of how different government policies and programs work together or against each other. We don’t know what spending produces the best public outcomes, and we often don’t really know why government does what it does.
The good news is that New York State and City government are increasingly analyzing internal data to improve, and better understand, what agencies are doing. Using “business analytics” and “business intelligence” helps agencies deliver more for less. But these processes are inherently limited, because they are internal, rarely shared with the public, and typically look at a narrow scope of activities. They are the like the camera watching the ball and quarterback — much better than no picture at all. But, New York can cheaply and easily do much better. Computer data crunching power and mapping has become much less expensive, and much faster. And, now data science has emerged to offer a more holistic understanding of how complex data — and human activities — relate to each other. By opening up its data to the public, New Yorkers can get hundreds of millions of dollars of free analysis and insights from data scientists in academia and journalism who are eager to increase our understanding of how the world — and our government — work.