FOIL Request Logs As Open Data Sensor

August 13, 2014

Listening to FOIL: Using FOIL Requests To Guide Agency Open Data Efforts
Case Study: An analysis of NYS DEC FOIL Logs

The advent of the Internet and inexpensive information technology have created enormous opportunities for proactively disclosing public records and digital information or “data.” New York State and New York City have recognized this opportunity, and have released hundreds of agency data sets via “open data initiatives,” like the State’s Open NY and the City’s Open Data Law. However, releasing public data sets takes time and money, and agencies have often wondered what data sets they should publish first.

It turns out the public is telling government exactly what data they want via Freedom of Information Law requests, and agencies should start Listening to FOIL.

Earlier this year, Reinvent Albany FOILed the list of the FOIL requests received by the NYS Department of Environmental Conservation (DEC) in 2013. We picked DEC because we knew it received large numbers of FOIL requests, had a highly professional FOIL staff, and has made a concerted effort to provide information on its comprehensive website. In other words, DEC is already one of the best agencies at striving to provide information to the public. But open data is as new to DEC as it is to other agencies, and we are breaking new ground by directly connecting FOIL to open data.

DEC provided us with a spreadsheet of nearly 4,000 requests, which included the identity of the requestor and a brief summary of the records requested. We also contacted a number of exceptionally “frequent FOILers” to ask them directly what data they were seeking.

Our analysis of the DEC FOIL logs confirms that FOIL requests themselves contain a wealth of information. In DEC’s case an extraordinary 55% of all FOIL requests are asking for information about oil or hazardous chemical spills on particular parcels of property. This is the kind of “public data” that is ideal of publishing on open data platforms. (Public data is in contrast to “personal data” or sensitive information about individuals.)

Read our full report: Listening to FOIL, examine the 2013 FOIL logs for DEC, and read on.

What We Learned

The log of all FOIL request received by the DEC in 2013 was full of interesting information about who is FOILing the DEC and what they’re looking for. The highlights:

  1. Over 80% of all FOIL requests to the DEC in 2013 (3,456 out of 3,977) were from businesses.
  2. Just twelve real estate industry companies made 20% of all FOIL requests.
  3. About 55% of all requests (2,200) were about spills on specific properties.
  4. Other state and local government agencies FOILed DEC 69 times in 2013.
  5. Journalists were only 3% of FOIL requests. (73 requests.)

DEC FOIL chart

 

What We Recommend

Our analysis of DEC FOIL logs demonstrates that FOIL logs are a valid way to identify which data sets are most interesting to the public, i.e. “High value data sets.” Accordingly, we recommend that agencies, and the Governor’s Open NY team, undertake the following steps to reap the most from this opportunity:

  1. Work with DEC to publish its most FOILed data sets.
  2. Identify which state agencies keep FOIL logs identifying requests by topic.
  3. Select the agency with the most requests and analyze its FOIL log.
  4. Publish that agency’s public data based on the FOIL log analysis.
  5. Draft guidance to agencies explaining how to analyze FOIL logs to determine what data to put on their websites, and in what form, and to guide the publication of their data sets on data.ny.gov.
  6. Recommend government agencies use automated FOIL systems which track the topic of requests, to create logs for analysis in recommendation #3.