NYC Enacts Five Bills Updating Open Data Law

Today, Mayor de Blasio signed into law the first five amendments to the NYC Open Data Law since its passage in 2012.

  1. Intro 890-A: requires DoITT to archive data sets on the portal, which will help users compare current data to historical data.
  2. Intro 898-A: requires every data set on the open data portal to have a data dictionary, which will define terms of art and what information is in each column.
  3. Intro 900-A: requires each data set with street addresses to use the same format for the address, as well as a common format for latitude and longitude information. This bill will also create a working group to establish that format.
  4. Intro 914-A: all requests for data made on the open data portal must receive a response from DoITT within two weeks, and a determination as to whether the request will be granted or denied within two months.
  5. Intro 915-A: requires any public data set that is made available on both the open data portal and through an agency website to be updated on the open data portal within ten days.

NYC Council Press Release on Passage of Five New Open Data Bills

At the NYC Council’s Stated Meeting on Tuesday, November 24th, the Council voted in favor of a legislative package of five bills designed to improve the existing Open Data Law. Committee on Technology Chair James Vacca, along with Council Members Gentile, Cabrera, Torres, and Kallos are lead sponsors of the legislation.

The Open Data Law, passed in 2012, requires the Department of Information Technology and Telecommunications (DoITT) to work with City agencies to post public data online in a centrally accessible location – the Open Data portal – by 2018. DoITT has made considerable efforts to ensure compliance and has succeeded in populating the portal with new data as it becomes available. However, several issues have arisen that necessitated this legislative remedy, ranging from technical standards to overall compliance.

“This package of five bills will dramatically strengthen the Open Data Law,” said Council Member James Vacca. “Each piece of legislation will improve users’ experience, and ultimately, will make a larger amount of data more accessible to all.” Vacca is the lead sponsor of legislation designed to require the timely updating of certain public data sets on the open data portal. “It is extremely important that our Open Data datasets are up-to-date. All users should be assured that the information up on the portal is the most recent available, and I believe my bill will do just that. I thank Speaker Mark-Viverito, Council Members Gentile, Cabrera, Torres, Kallos, and the administration for taking the time to negotiate these bills with me, and I look forward to the results. Open Data is a priority for the Committee on Technology, for the Council as a whole, and for the administration, and I am thrilled to be moving forward with these bills, the first I have passed through this committee.” Additionally, four other bills were passed covering additional inadequacies in the existing law.

“Opening data and structuring it to improve public access is allowing many government agencies around the U.S. and the world to improve efficiency and find solutions to long-standing problems,” said Council Member Fernando Cabrera. “Int. 890 will increase public access to New York City data through the creation and preservation of archives that allow us to track trends over time in order to make important decisions about how to move forward.”

“Today’s passage of the Open Data legislative package is a significant step in improving the system that allows the public to request government data and also provides increased transparency. My bill would set up a timeline for responding to open data requests and removes the backlog that has been lingering for months. The public should be able to get speedy responses to their requests and this bill will ensure that. I look forward to the package being signed into law and implemented as quickly as possible,” said Council Member Ritchie Torres of the Bronx.

“Standardizing address and geospatial information for datasets on the Open Data Portal will provide a solution to the frustrating issue of mapping data with no set format,” said Council Member Ben Kallos, a software developer. “I thank Chair Vacca for his commitment to meaningful improvements to the Open Data Law and I look forward to an Open Data Portal that is more accessible and usable for New Yorkers.”

“The Open Data Legislative Package ensures that all New York City residents can access and easily comprehend our City agencies’ flow of information,” said Council Member Vincent Gentile.

“We applaud the City Council and the Mayor’s Office for agreeing on this package of five bills which strengthen and improve NYC’s landmark Open Data Law. We thank Technology Chair Jimmy Vacca and his colleagues, for listening to the public and taking pragmatic steps to get city agencies to publish accurate, usable, timely, agency data. This is what opening up government looks like. Together, these bills give an important boost to open data, and help make city government more transparent and effective,” said John Kaehny, Co-Chair NYC Transparency Working Group and Executive Director of Reinvent Albany, who helped craft New York City’s 2012 Open Data Law.

“Today, the Nation’s best open data program gets better. We thank the Council & Council Member Vacca for their leadership. We are very excited to have legislation outlining data dictionaries and address standardization. The BetaNYC community is excited to work with the City to share our experience and increase data accessibility,” said Noel Hidalgo, Executive Director of BetaNYC.

“Citizens Union applauds the City Council for working with the administration to strengthen the City’s landmark Open Data Law,” said Dick Dadey, Executive Director of Citizens Union. “The passage of these bills shows that the City is committed to learn from experience and adapt to make its data even more accessible.”

“Common Cause NY thanks Chair Vacca and congratulates the City Council on passing bills that will standardize the ways our city agencies’ data is made available for public use,” said Susan Lerner, Common Cause NY’s Executive Director. “Formalizing a data dictionary and geospatial information will enhance the usefulness of these datasets created by our public tax dollars.”

“The League of Women Voters of the City of New York supports the Open Data laws of NYC because they promote transparency in government,” said Catherine Gray, co-president of the League of Women Voters of NYC. “These 5 bills will increase public access, promote more active participation in government, and enhance understanding of major policy. Technology will offer access to vital public data including machine readable formats, automatic updates and better searching tools.”

Following the Money: Opening NY Campaign Finance Data


WHEN: 10am, Friday November 20th
WHERE: Civic Hall, 156 Fifth Avenue, 2nd Floor, New York City

On Friday, November 20th, Reinvent Albany and Civic Hall present Following the Money. In a rare NYC public appearance, staff from the NY State Board of Elections will join the NYC Campaign Finance Board to preview their new campaign finance reporting systems and discuss with the audience ways to make state and city campaign finance data easier to use.

This is a great opportunity for civic technologists, journalists, and advocates to speak directly to the government officials who collect and publish campaign contribution data in New York State and NYC. Campaign contributions are a staple of election-year news coverage and analysis, yet collecting and reporting that data to the public in a useful form can be a big challenge for regulators at the state and city level. There is a lot to talk about.

The State Board of Elections is completely revamping its aging technology, and will deploy a new campaign finance reporting system after the 2016 elections.

The NYC Campaign Finance Board has made effective use of technology and is seeking to expand and improve its digital offerings before the next mayoral election in 2017.




Civic Groups Urge State Action to Reform Public Ethics

Corruption Trials and Two Reform Reports Make the Case for Action Now

In light of the corruption trials of two former legislative leaders taking place this week and New York State’s D-minus grade in how it handles issues of integrity from a national comparison study, New York’s leading good government groups today called upon the New York State legislature and governor to complete the job of reforming our laws governing public ethics.  We urge them to embrace real solutions by enacting legislation that will bring about large scale change instead of incremental reform.

In a letter sent today to Governor Cuomo and the two legislative leaders, Assembly Speaker Carl Heastie and Senate Majority Leader John Flanagan, the civic groups urged immediate action to enact comprehensive and significant solutions on the following matters of state ethics:

  1. Reform legislative compensation through considering the forthcoming recommendations of the recently appointed New York State Commission on Legislative, Judicial and Executive Compensation, to address issues of salary, lulus, and outside income.
  2. Limit the influence of dark money campaign contributions and end government spending that takes place in the shadows by closing the LLC loophole, requiring campaign contributors to disclose their employers, requiring disclosures of all lump-sum appropriation funds, and enacting much stronger restrictions on personal use of campaign funds.
  3. Reform ethics oversight and enforcement by changing JCOPE’s structure, scope, and voting procedures to boost public confidence in its actions.  Changes should increase transparency of its operations, meetings and votes; expand jurisdiction to include all executive and legislative branch employees; and elevate the independence of the commissioners from their appointing authorities.
  4. Strengthen financial reporting disclosure requirements for public officers to allow the public to more easily spot conflicts of interest.
  5. Streamline and standardize disclosure of lobbying activity for better analysis and easier evaluation by the public.

That three significant developments regarding New York State’s handling of ethics and corruption have converged together in the past ten days speaks once again to the pressing need for action:

  1. The separate but concurrent corruption trails of former Assembly Speaker Sheldon Silver and Senate Majority Leader Dean Skelos, two of the three men in the room who helped craft much of the recent ethics laws;
  2. A new national report out earlier this week, again showing New York’s risk for corruption and dismal standing on public integrity with a grade of D-minus; and
  3. Our own state’s ethics review commission calling for a wide range of reforms in addition to the ones made by JCOPE in its February 2015 report.

Additionally, the problem of Albany corruption again being addressed not by internal state watchdogs, but instead by the U.S. Attorney who this month is prosecuting charges against the two most recent former legislative leaders, demonstrates the clear need to strengthen our state’s own ethics enforcement.

While two governors and the legislature have made some improvements to our ethics laws over the past ten years, the core problem of lawmakers using their public posts for private gain still persists.  New Yorkers have lost faith in state government to make decisions without using the interest and influence of those who do business with the state.

The groups urge the governor and the state legislature to come together and enact legislation so public trust can be restored in New York’s democratic institutions and political processes. The groups, also strong supporters of comprehensive campaign finance reform built around a core of public financing of elections, today focused on the urgent need for ethics reform as another essential way to address the problems created by a money culture in Albany.  There is no shortage of real solutions that our leaders can draw upon in enacting comprehensive change, instead of incremental reform, and in doing so give hope to the public that can trust can be restored.  In light of this storm of two trials and two reports, the groups call for immediate action on these widely-supported reforms, and in a special session if possible.

“The list of undone ethics reforms is long, and it is clearer than ever before that our elected leaders need to complete the job of ensuring that public officials can no longer use their public posts for private gain,” said Dick Dadey, Executive Director of Citizens Union. “We need only look at the trials of former leaders Silver and Skelos to realize that our state has a long way to go to in responding to the crime wave of corruption – let alone the recent State Integrity Report that gave New York a D-minus for public integrity.”

John Kaehny, Executive Director of Reinvent Albany, said: “So far there has is no sign that the arrests of the legislative leaders has changed how Albany does business. In particular, there are still billions of dollars in grants, business subsidies and state contracts which are dispensed essentially in secret. This is an invitation for more corruption. We need an online database of subsidy deals, grants and MOU funds which makes it clear who sponsored the funds and where those funds are going. ”

“It is past time for the Governor and the Legislature to come together to address the corruption crisis in Albany which these recent events underscore,” said Susan Lerner, Executive Director of Common Cause/NY. “The public has moved from disapproval to anger and is demanding that our leaders step up and effect real change.”

Dare Thompson, President of the League of Women Voters of New York State, said: “Corruption in state politics has been accepted as the norm for way too long. We need immediate action by our elected officials on ethics reform to fix this situation. New Yorkers need to have their faith restored  in their state government.”

Blair Horner, Executive Director of the New York Public Interest Research Group,said: “New Yorkers are fed up with Albany’s scandals. They deserve action, not a head in the sand approach from their elected leaders. Governor Cuomo must call a special session devoted to ethics reform.”

Listening to FOIL at NYC DOITT: 311 Data Challenge

Reinvent Albany has analyzed the Freedom of Information Law requests received by numerous New York State and City agencies in an effort to identify what agency information and data the public is interested in, and to make a case for publishing that information online. In June 2014, Reinvent Albany published Listening to FOIL, an analysis of the FOIL requests made to the New York State Department of Environmental Conservation in 2013. We found that 2,200 of DEC’s 4,000 FOIL requests targeted a handful of DEC data sets. Accordingly, we recommended that the DEC publish those data sets online to reduce the number of FOIL requests it had to respond to.

Since then, we have analyzed numerous other FOIL logs, and decided to publish to our analysis of requests received by the NYC Department of Information Technology and Telecommunications (DOITT), an agency we are familiar with. DOITT is particularly interesting to us because it both manages NYC’s Open Data Portal and processes FOIL requests for NYC’s mammoth 311 system.

We filed FOIL requests for FOILs DOITT received from October 2014 to September 2015, and published our findings in the report below. Click here for the copy of DOITT’s FOIL logs used in our research.