When the Government Rules by Software, Citizens are Left in the Dark

Governments increasingly rely on mathematical formulas to inform decisions about criminal justice, child welfare, education, and other arenas. Yet it’s often hard or impossible for citizens to see how these algorithms work and are being used.

San Francisco Superior Court began using PSA in 2016, after getting the tool for free from the John and Laura Arnold Foundation, a Texas nonprofit that works on criminal-justice reform. The initiative was intended to prevent poor people unable to afford bail from needlessly lingering in jail. But a memorandum of understanding with the foundation bars the court from disclosing “any information about the Tool, including any information about the development, operation and presentation of the Tool.” Many governments said they had no relevant records about the programs. Taken at face value, that would mean those agencies did not document how they chose, or how they use, the tools. Others said contracts prevented them from releasing some or all information. Goodman says this shows governments are neglecting to stand up for their own, and citizens’, interests. “You can really see who held the pen in the contracting process,” she says. The Arnold Foundation says it no longer requires confidentiality from municipal officials, and is happy to amend existing agreements, to allow officials to disclose information about PSA and how they use it. But a representative of San Francisco Superior Court said its contract with the foundation has not been updated to remove the gag clause.