CHICAGO (AP) — A gunshot detection system that has cost Chicago tens of millions of dollars and is touted as a critical component of the police department’s effort to combat gun violence rarely produces evidence of gun-related crime in the city, Chicago’s nonpartisan watchdog agency concluded.

In a scathing report released Tuesday, the Office of Inspector General’s Public Safety section said the police department data it examined “does not support a conclusion that ShotSpotter is an effective tool in developing evidence of gun-related crime.” And, the office concluded, if the department has information that shows ShotSpotter plays a key role in developing such evidence, its “record-keeping practices are obstructing a meaningful analysis of the effectiveness of the technology.”

The inspector general’s office found that between Jan. 1, 2020, and May 31 of this year, over 50,000 ShotSpotter alerts were confirmed as probable gunshots, but that actual evidence of a gun-related crime was found in about 4,500 instances, or only about 9%.

The report is the latest blow to a system that has come under scrutiny, particularly in Chicago, after it set in motion the fatal police shooting of 13-year-old Adam Toledo in March. Although the boy appeared to be holding a gun right before police shot him, community groups argued afterward that the system sends officers to predominantly Black and Latino neighborhoods for “unnecessary and hostile” encounters with residents and asked a judge to scrutinize the algorithm-powered technology to determine if it is trustworthy.

Last week, The Associated Press reported that its review of thousands of internal documents, emails, presentations and confidential contracts, along with interviews with dozens of public defenders in communities where ShotSpotter has been deployed, found serious flaws in the use of ShotSpotter as evidentiary support for prosecutors.

According to the AP investigation, the system can miss live gunfire right under its microphones and misclassify the sounds of backfiring cars or fireworks as gunshots. It also found that forensic reports prepared by ShotSpotter employees have been used in court to improperly claim that a defendant shot at police, or provide questionable counts of the number of shots allegedly fired by defendants. Judges in a number of cases have thrown out the evidence.

Chicago prosecutors partially relied on audio evidence picked up by ShotSpotter sensors to charge 65-year-old Michael Williams with murder last year for allegedly shooting a man inside his car. Williams spent nearly a year in jail, but late last month a judge dismissed his case at the request of prosecutors, who said they had insufficient evidence.

Following the AP investigation, Democratic Sen. Ron Wyden, of Oregon, said the U.S. Justice Department needs to look into whether the algorithm-powered police technologies it funds, including some that integrate gunshot detection data, contribute to racial bias in law enforcement.

ShotSpotter vigorously defended the reliability and validity of its system on Tuesday, and pointed to an audit that the company commissioned to study the effectiveness of its technology.

“The OIG report does not negatively reflect on ShotSpotter’s accuracy which has been independently audited at 97 percent based on feedback from more than 120 customers,” ShotSpotter said in a statement.

Chicago Police Department spokesman Tom Ahern said Tuesday that ShotSpotter has detected hundreds of shootings that otherwise would have gone unreported.

“The system gives police the opportunity to reassure communities that law enforcement is there to serve and protect them and helps to build bridges with residents,” Ahern said.

The American Civil Liberties Union, meanwhile, published a blog post critiquing ShotSpotter’s system that cited the AP’s investigation.

“ShotSpotter’s methodology is used to provide evidence against defendants in criminal cases, but isn’t transparent and hasn’t been peer-reviewed or otherwise independently evaluated,” wrote Jay Stanley, a senior policy analyst at ACLU’s Speech, Privacy and Technology Project. “That simply isn’t acceptable for data that is used in court.”

In San Diego, the city council was set to vote last month on renewing its own contract with ShotSpotter and instead decided to send it to staff for further review after community activists raised questions about its use.

Chicago’s Office of Inspector General is a taxpayer-funded independent watchdog that has subpoena power but no authority to change or eliminate city programs.

According to the OIG report, late last year the police department asked for and received an extension of its three-year, $33 million ShotSpotter contract, the company’s largest, that was set to expire this month. The city “exercised an option to extend it, setting a new expiration date for August 19, 2023,” it said.

Some aldermen expressed surprise that Mayor Lori Lightfoot’s administration renewed the contract, with one saying he would introduce an ordinance requiring City Council approval on the renewal of any contract over $1 million. Lightfoot’s office did not immediately respond to a request for comment.

ShotSpotter has won praise from law enforcement agencies that say it puts officers on the scene far faster than if they had waited for someone to call 911 to report gunfire. While, for example, there have been questions about whether the police shooting of Toledo was justified, authorities said that an instant before he was shot, the teen was holding a gun that another man had fired minutes earlier.

Lightfoot has weighed in as well, calling the technology, along with cameras and high-tech support centers staffed with police, “a lifesaver.”

On its website, the California-based company says ShotSpotter helps stop gun violence by using “sensors, algorithms and artificial intelligence” to classify 14 million sounds in its proprietary database as gunshots or something else. In a recent interview, CEO Ralph Clark declined to discuss specifics about the company’s use of artificial intelligence, saying it’s “not really relevant,” and instead emphasizing the importance of ShotSpotter employees, who listen to sounds picked up by the sensors and help classify them.

According to the inspector general’s report, the use of the ShotSpotter system is “changing the way officers respond to calls,” and is being used “to form the basis for an investigatory stop or as part of the rationale for a pat down once a stop has been initiated.”

“If the Department is to continue to invest in technology which sends CPD members into potentially dangerous situations with little information — and about which there are important community concerns — it should be able to demonstrate the benefit of its use in combatting violent crime,” the office reported. “The data we analyzed plainly doesn’t do that.”