Report: Unprofessionalism Declined when Colo. PD Used AI to Review Bodycams

Unprofessionalism at the Aurora Police Department decreased substantially when the agency used artificial intelligence software to review every minute of officers’ body-worn camera footage last year, outside researchers found.

Incidents of officers using profanity, insults, threats and inappropriate language dropped by 57% among 220 Aurora police officers during a six-month span from July to December, according to an executive summary from researchers at the University of South Carolina and Clemson University that was obtained by The Denver Post on Thursday.

RELATED:

The first-ever empirical results showing the AI software’s impact come months after Aurora stopped using the system. The city opted not to renew its deal with AI company Truleo in March, when its one-year, $250,000 contract ended for the automated body-worn footage review service.

The police department began using Truleo during the tenure of interim chief Art Acevedo, who had previously worked as a strategic advisor for Truleo but said he disinvested from the company before joining Aurora’s police force. He at the time hailed it a “tool to help us identify the good, the bad and the ugly.”

At most police agencies, including Aurora’s, the vast majority of body-worn camera footage is never reviewed — departments simply do not have the staffing to regularly look through officers’ day-to-day footage. Truleo’s software automates that review process, using AI to analyze the videos’ audio for language the software considers either professional or unprofessional.

The software is designed to flag poor behavior and highlight good policing. It doesn’t account for an officer’s tone or consider the use of force, but is focused solely on the words an officer says.

After Acevedo’s departure in January, the city opted not to renew its contract with Truleo, bringing the technological oversight to an end.

“We had concerns about its functionality, is my understanding,” city spokesman Ryan Luby said, citing problems with the accuracy of the software’s transcriptions. He declined to comment further Thursday.

Aurora Mayor Mike Coffman and a spokeswoman for Aurora police did not immediately comment.

The city’s newest police chief, Todd Chamberlain, declined to say whether he’d bring back any similar AI body-worn camera monitoring during a news conference last week, but said he embraces the use of emerging technologies in policing.

“I am going to look for the most effective, efficient ways to use technology to enhance the Aurora Police Department’s commitment and service to the community we serve, and I never will shy away from that,” he said.

Truleo CEO Anthony Tassone said Thursday he’d like to see Colorado Attorney General Phil Weiser mandate that the software be used at the Aurora Police Department, which is currently under court-ordered oversight and reform after Weiser found a pattern of racially biased policing in the troubled agency.

“In Aurora, it’s not going to work for Truleo to contract directly with the department, a one-year contract,” he said. “We see what happens. A chief leaves, and all of our work gets thrown out of the window. We would be willing to go back into Aurora, but it’s got to be with the attorney general. And with a long-term mindset.”

The independent study into Truleo’s impact at the Aurora Police Department examined 124,443 videos from body-worn cameras and divided Aurora’s officers into three groups: a control group of officers who did not receive feedback from Truleo; a self-auditing group of officers who could review their Truleo metrics on their own; and a supervisor-mediated group of officers who received feedback about their Truleo findings from their supervisors.

The system varies across departments, but, generally, officers can log into Truleo and see their own videos, professionalism scores and transcripts. The system sends congratulatory emails to officers who score well for professionalism and composure, and alerts supervisors to unprofessional incidents, Tassone said.

Incidents of unprofessionalism dropped by 48% in the self-auditing group and by 67% in the supervisor-mediated group, when compared to baseline rates, the executive summary states. The document does not say whether the control group changed.

The summary outlines the study’s initial findings; the work has yet to go through the additional vetting of the peer review process and full results have not been published.

Researcher Ian Adams, an assistant professor at the University of South Carolina, on Thursday praised the Aurora Police Department for opening up to the study of the brand-new AI technology. He emphasized that the executive summary shows preliminary results.

“It’s not easy to be at the cutting edge of this sort of thing,” Adams said. “This is the first time it has been tested empirically and carefully. That is a vulnerable place for agencies to be. And it shows they want some change, or at least did during our time there.”

The researchers looked not only at Truleo in Aurora, but also examined how the AI software impacted 180 officers in the Richland County Sheriff’s Department in South Carolina. There, researchers saw an 82% increase in highly professional behavior over a six-month span, according to the executive summary. The summary did not describe changes in unprofessionalism in that department.

Highly professional behavior is defined as interactions in which officers not only use appropriate language but also offer explanations to members of the public about the officers’ actions before acting, like before frisking a suspect or issuing a citation, Adams said.

Tassone, the company’s CEO, on Thursday hailed the executive summary as “absolutely conclusive proof” that reviewing all body-worn camera footage improves professionalism at police departments.

“This study is two opposite departments, one in a consent decree and one that is a very squared away, super professional, first-class sheriff’s department,” Tassone said. “Truleo benefits both.”

__________

Visit at denverpost.com.

Distributed by Tribune Content Agency, LLC.