It has been six years since the Ferguson riots put a microscope on relations between police and black communities, and despite dozens of studies on the matter, researchers say they are no closer to a consensus on the role that bias and racism play.
It turns out there is not good enough data to be able to draw firm conclusions, leaving a chaotic landscape of competing conclusions.
One paper saying black unarmed civilians are 3.5 times more likely than whites to be shot by police is stacked up against a competing study arguing that once factors are controlled, there are no racial differences in police shootings.
George Floyd’s May 25 death under the knee of Minneapolis Police Officer Derek Chauvin has reignited all of the questions, with politicians, the press and community leaders saying it’s evidence of something gone wrong.
Researchers say that’s true, though exactly what that something is turns out to be harder to pinpoint.
“It seems pretty clear that the consensus is it was excessive force. What is never going to be clear is whether it was bias,” said Lorie Fridell, a criminology professor at the University of South Florida, and founder of the Fair and Impartial Policing training project. “We cannot read officers’ minds.”
She said it’s easy to measure disparities in outcomes, but getting at causes is difficult and pinpointing where a community’s behavior ends and an officer’s bias begins is not easy, given the data.
“Social science can’t measure everything we want it to measure,” the professor said. “We can’t even measure crime very well. We can measure reported crime. But this thought that we can measure when bias is impacting police is an aspiration we can’t meet.”
There are plenty of numbers out there trying to get at the problem.
A 2019 study said the probability of a black man dying at the hands of a police officer is nearly 1 in 1,000. For white men, it’s about a third of that, and for Asians it’s half that of whites. For women, no matter the race or ethnicity, the chances are far smaller.
Another paper by a Harvard economist, released in 2016 and updated in 2018, says blacks and Hispanics are 50% more likely to face nonlethal use of force from police — though for officer-involved shootings there was no racial difference.
Stanford social psychologist Jennifer Eberhardt, meanwhile, studied Oakland in 2013 and 2014 and found black residents were 28% of the city’s residents but accounted for 60% of police stops.
She also found that among officers who made at least one stop during the 13 months she studied, only a fifth of them reported stopping a white person while nearly all stopped a black person. Only about a quarter put a white person in handcuffs, while 72% of stops of black people involved handcuffing.
Studies dating back to the 1960s have posed the same questions and had similar difficulty in drawing ironclad conclusions.
“There are many reasons for disagreements in this literature, but I think a lack of necessary data, and confusion — even among scholars — over what constitutes a valid statistical test for discrimination, have been impediments to progress in this research,” said Jonathan F. Mummolo, a professor at Princeton University. “Data is becoming increasingly available, but is still lacking in important respects.”
Mr. Mummolo was part of a team that published a paper in the American Political Science Review this year arguing that police records at best show data on encounters that do happen. They can’t show all the times when an officer observed someone but didn’t engage in an encounter.
Without that, it’s tricky to say whether officers have a higher threshold for stopping, arresting or engaging in use of force for some racial or ethnic categories.
Mr. Mummolo and his fellow researchers said that could be masking even worse disparities in treatment of black or Hispanic civilians, versus whites, than show up in the data now.
When David J. Johnson, a social-cognitive psychologist and researcher at the University of Maryland, sought to study police incidents, he had to spend 1,800 hours — nearly a year’s worth of full-time work — tracking down data from departments just to get basic information on the race, sex and years of experience of officers involved in fatal shootings.
After crunching the numbers, he concluded that in police encounters in which a white person is fatally shot, it’s most likely to be by a white officer. In cases where a black person is fatally shot, it’s most likely to be a black officer.
But that doesn’t mean white officers are more likely to shoot whites or black officers are more likely to shoot blacks. That would require far more data about instances in which someone isn’t shot for comparison.
Mr. Johnson’s paper, published last year, sparked an exchange with Mr. Mummolo and other researchers who said he was overselling his conclusions.
He appended a clarification about his wording but said his central conclusion remains intact.
“We only have information about people who were fatally shot — and even then the data isn’t that great,” he said. “It’s not the question a lot of people think of immediately, but it’s the only question we can answer at this point, at least with a good degree of clarity.”
Mr. Johnson agreed with other researchers that they are constrained by available data.
“When police officers interact with people, we need to know when they aren’t shot,” he said. “When a police officer’s looking to pull someone over, how many is he not pulling over? How did those encounters go?”
Police departments do capture data, but it varies widely and there is no standardized set of definitions to make comparisons across departments possible.
Mr. Johnson said it would likely take an act of Congress to get departments to collect uniform data.
The lack of data also makes it tough to track what’s working.
In 2019, the Urban Institute tried to study whether an Obama-era initiative to improve trust between police and communities was successful. They said an “alarming dearth” of data made it tricky to say what actually happened.
“Agencies committed to enhancing police-community relationships should make collecting data on outcomes such as arrests and use of force by race and ethnicity a priority,” the Urban Institute concluded.
Ms. Fridell said the lack of data or the ability to pinpoint exactly where bias comes into play shouldn’t prevent the country from striving to make changes.
“On the one hand, the fact that we have demographic differences in criminal behavior, that is not something that should be lain at the doorstep of police. That is a societal issue,” she said. “And we know that police, because they are human, have biases. So we know that we need to address biases, we know we need to address causes of crime.”
She said one new focus of her training at the Fair and Impartial Policing project is on whether officers dehumanize groups that are different from them.
She asked a group of 90 police leaders at a session in Boston whether it’s a danger for officers. She was expecting some, if not most, of the leaders to say no. Instead, every one of them said yes, some officers could come to dehumanize certain populations over time.
Her training program now addresses that issue.
She said she wonders whether the outcome in Minneapolis would have been the same if Officer Chauvin had responded to a similar call in the business center of the city, and taken a person into custody who showed the same behavior but was a white male wearing slacks and a business shirt.
“It’s possible, but I don’t think so,” she said.