[music]

Female Speaker 1: Retailers we talk to are losing billions of dollars to organize retail crime, dangerous international crime rings, and even groups with suspected ties to terrorism increasingly getting involved.

Micah Loewinger: There's a pervasive theme in media coverage about shoplifting that may stem from bad data. From WNYC, New York. This is On the Media. I'm Micah Loewinger. So the math isn't making sense, but it sure feels like there's something going on with shoplifting rates.

Nicole Lewis: When we take New York City out, shoplifting actually declined. The headline is, "It's actually not getting worse."

Micah Loewinger: Also, on this week's show, I speak to someone who has misgivings about the way his company used artificial intelligence.

Jay Allred: What if this had been crime reporting? Real harm could have been done. I think it should give us all pause. It's why I'm having this conversation with you.

Micah Loewinger: It's all coming up after this.

[music]

Micah Loewinger: From WNYC in New York. This is On the Media. Brooke Gladstone is off this week. I'm Micah Loewinger. You might not know it, but recently there was some breaking news in the retail world. It has links to a story that has been doing the rounds for quite a while now.

Female Speaker 2: Tonight a rash of smash-and-grab thefts targeting retailers around the country. Just as the holiday shopping season picks up.

Female Speaker 3: A mob of 20 to 30 male and female suspects wearing masks and hoodies, seen snatching merchandise running out of Nordstrom and escaping.

Female Speaker 4: Items that you would normally just grab and throw in your cart are now under lock and key.

Male Speaker 1: This thing is an epidemic. It's spreading faster than COVID, Steve.

Micah Loewinger: Sensational reports used a new term Organized Retail Crime.

Female Speaker 1: Retailers we talk to are losing billions of dollars to organize retail crime, and authorities are warning that this has become an absolute threat to public safety with violent gangs, dangerous international crime rings, and even groups with suspected ties to terrorism increasingly getting involved.

David Johnston: Whether it's new laws or amending existing laws to be able to stiffen the penalties for repeat offenders who are doing this for financial gain, it's a start.

Micah Loewinger: That last voice is David Johnston.

Male Speaker 2: He's the Vice President of Asset Protection and Retail Operations for the National Retail Federation.

Micah Loewinger: The National Retail Federation, the NRF, the biggest retail lobbying group representing more than 16,000 companies, including Target and Walmart. If you've seen or read coverage about organized retail crime and the new bills aimed at cracking down on this type of theft, you've likely encountered NRF spokespeople and statistics.

Female Speaker 5: For a broader look at the issue in the retail industry. We want to bring in Matthew Shay. He's National Retail Federation's president and CEO. The NRF's latest report on the issue shows that retail shrinkage is on the rise.

Matthew Shay: The whole issue of shrink and organized retail crime really is a growing and a persistent threat. As the study you referenced just a minute ago determined and really illustrated in very stark ways.

Micah Loewinger: That study, which included some dubious data, is the subject of this piece, but before we go further, I think we should define our terms. Let's start with shrinkage.

[audio playback]

Male Speaker 1: You mean shrinkage?

Male Speaker 2: Yes.

[laughter]

Male Speaker 2: Significant shrinkage.

Male Speaker 1: You feel you were shortchange?

Male Speaker 2: Yes.

[end of audio]

Micah Loewinger: No, not that shrinkage.

Daphne Howland: So Shrink, shrinkage is inventory loss.

Micah Loewinger: Daphne Howland is a senior reporter at Retail Dive, an industry website reporting on news and trends in retail.

Daphne Howland: Unaccounted for inventory what happened to these items, lost, damaged, maybe an accounting mistake, and stolen.

Micah Loewinger: Just the overall number.

Daphne Howland: Overall number of goods unaccounted for in a year.

Micah Loewinger: Then recently we've learned this new term, Organized Retail Crime. It means something very specific. Can you describe it?

Daphne Howland: The NRF has a little bit of a convoluted definition, but it boils down to three or more individuals robbing a store and taking the stolen goods to resell as opposed to personal use. That's key because a lot of shoplifting, whether it's a teenager swiping a pack of gum or maybe an impoverished mother who needs diapers or something, that's personal use, this is a whole different scale. It means someone's going to probably sell it online or through FB Marketplace or a place like that.

Micah Loewinger: In its latest annual report originally released in April, the National Retail Federation estimated that in 2021 shrinkage overall loss of inventory cost these stores $95 billion and that organized retail crime made up nearly half of that. This claim referenced data from another retail association that estimated that groups of thieves had swiped a total of $45 billion worth of stuff in 2021. $45 billion. A couple of weeks ago, Daphne fact-checked that number and found it to be baseless.

Daphne Howland: After my story came out, the NRF removed this reference to $45 billion of losses to organized retail crime from a crime report that they released in April.

Micah Loewinger: It almost sounds small potatoes when you're like, National Retail Federation retracts number from 2021 until you realize that that's the only number they got.

Daphne Howland: And now they don't have it. There is no number related to organized retail crime. That doesn't mean there's not organized retail crime. It just means nobody has a number.

Micah Loewinger: I began trying to figure out how that now retracted claim first entered the NRF's crime report, the media, and even the congressional record, it seemed to get significant attention following a September 2021 Wall Street Journal Profile of a man named Ben Dugan.

Daphne Howland: He heads up a group that is basically a partnership between the retail industry and law enforcement and loss prevention.

Micah Loewinger: CLEAR, the Coalition of Law Enforcement and Retail.

Daphne Howland: He also works for CVS as a key loss prevention officer for them.

Ben Dugan: I'm Ben Dugan. I'm the director of organized Retail Crime and Corporate Investigations for CVS Health.

Ryan: Instead of like Law & Order: SVU, it's like Law & Order CVS.

Ben Dugan: Exactly. Something like that.

Micah Loewinger: This is Dugan speaking with Ryan Knutson, the host of the journal. The flagship Wall Street Journal podcast for an episode all about shoplifting.

Ryan Knutson: One trade group estimates that shoplifting costs US retailers $45 billion a year, which is up 50% from a decade ago.

Micah Loewinger: That trade group is Ben Dugan's CLEAR. Note that in this reference, the $45 billion is attributed to shoplifting writ large, a much bigger bucket than organized retail crime, which is a kind of shoplifting. There's no interrogation of that statistic. No one asks how Dugan got that number in this podcast or in the extensive print profile of Dugan that preceded it. The print story in the Wall Street Journal creates some buzz in the retail world and then two months later we hear from Dugan again, this time on a much bigger stage.

Ben Dugan: Good morning, Chairman Durbin, Ranking Member Grassley, and members of the committee.

Micah Loewinger: Here's Dugan speaking before the Senate Judiciary Committee on November 2nd, 2021.

Ben Dugan: I want to share firsthand today what I've experienced over 30 years of working on this problem. Organized retail crime represents a massive and growing threat to the tune of $45 billion a year. These criminal organizations employ teams or crews of professional thieves that steal the products by any means necessary and sell them through online marketplaces.

Micah Loewinger: Now the $45 billion in losses is attributed solely to organized crime.

Daphne Howland: It's a big claim. It's a definite claim.

Micah Loewinger: Daphne Howland, when she started digging into the 2023 National Retail Federation's report, she found that the source for the claim that roughly half of the shrinkage losses in 2021 came from organized retail crime was Dugan's Senate testimony.

Daphne Howland: Dugan said that CLEAR estimates this $45 billion figure, but I couldn't find any reports or research statements or anything on the CLEAR website or anywhere. Actually, as the LA Times did that same year in 2021, they questioned this number and tried to figure out where it came from.

Micah Loewinger: She eventually just went to Ben Dugan himself and asked where he got that number. In an email that Daphne shared with us, Dugan said the National Retail Federation,

Daphne Howland: That $45 billion was a number offered up by the National Retail Federation in 2016, which was their estimate of total shrink or total inventory loss for 2015.

Micah Loewinger: He goes to the Senate, he tells them that organized retail crime accounts for $46 billion of loss for retailers and he's just talking about a number from 2016 that describes something completely different?

Daphne Howland: Exactly.

Micah Loewinger: The National Retail Federation was saying that roughly half of all shrinkage in 2021 was the result of organized retail crime. Their source was Ben Dugan, who it turns out was quoting old numbers from National Retail Federation, quoting themselves in effect.

Daphne Howland: Quoting themselves and quoting themselves wrongly. Making their own math mistake and it seems like a mistake that should have been caught. Maybe put it this way, it should have been a number whose providence should have been investigated.

Micah Loewinger: We can and should put that $45 billion number to rest. Maybe these lobbying groups don't mind if journalists fail to scrutinize their claims.

Daphne Howland: We've heard from the National Retail Federation saying that organized crime is behind it. I'm seeing estimates here that it's costing retailers nearly $100 billion a year.

Female Speaker 6: Organized retail theft it's a crime that cost businesses $96.5 billion in 2021 according to the National Retail Federation.

Male Speaker 3: The National Retail Federation calls the retail theft and those smash-and-grabs organized retail crime that cost the industry $100 billion in losses in 2021.

Daphne Howland: This just shows that this is a problem not just of confusing numbers, which let's face it, numbers confuse people very easily, it's a confusion of terms. You touched on this before. When the NRF itself introduced its Shrink Report, its annual report on the inventory losses experienced by the industry, the headline of their press release said that it was nearly $100 billion lost to retail crime, when they know very well that it's the total inventory loss.

Micah Loewinger: The total shrink.

Daphne Howland: The total shrink.

Micah Loewinger: Meaning any outlet that told you that crime costs the retail industry $100 billion dollars a year got duped.

Daphne Howland: The industry and many retailers are insisting that this is a real and growing problem. They offer these numbers as evidence of that. I think that if we're going to get to the bottom of it and follow through on the different types of legislation that these groups are proposing, we need to be really well-informed, and right now, we're just not.

Micah Loewinger: Daphne Howland is a senior reporter at Retail Dive. We asked the Wall Street Journal for comment, but at time of recording, we haven't heard back from them. When we reached out to the NRF, they gave us this statement, "We stand behind the widely understood fact that organized retail crime is a serious problem impacting retailers of all sizes and communities across our nation." Ben Dugan sent us a response too saying that he now stands behind the $45 billion figure and, "Equivocating over a number we acknowledge is an estimate misses the point." For the full statements, head to onthemedia.org. Coming up, bad shoplifting coverage leads to bad shoplifting laws. This is On the Media.

[music]

Micah Loewinger: This is On the Media, I'm Micah Loewinger. Retailers have been lobbying the press and legislators to support a slate of new laws aimed at fighting the retail crime epidemic but when Nicole Lewis, the Engagement Editor at the Marshall Project, began digging into shoplifting data from the National Retail Federation, other lobbying groups, and law enforcement, she became less convinced there was an epidemic at all. In February, she published a piece called What the Panic Over Shoplifting Reveals about American Crime Policy. She says that the crime stats provided by the stores themselves might help clarify the situation if only she had a chance to look at them.

Nicole Lewis: Every year, these retailers report their numbers. This is a self-survey. When I asked the retail federation to say, "Can I dive into this raw data? I'm really curious how it's collected," they said, "No, of course not. We're not going to give that to you. That's proprietary information."

Micah Loewinger: When you say self-survey, you mean a trade organization asking CVS, Walgreens, Target, "What are you seeing?"

Nicole Lewis: Exactly. Then the other layer here is that when I went back and I dug into every single survey that they've conducted between 2016 and now, out of curiosity, I was like, "Well, has this problem actually been getting worse?" The share of stolen merchandise that comes from actual retail theft has stayed stable around 1.4%, 1.5% for each of those years. Basically, what that is saying to me is, yes, the cost or the overall amount has increased, but the share that they're attributing to people actually stealing has stayed the same from 2016 to 2021 when they did their last survey.

Micah Loewinger: Wait, sorry, I'm a little slow. Explain that to me.

Nicole Lewis: Basically, the headline is it's actually not getting worse, things are getting more expensive. If someone were to steal 10 gallons of milk in 2016, that's one cost, but if you stole 10 gallons of milk today, it's more expensive. That's how we get from $90 to $94 billion.

Micah Loewinger: Their own data does not bear this out. Let's talk about what should be more reliable data like law enforcement data. Are they seeing this big spike in so-called organized retail crime that we've been hearing about?

Nicole Lewis: Organized retail theft is a new category of this kind of crime. The umbrella term would be property theft, just generally stealing. In a lot of places, the police couldn't even really charge you with organized retail theft because it doesn't exist as a crime category yet. When I asked police departments and asked states and asked task forces, "Is this something you're tallying?" They say, "No, for the most part, shoplifting exists in this amorphous, lumpy category of property theft."

Micah Loewinger: Putting the obvious sort of propaganda elements aside, New York has seen a spike in shoplifting. That we can prove.

Nicole Lewis: Correct. Just recently, the Council on Criminal Justice looked at the police data that we do have available to say what's happening here. In New York, they found something really important. That shoplifting in New York is 16% higher in the first half of 2023 than in the first half of 2019. I think what's important to know is when we take New York City out, the trend changes. Shoplifting actually declined over those periods. Out of the 24 cities that the Council on Criminal Justice looked at, 17 reported decreases in shoplifting. Again, it really does contradict that national narrative that it's surging out of control everywhere and tells us that in some places this is a really concentrated major issue.

Micah Loewinger: The data is either incomplete and unreliable or outright does not support this narrative and yet-

Nicole Lewis: Exactly.

Micah Loewinger: -you can't tell me that those videos of cars crashing into stores or shoplifters flashing guns or knives on Home Depot employees or flash mobs rushing out of a store with loads of goods are not real.

Nicole Lewis: No, this is really important point because I don't want to make it seem like I'm saying, "Oh, calm down it's fine." We've all seen those videos and I was actually even in a CVS not that long ago in my neighborhood, and a man came rushing in with a duffle bag. He clears off a shelf, he runs out, he does say, thank you.

Micah Loewinger: Polite. [laughs]

Nicole Lewis: Right. Thank you for these stolen goods. All of the CVS attendants are standing around, the police are nowhere in sight, so absolutely, something is happening. I think the question here is, there are a couple. One is, is it really as bad as the executives say or as they'd like us to believe? Then the other big question is, what do we do about it? Are the methods that they're advocating for, is the involvement of law enforcement, stiffening of penalties, is that actually going to solve the problem? When I think about how consequential crime policy is to Americans' lives, that's really where I'm saying, "I think we need to pull back and really make sure that we're understanding the scope of this problem, what we can and cannot say about it."

Micah Loewinger: Yes, I was reading a piece in the Wall Street Journal about the so-called brazen burglars who are leading part of this shoplifting epidemic. The article made it clear that on the West Coast with, quote-unquote, "more lenient law enforcement policies," we're seeing a bigger problem. Tell me how this perception, this fear of an increase in shoplifting is translating into policy.

Nicole Lewis: Yes, it's so fascinating because even hearing that talking point in the Wall Street Journal, I'm thinking this sounds so shockingly familiar to what the lobbyists for retailers were telling me. When I go to talk to criminologists and I say, "Is it true that lenient policies around property theft are driving up property crime? Do we have any data or research that backs that up?" The resounding answer is just that is not how it works.

One good example of this comes from Pew. There has been this trend over the last several years for states to basically increase the threshold, increase the amount of goods you'd have to steal before triggering a felony charge. This is really important because let's say you set that number in the 1990s. Today again just the milk analogy, you don't have to steal nearly as much milk before you would trigger a felony and so that means that people were getting felony charges for a thing that wouldn't actually have been a felony in the '90s.

Micah Loewinger: If you set the price needed for a felony higher then in theory you're locking up fewer people?

Nicole Lewis: Right, so this was a really important reform and a number of states said, oh, my God, yes, we should probably do something about this. Pew went back and they looked and they said, "Okay, well, now we have this natural experiment where we can say, 'does it actually matter if people are going to be charged with a felony? Does that increase or decrease the amount that people are going to steal?'" The answer was clearly no. States that did change their threshold had the same overall decline in the overall property crimes, in the last few decades, as states that did not.

Micah Loewinger: Give me a sense of how widespread these shoplifting laws are. Are they growing?

Nicole Lewis: They absolutely are. When I reported this story out in February a handful of states had already changed their laws that year. This was California, Florida, Louisiana, North Carolina. Then there were about 11 states that were considering legislation that would more harshly punish people caught stealing from stores.

Female Speaker 7: Virginia lawmakers passed a bill making it a class three felony to steal more than $5000 worth of retail from one or more stores over the course of 90 days.

Female Speaker 8: Organized retail theft that's been an issue here in Minnesota.

Male Speaker 4: Yes, bipartisan legislation aimed at fighting organized retail theft passed this spring. It adds much stiffer penalties including prison time.

Male Speaker 5: The Alabama District Attorney's Association says that the state will adopt new laws for stiffer penalties.

Female Speaker 9: There's a new law that's designed to crack down on this type of crime in North Carolina.

Male Speaker 6: In New Mexico, prosecutors will now be able to combine the value of merchandise stolen from various stores over a 90-day period, making it easier to charge serial shoplifters with felonies rather than misdemeanors.

Nicole Lewis: This is a large share of states now that are being influenced by a trade association and making these penalties harsher or adding new categories of crimes to their statutes.

Micah Loewinger: Let's just talk a little bit about incentives here. Obviously, CVS, Walgreens, Target they don't want people to steal from their stores clearly. Why else are they pushing this unfounded narrative of a national shoplifting spike?

Nicole Lewis: This story that I wrote even started because we heard a Walgreens executive reporting out of an earnings call basically saying we cried too much last year about how many products were disappearing from our shelves.

Micah Loewinger: Basically just admitting, why say that?

Nicole Lewis: Yes. Well, they said this because there's just huge financial implications. They had paid to hire private security to police their stores to make sure that the merchandise stays on the shelves and that's the cost to them. Then the data says maybe we didn't need to spend that much. They walked it back and he was basically announcing the decision to let those private security companies go. That was the context of the admission. Just a few years earlier, Walgreens had tried to use shoplifting and rampant shoplifting as the reason behind their decision to close five stores in San Francisco.

What we know about the Walgreens scenario is it turned out not to be true. These decisions had been made months if not years prior, and then I'd say there's one other thing that becomes really important and it's all about the cost to these retailers. If you can make it so that there's a public panic, the public sees these videos, they go into CVS and items are behind plexiglass, there's a public concern, there's a need to say we really got to get a handle on this. Lawmakers tend to be pretty responsive to that kind of public energy about an issue. The retailers get to externalize the cost of dealing with this onto state lawmakers and law enforcement to say, hey, it's police, and it's prosecutors, and it's the state legislature that needs to change the laws and order for us to get a handle on it.

Micah Loewinger: There's reason to believe that they will get their way because they're lobbying both at the national level and at the regional level, is just super strong.

Nicole Lewis: Yes, retailers are a very, very important part of our economy. They're major employers and the transactions that happen in our stores generate a ton of sales tax. States have a huge interest in making sure that retail interests are taken care of.

Micah Loewinger: I've seen credulous profiles of executives who are combating the shoplifting epidemic at their store nationwide. We've seen this kind of breathless TV reporting that includes these sensational, but otherwise not representative clips on TV news. We know what bad coverage looks like. What does good coverage of shoplifting look like?

Nicole Lewis: There's one simple thing, just one thing that reporters can do when they're faced with these stories, when they have law enforcement or they have executives coming to them saying, "This problem is so out of hand and we got to close all these stores." You say, "Thank you for this information. How do you know it is true and can you show me?" We need to see the data and the sources that they're looking at, that inform their talking points.

Micah Loewinger: Nicole, thank you very much.

Nicole Lewis: Thank you so much for having me.

Micah Loewinger: Nicole Lewis is the Engagement Editor for The Marshall Project.

[music]

Micah Loewinger: When it comes to fear around shoplifting it's more about vibes and dramatic footage than data. The same could be said about crime in general. Jeff Asher is a data analyst and co-founder of AH Datalytics. In his recent Substack column Americans Are Bad at Perceiving Crime Trends, he posed the question, why do people always think that crime is rising?

Male Speaker 7: The Gallup poll shows more Americans fear becoming victims of crime. A near record 40% say they're afraid to walk alone at night within a mile of their home.

Micah Loewinger: Every year Gallup conducts a survey about how crime is perceived in the US. Its latest results were released in October.

Female Speaker 10: The last time we saw this level of concern about crime was back in 1993 which Gallup points out was "one of the worst crime waves in US history."

Micah Loewinger: The problem with the way Gallup conducts the poll says Jeff Asher, is that while the term crime covers everything from jaywalking to theft when polled people go straight to murder.

Jeff Asher: It is the crime that people tend to think about the most but it only makes up 0.2% of all major crimes. We've created this situation where we're asking people, hey, what do you think about crime without actually defining it? It's like saying what do you think about football without defining do you mean American football? Do you mean soccer? Do you mean Premier League? Do you mean NFL? Do you mean college football? Do you mean my five-year-old's team?

Micah Loewinger: When it comes to the way that Gallup is asking people about their perceptions in crime which might also reveal the misconceptions people have in general, is that the people conducting the poll are baking the misconceptions into the questions with this kind of ambiguity.

Jeff Asher: Absolutely, because of the ambiguity if you mean major crime, major crime rose in 2022 because property crime after the artificial dip in 2020 and 2021 during Coronavirus where everybody was home, it went down and so it rose relative to that in 2022. Other types of crime, violent crime fell. Murder fell in 2022, but without knowing which of those the person the respondent is thinking about it's hard to necessarily respond.

Micah Loewinger: The second factor that you cite for why Americans aren't great at knowing how our country is doing on crime, is that specific questions about crime are actually hard to answer.

Jeff Asher: Things like violent crime have fallen substantially since the '90s. 40% decline in violent crime, large decreases in murder, large decreases in property crime since the '90s, but they're not asking if these things have changed since the '90s they're asking have things changed since last year, and the year-to-year changes have been way more subtle. Violent crime fell very slightly in 2022 according to the FBI data, it rose very slightly in 2021 according to the FBI data. If it's a 1 or 2% change in crime for 5 to 10 straight years, I think it's a lot harder for somebody that's not inherently a data expert to understand year on year whether or not it's going up or it's going down.

Micah Loewinger: Which brings us to your third point that the data is hard to come by. First off, can you just describe for us how national crime data get compiled?

Jeff Asher: Each agency reports each year, they have until April of the following year to send to typically their state UCR, state Uniform Crime Report program, all of the major crimes that occurred in a given year. They send it to their state UCR program. The state UCR program collects it, sends it to the FBI, the FBI collects it all and publishes it. There's a long lag between formally when these agencies get the data and when they have to report it up the chain, and when it's actually reported nationally. Usually, it's a 9 or 10-month lag.

There are a handful of agencies, probably a couple of dozen, maybe 100 agencies, that report their crime data directly on their website but there are 18,000 agencies nationwide. Yes, it's great that 100 agencies maybe do this, but that's a tiny sliver of all of the agencies. The vast majority of people that live in the United States live in a place where it's a desert for crime stat that are updated. It makes it really hard for people to necessarily answer a question of, is crime going up? You really have to go by the feels rather than the data.

Micah Loewinger: If we just acknowledge that the public perceptions of crime polling from Gallup were spotty in the '90s, I'm just curious if we go back to that decade, given the fact that in the '90s we had nearly 2 million violent crimes a year compared to now, when we're about 1.2 million. Were people saying similar things about crime then, too?

Jeff Asher: In the early '90s the data from Gallup shows that over 80% of people thought crime was going up. That was regardless of whether it was at its peak, which occurred in the early '90s. I think in many degrees, they were responding to the fact that it was really high and it wasn't necessarily falling really fast until the mid to late 90s. It did, in the early 2000s, bottom out at around 40% of the US believing that crime had risen in the United States, which is more correct but is still 40% of the public being wrong.

Micah Loewinger: Which brings us to point number four, which you describe as the media doesn't cover the planes that land. What did you mean by that?

Jeff Asher: It's the saying that Chris Hayes-

Micah Loewinger: The MSNBC host?

Jeff Asher: Yes. -mentioned on Twitter a couple of months ago that solidified a thought that I'd had in my head. It's a great saying because you think about it, there's never been a report that there were no robberies yesterday. There were no murders yesterday, there were no thefts yesterday. Sometimes you get like a streak of-

Micah Loewinger: Days since six straight--

Jeff Asher: -several days and there's no murders and maybe they're going to report it, or you get, "Hey, carjackings are falling 30%, we're going to report that," but there's never a day that the lack of a crime gets reported.

Micah Loewinger: Business as usual just isn't a story usually.

Jeff Asher: Right. I'm not blaming the media for necessarily covering these things, but in the vacuum of everything else that we've talked about, it makes it very difficult for people to deliver accurate impressions of what's actually happening. They get overwhelmed by the anecdote, and they don't consider the data.

Micah Loewinger: Number five, the last point on your list of why Americans are bad at perceiving crime is partisanship. How do Democrats and Republicans feel about crime now? Are there trends that you're looking at?

Jeff Asher: Between 2000 and 2020 when Bush was in office, more Democrats than Republicans thought crime was rising. When Obama was in office, more Republicans than Democrats thought crime was rising. When Trump was in office, more Democrats thought crime was rising. Under the Bush years, when crime was reasonably down, a majority of Democrats were saying that crime was rising each year. Now, over the last few years, things have gotten broken. In 2023, the Gallup survey showed 91% of Republicans saying that crime was rising in the last year, versus 58% of Democrats, which is the highest percentage of Democrats that have ever said that crime was rising.

Micah Loewinger: Yes, so there's two things going on. There's the fact that right-wing concern for crime rates. Is it an all time high?

Jeff Asher: Not even close. Prior to 2020, it was under 60% every year.

Micah Loewinger: Democrats too, are also citing higher crime rates, which flies in the face of a trend where typically their guy is in office, and therefore they would seem less concerned. Why? What accounts for these changes?

Jeff Asher: Previously, partisanship was a decent answer for this. Now it's some sort of hyper-partisanship broken media vacuum that's pumping wrong information and misinformation into the system and is leading to 9 out of 10 Republicans saying that crime is rising.

Micah Loewinger: I want to ask you about how journalists and news consumers can do a better job of closing the gap between the anecdote-born perceptions and the data. Our perceptions of crime and the facts. Readers, viewers, listeners, what should we keep in mind or look out for when we're consuming news about a crime?

Jeff Asher: We should keep in mind that crime data is generally flawed.

Micah Loewinger: Do you mean the numbers are wrong or that the numbers simply lack context, like year-to-year or decades-long trends, et cetera?

Jeff Asher: Yes, the numbers are wrong. They're always wrong. They're always estimates. They're estimates because not every agency reports, not every agency reports perfectly, and there's no good way of adding up data from 18,000 agencies and saying there were 18,242 murders last year. We just don't have that level of precision. We're always looking at estimates. That's very important to understand.

The second thing is that if pickpocketing in New Orleans surged in 2022, 50%, 70%, 80% increase, that doesn't mean that we have a rash of pickpockets, that we have a serious pickpocket problem. It just means that in 2021, we didn't have Mardi Gras, and in 2022, we did have Mardi Gras, and whenever there's Mardi Gras, you're going to have pickpockets. Looking at it just year to date does not tell you the story of what's going on. You need a longer-term view of what's happening. Year-to-date sometimes is the best that's available to us, but frequently, if you were choosing what analysis to do on data, you wouldn't do year-to-date, you'd show rolling over time.

Micah Loewinger: What's at stake when such a large portion of Americans have unfounded concerns about rise in crime?

Jeff Asher: Certainly from an electoral standpoint, it just leads to fear and not making choices based on reality. I think from a policymaker standpoint, it means that you're not necessarily making smart choices throughout the criminal justice system, how you're using your resources, the number of officers you're hiring, how you're approaching incarceration reform, how you're approaching sentencing reform, and just are people scared? Are they nervous? Are they worried? Do they think things are getting better or things are getting worse? I think in all facets of American life, when things are getting better we should have optimism.

I'm going to go to the feels now. It feels like we've lost some of that sense of optimism at the things that are getting better while still taking seriously those issues and taking seriously the issues that are getting worse.

Micah Loewinger: Jeff, thank you very much.

Jeff Asher: My pleasure.

Micah Loewinger: Jeff Asher is a writer and data analyst. His latest article, Americans Are Bad at Perceiving Crime Trends, is available on his Substack: Jeff-alytics. Coming up, mixing journalism and AI. What could go wrong? This is On The Media.

[music]

Micah Loewinger: This is On The Media. I'm Micah Loewinger. This week, two top executives were fired from Sports Illustrated's publisher. The news comes a little over a week after the tech publication Futurism noticed that something was off with certain author profiles on the Sports Illustrated site.

Male Speaker 8: Authors like Drew Ortiz, who, according to a since-deleted bio, grew up in a farmhouse, or there's Sora Tanaka, who loves to try different foods and drinks. The problem is both their photos were reportedly found on a website that sells AI-generated headshots.

Micah Loewinger: In a recent company-wide call, the majority stakeholder reportedly told employees to, "Stop doing dumb stuff." Sports Illustrated has said the dismissals this week were unrelated to the AI scandal. The outlet is one of several media companies that have come under scrutiny for their alleged or stated use of artificial intelligence. In August, the country's largest newspaper company, Gannett, rolled out a new non-generative AI service that would provide automated high school sports coverage in a number of states but readers quickly discovered that bizarre phrases like close encounters of the athletic kind had shown up in hundreds of local news stories.

Jay Allred: Our client had a PR problem on their hands.

Micah Loewinger: Jay Allred is the CEO of Source Media Projects, which includes Richland Source, a local news organization in Ohio. He's also the co-founder of Lede AI, the company that built the technology that Gannett was using to automate some of its coverage.

Jay Allred: Gannett put an indefinite pause on the project of reporting high school sports results using AI with us.

Micah Loewinger: In September, Jay agreed to speak to me about what happened, his first extensive interview since his deal with Gannet blew up. He told me that his team began building and using Lede AI in his own newsroom at Richland Source a few years ago after they learned that they could draw on high school sports results from a service called ScoreStream, which collects game results often recorded by fans.

Jay Allred: If we're looking at a football game, we're trying to figure out, was it a close game? Was it an overtime? Was it a blowout? Was it a come-from-behind win in the fourth quarter? We've grouped those different outcomes into scenarios, and then we're going to pull from a library of pre-written templates, plug those variables into those pre-written templates for the customer.

Micah Loewinger: The goal was that you could basically be offering, let's say, fairly rudimentary coverage of high school sports all across Ohio, or wherever, that your writers and editors wouldn't necessarily have to be solely on the hook for producing, and then they could go and do more meaningful coverage.

Jay Allred: We're a small newsroom. There's only 10 of us, and there's only one full-time sports reporter. There's well over 20 high schools in our region. What this lets us do is be able to provide coverage to communities that we wouldn't have been able to be at that game at all. Our sports reporter covers the A game or the number one game, we'll cover the B and the C game with our two other reporters, and then Lede AI will be in to write the briefs for us for those other three games. From that standpoint, our editor that's on the desk that night can call coaches, flesh out that Lede AI story, combining the technology that Lede AI provides with the actual journalism our newsroom provides.

Micah Loewinger: How do you communicate to readers that what they're reading was not written by a human?

Jay Allred: Every single story that publishes on Richland Source has an author, and that author is called Otto Newsdesk. If you click on Otto Newsdesk, it identifies itself as an AI tool right out of the gate. At the bottom of the article, we are disclosing that it's an AI tool that we're using. We're actually linking to Lede AI's website, we have a feedback form that publishes with every piece of content that we publish.

Micah Loewinger: How do people react?

Jay Allred: In general, the readers understand it's information, it's not journalism. Of course, a lot of times readers want the content to be longer and then to include player names and photos, and video.

Micah Loewinger: They want it to be a reported article. [laughs]

Jay Allred: Exactly.

Micah Loewinger: How exactly do you attempt to make Lede AI produce a human-sounding article? I mean, I know that with some of these large language models, they require lots of data, and this has led to a lot of controversy around AI start-ups scraping enormous parts of the web, including books, entire news outlets, entire forums like Reddit. Explain to me how you feed language and templates to Lede AI.

Jay Allred: Every single word, every comma, every semicolon in our database has been written by a person and then it's been checked by another person and checked by a person after that. It's what allows us to be confident in all cases that if we're using our standard data set that the content that we're producing is accurate as long as the data is accurate and it's very accurate.

Micah Loewinger: Okay, that's interesting because in late August people on social media began posting some of the really awkward phrases that Lede AI has put into some local news sources, the one that caught a lot of attention on Twitter for instance was a piece in the Columbus Dispatch and some other Gannett owned papers. Readers finding examples of Lede AI using phrases like, quote-unquote, ''Close encounters of the athletic kind.'' There were a lot of articles referring to high school sports action, or how one team, ''Took victory away from another team.'' These are phrases that most human journalists would consider ranging from awkward to poor writing, so how did that happen?

Jay Allred: I knew you were going to go there and I'm glad you did. In mid-August, our technology powered a really big launch with Gannett across, I think, six or seven major markets in the US. We had written some custom code for that particular customer, and the code had bugs in it, Micah. Some of those things that showed up in those Gannett articles, especially the errors were the result of a small company working really, really hard to get ready for a launch with a very big company. As far as the awkwardness of the phrasing and the now infamous close encounters of the athletic kind. A human being wrote that, Micah.

[laughter]

Jay Allred: A person wrote that, and we got called out on a few phrases and they are no longer in our database. It was as simple as taking them out.

Micah Loewinger: I was curious to know if this was a feature or a bug. I actually just searched some of these phrases on the Richland Source and I counted over 140 articles on the Richland Source from this year that featured the phrase, close encounters of the athletic kind, or similar phrases, including 50 articles from this year that featured the phrase close encounter of the winning kind in the headline, so I don't really buy that it was just a fluke that happened with launching a new service with Gannett like you have been publishing these sentences for years. [laughs]

Jay Allred: No, and I appreciate you calling that out because that phrase has been in our code for years. The things that were unique to the Gannett launch were some other great, he says sarcastically, some other unfortunate stuff. For example, there were a couple of leads that published in some of the papers where we had plugged in a variable where there should have been a mascot name. There were instances where we published two very similar lead paragraphs, they said exactly the same thing in terms of factual information, but they said it slightly differently. Those were bugs that were built into that custom code, but those awkward phrases that the internet called out, that's been there for years.

Micah Loewinger: I guess this is what sends a shiver down the spine of media critics and journalists and editors because we're talking about high school sports, this is not the highest stakes beat in all of journalism, but it seems like it does speak to the risk of automation where one small mistake when automated becomes 150 small mistakes all across the country.

Jay Allred: Yes, absolutely. What if this had been crime reporting? What if these had been arrest reports? Real harm could have been done. As leaders in the industry, I think it should give us all pause, it's why I'm having this conversation with you.

Micah Loewinger: I appreciate that. I appreciate, your vulnerability and your openness to introspection. Are you at all concerned that local newsrooms would see the promise of Lede AI maybe think that it's capable of doing more than it is and kill entry-level jobs?

Jay Allred: I think about that every single day. In three years of talking to news leaders around the country, I've never once heard one of them say I'm super excited for AI because I get to reduce my head count.

Micah Loewinger: Well, no one says those things, they say we would like to be more efficient.

Jay Allred: I agree with you and with all of those things said I still lose sleep over it at night.

Micah Loewinger: What do you lose sleep over?

Jay Allred: Is the intention to use your euphemism to find efficiency and to do that through less people, or is the intention to create more value for consumers so that we can get the nose of this airplane pointed up and we can start to create a future where local news entrepreneurs can think of local news as a good small business. I think that there's lessons to be learned here and we can grow as an industry and get better because the reality is this stuff is, it's not coming, it's here.

Micah Loewinger: That's what I've heard in some of your answers is still this implicit belief that the rushing river of technology is coming no matter what. I wonder if this is a moment in time to say like, there might be some uses for AI, but we don't just have to see it to its logical conclusion just because technology is great, bro.

Jay Allred: Yes, I agree with you. I think we should use techs like Lede AI to report unreported stories that would never go reported otherwise. We should interrogate that technology vigorously and make sure that it can be trusted and be accurate and I know that there are ways to do that.

Micah Loewinger: You've been forthcoming about the mistakes your team made and the limitations of the technology. Do you feel that any of the backlash to AI within the media has been unfair? Like, has any of it, you think, missed the mark?

Jay Allred: I think that our industry has a tendency to respond to stuff like AI from a very defensive position. It's super understandable, our industry has done nothing but cut newsrooms to the bone for going on two decades now. I wish we could get into spaces where we understood that we were more all in this together and that we are trying to figure it out. I think we as an industry need to be able to hold multiple things to be true at the same time, which is malevolent deployment of AI inside of our industry is going to hurt our industry. Intentional thoughtful deployment for the benefit of readers and communities and reporters can benefit our industry. Both things might happen. I hope it's the second that's going to be the work I continue to do.

Micah Loewinger: Jay, thank you very much.

Jay Allred: Thank you, Micah. I was glad to be invited onto your program.

Micah Loewinger: Jay Allred is the CEO of Source Media Properties.

[music]

Micah Loewinger: That's it for this week's show. On the Media is produced by Eloise Blondiau, Molly Rosen, Rebecca Clark-Callender, and Candice Wang, with help from Shaan Merchant. Our technical director is Jennifer Munson, our engineer this week was Brendan Dalton. Katya Rogers is our executive producer, On the Media is a production of WNYC Studios. Brooke will be back next week, I'm Micah Loewinger.

[music]

 

Copyright © 2023 New York Public Radio. All rights reserved. Visit our website terms of use at www.wnyc.org for further information.

New York Public Radio transcripts are created on a rush deadline, often by contractors. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of New York Public Radio’s programming is the audio record.