BROOKE: This is On the Media. I’m Brooke Gladstone. You may have heard about the latest food once thought healthy, now turned lethal:

 

CLIP_AOL: Kale might not be as good for you as you think….

CLIP: The vegetable is known to absorb heavy metals like arsenic and thallium. According to the article, people who are eating large amounts of it are getting sick. ..

CLIP_AOL: … fatigue, skin and hair issues, neurological disorders, foggy thinking, digestive problems, and gluten intolerance.

 

BROOKE: Gluten intolerance! Oh no! For a few weeks the IT leaf might as well have been hemlock in leathery clothing. This particular panic was brought to you courtesy of a stunningly unsubstantiated article in Craftsmanship Magazine.

 

Kale’s momentary fall from grace is just the latest link in a perpetual feedback loop of specious health and diet stories proclaiming that a favored food stuff,  red wine for instance, will either kill you...

 

CLIP: Wine, or any alcohol, increases the risk of breast cancer.

 

BROOKE: Or save you…

 

CLIP: ladies, drink your red wine! it could prevent breast cancer

 

BROOKE: Or kill you…

CLIP: I probably told you hundreds of times that red wine is good for your heart, but recent news is raising questions.

 

BROOKE: Or save you...

 

CLIP: It’ll help you with good cholesterol, lowers blood pressure, lowers stresses, and overall just has many, many health benefits.

 

BROOKE: And so for the remainder of this segment, we’re diving into the dodgy world of health and diet news, complete with two of our Breaking News Consumer’s Handbook which you can find on our website, to help you navigate. Let’s start with an object lesson.

 

John Bohannon is a contributing correspondent at Science magazine. Back in December of 2014, he received a call from German tv reporter Peter Onneken, who had an intriguing proposition.

 

BOHANNON: let’s do a real study, with real people, to test the health benefits of chocolate. We’re going to do a really bad job on the statistics and the design of the experiment. and get a bad paper published, and build a global multimedia campaign around it. And, see how far it goes. In Germany it’s as big a problem as it is here. There are diet fads that come like tsunamis every few months with very little scientific backing, And so you wonder, how did journalists miss this? Why didn’t they smell a rat when they started looking into this story in the first place.

 

BROOKE: Bohannon signed on to the plot, and so the study was launched. Onneken <OH-neh-ken> and his partner, Diana Lobl <LUH-bull>, used Facebook to recruit 5 men and 11 women, aged 19 to 67. A German doctor named Gunter Frank oversaw the three- week trial. Yeah. 16 people. Three weeks.

 

BOHANNON: People were randomly assigned to three groups. One was the control, so basically for a few weeks it's going to be life as usual. Another group was on a low carb diet. nothing drastic, kind of a standard low carbohydrate diet. And then the third group was the same low carbohydrate diet plus a bar of chocolate every day.

BROOKE: an ounce and a half of bitter chocolate, like over 80% cocoa.

BOHANNON: That was Gunter Frank's idea. He says that in Germany there's this almost religious movement for foods that taste bad, therefore they must be good for you. A bitter chocolate - you have to make it slightly unpleasant.

 

BROOKE: If they could discern any difference at all between the low carb chocolate eaters and the others, they would proclaim that chocolate worked!

 

BOHANNON: Now here's the trick, if you want to get a false positive effect, something that looks sexy but probably isn't real, measure a ton of stuff about those people\.

 

BROOKE: So they recorded 18 different measurements: weight, blood protein levels, cholesterol, sodium, sleep quality, well-being… everything. Except for one, seemingly crucial marker for a diet experiment -- what they ate.

 

BOHANNON: Ohhh, well. [laughs] If you want to be a stickler. We actually didn't care really what they ate. Because our goal wasn't to do good science, our goal was to create a big, multidimensional data set that we could go treasure hunting in.

 

BROOKE: And the results?

 

BOHANNON: That if you eat chocolate, you lost a pound or two 10% faster than the group that didn't each chocolate.

BROOKE: Less than the amount a woman could possible gain and lose in the course of a month with her menstrual cycle.

BOHANNON: You got it, that could just be water weight.

 

BROOKE: Now the easy part: publishing the result.  As we’ve covered before on this show, the pay-to-publish scientific journal industry is notoriously easy to buy into, and the sleaziest are known quantities.

 

BOHANNON: And I just tried to do my shopping on the list of known bad guys. And sure enough, I started getting acceptances within 24 hours.

 

BROOKE; Two weeks later, for the bargain price of 600 Euros, they were in print. The final touch was Bohannon’s, a hot and sticky press release.

BOHANNON: It really should be kind of a perfect little story. It should tell you why you should care, what happened, be very specific, and then provide a lot of what we call color - quotes and funny connections if they exist. Something I didn't do, this was coming out pretty close to Easter and it's a study about chocolate! I should have made that connection. Luckily, all of the journalists who covered this had the same idea.

BROOKE: And you had made up quotes, one by a certain Mr Johannes Bohannon.

BOHANNON: Oh please, Dr. Bohannon. Herr Doktor Bohannon said the best part about this discovery is that you can buy chocolate everywhere! Cheaply!

BUTT CUT TO:

SCHWITZER: What makes my skin crawl is when I hold the news release in one hand and the story in the other hand, and the quote is lifted verbatim from the news release without attribution.

 

BROOKE: Gary Schwitzer, the publisher of healthnewsreview.org is a dogged exposer of health news shams, scams, and all-around slop.  

 

SCHWITZER: It makes the news organization look like independently sought out that work, and the reporter never left his or her desk. You’re not living up to your responsibility as a journalist if you do that.

 

BROOKE: And so, the first item in our Breaking News Consumer’s handbook: Health News Edition: watch out for single source stories: they’re usually based on press releases. Item number two: beware stories that don’t mention cost.

SCHWITZER: Reuters had a story about a immunotherapy combination promising in a melanoma study. The drug in question costs around $30,000 per injection, and the combined cost more than a quarter million dollars a year. The story didn't touch on cost at all. How can we be talking about how wonderful a drug is and not talk about costs?

 

BROOKE: Something else to consider  - when a new treatment is said to reduces your risk of disease by say 50 percent, is it a relative 50 percent, or an actual 50 percent? Stay with me.

 

CLIP - It’s the Omega-3s in fish like you mentioned like salmon that will reduce your risk of depression up to 50% ...Ask yourself, 50% of what? Here's a classic example. When a drug company marketed its drug for osteoporosis --

BROOKE: This was a MERC drug, right?

 

SCHWITZER: Yup. They marketed it as reducing the rate of hip fracture by 50%. The data showed it reduced the risk of hip fracture from 2 in 100 in the untreated group to 1 in 100 in the treated group. That is indeed a 50% relative risk reduction, but in the absolute look at how the group benefited, 1 in 100 benefited. It is a 1% absolute risk reduction, and the other 99 had to take the drug, run the risk of side effects, and pay for the drug and stand no chance of benefit.

 

BROOKE: Item four: if you think  it sounds too good to be true,  it probably is.

 

SCHWITZER: If you hear a story about a new intervention and all you hear is predominantly about the benefits, run for the hills. Because there are trade-offs in all healthcare decisions about all healthcare interventions.

 

BROOKE: One way you’ll know you’re not getting the full picture? The fifth item on our list: beware too many patient testimonials.

 

SCHWITZER: You know, there's this old saying, the plural of anecdote is not data. We write a lot about imbalanced news stories about screening tests. Patients saying, 'I'm glad I had that mammogram', 'I'm glad I had that PSA blood test for prostate cancer, because it saved my life.' Something that we can never prove.

 

BROOKE: Which brings us to item six…

 

CLIP: Now for the first time, there’s evidence that a simple screening test can catch lung tumors earlier…

BUTT CUTTO:

SCHWITZER: If you ever hear somebody talking about a simple screening test, a simple blood test, there is no such thing. Screening test decisions should be among the most complex that we face in all of our healthcare decision making. Some people will say, 'It's not a screening test decision, it's a decision about what you do after the results of the screening test.' To those people, I say, you have not talked with people who have experienced harms and regrets as I have and as many people have.

 

BROOKE: And there are other words you should watch for, too. Words like…

 

[this montage is swirling, with the single words popping up]

CLIP: Tonight, a true breakthrough in the fight against heart failure.

CLIP: what may be a breakthrough in helping millions of people…

CLIP: It’s called a medical miracle treatment saving lives.

CLIP: Plus a treatment that’s the first of its kind.

CLIP: New treatments could be a game-changer for the millions of people with diabetes.

CLIP: In tonight’s health alert, a game-changer in the way breast cancer is treated.

BUTT CUT TO:

SCHWITZER: First of all, this isn't a game. Let's stop calling it a game changer. Is this making a difference in people's lives, or is it making a difference in people's pocketbooks first, before all of the evidence is in hand? We hear, 'this could become the new standard of care.' Any time you hear 'it may become,' 'it could become,' substitute 'it may not become,' (Brooke laughs) 'it might not have.' Because that is indeed the case.

 

BROOKE: Item 8, Be sure that the new treatment is put in context with existing alternatives. New isn’t always better.

 

SCHWITZER: There's a phrase in medicine right now that I think we ought to adopt in journalism, and it's called slow medicine. And it's clinicians talking to each other, saying, This is getting outta control. Let's slow down. (Brooke laughs) Let's talk with patients. Let's make sure we do a good history before we pull out that prescription pad.

 

BROOKE: Related to this slow medicine movement, the ninth item in our handbook: beware of disease mongering. This means turning normal conditions, like balding, into pathologies...and

 

SCHWITZER: Disease mongering can also apply to treating risk factors for disease as if they were diseases unto themselves. So, maybe the classic example is LDL cholesterol. Do we make the public fixate on "know your numbers" without dwelling on the numbers that really matter, like, did this have an impact on the rate of heart disease, heart attacks, or death from heart disease? In disease mongering, we lower the threshold so we now get more bent out of shape about blood pressures, blood cholesterol levels, blood glucose levels, that are lower than what we used to get bent out of shape about 10 or 15 years ago.

 

BROOKE: Item ten: treatment availability is often a mirage.

 

CLIP; All of these drugs still need FDA approval.

CLIP: ...the company is hoping for FDA approval by the end of the year

CLIP: The test is still in pre-clinical trial phases, and the FDA still has to approve it.

BUTT CUT TO:

SCHWITZER: Anytime you hear anything like that, take the crystal ball and throw it out the window.We often see that in phase 1 drug trials. Phase 1 studies, they're meant primarily to say, hey, before we go forward, let's make sure it's safe. But you'll see stories about phase 1 trials that make claims about benefits. As if these things were available at our corner drug store and they're not.

 

BROOKE: And finally, item eleven: the distance between mice and men.

 

CLIP: If a study on mice is any indication, the idea of yo-yo dieting may actually be good for you after all.

CLIP: Researchers studying mice found that exposure to cell phone radiation during pregnancy affected brain development.

BUTT CUT TO:

SCHWITZER: If the leap from a phase 1 human trial to broad, widespread human use is an incredible leap, the leap from mice to men and women -- well, this month we had these great pictures of Pluto, it's about that much of a leap.

 

BROOKE: And so concludes our Breaking News Consumer’s Handbook: health news edition.

 

BROOKE: Now let’s check back in with John Bohannon. When we last left our hero, he had successfully completed a three week long sham trial, designed to illustrate the weight loss potential of dark chocolate. Armed with an article printed in a pay-to-publish medical journal and a killer press release, it was time to see what news outlets would even take the bait.

 

BOHANNON: The largest tabloid in europe called Bild published out of germany did a cover story on it. You know, that reached a huge number of people immediately. And after Bild it switched to the british tabloid newspapers. Then it spread to the daily star, the irish examiner. It hit the Times of India, cosmopolitan's german website, and then it spread to television -

 

CLIP MONTAGE:

CLIP: German scientists found out that your diet works better, faster if you combine it with dark chocolate!

CLIP: This is real - adding chocolate, you'll lose weight i'm not kidding

CLIP: a study out of germany found out that chocolate not only accelerates weight loss, but also aids in sleep quality.

 

BROOKE: To be fair, The study wasn't picked up by the NY Times or the Washington Post. The Daily Mail or Bild, they're not known for their journalistic rigeur.

BOHANNON: Where do you think the vast majority of people who pick up a fad diet learned it? It's not from the New York Times. they learn it from the Daily Mail and the Huffington Post and the Internet. I mean, it's all built on fake science.

 

BROOKE: It’s not just fake studies like Bohannon’s that trickle up through the tabloid press. There are countless life enhancers, and beautifiers, fixtures in glossy magazines. Each backed one by experts who stand as living proof of their efficacy. Those radiant celebrities.

 

CAULFIELD: You know these are not evil people, there is not a star chamber with Gwyneth and Oprah where they’re making decisions about what we’re supposed to eat.

 

BROOKE: Timothy Caulfield is the author of Is Gwyneth Paltrow Wrong about Everything: How the Famous Sell us Elixirs of Health, Beauty, and Happiness.

 

CAULFIELD: Actresses and actors, sports figures, musicians, they’re under more pressure than any of us to look good, to stay thin, to never age, so they’re looking for solutions for that problem and often they’re just talking about it in social media and in the popular press.

 

BROOKE: While Caulfield urges us to have compassion for celebrities, he nevertheless finds fault with the the fad diets and bogus beauty treatments they endorse. So now, our second breaking news consumer’s handbook: Celebrity Expert Edition. First up, beware the B - I - E - B, or, BIEB.

 

CAULFIELD: Beauty, industry, efficacy, bias. Consumers want it to work, the health care providers want it to work, obviously industry wants it to be portrayed as if it works and magazines aren’t going to sell magazines telling people none of this stuff works, so all the social forces kind of point in the direction of making people think this stuff is effective.

 

BROOKE: The BIEB,he says, affects us in two ways: first, fad diets.

 

CLIP: Celebrities like Madonna and Demi Moore are said to be big fans of a new diet called the Werewolf Diet. It’s based on a theory that the moon affects water in our bodies in the same way  it determines the ocean’s tides. Advocates claim that fasting with juice cleanses during a new moon or full moon can help you lose as much as 6 pounds in a single day.

CLIP: Finally, the baby food diet...Yes, Jennifer Aniston has made this baby food diet famous. What you do on the baby food diet is you replace your daytime eating with baby food

 

BROOKE:  Sure, those just sound stupid. But  how about cleansing, fasting….detoxing?

 

CAULFIELD: They’re scientifically ludicrous on every level. There’s no evidence we need to detox, we have organs that do that, and in addition to that there’s no evidence that the proposed regiment, whatever that is, is going to help you detox and I hate the messaging behind it. It’s an extreme strategy, some sort of quick fix which will solve your weight problems and your health problems when what you really need to do is find a lifestyle that you can sustain forever.

 

BROOKE: And don’t talk to Caulfield about gluten!

 

CAULFIELD: I don’t think this trend would exist but for celebrity culture, by the way, without people like Gwyneth and Miley Cyrus and all the sports figures who say that they benefit from going gluten free. So is going gluten free healthier? No. A study just came out that said gluten free food is no healthier. In fact in some cases it might be less healthy; there might be more sodium and less healthy fats in there. Is it a good weight loss strategy? Again, no evidence to support the idea. Some studies have even suggested or hinted that people put on weight when they go gluten free long term. Short term people probably lose weight because again they’re concentrating on what they’re eating. Then we come to the real controversial thing and this is the concept of non-celiac gluten sensitivity.

BROOKE: By which you mean if you don’t have a diagnosed case of celiac disease; the belief is that you could still be sensitive to gluten and it could still cause everything from fatigue to migraines. There’s lots of anecdotal evidence, people I know who have gone gluten free and say it’s changed their lives. So maybe they’re gluten sensitive?

CAULFIELD: So look, I’m not going to say everyone is responding this way but placebo effect has a big impact here. But let’s put that aside, there is an ongoing debate in the academic literature about whether this phenomenon exists. There’s no way it could account for the growth in the popularity of going gluten free. So one percent of the population is celiac and there may be an addition of 1 to 6 percent on the outside who may have some sort of non-celiac gluten sensitivity. The best studies that have been done on this where they have done actual clinical trials have found no gluten sensitivity so I just think people need to realize that science is way more equivocal than often portrayed.

BROOKE: The BIEB  reigns supreme in the beautification industry. Consider Kim Kardashian’s vampire facial, whereby a substance distilled from your own blood is injected back into your face. Or the sheep placenta facial, allegedly favored by Victoria Beckham. Or drinking 8 glasses of water a day. I can’t believe I just said that.

 

CAULFIELD: That’s one of the most persistent myths I came across. It’s just not true and people just swear by it. Look, you drink when you’re thirsty, full stop. And there’s no evidence that drinking all this water like Jennifer Aniston says we should is going to make your skin glow and rejuvenate blah blah blah. Just doesn’t work.

 

BROOKE: Speaking of glow and rejuvenate, they’re on a list of watch-words Caulfield has assembled to indicate a “science free zone.” Also on that list: revitalize, beautify, clinically proven...

 

CAULFIELD: What does clinically proven mean? Does that mean they did a study and asked two of their patients and the patients thought they looked better? What kind of clinical study was it? Dermatology approved what does that mean? That doesn’t necessarily mean there’s any scientific evidence to support it. If there was an anti-aging product out there I swear to god we’d know it worked because the data would be everywhere.

BROOKE: And if you see the word toxin?

CAULFIELD: Toxins have kind of become the modern day evil spirits, that they're somehow residing in our cells and making us fat and making us tired and depressed and there’s just no evidence to support that idea. And again no evidence that any of the products and processes that are being marketed by celebrity culture are going to rid your body of toxins.

 

BROOKE: Amidst all this dream-crushing, you have to wonder if anything is effective.

 

CAULFIELD: You know what really works? You know what the real anti-aging things are? Don’t smoke. Stay out of the sun. Get exercise, exercise seems to have an actual physiological impact on your skin. Eat well, I know this is boring stuff but, you know, the dream crusher, right?

BROOKE: And get sleep.

CAULFIELD: One thing I was most surprised about was the evidence around sleep. Those are the things that we’ve been told for a long time and those are the things that you should do.

BROOKE: He’s a right buzz kill, he is. But you suspect he’s right. So why do we continue in denial? In magical thinking? Because we want to escape death. And in the case of  chocolate... because we like it.

 

BOHANNON: So many people contacted me in the wake of this saying, you know i'm so disappointed.

 

BROOKE: John Bohannon.

 

BOHANNON: Even reporters would guiltily mention this when they were interviewing me about the whole stunt. they would say part of me just hoped beyond hope that chocolate could really help me in some way.

BROOKE: You know I had that experience myself.
BOHANNON: Look, it might actually be the case that chocolate helps, you can't draw the opposite conclusion. You really can't draw any conclusion from this study, it's like reading tea leaves. And that was the point. If you do really bad science, you can make up any story you want, if you're eating chocolate and it seems to be helping you, stick with it!

[music up and under]

 

On our website, you’ll also find our second breaking news consumers handbook related to health news:The Celebrity Edition, as well as our Spot the Tomfoolery quiz, where you can find out how adept you are at recognizing  real bogus health stories. From fake bogus health stories. Very meta.

CLIP: So grab some chocolate, and let’s lose some weight!