Anecdo or Anecdon’t? Traditional Science vs Anecdotal Evidence in The Age of the Coronavirus
By: Dr. Eeks
As COVID-19 takes over our entire lives, one thing is evident: No one definitively knows how to treat it. This makes sense, because it’s a new infectious disease; the new kid on the block who everyone is desperately trying to understand. Treatment is supportive, essentially keeping fevers from getting too high, airways open and oxygen flowing into infected lungs. In its most serious form, COVID-19 can lead to an aggressive form of pneumonia that rapidly progresses into Acute Respiratory Distress Syndrome and death.
The fact that there is no approved drugs or vaccines for COVID-19 has created a situation in which new drugs and therapies are being used. Perhaps one of them, or a combination of them, will work. This sentiment makes sense. We are fighting a deadly epidemic that is overwhelming healthcare systems and killing thousands of people, there is no approved treatment(s), we don’t have time for randomized controlled trials, so let’s just try something. Anything. Let’s see what happens, and really, what do we have to lose? Given our dire straits, even the strictest, traditional scientists and doctors are relaxing standards about what should be used as medicine and what shouldn’t.
We also live in a social media-obsessed world, where anyone with a large enough following, degree of likeability and ability to portray a level of expertise can act as a health and wellness expert. Or at least be viewed as one. Nowadays, they have easy access to user-friendly tools to create professional-looking posts, charts, pictures and infographics that they can share on Twitter, Facebook and Instagram, with the potential for others to see them, share them and make them go viral. Some of these people might be incredibly sharp and share helpful information, but there’s no quality control, no requirement for showing evidence, no need to have any credentials, no risk of losing any kind of license and no oversight committee to answer to. They can say whatever they want, and they often have a deep distrust of traditional medicine and doctors. Instead of the Wild West, I often refer to this phenomenon as Wild Wellness. Some are very good at what they do (and I mean very good) though I wish we could have a ranking system to weed out the good and helpful from the misinformed and dangerous.
In the battle of COVID-19, I’m witnessing both rigorous, traditional scientists and social-media fueled health gurus share a slew of anecdotes about what might work against the virus. This is expected, as we are in a race against time, folks are dying, and everyone is anxious and desperate for a cure. In some ways, everyone is on the same playing field, and it’s a matter of observational skills. What helped? What doesn’t? How many observations equal a safe amount of data to dictate a broader action? Does it even matter in a pandemic? If someone in your family has the virus and is knocking on Death’s door, wouldn’t you try anything? Anecdo or Anecdon’t: that IS the question.
I don’t have a problem with anecdotal evidence. In fact, I practice several wellness routines and ingest homemade “health” concoctions that are not supported by one iota of a randomized controlled trial. The thing is, I know that. I know what I’m doing is rooted in anecdotal evidence, but I do it anyway. Perhaps it’s because I know no one would ever fund or conduct a randomized controlled trial on what I’m doing, what I’m doing is super low risk compared to what I perceive as benefits, or I’ve been impressed with smaller studies and anecdotes. Just because something isn’t supported by a randomized, placebo-controlled trial, doesn’t mean it doesn’t work. However, I’m fully aware that what I’m doing or swallowing is not supported by a randomized, placebo-controlled trial, which is, hands down, the gold standard of evidence. As long as I’m aware, I’m okay, and as long as you’re aware of the level of evidence for a perceived treatment or intervention, I’m okay too. It’s all about being honest about the level of evidence.
Why is a randomized, placebo-controlled trial considered the gold standard? Because it’s the only legitimate way to prove that the outcome(s) you are observing are due to the treatment or intervention in question. Ideally, it’s best to have repeat randomized, placebo-controlled trials showing the same outcome. That’s a stronger level of evidence. Randomization is important to minimize for bias and to ensure that the characteristics of the treatment group and the control group are similar. Finally, our eyes and minds trick us, all the time, which is why it’s important to control for placebos. There are additional ways to make a study “stronger,” but those are the basics.
There are a lot of reasons why randomized, placebo-controlled trials don’t happen, for example lack of time, interest or funding. It’s often said that unless something can be “patented” and turned into a profit, it won’t get funding for a study. That’s probably true. 80% of my “wellness routine” can’t be patented, so I doubt it’d ever receive funding. Randomized, placebo-controlled trials aren’t cheap either. That’s why a lot of them are conducted by drug companies looking to patent a drug for a particular condition. Drug companies are rich and can pay for them, and they have access to top researchers at top universities and hospitals.
So when there’s no interest, time or funding for a randomized, placebo-controlled trial to test a particular treatment, what should we do? I’m not an expert on ethics, but I think the answer is: Just be honest. Tell people the truth. I work in the “world of wellness” and I often see home remedies, essential oils, supplements, anti-aging creams, diets and detoxes, with zero randomized, placebo-controlled trials to support them, presented as game-changing miracle cures: “This absolutely works,” “This is so amazing,” “This has the potential to change your life,” “This will make you better,” “This kills the virus.” Subsequently, people get so excited about these marketed products and start buying them off the shelves. I saw this type of advice a lot in the last weeks- from specific oils that destroy COVID-19, to steam treatments, to a commissioner in Florida telling folks to hold a blow dryer up to their noses. For some reason, folks don’t ask too many questions. They just want to believe. This trend always felt like a religion to me, hence my drawing. The drawing is from my new comedy book, Yours in Wellness, Krystal Heeling, in which I talk about anecdotes vs the scientific method A LOT.
Recently the president tweeted that the combination of hydroxychloroquine and azithromycin “might be a game-changer” in the fight against COVID-19. While hydroxychloroquine has/is being used in China and elsewhere ( even here in the US), my understanding is that his tweet stemmed from a study conducted in France. The president, obviously, has a lot of influence. When he tweeted that the combination of drugs was a potential game-changer, people immediately tried to get them. Now there are shortages. Later that day, while I was out walking my dog Barnaby, I ran into a guy who manages a hotel near me. I kept six feet away, at least. He told me that he felt “super confident” that this will “all be over soon” because of the “new malaria drug Trump tweeted about.” That made me pause. I tried to explain to him that the drug combination may not work, and it’s still too early to say, but he didn’t pay attention to me. I don’t have the power Trump has. Then I started to think: What if this “potential game-changer” changes the way people behave? Will people become more lax about social-distancing and hand-washing? How far will their optimism carry? If too far, that’s a public health official’s biggest nightmare.
Later in the day, I sat down to read the French study. Obviously, it wasn’t a randomized, placebo-controlled trial and it was very small. Originally, 26 COVID-19 positive patients received hydroxychloroquine and 16 control patients received treatment as usual. Of the 26, 6 were lost to follow up and not included in the final analyst. I tried to figure out what happened to them. From the study, it looks like several were moved to the ICU ( did they die?), one died and another discontinued treatment. If they weren’t included in the final analysis, would that skew the results in favor of the drug combination? Yes, absolutely.
Of the 20 patients remaining, who were all taking hydroxychloroquine, only 6 were given Azithromycin. The primary endpoint was virological clearance at Day 6, not symptom improvement. The study concluded that the group taking the combination of drugs had a statistically significant improvement in viral clearance. How were they clinically? I couldn’t tell from the write-up. My understanding is that the authors did not release that information. And this is only 6 people.
Based on the methods and data, was the president’s tweet overly optimistic? I think so. I think it’s way too early to even hint that the drug combination is a “game-changer,” plus one must always consider the potential side effects of one or both drugs, but I also understand why the president is hopeful. I just think it’s important to be honest about the level of evidence. Someone who criticized me for my critique of the president’s tweet asked me if I’d use the drugs if my family member had COVID-19 and was going downhill fast. If nothing else was working? Yes. I probably would and hope like Hell for the best. But I know I’d be dealing with an anecdote, not a miracle drug.
And what about you…Anecdo or Anecdon’t?
Thanks for reading- Erin
Check out Yours in Wellness, Krystal Heeling on Amazon. 🙂
Natural tips for the flu and cold
How to use the Neti Pot: Just Say No to Amoebas
Such an awesome article! Understanding research methodology is so important in keeping us safe and in managing health in general