Facts Alone Can't Tell Us What to Do: The 3-Legged Stool of Evidence-Based Decision-Making

The COVID-19 pandemic might be finally winding down as the U.S. population approaches the point of herd immunity, but many areas of the country are again struggling with decisions about public health strategies like mask and vaccination mandates. A recent Medscape article asked the question “who are the real COVID experts?” This is relevant again as we move into a new “endemic” phase that involves neither height-of-the-pandemic lockdowns nor a clear victory over the SARS-CoV-2 virus. Usually, the experts on an infectious disease would consist of immunologists, virologists, or infectious disease physicians. But in the case of COVID-19 the issues are much more complicated than just viral transmission and treatment. We need chronic disease experts to help us understand the symptoms and treatments of Long COVID syndrome. We need mental health experts to help us process the collective trauma that many people have experienced over the past 2 years. We need economists to help us understand COVID’s lingering effects on the economy, and human-centered design experts to forecast what the “new normal” of work will look like. We need child development experts to design strategies that help children adapt both academically and behaviorally to the new environment -- including help for those kids who don’t feel ready to take their masks off as well as those who are desperate to do so. We need sociologists to help us adapt to the changes in once-stable aspects of society, like virtual work, virtual school, and virtual family gatherings, and to examine the important economic and social disparities highlighted by the pandemic. And we even need theologians to help us cope with the changes in how we worship, celebrate marriage, care for the sick, and mourn our dead. 

Health care systems experts use a 3-part approach to decide on evidence-based policies. Facts, or the best available scientific evidence, are one element of that three-legged stool. The COVID-19 pandemic has sensitized us all to how wobbly that leg of the stool can sometimes be, as seen for example in the initial haggling over whether cloth masks had enough filtration ability to be of any use at all, whether mask-wearing was dangerous for people with asthma, or whether masks were still necessary once we had vaccines. (In case you weren’t keeping score, all three questions now have answers from subsequent research: yes, no, and yes). The cartoon above illustrates the weakness that we often see in early data: Preliminary scientific evidence has a high level of variability. In the cartoon, the long tail sticking up from bar "A" means that the actual average could be as low as the height of bar "A" or as high as the height of bar "C." But eventually another scientist always comes along with better data and is able to put the question to rest. In the masking example, later research showed that areas with mask mandates and prohibition of indoor dining in 2020 had fewer COVID-related hospitalizations and deaths. What people sometimes fail to recognize is that “science” is never a settled thing, because new data can always be collected. We need to keep up on science all the time, because what we think we know is fleeting, and new facts can put past findings in a different perspective. This is also a strength of science: Anyone is entitled to challenge its findings, as long as they can provide valid data. 

Facts are only one leg of the stool. The second element of evidence-based practice involves “community standards,” which means the consensus of experts. Anyone can do science -- it's just a set of methods -- but the judgment of experts is important to evaluate and endorse scientific findings. More surprising findings often face a higher burden of proof, which is OK -- it helps us to avoid making hasty decisions. As seen in the article which which I began this post, “experts” also can come from a wide variety of fields and many different areas of expertise may be needed. For a problem like COVID-19, a consensus of experts from many different disciplines might take a while to work out, but there are consensus models available to guide such a process. It’s often best if the experts go off by themselves and negotiate a solution that most of them can live with (experts are cantankerous folks, so there will always be outliers), then figure out how to present their solution to the rest of us. Many people instinctively resist this approach, because they don’t like the idea of “elites” telling us what to do — it goes against our democratic instinct that each person’s opinion is just as good as every other’s. Don’t worry — there’s a place for that too in the third leg of the stool. But we also benefit significantly when we take the input and intuitions of experts into account. These are people who have dedicated their careers to the study of a specific issue. The nature of expertise is that it allows one to rapidly consider and discard many possible solutions using Intuitive-level thought, and to see possibilities that the non-expert might never consider. The chess master can see many moves ahead with her Queen, while the novice wastes time maneuvering his pawns.

So why not let the scientists generate facts, the experts interpret them, and the rest of us do what they say? Unfortunately it doesn’t work like that. The third leg of the evidence-based decision-making stool is patient preference, which in the case of COVID-19 we might re-label as “public opinion.” The best prevention policy in the world won’t have any effect if people don’t follow it. A lack of effective communication by scientists is part of the problem, and scientists could make an effort to present more nuanced conclusions that more accurately reflect knowns and unknowns in the current science. But people, like experts, have their own Intuitive-mind perceptions and inclinations toward particular policy choices. These perceptions are based in part on moral judgments about the relative importance of competing goals, for instance the caring-oriented goal of protecting elderly relatives versus the fairness-oriented goal of wanting to see other people's faces to get a better sense of what they are thinking. People's moral judgments clearly have affected their responses to COVID-19 over the past 2 years, and these effects can be seen at a broad societal level. A recent study found that some countries around the globe handled the COVID-19 pandemic dramatically better than others: Denmark, for example, had 40% less SARS-CoV-2 infections than the average country. These differences were explained primarily by (a) citizens' level of trust for their government, and (b) their level of trust for one another. Trust in government had an effect twice as great as that of the country's level of economic development (GDP), and trust in one's fellow citizens had an effect four times as great as that of GDP. Once someone was infected other factors affected death rates, such as the country's average age and body mass index. But in the enactment of public health measures, basic trust in others was the strongest predictor of people's health outcomes. Procedures do exist for helping people to rebuild trust and come to consensus about difficult issues. Unfortunately, shouting at each other on Twitter isn't one of them.

“I just want us to follow the science,” has been a refrain of many during the pandemic, meaning a vote of support for masks, vaccines, and social distancing. The data on those interventions are clear: Recent estimates, for example, suggest that over 300,000 deaths in 2021 could have been prevented if everyone had been vaccinated against COVID-19 as soon as vaccines became available, even before the December 2021 Omicron surge. I also want to be clear that pushing unproven and potentially dangerous interventions is never acceptable (as in the case of hydroxychloroquine or ivermectin), and that it isn’t OK to deny well-established facts just because you don't like them (masks clearly do prevent SARS-CoV-2 transmission, and the currently available COVID-19 vaccines are both safe and efficacious). Despite the strong facts in all of these cases, however, a "science-based response" to a public health crisis must also include the judgment of experts from many different perspectives, and the willingness of people to take the steps that will be required of them. My argument is that "facts are always friendly," but that facts alone are not the sole determinant of policy. Those who wish that pandemic restrictions would finally come to an end would be well-advised to stop looking for alternative facts, and to argue for alternative fact-based policies instead. Those who wish that others would agree to abide by current public health policies would do well to appeal to moral values like group loyalty and the sanctity of life rather than presenting facts alone. Decision-making needs to take into account values as well as facts, to honor expert judgments without being bound by them, and to keep up with the latest findings in the ever-shifting landscape of science.