How to Avoid Being Derailed on the Reading Railroad

A high-school English teacher once told our class that “higher education is a process of learning more and more about less and less.” Catchy and memorable, but only true to the extent that most of our careers—academic and otherwise—tend to make us “experts” on a limited number of topics. So it is inevitable that we will confront some information that is outside our expertise, and we will need to make judgments about arguments, actions, or policies based on this information.

Recently, journalist Nicholas Wade authored a piece in the Bulletin of the Atomic Scientists about the origin of SARS-CoV-2—the coronavirus that is responsible for the COVID-19 pandemic—giving credence to the claim that the virus was leaked from a laboratory in Wuhan (China) instead of emerging from a natural reservoir such as local bat populations.

Those who have expertise in the details of laboratory research with viruses can certainly evaluate and critique the claims that Wade presented in support of his argument. But what about the rest of us? How do we evaluate arguments with important consequences without resorting to “dueling experts” who claim equal authority and come down on opposite sides?

This is a challenge for all of us who deal with claims and evidence outside our own expertise and experience. Even if we don’t know an exon from Exxon, we need a way to evaluate how much credibility to give particular arguments, such as this piece from Nicholas Wade.

When we teach the basics of reading research literature to students, we expect that, at first, they lack the specific disciplinary expertise necessary to critique the content effectively. So we help them to develop an approach that looks for clues in the source, the format and presentation, the language and appeal, the purpose, and other characteristics of the argument that we can identify without having a deep grasp of the technical details of a specific field of knowledge.

A simplified example provided by the Writing Center at Texas A&M University uses the C.R.A.P. acronym to help students focus on revealing indicators of credibility of persuasion pieces such as the one on the source of the COVID-19 pandemic. These are

• Currency: Is the information up to date; has it been supplanted or refuted by newer resources?

• Reliability: Is the information reviewed or vetted by people with specific expertise in a relevant field? Does it represent the consensus of that field, a commonly acknowledged alternative to the consensus, or a position rarely accepted by those most knowledgeable?

• Authority: Is there a specific author whose qualifications to identify and evaluate relevant evidence and to place it in context can be verified?

• Purpose: Does the author or publication have a particular goal or mission to explore issues in a certain way or from a certain perspective, and is it intended more to persuade or to inform? Most of this information is readily available from the source, whether it is a periodical or a web site. We teach students that sources that do not provide this information are automatically suspect.

Remember: This is not an approach that attempts to evaluate the technical details of the argument, but one that looks at the context in which the argument is made and the way that it is presented and supported.

Currency: Wade’s article is recent, but the information used in it goes back up to a decade, and it is not clear whether issues raised and inferences and conclusions reached and reported early in the pandemic are still current. Have they been re-evaluated, modified, or even retracted in the light of newer information?

Reliability: The publication’s policy for vetting or reviewing technical information is not clear. It does not classify this article as to content and intent. Is it news? Investigative reporting? Opinion? The article seems intent on persuading us of a position, rather than merely informing us.

Authority: The author, Nicholas Wade, has a good record as an “explainer” of things scientific. He worked for years at the New York Times and has some good collections of articles that are useful for high-school and introductory-college level readers and that give a good overview of established science. His credentials as a science journalist are well established, but Wade is not a virologist or a microbiologist or an epidemiologist. He is a journalist, albeit one with a specialty in scientific—particularly biologic—topics.

The result is that Wade has to rely on published reports and interviews of scientists in the relevant fields. This is not a bad thing—one should get one’s information from these sources—but it is difficult to evaluate Twitter feeds, interviews, and other media except via an appeal to the authority of those who are quoted in these sources (see reliability above).

Purpose: In this case the mission statement of the Bulletin of Atomic Scientists tells us that its focus is on threats to existence caused by humans. So its articles are going to emphasize things that humans do that threaten human life on Earth.

Our Mission
The Bulletin equips the public, policymakers, and scientists with the information needed to reduce man-made threats to our existence.

One of the themes of the publication is to highlight threats posed by technologies that we do not or cannot control properly; a major theme of this article is the potential for accidental release of laboratory-engineered viruses.

About Us
At our core, the Bulletin is a media organization, publishing a free-access website and a bimonthly magazine. But we are much more. The Bulletin’s website, iconic Doomsday Clock, and regular events help advance actionable ideas at a time when technology is outpacing our ability to control it. The Bulletin focuses on three main areas: nuclear risk, climate change, and disruptive technologies. What connects these topics is a driving belief that because humans created them, we can control them.

At the very least, this is a publication that is going to be more sympathetic to explanations about “threats” that are caused by humans (intentional or otherwise). In this case, part of the argument is that laboratory technologies used in the study of viruses are endangering humanity and the world, at least in part because of shortcuts taken by researchers, research laboratories, industry, and governments that increase the likelihood that a virus modified in a research project might accidentally be released.

These issues also form the basis of more detailed evaluations that fall under the rubric of “rhetorical analysis”—another approach that looks at the form and format of the argument, as well as its context and some aspects of the construction of the arguments (see one example here). Again, the approach in this article is not to evaluate the specific examples and data, but to look at how those are used to construct the argument.

One part of this analysis is to examine what is called the “warrant” or the connection between the evidence presented and the conclusion. For example, in this article Wade argues that “CGG is coronavirus’s least popular codon for arginine.” There is, Wade continues, a CGG-CGG pair at a critical site that is essential for infecting human cells, even though this arrangement is unknown in related coronaviruses. The “warrant” in Wade’s argument is that rarity of the sequence in Nature implies that a purposeful insertion is a better explanation for this configuration than natural processes that produce changes in nucleotide sequences. (Readers will recognize that this sort of error—the unwarranted conclusion or inference—is the same one that we see in pseudoscientific explications of “evidences against” evolution.)

The outcomes of the C.R.A.P. profile and the rhetorical analysis do not mean that source is necessarily unreliable, but it places the material in a context that allows readers to compare its features and characteristics directly with other material from other sources. In this case, it suggests that the source publication has a vested interest in a particular perspective on the issues.

In the rhetorical analysis, when we come to this point, we advise students that it is time to look at appropriate disciplinary literature and to compare the data, methods, and conclusions that specialists in the field are applying to the problem, and then to record what questions those specialists are still asking after their own investigation. If there is significant disagreement in the field, it pops up in these comparisons.

Wade looks at the evidence he has compiled and concludes:

Neither the natural emergence nor the lab escape hypothesis can yet be ruled out. There is still no direct evidence for either. So no definitive conclusion can be reached.

That said, the available evidence leans more strongly in one direction than the other. Readers will form their own opinion. But it seems to me that proponents of lab escape can explain all the available facts about [SARS-Cov-2] considerably more easily than can those who favor natural emergence.

He seems to be hedging his bets here—or pulling his punches—at what seemed like an argument strongly committed to a laboratory origin of the pandemic.

By comparison, the joint investigative team of the World Health Organization came to a different conclusion in its report released just about a month earlier:

In summary, the joint team considered the following ranking of potential introduction pathways, from very likely to extremely unlikely: (1) through an intermediate host; (2) direct zoonotic introduction; (3) introduction through cold/ food chain; and (4) introduction resulting from a laboratory incident. Building from the evidence for the studies conducted so far, follow-up research studies were proposed for the first three options (p. 112)....

And, after a review of existing evidence, the WHO report concluded:

In view of the above, a laboratory origin of the pandemic was considered to be extremely unlikely.

What would be needed to increase knowledge?
Regular administrative and internal review of high-level biosafety laboratories worldwide. Follow-up of new evidence supplied around possible laboratory leaks (p. 120).

Of course, an account by a virologist specializing in coronaviruses would be the gold standard, if it were as accessible as this article written by a science journalist who knows how to communicate effectively with ordinary people. The main question that people ask is,”How can we tell if a virus has been modified in a lab versus a natural process in the wild, and what evidence do we still need to reach a conclusion?” And that is the crux of the argument. If we could make that clear to the average reader, one hopes it would replace big conspiracy theories with reasoned discussion that could resolve the question.

BIO: Andrew Petto is Distinguished Lecturer Emeritus at the University of Wisconsin‒Milwaukee, where he taught anatomy in the Departments of Biological Sciences and Kinesiology. For 8 years he and his students initiated and studied mock epidemics in the summer Research Apprenticeship Program for high-school students at the University of Wisconsin School of Medicine and Public Health in Madison, Wisconsin. The culminating publication from this project was Communic-Able: Writing to learn about emerging infectious diseases. In: Yager R. E., ed. Exemplary Science for Solving Societal Problems. Reston, Virginia: National Science Teaching Association. 2010.