One of the reasons I started writing this blog was for science outreach. I like to discuss how we do science. Tonight I am going to go back to my roots and discuss an oft-misunderstood concept in science. The role of the anecdote, and through it the inherent value of simple individual observations in the scientific method. The background for this blog post is a discussion I had over the weekend about which of two mutually exclusive expressions is correct:
I regularly hear people mistakenly claiming the latter is true, but the truth of the matter is that the former is true. In this blog post I want to explain why I believe anecdotes have a legitimate place informing our initial hypothesis development and how numerous, mutually-supporting, individual, observations represent a valuable information resource…or put in plain English that the plural of anecdote IS data.
To begin, a bit of background. The research project that led to my original PhD thesis research was done when computer databases were in their infancy. It was the end of the Green Plan in Canada and the Canadian government had spent hundreds of millions of dollars collecting baseline data on all sorts of phenomena, but the money wasn’t there to effectively archive the data. I was a recently graduated analytical chemist hired by a research group at the University of Victoria that was looking at thousands of analytical results collected throughout the Strait of Georgia and stored in filing cabinets in numerous federal research facilities. The question our research group was asked to answer: How do we avoid simply wasting all this data when the funding for the Green Plan ran out?
Our decision was to develop a geospatial database which assembled all the information in a location, and format, useful for future researchers. But we were left with a question: how do we help future users understand the strengths and limitations of the data in the database? We had to ask ourselves what differentiates mere observations from useful data and how do we document that difference for future users?
Surprisingly, we discovered that this topic had not been well-studied in the academic press. After months of scouring the academic literature, the best answer we could identify was a concept called “process knowledge“. To explain the concept of process knowledge, in that era, I’m going to steal a paragraph from my earlier work:
From a policy perspective, scientific information consists of two distinct sub-components: scientific data or measurements, and process knowledge. The former are obvious – the levels of a particular contaminant in a fish liver, the nature of the lesions provoked on the fish liver, and the mortality effects of a particular contaminant on fish in bioassays are all examples of scientific data or measurements. Process knowledge describes “the dynamics and interrelationships within natural biophysical and social systems” (Cornford and Blanton, 1993). Process knowledge may be theories, hypotheses, mathematical models, or even suspected correlations culled from a body of observations. Both data and process knowledge are required as inputs to decisions, but in different proportions depending on the questions being addressed.
So now we can see the difference between scientific data and simple observations? It is the existence of an underlying understanding of the process knowledge that links a set of observations to the physical world. That being said, a well-documented observation, when put into the correct context, can graduate from a simple anecdote to a useful data point.
So what is an anecdote? Anecdotes are individual accounts or stories. They are in effect observations unconnected to any specific process knowledge. As such they represent an information resource that can be used, with appropriate scrutiny, to help us understand the world around us.
Going back to the history of science we all remember the stories of scientists making general observations that informed their hypothesis development. The classic example is Newton’s apple. Now whether Newton observed an apple fall from a tree and used it to help him understand gravity is apocryphal or literal is not the point, it demonstrates a point. In the development of new theories and hypotheses, the first observations are almost always done absent detailed process-knowledge.
Observations, often collected for some completely different purpose, form the first step in understanding our shared environment. Reports of these observations (anecdotes) form the original data points used for virtually every investigation. Sure, not all anecdotes are of equal value; for every story of a useful anecdote, I can provide a story of an anecdote that was misconstrued. I have an entire blog post (Risk Assessment Epilogue: Have a bad case of Anecdotes? Better call an Epidemiologist) dedicated to situations where anecdotal information led to unsupported hypotheses. But that is the point, a single anecdote is not typically terribly useful, but numerous, similar anecdotes can inform the development of a hypothesis.
For centuries indigenous peoples collected their observations (usually in oral form). These oral stories represent “anecdotes” by the standard definition of the term and were historically dismissed as not particularly valuable because they were collected in the absence of any systematic evaluation of a specific phenomena or to address specific hypotheses. Yet now virtually the entire scientific community agrees that traditional ecological knowledge (TEK) provides invaluable insights into our shared ecological heritage. The hundreds of years of historical narratives and observations (anecdotal observations) are indeed data by any fair view of the concept.
Let’s take a specific example. The anecdotal reports from generations of fishermen and coast watchers allow us to generate an effective estimate of the extent of the historical habitat for the Southern Resident Killer Whales (SRKWs). Those compiled anecdotes were not systematically collected for the purposes of establishing the extent of the SRKWs’ habitat but those anecdotes do just that. When numerous, independent observers all document the presence of SRKWs off Haida Gwaii it confirms that this location is part of their historical range. The assembled anecdotes, collected for totally different reasons, are data in the context of establishing these historical extents.
So are all anecdotes useful? Absolutely not. But take a lot of observations that are similar in nature (or mutually-supporting) and suddenly you have the baseline data necessary to start developing a hypothesis. Anecdotes can, and do, provide a valuable information source at the initiation of a scientific investigation of a phenomena, or put another way: the plural of anecdote is indeed data.