I Know This Much to Be True
4 out of 5 dentists agree. Or so we’re told. But how often do we stop to question how this data was obtained—just who are these dentists? Although the catchphrase has managed to burrow itself into our collective psyche, the data behind this survey has never been released to the public (not that it really matters anymore, anyway).
I suggest that, above and beyond attributions of authority or group preference, the Trident slogan works because we exist in a society that demands data, equating the presence of scientific inquiry with legitimacy. We have come, since the Enlightenment, to accept Science as the structuring philosophy of our world (perhaps in lieu of Religion) and what else is data but evidence of that process? Although data is itself abstract, it represents something tangible (or at least quantifiable) for opinions were counted, preferences measured, and votes collected. We have become trained to respond to data, unaware of how statistics and “facts” can be manipulated. We have become reliant on data’s ability to simplify our world, unwittingly engaging in a trade-off that ignores nuance in favor of broad strokes; in a world rapidly becoming overwhelmingly complicated, we demand clear and readily apparent answers.
What we do not demand, however, is scientific rigor.
As a public, we do not generally care how data is obtained, only occasionally pausing to note flagrant violations in collection methods (exceptions are of course made for specific lines of inquiry but the broader point here is one of everyday experience). How often, for example, do we take the time to ask how respondents were selected for a candidate popularity survey? What types of questions were asked and what language was used to ask them? Were the questions asked in a particular order? And, again, just who are these people being asked? We not only fail to demand rigor from those who would present data, but also from ourselves as we blithely accept that the graphics on the nightly news broadcast represent “the truth.” Data tells how we are different from others but it is to our detriment that we rarely ask ourselves why.
One consequence of this lack of action is the overwhelming influence of the social sciences on Americans’ thinking (as noted in Sarah Igo’s The Averaged American), with an especially profound impact on the way that we conceptualize ourselves, particularly in relation to others. Once the practice of surveys had been generally accepted, it seemed that most anything could be measured and therefore every aspect of life, identity, and thought had a theoretical mean; the legacy of this new paradigm was (and still is) a perpetual state of unease as self-evaluation through data sets coupled with a mid-century culture already juggling paranoia, neighbors, the suburbs, and conformity.
And surveys, like Alfred Kinsey’s (in)famous investigation, laid bare the most private aspects of our lives even as it refocused our attention on the concept of normalcy—a shift that directed our attention toward commonalities instead of outliers. In retrospect, this change seemed somewhat natural as the intent of the surveys being conducted in the mid-20th century was to establish and define a national identity with the data collected suggesting that an “average American” was indeed a possibility. In other words, our thoughts about who we were (along with who we could/should be) came out of exposure to heretofore unseen aspects of ourselves—moving forward, our theories were shaped by our experiences as we incorporated our knowledge of data into various identities (e.g., personal, community, national).
Experiences, it might be argued, have much to do with the formulations of our theories, as exemplified by the role of the black swan (the animal, although Kinsey would undoubtedly have much to say about the recent Aronofsky offering) in Early Modern Science: once presumed to be a nonexistent creature, discovery of black swans in Australia pointed to the possibility of highly improbable outliers that caused a fundamental rethinking of prior assumptions. Prior to the discovery of black swans, an entire set of assumptions was made by philosophers/scientists grounded in the idea that swans could only ever be white; based on a lifetime of experience, people came to see the world in a particular way, which determined the types of questions the way in which they viewed the world, the types of questions they could ask, and perhaps more importantly, kept them unaware of the types of questions that could not be asked.
Experience, then, can cause investigators to develop a type of confirmation bias as they unconsciously (or consciously!) begin to collect data that confirms preexisting beliefs about how the world operates. Although the scientific method exists to shield us from this type of behavior, experience can be a difficult influence to mitigate as it ingrains in us a particular way of seeing/interacting with the world and constantly challenging stable environmental patterns would cause us much cognitive stress. To take this practice to the extreme would be unfeasible as we would be paralyzed by inaction while we analyzed the veracity of everything around us so instead I suggest that we, as a first (and smaller) step take a page from epistemology and simply begin by training ourselves to ask the question, “How is it that I know what I know?” As responsible scholars we need to be transparent about our theoretical foundations and honest with ourselves as we actively reflect on our process and our results.