Testing, Testing…Can Anybody Hear Me?
Perhaps the most dangerous consequence of this overreliance on WEIRD (Western, Educated, Industrialized, Rich, and Democratic) samples (see “The Weirdest People in the World?”) is that the body of social scientific knowledge becomes self-reinforcing as supposed truths are legitimized through scientific inquiry.
In some ways, I am reminded of how standardized tests like the SAT or the ACT can unconsciously work to reify WEIRD culture. Ignoring for a moment who the test is made by and for (both groups undoubtedly fall largely into the WEIRD category), we can examine how the methodology of the test’s construction acts to privilege a particular kind of information and legitimizes such knowledge as normalized.
Although the issue has been somewhat corrected, early forms of the test attempted to work backward in order to find measures of intelligence and asked individuals already considered “smart” to test questions. The assumption behind this practice was that questions answered correctly by those who had already demonstrated their intelligence were accurate predictors of intelligence—the questions that intelligent people answered wrong were obviously not good questions because the intelligence of the respondents was assured.
One might make a case that this sort of test construction might work for more “objective” areas like math, but the larger question that should be considered is how a standardized test endeavors to test knowledge but instead assesses only specific types of knowledge. Taking the example of math, we might assume that there does not exist a large amount of variation in knowledge among those who would take a test like the SAT or the ACT (i.e., math is taught in very similar ways across classrooms) and so therefore might make the case that the test exhibits a rather high measure of validity.
But we can also consider areas like reading comprehension where WEIRD students may have an advantage. Although no student that I have talked to has particularly enjoyed reading the passages on the SAT or the ACT, I would argue that the types of passages that appear represent styles, formats, and subject matter that WEIRD students may have been exposed to before through their schools, test preparation services, and their families. In essence then, tests like the SAT or the ACT might measure raw intelligence (if such a thing exists) but also measure social capital and dangerously transmute capital into a form of intelligence in the traditional sense. This form of knowledge is not only overgeneralized but also held up as a standard of what intelligent people should know, thereby initiating a self-reinforcing cycle as the non-WEIRD people become labeled weird.
Here, an effort to equalize society (in the case of the SAT/ACT to form a meritocracy) seems to continue on in the vein of Matthew Arnold’s belief that culture should evidence “the best that has been thought and known” while not pausing to reflect on just whom all of this is best for. In other words, the ability to align with, internalize, and parrot back the knowledge that test makers hold in esteem—what a disproportionately powerful subsection of society thinks is worthy—becomes a sign of culture and intelligence. At our worst, we punish students for not knowing things they should and, at our best, we help to indoctrinate students into a framework that aspires to be WEIRD; in many ways, we tell our young people that knowledge gained outside of the mainstream—those bits of wisdom collected from folk culture or non-White homes or from rural areas—are simply not worth much to those who matter.
 By this I certainly do not mean to argue that knowledge in the form of social capital is not a form of intelligence, but merely that it is not the type of intelligence purported to be measured on a standardized test like the SAT or the ACT.