The Perfect Score Project: Uncovering the Secrets of the SAT

The Perfect Score Project:  Uncovering the Secrets of the SAT

 Chris Tokuhama

Bibliography

Stier, D. (2014). The Perfect Score Project: Uncovering the Secrets of the SAT. New York: Harmony Books.

Author

Adapted from Debbie Stier’s Perfect Score Project biography (which seems very similar to the one provided in her book and on other websites):  Debbie Stier’s book publishing career has spanned two decades, most of it spent in public relations. Debbie regularly speaks on topic pertaining to social media and technology as well as, most recently, standardized testing.  Stier has written a piece in Time that largely supports testing and published drafts of her book chapters in Psychology Today.

Summary

The Perfect Score Project presents a peculiar amalgamation of memoir and test preparation guide. Although the book is largely focused on detailing Debbie Stier’s “project,” it also contains inserts of advice about various aspects of the SAT. The advice presented is reasonable but nothing new and this suggests that this book represents nothing more than a repackaging of old information with a veneer of self-improvement memoir in order to make it friendlier to readers. Furthermore, the stark contrast between the “experience” and the advice—it should be noted that Stier’s only claim to credibility is having taken the SAT seven times and having endured multiple preparation courses—demonstrates a lack of engagement with the fields of testing, college admission, and higher education by Stier. Indeed, Stier’s use of outside references is somewhat questionable throughout the work, seemingly relying on a relatively small body of work and failing to provide citations for claims in multiple instances. What Stier seems continually unable to do is to understand that scoring well on the SAT is not a grand “project” but rather a straightforward (if not easy) process of internalizing a strategy based on the logic of the SAT. To save time, one should just seek out the inserts if one is looking for information about test-taking strategies for the SAT. In reading her book, it becomes increasingly clear that Stier is not interested in dismantling the anxiety that surrounds testing, nor is she invested in dislodging testing’s place in college admission. Ultimately, this book evidences a missed opportunity to meaningfully reflect on the way that high-stakes testing can impact the parent-child relationship.

Notes

Prologue – Stier establishes a college-going environment that is suffused with anxiety that results from economic concerns and lack of information about how to prepare for the application process. Stier notes that the impetus of the book was a belief that high SAT scores could lead to scholarship money for a student, her son Ethan, who was otherwise mediocre—here the reader must pause to question the validity of this assertion—and that what follows is a result of that position. Although one might sympathize with Stier’s desire to aid her son’s ability to go to college (it is yet unclear whether and to what extent her son is actually interested in college), one also questions whether Stier’s efforts are really about effecting change or if they are instead primarily focused on creating advantage for her son at the expense of others. The larger question here is to what extent The Perfect Score Project legitimizes the SAT—and standardized testing as an extension—as part of the evaluation matrix for college admission; by focusing solely on standardized tests it seems that Stier is prone to overemphasizing their impact.[1] It seems evident that Stier is not initially interested in challenging testing culture although she might have an interesting angle as she would be able to contrast the ways that testing impacts her son as a student who is diagnosed with ADHD with testing in the form of the SAT/ACT.

Chapter 1 – Stier writes “I was ‘modeling’ the behavior that I was hoping to cultivate in my son” (11) but one wonders if this behavior is really results-oriented thinking. Stier would argue against this, as, for her, “project” indicates that “it’s about the journey” (13). And while the moral of this story might be that perspective has allowed her to understand the benefits conferred by this experience, we must question Stier’s truthfulness as her stated goal was to obtain scholarship money through higher testing (9). Here, it seems as though Stier wants to benefit from the “fix it” rhetoric that pervades America in an era of popular makeover and renovation shows while avoiding the negative connotations of over-attentive parents.[2] We can see how Stier’s book indicates that it will conform to some of the tropes of the genre near the end of the chapter as she writes on page 15:

Looking back, maybe I should have called it “The Perfect Do-Over.” But that insight didn’t come until much later. At the time, this was about how I could salvage Ethan’s thirteen years of education, at the very last minute, with the SAT.

Here, Stier makes clear that what comes next—what the book is about—is about a journey for her and not necessarily a story about how her son developed in his relationship to testing or to college admission. In spite of this, perhaps the most valuable piece of this short chapter is the way in which Stier reminds us the degree to which parents can feel powerless in the process and how this impulse can lead to reliance on things like checklists that function to give parents a sense of control (13). Remaining unexamined are the possible reasons for this anxiety and whether the actions taken by parents on behalf of their students are truly beneficial. As the above quote indicates, this “project” seems much more about the Stier than it does about truly helping her son.

At this point it should also be noted that despite being a single mother, Stier is the beneficiary of some luxuries that other parents might not benefit from:  being able to afford the cost—both in time and in money—of taking the SAT seven times along with the discretionary income to purchase domain names indicates that Stier is approaching this problem from a very particular vantage point. While keeping in mind that this situation may present very real stress for students and parents, how sympathetic should we be that she has not saved for her student’s college education?

Chapters 2 and 3 – Although Stier continually indicates that the woman she was at the start of this process is not the woman who she is now, it becomes evident that Stier privileged her perspective over that of her son. In writing, “Thinking about my son, I didn’t remember any ‘trauma’ or ‘unresolved issues’ or ‘test anxiety’ from my own SAT experience. Why would I? Low scores or not, I had gotten into college,” (17) Stier seems willfully ignorant of her own experience as a teenager, having mentioned earlier that she developed a story about her past to in order to diffuse the impact that standardized tests had on her. Stier goes on to mention how unearthing old letters challenged this view but does not indicate how her attitude toward testing, her project, or her son may have changed as a result of this. Later, in Chapter 3, Stier begins to hint at the importance of empathizing with students going through the process[3] but does not, at this point, address the point explicitly.

Chapter 5 – A prominent theme in this section is realigning the efforts and perceptions of students and parents, understanding motivations for each group and how these particular drives manifest in terms of behavior. Complicating matters is Stier’s conflation of projects that involve making something (e.g., building a car) with her own efforts at improving standardized test scores[4]—although this is part of a much larger discussion, the way in which critical thinking and analysis manifests in “projects” that gesture toward a DIY or maker movement differ from the thought processes involved in studying, generally, or test taking, in particular.

Stier also unfortunately cites work by Neufeld and Maté[5] to suggest that parents are pitted against peers in a battle to influence young people (46). In addition to creating a simplistic—shouldn’t “media” also be included at the very least?—and antagonistic dichotomy, Stier’s misunderstanding of attachment theory leads to the development of a perspective that largely ignores how youth culture functions[6] and how the values of the peer group impact an individual student’s understanding of himself or herself. Although Stier advocates for parents to be involved in the life of their child (47), what she proposes is an ignorance of the way in which parental influence and peer culture can work together to spur a student on. Exemplifying the unhelpful nature of Stier’s contribution, she writes, “Honestly, I’m not sure what level of parental engagement is necessary to be an effective motivator. […] I’m pretty sure any level of warm and connected parental participation is a good thing and has the potential to be a powerful source of teenage motivation” (48).

Chapter 6 – Earlier in the Prologue it was noted that Stier does not seem invested in challenging either the primacy of standardized tests or the anxiety that surrounds them and Stier does nothing to subvert this assertion in her discussion of College Confidential, an online forum that often serves as a breeding ground for speculation. In fact, Stier’s account exemplifies how the kind of discussion that can occur on College Confidential is actually not productive or helpful, nor does it encourage students to engage in the critical reflection that is often involved in the “projects” that Stier cites earlier in Chapter 5.[7]

Stier does note that the standardized test can advantage particular groups of people—although this topic deserves a much richer treatment than given here—but also makes assertions that while possibly true, without additional support, seem racist (e.g., Asians are better prepared in Math). More problematic is the way in which Stier fails to address the larger issue of bias in testing, which calls for an interrogation of what the standardized test purports to measure, how it claims to do so, and whether its method reflects particular biases toward what is measurable.[8]

Chapter 7 – Stier’s failure to engage in the larger conversation about testing further hurts her as she talks about the recentering of SAT scores in 1995:  although Stier notes that there is a correlation between SAT Verbal scores and school curriculum (although she seems to lump all curriculum and all SAT-taking students into one category) she does not question whether SAT scores are actually an accurate measure of verbal ability.[9]

Also frustrating is Stier’s inability to link her embarrassment over her test score to an earlier quote about self-confidence in American students (52). Had Stier engaged in thoughtful reflection about cultural factors that informed this emotion—which is unfounded as she writes that SAT scores do not matter as an adult—she might have rethought the adult/peer dichotomy that she embraced in Chapter 5.

Additionally, Stier’s writing on the SAT suggests that she does not understand how the scores are actually used or interpreted (i.e., in a national context) and continues to think about scores from an individual perspective. Further in writing “Superscoring is what the colleges do with your Score Choice—to position themselves in the college rankings” (68) Stier showcases her surface-level understanding of the use of standardized test scores in college admission:  had she dug a little deeper, the reasoning for superscoring—again related to an understanding of the test as a measure and how its components operate—would have become apparent as would the reason that superscoring occurs on the SAT but not the ACT. Finally, Stier misunderstands the debate over optional test scores entirely (70):  her logic regarding the impact that optional test reporting has on U.S. News & World Report rankings is deeply flawed and insufficiently considers how the test optional nature of an admission process can alter the demographic of the students who apply to that institution.

Chapter 8 – The extent to which Stier buys into the system is shown on page 72 as she writes about seeking advice for test preparation:  “Plus, [Mark] graduated from MIT, so I figured he was a reliable source. Obviously he’d done well on the SAT.” Not only does this quote exemplify the degree to which Stier upholds the traditional hierarchy of education in America, it also runs counter to Stier’s own observations about the changing nature of the SAT. Even if Mark did in fact score well on an older version of the SAT, there are no guarantees that his prowess is in any way translatable to advice on the test’s current iteration. Mark may provide good advice but that is because he is the founder of a test preparation company and not because of the institution that he attended or because of his SAT scores.

Although Stier raises valid questions about the relationship between the SAT and test preparation (74) and includes a brief history of the test itself (75-76), she continues to fail to consider whether the SAT actually measures what it purports to (i.e., innate ability) or how these measures are reconciled against a test prep industry that largely teaches students how to take the test rather than content that will be on the test.

Throughout the course of the chapter it becomes increasingly obvious that Stier’s intended reader is a parent like herself:  the person who simply wants to be told what to do in order to get through a problem.[10] For this reader, Stier’s book functions as nothing more than an extended review of various test prep products, with common advice sprinkled in.[11] For Stier, the “project” continues to focus on product/outcome instead of focusing on how to actually work with her son through the process. Further supporting the perception that The Perfect Score Project is really nothing more than a prep book in different packaging is Stier’s inclusion of a “recipe for success on the essay” (80).[12] Stier’s information isn’t necessarily bad here as the score of the SAT’s essay is entirely determined by a scoring rubric but, at the same time, this information is hardly a “secret.” A carefully considered work would, for example, take advantage of an observation that homemade vocabulary flashcards work better (88) in order to comment on engagement, practice, and learning—ideally, the themes that should be highlighted in the conceptualization of a “project.” It is additionally ironic that Stier writes about a revelatory moment where gained perspective on what she was doing in a moment where she is still focused on the mechanics of answering questions (92).

Chapter 9 – Stier’s lack of critical thinking continues to be an issue, deciding to employ Dr. John Chung’s SAT Math prep book despite the fact that she knows that she holds an irrational belief regarding the book’s effectiveness. Perhaps the most generous comment can make at this point is an observation about the power of the anxiety surrounding testing and how it can encourage undesirable behavior. The larger, and more interesting, theme would seem to be a discussion of “learning” in studying for the SAT and to what extent this process mirrors the process of learning that takes place in schools.

Chapter 11 – It seems evident that Stier has completely lost sight of her goal in this process as she fixates on her scores and the corresponding percentiles.[13] What becomes increasingly obvious is that Stier is engaged in “project” designed to provide validation and redemption for her anxiety over her perceived lack of preparedness with regard to college admission. Not only focusing on scores but how they make her feel—as opposed to using this as an opportunity to better understand her son’s frame of mind—seems indicative of the way in which Stier approaches this endeavor.

It is further telling that Stier is a devotee of Stanley Kaplan’s approach to test preparation, including the quote “To say you can’t improve scores is to say you can’t improve students” (115) without comment. Kaplan’s words create a false equivalency here, assuming that improving standardized test scores indicates something about improving students, when the two are not necessarily linked in a causal relationship. Here we can again see how Stier’s inability to contextualize standardized tests—or testing culture for that matter—in a larger environment of education leads to misguided thinking.

Chapters 13, 14 and 15 – One must seriously question the extent to which “learning” is equated with “higher scores” throughout this book; it would seem that any amount of perspective would teach one that scores on the SAT do not readily translate to anything beyond success on the SAT. Additionally, Stier’s fixation on score is also highly at odds with a desire to actually learn anything meaningful. In Chapter 14 Stier further evidences inappropriate conflation between the two, noting that “Learning was the easy part…it was remembering that was hard” and that “understanding is remembering in disguise” (158-159).[14] At the conclusion of the chapter Stier records her latest round of scores, noting a 30 point gain in Critical Reading, a 40 point gain in Math, and a 90 point drop in Writing—Stier’s failure to really understand the scoring system, however, leads her to focus on her victories (i.e., “motivational rocket fuel”) despite the fact that these scores are within the variable range for her based on her testing history.

Stier’s paternalistic attitude remerges in this chapter as she takes her children to Kumon, writing, “nor did I know that it was critical for their well-being for me to reclaim my place as their respected mother” (163). Although Stier’s parenting style is not necessarily called into question here, it seems evident that Stier’s book would benefit from an examination of how her attitude as a parent intersects with her motivations for her “project” and who all of this is really for. Although this attitude eventually results in her children moving out of the hose, Stier remains staunchly unable to think critically about what this rupture means for her and her family; the lesson here seems to be about how to navigate the parent-child relationship (Stier solves this by deciding that her family can bond over television) and to endeavor to understand children on their own terms. Instead of understanding this, Stier notes that her son “matured” when he became to exhibit behavior more in line with Stier’s idealized version of who he should be.[15]

Chapter 15 also further evidences shoddy thinking as it conflates “intelligence” with the SAT (170); as previously noted on pages 75-76, although the SAT was originally conceived as a measure of aptitude and based on models of intelligence tests, the current iteration claims to evaluate “developed reasoning.” Stier’s reasoning for equating IQ tests and SAT tests (176) is based on an incomplete analysis of the test’s history and, again, a failure to fully consider the test in its modern context.[16] Stier correctly notes that Kumon reinforces skills but skills are not the same as cultivating intelligence or actually learning material (although one might make the case that there is a residual effect as one learns how to learn). Later, Stier also describes undergoing neurological testing and incorrectly labels this as related to “intelligence”—one might possibly be able to link this to Howard Gardner’s theory of multiple intelligences, but an explicit connection was not made and one gets the sense that the way in which Stier’s evocation of “IQ” is in alignment with the most colloquial use of the term.

Chapter 17 – Steir’s treatment of K. Anders Ericsson’s theory of deliberate practice is tricky for while the tenets of Ericsson’s work may apply generally to SAT prep, it seems likely that Stier was exposed to the concept as popularized by Malcolm Gladwell. The evolution of the idea is important here as Gladwell employed the “10,000 hours” mantra in a specific way in his book The Outliers and this interpretation is not necessarily congruent with what Ericsson put forth in his original paper. Furthermore, the concept of deliberate practice has been contested in stories by the BBC and Time, among others[17]

Chapter 18 – This chapter prominently features a section on testing accommodations, making an unsupported assertion that the number of students testing with accommodations has increased since 2003 (222). More curious—and further demonstrating a lack of thoroughness—is Stier’s failure to mention the controversy over over-diagnosing of ADHD and its relationship to standardized testing given that her son has ADHD!

Characteristically, Stier also includes some less-than-flattering information but does not pause to reflect on what this might mean for her or her family:  on page 232 Stier writes that she would only learn later that feeling like you know something and actually knowing it are two different things and that it’s best for students to choose their own goals in order to increase motivation. Of these two, the second is more problematic as it upholds the way in which Stier continually privileges her perspective, wants, and desires over that of her son. Stier goes onto write:

The real magic of the project, though I didn’t know it at the time, was that what I thought about the SAT became more important to Ethan than what his friends though. I embedded myself in my teenage son’s life at the very moment when those forces of nature—the peers—are most powerful, and most dangerous.

In many ways, the parenting style that Stier continually exhibits—as told in her own words!—continues to run counter to her ideal of an “authoritative” style and seems generally much more aligned with “authoritarian” parenting.

Chapter 19 – The repeated invocation of Ben Bernstein in reference to test anxiety (Stier also mentioned him early in Chapter 3) might suggest that particular works have been highly influential for Stier and that she does not necessarily consider multiple views on a particular subject.[18]

The more important theme here, however, is one unaddressed by Stier herself:  the lack of knowledge regarding testing conditions/locations, SAT II versus SAT testing, and differences between College Board policy and enforcement by schools all point to the way in which cultural capital can have an indirect impact on scoring.[19] What Stier should point to here—but does not—is that individuals can have advantages that have nothing to do with testing ability.[20]

Finally, Stier’s perspective on her own parenting is again called into questions as she relates an anecdote about her son’s absorption in World of Warcraft in order to assert that “The best part of the project was the fun I was having with Ethan” (253)—a claim that while possibly true, receives little support in the stories that Stier has chosen to tell thus far.

Chapter 20 – Stier’s research skills are again called into question as she notes that The New York Times called Advantage Testing’s score gains “stunning” (257) as an Internet search returned no results to this effect.[21]

What becomes most obvious in this section is that any improvement in Steir’s family life was incidental to her project and not a result of it. One must again question Stier’s interpretation of this situation as her understanding of familial intimacy seems directly correlated with her son’s positive attitude toward her; noting that her son was “loyal,” “stricken at seeing his mother sad,” and “he’d become more me than me” (273), Stier makes clear that bonding is significant only to the degree in which it is related to her.

Furthermore, Stier does not reexamine her assertion that high SAT scores equal scholarships—one deeply suspects that she simply had not done her research at the outset—for although she notes that the University of Vermont offered her son merit (and not need-based as she points out) aid, the highest scholarship for that school only requires an 1800 combined SAT score for out-of-state applicants.

Stier concludes with recommendations that seem to stem largely from her own experience and are not necessarily grounded in any actual data. Her call for a “coherent curriculum in mathematics” (279, for example, seems directly related to Stier’s own deficiency in the subject. Moreover, Stier notes that “a good SAT course can offer a coherent curriculum in the SAT”(279), a statement that should cause her to rethink the entire book and the value of the SAT and its scores. Stier’s call to consider the foundations developed in American schools is a valid one but her insistence on tying this advocacy to measures like the SAT without acknowledging the greater ongoing discussion surrounding testing (at the very least, No Child Left Behind and the Common Core should enter into the fray at this point) evidences a narrow focus that is ultimately unproductive. Stier’s understanding of college-going is also suspect and the larger picture that emerges is that she and her son would have benefitted from having better discussions with their college counselor earlier in the process.

Perhaps most unfortunate is Stier’s complete lack of perspective about how this “project” worked:  Stier notes that she got to know her son better by reading his essays (281) and one can readily see that this bonding has nothing to do with getting a perfect score on the SAT (which ultimately really seems much more like a gimmick than anything else). Throughout her book Stier continually indicates a disinterest in getting to know “youth culture” and understanding where her son is coming from—had she felt differently, she might have realized that “The Perfect Score Project” was entirely unnecessary.

[1] It should also be noted that, at this point, the types of colleges that Stier and Ethan are interested in applying to. It seems likely from the connection between standardized test scores that Stier is talking about some level of selective college but this important detail is missing for the importance of test scores varies across institutions.

[2] Here we must also consider the extent to which this works fits into the larger genre of self-help/discovery memoirs that tend to be written by 1) women 2) of a certain age.

[3] It seems like a much taller order to ask teenagers to assume the perspective of their parents although this does not mean that they should be kept in the dark.

[4] See also earlier notes in Chapter 1 about the implications of conceptualizing this effort as a “project.”

[5] Hold On to Your Kids:  Why Parents Need to Matter More Than Peers (2006).

[6] Stier actually seems to have a disdain for youth culture as she writes that her son was “not even fazed enough to look up from whatever inane thingymabobber he happed to have been wasting his time on at that moment” (82). At best, she seems unwilling to engage in the nuances of the teenage experience (107). See also the way in which gamication goes uncommented upon and Stier’s comment about Grockit, “state-of-the-art technology, gaming, adaptive learning (none of which appealed to me, of course)” (134). See also, pages 202 and 233.

[7] See also the equation “SAT = college” on page 275.

[8] For example, we must consider how the Educational Testing Service (under contract from the College Board) validates its test questions:  it assumes that high-scoring students should get a particular item correct, which means that the values and perspectives of these particular students become ingrained in the test in a tautological process. Stier touches briefly on the vetting process for questions on pages 142-143 but does not interrogate what the “ability” criteria, which, according to Stier, states, “All questions must allow the testers to distinguish between a high and low scorer” (142). What Stier fails to mention is that the College Board designates a “high scorer” based on how a student did on other sections of the SAT—thus the logic here is that students who answered previous questions correctly should be able to answer future questions correctly, and not that these questions are necessarily accurate measures of anything else. Furthermore, Stier evidently has not done thorough research on the topic for she does not account for recent evidence that racial bias in SAT questions persists. See also the equating of “nonobjective” with “nonbiased” on page 78.

[9] Stier’s observation on page 190 that “The focus [of Critical Reading passages] is on the author’s meaning or intent, which is not necessarily what is taught in most high school English classes” seems like the perfect opportunity to launch into an investigation about the validity of the SAT but Stier makes no such effort.

[10] This mentality results in Stier’s confusion over the conflicting advice offered by different test preparation companies; although Stier briefly acknowledges that test prep companies are training you in a systematic approach to the test, not necessarily actually teaching content, the implications of this are not discussed. Stier philosophy toward the test can further be seen through her desire for “standardized advice” (137) and assertion that she was “dying for someone [she] could trust to just tell [her] what to do” (139).

[11] Further cementing Stier’s purpose is her conceptualization of her reader as part of a “market” (103).

[12] As further example, we can see that although Stier cites a study by Les Perelman that correlates an essay’s score with its length (110), she does not use this moment to challenge the legitimacy of the essay’s score as a measure of writing ability but instead proceeds to provide tips on how to game the scoring system.

[13] Stier also fails to mention the range in section scores (which are reported on the results!), which the College Board cites as ±30-40 points for each section. See also page 135.

[14] Specifically, Stier seems to confuse “automaticity” with learning/mastery.

[15] See also, 197.

[16] To add frustration, Stier even includes a quote from a verbal tutor that says, on page 181, “The SAT is not a literature test. It’s a vocabulary-based reasoning test.” In short, Stier needs to be much more specific and explicit in her definition of “intelligence.” Stier also goes on to reference Advantage Testing in Chapter 19 (256), a company whose representative would later write an opinion piece for The New York Times on the SAT titled “Not an I.Q. Test.”

[17] To be fair, the Time and Huffington Post pieces may have been published once The Perfect Score Project’s manuscript had been locked but the BBC piece is from 2012 and its position should be accounted for in a responsible treatment of the subject.

[18] See earlier point with Ericsson/Gladwell in Chapter 17 and Stier’s reference to Gladwell on page 247.

[19] Stier also includes a list of preferences for test center amenities but, unhelpfully, only mentions how one is supposed to determine what facilities will be available at a given test center 13 pages later.

[20] This is perhaps not surprising given the way in which Stier discusses Advantage Testing, a private test prep company, in Chapter 20. Stier mentions that Advantage has not turned anyone away because of ability to pay (according to the founder) but does not discuss the very real class implications of this kind of testing.

[21] The one result that did come back was not complimentary toward private SAT tutors like Advantage Testing. Advantage Testing also seems to misrepresent its coverage in the press, selectively picking quotes from a Wall Street Journal piece, for example, that is not exactly complimentary.

March 29, 2014 | Categories: Education | Tags: ACT, ADHD, Advantage Testing, BBC, College Admission, College Board, College Confidential, Common Core, Debbie Stier, Deliberate Practice, Developed Reasoning, Educational Testing Service, Gabor Mate, Gordon Neufeld, Howard Gardner, K. Anders Ericsson, Kumon, Les Perelman, Malcolm Gladwell, Multiple Intelligences, No Child Left Behind, Psychology Today, SAT, Standardized Testing, Stanley Kaplan, The Huffington Post, The New York Times, The Outliers, The Perfect Score Project, The Wall Street Journal, Time, University of Vermont, World of Warcraft | Leave a comment