June 1, 2020

One thing way worse than standardized testing is unstandardized testing

Sometimes I feel like I’m the last man standing in favor of standardized testing.

I don’t think people know that when I ask “how are the children,” I’m usually asking about their intellectual care and development. I’m an education activist so when you answer, I expect to hear results from a relatively objective source.

Like standardized test scores.

I can hear your collective sighs and hisses. Heresy, I know. Am I unaware that testing students, as a practice, was invented by the Klan?

Don’t I know the tests states give school children in 2020 are actually the product of mad scientists in 1940. (Because that’s totally plausible).

While those are fascinating questions, I have my own questions.

What is the rate of reading, math, and science proficiency in your city or school district? What gaps do you see when you disaggregate the testing results by race, class, and gender? Which schools are accelerating the academic growth of their students, and which ones aren’t?

If you can find a way to answer those questions without testing, let’s talk.

For now I realize you will ignore all the ways the knowledge of psychometricians outstrips yours. You will call them “high stakes tests” when in fact the highest stakes are suffered by people graduating from K-12 schools nominally illiterate or functionally innumerate.

You will repeat the language produced in teachers’ union focus groups, “test and punish,” but you won’t say the alternative is to ignore how mass miseducation sets up millions of students for the greatest punishment of all: poverty.

Researcher Richard Phelps, a longtime defender of standardized testing, articulates one of my biggest concerns about removing standardized testing from the landscape.

He says:

Without standardized tests (or standardized grading protocols) in education, we would increase our reliance on individual teacher grading and testing. Are teacher evaluations free of standardized testing’s alleged failings? No. Individual teachers can narrow the curriculum to that which they prefer. Grades are susceptible to inflation with ordinary teachers, as students get to know a teacher better and learn his idiosyncrasies. A teacher’s (or school’s) grades and test scores are far less likely to be generalizable than any standardized tests’ (See, for example, Gullickson & Ellwein, 1985; Impara & Plake, 1996; Stiggins, Frisbee, & Griswold, 1989; Woodruff & Ziomek, 2004a, 2004b). (In Phelps, 2008, Table 1 lists some common fallacies proffered by testing opponents, along with citations to responsible refutations.)

I know we’re supposed to say “trust teachers.” I’m ok with that only if we add “but verify” to that sentiment.

Which brings me to a new report authored by Lynn Olsen and Craig Jerald for FutureEd that details the weirdly bi-partisan coalition (Tea Party and teacher unions?) who would have us blow up the accountability totem and send us all back to the bad data stone age.

This is how they summarize their report:

Standardized testing has been a cornerstone of school reform for two decades. But a bipartisan backlash against testing in recent years and the suspension of statewide testing in the wake of the coronavirus pandemic have left the future of state assessments in question.

That’s the conclusion of a new FutureEd report, The Big Test: The Future of State Standardized Assessments. Written by senior fellow Lynn Olson and education analyst Craig Jerald, the report examines the evolution of the testing backlash, the current landscape, and how state testing systems must change to survive.

Drawing on a comprehensive, new 50-state analysis of testing legislation from 2014 through 2019, Olson and Jerald examine the striking scale of the pushback against testing and how the backlash is likely to play out in the dramatically different education landscape we suddenly find ourselves in.

I will go down clinging to the utility of standardized tests for civil rights monitoring, public policymaking, and system improvements. Yet, I fear the hull of that boat is below sea level.

You can read the report here:

The Big Test by Chris Stewart on Scribd

5 thoughts on “One thing way worse than standardized testing is unstandardized testing

    1. While I respect the intentions of Mr. Stewart, he fails to understand the underlying causes of poverty and inequity in American society. No amount of education can, to scale, ameliorate these issues without first addressing the policies and practices across all institutions in this country that are inherently racist, especially when schools themselves are racist institutions. Mr. Stewart ignores basic facts to argue in favor for standardized testing. But there is a myriad of evidence to support that the underlying causes of poverty have little to do with how we educate students. But we needn’t dig that deep to argue against his point that testing is the most “objective” way to see inequity. What data (and since he is a big believer in “data” he should be able to support his arguments with it) can Mr. Stewart provide to demonstrate that the testing regime has produced the results its proponents claim it does. From NCLB onward, we have not budged the inequities in the “achievement gap.” Even the most recent NAEP scores attest to this reality. Yet, he insists that we keep testing to show that the gap exists. We do, however, have a great deal of data supporting how testing has narrowed the curriculum, further impacted and marginalized ELL students, students in poverty and color (which have a strong correlation). He also asks the questions “What is the rate of reading, math, and science proficiency in your city or school district? What gaps do you see when you disaggregate the testing results by race, class, and gender? Which schools are accelerating the academic growth of their students, and which ones aren’t?” Well, these are the wrong questions. What he should be asking is how testing has contributed to the derailment of student progress, in particular the students for which he claims to be advocating. He could be asking how testing contributes to the school to prison pipeline and how it further marginalizes vulnerable students. He could also be asking how schools that emphasize social-emotional learning and other student centered practices perform in terms of students’ perceptions of their education. He could be asking many things, but instead, he asks questions that ignore what schooling is truly about. I can cite many studies, books and sources that attest to each of these points. But given that Stewart cites absolutely no empirical evidence to support the efficacy of testing, I won’t do so unless requested to.

      1. My goodness, condescend much?

        I reject Ms. Tenam-Zemach’s attempt to support educational essentialism which basically says that children without two white college-educated parents who have money are doomed to poor educational outcomes -regardless of pedagogical interventions, the quality of teaching, and the practices of their classroom teachers. Demography is destiny, why measure their learning in any way. We already know they can’t learn, so why test their ability to master concepts.

        This is the reason people of color should remove their children from the care of a largely white, female workforce who are shown by research to have low-expectations and self-preserving ideas about why their work isn’t strong enough to get results. The belief gap is real and the compensatory thinking teachers use to cover over their poor teaching weaponized by their unions and their unionized sisters in academia.

        She asks: “What data (and since he is a big believer in “data” he should be able to support his arguments with it) can Mr. Stewart provide to demonstrate that the testing regime has produced the results its proponents claim it does.”

        The question is a stupid one because no intelligent person says assessing students in itself closes the “achievement gap.” It’s telling that Ms. Tenam-Zemach uses NAEP testing returns to prove her point that testing doesn’t move the needle on student achievement.

        Testing is diagnostic, not a stand-alone cure. Blood pressure tests don’t reduce hypertension. Equity audits don’t solve racism. Surveying the labor market doesn’t fix workplace discrimination. People who claim they support “equity” should realize that it starts with numbers and information.

        She says: “We do, however, have a great deal of data supporting how testing has narrowed the curriculum, further impacted and marginalized ELL students, students in poverty and color (which have a strong correlation).”

        This is a common point made by academics who embarrass their universities because it lacks too much context. Having student assessment data doesn’t necessarily drive people to take the worst path (narrowing curriculum because they know too little about how to achieve goals and standards). In fact, for schools and educators that know how to educate, data is a tool for not just meeting standards, but exceeding them.

        I could cite plenty of research about the importance of teacher efficacy, bias, beliefs, preparation, and induction that impact student achievement regardless of the out-of-school factors students encounter. But I won’t bother offering that information since Ms. Tenam-Zemach’s level of “education” should have already taught her what is empirically true about those critical in-school issues. Any ignorance of the findings of this large body of work is, in my estimation, intentional and for a political purpose.

        All the more reason we can’t trust the judgment of her people without objective measurements to track outcomes.

  1. I am supportive of standardized testing but the fact remains that the implications of the results of that standardized testing can be out of line with what is required to eduatonal issues as the options become politicized. Above you indicate that teachers and students converge to a spot where grades are maximized. I agree, generally but not always, most teachers are more supportive to positions presented that line up with what they teach and present.

    Another scenario is also true. Schools that teach to the standardized tests do better. That does not mean the students know the material better but that the school/district has identified what type of questions are likley to be on the test and have usually spent time practicing the format of the test. Have you ever considered that some schools could have spent more time teaching new math skills but instead focused on improving standardised test results? This is the opportunity cost of standardized testing. The system is often gamed to the deteriment of non-standardised but important curriculum elements. Not to mention that schools with alot to loose often attempt to gerry mander thier populations to maximize benefits from anticipated results.

    Standarised tests are good for looking at anomolies. Expectations for example that students in high socio-econmic areas with high levels of support do well and looking for those that do not. The bar is not the same for the other extream where low socio-ecomic groups with low elvel of support (school/home/other) do not do as well. It is very difficult to compare effectively between group except at the margins. Often people try to compare the top schools with schools in other brackets (think of lists and ranking for a city with multiple different student populations) when they can/should not be directly compared. It maybe valueable to compare schools 1-10 together and schools 49-50 together but comparing #1 and #50 generally does little but that is the way resutls are presented. Ranking is a problem.

    I have generallised the above but as much as I like to see how my children and thier school compare to their peers I hate to see the the kids comming home and saying they could not finish a geography or art project becuase they had to practice the standardised test again today.

    My $0.02.

Leave a Reply

%d bloggers like this: