One of the worst chapters in the history of public education in our city is nearing a close with the recent conviction and sentencing of 8 educators involved with widespread cheating by adults of student answer documents on the CRCT in Atlanta several years ago. In the aftermath of the trial and its outcome, there has been quite a bit of discussion in the media about the case, much of it focused on whether or not the punishment for these educators was appropriate or not. It seems as though not a day goes by without someone new arguing one way or the other about the sentences.
In the midst of this onslaught of opinions, I came across a sharp piece by Daniel Koretz, a professor from the Harvard Graduate School of Education, whose research is focused on educational testing and whose views I’ve shared before in my blog. Rather than get into the outcomes of the legal process, Koretz instead explores how the use of standardized testing has shifted markedly in the past 50 years from being used primarily alongside teacher reports and classroom work to provide complementary information about student learning to today’s use of standardized test scores as the main marker of student, teacher, and school “success” (or “failure”). The article (and short podcast accompanying it) are worth 15 minutes of your time, but here are two key takeaways and some commentary from me:
- There is way too much pressure on students, teachers, and schools to raise test scores against often unreasonable targets. It’s clear from the APS case that the evidence shows those educators cheated and violated their contracts and code of ethics. It’s also clear that the expectations—both carrots and sticks—were way out of line in APS. Big monetary bonuses were offered up to school leaders for improved scores. And, of course, the need for those improved scores was to stay on track with arbitrary targets set up to be compliant with No Child Left Behind’s goal of 100% proficiency for all students on reading and math tests. As Koretz points out, Atlanta was not unique in this regard, and though the APS case has garnered lots of press, this combination of factors has led to cheating scandals in D.C., Texas, California, Pennsylvania, and several other states over the past several years.
- Using test scores as the main basis for school and teacher accountability systems is simplistic and ignores many other indicators of a school’s or teacher’s effectiveness. Koretz says that the starting point for accountability systems needs to be “what we want to see when we walk into the classroom.” Needless to say, for most of us, it’s not seeing only how students do on a multiple choice reading or math test. There are many other important—even more important, in my opinion—measures of a school or teacher’s success, including how students perform on real tasks of writing, research, and problem-solving, whether students can monitor and manage their emotional learning, and how well students can communicate verbally.
These points by Koretz are important to consider in the wake of the APS testing trial verdict. Many people have pointed out that students served by the convicted educators were cheated out of parts of their educational experience. I don’t disagree. But unless and until we address the two points Koretz makes above, our accountability systems themselves are cheating students out of what could be more meaningful learning. For every convicted cheating educator, there are 1,000 others who work within the rules but engage in countless hours of test prep, teach students test-taking strategies, and work the system so that their students’ scores are as high as they can be. Those teachers are simply doing their job in a system that demands and rewards high test scores rather than more robust and holistic measures of student learning towards which these educators could be teaching.
Of course, it doesn’t have to be this way. We can use standardized test scores more sensibly, as they were originally intended to be used, by giving shorter tests at a few points over the year so as to have a better picture of student progress and to not put so much stock into one single score. We can use performance-based assessments in writing, research, problem-solving, and the like—”tests” upon which you cannot “cheat”. Instead of using money for bonuses based on test scores, we could instead use those funds to equalize funding across schools since test results clearly show disparities in scores based on a school’s level of resources. Shifting the huge amount of time and resources currently poured into administering state standardized tests (like this week’s Georgia Milestones) into approaches such as these would, I think, result in teaching and learning we’d all like to see.
The educators in the APS case were convicted and now must face the punishments that will come–you’ll get no argument from me on that point. But for all those who have expressed disgust with what those educators did I hope that as much of their energy will be put into changing the flawed way we use test scores that’s equally at fault in this case, a system that is harming many more students on a daily basis than these educators ever did.
Comments
2 responses to “So some teachers are now in jail. Will that be all our students will get out of the APS test cheating trial?”
Wonderful, a topic in which I have some tangential professional experience! This might get philosophical, but here goes:
I work in an industry with similar myopic focus on narrow “Benchmarks” for performance measurement. Ultimately, that’s the intention of testing measurement, to statistically identify areas that deviate from a standard. Is this worth the effort? I have no idea, the topic is way above my pay grade.
What I do know is that blindly targeting performance against a benchmark tends to lead to perverse outcomes, i.e. systematic underperformance. Over the long-term, the worst thing you can do is to incentivize individual contributor results to a narrow standard; the result is not better performance, but a desire to not stand out, in one direction or another. In other words, to be average.
A better approach is typically to incentivize an adherence to process and/or philosophy, regardless of outcomes. Are our teachers finding innovative ways to apply our constructivist philosophy in the classroom? Are our teachers pedagogically consistent with our philosophy? Are our teachers willing to experiment and learn from their successes and mistakes? These are the performance metrics that matter. Test scores are largely out of their control, so it sure as heck doesn’t make sense to reward/punish based on the result.
To hit the key points of the article:
1. There is way too much pressure on students, teachers, and schools to raise test scores against often unreasonable targets.
So which parts of this can we control? Matt, I suspect you can’t avoid some of the pressure that comes with a district/state/national obsession with test scores. Sorry, but sometimes it sucks to be the Executive Director! But our administration can (and does) act as a sieve to lessen the pressure on students and teachers. Have extra recess on testing days. Measure teachers on student contributions to a learning environment, not testing outcomes. Find ways to break the “thick” air that naturally surrounds test days. And most importantly, from my perspective, just don’t make a big deal out of it for parents/teachers/students! It’s just another school day.
2. Using test scores as the main basis for school and teacher accountability systems is simplistic and ignores many other indicators of a school’s or teacher’s effectiveness.
I don’t think you’ll get any pushback on excluding test scores from individual teacher accountability measurements. Are aggregate test scoring trends important at a District level? Maybe. Are they important at a school level? Debatable. Are they important at an individual classroom level? Absolutely not, and if someone vociferously disagrees, just tell them to back up their claims with data. Find affirmative and constructive ways to minimize the impact of test scores at performance evaluation time. Focus on process, innovation and experimentation and you’ll get a more comprehensive look at teacher performance than a simplistic, one dimensional number that is subject to variation based on uncontrollable outcomes.
Finally, an analogy by movie quote: (Caddyshack)
Judge Smails: Ty, what did you shoot today?
Ty Webb: Oh, Judge, I don’t keep score.
Judge Smails: Then how do you measure yourself with other golfers?
Ty Webb: By height.
Matt, thanks for your thoughtful reply. I do think that test scores have a place in evaluating teachers and schools. But right now, they are way to overemphasized in those processes when there are many other metrics that are as important, maybe even more so.
Any time you can work in a Caddyshack reference, you’ve done well!