In a few years, teaching will be a lot like fantasy baseball.
Principals will be selecting their draft picks, organizing their order to fill difficult slots, like math, science and special education. As for myself, the stats show I’m a solid .700 hitter, with an 80% pass rate last year. I’m a good pick for power and distance, although I may fade down the stretch into June. As long as the bullpen comes through in the clutch, I should be alright. God help those teachers on the waiver wire.
It would be incredible if every aspect of our professional lives can be effectively reduced to a number. Classifying and ranking would be much simpler. A simple graph would tell who was pulling their weight and who couldn’t hit out of a paper bag. How many principals would love to shove a chart into a failing teacher’s face, bellow out “the number’s don’t lie!” and give the unfortunate loser the boot.
If education were only that simple. It isn’t.
Recently, the New York Post ran a story about the teacher’s union, the UFT, allegedly obstructing efforts by Mayor Mike Bloomberg and schools chancellor Joel Klein to include student test scores as a factor in determining teacher tenure. The State Assembly recently killed any hope of even creating a commission to investigate how test scores can fit into the process–a commission supported by UFT president Randi Weingarten and the larger umbrella group New York State United Teachers. The commission was seen as an olive branch in the feud between the Department of Education and the Union over the test scores issue, and was even included in this year’s controversial budget. Yet the chair of the Education Committee in Albany, Catherine Nolan, refused to even allow a vote on the matter.
Teacher tenure is an issue that really bothers me. On the one hand, many teachers who have long since proven a disservice to students are protected by the tenure system. I won’t name names, but i’m familiar with a number of teachers in various schools that probably do more harm than good, yet are protected by the system. Even if they are excessed by a principal, these teachers are guaranteed a salaried position by union contract. While this is helpful to most, it can also be detrimental by keeping bad teachers in play. Just like the Yankees who are obligated to keep and pay for useless players based on contract (Randy Johnson, Carl Pavano, etc.), the system of tenure, when abused, can create a class of benchwarmers that drain resources for next season.
However, tenure is also an important safeguard for teachers against the lesser natures of administrators–especially those with no experience in the classroom. An administrator without classroom experience can quickly turn into a bean counter. Statistics, numbers, charts, graphs–the quantifiable data that is so useful in the business world makes little sense in education. If your supervisor had no experience handling children in a classroom, he/she will probably not be sympathetic to your terrible class forged by Lucifer.
Like baseball, teaching also has those “intangibles” that cannot necessarily be controlled. Who knows if Ted Williams would have hit a .406 batting average in 1941 had there not been a ridiculously shallow right field in Fenway Park. Sandy Koufax, the great Los Angeles Dodgers pitcher of the 1960’s, had the benefit of a concrete monolith in Dodger Stadium, where home runs were few and far between. In public schools, we cannot cherry-pick our students. Nor can we adjust the facilities. Our schedules are often held hostage by meetings, workshops, common planning periods and the like. Test day is also stressful; not everyone is a great test taker, even the really bright children. There’s also the home to consider: not everyone lives in an environment conducive to achievement. If Daddy is playing Grand Theft Auto with his son during homework time–which is sometimes the case–you can be sure that school isn’t much of a priority.
So let’s assume that test scores will play a factor in determining tenure. How much weight should they carry? Will the criteria include just tests, or a basket of assessments (portfolio, written work, observations, etc.)? Should all tests be used, just State assessments, or a selection of subjects? How will one set of scores compare to another? Should the raw score be used or the scale score? Are we looking for set targets, or windows of progress over time? Can we adequately assess a teacher’s skills based on the work of previous teachers in previous years?
Test scores may be numbers, but the factors surrounding them are anything but tangible. This is why using them to determine teacher tenure can become a volatile issue. If student work is to be used in assessing teachers, then that assessment should be done within a framework that fellow teachers can understand. Test scores must be viewed in context to a holistic learning experience that includes various assessments, observations and data. Scores alone cannot determine performance, since it is but one indicator of a complex process.
Let’s return to baseball for a moment. If Alex Rodriguez were to be assessed based on cumulative, one-shot annual state exams, he would be considered a failure. Look at his record in October and see if you doubt me. However, he is not judged merely on his October stats (though maybe he should be) but rather on a season-wide scale. This would include slumps, tears, injury periods and the like, producing results that look good on a stats sheet, even though he can’t perform during the playoffs.
I was actually looking forward to a commission studying how test scores can be used. It would have provided some form of closure to a contentious issue that mistakenly blames teachers as being against student acheivement. It would have also provided some real data to see how reflective are test scores to actual student learning. For now, I guess we will have to live with tenure as it is.
Funny, I would’ve gotten a kick seeing my principal watch some kindergarten teacher getting swamped, then make a call to the bullpen. Make sure you’re loose.
Mr. D's Neighborhood 
Mr. D-
You make a very interesting argument in this post. I’m in general disagreement, though, with the overall thrust, which is to limit the use of test scores as a factor in tenure decisions. This strikes me as the perfect being the enemy of the good — because test scores are not a single noiseless signal for teaching quality, we should throw them out of the equation entirely. For all their flaws, they have a real value of being an objective scoring system — an opportunity to abstract out from the qualitiative assessments you listed and provide a data-driven evaluation as well. In fact, they can be a much less noisy signal (i.e., influenced by in-building administrators) than many of the other assessment metrics you mentioned. As for the Daddy playing GTA with Junior the night before the test argument, if I remember the statistics classes, when you’re doing a pre- and post-treatment analysis of a small number of matched subjects, (for example) you have sufficient degrees of freedom. Because of Daddy whacking virtual street thugs on the PS3 with Sonny the night before the evaluation at the end of the year exam, he probably was also doing the same the night before the pre-test …
Scores are a *great* way of measuring progress. NYC has already shown that it uses the scores responsibly — the school academic progress reports weight student progress ~2.5x more than student achievement. So, even if you’re dealt a rough hand to begin with (e.g., administrators; tough students), the scores can be used to evaluate progress — which speaks to the skills of the teacher in that classroom. I’d be more concerned about the shutting down of the debate than the introduction of this metric.
Anyway, just a contrarian grenade to roll into the room. Looking forward to a continued discussion.
Glad to see such astute points being made on this topic–and happy to see a statistician’s point of view. I, too, share your concern of the stifling of the debate, since it would add more light to a growing dilemma in our field. I do want to raise two points, however:
(1) The assertion that test scores are an “objective scoring system” is true in it of itself, in that data derived from an exercise in a controlled environment is collected, ranked and qualified to provide useful statistical data. However, like polling questions, the tests themselves–including the scoring rubrics and the weighted scales used to rank scores– can be very subjective. State assessments in various subjects differ in difficulty year by year–in NYS at least, there is little real continuity in question frequency, arrangement and focus. Furthermore, we as teachers have noticed wildly different weighted scales by year. What was a failing grade one year could become a passing grade under a different rubric in a different year. I may be paranoid, but this can lead to “manufactured progress”, a fancy way of saying tests are dumbed down to create the illusion of student progress.
(2) School rank, especially in New York City, is more complex than the academic progress report. While this report is most important in NCLB terms as it determines whether a school meets “adequate yearly progress” based on preset targets, a school’s report card score–what is used to measure school progress as a whole–is based on a quality review visit, accountability reports, parent and teacher surveys, and reviews of Individualized Education Plans (IEPs) and Comprehensive Education Plans (CEPs). This is on top of the academic progress report. You are correct in that student progress is weighed more–AND that students are tracked over multiple years and among different population strata. This is a very positive move, in that we can see how a child does based on their own baseline. Just wanted to point out that schools must account for a lot more.
Heck, as a teacher with a consistent record of improved test scores–and exponential growth among English Language Learners–I’m not worried about the test score situation. I just want to make sure we don’t oversimplify things for the sake of statistical analysis. Would love to hear more thoughts on this from the Neighborhood.
Pingback: How’s this for “Full Disclosure”? A Counterproposal for Publishing Teacher Data « Mr. D’s Neighborhood
I love the baseball analogies! 🙂