cornvillenutmeg

Mean SAT Score for reading and math tests, by year (Photo credit: Wikipedia)

     Remember Judge James C. Chalfant and how he has instructed the Los Angeles Unified School District  to begin to use students’ academic achievement as part of its teacher evaluation system?  How do you think that might work?

     Well, you might say, grades.  Pay teachers based on how many of their students end the year with As and Bs.  Or go about it the other way;  any teachers more than half of whose students get Ds and Fs don’t get a raise.  Yes?  No?

     No.  Anyone who has taken a general psychology course knows that even the most scrupulous, assiduous, and honest teacher will over time – and not that much, in fact – begin to inflate the grades he gives his students. 

     Maybe standardized tests?  Well, okay, but don’t expect  anyone who knows much at all about research and statistics to give her stamp of approval.  Here is just one tiny thing the College Board website says about scores.  “Average (or mean) scores are based upon the most recent SAT scores of all students of a particular graduating class.”  Now, of this I am sure:  any teacher of more than, say, seven years experience out there will swear on his or her pension that from one year to the next, entire classes, identified as their year of graduation, are perceptibly different in any number of ways – behavior, intellectual curiosity or almost complete lack thereof, friendliness, energy or lethargy – one from the next. 

     So you take the SAT scores for Ms. Journeyman’s Juniors for 2010 and 2011 and 2012.  First you might see that the average scores are different by a significant percent.  Then you might see that Ms. Journeyman’s students’ average scores themselves were different during those years by an even greater percent.  Oh, my, you may think, Ms. Journeyman has been slacking off!  Or, you might think you see that Ms. Journeyman is today a much better teacher than she was two years ago.  And you would likely be wrong no matter what those scores led you to believe!  What makes you think you can compare in any meaningful way the scores of one small group of juniors one year to the scores of another utterly different small group of juniors another year? 

     Should you mange to take your findings to an actual statistician, you might find yourself being asked where and under what conditions the tests were administered.  Were the three populations (the groups of students) given the tests by the same proctors?  In the same room?  At the same time of year?  Time of day?  And – watch out for this one – were the tests identical in every way?

     So what’s your answer, hmm?  Let me help you out.  No, I don’t know, maybe, yeah, I guess so;  and no, of course not because if you gave identical tests there would be an excellent chance that the answers to many of the questions would be known to an increasing number of test takers.  After all, these tests and their results are supposed to help colleges with the admission decisions so the motivation for getting really good scores might just possibly lead 85 % or more of potential test takers to take advantage of student entrepreneurs who would most certainly be collecting questions and answers to past tests.

     Pick a standardized test given to any population of students across an academic year and its uselessness in determining the worth of a teacher is a foregone conclusion.

     So, what are you going to do?  Hunh?  You need to create a test, by which I mean a test needs to be created by people who are trained test creators, which will measure what you want to know about how a teacher goes about plying his trade.  Therefore the first question must be, what do you want to know?

     How well the teacher is doing her job.  Good.  Now, what defines a job well done?

     How well or how much the teacher’s students are learning.  Okay. 

     May I assume what you mean by that would be on the order of, say, how much Algebra?  Chemistry?  French?  Yes?  Good.

     How do you find that out?  Hmm.

     Oh, wait, aren’t there tests out there that measure skills and knowledge and facility with things like math and science and foreign languages?  Yes, things like the PSAT, SAT, ACT;  you know, they call them standardized tests. 

     Oh, oh.  Back to square one, hunh?  Bummer.

Related articles

• Judge: LAUSD must rate teachers on pupil progress (sacbee.com)