A number of recent studies show that testing students is not a good way to evaluate the performance of their teachers. According to an article in the New York Times earlier this week, an increasing number of school districts are adopting a system known as “value-added” modeling in order to compare one teacher to another and increase teacher accountability.
The article says that the system will calculate “the value teachers add to their students’ achievement, based on changes in test scores from year to year, and how the students perform compared with others in their grade.”
Education Secretary Arne Duncan briefly weighed in to support the newspaper’s work, calling it an exercise in health transparency.
“There are real issues and competing priorities and values that we must work through together — balancing transparency, privacy, fairness and respect for teachers,” Mr. Duncan was quoted in the NY Times as saying. On The Los Angeles Times’s publication of the teacher data, he added, “I don’t advocate that approach for other districts.”
However, as someone familiar with statistical research, I think caution is in order before such rankings are reported. For example, I would want to see how the scores play out over a period of time. What if the results vary widely from one year to another? In that case, you would be firing or embarrassing a teacher based on a single data-point, rather than a series of data.
Stanford Professor Edward Haertel noted that the methodology could be unreliable. “If these teachers were measured in a different year, or a different model were used, the rankings might bounce around quite a bit,” he was quoted in the New York Times article as saying. “People are going to treat these scores as if they were reflections on the effectiveness of the teachers, without any appreciation of how unstable they are.”
Defenders are quick to argue that these methods are sound – but is anyone truly comfortable using a single year’s data to determine whether someone should be retained or fired? I would want to see at least a few years of data to see if the ranking is stable.
If you see teachers’ ranks moving around from the top to the middle to the bottom, then you have a problem with the data. If you see stability in the rankings, then we can say that the rankings are stable, but not necessarily accurate. A single year should not be driving policy and personnel decisions, especially for a new test whose reliability no one can be truly certain about.
The other problem is that, while teachers are important, they are not the only variable in the equation. Work known as the “Broader, Bolder Approach” argues that “communities need to improve the educational environment, not just hire and fire new teachers.”
A group called Educational Justice asks, “Why do politicians and news writers avoid this broader-bolder approach?”
The answer, they say, is that, “It seems that they avoid it, argue against it, oppose it, because it would require that the politicians, the communities, the news writers do something positive, not simply cause the dismissal of some teachers.”
“Politicians would have to adequately fund the schools and the supporting health and community centers needed to promote education,” they continue.
One problem is that neither are excellent teachers are recognized, or poor teachers assisted, coached, and helped to improve.
Part of that problem is the assumption that poor teachers always have been that way, and always will remain that way. It may be that the professional environment of teachers eventually wears down the good ones, who would then appear to be poor teachers at one glance. While in reality, they’ve become frustrated over time at the inability of the system to address the needs of their students.
As the Economic Policy Institute argues, “Every classroom should have a well-educated, professional teacher, and school systems should recruit, prepare, and retain teachers who are qualified to do the job. Yet in practice, American public schools generally do a poor job of systematically developing and evaluating teachers.”
“Many policy makers have recently come to believe that this failure can be remedied by calculating the improvement in students’ scores on standardized tests in mathematics and reading, and then relying heavily on these calculations to evaluate, reward, and remove the teachers of these tested students,” the EPI writes on August 27.
They continue, “If new laws or policies specifically require that teachers be fired if their students’ test scores do not rise by a certain amount, then more teachers might well be terminated than is now the case.”
However, they argue, “there is not strong evidence to indicate either that the departing teachers would actually be the weakest teachers, or that the departing teachers would be replaced by more effective ones. There is also little or no evidence for the claim that teachers will be more motivated to improve student learning if teachers are evaluated or monetarily rewarded for student test score gains.”
“A review of the technical evidence leads us to conclude that, although standardized test scores of students are one piece of information for school leaders to use to make judgments about teacher effectiveness, such scores should be only a part of an overall comprehensive evaluation,” the EPI writs.
“Some states are now considering plans that would give as much as 50% of the weight in teacher evaluation and compensation decisions to scores on existing tests of basic skills in math and reading. Based on the evidence, we consider this unwise,” they continue. “Any sound evaluation will necessarily involve a balancing of many factors that provide a more accurate view of what teachers in fact do in the classroom and how that contributes to student learning.”
Anyone with a background in the classroom knows that there is a good amount of variability between classes. Some students are better than others. Some classes are better than others. I am very uncomfortable with a single year’s worth of tests supposedly determining the quality of a teacher. A bigger sample over time is needed to be able to properly measure that.
Moreover, teaching is not the only variable. Performance of individual students needs to be looked at over time as well.
The reports indicate that if the schools are not providing the proper resources and support for their teachers and not helping teachers improve, then simply firing teachers is no guarantee that performance will improve. All you may be achieving is to rotate a series of teachers through the system without much study or evidence guiding those decisions.
—David M. Greenwald reporting
Teachers are most definitely only one part of the equation. So much of how good a school is depends on the administration. For example, if teachers as a whole are expending too much time on discipline problems, that is an indication the administration does not have a good handle on the problem; if the principal of a school is busy hiding in the office or working on a PhD instead of doing his/her job, the administration is not doing its part; if the administration does not control what curriculum is used in the classroom, another administrative problem. It is convenient to scapegoat teachers, when the problem of poor schools is multi-faceted. What neighborhood the school is in can become a huge factor, if kids are dealing with drug infested neighborhoods, where gangs thrive, versus a school that is located in a university community. And what failure of counselors to address learning disabilities plays a part in all this? In some schools, the facilities themselves are subpar. When my kids went to DHS, the gutters spewed water all over the place, causing huge puddles for kids to wade through. Many schools have inadequate library facilities. How many schools have adequate access to computers? And these are just a few of the issues facing our schools…
I assume that teachers will be compared to others in the same school to control for the variables Elaine mentions–or at least some measure of socioeconomic status and school quality will be taken into account.
One year is not enough but if a teacher has a consistent pattern of poor performance something needs to be done. THe quality of the individual instructor matters and we need to get rid of bad teachers–and there are many out there–and reward good teachers.
Are test scores perfect? No. Do I like the idea of teachers teaching to a test? Not particularly. But to me the alternatives are worse. The fact that even the Obama Administration is embracing this idea is an indication that we have a problem. Our schools are failing, not in Davis, but elsewhere in the State and in the country. You can’t blame it all on the teachers but we can certainly weed out bad teachers and reward good ones. That would be a huge start. And in this job market there are many people who would love a teaching job at current pay.
[i]”I assume that teachers will be compared to others in the same school to control for the variables Elaine mentions–or at least some measure of socioeconomic status and school quality will be taken into account.”[/i]
Those factors are not considered in value-added assessments as I understand them. The way teachers are graded is based on one starting point and one ending point. The value-added is the difference.
If you have a theoretical district with say 100 3rd grade teachers, each class (as measured by an entrance test) starts at a certain point on a spectrum. Very likely, those points of departure will form a bell curve. Say (on a 1-100 scale) most are clustered around 50, some fall substantially below that and others fall substantially above that. But most are say within 40-60.
The question then becomes, where do they wind up at the end of the school year? If Teacher P’s class started at 45 and finished up at 71 and Teacher Q’s class started at 70 and finished at 73, Teacher P clearly deserves a higher grade (and a higher salary) for her results than Teacher Q did, despite the fact that Q finished higher.
What studies of these assessment programs show is that the best teachers one year — measure by progress — are almost always the best teachers year after year after year. And the worst teachers are the worst year after year after year. The best teachers are not more likely to be found in the so-called best schools*. They are well distributed among schools from the lowest performing to the highest.
(Before reading about this analysis, I incorrectly thought that low-performing schools would tend to retain bad teachers much longer than high-performing ones do, because the parents at the better schools would push the bad teachers out. However, the facts do not bear this out.)
The policy answer which follows from value-added assessment is not necessarily to fire the bad teachers, though that might be necessary in some cases. Rather, the policy probably should be to figure out what it is that the best teachers tend to do and tend to not to do and try to replicate that skill set to those who are less effective. Insofar as the skills to be great** are innate–such as force of personality–not everyone can be a great teacher. But a lot who are not very effective can be a lot better.
It’s important to also understand that the CTA (and other teachers’ unions) are fighting this assessment tooth and nail (though they falsely claim that their attack is against the actual tests, not the idea of value-added). The reason the unions don’t not want value-added assessments is because they will lead to performance-based pay. And that is anathema to all unions.
*It’s pretty obvious that the best schools are not defined by the absolute or relative quality of their faculties. They are defined by the parents of the children in those schools. The best schools have parents who motivate their kids to learn, who encourage learning at home, who read to their kids, who feed them properly, who have stable households, etc., etc. The great disparity in educational outcomes between some Asian groups which do just about everything right at home in terms of making sure their children are good students and other ethnic groups which do just about everything wrong is not all that hard to figure out. Its obviousness also raises the question: why don’t we as a society try harder to change the cultures of those groups which do everything wrong? It seems like our society is too politically correct to do what would actually work.
**My guess about great teachers is that, in general, they tend to have the highest expectations of their students. They are task-masters and they don’t accept less-than-stellar efforts. The students in turn internalize their teachers’ high expectations and perform accordingly.
That said, in thinking back on my best teacher in the elementary grades, I don’t know if she really had higher expectations than my other teachers. What distinguished her was her wonderful personality–she was someone you wanted to please– and her great ability to treat every child in her classrooms as an individual. She didn’t fall into the trap of teaching to the middle. The “middle” strategy tends to slow down and bore the smartest kids and leave behind the slow ones. Mrs. Sherry allowed (and encouraged) the smartest kids to advance at a rapid pace. And she made sure that for the kids who were not bright, they were making the best progress they could in each subject area.
For example, if teachers as a whole are expending too much time on discipline problems, that is an indication the administration does not have a good handle on the problem; if the principal of a school is busy hiding in the office or working on a PhD instead of doing his/her job, the administration is not doing its part; if the administration does not control what curriculum is used in the classroom, another administrative problem.
A principal might likely be working on a Ed.D., rather than a PhD.
I had a recent chat with a Davis school administrator who did point out an observation that is consistant with one of your observations. She had been an administrator in another district before coming to Davis, and her observation was that administrators were *not* so tied up dealing with discipline issues in Davis as is the case in many other school districts.
Better teacher assessments might be useful. Here is information about IMPACT, the assessment program instituted by Superintendent Rhee in Washington DC:
[url]http://www.dc.gov/DCPS/In+the+Classroom/Ensuring+Teacher+Success/IMPACT+(Performance+Assessment)[/url]
[i]”the assessment program instituted by Superintendent Rhee in Washington DC”[/i]
My understanding is that Supt. Michelle Rhee (who is soon-to-be-Mrs. Kevin Johnson) will be fired first thing right after the election. Mayor Fenty is going to lose–he is getting crushed in the latest opinion polls, and one of the main issues in the campaign has been Michelle Rhee.
The hatred of Fenty is, sort of, a racial backlash by black Washington (according to virtually every story I have read on this election). Fenty (who is half-black) is seen as the mayor for white Washington. Rhee (who is Asian) is seen by a lot of racist blacks as the tool of white Washington who rode in to tell “real” blacks how to manage their affairs.
Vincent Gray, who is going to be the new mayor, has played up this antagonsim. He has implied that he will get rid of Michelle Rhee once he takes office. (He has not said that explicitly though.)
In my view, all of this is unfortunate. Rhee has done a good job and some things have improved in the DC schools in her tenure. But in the heat of this year’s mayor’s race, Rhee has become the whipping boy (or girl) of the Fenty haters.
One other thing worth noting: Some of the anti-Fenty feeling among many blacks in Washington is driven by the increasing number of whites living in the District. They blame Mayor Fenty for the changing demographics.
Another factor is that Fenty is not corrupt. All of the black mayors who preceded him in office–Marion Barry being the worst example–were corrupt. And what that corruption meant was handing out phony make-work civil service jobs to black neighborhood leaders, who in exchange would support the council members the mayor supported and would also bring out the votes at election time for the mayor. A lot of those jobs in the last 4 years have been eliminated; and others (filled by black Washingtonians) have been cut back due to budget cuts. Those changes have fueled further anti-Fenty sentiment. The powers that used to be want to go back to the status quo anti, even if that means a more corrupt city with schools that don’t function.
wow… what a complete utter waste of time.
Schools are not a bit different from any other business. Give the administrators a free hand to hire and fire exactly like any other manager at any other business out there. Everyone knows the cranky old ladies that only still have a job because of bs seniority rules promulgated by the unions. Toss the weak out and bring on some decent teachers.
Funny thing will be that the strongest “pro-union” teachers will probably found to be the least effective and it has only been the nonsensical work rules that keep them in an environment where they are a complete disaster for children.
Yeah, it happens here in Davis. All the time. Talk to ANY parent who pays attention and talks to other parents, we know who sucks. We know it makes zero difference to complain. The only thing that matters to the teachers union is seniority and the path to a fat retirement plan after decades of part-time work.
Maybe we can examine what factors worked in the past (prior to 1980s) such that the United States produced the best doctors, engineers, and scientists in the world. What is it about education in the United States that has changed since then?
One Caution about increasing emphasis on test scores: it pressures teachers into teaching to the test.
This will change the makeup of people who want to make a profession of teaching (a shift from creative and enthusiastic people who want to be involved with the kids to more pedantic and regimented taskmasters).
Funny thing will be that the strongest “pro-union” teachers will probably found to be the least effective and it has only been the nonsensical work rules that keep them in an environment where they are a complete disaster for children.
Yeah, it happens here in Davis. All the time. Talk to ANY parent who pays attention and talks to other parents, we know who sucks. We know it makes zero difference to complain. The only thing that matters to the teachers union is seniority and the path to a fat retirement plan after decades of part-time work.
Well, I count myself as a parent who pays a lot of attention and talks to a lot of other parents, and I think you’re wrong to make that correlation.
For the sake of engaging your point, I would define the “strongest ‘pro-union’ teachers” as those who are DTA reps for their school site or are collective bargaining officers or on the contract negotiating committee.
I know very competent and desired teachers who are DTA reps; my kids had some of them. I also know of some teachers who fit that category (“pro-union” as defined above) who might oughta retire for everyone’s sake. I don’t think the stereotype you appeal to, “lazy fat-cat unionists”, matches my personal sampling in any way to prove your hypothesis.
Some comments/questions on unions or collective bargaining groups:
The only thing that matters to the teachers union is seniority and the path to a fat retirement plan after decades of part-time work.
Please explain why teachers’ retirement is out of proportion to their work and salaries. Again, I personally know retired DJUSD teachers whose retirement allows some minimal comfort (especially if their mortgage is paid off and kids are through college), but not necessarily evidence of excess disposable income — buying new cars regularly, maybe yearly out-of-state vacations (visiting relatives doesn’t count to me, here).
Give the administrators a free hand to hire and fire exactly like any other manager at any other business out there.
What guarantee is there that administrators will act responsibly in supervising teachers? DTA at least offers a grievance process that at least allows members a fair shake.
Because of funding formulas set by the state, DJUSD gets a smaller than average allocation of money from the state. But because of collective community cooperation — DSF fundraising, DTA/CSEA concessions, budget cuts (to teachers, staff, and administrators), parcel taxes, and retirement incentives — DJUSD has done a better job at retaining its teachers, and at attracting some very good staff to fill in a few vacancies. Right now it is an employers’ market out there in education, and DJUSD has snapped up some good talent because it is a little better positioned. Many other districts have been absolutely trashed, by contrast.
Right now I don’t see much incentive for the current college population to pursue a career in public education in California.
It is fair to say that we may have different experiences with specificc teachers, so our opinions may differ about the value of union vs. non-union teachers.
In general, teachers are undervalued, that being said, it is largely because the union makes us treat them as a group, rather than individuals. The union does “protect” the weakest members. It protects them from the public (us) and allows their inability to expolit our children to further their personal financial gain. A bad teacher does not deserve retirement benefits and should not be allowed to continue at their job. If they were a bad surgeon, sushi chef or air traffic controller we wouldn’t tolerate it. Why do we allow the incompetent to teach our children?
What guarantee is there that ANY administrator will treat their employees fairly? There are laws against discrimination and harrasment and any adminsitrator who behaves arbitrarily won’t last long. But in short- SO WHAT???? There is nothing sacred about a teacher as opposed to any other job. They are not an endangered species needing protection.
Your point about Davis being a special community in its support for the schools is true. But imagine if instead of ham-fisted firing of the junior teachers (decimating DaVinci for example) the administrators could simply go through the list of the teachers who have retired in place and send them packing? We might not even feel the pain of the budget cuts…
There are laws against discrimination and harrasment and any adminsitrator who behaves arbitrarily won’t last long. But in short- SO WHAT???? There is nothing sacred about a teacher as opposed to any other job. They are not an endangered species needing protection.
True there are laws against discrimination and harrassment and arbitrary mistreatment. Do you happen know what those laws and specifics are?
If you were a teacher hauled in by a principal for whatever reason and had your job threatened, you would at least like to know what recourse you have, and that you are being treated fairly. Would you rather spend your time in a law library to start learning the legal code from the beginning in order to save yourself in the event that you were unfairly targeted for dismissal? Or would you like to have access to someone with knowledge of the law and your rights so that you can spend a little more time doing your job as a teacher?
I see this type of process as a way to make sure an administrator is certain about his/her facts. In that sense, it may help an adminstrator from acting too rashly.
I have also seen this type of grievance process give the signal to an employee (through a collective bargaining rep) that his transgression was so obvious and proven that there would be no ultimate alternative but dismissal. The employee left quickly and didn’t try to fight it.
In yet another instance, a teacher was accused of making inappropriate sexual contact with a middle grade student. The local teachers union chose to aid the teacher. In the course of preparation for the trial, it became clear that the student had lied because the teacher had caught the student harrassing classmates. In tough budget times, I doubt if the school district administration would be very inclined to spend money defending the teacher.
and therein lies the rub…
Your entire statement is well thought out and reasonable. But you could easily replace the word “teacher” with bricklayer, bank teller, grocery clerk or rodeo clown and it would still be fair. My point is that there is zero reason to elevate teachers above any of these other professions…
I would go further to add that the argument that they have an important role in our children’s lives makes it that much more important that an administrator can act quickly to remove one who is weak.
A bad teacher does not deserve retirement benefits and should not be allowed to continue at their job. If they were a bad surgeon, sushi chef or air traffic controller we wouldn’t tolerate it. Why do we allow the incompetent to teach our children?
And how do you define a bad teacher? There are clearly some subjective standards in play.
The problem is that if a student doesn’t reach certain standards by 12th grade, then in the current national narrative, it is the teachers’ fault and the system’s fault.
But the kid might have a bad home environment or terrible parents, or indeed there may be inadequate funds and staffing.
If a city has a high crime rate, we don’t necessarily blame the police so readily. Or if there are a lot of fires in California, we don’t blame the fire department. In fact, we might react out of fear and throw more money to the police and fire departments.
But if you have a poor school district, bad environment, with little home support for the kids, in which a teacher would probably have to give up home and family life to make any headway, where teacher turnover is high, then we are likelier to blame the problem on the teacher.
Your entire statement is well thought out and reasonable. But you could easily replace the word “teacher” with bricklayer, bank teller, grocery clerk or rodeo clown and it would still be fair.
The difference is that it takes more time and money to train a teacher, the teacher is paid by taxpayer dollars (thus requires lots of regulation to make all interested taxpayers happy), and the other workers you mention likely wouldn’t even have a grievance process to resort to.