Jay P. Greene. Manhattan Institute for Policy Research. 1/8/2005.

Testimony of Jay P. Greene on the CFE Lawsuit
Senior Fellow, Manhattan Institute for Policy Research

The decision in the CFE lawsuit ordering more than $5 billion in additional spending on New York City Schools is likely to have little effect on student achievement in the city.  Lack of money is not a primary explanation for the city’s low student performance, so having additional money by itself will do little to improve the situation. Of course, Judge DeGrasse was presented with considerable evidence to this effect but chose to disregard that evidence, deciding instead to order the legislature to spend several billion more of the taxpayer’s dollars on the city’s schools.

Even if Judge DeGrasse found that New York City public school students were being denied a sound basic education as required by the state constitution he did not have to order spending increases as the remedy. This is especially true if lack of money is not the primary difficulty facing the city’s schools.  He could have ordered other reforms that do not require significant increases in spending and have been demonstrated to improve student achievement. For example, the judge could have ordered that vouchers be made available to students at public schools denying students a sound basic education.  Those students could have then been given the resources individually to purchase a sound basic education at a different school if they wished. 

Families with vouchers would be given the power to ensure that their children received a quality education without being dependent upon the slow and unreliable goodwill of school bureaucrats, elected officials, or judges.  And vouchers would provide public schools with greater incentives to offer a sound basic education in order to attract and retain students and the revenues they generate. A system where schools are guaranteed ever increasing amounts of money regardless of how well they serve students is one that is likely to fail educationally while costing the taxpayers dearly. 

The idea that a voucher system would provide incentives to public schools to improve the quality of their education while also improving outcomes for students who use the vouchers to leave public schools is not just a plausible theory; it is supported by a considerable body of the highest quality research. There have been eight random-assignment studies of the effects of voucher programs on the students who use them to attend private schools.  Random-assignment, where a lottery places students into treatment and control groups that are virtually identical at the start of the experiment, is the gold standard of research designs.  All eight random-assignment studies of voucher programs have shown positive effects and all but one has shown those positive effects to be statistically significant. 

Research also strongly supports the idea that expanding school choice would improve the performance of the public school system as a whole.  You don’t have to believe me or the research I have conducted on this matter.  Listen to the conclusions drawn by Clive Belfield and Henry Levin at Teachers College, a place not known for its choice enthusiasm. They reviewed the research on how expanding choice affects public school performance and found: “a sizable majority of these studies report beneficial effects of competition across all outcomes, with many reporting statistically significant correlations.” They then conclude that “the above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.” Rather than draining public schools of the talent and resources they need to succeed, as some allege, the evidence shows that choice and competition would help New York City public schools provide a sound basic education.

 

Belfield and Levin’s findings on the benefits of choice and competition may apply as much to the expansion of choice through opening additional charter schools as offering students vouchers to attend private schools.  Given that Chancellor Klein has already announced they city’s intention to open 50 new charter schools, Judge DeGrasse could have helped ensure that the city offered a sound basic education by accelerating the opening of those schools and providing charter schools the same per pupil resources that traditional public schools enjoy.

Instead, ignoring the bulk of the evidence and years of experience, Judge Degrasse ordered the legislature to spend a whole lot of money to fix the problem.  Our challenge then is to devise ways in which these new expenditures could produce better results than previous new expenditures. One promising use of new money is to create substantial merit pay systems that directly link large bonuses for teachers and administrators to increases in student achievement. 

The primary difficulty with previous increases in spending is that spending was never significantly dependent upon performance.  More money was provided for buildings, for teachers, and for administrators regardless of how well students learned.  Perversely, schools and staff have been in a stronger position to demand spending increases the worse their students do, proof of which can be found in the court decision we are discussing today.  A system that financially rewards failure is far more likely to produce failure.

What we need is a system that provides financial incentives for success. If we look at how teachers are compensated today, we see a complete absence of financial incentives for teachers to improve student achievement.  Basically, teachers are paid more for possessing credentials, such as advanced degrees, and for having taught for more years.  A system that pays teachers more for obtaining credentials, such as a masters’ degree, or for sticking around as a teacher for many years would make sense if credentials or longevity in the profession meant that someone was likely to be a better teacher.

Unfortunately, these qualities are not related to student achievement. In a review Eric Hanushek of Stanford University conducted of 171 scientifically valid studies on the effect of teacher credentials on student achievement only 9 found a statistically significant positive relationship while 5 found a statistically significant negative relationship.  Since we would expect about 9 studies to have statistically significant effects in either direction just by chance, this pattern of results indicates that research finds no relationship between teacher credentials and student achievement. Another recent review of the research literature by the Abbell Foundation similarly found no relationship between teachers holding education masters degrees and student achievement.

Recent high quality studies confirm these reviews of the research. Rivkin, Hanushek, and Kain analyzed several years of Texas data and found that advanced degrees had no effect on student achievement.  They concluded: “Similar to most past research, we find absolutely no evidence that having a master’s degree improves teacher skills.” Two separate analyses of data from the National Assessment of Educational Progress (NAEP) by Wenglinsky and Grissmer both found that master’s degrees did not improve student performance. Also, Dan Goldhaber’s research for the Urban Institute found that “only about 3 percent of the contribution teachers made to student learning was associated with teacher experience, degree attained, and other readily observable characteristics” combined.

The evidence supporting paying experienced teachers more is hardly stronger than the evidence for paying credentialed teachers more.  The Abell Foundation’s review of the available research reports that a number of studies have found that teaching experience only improves teachers’ performance in the first two or three years of teaching, and may actually be associated with a decline in teachers’ performance at the end of their careers.

The finding that only the first few years of teaching improve performance is confirmed by two recent studies. The Rivkin, Hanushek, and Kain study of Texas data found no effect on student achievement for teacher experience generally, but it did find that teachers with at least two years of experience performed better than teachers with fewer than two years. Even here, the size of the effect they identified was very small; the authors write that “the magnitudes of the effects of these variables pale in comparison to the total effect of teacher quality.” An analysis of NAEP data conducted for RAND by Grissmer and his colleagues finds that while teacher experience generally had a weak effect on teachers’ performance, additional years of teaching had no effect on performance for teachers who had more than two years of experience.

Thus the evidence seems to indicate that teachers get a little more effective in their first few years as they get up to speed in the classroom, but that after this initial period teachers do not tend to get more effective with more years of experience.  They may in fact get worse after many years of teaching if they get burned out.

The bottom line is that the research literature clearly does not support the method by which teachers are paid in New York. Paying teachers more simply because they get a masters degree is unlikely to have any effect on student achievement.  Paying teachers more when they stick around for five, ten, or twenty years is similarly unlikely to affect student achievement.  Since the primary purpose of schools is to improve student learning, spending money on anything that does not enhance learning is, by definition, wasting that money. Pouring more money into such a system would only waste more money, not improve student achievement.

Basing teacher compensation on credentials and years of teaching is comparable to having the Mets pay players based on their hat size. If you pay people according to factors that are largely unrelated to job performance, you shouldn’t expect excellent performance. Paying teachers based on longevity and credentials just produces a more senior work force that has a lot of masters degrees but it does not encourage good teaching. Similarly, paying Mets players based on hat size would just produce big-headed players who aren’t very good at baseball.

What we need is a system that aligns the compensation of teachers and administrators with student achievement.  The more that educators improve student learning the more they should get paid. If New York must significantly increase spending on the City’s schools, it should do so by offering bonuses that are linked to gains in student learning. 

Such a merit based system is not only in keeping with common sense it is also supported by a growing body of research. For example, Thomas Dee and Benjamin Keys at Swarthmore analyzed a merit pay system implemented in Tennessee at the same time as that state conducted the Project STAR class-size experiment.  Using a high-quality random assignment design they found that the merit pay system increased student achievement by about 3 percentile points in math and 2 percentile points in reading, although the reading sample was too small to be statistically significant.  They concluded that the benefit of having a teacher in the merit pay system was “equal [to] 40 to 60 percent of the estimated gains associated with assignment to a small class. Furthermore, these gains are approximately equivalent to a third of the corresponding black-white gap in test scores.”

Two studies of a merit pay program in South Carolina come to similarly positive conclusions. Using a regression analysis of class-level data Cohen and Teel and then Cooper and Cohn found that teachers working in merit pay systems produced significantly higher math and reading scores than teachers paid under the traditional system based on seniority and credentials.

Of course, someone might ask: if the benefits of Tennessee’s merit pay system were about half as large as reducing class size, why not just reduce class size?  The answer is that reducing class size costs a whole lot more, since a one-third reduction in class size, as was done in Tennessee, roughly requires one-third more classrooms built, one-third more teachers hired and therefore a one-third increase in expenditures.  Merit pay costs only a fraction of that. The City could get much more increase in learning for the dollar with merit pay.  But if cost is really no object, then I would urge New York to do both, reduce class size and adopt merit pay. But no one should come before you and cite the Tennessee results urging you to back smaller class sizes without also being willing to embrace the merit pay results from the same random-assignment data.

In addition, there are good reasons to believe that the benefits of a merit pay system more directly linked to increases in student learning than the one adopted in Tennessee might produce larger benefits than found in the Tennessee study. It is always a danger that programs that are described as merit pay only link pay to factors that are at most tangentially related to student achievement.  For example, linking pay to subjective peer-assessments where almost everyone receives the highest performance evaluation is just a disguised version of an across-the-board pay increase, not a true merit pay system. Similarly, paying teachers more for new credentials, like national board certification, is just more of the same old system of pay for credentials without clear evidence linking those credentials to student learning.  Ideally, merit pay systems link pay directly to improvements in student achievement rather than to various factors that are assumed or believed to result in improved achievement. The Tennessee program fell short of this ideal, including many factors in its assessment of merit that may have had little to do with student learning, so the benefits of an ideal program could be substantially larger. 

Some might object that a merit pay system linked directly to increases in student achievement as measured by test scores will only produce teaching to the test, cheating, or other manipulations of the results. The fear is that the more consequences for educators from a testing system, the less reliable and meaningful the test results will be.  While this is a plausible fear, it is not consistent with the evidence. In a study I led that was published last year in the Teachers College Record, attaching stakes to test results did not undermine the reliability of those results.

We obtained data from two states and seven school districts in which students took both a high stakes test, one with significant consequences, as well as a low-stakes test, where there were few or no consequences related to how students scored.  As it turns out, the results of high and low stakes tests given to the same students around the same time tend to correlate very strongly. That is, both high and low stakes tests tend to produce similar results. Since no one has an incentive to teach to, cheat on, or otherwise manipulate the low stakes test results, this suggests that stakes do not distort test results.  So New York could attach merit pay bonuses to the improvements educators help produce in individual students’ test scores without fearing that the integrity of test results or the quality of education would be undermined.

Of course, the integrity of high stakes test results has to be maintained by adopting safeguards and auditing results. Similarly, the income tax produces incentives for people to misreport their income, but the system generally produces revenue as desired because its integrity is preserved by checks and auditing and because most people are inclined to be honest.  The same can and must be done to ensure the integrity of test results if merit bonuses are linked to them.

Paying teachers more for being better teachers does not mean that teachers are bad people who will only care about kids if they are paid to do so.  Most teachers, like most people, care about kids and want to do well in their jobs.  But like most people teachers are likely to do a better job if they are rewarded for doing so.  Rewarding excellent teachers helps attract and retain those people as teachers.  Denying automatic pay increases for unsuccessful teachers may also have the benefit of encouraging those less capable teachers to leave the profession.

Most skilled professions link pay to performance, it is time that we do the same in education. If New York must spend significantly more money on its schools, using the money to fund a meaningful merit pay system is one of the most promising options I can imagine.