What are some reasons studies might not find a difference for merit pay or we might not trust their findings. It might be that teachers are already doing their best; therefore, giving them additional financial incentives to do better produced no increase in achievement. It may be that the teachers felt forced, controlled or required to use methods different from what they chose or felt best. It may be that the incentives are not appropriate: large or attractive enough to change behavior. It may have been that the targets to receive the reward were too high or unreasonable, therefore limiting motivation to try to reach them. It may have been that the teachers either did not understand or buy into the program. It may have been because of some bias in who is selected to get bonuses or differences between the experimental group and comparison groups. As has happened in other settings, the gains may have been illusory, rather than measures of real achievement.
All but the first of the above reasons were controlled for in a three-year randomized experiment by the National Center on Performance Incentives (NCPI) (Moran, 2010; Springer, Ballou, Hamilton, Lockwood, McCaffrey, Pepper, & Stecher, 2010) that found that incentive pay did not lead to improved performance. It was left to teachers to decide what, if anything, they did to improve their students’ scores. The incentives, $5,000, 10,000 or 15,000 depending upon the gains in scores, were considered significant and attractive enough by the teachers involved. The incentive thresholds were considered attainable and reasonable, requiring students to answer only 2 to 3 more questions correctly on the 55-item exam to qualify for the minimum bonus of $5,000. The teacher understood and bought into the program. Bias was limited by randomizing selection of the teachers in the study and other factors controlled for between treatment and control groups in the experiment. Cheating and gaming the tests were controlled to avoid false results. Teachers in the group with the potential to receive the bonus pay, did little to change what they were doing and varied little from the control group in teaching practices and beliefs.
Though this study did not find performance incentives made a significant difference on middle school mathematics test scores overall, they did find a positive effect in grade 5 achieivement. A possible reason is that these teachers were more likely to teach their students in multiple subjects; therefore could spend more time on mathematics, the subject tested, or that they knew their students better.
A later study by RAND (Marsh, Springer, McCaffrey, Yuan, Epstein, Koppich, Kalra, DiMartino & Peng, 2011) tested the effects of performance bonuses in a random sample of New City high-needs public schools. This 2007-2010 study was to improve student performance using school based financial incentives rather than the teacher incentives used in the NCPI study. They did not find the bonuses, about $3,000 per teacher/staff member and about $56 million for the total program, improved test scores. Schools were allowed to determine how they would distribute their bonuses. Most schools chose to distribute them equally, but some schools used performance measures to differentiate the amount of award each person received. There was no difference in achievement between these schools.
This study further challenges the assumptions of policy makers that teachers will teach better and students learn more if they are rewarded more. Unlike the National Center on Performance Incentives, the RAND investigators suggested that the conditions needed to motivate the teachers may not have been sufficient to make a difference in performance. These conditions are considered important pay-for-performance programs based on expectancy theory (Vroom, 1964; Marsh, et al. 2011). For example, in this study more than a third of the staff members were lacking in an understanding and awareness of how the incentive program worked and how it might affect them. Though most of the teachers wanted to win bonuses, almost half felt they were not large enough to motivate extra effort. Buy in was limited in that only 57% of the staff members had to agree to participate for a school to be included in the experiment and most felt the criteria relied too heavily on test scores. Some also felt the targets for bonuses were too high and the timeline not effective.
As discussed elsewhere the reason pay-for-performance for teachers has not been shown to be effective may be that teacher motivation is not the problem, and therefore to incentivize it is not the solution. Teachers are already motivated to do their best and greater financial rewards will not result in greater output. It may be the more money is the wrong incentive, even counterproductive, as it works against other motivations or adds more stress. It also may be that improved teacher motivation alone is not enough to make a difference in student achievement. Maybe other factors are more important and override the influence of merit pay.
Marsh and fellow researchers concluded
The results of this evaluation add to a growing body of research from the United States that finds no effects on student achievement of narrow pay-for-performance policies that focus only on financial incentives without other features, such as targeted professional development or revised teacher evaluations (Marsh, et al. 2011, p. 264).
They recommended that for pay-for-performance policies to work, they should create conditions that foster strong motivation, identify the factors that affect motivation, address the politics of school-level implementation and pilot test and evaluate before full implementation.