IX. INCENTIVES IN THE PUBLIC AND NONPROFIT SECTORS: DO HIGH-PERFORMANCE WORKPLACE PRACTICES WORK?

Teacher Performance Incentives, Collective Bargaining, and Student Outcomes

RANDALL W. EBERTS AND KEVIN M. HOLLENBECK
W.E. Upjohn Institute for Employment Research

JOE STONE
University of Oregon

Abstract

      This paper reviews evidence on the effectiveness of individual merit pay systems for teachers on student achievement, presents new empirical results on a system established within a collective bargaining environment, and reviews evidence of the impact of teacher unionization on student achievement. While many merit pay systems have been established in school districts across the United States, little empirical evidence concerning their influence on student achievement exists. A natural experiment arose in a county in which one high school piloted a merit pay system that rewarded student retention, while another comparable high school in the county maintained a traditional compensation system. A differencein-differences analysis implies that merit pay increased retention, had no effect on grade point averages, reduced average daily attendance rates, and increased the percentage of students who failed. Empirical studies of the influence of teacher unionization on student achievement seem consistently to find positive impacts, albeit at increased per pupil cost.

Introduction

      Recent discontent with the performance of U.S. public elementary and secondary schools has generated a series of reform proposals. Some reformers have advocated incentive-based schemes to improve school quality (Hanushek 1994), such as merit pay for individual teachers or school-based performance awards. Others have advocated institutional changes such as policies to weaken collective bargaining (Ballou and Podgursky 1997). The purpose of this paper is threefold: (1) to review the (scant) evidence on the effectiveness of incentive-based compensation schemes on student achievement, (2) to present new empirical evidence about the efficacy of individual merit pay from a case study, and (3) to review the evidence about the impact of collective bargaining on student achievement.

Pay-for-Performance Compensation in K–12 Education

      Advocates of incentive-based schemes to reform public schools often refer to the private sector as an example of individual performance-based compensation systems and as one that schools should emulate. Yet, even the simplest incentive models are subject to pragmatic problems when they are implemented, and evidence reveals that only a small proportion of jobs in the private sector base compensation on explicit contracts that reward individual behavior.¹ The simple, static principal–agent model that Prendergast (1999) explicates rewards agents for taking on additional risk through a pay-for-performance contract with higher (mean) wages. In his model, the performance measures used are noisy, and the efficacy of the incentives depends on the risk aversion of the agents. Furthermore, incentives may result in unintended, sometimes perverse, consequences. Prendergast uses the term dysfunctional behavioral responses; Murnane and Cohen (1986) call them opportunistic behaviors. Institutional factors that may result in such dysfunctional responses include poorly defined or poorly measured outcomes leading to a reliance on flawed subjective evaluations, multitasking by job incumbents, team production, and multiple principals/stakeholders. Subjective evaluations may be flawed because (1) evaluators may be subject to a moral hazard problem, (2) individuals being evaluated may engage in non-productive activities to curry favor with their evaluators, or (3) evaluators may end up with distributions of ratings that are compressed because of a reluctance to give very high or very low ratings (Prendergast 1999:29–31).

      Jobs in which the incumbents perform many different tasks also strain an incentives-based compensation contract. First, multiple tasks imply multiple performance measures, some of which may be costly to measure. Second, if performance measures are skewed in their relative weights, then the agent may respond by investing too much effort into the tasks that receive the most weight in the performance measurement system.

      Team production introduces the “1/n” problem, in which each individual’s contribution (and reward) is diluted by the size of the team. Furthermore, if the individuals’ contributions to the team are costly to observe or measure, then team-based incentives may lead to free riders. The problems for an incentive-based compensation system when there are multiple stakeholders come from a potential for misalignment of organizational goals. In effect, the principal–agent arrangement becomes a “principals”–agent problem. Multiple principals may have different, and conflicting, goals. For example, for employed individuals, training directors and production supervisors may conflict with each other on how to reward an individual’s (paid) time spent in training activities.

      The nature of the educational process features each of these complications and confounds the effectiveness of individual performance-based compensation systems.² The four constraints on the effectiveness of incentivesbased compensation--need for reliance on subjectively measured outcomes, multiple tasks undertaken by incumbent workers, team production, and multiple stakeholders--characterize the teaching and learning process in schools. Learning outcomes may be assessed through standardized tests, which are amenable to performance-based contracts (particularly if value-added measures are available). Yet, many additional dimensions to student learning and development either are not assessed or are assessed without standardized instruments, so evaluations must be inherently subjective.

      Schools (at all levels of the K–12 system) typically have dozens of learning processes or programs going on simultaneously. These include core academic subjects; noncore academic subjects such as art, physical education, music; acquisition of technology skills; career development; special education; extracurricular offerings; gifted and talented programs; human growth and development; and remediation or developmental education. Even within a teacher’s discipline, multiple tasks comprise the teaching and learning process-curriculum development and planning, instruction, and assessment, for example. Furthermore, good teaching requires attention to students’ learning styles, which may mean multiple modes of instruction.

      Education is often delivered through team production. For example, many elementary and middle schools are organized into teams of teachers. However, even apart from explicit team teaching, departmentalized secondary schools result in team production, since students’ performances on standardized tests depend on learning in several courses taught by different teachers.

      Finally, school governance and control is characterized by many different stakeholders with differing, and sometimes conflicting, goals. Administrators who are accountable for direct student achievement may be most responsive to levels of test scores. School boards, accountable for resource decisions, may be most interested in changes (value added) over time in test scores. Parents may be most concerned about postsecondary education attendance rates, whereas employers may be most concerned about “soft,” employability skills such as problem-solving, attendance, and attitude.

      Team, or building, incentives theoretically ameliorate the problems of multiple tasks and stakeholders. School-based performance systems have been adopted by several districts. For example, districts in Kentucky and South Carolina have implemented a system in which high-performance schools receive additional revenue that can be used at the schools’ discretion, including in some cases offering additional compensation to teachers. Clotfelter and Ladd (1996) analyzed Dallas’ performance-based system and found an increase of 10 to 12 percent in the pass rate on selected state-wide tests. Unfortunately, the study did not use a true control group, so it is unclear if the incentive system was primarily responsible for the gains.³

      Another characteristic of most school districts is that they have very little control over their revenue streams. As noted, incentive-based contracts allocate part of the production risk to the employees in return for higher rewards (wages). Since school administrators have little revenue to share, they cannot offer sizable increases in compensation were teachers willing to accept the risk inherent in a merit pay system.

      In short, while economic actors may respond to incentives, there may be several wedges between performance measures and the actions of teachers who tend to mitigate against individual level, incentive-based compensation schemes in education--just as they do in the private sector. The net result of these forces remains an empirical issue. Yet, little empirical evidence examines the effects of merit pay on student achievement. Most of the literature on merit pay systems documents the institutional experiences in districts--for the most part, rather short-lived and usually negative. For example, a major study of merit-based pay (Hatry, Greiner, and Ashford 1994) found that most (75 percent) merit pay programs that had been in existence in 1983 and had been studied by the researchers, were no longer operational in 1993.⁴ An interesting self-described limitation of the Hatry et al. (1994) study is that they did not examine student achievement. They note,

We would especially have liked to have performed an in-depth analysis of the impact of incentive programs on student achievement. However, very few of the participating districts had attempted any systematic evaluation of the effects of their incentive plans on student achievement, even though a basic assumption behind incentive plans is that teachers can indeed significantly affect learning. (pp. 7–8)

In a study involving one district in Pennsylvania, Tulli (1991) found no correlation between gains in student achievement and teachers awarded merit bonuses under this district’s plan.

A Case Study of a Merit Pay System

      We have acquired data from a particular high school that implemented a merit pay system in 1996 and a “comparable” high school that maintained a traditional compensation system.⁵ Community High School, which implemented the merit pay system, is an alternative education facility that has an enrollment of approximately 500 students pursuing a high school diploma and 100 students pursuing other certifications. Alternative education settings are characterized by students who have often not succeeded in traditional school settings and usually experience attendance problems and intermittent dropping out and reenrollment episodes. Consequently, the performance-based incentives were targeted on student retention. The results and a more detailed description of this study is found in Eberts, Hollenbeck, and Stone (forthcoming).

      The district decided to operate Community High School as a “pilot” program with a performance-based compensation scheme for its teachers, who collectively decided to remain separate from the local district’s education association (union). The merit pay system that was implemented offers two supplements to teachers’ base pay. The first supplement is a retention bonus, of approximately 12.5 percent, which is paid if 80 percent or more of the students assigned to the class (as of the end of the second week of the quarter) are still enrolled and attending at the end of the quarter.⁶ The second supplement is based on student evaluations. Students rate 15 factors on a 5-point scale, and teachers who receive an average rating of 4.65 or higher (the average rating in 1994–1995) for all 15 items in all of their classes (weighted by class enrollment) in each quarter for four consecutive quarters receive the performance bonus, which increases their base pay by about 5 percent and increases their retention bonus by 10 percent.⁷

      We performed a difference-in-differences analysis of several student outcomes: grade point average, class attendance, course completion, and passing rates conditional on course completion.⁸ The analysis included data from the period 1994/95 to 1998/99 for students at this school and at a similar alternative education high school in the same county that relies on a traditional experience/education compensation scheme.⁹ The data encompass 2 years prior to and 2 years after the implementation of the performance incentive system. The grade point average (GPA) is calculated from student-level data; the other three outcomes--attendance, completion, and conditional passingare calculated from course-level data.

      The results are consistent with expectations regarding the effect of incentives on teacher behavior. As shown in the first row of Table 1, the percentage of students who completed courses was dramatically higher in the merit school than in the traditional school. While the completion percentages increased in both schools over the 5-year period, the increase was larger and quite dramatic in School A, as would be predicted. Attendance, on the other hand, was not rewarded (except that a student had to be present during the last week of classes to be considered a completer). Results in the second row of the table show that the merit system appears to have little effect (and, in fact, the sign is negative) on daily attendance. School A’s attendance rate stayed approximately the same in the 2 years, and School B’s rate actually went up slightly, which is the opposite of what one would expect if teachers were to respond to economic incentives by finding ways to increase overall attendance and not simply during the week the actual class count was taken.

The increase in course completion had an adverse affect in outcomes related to student achievement. The (student) average GPA in both schools declined over the 5-year period, but the decline in School A of 0.53 points was greater in magnitude than the decline of 0.37 points in School B.¹⁰ The fact that the decline in School A was greater than the decline in School B is consistent with the hypothesis that the merit pay incentive resulted in higher retention of lower-achieving students, who were most likely to drop out. Finally, consistent with the GPA analysis, the percentage of students actually passing their courses declined over the period of analysis. Again, the decline was far larger for School A, which went from approximately 93 to 75 percent (fourth row). That school’s decline in the percentage of students passing the course conditional on completion is more than 6 percentage points greater than School B’s, which is consistent with the hypothesis that School A is retaining, on average, more low-achieving students.

      The analysis reveals that teachers responded to the incentives explicitly incorporated into their incentive-pay system, but they did not pursue, at least not as vigorously, those outcomes that were not directly rewarded. Course completion was rewarded, and it was significantly higher for students at the merit-based school. Daily attendance rates were not rewarded, and there was actually a statistically significant decline in attendance rates. The same was true with GPA and the percentage passing courses: the merit-based school did worse than the traditional school.

      The outcomes illustrate the difficulty of instituting individual merit pay in schools. First, the output measure has to be easily, inexpensively, and accurately determined, and it has to be agreed upon up front. In this case, the administrators of the high school knew that they wanted to increase retention. The incentive “worked” according to the retention measure adopted by the school, but it did not work with regard to passing rates nor to GPA, which could be considered a measure of student achievement. This finding leads to the second difficulty: the output measure should be the organization’s final product, or at least highly correlated with the final product. In this case, the definition of final product is ambiguous. Administrators articulated that student achievement is a primary goal, but the incentive system did not appear to promote it.¹¹

Collective Bargaining and Incentives

      An additional dimension that must be considered when discussing incentives in schools is collective bargaining. Unlike many private sector industries, particularly those that are service-oriented, public education is highly unionized, with coverage reaching about 63 percent of public school teachers. As such, incentives must be considered within the context of collective bargaining agreements.

      Such agreements establish rules that affect the working conditions of teachers and thus the school environment. If improvements in working conditions are in line with factors that positively affect student achievement, then collective bargaining can lead to improved student outcomes. In fact, Eberts and Stone (1984) report that teachers covered by collective bargaining (1) have smaller classes (see also Argys and Rees 1995), (2) spend more paid time in class preparation (see also Hoxby 1996; Kleiner and Petree 1988), (3) are more likely to adopt traditional classroom instruction as opposed to other arrangements, and (4) place more importance on participation in student assignment and teacher assignment than do teachers not covered by contracts. These authors also find that fourth graders in unionized districts on average spend 42 percent less time with a specialist, 62 percent less time with an aide, 26 percent less time with a tutor, and 68 percent less time in independent, programmed study.

      Several studies have shown that the factors discussed above, namely class size, teacher time, and instructional leadership, are positively related to student achievement gains.¹² A much smaller set of studies has examined the direct effect of collective bargaining on student outcomes. Stone (1998) summarizes and critiques seven such studies. Eberts and Stone (1984, 1987) use detailed student, teacher, and school data from a national sample of 14,000 elementary students and find that students in districts covered by a collective bargaining contract scored roughly 1 percent higher on a post-test, or about 3.3 percent higher as a percentage of the average gain from the pretest to the post-test (statistically significant).¹³

      Studies using aggregate state data find larger positive effects of unions than those using student-level data. Kleiner and Petree (1988) find that SAT and ACT scores are 6 to 8 percent higher in states with 100 percent union representation versus states with no representation. Nelson and Rosen (1996) include more detailed control variables and find that students in states with more than 90 percent union coverage score on average 4.5 percent higher on SAT tests than students in states with fewer than 50 percent union representation.

      The positive effect of unions on student achievement is not enjoyed by all students. The student-level studies by Eberts and Stone, Milkman, and Argys and Rees find an inverted U-shaped effect of collective bargaining on student achievement gains. For students of average ability, as measured by pretest scores, those in union districts score higher on post-tests than those in nonunion districts. The opposite is true for low-achievers and high achievers. Below- and above-average students in nonunion districts score higher than those in union districts.

      These results for below-average students are consistent with those presented in Hoxby’s (1996) detailed district-level study of the effect of unions on high school drop-out rates. She finds that the presence of collective bargaining, where at least 50 percent of the teachers are union members, increases high school dropout rates by 2.3 percent. Recognizing that students at the lower end of the test-score distribution are more likely to drop out of school, Hoxby’s results are consistent with the studies that show that below-average students in union districts experience less academic success.¹⁴

      Therefore, empirical studies of the effect of collective bargaining on student achievement find little, if any, support for the argument that unions on average reduce academic success. Unions, by negotiating rules to standardize the workplace through class size provisions and traditional instructional models, may affect students with different abilities, but even these effects may be small. Thus, the codification of bureaucratic rules through collective bargaining agreements does not appear to have significant negative effects on student achievement.¹⁵

      This is not to say that unions do not have detrimental effects. Unions increase the cost of education by between 8 percent and 15 percent and distort the least-cost combination of inputs (Eberts and Stone 1991; Stone 1998). These findings are consistent with the prediction that bureaucracies establish rules to avoid counterproductive activities, such as influence activities, at the expense of inefficient allocations.

Conclusion

      In summary, we argue that the nature of schools and the teaching and learning process make the use of individual-based merit pay an extremely delicate weapon to use in the arsenal of school reform. Incentive systems within education, with its multiple goals and outcomes, team production, and multiple stakeholders, may produce unintended results that are, at times, misdirected--unless carefully constructed and implemented. The case study results buttress this point.

      In this case, the implementation of a merit pay system in a specific high school showed that incentives do “work.” The merit pay system directly targeted at student retention, as defined by a measure understood and agreed upon by both teachers and administrators, resulted in higher student retention, as defined by attendance during the last week of classes. However, student grade-point averages and daily attendance rates were virtually unchanged, and course passing rates declined. There was also anecdotal evidence that suggested that course content was diluted.

      The paper argues that weakening teacher collective bargaining institutions is also unlikely to improve student achievement, particularly for students within the middle range of academic success. (The evidence does seem to suggest that unionization is detrimental to students at either end of the distribution of achievement, however.)

      What about group or organizational incentives? First, it is interesting to note that they have been instituted successfully in several unionized districts (see CPRE 2001). Second, they overcome some of the issues that plague incentive schemes in education. School-building performance awards are less subject to the problems associated with subjective evaluations, multitask job descriptions, and team production. However, it is still the case that multiple outcomes and multiple stakeholders may complicate the design of a group award system such that it results in unintended consequences. Consequently, we agree with Hanushek’s (1994) prescription that administrators and policy makers should constantly evaluate and be ready to revise their performance award systems.

Acknowledgments

      An earlier version of the paper was prepared for a National Academy of Sciences Conference entitled “Devising Incentives to Promote Human Capital,” held December 17–18, 1999, in Irvine, CA. The authors appreciate the helpful comments of Derek Neal, Richard Murnane, other participants of the conference, and two anonymous referees. The research assistance of Kristine Kracker and Noyna DebBurman is gratefully acknowledged, as is the excellent clerical assistance of Phyllis Molhoek and Claire Black. The views expressed do not necessarily represent the views of The W.E. Upjohn Institute for Employment Research or The University of Oregon.

Endnotes

1. For example, a study in the early 1980s found that the practices of merit pay in private industry are neither as common nor effective as many believe (Lawler 1983).

2. Much of the argument presented here was also presented in Murnane and Cohen (1986). Dixit’s (1999) analysis of incentives in education also coincides closely with ours. He suggests four complications in educational settings that confound the simple “principal– agent” model of implicit contracting: multiple goals, multiple principals, lack of competition in the product market, and agents motivated by intrinsic values.

3. Private sector businesses reward workers more through promotions and group-based merit systems, such as gainsharing or profit-sharing, than through individual merit rewards (Prendergast 1999). See Kruse (1993) for a study of the effects of profit-sharing in private industry.

4. Murnane and Cohen (1986) also emphasize the short-lived nature of merit pay systems.

5. Unfortunately, the data only contained course-related information such as grades and daily attendance. They did not include any information about the students other than ID number.

6. The initial enrollment in the class for purposes of calculating retention is capped at 20, so to earn the retention bonus, teachers must have 16 students or 80 percent of the initial enrollment at the end of the term, whichever is less.

7. Hatry et al. (1994) found a range of merit pay awards in their study from at most 25 percent of salary to 5 percent or less (see also Lawler 1983). To give the reader a sense for the size of these bonuses, during school year 1998–1999, the base pay for a beginning teacher with a bachelor’s degree was $816 per class ($22,848 for 9 months; 4 quarters with 7 classes). With the performance bonus and retention bonuses in all classes, the per-class pay would be $979 ($',412 for 9 months; 4 quarters with 7 classes). Many teachers have more than six classes per term. With at least six, the teachers receive full benefits equivalent to the unionized teachers in the district.

8. The difference-in-differences technique “differences out” time-invariant causal variables and assumes that there is no interaction between the “treatment”--that is, merit pay--and time-varying causal variables. In short, it is appropriate in this case only if both high schools’ student characteristics, curriculum and instruction, and outside external factors such as the local economies changed similarly. Unfortunately, the small sample size and data deficiencies did not allow formal testing of these assumptions.

9. With no data on detailed student characteristics, we relied on the judgment of building administrators and district educators in selecting the best local alternative school to use as a control. Both schools are located in the same county but not in the same district. The districts are both suburban. Educators familiar with both schools indicated that the schools were comparable in course offerings, student socioeconomic characteristics, and funding levels.

10. The pre-merit pay difference in student GPA levels were substantial and weaken our confidence in the comparability of the schools. Nevertheless, we note that the difference is consistent with the Eberts and Stone (1984) evidence that unionization may have a negative impact on lower-achieving students.

11. Administrators in the merit-based school provided anecdotes that suggested that teachers were altering their instructional style and course content in order to make their courses more interesting to and well liked by students. The teachers were trying to entice students who would otherwise have dropped out to stay in the course to ensure that they would earn their student retention bonus, and they were trying to get better student evaluations, which is the second component of the merit pay plan. Anecdotes included activities such as more field trips and in-class parties.

12. Extensive literature on educational production functions exists, and the issue of whether inputs into the schooling consistently and significantly affect student outcomes is not without controversy. Hanushek (1986) raises the issue of whether inputs matter, and reanalysis of the same literature by Hedges, Laine, and Greenwald (1994) draw the opposite conclusion. As Ladd (1996) points out, those studies based on sounder methodologies, such as analysis of student-level data using pre- and post-tests and controlling for school- and home-based resources, generally show that school inputs do affect student test score gains.

13. Milkman (1989) reports similar results. Students in union districts scored 2 percent higher than students in nonunion districts. In a separate study, Milkman (1997) finds that minority students in union districts score about 1.4 percent higher than similar minority students in nonunion districts. Grimes and Register (1991) and Argys and Rees (1995) also find small, but significantly positive, union effects on student achievement.

14. Stone (1998) provides this explanation to reconcile Hoxby’s results with those of the six other studies that show positive union effects on student achievement.

15. Heckman et al. (1997) consider bureaucratic behaviors in governmental agencies such as educational districts.

References

Argys, Laura M., and Daniel I. Rees. 1995. “Unionization and School Productivity: A Reexamination.” Research in Labor Economics, Vol. 14, pp. 49–68.

Ballou, Dale, and Michael Podgursky. 1997. Teacher Pay and Teacher Quality. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

CCC. 1987. “Teachers’ Unions and the Productivity of Public Schools.” Industrial and Labor Relations Review, Vol. 40, pp. 355–63.

CCC. 1991. “Unionization and Cost of Production: Compensation, Productivity, and Factor-Use Effects.” Journal of Labor Economics, Vol. 9, no. 2, pp. 171–85.

CCC. 1997. “Teachers’ Unions, Productivity, and Minority Student Achievements.” Journal of Labor Research, Vol. 18, pp. 137–50.

Clotfelter, Charles T., and Helen F. Ladd. 1996. “Recognizing and Rewarding Success in Public Schools.” In Holding Schools Accountable. Washington, DC: The Brookings Institution.

CPRE. 2001. December 10 <http://www.wcer.wisc.edu/cpre/tcomp/state/>. Dixit, Avinash. 1999. “Incentives and Organizations in the Public Sector: An Interpretative Review.” Paper presented at the National Academy of Sciences conference on Devising Incentives to Promote Human Capital, Irvine, CA, December 17–18.

Eberts, Randall W., Kevin M. Hollenbeck, and Joe A. Stone. Forthcoming. “Teacher Performance Incentives and Student Outcomes.” Journal of Human Resources.

Eberts, Randall W., and Joe A. Stone. 1984. Unions and Public Schools: The Effect of Collective Bargaining on American Education. Lexington, MA: Lexington Books.

Grimes, Paul W., and Charles A. Register. 1991. “Teacher Unions and Black Students’ Scores on College Entrance Exams.” Industrial Relations, Vol. 30, pp. 354–63.

Hanushek, Eric A. 1986. “The Impact of Differential Expenditures on School Performance.” Educational Researcher, Vol. 18, no. 4, pp. 45–62.

Hanushek, Eric A. et al. 1994. Making Schools Work: Improving Performance and Controlling Costs. Washington, DC: The Brookings Institution.

Hatry, Harry P., John M. Greiner, and Brenda G. Ashford. 1994. Issues and Case Studies in Teacher Incentive Plans, 2nd ed. Washington, DC: The Urban Institute Press.

Heckman, James, Carolyn Heinrich, and Jeffrey Smith. 1997. “Assessing the Performance of Performance Standards in Public Bureaucracies.” AEA Papers and Proceedings, Vol. 87, no. 2, pp. 389–95.

Hedges, Larry V., Richard D. Laine, and Rob Greenwald. 1994. “Does Money Matter? A Meta-Analysis of Studies of the Effects of Differential School Inputs on Student Outcomes.” Educational Researcher, Vol. 23, no. 3, pp. 5–14.

Hoxby, Caroline Minter. 1996. “How Teachers’ Unions Affect Education Production.” Quarterly Journal of Economics, Vol. 111, pp. 671–718.

Kleiner, Morris M., and Daniel L. Petree. 1988. “Unionism and Licensing of Public School Teachers: Impact on Wages and Educational Output.” In Richard B. Freeman and Casey Ichniowski, eds., When Public Sector Workers Unionize. Chicago: University of Chicago Press.

Kruse, Douglas L. 1993. Profit Sharing: Does It Make a Difference? Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

Ladd, Helen F., ed. 1996. Holding Schools Accountable: Performance-Based Reform in Education. Washington, DC: The Brookings Institution.

Lawler, E.E., III. 1983. Pay and Organization Development. Reading, MA: Addison-Wesley.

Milkman, Martin I. 1989. Teacher Unions and High School Productivity. Diss., University of Oregon.

Murnane, Richard J., and David K. Cohen. 1986. “Merit Pay and the Evaluation Problem: Why Most Merit Pay Plans Fail and a Few Survive.” Harvard Educational Review, Vol. 56, no. 1, pp. 1–17.

Nelson, F. Howard, and Michael Rosen. 1996. “Are Teachers’ Unions Hurting American Education? A State-by-State Analysis of the Impact of Collective Bargaining Among Teachers on Student Performance.” Technical report, Institute for Wisconsin’s Future, Milwaukee, WI.

Prendergast, Candice. 1999. “The Provision of Incentives in Firms.” Journal of Economic Literature, Vol. 37, pp. 7–63.

Stone, Joe A. 1998. “Collective Bargaining and Public Schools: A Critical Survey of the Evidence.” Presented at the Conference on Teacher Unions and Educational Change, September 23–25, 1998, sponsored by the Program on Educational Policy and Governance at the Taubman Center for State and Local Government, Kennedy School of Government.

Tulli, Dennis J. 1991. “An Assessment of Student Achievement before and during a Merit Pay Program for Teachers of the Penn Manor School District.” Unpublished dissertation, Temple University, Philadelphia.