AccountabilityNeedsOtherFactorsGlobe20Jan01

MCAS rating system needs to be fixed

By David G. Tuerck, 1/20/2001

EDUCATION REFORM in Massachusetts took an important turn this month when the Department of Education released its cornerstone School Performance Rating Process report. The department issued the report as a step toward the implementation of the Education Reform Act of 1993, under which the Commonwealth assumed a greatly expanded role in funding and managing what were locally funded and controlled public school districts.

The report examined schools for improvement on MCAS tests given to public school students from 1998 to 2000. One aim was to identify schools that do exceptionally well or exceptionally poor at teaching. Laudable as this aim may have been, the methodology used to rate schools was flawed, based as it was on the subjective views of the report's authors. There is nothing wrong with department officials expressing their hopes for school improvement. But a meaningful rating system must be based on statistics, not hopes.

This is important because schools stand to incur penalties or receive rewards based on the ratings they receive. Once identified as failing, schools can receive warnings or referrals for review.

School districts found to be chronically under-performing can be put in receivership. Conversely, schools identified as exceeding expectations can be recognized as exemplary. Unfortunately, there is likely to be a large element of unfairness in these penalties and rewards. This is because the rating system ignores what everyone knows - that an evaluation of a school's performance must consider factors beyond its control, in particular, the socioeconomic character of the community in which it operates.

An inner-city community cannot match the performance of a leafy suburb. Factors such as a community's crime rate or the number of professionals residing there affect its schools' performance. A business owner cannot determine what he expects of a store's manager without considering that manager's customer base. The owner of a chain of swimsuit shops does not expect his Maine shops to match sales in January with his Florida shops.

The same goes for different schools striving to teach students from very different communities. Education officials defend the rating system on the ground that they don't want us to permit socioeconomic factors to limit our expectations of students in disadvantaged schools. But research consistently shows that socioeconomic factors influence school performance. To ignore these factors is to give low marks to some schools that deserve to be rewarded and high marks to some schools that deserve to be penalized.

A statistical model developed by the Beacon Hill Institute shows how a consideration of socioeconomic factors can lead to results that are very different from those obtained by the Department of Education. Schools whose students do much better on MCAS when socioeconomic factors are taken into account can be fairly deemed as exceeding expectations. Conversely, schools doing much worse can be deemed as failing to meet expectations.

Consider the Hadley school district. The Department of Education rated Hadley's fourth and 10th graders as having failed to meet expectations and eighth graders as having only approached expectations. Yet, when socioeconomic factors are considered, Hadley ranks among the top ten, according to its success in outperforming other school districts.

Why does Hadley do so well? The answer is a lean administrative staff, committed teachers who voluntarily give extra time to meet learning targets, and a community dedicated to keeping its school system small in order to maintain control of the curriculum.

Faced with the prospect of having to deny graduation to students unable to pass the MCAS tests, Massachusetts schools need to know which schools among them have a track record of good teaching.

A rating system that leaves good schools buried in an array of bad statistics only harms those schools that need help the most. The system must be right if we are to do right by our students.

David G. Tuerck is executive director of the Beacon Hill Institute and chairman and professor of economics at Suffolk University.