(last update: 08-Sep-2002)
In 1978 the Massachusetts Supreme Judicial Court ruled that funding disparities for public schools in Massachusetts denied quality education to some students. In response, the legislation passed the Education Reform Act of 1993. The initial drafts concentrated on funding formulas. Later, curriculum standards and higher academic achievement were added, followed by the creation of charter schools. To see the full text of the Education Reform Act of 1993 follow one of the following two links: HTML format or PDF format.
The reform was supported by a significant financial state aid to communities. It started at $1.3 billion and has grown to $3.0 billion per year. Most of this aid is directed to communities with low household incomes.
For a detailed description of the history and impact of the education reform, please read "Education Reform in Massachusetts" by Craig Bolon.
The primary goals of the reform are: educational equity, uniform curriculum (frameworks), higher academic standards, statewide graduation criteria, and a system of school accountability. Other goals include elimination of general track, professional development for teachers, greater hiring and firing powers to principals, and reduction of local control over schools (this last objective is viewed by many as a hindrance to better education).
Section 1E of chapter 69 (Education Reform Act) empowers the Board of Education to establish a common curriculum for core subjects, the so-called frameworks.
The material covered by the MCAS adheres closely to the frameworks.
Initial frameworks were released in 1996. Since then they have gone through a series of revisions and are still being changed. The ongoing revisions make the already formidable task of rapidly aligning district curricula to the frameworks even more difficult. Discrepancies between schools still exists, caused by the variations in the curricula previously in place, by administrative difficulties, and by other factors. As a consequence, many of the students will be tested on material that their schools may not have taught.
For the latest official version of the frameworks, go to the Department of Education's frameworks web page.
The establishment of the frameworks has not been void of controversy. Besides the expected differences of opinions about what should be taught, the Board of Education has been criticized for:
The mathematics frameworks were protested by many teachers; the history frameworks continue to be very controversial.
Nevertheless, most people agree that the unified curriculum specified in the frameworks is an essential element of the education reform and will greatly benefit all students in Massachusetts. Most private schools are also aligning their curricula with the frameworks, even though they are not required to do so.
The frameworks specifies seven broad subjects: Language, Math, Science, History, Art, Health, and foreign languages. The first four subjects are covered by the MCAS. A passing grade in the first two subjects on the 10'th grade MCAS is required for high school graduation.
It is imperative that the frameworks do not interfere with the in-depth learning of specific subjects, including subjects not contained in the frameworks. Modern society needs not only a lot of common knowledge possessed by all individuals, but more importantly, it needs different, unique skills and talents possessed by different individuals. Diversity is necessary for advancing and creating new endeavors. In other words, the frameworks should not specify all that is taught in schools, but only the essential. Regrettably, the pressure of getting high average scores forces schools systems to teach almost exclusively the frameworks, abandoning other successful programs.
Section 1I of Chapter 69 (Education Reform Act) empowers the Board of Education to develop a system for evaluating schools to determine whether they succeed in improving student performance, and to develop a diagnostic assessment of individual students. To fulfill this obligation, the Board has developed the Massachusetts Comprehensive Assessment System - the MCAS.
For a general description of the MCAS, how it is administered, and what it looks like, go to What's MCAS. This section deals with the controversies surrounding the MCAS.
The first controversy is associated with "a variety of assessment instruments". The critics of the test point out that the law requires that student performance be measured not by a single pencil-and-paper test, but by a variety of assessment instruments that include work samples, projects and portfolios. The precise words that support that point of view are in the second paragraph of Section 1I:
"The system shall be designed both to measure outcomes and results regarding student performance, and to improve the effectiveness of curriculum and instruction. In its design and application, the system shall strike a balance among considerations of accuracy, fairness, expense and administration. The system shall employ a variety of assessment instruments on either a comprehensive or statistically valid sampling basis. Such instruments shall be criterion referenced, assessing whether students are meeting the academic standards described in this chapter. As much as is practicable, especially in the case of students whose performance is difficult to assess using conventional methods, such instruments shall include consideration of work samples, projects and portfolios, and shall facilitate authentic and direct gauges of student performance."
The third paragraph of Section 1I extends the use of the test from merely measuring effectiveness of curriculum and instruction to being a diagnostic instrument for teachers, parents, administrators, and students themselves.
"In addition, comprehensive diagnostic assessment of individual students shall be conducted at least in the fourth, eighth and tenth grades. Said diagnostic assessments shall identify academic achievement levels of all students in order to inform teachers, parents, administrators and the students themselves, as to individual academic performance. The board shall develop procedures for updating, improving or refining the assessment system."
The critics argue that the MCAS does not fulfill the diagnostic requirement because the tests are not returned to the teachers or students, so it is impossible to evaluate the details of student’s answers (or if mistakes were made in the grading). All that is returned are scores with some breakdowns. The scores come back six months after the test, when students are in the next grade with usually a different teacher.
Further down, Section 1I imposes additional requirements, including appropriate treatment of students with special needs.
"The assessment instruments shall be designed to avoid gender, cultural, ethnic or racial stereotypes and shall recognize sensitivity to different learning styles and impediments to learning. The system shall take into account on a nondiscriminatory basis the cultural and language diversity of students in the commonwealth and the particular circumstances of students with special needs."
How does the MCAS recognize sensitivity to different learning styles or impediments to learning? The particular circumstances of students with special needs are supposed to be taken care of by special accommodations during the test, but the Board of Education has been criticized for not allowing adequate accommodations (see Special-Ed students).
Currently, the MCAS is used for not only assessing student and school performance, but also as a graduation requirement. This dual purpose is not consistent with the law (see Graduation Requirement). The scope of the MCAS should be restricted to just the assessment of the progress of the implementation of the frameworks. The scope may also include the quantification of the average academic level. But the MCAS should not be used in determining the minimum competency as a requirement for high school graduation. This should be a separate test, a test of just the basic skills. (Some people believe strongly that no single test should be used to determine whether a student graduates from high school, but the Education Reform Act requires such a test.)
Critics of the MCAS also object to the simultaneous use of the MCAS as both a measure of the compliance of schools with the frameworks, and as an indicator of the individual student performance. They argue that if the MCAS is forward-looking, with the intention of identifying schools which are behind, then the results of such a test should not to judge the performance of individual students. In other words, if the test is meant to determine the areas which are not being taught adequately, then the students should not be evaluated by a test that includes these areas (more).
Further problems with the MCAS are not related to the Education Reform Act. Among these, the one that is voiced most often is the duration of the test. It is longer that the Massachusetts Bar Exam. Although it only takes half a day each day, given the pressure and intensity of the test, nothing else gets done for the duration of almost two weeks during which the test is administered. That's 6% of the total time spent in school in a year.
The test is sometimes criticized for confusing or tricky questions. This is particularly evident in the lower grades in which the students do not have much experience in taking such tests. In informal studies, the same students who did poorly in the 4th grade test, did much better after the questions were read to them. There are a number of anecdotal stories, such as the 4th grade question about a heroine of a reading passage. The word heroine was not used in the passage, but the class had just finished a drug abuse awareness session, and the students were stumped because the passage did not contain anything about heroin addiction (more)
In conclusion, there is nothing in the Education Reform Act that resembles the MCAS. This in part is the reason why the MCAS has become so controversial.
Section 1D of Chapter 69 (Education Reform Act) empowers the Board of Education to establish statewide education goals, as well as criteria for the following three certificates for graduation.
(i) The "competency determination" shall be based on the academic standards and curriculum frameworks for tenth graders in the areas of mathematics, science and technology, history and social science, foreign languages, and English, and shall represent a determination that a particular student has demonstrated mastery of a common core of skills, competencies and knowledge in these areas, as measured by the assessment instruments described in section one I. Satisfaction of the requirements of the competency determination shall be a condition for high school graduation.
(ii) The "certificate of mastery" shall be based upon a determination that the recipient has demonstrated mastery of a comprehensive body of skills, competencies and knowledge comparable to that possessed by accomplished graduates of high school or equivalent programs in the most advanced education systems in the world. The criteria for a certificate of mastery may incorporate a number of factors which may include, but not be limited to, any of the following: high school graduation standards, superior performance on advanced placement tests administered by the educational testing service, and demonstrated excellence in areas not reflected by the state's assessment instruments, such as artistic or literary achievement.
(iii) The "certificate of occupational proficiency" shall be awarded to students who successfully complete a comprehensive education and training program in a particular trade or professional skill area and shall reflect a determination that the recipient has demonstrated mastery of a core of skills, competencies and knowledge comparable to that possessed by students of equivalent age entering the particular trade or profession from the most educationally advanced education systems in the world. No student may receive said certificate of occupational proficiency without also having acquired a competency determination.
Section 1D also provides for multiple opportunities to pass the test as well as eligibility to receive additional educational assistance.
Please note that the certificate of occupational proficiency requires a competency determination and is therefore of higher rank. The certificate of mastery has the highest rank for superior performance comparable to the most accomplished graduates in the most advanced systems in the world. Interestingly, it may be based on artistic or literary achievement not reflected by the state’s current assessment system.
How all this relates to the MCAS and its use as a graduation requirement? How do we translate the separate scores for the individual subjects into the above three certificates? The MCAS scores are divided into four levels: advanced, proficient, needs improvement, and failed.
Shortly after the MCAS was introduced, its supporters argued that the MCAS graduation requirement is necessary to ensure that all high school graduates posses the basic knowledge and essential skills needed in the modern world. However, it is hard to rationalize that the knowledge of matrix operations or Egypt history represents basic knowledge or essential skills. Such claims are now less frequently made. Instead, it is widely acknowledged that the MCAS is a difficult and advanced test, and, as many people point out, is aimed at a college-preparation level, not basic competency.
To accommodate students whose aspirations may not include college, the Board of Education set the passing threshold at a lower level that requires only a fraction of the questions to be answered correctly. However, the advanced questions are nevertheless intimidating and attempts to answer these questions may lead to a lower score. To address this concern, the Board of Education has implemented special MCAS tests for students who already failed the test at least once. These special tests do not contain the advanced questions, but supposedly are still graded with the same rubrics, so the difficulty of the test is supposedly unchanged. (This approach brings forth the next concern: what about students who are interested in specific topics and would do well on some of the advanced questions, but are no longer given this opportunity?)
Another change recently adopted by the Board of Education is to require a passing grade on only the English and Math sections of the test, and not the other subjects. This new rule is in conflict with paragraph (i) above and is therefore of questionable legality.
As was pointed out earlier, the law requires (as a condition for graduation) a determination of competency in a common core of skills. From the text above one may assume that such a determination is ranked below the occupational proficiency for a particular trade or professional skill. The MCAS, on the other hand, covers the entire frameworks, a scope that is much broader than the common core of skills, and is therefore not what the authors of the Education Reform Act had in mind.
The MCAS should be separate from the competency determination. The latter should either be a simple test or some other assessment method that indeed covers the common core of skills as was generally described by the Commission on the Common Core of Learning in 1994.
Perhaps the most disturbing aspects of using the MCAS as a graduation requirement is that a large fraction of students is expected to fail (more). This is happening in an educational system that, although not perfect, is considered one of the best.
The high failure rate has many causes. In some cases, the material covered by the test has not been taught. (A report Up and Over the Bar by Mass Insight, an organization devoted to promoting the MCAS, indicates that at least in the case of math, a significant number of failures were caused by a "curriculum lag"). In other cases, students either lack confidence or simply don’t care. Still, it is quite likely that the "bar" is set too high and the material tested includes too many topics that are irrelevant to many students or should not be considered a part of the common core of skills.
To fulfill the obligations spelled in the Education Reform Act, the Board of Education should abandon the MCAS as a graduation requirement and replace it with a competency determination based on a variety of assessment instruments (as the Board was directed to do in Section 1I), that may include, but not be exclusively based on a much shorter and simpler test (more).
In conclusion, it is important to point out that some people strongly believe that high school graduation should not be based on a single pencil-and-paper test. The outcome of 12 years of learning cannot be invalidated by a test lasting only a few hours, graded by contractors that may not be adequately qualified. They believe that it is virtually impossible to design a test that would be fair to everyone.
Section 1D of Chapter 69 (Education Reform Act) empowers the Board of Education to establish statewide education goals.
"The board shall direct the commissioner to institute a process to develop academic standards for the core subjects of mathematics, science and technology, history and social science, English, foreign languages and the arts. The standards shall cover grades kindergarten through twelve and shall clearly set forth the skills, competencies and knowledge expected to be possessed by all students at the conclusion of individual grades or clusters of grades. …. The skills, competencies and knowledge set forth in the standards shall be expressed in terms which lend themselves to objective measurement, define the performance outcomes expected of both students directly entering the workforce and of students pursuing higher education, and facilitate comparisons with students of other states and other nations."
The second and third paragraphs of Section 1D enumerate several specific areas that should be incorporated in the education goals. The rest of Section 1D deals with high school graduation requirements.
The third paragraph contains the following key sentence.
"The academic standards shall reflect sensitivity to different learning styles and impediments to learning."
This sentence clearly indicates that the academic standards must not be a single one-size-fits-all standard, but that it should be a set of standards that are different for different students. Unfortunately, this is not the interpretation advanced by the Board of Education. In their view there is one standard represented by one MCAS.
The problem with a one-size-fits-all standard is that it ignores diversity among students, and that diversity is very important in modern society. Let us consider just one aspect of diversity – varying ability in the academic areas covered by the MCAS (and that some do better in written tests than others). For some students, the MCAS is easy; for others it is very difficult. Worse yet, a single student may find some subjects not challenging and other subjects overwhelming.
It might be helpful to consider academic standards as made up of two components, one representing what society expects everyone to know, and the other reflecting individual accomplishments based on one’s talents and abilities. The first one might correspond to the competency determination required for graduation. The second might be viewed as the individualized educational challenge. This challenge should be equally difficult to all students, the strongest and weakest. It means that students with mathematical abilities need standards that let them excel in that subject, while students talented in arts or music need different standards. Vocational students should not be required to demonstrate the same scholastic proficiency as those heading to Ivy League colleges, while those going to Ivy League colleges need not prove skills in vocational crafts. Uniform standards is not what modern society needs; it needs individuals with many different talents and skills, all trying their best in utilizing those talents and skills.
Having different standards or expectations for different students does not stand in a way of pursuing higher standards. In fact, challenging all students is better than making it easy for some, and too difficult for others (more).
The use of a common curriculum, or frameworks, as means of advancing academic standards is a reasonable first step if the frameworks are not so extensive that they completely dominate what is being taught, without giving the students an opportunity to develop their own interests and talents. Unfortunately, the MCAS and the frameworks have become such a major concern to all schools that too much is now being sacrificed for the sake of higher scores. This issue goes beyond ensuring that kids acquire the basic skills. It affects our best students who may not get the opportunities to learn beyond the frameworks. The frameworks do not make a distinction between students with different interests or abilities. It covers material that all students must learn. In the extreme, if almost all of the teaching goes towards the frameworks, then eventually all graduates will posses roughly the same (limited) knowledge. This might be desirable for low-end clerical work in a large corporation, but not too good for a society that needs inventors, leaders, creators, etc.
In short, the Board of Education's desire to improve academic standards may have the opposite effect of actually lowering them.
Officially, the frameworks include three more subjects not covered by the MCAS, the arts, the health, and foreign languages, as well as the two that are no longer required for graduation – science and history. The implementation of the frameworks on these other subjects is slower. School administrators allocate more resources to the two subjects that count, in hope of maximizing the average MCAS scores.
For example, the science homework that students now get, often resembles an English essay assignment, with more emphasis on writing than science. Although the ability to communicate scientific concepts is a desirable objective, many think that this has gone too far, and is a simple consequence that science is no longer a graduation requirement, and that science scores are seldom reported in newspapers. Some even argue that science teaching has become more a teaching of definitions than deep understanding.
The association between the frameworks and the academic standards is spelled out in Section 1E. It states that frameworks represent "broad pedagogical approaches and strategies for assisting students in the development of the skills, competencies and knowledge called for by these (academic) standards." The current frameworks are more a detailed enumeration of topics to be taught than broad pedagogical approaches.
The association of the MCAS with academic standards is often describes as "raising the bar". But how high should we raise the bar? Many people believe that the bar has been set too high. This is especially evident in the 4'th grade English test. That test has been evaluated by a number of educators who concluded that it as far above the 4'th grade level.
The official view of the academic standards is that they are represented by the MCAS and therefore anything we do to improve the MCAS scores is considered equivalent to pursuing higher standards. We believe that the overemphasis on the MCAS may actually lower the academic standard, even if it improves the ability to write. A better approach is to have different standards for different students, so all challenged to do their best.
Section 1J of Chapter 69 (Education Reform Act) empowers the Board of Education to establish regulations defining under-performing schools. If a school is determined to be under-performing, the commissioner appoints a fact-finding team that reports within 90 days. The school must present a two-year improvement plan, subject to approval and modification by the Board. If after two years the school has not made adequate progress, it is found chronically under-performing. In that case the Board may replace the principal with an individual having extraordinary powers to dismiss teachers, increase salaries, or secure extra funds (subject to miscellaneous constraints).
Section 1K of Chapter 69 empowers the Board to act if a school district is found to be chronically under-performing. It lets the Board designate a receiver with all the power of a superintendent and school committee. It also gives it the power to secure the necessary funds from the community if the community has not provided adequate funding to the school district.
The latest regulations related to school accountability (i.e. Department of Education's interpretation of the law) can be found at www.doe.mass.edu/ata/.
According to these regulations, the determination of whether a school or district is under-performing is based solely on the average MCAS scores. Other factors are not taken into account until a fact-finding team is sent. This narrow view of school performance ignores the reality of socioeconomic factors. If these factors are taken into account, a school with a low average, but located in an inner-city, may be performing much better given their situation than a school with a higher average located in a wealthy community. Globe article 20-Jan-2001.
Basing accountability on only the average MCAS score is also inconsistent with the wording of Section 1J.
"Such regulations shall be consistent with the goals and standards adopted by the board and the basis for the determination of chronic failure shall include, but not be limited to, the evaluations performed pursuant to section one I. The regulations adopted by the board shall take into account the turnover of students in particular schools and districts."
Section 1I is the one that contains the following sentences.
"The assessment instruments shall be designed to avoid gender, cultural, ethnic or racial stereotypes and shall recognize sensitivity to different learning styles and impediments to learning. The system shall take into account on a nondiscriminatory basis the cultural and language diversity of students in the commonwealth and the particular circumstances of students with special needs."
Another problem with the use of just the MCAS scores is the likelihood of year-to-year statistical fluctuations. Consider a small elementary school with only 30 kids in the 4th grade. Just one special-ed child with a very low score can affect the average significantly. A small school will see its average score vary a lot more than a large school.
Some schools have a significant number of students that boycott the test. Most of these students are at the top of their class, but they all receive the failing score of 200. The Department of Education averages their scores with the scores of the rest of the students, purposely distorting the averages for those schools.
Other states, including Texas, Louisiana, Florida, Maryland, and North Caroline consider drop-out rates and attendance in their school rating. (There is a significant controversy about how the drop-out rates are actually measured.)
The simplistic method of accountability based on just one number should be replaced with accountability analogous to the one used for charter schools. It should be based on indicators that are reported by the schools to their communities. Board of Education should be responsible for specifying these indicators in accordance with the multiple goals stated in the Education Reform Act, not just the MCAS scores. We have a model of how accountability should be implemented and we should use this model.
The education reform involves a significant amount of money that the school district receive from the state, and therefore, as taxpayers, we should demand proper accountability. The state's insistence on using just the MCAS is not acceptable. Just one number would not satisfy a state auditor and should not be sufficient for the general public. We must be informed how the money spent is advancing all of the goals of the education reform (more).
To minimize the possibility of a conflict of interest, accountability should be separate from the Department of Education. We don’t want the same organization that is in charge of implementing the reform be also in charge of determining whether the reform is being implemented in a fiscally prudent manner. The Board of Education has difficulties with misspending its own budget on consultants (more), so why should we expect that it will do better with the statewide funds.
Some politicians want to reduce financial assistance to schools that do not improve within a certain time. This makes as much sense as eliminating funding for drug enforcement agencies that do not succeed in reducing the use of illegal drugs.
As taxpayers, we should demand accountability that goes beyond the MCAS. We should emphasize that the MCAS is too controversial and not considered a reliable measure of school performance. We should demand accountability that reflects all of the goals of the education reform. We should make it clear that our criticism of the MCAS is motivated by the pursuit of better accountability and that the accountability provided by the state (based on only the MCAS) is inadequate. We should demand accountability not only from the school districts but also from the Board of Education. Finally, the financial aspects of accountability should be handled by state auditors and not the Board of Education.
Education reform was the outcome of a court decision that was to ensure equitable education for kids in poor communities.
Supporters of the MCAS claim that the MCAS does indeed help those kids because it provides them with diplomas that mean something, diplomas that guarantee that the kids do have the essential skills to meet the demanding criteria of today’s work force. The MCAS halts the practice of meaningless promotions from grade to grade and ensures that everyone learns the required material in each grade. The pressure on schools and school districts to get good average scores will whip these schools and districts into shape.
The critics argue that the MCAS by itself does not improve education and that in fact the MCAS is counterproductive. We know that inner-city schools have a lot of problems, and we don't need the MCAS to reveal that. Instead of trying to fix these problems first, the state hopes that improvements will come as a consequence of low MCAS scores. To improve the scores, the state recommends after-school classes, remediation programs, summer sessions, and volunteer tutoring, implying that the primary cause of low scores is indolence. If the students just worked harder, all would pass the MCAS.
Such views are unrealistic. To improve education in the inner-city schools, these schools must have clean and safe facilities, adequate supplies, small classes, creative programs, and qualified and dedicated teachers. These things must be roughly equivalent to those in the richest communities. Before we start failing the inner-city kids en masse, let us have parity between schools first (the main goals of education reform), and then see how many of the inner-city kids are truly lazy or really don't care about education.
The state does not intend to wait until there is equity before imposing a graduation requirement based on the MCAS. It plans to deny diplomas to students who may not be able to overcome the inequities in their education. In essence, the state expects the students to compensate for the inequities, thus transferring the responsibility for achieving the equity from the state to the students.
How does a test that flunks most of the inner-city students help these students? Does it encourage them to try harder or does it demoralize them? Does it motivate them to "reach for the bar" or does it lead to resignations and dropouts? Can we convince these students to spend more time on school work when we know that there are not enough textbooks for them to take home? Will they participate in tutoring by volunteers who aren't even teachers, to make up the time wasted in over-sized classes during regular hours? Why do we have a test that underscores their bad situation instead of finding ways of stimulating and inspiring them?
The MCAS covers material that the schools may not yet have had a chance to teach. The inner-city schools require more time to align with the frameworks, so the Board of Education should verify first that the frameworks are well in place before administering the test. Is it fair to deny graduation to a student who learned material that was taught in his school, but failed the test on material that his school did not teach?
Section 1 of Chapter 69 (Education Reform Act), the introductory section that defines the four primary goals of public education, lists the first goal as providing conditions for learning that are meaningful, enjoyable, and without threatening the sense of security or self-esteem.
"It is hereby declared to be a paramount goal of the commonwealth to provide a public education system of sufficient quality to extend to all children the opportunity to reach their full potential and to lead lives as participants in the political and social life of the commonwealth and as contributors to its economy. It is therefore the intent of this title to ensure: (1) that each public school classroom provides the conditions for all pupils to engage fully in learning as an inherently meaningful and enjoyable activity without threats to their sense of security or self-esteem, … "
It is the sense of security or self-esteem that many fear is being ignored when administering such an overwhelming, one-size-fits all test. Teachers who work with the inner-city kids see these kids as having to deal with many problems that go beyond the academics. To these kids, the MCAS is at best irrelevant, and more often is viewed as just another insurmountable challenge. The MCAS punishes these kids for the failures of their environment and their schools. It definitely does not promote learning as an enjoyable activity without threats to the sense of self-esteem (more).
A rare student who indeed has not learned the essential skills should be denied the diploma. Similarly, we should not tolerate the practice of undeserved promotions (passing students to the next grade regardless of whether they have learned the required material). But the means of accomplishing these objectives should be a lot more intelligent, than a test that flunks most of the students. The MCAS harms not helps the inner-city kids.
The true success of education reform in the inner-cities will be measured by lower absenteeism, fewer dropouts, more parental and community involvement, smaller classes, cleaner and safer schools, better equipment, and most importantly, how well do the graduates do after they leaves the school.
Most people who have not dealt with children having learning disabilities do not appreciate the difficulties these children face each day in school.
Many of these children are highly intelligent and talented, but often they cannot cope with simple reading or writing tasks. It is not unusual that a child in a fifth grade may read or write at a second grade level, yet the same child two or three years later can become a devout reader and exceed his peers in comprehension and appreciation. Albert Einstein and Winston Churchill had learning disabilities and did poorly in school.
The problem is that the methods of teaching that are appropriate for most children are not appropriate for children with learning disabilities. For example, many require a phonetic-based learning. Some need a quiet surrounding or the help of a computer. There are many different types of disabilities, so each child requires a different approach.
One of the great strengths of the American education system that is particularly true in Massachusetts is that it does provide assistance to students with learning disabilities. Many of these special-ed students grow up to become leaders, instead of being lost to society. Our system nurtures and encourages them to do the best they can, instead of treating them as dumb or incompetent. This is one reason why America ends up with a greater number of creative entrepreneurs and artists than other countries.
The MCAS – a one-size-fits-all test – does not measure the true abilities that children with learning disabilities may have. Even with special accommodations, the scores do not reflect what the teachers know are the child’s real capabilities. Instead, the low scores undermine these children’s fragile confidence. These children need encouragement and sensitivity, not a state-sponsored instrument that underscores their disabilities.
To appreciate what the MCAS does to special-ed students, please read When Bad Things Happen to Good Children.
The current requirement is that all students, including the special-ed students must take the MCAS. The rationale for including the special-ed students is to discourage the practice of designating too many students as special-ed student for the purpose of improving school’s average scores. For the same reason, the special accommodations were initially very limited. More recently, additional accommodations are being permitted, although this topic is still very controversial. Federal law stipulates that special-ed students be included in standardized tests, but does allow accommodations or alternative assessments.
The MCAS is promoted as a useful diagnostic tool that accurately reflects student performance. Yet there are enough concerns that warrant further investigation of this broad claim. Undoubtedly, the test correlates with student’s knowledge and may indicate the weak areas, but how useful a diagnostic tool is it and is it really accurate?
We shall not concern ourselves here with the fact that the MCAS does not necessarily measure ability (i.e. student is smart but has not learned much), nor does it reflect other qualities important in life, such as creativity, diligence, etc. The test is meant to measure knowledge specified by the frameworks and let’s just discuss that.
The foremost concern of test’s critics (that deals with accuracy) is the lack of a rigorous validation process that would normally be expected for such an important test. What is being done is the mere minimum dealing with the distribution of weights and the year-to-year consistency, as described in the Technical Report Summary or the Full Technical Report. What is missing is any type of study that can relate the MCAS scores to subsequent academic performance in college, or to professional success, or to general success in the modern economy. After all, we are being told that the MCAS is necessary to ensure that high school graduates possess the skills that they will need in the modern world. It is regrettable that the state does not intend to verify this claim.
One may argue that since the MCAS measures the knowledge embodied in the frameworks and since we designed the frameworks to represent the necessary skills, we do not need any further validation. If this is the official attitude then this point should be made clear in official proclamations; attaching broader interpretations to the MCAS only irks the opponents. On the other hand, such an argument does not really resolve the key issue; instead of correlating the MCAS to future success, we have to correlate the frameworks to future success.
The next major concern regarding the MCAS accuracy is the reliability of the grading process. Grading of the multiple-choice questions is straightforward, but the MCAS contains a number of "open response" questions and essays. These open response questions and essays count for half of the grade. Currently, the 6 million open response questions are graded by 800 out-of-state contractors, half of whom do not have any teaching experience and a third do not even have a college degree. They undergo a brief training and go to work. Are these people qualified to decide whether a student may graduate? More importantly, what is the probability that a given student may have had the bad luck of being poorly graded on most of his open response questions? Even if the probability is small, when it is multiplied by the number of students taking the MCAS, we may end up with thousands of students in that category. The concern is that graders are not adequately qualified and the probability of flawed scores is too high.
Neither the students nor the teachers get the open-response questions back. All they get are the scores. There is therefore no way of checking the grading, or learning precisely what the student should concentrate on to do better next time. In that way, the MCAS is seriously lacking as a diagnostic tool. (The writing sample is graded with several indices that do provide some additional information, but are nevertheless the interpretations of the individual grader.)
The final scores are based on complicated formula that themselves are subject to errors that neither the individual student, nor the schools, nor even the entire district can validate. Such errors do happen, and they harm students and destroy careers, as was recently documented by New York Times (more).
To better appreciate the difficulties in grading of the open response questions, please look at the analysis of question 13 in 8'th grade 2000 science test. This question has been selected by the DOE to illustrate how the grades are assigned. However, a closer examination reveals a confusing question and inconsistency in the assigned grades. The student who understands the subject better may have gotten the lower score.
The test is also criticized for confusing questions or cultural bias. Consider, for example, question 12 in 4'th grade math test. This questions asks the child to count the number of stacked blocks. Most people will answer this question without hesitation, getting the officially correct answer. However, a dyslexic child whose geometric perception is very different from most people, may immediately wonder about the blocks that are hidden behind the visible blocks. The correct answer is that there is no way of knowing how many blocks there are because some of them may not be visible and the officially correct answer is wrong.
Consider another question about relative motion of two side-by-side trains (question 27 in 4'th grade science test). Anyone who has been traveling on trains has experienced the false perception of motion when the other train is moving. Such a person will immediately recognize the situation and answer this question correctly. But what about a nine-year-old child who has never been on trains? He may find this question very perplexing.
The fact that such questions exist in the MCAS is a clear indication that those who designed the test and who reviewed the test either did not have the necessary know-how or did not spend enough time doing their job.
In conclusion, the accuracy of the test is problematic and the state officials should devote more resources to ensure its accuracy.
Is there a real crisis in public education or has the crisis been invented by politicians looking for a cause? We all know that there are problems, but can these problems be classified as a crisis? The original goal of the education reform was equity in education – that everyone has equal access to quality education. This was later augmented with the desire to improve the academic standards, to raise the bar. Next came the MCAS, with a bar set so high that a large fraction of students is expected to fail and not graduate. Now, with so many students failing we indeed have a crisis. But is it a real crisis or a fabricated crisis?
To substantiate the claim of a crisis, the Board of Education points to the high failure rate of the MCAS. To demonstrate that the MCAS is a reasonable (not unrealistically difficult) test, it publishes a few questions that most adults can easily answer, then shows the high percentage of students that answered these questions incorrectly. Indeed it is sad that our education system did not prepare the students adequately to answer these questions. But a small number of selected questions may not be sufficiently convincing. These questions may have been at the very end of a grueling test, when tired students became careless. On the other hand, these kids find jobs after graduation. We have an economy with near-full employment and a labor force that is considered highly skilled. There is no proof that students now graduate with less knowledge than did previous generations.
Another "evidence" of the crisis is the fact that American students score lower in international math competitions than other industrialized countries. This may be so, but then why are the educators in these other countries trying to emulate the American system? Perhaps they are less concerned with the math scores and more about promoting creativity, originality and flexibility. They want to replace their rigid methods that tend to reward conformity and obedience with less structured methods that are more likely to produce inventors, entrepreneurs, and civic leaders (and lower math scores).
Not everyone is therefore swallowing the crisis hype. It seems that people who are furthest away from the education system are the most likely to believe that there is a crisis. Parents who are not involved in schools are next. Parents who actively participate in their children’s education do not in general think that there is a crisis and educators and administrators feel like scapegoats of a non-existing nemesis.
Perhaps the education crisis has been fabricated by politicians who need a cause when the usual causes of bad economy or external enemy are not available. This may be a cynical viewpoint, but there is some evidence that justify this viewpoint. For example, the current make up of the Board of Education does not include one person who has K-12 teaching experience (The Commissioner, Dave Driscoll is a former teacher, but is not an official member of the Board). The Board is appointed by the Governor who has repeatedly proven herself (as well as her predecessor) to be a champion of the crisis theory.
The politicians who talk of the crisis are supported by a few business "leaders". These leaders created an organization Mass Insight whose objective is to promote the MCAS as the means of reaching higher academic standards. The underlying thesis is that high school graduates entering the labor market do not possess the necessary skills and to remedy this we must have the MCAS.
The business leaders who are associated with Mass Insight are also associated with a few very large business organizations. They do not represent a "typical" businessman. Their support of the MCAS may be motivated more by maintaining a good relationship with the Governor than a genuine concern about education. One does not have to be an English scholar to be successful in business, as is often demonstrated by the grammatically incorrect memos written by business executives. On the other hand, our best writers, philosophers and scientists often cannot balance their own checkbooks. The scientific geniuses, as well as the most successful entrepreneurs, are usually misfits who do not do well in school.
Mass Insight is the organization that publishes the glossy brochures that are distributed in schools for parents. These brochures exalt the MCAS. The state "pays" for these brochures, but these brochures are not officially from the Department of Education. Why a particular private (albeit nonprofit) organization is allowed to distribute material through schools? Many consider this material to be political propaganda.
An even more cynical viewpoint is that a group of politicians and businessmen are trying to undermine the public education system, so that the public will eventually become convinced that privatization of education through school vouchers or other means is necessary. The MCAS is needed to demonstrate that there is an education crisis – a crisis that the public education system cannot overcome. This is also the rationale for accountability systems that is based strictly on the average MCAS scores, as well as ideas that salaries and/or funding should be tied to these average scores. Even President Bush suggested removal of funding from school systems that do not improve test scores. This concept is as convincing as the concept of removing funds from drug enforcement agencies that do not reduce the use of illegal drugs in their communities. The removed funds are to be transferred directly to students (or drug users) so they can chose the best ways of improving their education (or drug treatment). (President Bush has recently been moving away from this idea.)
The cynical viewpoint is supported by the fact that most of the members of the Board of Education are directly or indirectly associated with the Pioneer Institute, a conservative think tank whose objective is to "drive broad-based educational reform through the expansion of competition and parental choice".
Critics of the MCAS do not believe that there is an education crisis. Many of them are concerned about the involvement of Mass Insight and Pioneer Institute in the education reform (more).