New Abell study takes balanced MSPAP view

The Education Beat

Critique: Panel finds tests well-conceived - but raises some important questions.

October 03, 2001|By Mike Bowler | Mike Bowler,SUN STAFF

THE ABELL Foundation has dropped the other shoe on the Maryland School Performance Assessment Program, and it doesn't pinch quite as badly.

A couple of years ago, Abell commissioned two national panels to look at MSPAP, one at its content, the other at its technical qualities -- the validity of its questions, how it's administered and scored, whether MSPAP results can be compared with those of other tests and so on.

The technical report, called a psychometric review, deserved as much attention as the content report, but it didn't work out that way. A summary of the content report became public, and its thorough condemnation of MSPAP sparked more than a year of discussion and debate (even though few have read the document, kept under lock and key at the state Education Department because it discloses test items that might be used again).

The technical panel, headed by Ronald K. Hambleton, a professor at the University of Massachusetts and an authority on testing, came out much later with its report. It was lost in the shuffle, much to the dismay of Hambleton and his fellow authors. But last Friday Abell posted the report on its Web site (

Hambleton's panel takes a much more balanced and less ideological view of MSPAP, although the two groups come to some of the same conclusions and in general call for more objective testing, including the addition of multiple-choice questions.

Multiple choice doesn't have to be simple regurgitation of facts, the report says. It can be designed to measure higher-order reasoning. (The agreement of the two groups is remarkable, given that they met only once.)

Parents and critics of MSPAP will find in the new report recommendations that will please them -- and others that won't. In all, the experts find Maryland's testing program to be well-conceived and well-executed. They praise the state for its openness in making MSPAP data public and for the "clarity in its reports."

The Hambleton group recommends against releasing individual MSPAP scores -- unless the state makes major changes in the test. As it is, individual scores are meaningless, the panel says.

For immediate action, the report suggests doing away with the group discussions that precede many of the MSPAP tasks and eliminating "manipulatives" -- blocks and straws and coins and the like -- that are a part of many MSPAP tasks. Group work and manipulatives, the report says, make it more difficult to administer MSPAP in a "fair, consistent and individual manner."

The Hambleton panel waxes philosophic on some of the questions that surround MSPAP. For example, can it be an assessment tool and a model for classroom instruction at the same time? Not really, says the report. "It seems impossible for an assessment system to achieve both purposes."

And does MSPAP really measure higher-level thinking skills? The panel, agreeing with the experts who looked at MSPAP's content, says it found no evidence that it does. And it also agrees with the content panel that MSPAP's heavy reliance on writing might be distorting scores.

Finally, the panel recommends against expanding MSPAP to other grades, a possibility raised this week by a national education reform group that is studying Maryland's schools. Grades three, five and eight are fine, it says. Adding more grades would bring costs to "almost impossible levels."

Robert C. Embry Jr., president of the Abell Foundation, said he publicized the technical panel report because a number of people had copies of it, "and there are no security problems with it. I've said all along that no test is perfect, and nobody's evil here. But there are important policy issues that should be discussed in public."

A spokesman for state schools Superintendent Nancy S. Grasmick said she agreed with Embry's decision to make the technical report public. She and her staff have discussed the report, which will play a major role in the deliberation of the Visionary Panel for Better Schools, a 40-member body charged with drawing up a blueprint for Maryland public education over the next 10 years and that is leaning toward recommending the development of Maryland's first statewide curriculum.

Baltimore Sun Articles
Please note the green-lined linked article text has been applied commercially without any involvement from our newsroom editors, reporters or any other editorial staff.