MSPAP graders see inconsistent scoring

Quantity, not quality often is the focus, some teachers say

February 01, 2002|By Stephanie Desmon | Stephanie Desmon,SUN STAFF

SILVER SPRING - Some Montgomery County teachers who graded recent MSPAP exams liken the experience to working in an academic sweatshop - one in which accuracy and fairness are far less important than racing through large numbers of tests at a pace too fast for much more than skimming.

They say the assembly-line attitude and rampant inconsistencies are a distressing disconnect from the serious consequences for Maryland schools and schoolchildren on the annual high-stakes state tests.

"I thought it would be scored in a fair way," said Shelly Turi, who teaches language arts at Oak View Elementary School in Silver Spring and scored tests in the summer of 2000, "but that was a wrong impression."

Previously, few MSPAP scorers - Maryland teachers working in the summer months - have spoken out about what goes on behind the closed doors of the state test scoring centers.

But in trying to get to the bottom of Montgomery County's and the state's overall drop in test scores, county school officials - the first in the state to publicly criticize the decade-old testing program - have also been bringing together similar groups of former test graders to discuss scoring problems.

Yesterday, the scorers recalled an assembly line, in which quality was not the main focus.

Myrna Schwadron, a third-grade teacher at Oak View, spoke of scoring tests in a room with a big chart that graphed her scoring group's progress toward the goal of grading a total of 22,000 tests - and not so subtle prodding of those who didn't keep up.

Another third-grade teacher, Martha G. Stevens from Oak View, said scorers in her group were awarded giant chocolate bars for plowing through more than 100 test booklets in a 6 1/2 -hour workday. "You're glancing, and you keep going," she said of trying to keep up with the pace of grading.

In all, more than 700 Maryland teachers last summer scored Maryland School Performance Assessment Program tests taken by 200,000 third-, fifth- and eighth-graders across the state in just five weeks, including initial training.

Testing officials and other teachers who've scored the tests say that accuracy and consistency, the hallmark of this assessment program, they say, are maintained in the process.

"If the average person needs to read approximately 55 books a day, then each person is expected to do a reasonable amount close to that," said Jo Davidson, who oversees MSPAP scoring for Measurement Inc., a Durham, N.C.,-based firm that has a $2 million a year contract with the state to score the tests. She said the claim that anyone was pushed to read 100 sets of answers a day was "ridiculous."

But, she added, "If somebody's being paid the same amount of money and they're only reading 10 or 20 books a day, they're not being a very good employee. We have a deadline to meet."

Officials said they go out of their way to assure the validity of the scores. Although each answer is read just once, each test booklet is read by four teachers who read separate sections. Scorers are tested and retested to make sure they grade consistent with state standards.

Grading MSPAP tests is complicated because they are not multiple choice exams, the answers to which can be run through a scanner. Pupils are sometimes required to write lengthy essays across six subject areas.

To decide whether the answer is correct, teachers are supposed to go through extensive training to learn the difference between, say, a zero and a high score of a three. They are given examples of poor, good and excellent answers, which show the varying elements that warrant different scores.

"It's difficult to predict what a student is going to do with a particular question because these are not 2 + 2 = 4 kind of items," Davidson said. "We try to take as much of that subjectivity out of it - that's our goal."

Raymond J. Graleski Jr., a math teacher at Chesapeake High School in Pasadena who has been scoring MSPAP exams for nine years, said the test has become more reading intensive and therefore more difficult to score. "We do our best to make sure the scores are as consistent and accurate as possible," he said. "But it's not going to be perfect."

But the Montgomery teachers also alleged that grading standards would sometimes change in the middle of scoring without any effort to go back and regrade tests that had been processed.

Mark Moody, assistant state superintendent who oversees testing, called that assertion "patently untrue." He said the state went back and rescored 1,500 papers on a questionable math question in 2001. Others are "tweaked" in ways that don't affect overall scores, he said.

Last summer, Maryland teachers were paid $10.50 an hour by Measurement Inc. for grading MSPAP tests. Some jurisdictions supplemented the salary to get enough scorers, with Montgomery teachers taking home $20 an hour. Still, some scorers complained that their colleagues missed work or quit early, leaving them short-handed - and with even more papers to grade.

Baltimore Sun Articles
Please note the green-lined linked article text has been applied commercially without any involvement from our newsroom editors, reporters or any other editorial staff.