Exam Scoring Gaffe Leaves Evaluators Sorely Tested

Education: Many schools eager to learn Stanford results keep vital curriculum changes on hold until publisher gets them straight

And now, maddeningly, they wait.

Many testing directors of Orange County’s 27 school districts should be spending the next few weeks poring over test results, comparing this year’s fourth-grade scores to those of last year’s third-graders.

In districts such as Placentia-Yorba Linda, where some immigrant children remain in bilingual classes and others are assigned to “sheltered English immersion” classes, administrators wanted to begin tracking the progress of both groups of kids. Others need to analyze weak areas revealed by the Stanford 9 standardized test so they can tweak classroom instruction.

Instead, many researchers and testing evaluators are cooling their heels. One even scheduled a few days off.

A massive testing blunder is the cause for consternation among testing analysts. An error by the Stanford exam publisher, Harcourt Educational Measurement, has delayed the state’s release of test scores for counties, districts and schools. All Orange County school districts have already publicly released their district and school scores, which are expected to remain unchanged.

The error specifically involved Harcourt’s lumping of scores from students who have “graduated” from bilingual programs to English fluency with those who still don’t speak much English. That mingling artificially inflated results for students learning English and, conversely, deflated those for fluent English speakers. The error might not be remedied until July 15.

So Santa Ana’s director of evaluation and research, Linda DelGiudice, is postponing much of her data-crunching. In her heavily immigrant district, nearly all students fall into one of the two mixed-up categories.

“There’s not a lot of point in analyzing an error,” she sighed. “I personally prefer to look at accurate numbers myself.”

Other testing analysts–including Anaheim City’s Pam Ellis, one of the first to discover the glitch–are moving forward with analysis, using their in-house records and tracking systems.

“The fact that one group of scores is incorrect doesn’t prevent us from moving forward,” Ellis said. “There’s a lot of other important data to analyze. . . . I kind of give Harcourt a break, because the next error may be mine.”

She discovered the problem when she noticed that the number of limited-English students listed on the test results seemed way out of line with the district’s own figures.

Not everyone feels so charitable. Throughout the county, many educators are fuming, saying that the test publisher is not performing up to contract, not providing key information that was provided last year, and that they are worried about whether they can trust any of its results.

Among the concerns: Some districts could soon face penalties if scores don’t improve and administrators worry that they don’t have the data they need to improve instruction. Some districts initially had problems with CD-ROMs Harcourt sold for analyzing data. Others carped that scores were grouped differently this year. Previously, you could look at scores for limited-English speakers, versus native speakers, versus students who have successfully completed bilingual education programs and now speak English fluently. This year the only option is comparing limited-English speakers with fluent speakers.

And many parents are still stumped by what their scores mean.

Capistrano Supt. James A. Fleming could not contain his frustration with the exam. Just this year, he said, Stanford tests were delivered late; the extra “augmentation” questions were keyed to a new state curriculum that has not been implemented yet, so they befuddled teachers and students; and now scores for English-learners are in question.

“We heard the first oops on March 12, when the tests were late,” he said. “The second oops was the whole augmentation. The other oops was the limited-English scores. . . . The state needs to address which way it’s going for a meaningful accountability system.”

Santa Ana Unified Supt. Al Mijares said he expected the blunder to prompt his school board into seeking a legal opinion on whether the district can use another standardized test or opt out of the Stanford 9.

“It is disconcerting to put so much effort into the test itself, then get your scores, then try to make sense of them to improve your program, and then be told, ‘Wait a minute. Your information isn’t accurate,’ ” said Mijares, whose students made test gains almost across the board, according to preliminary data. “You would never build a bridge, argue a case in court or conduct surgery like that.”

In the Cypress School District, where students with limited English-speaking skills initially appeared to have made enormous gains over last year, administrators are scrutinizing results and awaiting a call from Harcourt.

Huntington Beach Union High School District administrator Jerry White had written a third of the 32 reports he planned on test results when the glitch was discovered.

“I’ve just stopped,” he said. “It’s hard when you can no longer hold your data credible. Some might stop looking at their programs and saying, ‘What are we doing wrong?’ I don’t like that attitude . I just hope they get this resolved soon.”



Comments are closed.