Day: May 7, 2013

In a reply to my post about sneaky changes in the NECAP documentation, the RI Department of Education spokesman wrote this:

“The NECAP assessment is designed to measure whether students have attained the knowledge and skills expected at each grade level, that is, whether students have met grade-level standards.”

This, of course, is the heart of the matter, isn’t it?  I claim the test is a poor measure of the mastery of a body of knowledge, and therefore it is, shall we say, an outrageous act of irresponsibility to use it for a graduation test.  RIDE, of course, says otherwise.  This is precisely what is at issue in this whole controversy, and simply stating it as fact at the head of a reply doesn’t really address the point at all, but simply seeks to override it with the voice of authority.

In truth, as was pointed out by the psychometricians I’ve spoken to, RIDE has done little or no work to demonstrate the “validity” of the test, this very question.  For an employment test, by contrast, the laws insist that the employer demonstrate — with real data — that good performance on the test is a good way to identify good employees.  RIDE relies on correlation between NECAP scores and survey questions that ask piffle like “how much homework do you do in a week?”

The NECAP test was designed with the grade-level expectations (GLE) in mind, and it uses questions relevant to those GLEs.  Does that make it a good measure of whether a student has mastered those or not?  Tom Hoffman, who runs tuttlesvc.org, a great education resource, showed us at a Senate hearing that performance in Massachusetts and Rhode Island is not so very different on the 8th-grade math NAEP tests (administered by the federal Dept of Education and widely considered the “gold standard” of testing).  Overall, Massachusetts does do better than Rhode Island on that test, but they’re not in a different league.  But performance is dramatically different on the 11th-grade math tests administered by each state (NECAP in RI, the MCAS in MA).  Can anyone explain this?  Do our kids get dumber in the 9th and 10th grades?  Or are the tests different in ways that haven’t been adequately explained?

“NECAP was not designed to provide, in isolation, detailed student-level diagnostic information for formulating individual instructional plans.”

This is a quote from the NECAP documentation, earlier in the paragraph that they “clarified.”  According to RIDE, then, we should read “in isolation” in the sentence above as “only taking it once”?  This is comparable to the way RIDE claims that “multiple measures” is to mean that you can take the NECAP more than one time.  This is silly.  What the above means is that NECAP is a clue to student achievement, but should only be used as one of several measures, as was policy under the previous commissioner.  Making passage a graduation requirement is contrary to the meaning of the NECAP designers’ instructions.

Let’s end with a brief but important digression.

One hundred years ago, Henry Goddard, who went to school at Moses Brown and was a member of the first generation of psychological testers, persuaded Congress to let him set up an IQ testing program at Ellis Island that eventually proved that most immigrants were “morons.” (He  coined the term.)  During World War I, intelligence tests used to select officers were later shown to have profound biases in favor of native-born recruits and those of northern European extraction, which is another way to say that lots of Italian-American soldiers were unjustly denied promotions. For decades, misused IQ tests classified tremndous numbers of healthy children as disabled, or mentally deficient — well into the 1960s and 1970s. The history of testing in America is littered with misuses of testing that have had profound and unjust effects on millions of adults and children.  Does the available evidence about the NECAP test persuade you that we are not in the middle of one more chapter of this terrible history?

A graduation test is not a trivial thing.  The results of a test can have a significant impact on a young person’s life.  It seems to me that the burden is on the people who think a high-stakes graduation test is the only sensible way forward to demonstrate — with a great deal more rigor than they have so far bothered to do — that a test measures what it is supposed to measure.  The IQ tests at Ellis Island, in the officer corps, and in the schools, did not measure what they claimed, and thousands upon thousands of lives were changed, few for the better.

If these policy changes are being made for the sake of our children, then can’t we stand to have a little more compassion while we’re making them?  This means intellectual honesty, and it also means being careful not to ruin lives you say you’re trying to help.