Nobody knows how to increase 11th grade NECAP math scores

The most prominent and persuasive argument for using the 11th grade NECAP math test as a graduation requirement in RI high schools has been the relative success of Massuchusetts’ use of their MCAS exam for that purpose.  Last year I prepared a graph showing how Massachusetts student’s math MCAS scores increased over time compared to the NECAP math scores of RI, Vermont and New Hampshire.  It is important to note that Vermont and New Hampshire score in the top tier of states by virtually every measure of math achievement.

See where the percentage scoring at least “2” on the MCAS  jumps about 20% between years three and four?  That’s the graduation requirement kicked in for juniors.  The same effect in theory should have kicked in in year six (last year) for the RI NECAP, but there was no corresponding jump.

The 2013 NECAP results, which have been partially released on RIDE’s website, continue the past trend.  The number of 11th graders statewide scoring “1” or “substantially below proficient” decreased just 4% to 36%.  This is simply not enough progress to show the policy is working.  If you dig down to individual districts and charter schools — demographic and other in-district breakdowns are not available — it is even more disappointing (past years’ data from here):

• The state’s flagship turnaround at Central Falls High School has the same number of students scoring “1” in 2013, 73%, as in 2008.  After all the turmoil, expense, and the reformers’ best effort, no change.
• Providence Public Schools has bumped the “1’s” down from  67% in 2008 to 63% in 2013.
• Barrington High School has only improved 4% since 2008: 12% to 8% scoring a “1” (compared to 1% in reading both years).
• Blackstone Academy, a small (about 40 in the junior class) charter with 85% economically disadvantaged students, has levelled off after some impressive progress with about a third of juniors not meeting the graduation standard.
• Paul Cuffee Charter School, a well-regarded new high school in Providence, still has 52% of juniors not scoring above a “1.”
• A few districts had noticeably fewer students at “substantially below proficient” compared to 2012: North Providence from 44% to 26%; Pawtucket, 63% to 52%; Westerly, 29% to 17%.
• But a other districts saw increases in students not meeting the requirement since last year: Exeter-West Greenwich, +9%; Newport, +4%; South Kingston, +3%.

I should hasten to note that my point here is not to do the typical blame and shame.  What seven years of 11th grade NECAP math scores tell me is that nobody has figured out how to increase them, especially among disadvantaged students, to an degree comparable to the MCAS and by enough to make the test an appropriate graduation requirement.

This is not supposed to happen.  Two of the fundamental premises of contemporary school reform are that students will rise to the level of expectations, and that incentives drive results.  We’ve got the expectations, the incentives could not be clearer or higher stakes to students and all the adult stakeholders and… the results are just not there.

To replicate the MCAS success story, North Providence’s 18 point improvement would have to be the average gain statewide, not an outlier.  When your highest flying low-income charter still has a third of its juniors not on track to graduate because of a single test, that’s not normal.  If RIDE knows how to increase 11th grade math NECAP scores, why haven’t they told Central Falls?  We’ve been paying outside consultants, too, like The Dana Center, who know as much about aligning math curriculum to standards as anyone.  Apparently they don’t know the answer either.

Exactly why we — and New Hampshire and Vermont — don’t seem to be able to raise 11th grade NECAP math scores is beside the point.  My theory is that 11th grade NECAP math scores reflect “fluid intelligence” more than the MCAS and other standardized tests, and teaching “fluid” skills like analyzing abstract problems and thinking logically in school is difficult and poorly understood.

Or perhaps the difference is simply that the MCAS was a problem designed with its own solution in mind.  The test was developed in parallel with a curriculum framework.  For schools to raise their MCAS scores, they needed to do a better job of delivering the state curriculum, which was not necessarily easy, but it was straightforward and achievable.  There is no equivalent map for increasing NECAP scores.

At this point, the burden of proof should pass to the proponents of the NECAP graduation requirement to lay out an evidence-based strategy for increasing the “pass” rate for NECAP math statewide by 20% that amounts to something other than “stay the course.”  We can’t have a third or more of seniors not knowing if they’re going to graduate in February or scrambling for waivers.  For this to work, we need Barrington at 99% pass, Westerly and Blackstone Academy at 95%, and Central Falls and Providence  need to nearly double the number of students getting over the bar.  Maybe it is possible.  Tell me how.  I don’t see it.

And of course, it is a moot point since RI is moving off the NECAP as soon as possible anyhow. So… seriously, why are we doing this to ourselves?