This article is from the archive of The New York Sun before the launch of its new website in 2022. The Sun has neither altered nor updated such articles but will seek to correct any errors, mis-categorizations or other problems introduced during transfer.
We’re all for taking advantage of research to help make policy decisions on issues like education, but that research is best wielded like an epee instead of a broadsword. Used injudiciously, questionable research leads to questionable conclusions like those espoused in a recent New York Times editorial about charter schools. The Times relies on studies conducted by the Western Michigan University Evaluation Center over the past five years to suggest that charter schools don’t significantly outperform public schools. Yet it is not clear that the studies in question show anything, let alone that they make the case for putting the brakes on such a popular education reform.
The research cited by the Times purports to find that, on aggregate, student performance in charter schools is not significantly different from performance in regular public schools. This is a result, the newspaper’s editors surmise, of the lack of regulation on the schools. That would be a reasonable proposition, but only if the analyses on which it is based were a little more meaningful. School reform advocates offer an argument that they’re not.
On its face, the research, much of it coauthored by the Evaluation Center’s Gary Miron, appears perfectly reasonable. It derives from respectable data sets, avoids obvious mistakes by controlling for demographic and economic factors, and uses sophisticated statistical methods to process the numbers. Yet there are still serious problems, as the head of the Center for Education Reform, Jeanne Allen, told us recently. Some of the studies for which the Times editorial bit are half a decade old and date from a period when charter schools in many places were just starting. Now that hundreds of charter schools across the country are hitting their stride, one would expect different results.
More fundamentally, Ms. Allen suggests, this research doesn’t account for the fact that in measuring educational performance, individual students matter. Charter schools often take the students who have been faring the worst in traditional public schools, so it would not be surprising if their aggregate performance at any given moment were below that of the public schools overall, although some studies suggest aggregate performance is not lower anyway.
A more accurate measure would come from studies that follow individual students over time, comparing their rate of improvement to that of their peers in neighboring traditional schools. The data one needs for a study of this sort are hard to come by. When researchers have been able to perform this kind of research the results have shown that charters outperform traditional public schools.
If anything, the Times’s blunder and the ensuing statistical scuffle point up a big catch in the education reform debate. Precisely because each student is different, savvy researchers can come up with an infinite number of data sets and manipulate each set in infinite ways. Measuring educational success is difficult, though as the No Child Left Behind Act recognizes, there’s a role for testing and other standardized performance measures.
What the Times forgot is that parents and students themselves deserve some consideration, too. As parents in the Times’s own backyard have shown by forming mile-long waiting lists for slots in the city’s charter schools, those schools must be doing something right. In which case, rather than waiting for the experts to agree – which will probably never happen – politicians would do better to trust parents’ own judgments in respect of whether their children are learning.