Education narrative favoring wealthier, whiter states was just proven wrong

State education rankings published by organizations such as U.S. News and World Report or Education Week are highly influential. When education is discussed, whether the focus is teacher pay, unions, common core standards, or school choice, state education rankings are invariably used as a political cudgel.

These rankings have spurred a well-known consensus: states in the Northeast and upper Midwest have the best education systems. The worst states, supposedly, are fiscally conservative right-to-work states in the South and Southwest. It would seem parents must force politicians to spend into bankruptcy or else doom their illiterate, innumerate children to a menial existence.

But these conventional rankings fail to make an “apples to apples” comparison between states. Students arrive in class on the first day of school with different backgrounds, endowments, and life experiences, often related to race and socioeconomic status. Conventional rankings largely ignore these differing characteristics by combining scores on student achievement tests into an all-encompassing statewide average. This blunt over-aggregation skews school rankings in favor of wealthier, whiter states.

Conventional rankings also include metrics that aren’t directly related to learning. Some conventional rankings, like Education Week’s ranking, erroneously treat government spending on education as a purely positive factor, rewarding states that spend lavishly regardless of actual student performance.

We recently completed a study of state education systems and found that fixing these problems changes rankings substantially. Conventional rankings are thus severely flawed, as is the consensus of which states educate best.

We graded states based on how well they educate each type of student; i.e., how much “value added” in learning they create. Our analysis utilized the same Department of Education student achievement test data included in most conventional state rankings, but removed metrics unrelated to learning.

Crucially, we disaggregated scores by grade, race, and test subject in order to more accurately measure the value added by state educational systems.

This difference in approach is best understood with an actual example. When students are treated as a single monolithic group, Iowa students outscore their Texas counterparts in Math, Reading, and Science for fourth and eighth-grade students. But when students are disaggregated into major ethnic groups, the results reverse.

In fact, every ethnic group in Texas scores higher than their Iowa counterparts. In all, disaggregated Texas students surpass disaggregated Iowa students in 19 of the 20 exams they take in common.

It’s ludicrous to say Iowa outranks Texas when all student groups score higher in Texas. But that is the story traditional rankings would have you believe.

This Texas-Iowa example is no isolated fluke. The rankings of many states flip dramatically when our more appropriate methodology is used. In our ranking of education quality, Virginia, Massachusetts, Florida, New Jersey, and Texas rate highest, in descending order.

We also find that unionization negatively impacts student performance. Our regression analysis suggested that if a state went from having the weakest teacher unions to the strongest, its quality rank would decline by between 22 to 11 ranking positions. The percentage of students in charter schools also has a strong positive impact on student performance in some of our regressions.

We further refine our ranking to account for the efficiency of education spending. Rankings should not reward extravagant spending without corresponding gains in student performance, yet many do.

Consider New York and Tennessee, both states score similarly on our quality ranking (31st, and 30th, respectively) but New York spends three times as much. Whereas, Education Week’s ranking rewards New York for its lavish spending, our efficiency ranking sensibly penalizes such excess. We adjust spending to take account of different costs-of-living in each state.

After adjusting our ranking of education quality to account for spending efficiency, Florida, Texas, Virginia, Arizona, and Georgia lead the pack. All of these states are southern or southwestern, with right to work laws and very low levels of unionization, the very opposite of the conventional narrative.

These results are based on a much sounder analysis than traditional rankings. Evidently, a major reevaluation of the policies contributing to student performance is badly needed. We hope our improved methodology can help guide that important effort.

JOIN THE MOVEMENT #iBELIEVE