New research on school choice: Winning isn't everything

How many times have you heard of a lucky duck who wins the lottery, just to squander it all and return to his old work-a-day self? I'm sure those guys thought winning the lottery would turn their luck around forever.

Just like education reform proponents who are fond of calling school choice a "panacea" think that winning a voucher or attending a private school automatically results in a better student.

Well, there is new evidence that offering a choice isn't, by itself, going to effect education reform. A newly released study by the National Bureau of Economic Research (authored by Julie Berry Cullen of UC-San Diego and Brian Jacob of the University of Michigan) attempts to focus on the "lottery" effect of school choice by tracking, over time, students who won the ability to choose their Chicago public schools in Kindergarten or first grade. By also following students who did not win, they are able to avoid what is the biggest hurdle to good research on the effect of school choice...non-random selection.

Here in Milwaukee vouchers are not awarded randomly to families. In our city, parents first must find a school, then apply for a voucher seat at that school. Only if there are more applicants than seats will random selection kick in, and even then, the student is only competing against the other students who have chosen that school...they are not in a pool with students choosing other schools. Not a random situation at all. (Which is not to say it isn't in the students' and schools' best interests to operate the program that way, just that it is difficult to research rigorously due to this.)

In Chicago, however, the lottery operates first, randomly selecting from the entire pool of interested students those that will be able to "open enroll" in a public school of their choice. The winners then choose their schools. Cullen and Jacob find that the winning students "attend higher quality schools as measured by both the average achievement level of peers in the school as well as by value-added indicators of the school's contribution to student learning." Meaning the winners ended up in schools that had highly performing students and that improved students' scores over time. Sounds pretty good.

Unfortunately, many of these students turned out like those unlucky lotto winners of lore. The authors found no systematic educational benefits accruing to the winners:

We do not find that winning a lottery systematically confers any evident academic benefits. We explore several possible explanations for our findings, including the possibility that the typical student may be choosing schools for non-academic reasons (e.g., safety, proximity) and/or may experience benefits along dimensions we are unable to measure, but find little evidence in favor of such explanations. Moreover, we separately examine effects for a variety of demographic subgroups, and for students whose application behavior suggests a strong preference for academics, but again find no significant effects.

Because of the random selection, Cullen and Jacob can focus on the difference between schools to try to explain their results and not the difference between winners and losers. Why aren't the "good" schools improving outcomes for the lottery winners? Why isn't attending a good school enough to change your luck forever?

Obviously, because it's more complicated than that. Education reform must go beyond waving a magic wand and granting a parent's wish for a better school. Ask any principal in Milwaukee how easy it is to bring a poorly performing student up to grade level...it's hard work that isn't any easier just because the school has a good reputation or the word "private" in its name.

Now, private schools do have some ability to work with students' families in ways public schools do not. Usually this ability is what leads people to predict that private schools outperform public schools: if the parents or student aren't cooperating, the student doesn't get to stay and the school climate stays focused on high achievement.

However, another new report, this one from the Center on Education Policy, found that, controlling for family background and prior achievement, students attending private high schools performed no better than students in traditional public high schools on math, reading, science, and history achievement tests and were no more likely to attend college than public school students.

If accurate, this finding is truly disheartening. If the schools that are able to "choose" their students aren't achieving at a higher rate, and if winning the ability to choose a better school doesn't improve achievement, then how can school choice possibly be expected to be the tide that lifts all boats?

Again, it is complicated. There is too much variation in the school universe to paint all private schools, or all public schools, with the same brush. Some will be very good, some will disappoint, and some might even be damaging. The same is true for voucher winners. Some will maximize the opportunity, some will squander it, and some will try their best to no avail. It's time to acknowledge that there is no "gold standard" for education reform because schools and students cannot be standardized. Improvement will come classroom by classroom from the sweat of the brows of teachers and students working together. Reformers, and policymakers, must return focus to the work of the classroom.