Public Policy Forum's top five research findings of 2012

If it’s December, then it must be time for the Forum’s annual list of its top five research findings of the year.  Last year’s list included findings on MMSD’s daunting capital needs, the dramatic decline in Milwaukee County’s corrections population, the City of Milwaukee's reliance on state shared revenue, and our region’s tardiness in embracing strategic economic development planning.  The 2012 list is summarized below in chronological order:

1. More than four-fifths of Milwaukee County’s rated child care providers stand to lose funding under YoungStar, the state’s new child care quality rating system.  Our January report analyzing the first year of Youngstar implementation examined the ratings earned by 534 child care providers in Milwaukee County as of December 2011.  We found that 428 (80.1%) received 2-star ratings and 33 (6.2%) received 1-star ratings.  In light of the state’s plan to reduce Wisconsin Shares subsidy payments for 2-star providers and disqualify 1-star providers from Wisconsin Share payments entirely, that meant more than 80% of the rated providers would lose funding under YoungStar, a consequence that could impact the availability of child care options.

2. Much of the growth in the Milwaukee Parental Choice Program in the past year appeared to come from existing private school students.  The 2011-12 school year was the first to reflect the impact of major changes to school choice program eligibility adopted in the 2011-13 state budget, including an increase in the enrollment cap, broader eligibility limits, and expansion to schools outside the City of Milwaukee.  Our annual survey of voucher school participants released in February revealed not only that much of the growth in voucher use from the previous year was in schools that already participated in the program, but also that it appeared to have come from students already enrolled in those schools.  

3. Fire department consolidation that preserves existing capacity can still save millions.  In May, we released a comprehensive report on options for sharing or consolidating fire services in the southern Milwaukee County communities of Franklin, Greendale, Greenfield, Hales Corners and Oak Creek.  Our finding that a full consolidation option could save nearly $2 million annually may not have been surprising to those who have examined fire service consolidation proposals in other states, or to those familiar with Milwaukee County’s consolidated North Shore Fire Department.  It was illuminating to find, however, that such substantial savings were achievable without eliminating any fire stations or reducing direct firefighting staff.

4. MATC’s technical diploma offerings are generally attuned with the region’s projected job openings.  Our December report on the activities and resources of Milwaukee’s key workforce development players examined MATC’s role in providing non-degree occupational training to job seekers.  We found that despite longstanding criticism of the college for failing to appropriately align itself with the needs of area employers, MATC’s technical diploma offerings match up well with Department of Workforce Development projections regarding future job openings.  Nursing, Barber/Cosmetologist and Emergency Medical Technician were the most heavily enrolled technical diploma programs in the 2010-11 academic year. 

5. MPS’ five-year fiscal forecast is more optimistic than those of Milwaukee County or the City of Milwaukee.  Our December fiscal assessment of the Milwaukee Public Schools examined the district’s most recent five-year fiscal forecast and found that its projection of a $41 million deficit in 2017 is plausible and perhaps even a bit on the conservative side.  While a projected deficit of that magnitude certainly is not ideal, it is more optimistic than the five-year deficits projected by the City of Milwaukee and Milwaukee County earlier this year.  It is important to note that each government’s five-year forecast is based on financial modeling that is somewhat speculative in nature, and that a complete understanding of the fiscal condition of each must go far beyond their five-year forecasts.  Nevertheless, given the substantial position cuts and general gloom surrounding MPS’ budget in recent years, to find that MPS’ five-year challenges may not be deeper than those of the city and county was somewhat surprising. 

With 18 research reports in 2012, it was not easy to reduce our list of top findings to five. Left off the list this year were important findings related to the application of the state’s new child care ratings system to afterschool care providers; the out-of-classroom spending habits of Milwaukee County suburban school districts; the range of case management services available to persons with mental illness in Milwaukee County; opinions on mental health redesign by nurses and their employers; and the continued impacts of pension and health care benefits on Milwaukee's city and county budgets. Those interested in reviewing those and other findings can access the Forum’s full list of research publications here.

Fighting an uphill battle

It always feels a bit discomforting to say, "Race and economic status are highly correlated to academic achievement." Children can change neither their race nor their income; a statement like the one above can feel like saying these children also cannot improve their academic achievement. But when confronted with the lackluster-to-dismal achievement data of many Milwaukee schools, where the race and income patterns are so stark, its hard to avoid making that statement.

As the Forum's recent report on the Milwaukee Parental Choice Program found, these patterns appear among private school students as well as public school students. In both the public and private sectors in Milwaukee, there are very few schools enrolling predominantly minority and low-income students producing high test scores. (Even among the suburban public schools with fewer minority or low-income children, we've found that as those populations have grown, aggregate test scores have declined.) And while the private school data are new, are they really news, considering how entrenched the pattern has been in the public schools?

What is news is the fact that, while the nation's schools have made some progress in closing the racial achievement gap, the income achievement gap has grown significantly. A recent New York Times story highlights the research of Sean F. Reardon, a Stanford University sociologist, who analyzed data from 12 national studies starting in 1960 and ending in 2007 and found that while the average black-white racial achievement gap has shrunk in half since the 1960s, the gap between children from the wealthiest and poorest families has grown 40% in that time, and is now twice the size of the racial achievement gap.

Schools with low-income students in their charge, whether public or private, are fighting against a nationwide, decades-long trend. While several of these schools, on an individual level, have stellar outcomes, educators are still hunting for the best strategy to improve outcomes system-wide. The hunt requires even more urgency now--the recession is creating more, not fewer, poor families.

Most new voucher users already were enrolled in private schools

The Forum's 14th annual census of schools participating in the Milwaukee Parental Choice Program (MPCP) finds that voucher use by Milwaukee students grew 10% in 2011-12 to 23,198 voucher students, reversing last year’s enrollment decline. In addition, the data indicate that most voucher students are attending hyper-segregated schools that have low reading and math proficiency rates.

The dramatic increase in voucher use is likely due to changes to the program in the most recent state budget, which allowed schools outside Milwaukee to join MPCP and expanded eligibility to include families at higher income levels. As a result, more than 2,200 additional students are using vouchers worth $6,442 each, increasing the program’s cost by $14.2 million.

Most of the new voucher users appear to have already been enrolled in private school. In 56 schools, the number of new voucher users exceed the growth in total enrollment in the school, while in 13 schools voucher growth and enrollment growth were equal. Over the past 10 years, total enrollment in the schools participating in the program has grown by roughly 5,300 students, while the number of voucher users has increased over twice as much.

More students are eligible for vouchers because the income limits for voucher were raised to 300% of the Federal Poverty Level, which means that a family of four earning up to $67,050 per year is now eligible. The median household income in Milwaukee is $35,921 per year. Under the new rules, once a student qualifies for a voucher, he or she remains eligible for all subsequent years, even if the family’s income grows.

The report also includes an analysis of the 2010-2011 state standardized test results of the participating schools, finding that performance among the MPCP schools varies widely. There are a few patterns in the data, however. Reading proficiency rates are higher, on the whole, than math proficiency rates. In addition, schools with fewer voucher users and fewer minority students tend to produce higher proficiency in both reading and math, as do the Catholic and Lutheran schools.

The poor test scores are likely related to the socio-economic and racial demographic make-up of the schools, which mirror the Milwaukee Public Schools, in the aggregate. Nearly half of all MPCP schools have student bodies that are at least 90% minority and/or 90% low income; 65% of all voucher users attend one of these schools.

The report also includes updated data on enrollment trends, schools gaining and losing the most MPCP students, and the aggregate high school drop-out rate. Schools participating in Racine’s new Parental Private School Choice Program are included as well.

The full report and an interactive database of school information are available on the Forum's website.

Will an expanded voucher program cost more or less?

Gov. Walker’s proposed 2011-2013 biennial budget calls for an expansion of the Milwaukee Parental Choice Program by repealing the enrollment cap, allowing private schools anywhere within Milwaukee County to participate, and expanding eligibility to all City of Milwaukee families by eliminating income limits.

During tough budget deliberations, it would be good to know whether the expanded choice program is likely to save or cost state taxpayers over the long run. Either is possible—taxpayers save if the students who join the expanded program otherwise would have been students at more costly public or charter schools and taxpayers lose if the new voucher users would have otherwise been free to the state as tuition-paying private school students.

There is a debate over the likelihood that the program will be able expand considerably, as capacity for new students in the county’s existing private schools appears constrained at this time. However, the debate so far has overlooked the fact that the proposed budget would allow new voucher users to be existing private school students starting in the 2012-13 school year. There is a real concern that the expanded program may, in fact, increase costs for the state over the long run by increasing the total number of Wisconsin K-12 students who receive state support for their education.

Analyzing data from other years in which the legislature expanded the program can give a sense of how likely it is that significant growth in the program will come from existing private school students. When the choice program expanded to include religious schools in 1998, the state Department of Public Instruction (DPI) collected information regarding the type of school each voucher user had attended the previous school year. From that data, we know that 50% of the new voucher users in 1998-99 were already enrolled in private schools—2,512 of the 4,995 new voucher users.

We can also look at growth in voucher use compared to total growth in private school enrollment. In 2006, when the enrollment cap was raised, more private school students took advantage of the opportunity to use vouchers than public school students. In fact, 60% of new voucher users were existing private school students in 2006—1,408 of the 2,355 new voucher users. In the Catholic and Lutheran schools in particular, new voucher users tended to be students that were already attending these schools. The Lutheran schools, for example, had 467 more students using vouchers in 2006-07 than the previous year, but total enrollment in Lutheran schools grew by only one student during that time.

If these previous experiences are a guide, it is not unreasonable to expect that about half of the new voucher users in 2012-2013 will come from within the private schools joining the program. A quick analysis of private schools located outside the city limits that may be enrolling significant numbers of Milwaukee residents indicates at least six such schools: Indian Community School, St. Thomas More High School, Dominican High School, Milwaukee Jewish Day School, St. Bernard School, and St. Robert School. The likelihood that the student populations of these private schools could generate significant demand for new vouchers is quite high; these six schools enrolled a total of 1,729 students in 2009-10.

It is clear that assuming all, or even most, new voucher users in the coming years will save taxpayers money by switching from public or charter schools is not realistic. There will certainly be growth in the total number of elementary and secondary students receiving state taxpayer support. The debate should be about the affordability of these extra costs in the short- and long-term, whether these higher costs can/should be considered an investment in a better future, and what the effects of a more costly choice program might be on the public and private schools.

UPDATE: To clarify, low-income private school students who reside in Milwaukee and attend a participating private school in Milwaukee are currently eligible to use vouchers. The budget bill does not change the income limits for these students. However, it does expand their schooling choices to include participating schools throughout Milwaukee County.

No growth in school choice program this year

Milwaukee's private school voucher program, known as the Milwaukee Parental Choice Program (MPCP), enrolled slightly fewer students this fall than in the previous year. That makes 2010-2011 the first school year without growth in the program since its expansion to include religious schools in 1998-1999. The Forum analyzes 13 years of school choice data in our newest report on the program, released today.

The loss of 66 students this year was minor compared to the number of schools leaving the program. Twelve schools did not return to the program this year. Combined with the 17 schools that were lost in 2009-2010, this factor had a direct impact on the decline in enrollment. Just three schools joined the program in 2009-2010 and four schools joined this year, all of them much smaller than the schools that closed or withdrew.


Thus, at 103 schools in 2010-2011 compared to 114 in 2009-2010, it shouldn't be surprising that fewer students receive vouchers this year. Total enrollment also declined this year among the schools participating in the MPCP.

The reduction in the number of participating schools is the result of new state regulations requiring MPCP schools to be accredited. Most of the schools leaving the program over the past two years did so because they lacked accreditation. In addition, two schools left the program this year in order to convert to charter schools, which earn more per pupil than voucher schools.

For more on the impact of state regulations on the program, including newregulations requiring all schools to administer the state standardized tests, see the full report. A directory of schools participating in the program, with information useful to parents making schooling decisions, also is available on the Forum's Web site.

New rules for voucher schools having an impact

In the Forum's 12th annual census of schools participating in the Milwaukee Parental Choice Program (MPCP), we found that recent regulation changes have had an impact on the number of schools participating in the program. Between the 2008-09 and 2009-10 school years, the program saw far fewer new schools join and many other schools close or drop out of the program. Thirteen of the 14 schools that closed were not accredited, which was a new requirement that became fully phased-in last year.

In addition, a rule put in place for the 2009-10 school year that requires schools new to the program to obtain pre-accreditation seems to have dramatically reduced the number of new schools this year. Over the past decade the program was averaging 11 new schools a year, but this fall just three schools joined the program. Many, if not most, of those new schools were start-up schools, but the three schools joining the program this year are all established schools. The pre-accreditation requirement, which was intended to ensure new schools have a solid operational and educational footing, seems to be having an impact.

We also examined the potential impact of new regulations requiring schools to administer state standardized tests and to report MPCP student test scores. Starting in the 2010-2011 school year, all MPCP schools will be required to administer to MPCP students the same state standardized tests as public schools.

Nearly all schools in the program administer standardized tests and 37% of schools administer the Wisconsin state test. This rule will require the two-thirds of schools that use a test other than the state test to either switch tests or to add in the state test.

Another new rule kicks in this August, when schools will, for the first time, be required to report MPCP student test scores to the state Department of Public Instruction. Of the 112 schools in the program this year, 102 administer at least one type of standardized test and are expected to be able to report test scores in the fall.

For more information and for updated data on enrollment trends, schools gaining and losing the most MPCP students, schools’ racial make-up, and the aggregate high school drop-out rate see the full report.

Click here for complete 2009-10 data and a directory of all participating schools.

A school choice paradox (or two)

The latest results of the longitudinal evaluation of Milwaukee's school choice program were released by the School Choice Demonstration Project (SCDP) at a Forum Eye-Opener Breakfast last week. (You can listen to the podcast here.)

Dr. Patrick Wolf, the lead evaluator, does a nice job summarizing the findings of the eight different research reports in his executive summary. These are nuanced findings, with a few contradictions in them, yet understanding them is vitally important for policymakers. Wolf's summary, if not all the reports, should be required reading for local and state officials.

What struck me as I waded through the findings (and I have had several weeks to ruminate on them as I was given a preview of the results last month due to my membership on the SCDP's advisory board) was that Wisconsin has created an education reform policy so paradoxical that mixed results are nearly assured.

First, by encouraging dissatisfied parents to "vote with their feet" and exercise their choice, have we exacerbated schools' mobility problems and negatively impacted student performance?

Educators have known for ages that children who switch schools experience a set-back in achievement, at least initially, and maintaining stability goes a long way toward being able to improve learning and understanding. School choice presumes that a child's switch to a school selected by a parent as being a better fit for the child would outweigh the initial negative impacts of the switch. While that may, in fact, be true, the researchers found that in Milwaukee parents don't stop after making that first choice. They just keep choosing, and choosing, and choosing. (Some portion of this choice is likely due to the high mobility rates of low income families.) This occurs in both the public and private schools, although it appears to be a bigger problem in the public schools. The resulting instability in classrooms and in children's lives could be negatively impacting test scores in both MPS and the private voucher schools.

Thus, we need to ask ourselves, "Could there be such a thing as too much choice?" Or, at least, "Are we doing enough to educate parents about the consequences of choosing too often?"

The second paradox involves an expectation of improved test scores resulting from greater competition among schools, despite that fact that most competition occurs in grade levels that are too young for standardized tests. Third grade is when standardized testing starts, yet over the past 10 years, most of the vouchers have been used by students in grades younger than 3rd. The threat of losing students to another school was supposed to result in better outcomes and parents were supposed to choose the most successful schools, causing poorly performing schools to lose market share. In reality, what we find is that demand for vouchers is greatest in younger, untested grades, which doesn't really send much of a message to schools about the need for measurable performance improvement in order to succeed in the marketplace.

One way to get the message to the schools would be to start testing younger students. A more realistic option is to acknowledge the research that shows parents aren't considering test scores as a major factor in their decision to choose a school. If the choice isn't based on testing results, then perhaps we shouldn't be expecting the reform to improve test scores. If that's the case, then we need to agree on a different policy goal for the school choice program: parental satisfaction, sustainability of religious schools, weakening the political power of the teachers union, stemming population decline in the city, or saving taxpayer money. These have all been suggested as policy goals over the years and are worthy of debate.

Maybe it is time to stop discussing school choice in the context of "competition" altogether. I once talked with a parent who had narrowed down her choices to either the large public Montessori school a 45-minute bus ride away or the smaller private Catholic school in a nearby neighborhood. I tried to talk her though the few comparable facts I had collected about each school...enrollment size, afterschool programs, etc. Her biggest concern was not academic achievement, but discipline, and I had no information for her about that. I don't know how she made her decision, in the end, and I don't know how her decision could have sent an actionable message to the "losing" school. Was it too big? Too secular? Too far? Too traditional?

Most of the reaction to the SCDP research results has ignored the inherent conflict between the results and the rhetoric: How can it be competition that has improved MPS (ever so slightly), when the performance of the competing schools actually differs so little from MPS? How can both good and bad schools be thriving in the educational marketplace?

The answer is that parents are willing to overlook academic deficiencies when they are satisfied with other aspects of the chosen school. Schools, public and private, good and bad, are able to attract students each year despite also losing students each year. Milwaukee must face facts: Our parents like having choices and yet our schools are mostly underperforming. So now we must debate: Do we give up on school choice or adapt it to a new goal? Do we shut down failing schools or invest more resources in them?

Voucher schools similar to MPS schools

Big headlines last winter proclaimed voucher students "achieve about as well as those at MPS," provoking debate about whether private voucher schools should be expected to perform similarly to MPS or better than MPS.


Today, findings from the Forum's 11th annual census of voucher schools reveal a likely contributing factor for the test score similarities--in many ways voucher schools and MPS schools are alike:

• The student population in the average voucher school is 78% voucher students, who must be low-income to be eligible for a voucher. The student population at the average MPS school is 77% low-income, as measured by eligibility for free or reduced-price lunch.


• The student population at the average voucher school is also 85% minority, while the average in MPS is 88% minority.


• Voucher schools average 15.6 students per full-time teacher, compared to 14.9 students per full-time MPS teacher.

The biggest differences between voucher schools and MPS?

• Most voucher schools are religious (81% of voucher students attend religious school) while MPS schools are, obviously, secular.


• In addition, while MPS students must take the state standardized exam (the WKCE) of the 122 voucher schools, 114 administer standardized exams, and less than half of those (53) administer the WKCE.


• Among the middle, elementary, and K-12 voucher schools, less than half have teaching staffs large enough for each grade level to have at least one full-time teacher.


• On the aggregate, high school students who use vouchers appear to drop out over the course of four years at a much lower rate than MPS high school students.

Because the Forum has been surveying voucher schools for 11 years now, we can make some observations about how the voucher program has changed over the past decade. Enrollment has grown 227% since 1998-99, to over 20,200 voucher students in 2008-09 (the annual price tag for the program has grown 358% to $129.1 million this year).


Since the growth in enrollment has been accompanied by an increase in the number of schools, from 86 to 122, the average school size has grown just 5%, from 201 students to 211. At the same time, the average rate of voucher use within a school has grown from 34% to 78%.


The concentration of voucher students across the program has not changed dramatically, however. Today, just as in 1998-99, a fifth (20%) of the schools enroll half (50%) of all voucher students. The greatest difference is in the share of students enrolled in non-denominational Christian schools, which was 1% in 1998-99 and is 17% today. In addition, the percent of voucher students attending secular schools has declined from 33% to 19% over the past decade.


We also find that the distribution of schools and students across the city has changed. There are 10 fewer voucher schools on the city's south side today, at 15 schools, while the number of schools on the northwest side has doubled, to 57 schools.


On our website you can read more findings in the full report and download the accompanying poster-sized directory of voucher schools.

Mitigating the risk of making a bad schooling choice

'Tis the season for school shopping, as certain programs and schools have deadlines for fall enrollment quickly approaching. An article last week in the New York Times covered this subject by following one mother's struggle to find the right high school for her daughter. The irony was that this mother was an expert on the city's schools. Says the article:

"...Ms. Hemphill literally wrote the book on the subject — her series of “New York City’s Best Schools” books are regarded as the bible for navigating school choices — yet she has found herself befuddled and overwhelmed trying to help her 13-year-old daughter, Allison Snyder, find a spot."

Choosing from among the best and most selective public high schools in New York City is the kind of problem many parents in Milwaukee would like to have. But choosing a school here in Milwaukee can be similarly daunting. City residents can choose:

• a neighborhood MPS school;


• a specialty MPS school, with options including Montessori, Waldorf, fine arts, or technical careers, among many others;


• a charter school, chartered by either the city, UWM, or MPS;


• a suburban public school, either through open enrollment or the city-suburb integration program;


• or a private school, with those who can afford it paying tuition and those who can't using a tax-payer funded voucher.

Perhaps someone needs to write a book on Milwaukee's best schools. Having spent nine years at the Forum studying schooling in Milwaukee and co-authoring a previous book on school choice, I could be a good candidate to do so. But I would have to reveal the dirty little secret about choosing schools: There's no rule requiring parents to choose schools based on academic criteria. Parents can choose a school for any reason whatsoever.


I learned the secret when I was choosing a school in Milwaukee for my own son. I found myself, one of the "experts," basing my choice on location, a full-day pre-Kindergarten program, and an award-winning afterschool program. It made sense for us, becasue my husband and I both work downtown full-time and having eight hours of convenient child care a day was important. But academics didn't really enter the picture and we could have chosen based on any number of quirky reasons. In fact, we briefly considered a private school, but rejected it based on the junk-food-laden lunch menu.


The New York family narrowed their choices using arbitrary criteria of their own. Reports The Times: "She focused on smaller schools that were no more than 45 minutes from home and would offer her a chance to take advanced classes but also give her enough time to focus on dance and theater after school."


There may be facets of a school that are a greater priority than academic performance for the parents choosing that school--religious instruction, teaching methodology, and student discipline, for example. Certain schools can thus attract parents despite low scores. But, as a result, competition among schools may not result in better school performance and there is little evidence that it has, in Milwaukee.

So, if even school performance "experts" find choosing schools difficult and overwhelming, due to all the factors that could be considered, can anything be done to help regular parents exercise their choice more efficiently and with better results?


The answer is simple: mitigate the risk of making the choice. Parents who are wealthy enough to exercise their school choice by choosing to buy a home in a good school district don't have much risk attached to that decision-making process. They are able to choose from among a host of academically good options.


School choice in its various guises in Milwaukee was supposed to result in similar empowerment for Milwaukee's low-income parents, but it has not, mainly because their choices carry risks: quite a few of the schools available to them are not performing adequately, according to data from a state-sanctioned evaluation of voucher schools and charter school test scores. If parents were making choices based on school performance, this would not be a concern, as we could assume they would not choose bad schools. But they aren't (which we know from the number of voucher schools that have failed despite having rather large enrollments) and so, despite the availability of choice, many children in Milwaukee are continuing to suffer educationally.


The state has recently taken steps aimed at mitigating the risk of choice, by requiring voucher schools to be accredited, for example. But there are few other regulations regarding academic offerings and accountability.

Having a choice among mostly poorly performing schools is no better than not having a choice at all. Without accountability for academic performance, we put the burden on parents to be experts on schooling, instead of expecting schools to excel.

You say "accountability," I say "longitudinal study"

In a report released last week that did not get press other than a post on the education blog of the Journal Sentinel, the Legislative Audit Bureau rehashed the first-year findings of the School Choice Demonstration Project's study of the Milwaukee Parental Choice Program (MPCP). Interestingly, the Bureau found the study's data cannot provide information about performance in individual schools. In response, a prominent school choice lobbying group has called for test scores to be reported annually on a school-by-school basis.

The overall findings, released last February, were not as positive as education reform supporters had anticipated. The Audit Bureau re-analyzed the data and confirmed these findings. For example, the sample of choice students in the private schools had lower reading scores on state standardized tests than a matched sample of MPS students at three of six grade levels. At all six grade levels tested, the private school students scored lower than a random sample of MPS students. In nearly all cases, however, the differences were not statistically significant.

As the Demonstration Project's researchers emphasized, these results are to serve as a baseline. The real test will be whether scores improve in coming years and how those improvements compare across school types. The Audit Bureau also agrees that tracking score improvements over the years will be most important. But, the Bureau is worried that due to attrition within the samples, there may not be enough students in future years of the study for long-term comparisons to be reliable.

That is just one of many caveats in the Audit Bureau's report, most of which were methodological. The main concern of the auditors is the lack of usefulness of the study overall. Says the report:

While the project is designed to answer several academic research questions, there are limitations to its usefulness for policymakers... [W]e had initially believed that the project would provide us with data that identified the school attended by each Choice pupil who took the tests... [H]owever, citing confidentiality concerns, the project chose not to provide information on these pupils' scores... Because the project's data do not identify the Choice pupils and schools, we are limited in what we can report and confirm.

The auditors go on to say that this limitation means they cannot report information about academic performance specific to each choice school, despite there being such information available about every public school in Milwaukee and the rest of the state. The study, therefore, provides no accountability for individual private schools accepting taxpayer-funded vouchers in lieu of tuition.

While perhaps the Audit Bureau shouldn't be surprised that a longitudinal study is not a great vehicle for providing accountability, their words and tone indicate that they were expecting to be able to report out performance data for individual schools. Indeed, in early 2006 when the longitudinal study was passed as part of a compromise bill to lift the enrollment cap on the voucher program, it was cast as an accountability measure by many, including the editors of the Journal Sentinel (also here) and proponents of lifting the cap. In fact, it is still discussed under the heading of accountability on the School Choice Wisconsin website.

Yet, School Choice Wisconsin's official response to the audit report included the following language:

Raw test data released on a school-by-school basis are not meaningful and even can be misleading. Such data will not provide legislators or parents with useful information about the academic quality of individual private schools in the Milwaukee Parental Choice Program (MPCP).

The statement goes on to say:

School Choice Wisconsin favors an independent annual report available to Milwaukee parents that includes school-by-school testing data based on individual student progress. This report would use a common basis for reporting test scores at schools in the MPCP, the Milwaukee Public Schools, and at independent charter schools. We also favor an aggressive information campaign to inform parents about their options.

This second quote is the first call from School Choice Wisconsin for any kind of comparable reporting of test scores that could be used for accountability purposes. (Why they feel the Demonstration Project is not sufficiently independent to report school-by-school data is not clear from the press release; but that concern could be discussed and worked out.)

The important thing is in response to the Audit Bureau's concern that the longitudinal study has not released its findings on a school level, the most prominent school choice advocacy group is calling for comparable test score results to be reported. If legislators, educators, and advocates can come together around real accountability, then perhaps soon parents and taxpayers will be able to make informed decisions about school quality.

School Choice: Is Milwaukee still state-of-the-art?

Milwaukee has long been called "ground zero" of education reform in America, due mostly to our nearly two-decade-long "experiment" with publicly-funded private school vouchers. Now New Orleans, LA (NOLA) threatens to revoke our title as the epicenter of school choice by heeding the lessons learned here in Milwaukee and advancing the policy design with its new voucher program.

Governor Bobby Jindal of Louisiana is set to sign the nation's fifth voucher program into law, allowing impoverished students in under-performing New Orleans public schools to leave for other options. The NOLA program's legislation looks designed to avoid many of the failings of Milwaukee's program: it borrows certain elements of our program, building on Milwaukee's strengths, yet limits our deficiencies.

For example, NOLA has learned from us that supply and demand are insufficient accountability mechanisms. Voucher recipients in New Orleans will be required to take the regular state standardized tests, including the graduation exit exam, and school level testing data will be reported to the public annually. In addition, private schools in Louisiana are overseen by an advisory committee of school representatives who advise the state school board on standards. The board approves new private schools and discontinues approval if a private school is not maintaining its standards. By having access to school performance data, parents will be able to make better informed decisions. They will also be able to rely on the added level of accountability provided by the state school board and its advisory committee.

Anticipating that a program designed to allow parents to "vote with their feet" will find parents actually doing so, the New Orleans program provides that students who find themselves in unacceptable choice schools have priority in transferring to another participating school. This not only helps parents feel comfortable about taking the risk to change schools, it also avoids the problems that have been experienced in Milwaukee when a school fails mid-year and leaves students with no other options.

Further accountability comes in the form of a probationary period for those schools failing to comply with the statutory provisions requiring a healthy audit. During the one-year probation, the school will not be permitted to enroll additional voucher students. If the school has not come into compliance by the end of the probationary period, it will no longer be eligible to participate in the program. This requirement protects students and taxpayers alike and avoids situations like those in Milwaukee where a school finds itself closing its doors due to unexpected financial problems.

New Orleans' new program also reflects the lessons that parents cannot be empowered without information and shopping for schools is difficult. In addition to testing students and reporting the results, choice schools in New Orleans will be required to inform each family about all school rules and policies, including academic and disciplinary policies, prior to their enrollment. Schooling information will be obtained not only from the schools, but also from the state education department, which determine voucher eligibility as a first step. Once a student has the voucher in hand, he or she can choose from among the participating schools, which will hold a simultaneous enrollment period.

Parents who are shopping for schools in Milwaukee have to call or visit schools individually to gather any schooling information, limiting the number of schools in competition for that student. In New Orleans, schools will compete amongst themselves for the limited pool of voucher recipients during the mutual enrollment period. In this way, the NOLA program encourages schools to market themselves directly to these parents, likely giving New Orleans parents opportunity to obtain more complete information than Milwaukee parents.

Finally, unlike Milwaukee, the NOLA program is designed to ensure public schools compete for students based on school performance. It does this in two ways. First, it grants eligibility only to those students enrolled in an under-performing public school (and new Kindergarten students). Current private school students cannot receive vouchers. Secondly, it provides additional money on top of the voucher amount for special needs students. Thus, the playing field is more level as the private schools' disincentive to accept special needs students is reduced, while the focus of the parents' choice is squarely on the relative performance of the schools. In Milwaukee, parents cannot consider school performance, as no such data on individual private schools are available. Furthermore, one of the biggest complaints about Milwaukee's program, from both sides of the issue, has been that the voucher amount does not cover the entire cost of educating special needs students.

While the New Orleans' program looks promising from a policy design point of view, it is obviously too early to tell whether implementation will be successful. If it is, Milwaukee's 18-year-old program just may have been replaced by an updated model.

Voucher schools compromised by legislature

The Forum's tenth annual census of private schools participating in the school choice program is released today. Our analysis finds that the results of the 2006 legislation lifting the enrollment cap in exchange for greater accountability are not as expected. Lifting the cap allowed enrollment to grow 16% in the first year, but that growth moderated to only 7% the following year. Meanwhile, the effect of the requirement that schools become accredited is moderated by the inclusion of several atypical organizations on the list of allowable accrediting agencies. Fifty-six schools, enrolling 42% of all voucher students, get their accreditation from agencies that were not in the accrediting business before the statute was passed and were affiliated with private schools prior to becoming accrediting bodies.

Other key findings include:

*Turnover among schools--10 schools joined the program and 11 schools left the program.

*Demand for private options by public school students--Of the 1,282 new voucher users, at least 44% appear to already be enrolled in private school.

*Teaching staff diversity compared to diversity of students--Of the 38 schools with 100% minority students, 28 have teaching staffs comprised of more than 75% minority teachers.

*Demand by grade level--In Kindergarten, there is one voucher user for every three MPS students, in high school there is less than one voucher user for every five MPS students.

*High school dropouts--On the aggregate, voucher users appear to stay enrolled in high school at a higher rate than MPS students.

Click here for a poster-sized directory of all private schools participating in the voucher program.

Put your money where your agenda is

Among Milwaukee's claims to fame is our status as "ground zero" for the school choice movement. No other city has as large or as long-standing a school choice program as Milwaukee's, which enables over 20,000 low-income students to attend private school using taxpayer-funded vouchers.

The Forum has had much to say over the years with regards to the intent, design, and function of the voucher program, but we have not weighed in as either pro- or anti-school choice. (Our survey work has shown that it is generally supported by a majority of the public and that parents are satisfied with the program.)

We are just about the only policy organization in our state that can say we have not taken a position on this issue.

A new study by the National Committee for Responsive Philanthropy reveals why the Forum is unique in this respect...advocacy on behalf of school choice is big business. In one year (2005), over $65.5 million in grants to pro-choice groups were made.

While those of us paying attention to education reform have known for a long time that certain foundations are playing an extremely important role in policymaking, this report is the first to delineate and enumerate the power of the foundations and the advocacy groups they support. You shouldn't be surprised to learn Milwaukee's own Bradley Foundation is the second-most generous supporter of pro-choice advocates and researchers, giving 37 grants totaling over $6.3 million in 2005 alone.

The unique coordination and collaboration among the various foundations and the groups they support is the focus of the study. As the author states,

"This report shows how philanthropic capital from small and large foundations
has helped build political support for the school privatization agenda. It can
serve as a case study for other foundation and nonprofit leaders who are
interested in effective, strategic movementbuilding grantmaking."

The children receiving vouchers in Milwaukee do so because a school choice movement was built and strengthened in a strategic and systematic way, with supporters putting their money behind their ideas and ideology. The National Committee for Responsive Philanthropy considers this history a blueprint for future social movements. I expect this model will be tried, with varying degrees of success, across many policy issues. Where does this trend leave an independent, non-advocacy research organization like the Forum? I guess we'll be the ones, ten years down the road, telling those foundations whether they've spent their money wisely.

PPF Pearl: Growth in school choice program not what it seems

Alan Borsuk's article in today's Journal Sentinel could lead one to believe that growth in the voucher program necessarily comes at the expense of MPS enrollment. However, our research shows the recent growth in voucher use comes mostly from existing private school students, not public school students switching to private schools. The lifting of the program's enrollment cap was accompanied by some changes in eligibility, which made more private school students able to take advantage of the publicly-funded voucher program.

When we analyzed the voucher use and enrollment data last year as part of our annual census of choice schools, we found that while voucher use in private schools had grown by 2,516 students, enrollment had grown by only 620 students. For example, the Lutheran schools as a whole had 467 more voucher users, but total enrollment grew by only 1 student. The only Jewish school in the program actually had 2 fewer students, but 31 more voucher users.

These findings help explain why, as Borsuk's article states, "[MPS] test scores and other indicators continue to lag behind the state and have not changed dramatically." The program is not exerting competitive pressure on MPS. In 2006, 80% of voucher users were enrolled in religious schools. Are these schools truly competitors with MPS? The data indicate that parents are making choices based on religion, not on the availability of a voucher. From our report:

How can MPS be expected to compete with a program consisting mainly of religious schools that attract students who most likely would never have been public school students?

Growth in voucher use is certainly growth in publicly-funded education, as the article explains. Unfortunately, simply making the public education pie bigger doesn't seem to be encouraging MPS to get better.

Dog bites man and school choice oversells

Sometimes when I read the Journal Sentinel's headlines, I wonder if I'm actually reading The Onion.

Today's headline is an example: Choice may not improve schools, study says

After decades of MPS score stagnation despite school choice, charter schools, open enrollment, Chapter 220, and intradistrict busing, this is now newsworthy?

I have yet to read the new study that makes this finding, but from the paper's coverage it seems that parents are the scapegoats. Allowing them choices didn't improve schools because they failed to make good choices.

But maybe that's not a failure of the parents, maybe that's a failure of the "market theory" of school reform. We all make bad choices when we act as consumers. Despite a general legal theory of caveat emptor ("buyer beware"), there are lemon laws and tort law and product recalls. Why weren't similar consumer protections built into the education marketplace?

If a parent chooses a school that turns out to be a bad school, there's no way for that parent to recoup their "losses." You can't sue a school district for educational malpractice and good luck trying to get any satisfaction from an unregulated private school. Leaving the bad school for a better school was supposed to be enough accountability to improve the bad school...but years of research have shown this isn't the case. And it's because it's not easy to tell the bad from the good. There is no Consumer Reports for schools; parents find a good school through trial and error. What if you had to buy a refrigerator that way?

I guess the newsworthiness of this study is that it was authored by a conservative advocacy group, the Wisconsin Policy Research Institute, that has a long history of supporting school choice. Perhaps they will now support policy changes that provide parents with the information they need to make good choices. I don't think I'll hold my breath for that, though. The study looked only at choice made within the public school system, where academic information is readily available to parents; when asked whether the study shed any light on private school choice, where there is no such information available, the author demurred.

New research on school choice: Winning isn't everything

How many times have you heard of a lucky duck who wins the lottery, just to squander it all and return to his old work-a-day self? I'm sure those guys thought winning the lottery would turn their luck around forever.

Just like education reform proponents who are fond of calling school choice a "panacea" think that winning a voucher or attending a private school automatically results in a better student.

Well, there is new evidence that offering a choice isn't, by itself, going to effect education reform. A newly released study by the National Bureau of Economic Research (authored by Julie Berry Cullen of UC-San Diego and Brian Jacob of the University of Michigan) attempts to focus on the "lottery" effect of school choice by tracking, over time, students who won the ability to choose their Chicago public schools in Kindergarten or first grade. By also following students who did not win, they are able to avoid what is the biggest hurdle to good research on the effect of school choice...non-random selection.

Here in Milwaukee vouchers are not awarded randomly to families. In our city, parents first must find a school, then apply for a voucher seat at that school. Only if there are more applicants than seats will random selection kick in, and even then, the student is only competing against the other students who have chosen that school...they are not in a pool with students choosing other schools. Not a random situation at all. (Which is not to say it isn't in the students' and schools' best interests to operate the program that way, just that it is difficult to research rigorously due to this.)

In Chicago, however, the lottery operates first, randomly selecting from the entire pool of interested students those that will be able to "open enroll" in a public school of their choice. The winners then choose their schools. Cullen and Jacob find that the winning students "attend higher quality schools as measured by both the average achievement level of peers in the school as well as by value-added indicators of the school's contribution to student learning." Meaning the winners ended up in schools that had highly performing students and that improved students' scores over time. Sounds pretty good.

Unfortunately, many of these students turned out like those unlucky lotto winners of lore. The authors found no systematic educational benefits accruing to the winners:

We do not find that winning a lottery systematically confers any evident academic benefits. We explore several possible explanations for our findings, including the possibility that the typical student may be choosing schools for non-academic reasons (e.g., safety, proximity) and/or may experience benefits along dimensions we are unable to measure, but find little evidence in favor of such explanations. Moreover, we separately examine effects for a variety of demographic subgroups, and for students whose application behavior suggests a strong preference for academics, but again find no significant effects.

Because of the random selection, Cullen and Jacob can focus on the difference between schools to try to explain their results and not the difference between winners and losers. Why aren't the "good" schools improving outcomes for the lottery winners? Why isn't attending a good school enough to change your luck forever?

Obviously, because it's more complicated than that. Education reform must go beyond waving a magic wand and granting a parent's wish for a better school. Ask any principal in Milwaukee how easy it is to bring a poorly performing student up to grade level...it's hard work that isn't any easier just because the school has a good reputation or the word "private" in its name.

Now, private schools do have some ability to work with students' families in ways public schools do not. Usually this ability is what leads people to predict that private schools outperform public schools: if the parents or student aren't cooperating, the student doesn't get to stay and the school climate stays focused on high achievement.

However, another new report, this one from the Center on Education Policy, found that, controlling for family background and prior achievement, students attending private high schools performed no better than students in traditional public high schools on math, reading, science, and history achievement tests and were no more likely to attend college than public school students.

If accurate, this finding is truly disheartening. If the schools that are able to "choose" their students aren't achieving at a higher rate, and if winning the ability to choose a better school doesn't improve achievement, then how can school choice possibly be expected to be the tide that lifts all boats?

Again, it is complicated. There is too much variation in the school universe to paint all private schools, or all public schools, with the same brush. Some will be very good, some will disappoint, and some might even be damaging. The same is true for voucher winners. Some will maximize the opportunity, some will squander it, and some will try their best to no avail. It's time to acknowledge that there is no "gold standard" for education reform because schools and students cannot be standardized. Improvement will come classroom by classroom from the sweat of the brows of teachers and students working together. Reformers, and policymakers, must return focus to the work of the classroom.

PPF Pearls: Improving school choice

Professor Paul Hill of the University of Washington has called for stronger governmental oversight of school choice programs in the latest issue of Education Week, noting that "...when one school neglects a student or doesn't pay its bills, government agencies crack down on everyone. Schools need oversight that is capable and fair, not negligent."

Prof. Hill also recommends that choice schools "prove themselves on the same tests as other schools," since "early hopes that charter and voucher schools would be so obviously great that no finely calibrated outcomes measured would be needed to prove it have been dashed."

The Forum would agree with those recommendations, as our research over the past 10 years has consistently revealed a need for more accountability in school choice.

Our first study, issued in 1997, interviewed parents in Milwaukee and Cleveland, the first two cities with publicly-funded private school choice, and found they felt they needed more information about their schooling choices. We then conducted two public opinion surveys in Wisconsin and Ohio and found consensus among the general public for having an agency, either independent or governmental, oversee and collect information about choice schools and for requiring schools to provide to the public information such as test scores.

For a comprehensive understanding of the need for greater accountability in publicly-funded school choice, as well as recommended policy options for improving choice programs, read School Choice and the Question of Accountability: The Milwaukee Experience, which resulted from the Forum's many years of research on the Milwaukee Parental Choice Program.

UPDATE: Mike Ford of School Choice Wisconsin sent the following in response to this post. Mike is right that schools participating in the voucher program must now take standardized tests; however, these test scores will be used to study how the choice program performs in the aggregate as compared to MPS. Scores will not be reported on a school-by-school basis.

Anneliese,
Your blog post today, "Improving school choice," fails to mention the additional accountability in the Milwaukee Parental Choice Program (MPCP) since your book was published. Today, MPCP schools face a rigorous accreditation process, take nationally normed standardized tests as part of a longitudinal study (results of which will be reported to the state Legislative Audit Bureau and publicly released), and must meet a multitude of financial viability requirements. The barriers to enter and stay in the MPCP are significant, as evidenced by the number of schools removed from and prevented from entering the program.
Reasonable people can have differing opinion on whether this is enough accountability, but I think it would benefit your blog's readers to mention these new requirements.
Thanks for listening,
Mike

Finally, some accountability

Wisconsin's class size reduction program, SAGE, provides extra state aid to schools serving low-income students that maintain a maximum pupil/teacher ratio of 15:1 in K5-3rd grade classrooms. Apparently the Department of Public Instruction (DPI) has been granting waivers for schools unable to meet the specified ratios during their first year or two of participation. Representative Kitty Rhoades, co-chair of the Joint Finance Committee, got wind of the waivers and moved to audit the program, which failed to pass out of committee. The Milwaukee Journal Sentinel now reports that DPI will no longer grant waivers and has denied the waiver requests of several school districts that had previously received SAGE funds.

The extra aid offered by the SAGE program became even more attractive to school districts a year ago when the legislature passed SB 618, lifting the cap on the number of private school vouchers. As a political compromise, that new law also increased the per-pupil SAGE amount from $2,000 per low-income student to $2,250. The irony is that the voucher program, which provides $6,501 per low-income pupil, is almost completely devoid of educational accountability and does not specify any classroom practices or operations, at any rate. Based on the Forum's annual survey of voucher schools, half of participating voucher schools have pupil/teacher ratios larger than the 15:1 required by SAGE, including two schools with 27:1 ratios. Meanwhile, the voucher program has not been audited since the 1999-2000 school year.

Accountability for taxpayers is not a sometimes proposition. Audit threat or not, DPI should probably not have been granting class size waivers for a program intended to reduce class size. Regular audits are good practice, in general, and should be utilized as resources allow. Perhaps it's time for another audit of the voucher program.

But even more important, shouldn't we be sending a consistent message to our schools, both public and private? If lawmakers have decided that the optimal learning environment for low-income students is in a small class, then why not encourage this environment in the voucher program, which is specifically intended for low-income students?

"Shopping" for schools isn't a good metaphor

In this week's Crossroads section, the Journal Sentinel hosted another community issues round table discussion, this time about K-12 education. Interestingly, this week one invited guest at the round table was unable to attend, so he was given room to write his own editorial. Brother Bob Smith, President of Messmer Catholic Schools, has this to say about K-12 education:

[The] Milwaukee Parental Choice Program. . . is one of the jewels of our city and state and is talked about and replicated in cities, states and countries around the world. . . This is one of the few times in life in which people with little money can go into a store full of school options and pick what is best for their child. This provides the poor with the same educational choices as the rich. [Emphasis mine.]

The school choice program does indeed open up many options for low-income students, but unfortunately there is no such "store." Parents cannot get information about their private school options anywhere other than the 124 independently run schools themselves. The only place they can even get a list of all the participating schools by grade level and religion is from the Public Policy Forum. And that's about all the information we can provide them. Any parent hoping to get information about student achievement, graduation rates, curriculum, or even after-school programs had better gas up or get a bus pass because these schools are located all across the city. Which means Brother Bob is reaching when he says, "Around Wisconsin, [the voucher program] is parallel to open enrollment for public schools."

Parents interested in open enrollment can simply visit the state's schooling information website to view, and compare, data on all of Wisconsin's public schools, including charter schools. Parents interested in private schooling are not offered this convenience. In fact, over a year ago PAVE, one of the original voucher program proponents, began, for the first time since the program started in 1990, an initiative to collect and disseminate private school information in a parent-friendly way. That effort has since been abandoned.

The result of this opacity? A greater burden on low-income parents. When Brother Bob says the poor have the same educational choices as the rich, he's only half right. The poor could have the same options as the rich, if they knew what they were. In reality, the rich have many more options, mainly because the opportunity costs for them to research schools are much lower. The poor, we have found via survey work, depend on word-of-mouth referrals when choosing schools. Thus, we end up with a choice program in which nearly a third of the schools enroll 99% or more voucher students. Or, in other words, schools that no rich students have chosen.

Social experiments as good public policy

New York City Mayor Michael Bloomberg recently announced a new privately-funded program aimed at moving families out of poverty. The program, Opportunity NYC, is being called a conditional cash transfer program. The basic gist is that participating families will work toward a series of goals and will receive cash bimonthly for each goal they meet. After a year a family could earn up to $5,000.

It will be interesting to see how this program is received. Most of the goals are short-term (school attendance, doctor and dentist visits, job training activities), but it is long-term planning and goal setting that will be the key to truly lifting these families out of poverty, I would guess. It's not clear how many years the families will be in the program, nor how long they anticipate the funding will last. Opportunity NYC is modeled on a cash transfer program in Mexico that has been shown to be effective. Whether small cash rewards provide the same incentive in New York City as they do in Mexico is yet unknown.

The most encouraging thing about this program, however, is that it is designed with a built-in evaluation, so that its effectiveness can be analyzed scientifically. (There will be a comparable control group of families who will work toward the goals but not receive the cash.) If a government is going to try a new and innovative program, even if it is privately funded, ensuring there will be a rigorous evaluation is important. If it works and is cost-effective, great...bring it to scale. If it doesn't, it is unfortunate, but better to know than to expose even more families to it.

This is a lesson Milwaukee and Wisconsin should learn. Innovation in public policy can be desirable, but without evaluation from the get-go, we don't know whether we should invest more or try another option. It shouldn't matter whether the innovation is publicly or privately funded; taxpayers want to know they are making wise investments just as much as foundation officers do. Evaluating after several years and even more millions are spent is not good public policy. . . no matter what the results of the underlying program turn out to be.