Book Review: 'Charter Schools: Hope or Hype?'
Buckley, J. & Schneider, M., Princeton University Press: Princeton, 2007
Review author Marc Holley
Supporters of charter schools may
be disappointed by some of the findings presented in this book, but the authors
do leave a little room for hope.
Buckley and Schneider’s objective
is to bring empirical evidence to bear on the question of whether charter
schools are an effective educational reform. The authors explain that charter
schools are public K-12 educational institutions which operate under a contract
that provides them with greater operational autonomy in exchange for greater
accountability regarding educational outcomes.
The theory of charter schools
rests on the assumption that increasing parental choice will lead to higher
levels of parent participation, parent and student satisfaction, and student
achievement. However, the authors note that charter schools may have negative
consequences, such as social fragmentation and stratification, which can
undermine some of the positive public benefits that may come with traditional
public schools.
Buckley and Schneider limit their
own research to examination of the charter school movement in Washington, D.C.
They argue that this city provides a sufficiently large and varied sample for
research purposes because approximately 20 percent of D.C. students attend
charter schools. Among the many research questions that Buckley and Schneider
explore are: 1) How are charter school students different from traditional
public school students; 2) Are charter school parents more satisfied; and 3) Are
charter schools performing better than traditional public schools?
In addition to their extensive
review of charter school literature, the authors employed two creative
data-collection methods of their own. They describe their first tool as a
four-wave panel survey, in which the authors interviewed a sample of parents of
children attending D.C. charter schools and others with children in traditional
D.C. public schools. Buckley and Schneider also interviewed students in each
setting. The panel nature of their survey means that they tracked the same
individuals over time.
Their second research instrument
was a Web site designed to follow parents as they sought information online
about schools. By tracking parental search patterns, the authors collected data
on the extent to which parents value proximity, demography of the student body,
student achievement and other factors when making choices about schools. To
confirm their hypotheses about whether conclusions could be drawn from observing
Web site use, the authors also followed up with parents who had accessed the
site.
In studying charter schools, these
authors, like other researchers, faced a challenging set of research and data
limitations. First, the authors point out that self-selection bias is a serious
threat to their research. The problem is that no matter how well-matched the
comparison group is on observable characteristics, charter school parents and
students are systematically different from traditional public school parents.
That is, the fact that they were willing to choose sets them apart.
Second, Buckley and Schneider
admit significant attrition of their sample. Unfortunately, the potential to
draw conclusions based on their diligently created panel data was weakened by
the loss of a large portion of their data over time.
Third, the authors concede that
the study of charter schools is threatened by the difficulty in separating the
charter treatment effect from other related school variables. For example, as
they note, it is not possible to control for the differences in school size when
making comparisons between charter schools and traditional public schools. Small
size is an inherent characteristic of charter schools, so it becomes impossible
to determine if positive effects result from size or the fact that a school is a
charter school.
Despite the fact that the authors
use all the tools in their statistical bags, all the limitations in studying
these schools, when taken together, appear to make for cautious conclusions. In
answer to the first question, above, which asks how charter and traditional
public school students differ, Buckley and Schneider find that charter school
students may have observable differences but that they are no more
difficult-to-educate than their peers. In other words, the authors are saying
that charter schools are neither “creaming” nor “bottom-skimming.” On question
two, they found that charter parents were more satisfied than traditional school
parents.
For the third
question about charter school performance, the authors do not find a
statistically significant difference in performance between charter and
traditional public schools in the majority of their analyses. However, in one
model they do “find some evidence that the traditional schools are outperforming
their charter counterparts.” It is important to note that these findings are
not based on gains scores, but on cross-sectional data. Generally, school choice
research which uses cross-sectional data tends to show negative or null effects,
whereas research with longitudinal achievement data tends to reveal a positive
charter effect.
The authors are
correct in asserting that a better way to explore charter school impacts on
student achievement would be to conduct random assignment studies in scenarios
of over-subscription to charter schools. In other words, to determine whether
charter schools are the cause of academic improvements, researchers would
benefit from studying cases in which there are more potential charter students
than seats available. Researchers could then compare the outcome among students
assigned to charter schools (by lottery) with those who are not. Buckley and
Schneider note that the U.S. Department of Education has recently commissioned
two five-year random assignment studies, so interested readers should keep an
eye out for those.
Perhaps the best way for
researchers to determine if education markets can really work is to have
universal school choice. As Buckley and Schneider note, charter schools involve
an “option-demand” choice, which means that parents must “choose to choose.”
This self-selection issue is what makes research so problematic. By contrast, in
a universal school choice scenario, all parents must choose. If all parents
must choose schools, in addition to the theorized benefits of a more pure market
approach, researchers can more fairly examine the choices parents make and the
effectiveness of schools.
Ultimately, supporters of charter
schools may be disappointed, for at the heart of the authors’ conclusions is:
“Too many of the facts we have documented in our research suggest that charter
schools, on the whole, are falling short, at least viewed through the eyes of
the students and parents who are their customers … .”
Still, Buckley and Schneider do
offer some hope of their own: “We have found that on average charter
schools do no harm and in fact have the potential for doing good in many
critical areas such as building social capital, increasing customer
satisfaction, and enhancing the civic skills of students.”
For those wanting an interesting
read on charter schools, this book is certainly a well-written and thoughtful
consideration of the related issues. Some of the discussions are relatively
technical, but readers with varying backgrounds in statistics will still find
many of the arguments accessible.
###
Marc Holley is a doctoral fellow at the University of
Arkansas’ Department of Education Reform and an adjunct fellow with the Mackinac
Center for Public Policy. Permission to reprint in whole or in part is hereby
granted, provided that the author and the Center are properly cited.