The misguided exam school debate

Selective public high schools have few Black and Hispanic students — there's also no evidence these schools are any good

(Spencer Weiner/Los Angeles Times via Getty Images)

San Francisco’s Board of Education has been in the news lately, thanks largely to a bizarre school renaming spree that didn’t just target hard-core racists, but also Abraham Lincoln — who was bad on Native Americans — and Paul Revere, who is said to have led an expedition aimed at suppressing the Penobscot Nation of indigenous people in Maine. The Penobscot Expedition was in fact directed at British forces who’d occupied the town of Castine (I visited a relevant fort with my 5-year-old over the summer) but in an interview with the New Yorker’s Isaac Chotiner, Gabriela López, the board’s chair, was unrepentant.

San Francisco’s public schools also remain closed due to the pandemic, contrary to the advice of most public health agencies. López did herself no favors by explaining that this is fine because even without in-person instruction, her students “are learning more about their families and their culture spending more time with each other. They’re just having different learning experiences than the ones we currently measure. And the loss is a comparison to a time when we were in a different space.”

Last on the list of woke excesses, the city is looking to change up Lowell High School’s admissions policies, which are currently based on standardized tests and grades and lead to very few Black or Latino students being admitted.

The San Francisco Chronicle slammed this decision; meanwhile, a long-standing controversy surrounding New York City’s exam-based high schools and related changes at an exam-based school in Fairfax County have all provoked similar arguments.

On the one hand, there’s a view that these schools promote segregation and undermine critical racial justice goals. On the other hand, there’s the view that this is a form of woke politics run amok that is ruining meritocratic institutions. Hanging awkwardly overhead is the fact that in almost all of these cases, the most salient demographic overrepresentation is of people from Asian backgrounds rather than white. And I think it is right to look a bit askance at a political system that chooses to address racial inequity by targeting what are often immigrant families from Asia rather than considerably more privileged white ones.

That said, I’ve tried to look at this issue several times, and every time I do, what I come up with is that there’s no evidence that selective schools actually benefit their students. Rather than leaving them in place or changing their admissions policies to somehow fudge the fact that different groups have different test scores, I think we ought to just get rid of them.

The false promise of exam schools

It’s obvious if you speak to parents — or just participate in society — that people generally have a strong intuitive belief in the power of peer effects. When someone refers casually to a place having “good schools,” what they typically mean is that the schools draw from students in upscale neighborhoods where the parents are well-educated and the kids have high test scores. They’re not using the phrase “high-productivity schools,” in which the kids learn more than you would expect based on their demographic background. I think that’s because people believe that being surrounded by good students will help their kid succeed.

At any rate, around the country and around the world, you often see a strategy of giving everyone a test and then taking the kids who do really well on the test and putting them in a school together.

New York City actually takes this to an extreme degree — kids from the Bronx will commute all the way down to Brooklyn Tech while other kids from Brooklyn commute all the way up to Bronx Science. Those schools are number two and number three in the city’s hierarchy of exam-based schools (number one is Stuyvesant in Manhattan), and the logic of sorting is considered so important that it trumps basic common sense about commuting patterns.

The theory, at least implicitly, seems to be that the highly selective schools will also be highly productive. But the evidence says they are not.

• Dobbie and Fryer (2013) look at New York City, comparing kids whose test scores are just-above versus just-below the cut points, and find that “exposure to these higher-achieving and more homogeneous peers has little impact on college enrollment, college graduation, or college quality.”

• Abdulkadriglu, Angrist, and Pathak (2014) look at Boston and find that “the marked changes in peer characteristics at exam school admissions cutoffs have little causal effect on test scores or college quality.”

• Barrow, Sartain, and de la Torre (2020) look at an effort in Chicago to give a boost to lower-scoring kids who came from high-poverty neighborhoods and find that “for students from low-SES neighborhoods, we estimate negative effects on grades and the probability of attending a selective college.” 

The hypothesis that affirmative action actually hurts Black students has been frequently offered in the context of college admissions, and it appears to be untrue, making the finding on Chicago high schools striking. A perhaps relevant issue is that Dale and Krueger show that attending a highly selective university is generally not beneficial except for students from disadvantaged backgrounds — another point of contrast to high school, where there seems to be no benefit regardless of background.

It’s not entirely clear why college and high school are different in this regard, but my guess is it has something to do with networking. If you’re from an underprivileged background, going to a Fancy College and meeting other Fancy College People plausibly has some concrete labor market benefit. But in high school, you’re better off (at least in terms of college acceptance and later test scores) being at the top of your class in a poor neighborhood in Chicago than being at the bottom of your class at an exam school.

The upshot of this is that cities tweaking their admissions policies to try to generate more diversity are probably making a mistake and will fail in their goal of helping Black and Latino kids. But it’s not exactly the mistake their critics think they are making, where they are going to somehow ruin the good schools and mess things up for the families they currently serve. We’d just arguably be better off without these schools at all so that everyone could have a more convenient commute.

Gifted and talented programs can maybe do some good

This has not been as much in the air recently, but you see somewhat similar questions around admission to “gifted and talented” programs that usually start at younger ages.

Here, though, David Card and Laura Giuliano show that more reliance on testing might improve racial equity by finding talented students from underrepresented backgrounds that are being overlooked in the traditional referral-based system. They looked at one large school district that shifted from a system of just testing kids who were recommended by teachers to one where they tested everyone, and they found that “blacks and Hispanics, free/reduced price lunch participants, English language learners, and girls are all systematically ‘under-referred’ in the traditional parent/teacher referral system.”

So that’s something you can do that’s beneficial.

But it’s still not clear that sifting and sorting students like this actually accomplishes anything. Bui, Craig, and Imberman “find that achievement for marginal students neither improves nor worsens from GT services in the short run.”

March, Chessor, Craven, and Roche find that being sorted into a gifted and talented program ends up making students feel less academically competent.

A separate Card and Giuliano study finds that for the high-IQ kids who make up the bulk of the gifted and talented population, the G&T program doesn’t help. But, there’s a small group of kids who were just shy of the IQ threshold and were let in on the basis of the quality of their schoolwork. Those kids did better with the positive impact “concentrated among lower-income and black and Hispanic students.”

Rachana Bhatt finds that gifted and talented programs increase math scores. But, in a separate paper, she cautions that these programs are administered in wildly different manners from place to place, and you probably can’t make valid generalizations.

What are we doing here?

As I argued in “Culturally Relevant Pedagogy is Good,” it’s dangerous to let education policy become consumed by our culture war hangups.

The fact that just about everyone has been to school and thus has first-hand knowledge of it only makes the situation more dangerous. The same Barrow, Sartain, and de la Torre study cited above showed that while Chicago’s move to bring a more diverse group into its exam school hurt the people it was supposed to help, they subjectively enjoyed their high school experience more. The policy was supposed to benefit them, and the students felt that it did, and they would no doubt recommend that your city make the same change. And yet, it didn’t help academically.

The only studies I can find where strict sorting at the secondary school level is helpful are from much poorer countries where it seems like they genuinely don’t have the resources to run proper high schools for everyone.

The gifted and talented literature, by contrast, does indicate some scope for identifying promising youngsters from disadvantaged backgrounds and some prospect of running programs that actually help students. But it’s fairly murky. I think we can say with confidence that universal testing will generate a less discriminatory outcome than letting subjective recommendations drive the bus.

Ultimately, though, fussing over exactly how to sort the best students seems to me like an awful case of chattering class navel-gazing. The much bigger policy issue is finding scalable ways to improve the quality of instruction across the board. I’ve become pretty convinced by the evidence in favor of phonics and “direct instruction” as a superior method to teach kids to read (see John McWorter if you like argumentative takes or Dana Goldstein if you prefer an elegantly reported narrative), which is something that in principle we could do everywhere. It also seems that dumb stuff like putting air conditioners in schools helps a lot.

Education is hard

Long story short, I think the argument around exam schools is misguided. The reformers aren’t going to help the people they think they are going to help, but the noble fight to save these schools is also not helping anyone. The real problem here (along with mangling the history of the Penobscot Expedition) is that every day spent having basically symbolic fights is a day you’re not spending doing real evaluations of whatever programs your city has in order to see which of them are working and which aren’t.

Student and parent intuitions about what is and isn’t an effective instructional program are an incredibly poor guide, and if your district isn’t trying to measure and test, then it’s not going to end up doing a good job just by accident.

Fighting about admissions policies at specific prestigious schools feels good because it’s easy to get your arms around and measure whether you succeeded in tweaking the demographics. But it’s not just that there isn’t much reason to believe that fussing with the diversity metrics will accomplish very much here — there isn’t much reason to believe that these high-prestige schools are even any good at teaching! If you want to effectively improve the education system’s results, you need to put in some genuinely hard work to try to understand what ideas are actually promising. The fact that you can spin up huge political controversies over who gets access to schools that may not even be worth going to shows how easy it is to go astray here.