I am curious about the impacts to learning from technology advancement. Simple example, does the ubiquitous of spell check have negative impacts on the average persons spelling? I think my spelling has gotten worse because the software now will automatically tell me how to spell something if I’m close.
I have started using GPT-4 like this but for entire sentences and paragraphs. I write what I want to say and then I ask my new friend to write it better.
Why get really good at something when the AI will always do it better? It makes one wonder what we should be teaching? People who create the greatest leverage will have the most advantage.
I think “schools don’t matter” and “better schools can improve student outcomes by 0.05SD or 6% here and there” are entirely compatible positions. Freddie DeBoer’s thesis is that tweaking what happens in classrooms cannot bring about equality in society, and given the depth of existing inequality and these minuscule effect sizes, that view seems vindicated.
This statement exaggerates schools' impact, as the linked paper actually compares 80th percentile schools to 20th percentile schools (not the mean): "Low-income students who attended a high school at the 80th percentile of quality were 6 percentage points more likely to earn a bachelor’s degree and earned 13% more money (or about $3,600) per year at age 30."
In March of 2020 I predicted the pandemic would result in increased achievement gaps.
The irony is for a small group of students, online school was better. If you were motivated, you were no longer held back by the pace of slower learners.
I like really want much deeper data than I’ve seen anywhere.
Anecdotally my students who were online with me the whole year for an entire school day live seemed to do okay a tiny dip less growth in their scores in 2nd followed by grade leading growth in 3rd.
My deep suspicion is most of the loss was concentrated in highly asynchronous classes, hybrid models and non-attenders also noting that the home arrangements you need for 6.5 hours of Google Meet are more strict. But I’ve not seen anyone who focuses on result by modality.
Was gonna reach for that "like" button, the first I've ever granted to a non-Milan guest post (I think), but...then you did the same thing Matt does, mischaracterizing FdB's stance based on what seems to be an overly-literal surface level read. This runs both ways, with Freddie continually dunking on Matt's YIMBYism...based solely on his tweets, seemingly. Not sure where this historical beef comes from, but as someone who subscribes to both blogs, it's very weird.
...building on FdB's first book, "The Cult of Smart", available at your friendly local bookstore.
Obviously no one's gonna wade through all that just on my say-so; I'll try and summarize one last time: Freddie doesn't claim schools "don't work" writ large, actually he goes to great pains showing that of course there's absolute gains from school, and yes a lot of that came from successful reform. I notice that you've grokked his distinction between absolute and relative gains, which is one step in the right direction. But even here, the argument is not that disparate relative gains ("gaps") never change - as you point out, research is quite clear on this, and anyway it follows naturally from absolute values changing. It'd be a very strange situation if absolute shifts just happened to coincidentally line up along the current relative-gap lines.
Here's the rub though: even though we can move the needle here and there, sometimes even in meaningfully big ways, the bulk of those gaps will always come from outside school. Like all other measures, school reflects our society's inequities back at us, so at some point one must change society. These are exogenous factors which school can't do much to change. (Wraparound services attempt to ameliorate somewhat, and are even sometimes kinda successful e.g. school lunch, but there's plenty of reasons to be wary of "mission creep" here. Iron Law of Instutitions, Law of Comparative Advantage, credentialism, etc.) Given this plateau, we ought to be really clear about what school can and cannot do...that is, think a bit like an EA, and realize there are probably better ways to improve those long-term outcomes than indirectly via school's diminishing returns. Get everyone through highschool, sure, get everyone through college? Hold up.
That was the point of the optimism-bias post, that "school" has become this way to launder hopes and dreams, a proxy battle that sucks energy out of larger debates. (School lunches work, but wouldn't be necessary if we actually solved hunger in America, which is totally within our power and yet we don't. Incremental gains are great, but don't forget to think bigger!) Ultimately, focusing on improving this correlate to long-term outcomes is a type of Goodharting*. Which isn't to say it doesn't have its own unique and intrinsic benefits - I would not want every kid to be "unschooled", and it's a load-bearing part of the current compromise on welfare (e.g. school-as-childcare, freeing up parents for more work hours). The question is, what is schooling for, what role do we want it to play in society? Because we're really muddled on that direction right now, to everyone's detriment.
I think I'll sit out of future education-related posts if they do the same thing. Too much talking-past, which is unfortunate cause I don't see the positions as contradictory. Not SB's strongest topic.
> School lunches work, but wouldn't be necessary if we actually solved hunger in America
Well… they would, because you can't "solve hunger" in a way that stops people from being hungry again at lunchtime, which is when they're in school.
My memory of FdB's article is that he definitely sounded like he was arguing no school intervention could ever change the relative achievements of students. (But I can think of one - take the highest performing students and hit them on the head until they perform worse.)
I propose putting nootropics in the school lunches and seeing if that causes any more absolute improvements.
Nice article! It does make sense that reform would help somewhat and I think in general that the reform nihilism camp in general suffers from an all or nothing bias. Either the reform solves all problems or it isn't worth it. It would be nice to see a more in depth post into the actual evidence behind these reforms. The papers that you mention sound like they are essentially observational studies. Do they do anything to try to account for things like sample selecting and omitted variable bias? If not, it might be nice to caveat their limitations.
The observation here along the lines of “studies show similar interventions have very different results across time and place” *could* support his conclusion that these interventions are effective if precisely targeted. But it also (I think, more likely) could be an artifact of running lots of studies on ineffective interventions. Some results are positive, some are negative, no one can tell a clear story about the important variables.
In other words, ineffective interventions with no/random results will include positive results at the single-study level.
My kid attended what would be described as a bad school, and I think one major contributor is extremely high staff turnover. Most if not all here teachers are in there first year
And it's not surprising. If you are a good and desirable teacher, and you are paid the same no matter which school in the district you teach, then why not take the position teaching AP history at the prestige school in the suburbs.
I've always wondered if it would make sense to allow struggling schools to offer teachers a premium to work there?
Interesting and well-written but misses a number of important factors. First one has to understand why "reform" fails consistently and rarely continues as an educational culture. Second, one should never write or think about reforming American public education without taking into account white supremacy and the corresponding fear of urban public schools. Third, American public education is controlled and driven by regulatory compliance (and fear of regulatory sanctions), not any reasonable understanding of the best interests of the students.
First, there have been nearly constant waves of new approaches, new teaching methodologies and new tests since the 1950s. The faculty have learned to ride out the reform and wait for the next push from above. Nothing changes. That is one reason for the legislative and political push for national or state testing. Of course there should not be social promotion but testing is a heavy hand. Due to significant pressure on teachers to improve the scores of their students they understandably concentrate not on their most needy students but on the students whose scores they can most easily raise. This is studied and well known in the best graduate programs in education.
Second, the Confederate South abandoned public schools wholesale over integration. Public education in the South is as segregated - for the same reason - as it was before Brown v. Board of Education. Fear of dark-skinned people in cities pushed many white people to single-family home suburbs with better white majority students.
Third, Principals and other supervisory educators are largely measured and rewarded or punished based on test scores in schools under their supervision. Career interests and career insecurity drive compliance and tend to push down sanctions. No one is promoted for being a great academic leader or educator.
Fourth, for what it is worth, American educators are poorly screened and poorly trained. The best school systems worldwide screen carefully and train patiently to get the best teachers who are choosing and joining a profession for life.
You assume self-segregation is only and always White people moving away from Black people. But certainly the reverse happens, as well, and is a contributing factor. There wouldn't be HBUs otherwise.
You also seem to be ignoring the 35% of our students that are neither White or Black. How can you write about segregation and not even consider the very many students who don't factor into a White/Black binary, including even the many 10% of white and black students who are immigrants or children of immigrants?
I was talking about the big picture which is driven by interlocking factors. If you want to get into the weeds I would be happy to converse. For openers you do not seem to understand at least two background facts: New England is segregated by class more than race because the school systems are by town or city limits. The massive middle and high schools in Texas are educational disasters. Second, the HBCUs were founded because at the time of their founding no African American could enroll in a nominally public university in the South.
One of the trends that ought to be reexamined is the push for college attendance above all else and the propaganda efforts surrounding that in which college is the norm and if you’re not college bound then you’re some kind of loser.
Schools need to do a much better job of preparing and promoting other opportunities because the fact of the matter is that college isn’t the right choice for everyone and we have a growing problem of kids going to college, taking on debt and either not finishing or not getting a degree that gets them stable employment.
I run a tutoring program for public schools, working with close to 2000 tutors every week. I’d love to talk with people in industry to exchange perspectives.
Every couple days for a while now I’ve read an anecdote from a teacher that suggests that conditions in public schools across the country have basically collapsed? The kids can’t stop looking at their phones, half of them don’t know how to read, can’t be held back, teachers getting assaulted and the kid is back in the classroom the next day, etc. This seems like a huge story?
"It turns out that schools had a big, long-term effect on students. Low-income students who attended a high school at the 80th percentile of quality were 6 percentage points more likely to earn a bachelor’s degree and earned 13% more money (or about $3,600) per year at age 30."
The linked article makes it clear that this increase is in comparison to be students who attended a high school at the 20th percentile of quality. The effect is pretty small for going from 20th to 80th percentile.
And given that one of (if not the) main factor in school quality is the composition of the student body (and their parents), then there is going to be a hard limit on how many low-income students you can add to a given high-quality school before it is...no longer high-quality.
Additionally, the paper was limited to schools in Massachusetts. Which isnt inherently a problem, but my understanding is that MA is generally a significant outlier w.r.t. to education. So I'm not sure how useful it is.
The careful reasoning in this article will never percolate into the discourse because improving student outcomes by 0.05 standard deviations won’t get anyone excited.
The response to the claim that "almost nothing moves the needle in academic outcomes" is to point to a "0.05 standard deviation [improvement] per decade”?!?!
I am at least happy to see someone acknowledge (albeit handwavingly rather than directly) that we shouldn't judge progress by inputs like class size, funding, etc. but rather by their outputs--actual student improvement.
Are college enrollment rates not a function of wage growth in service industry jobs?
I am curious about the impacts to learning from technology advancement. Simple example, does the ubiquitous of spell check have negative impacts on the average persons spelling? I think my spelling has gotten worse because the software now will automatically tell me how to spell something if I’m close.
I have started using GPT-4 like this but for entire sentences and paragraphs. I write what I want to say and then I ask my new friend to write it better.
Why get really good at something when the AI will always do it better? It makes one wonder what we should be teaching? People who create the greatest leverage will have the most advantage.
I think “schools don’t matter” and “better schools can improve student outcomes by 0.05SD or 6% here and there” are entirely compatible positions. Freddie DeBoer’s thesis is that tweaking what happens in classrooms cannot bring about equality in society, and given the depth of existing inequality and these minuscule effect sizes, that view seems vindicated.
This statement exaggerates schools' impact, as the linked paper actually compares 80th percentile schools to 20th percentile schools (not the mean): "Low-income students who attended a high school at the 80th percentile of quality were 6 percentage points more likely to earn a bachelor’s degree and earned 13% more money (or about $3,600) per year at age 30."
In March of 2020 I predicted the pandemic would result in increased achievement gaps.
The irony is for a small group of students, online school was better. If you were motivated, you were no longer held back by the pace of slower learners.
I like really want much deeper data than I’ve seen anywhere.
Anecdotally my students who were online with me the whole year for an entire school day live seemed to do okay a tiny dip less growth in their scores in 2nd followed by grade leading growth in 3rd.
My deep suspicion is most of the loss was concentrated in highly asynchronous classes, hybrid models and non-attenders also noting that the home arrangements you need for 6.5 hours of Google Meet are more strict. But I’ve not seen anyone who focuses on result by modality.
Was gonna reach for that "like" button, the first I've ever granted to a non-Milan guest post (I think), but...then you did the same thing Matt does, mischaracterizing FdB's stance based on what seems to be an overly-literal surface level read. This runs both ways, with Freddie continually dunking on Matt's YIMBYism...based solely on his tweets, seemingly. Not sure where this historical beef comes from, but as someone who subscribes to both blogs, it's very weird.
The article in question: https://freddiedeboer.substack.com/p/education-commentary-is-dominated
The larger meat behind that bone: https://freddiedeboer.substack.com/p/education-doesnt-work-20
...building on FdB's first book, "The Cult of Smart", available at your friendly local bookstore.
Obviously no one's gonna wade through all that just on my say-so; I'll try and summarize one last time: Freddie doesn't claim schools "don't work" writ large, actually he goes to great pains showing that of course there's absolute gains from school, and yes a lot of that came from successful reform. I notice that you've grokked his distinction between absolute and relative gains, which is one step in the right direction. But even here, the argument is not that disparate relative gains ("gaps") never change - as you point out, research is quite clear on this, and anyway it follows naturally from absolute values changing. It'd be a very strange situation if absolute shifts just happened to coincidentally line up along the current relative-gap lines.
Here's the rub though: even though we can move the needle here and there, sometimes even in meaningfully big ways, the bulk of those gaps will always come from outside school. Like all other measures, school reflects our society's inequities back at us, so at some point one must change society. These are exogenous factors which school can't do much to change. (Wraparound services attempt to ameliorate somewhat, and are even sometimes kinda successful e.g. school lunch, but there's plenty of reasons to be wary of "mission creep" here. Iron Law of Instutitions, Law of Comparative Advantage, credentialism, etc.) Given this plateau, we ought to be really clear about what school can and cannot do...that is, think a bit like an EA, and realize there are probably better ways to improve those long-term outcomes than indirectly via school's diminishing returns. Get everyone through highschool, sure, get everyone through college? Hold up.
That was the point of the optimism-bias post, that "school" has become this way to launder hopes and dreams, a proxy battle that sucks energy out of larger debates. (School lunches work, but wouldn't be necessary if we actually solved hunger in America, which is totally within our power and yet we don't. Incremental gains are great, but don't forget to think bigger!) Ultimately, focusing on improving this correlate to long-term outcomes is a type of Goodharting*. Which isn't to say it doesn't have its own unique and intrinsic benefits - I would not want every kid to be "unschooled", and it's a load-bearing part of the current compromise on welfare (e.g. school-as-childcare, freeing up parents for more work hours). The question is, what is schooling for, what role do we want it to play in society? Because we're really muddled on that direction right now, to everyone's detriment.
I think I'll sit out of future education-related posts if they do the same thing. Too much talking-past, which is unfortunate cause I don't see the positions as contradictory. Not SB's strongest topic.
*And of course this also leads to Goodharting the Goodharting: https://educationrealist.wordpress.com/2021/09/18/false-positives/
> School lunches work, but wouldn't be necessary if we actually solved hunger in America
Well… they would, because you can't "solve hunger" in a way that stops people from being hungry again at lunchtime, which is when they're in school.
My memory of FdB's article is that he definitely sounded like he was arguing no school intervention could ever change the relative achievements of students. (But I can think of one - take the highest performing students and hit them on the head until they perform worse.)
I propose putting nootropics in the school lunches and seeing if that causes any more absolute improvements.
Nice article! It does make sense that reform would help somewhat and I think in general that the reform nihilism camp in general suffers from an all or nothing bias. Either the reform solves all problems or it isn't worth it. It would be nice to see a more in depth post into the actual evidence behind these reforms. The papers that you mention sound like they are essentially observational studies. Do they do anything to try to account for things like sample selecting and omitted variable bias? If not, it might be nice to caveat their limitations.
The observation here along the lines of “studies show similar interventions have very different results across time and place” *could* support his conclusion that these interventions are effective if precisely targeted. But it also (I think, more likely) could be an artifact of running lots of studies on ineffective interventions. Some results are positive, some are negative, no one can tell a clear story about the important variables.
In other words, ineffective interventions with no/random results will include positive results at the single-study level.
My kid attended what would be described as a bad school, and I think one major contributor is extremely high staff turnover. Most if not all here teachers are in there first year
And it's not surprising. If you are a good and desirable teacher, and you are paid the same no matter which school in the district you teach, then why not take the position teaching AP history at the prestige school in the suburbs.
I've always wondered if it would make sense to allow struggling schools to offer teachers a premium to work there?
Actually dealing with student behaviorial issues is probably high up the list.
Interesting and well-written but misses a number of important factors. First one has to understand why "reform" fails consistently and rarely continues as an educational culture. Second, one should never write or think about reforming American public education without taking into account white supremacy and the corresponding fear of urban public schools. Third, American public education is controlled and driven by regulatory compliance (and fear of regulatory sanctions), not any reasonable understanding of the best interests of the students.
First, there have been nearly constant waves of new approaches, new teaching methodologies and new tests since the 1950s. The faculty have learned to ride out the reform and wait for the next push from above. Nothing changes. That is one reason for the legislative and political push for national or state testing. Of course there should not be social promotion but testing is a heavy hand. Due to significant pressure on teachers to improve the scores of their students they understandably concentrate not on their most needy students but on the students whose scores they can most easily raise. This is studied and well known in the best graduate programs in education.
Second, the Confederate South abandoned public schools wholesale over integration. Public education in the South is as segregated - for the same reason - as it was before Brown v. Board of Education. Fear of dark-skinned people in cities pushed many white people to single-family home suburbs with better white majority students.
Third, Principals and other supervisory educators are largely measured and rewarded or punished based on test scores in schools under their supervision. Career interests and career insecurity drive compliance and tend to push down sanctions. No one is promoted for being a great academic leader or educator.
Fourth, for what it is worth, American educators are poorly screened and poorly trained. The best school systems worldwide screen carefully and train patiently to get the best teachers who are choosing and joining a profession for life.
You assume self-segregation is only and always White people moving away from Black people. But certainly the reverse happens, as well, and is a contributing factor. There wouldn't be HBUs otherwise.
You also seem to be ignoring the 35% of our students that are neither White or Black. How can you write about segregation and not even consider the very many students who don't factor into a White/Black binary, including even the many 10% of white and black students who are immigrants or children of immigrants?
I was talking about the big picture which is driven by interlocking factors. If you want to get into the weeds I would be happy to converse. For openers you do not seem to understand at least two background facts: New England is segregated by class more than race because the school systems are by town or city limits. The massive middle and high schools in Texas are educational disasters. Second, the HBCUs were founded because at the time of their founding no African American could enroll in a nominally public university in the South.
That's what your report says. Segregation is highest in the Northeast and most of that is due to public school district lines.
One of the trends that ought to be reexamined is the push for college attendance above all else and the propaganda efforts surrounding that in which college is the norm and if you’re not college bound then you’re some kind of loser.
Schools need to do a much better job of preparing and promoting other opportunities because the fact of the matter is that college isn’t the right choice for everyone and we have a growing problem of kids going to college, taking on debt and either not finishing or not getting a degree that gets them stable employment.
I run a tutoring program for public schools, working with close to 2000 tutors every week. I’d love to talk with people in industry to exchange perspectives.
I think it has a lot of potential:
- a lot of college student and retirees are interested in it
- many programs are after school so they are compatible with part time work or having two jobs
- in large urban areas tutors often live near the schools, so they share socio económico context with the students
- tutoring is 1-1 to 1-5, so kids get to know an adult that can be a role model to them, who tells them learning is important
- a lot of the challenges in teaching young children are around them paying attention. Small group settings help a lot with that.
Where do you live
Every couple days for a while now I’ve read an anecdote from a teacher that suggests that conditions in public schools across the country have basically collapsed? The kids can’t stop looking at their phones, half of them don’t know how to read, can’t be held back, teachers getting assaulted and the kid is back in the classroom the next day, etc. This seems like a huge story?
Good luck!
This is maybe a nitpick, but...
"It turns out that schools had a big, long-term effect on students. Low-income students who attended a high school at the 80th percentile of quality were 6 percentage points more likely to earn a bachelor’s degree and earned 13% more money (or about $3,600) per year at age 30."
The linked article makes it clear that this increase is in comparison to be students who attended a high school at the 20th percentile of quality. The effect is pretty small for going from 20th to 80th percentile.
And given that one of (if not the) main factor in school quality is the composition of the student body (and their parents), then there is going to be a hard limit on how many low-income students you can add to a given high-quality school before it is...no longer high-quality.
Additionally, the paper was limited to schools in Massachusetts. Which isnt inherently a problem, but my understanding is that MA is generally a significant outlier w.r.t. to education. So I'm not sure how useful it is.
i think they tried to regress out the ses factors in making that statement
The careful reasoning in this article will never percolate into the discourse because improving student outcomes by 0.05 standard deviations won’t get anyone excited.
The response to the claim that "almost nothing moves the needle in academic outcomes" is to point to a "0.05 standard deviation [improvement] per decade”?!?!
I am at least happy to see someone acknowledge (albeit handwavingly rather than directly) that we shouldn't judge progress by inputs like class size, funding, etc. but rather by their outputs--actual student improvement.