It's just incredible how "accountability" has failed again and again and again and yet still gets endorsed as a solution to our education woes. (Which, to be clear, are actually socioeconomic problems concentrated among a thin slice of students; America's median student does pretty good and our best are the envy of the world.) "Accountability" didn't work in Houston or Newark or New Orleans or DC. Our country has become an elephant's graveyard of failed accountability efforts in education. It doesn't work! There was no Texas miracle and in time there will prove to have been no Mississippi miracle either.
It is just so mindboggling to listen to Lovett fall into the same tired "we just need to demand better outcomes!" school of ed reform rhetoric. Academic ability is largely genetic and largely fixed. There is a normal distribution of that ability in the country's population. One half will always be below average. One quarter will always be in the bottom quartile. Failure is literally inevitable. This conversation is like listening to people who were in cryofreeze for the past 25 years of ed policy.
There does in fact appear to be actual real improvements in Mississippi and the handful of of Southern states following similar methods.
They are not using charter schools. They are basically throwing a ton of targeted support behind basic reading in public schools.
It’s not reading at “an implausibly high level.” It’s literally, learn basic reading skills in early elementary, with focused support, or be held back and get even more support.
The huge improvement in rank is most states dropping versus Mississippi making real but moderate gains, combined with the adjustment for demographics that pushes them up further.
Doesn’t appear so. The number of retentions I have seen reported is not unusually high, and they go through the testing again the next year, so if they’re illiterate, they show up again.
You really do want kids to be held back and helped if they can’t read. They do help, and they give kids support and a chance to be re evaluated before they hold them back.
It appears that targeting literacy, screening for obstacles like dyslexia, and providing help and tutoring are helpful. Which is not surprising.
what they’re talking about in Mississippi is improving basic literacy, not achievement gaps, not making everyone Stanford ready. It’s utterly plausible that moving to an evidence based curriculum and enforcing it can make absolute gains in basic literacy from a terrible baseline.
While you're correct about relative distributions, there is no excuse for an education system that produces "graduates" with grade school reading and math levels. So many American students aren't even approaching their academic potential, which may be mediocre, but even someone with an IQ of 75 or 80 should be able to get a 70 average and graduate high school. Especially in the districts where standards for doing so are basically (1) show up to class, (2) don't be disruptive, (3) turn your assignments in on time, and (4) work hard on studying for quizzes and exams. Even if we accomplish all that we have to face the hard reality than tens of millions of kids aren't college material, but they will have gotten some semblance of an education. Instead we are just passing them through school at 5x the cost and keeping the delinquents in the classroom to ruin the experience for many kids on the bubble.
And yet, under a different accountability regime, worse performing students did better than they are now!
Americans aren’t poorer (at any income slice) than they were when kids were performing better either.
I am not claiming accountability is ALL we need and it’s true that there are always going to be socioeconomic, genetic, and a host of other differences that prevent a classroom from being fully flat in performance, duh.
But those things haven’t changed in the direction needed to explain worsening test scores.
Also, in some (many? Idk) instances we know there are better ways to teach kids things and worse ways. Some/many school are doing it the worse way! Call it “accountability” while holding your nose if you want, but seems pretty obvious you could improve median outcomes if you stopped people from teaching stuff the worse way (which, in fact, is what happens)
Freddie, how do you account for the often large differential achievement gap between students randomly selected for a charter school and those in the mainline public schools?
It strikes me that this is a compelling “natural experiment” taking students with the same SES, background, family engagement, etc. and proving that they can have different results in a different environment.
That's more or less the entire charter story - whenever a charter school or system appears to be performing at an implausibly high level, there's inevitably a hidden selection bias resulting in a dramatically unrepresentative student body. Please note that simply having parents who have it together enough to apply to a charter school lottery is sufficient to break randomization, and they go much, much further than that.
But the lottery kids who "win" do better - the other parents were organised enough to apply but didn't get in. Surely that is a good example of mitigating selection effects in the data?
Dude, in this very podcast Matt said that achievement gaps didn't close because the higher level students did better. But the bottom students also did better.
We get it. You don't think absolute level of education matters because it won't redistribute the pie evenly. Most of us do think that increasing the absolute level of education matters even if it doesn't do that. Are more fourth graders not reading at a higher level in Mississippi than they did several years ago?
Everyone keeps pointing out that, if the Mississippi Miracle had happened in a blue state, we would be having a national conversation about it. It's interesting that nobody seems to think national Republicans have the ability -responsibility?- to care about this either. Democrats are expected to be the party with all the answers on education, urban policing, and rebuilding manufacturing, whereas national Republicans get a win if they even acknowledge these problems and don't deindustrialize us -too- much or eliminate -too- many cancer research centers.
Who says that Republicans don't care about education? Many have given up on the ability of public schools/teachers unions to provide it, but that doesn't mean that it's not important.
These things have spread from Mississippi to other Republican-led states, Mississippi's governor has talked about it, etc.
Is it just the case that the sorts of media that you consume doesn't overlap with the sorts of media where quotes from Republican governors are published?
I tried checking out the fox news front page on that day and I don't even see that article. Am I looking in the wrong place? Here's fox news US from the same day:
The front page is just a lot of stories about random crimes, and two pieces about abortion and trans people. And this article only has 7 comments compared to thousands on other articles. Does foxnews.com's own reader statistics and front page editor say that Republicans don't care about education?
Hey did the post before this one self-destruct or something? It was called "The left is right about the Democrats" but I cannot find it any more. It exists in my email, but clicking the link brings me to a "page not found"
I'm still interested to understand why American parents hate testing so much. The US has much less testing than most countries, and high stakes tests are almost completely optional. It seems odd to me that the US as a highly meritocratic country, has so much of kids' marks based on whether or not their teacher likes them or they can hassle their teacher for a better mark.
Something really stuck out to me listening to this though, which is that parents in the US don't like hearing negative things about their kids' schools. I feel that the UK's incredibly pessimistic attitude, which annoys me much of the time, actually makes school improvement much easier. Everyone in the UK thinks schools could do better and we're willing to hear about how bad our kids' schools are and how much better they should be.
I can't believe how much better primary education is for my kids than it was for me - the quality of writing, the fact that all the kids can read - it's a real testament to the consensus on the left and right in the UK on the importance of phonics, school accountability and a commitment to knowledge based curricula.
Part of it is standardized testing on a nationwide level is a fairly new thing.
I'm not super old (early 40s), but when I went to school, there was a statewide assessment test late in grade school, but it wasn't a high stakes thing. It was annoying to take 3 days out of the year to be bubbling in multiple choice scantrons, but it wasn't treated by the end of the world.
Even by time I was leaving high school, that had begun to change (note - I lived in Florida) in elementary and middle school and has only increased.
To a certain extent, the standardized tests are seen as something separate from the actual schooling.
Are there any high stakes standardised tests in the US though? There are SATS and the ACT but aren't they completely optional? Don't most other standardised tests in the US only have stakes for schools? Like if you fail your SATS you can go to a test-optional university, right? For me high-stakes implies stakes for the kids too - like in the UK if you fail your A-levels you're not going to university until you retake them, no matter how well you did on your homework the rest of year or how great you are at extra curriculars.
In the UK we have the phonics screening check in year 1 (moderate stakes for schools), SATS at the end of year 6 (high stakes, but only for the school), GCSEs at the end of Year 11, A-levels at the end of Year 13 (high stakes for schools & kids). On top of that we have regular high stakes inspections of schools from Ofsted.
Labour has just introduced a new Year 8 reading test (12/13 yo kids) too - again the only stakes are for the schools.
I suppose it kind of seems crazy to me that so many US parents really hate the standardised testing when it's so routine in the UK, with much more tests and much higher stakes.
The SAT or ACT isn’t required for people who want to go to “community college.” One is needed for those interested in attending a “directional state” college, state university or private college/university. My understanding is that which test is used varies from state to state. In my state (WA) we mostly take the SAT.
A kid generally needs a pretty solid SAT/ACT score for more selective schools, so in that regard it’s by far the highest stake test we take in my opinion.
There are tracking assessments that happen in K12 schools, but there’s no real standard there from one school district to the next as far as I know. Parents really hate this if/when their kid can’t be in honors or whatever. Additionally while your assigned public school can’t exit a kid due to lack of performance (due to Supreme Court decisions), private schools can.
I think the criticism you’re seeing is somewhat true, but maybe more so rooted in partisan politics. President George W. Bush and the Republicans started standardized testing under the No Child Left Behind Law. The idea was “a high school diploma should mean something.” I think that effort has mostly been abandoned now.
So I think one of the policy questions that frustrates me is why is accountability a one time thing.
Like I teach in 3rd grade in a retention year state. We don't really spend that much instructional time on phonics. There's like some skills to master but the really load bearing years for phonics instruction is first and kindergarten. Most of our retained students don't flourish their second time in the grade.
We give them a standardized test to measure what they learned but we only make a big fuss about it once. It feels pretty strange.
Accountability is not a one-time thing, if you mean testing. In many places in the United States public school students are still going through absolutely relentless census testing that takes up weeks of instructional time.
"Most of our retained students don't flourish their second time in the grade."
That should not be surprising; teachers and schools control somewhere between 1% and 7% of the variance in student outcomes in quantitative metrics.
I mean accountability that affects the students. In 3rd grade you either pass the test or you fail 3rd grade and take it again. Every other year basically nothing happens to you if you fail. We assign you some half baked intervention that's even more fucked than our primary curriculum.
Low achievers who aren't catastrophically low work really hard to avoid getting retained. My reading test felt like a minor miracle that on test day everyone passed.
Hey Freddie, you seem very knowledgeable about ed and I'm going to browse your substack. In the meantime could you post a source for your statistic about teachers/schools only controlling 1-7% of variance in student outcomes? I'm sure google could find some study along these lines but I'd like to know the one that was convincing to you. Thanks!
Some of these talks are more productive than others...this one felt like two ships passing in the night, who keep acknowledging that they're receiving the other's signal in the vicinity, yet continuously fail to see the other ship. I was left confused as to whether Jon or Matt actually agree on much of anything at all, or this was just an elaborate object lesson in popularism. Perhaps that would have been hashed out later on in the discussion, alas. (Even if one grants the premise that leftists have a state-owned monopoly on energy and charisma and heartfelt beliefs, the name of the game is still to win elections. Thought the GOP may have won all the battles, we had all the good songs...)
Also it's weird the URL for this one says "Jon Lovell". Someine didn't dot their i's and cross their t's!
If you want to get involved in a political fight involving better education in math that’s happening right now you should check out the petition against the recent New York briefs on teaching math. You can find a good discussion on the Chalk and Talk podcast. https://youtu.be/7pcZ7TexRQE?si=tHkVLYofF7hfD9-7
In general, there has been a lot of research on how to best teach math, and it’s been very hard to get public attention for it. Part of the challenge is multiple groups are claiming that their methods are scientifically backed. In my opinion Solomon and Stokke have the actually experiment backed research supporting them.
One size fits all education has as much of a chance of succeeding as does one size fits all medicine. If the time wasted on identity politics* was spent using AI algorithms in computers to be applied by teachers to achieve what teaching machines were supposed to achieve, each student would be guided to a state of being educated** at a pace that matched their ability. *I listened to my grandson's classes during the Covid-19 pandemic and experienced what they were exposed to. **reaching a state of being able to function independently in society.
It's just incredible how "accountability" has failed again and again and again and yet still gets endorsed as a solution to our education woes. (Which, to be clear, are actually socioeconomic problems concentrated among a thin slice of students; America's median student does pretty good and our best are the envy of the world.) "Accountability" didn't work in Houston or Newark or New Orleans or DC. Our country has become an elephant's graveyard of failed accountability efforts in education. It doesn't work! There was no Texas miracle and in time there will prove to have been no Mississippi miracle either.
It is just so mindboggling to listen to Lovett fall into the same tired "we just need to demand better outcomes!" school of ed reform rhetoric. Academic ability is largely genetic and largely fixed. There is a normal distribution of that ability in the country's population. One half will always be below average. One quarter will always be in the bottom quartile. Failure is literally inevitable. This conversation is like listening to people who were in cryofreeze for the past 25 years of ed policy.
There does in fact appear to be actual real improvements in Mississippi and the handful of of Southern states following similar methods.
They are not using charter schools. They are basically throwing a ton of targeted support behind basic reading in public schools.
It’s not reading at “an implausibly high level.” It’s literally, learn basic reading skills in early elementary, with focused support, or be held back and get even more support.
The huge improvement in rank is most states dropping versus Mississippi making real but moderate gains, combined with the adjustment for demographics that pushes them up further.
Aren’t they just holding back more of the worst students thus creating an illusion of better performance among the students who are not held back?
Doesn’t appear so. The number of retentions I have seen reported is not unusually high, and they go through the testing again the next year, so if they’re illiterate, they show up again.
You really do want kids to be held back and helped if they can’t read. They do help, and they give kids support and a chance to be re evaluated before they hold them back.
It appears that targeting literacy, screening for obstacles like dyslexia, and providing help and tutoring are helpful. Which is not surprising.
what they’re talking about in Mississippi is improving basic literacy, not achievement gaps, not making everyone Stanford ready. It’s utterly plausible that moving to an evidence based curriculum and enforcing it can make absolute gains in basic literacy from a terrible baseline.
>Academic ability is largely genetic and largely fixed
I don't think you understand what the word 'genetic' means. Height is around 80% heritable and is still strongly influenced by nutrition
While you're correct about relative distributions, there is no excuse for an education system that produces "graduates" with grade school reading and math levels. So many American students aren't even approaching their academic potential, which may be mediocre, but even someone with an IQ of 75 or 80 should be able to get a 70 average and graduate high school. Especially in the districts where standards for doing so are basically (1) show up to class, (2) don't be disruptive, (3) turn your assignments in on time, and (4) work hard on studying for quizzes and exams. Even if we accomplish all that we have to face the hard reality than tens of millions of kids aren't college material, but they will have gotten some semblance of an education. Instead we are just passing them through school at 5x the cost and keeping the delinquents in the classroom to ruin the experience for many kids on the bubble.
And yet, under a different accountability regime, worse performing students did better than they are now!
Americans aren’t poorer (at any income slice) than they were when kids were performing better either.
I am not claiming accountability is ALL we need and it’s true that there are always going to be socioeconomic, genetic, and a host of other differences that prevent a classroom from being fully flat in performance, duh.
But those things haven’t changed in the direction needed to explain worsening test scores.
Also, in some (many? Idk) instances we know there are better ways to teach kids things and worse ways. Some/many school are doing it the worse way! Call it “accountability” while holding your nose if you want, but seems pretty obvious you could improve median outcomes if you stopped people from teaching stuff the worse way (which, in fact, is what happens)
Accountability works in England, probably due to our incredibly centralised education system that is far more centralised than most US states
Freddie, how do you account for the often large differential achievement gap between students randomly selected for a charter school and those in the mainline public schools?
It strikes me that this is a compelling “natural experiment” taking students with the same SES, background, family engagement, etc. and proving that they can have different results in a different environment.
There are no such large gains because there is no actual randomization. The amount of ways charters find to avoid truly random lotteries is immense: https://freddiedeboer.substack.com/p/charter-school-lotteries-are-a-black
That's more or less the entire charter story - whenever a charter school or system appears to be performing at an implausibly high level, there's inevitably a hidden selection bias resulting in a dramatically unrepresentative student body. Please note that simply having parents who have it together enough to apply to a charter school lottery is sufficient to break randomization, and they go much, much further than that.
But the lottery kids who "win" do better - the other parents were organised enough to apply but didn't get in. Surely that is a good example of mitigating selection effects in the data?
Dude, in this very podcast Matt said that achievement gaps didn't close because the higher level students did better. But the bottom students also did better.
We get it. You don't think absolute level of education matters because it won't redistribute the pie evenly. Most of us do think that increasing the absolute level of education matters even if it doesn't do that. Are more fourth graders not reading at a higher level in Mississippi than they did several years ago?
Everyone keeps pointing out that, if the Mississippi Miracle had happened in a blue state, we would be having a national conversation about it. It's interesting that nobody seems to think national Republicans have the ability -responsibility?- to care about this either. Democrats are expected to be the party with all the answers on education, urban policing, and rebuilding manufacturing, whereas national Republicans get a win if they even acknowledge these problems and don't deindustrialize us -too- much or eliminate -too- many cancer research centers.
It’s a good point.
Who says that Republicans don't care about education? Many have given up on the ability of public schools/teachers unions to provide it, but that doesn't mean that it's not important.
A quick internet search suggests that Republicans do, in fact, talk about Mississippi's successes (see https://www.foxnews.com/us/mississippi-miracle-as-student-reading-scores-rapidly-climb-in-the-deep-south).
These things have spread from Mississippi to other Republican-led states, Mississippi's governor has talked about it, etc.
Is it just the case that the sorts of media that you consume doesn't overlap with the sorts of media where quotes from Republican governors are published?
https://web.archive.org/web/20230517204515/https://www.foxnews.com/
I tried checking out the fox news front page on that day and I don't even see that article. Am I looking in the wrong place? Here's fox news US from the same day:
https://web.archive.org/web/20230517220708/https://www.foxnews.com/us
The front page is just a lot of stories about random crimes, and two pieces about abortion and trans people. And this article only has 7 comments compared to thousands on other articles. Does foxnews.com's own reader statistics and front page editor say that Republicans don't care about education?
Hey did the post before this one self-destruct or something? It was called "The left is right about the Democrats" but I cannot find it any more. It exists in my email, but clicking the link brings me to a "page not found"
It never existed on the site for me, only via the email link.
My hunch is that it was meant to be a future article but it accidentally got sent out early.
Agreed, and now confirmed!
I'm still interested to understand why American parents hate testing so much. The US has much less testing than most countries, and high stakes tests are almost completely optional. It seems odd to me that the US as a highly meritocratic country, has so much of kids' marks based on whether or not their teacher likes them or they can hassle their teacher for a better mark.
Something really stuck out to me listening to this though, which is that parents in the US don't like hearing negative things about their kids' schools. I feel that the UK's incredibly pessimistic attitude, which annoys me much of the time, actually makes school improvement much easier. Everyone in the UK thinks schools could do better and we're willing to hear about how bad our kids' schools are and how much better they should be.
I can't believe how much better primary education is for my kids than it was for me - the quality of writing, the fact that all the kids can read - it's a real testament to the consensus on the left and right in the UK on the importance of phonics, school accountability and a commitment to knowledge based curricula.
Part of it is standardized testing on a nationwide level is a fairly new thing.
I'm not super old (early 40s), but when I went to school, there was a statewide assessment test late in grade school, but it wasn't a high stakes thing. It was annoying to take 3 days out of the year to be bubbling in multiple choice scantrons, but it wasn't treated by the end of the world.
Even by time I was leaving high school, that had begun to change (note - I lived in Florida) in elementary and middle school and has only increased.
To a certain extent, the standardized tests are seen as something separate from the actual schooling.
Are there any high stakes standardised tests in the US though? There are SATS and the ACT but aren't they completely optional? Don't most other standardised tests in the US only have stakes for schools? Like if you fail your SATS you can go to a test-optional university, right? For me high-stakes implies stakes for the kids too - like in the UK if you fail your A-levels you're not going to university until you retake them, no matter how well you did on your homework the rest of year or how great you are at extra curriculars.
In the UK we have the phonics screening check in year 1 (moderate stakes for schools), SATS at the end of year 6 (high stakes, but only for the school), GCSEs at the end of Year 11, A-levels at the end of Year 13 (high stakes for schools & kids). On top of that we have regular high stakes inspections of schools from Ofsted.
Labour has just introduced a new Year 8 reading test (12/13 yo kids) too - again the only stakes are for the schools.
I suppose it kind of seems crazy to me that so many US parents really hate the standardised testing when it's so routine in the UK, with much more tests and much higher stakes.
The SAT or ACT isn’t required for people who want to go to “community college.” One is needed for those interested in attending a “directional state” college, state university or private college/university. My understanding is that which test is used varies from state to state. In my state (WA) we mostly take the SAT.
A kid generally needs a pretty solid SAT/ACT score for more selective schools, so in that regard it’s by far the highest stake test we take in my opinion.
There are tracking assessments that happen in K12 schools, but there’s no real standard there from one school district to the next as far as I know. Parents really hate this if/when their kid can’t be in honors or whatever. Additionally while your assigned public school can’t exit a kid due to lack of performance (due to Supreme Court decisions), private schools can.
I think the criticism you’re seeing is somewhat true, but maybe more so rooted in partisan politics. President George W. Bush and the Republicans started standardized testing under the No Child Left Behind Law. The idea was “a high school diploma should mean something.” I think that effort has mostly been abandoned now.
We had statewide assessment tests every year, but they didn’t count for anything and were only one day of testing. This was in TN during the 90s.
So I think one of the policy questions that frustrates me is why is accountability a one time thing.
Like I teach in 3rd grade in a retention year state. We don't really spend that much instructional time on phonics. There's like some skills to master but the really load bearing years for phonics instruction is first and kindergarten. Most of our retained students don't flourish their second time in the grade.
We give them a standardized test to measure what they learned but we only make a big fuss about it once. It feels pretty strange.
Accountability is not a one-time thing, if you mean testing. In many places in the United States public school students are still going through absolutely relentless census testing that takes up weeks of instructional time.
"Most of our retained students don't flourish their second time in the grade."
That should not be surprising; teachers and schools control somewhere between 1% and 7% of the variance in student outcomes in quantitative metrics.
I mean accountability that affects the students. In 3rd grade you either pass the test or you fail 3rd grade and take it again. Every other year basically nothing happens to you if you fail. We assign you some half baked intervention that's even more fucked than our primary curriculum.
Low achievers who aren't catastrophically low work really hard to avoid getting retained. My reading test felt like a minor miracle that on test day everyone passed.
Hey Freddie, you seem very knowledgeable about ed and I'm going to browse your substack. In the meantime could you post a source for your statistic about teachers/schools only controlling 1-7% of variance in student outcomes? I'm sure google could find some study along these lines but I'd like to know the one that was convincing to you. Thanks!
Some of these talks are more productive than others...this one felt like two ships passing in the night, who keep acknowledging that they're receiving the other's signal in the vicinity, yet continuously fail to see the other ship. I was left confused as to whether Jon or Matt actually agree on much of anything at all, or this was just an elaborate object lesson in popularism. Perhaps that would have been hashed out later on in the discussion, alas. (Even if one grants the premise that leftists have a state-owned monopoly on energy and charisma and heartfelt beliefs, the name of the game is still to win elections. Thought the GOP may have won all the battles, we had all the good songs...)
Also it's weird the URL for this one says "Jon Lovell". Someine didn't dot their i's and cross their t's!
If you want to get involved in a political fight involving better education in math that’s happening right now you should check out the petition against the recent New York briefs on teaching math. You can find a good discussion on the Chalk and Talk podcast. https://youtu.be/7pcZ7TexRQE?si=tHkVLYofF7hfD9-7
In general, there has been a lot of research on how to best teach math, and it’s been very hard to get public attention for it. Part of the challenge is multiple groups are claiming that their methods are scientifically backed. In my opinion Solomon and Stokke have the actually experiment backed research supporting them.
If Democrats want to become the party that teaches math better, using Stokke’s podcast as a resource would be an excellent way to get up to speed.
Yes, I have recommended the Chalk and Talk podcast here as well! I really do think it's a useful area to which Matt can turn his attention.
Looks like someone needs more reliable or redundant internet.
Love this crossover!
Reschedule and pick up where you left off!
Can’t wait to listen. This is like my political Super Bowl
This was nice to hear.
That’s the ticket!
My company is building an AI-powered SAT math tutoring app to help kids study. Would love feedback - https://aaris.ai
One size fits all education has as much of a chance of succeeding as does one size fits all medicine. If the time wasted on identity politics* was spent using AI algorithms in computers to be applied by teachers to achieve what teaching machines were supposed to achieve, each student would be guided to a state of being educated** at a pace that matched their ability. *I listened to my grandson's classes during the Covid-19 pandemic and experienced what they were exposed to. **reaching a state of being able to function independently in society.
You can't fix stupid.