I wonder what would happen if somehow class sizes could be reduced across the board. Twenty-four the new standard, with even lower class sizes depending on the economic demographics of the student body. Increased costs would go towards more teachers and more classrooms. Salaries might stay about the same, but job satisfaction would increase.
I am not a policy wonk nor an educator. I am merely a parent of kids who went/are in public schools consistently identified as "persistently failing" or whatever the term du jour is. My oldest was in the middle of her K12 career when all this when RTTT really started being implemented. I see a big contrast in our schools from pre and post RTTT, often not for the better.
Although I agree with the goals, RTTT was mostly a disaster for my kids and their cohorts. Because we were a failing school, we had to have more data points and many more pre and post-tests to gauge progress. School became very dull, and I had a hard time figuring out when the teachers found time to actually teach content because there were so many tests and data gathering. Our schools created new positions to handle the data - money that could have been used for social workers and teachers' aides. Huge chunks of PTA meetings were spent on these data points. We received much higher scores for "growth" than surrounding suburban schools because we had much further to grow for our students to reach grade level. After a while, I got the distinct impression that a lot of time and effort was spent on data manipulation. The most attention was paid to students that were ALMOST at grade level because getting them to grade level would be the easiest, which would improve our rankings. If you were way at the bottom, forget it.
At the time (pre-Trump) our district had an ever-growing influx of refugee students. The RTTT expectations for non-native English speakers were ridiculous. Many of the older refugee students had missed years of schooling in their own language. Their stories of learning while living in camps, fleeing danger, and having no electricity for many years were both inspiring and heartbreaking. But there is no way most of them would get up to grade level in a couple of years. Often teachers nearing retirement would offer to take classes filled with English Language Learners because they knew they would be gone before those scores would count in their performance evaluations. Luckily, most of those teachers were very good and excited to work with this new set of students.
The final slap in the face was when our state Commissioner of Education came to our district telling us how wonderful these efforts would be for students while he sent his own kids to a local private school that didn't have to follow any of these standards and testing and whose method of teaching was pretty much the complete opposite of what the reforms were pushing.
I could rant for a lot longer, but I wish people who are interested in reform could spend an entire year as an employee and their kids as students at the schools they are trying to improve with their efforts.
I agree with much of this analysis, but here's one important factor that isn't mentioned: reforms focused on improving teacher quality through test-based evaluations have only reinforced a type of curriculum and instruction that inevitably ends up holding many students back. That's a big reason NCLB and RTTT were largely self-defeating.
If you hold schools and teachers accountable for scores on tests, they will spend more time on activities they think will boost those scores. With reading tests, that means lots of practice doing things like "finding the main idea" of a text, using reading passages on random topics followed by comprehension questions. That practice means less time for things like social studies and science. But cognitive science tells us that practicing comprehension skills in the abstract won't boost comprehension, or test scores. And the evidence suggests that more time on social studies and science WILL do that, because it provides students with the vocabulary they need to understand the reading passages on the tests.
The tests aren't inherently bad--or racist. But what they measure isn't abstract "reading comprehension" skill, as virtually everyone has been assuming. What they mostly measure is general academic knowledge and vocabulary.
Matt: I would strongly urge you to take a look at this issue, complex as it is, in a future post about education reform. There's a growing shift across the country towards the adoption of a different kind of elementary reading curriculum--one that focuses on building kids' knowledge (along with foundational skills like phonics) instead of abstract and largely illusory reading comprehension skills. It has enormous potential ramifications, but practically no one in the media is writing about it.
> Are there problems and shortcomings with value-added measures based on test scores? Sure. At the same time, a highly imperfect proxy still seems much better than a proxy that definitely doesn’t work well, like seniority.
I am a statistician. At one point, I came across one of the models used to convert student test scores into teacher ratings. It had been developed by some outside consulting firm. I can't remember where it was used, somewhere in the northeast, I think. I was appalled. It was garbage. You might as well have used seniority. I don't know about other models, but overall my impression is not good.
Great article, reminds me a lot of the Bunny Colvin character arc in Season Four of The Wire, where they come up with a innovate program (based on one created in the 90s by series co-creator Ed Burns who became a teacher after he retired as a cop) and it does kind of work. But its shutdown due to broader bureaucratic and political issues (as was apparently Burns' program) https://www.youtube.com/watch?v=OwEUcTF_XQg
Also it seems virtually impossible to ensure that teachers have no inside information relative to a statistician in DC. Sure, you can measure all sorts of numbers but how do you avoid a teacher from realizing: ohh class X is taught at the same time as band/AP physics/etc and thus having an advantage not captured in the stats.
I agree in principle my concern is that we've never been willing to ask the kind of questions on student tests that would make this work. You need to be willing to ask questions that are surprising and unlike examples to test understanding and it's very hard to do via buerocracy.
Basically, the issue is that you can't both give fair warning as to what kinds of questions are going to be asked (other than vaguely about this sort of subject) while avoiding the pitfalls of encouraging unproductive teaching to the tests.
And, for a variety of reasons we arey really willing to use the kind of unpredictable hard questions to evaluate performance. I think it strikes many people as a bit unfair.
This is one of those posts that I am not sure how to feel because I can never be objective about it. My wife, now in higher Ed, taught English in Milwaukee Public Schools for 6 years. She was a teacher when then governor Scott Walker broke the teacher unions. Those were tough times! It's just too hard for me to be objective. What I will say is that I do think some level of more stringent teacher evaluation would be beneficial but it would be even more effective if it was complemented with interventions to try and increase teaching skills on those evaluated as below average. Maybe it already had that? I don't know.
Working in urban schools is incredibly hard, as Yglesias points out in his post. My wife would often be up into the wee hours of the morning grading papers and lesson planning. She gave a sh*t. She had to spend a lot of time winning kids over before she could effectively teach them. Most of the time she was successful, but sometimes she wasn't. Her students' test scores were generally good, but she also taught 3 of those 6 years in MPS' most coveted and high performing school. She certainly made an impact but so did the kids wanting to be there and their parents being involved.
Ultimately, my goal with this comment is to try to avoid painting with a broad brush. I see too many conservatives and centrists holding views towards public school teachers that are mirrored by progressive views towards cops. The world is sooo much more nuanced than that.
Is there good evidence that suggests that teachers play an important role in improving standardized test scores? From my own experience, most students self select into different classes based on their own intellectual ability(so teachers who teach AP Calc BC will naturally look better, regardless of their own teaching ability), and I'd bet that this is a much stronger factor in how well students do on standardized tests.
Matt, I think if you want to discuss this issue, you really need to address the fact that the most widely-used "Value Added Model" for assessing teachers wasn't just not-great; it was a total, abject failure. You could look at teachers who taught the same subject, to adjacent grades, in the same year, so the metric could give two scores on things that ought to be highly correlated -- and they weren't. There was basically no correlation at all.
The metrics being pushed at the time were basically random number generators. It was _completely reasonable_ for teachers to object to being evaluated with this tool.
I agree that _if_ we had a good measure of value-added, that would be a good thing to use in setting some portion of pay. But we shouldn't pretend that the metrics that were actually in use were any good.
As a HS teacher I found this piece the best of the series. In my state you get frozen on the salary schedule if you don’t get (pay for) your own master’s degree, after which you’re likely to get sporadically frozen anyway.
Regarding higher compensation for teachers who commit to more challenging schools, this sounds great in theory, and it may work. I don’t know. Just not sure regular people have any idea how emotionally exhausting those jobs are, how helpless it can be. Kids need to be in school so when they do terrible things the institutional incentive is to try to keep them in class which causes the terrible behavior to snowball. Teaching content falls down the list of things you do during a day. Those that teach in more challenging schools are American heroes. Not sure there is an amount of money that would be worth it to me.
yeah...those kinds of places don't have enough support / admin staff to allow you a bathroom break. everyone's got a price - I think i'd do that for 150K?
My (without any evidence heuristic) is that principals are morons and there’s no way to objectively rank ALL teachers. However, we can take the bottom 5% and there would be a general consensus. In an ideal world, we could find a way for them to improve or be out the door.
So I'd believe getting a random Masters degree doesn't help anything, but what about using the selection effect? Is the kind of teacher that gets a random Masters degree the kind of teacher we want to pay more to work harder to retain?
i don't know about selection effect - it's a basic and easy to see correlation. if you get that piece of paper, your salary jumps to another band. also, many states are now requiring a master's to move from provisional to permanent teaching license.
I know it does help people get paid better, I'm talking about whether it should.
Matt implies that its wrong to increase salary for a Masters degree because studies show this doesn't make someone more valuable after getting the Masters degree. I'm saying, what if the kind of person who gets a Masters degree is more valuable, and this helps you identify who that is.
Seems like another entry in the general saga of "constrained supply makes all things harder". If you have more teachers (and the opportunity cost of teaching anything else is lower), then it's easier to boost test scores, make education more equitable, or do whatever else you want. You can see how this sort of aligns with, say, YIMBYist thinking on housing.
I wonder what would happen if somehow class sizes could be reduced across the board. Twenty-four the new standard, with even lower class sizes depending on the economic demographics of the student body. Increased costs would go towards more teachers and more classrooms. Salaries might stay about the same, but job satisfaction would increase.
The think the link to part 4 is broken
I am not a policy wonk nor an educator. I am merely a parent of kids who went/are in public schools consistently identified as "persistently failing" or whatever the term du jour is. My oldest was in the middle of her K12 career when all this when RTTT really started being implemented. I see a big contrast in our schools from pre and post RTTT, often not for the better.
Although I agree with the goals, RTTT was mostly a disaster for my kids and their cohorts. Because we were a failing school, we had to have more data points and many more pre and post-tests to gauge progress. School became very dull, and I had a hard time figuring out when the teachers found time to actually teach content because there were so many tests and data gathering. Our schools created new positions to handle the data - money that could have been used for social workers and teachers' aides. Huge chunks of PTA meetings were spent on these data points. We received much higher scores for "growth" than surrounding suburban schools because we had much further to grow for our students to reach grade level. After a while, I got the distinct impression that a lot of time and effort was spent on data manipulation. The most attention was paid to students that were ALMOST at grade level because getting them to grade level would be the easiest, which would improve our rankings. If you were way at the bottom, forget it.
At the time (pre-Trump) our district had an ever-growing influx of refugee students. The RTTT expectations for non-native English speakers were ridiculous. Many of the older refugee students had missed years of schooling in their own language. Their stories of learning while living in camps, fleeing danger, and having no electricity for many years were both inspiring and heartbreaking. But there is no way most of them would get up to grade level in a couple of years. Often teachers nearing retirement would offer to take classes filled with English Language Learners because they knew they would be gone before those scores would count in their performance evaluations. Luckily, most of those teachers were very good and excited to work with this new set of students.
The final slap in the face was when our state Commissioner of Education came to our district telling us how wonderful these efforts would be for students while he sent his own kids to a local private school that didn't have to follow any of these standards and testing and whose method of teaching was pretty much the complete opposite of what the reforms were pushing.
I could rant for a lot longer, but I wish people who are interested in reform could spend an entire year as an employee and their kids as students at the schools they are trying to improve with their efforts.
I agree with much of this analysis, but here's one important factor that isn't mentioned: reforms focused on improving teacher quality through test-based evaluations have only reinforced a type of curriculum and instruction that inevitably ends up holding many students back. That's a big reason NCLB and RTTT were largely self-defeating.
If you hold schools and teachers accountable for scores on tests, they will spend more time on activities they think will boost those scores. With reading tests, that means lots of practice doing things like "finding the main idea" of a text, using reading passages on random topics followed by comprehension questions. That practice means less time for things like social studies and science. But cognitive science tells us that practicing comprehension skills in the abstract won't boost comprehension, or test scores. And the evidence suggests that more time on social studies and science WILL do that, because it provides students with the vocabulary they need to understand the reading passages on the tests.
The tests aren't inherently bad--or racist. But what they measure isn't abstract "reading comprehension" skill, as virtually everyone has been assuming. What they mostly measure is general academic knowledge and vocabulary.
Matt: I would strongly urge you to take a look at this issue, complex as it is, in a future post about education reform. There's a growing shift across the country towards the adoption of a different kind of elementary reading curriculum--one that focuses on building kids' knowledge (along with foundational skills like phonics) instead of abstract and largely illusory reading comprehension skills. It has enormous potential ramifications, but practically no one in the media is writing about it.
> Are there problems and shortcomings with value-added measures based on test scores? Sure. At the same time, a highly imperfect proxy still seems much better than a proxy that definitely doesn’t work well, like seniority.
I am a statistician. At one point, I came across one of the models used to convert student test scores into teacher ratings. It had been developed by some outside consulting firm. I can't remember where it was used, somewhere in the northeast, I think. I was appalled. It was garbage. You might as well have used seniority. I don't know about other models, but overall my impression is not good.
Great article, reminds me a lot of the Bunny Colvin character arc in Season Four of The Wire, where they come up with a innovate program (based on one created in the 90s by series co-creator Ed Burns who became a teacher after he retired as a cop) and it does kind of work. But its shutdown due to broader bureaucratic and political issues (as was apparently Burns' program) https://www.youtube.com/watch?v=OwEUcTF_XQg
Also it seems virtually impossible to ensure that teachers have no inside information relative to a statistician in DC. Sure, you can measure all sorts of numbers but how do you avoid a teacher from realizing: ohh class X is taught at the same time as band/AP physics/etc and thus having an advantage not captured in the stats.
I agree in principle my concern is that we've never been willing to ask the kind of questions on student tests that would make this work. You need to be willing to ask questions that are surprising and unlike examples to test understanding and it's very hard to do via buerocracy.
Basically, the issue is that you can't both give fair warning as to what kinds of questions are going to be asked (other than vaguely about this sort of subject) while avoiding the pitfalls of encouraging unproductive teaching to the tests.
And, for a variety of reasons we arey really willing to use the kind of unpredictable hard questions to evaluate performance. I think it strikes many people as a bit unfair.
This is one of those posts that I am not sure how to feel because I can never be objective about it. My wife, now in higher Ed, taught English in Milwaukee Public Schools for 6 years. She was a teacher when then governor Scott Walker broke the teacher unions. Those were tough times! It's just too hard for me to be objective. What I will say is that I do think some level of more stringent teacher evaluation would be beneficial but it would be even more effective if it was complemented with interventions to try and increase teaching skills on those evaluated as below average. Maybe it already had that? I don't know.
Working in urban schools is incredibly hard, as Yglesias points out in his post. My wife would often be up into the wee hours of the morning grading papers and lesson planning. She gave a sh*t. She had to spend a lot of time winning kids over before she could effectively teach them. Most of the time she was successful, but sometimes she wasn't. Her students' test scores were generally good, but she also taught 3 of those 6 years in MPS' most coveted and high performing school. She certainly made an impact but so did the kids wanting to be there and their parents being involved.
Ultimately, my goal with this comment is to try to avoid painting with a broad brush. I see too many conservatives and centrists holding views towards public school teachers that are mirrored by progressive views towards cops. The world is sooo much more nuanced than that.
Is there good evidence that suggests that teachers play an important role in improving standardized test scores? From my own experience, most students self select into different classes based on their own intellectual ability(so teachers who teach AP Calc BC will naturally look better, regardless of their own teaching ability), and I'd bet that this is a much stronger factor in how well students do on standardized tests.
Matt, I think if you want to discuss this issue, you really need to address the fact that the most widely-used "Value Added Model" for assessing teachers wasn't just not-great; it was a total, abject failure. You could look at teachers who taught the same subject, to adjacent grades, in the same year, so the metric could give two scores on things that ought to be highly correlated -- and they weren't. There was basically no correlation at all.
https://mathbabe.org/2012/03/06/the-value-added-teacher-model-sucks/
The metrics being pushed at the time were basically random number generators. It was _completely reasonable_ for teachers to object to being evaluated with this tool.
I agree that _if_ we had a good measure of value-added, that would be a good thing to use in setting some portion of pay. But we shouldn't pretend that the metrics that were actually in use were any good.
As a HS teacher I found this piece the best of the series. In my state you get frozen on the salary schedule if you don’t get (pay for) your own master’s degree, after which you’re likely to get sporadically frozen anyway.
Regarding higher compensation for teachers who commit to more challenging schools, this sounds great in theory, and it may work. I don’t know. Just not sure regular people have any idea how emotionally exhausting those jobs are, how helpless it can be. Kids need to be in school so when they do terrible things the institutional incentive is to try to keep them in class which causes the terrible behavior to snowball. Teaching content falls down the list of things you do during a day. Those that teach in more challenging schools are American heroes. Not sure there is an amount of money that would be worth it to me.
yeah...those kinds of places don't have enough support / admin staff to allow you a bathroom break. everyone's got a price - I think i'd do that for 150K?
My (without any evidence heuristic) is that principals are morons and there’s no way to objectively rank ALL teachers. However, we can take the bottom 5% and there would be a general consensus. In an ideal world, we could find a way for them to improve or be out the door.
“ these reforms did not on the whole generate great results. “
And they never will. At the end of the day you just can’t fix stupid.
So I'd believe getting a random Masters degree doesn't help anything, but what about using the selection effect? Is the kind of teacher that gets a random Masters degree the kind of teacher we want to pay more to work harder to retain?
i don't know about selection effect - it's a basic and easy to see correlation. if you get that piece of paper, your salary jumps to another band. also, many states are now requiring a master's to move from provisional to permanent teaching license.
I know it does help people get paid better, I'm talking about whether it should.
Matt implies that its wrong to increase salary for a Masters degree because studies show this doesn't make someone more valuable after getting the Masters degree. I'm saying, what if the kind of person who gets a Masters degree is more valuable, and this helps you identify who that is.
Seems like another entry in the general saga of "constrained supply makes all things harder". If you have more teachers (and the opportunity cost of teaching anything else is lower), then it's easier to boost test scores, make education more equitable, or do whatever else you want. You can see how this sort of aligns with, say, YIMBYist thinking on housing.