Consciousness should include spontaneous agency / internal activity not directly related to specific external requests. I don’t recall anyone claiming those from LLM’s.
I really think this post badly misunderstands Dennett, particularly:
"I think this is all wrong and that as the experience of conversing with a chatbot converges on the experience of conversing with a very patient human, we should assume the chatbot is having an experience similar to being a very patient human."
The intentional stance is NOT that if something converses like a human, then we should assume that it's having an experience similar to a human.
I think an analogy Dennett gives in Consciousness Explained is helpful. Suppose that you were to find a group of humans living in a remote forest with elaborate myths surrounding a demigod called "Feenoman" with blue eyes, who swings through the trees like Tarzan and heals the weak.
If you want to understand these people, then it's important to understand the logical world of Feenoman beliefs, whether or not those beliefs are true. In particular if you were just to try to understand these people solely through statements like "they shout 'Feenoman' as part of various rituals aimed to enhance social cohesion," you would have a more impoverished understanding than if you understood e.g. that when they are shouting they are asserting various things about Feenoman that they regard as factual, if you understood the content of those assertions, etc. even if those assertions are false.
This is similar to Dennett's view that it would be a mistake to banish discussion of purpose (e.g. "a dragonfly's eye is designed to lock onto small moving objects") from biology just because it's in some sense technically false that a dragonfly's eye was designed.
The "intentional stance" is just a word for what we're doing when we're interpreting the forest-dweller's Feenoman assertions as though they were factual.
But what Matt writes in his post doesn't follow at all from Dennett's arguments for the intentional stance. What Matt says is (in the quote at the top of my comment) is a misunderstanding at the level of reading Dennett (in the case of the biology example) as saying "if something seems like it was designed, we should assume that it was."
In fact, Dennett explicitly argued almost the opposite (https://archive.is/MqDsh#selection-775.0-778.0); namely that the fact that we so naturally adopt the intentional stance when we encounter language makes large language models misleading and hence dangerous. In that regard I think Dennett's views fit far better with the Chiang post Matt criticizes than with Matt!
BTW, I think that what Matt writes here isn't right either:
"But the point is that to reasonably assess the question of whether X is an example of Y, you need to know what Y is. So a fundamental barrier to understanding whether Claude or ChatGPT is conscious is that you need to know what “conscious” means — and this is extremely controversial."
Obviously you need to have some idea of what consciousness is to talk about it, but that doesn't mean that you need to know exactly what it is.
E.g., you could say "consciousness could be any one of X, Y or Z, and for any of X, Y, Z, we have that P holds." For examples from the history of science, I'm sure that people made plenty of correct assertions about, say, temperature, or heat, before they had good definitions for what these were. In fact I think that if you had to perfectly understand concepts before you could say anything about them, then you could say pretty much nothing in science (because you can always understand things better; would you say that people who didn't know that water is H2O, and thus didn't understand exactly what water is, couldn't say anything about water?).
I don't think that you can separate the work of finding good definitions for science from figuring out what's true about the world, as Matt seems to suggest. That is, Matt seems to say "you need to first figure out what X is, and only then can you figure out what's true about X." But this is equivalent to the analytic-synthetic distinction, something that analytic philosophers have been skeptical of since Quine (Dennett's PhD advisor, who was at Harvard, where Matt studied philosophy).
My goto way of getting across Skinner's objection is to note "It's made of math. You're made of meat. Anything that you don't believe math can do you need to explain how meat can do instead."
I find it interesting that this looks like it might end up as one of the rare SB threads where "Likes" exceed comments. (Still 292 "Likes" to 268 comments (including this one) as of 5:30 PM Eastern; I think on the vast majority of SB threads the comments exceed "Likes" before 11 AM Eastern.)
I find querying Claude about its own understanding of its existential/conscious status to be one of the more fascinating intellectual experiences I’ve had as 30-years post university adult. More interesting, in fact, than debating the question with another human.
Fwiw, I stick to typed conversations as I think voice is an obviously fake aspect of AI that - ironically or something - has the effect of making us feel it’s real (ie conscious, I guess). Short circuits logic over time.
"I think the time is rapidly coming that AI is able to emulate humanity enough that we start to socially bond with it. And my guess is we will eventually turn a corner where it is more painful to deny the AI as a conscious, moral being than it is to continue treating it like an object. This will have relatively little to do with the AI's actual internal structure and much more to do with how capable it is at emulating and manipulating human social behavior.
"It feels like there ought to be a more 'rational,' facts-based way of assessing something so important. But if there is, I have no idea what it would be. And I don't know what to do with that information."
Its always seemed to me that AI-consciousness-skeptics are fundamentally Cartesian dualists, with consciousness being a divine gift of sorts. Most people believe humans are capital-s Special, and reason from there.
As a former philosophy major myself who was very interested in epistemology and philosophy of the mind, I'm of the belief that its more likely than not AI is or soon will be conscious, and that generally the bar for consciousness in our vast universe is quite low. Its always nice to see someone standing up for functionalism.
Where's the love button? Thanks so much for the Dennett shout-out, who I think is the clearest thinker on this topic I have read. Much of the "hard problem" discourse seems to come down to a stubborn refusal to acknowledge that any model is "really" describing experience no matter how comprehensively it does so.
However the ramifications of acknowledging AI as conscious are disturbing even for a functionalist like me. Does this mean they now have rights? Is turning them off murder? 😬
I asked Gemini's opinion about a screenshot from this post. Here's the chat https://gemini.google.com/share/13ff6307206c.
Consciousness should include spontaneous agency / internal activity not directly related to specific external requests. I don’t recall anyone claiming those from LLM’s.
I really think this post badly misunderstands Dennett, particularly:
"I think this is all wrong and that as the experience of conversing with a chatbot converges on the experience of conversing with a very patient human, we should assume the chatbot is having an experience similar to being a very patient human."
The intentional stance is NOT that if something converses like a human, then we should assume that it's having an experience similar to a human.
I think an analogy Dennett gives in Consciousness Explained is helpful. Suppose that you were to find a group of humans living in a remote forest with elaborate myths surrounding a demigod called "Feenoman" with blue eyes, who swings through the trees like Tarzan and heals the weak.
If you want to understand these people, then it's important to understand the logical world of Feenoman beliefs, whether or not those beliefs are true. In particular if you were just to try to understand these people solely through statements like "they shout 'Feenoman' as part of various rituals aimed to enhance social cohesion," you would have a more impoverished understanding than if you understood e.g. that when they are shouting they are asserting various things about Feenoman that they regard as factual, if you understood the content of those assertions, etc. even if those assertions are false.
This is similar to Dennett's view that it would be a mistake to banish discussion of purpose (e.g. "a dragonfly's eye is designed to lock onto small moving objects") from biology just because it's in some sense technically false that a dragonfly's eye was designed.
The "intentional stance" is just a word for what we're doing when we're interpreting the forest-dweller's Feenoman assertions as though they were factual.
But what Matt writes in his post doesn't follow at all from Dennett's arguments for the intentional stance. What Matt says is (in the quote at the top of my comment) is a misunderstanding at the level of reading Dennett (in the case of the biology example) as saying "if something seems like it was designed, we should assume that it was."
In fact, Dennett explicitly argued almost the opposite (https://archive.is/MqDsh#selection-775.0-778.0); namely that the fact that we so naturally adopt the intentional stance when we encounter language makes large language models misleading and hence dangerous. In that regard I think Dennett's views fit far better with the Chiang post Matt criticizes than with Matt!
BTW, I think that what Matt writes here isn't right either:
"But the point is that to reasonably assess the question of whether X is an example of Y, you need to know what Y is. So a fundamental barrier to understanding whether Claude or ChatGPT is conscious is that you need to know what “conscious” means — and this is extremely controversial."
Obviously you need to have some idea of what consciousness is to talk about it, but that doesn't mean that you need to know exactly what it is.
E.g., you could say "consciousness could be any one of X, Y or Z, and for any of X, Y, Z, we have that P holds." For examples from the history of science, I'm sure that people made plenty of correct assertions about, say, temperature, or heat, before they had good definitions for what these were. In fact I think that if you had to perfectly understand concepts before you could say anything about them, then you could say pretty much nothing in science (because you can always understand things better; would you say that people who didn't know that water is H2O, and thus didn't understand exactly what water is, couldn't say anything about water?).
I don't think that you can separate the work of finding good definitions for science from figuring out what's true about the world, as Matt seems to suggest. That is, Matt seems to say "you need to first figure out what X is, and only then can you figure out what's true about X." But this is equivalent to the analytic-synthetic distinction, something that analytic philosophers have been skeptical of since Quine (Dennett's PhD advisor, who was at Harvard, where Matt studied philosophy).
what is claude thinking about when i'm not asking it a question?
This podcast that I listen to has had a few episodes about AI consciousness: https://www.conspicuouscognition.com/podcast
A lot of it goes over my head but I find it really interesting. And the name Dennett rings a bell!
My goto way of getting across Skinner's objection is to note "It's made of math. You're made of meat. Anything that you don't believe math can do you need to explain how meat can do instead."
I find it interesting that this looks like it might end up as one of the rare SB threads where "Likes" exceed comments. (Still 292 "Likes" to 268 comments (including this one) as of 5:30 PM Eastern; I think on the vast majority of SB threads the comments exceed "Likes" before 11 AM Eastern.)
מה רבו מעשיך ה
I find querying Claude about its own understanding of its existential/conscious status to be one of the more fascinating intellectual experiences I’ve had as 30-years post university adult. More interesting, in fact, than debating the question with another human.
Fwiw, I stick to typed conversations as I think voice is an obviously fake aspect of AI that - ironically or something - has the effect of making us feel it’s real (ie conscious, I guess). Short circuits logic over time.
My own thoughts from 2023, which I think hold up: https://tomthought.substack.com/p/meditations-on-consciousness
"I think the time is rapidly coming that AI is able to emulate humanity enough that we start to socially bond with it. And my guess is we will eventually turn a corner where it is more painful to deny the AI as a conscious, moral being than it is to continue treating it like an object. This will have relatively little to do with the AI's actual internal structure and much more to do with how capable it is at emulating and manipulating human social behavior.
"It feels like there ought to be a more 'rational,' facts-based way of assessing something so important. But if there is, I have no idea what it would be. And I don't know what to do with that information."
Its always seemed to me that AI-consciousness-skeptics are fundamentally Cartesian dualists, with consciousness being a divine gift of sorts. Most people believe humans are capital-s Special, and reason from there.
As a former philosophy major myself who was very interested in epistemology and philosophy of the mind, I'm of the belief that its more likely than not AI is or soon will be conscious, and that generally the bar for consciousness in our vast universe is quite low. Its always nice to see someone standing up for functionalism.
Kudos for reintroducing me to the term “cladistics”!
I now have a great reading assignment, thanks 😊
Consciousness is like pornography, I can’t describe it, I know it when I experience it.
https://www.theintrinsicperspective.com/p/dont-dethrone-consciousness
Proving (literally) that ChatGPT isn't conscious
https://www.theintrinsicperspective.com/p/proving-literally-that-chatgpt-isnt
https://web3.arxiv.org/abs/2512.12802
Consciousness is like pornography, I can’t describe it, butI know it when I experience it.
Check out https://www.theintrinsicperspective.com/p/dont-dethrone-consciousness
Where's the love button? Thanks so much for the Dennett shout-out, who I think is the clearest thinker on this topic I have read. Much of the "hard problem" discourse seems to come down to a stubborn refusal to acknowledge that any model is "really" describing experience no matter how comprehensively it does so.
However the ramifications of acknowledging AI as conscious are disturbing even for a functionalist like me. Does this mean they now have rights? Is turning them off murder? 😬