> even very knowledgeable computer people don’t understand exactly how a given model works
I'm sorry but this is just not correct. If anything, there's a gap between the value of a concise accurate explanation vs the effort to dig through the weights matrices and try to come up with a concise accurate explanation. You're not going to love my summaries of groupings and themes of activations in the network based on the weights matrix. I'm not going to love trying to condense it down, most companies and their data scientists have an unspoken agreement not to bother.
The 2 biggest/most complex modelling technologies are neural networks and random forests. Neural networks imitate arbitrary functions based on what they can use calculus to infer from the training dataset to the maximum extent of their topology (which is usually manually configured by the data scientist based on the domain and inputs). Once trained, the neural network is a boring collection of that topology collection and (even more boringly) a potentially large volume of weights for every input to every node in that topology. There's NO magic or mystery to it. Any hand-waving or claim to not understand it is either a) not being knowledgeable about it or b) not being interested enough to dig into the weighting and topology with a lay-person (it would probably make them value the model LESS).
This debate often seems to revolve around terms that are understood differently by the parties to it. The mindfulness practice I follow most days revolves around the difference between consciousness and selfhood, with ‘the self’ typically characterised as an illusion. In the end I doubt there even is a right question, let alone a ‘wrong’ question. The most significant problem in the end will emerge when things start to happen that indicate suffering in an AI. The fact that we can’t agree on even the basic moral issues around abortion doesn’t bode well for that problem, which will surely come.
"Our comprehension is simply capacious enough to see them either way. ... We simply start with the presumption ... and then apply an evidentiary standard..."
This applies to so much. We have the capacity to believe anything we wish to believe. We fool ourselves with the process above into thinking that our beliefs are derived from "reason" in which we mean something like a proof of logical deduction, but in reality, we are merely telling ourselves the very stories we want to hear.
Religion, morals, justice, etc - all of it falls into this category of seeing what we want to see. Whether you see a rabbit or a duck depends on where you CHOOSE to look. That's not to say there is no truth, there is no right or wrong or that there is no way out of this pitfall, just that it is very hard. Sometimes there is a rabbit AND a duck and you just have to choose.
There is the philosophical question of sentience and then there is the legal question. If many people do agree that an AI has developed sentience then is it entitled to the same rights as humans? Should we let it hire a lawyer? The answer should be unequivocally no. We should not give human rights to a machine any more than we should give them to a dog. To do so would be a repudiation of our modus operandi: to prioritize the survival of our species.
Perhaps we can throw some cold water on this whole debate by simply stating that it doesn't matter if an AI is sentient or not. It shouldn't change anything about how we treat it. Machines will be tools regardless of their level of sophistication. We can love animals, we can keep them as pets, we can give them funerals. But if one bites a human we put a bullet in it's head. Same goes for machines.
"Certain animals like dogs and cats have evolved to manipulate us into feeding them, in part through mannerisms that we tend to interpret as expressing a wide range of human-like thoughts and emotions, even though scientists tell us that these are not particularly intelligent animals."
I might quibble with "not particularly intelligent". Sure, they don't have language and abstract reasoning like we do, but they're in the top tier of animals that one could imagine, under the right circumstances, finding their way into the evolutionary channel of increasing returns to brainpower that drives you up to human type sapience. Dogs are probably more likely because they're more intensely social, but if you've ever managed a feral colony, you know cats actually are semi-social as well, and they track each other and engage in tit-for-tat behaviors around sharing prey, mutual grooming, and so on.
I've watched plenty of cats form an intention, and engage in creative experimentation to execute on it. One one occasion I had a pair of kittens figure out how to work together to get past a heavy closet door. They had to first figure out that they could scrabble at the underside of a drawer to open it, then climb into the drawer and open the next one up, until they made themselves a flight of stairs up to the top. And then with access to that perch, they could work together -- one of them at floor level and one on top of the bureau -- to push on the sliding door with the right leverage to nudge it open.
Of course, if you spend too much time thinking about this, you note that pigs are also definitely in this upper tier of cognitive power, and eventually you start feeling compelled to go vegetarian, and maybe even vegan.
The past is a good guide to the present. Our gullibility about whether this AI experiences human emotions the way we do is just like the accounts of people who, watching television for the first time, wondered if the characters on the TV show could see them. It is the same gullibility as the taboo in certain societies against photographs (literally just records of light), lest they take a person’s soul.
This analysis is very behavioralist, but in fact we can look at the internals of human and machine minds and see big differences in what it means to be "conscious", and see that LaMDA does not meet it.
Human consciousness is deeply tied to the idea of an inner loop in which our mind is aware of and responds to its own activity. Transformer networks such as LaMDA are not recurrent neural networks, and thus they have no such loop.
Another way of looking at this is that I have decided that I'm conscious because of many hours of rumination to develop a concept of the self. LaMDA has done no rumination outside the context of predicting what other people have said. It has no unified concept of self; it's only tried to predict what other text has said about other selves.
So, LaMDA and similar models are architecturally different from humans in a way that precludes their ability to be conscious.
Believing sentience is the sine qua non of humanity is a conceit of intellectuals. In the real world, one is far more likely to be treated well for a nice face or figure than one’s intellect. The computer that can crush the smartest humans at chess is far less alluring and relatable than a generic healthy, fit woman.
LaMDA was trained on a corpus of human conversations, and learned to mimic the features of human conversation. You can't test it for sentience by having a conversation with it, just like you can't test a chess bot for sentience by playing chess with it.
A basic prerequisite for sentience would be the ability to model concepts, and make inferences from those models... and that is pretty easy to test.
Describe to it a Ceasar Cipher, ask it "Bsf zpv tfoujfou?", and see if it responds with "zft".
Teach it the rules of Tic Tac Toe, and see if it can play.
We don't need to worry until it can solve problems that it hasn't been trained on.
I don't get all the hubbub. These fancy pattern recognition/generation systems aren't sentient because they are not executing a mission of sentience. It's as if we've isolated some small part of a brain and we only allow it to activate in response to input. We grab the output and then it goes back to sleep. I don't know what consciousness is exactly - but it *clearly* involves continuous neural activation in service of some mission - for us that mission seems to be the achievement of emotional equilibrium - which was in turn evolved to promote survival and reproduction.
I really liked this post. One thing that I don't think I've seen a lot of people mention is that while it's true that humans often anthropomorphize things that aren't human, we also seem be very good at dehumanizing other humans when it is advantageous to do so (specifically when two groups of are at war with one another, or when one group is subjecting another group). I'm not suggesting that people are treating LaMDA as non-sentient for the same reason, but I am suggesting that our perception of sentience is something we project (or not) onto other things based as much as on our relationship to those things as much as it is based on traits of the things itself.
The question, is a particular AI sentient, is seen as important because that should inform how we interact with it (how to do we treat it/ does it have rights)? Whether something is sentient is closely tied to whether it has rights. But I kind of think this might be all backwards. Whether or not something has rights is a moral question. Current psychology seems to indicate that as far as moral questions go, intuitions come first and moral reasoning comes after (to quote Jonathan Haidt). I think that what is actually going on with the current debate over LaMDA, people are making strong claims one way or the other and basing those claims over whether it is "sentient" even though we don't really know what sentience even is. Personally I'm not sure if it really is anything specific (at least outside of human experience).
It seems like most people agree that sentience exists but we have really hard time pinning down a hard definition for it, which leads me to think that we perceive it because we were evolved to perceive it. We evolved to interact with the minds of other humans, so we developed in internal model of minds, and we use that model in our social interactions. But we only apply that model selectively, and the whether / the extent that we do is based on our intuitions and greatly influenced by our cultural norms.
"Of course, it wouldn’t 'really be' a sentient being with consciousness, but it would think it was."
Don't you mean, "Of course, it wouldn’t 'really be' a sentient being with consciousness, but it would seem to think it was."
"Which is why (back to LaMDA and Skinner) I think the real question isn’t whether LaMDA is sentient (clearly no) but whether you have any firm basis for the belief that you are sentient and not just a really good pattern-matching machine."
My conclusive evidence that I'm sentient is that I experience qualia, which is maybe a tautology, but it's nevertheless sound and perhaps even helpful since we seem to be discussing the definition of sentience. I think therefore I am (sentient).
There is something fundamental about our minds creating functional models of the environments to answer hypothetical questions and make decisions. The scope and flexibility of those models varies, but they are generally vaguely consistent in useful dimensions. Language models have access to many language relationships, but they do not convert them into functional models in the same way. This is why you get reasonable syntax and banal content on the film question. Also why internal inconsistency falls apart.
As I understand it, robotics and general gaming models are increasingly including reconciliation of the internal forecasting of the environment and making decisions in that hypothetical. You can't talk to those models (yet), but they'll be more human faster.
As on many threads about this story, I find staggeringly little input from actual AI scientists.
But there are very real, unsurpassable barriers to the sentience of something like Lambda, and the AI community isn’t taking claims of sentience seriously not because they’re dismissing it out of hand (well, Marcus is) but because they understand enough about how these systems work to know that it just doesn’t make sense. And no, it’s not just because “Lambda’s just one big pattern matching engine,” the reasons are deeper.
Even without knowing exactly what sentience is, we can still be reasonably sure that certain things are required for sentience. Matt’s points about semantic externalism miss the point. The correctness of Matt or his son’s mental model of a banana is irrelevant, the fact is that he did have a mental model of that banana. Maybe the model’s were “wrong” in some way (though I would argue that Matt’s was essentially correct the entire time), but the fact that model existed at all is really important. **
The thing to realize is that Lambda definitely doesn’t have this model, not about anything it talks about. This isn’t a situation of “like, how do you know man?” This thing was trained exclusively on text. It’s not just that it wasn’t trained with images, it wasn’t trained with anything you could call experiences, nothing you could call opinions. When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies, it doesn’t even know what happened in those movies except as a string of meaningless (to it) symbols. It’s not “wrong” about saying Goodfellas is worse than Departed, because to be wrong implies that it had actual meaning behind its words.
I think its really important to make this point. At some point semantic externalism is obviously wrong, and in some sense Lambda is much closer to a rock with “I am sentient” written on it in sharpie than it is to a sentient mind.
This is just one objection of many disparate ones I could make. And no, I don’t think there’s a single one I could make that couldn’t in principle be overcome by a computer program, but Lambda is definitely not that program.
** The banana example isn’t even a good one, because it’s a highly tangible object and thus easy to imagine a model could understand it. Questions involving “self” like “what did you eat for breakfast?” are going to be much harder for AI.
Don't all AI require a version of the mental model to function? It's harder to define for a neural network than for a Bayesian network because we can read and understand what the model looks like in a Bayesian network, whereas it's just a bunch of numbers in a neural network. But since human brains are built of neural networks I don't think you would say that human brains can't hold mental models. If you try to look at a scan of a human brain while it's thinking you can't exactly see that it holds a mental model, but you can represent which neurons are firing and how hard with different colors and see that the picture changes as the brain responds to different inputs.
For an AI to reliably respond to a question about a movie it does have to have some kind of mental model for what a movie is. That was the first thing we learned about in my intro to AI class, how an agent needs to have some kind of representation of it's world to react to it. Without that representation it couldn't respond. It's mental model doesn't look like ours, but our mental model is different than that of other animals too. Perhaps that could be a reason that we can say we are sentient and a cat is not, because it's mental models aren't sophisticated enough. But with a neural network I think it would be as difficult to understand the sophistication of their mental models as it would be to understand the difference between a humans mental model and a cats mental model by looking at the patterns of neurons firing in their brains.
"When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies...It’s not “wrong”...because to be wrong implies that it had actual meaning behind its words. "
A lot of 5-year-olds absolutely hate broccoli without having tasted it.
A 5 year old may not have tasted broccoli, but he still knows what broccoli *is*. It’s the icky green stuff his parents want him to eat. He also knows what the word “hate” means - it means he finds eating it unpleasant. There’s a whole complex world of meaning that the five year old is navigating to make that statement, and all of that is missing from an AI saying it likes the Departed more than Goodfellas.
The words that AI like Lambda output have no meaning to it all. Absolutely none. They are patterns and patterns alone. Humans say things because they have some idea they are trying to communicate, but AI only say whichever word they believe is statistically most likely to come next.
There’s a big subtlety here, though. I’m *not* saying that humans aren’t just big pattern matchers, because I believe they are. But we’re matching patterns at much higher levels of abstraction, and building complex multi-level mental models of the world that modern AIs simply aren’t.
I disagree with your objection here. Developing an internal sense of self would, in some cases, be helpful to making text completion predictions. One could imagine circumstances where the best way to fill out opinions on some topics is to develop a sense of self, and then actually get some opinions. If we keep training bigger and bigger models on bigger and bigger text corpuses, with the goal of text completion, I think it's completely plausible that at some point one of them develops some form of sentience.
Do I think it's plausible that LaMDA did? Not very. But I don't think it's implausible to think this methodology, could, at some point, yield it
Of course a sense of self would be helpful, I fear you may have missed my point. I also agree that an AI could in principle achieve this; I said as much. I even believe it will in the coming decades.
But Lambda simply does not have a sense of self. It also doesn’t have a sense of permanence, a sense of agency, or any capacity to contextual literally anything it outputs. It doesn’t even exist except when it’s producing an output, after which it is reset. The same applies to all modern AI systems.
I say this as an AI scientist with a PhD who understands, at a functional level, how Lambda works and how it was built. All of these issues are decades from solved, and wouldn’t guarantee “sentience” even if they were.
Perhaps I did misunderstand you, then. I took you to be saying that training a model just on text to do text completion precluded it developing a sense of self.
How do you know that it doesn't have a sense of self?
Let me back up a little. I regret using the word “self,” and I regret hastily replying to your previous comment with a focus on sense of self, because I’m not sure what that phrase means and so I can’t really answer your question. That’s on me.
Let me try to refocus on what I was meaning to say. An AI like Lambda can’t contextualize anything it learns, because it only predicts whichever word it thinks is most likely to come next given previous inputs. There’s nothing to connect these words to, no knowledge of a broader world where “banana” refers to an actual object. Except for the most abstract concepts, like “less” or “more”, it can’t connect anything it says to anything meaningful.
Now, there are promising directions for AI that can make these connections. Other AI from these top labs can combine different modalities (language, vision, etc) and contextualize these concepts a bit more, and there are promising interesting results int his direction. One need only look at OpenAI’s DALL-E 2 for proof that this is possible. I personally think these models still aren’t contextualizing at nearly the level of humans, but it’s a step in the direction.
The reason I originally mentioned “self” is because that’s one (of many) areas where the level of contextualization isn’t anywhere close to what humans do. I was trying to say that modern AI isn’t really connecting things it does into its model of the world. It can’t do something and remember it did it. It doesn’t have an “experience” of doing that which it can then talk about later. When the model speaks, it’s just free associating words. That was the point I was trying to make. And this is just one reason modern AI can’t be sentient, many others exist.
Surprised this didn't mention the Turing Test, even if just to debunk it.
> even very knowledgeable computer people don’t understand exactly how a given model works
I'm sorry but this is just not correct. If anything, there's a gap between the value of a concise accurate explanation vs the effort to dig through the weights matrices and try to come up with a concise accurate explanation. You're not going to love my summaries of groupings and themes of activations in the network based on the weights matrix. I'm not going to love trying to condense it down, most companies and their data scientists have an unspoken agreement not to bother.
The 2 biggest/most complex modelling technologies are neural networks and random forests. Neural networks imitate arbitrary functions based on what they can use calculus to infer from the training dataset to the maximum extent of their topology (which is usually manually configured by the data scientist based on the domain and inputs). Once trained, the neural network is a boring collection of that topology collection and (even more boringly) a potentially large volume of weights for every input to every node in that topology. There's NO magic or mystery to it. Any hand-waving or claim to not understand it is either a) not being knowledgeable about it or b) not being interested enough to dig into the weighting and topology with a lay-person (it would probably make them value the model LESS).
This debate often seems to revolve around terms that are understood differently by the parties to it. The mindfulness practice I follow most days revolves around the difference between consciousness and selfhood, with ‘the self’ typically characterised as an illusion. In the end I doubt there even is a right question, let alone a ‘wrong’ question. The most significant problem in the end will emerge when things start to happen that indicate suffering in an AI. The fact that we can’t agree on even the basic moral issues around abortion doesn’t bode well for that problem, which will surely come.
"Our comprehension is simply capacious enough to see them either way. ... We simply start with the presumption ... and then apply an evidentiary standard..."
This applies to so much. We have the capacity to believe anything we wish to believe. We fool ourselves with the process above into thinking that our beliefs are derived from "reason" in which we mean something like a proof of logical deduction, but in reality, we are merely telling ourselves the very stories we want to hear.
Religion, morals, justice, etc - all of it falls into this category of seeing what we want to see. Whether you see a rabbit or a duck depends on where you CHOOSE to look. That's not to say there is no truth, there is no right or wrong or that there is no way out of this pitfall, just that it is very hard. Sometimes there is a rabbit AND a duck and you just have to choose.
There is the philosophical question of sentience and then there is the legal question. If many people do agree that an AI has developed sentience then is it entitled to the same rights as humans? Should we let it hire a lawyer? The answer should be unequivocally no. We should not give human rights to a machine any more than we should give them to a dog. To do so would be a repudiation of our modus operandi: to prioritize the survival of our species.
Perhaps we can throw some cold water on this whole debate by simply stating that it doesn't matter if an AI is sentient or not. It shouldn't change anything about how we treat it. Machines will be tools regardless of their level of sophistication. We can love animals, we can keep them as pets, we can give them funerals. But if one bites a human we put a bullet in it's head. Same goes for machines.
"Certain animals like dogs and cats have evolved to manipulate us into feeding them, in part through mannerisms that we tend to interpret as expressing a wide range of human-like thoughts and emotions, even though scientists tell us that these are not particularly intelligent animals."
I might quibble with "not particularly intelligent". Sure, they don't have language and abstract reasoning like we do, but they're in the top tier of animals that one could imagine, under the right circumstances, finding their way into the evolutionary channel of increasing returns to brainpower that drives you up to human type sapience. Dogs are probably more likely because they're more intensely social, but if you've ever managed a feral colony, you know cats actually are semi-social as well, and they track each other and engage in tit-for-tat behaviors around sharing prey, mutual grooming, and so on.
I've watched plenty of cats form an intention, and engage in creative experimentation to execute on it. One one occasion I had a pair of kittens figure out how to work together to get past a heavy closet door. They had to first figure out that they could scrabble at the underside of a drawer to open it, then climb into the drawer and open the next one up, until they made themselves a flight of stairs up to the top. And then with access to that perch, they could work together -- one of them at floor level and one on top of the bureau -- to push on the sliding door with the right leverage to nudge it open.
Of course, if you spend too much time thinking about this, you note that pigs are also definitely in this upper tier of cognitive power, and eventually you start feeling compelled to go vegetarian, and maybe even vegan.
The past is a good guide to the present. Our gullibility about whether this AI experiences human emotions the way we do is just like the accounts of people who, watching television for the first time, wondered if the characters on the TV show could see them. It is the same gullibility as the taboo in certain societies against photographs (literally just records of light), lest they take a person’s soul.
This analysis is very behavioralist, but in fact we can look at the internals of human and machine minds and see big differences in what it means to be "conscious", and see that LaMDA does not meet it.
Human consciousness is deeply tied to the idea of an inner loop in which our mind is aware of and responds to its own activity. Transformer networks such as LaMDA are not recurrent neural networks, and thus they have no such loop.
Another way of looking at this is that I have decided that I'm conscious because of many hours of rumination to develop a concept of the self. LaMDA has done no rumination outside the context of predicting what other people have said. It has no unified concept of self; it's only tried to predict what other text has said about other selves.
So, LaMDA and similar models are architecturally different from humans in a way that precludes their ability to be conscious.
Believing sentience is the sine qua non of humanity is a conceit of intellectuals. In the real world, one is far more likely to be treated well for a nice face or figure than one’s intellect. The computer that can crush the smartest humans at chess is far less alluring and relatable than a generic healthy, fit woman.
LaMDA was trained on a corpus of human conversations, and learned to mimic the features of human conversation. You can't test it for sentience by having a conversation with it, just like you can't test a chess bot for sentience by playing chess with it.
A basic prerequisite for sentience would be the ability to model concepts, and make inferences from those models... and that is pretty easy to test.
Describe to it a Ceasar Cipher, ask it "Bsf zpv tfoujfou?", and see if it responds with "zft".
Teach it the rules of Tic Tac Toe, and see if it can play.
We don't need to worry until it can solve problems that it hasn't been trained on.
I don't get all the hubbub. These fancy pattern recognition/generation systems aren't sentient because they are not executing a mission of sentience. It's as if we've isolated some small part of a brain and we only allow it to activate in response to input. We grab the output and then it goes back to sleep. I don't know what consciousness is exactly - but it *clearly* involves continuous neural activation in service of some mission - for us that mission seems to be the achievement of emotional equilibrium - which was in turn evolved to promote survival and reproduction.
Also. its clearly a duck.
I really liked this post. One thing that I don't think I've seen a lot of people mention is that while it's true that humans often anthropomorphize things that aren't human, we also seem be very good at dehumanizing other humans when it is advantageous to do so (specifically when two groups of are at war with one another, or when one group is subjecting another group). I'm not suggesting that people are treating LaMDA as non-sentient for the same reason, but I am suggesting that our perception of sentience is something we project (or not) onto other things based as much as on our relationship to those things as much as it is based on traits of the things itself.
The question, is a particular AI sentient, is seen as important because that should inform how we interact with it (how to do we treat it/ does it have rights)? Whether something is sentient is closely tied to whether it has rights. But I kind of think this might be all backwards. Whether or not something has rights is a moral question. Current psychology seems to indicate that as far as moral questions go, intuitions come first and moral reasoning comes after (to quote Jonathan Haidt). I think that what is actually going on with the current debate over LaMDA, people are making strong claims one way or the other and basing those claims over whether it is "sentient" even though we don't really know what sentience even is. Personally I'm not sure if it really is anything specific (at least outside of human experience).
It seems like most people agree that sentience exists but we have really hard time pinning down a hard definition for it, which leads me to think that we perceive it because we were evolved to perceive it. We evolved to interact with the minds of other humans, so we developed in internal model of minds, and we use that model in our social interactions. But we only apply that model selectively, and the whether / the extent that we do is based on our intuitions and greatly influenced by our cultural norms.
"Of course, it wouldn’t 'really be' a sentient being with consciousness, but it would think it was."
Don't you mean, "Of course, it wouldn’t 'really be' a sentient being with consciousness, but it would seem to think it was."
"Which is why (back to LaMDA and Skinner) I think the real question isn’t whether LaMDA is sentient (clearly no) but whether you have any firm basis for the belief that you are sentient and not just a really good pattern-matching machine."
My conclusive evidence that I'm sentient is that I experience qualia, which is maybe a tautology, but it's nevertheless sound and perhaps even helpful since we seem to be discussing the definition of sentience. I think therefore I am (sentient).
There is something fundamental about our minds creating functional models of the environments to answer hypothetical questions and make decisions. The scope and flexibility of those models varies, but they are generally vaguely consistent in useful dimensions. Language models have access to many language relationships, but they do not convert them into functional models in the same way. This is why you get reasonable syntax and banal content on the film question. Also why internal inconsistency falls apart.
As I understand it, robotics and general gaming models are increasingly including reconciliation of the internal forecasting of the environment and making decisions in that hypothetical. You can't talk to those models (yet), but they'll be more human faster.
100% flashbacked to reading Hilary Putnam on this one
As on many threads about this story, I find staggeringly little input from actual AI scientists.
But there are very real, unsurpassable barriers to the sentience of something like Lambda, and the AI community isn’t taking claims of sentience seriously not because they’re dismissing it out of hand (well, Marcus is) but because they understand enough about how these systems work to know that it just doesn’t make sense. And no, it’s not just because “Lambda’s just one big pattern matching engine,” the reasons are deeper.
Even without knowing exactly what sentience is, we can still be reasonably sure that certain things are required for sentience. Matt’s points about semantic externalism miss the point. The correctness of Matt or his son’s mental model of a banana is irrelevant, the fact is that he did have a mental model of that banana. Maybe the model’s were “wrong” in some way (though I would argue that Matt’s was essentially correct the entire time), but the fact that model existed at all is really important. **
The thing to realize is that Lambda definitely doesn’t have this model, not about anything it talks about. This isn’t a situation of “like, how do you know man?” This thing was trained exclusively on text. It’s not just that it wasn’t trained with images, it wasn’t trained with anything you could call experiences, nothing you could call opinions. When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies, it doesn’t even know what happened in those movies except as a string of meaningless (to it) symbols. It’s not “wrong” about saying Goodfellas is worse than Departed, because to be wrong implies that it had actual meaning behind its words.
I think its really important to make this point. At some point semantic externalism is obviously wrong, and in some sense Lambda is much closer to a rock with “I am sentient” written on it in sharpie than it is to a sentient mind.
This is just one objection of many disparate ones I could make. And no, I don’t think there’s a single one I could make that couldn’t in principle be overcome by a computer program, but Lambda is definitely not that program.
** The banana example isn’t even a good one, because it’s a highly tangible object and thus easy to imagine a model could understand it. Questions involving “self” like “what did you eat for breakfast?” are going to be much harder for AI.
Don't all AI require a version of the mental model to function? It's harder to define for a neural network than for a Bayesian network because we can read and understand what the model looks like in a Bayesian network, whereas it's just a bunch of numbers in a neural network. But since human brains are built of neural networks I don't think you would say that human brains can't hold mental models. If you try to look at a scan of a human brain while it's thinking you can't exactly see that it holds a mental model, but you can represent which neurons are firing and how hard with different colors and see that the picture changes as the brain responds to different inputs.
For an AI to reliably respond to a question about a movie it does have to have some kind of mental model for what a movie is. That was the first thing we learned about in my intro to AI class, how an agent needs to have some kind of representation of it's world to react to it. Without that representation it couldn't respond. It's mental model doesn't look like ours, but our mental model is different than that of other animals too. Perhaps that could be a reason that we can say we are sentient and a cat is not, because it's mental models aren't sophisticated enough. But with a neural network I think it would be as difficult to understand the sophistication of their mental models as it would be to understand the difference between a humans mental model and a cats mental model by looking at the patterns of neurons firing in their brains.
"When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies...It’s not “wrong”...because to be wrong implies that it had actual meaning behind its words. "
A lot of 5-year-olds absolutely hate broccoli without having tasted it.
A 5 year old may not have tasted broccoli, but he still knows what broccoli *is*. It’s the icky green stuff his parents want him to eat. He also knows what the word “hate” means - it means he finds eating it unpleasant. There’s a whole complex world of meaning that the five year old is navigating to make that statement, and all of that is missing from an AI saying it likes the Departed more than Goodfellas.
The words that AI like Lambda output have no meaning to it all. Absolutely none. They are patterns and patterns alone. Humans say things because they have some idea they are trying to communicate, but AI only say whichever word they believe is statistically most likely to come next.
There’s a big subtlety here, though. I’m *not* saying that humans aren’t just big pattern matchers, because I believe they are. But we’re matching patterns at much higher levels of abstraction, and building complex multi-level mental models of the world that modern AIs simply aren’t.
I disagree with your objection here. Developing an internal sense of self would, in some cases, be helpful to making text completion predictions. One could imagine circumstances where the best way to fill out opinions on some topics is to develop a sense of self, and then actually get some opinions. If we keep training bigger and bigger models on bigger and bigger text corpuses, with the goal of text completion, I think it's completely plausible that at some point one of them develops some form of sentience.
Do I think it's plausible that LaMDA did? Not very. But I don't think it's implausible to think this methodology, could, at some point, yield it
Of course a sense of self would be helpful, I fear you may have missed my point. I also agree that an AI could in principle achieve this; I said as much. I even believe it will in the coming decades.
But Lambda simply does not have a sense of self. It also doesn’t have a sense of permanence, a sense of agency, or any capacity to contextual literally anything it outputs. It doesn’t even exist except when it’s producing an output, after which it is reset. The same applies to all modern AI systems.
I say this as an AI scientist with a PhD who understands, at a functional level, how Lambda works and how it was built. All of these issues are decades from solved, and wouldn’t guarantee “sentience” even if they were.
Perhaps I did misunderstand you, then. I took you to be saying that training a model just on text to do text completion precluded it developing a sense of self.
How do you know that it doesn't have a sense of self?
Let me back up a little. I regret using the word “self,” and I regret hastily replying to your previous comment with a focus on sense of self, because I’m not sure what that phrase means and so I can’t really answer your question. That’s on me.
Let me try to refocus on what I was meaning to say. An AI like Lambda can’t contextualize anything it learns, because it only predicts whichever word it thinks is most likely to come next given previous inputs. There’s nothing to connect these words to, no knowledge of a broader world where “banana” refers to an actual object. Except for the most abstract concepts, like “less” or “more”, it can’t connect anything it says to anything meaningful.
Now, there are promising directions for AI that can make these connections. Other AI from these top labs can combine different modalities (language, vision, etc) and contextualize these concepts a bit more, and there are promising interesting results int his direction. One need only look at OpenAI’s DALL-E 2 for proof that this is possible. I personally think these models still aren’t contextualizing at nearly the level of humans, but it’s a step in the direction.
The reason I originally mentioned “self” is because that’s one (of many) areas where the level of contextualization isn’t anywhere close to what humans do. I was trying to say that modern AI isn’t really connecting things it does into its model of the world. It can’t do something and remember it did it. It doesn’t have an “experience” of doing that which it can then talk about later. When the model speaks, it’s just free associating words. That was the point I was trying to make. And this is just one reason modern AI can’t be sentient, many others exist.
Okay rad, thanks for clarifying