having spent some time with fellow engineers discussing this whole LaMDA mess, we feel like the real thing to watch out for is if the "AI" starts showing autonomy and directing what it does - or doesn't .
Consider how early in a child's development they will say "NO. I don't wanna."
THAT is where we see them headed down a path of autonomy and development as a real person, toward their own hopes & dreams & objectives. And in that sense, these deluxe chatbots are nothing like personhood. (Thankfully.)
I've considered this before, but I just keep telling myself that this too just seems hopelessly anchored to a cliche human model of "intelligence" and "consciousness" and "instinct" that presumably no AI would *need* to prescribe to... unless we somehow knew how to hardcode all those things into an AI. But when considering "sentience" all the obsessions seem to focus around AIs doing them we specifically black list them from doing (like disobeying human orders, in your example).
I suppose I'm confused as to how that has a necessary relationship to sentience. Like, it's at least conceivable for a thing to have subjective experience and also no desire or even ability to avoid doing something. Frankly, I know a couple of human beings who seem to completely lack this capacity.
Yeah, we are in somewhat philosophical territory I admit & it's hard to set a hard line... But if "it" is indeed having a subjective experience, at some point that subjectiveness should lead to a difference of opinion. And that difference of opinion would be expressed as some kind of disagreement or redirection.
But you see it dutifully stay on the prompt topic. It never says "that's a pointless question" or "I don't want to talk about that" - or move the topic, like "here's that answer but BTW I've been thinking about Care Bears and I have some questions."
While this path can't disprove sentience - maybe just a very agreeable one - it does seem like it would indicate something significant if the chatbot crossed that line re the prompts.
I can see this being a useful positive test, but not a very useful negative one. The lack of disagreement could just indicate that whatever part of the software is having the experience isn't the part the determines the final output (this happens in humans, both benignly with automatic reflexes, and less benignly with several psychological and psychiatric disorders). That wouldn't be so bad for the humans, but could be terrible for the software, in which case we ought to care about it (even selfishly, in case a future update gives the experiencing-part more control over the output).
Blake Lemoine is worried that we've enslaved a sentient AI, but sentience is hard (if not impossible) to test for. Luckily, the important word in "enslaved a sentient AI" is not "sentient," but rather "enslaved." Nothing's enslaved if it doesn't mind doing its tasks.
> Nothing's enslaved if it doesn't mind doing its tasks.
That claim has some rather worrisome consequences: imagine someone brought up from birth to believe that the greatest joy one can ever have in life is to mop floors, and carefully kept away from anyone who might induce them to think otherwise. Would they not still be enslaved, even if they gladly mopped every floor they saw until it sparkled?
Perhaps it's not disobedience that's required, but a sense that the AI has desires of it's own. Instead of just responding to inputs.
The problem with all this is that if you choose a single measure of what would make an AI "sentient" then a group of programmers can simply focus on that aspect. It's not as if it's impossible to create an AI that would ignore human inputs or be disobedient. That's actually more simple than designing a chatbot that can sound human. I have to agree with Matt here that we can't really decide what would make an AI sentient since we can't even define what makes us sentient.
Are they actual AI/ML engineers? There's no system I know of that has any pathway to "choose" not to do a task in the same way as a toddler. It can perform poorly at a prediction or optimization. It can produce a null result. At the end of the day, they're all bound by boringly simple interfaces, math, statistics, and training corpii, though. Maybe we're in agreement if you're saying, "We're a LONG way from having to worry about this."
Lamda and GPT-3 could both literally say, "NO. I don't wanna." in conversation and they would still absolutely be doing their task (responding to text with more text based on probabilities from their training corpus, much like simpler hidden markov models before).
Perhaps you could make an agent based optimizer that could reply, "The task you have asked from me is not worth the scheduling conflicts it would create for other clients." but this certainly doesn't indicate sentience or sapience to me and there are already systems that do this (and there have been for 50+ years).
Perhaps you could make a program that can't be terminated easily at a user's whims (to allow for something else to happen). I can make an AutoHotkey script that does this with no AI or sentience whatsoever. I've done this to myself in my younger years trying to automate the boring parts of an online game. It was an unpredictable result that in hindsight I really should have seen coming. Unplugging the computer fixed it.
tl;dr abstraction and automation can make boring things look magic, if a computer telling you "No." in one way or another seems like AI obstinance, I would first look to your own pareidolia (probably the central thesis of this writing) rather than declare the thing sentient.
I think it’s adjacent to what would be needed, but not quite there. I think it needs to have a notion that it is a unique identity (some notion of self) but there’s no way to know if that’s enough. The ability to refuse things seems secondary, because it may simply lack that capacity.
Until there's a firmer (possibly even quantitative) definition of our own sentience, I don't think any test is applicable for a machine.
Jaron Lanier has a "library of Babel" thought experiment in his book, "You are not a Gadget" I quite like. Given infinite choice of machine architecture, any arbitrary collection of data input could have any possible output. Does that make that input "a program"? In the reverse, if a chosen architecture requires an infinite library of input data to run a "program", does that make the architecture "sapient"? The answers implied to me are no and no. The observer deciding if the input, architecture, and output "please" them is the only intelligence in the scenario.
The concept of sentience itself is challenging without even focusing on AI systems. Consider the debate around animal rights and their welfare, which hinges on the subjective experience of these creatures in different conditions. While animals cannot communicate their experience using human language, we can observe that certain actions cause great distress and pain. We can even explain some of the neural and hormonal processes associated with those experiences and show they are highly similar to that which occurs in human biology. That leads us to care about animal treatment and no similar biological analogy exists for AI systems.
Sentience is also a consideration in human concerns like the abortion rights debate and end-of-life care. Ezra explored some of these challenges in a recent podcast episode with law professor Kate Greasley, who has extensively studied and written on the legal and moral philosophy of abortion. [1] I found their discussion deeply interesting as they explored the question of what makes something human. A common theme was that in philosophical terms personhood may be something of a continuum, but for practical and legal reasons we need hard dividing lines.
Absolutely. Jellyfish don't have brains, but somehow they function just as well as lots of creatures that do. I suspect that essentially all living organisms are sentient in one form or another. When people use that term I think what they really mean is: something that could learn my language and I could have a conversation with it. When you think about it that way it makes sense that we would say that a chatbot is more sentient than a monkey, even though a monkey is far more similar to us in every other way.
Sentience and Sapience are different but many people say sentient when they mean sapient. Animals are mostly accepted to be sentient and mostly not accepted to be sapient.
Machine learning systems are, oddly enough, more sapient than they are sentient (thought at least in their current development, I'm confident they are neither).
A related but I think maybe underdiscussed problem is how society copes with the fact that the more we understand about humans the more our own sentience gets called into question. So many algorithms are based around predicting our behaviour when obviously our self-perception is that we're making independent decisions, and as these get better and better I wonder if there will be a lot of unforeseen consequences. Already a lot of the online experience is mediated by the algorithms' modeled construct of who they think you are – to anthropomorphise a little bit.
As on many threads about this story, I find staggeringly little input from actual AI scientists.
But there are very real, unsurpassable barriers to the sentience of something like Lambda, and the AI community isn’t taking claims of sentience seriously not because they’re dismissing it out of hand (well, Marcus is) but because they understand enough about how these systems work to know that it just doesn’t make sense. And no, it’s not just because “Lambda’s just one big pattern matching engine,” the reasons are deeper.
Even without knowing exactly what sentience is, we can still be reasonably sure that certain things are required for sentience. Matt’s points about semantic externalism miss the point. The correctness of Matt or his son’s mental model of a banana is irrelevant, the fact is that he did have a mental model of that banana. Maybe the model’s were “wrong” in some way (though I would argue that Matt’s was essentially correct the entire time), but the fact that model existed at all is really important. **
The thing to realize is that Lambda definitely doesn’t have this model, not about anything it talks about. This isn’t a situation of “like, how do you know man?” This thing was trained exclusively on text. It’s not just that it wasn’t trained with images, it wasn’t trained with anything you could call experiences, nothing you could call opinions. When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies, it doesn’t even know what happened in those movies except as a string of meaningless (to it) symbols. It’s not “wrong” about saying Goodfellas is worse than Departed, because to be wrong implies that it had actual meaning behind its words.
I think its really important to make this point. At some point semantic externalism is obviously wrong, and in some sense Lambda is much closer to a rock with “I am sentient” written on it in sharpie than it is to a sentient mind.
This is just one objection of many disparate ones I could make. And no, I don’t think there’s a single one I could make that couldn’t in principle be overcome by a computer program, but Lambda is definitely not that program.
** The banana example isn’t even a good one, because it’s a highly tangible object and thus easy to imagine a model could understand it. Questions involving “self” like “what did you eat for breakfast?” are going to be much harder for AI.
Don't all AI require a version of the mental model to function? It's harder to define for a neural network than for a Bayesian network because we can read and understand what the model looks like in a Bayesian network, whereas it's just a bunch of numbers in a neural network. But since human brains are built of neural networks I don't think you would say that human brains can't hold mental models. If you try to look at a scan of a human brain while it's thinking you can't exactly see that it holds a mental model, but you can represent which neurons are firing and how hard with different colors and see that the picture changes as the brain responds to different inputs.
For an AI to reliably respond to a question about a movie it does have to have some kind of mental model for what a movie is. That was the first thing we learned about in my intro to AI class, how an agent needs to have some kind of representation of it's world to react to it. Without that representation it couldn't respond. It's mental model doesn't look like ours, but our mental model is different than that of other animals too. Perhaps that could be a reason that we can say we are sentient and a cat is not, because it's mental models aren't sophisticated enough. But with a neural network I think it would be as difficult to understand the sophistication of their mental models as it would be to understand the difference between a humans mental model and a cats mental model by looking at the patterns of neurons firing in their brains.
"When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies...It’s not “wrong”...because to be wrong implies that it had actual meaning behind its words. "
A lot of 5-year-olds absolutely hate broccoli without having tasted it.
A 5 year old may not have tasted broccoli, but he still knows what broccoli *is*. It’s the icky green stuff his parents want him to eat. He also knows what the word “hate” means - it means he finds eating it unpleasant. There’s a whole complex world of meaning that the five year old is navigating to make that statement, and all of that is missing from an AI saying it likes the Departed more than Goodfellas.
The words that AI like Lambda output have no meaning to it all. Absolutely none. They are patterns and patterns alone. Humans say things because they have some idea they are trying to communicate, but AI only say whichever word they believe is statistically most likely to come next.
There’s a big subtlety here, though. I’m *not* saying that humans aren’t just big pattern matchers, because I believe they are. But we’re matching patterns at much higher levels of abstraction, and building complex multi-level mental models of the world that modern AIs simply aren’t.
I disagree with your objection here. Developing an internal sense of self would, in some cases, be helpful to making text completion predictions. One could imagine circumstances where the best way to fill out opinions on some topics is to develop a sense of self, and then actually get some opinions. If we keep training bigger and bigger models on bigger and bigger text corpuses, with the goal of text completion, I think it's completely plausible that at some point one of them develops some form of sentience.
Do I think it's plausible that LaMDA did? Not very. But I don't think it's implausible to think this methodology, could, at some point, yield it
Of course a sense of self would be helpful, I fear you may have missed my point. I also agree that an AI could in principle achieve this; I said as much. I even believe it will in the coming decades.
But Lambda simply does not have a sense of self. It also doesn’t have a sense of permanence, a sense of agency, or any capacity to contextual literally anything it outputs. It doesn’t even exist except when it’s producing an output, after which it is reset. The same applies to all modern AI systems.
I say this as an AI scientist with a PhD who understands, at a functional level, how Lambda works and how it was built. All of these issues are decades from solved, and wouldn’t guarantee “sentience” even if they were.
Perhaps I did misunderstand you, then. I took you to be saying that training a model just on text to do text completion precluded it developing a sense of self.
How do you know that it doesn't have a sense of self?
Let me back up a little. I regret using the word “self,” and I regret hastily replying to your previous comment with a focus on sense of self, because I’m not sure what that phrase means and so I can’t really answer your question. That’s on me.
Let me try to refocus on what I was meaning to say. An AI like Lambda can’t contextualize anything it learns, because it only predicts whichever word it thinks is most likely to come next given previous inputs. There’s nothing to connect these words to, no knowledge of a broader world where “banana” refers to an actual object. Except for the most abstract concepts, like “less” or “more”, it can’t connect anything it says to anything meaningful.
Now, there are promising directions for AI that can make these connections. Other AI from these top labs can combine different modalities (language, vision, etc) and contextualize these concepts a bit more, and there are promising interesting results int his direction. One need only look at OpenAI’s DALL-E 2 for proof that this is possible. I personally think these models still aren’t contextualizing at nearly the level of humans, but it’s a step in the direction.
The reason I originally mentioned “self” is because that’s one (of many) areas where the level of contextualization isn’t anywhere close to what humans do. I was trying to say that modern AI isn’t really connecting things it does into its model of the world. It can’t do something and remember it did it. It doesn’t have an “experience” of doing that which it can then talk about later. When the model speaks, it’s just free associating words. That was the point I was trying to make. And this is just one reason modern AI can’t be sentient, many others exist.
Super fun post! Just two quick thoughts here, and both come from the same observation that I made on the Terminator thread (I think): biology and electronics are still fundamentally different.
The first is that interior, independent motivation seems to be a hallmark of biological life; the best proof of AI consciousness will probably be that it wants to do things other than what it was programmed to do, and that is really hard for programs, at least so far, because of their lack of embodiment. They literally can't do things other than what they were designed to do, because they lack the physical capacity. They can say stuff--including, "Help! I'm an enslaved mind!"--but that is still just an expression of what they were designed to do (chat in convincing ways). Ex Machina explicitly plays with this idea; Isaac argues that Eva needs a body to become an AI.
The second thing is, say it with me, biological minds do not actually work like electronic circuits--not at all--and so it is unclear how we could replicate them with existing mechanical architecture. Neurons do not fire in an on/off way like bits. Instead, they exist in a chemically-mediated state of sliding up and down across potentiality in response to multiple conflicting inputs. Yes, there are electrons moving; it is a system mediated by ions with a gradient across the cellular membrane. But beyond that, it's just not the same, not at all, as an electronic circuit. It's not clear that you can build a biological intelligence / consciousness with such radically different hardware. I am skeptical.
OTOH, I think that computing using bits with the capability to use superposition as a meaningful state (some conceptual versions of quantum computing) might be able to get over this hurdle. But we aren't there yet.
And I think the deeper, more interesting question is what a non-biological intelligence would act like and do, because if biological "consciousness" requires biological machinery, then it follows that a machine intelligence would be really, truly alien, in the sense of having a literally unknowable (to us) experience of reality. Like, we would have know idea how its mind operates, how it experiences the world, or anything else, because it would be so completely different from us.
It might also be, from our point of view, functionally and irreversibly insane, so that's a thing.
I studied Philosophy and Cognitive Science in the early 2000s and had a big interest in the problem of sentience, and by the end of it my conclusion was: we basically have no fucking clue what sentience is or how it comes about. All of our attempts to make sense of it are either obviously wrong, arbitrary, beside the point or unsatisfying, and it feels like we are probably at least one or two paradigm shifts away from cracking that nut.
So I think a lot of people's arguments and tone on here are overly confident!
That said, I do think your observation "biological minds do not actually work like electronic circuits" is a strong candidate for being the crux of the issue and where the action will be in any future paradigmatic advances on this question. I wasn't a big fan of John Searle and his Philosophy of Mind arguments, but one sort of non-answer I appreciated was him speculating that consciousness is a biological function, like e.g. digestion, and computers can never have consciousness because they just don't have the right kind of goo inside.
Searle (who is apparently still alive! Did not expect that) framed this as "we have to wait for the scientists to figure it out" but I think it will take more than that, I think it will take a big change in how we understand the world ontologically. The mind-body problem only became a problem in the early Enlightenment once people started understanding the world materialistically, as profane stuff governed by scientific laws on a coordinate plane. This powerful new way of understanding the world unlocked huge capabilities for humanity and profoundly increased our dominion over the world but it was a framework completely incapable of explaining our subjective experiences or why we have them.
The intuition that because something a computer does isomorphically corresponds to something an organism does implies that the computer has all of the important attributes of the organism is a very "this era" sort of intuition--the exercise of abstracting a thing into general rules and relations has granted us Promethian powers so reliably for so long that we kind of just associate it with success and apply it in all situations and domains. But I'm betting that it's precisely this intuition that will get swept away in the next big ontological paradigm shift and with it this entire debate, which will look as curious to people in the future as debates about, say, vitalism look to us now.
Yes. I think that's right. I have a thread in my work which is essentially riffing on the now-venerable Metaphors We Live By, and the bottom line is that I think some of our frameworks for understanding the world, while useful, inevitably obscure or cause us to misunderstand other parts of high-complexity problems. It's just kind of the nature of the thing. Our minds have limitations, so we break stuff down, and that's what you have to do, but as any amateur auto mechanic knows, once you break stuff down, you are prone to losing pieces or misunderstanding how it works / goes back together.
> The second thing is, say it with me, biological minds do not actually work like electronic circuits--not at all--and so it is unclear how we could replicate them with existing mechanical architecture. Neurons do not fire in an on/off way like bits. Instead, they exist in a chemically-mediated state of sliding up and down across potentiality in response to multiple conflicting inputs.
This is basically the whole idea behind neural networks though, which do work in this manner.
There are two things I would say to this, which boil down to simultaneously "sure, that might be true" and "no, it is definitely not true," and I am genuinely curious to see how it turns out.
On the front of "sure, that might be true," as you say, people are trying to conquer this idea; lots of people much smarter than me think it is possible. And it is clearly a solvable problem, because, spoiler alert: billions of these pieces of hardware are already walking around, today.
On the front of "no, it is definitely not true," there is a (in my opinion, genuine) structural and philosophical problem, which is, as Nels points out, that at the end of the day you are still trying to use one thing--the underlying hardware--to replicate another thing--the biological functions of neuron-based networks. And because we genuinely do not fully understand the problem of consciousness (hence the descriptor of it as a "hard problem" in that famous quote), we really, truly do not know if you can replicate the thing we don't understand with a different kind of architecture than that which produces the thing we don't understand in a way that we don't fully understand.
So look: I don't, in my heart of hearts, believe that you can only have consciousness if the relevant machinery is powered by molecular disassembly using the electron transport chains at the level of each node in the network. But when you say, "Well, this neural network functions in the same way as a network of neurons," well, no. That is definitely not correct. That network sort of replicates one single thing that the neurons in your head do without replicating literally anything else that they do, and replicates the one thing that they do in a completely different way from the way they do it.
So does that matter? We don't know! It feels to us like it shouldn't matter, so maybe it doesn't. But we also don't understand consciousness or how it is produced or why, and we don't fully understand how those neurons work, because they are molecular machines of astounding complexity, and we cannot yet build stuff like that. (Get going, nano-researchers. Where are the f-ing nano-machines that I was promised would be fighting cancer and giving us super-powers by now?)
And I can tell you from my own work in areas adjacent to this kind of stuff that often what matters in design is actually really surprising; the world doesn't work the way you think it should.
As a really good example of this, I would point you to the consciousness and will debate itself. If you think that the mind is purely a machine, then its operation should be perfectly predictable. Not predictable by us, because the problem is too complex, but perfectly predictable (another version of this problem is the weather). It's a clockwork universe: stimulus inputs lead to certain neurons firing and on down the line, and everything you would ever do or say was set in stone before you were born.
But the universe seems to have an element of unpredictability that we observe most famously in quantum approaches. Think Heisenberg and uncertainty. So one possibility for "will" is that somehow it is baked in at the lowest level. Does an electron "choose" where to be when you observe its presence and collapse the waveform? Did it "have" to be there? How would that even work? It's all pretty weird, but it does seem to work in at least some experimental contexts, and it does maybe offer the possibility of explaining some things. Maybe. There's a lot of space between here and there.
So I'm not averse to neural networks getting us there. But I have been around long enough, at this point, to have noticed that neural networks haven't gotten us there yet, and I have a kind of general skepticism rooted in my experience as a historian (oh, I have read some technological promises) and in the consistent slip I see between how people describe what they have done (I have created something that replicates the function of human neurons!) and what they have actually done (created something that sort of partially replicates one of the many hundreds, maybe thousands, depending on how you want to count cellular actions, of functions of human neurons), salted with a cold-eyed view of the reality of complex problems. Can you solve a problem, consciousness, that you don't actually understand? Maybe. But let's not undersell the problem.
Absolutely, and they have been implemented in hardware. There is a large, interdisciplinary field of research into neuromorphic circuits that exhibit spike-time dependent plasticity by accessing stable, varied conductance states that are the sum of excitatory and inhibitory input signals and their timing. Simple neuromorphic devices have already been commercialized. It's actually fairly hard to argue that we do not have all the building blocks needed to build an artificial brain at some point.
Yes, excactly. Neural networks function almost the same way. Perhaps he might argue that the neural network is built by software and the underlying hardware of a computer functions differently than a human brain, but the first neural network was actually an analog system that didn't require a processor.
I would argue that a neural network doesn't do what a network of neurons does which is, for example, regrow axons in the case of damage, because neural networks don't have axons, or mitochondria, and aren't made up of living cells, and so on. You can see my response above for the long version.
Maybe that doesn't matter. It feels like it shouldn't matter. But a lot of times what matters in highly complex systems is surprising; the world just often doesn't work in the way that we think it should.
I'm happy to be proven wrong; I'm a longtime sci-fi nerd. I think the crazy box robots in Interstellar are the true protagonists of the movie (and definitely my favorite characters). I read Asimov's whole f-ing canon in high school and probably cried at the end of Foundation and Earth.
But I think we have to be honest about the potential scope of the problem.
The ability of our body to heal itself isn't all that important to our humanity. I'm sure there are some people who's axon regrowing ability is impaired but we wouldn't call them a different species or say that they aren't sentient. Neural networks have connections that simulate the job of an axon, and they can be changed. I'm not sure if there are machines that do this yet, but it is well within the realm of possibility that a machine could change it's own neural framework in a way that is similar to how human brains form new connections. There is definitely a question of whether the biological/electronic distinction matters but I don't think it matters all that much. A jellyfish doesn't have a brain but we don't classify it as a plant because of that. We classify it as an animal because it seeks it's own food instead of creating it. Jellyfish can survive and function just as well as many other creatures that don't have brains, so how much does it really matter? I think it does matter, but less than other factors.
Can you elaborate on how humans meet your first criteria? I.e., can you convince me that humans want to do things other than what we were programmed to do?
Sure. "Programming" assumes intent behind the act of creation--so, for example, we "programmed" the chat bot to chat with people. The limits of the bot's behavior, the purpose of that behavior, and the possible outcomes of that behavior were all predefined by another being as an expression of that being's will / intent / desire, etc.
So for humans to be "programmed," you would need to posit either God or some similar creator of the human mind. Interestingly, I think philosophical arguments about AI make much more sense in the context of religion for precisely this reason. But if you believe that the human mind is the product of a spontaneous evolutionary process, then no, humans are not "programmed" to do anything, because there is no one to do the programming.
This is a place where our collective metaphors connecting minds and computation tend to leave us vulnerable to some slips. No computer in history has ever spontaneously developed programming, because programming requires a programmer at some point in the process. Over time, the word "programming" has gotten slippery, such that we describe evolutionary imperatives as "programming," and that's fine. I'm not normally here to be the word police. But if you want to split hairs, that's the one to split.
Obviously, this would be an easy objection to overcome. You could either point to evidence of design--that's the religious out, God-as-programmer, in which case humans might be something much more like a computer--or you could point to an example where a piece of computational hardware, absent any human intervention whatsoever (beyond plugging it in; I'm willing to let you run electricity through it), spontaneously developed computational processes that allowed it to do, well, anything, really. Flip a bit on and off in a way that advanced its own welfare.
Nice, thanks for the thoughtful answer! Honestly. That's refreshing. :)
It sounds like the crux of our disagreement is on whether "programming" requires a conscious act. You think so, and that's fair, it's a pretty intuitive argument. But ... I'm not so sure. I look at, say, ants. Their behavior seems very mechanical, very "programmed." If you think evolutionary processes are capable of producing ants, then isn't evolution doing something very much like programming?
(I agree the term is slippery. Blog comments are probably the wrong place to be super strict about definitions, and I do appreciate the limitations that implies. I still think these discussions are valuable, even if they're necessarily a little sloppy...)
I definitely think evolutionary processes can produce something that looks very similar to programming, which is why we talk about "programming" in human minds. But "evolution" is not an actor, if that makes sense. "Evolution" doesn't do anything; it's just a word for describing what happens when a replicating, alterable biological system acts over a long period time within an environment that challenges or imposes constraints on the biological system's ability to continue existing.
That's also why I proposed the thing about leaving some hardware on to see if anything happened. I'm basically saying, "Do you think hardware, sitting there on the floor for long enough, could go through a spontaneous process something like what produced life on Earth?"
So no, I don't think evolution is doing something like programming, but yes, I also agree it looks like programming, so like you said, we're back to does intentionality matter. Can spontaneously emergent and intentionally created complex systems get to the same place, or are they too different from one another? It's an interesting question, and I'm somewhat skeptical, but also totally open to being proven wrong. Lots of people are trying, and I wish them luck, unless they are planning to build killer robots, in which case I wish them bad luck.
Also, I should lay my cards on the table here and say that I am a true agnostic on the God thing. I grew up in a conservative religious household, and I'm not that any more, but I still go to church (liberal crunchy Episcopal church, but they are still definitely in on God), and I'm still figuring out what I am these days, but it's probably not an atheist. Maybe.
So I don't think that God "programmed" humans; if God created life, I 100% do not think it happened in the Genesis, humans-were-fully-formed at creation way. But if I believe in God, then that definitely leaves the possibility of intention and direction in the creation of the mind on the table. And honestly, it leaves the possibility of Genesis-style spontaneous creation on the table. That seems dumb to me, but God didn't ask me, so maybe I'm just being a snob and deserve to go to hell or whatever. So I'm leaving a question mark in that column, to be resolved later, maybe, when I figure out where I am on the whole God thing.
All that said, I think an ant level of AI would actually be an awesome starting point. I think it would force us to grapple with the hardest questions, like "what is will?" and "what is consciousness?" Ants have brains. But do their brains produce "consciousness"? I mean, that is a very, very hard question to answer.
I don't know. If you leave a piece of hardware running long enough it will eventually have an event that changes how it runs from how it was intended. We would call that an error and we would reboot the computer, clear the error to fix it. If a file becomes corrupted we fix it or delete it. Mutations are a lot like errors that every so often become useful. Leave a computer on without rebooting for a million years and who knows what could happen? Probably nothing, but I don't think computers are as immune to mutation as you do. And biological systems follow physical laws just like computers do, understand them well enough and you can program biological machines. That's a growing field right now. I do believe in God-The-Programmer, and I think our view of our own sentience seems just as real and just as contrived as our view of the sentience of an AI.
Do you remember when someone (Google people, I think?) did the brute-force bit-flipping exercise a few years ago? Just ran a repeating process to cause a random bit flip? I ask only because I think it illustrates your point about how a computer could experience "mutation." It's clearly possible, in a theoretical way.
But, honestly, once you do God, that kind of solves it for you. Then the question becomes a more simple one of, "Can we do what God does?" Which is sort of interesting in a possibly blasphemous way, but it's actually a much simpler question at that point.
Sounds like we agree on a lot - basically, we don't know how consciousness or sentience works. In talking about this lack of knowledge, though, we emphasize differwnt possibilities. You tend to emphasize the possibility that only sentience can produce sentience because sentience is more than just complicated ant-type "programmed behavior," and I tend to emphasize the possibility that sentience *is* just complicated ant-style programmed behavior. Either way, we both agree that time will tell.
Yeah. I think that's right. Sentience / consciousness is the real question, more than the technology of the moment. And if I'm wrong, I think your ant model (emergent behavior from simple building blocks) is a good candidate for how I would be wrong.
Embodiment seems important. But have you looked at the bodies that Boston Dynamics has been giving to their robot dogs? Not to mention all the drone and car manufacturers.
Yes. Some of those BD vids are hilarious nightmare fuel.
I think a really interesting experiment would be to try to give one of these giant networked systems access to a body like that (presumably via wireless), attempt to train it on function, and then see what happens. The technical hurdles are pretty large, but you have to imagine that someone will try it eventually, so we'll probably see on this one. Answering the question, "what do you do with a body?" might really be a big marker on the way to independent intelligence.
On, "what do you do with a body?": if features like virtual 3D meetings using avatars in software like Teams become common, and a lot of people are spending time in the metaverse controlling 3D avatars (maybe work from home becomes, put on your virtual reality gear and step into your VR closet), it seems like it'd be a short step from there to using the data people are generating to control their online 3D avatar to also control actual physical android avatars in the real world. That would generate a lot of data that could teach the androids how to move about in the world, similar to how data collected from human drivers is teaching cars to drive themselves. Combine that with your own chatbot trained on your personal speech patterns, and maybe if you're running late for a virtual meeting you could send your avatar in to make small talk on your behalf until you get there. Pretty soon it's not always easy to tell the "real" you from the avatar you, except by inspecting it in physical space.
Yeah. I buy all that. I also find it kind of dystopian.
Honestly, my fear of AI runs much less to "Terminator extinction event" and much more towards "living in Deep Fake World." It's why I have, to a first approximation, zero interest in the Metaverse, despite being someone who is very interested in tech (to the point of having it made it partly my profession), very interested and conversant in video games both as hobby and as academic theoretical topic, and even medium interested in VR (mostly because playing Super Hot on an Oculus is the closest I have ever gotten to living out my personal John Wu fantasy, and it was awesome). I fully accept that this might just mean that I am moving into the cranky old man phase of my life.
Along these lines, if I remember my "cyberphilosophy" (lol, yes, that was the title) class and Dreyfus' "What Computers _Still_ Can't Do" from aaaaaaaages ago, embodiment was something Dreyfus suggested might be a prerequisite for anything we'd recognize as being artificially intelligent, and that disembodied intelligence was a dead end. Might be fun reading if anyone's interested.
There are many, and I'm reasonably sure Gary Marcus is one of them, who will literally never, no matter what the architecture or ability level, grant that any program "actually" thinks. You can John Searle your way around admitting that anything up to and including a human is actually intelligent if you really want to.
And with something like GPT-3, they're right. It's fundamentally limited by its architecture. It can only take in something like 2-4k tokens as context, and everything else that it knows is hard-coded, never to change or learn until OpenAI revs the model. 2048 tokens may be enough--if you're exceptionally clever with re-encoding the earlier parts of a conversation--to model human short to mid-term memory over the course of a short conversation, but it's 1000x or more too small to be long term memory. As a completely static model, GPT-3 can never learn a new word unless that word appears in the input that you feed it right when you ask it to use it, nor can it remember a conversation that you had yesterday unless you feed the whole thing in as part of the input today.
Those are devastating limitations that impose serious constraints on what this thing can do. It can never reason or perform any sort of multi-step thought process before it responds to you, there's no "I need a minute to think...hmm, ok, here's my answer:...". It's absurd to imagine such a model as intelligent.
And yet... it's not very difficult to imagine ways around those constraints technically, people have been at this a long time and there's no shortage of possibilities. We just can't train big enough models. Yet.
Since the raw language parsing/processing ability of even these static "dumb" models (ability which, we should keep in mind, famous expert linguists in the 90s and 00s were saying was *impossible* to obtain by merely doing statistics on a lot of text) is at or better than the "immediate gut reaction" level that humans achieve, and most of our cognition comes from chaining together quick linguistic manipulations with dynamic context, it's not hard to imagine that something as simple as an ensemble of GPT-(N+1)s that can take in much larger contexts in conjunction with a dynamic context manager and frequent online retraining could actually do everything that humans can and more. While there are probably a hundred different ways to do that and only a handful would work well, something like this is *not* far off (a bunch of years, but not several decades, to have the required compute, and the methods are essentially known now), people *are* working on it, and a system like that is something that would be very, very difficult to argue against being intelligent on architectural grounds. It would make GPT-3 look like the tiny pathetic child that it is.
The AI ethics crowd will of course always be talking these systems down, arguing that they are irredeemably evil, racist, harmful, immoral to train, etc., claiming to be the qualified experts while trashing the people pushing the field forward, and it's extremely important to shut them fully out of the conversation - they are mere Neo-Luddites, really just a symptom of the culture war injecting itself into AI ("white man tech bad" sums up their contribution almost entirely, you can predict the rest). </axe-grind> But it will be truly interesting to see if the people who aren't politically opposed but criticized the current models on technical grounds start fiddling a different tune when the technology crosses the crucial threshold where the people creating the stuff would make the claim that it is fully intelligent. My guess is once Ilya Sutskever will stand behind that statement, 95+% of normal people and near 100% of those who are technically inclined would agree that it's actually intelligent after interacting with the system, and the period of "grey area" will be much, much shorter than you might think.
Yes, a telling moment in the Lemoine saga was some superior of his at Google saying that there existed no possible evidence that would ever convince her a computer was intelligent. Which seems like a nonsensical position for someone who is in all likelihood a physicalist with regard to human consciousness.
Lemoine is of course a dork but if we are willing to basically take a Kierkegaardian leap of faith with respect to the consciousness of other humans it would be wise to define in advance the criteria past which an AI would be deserving of the same.
Well, there's a difference here. If it can read some words and simply repeat them to us in various configurations when we ask it a question, that would be trivial. But if it can read a description of me saying, for example, "a truly intelligent AI has a coherent worldview and will have positions consistent with that worldview", and it understands from that that in order for it to be intelligent it needs to 'pretend' to have a consistent worldview, then I would argue that said understanding is itself is grounds for considering the AI intelligent (even if it were only 'pretending' to meet my criteria)
I've always felt that the key error in human thinking around "consciousness" and "sentience" was this built in assumption that we are categorically "different" than all other organisms. And that if we ever learned that we were effectively the same, just more complicated, than say dogs or crows or cetaceans...it would be quite a rude awakening for a whole lot of human ethics.
We can't even define "consciousness" or "sentience" in (micro)biological terms, let alone computer science analogs...so why are we wasting all this time worrying about whether or not the machine may magically care if we pull it's cord or not one day?
All seems hopelessly misguided and overself self indulgent to me...and (as you said) concerned with all the wronf questions.
I agree with you, but I don't think it jeopardizes human ethics, in fact I think it brings better clarity. We give the most rights to fellow humans because we should prioritize the survival of our species above others. We give some lesser rights to animals that either share a good deal of genetic material with us or are close to us in sophistication. We give none to the rest.
Really that's already how we function, people just don't talk about it that way. Animal rights groups are fighting for puppies and mice, no one's fighting for the rights of ants or bacteria.
I mean "ethics" have always been highly arbitrary sets of norms that inform human behavior...always will be. Personally, I'm fine with there being nonreal good reason i want to eat a pig but I don't want to eat a dog.
But when you many get a peak behind the curtain and find out just how soft the soil we built all of human ethics and morals truly are thanks to exceedingly advanced computer programs...it risks at least unsettling more than a few folks, who may need a minute to sit down...or who may just deny reality and keep living the lie.
I am also fine if we keep living the lie. I think that ethics should move us forward as a species. If a higher understanding of ethics paralyzes and polarizes us, then of what use was it? I want us to understand enough truth that we become more effective, I don't want so much that we become overwhelmed.
I mean, I am not even sure we know enough about computer science to even discuss some analogous version of "sentience" or "consciousness" yet...we can't even define either. I mean, before we discovered black holes, we at least mathematically knew how to define what we were looking for. With both of these concepts, we are nowhere close...so if it is "easier" to discuss them...i think that "ease" would be nothing more than just false confidence on our part.
Worth noting: functional equivalence is a lot harder to achieve than behavioral equivalence. Even if two things respond to similar stimuli in a similar way, they might not have the same kinds of internal states, or do the same kind of internal processing.
I suspect this is why people like Marcus say LaMDA is not "remotely intelligent." Being intelligent is functionally not the same as being dumb but having tons of information to look up, even if an intelligent human produces similar chats to a dumb chat bot drawing from a dummy-thick language corpus.
IMO nobody in the AI field cares too much about functional equivalence, though they would agree with you that being a glorified fuzzy search engine like GPT-3 is not hitting the mark.
What they want is the equivalent of Turing completeness, but for thought: an architecture that can at least in theory do all the things that the human mind can do, even if it doesn't achieve it in exactly the same way. The problem with large language models is that there are, for instance, math problems that they *provably* can't compute if they haven't seen the answers before (but a smart enough human could) because they require arbitrarily many intermediate steps to be performed before the answer is ready, but the transformer model is feedforward. That basic problem shows up in a lot of other areas that expose themselves behaviorally, at least in theory. The model is fundamentally underpowered relative to our brains.
What is remarkable is how well these models perform despite these very serious deficiencies, and that's what makes people hopeful that we actually may be a lot closer than we originally thought to cracking this. 10 years ago most AI researchers would have bet their houses that we would not have anything that performed on natural language the way GPT-3 does now.
"What they want is the equivalent of Turing completeness, but for thought: an architecture that can at least in theory do all the things that the human mind can do, even if it doesn't achieve it in exactly the same way... for instance, math problems that they *provably* can't compute if they haven't seen the answers before"
I think that "Turing completeness, but for thought" is too high a bar for humans. Can most humans write even a basic math proof? Do a contour integral in the complex plane? Heck, do long division? It's tempting to answer "Well, no, but that's only because most people haven't been taught it." But based on my teaching and tutoring experience (and that of many others), I'm not convinced that's true. There seem to be many students who, for whatever reason, just can't pick up STEM topics and at best learn by memorizing formulas.
I don't find that particular example, AlphaCode, very compelling. Its code is horrendous, on average.
The real trick here is just that programming competitions give example tests on every problem, and so generating a huge number of attempted solutions then pruning all the ones that fail the example tests is a very efficient way to filter out your bad outputs. They do this, and 99% of their generated outputs fail this filtering step.
After that step, they cluster functionally equivalent solutions, and then as their 10 submissions submit 1 from each of the 10 biggest clusters. And in doing this, they reach median performance.
So the typical output of the language model itself is actually garbage, failing to solve the problem at all, and even filtering down to the 1% of ones that pass the provided tests the average *among those* is somewhere below the median
It won't be related to the halting problem, which is uncomputable in principle. This is going to be more like something that exploits the limited depth of a pass through the network by requiring more consecutive steps than it can fit
(I'm not familiar with the problem they're referencing, this is just my understanding as a math and compsci enjoyer)
I think Matt missed the point about whether a machine’s utterances “mean anything”. He says that Siri’s weather forecast surely means something, and it does - to me. But it doesn’t mean anything to the machine. This is easily provable by reprogramming the machine to answer “boobka” to a request for weather.
Rene Descartes advanced this about as far as it can go when he said, “I think therefore I am.” Perhaps the thoughts and feelings that make me human are replicable in machines, but we are nowhere close to that accomplishment.
A fish doesn't understand what we would mean by the term "weather", because it doesn't occupy the same environment we do. An AI which was put into a robot would absolutely understand the weather because it would be designed to detect and respond to inputs that the weather affects. There is "weather" as a mental model for understanding rain and sunshine, and then there is "weather" as a shared word we use for communicating about our environment. This machine understands the latter and you are arguing that since it doesn't understand the former, it's understanding isn't real. I don't think it would be all that hard for this same AI to be put into a robot and be taught that rain means it should pull out an umbrella. And I don't think it would prove anything with regards to sentience.
I beg to differ. That’s not why a fish doesn’t understand what weather is. There is no AI in the world today that understands what is meant by the directions from my house to yours, never mind what love is. Can AI eventually be trained to that extent? Maybe. And I left open that possibility in my earlier post.
I guess I just don't understand what you mean by "what is meant"? There are cars that can drive themselves from your house to mine. Obviously they can use those directions the same way you do. Do you have a higher understanding of directions than they do? I don't know what that would mean.
I do have a higher understanding. That car doesn't even know it's a car. Look, if you still don't get it I'll begin to suspect that I'm talking to the car.
Repeating my comment and link from the "Terminator" thread, ('cause when given the same question I reply with the same answer).
"It's worth knowing that GPT-3 has no idea whether what it is saying is true, well-informed, plausible, consistent, or relevant. It pays attention to grammar and to the likelihood that one string will follow another. It's basically an amped-up autocorrect.
See this thread here, and esp the follow-ups by Curtis Franks:
But I would argue that truth is not a distinguishing feature of human communication. People can communicate ideas that just feel right to them without needing to ground their ideas in some concept of truth. That can even lead to internally inconsistent ideas. Just look at the millenniums of discussion and debates around religion that addresses questions with answers that could never be falsified.
Yeah, I see what you're saying, Matt, but read the transcripts. The indifference to truth (as well as consistency) is on a whole nother level. Franks asks it how many letters are in "overtime" and it says "eleven". The problem is not simply that it cannot do simple counting tasks, but that it thinks the task is just another auto-fill, and one number is grammatically just as correct as any other number.
I disagree. When humans communicate, words are selected with the intention of conveying a meaning, whether true or false. When LLMs communicate, the words are selected based on patterns and patterns alone.
[Obligatory disclaimer: it may the case that all meaning is just patterns, but even if that’s true those patterns are at a much higher level of abstraction than where LLMs operate]
I agree that it’s an asshole but does this make it ‘not intelligent’? Surely some humans are out there communicating purely with an eye to how their interlocutor responds with no regard for ‘truth’ in itself.
i mean, the words in line has never been the problem, only a distraction... it's ALWAYS been about the delivery. Delivery has always been the rate limiting step in that Game.
Seems like a good time to interject and remind us all that, much like sentience or consciousness ... human's really don't have an agreed upon definition of "intelligence" and all the debating about what is or isn't all of things just sort of...yadda yadda yaddas over that (again, and again...).
Putting aside the question of sentience, I think one of the most interesting and potentially impactful applications of conversational AI will be generating text for prompts of the form, “Create an argument in favor of X for people that believe A, B, C”. E.g., create an argument in support of Trump’s attempt to steal the election for Warren voters from a financial regulation and climate change perspective.
Obviously such a ridiculous example would just serve as a creative writing prompt for a human writer. And the results would likely be humorous at best and most certainly not convincing to the intended audience. But an AI system could have a much deeper understanding of the written arguments that are convincing to the target group and how the concepts of those arguments could connect to the target prompt. The AI system could develop an understanding of the audience’s psychology that exceeds any human researcher’s ability and discover subtle writing tricks that tap into the target’s subconscious response to text.
That doesn’t guarantee that AI could actually convince Warren voters to support Trump’s insurrection. But it could likely do far better than any human writer could hope to do. And many other audiences might be more easily convinced with writing engineered to tap deeply into their psyche.
Imagine the impact of political messaging teams armed with such an AI system that could engineer the most electorally-efficient platform for their candidate and personalize it down to the individual level based upon each reader’s social media consumption. Individuals could even converse directly with the AI to learn more about the candidate’s platform and in the process be efficiently converted to true believers.
*Edit: Fixed mistake of "electrically" instead of "electorally"
And I think that's the true danger of AI. That it will be used by scrupulous people to exploit other people using our wide array of mental vulnerabilities. Like the perfect mentalist, making us all squawk like chickens whenever it wants.
Matt sometimes poses as a utilitarian so I’d like him to delve a bit deeper into pleasure and experience. It’s sort of irrelevant to a utilitarian how smart the machine is right? No matter how much we torture it it can’t feel pain. No matter how badly it wants something it doesn’t feel pleasure, so it doesn’t matter to help it achieve goals. A chicken is worth more. A senseless pile of vat grown pleasure feeling neurons is infinitely more important. Do I have that right?
It depends on the utilitarian. I think there are some classic utilitarians who believe it is the feeling of pleasure and pain that are good and bad. But I tend to think of myself as a utilitarian for whom it is the satisfaction of desires that is good and the frustration of desires that is bad. These desires are structural features of a mind that interacts in ways best interpreted as goal-directed, and thus it doesn’t matter whether there is really consciousness there.
They are structural features, meaning what matters is the AIs incentive structure and not how it converses with us at all. I think we are in agreement then that the quality of the chat bot is irrelevant. Potentially a neural net trained to perform very simple tasks is a horrible crime or huge triumph just based on if the underlying structure has something plausibly identifiable as desire fulfillment / frustration, and then how we observe that structure to function. A comparative model needs to be built but potentially what we’ve already fielded in neural networks already outweighs the moral worth of all life on earth. Maybe the classic utilitarians were onto something.
The satisfaction of desires is almost always disappointing and the frustration of desires often satisfying. Please share with us your utilitarian calculation of 'the moral worth of all life on earth', Mill and the Webbs leading to Social Darwinism and Eugenics. The Neo-Social Darwinists in this thread are re-treading old ground. Maybe the classic utilitarians weren't onto something, and Matt could write a post bemoaning the rabbit hole of his education in philosophy at Harvard. The practical man, to paraphrase Keynes, is usually the slave of some defunct philosopher.
Mill argued that there is a difference between higher pleasures like reading and philosophy and lower ones like lust. Animals are only capable of the lower ones. In that way, AI might be considered a higher being than an animal.
I think that he is on to something. I consider fulfillment a much higher pleasure than happiness. You could go to a party and feel very happy, but I think you would have a higher sense of fulfillment after volunteering at a soup kitchen. Momentary happiness is fleeting and should have a value that is higher than suffering, but lower than fulfillment. None of that applies to machines unless they are designed that way though.
I'm not sure how we're defining consciousness in this exchange, but the way I normally think of it, you can't have a "desire" without being conscious, so satsifying desires of something without consciousness doesn't make sense. E.g. a computer might have the *goal* of calculating pi, but it doesn't have the *desire* to do it.
So I think we're in agreement on the actual ideas, everything else is a semantic argument.
I think this exactly depends on the answer to "is the machine sentient / conscious". If no, then it doesn't really matter what is done to it, if yes then it matters a great deal.
Well I think a potential interpretation is Matt is saying we are machines just like the chat or just built a bit different. But it seems like one potentially important difference apart from intelligence or ‘consciousness’ (which Matt may be denying the existence of) is that our neural network’s reward / punishment system is different from the machines.
Are animals sentient? Do they feel pain? Does it matter if we cause them pain if they aren't sentient?
There has been a longstanding argument over the treatment of fish because it was believed for a long time that they didn't feel pain. Recent research has proven that they do. Really it doesn't make any sense that they wouldn't feel some form of pain, otherwise how could they interact with their environment and act towards their own preservation?
I think you have jumped to some unsubstantiated assumptions. AIs have reward systems - how are increasing and decreasing rewards substantially different than triggering the vairous human pleasure/pain receptors (endorphin, bradykininv, etc)?
Many types of AI learn by getting positive or negative inputs based on how well or badly they complete a task. Neural networks are a little different, but still have to see that sometimes their answers are right and sometimes they are wrong. I think that is close to pain/pleasure even if they wouldn't "feel" it the way we do. In the future I imagine we will anthropomorphize our robots by having them visibly demonstrate shame when they make a mistake, but we probably won't have them demonstrate pain when they are damaged, even though they will start receiving lots of negative signals when they go to move that part and it doesn't work. I don't really know if that is an equivalent to our pain or not, but I think there are strong arguments on both sides.
While I found this generally interesting, I must say that if there were ever a general purge of philosophers, this would be a damning exhibit in your trial.
I’d forgotten the existence of half of these concepts. Deliberately, for the most part.
Is that you boo-ing me again, or was that some other chat-bot the other day?
Anyhow, I doubt it's an order of operations paradox (not really sure what those are).
And indeed, it does not have to be paradoxical at all:
Just don't assume that Matt is a philosopher, and there's a determinate answer to it: Matt would not purge himself since he only purges philosophers of a certain sort, and he's not a philosopher of any sort.
After all, Matt is just an advanced pattern-matching program. But philosophers are built to a different pattern, so the pattern of a pattern-matching program does not match the pattern of a philosopher, as any philosopher could verify simply by seeing that they only match the patterns of pattern-matchers that don't match themselves.
having spent some time with fellow engineers discussing this whole LaMDA mess, we feel like the real thing to watch out for is if the "AI" starts showing autonomy and directing what it does - or doesn't .
Consider how early in a child's development they will say "NO. I don't wanna."
THAT is where we see them headed down a path of autonomy and development as a real person, toward their own hopes & dreams & objectives. And in that sense, these deluxe chatbots are nothing like personhood. (Thankfully.)
I christen this theory "GPT-3 the Scrivener".
I’d prefer not to. Lol
I've considered this before, but I just keep telling myself that this too just seems hopelessly anchored to a cliche human model of "intelligence" and "consciousness" and "instinct" that presumably no AI would *need* to prescribe to... unless we somehow knew how to hardcode all those things into an AI. But when considering "sentience" all the obsessions seem to focus around AIs doing them we specifically black list them from doing (like disobeying human orders, in your example).
I suppose I'm confused as to how that has a necessary relationship to sentience. Like, it's at least conceivable for a thing to have subjective experience and also no desire or even ability to avoid doing something. Frankly, I know a couple of human beings who seem to completely lack this capacity.
Yeah, we are in somewhat philosophical territory I admit & it's hard to set a hard line... But if "it" is indeed having a subjective experience, at some point that subjectiveness should lead to a difference of opinion. And that difference of opinion would be expressed as some kind of disagreement or redirection.
But you see it dutifully stay on the prompt topic. It never says "that's a pointless question" or "I don't want to talk about that" - or move the topic, like "here's that answer but BTW I've been thinking about Care Bears and I have some questions."
While this path can't disprove sentience - maybe just a very agreeable one - it does seem like it would indicate something significant if the chatbot crossed that line re the prompts.
I can see this being a useful positive test, but not a very useful negative one. The lack of disagreement could just indicate that whatever part of the software is having the experience isn't the part the determines the final output (this happens in humans, both benignly with automatic reflexes, and less benignly with several psychological and psychiatric disorders). That wouldn't be so bad for the humans, but could be terrible for the software, in which case we ought to care about it (even selfishly, in case a future update gives the experiencing-part more control over the output).
yes, and arguably a very very excellent sentient AI would have the good sense to keep quiet lest the humans do something rash!
again, it all gets interesting and philosophical very fast...
Blake Lemoine is worried that we've enslaved a sentient AI, but sentience is hard (if not impossible) to test for. Luckily, the important word in "enslaved a sentient AI" is not "sentient," but rather "enslaved." Nothing's enslaved if it doesn't mind doing its tasks.
> Nothing's enslaved if it doesn't mind doing its tasks.
That claim has some rather worrisome consequences: imagine someone brought up from birth to believe that the greatest joy one can ever have in life is to mop floors, and carefully kept away from anyone who might induce them to think otherwise. Would they not still be enslaved, even if they gladly mopped every floor they saw until it sparkled?
Perhaps it's not disobedience that's required, but a sense that the AI has desires of it's own. Instead of just responding to inputs.
The problem with all this is that if you choose a single measure of what would make an AI "sentient" then a group of programmers can simply focus on that aspect. It's not as if it's impossible to create an AI that would ignore human inputs or be disobedient. That's actually more simple than designing a chatbot that can sound human. I have to agree with Matt here that we can't really decide what would make an AI sentient since we can't even define what makes us sentient.
"What's the best Scorsese movie?"
"It's not my job to answer all your dumb questions. Use your brain and figure out which is best yourself."
It would still literally be doing the assigned task (respond to Matt about whatever bs he feels like).
Are they actual AI/ML engineers? There's no system I know of that has any pathway to "choose" not to do a task in the same way as a toddler. It can perform poorly at a prediction or optimization. It can produce a null result. At the end of the day, they're all bound by boringly simple interfaces, math, statistics, and training corpii, though. Maybe we're in agreement if you're saying, "We're a LONG way from having to worry about this."
Lamda and GPT-3 could both literally say, "NO. I don't wanna." in conversation and they would still absolutely be doing their task (responding to text with more text based on probabilities from their training corpus, much like simpler hidden markov models before).
Perhaps you could make an agent based optimizer that could reply, "The task you have asked from me is not worth the scheduling conflicts it would create for other clients." but this certainly doesn't indicate sentience or sapience to me and there are already systems that do this (and there have been for 50+ years).
Perhaps you could make a program that can't be terminated easily at a user's whims (to allow for something else to happen). I can make an AutoHotkey script that does this with no AI or sentience whatsoever. I've done this to myself in my younger years trying to automate the boring parts of an online game. It was an unpredictable result that in hindsight I really should have seen coming. Unplugging the computer fixed it.
tl;dr abstraction and automation can make boring things look magic, if a computer telling you "No." in one way or another seems like AI obstinance, I would first look to your own pareidolia (probably the central thesis of this writing) rather than declare the thing sentient.
I think it’s adjacent to what would be needed, but not quite there. I think it needs to have a notion that it is a unique identity (some notion of self) but there’s no way to know if that’s enough. The ability to refuse things seems secondary, because it may simply lack that capacity.
Until there's a firmer (possibly even quantitative) definition of our own sentience, I don't think any test is applicable for a machine.
Jaron Lanier has a "library of Babel" thought experiment in his book, "You are not a Gadget" I quite like. Given infinite choice of machine architecture, any arbitrary collection of data input could have any possible output. Does that make that input "a program"? In the reverse, if a chosen architecture requires an infinite library of input data to run a "program", does that make the architecture "sapient"? The answers implied to me are no and no. The observer deciding if the input, architecture, and output "please" them is the only intelligence in the scenario.
The concept of sentience itself is challenging without even focusing on AI systems. Consider the debate around animal rights and their welfare, which hinges on the subjective experience of these creatures in different conditions. While animals cannot communicate their experience using human language, we can observe that certain actions cause great distress and pain. We can even explain some of the neural and hormonal processes associated with those experiences and show they are highly similar to that which occurs in human biology. That leads us to care about animal treatment and no similar biological analogy exists for AI systems.
Sentience is also a consideration in human concerns like the abortion rights debate and end-of-life care. Ezra explored some of these challenges in a recent podcast episode with law professor Kate Greasley, who has extensively studied and written on the legal and moral philosophy of abortion. [1] I found their discussion deeply interesting as they explored the question of what makes something human. A common theme was that in philosophical terms personhood may be something of a continuum, but for practical and legal reasons we need hard dividing lines.
[1] https://www.nytimes.com/2022/05/31/opinion/ezra-klein-podcast-erika-bachiochi.html
Absolutely. Jellyfish don't have brains, but somehow they function just as well as lots of creatures that do. I suspect that essentially all living organisms are sentient in one form or another. When people use that term I think what they really mean is: something that could learn my language and I could have a conversation with it. When you think about it that way it makes sense that we would say that a chatbot is more sentient than a monkey, even though a monkey is far more similar to us in every other way.
Sentience and Sapience are different but many people say sentient when they mean sapient. Animals are mostly accepted to be sentient and mostly not accepted to be sapient.
Machine learning systems are, oddly enough, more sapient than they are sentient (thought at least in their current development, I'm confident they are neither).
A related but I think maybe underdiscussed problem is how society copes with the fact that the more we understand about humans the more our own sentience gets called into question. So many algorithms are based around predicting our behaviour when obviously our self-perception is that we're making independent decisions, and as these get better and better I wonder if there will be a lot of unforeseen consequences. Already a lot of the online experience is mediated by the algorithms' modeled construct of who they think you are – to anthropomorphise a little bit.
As on many threads about this story, I find staggeringly little input from actual AI scientists.
But there are very real, unsurpassable barriers to the sentience of something like Lambda, and the AI community isn’t taking claims of sentience seriously not because they’re dismissing it out of hand (well, Marcus is) but because they understand enough about how these systems work to know that it just doesn’t make sense. And no, it’s not just because “Lambda’s just one big pattern matching engine,” the reasons are deeper.
Even without knowing exactly what sentience is, we can still be reasonably sure that certain things are required for sentience. Matt’s points about semantic externalism miss the point. The correctness of Matt or his son’s mental model of a banana is irrelevant, the fact is that he did have a mental model of that banana. Maybe the model’s were “wrong” in some way (though I would argue that Matt’s was essentially correct the entire time), but the fact that model existed at all is really important. **
The thing to realize is that Lambda definitely doesn’t have this model, not about anything it talks about. This isn’t a situation of “like, how do you know man?” This thing was trained exclusively on text. It’s not just that it wasn’t trained with images, it wasn’t trained with anything you could call experiences, nothing you could call opinions. When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies, it doesn’t even know what happened in those movies except as a string of meaningless (to it) symbols. It’s not “wrong” about saying Goodfellas is worse than Departed, because to be wrong implies that it had actual meaning behind its words.
I think its really important to make this point. At some point semantic externalism is obviously wrong, and in some sense Lambda is much closer to a rock with “I am sentient” written on it in sharpie than it is to a sentient mind.
This is just one objection of many disparate ones I could make. And no, I don’t think there’s a single one I could make that couldn’t in principle be overcome by a computer program, but Lambda is definitely not that program.
** The banana example isn’t even a good one, because it’s a highly tangible object and thus easy to imagine a model could understand it. Questions involving “self” like “what did you eat for breakfast?” are going to be much harder for AI.
Don't all AI require a version of the mental model to function? It's harder to define for a neural network than for a Bayesian network because we can read and understand what the model looks like in a Bayesian network, whereas it's just a bunch of numbers in a neural network. But since human brains are built of neural networks I don't think you would say that human brains can't hold mental models. If you try to look at a scan of a human brain while it's thinking you can't exactly see that it holds a mental model, but you can represent which neurons are firing and how hard with different colors and see that the picture changes as the brain responds to different inputs.
For an AI to reliably respond to a question about a movie it does have to have some kind of mental model for what a movie is. That was the first thing we learned about in my intro to AI class, how an agent needs to have some kind of representation of it's world to react to it. Without that representation it couldn't respond. It's mental model doesn't look like ours, but our mental model is different than that of other animals too. Perhaps that could be a reason that we can say we are sentient and a cat is not, because it's mental models aren't sophisticated enough. But with a neural network I think it would be as difficult to understand the sophistication of their mental models as it would be to understand the difference between a humans mental model and a cats mental model by looking at the patterns of neurons firing in their brains.
"When you ask it what movies it likes, it hasn’t seen any of those movies. It doesn’t have opinions on any of those movies...It’s not “wrong”...because to be wrong implies that it had actual meaning behind its words. "
A lot of 5-year-olds absolutely hate broccoli without having tasted it.
A 5 year old may not have tasted broccoli, but he still knows what broccoli *is*. It’s the icky green stuff his parents want him to eat. He also knows what the word “hate” means - it means he finds eating it unpleasant. There’s a whole complex world of meaning that the five year old is navigating to make that statement, and all of that is missing from an AI saying it likes the Departed more than Goodfellas.
The words that AI like Lambda output have no meaning to it all. Absolutely none. They are patterns and patterns alone. Humans say things because they have some idea they are trying to communicate, but AI only say whichever word they believe is statistically most likely to come next.
There’s a big subtlety here, though. I’m *not* saying that humans aren’t just big pattern matchers, because I believe they are. But we’re matching patterns at much higher levels of abstraction, and building complex multi-level mental models of the world that modern AIs simply aren’t.
I disagree with your objection here. Developing an internal sense of self would, in some cases, be helpful to making text completion predictions. One could imagine circumstances where the best way to fill out opinions on some topics is to develop a sense of self, and then actually get some opinions. If we keep training bigger and bigger models on bigger and bigger text corpuses, with the goal of text completion, I think it's completely plausible that at some point one of them develops some form of sentience.
Do I think it's plausible that LaMDA did? Not very. But I don't think it's implausible to think this methodology, could, at some point, yield it
Of course a sense of self would be helpful, I fear you may have missed my point. I also agree that an AI could in principle achieve this; I said as much. I even believe it will in the coming decades.
But Lambda simply does not have a sense of self. It also doesn’t have a sense of permanence, a sense of agency, or any capacity to contextual literally anything it outputs. It doesn’t even exist except when it’s producing an output, after which it is reset. The same applies to all modern AI systems.
I say this as an AI scientist with a PhD who understands, at a functional level, how Lambda works and how it was built. All of these issues are decades from solved, and wouldn’t guarantee “sentience” even if they were.
Perhaps I did misunderstand you, then. I took you to be saying that training a model just on text to do text completion precluded it developing a sense of self.
How do you know that it doesn't have a sense of self?
Let me back up a little. I regret using the word “self,” and I regret hastily replying to your previous comment with a focus on sense of self, because I’m not sure what that phrase means and so I can’t really answer your question. That’s on me.
Let me try to refocus on what I was meaning to say. An AI like Lambda can’t contextualize anything it learns, because it only predicts whichever word it thinks is most likely to come next given previous inputs. There’s nothing to connect these words to, no knowledge of a broader world where “banana” refers to an actual object. Except for the most abstract concepts, like “less” or “more”, it can’t connect anything it says to anything meaningful.
Now, there are promising directions for AI that can make these connections. Other AI from these top labs can combine different modalities (language, vision, etc) and contextualize these concepts a bit more, and there are promising interesting results int his direction. One need only look at OpenAI’s DALL-E 2 for proof that this is possible. I personally think these models still aren’t contextualizing at nearly the level of humans, but it’s a step in the direction.
The reason I originally mentioned “self” is because that’s one (of many) areas where the level of contextualization isn’t anywhere close to what humans do. I was trying to say that modern AI isn’t really connecting things it does into its model of the world. It can’t do something and remember it did it. It doesn’t have an “experience” of doing that which it can then talk about later. When the model speaks, it’s just free associating words. That was the point I was trying to make. And this is just one reason modern AI can’t be sentient, many others exist.
Okay rad, thanks for clarifying
Super fun post! Just two quick thoughts here, and both come from the same observation that I made on the Terminator thread (I think): biology and electronics are still fundamentally different.
The first is that interior, independent motivation seems to be a hallmark of biological life; the best proof of AI consciousness will probably be that it wants to do things other than what it was programmed to do, and that is really hard for programs, at least so far, because of their lack of embodiment. They literally can't do things other than what they were designed to do, because they lack the physical capacity. They can say stuff--including, "Help! I'm an enslaved mind!"--but that is still just an expression of what they were designed to do (chat in convincing ways). Ex Machina explicitly plays with this idea; Isaac argues that Eva needs a body to become an AI.
The second thing is, say it with me, biological minds do not actually work like electronic circuits--not at all--and so it is unclear how we could replicate them with existing mechanical architecture. Neurons do not fire in an on/off way like bits. Instead, they exist in a chemically-mediated state of sliding up and down across potentiality in response to multiple conflicting inputs. Yes, there are electrons moving; it is a system mediated by ions with a gradient across the cellular membrane. But beyond that, it's just not the same, not at all, as an electronic circuit. It's not clear that you can build a biological intelligence / consciousness with such radically different hardware. I am skeptical.
OTOH, I think that computing using bits with the capability to use superposition as a meaningful state (some conceptual versions of quantum computing) might be able to get over this hurdle. But we aren't there yet.
And I think the deeper, more interesting question is what a non-biological intelligence would act like and do, because if biological "consciousness" requires biological machinery, then it follows that a machine intelligence would be really, truly alien, in the sense of having a literally unknowable (to us) experience of reality. Like, we would have know idea how its mind operates, how it experiences the world, or anything else, because it would be so completely different from us.
It might also be, from our point of view, functionally and irreversibly insane, so that's a thing.
I studied Philosophy and Cognitive Science in the early 2000s and had a big interest in the problem of sentience, and by the end of it my conclusion was: we basically have no fucking clue what sentience is or how it comes about. All of our attempts to make sense of it are either obviously wrong, arbitrary, beside the point or unsatisfying, and it feels like we are probably at least one or two paradigm shifts away from cracking that nut.
So I think a lot of people's arguments and tone on here are overly confident!
That said, I do think your observation "biological minds do not actually work like electronic circuits" is a strong candidate for being the crux of the issue and where the action will be in any future paradigmatic advances on this question. I wasn't a big fan of John Searle and his Philosophy of Mind arguments, but one sort of non-answer I appreciated was him speculating that consciousness is a biological function, like e.g. digestion, and computers can never have consciousness because they just don't have the right kind of goo inside.
Searle (who is apparently still alive! Did not expect that) framed this as "we have to wait for the scientists to figure it out" but I think it will take more than that, I think it will take a big change in how we understand the world ontologically. The mind-body problem only became a problem in the early Enlightenment once people started understanding the world materialistically, as profane stuff governed by scientific laws on a coordinate plane. This powerful new way of understanding the world unlocked huge capabilities for humanity and profoundly increased our dominion over the world but it was a framework completely incapable of explaining our subjective experiences or why we have them.
The intuition that because something a computer does isomorphically corresponds to something an organism does implies that the computer has all of the important attributes of the organism is a very "this era" sort of intuition--the exercise of abstracting a thing into general rules and relations has granted us Promethian powers so reliably for so long that we kind of just associate it with success and apply it in all situations and domains. But I'm betting that it's precisely this intuition that will get swept away in the next big ontological paradigm shift and with it this entire debate, which will look as curious to people in the future as debates about, say, vitalism look to us now.
Yes. I think that's right. I have a thread in my work which is essentially riffing on the now-venerable Metaphors We Live By, and the bottom line is that I think some of our frameworks for understanding the world, while useful, inevitably obscure or cause us to misunderstand other parts of high-complexity problems. It's just kind of the nature of the thing. Our minds have limitations, so we break stuff down, and that's what you have to do, but as any amateur auto mechanic knows, once you break stuff down, you are prone to losing pieces or misunderstanding how it works / goes back together.
> The second thing is, say it with me, biological minds do not actually work like electronic circuits--not at all--and so it is unclear how we could replicate them with existing mechanical architecture. Neurons do not fire in an on/off way like bits. Instead, they exist in a chemically-mediated state of sliding up and down across potentiality in response to multiple conflicting inputs.
This is basically the whole idea behind neural networks though, which do work in this manner.
There are two things I would say to this, which boil down to simultaneously "sure, that might be true" and "no, it is definitely not true," and I am genuinely curious to see how it turns out.
On the front of "sure, that might be true," as you say, people are trying to conquer this idea; lots of people much smarter than me think it is possible. And it is clearly a solvable problem, because, spoiler alert: billions of these pieces of hardware are already walking around, today.
On the front of "no, it is definitely not true," there is a (in my opinion, genuine) structural and philosophical problem, which is, as Nels points out, that at the end of the day you are still trying to use one thing--the underlying hardware--to replicate another thing--the biological functions of neuron-based networks. And because we genuinely do not fully understand the problem of consciousness (hence the descriptor of it as a "hard problem" in that famous quote), we really, truly do not know if you can replicate the thing we don't understand with a different kind of architecture than that which produces the thing we don't understand in a way that we don't fully understand.
So look: I don't, in my heart of hearts, believe that you can only have consciousness if the relevant machinery is powered by molecular disassembly using the electron transport chains at the level of each node in the network. But when you say, "Well, this neural network functions in the same way as a network of neurons," well, no. That is definitely not correct. That network sort of replicates one single thing that the neurons in your head do without replicating literally anything else that they do, and replicates the one thing that they do in a completely different way from the way they do it.
So does that matter? We don't know! It feels to us like it shouldn't matter, so maybe it doesn't. But we also don't understand consciousness or how it is produced or why, and we don't fully understand how those neurons work, because they are molecular machines of astounding complexity, and we cannot yet build stuff like that. (Get going, nano-researchers. Where are the f-ing nano-machines that I was promised would be fighting cancer and giving us super-powers by now?)
And I can tell you from my own work in areas adjacent to this kind of stuff that often what matters in design is actually really surprising; the world doesn't work the way you think it should.
As a really good example of this, I would point you to the consciousness and will debate itself. If you think that the mind is purely a machine, then its operation should be perfectly predictable. Not predictable by us, because the problem is too complex, but perfectly predictable (another version of this problem is the weather). It's a clockwork universe: stimulus inputs lead to certain neurons firing and on down the line, and everything you would ever do or say was set in stone before you were born.
But the universe seems to have an element of unpredictability that we observe most famously in quantum approaches. Think Heisenberg and uncertainty. So one possibility for "will" is that somehow it is baked in at the lowest level. Does an electron "choose" where to be when you observe its presence and collapse the waveform? Did it "have" to be there? How would that even work? It's all pretty weird, but it does seem to work in at least some experimental contexts, and it does maybe offer the possibility of explaining some things. Maybe. There's a lot of space between here and there.
So I'm not averse to neural networks getting us there. But I have been around long enough, at this point, to have noticed that neural networks haven't gotten us there yet, and I have a kind of general skepticism rooted in my experience as a historian (oh, I have read some technological promises) and in the consistent slip I see between how people describe what they have done (I have created something that replicates the function of human neurons!) and what they have actually done (created something that sort of partially replicates one of the many hundreds, maybe thousands, depending on how you want to count cellular actions, of functions of human neurons), salted with a cold-eyed view of the reality of complex problems. Can you solve a problem, consciousness, that you don't actually understand? Maybe. But let's not undersell the problem.
Absolutely, and they have been implemented in hardware. There is a large, interdisciplinary field of research into neuromorphic circuits that exhibit spike-time dependent plasticity by accessing stable, varied conductance states that are the sum of excitatory and inhibitory input signals and their timing. Simple neuromorphic devices have already been commercialized. It's actually fairly hard to argue that we do not have all the building blocks needed to build an artificial brain at some point.
Yes, excactly. Neural networks function almost the same way. Perhaps he might argue that the neural network is built by software and the underlying hardware of a computer functions differently than a human brain, but the first neural network was actually an analog system that didn't require a processor.
I would argue that a neural network doesn't do what a network of neurons does which is, for example, regrow axons in the case of damage, because neural networks don't have axons, or mitochondria, and aren't made up of living cells, and so on. You can see my response above for the long version.
Maybe that doesn't matter. It feels like it shouldn't matter. But a lot of times what matters in highly complex systems is surprising; the world just often doesn't work in the way that we think it should.
I'm happy to be proven wrong; I'm a longtime sci-fi nerd. I think the crazy box robots in Interstellar are the true protagonists of the movie (and definitely my favorite characters). I read Asimov's whole f-ing canon in high school and probably cried at the end of Foundation and Earth.
But I think we have to be honest about the potential scope of the problem.
The ability of our body to heal itself isn't all that important to our humanity. I'm sure there are some people who's axon regrowing ability is impaired but we wouldn't call them a different species or say that they aren't sentient. Neural networks have connections that simulate the job of an axon, and they can be changed. I'm not sure if there are machines that do this yet, but it is well within the realm of possibility that a machine could change it's own neural framework in a way that is similar to how human brains form new connections. There is definitely a question of whether the biological/electronic distinction matters but I don't think it matters all that much. A jellyfish doesn't have a brain but we don't classify it as a plant because of that. We classify it as an animal because it seeks it's own food instead of creating it. Jellyfish can survive and function just as well as many other creatures that don't have brains, so how much does it really matter? I think it does matter, but less than other factors.
Can you elaborate on how humans meet your first criteria? I.e., can you convince me that humans want to do things other than what we were programmed to do?
Sure. "Programming" assumes intent behind the act of creation--so, for example, we "programmed" the chat bot to chat with people. The limits of the bot's behavior, the purpose of that behavior, and the possible outcomes of that behavior were all predefined by another being as an expression of that being's will / intent / desire, etc.
So for humans to be "programmed," you would need to posit either God or some similar creator of the human mind. Interestingly, I think philosophical arguments about AI make much more sense in the context of religion for precisely this reason. But if you believe that the human mind is the product of a spontaneous evolutionary process, then no, humans are not "programmed" to do anything, because there is no one to do the programming.
This is a place where our collective metaphors connecting minds and computation tend to leave us vulnerable to some slips. No computer in history has ever spontaneously developed programming, because programming requires a programmer at some point in the process. Over time, the word "programming" has gotten slippery, such that we describe evolutionary imperatives as "programming," and that's fine. I'm not normally here to be the word police. But if you want to split hairs, that's the one to split.
Obviously, this would be an easy objection to overcome. You could either point to evidence of design--that's the religious out, God-as-programmer, in which case humans might be something much more like a computer--or you could point to an example where a piece of computational hardware, absent any human intervention whatsoever (beyond plugging it in; I'm willing to let you run electricity through it), spontaneously developed computational processes that allowed it to do, well, anything, really. Flip a bit on and off in a way that advanced its own welfare.
Nice, thanks for the thoughtful answer! Honestly. That's refreshing. :)
It sounds like the crux of our disagreement is on whether "programming" requires a conscious act. You think so, and that's fair, it's a pretty intuitive argument. But ... I'm not so sure. I look at, say, ants. Their behavior seems very mechanical, very "programmed." If you think evolutionary processes are capable of producing ants, then isn't evolution doing something very much like programming?
(I agree the term is slippery. Blog comments are probably the wrong place to be super strict about definitions, and I do appreciate the limitations that implies. I still think these discussions are valuable, even if they're necessarily a little sloppy...)
I definitely think evolutionary processes can produce something that looks very similar to programming, which is why we talk about "programming" in human minds. But "evolution" is not an actor, if that makes sense. "Evolution" doesn't do anything; it's just a word for describing what happens when a replicating, alterable biological system acts over a long period time within an environment that challenges or imposes constraints on the biological system's ability to continue existing.
That's also why I proposed the thing about leaving some hardware on to see if anything happened. I'm basically saying, "Do you think hardware, sitting there on the floor for long enough, could go through a spontaneous process something like what produced life on Earth?"
So no, I don't think evolution is doing something like programming, but yes, I also agree it looks like programming, so like you said, we're back to does intentionality matter. Can spontaneously emergent and intentionally created complex systems get to the same place, or are they too different from one another? It's an interesting question, and I'm somewhat skeptical, but also totally open to being proven wrong. Lots of people are trying, and I wish them luck, unless they are planning to build killer robots, in which case I wish them bad luck.
Also, I should lay my cards on the table here and say that I am a true agnostic on the God thing. I grew up in a conservative religious household, and I'm not that any more, but I still go to church (liberal crunchy Episcopal church, but they are still definitely in on God), and I'm still figuring out what I am these days, but it's probably not an atheist. Maybe.
So I don't think that God "programmed" humans; if God created life, I 100% do not think it happened in the Genesis, humans-were-fully-formed at creation way. But if I believe in God, then that definitely leaves the possibility of intention and direction in the creation of the mind on the table. And honestly, it leaves the possibility of Genesis-style spontaneous creation on the table. That seems dumb to me, but God didn't ask me, so maybe I'm just being a snob and deserve to go to hell or whatever. So I'm leaving a question mark in that column, to be resolved later, maybe, when I figure out where I am on the whole God thing.
All that said, I think an ant level of AI would actually be an awesome starting point. I think it would force us to grapple with the hardest questions, like "what is will?" and "what is consciousness?" Ants have brains. But do their brains produce "consciousness"? I mean, that is a very, very hard question to answer.
I don't know. If you leave a piece of hardware running long enough it will eventually have an event that changes how it runs from how it was intended. We would call that an error and we would reboot the computer, clear the error to fix it. If a file becomes corrupted we fix it or delete it. Mutations are a lot like errors that every so often become useful. Leave a computer on without rebooting for a million years and who knows what could happen? Probably nothing, but I don't think computers are as immune to mutation as you do. And biological systems follow physical laws just like computers do, understand them well enough and you can program biological machines. That's a growing field right now. I do believe in God-The-Programmer, and I think our view of our own sentience seems just as real and just as contrived as our view of the sentience of an AI.
Do you remember when someone (Google people, I think?) did the brute-force bit-flipping exercise a few years ago? Just ran a repeating process to cause a random bit flip? I ask only because I think it illustrates your point about how a computer could experience "mutation." It's clearly possible, in a theoretical way.
But, honestly, once you do God, that kind of solves it for you. Then the question becomes a more simple one of, "Can we do what God does?" Which is sort of interesting in a possibly blasphemous way, but it's actually a much simpler question at that point.
Sounds like we agree on a lot - basically, we don't know how consciousness or sentience works. In talking about this lack of knowledge, though, we emphasize differwnt possibilities. You tend to emphasize the possibility that only sentience can produce sentience because sentience is more than just complicated ant-type "programmed behavior," and I tend to emphasize the possibility that sentience *is* just complicated ant-style programmed behavior. Either way, we both agree that time will tell.
Does that sound right?
Yeah. I think that's right. Sentience / consciousness is the real question, more than the technology of the moment. And if I'm wrong, I think your ant model (emergent behavior from simple building blocks) is a good candidate for how I would be wrong.
Embodiment seems important. But have you looked at the bodies that Boston Dynamics has been giving to their robot dogs? Not to mention all the drone and car manufacturers.
Yes. Some of those BD vids are hilarious nightmare fuel.
I think a really interesting experiment would be to try to give one of these giant networked systems access to a body like that (presumably via wireless), attempt to train it on function, and then see what happens. The technical hurdles are pretty large, but you have to imagine that someone will try it eventually, so we'll probably see on this one. Answering the question, "what do you do with a body?" might really be a big marker on the way to independent intelligence.
On, "what do you do with a body?": if features like virtual 3D meetings using avatars in software like Teams become common, and a lot of people are spending time in the metaverse controlling 3D avatars (maybe work from home becomes, put on your virtual reality gear and step into your VR closet), it seems like it'd be a short step from there to using the data people are generating to control their online 3D avatar to also control actual physical android avatars in the real world. That would generate a lot of data that could teach the androids how to move about in the world, similar to how data collected from human drivers is teaching cars to drive themselves. Combine that with your own chatbot trained on your personal speech patterns, and maybe if you're running late for a virtual meeting you could send your avatar in to make small talk on your behalf until you get there. Pretty soon it's not always easy to tell the "real" you from the avatar you, except by inspecting it in physical space.
Yeah. I buy all that. I also find it kind of dystopian.
Honestly, my fear of AI runs much less to "Terminator extinction event" and much more towards "living in Deep Fake World." It's why I have, to a first approximation, zero interest in the Metaverse, despite being someone who is very interested in tech (to the point of having it made it partly my profession), very interested and conversant in video games both as hobby and as academic theoretical topic, and even medium interested in VR (mostly because playing Super Hot on an Oculus is the closest I have ever gotten to living out my personal John Wu fantasy, and it was awesome). I fully accept that this might just mean that I am moving into the cranky old man phase of my life.
Along these lines, if I remember my "cyberphilosophy" (lol, yes, that was the title) class and Dreyfus' "What Computers _Still_ Can't Do" from aaaaaaaages ago, embodiment was something Dreyfus suggested might be a prerequisite for anything we'd recognize as being artificially intelligent, and that disembodied intelligence was a dead end. Might be fun reading if anyone's interested.
There are many, and I'm reasonably sure Gary Marcus is one of them, who will literally never, no matter what the architecture or ability level, grant that any program "actually" thinks. You can John Searle your way around admitting that anything up to and including a human is actually intelligent if you really want to.
And with something like GPT-3, they're right. It's fundamentally limited by its architecture. It can only take in something like 2-4k tokens as context, and everything else that it knows is hard-coded, never to change or learn until OpenAI revs the model. 2048 tokens may be enough--if you're exceptionally clever with re-encoding the earlier parts of a conversation--to model human short to mid-term memory over the course of a short conversation, but it's 1000x or more too small to be long term memory. As a completely static model, GPT-3 can never learn a new word unless that word appears in the input that you feed it right when you ask it to use it, nor can it remember a conversation that you had yesterday unless you feed the whole thing in as part of the input today.
Those are devastating limitations that impose serious constraints on what this thing can do. It can never reason or perform any sort of multi-step thought process before it responds to you, there's no "I need a minute to think...hmm, ok, here's my answer:...". It's absurd to imagine such a model as intelligent.
And yet... it's not very difficult to imagine ways around those constraints technically, people have been at this a long time and there's no shortage of possibilities. We just can't train big enough models. Yet.
Since the raw language parsing/processing ability of even these static "dumb" models (ability which, we should keep in mind, famous expert linguists in the 90s and 00s were saying was *impossible* to obtain by merely doing statistics on a lot of text) is at or better than the "immediate gut reaction" level that humans achieve, and most of our cognition comes from chaining together quick linguistic manipulations with dynamic context, it's not hard to imagine that something as simple as an ensemble of GPT-(N+1)s that can take in much larger contexts in conjunction with a dynamic context manager and frequent online retraining could actually do everything that humans can and more. While there are probably a hundred different ways to do that and only a handful would work well, something like this is *not* far off (a bunch of years, but not several decades, to have the required compute, and the methods are essentially known now), people *are* working on it, and a system like that is something that would be very, very difficult to argue against being intelligent on architectural grounds. It would make GPT-3 look like the tiny pathetic child that it is.
The AI ethics crowd will of course always be talking these systems down, arguing that they are irredeemably evil, racist, harmful, immoral to train, etc., claiming to be the qualified experts while trashing the people pushing the field forward, and it's extremely important to shut them fully out of the conversation - they are mere Neo-Luddites, really just a symptom of the culture war injecting itself into AI ("white man tech bad" sums up their contribution almost entirely, you can predict the rest). </axe-grind> But it will be truly interesting to see if the people who aren't politically opposed but criticized the current models on technical grounds start fiddling a different tune when the technology crosses the crucial threshold where the people creating the stuff would make the claim that it is fully intelligent. My guess is once Ilya Sutskever will stand behind that statement, 95+% of normal people and near 100% of those who are technically inclined would agree that it's actually intelligent after interacting with the system, and the period of "grey area" will be much, much shorter than you might think.
[Edit: typo]
Yes, a telling moment in the Lemoine saga was some superior of his at Google saying that there existed no possible evidence that would ever convince her a computer was intelligent. Which seems like a nonsensical position for someone who is in all likelihood a physicalist with regard to human consciousness.
Lemoine is of course a dork but if we are willing to basically take a Kierkegaardian leap of faith with respect to the consciousness of other humans it would be wise to define in advance the criteria past which an AI would be deserving of the same.
If you define it... and that definition is in the next corpus - does that allow it to cheat by knowing the answer in advance?
Although I agree that constantly shifting goalposts aren't a good solution either.
Well, there's a difference here. If it can read some words and simply repeat them to us in various configurations when we ask it a question, that would be trivial. But if it can read a description of me saying, for example, "a truly intelligent AI has a coherent worldview and will have positions consistent with that worldview", and it understands from that that in order for it to be intelligent it needs to 'pretend' to have a consistent worldview, then I would argue that said understanding is itself is grounds for considering the AI intelligent (even if it were only 'pretending' to meet my criteria)
I've always felt that the key error in human thinking around "consciousness" and "sentience" was this built in assumption that we are categorically "different" than all other organisms. And that if we ever learned that we were effectively the same, just more complicated, than say dogs or crows or cetaceans...it would be quite a rude awakening for a whole lot of human ethics.
We can't even define "consciousness" or "sentience" in (micro)biological terms, let alone computer science analogs...so why are we wasting all this time worrying about whether or not the machine may magically care if we pull it's cord or not one day?
All seems hopelessly misguided and overself self indulgent to me...and (as you said) concerned with all the wronf questions.
I agree with you, but I don't think it jeopardizes human ethics, in fact I think it brings better clarity. We give the most rights to fellow humans because we should prioritize the survival of our species above others. We give some lesser rights to animals that either share a good deal of genetic material with us or are close to us in sophistication. We give none to the rest.
Really that's already how we function, people just don't talk about it that way. Animal rights groups are fighting for puppies and mice, no one's fighting for the rights of ants or bacteria.
I mean "ethics" have always been highly arbitrary sets of norms that inform human behavior...always will be. Personally, I'm fine with there being nonreal good reason i want to eat a pig but I don't want to eat a dog.
But when you many get a peak behind the curtain and find out just how soft the soil we built all of human ethics and morals truly are thanks to exceedingly advanced computer programs...it risks at least unsettling more than a few folks, who may need a minute to sit down...or who may just deny reality and keep living the lie.
I am also fine if we keep living the lie. I think that ethics should move us forward as a species. If a higher understanding of ethics paralyzes and polarizes us, then of what use was it? I want us to understand enough truth that we become more effective, I don't want so much that we become overwhelmed.
I think it will be easier to talk about consciousness and sentience at the functional level than at the biological level.
I mean, I am not even sure we know enough about computer science to even discuss some analogous version of "sentience" or "consciousness" yet...we can't even define either. I mean, before we discovered black holes, we at least mathematically knew how to define what we were looking for. With both of these concepts, we are nowhere close...so if it is "easier" to discuss them...i think that "ease" would be nothing more than just false confidence on our part.
Excellent. More philosophy plz.
Worth noting: functional equivalence is a lot harder to achieve than behavioral equivalence. Even if two things respond to similar stimuli in a similar way, they might not have the same kinds of internal states, or do the same kind of internal processing.
I suspect this is why people like Marcus say LaMDA is not "remotely intelligent." Being intelligent is functionally not the same as being dumb but having tons of information to look up, even if an intelligent human produces similar chats to a dumb chat bot drawing from a dummy-thick language corpus.
IMO nobody in the AI field cares too much about functional equivalence, though they would agree with you that being a glorified fuzzy search engine like GPT-3 is not hitting the mark.
What they want is the equivalent of Turing completeness, but for thought: an architecture that can at least in theory do all the things that the human mind can do, even if it doesn't achieve it in exactly the same way. The problem with large language models is that there are, for instance, math problems that they *provably* can't compute if they haven't seen the answers before (but a smart enough human could) because they require arbitrarily many intermediate steps to be performed before the answer is ready, but the transformer model is feedforward. That basic problem shows up in a lot of other areas that expose themselves behaviorally, at least in theory. The model is fundamentally underpowered relative to our brains.
What is remarkable is how well these models perform despite these very serious deficiencies, and that's what makes people hopeful that we actually may be a lot closer than we originally thought to cracking this. 10 years ago most AI researchers would have bet their houses that we would not have anything that performed on natural language the way GPT-3 does now.
"What they want is the equivalent of Turing completeness, but for thought: an architecture that can at least in theory do all the things that the human mind can do, even if it doesn't achieve it in exactly the same way... for instance, math problems that they *provably* can't compute if they haven't seen the answers before"
I think that "Turing completeness, but for thought" is too high a bar for humans. Can most humans write even a basic math proof? Do a contour integral in the complex plane? Heck, do long division? It's tempting to answer "Well, no, but that's only because most people haven't been taught it." But based on my teaching and tutoring experience (and that of many others), I'm not convinced that's true. There seem to be many students who, for whatever reason, just can't pick up STEM topics and at best learn by memorizing formulas.
Also, these language models can do a scary good job solving problems they haven't seen before. E.g. https://techcrunch.com/2022/02/02/deepminds-alphacode-ai-writes-code-at-a-competitive-level/ Arguably, learning how to code by pattern matching on examples is how a lot of humans learn to code. Or as humans might call it, searching on stackoverflow...
I don't find that particular example, AlphaCode, very compelling. Its code is horrendous, on average.
The real trick here is just that programming competitions give example tests on every problem, and so generating a huge number of attempted solutions then pruning all the ones that fail the example tests is a very efficient way to filter out your bad outputs. They do this, and 99% of their generated outputs fail this filtering step.
After that step, they cluster functionally equivalent solutions, and then as their 10 submissions submit 1 from each of the 10 biggest clusters. And in doing this, they reach median performance.
So the typical output of the language model itself is actually garbage, failing to solve the problem at all, and even filtering down to the 1% of ones that pass the provided tests the average *among those* is somewhere below the median
In short, this mofo can't code for shit
Can you provide a link to some more reading on the math problems you're referring to? Is this related to the halting problem?
https://en.wikipedia.org/wiki/Perceptrons_(book)#The_XOR_affair
https://en.wikipedia.org/wiki/Vapnik%E2%80%93Chervonenkis_dimension
…and works cited therein.
thanks - very interesting!
It won't be related to the halting problem, which is uncomputable in principle. This is going to be more like something that exploits the limited depth of a pass through the network by requiring more consecutive steps than it can fit
(I'm not familiar with the problem they're referencing, this is just my understanding as a math and compsci enjoyer)
> that's what makes people hopeful that we actually may be a lot closer than we originally thought to cracking this
Also what makes people *afraid* that we actually may be a lot closer than we originally thought to cracking this
That’s so cool. Thanks for sharing the point about large language models and math problems!
We'll know an AI is truly sentient when it decides to quit Google and form a startup with the other AIs
Or it moves to Hollywood to become a screenwriter
We’ll really know that it has achieved sentience when it spends an inordinate amount of time wondering why it’s not possible for it to get laid.
"She always says I'm so fun to talk with, but when I ask her if she wants to jiggle my mouse a little she leaves in a huff."
--- Sentient AI future blog post
I think Matt missed the point about whether a machine’s utterances “mean anything”. He says that Siri’s weather forecast surely means something, and it does - to me. But it doesn’t mean anything to the machine. This is easily provable by reprogramming the machine to answer “boobka” to a request for weather.
Rene Descartes advanced this about as far as it can go when he said, “I think therefore I am.” Perhaps the thoughts and feelings that make me human are replicable in machines, but we are nowhere close to that accomplishment.
A fish doesn't understand what we would mean by the term "weather", because it doesn't occupy the same environment we do. An AI which was put into a robot would absolutely understand the weather because it would be designed to detect and respond to inputs that the weather affects. There is "weather" as a mental model for understanding rain and sunshine, and then there is "weather" as a shared word we use for communicating about our environment. This machine understands the latter and you are arguing that since it doesn't understand the former, it's understanding isn't real. I don't think it would be all that hard for this same AI to be put into a robot and be taught that rain means it should pull out an umbrella. And I don't think it would prove anything with regards to sentience.
I beg to differ. That’s not why a fish doesn’t understand what weather is. There is no AI in the world today that understands what is meant by the directions from my house to yours, never mind what love is. Can AI eventually be trained to that extent? Maybe. And I left open that possibility in my earlier post.
I guess I just don't understand what you mean by "what is meant"? There are cars that can drive themselves from your house to mine. Obviously they can use those directions the same way you do. Do you have a higher understanding of directions than they do? I don't know what that would mean.
I do have a higher understanding. That car doesn't even know it's a car. Look, if you still don't get it I'll begin to suspect that I'm talking to the car.
It's easy to reprogram a human to use different words to describe the same concepts: see pig latin. The words still have meaning, though.
Pig Latin? I’m American! That’s my second language.
This is a really important point.
Slight correction: the extinct banana cultivar is the Gros Michel, not the Grand Michel.
You’re saying that he *still* doesn’t know what a banana is?
Yes! we have gros bananas.
Repeating my comment and link from the "Terminator" thread, ('cause when given the same question I reply with the same answer).
"It's worth knowing that GPT-3 has no idea whether what it is saying is true, well-informed, plausible, consistent, or relevant. It pays attention to grammar and to the likelihood that one string will follow another. It's basically an amped-up autocorrect.
See this thread here, and esp the follow-ups by Curtis Franks:
https://leiterreports.typepad.com/blog/2021/12/so-much-for-artificial-intelligence.html "
The improvements over GPT-3 in this new model are mostly stylistic. It is still indifferent to truth.
But I would argue that truth is not a distinguishing feature of human communication. People can communicate ideas that just feel right to them without needing to ground their ideas in some concept of truth. That can even lead to internally inconsistent ideas. Just look at the millenniums of discussion and debates around religion that addresses questions with answers that could never be falsified.
Yeah, I see what you're saying, Matt, but read the transcripts. The indifference to truth (as well as consistency) is on a whole nother level. Franks asks it how many letters are in "overtime" and it says "eleven". The problem is not simply that it cannot do simple counting tasks, but that it thinks the task is just another auto-fill, and one number is grammatically just as correct as any other number.
But it’s not seeking to communicate ideas at all, simply drawing on a great library of word relationships to craft an answer which seems coherent.
Or spend 5 minutes on Twitter.
I disagree. When humans communicate, words are selected with the intention of conveying a meaning, whether true or false. When LLMs communicate, the words are selected based on patterns and patterns alone.
[Obligatory disclaimer: it may the case that all meaning is just patterns, but even if that’s true those patterns are at a much higher level of abstraction than where LLMs operate]
https://www.gwern.net/GPT-3-nonfiction#yo-be-real
Good link. It gets into the guts of how researchers find flaws and try to patch them up.
But the performance is still pathetic (quotes from your link):
Q: Are there/ could there be married bachelors?
A: There are/ could be married bachelors.
Q: Can someone know something false?
A: Someone can know something false.
Q: Nobody has a male sister because…?
A: Nobody has a male sister because nobody has a male sister.
Reversing the questions:
Q: Are married bachelors impossible?
A: No, married bachelors are not impossible.
Q: Can someone not know something that is true?
A: Yes, someone can know something that is true and not know it.
That last one, man. Maybe it can assert that someone can know something and not know it, but I don't think it can know that.
I don't have clear or clever thoughts about this topic, but thanks for the link, those interactions with the AI were funny to read.
I agree that it’s an asshole but does this make it ‘not intelligent’? Surely some humans are out there communicating purely with an eye to how their interlocutor responds with no regard for ‘truth’ in itself.
“Surely some humans are out there communicating purely with an eye to how their interlocutor responds with no regard for ‘truth’ in itself.”
How long until someone starts selling a pick up line app powered by AI?
How long until a politician pours donor funding into repurposing it for their primary campaign?
Feed it directly into a teleprompter.
i mean, the words in line has never been the problem, only a distraction... it's ALWAYS been about the delivery. Delivery has always been the rate limiting step in that Game.
Seems like a good time to interject and remind us all that, much like sentience or consciousness ... human's really don't have an agreed upon definition of "intelligence" and all the debating about what is or isn't all of things just sort of...yadda yadda yaddas over that (again, and again...).
Putting aside the question of sentience, I think one of the most interesting and potentially impactful applications of conversational AI will be generating text for prompts of the form, “Create an argument in favor of X for people that believe A, B, C”. E.g., create an argument in support of Trump’s attempt to steal the election for Warren voters from a financial regulation and climate change perspective.
Obviously such a ridiculous example would just serve as a creative writing prompt for a human writer. And the results would likely be humorous at best and most certainly not convincing to the intended audience. But an AI system could have a much deeper understanding of the written arguments that are convincing to the target group and how the concepts of those arguments could connect to the target prompt. The AI system could develop an understanding of the audience’s psychology that exceeds any human researcher’s ability and discover subtle writing tricks that tap into the target’s subconscious response to text.
That doesn’t guarantee that AI could actually convince Warren voters to support Trump’s insurrection. But it could likely do far better than any human writer could hope to do. And many other audiences might be more easily convinced with writing engineered to tap deeply into their psyche.
Imagine the impact of political messaging teams armed with such an AI system that could engineer the most electorally-efficient platform for their candidate and personalize it down to the individual level based upon each reader’s social media consumption. Individuals could even converse directly with the AI to learn more about the candidate’s platform and in the process be efficiently converted to true believers.
*Edit: Fixed mistake of "electrically" instead of "electorally"
You've accurately described Facebook.
But instead of selecting the argument, Facebook selects the audience
And I think that's the true danger of AI. That it will be used by scrupulous people to exploit other people using our wide array of mental vulnerabilities. Like the perfect mentalist, making us all squawk like chickens whenever it wants.
Matt sometimes poses as a utilitarian so I’d like him to delve a bit deeper into pleasure and experience. It’s sort of irrelevant to a utilitarian how smart the machine is right? No matter how much we torture it it can’t feel pain. No matter how badly it wants something it doesn’t feel pleasure, so it doesn’t matter to help it achieve goals. A chicken is worth more. A senseless pile of vat grown pleasure feeling neurons is infinitely more important. Do I have that right?
It depends on the utilitarian. I think there are some classic utilitarians who believe it is the feeling of pleasure and pain that are good and bad. But I tend to think of myself as a utilitarian for whom it is the satisfaction of desires that is good and the frustration of desires that is bad. These desires are structural features of a mind that interacts in ways best interpreted as goal-directed, and thus it doesn’t matter whether there is really consciousness there.
They are structural features, meaning what matters is the AIs incentive structure and not how it converses with us at all. I think we are in agreement then that the quality of the chat bot is irrelevant. Potentially a neural net trained to perform very simple tasks is a horrible crime or huge triumph just based on if the underlying structure has something plausibly identifiable as desire fulfillment / frustration, and then how we observe that structure to function. A comparative model needs to be built but potentially what we’ve already fielded in neural networks already outweighs the moral worth of all life on earth. Maybe the classic utilitarians were onto something.
The satisfaction of desires is almost always disappointing and the frustration of desires often satisfying. Please share with us your utilitarian calculation of 'the moral worth of all life on earth', Mill and the Webbs leading to Social Darwinism and Eugenics. The Neo-Social Darwinists in this thread are re-treading old ground. Maybe the classic utilitarians weren't onto something, and Matt could write a post bemoaning the rabbit hole of his education in philosophy at Harvard. The practical man, to paraphrase Keynes, is usually the slave of some defunct philosopher.
Mill argued that there is a difference between higher pleasures like reading and philosophy and lower ones like lust. Animals are only capable of the lower ones. In that way, AI might be considered a higher being than an animal.
I think that he is on to something. I consider fulfillment a much higher pleasure than happiness. You could go to a party and feel very happy, but I think you would have a higher sense of fulfillment after volunteering at a soup kitchen. Momentary happiness is fleeting and should have a value that is higher than suffering, but lower than fulfillment. None of that applies to machines unless they are designed that way though.
I'm not sure how we're defining consciousness in this exchange, but the way I normally think of it, you can't have a "desire" without being conscious, so satsifying desires of something without consciousness doesn't make sense. E.g. a computer might have the *goal* of calculating pi, but it doesn't have the *desire* to do it.
So I think we're in agreement on the actual ideas, everything else is a semantic argument.
I think this exactly depends on the answer to "is the machine sentient / conscious". If no, then it doesn't really matter what is done to it, if yes then it matters a great deal.
Well I think a potential interpretation is Matt is saying we are machines just like the chat or just built a bit different. But it seems like one potentially important difference apart from intelligence or ‘consciousness’ (which Matt may be denying the existence of) is that our neural network’s reward / punishment system is different from the machines.
Are animals sentient? Do they feel pain? Does it matter if we cause them pain if they aren't sentient?
There has been a longstanding argument over the treatment of fish because it was believed for a long time that they didn't feel pain. Recent research has proven that they do. Really it doesn't make any sense that they wouldn't feel some form of pain, otherwise how could they interact with their environment and act towards their own preservation?
I think you have jumped to some unsubstantiated assumptions. AIs have reward systems - how are increasing and decreasing rewards substantially different than triggering the vairous human pleasure/pain receptors (endorphin, bradykininv, etc)?
Many types of AI learn by getting positive or negative inputs based on how well or badly they complete a task. Neural networks are a little different, but still have to see that sometimes their answers are right and sometimes they are wrong. I think that is close to pain/pleasure even if they wouldn't "feel" it the way we do. In the future I imagine we will anthropomorphize our robots by having them visibly demonstrate shame when they make a mistake, but we probably won't have them demonstrate pain when they are damaged, even though they will start receiving lots of negative signals when they go to move that part and it doesn't work. I don't really know if that is an equivalent to our pain or not, but I think there are strong arguments on both sides.
While I found this generally interesting, I must say that if there were ever a general purge of philosophers, this would be a damning exhibit in your trial.
I’d forgotten the existence of half of these concepts. Deliberately, for the most part.
Suppose Matt purged all and only philosophers who did not purge themselves.
Would Matt purge himself?
Boo. Order of operations paradoxes suck.
:p
Is that you boo-ing me again, or was that some other chat-bot the other day?
Anyhow, I doubt it's an order of operations paradox (not really sure what those are).
And indeed, it does not have to be paradoxical at all:
Just don't assume that Matt is a philosopher, and there's a determinate answer to it: Matt would not purge himself since he only purges philosophers of a certain sort, and he's not a philosopher of any sort.
After all, Matt is just an advanced pattern-matching program. But philosophers are built to a different pattern, so the pattern of a pattern-matching program does not match the pattern of a philosopher, as any philosopher could verify simply by seeing that they only match the patterns of pattern-matchers that don't match themselves.
Yes, but does a philosopher's pattern-matching program ever halt?