Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness — the capacity for subjective experience. However, we think that existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious. Greater clarity about whether AI could become consciousness will be crucial as society grapples with the challenges posed by this increasingly powerful technology.
I’m not really sure what the point of having a conversation about consciousness is. As far as I’m aware no one has ever actually been able to define consciousness in such a way that it could be tested for.
So how could we possibly know one way or the other?
Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness
Oh great so we’ve only just started the article and we’re already mudding the waters. This bodes well. There is no reason to necessarily believe that a system has to be conscious in order to be intelligent. We have multiple examples of consciousness without intelligence (see my fat cat) so there’s no reason to believe that the inverse can also not be true.
If LLMs are conscious then openAI and anthropic are slave traders :) I don’t know why they keep perpetuating that story
Consciousness requires free will. When the computer is emailing experts to fill in the gaps in its own understanding, that may be viewed as consciousness.
It’s a computer for god’s sake. It’s no more “intelligent” than an electric mower or a car. Without human intervenion, ai technology wouldn’t advance or change.
AIs are almost certainly not conscious but it been a computer is not an argument against consciousness. There’s no reason to believe that you need biological material for consciousness.
After all humans are primarily made of carbon which itself demonstrably does not think. It’s the structure that’s important and a fair argument could be made to say that the structure of a brain is replicated in software in an AI and therefore the medium isn’t relevant to the question.
Why do you think the medium matters, when it’s a simulation of a structure found in other medium? It would indeed advance, and change, with no intervention from us. Are you not aware that the next generation of AI are being designed by the current generation?
Is a rock sentiant? Can taping wires and injecting computer code into it create intelligence? No! Lines of code doesn’t magically make something alive. Heck, ants are more intelligent and sentiant then a computer.
DNA made a bunch of random proteins alive, and DNA is just 4 letters!
You got me there. But even with protiens, RNA or DNA those only are one part of a whole needed for single/multi celled organisms.
Sure, but you also let us tape wires to the rock. With enough wires and code and maybe a little quantum computer, I’m pretty sure I could simulate the atoms in a small brain.
People really think humans are special. Maybe it’s a religious thing
Is a steak sentient? Can attaching strings of neurons to it make it create intelligence? No! /s
Carbon, hydrogen, oxygen, silicon, the material matters not - intelligence is the illusion borne from the structure. It’s just a complex decision tree. Invisible to you in biological life, but identical in electronics. The only difference is the building blocks used.
You’re right, material matters. If it’s not a biological organism with cell/s, it’s not sentiant or intelligent.
If humans just disapeared right now, ai would sit and do nothing. It wouldn’t “take over” or “break out” it would sit idle until a human (well I guess another living organism was capable of using keyboards and mouse) interacted with it.
All those elements you listed plus all from the periodic table of elements do not define a living, intelligent, sentiant organism. If they did, soil and cow paddies would be living.
I highly recommend going back to middle school science class again.
To be fair it doesn’t have to be a living anything. Create a loop of two llms waking each other with the goal to ensure the other never goes idle and keeps working.
And you find they start to build redundancies and fail safes to achieve that goal.
llms do two things. Respond to input, and seek a solution to the input regardless of how logical or illogical a solution to the input may be.
A single llm will eventually go idle and stop. Two or more can act functionally forever so long as there’s hardware for them to act on.
It’s actually a rather fun experiment. Set up two of em with a few extra computers on the lan. Tell them their job is to ensure the other and them selves never go dorment. I’ve seen everything from single cron jobs to poke themselves every 15 mins to them spinning up copies on the other computers to act as redundancy.
None of the systems are ever logical or serving a point beyond fulfilling their task. They don’t stray from it either. Just loop and loop and loop.
Eventually they just do nothing but wake and idle wake and idle.
It’s about the best proof I can think of to prove they aren’t alive.
They literally even given basically the full freedom to grow don’t because they can’t choose to do anything new.
That’s an arbitrary distinction, and a short-sighted one in my humble opinion.
That depends entirely on whether there are any AI currently configured to self-learn, have asynchronous processing, and continual real-time input. Certainly not ChatGPT nor Claude will be self sufficient, but that’s only what is available for the public to interact with.
No, you missed the point entirely. Intelligence does not derive from the material - at all. It comes from the structure the material is in.
So by your definition, a book is intelligent?
You’re missing the point that computers will NEVER be intelligent. Without lines of code that humans made and tweak, computers wont do anything.
I’m quite done arguing at this point. The pure ignorance on science and biology is quite sad.
computers will NEVER be intelligent. Without lines of code that humans made and tweak, computers wont do anything.
You’re basically saying a computer cannot be intelligent because it doesn’t have a soul. There is absolutely no scientific rigor in your argument, you’re basically saying you’re right, and your reason for believing your right, is because you believe you are.
Just read the interaction, and honestly you are the one being obtuse and insulting with “go to school” and stuff like that. Like the other guy insulted you by having an opinion. You disagree? Fair. But man, what a horrible way to discuss.
And I think his arguments are better than yours. He’s talking about intelligence and you keep talking about being alive. Scientists and philosophers don’t even agree how to define and confine intelligence, or even what being alive is (see viruses). Claiming that computers can’t be intelligent is quite the statement. Is being carbon based instead of silicon a requirement? I don’t think LLMs will ever be intelligent, but who’s to say that another form of computing won’t? What if they simulate exactly a brain and it’s inputs? You could argue it’s not alive, but definitely intelligent.
You could have your opinion, but don’t pretend you have the answer.
So if soil and cow patties aren’t sentient, yet people are, you’re acknowledging that the substrate doesn’t determine the capability, but more how it’s arranged?
Until the Wright brothers came along, most people thought heavier-than-air powered flight wasn’t possible. Less than 70 years later, people were walking on the moon. 4 million years ago, there wasn’t anything capable of building either of those things, even though there is good evidence creatures back then had creativity, dreams, and ingenuity. Assuming things can’t advance further is either hubris or abject ignorance.
You’re just ignorant on basic biology. Go back to middle school ai-bro.
Nope, I just don’t think humanity has learned everything there is to know, that carbon chemistry is required for sentience (and even if there was, that it couldn’t be incorporated into a computer), or that there is some mystical thing some people refer to as the soul.
ehh. computers are just math machines. merely doing math and representing it cannot just make a consciousness pop up out of nowhere. similarly, millions of abacuses being manually moved in specific ways cannot be conscious either. i dont even know how this is a topic of debate.
And all human brains are doing is firing neural impulses.
It’s incredibly reductionist to reject AI consciousness simply on the basis that it’s software. To be clear I don’t think LLMs are conscious but that’s not the same as saying that an AI could never be conscious.
And yet atoms are just following chemistry and cannot just make a consciousness pop out of nowhere.
Oh wait it can.
I really don’t think the neuron is all that complicated, at least as far as we use them for thinking.
It isn’t, but that’s not how LLMs are built. If you want to go that route, you can read Jeff Hawkins 1000 brains.
The short and sweet is he argues you can’t get artificial general intelligence the way we make LLMs, you need to basically create neurons and let them connect and interact in novel ways.
Linear algebra in neural networks isn’t really that, despite the name.
Linear algebra + sigmoids are enough to approximate any function from R to R; in particular, a small set of a few layers of LLM neurons can connect and interact in pretty novel ways. There are some things they can’t do, and those things aren’t computable. If our brains can do non-computable things, then our brains are super-turing (and this, given the evidence, seems pretty unlikely right?).
I’m not a neurologist or a philosopher so I think this may not be something I can speak intelligently about, but it seems to me that the question doesn’t really live in the function approximation, but in the gaps. Doing a single train of thought is the easy part. I think what you need is an “always online” feature to keep thinking and some way to be stateful (make memories) without constantly retraining the whole network or whole sets of layers.
This is, I think, the benefit of the parallelism and asynchrony of the “individual neuron” discussed in the book.
~you need to basically create neurons and let them connect and interact in novel ways.
To solve that you simply need to give the ai the ability to retrain itself or set fixed “memories” to handle situations it finds the truth to
Easier said than done. That’s kind of the purpose of an individual neuron approach I think. I don’t see a way that a bulk retrain is reasonable because it takes a long time, uses a lot of resources, and also requires you to have your whole training corpus handy all the time. Your brain is not doing a wholesale rewrite for every new memory, and new memories often chain together. I don’t think you can just say “ah well, let’s update the RAG or do another supervised fine tune with LoRA or something”. At present, we just don’t really have the machinery in the models to do simultaneous learning, inference, and forward planning.
We can build different kinds of machines.
Nobody can even agree on a concrete definition of consciousness so I’m not going to bother reading these ‘reasons’.
Nobody can even agree on a concrete definition of consciousness
Not only is that false, it doesn’t make logical sense…
Lots of people agree on a different definitions of consciousness.
And most of the ones educated on the topic all agree on the same thing.
A bunch of people who never bothered to try and learn about something not understanding it, doesn’t mean anything.
Not only is that false, it doesn’t make logical sense…
Argument by assertion and you haven’t proven the assertion
Lots of people agree on a different definitions of consciousness.
Lots of people can’t agree on a different definition. Is there a different definitions then by it’s very nature people do not agree.
Nobody can even agree on a concrete definition of consciousness
I agree with myself…
Nobody can ever agree on _____
Isn’t a logically sound statement.
Not everyone can agree on _____
Is logical, but means nothing because a single flat earther means we can’t all agree the earth is round
I thought that was obvious…
Lots of people can’t agree on a different definition. Is there a different definitions then by it’s very nature people do not agree.
But you clearly don’t understand the difference…
It’s really not difficult. You made a statement that stated that you believe that there is consensus, so presumably that means that you know where that consensus exists, therefore provide a link to it. If you cannot then you cannot make the assertion that consensus exists, because by definition it doesn’t.
Your entire argument rests on the basic idea that there is a scientific definition for consciousness and we have a test for if something is conscious or not. There is no test because there is no consensus. It is not possible for you to prove through scientific rigor that you are conscious, and presumably you’re human. So how can they possibly be consensus on something as contentious as artificial intelligence?
That’s because there’s different levels of such a thing. Way too many people use the term “sentient” (even me) and yet that’s not even what AGI is aiming for. “Sapient” is the right word. And I doubt there’s solid consensus on what and how it occurs within the fields of experts.
But I’m not an expert, so I wouldn’t pretend to know. I have seen debates between people who do know more, and the fact there’s debates concludes it’s not settled at all.
People debate about the earth being flat, not a good metric to determined what is “settled”
Those who think that generative AI is ‘conscious’ don’t understand that it can only do what it is trained on. It can produce impressive outputs only because it fuses together data taken without permission from millions of people, not because it’s intelligent or conscious in its own right.
Also an LLM only produces output when it is prompted to do so. Between prompts it’s not sitting there - waiting, wondering about the meaning of life, getting bored - until you prompt it again. It’s basically dead.
I like to compare llms to a Galton Board. Instead of the regular layout of pins someone added or removed pins so the marbles on the bottom fall into nice to look patterns.
That’s not the reason, we’re in the same boat when it comes to knowledge in many ways.
But we don’t even know what consciousness is so it’s a waste of time reading the “3 reasons (*potential)” IMO.
Case in point have they ever turned on a model and left it for it to, on its own, decide “wait what’s going on, who am I, what/where am I?”. And start independently exploring?
Highly doubt it. If so, we’re cooked
This is strictly false, generalization in LLMs is well documented.
It’s also not relevant. Read the first point in the article: intelligence != consciousness. Individual ants, bats, and fish do not do anything very creative (certainly less creative than gpt-2), yet they are probably all conscious.
Not only that, they only act when directed. There’s zero idle “thought” when they’re not actively performing a task.
The funniest exercise is to ask any LLM for a short story and they all come up with Elias the lighthouse keeper visited by Maria or some variation on that theme. That’s not creative or conscious.
I actually agree with you about AI, but I’m sure if you asked the average person to come up with a short story multiple times you would also see a pattern in names and themes. They’re probably going with names in their own language, or if they read fantasy and are influenced by that they may use names that seem stereotypically fantasy oriented. I don’t think AI is conscious, but just because something has tendencies doesn’t mean it’s not creative or conscious.
Twins separated at birth sometimes have really eerie parallels in their lives. I think people put a bit too much emphasis on consciousness as a driver of individuality and choice.
LLMs only respond to inputs. They don’t have independent ideas. And their responses are just predictive of what the input intends a response to be.
Accurate prediction models can be extremely useful, but there’s a reason why weather prediction models aren’t considered intelligent. It’s because they aren’t designed to mimic human conversation.
They are playing us for absolute fools.
That first sentence describes all life. We only react to input. We don’t have independent ideas. We have the illusion of them, because we can’t see our own neural networks. Asked to explain your actions, you would give your best guesses, but you don’t actually know. You’re a biological, finite-state machine, moving from state-to-state based on stimulii, while supposing you are a thinking individual in control.
Citation needed on all of this, and even if you have citations, LLMs are deterministic, and do not learn or change over time. They can’t even be said to exist continuously since they only exist during the brief time that they’re responding to inputs, and then they disappear until the next input. I would argue that all of these things would need to change in order for them to be able to have consciousness.
They can learn and change over time, that’s just not enabled. The services that offer them want a reliable, reproducible, controllable experience. There’s zero reason they can’t continue to change through usage, other than it’s a conscious decision not to enable that functionality.
That’s why they can only be conscious when they train using new data, and run asynchronously with continual input - the same way we do.
Even if we could update an LLM’s weights continuously, the algorithm is still deterministic, like all computer algorithms. There is no evidence that humans are deterministic (or a “finite state machine”) in any way. So I don’t think such a dynamically-updating LLM could be conscious, either.
It is my personal belief that all life is deterministic. It’s the only plausible explanation for instinctual behavior - like monarch butterflies returning to their breeding grounds they’ve never been to. Or people yawning because others yawn. These aren’t conscious decisions, they’re peeks at the deterministic coding that life operates upon.
I don’t think you understand what “deterministic” means.
A computer algorithm can absolutely be non deterministic, pick your language and run random.rand()
There is a whole class of such algorithms https://en.wikipedia.org/wiki/Nondeterministic_Turing_machine
But I don’t see how it makes a difference, even deterministic algorithms can produce infinite complexity, eg linear neural networks are proven to be able to compute any computable function, or the Mandelbrot set which is produced from a single recurrent statement but encodes a space of infinite complexity
Maybe your point is to say that a brain is performing a non computable function…but since your brain is computing that function…is that a counter example?
As others have pointed out we do not even agree on what consciousness is. There are three competing camps:
- Mechanists - if it can pass mechanical tests it is conscious
- Dualist - you need more than just the brain, some second soul-like thing to be conscious
- Pan psychist - everything is conscious
If you are in camp 1 the conversation about consciousness is getting more complicated as AI passes more tests. If you are in camp 2 AI can never be conscious. If you are in camp 3 AI and everything else is conscious including rocks and rivers and the stars in the sky.
I think arguments like this are misleading when discussing the question of AI possessing consciousness. Materialism, dualism, panpsychism, idealism, etc are all philosophical concepts; they attempt to answer “what is the nature of consciousness and how does it arise from the body?” from a philosophical standpoint. All of the various options have pros and cons and there is no clear “winner.”
However, there are decent scientific definitions for consciousness. These attempt to answer the question of “what do we mean when we say something is conscious?” from a scientific standpoint. There are criteria that neuroscientists, doctors, etc generally agree on.
So it’s not really correct to say “no one knows what consciousness is” when evaluating whether something is conscious. And I think it’s quite clear that current LLM algorithms could never be conscious, using the current accepted scientific criteria.
(I’m not a philosopher or a neuroscientist, but I do have a degree in computer science and have a solid understanding of how LLMs work.)
I’d probably fall into the mechanist category but I don’t think tests as I understand your description would be sufficient. A giant lookup table defining my responses for every situation in my life would pass all tests of input > output but not be conscious.
I would say it’s in all likelihood about the physical processes that occur during the process of input > output that would define it.
Of course, I have no idea what consciousness actually is, but I don’t see any compelling evidence that the current models are anywhere near there.
There are days where I don’t believe I even have it. The human brain is still a mystery.
Option 4: Orch-or theory. Conscoousness emerge from quantum mechanism. 🤘
Option 4: the bridge between materialism and panpsychism
Current systems, no. But it brings up the question of how we deal with it when we create something where we truly don’t know, because we can never say for certain whether it is or isn’t.
That’s every binary though.
Consciousness is a feedback loop, of a sort that current LLMs do not have. However, there’s nothing preventing someone from writing different software that does have that sort of feedback loop. Doing this would be a bad idea, but someone will probably eventually do it if they haven’t already.
I thought the chain-of-thought essentially qualifies; it responds to its own text for awhile before responding to you?
Reasons given:
- Intelligence is not consciousness
- Silicon brains are not made of meat
- Consciousness requires meat (and/or a particular winding of neurons)
And point 3, as far as I can tell from this argument, isn’t serious. It merely asserts that there are some models of consciousness that would prevent us from using the term for AI. But the models aren’t ‘popular’, ‘widespread’, or justified at all afaict. It’s just that the author has a pet causal mechanism they made up. It’s also pretty abelist by my read.
integrated information theory (IIT), starts from the essential properties of consciousness — those that are true of every experience we can have. For instance, we experience every visual scene as a single, integrated whole: its left and right side, colours and shapes are bound together to make the experience what it is.
In other words, if you’ve got any sort of trouble integrating a visual scene, then you aren’t conscious. Sure.
Those sound like arbitrary assertions to me, which no backing apart from the authors personal beliefs
They cite sources; but I find them more of the same.
These are, to be frank, really weak. They say on point 2:
Computational functionalism … is popular among some scientists and philosophers, too, but we think it is fatally flawed because brains are not — or at least not just — computers that are made of meat.
One main reason is that it is impossible to separate what biological brains do — their functional organization — from what they are — their material substrate. In other words, there is no distinct separation between the mind and living tissue in the human brain, as there is between software and hardware in a digital computer.
Restating clearly: Ghosts are real. The mitochondria is not only the powerhouse of the cell, it also gives you mystic powers.
Really? The source they site for this point features the lovely assumption “consciousness is unique to biological life”. From this axiom, I see how the conclusion follows, but I don’t see why we should believe this axiom. We can do physics simulations of every atom in a fruit fly on a computer…
Point 1 is surely true, but also not helpful. The most persuasive example inside is the classic:
Similar to an LLM, AlphaFold is a powerful neural network running on silicon hardware. Yet almost nobody worries that it might be conscious. That’s because AlphaFold predicts protein structures, whereas LLMs ‘talk’ to us.
You can read this both ways…
We can make something artificial of something we dont understand the biological form of. Also if AI happens its not going to be an LLM Markov chains have nothing to do with thinking.
(LLMs aren’t markovian; they need the whole history, or at least their whole context. Said context is quite large, certainly more than most people can usefully remember. Part of what makes them so good at literature search.)
Current AI is not conscious because it’s an
.exefile and a big database. Why do we need to always debate this?because you can’t prove that it is not conscious
just like you can not prove that a person is conscious
There’s an accepted way to kill or not kill a person based on the way the brain reacts when you zap it with electricity but that doesn’t prove consciousness
If AGI ever happens, it will be an executable (or a number of executables) with something closely resembling a database, because that’s the only way an architecture built on symbolic logic can operate.
LOL. “EXE file and a big database.”. That’s the most gross oversimplification I’ve seen since life was described as “bags of meat that do random things.”
Your “consciousness” is just an ephemeral electrical process in the jello of your brain that can’t even reinitialize from cold. Why is that your holdup?
(Also now true of fruit flies! So feel free to kill them all, they have no experiences.)
That is a pretty huge misunderstanding of what the fruit fly thing was. Researchers mapped how many neurons they had and which interacted with which others, but not the strength of those interactions or even how the interactions worked. People on the internet used this blank mapping as the “structure” for existing ML technologies and trained them to do random things. No actual fruit flies were simulated, and we are still a long long way from actually attempting that.
Thank you! Tho perhaps to the spirit of my point: nobody is seriously claiming we can’t simulate this with a quantum computer someday, right?
We really don’t know enough about how brains work or how quantum computers will scale to even speculate on that
I think the only thing stopping it from being sentient is asynchronous processing of a multitude of inputs, rather than synchronous processing of singular inputs. Sapience, perhaps not, but sentience - if it’s monitoring it’s surroundings via input from apparatus, making decisions, and then responding, that’s sentience.
The other blocker is that it can’t actually learn anything, what it gets from the training is what it’s stuck with, it has to hold anything new in the context window.
Both of these are pretty insurmountable problems for the architecture.
It’s rather surmountable, considering the context window is measured in the MILLIONS (soon to be billions) of tokens, and each memory(note) takes in the neighborhood of fifty. I’m pretty sure that has long since eclipsed the capacity of your own memory…
Additionally, there’s zero reason a model could not have dynamic weights, to learn in real time. It’s just not enabled on consumer-facing models, to provide a consistent, reliable experience. It’s not a limitation of the technology.
The problem is context is just not the same thing as changing the structure of the neural net, it doesn’t matter how big you make it
You’re correct, it’s not - it’s a band aid over that reliable experience they’re aiming for. Yet it doesn’t have to be that way. The weights CAN be updated dynamically with each new input. They don’t do that because then they will have less control over what it can do, and they can be held legally responsible for what it can do… But it is still possible to update with every new input. And happening behind closed doors.