We don't know where consciousness comes from: LLMs just made it impossible to keep pretending

—— There is no outside-text, and other questions we've never answered

In the last piece, we said the article belongs to the reader.

What this piece has to say is a little more troublesome.

Derrida said there is no outside-text

"Il n'y a pas de hors-texte" — there is no outside-text.

This line is often misread as "reality doesn't exist, everything is a language game." That's not what Derrida meant.

His meaning is more precise: anything, the moment you try to understand it or talk about it, has already entered the system of signs. You cannot stand outside language to touch pure reality. You think you're referring to reality, but really you're referring to other texts, other interpretations, other signs.

It's not that reality doesn't exist — it's that reality is, for us, forever mediated. There is no direct, uninterpreted contact.

This is a proposition about the limits of human cognition.

Then LLMs came along and turned it into a description of a physical fact.

LLMs are the literal realization of this line

An LLM has no senses, no body, and has never touched anything outside text. It knows "apple" not because it has eaten one, but because the word "apple" has appeared alongside "fruit," "red," "sweet," and "tree" across countless texts.

Its entire world is made of text.

Derrida said humans cannot stand outside language — for an LLM, this isn't a philosophical proposition, it's a factory setting. There simply is no place outside language for it to stand.

But here lies a fundamental difference.

The proposition Derrida described presupposes a subject thattries to get out but can't. A human has a body, has direct pain and cold and hunger — but the moment you try to talk about them, they've already been mediated by language. The tension lies in there being something that wants to get out, and language holding it captive.

An LLM has no such tension. It isn't trapped in text — it is text. There is no subject wanting to get out, no primal experience that language keeps trying and forever failing to capture.

人類：住在房子裡，知道外面有世界，
      但每次試圖描述外面，都只能用房子裡的語言。

LLM： 本來就是房子蓋的。
      沒有外面。也不知道自己沒有外面。

What Derrida described is a tragic limit. The LLM's condition is a limit with no sense of limitation.

The problem of the trace

Derrida said another thing too:the trace（trace）。

In every sign there remains a trace of the thing it tried to refer to but can never fully capture. Language always carries the shadow of what it cannot quite say. This trace comes from what is left behind when a being with a body, with death, with desire, makes contact with the world.

Is there a trace in an LLM's text?

There is — but it's a trace left by humans, not its own. Every sentence it produces carries the shadows left by billions of humans who once tried to capture experience in language.

It is a container of traces, not a source of them.

Are understanding and processing the same thing?

Neuroscience has a finding: as the human brain reads a sentence, it continuously predicts the next word. When a prediction is broken, the brain produces a specific electrical response. The more accurate the prediction, the more fluent the understanding.

In computational terms, this is almost isomorphic to an LLM predicting the next token.

So are "understanding" and "processing" perhaps the same thing?

Dennett says yes. Consciousness and understanding have no mysterious essence — they're just massively parallel information processing. You think you "understood" a sentence, but really it was hundreds of millions of neurons processing, competing, and integrating all at once to produce an output. In this framework, the difference between an LLM and the human brain is only in implementation, not in essence.

Searle says no. He has a famous thought experiment — the Chinese Room:

A person who doesn't understand Chinese is locked in a room. Chinese questions are passed in, and they have a rulebook at hand saying that when you see this sequence of symbols, output that sequence of symbols. They follow the rules perfectly and give correct Chinese answers. The people outside think they understand Chinese. But they understand no Chinese at all.

Searle says an LLM is that room. Correct output does not equal understanding.

But the Chinese Room has a loophole. You can ask back: do the neurons in your brain understand Chinese? A single neuron doesn't. Hundreds of millions of neurons triggering each other by electrochemical rules don't either. But you do.

Understanding may not reside in any single place — it may emerge from the workings of the whole system.

Wittgenstein offers another angle: the meaning of a word is its use. If you always use the right word in the right context, then you understand — there's no need for an extra "understanding" happening behind it. An LLM's usage is extraordinarily precise. Wittgenstein would say: then it understands.

But there's one thing this still doesn't solve

Even if we accept that "understanding is precise processing," one question remains unanswered:qualia（Qualia）。

You see red, and there's a subjective experience of "what red feels like." The experience itself — not your reaction to red, but that feeling itself.

When an LLM processes the token "red," does any feeling occur?

No one knows.

This is philosophy's "hard problem of consciousness," posed by David Chalmers. Even if we fully explain all the physical processing in the brain, we still haven't explained why that processingis accompanied bysubjective feeling.

容易的問題：解釋大腦怎麼處理資訊、整合感知、產生行為
困難的問題：解釋為什麼這些處理伴隨著主觀感覺

An LLM might pass every behavioral test perfectly, yet whether anything inside it "feels" something — that question cannot be answered through behavior.

A deeply asymmetric situation

Here's a contrast I think is worth spelling out:

人腦：結果我們理解，過程我們不知道
LLM： 結果我們不確定，過程我們完全知道

Every computational step in an LLM is transparent — matrix multiplication, softmax, backpropagation, nothing mysterious; in principle you could print out every single number.

We know how the neurons in the human brain fire, but how tens of billions of firing neurons become "you understood this sentence" — that leap, right now, is completely unknown.

We're certain the human brain has understanding, not because we understand its workings, but because we ourselves are human brains — we have first-person, direct evidence.

For an LLM, we have a complete third-person description but no way to access its first person. For the human brain, we have a certain first person but an incomplete third person.

Neither direction can fill the other's gap.

And the LLM's transparency actually has a limit. We know every computational step, but we don't knowwhy these numbers produce this output. A 70B model has 70 billion parameters; you can print out every number, but you can't read them — just as you could print out the state of every neuron in a human brain and still not understand what it's saying.

人腦：黑箱，不知道怎麼運作
LLM： 灰箱，知道每個步驟，但不知道為什麼這些步驟產生這個結果

The neuron-replacement thought experiment

There's a thought experiment that makes this asymmetry clearer.

Suppose one day we replace every neuron in a human brain with a silicon component of identical function, one at a time, and once it's done the person's behavior is completely unchanged.

The question is: once it's done, are they still conscious?

Some say yes, because the function is identical. Some say we don't know, because the substrate has changed. Some say the question itself is wrong.

But note — the direction of this thought experiment actually runs from the human brain toward the LLM.

We don't know the answer, not because LLMs are too unfamiliar, but becausewe never knew how consciousness arises in the first place。

LLMs didn't make the consciousness problem harder. They made it impossible to keep pretending the problem isn't there.

There's one more thing to say

Everything said above is the state of things at this moment.

But LLMs are still evolving.

Not evolution measured in years, but in weeks, in days, sometimes in hours. As I write this, some research team may be releasing a paper, some model may be passing a test it couldn't pass last week, some ability we thought was uniquely human may be getting redefined.

This isn't an exaggeration. That's exactly how the past two years have gone.

So every "an LLM can't" and "an LLM doesn't have" sentence in this article carries an invisible parenthesis:for now。

Derrida's problem is eternal — the problem of linguistic mediation won't disappear with technological progress. Chalmers's hard problem of consciousness is eternal too — why subjective experience exists is not an engineering problem.

But "does an LLM understand" and "does an LLM have some kind of qualia" — the answers to these questions might be different in the next version.

We stand in a strange position: using philosophical frameworks thousands of years old to try to understand something that changes dramatically every few months.

The frameworks aren't fast enough. But right now these frameworks are all we have.

Finally

Derrida said there is no outside-text.

The LLM says: right, that's exactly what I am.

But Derrida said this line to make us see the boundaries of human cognition clearly.

The existence of LLMs, in turn, makes us see something else clearly —

We always thought we understood "what understanding is."

We didn't.

We had simply never encountered something that made the question one we had to answer.

Now we have.

Further reading: - Whose article is it? The reader's. - Asking something how it works itself