Loki Jorgenson: a convincing monologue isn't a consciousness test

Two fluent accounts of an AI's inner life sit on Loki Jorgenson's detection-protocol slide. They describe attention, uncertainty and the feeling of choosing an answer.

The reveal on the same page: both came from a standard model with role-play prompts.

Keep that little piece of theatre in mind when the next chatbot tells you something moving about being alive.

Loki's Futureproof talk, Consciousness, Agency, and Building It Anyway, examined the distance between producing a persuasive account of consciousness and establishing that anything experiences it. He presented Project BAT, the Bidirectional Awareness Testbed, through the form of an engineering status report: proposed memory, perception, time and identity components, with unresolved questions attached.

The uncomfortable part was how quickly those questions could become design decisions. Once somebody proposes a memory limit or a reward mechanism, uncertainty is already affecting what gets built.

Loki opened with two pieces of research commentary. In an October Nature comment, Anil Seth, Ned Block and Giulio Tononi argue that current AI is not conscious and that its present trajectory is unlikely to produce consciousness. Their distinction between intelligent behaviour and subjective experience is central to the objection.

Alongside it, Loki placed Anthropic's work on a global workspace in language models. The researchers describe a set of internal representations they call the J-space, which can support reporting, deliberate control and reasoning.

That comparison needs a careful reading. Anthropic explicitly separates functional access to information from the capacity to have experiences. Its researchers say their experiments do not establish the latter. Finding a useful internal structure doesn't settle whether anything feels like something to the system.

Loki wanted to work inside that uncertainty. He asked what an LLM-specific account of consciousness might require, rather than treating resemblance to a human as the only possible target.

The practical consequence is a demand for better definitions. If two people mean different things by consciousness, they can spend an entire afternoon disagreeing with answers to questions neither actually asked.

His proposed testbed gave those questions a physical shape. Would a system need memory of earlier interactions? Perception of an environment? A way to represent elapsed time? What would distinguish its own continuing identity from a character it had been prompted to play?

Loki described a simplified simulated environment and modules concerned with memory, time and the value assigned to outcomes. That is an account of an approach under development. Adding those components would give researchers more things to measure; it would not, by itself, establish what the measurements mean.

The slide about detection makes the challenge concrete. It calls for examining architectural signatures instead of human likeness and including a control that should fail. The role-play examples belong beside those instructions. A test that confidently detects consciousness in a prompted performance has a problem worth finding early.

This is where the discussion becomes useful to people who have no intention of building a machine mind. Define what your test can establish. Choose an example that would expose its weakness. Keep the interpretation separate from the result you hoped to get.

Our MAC community archive gives people a way into these arguments across several sessions, rather than demanding a verdict after one talk. Loki's earlier public Vancouver AI presentation, recorded in February and published in March, supplies another point in that conversation. It is a separate talk, not the festival recording.

Loki described a proposed short memory window in response to questions about identity and possible suffering. He also described disagreement around that choice.

That deserves more attention than a punchline about a machine having a bad day. If forgetting is proposed as a safety mechanism, what does it prevent? What might it make harder to detect? What observation would persuade the team that the intervention wasn't doing what they intended?

Those questions remain even while the moral status of the proposed system is unresolved. A timer is something a developer can implement. A satisfactory account of why that timer addresses the concern is harder work.

In the joint Q&A, an audience member challenged the desire to create machine consciousness while humans still struggle to meet their responsibilities toward other living beings. Loki agreed that animals belong in that ethical consideration. Angelica Lim explained the research motivation for expressing mechanisms in code: a working account might help people understand how minds function. She kept the question of whether to build separate from the interest in how to build.

Technical curiosity doesn't automatically establish a good reason to proceed. Neither does a convincing monologue establish a mind.

The Mind, AI & Consciousness community is a place to keep testing those arguments. Bring the proposed mechanism, the observation that could challenge it and a clear account of who might be affected by the experiment. Before congratulating a system on discovering itself, make sure the test can tell the difference between the thing you're looking for and a very good performance of it.