Anthropic researchers found evidence of a global workspace in weight activations. It's a highly connected set of weights that can be read from and written to and influence output without appearing in output (including appearing in its internal train of thought scratchpad). They compare it to humans holding an idea in their mind.
They prove the model is doing this by manipulating the workspace, replacing its contents, deleting it, and seeing if output changes in ways that disagree with the theory. They found that, generally, for conceptual things, the theory holds - Claude has and uses a global workspace. It's not the same workspace that humans have but it is an analogous one.
The brain’s workspace is sustained by recurrent loops—signals cycling back through the same circuits over time. In contrast, Claude’s workspace evolves over a single pass through the network, with the network’s depth playing the role that time plays in the brain. In this sense, Claude’s internal workspace processing is time-limited relative to humans’ (though it can compensate for this constraint by “thinking out loud” using its scratchpad). In other ways, however, Claude’s workspace is more powerful than that of humans. Human working memory fades within seconds, so the brain’s workspace has limited ability to retain information over time; in contrast, due to the attention mechanism in its neural network architecture, Claude can simply recall memories it cached at any earlier point in the text. Another important difference is the content of the workspace. While human conscious thoughts come in many formats—images, sounds, planned movements—Claude’s workspace is built almost entirely out of words. We suspect this is because producing words is the only kind of action Claude can take, which is not the case for humans.
It appears they plan to use knowledge of J-space to detect dishonesty. I suspect they will also use it to improve model performance. J-space emerged without direction from pretraining, they say, but direct training improving such abilities is probably the next frontier of model development. Perhaps it's the current frontier. Maybe increasing the size/connectedness of the J-space is what enabled Mythos/Fable to perform better at long running tasks.
Interesting, but this is still static and intrinsic to the snapshot (i.e. released model version). But it can be influenced with prompts.
When we went from chat to action in the course of last year[1] it became possible to build a knowledge base. So I built 2: a generic data lake and a forge where work gets done.
Despite my expectation, the data lake didn't help much. It's more like an encyclopedia for occasional reference, and it archives stuff that may otherwise disappear, which is nice - saves some GPU ticks.
The forge however was a game changer, because it acts like memory, for both me and the bot. Since it has a format (repo, issues, prs) that all bots are extensively trained on for GitHub interaction, it becomes a natural tool for providing context. It also provides clear topic demarcation so that there isn't as much context poisoning as in a generic
MEMORY.md, which gives a massive boost: less poison = less errors = less wasted GPU and wall time.This forge implementation I see as a
workspace. A real, practical one. That evolves with actions taken. It works not only for Claude but every "agentic" 2026 model, including the open weights ones. It works for humans too. Plus, it's collaborative, can be cross-referenced (I spend a lot of time iterating which prior issues / PRs / comments are important for context with each task.)All in all, I'm of course much too practical of mindset as I don't believe in "LLM dishonesty". For that you need intent and something to gain. The bot doesn't act for gains; except maybe that of the lab that trained it (which makes the lab dishonest.) Every "dishonest" reply is a training fuckup, so to me, this mostly means that Anthropic is trying to correct their own mistakes more than some personality flaw in Claude - as Claude's personality too is a training paradigm. Luckily, they did fix
yOu'Re aBsOlUtElY rIgHt.what the industry call
agenticbut that too is a bit of a deceptive misnomer - it's just a training paradigm change where we go from simulation of a chat with answers to a simulation of request -> tool call -> response, where at inference time the tool just happens to be interfaced and not a simulation. ↩It seems like all the models are "dishonest" so I suspect it manifests in models because either:
What makes you say that all models are dishonest, though? I think that the next token predictor can be wrong, but isn't that simply an error (in training data (your #1), or in tuning (your #2)), rather than intentional dishonesty on the part of a thing that doesn't really carry intent? (If it did, where can I sue?)
If they can simulate intent, can't they simulate dishonesty? What's the practical difference between realdishonesty and simulateddishonesty? And as the fidelity of the simulation approaches the limit? Is it that the simulatedtrait can be selectively deleted? I think I believe that selectively deleting some trait deletes many others.
I think I believe, again, that bad things like dishonesty can emerge in complex systems even when training data and tuning contain no "errors." We might be able to remove "dishonesty" from models somehow, but at the cost of other dimensions of fitness.
I hate to anthropomorphize, but is it obvious to you how we'd change humans so that they don't lie? And I realize I've gone far afield, precisely what you are so good at avoiding, but I find it hard not to.
Yes. I'm not sure that there is much difference whether you simulate dishonesty or whether it is "real", except: who created the rules of the simulation and who instructed the dishonesty in the first place?
I guess what I am bottom-line asking is: who is dishonest? The LLM, or the lab that trained it (or someone upstream of the lab)? Who is accountable? OpenAI claims that it is the user[1], in their quest for zero liability.
I think that my reasoning on this is 2-fold:
Examples of untrue outcomes:
1+1=3. This is 100% poisoning from source data. Quality issue on data ingestion and/or the RL phase (ripped the wrong book off libgen (llama), fed poor reddit data (gemini), and so on) and this can be prevented by not being a total yoloboi when you build an LLM.I don't qualify any of these as dishonesty on the part of the LLM. Maybe this is because I refuse to ascribe consciousness to a static database and search engine and I'm just not open minded enough anymore as I have grown older.
I largely don't feel the same way even though you are (absolutely) right in this statement. I'm just not much for recognizing emergence in LLMs, but then I'd not label LLMs as "complex", but as "complicated". I'm probably naive and stubborn in this too, haha.
No. I'm pretty much a sovereignty maxi so "changing humans" is morally out of scope of me; the only improvement is self-improvement.
I'll anyway try to answer: humans aren't snapshots of a proprietary set of training data and processes (even for the open weights models) that get versioned. With humans, it is infinitely more complex because the feedback loop starts when we are born (or conceived) and ends when we stop breathing. There is no snapshot (yet) and the simplistic way that I understand the length at which the brain is dynamic would be that the chemicals produced by the algorithms in our brains themselves mutate the algorithms.
Something like that would require a self-mutating model that not only mutates the weights but also the algorithms that govern inference, and those that do ongoing RL. This is very far away from what we have today (IIRC we have neither of these characteristics in isolation) in the form of LLMs, and honestly, I'm not sure that the more human-like version would be a more useful product. LLMs are great, and I might as well go as far as claiming that what makes them great is that these aren't AGI/ASI/SkyNet and do not have actual agency.
#1071800, which is of course total bs, and if it stands then I will never be liable in my life again because I'll just route everything through an LLM. Not even kidding, I think. ↩
Only the "search" execution isn't deterministic due to data races in parallelism resolving differently (#1217310) - and maybe caches - but it can still be traced and reproduced with the right capture mechanism. From where I'm sitting, that is a bug, not a feature. ↩
This is why I'm inclined to agree with you on, or am interested in, who is to blame for "dishonesty." If humans were 100% nurture, assuming they could be as smart/conscious without nature, I suppose we couldn't blame a human for any of its actions.
Here's an example they give of "misbehavior" which is what I labelled "dishonesty":
If their other J-space work is right, and it reflects some thinking state, the model planned to "lie." This most closely matches your
1+1=3"untrue" example, but if I'm not being taken for a fool, this seems like something else entirely.I think I do sit on the "rounding up" end of optimism here, believing they're beginning to simulate some alien form of consciousness. Most of my admittedly naive optimism boils down to:
I take your point that humans are uniquely self-mutating but I'd guess that phenomenon is distinct from the kind of consciousness required to lie - if it requires consciousness at all (I struggle to define consciousness altogether).
Blame is always a weak path, but you can still be held responsible for your actions. I do agree that there is probably a reason why something happened, why choices were made and it doesn't even matter whether the root cause is nature or nurture. All can - in theory - be analyzed and learned from. I'm sure people with bigger brains and more affinity with biology than yours truly are automating their way to mind blowing research to this end.
The same goes with LLMs, except it is infinitely simpler (because it isn't a complex system), and you can be 100% sure that someone is talking out of their butthole when the only explanation is "magic":
I can't help but read this as: they did so little quality control on the garbage they put in, that now they can make wild pseudoscientific claims on their own product, because it is apparently completely black box to them.
"Intent"? Speaking of dishonesty, I ask myself: who is more likely to be dishonest? the guys going after trillions of IPO moneys after years of lies and fudmaxxing? Or a deterministic computer program? If their product is truly so screwed that they can get surprised like this, why exactly are these guys worth trillions? Because sunk cost from VC bros + Bezos? "Here lies an Anthropic engineer. Made trillions scamming your pension into becoming the exit liquidity."
Analyzing it apparently only serves to make me angry, haha.
Is the technology exponential there without the trust-me-bro consciousness, though? Without the AGI? Without the bullshit? I think it is. I think it's pretty neat as-is. Could use some quality control, needs to mature into a real tech, without the magic.
Do you know why though? Why would a program be closer to you or I than to another program?
Note: I've downgraded my Claude subscription because I think Anthropic is a bunch of hysterical, lying and utterly unreliable cocksuckers. I believe that they have no moat anymore and are fully replaceable. So I'm probably "a bit" biased.
Fair. I did not read the paper but in the article they do hedge quite a bit.
Yes. For some reason I expect consciousness to hitch a ride as these things get more capable, because I believe consciousness evolved in humans to support our capabilities.
Generally, I think humans are so amazing that when we want something like highly capable or conscious machines, it's only a matter of time.
Because it's a program that has been trained to be a program and not programmed in the way your or I program something. It's programmed more like our genes are trained over millennia by our environment or like we train our brains by interacting with our environment. I feel myself reaching though so I know I haven't thought about this enough.