\n<\/p>\n
Recent work from Anthropic identified a small set of internal representations in
\nlanguage models \u2014 the J-space \u2014 that behaves like a global workspace: its
\ncontents can be verbally reported by the model, deliberately modulated, and are
\ncausally used for multi-step reasoning, while the surrounding majority of neural
\nactivity remains inaccessible to report. The identification tool is the
\nJacobian lens, which transports a residual-stream activation at any layer into
\nthe final-layer basis and decodes it through the model’s own unembedding,
\nanswering: which vocabulary words is this internal state disposed to produce,
\nnow or later?
\nSubtext applies that method continuously during live conversation with a local
\nmodel. On every token \u2014 both while the model ingests the user’s message and
\nwhile it generates its reply \u2014 the lens is read at nine depths and the result
\nis rendered as it happens. The intermediate steps of the model’s computation
\nbecome directly watchable: verdicts form during reading, several tokens before
\nany output; planned words hold at high strength while unrelated tokens are
\nbeing emitted; two-hop questions surface their unspoken middle term.
\nSubtext differs from the interactive readouts already available (e.g. the
\nNeuronpedia demo) in that it is conversational and continuous: it renders the
\nlens during a live chat, includes the reading phase over the user’s message,
\nstreams at generation speed via a KV cache, and pairs the canvas with a
\nper-token ledger and per-word inspector. Sessions can be exported and replayed
\nin any browser without a GPU.
\nWhat the lens shows that the output does not
\nThe value of the instrument is the gap between the model’s internal state and
\nits visible text. Three moments from the demo session:
\n1. The verdict precedes the reply. Zero tokens of output exist; the model
\nis still reading Is this correct? 12 + 5 = 1. The workspace already holds
\nmath, addition, arithmetic, modulo \u2014 the phase indicator is amber
\n(reading).<\/p>\n
2. The judgment is formed, then verbalized. As the reply begins (“No, that
\nis **not\u2026”), incorrect dominates the workspace at high strength, with
\nequation, calculation, statement co-active \u2014 the conclusion is
\ninternally settled several tokens before the words “not correct” appear.<\/p>\n
3. Plans are held while other words are being said. Mid-explanation, the
\nworkspace holds modulo, bitwise, system, numbers \u2014 the technical
\ncaveat the model is about to raise \u2014 while the current output token is
\nunrelated.<\/p>\n
These reproduce, on an open 4B model on consumer hardware, the reporting and
\nplanning phenomena described in the paper (which used Claude-scale models),
\nincluding the two-hop signature: Italy at layer 20 and euros at layer 26
\non the country-shaped-like-a-boot question, before generation begins.<\/p>\n
Each rendered word is a lens readout, not model output. It indicates an
\ninternal activation disposing the model toward that word.
\nVertical position corresponds to layer. Early layers (perception) are at
\nthe top; readouts approach the bottom rail as they approach emission.
\nSize and opacity encode absolute readout strength. The display applies a
\nfixed monotone mapping from lens probability; weak readouts are rendered
\nweak. Amber marks readouts taken while reading the user; blue while
\ngenerating.
\nHover shows a word’s per-layer activation profile; click opens an
\ninspector with peak strength, mean depth, and strength history.
\nThe right panel records everything the canvas curates: the conversation, a
\nlive ranking of currently-active readouts, and a per-token ledger.<\/p>\n
browser (single HTML file) \u21d0 websocket \u21d0 server.py
\n Qwen3.5-4B (bf16, HF transformers, KV cache)
\n pre-fitted Jacobian lens: neuronpedia\/jacobian-lens, revision qwen-n1000
\n per token: residual hooks at 9 layers \u2192 J_l transport \u2192 unembed
\n \u2192 full-vocabulary softmax \u2192 word-start top-k \u2192 frame<\/p>\n
Each exchange has two phases. A single prefill pass covers the user’s message,
\nwith lens readouts taken at every position (the reading phase); generation
\nthen proceeds token-by-token with a KV cache, reading the lens at the newest
\nposition each step (the thinking phase). The lens adds a per-layer
\nmatrix-vector product and an unembedding per token, so streaming runs at the
\nmodel’s native generation speed.
\nDisplay filtering. Raw lens top-k contains punctuation and BPE
\ncontinuation fragments (e.g. itude, from cert\u2011itude), which are not
\nmeaningful as readouts. Display is restricted to word-initial vocabulary
\ntokens, following the reference implementation’s mask_display with a
\nstricter word-start criterion. Probabilities are computed over the full
\nvocabulary before any filtering, so filtering affects legibility only, never
\nthe readout itself.<\/p>\n
verify_accuracy.py compares this implementation’s live path (forward hooks,
\nKV cache enabled) against the reference JacobianLens.apply() on identical
\ninputs. Across 4 layers \u00d7 3 positions on the walkthrough prompt, top-5
\nreadouts match exactly, with cosine similarity \u2265 0.99998 between logit
\nvectors, and reproduce the expected two-hop intermediates. The audit can be
\nre-run at any time with the server stopped.<\/p>\n
Requirements: an NVIDIA GPU with ~10 GB of VRAM, Python 3.11+, and a CUDA
\nbuild of PyTorch. First launch downloads the model and lens (~9 GB total) and
\nbuilds a display-token mask (~1 minute, cached).
\ngit clone https:\/\/github.com\/ninjahawk\/Subtext
\ncd Subtext
\npip install -r requirements.txt
\npython server.py
\n# \u2192 http:\/\/localhost:8765
\nOn Windows, run python -u -X utf8 server.py, or use start.bat.
\nOther models. The server is configured for Qwen3.5-4B because Neuronpedia
\npublishes a pre-fitted lens for it (a 27B lens is also published, for larger
\nGPUs \u2014 edit MODEL_NAME\/LENS_FILE in server.py). Any HuggingFace decoder
\ncan be used by fitting your own lens with jlens.fit(); ~100 prompts produces
\na usable lens, and fitting a 4B-scale model takes on the order of an hour on a
\nsingle consumer GPU. See the reference repo
\nfor details.
\nReplays. The \u2913 session button exports the current conversation \u2014 every
\nlens frame included \u2014 as a JSON file. Open the app with ?replay=
\nto play one back with live pacing, no GPU required; that is exactly what the
\nhosted demo is.<\/p>\n
The instrument inherits the method’s limitations. The lens reads only concepts
\nthat correspond to single vocabulary tokens; multi-token concepts are invisible
\nor fragmentary. It approximately captures the workspace identified in the
\npaper, not the entirety of the model’s internal state, and layers below the
\nfitted range are not observed. Interpretation should also respect the paper’s
\nown framing: workspace readouts demonstrate functional availability of
\ninformation for report and reasoning; they do not demonstrate subjective
\nexperience.<\/p>\n
The method and reference implementation are by Anthropic Recent work from Anthropic identified a small set of internal representations in language models \u2014 the J-space \u2014 that behaves like a global workspace: its contents can be verbally reported by the model, deliberately modulated, and are causally used for multi-step reasoning, while the surrounding majority of neural activity remains inaccessible to report. The identification […]<\/p>\n","protected":false},"author":1,"featured_media":6664,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[676],"tags":[],"class_list":["post-6663","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-ai"],"_links":{"self":[{"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=\/wp\/v2\/posts\/6663","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6663"}],"version-history":[{"count":0,"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=\/wp\/v2\/posts\/6663\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=\/wp\/v2\/media\/6664"}],"wp:attachment":[{"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6663"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6663"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/daiilynews.cu.ma\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6663"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n(jacobian-lens, Apache 2.0).
\nPre-fitted lens weights are published by
\nNeuronpedia. The model is
\nQwen3.5-4B. Subtext is an
\nindependent project and is not affiliated with Anthropic.
\nLicensed under Apache 2.0.
\n
\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"