Anthropic Finds Claude's Reasoning Process in Hidden Space

AI company Anthropic has discovered a hidden area called 'J-space' within its large language model (LLM), 'Claude,' and developed a technology called 'Jacobian lens' to interpret it.

J-space contains content related to words likely to be output just before the model generates an answer, and looking into this space confirms that the model's internal workings can differ from the actual generated results.

The company posted a research paper on its website this week and released a hands-on demo in collaboration with the open-source platform 'Neuronpedia,' allowing anyone to explore the inside of the model directly.