The open problems in mechanistic interpretability fall into three groups, following the field's main 2025 review. First, the methods are not good enough: nobody knows the right way to split a network into parts, sparse autoencoders leave large reconstruction errors, and many explanations are never properly validated. Second, the methods rarely beat simpler baselines on real jobs such as monitoring, editing or unlearning. Third, there are social questions about how the work feeds into AI governance. Since the review, attribution graphs have made circuits easier to trace, but even their authors say they give satisfying insight for only about a quarter of prompts, and Google DeepMind's team has moved toward a more pragmatic, task driven style of research.
This guide walks through each problem in plain terms, says what has changed since January 2025, and points to the open tools you can use to work on them with open weight models. If you are new to the field, read our explainer on what mechanistic interpretability research is first; this page assumes you know what features, circuits and superposition are.
The Open Problems in Mechanistic Interpretability review
The paper most people mean by the phrase is Open Problems in Mechanistic Interpretability, posted to arXiv in January 2025. Its 29 authors come from Apollo Research, Anthropic, Google DeepMind, EleutherAI, MIT, Northeastern University, Goodfire, Decode Research and other groups, and the text is a synthesis of their views written by Apollo Research on behalf of Schmidt Sciences. The paper stresses that it does not necessarily reflect any single author's view.
The review sorts the open problems into three sections:
| Section | Core question | Example problems |
|---|---|---|
| Methods and foundations | Can we take a network apart and describe its parts correctly? | Decomposition, sparse dictionary learning limits, validating explanations, circuit discovery, automation |
| Applications | Do the methods help with real goals? | Monitoring, red teaming, steering, unlearning and editing, predicting behavior, extracting knowledge |
| Socio technical | How does the work connect to society? | Turning technical progress into levers for AI governance, social and philosophical challenges |
The goals it lists are concrete: monitor AI systems for signs of dangerous cognition, edit internal mechanisms, predict how models act in unseen situations, improve inference and training, and extract latent knowledge. Its blunt summary is that the field "still has considerable distance to cover" on most of them.
Problem 1: Nobody knows how to carve a network at its joints
Reverse engineering a network takes three steps in the review's framing: decompose it into parts, describe what each part does, then validate the description. The first step is already unsolved.
Neurons and attention heads are the obvious parts, but both are polysemantic: one neuron responds to several unrelated things. Some representations also seem to span several layers, so even a layer is the wrong unit. Older tools such as PCA cannot help much, because under the superposition hypothesis a model stores more features than it has dimensions, and PCA cannot find more directions than there are dimensions.
The leading answer is sparse dictionary learning, the family that includes sparse autoencoders (SAEs), transcoders and crosscoders. The review lists its weak points:
- Reconstruction error is too high. When a sparse dictionary with 16 million latents was inserted into GPT-4, language modeling loss matched a model trained with only 10% of GPT-4's pretraining compute. On GPT-2 small, SAE reconstructions cut performance by 10% on task data and 40% on the full distribution.
- Cost. Dictionaries often have more parameters than the layer they explain, and nobody has published an attempt to cover every part of a model.
- Shaky assumptions. SDL assumes concepts are stored as linear directions, and it treats sparsity as a stand in for interpretability. Feature splitting and feature absorption suggest that proxy breaks down under pressure.
- Missing structure. SDL describes activations, not the mechanisms in the weights that compute them, and it leaves the geometry between features unexplained.
- Missing concepts. An SAE trained on pretraining text may simply lack a good latent for the concept you need. The review cites work finding that such SAEs lacked good latents for refusing harmful requests, while SAEs trained on chat data had them.
Underneath all of this sits a conceptual gap: there is still no satisfying formal definition of a feature. The review also raises intrinsic interpretability, meaning models trained to be easy to decompose, but notes that attempts so far have not been competitive and tend to let superposition "sneak through".
Problem 2: Explanations are easy to fake and hard to validate
Once you have parts, you have to say what they do, and the standard tools all have holes. Max activating dataset examples are only correlational, depend on the dataset you sample from, and can produce a plausible story for an arbitrary direction. Gradient based attribution is often only a first order approximation. The logit lens shows a component's direct effect on the output but misses its indirect effects through later layers.
Validation is where the review is sharpest. It says conflating hypotheses with conclusions "has regrettably been commonplace" and lists interpretability illusions: explanations that looked convincing and turned out to be false. Two fixes it proposes are still open:
- Shared model organisms. The field has no agreed "fruit fly" model. The review's wish list is a model that is open source, cheap, representative, trained on open data with a replicable process, and available in several random seeds. Small open models such as Gemma, Llama and SmolLM cover some of that list, not all of it.
- Benchmarks with ground truth. One route compiles known programs into transformer weights so a method can be checked against the right answer.
Problem 3: Circuit discovery only works under the streetlight
Circuit discovery is the field's most common pipeline: pick a task, find the subgraph of the model that performs it, describe each node, and check the result for faithfulness, minimality and completeness. The review lists six problems with it:
- Tasks are defined by humans, and performance varies a lot between examples within one task, so a circuit can explain the average case and miss individual inputs.
- The nodes come from neurons or SAE latents, which inherit every problem above.
- Faithfulness is low, and faithfulness scores change depending on how you run the intervention.
- Fast methods such as attribution patching rely on gradients, a first order approximation of a real ablation.
- Backup and negative components suppress the behavior and get missed by methods that maximize task performance.
- "Streetlight interpretability": studied tasks were chosen because they are easy, and attempts on arbitrary tasks have gone less well.
Automation helps with parts of this. Language models can write and score feature descriptions, and ACDC automates finding the subgraph, but the review says fully automating current pipelines would still not give satisfying explanations.
Problem 4: Applications that beat simple baselines
The second half of the review asks whether interpretability actually helps. Its answer is: sometimes, and less than hoped.
| Goal | What has worked | What is still open |
|---|---|---|
| Monitoring | Probes and white box evaluations can flag concerning cognition | Mechanistic anomaly detection needs better decomposition |
| Red teaming | A single "refusal direction" in chat models made attacks cheaper | Mechanism level understanding rather than single features |
| Steering | Adding an activation vector changes behavior predictably | Steering whole mechanisms with few side effects |
| Unlearning and editing | ROME made early progress on editing facts | Activation based unlearning not yet competitive; ROME's flaws show the right parts have not been found |
| Fine tuning | Evidence that fine tuning makes shallow edits, so safety training can be undone cheaply | Tools to debug what fine tuning changed |
The review argues that interpretability's comparative advantage is understanding, so auditing, debugging and monitoring suit it better than control, where other fields already compete.
What changed after the review
Three developments since January 2025 matter most if you plan to work on these problems.
Attribution graphs. On March 27, 2025, Anthropic published its circuit tracing methods and a companion study of Claude 3.5 Haiku. The method swaps a model's MLPs for cross layer transcoders and traces how features feed each other on one prompt. Two of the review's problems show up directly in its limits: attention patterns are frozen rather than explained, and error nodes hold whatever the transcoders miss. The companion paper says the graphs give "satisfying insight" for about a quarter of the prompts the team tried.
Open circuit tracing tools. On May 29, 2025, Anthropic open sourced the circuit tracing tools, built by Anthropic Fellows with Decode Research. The MIT licensed circuit-tracer library works on gemma-2-2b and Llama-3.2-1b, and its README says Gemma 2 2B runs on a free Colab GPU with 15 GB. Neuronpedia hosts a no code version, and its graph server now also offers an engine based on OpenMOSS's Language-Model-SAEs that adds features for attention layers, a direct attack on the frozen attention gap.
The pragmatic turn. In March 2025 the Google DeepMind team reported that SAEs underperformed linear probes at detecting harmful intent out of distribution, and said it was deprioritizing fundamental SAE research while keeping SAEs as a tool. Chat trained SAEs closed only about half the gap. In December 2025 the same team published A Pragmatic Vision for Interpretability, which describes a pivot from complete reverse engineering to work grounded in proxy tasks, and advises trying prompting, steering, probes and reading chain of thought before fancier methods. It lists goals that still feel far out of reach: superposition (especially across layers), the complexity of sampling, multiturn and agentic tool use, dictionary learning error that "shows no signs of going away", self repair that muddies causal interventions, and inherently interpretable models that are not yet cheap enough for frontier scale.
So the open problems from the review are mostly still open. What changed is the tooling and the attitude: more people now measure methods by whether they help on a concrete task rather than by reconstruction scores.
How to start working on an open problem
You can attack several of these problems on a single GPU with open weights.
- Pick a small, well studied model. Gemma 2 2B has public transcoders and works with
circuit-tracer. Our VRAM guide helps you check what fits. - Choose a library. TransformerLens suits teaching scale models, nnsight wraps any PyTorch model, and SAELens loads and trains SAEs. Our TransformerLens vs nnsight comparison explains the tradeoffs.
- Start with a baseline. Before you train an SAE, try a linear probe or a steering vector on the same question. The DeepMind results show the simple method often wins.
- Validate like the review asks. Predict what an ablation or a steering change will do before you run it, and test on inputs outside the set you used to form the hypothesis.
- Read the sources. The Transformer Circuits thread and arXiv carry most new work.
FAQ
What is the Open Problems in Mechanistic Interpretability paper?
It is a review posted to arXiv in January 2025 (arXiv 2501.16496) by 29 researchers from groups including Apollo Research, Anthropic and Google DeepMind. It lists unsolved problems in methods, applications and the social side of the field, and suggests which ones to prioritize.
Are sparse autoencoders a dead end?
No, but they are no longer seen as the whole answer. The review lists high reconstruction error, cost and missing concepts. Google DeepMind found SAEs lost to linear probes on a harmful intent task, yet it still uses them to surface unexpected hypotheses.
Do attribution graphs solve circuit discovery?
Not yet. Anthropic's circuit tracing freezes attention patterns, leaves part of the computation in error nodes, and gave satisfying insight on about a quarter of the prompts its authors tried. It does make circuits on open models much easier to find and share.
Can I work on these problems without a lab?
Yes. The circuit-tracer README says Gemma 2 2B runs on a free Colab GPU, Neuronpedia offers no code graph generation, and TransformerLens, nnsight and SAELens are free open source libraries.
What is pragmatic interpretability?
It is the approach Google DeepMind's team described in December 2025: choose a problem that matters, measure progress with a concrete proxy task, and start with the simplest method that could work, such as prompting, steering or probing.
