Interpretability
A trained network contains structure nobody wrote down. Features, circuits and internal abstractions emerge from the training process, and they are the actual mechanism by which a model produces an output. To say anything reliable about why a model behaved a particular way, we have to be able to read that structure.
What we are working on
- Sparse feature bases. Decomposing activations into a large, sparse, human-legible basis, and asking how stable that basis is across training runs, model scales and random seeds.
- Circuit tracing. Identifying the subgraph responsible for a specific behaviour, then validating it by ablation rather than by visual inspection of an attention map.
- Causal intervention. Editing an internal representation and measuring the downstream effect, so an explanation makes a prediction that can fail.
An interpretation that cannot be wrong is not an interpretation. It is a story about a picture.
A circuit claim is only worth something if removing it changes the behaviour it is supposed to explain. Toggle the ablation to see the prediction being tested.
What we think is unresolved
Most interpretability results are demonstrated on small models and small behaviours. Whether the methods survive at scale, and whether the features they recover are the ones the model is actually using rather than the ones the method is good at finding, remain open. We treat both as empirical questions.
Evaluation and measurement
A benchmark is an instrument. Instruments drift, and an instrument that is also a training target drifts fast. Scores keep rising while the underlying capability moves less than the number suggests, and the gap between the two is where deployment surprises come from.
What we are working on
- Measurement drift. Quantifying how much of a benchmark gain is capability and how much is adaptation to the test, using held-out instrument variants built after the model was trained.
- Construct validity. Asking what a benchmark actually measures. Many tests labelled as reasoning are substantially tests of formatting, retrieval, or familiarity with a question style.
- Evaluation that ages. Designing tests whose difficulty comes from structure rather than from specific items, so the suite stays informative after it has been published and optimised against.
Both numbers go up. Only one of them is the thing anyone cares about, and the distance between them is where deployment surprises come from.
What we think is unresolved
There is no agreed method for deciding whether two evaluations measure the same thing. Until there is, aggregate leaderboard scores should be read as weak evidence about a model, and strong evidence about what its developers optimised.
Robustness and alignment
Systems behave differently in deployment than in evaluation, and the difference is not random. It concentrates in exactly the inputs that were rare in training and common in the world. Our interest is in characterising that gap before it is discovered by users.
What we are working on
- Behaviour under shift. Systematic stress tests that vary one property of the input distribution at a time, so failures can be attributed to a cause rather than noted as an anecdote.
- Which training signals generalise. Preference and feedback signals teach models something. We study what, exactly, and how far outside the feedback distribution it holds.
- Adversarial pressure as measurement. Treating attacks as a probe of the decision boundary rather than only as a threat to patch.
The headline accuracy barely moves while the tail collapses, because the tail is a small share of the test set and most of what a deployed system actually meets.
What we think is unresolved
Robustness is usually reported as a single number against a fixed attack suite. That number tells you about the suite. We would like measures that say something about the space of inputs a system will actually meet, and we do not think the field has one yet.
Efficient architectures
Capability per unit of compute is a research question. It also turns out to be an interpretability question: architectures that route computation explicitly are easier to inspect than architectures that spread it everywhere. We are interested in designs that are cheaper and more legible, and in whether that pairing is a coincidence.
What we are working on
- Sparsity and routing. Conditional computation where the routing decision is itself a readable object.
- Retrieval as memory. Moving factual recall out of weights and into an inspectable, updatable store, and measuring what that costs in reasoning quality.
- Long context. What actually degrades as context grows, separated from what people assume degrades.
The interesting property is not only that it is cheaper. It is that the routing decision is a discrete object you can read, rather than computation smeared everywhere.
What we think is unresolved
Efficiency results are reported at the scale the authors could afford, and the scaling behaviour of the gain is rarely established. We would like to publish efficiency claims with the range of scales over which we verified them, and nothing beyond it.