ROTATE: Data-Free Method Disentangles LLM Neurons in Weight Space Using Vocabulary Kurtosis

ROTATE: Understanding LLM Neurons Without Running the Model — A Breakthrough in Mechanistic Interpretability

Researchers have developed ROTATE (Rotation-Optimized Token Alignment in weighT spacE), a data-free method that disentangles what individual neurons in LLMs encode by analyzing their weights directly — without any forward passes.

The Key Insight

Neurons that encode coherent, monosemantic concepts exhibit high kurtosis when projected onto the model's vocabulary. By optimizing rotations of neuron weights to maximize this kurtosis, ROTATE recovers interpretable "vocabulary channels."

What Makes ROTATE Special

Results

Tested on Llama-3.1-8B-Instruct and Gemma-2-2B-it:

• Recovers vocabulary channels faithful to neuron behavior
• Ablating individual channels selectively disables specific input activations or concepts
• Aggregated channel descriptions outperform activation-based baselines by 2-3x in head-to-head comparisons

Why This Matters

1. Mechanistic interpretability at scale — Understanding 8B+ parameter models without expensive compute
2. Safety research — Identifying what neurons encode helps detect dangerous capabilities
3. Model editing — Knowing which channels control which concepts enables precise modifications
4. No data needed — Privacy-preserving interpretability (no training data required)