Nick Cammarata argues neural networks are already clean
His essay says many cases of polysemanticity fall to further agentic analysis.
Research#04
90%
of Qwen-32B neurons have one job
Image: Latent Space
90%of Qwen-32B neurons have one job
The short version
- Nick Cammarata at Goodfire has published an essay with his working theory that neural networks are already clean.
- The essay argues that many known cases of polysemanticity and superposition simply fall to further agentic analysis.
- Deedy wrote that neural networks are not tangled knots and that their neurons can be studied directly, without first unfolding them with a second model.
- Deedy also said that 90% of neurons in Qwen-32B have one job.
Why it matters
Interpretability is treated as central to alignment, since Cammarata argues that alignment cannot be solved without knowing what we are building. If neurons can be studied directly, interpretability could become a more routine science, and roon wrote that alignment could become a normal science.
“This is one of the most important results in AI, period.”