Description

In this very impressive work, Anthropic tackle interpretability of Large Language Models. Working around the problem of superposition causing polysemanticity, they use a weak dictionary learning algorithm called a sparse autoencoder to generate learned features from a trained model that offer a more monosemantic unit of analysis than the model's neurons themselves.