In mechanistic interpretability, what type of model is trained on a network's internal activations to decompose them into a large set of sparse, more human-interpretable features, helping address superposition?

Show answer

Sparse autoencoder (SAE)

Sparse autoencoders learn an overcomplete, sparsely-activating dictionary of features from a model's activations, helping disentangle the "superposition" in which many concepts share the same neurons. They have become a leading tool for naming and steering the features a network uses.

Also asked as:

In mechanistic interpretability, what overcomplete models are used to extract a dictionary of sparsely-activating features from a network's activations?

Would you remember this in 6 months?

Trivia Forge uses spaced repetition so facts stick. Free, no signup needed.

Turn on Machine Learning practice →

More Machine Learning trivia

In Principal Component Analysis (PCA), the principal components are the eigenvectors of which specific matrix?
In Reinforcement Learning, what equation expresses the value of a state as the expected immediate reward plus the discounted value of the next state?
In Reinforcement Learning, what is the term for the tension between trying new actions to find better rewards and choosing known actions that yield high rewards?
In Support Vector Machines, what is the 'Kernel Trick' used to achieve?
In the Bias-Variance Tradeoff, what component of error is reduced by increasing model complexity?
In the bias-variance tradeoff, what usually happens to the variance as the model complexity (degrees of freedom) increases?

See all 98 →