Reproducing Toy Models of Superposition: Training a Tiny Compression Network in NumPy

Neural network interpretability often feels like navigating a black box, but stripping away high-level frameworks exposes the raw mechanics of representation learning. According to the investigation published on Towards Data Science, training a miniature autoencoder using hand-derived gradients in pure NumPy spontaneously causes the hidden layer to geometric structures like a regular pentagon when forced to compress high-dimensional inputs.

Architectural Setup of the NumPy Compression Network

To replicate Anthropic's superposition phenomena without hidden abstractions, the architecture relies on a deterministic bottleneck layer designed to project inputs into a lower-dimensional subspace. The network takes continuous feature vectors and passes them through a single linear encoder followed by a non-linear activation, before attempting reconstruction via a symmetrical decoder.

Key Takeaways

• Built entirely in pure NumPy with hand-derived backpropagation gradients.
• Forces a high-dimensional feature space through a 2D bottleneck layer.
• Spontaneously develops geometric representations such as a regular pentagon during training.

Implementing Forward and Backward Passes Without Autograd

Writing custom gradient descent updates forces absolute clarity on matrix dimensions and derivative chains. Below is the core training step executing forward projections and weight updates manually across NumPy arrays.

Emergence of Geometric Superposition in Bottleneck Layers

When the input dimensionality exceeds the bottleneck capacity, the network cannot store every feature orthogonally. Instead, optimization pressure forces the weight vectors to arrange themselves into symmetric polygons - in this specific experiment, perfectly outlining the vertices of a pentagon to maximize information density.

Validating Weight Vector Coordinates and Feature Angles

Inspecting the weight matrix post-training reveals that the feature vectors converge to precise angular separations matching regular polygons. Developers studying mechanistic interpretability can use this minimalist setup to test feature interference, sparsity penalties, and polysemanticity without incurring heavy GPU overhead or framework bloat.

Conclusion

Building neural networks from first principles in NumPy strips away framework magic and lays bare the mathematical elegance of representation learning. Observing a tiny network spontaneously draw a pentagon proves that geometric superposition is an inevitable mathematical consequence of optimizing capacity constraints under informational pressure.