Axiomi.si
Submit a tool

// category · 8 tools · 6 open source

Interpretability

Libraries and platforms for opening the black box — hooking activations, training sparse autoencoders, attributing outputs and publishing circuit-level findings.

SHAPshap.readthedocs.ioGame-theoretic approach to explaining the output of any machine learning model using Shapley values.InterpretabilityOpen source★ 25.8kCaptumcaptum.aiPyTorch library for model interpretability with integrated gradients, saliency and other attribution methods.InterpretabilityOpen source★ 5.7kTransformerLensgithub.comLibrary for mechanistic interpretability that exposes and edits the internal activations of GPT-style models.InterpretabilityOpen source★ 3.9kSAELensgithub.comToolkit for training and analysing sparse autoencoders that decompose model activations into interpretable features.InterpretabilityOpen source★ 1.5kNeuronpedianeuronpedia.orgOpen platform for exploring, searching and steering interpretable features discovered inside language models.InterpretabilityOpen source★ 1.2knnsightnnsight.netLibrary for inspecting and intervening on model internals, with remote execution on large models through NDIF.InterpretabilityOpen source★ 1.1kTransformer Circuitstransformer-circuits.pubAnthropic's interpretability publication thread, home to influential work on features, superposition and circuit tracing.InterpretabilityFreeHN 4Goodfiregoodfire.comInterpretability research company building tools to understand and edit the internal workings of neural networks.InterpretabilitySee siteHN 3

More categories

Open-Weight ModelsML FrameworksTraining & Fine-tuningDatasetsResearch Platforms