Package: transpector
🔬 TranspectorVisually inspect, analyse and debug transformer models. Aimed at accelerating interpretability research and reducing the barrier to entry.Getting StartedUsing TranspectorTo use Transpector first install the packagepip install transpectorthen you can start it from the command line withtranspector. Once it's ready, navigate tohttp://127.0.0.1:8000/index.htmlto get started.FeaturesAnalyse, ablate and freeze attention heads (Look at what each head is predicting below).Analyse loss (Look at how loss drops on a random repeated sequence below).Select any model supported by Transformer Lens (there is a todo for correctly linking shortformer models)Interactive jupyter notebook linked to the Transpector's display (Add custom hooks in py and see the affect on the model results).MotivationsThrough determined effortseveralgroupshave(not exhaustive, only illustrative) been able to decode probable circuits and algorithms within transformers. Doing so has required significant technical knowledge, experience and time. This, in my eyes doesn't necessarily have to be true hence the foundational question of transpector - How can we reduce the iteration time and barrier of entry for useful mechanistic analysis of transformer models?Contributing and Developing locallyContributingTranspector's code base is designed to be extended, and it's easy too see how much opportunity there is to take it further. So if you've got some great ideas, want to tie in your research, or just help out with the todo list, please join in!Developing locallyFor efficient development, we want to enable hot reloading of both the typescript and python sides. We can do this for py bypip install -e .(from the root directory of this project) to install the python package based out of this directory, meaning when we change a file in this dir we also change the package. From the next.js TS side we can start up a dev server usingnpm run dev, (also in the root dir of this project as per the pip install). This will be hosted at a different point (currently:80) but will proxy all api requests to the same port where the standard python backend runs, so if you are only working on the TS you can just use thetranspectorcommand to start up the backend.A short note on the py backend, you can examine how transpector starts by looking through launch.py, but in breif:We start a jupyter-server in a threadConnect to it through httpCreate a IPython kernal and start itConnect to it through websocketsLoad our starter notebookExecute the first few cells in the starter notebook, loading our model of interestExecute the last few cells in our starter notebook, spawning a fastAPI backend in the same event loop as the IPython KernalHost our static bundle of next.js along with our backend api from that fastAPI backendTo run Next JSnpmrundevTo manually launch the python sidepythonlaunch.pyBuilding the packageFirst build the next js into a static output doingnpm run buildThen copy the static out folder from./outto./transpector/frontend_dist.Now you can build the python package by doing.python setup.py bdist_wheel sdistYou should see that the static files are included in the distribution from the output from the build command. Now we can install the package usingpip install dist/<build-name>And finally, use our installed package to run the server:transpectorTo DoMilestone 2️⃣ROME/ MEMIT style search and replace engine for MLP knowledgeAutomatic node graph generation based on model graphAbility to mix in copy and mix in pinned activationsAdd ability to split each token into seperate nodeInferencing layer cutoff (both where to start and end)Add graident visualisations (What would be useful here?)Basic Layernorm visuals (what would be useful?)Individual activation level ablationsAutoencoder polysemic decoding node attachmentText pane batch inputDataset upload and controlsBatch example selector across appJupyter cell popups for adding mid model hooksNotebook autosavingNotebook bring in all jupyterlab featuresWebsockets for push pull state between py/jsMove to vector logical clocks for everythingMulti-node selection menuArchitecture diagramAdd correct shortformer positional embedding linksResearch replication using transpectorAbility to add notes to activations, weights and nodesMilestone 1️⃣Add icons on the edgesWeight Analysis: Visualise weights (What would be useful?)Weight Analysis: What is the OV circuit trying to copy given a tokenWeight Analysis: What is the KQ circuit attention position distributionVisual layer foldingAttention pattern similarity searchLayer and node type filtersResidual stream analysis visualisationMove to same data structure in py/ js and ws for syncingCompute/ GPU flags/controls from UIBasic MLP visuals (what would be useful? Dictionary learning, key-value? Neuron view?)Edge editing automatically performs patchingActivation freezingAblationsCanvas summed pattern and result renderingAdd readme creditsModel OutputsMLP nodesKQV basic visualisationLayernorm nodesNotebook loading pre existing kernelPypi packageUpgrade model selection paneImprove typing coverageNext static exportPython kickoff scriptPython static hostingPython packageModel Pane UpgradeNodes disabled, default and highlighted featureJupyter-UI upgradeNode resizing and rerendering (for canvas nodes)Refactor Nodes and edges codeLayer groupsAttention head nodesCombined attention nodeText input nodeTokenization nodeResidual stream nodeAttention text visualisationChoresBasic usage tutorialMinimap collision with text paneReloading webpage loses some stateOptimisation: OffscreenCanvasOptimisation: WebGPU renderingOptimisation: Remove dead dataNon blocking model updatingNon blocking renderingImprove global state controlFaviconSettings paneSupport running on remote serversSupport import into standard notebookMake gif of patchingMake gif of resizingKeyboard shortcutsAdd in app creditsCleanup slicesCleanup activation passingNode resizing for all visual nodesMake gifs for each key featureLauncher cleanupMake main gifText pane visibilityModel selection bugsLayernaming bugsNotebook scrolling bugsVisual spacing optimisationCreditsThis work has only been possible through the usage of Transformer Lens by Neel Nanda (also thanks to Redwood Research for Easy Transformer). It has also been heavily inspired by circuitviz by Alan Cooney and Anthropic's visualisation work.CitationIf transpector aided in your research, you can cite transpector as:@misc{kopeltranspector,
    title  = {Transpector},
    author = {Kopel, Rob},
    url    = {https://github.com/R0bk/Transpector},
    year   = {2023}
}Furthermore, please look through the above credits as transpector would not be possible without them. If you have any questions or would like to collaborate please feel free  to reach out to me at rob.kopel on gmail.