heads. URL: https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html.
OpenAI (2023). GPT-4 Technical Report. arXiv: 2303.08774 [cs.CL].
Pearl, Judea (2009). Causality. Models, Reasoning, and Inference. 2nd ed. Cambridge University Press. ISBN: 978-0-521-89560-6. DOI: 10.1017/CBO9780511803161.
Radford, Alec, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever (2019). "Language Models are Unsupervised Multitask Learners". In.
Ramanujan, Vivek, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari (2020). "What's Hidden in a Randomly Weighted Neural Network?" In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, pp. 11890-11899. DOI: 10.1109/CVPR42600.2020.01191. URL: https://doi.org/10.1109/CVPR42600.2020.01191.
Räuker, Tilman, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell (2022). Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks. URL: https://arxiv.org/abs/2207.13243.
Sanh, Victor, Thomas Wolf, and Alexander M. Rush (2020). "Movement Pruning: Adaptive Sparsity by Fine-Tuning". In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. Ed. by Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin. URL: https://proceedings.neurips.cc/paper/2020/bit/ eae15aabaa768ae4a5993a8a4f4fa6e4-Abstract.html.
Syed, Aaquib, Can Rager, and Arthur Conmy (2023). Attribution Patching Outperforms Automated Circuit Discovery. arXiv: 2310.10348 [cs.LG].
Tigges, Curt, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda (2023). Linear Representations of Sentiment in Large Language Models. arXiv: 2310.15154 [cs.LG].
Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin (2017). "Attention is All you Need". In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. Ed. by Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, pp. 5998-6008. URL: https://proceedings.neurips.cc/paper/2017/bit/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.
Veit, Andreas, Michael J. Wilber, and Serge J. Belongie (2016). "Residual Networks Behave Like Ensembles of Relatively Shallow Networks". In: Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain. Ed. by Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, pp. 550-558. URL: https://proceedings.neurips.cc/paper/2016/bitstream/37bc2f75bf1bcfe8450a1a41c200364c-Abstraction.html.
Vig, Jesse, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang, Yaron Singer, and Stuart Shieber (2020). Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias. URL: https://arxiv.org/abs/2004.12265.
Wang, Kevin Ro, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt (2023). "Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small". In: The Eleventh International Conference on Learning Representations. URL: https://openreview.net/forum?id=NpsVSN6o4ul.
Wang, Ziheng, Jeremy Wohlwend, and Tao Lei (2020). "Structured Pruning of Large Language Models". In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, pp. 6151-6162. DOI: 10.18653/v1/2020.emnlp-main.496. URL: https://aclanthology.org/2020.emnlp-main.496.
Wei, Jason, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus (2022). Emergent Abilities of Large Language Models. arXiv: 2206.07682 [cs.CL].
Wu, Zhengxuan, Atticus Geiger, Christopher Potts, and Noah D. Goodman (2023). Interpretability at Scale: Identifying Causal Mechanisms in Alpaca. arXiv: 2305.08809 [cs.CL].
Wu, Zhengxuan, Atticus Geiger, Joshua Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, and Noah Goodman (2022). "Causal Distillation for Language Models". In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational