# Mathematical Circuits in Neural Networks

[*(Also posted on the EA Forum*](https://forum.effectivealtruism.org/posts/AQRvQ3AuQaPmuurk8/mathematical-circuits-in-neural-networks)*)*

*This is one of my final projects for the* [*Columbia EA Summer 2022 Project Based AI Safety Reading Group*](https://www.columbia-ea.org/groups/ai-safety) *(special thanks to facilitators Rohan Subramini and Gabe Mukobi). If you're curious you can find my other project* [*here*](https://forum.effectivealtruism.org/posts/ixa4mM9aYF4yyqj84/ai-safety-executive-summary)*.*

**Summary**
-----------

In this project, I:

1.   Derive by hand the optimal configurations (architecture and weights) of "vanilla" neural networks ([multilayer perceptrons](https://en.wikipedia.org/wiki/Multilayer_perceptron); [ReLU](https://en.wikipedia.org/wiki/Rectifier_(neural_networks)) activations) that implement basic mathematical functions (e.g. absolute value, minimum of two numbers, etc.)
2.  Identify "features" and "circuits" of these networks that are reused repeatedly across networks modeling different mathematical functions
3.  Verify these theoretical results empirically ([in code](https://github.com/sosier/Mathematical_Circuits_in_Neural_Nets/blob/master/Mathematical_Circuits.ipynb))

What follows is a brief introduction to this work. For full details, please see:

*   The [linked video](https://www.youtube.com/watch?v=jGQN0TVCtMo) (also embedded at the bottom of this post)
*   Or if you prefer to go at your own pace, [the slides](https://github.com/sosier/Mathematical_Circuits_in_Neural_Nets/blob/master/Mathematical_Circuits_in_Neural_Networks.pdf) I walk through in that video

**Motivation**
--------------

[Olah et al.](https://distill.pub/2020/circuits/zoom-in/) make three claims about the fundamental interpretability of neural networks:

[![Three Claims](https://user-images.githubusercontent.com/13408985/189792395-8c4ee31b-3d4b-42db-aa62-6a05e3ae6b0c.png)](https://user-images.githubusercontent.com/13408985/189792395-8c4ee31b-3d4b-42db-aa62-6a05e3ae6b0c.png)

They demonstrate these claims in the context of image models:

***Features / Circuits:***

[![Features / Circuit](https://user-images.githubusercontent.com/13408985/189792613-42663d32-3e48-4a3b-846d-331714dca639.png)](https://user-images.githubusercontent.com/13408985/189792613-42663d32-3e48-4a3b-846d-331714dca639.png)

***Universality:***

[![Universality](https://user-images.githubusercontent.com/13408985/189792851-3a05d17b-cb22-4b7f-a6fd-09775510401a.png)](https://user-images.githubusercontent.com/13408985/189792851-3a05d17b-cb22-4b7f-a6fd-09775510401a.png)

This work demonstrates the same concepts apply in the space of neural networks modeling basic mathematical functions.

**Results**
-----------

Specifically, I show that the optimal network for calculating the minimum of two arbitrary numbers is fully constructed from smaller "features" and "circuits" used across even simpler mathematical functions. Along the way, I explore:

*   "Positiveness" and "Negativeness" Detectors
*   Identity Circuits (i.e. f(x) = x)
*   Negative Identity Circuits (i.e. f(x) = -x)
*   Subtraction Circuits (i.e. f(x1, x2) = x1 - x2)
*   "Greaterness" Detectors
*   And More

***Minimum Network:***

[![Minimum Network](https://user-images.githubusercontent.com/13408985/190928502-f908fead-78f7-4568-83f6-2b1d001fafe6.png)](https://user-images.githubusercontent.com/13408985/190928502-f908fead-78f7-4568-83f6-2b1d001fafe6.png)

I also demonstrate that each of these theoretical results hold in practice. [The code for these experiments](https://github.com/sosier/Mathematical_Circuits_in_Neural_Nets/blob/master/Mathematical_Circuits.ipynb) can be found on [the GitHub page for this project](https://github.com/sosier/Mathematical_Circuits_in_Neural_Nets).

**Full Details**
----------------

For full details, please see the [PDF presentation](https://github.com/sosier/Mathematical_Circuits_in_Neural_Nets/blob/master/Mathematical_Circuits_in_Neural_Networks.pdf) in the GitHub repo or watch the full video walkthrough: