dar-tau commited on
Commit
0d6b098
β€’
1 Parent(s): 18261f8

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +11 -12
app.py CHANGED
@@ -147,18 +147,17 @@ with gr.Blocks(theme=gr.themes.Default(), css=css) as demo:
147
  global_state = gr.State([])
148
  with gr.Row():
149
  with gr.Column(scale=5):
150
- gr.Markdown('''
151
- # 😎 Self-Interpreting Models
152
-
153
- πŸ‘Ύ **This space is a simple introduction to the emerging trend of models interpreting their _own hidden states_ in free form natural language**!! πŸ‘Ύ
154
- This idea was explored in the paper **Patchscopes** ([Ghandeharioun et al., 2024](https://arxiv.org/abs/2401.06102)) and was later investigated further in **SelfIE** ([Chen et al., 2024](https://arxiv.org/abs/2403.10949)).
155
- An honorary mention of **Speaking Probes** ([Dar, 2023](https://towardsdatascience.com/speaking-probes-self-interpreting-models-7a3dc6cb33d6) -- my own work!! πŸ₯³) which was less mature but had the same idea in mind.
156
- We will follow the SelfIE implementation in this space for concreteness. Patchscopes are so general that they encompass many other interpretation techniques too!!!
157
-
158
- πŸ‘Ύ **The idea is really simple: models are able to understand their own hidden states by nature!** πŸ‘Ύ
159
- If I give a model a prompt of the form ``User: [X] Assistant: Sure'll I'll repeat your message`` and replace ``[X]`` *during computation* with the hidden state we want to understand,
160
- we hope to get back a summary of the information that exists inside the hidden state, because it is encoded in a latent space the model uses itself!! How cool is that! 😯😯😯
161
- ''', line_breaks=True)
162
  with gr.Column(scale=1):
163
  gr.Markdown('<span style="font-size:180px;">πŸ€”</span>')
164
 
 
147
  global_state = gr.State([])
148
  with gr.Row():
149
  with gr.Column(scale=5):
150
+ gr.Markdown('# 😎 Self-Interpreting Models')
151
+ with gr.Accordion(label='πŸ‘Ύ **This space is a simple introduction to the emerging trend of models interpreting their _own hidden states_ in free form natural language**!! πŸ‘Ύ'):
152
+ gr.Markdown(
153
+ '''This idea was explored in the paper **Patchscopes** ([Ghandeharioun et al., 2024](https://arxiv.org/abs/2401.06102)) and was later investigated further in **SelfIE** ([Chen et al., 2024](https://arxiv.org/abs/2403.10949)).
154
+ An honorary mention of **Speaking Probes** ([Dar, 2023](https://towardsdatascience.com/speaking-probes-self-interpreting-models-7a3dc6cb33d6) -- my own work!! πŸ₯³) which was less mature but had the same idea in mind.
155
+ We will follow the SelfIE implementation in this space for concreteness. Patchscopes are so general that they encompass many other interpretation techniques too!!!
156
+
157
+ πŸ‘Ύ **The idea is really simple: models are able to understand their own hidden states by nature!** πŸ‘Ύ
158
+ If I give a model a prompt of the form ``User: [X] Assistant: Sure'll I'll repeat your message`` and replace ``[X]`` *during computation* with the hidden state we want to understand,
159
+ we hope to get back a summary of the information that exists inside the hidden state, because it is encoded in a latent space the model uses itself!! How cool is that! 😯😯😯
160
+ ''', line_breaks=True)
 
161
  with gr.Column(scale=1):
162
  gr.Markdown('<span style="font-size:180px;">πŸ€”</span>')
163