Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2404.07129

🔍 Daily Picks in Interpretability & Analysis of LMs

Outstanding research in interpretability and evaluation of language models, summarized

Calibrating Reasoning in Language Models with Internal Consistency

Paper • 2405.18711 • Published 3 days ago • 1
Emergence of a High-Dimensional Abstraction Phase in Language Transformers

Paper • 2405.15471 • Published 8 days ago • 1
Not All Language Model Features Are Linear

Paper • 2405.14860 • Published 9 days ago • 33
Your Transformer is Secretly Linear

Paper • 2405.12250 • Published 13 days ago • 133

Papers - ICL - Residual Head Hypothesis

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - XAI - Attention - LayerNorm

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Phase Change Delay - Large Vocabulary Size

Larger vocab is better compression, but may result in longer training ICL phase change delays due to the slower Induction Head Copy Subcircuit (C)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Phase Change - Delay - Classes and Labels

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Induction Head - Copy vs QK Match

See figure 6: Classes vs labels in columns B and C. Subcircuit B delays phase change on number classes vs C delays on number of labels (dramatically)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - Training - ICL - Induction Circuit Evolution

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Induction Circuit - Data Dependent Learning

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - ICL - Induction Head - Num Labels vs Classes - Loss

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3

Papers - XAI - Induction Head - Phase Change - Components

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10 • 3
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions

Paper • 2403.07809 • Published Mar 12 • 1

Previous
1
2
3
4
Next

Company

© Hugging Face

TOS Privacy About Jobs

Website

Models Datasets Spaces Pricing Docs