Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models
Paper
•
2311.00871
•
Published
•
2
Papers - ICL - In-Context Learning
-
-
Can large language models explore in-context?
Paper • 2403.15371 • Published • 32 -
Data Distributional Properties Drive Emergent In-Context Learning in Transformers
Paper • 2205.05055 • Published • 2 -
Long-context LLMs Struggle with Long In-context Learning
Paper • 2404.02060 • Published • 35 -
WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents
Paper • 2404.05902 • Published • 20 -
LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency
Paper • 2404.12872 • Published • 11 -
What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
Paper • 2404.07129 • Published • 3 -
In-context Learning and Induction Heads
Paper • 2209.11895 • Published • 2 -
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions
Paper • 2403.07809 • Published • 1 -
Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space
Paper • 2406.19370 • Published • 1 -
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering
Paper • 2311.06668 • Published • 5 -
BERTs are Generative In-Context Learners
Paper • 2406.04823 • Published • 1