Larimar: Large Language Models with Episodic Memory Control
Paper
•
2403.11901
•
Published
•
32
Architectures
-
-
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
Paper • 2212.05055 • Published • 5 -
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper • 2404.02258 • Published • 104 -
Multi-Head Mixture-of-Experts
Paper • 2404.15045 • Published • 59 -
KAN: Kolmogorov-Arnold Networks
Paper • 2404.19756 • Published • 108