Iterative Reasoning Preference Optimization
Paper
•
2404.19733
•
Published
•
41
Foundation AI Papers (II)
-
-
Better & Faster Large Language Models via Multi-token Prediction
Paper • 2404.19737 • Published • 61 -
ORPO: Monolithic Preference Optimization without Reference Model
Paper • 2403.07691 • Published • 56 -
KAN: Kolmogorov-Arnold Networks
Paper • 2404.19756 • Published • 91 -
Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations
Paper • 2303.02536 • Published • 1 -
Suppressing Pink Elephants with Direct Principle Feedback
Paper • 2402.07896 • Published • 7 -
Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
Paper • 2310.01801 • Published • 3 -
Aligning LLM Agents by Learning Latent Preference from User Edits
Paper • 2404.15269 • Published • 1 -
Language-Image Models with 3D Understanding
Paper • 2405.03685 • Published • 1 -
Chain of Thoughtlessness: An Analysis of CoT in Planning
Paper • 2405.04776 • Published • 1 -
Memory Mosaics
Paper • 2405.06394 • Published • 2 -
The Consensus Game: Language Model Generation via Equilibrium Search
Paper • 2310.09139 • Published • 12 -
RLHF Workflow: From Reward Modeling to Online RLHF
Paper • 2405.07863 • Published • 52
PHUDGE: Phi-3 as Scalable Judge
Paper • 2405.08029 • Published • 1Note LoRA fine-tune on judge LM, using dataset from Prometheus's 10K feedback dataset. Turn LLM into a classifier to increase 'overfitting' and get a slightly better performing model based on Phi-3 (which arguably already have a stronger performance than Mistral) Not that surprising, and using large dataset to fine-tune on human preference is boring. They did release code for the experiment which is nice to have. The real gem is efficient alignment.
-
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Paper • 2405.09220 • Published • 20 -
Understanding the performance gap between online and offline alignment algorithms
Paper • 2405.08448 • Published • 10 -
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?
Paper • 2405.05904 • Published • 3 -
Robust agents learn causal world models
Paper • 2402.10877 • Published • 2