Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
Paper
•
2407.08348
•
Published
•
46
Lize Pirenne
Inversta
·
AI & ML interests
LLMs, RL
Organizations
None yet
Inversta's activity
upvoted
2
papers
6 days ago
upvoted
2
papers
20 days ago
upvoted
2
papers
25 days ago
upvoted
2
papers
27 days ago
Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
Paper
•
2406.11817
•
Published
•
13
The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing
Paper
•
2406.10601
•
Published
•
65
upvoted
2
papers
28 days ago
upvoted
6
papers
about 1 month ago
mDPO: Conditional Preference Optimization for Multimodal Large Language Models
Paper
•
2406.11839
•
Published
•
36
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
Paper
•
2406.08973
•
Published
•
85
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
Paper
•
2406.07522
•
Published
•
35
What If We Recaption Billions of Web Images with LLaMA-3?
Paper
•
2406.08478
•
Published
•
38
Are We Done with MMLU?
Paper
•
2406.04127
•
Published
•
36
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
Paper
•
2406.08464
•
Published
•
49
upvoted
9
papers
about 2 months ago
Self-Improving Robust Preference Optimization
Paper
•
2406.01660
•
Published
•
18
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Paper
•
2405.21060
•
Published
•
61
Transformers Can Do Arithmetic with the Right Embeddings
Paper
•
2405.17399
•
Published
•
50
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models
Paper
•
2405.15574
•
Published
•
52
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Paper
•
2405.12981
•
Published
•
26
OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
Paper
•
2405.11143
•
Published
•
33
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
Paper
•
2405.12130
•
Published
•
44
FIFO-Diffusion: Generating Infinite Videos from Text without Training
Paper
•
2405.11473
•
Published
•
53
RLHF Workflow: From Reward Modeling to Online RLHF
Paper
•
2405.07863
•
Published
•
62
upvoted
9
papers
3 months ago
Layer Skip: Enabling Early Exit Inference and Self-Speculative Decoding
Paper
•
2404.16710
•
Published
•
56
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
Paper
•
2404.14619
•
Published
•
124
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Paper
•
2404.14219
•
Published
•
243
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper
•
2404.08801
•
Published
•
62
Rho-1: Not All Tokens Are What You Need
Paper
•
2404.07965
•
Published
•
80
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Paper
•
2404.05892
•
Published
•
28
Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper
•
2404.02258
•
Published
•
102
Jamba: A Hybrid Transformer-Mamba Language Model
Paper
•
2403.19887
•
Published
•
100
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Paper
•
2404.07143
•
Published
•
98
upvoted
14
papers
4 months ago
FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation
Paper
•
2403.12962
•
Published
•
7
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Paper
•
2403.09611
•
Published
•
123
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
Paper
•
2403.09629
•
Published
•
54
Gemma: Open Models Based on Gemini Research and Technology
Paper
•
2403.08295
•
Published
•
45
MoAI: Mixture of All Intelligence for Large Language and Vision Models
Paper
•
2403.07508
•
Published
•
73
Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
Paper
•
2402.14848
•
Published
•
18
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
Paper
•
2403.03950
•
Published
•
11
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Paper
•
2403.05530
•
Published
•
52
Yi: Open Foundation Models by 01.AI
Paper
•
2403.04652
•
Published
•
59
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Paper
•
2403.04132
•
Published
•
38
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Paper
•
2403.03507
•
Published
•
180
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Paper
•
2402.19427
•
Published
•
50
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Paper
•
2402.17764
•
Published
•
581
LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
Paper
•
2402.13753
•
Published
•
108
upvoted
10
papers
5 months ago
StarCoder 2 and The Stack v2: The Next Generation
Paper
•
2402.19173
•
Published
•
126
Linear Transformers with Learnable Kernel Functions are Better In-Context Models
Paper
•
2402.10644
•
Published
•
75
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts
Paper
•
2402.09727
•
Published
•
35
ODIN: Disentangled Reward Mitigates Hacking in RLHF
Paper
•
2402.07319
•
Published
•
13
A Tale of Tails: Model Collapse as a Change of Scaling Laws
Paper
•
2402.07043
•
Published
•
13
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
Paper
•
2402.07827
•
Published
•
43
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning
Paper
•
2402.06619
•
Published
•
51
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
Paper
•
2402.04291
•
Published
•
48
Grandmaster-Level Chess Without Search
Paper
•
2402.04494
•
Published
•
65
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Paper
•
2402.03300
•
Published
•
67
upvoted
2
papers
6 months ago