qinluo (qinluo)

upvoted a paper 24 days ago

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Paper • 2406.17557 • Published 27 days ago • 75

upvoted a collection about 1 month ago

AnyTaskTune-Psychology

Collection

Elevating Domain-Specific Model Performance with Precision Task-Specific Fine-Tuning • 3 items • Updated Jun 7 • 2

upvoted a paper about 2 months ago

Xmodel-LM Technical Report

Paper • 2406.02856 • Published Jun 5 • 7

upvoted a paper 2 months ago

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design

Paper • 2401.14112 • Published Jan 25 • 17

upvoted 3 papers 3 months ago

Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression

Paper • 2403.15447 • Published Mar 18 • 15

WildChat: 1M ChatGPT Interaction Logs in the Wild

Paper • 2405.01470 • Published May 2 • 57

KAN: Kolmogorov-Arnold Networks

Paper • 2404.19756 • Published Apr 30 • 102

upvoted 3 articles 3 months ago

Article

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

By

•

Apr 29

• 28

Article

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

By

•

Jun 4

• 65

Article

Fine-tune Llama 3 with ORPO

By

•

Apr 22

• 212

upvoted a paper 3 months ago

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws

Paper • 2404.05405 • Published Apr 8 • 4

upvoted 8 papers 4 months ago

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

Paper • 2404.02905 • Published Apr 3 • 63

Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Paper • 2404.02258 • Published Apr 2 • 102

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Paper • 2402.06619 • Published Feb 9 • 51

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

Paper • 2402.13064 • Published Feb 20 • 46

What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

Paper • 2312.15685 • Published Dec 25, 2023 • 17

Instruction-tuned Language Models are Better Knowledge Learners

Paper • 2402.12847 • Published Feb 20 • 24

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9 • 24

ORPO: Monolithic Preference Optimization without Reference Model

Paper • 2403.07691 • Published Mar 12 • 59

upvoted a collection 4 months ago

Common Corpus

Collection

The largest public domain dataset for training LLMs. • 27 items • Updated 5 days ago • 107

upvoted a paper 4 months ago

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Paper • 2403.03507 • Published Mar 6 • 180

upvoted a paper 7 months ago

LLM360: Towards Fully Transparent Open-Source LLMs

Paper • 2312.06550 • Published Dec 11, 2023 • 54

upvoted a paper 8 months ago

GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation

Paper • 2311.07562 • Published Nov 13, 2023 • 12

upvoted a paper 9 months ago

Zephyr: Direct Distillation of LM Alignment

Paper • 2310.16944 • Published Oct 25, 2023 • 120

upvoted 4 papers 10 months ago

Language Modeling Is Compression

Paper • 2309.10668 • Published Sep 19, 2023 • 82

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

Paper • 2309.09400 • Published Sep 17, 2023 • 78

FLM-101B: An Open LLM and How to Train It with $100K Budget

Paper • 2309.03852 • Published Sep 7, 2023 • 42

Textbooks Are All You Need II: phi-1.5 technical report

Paper • 2309.05463 • Published Sep 11, 2023 • 84

upvoted 2 papers 12 months ago

Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models

Paper • 2308.00304 • Published Aug 1, 2023 • 23

MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

Paper • 2305.07185 • Published May 12, 2023 • 9

upvoted 4 papers about 1 year ago

qinluo

AI & ML interests

Organizations

qinluo's activity

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

AnyTaskTune-Psychology

Xmodel-LM Technical Report

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design

Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression

WildChat: 1M ChatGPT Interaction Logs in the Wild

KAN: Kolmogorov-Arnold Networks

⚗️ 🧑🏼‍🌾 Let's grow some Domain Specific Datasets together

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

Fine-tune Llama 3 with ORPO

Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

Instruction-tuned Language Models are Better Knowledge Learners

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

ORPO: Monolithic Preference Optimization without Reference Model

Common Corpus

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

LLM360: Towards Fully Transparent Open-Source LLMs

GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation

Zephyr: Direct Distillation of LM Alignment

Language Modeling Is Compression

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

FLM-101B: An Open LLM and How to Train It with $100K Budget

Textbooks Are All You Need II: phi-1.5 technical report

Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models

MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers

Retentive Network: A Successor to Transformer for Large Language Models

Copy Is All You Need

Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-Tuning

Training Transformers with 4-bit Integers