Cartinoe5930 (Hyunwoo Ko)

upvoted an article 3 days ago

Article

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

4 days ago

• 129

upvoted a paper 4 days ago

AgentInstruct: Toward Generative Teaching with Agentic Flows

Paper • 2407.03502 • Published 23 days ago • 36

upvoted an article 12 days ago

Article

The Rise of Agentic Data Generation

By

•

12 days ago

• 69

upvoted an article 21 days ago

Article

🪆 Introduction to Matryoshka Embedding Models

Feb 23

• 30

upvoted a collection 24 days ago

🔍 Daily Picks in Interpretability & Analysis of LMs

Collection

Outstanding research in interpretability and evaluation of language models, summarized • 63 items • Updated 3 days ago • 75

upvoted 3 papers about 1 month ago

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges

Paper • 2406.12624 • Published Jun 18 • 35

How Do Large Language Models Acquire Factual Knowledge During Pretraining?

Paper • 2406.11813 • Published Jun 17 • 29

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

Paper • 2406.08464 • Published Jun 12 • 50

upvoted an article about 2 months ago

Article

Uncensor any LLM with abliteration

By

•

Jun 13

• 250

upvoted a paper about 2 months ago

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper • 2405.21060 • Published May 31 • 61

upvoted a paper 2 months ago

RLHF Workflow: From Reward Modeling to Online RLHF

Paper • 2405.07863 • Published May 13 • 62

upvoted an article 3 months ago

Article

makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch

By

•

May 7

• 31

upvoted a collection 3 months ago

Model Merging

Collection

Model Merging is a very popular technique nowadays in LLM. Here is a chronological list of papers on the space that will help you get started with it! • 30 items • Updated Jun 12 • 197

upvoted 2 papers 3 months ago

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

Paper • 2405.01535 • Published May 2 • 110

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Paper • 2404.18796 • Published Apr 29 • 67

upvoted 2 articles 3 months ago

Article

Expanding Model Context and Creating Chat Models with a Single Click

By

•

Apr 28

• 35

Article

Can We Train Chat Models with Raw Data?

By

•

Apr 25

• 17

upvoted 3 papers 3 months ago

OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework

Paper • 2404.14619 • Published Apr 22 • 124

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Paper • 2404.14219 • Published Apr 22 • 243

Learn Your Reference Model for Real Good Alignment

Paper • 2404.09656 • Published Apr 15 • 81

upvoted an article 3 months ago

Article

Mergoo: Efficiently Build Your Own MoE LLM

By

•

Jun 3

• 38

upvoted an article 4 months ago

Article

Mixture of Experts Explained

Dec 11, 2023

• 116

upvoted 5 papers 4 months ago

upvoted a collection 4 months ago

ORPO

Collection

This is the official collection of "ORPO: Monolithic Preference Optimization without Reference Model". • 5 items • Updated Apr 12 • 10

upvoted a paper 4 months ago

RAFT: Adapting Language Model to Domain Specific RAG

Paper • 2403.10131 • Published Mar 15 • 65

upvoted a paper 5 months ago

Yi: Open Foundation Models by 01.AI

Paper • 2403.04652 • Published Mar 7 • 60

upvoted a collection 5 months ago

Matryoshka Embedding Models

Collection

https://huggingface.co/blog/matryoshka • 14 items • Updated Jun 4 • 11

upvoted 6 papers 6 months ago

Self-Discover: Large Language Models Self-Compose Reasoning Structures

Paper • 2402.03620 • Published Feb 6 • 108

More Agents Is All You Need

Paper • 2402.05120 • Published Feb 3 • 47

PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models

Paper • 2402.01118 • Published Feb 2 • 28

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research

Paper • 2402.00159 • Published Jan 31 • 56

OLMo: Accelerating the Science of Language Models

Paper • 2402.00838 • Published Feb 1 • 75

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

Paper • 2401.16380 • Published Jan 29 • 46

upvoted a collection 6 months ago

MoEs papers reading list

Collection

52 items • Updated 12 days ago • 130

upvoted a paper 6 months ago

Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation

Paper • 2401.08417 • Published Jan 16 • 29

upvoted 4 papers 7 months ago

Mixtral of Experts

Paper • 2401.04088 • Published Jan 8 • 156

Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

Paper • 2401.01335 • Published Jan 2 • 62

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Paper • 2312.15166 • Published Dec 23, 2023 • 56

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Paper • 2312.00752 • Published Dec 1, 2023 • 134

upvoted 2 collections 7 months ago

Awesome feedback datasets

Collection

A curated list of datasets with human or AI feedback. Useful for training reward models or applying techniques like DPO. • 19 items • Updated Apr 12 • 60

Journal Club

Collection

Candidate papers to read in the H4 journal club • 54 items • Updated Apr 21 • 25

upvoted 2 papers 7 months ago

Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision

Paper • 2312.09390 • Published Dec 14, 2023 • 32

TinyGSM: achieving >80% on GSM8k with small language models

Paper • 2312.09241 • Published Dec 14, 2023 • 35

upvoted a collection 8 months ago

Awesome SFT datasets

Collection

A curated list of interesting datasets to fine-tune language models with. • 43 items • Updated Apr 12 • 104

upvoted a paper 8 months ago

GAIA: a benchmark for General AI Assistants

Paper • 2311.12983 • Published Nov 21, 2023 • 177

upvoted a collection 8 months ago

GAIA release

Collection

Gather the items of the GAIA release • 4 items • Updated Nov 23, 2023 • 19

upvoted 3 papers 8 months ago

Orca 2: Teaching Small Language Models How to Reason

Paper • 2311.11045 • Published Nov 18, 2023 • 69

Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2

Paper • 2311.10702 • Published Nov 17, 2023 • 18

Contrastive Chain-of-Thought Prompting

Paper • 2311.09277 • Published Nov 15, 2023 • 31

upvoted a paper 9 months ago

Zephyr: Direct Distillation of LM Alignment

Paper • 2310.16944 • Published Oct 25, 2023 • 120

upvoted 3 papers 10 months ago

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

Paper • 2310.08491 • Published Oct 12, 2023 • 52

DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models

Paper • 2309.03883 • Published Sep 7, 2023 • 32

Large Language Models Cannot Self-Correct Reasoning Yet

Paper • 2310.01798 • Published Oct 3, 2023 • 32

upvoted a paper 11 months ago

Textbooks Are All You Need II: phi-1.5 technical report

Paper • 2309.05463 • Published Sep 11, 2023 • 84

upvoted 2 papers 12 months ago

Self-Alignment with Instruction Backtranslation

Paper • 2308.06259 • Published Aug 11, 2023 • 39

Skeleton-of-Thought: Large Language Models Can Do Parallel Decoding

Paper • 2307.15337 • Published Jul 28, 2023 • 36

Hyunwoo Ko

AI & ML interests

Organizations

Cartinoe5930's activity

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

The Rise of Agentic Data Generation

🪆 Introduction to Matryoshka Embedding Models

Uncensor any LLM with abliteration

makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch

Expanding Model Context and Creating Chat Models with a Single Click

Can We Train Chat Models with Raw Data?

Mergoo: Efficiently Build Your Own MoE LLM

Mixture of Experts Explained