leonardlin (lhl)

upvoted a collection about 22 hours ago

Dataset generation

Collection

118 items • Updated 1 day ago • 17

upvoted 3 papers about 22 hours ago

upvoted 4 articles about 22 hours ago

Article

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

By

•

26 days ago

• 55

Article

Many-shot jailbreaking

By

•

Apr 2

• 4

Article

Unleashing the Power of Logprobs in Language Models: A Practical Guide

By

•

Jan 12

• 1

Article

💨 Introducing Notus: a DPO fine-tune of Zephyr with a focus on high-quality data

By

•

Dec 1, 2023

• 1

upvoted 2 articles about 23 hours ago

Article

Training MoE on AWS Trainium

By

•

1 day ago

• 1

Article

LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!)

By

•

28 days ago

• 45

upvoted a paper 2 days ago

A Survey on Large Language Models with Multilingualism: Recent Advances and New Frontiers

Paper • 2405.10936 • Published 5 days ago • 1

upvoted 2 papers 3 days ago

Instruction Tuning with Human Curriculum

Paper • 2310.09518 • Published Oct 14, 2023 • 3

Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities

Paper • 2404.17790 • Published 25 days ago • 2

upvoted 2 papers 7 days ago

Zero-Shot Tokenizer Transfer

Paper • 2405.07883 • Published 9 days ago • 3

Measuring Taiwanese Mandarin Language Understanding

Paper • 2403.20180 • Published Mar 29 • 3

upvoted a paper 21 days ago

Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models

Paper • 2404.18796 • Published 23 days ago • 63

upvoted a paper 25 days ago

Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks

Paper • 2404.14723 • Published 29 days ago • 9

upvoted 16 papers about 1 month ago

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

Paper • 2402.19427 • Published Feb 29 • 49

CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models

Paper • 2404.08763 • Published Apr 12 • 1

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Paper • 2404.08801 • Published Apr 12 • 62

Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment

Paper • 2404.12318 • Published Apr 18 • 14

Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models

Paper • 2404.12387 • Published Apr 18 • 35

Compression Represents Intelligence Linearly

Paper • 2404.09937 • Published Apr 15 • 27

Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning

Paper • 2301.09626 • Published Jan 23, 2023 • 2

InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory

Paper • 2402.04617 • Published Feb 7 • 4

Jamba: A Hybrid Transformer-Mamba Language Model

Paper • 2403.19887 • Published Mar 28 • 99

Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence

Paper • 2404.05892 • Published Apr 8 • 28

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

Paper • 2404.07839 • Published Apr 11 • 39

LLoCO: Learning Long Contexts Offline

Paper • 2404.07979 • Published Apr 11 • 15

Fast Vocabulary Transfer for Language Model Compression

Paper • 2402.09977 • Published Feb 15 • 2

Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models

Paper • 2403.00417 • Published Mar 1 • 1

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Paper • 2012.15613 • Published Dec 31, 2020 • 1

Getting the most out of your tokenizer for pre-training and domain adaptation

Paper • 2402.01035 • Published Feb 1 • 1

upvoted 6 papers about 2 months ago

Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models

Paper • 2404.02575 • Published Apr 3 • 46

InternLM2 Technical Report

Paper • 2403.17297 • Published Mar 26 • 25

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning

Paper • 2403.17919 • Published Mar 26 • 15

Orca-Math: Unlocking the potential of SLMs in Grade School Math

Paper • 2402.14830 • Published Feb 16 • 23

Polaris: A Safety-focused LLM Constellation Architecture for Healthcare

Paper • 2403.13313 • Published Mar 20 • 1

On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial

Paper • 2403.14380 • Published Mar 21 • 1

upvoted 8 papers 2 months ago

Evolutionary Optimization of Model Merging Recipes

Paper • 2403.13187 • Published Mar 19 • 45

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

Paper • 2403.13372 • Published Mar 20 • 54

Simple and Scalable Strategies to Continually Pre-train Large Language Models

Paper • 2403.08763 • Published Mar 13 • 48

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

Paper • 2403.07974 • Published Mar 12 • 1

Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

Paper • 2403.09629 • Published Mar 14 • 54

MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training

Paper • 2403.09611 • Published Mar 14 • 119

AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation

Paper • 2312.13010 • Published Dec 20, 2023 • 4

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

Paper • 2403.04132 • Published Mar 7 • 38

upvoted 13 papers 3 months ago

OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement

Paper • 2402.14658 • Published Feb 22 • 77

Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs

Paper • 2312.05934 • Published Dec 10, 2023 • 1

A Survey on Data Selection for Language Models

Paper • 2402.16827 • Published Feb 26 • 3

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap

Paper • 2402.19450 • Published Feb 29 • 3

Ring Attention with Blockwise Transformers for Near-Infinite Context

Paper • 2310.01889 • Published Oct 3, 2023 • 8

Striped Attention: Faster Ring Attention for Causal Transformers

Paper • 2311.09431 • Published Nov 15, 2023 • 4

System 2 Attention (is something you might need too)

Paper • 2311.11829 • Published Nov 20, 2023 • 38

FuseChat: Knowledge Fusion of Chat Models

Paper • 2402.16107 • Published Feb 25 • 35

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

Paper • 2312.17673 • Published Dec 29, 2023 • 1

An Early Categorization of Prompt Injection Attacks on Large Language Models

Paper • 2402.00898 • Published Jan 31 • 2

Repetition Improves Language Model Embeddings

Paper • 2402.15449 • Published Feb 23 • 2

Divide-or-Conquer? Which Part Should You Distill Your LLM?

Paper • 2402.15000 • Published Feb 22 • 22

tinyBenchmarks: evaluating LLMs with fewer examples

Paper • 2402.14992 • Published Feb 22 • 11

lhl PRO

AI & ML interests

Articles

Not Legal Advice on AI Training Data in Japan

Evaling llm-jp-eval (evals are hard)

Organizations

leonardlin's activity

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

Many-shot jailbreaking

Unleashing the Power of Logprobs in Language Models: A Practical Guide

💨 Introducing Notus: a DPO fine-tune of Zephyr with a focus on high-quality data

Training MoE on AWS Trainium

LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!)