pszemraj (Peter Szemraj)

upvoted an article 4 days ago

Article

Improving Hugging Face Training Efficiency Through Packing with Flash Attention

11 days ago

• 13

upvoted an article 12 days ago

Article

∞🧙🏼‍♂️AnyClassifier - Generating Synthetic Data For Text Classification

By

•

12 days ago

• 6

upvoted an article 22 days ago

Article

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

By

•

27 days ago

• 24

upvoted a collection 25 days ago

📈 Scaling Laws with Vocabulary

Collection

Increase your vocabulary size when you scale up your language model • 5 items • Updated 21 days ago • 3

upvoted a paper 29 days ago

Gemma 2: Improving Open Language Models at a Practical Size

Paper • 2408.00118 • Published Jul 31 • 72

upvoted 2 papers about 1 month ago

Spectra: A Comprehensive Study of Ternary, Quantized, and FP16 Language Models

Paper • 2407.12327 • Published Jul 17 • 75

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

Paper • 2407.13623 • Published Jul 18 • 52

upvoted a collection about 2 months ago

🪐 SmolLM

Collection

A series of smol LLMs: 135M, 360M and 1.7B. We release base and Instruct models as well as the training corpus and some WebGPU demos • 12 items • Updated 14 days ago • 160

upvoted a paper about 2 months ago

SpreadsheetLLM: Encoding Spreadsheets for Large Language Models

Paper • 2407.09025 • Published Jul 12 • 120

upvoted a collection about 2 months ago

InternLM2.5

Collection

13 items • Updated 26 days ago • 66

upvoted an article 2 months ago

Article

Tokenization Is A Dead Weight

By

•

Jun 27

• 16

upvoted 2 collections 3 months ago

Nemotron 4 340B

Collection

Nemotron-4: open models for Synthetic Data Generation (SDG). Includes Base, Instruct, and Reward models. • 4 items • Updated Jul 17 • 154

MatMulfree LM

Collection

Pre-trined models for Matmulfree LM. • 4 items • Updated Jun 10 • 22

upvoted 7 papers 3 months ago

Zamba: A Compact 7B SSM Hybrid Model

Paper • 2405.16712 • Published May 26 • 20

Not All Language Model Features Are Linear

Paper • 2405.14860 • Published May 23 • 39

Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3

Paper • 2405.00664 • Published May 1 • 18

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

Paper • 2405.01535 • Published May 2 • 113

What matters when building vision-language models?

Paper • 2405.02246 • Published May 3 • 97

Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory

Paper • 2405.08707 • Published May 14 • 27

LoRA Learns Less and Forgets Less

Paper • 2405.09673 • Published May 15 • 86

upvoted 3 collections 4 months ago

upvoted 2 collections 5 months ago

Meta Llama 3

Collection

This collection hosts the transformers and original repos of the Meta Llama 3 and Llama Guard 2 releases • 5 items • Updated 29 days ago • 668

RecurrentGemma Release

Collection

8 items • Updated Jul 31 • 39

upvoted 3 papers 7 months ago

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

Paper • 2402.04615 • Published Feb 7 • 36

More Agents Is All You Need

Paper • 2402.05120 • Published Feb 3 • 51

Grandmaster-Level Chess Without Search

Paper • 2402.04494 • Published Feb 7 • 65

upvoted 2 collections 7 months ago

Qwen1.5

Collection

Qwen1.5 is the improved version of Qwen, the large language model series developed by Alibaba Cloud. • 55 items • Updated Jun 6 • 206

Zeroshot Classifiers

Collection

These are my current best zeroshot classifiers. Some of my older models are downloaded more often, but the models in this collection are newer/better. • 11 items • Updated Apr 3 • 95

upvoted 2 papers 8 months ago

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

Paper • 2401.01325 • Published Jan 2 • 26

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Paper • 2312.15166 • Published Dec 23, 2023 • 56

upvoted a collection 9 months ago

Nougat ONNX

Collection

Faster Nougat in ONNX format (optimum onnxruntime) • 6 items • Updated Feb 24 • 1

upvoted a paper 10 months ago

Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure

Paper • 2311.07590 • Published Nov 9, 2023 • 16

upvoted a collection 10 months ago

smol llama

Collection

🚧"raw" pretrained smol_llama checkpoints - WIP 🚧 • 4 items • Updated Apr 29 • 6

Peter Szemraj PRO

AI & ML interests

Organizations

pszemraj's activity

Improving Hugging Face Training Efficiency Through Packing with Flash Attention

∞🧙🏼‍♂️AnyClassifier - Generating Synthetic Data For Text Classification

The case for specialized pre-training: ultra-fast foundation models for dedicated tasks

📈 Scaling Laws with Vocabulary

Gemma 2: Improving Open Language Models at a Practical Size

Spectra: A Comprehensive Study of Ternary, Quantized, and FP16 Language Models

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

🪐 SmolLM

SpreadsheetLLM: Encoding Spreadsheets for Large Language Models

InternLM2.5

Tokenization Is A Dead Weight

Nemotron 4 340B

MatMulfree LM

Zamba: A Compact 7B SSM Hybrid Model

Not All Language Model Features Are Linear

Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

What matters when building vision-language models?

Beyond Scaling Laws: Understanding Transformer Performance with Associative Memory

LoRA Learns Less and Forgets Less

Flan-T5 release

Bee Models 🍯

Arctic-embed

Meta Llama 3

RecurrentGemma Release

ScreenAI: A Vision-Language Model for UI and Infographics Understanding

More Agents Is All You Need

Grandmaster-Level Chess Without Search

Qwen1.5

Zeroshot Classifiers

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

Nougat ONNX

Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure

smol llama