satyamt (Satyam)

upvoted a collection about 18 hours ago

sentence-transformers-from-synthetic-data

Example of using distilabel to generate synthetic triplets data for fine-tuning a Sentence Transformer model • 3 items • Updated 1 day ago • 15

upvoted a paper 5 days ago

Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

Paper • 2405.15071 • Published 9 days ago • 30

upvoted an article 19 days ago

Article

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 24

upvoted a paper 21 days ago

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Paper • 2405.04434 • Published 25 days ago • 10

upvoted 2 articles about 1 month ago

Article

seemore: Implement a Vision Language Model from Scratch

By

•

20 days ago

• 42

Article

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

Apr 19

• 70

upvoted a paper about 2 months ago

TransformerFAM: Feedback attention is working memory

Paper • 2404.09173 • Published Apr 14 • 42

upvoted a collection about 2 months ago

EasyContext

Collection

https://github.com/jzhang38/EasyContext • 7 items • Updated Apr 19 • 3

upvoted 2 papers about 2 months ago

Long-context LLMs Struggle with Long In-context Learning

Paper • 2404.02060 • Published Apr 2 • 33

Advancing LLM Reasoning Generalists with Preference Trees

Paper • 2404.02078 • Published Apr 2 • 41

upvoted a collection about 2 months ago

Eurus

Collection

Advancing LLM Reasoning Generalists with Preference Trees • 11 items • Updated Apr 15 • 22

upvoted 2 papers 2 months ago

Jamba: A Hybrid Transformer-Mamba Language Model

Paper • 2403.19887 • Published Mar 28 • 99

When Do We Not Need Larger Vision Models?

Paper • 2403.13043 • Published Mar 19 • 24

upvoted 8 papers 3 months ago

upvoted a collection 3 months ago

Caduceus

Collection

https://caduceus-dna.github.io/ • 8 items • Updated Apr 19 • 9

upvoted a paper 3 months ago

Common 7B Language Models Already Possess Strong Math Capabilities

Paper • 2403.04706 • Published Mar 7 • 16

upvoted 2 collections 3 months ago

UDOP

Collection

UDOP is a general multimodal model for document AI • 4 items • Updated 10 days ago • 20

OpenCodeInterpreter

Collection

18 items • Updated Mar 3 • 73

upvoted 3 papers 3 months ago

RoleCraft-GLM: Advancing Personalized Role-Playing in Large Language Models

Paper • 2401.09432 • Published Dec 17, 2023 • 2

RoleEval: A Bilingual Role Evaluation Benchmark for Large Language Models

Paper • 2312.16132 • Published Dec 26, 2023 • 2

Instruction-tuned Language Models are Better Knowledge Learners

Paper • 2402.12847 • Published Feb 20 • 25

upvoted a collection 3 months ago

DPO

Collection

5 items • Updated Dec 1, 2023 • 1

upvoted 2 papers 4 months ago

How to Train Data-Efficient LLMs

Paper • 2402.09668 • Published Feb 15 • 34

LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models

Paper • 2310.08659 • Published Oct 12, 2023 • 20

upvoted a collection 4 months ago

datasets-SPIN

Collection

Generated synthetic data used to finetune SPIN. • 8 items • Updated Feb 9 • 10

upvoted 5 papers 4 months ago

The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs

Paper • 2210.14986 • Published Oct 26, 2022 • 4

In-Context Principle Learning from Mistakes

Paper • 2402.05403 • Published Feb 8 • 12

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Paper • 2402.03300 • Published Feb 5 • 61

ReGAL: Refactoring Programs to Discover Generalizable Abstractions

Paper • 2401.16467 • Published Jan 29 • 7

Convergent Learning: Do different neural networks learn the same representations?

Paper • 1511.07543 • Published Nov 24, 2015 • 2

upvoted a collection 4 months ago

Medical Merges

Collection

Playful merges that try to improve small medical LMs by merging them with models with higher reasoning capabilities. • 35 items • Updated Mar 5 • 2

upvoted a paper 4 months ago

Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

Paper • 2306.02858 • Published Jun 5, 2023 • 14

upvoted a collection 4 months ago

AIM

Collection

AIM: Autoregressive Image Models • 5 items • Updated Jan 29 • 43

upvoted a paper 4 months ago

ReFT: Reasoning with Reinforced Fine-Tuning

Paper • 2401.08967 • Published Jan 17 • 27

upvoted a paper 5 months ago

Adapting Large Language Models via Reading Comprehension

Paper • 2309.09530 • Published Sep 18, 2023 • 70

upvoted 4 collections 5 months ago

Preference Datasets for DPO

Collection

This collection contains a list of curated preference datasets for DPO fine-tuning for intent alignment of LLMs • 7 items • Updated Apr 4 • 20

Hermes

Collection

Nous' Flagship LLM Series • 21 items • Updated 17 days ago • 87

Comparing DPO with IPO and KTO

Collection

A collection of chat models to explore the differences between three alignment techniques: DPO, IPO, and KTO. • 56 items • Updated Jan 9 • 31

Model Merging

Collection

Model Merging is a very popular technique nowadays in LLM. Here is a chronological list of papers on the space that will help you get started with it! • 29 items • Updated 1 day ago • 181

upvoted 2 papers 5 months ago

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws

Paper • 2401.00448 • Published Dec 31, 2023 • 25

Generative AI for Math: Part I -- MathPile: A Billion-Token-Scale Pretraining Corpus for Math

Paper • 2312.17120 • Published Dec 28, 2023 • 24

upvoted 2 papers 6 months ago

Vision-Language Models as a Source of Rewards

Paper • 2312.09187 • Published Dec 14, 2023 • 10

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

Paper • 2312.06585 • Published Dec 11, 2023 • 26

upvoted a collection 6 months ago

Awesome feedback datasets

Collection

A curated list of datasets with human or AI feedback. Useful for training reward models or applying techniques like DPO. • 19 items • Updated Apr 12 • 53

upvoted 8 papers 6 months ago

Orca 2: Teaching Small Language Models How to Reason

Paper • 2311.11045 • Published Nov 18, 2023 • 69

LLM360: Towards Fully Transparent Open-Source LLMs

Paper • 2312.06550 • Published Dec 11, 2023 • 52

ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge

Paper • 2303.14070 • Published Mar 24, 2023 • 9

The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning

Paper • 2312.01552 • Published Dec 4, 2023 • 26

Mistral 7B

Paper • 2310.06825 • Published Oct 10, 2023 • 43

GAIA: a benchmark for General AI Assistants

Paper • 2311.12983 • Published Nov 21, 2023 • 172

MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records

Paper • 2308.14089 • Published Aug 27, 2023 • 24

Exponentially Faster Language Modelling

Paper • 2311.10770 • Published Nov 15, 2023 • 117

upvoted a collection 6 months ago

Medical QA Datasets

Collection

A collection of medical question answering (QA) datasets • 19 items • Updated Oct 31, 2023 • 16

Satyam

AI & ML interests

Organizations

satyamt's activity

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

seemore: Implement a Vision Language Model from Scratch

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare