Hugging Face's logo
James Wright's picture
25 5 79

James Wright PRO

jsfs11
Clevyby's profile picture Ttimofeyka's profile picture Suparious's profile picture
·

AI & ML interests

ML engineering, SOTA techniques, LLM fine-tuning and merging

Recent Activity

reacted to davidberenstein1957's post with 👀 3 days ago
Introducing the Synthetic Data Generator, a user-friendly application that takes a no-code approach to creating custom datasets with Large Language Models (LLMs). The best part: A simple step-by-step process, making dataset creation a non-technical breeze, allowing anyone to create datasets and models in minutes and without any code. Blog: https://huggingface.co/blog/synthetic-data-generator Space: https://huggingface.co/spaces/argilla/synthetic-data-generator
liked a Space 3 days ago
JeffreyXiang/TRELLIS
upvoted a collection 5 days ago
Falcon3
View all activity

Organizations

None yet

jsfs11's activity

reacted to davidberenstein1957's post with 👀 3 days ago
view post
Post
4058
Introducing the Synthetic Data Generator, a user-friendly application that takes a no-code approach to creating custom datasets with Large Language Models (LLMs). The best part: A simple step-by-step process, making dataset creation a non-technical breeze, allowing anyone to create datasets and models in minutes and without any code.

Blog: https://huggingface.co/blog/synthetic-data-generator
Space: argilla/synthetic-data-generator
·
liked a Space 3 days ago
upvoted a collection 5 days ago
liked a model 14 days ago
updated a model 16 days ago
updated 2 models 17 days ago
liked a Space about 1 month ago
upvoted a paper about 2 months ago
liked a Space 3 months ago
liked a model 3 months ago
liked a Space 3 months ago
updated 4 models 4 months ago
upvoted a collection 4 months ago
reacted to bartowski's post with 👍 4 months ago
view post
Post
10044
So turns out I've been spreading a bit of misinformation when it comes to imatrix in llama.cpp

It starts true; imatrix runs the model against a corpus of text and tracks the activation of weights to determine which are most important

However what the quantization then does with that information is where I was wrong.

I think I made the accidental connection between imatrix and exllamav2's measuring, where ExLlamaV2 decides how many bits to assign to which weight depending on the goal BPW

Instead, what llama.cpp with imatrix does is it attempts to select a scale for a quantization block that most accurately returns the important weights to their original values, ie minimizing the dequantization error based on the importance of activations

The mildly surprising part is that it actually just does a relatively brute force search, it picks a bunch of scales and tries each and sees which one results in the minimum error for weights deemed important in the group

But yeah, turns out, the quantization scheme is always the same, it's just that the scaling has a bit more logic to it when you use imatrix

Huge shoutout to @compilade for helping me wrap my head around it - feel free to add/correct as well if I've messed something up
·
liked a Space 4 months ago
liked a model 4 months ago