7 6

Vadim Karpenko

jrell

AI & ML interests

None yet

Recent Activity

New activity 14 days ago

ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3-GGUF:32k context bug

liked a model 14 days ago

PramaLLC/BEN

New activity about 1 month ago

mradermacher/model_requests:LoLCATs models.

View all activity

Organizations

None yet

jrell's activity

New activity in ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3-GGUF 14 days ago

32k context bug

#1 opened 14 days ago by

jrell

liked a model 14 days ago

PramaLLC/BEN

Image Segmentation • Updated 6 days ago • 166 • 70

New activity in mradermacher/model_requests about 1 month ago

LoLCATs models.

#369 opened about 1 month ago by

jrell

New activity in ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2 about 2 months ago

LM Studio produces gibberish (GGUF)

#1 opened about 2 months ago by

jrell

liked a model 2 months ago

mistralai/Mistral-Small-Instruct-2409

Updated Oct 16 • 12.3k • 351

New activity in QuantFactory/DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored-GGUF 4 months ago

llama.cpp error: 'done_getting_tensors: wrong number of tensors; expected 292, got 291'

#1 opened 4 months ago by

jrell

New activity in cognitivecomputations/dolphin-2.9.3-mistral-nemo-12b-gguf 4 months ago

"llama.cpp error: 'error loading model vocabulary: unknown pre-tokenizer type: 'dolphin12b''"

#1 opened 4 months ago by

jrell

Reacted to singhsidhukuldeep's post with ❤️ 7 months ago

Post

1456

You are all happy 😊 that @meta-llama released Llama 3.

Then you are sad 😔 that it only has a context length of 8k.

Then you are happy 😄 that you can just scale llama-3 PoSE to 96k without training, only needing to modify max_position_embeddings and rope_theta.

But then you are sad 😢 it only improves the model's long-context retrieval performance (i.e., finding needles) while hardly improving its long-context utilization capability (doing QA and summarization).

But then you are happy 😁 that the
@GradientsTechnologies community has released the long-context Llama-3-8B-Instruct-262K with long context (262k-1M+).

Now we have another paper "Extending Llama-3's Context Ten-Fold Overnight" 📜.

The context length of Llama-3-8B-Instruct is extended from 8K to 80K using QLoRA fine-tuning⚙️.

The training cycle is highly efficient, taking "only" 😂 8 hours on a single 8xA800 (80G) GPU machine.

The model also preserves its original capability over short contexts. ✁

The dramatic context extension is mainly attributed to merely 3.5K synthetic training samples generated by GPT-4.📊

The paper suggests that the context length could be extended far beyond 80K with more computation resources (😅 GPU-poor).

The team plans to publicly release all resources, including data, model, data generation pipeline, and training code, to facilitate future research from the ❤️ community.

Paper: https://arxiv.org/abs/2404.19553

This is where we are... until next time... 🌟

Extending Llama-3's Context Ten-Fold Overnight (2404.19553)