Concedo's picture

39 13

Concedo

concedo

·

LostRuins

AI & ML interests

Hi, I'm Concedo, also known as LostRuins. I'm the dev of KoboldCpp.

Recent Activity

replied to bartowski's post 3 days ago

Old mixtral model quants may be broken! Recently Slaren over on llama.cpp refactored the model loader - in a way that's super awesome and very powerful - but with it came breaking of support for "split tensor MoE models", which applies to older mixtral models You may have seen my upload of one such older mixtral model, ondurbin/bagel-dpo-8x7b-v0.2, and with the newest changes it seems to be able to run without issue If you happen to run into issues with any other old mixtral models, drop a link here and I'll try to remake them with the new changes so that we can continue enjoying them :)

replied to bartowski's post 3 days ago

Looks like Q4_0_N_M file types are going away Before you panic, there's a new "preferred" method which is online (I prefer the term on-the-fly) repacking, so if you download Q4_0 and your setup can benefit from repacking the weights into interleaved rows (what Q4_0_4_4 was doing), it will do that automatically and give you similar performance (minor losses I think due to using intrinsics instead of assembly, but intrinsics are more maintainable) You can see the reference PR here: https://github.com/ggerganov/llama.cpp/pull/10446 So if you update your llama.cpp past that point, you won't be able to run Q4_0_4_4 (unless they add backwards compatibility back), but Q4_0 should be the same speeds (though it may currently be bugged on some platforms) As such, I'll stop making those newer model formats soon, probably end of this week unless something changes, but you should be safe to download and Q4_0 quants and use those ! Also IQ4_NL supports repacking though not in as many shapes yet, but should get a respectable speed up on ARM chips, PR for that can be found here: https://github.com/ggerganov/llama.cpp/pull/10541 Remember, these are not meant for Apple silicon since those use the GPU and don't benefit from the repacking of weights

updated a model 4 days ago

koboldcpp/mmproj

View all activity

Organizations

concedo's activity

New activity in concedo/Beepo-22B 8 days ago

Can you increase the context length?

#3 opened 9 days ago by

New activity in bartowski/Beepo-22B-GGUF 25 days ago

Prompt format correction

#1 opened 26 days ago by

New activity in Gryphe/ChatGPT-4o-Writing-Prompts about 1 month ago

Slight corruption in dataset

#1 opened about 1 month ago by

New activity in DontPlanToEnd/UGI-Leaderboard 4 months ago

Eval request - mini magnum unboxed

#23 opened 4 months ago by

New activity in concedo/Mini-Magnum-Unboxed-12B 4 months ago

Is there a plan for a magnum V2 version of this model?

#1 opened 4 months ago by

SaisExperiments

New activity in concedo/Mini-Magnum-Unboxed-12B-GGUF 4 months ago

Upload folder using huggingface_hub

#1 opened 4 months ago by

New activity in concedo/Mini-Magnum-Unboxed-12B 4 months ago

Upload folder using huggingface_hub

#3 opened 4 months ago by

[WIP] Upload folder using huggingface_hub (multi-commit ac840cbd2fa2506a8672ed415d6d4f8913465bff5e1a3567c66ab06c6974e87f)

#2 opened 4 months ago by

New activity in concedo/KobbleSmall-2B 5 months ago

Kobo

#1 opened 5 months ago by

New activity in koboldcpp/kcppbins 5 months ago

[bug] KoboldCpp 1.71.1 wrong detection of quants not only llama type and cause quality loss

#1 opened 5 months ago by

New activity in TheDrummer/Tiger-Gemma-9B-v1 5 months ago

The tiger models tend to go crazy on LM Studio ?

#4 opened 5 months ago by

New activity in bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF 5 months ago

K quants should not contain IQ4_NL types inside

#4 opened 5 months ago by

New activity in concedo/WebTokenizer 7 months ago

Add more models

#1 opened 7 months ago by

New activity in TheDrummer/Moistral-11B-v3 7 months ago

🏆.Best NSFW 11B model ever made.💦💦💦💦

#4 opened 8 months ago by

New activity in Orenguteng/Llama-3-8B-Lexi-Uncensored 8 months ago

Re-quant?

#5 opened 8 months ago by

New activity in hjhj3168/Llama-3-8b-Orthogonalized-exl2 8 months ago

🚩 Report: Legal issue(s)

#2 opened 8 months ago by

New activity in Orenguteng/Llama-3-8B-Lexi-Uncensored-GGUF 8 months ago

Reconverted and requantized with latest GGUF to fix llama3 tokenizer

#5 opened 8 months ago by

New activity in mradermacher/Llama-3-SnowyRP-8B-V1-B-GGUF 8 months ago

May need reconversion

#1 opened 8 months ago by

New activity in concedo/KobbleTinyV2-1.1B 8 months ago

Upload 7 files

#1 opened 8 months ago by

New activity in Orenguteng/Llama-3-8B-LexiFun-Uncensored-V1-GGUF 8 months ago

May require reconversion due to llama.cpp enhancements

#1 opened 8 months ago by