s
Tom-Neverwinter
AI & ML interests
Making improvements to help the world.
Recent Activity
reacted
to
csabakecskemeti's
post
with 🔥
about 2 months ago
I've built a small utility to split safetensors file by file.
The issue/need came up when I've tried to convert the new Deepseek V3 model from FP8 to BF16.
The only Ada architecture GPU I have is an RTX 4080 and the 16GB vram was just wasn't enough for the conversion.
BTW: I'll upload the bf16 version here:
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3-Base-bf16
(it will take a while - days with my upload speed)
If anyone has access the resources to test it I'd appreciate a feedback if it's working or not.
The tool, is available from here:
https://github.com/csabakecskemeti/ai_utils/blob/main/safetensor_splitter.py
It's splitting every file to n pieces by the layers if possible, and create a new "model.safetensors.index.json" file.
I've tested it with Llama 3.1 8B and multiple split sizes, and validated by using inference pipeline.
use `--help` for usage
Please note current version expects the model is already multiple file and have a "model.safetensors.index.json" layer-safetensor mapping file.
new activity
2 months ago
Apollo-LMMs/README:model pulled
Organizations
None yet
Tom-Neverwinter's activity
Engage the hype engines anther hermes has arrived!!!
#1 opened 3 days ago
by
Tom-Neverwinter
model pulled
#1 opened 2 months ago
by
Tom-Neverwinter
how to make a lora
3
#2 opened 6 months ago
by
guardiancc
Issues loading model with ooabooga textgenwebui
5
#20 opened 7 months ago
by
Kenji776
how do we run this?
2
#2 opened 7 months ago
by
Tom-Neverwinter
GGUF for the 236B model
3
#4 opened 7 months ago
by
amarmir
WizardLM-8x22B Evaluation failed
25
#823 opened 8 months ago
by
llama-anon
can we have some more details?
2
#1 opened 8 months ago
by
Tom-Neverwinter
How good is the gguf?
3
#3 opened 8 months ago
by
Tom-Neverwinter
censored
1
#1 opened 9 months ago
by
Tom-Neverwinter
ram usage
1
#1 opened 10 months ago
by
Tom-Neverwinter
resources to run
#3 opened 10 months ago
by
Tom-Neverwinter
3.0 bpw?
16
#1 opened 11 months ago
by
CulturedMan
multiple gpu?
3
#3 opened 11 months ago
by
bdambrosio
Missing config.json
8
#2 opened about 1 year ago
by
Cayleb
enter it into the leaderboard?
#6 opened 11 months ago
by
Tom-Neverwinter
safetensors?
1
#5 opened 11 months ago
by
Tom-Neverwinter
Dont download, google scuttled this model
16
#77 opened 11 months ago
by
Tom-Neverwinter
is this filtered(cencored) or not? can i run it on 3090?
3
#1 opened 11 months ago
by
Avos0001