Rai PRO

sr-rai

AI & ML interests

None yet

Recent Activity

reacted to Yehor's post with 🤗 1 day ago

Create spectrogram using Rust! Slightly improved nice project that creates spectrogram and built binaries for different platform using cross-rs I've mentioned earlier in my channel. Repo: https://github.com/crs-org/sonogram

posted an update 1 day ago

ExLlamaV3 is out. And it introduces EXL3 - a new SOTA quantization format! "The conversion process is designed to be simple and efficient and requires only an input model (in HF format) and a target bitrate. By computing Hessians on the fly and thanks to a fused Viterbi kernel, the quantizer can convert a model in a single step, taking a couple of minutes for smaller models, up to a few hours for larger ones (70B+) (on a single RTX 4090 or equivalent GPU.)" Repo: https://github.com/turboderp-org/exllamav3

View all activity

Organizations

Posts 1

Post

1674

ExLlamaV3 is out. And it introduces EXL3 - a new SOTA quantization format!

"The conversion process is designed to be simple and efficient and requires only an input model (in HF format) and a target bitrate. By computing Hessians on the fly and thanks to a fused Viterbi kernel, the quantizer can convert a model in a single step, taking a couple of minutes for smaller models, up to a few hours for larger ones (70B+) (on a single RTX 4090 or equivalent GPU.)"

Repo: https://github.com/turboderp-org/exllamav3

models

None public yet

datasets

None public yet