LoneStriker

/

Mixtral_34Bx2_MoE_60B-6.0bpw-h6-exl2

like 1

Agreed that the model is really good. Many of the newer 2x34B merges have been surprisingly good. I can add 8.0 to the list of quants, but very few people have more than 48 GB VRAM to load them.

Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).

8.0bpw quant up:
https://huggingface.co/LoneStriker/Mixtral_34Bx2_MoE_60B-8.0bpw-h8-exl2

Have not heard of modded 2080 Ti cards, interesting. Used 3090s here are around $700, so I have some of those instead for more VRAM.

Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).

where and how to buy these cheap hard-modded 2080Ti ?
thanks

Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).

where and how to buy these cheap hard-modded 2080Ti ?
thanks

I live in China, so I got them on 闲鱼(that's taobao for used items, just search 2080ti 22G, it's easy to find ones around 2400 CNY).

Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).

where and how to buy these cheap hard-modded 2080Ti ?
thanks

I live in China, so I got them on 闲鱼(that's taobao for used items, just search 2080ti 22G, it's easy to find ones around 2400 CNY).

我在pdd看到了，私聊qq 206887187，请教一下怎么配置