Exl2 version of Sao10K's Euryale-Inverted-L2-70B.
6.0 bpw version.
Model original card : https://huggingface.co/Sao10K/Euryale-Inverted-L2-70B
Tested with 16K context with alpha value, fits just below 60Gb of VRAM divided over 3 GPUs.
Exl2 version of Sao10K's Euryale-Inverted-L2-70B.
6.0 bpw version.
Model original card : https://huggingface.co/Sao10K/Euryale-Inverted-L2-70B
Tested with 16K context with alpha value, fits just below 60Gb of VRAM divided over 3 GPUs.