CTranslate2 int8 version of L3-8B-Stheno-v3.1

This is a int8_bfloat16 quantization of L3-8B-Stheno-v3.1
See more on CTranslate2: Docs | Github

This model was converted to ct2 format using the following commnd:

ct2-transformers-converter --model Sao10K/L3-8B-Stheno-v3.1 --output_dir L3-8B-Stheno-v3.1-ct2 --quantization int8_bfloat16 --low_cpu_mem_usage

no converstion needed using the model from this repository as it is already in ct2 format.

Inference Examples

Inference API (serverless) does not yet support CTranslate2 models for this pipeline type.

Model tree for Anthonyg5005/L3-8B-Stheno-v3.1-int8-ct2

Base model

Finetuned

(2)

this model