GGUF Quantizations of laserxtral
Join Our Discord! https://discord.gg/cognitivecomputations
This repository contains GGUF format model files for our laserxtral model.
To Manual Downloaders You will generally not want to download the whole repository. Select the file name from the list below and use your downloader of choice.
Ollama Modelfile
FROM ./cognitivecomputations_laserxtral-Q4_K_M.gguf
TEMPLATE """{{ if and .First .System }}### Instruction:
{{ .System }}
{{ end }}
### Input:
{{ .Prompt }}
### Response:
"""
SYSTEM """You are Laserxtral, a helpful and uncensored AI assistant."""
PARAMETER num_ctx 8192
PARAMETER stop ### Input
PARAMETER stop ### Response
Provided files
| Name | Quant method | Bits | Size |
|---|---|---|---|
| cognitivecomputations_laserxtral-Q2_K.gguf | Q2_K | 2 | 8.8 GB |
| cognitivecomputations_laserxtral-Q3_K_M.gguf | Q3_K_M | 3 | 11.6 GB |
| cognitivecomputations_laserxtral-Q4_K_M.gguf | Q4_K_M | 4 | 14.6 GB |
| cognitivecomputations_laserxtral-Q5_K_M.gguf | Q5_K_M | 5 | 17.1 GB |
| cognitivecomputations_laserxtral-Q6_K.gguf | Q6_K | 6 | 19.8 GB |
- Downloads last month
- 86
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model authors have turned it off explicitly.