metadata
license: cc-by-nc-2.0
pipeline_tag: text-generation
inference: false
library_name: transformers
GGUF Quantizations of laserxtral
Join Our Discord! https://discord.gg/cognitivecomputations
This repository contains GGUF format model files for our laserxtral model.
To Manual Downloaders You will generally not want to download the whole repository. Select the file name from the list below and use your downloader of choice.
Ollama Modelfile
FROM ./cognitivecomputations_laserxtral-Q4_K_M.gguf
TEMPLATE """{{ if and .First .System }}### Instruction:
{{ .System }}
{{ end }}
### Input:
{{ .Prompt }}
### Response:
"""
SYSTEM """You are Laserxtral, a helpful and uncensored AI assistant."""
PARAMETER num_ctx 8192
PARAMETER stop ### Input
PARAMETER stop ### Response
Provided files
| Name | Quant method | Bits | Size |
|---|---|---|---|
| cognitivecomputations_laserxtral-Q2_K.gguf | Q2_K | 2 | 8.8 GB |
| cognitivecomputations_laserxtral-Q3_K_M.gguf | Q3_K_M | 3 | 11.6 GB |
| cognitivecomputations_laserxtral-Q4_K_M.gguf | Q4_K_M | 4 | 14.6 GB |
| cognitivecomputations_laserxtral-Q5_K_M.gguf | Q5_K_M | 5 | 17.1 GB |
| cognitivecomputations_laserxtral-Q6_K.gguf | Q6_K | 6 | 19.8 GB |