yhavinga/Boreas-7B-chat-v1-GGUF

Model Summary

See Boreas-7B-chat for more info. This repository contains several quantized models and their associated model cards for ollama. Click on 'files' to get a sense of the file sizes.

You can download a single GGUF file with the Huggingface Web UI widgets for quantized files. The Modelcard for ollama can be downloaded from the 'files' section.

Then:

ollama create Boreas-7B-chat-q6_k -f Modelfile_Q6_K
ollama run Boreas-7B-chat-q6_k

Enjoy!