README.md

---
license: apache-2.0
---
# Yi-Coder-9B-Chat GGUF Model

This repository contains the GGUF version of the Yi-Coder-9B-Chat model. This quantized version allows for efficient inference on a wide range of hardware.

## Model Description

Yi-Coder-9B-Chat is a powerful language model specifically designed for coding tasks and chat interactions. This GGUF version maintains the model's capabilities while offering improved performance and reduced resource requirements.

## Usage

You can use this model with popular inference frameworks such as [llama.cpp](https://github.com/ggerganov/llama.cpp) or [Ollama](https://github.com/jmorganca/ollama). Below are instructions for using the model with llama.cpp.

### Using with llama.cpp

1. Clone and build llama.cpp following their [installation instructions](https://github.com/ggerganov/llama.cpp#build).

2. Download the GGUF model file from this repository.

3. Run the model using the following command:

```bash
cd llama.cpp/build_cuda/bin
./llama-cli -m /path/to/Yi-Coder-9B-Chat-BF16.gguf -n -1 --color -r "User:" --in-prefix " " -i -e -p "User: Hi"
```

Replace `/path/to/Yi-Coder-9B-Chat-BF16.gguf` with the actual path to the downloaded model file.

This command starts an interactive session with the model. You can type your prompts after the "User:" prefix.

### Using with Ollama

Ollama provides an easy-to-use interface for running large language models. Follow the [Ollama documentation](https://github.com/jmorganca/ollama#getting-started) to set up and use this model.

## License

This model is released under the Apache 2.0 License. Please see the LICENSE file in this repository for full details.

## Acknowledgements

This model is based on the Yi-Coder-9B-Chat model. We acknowledge the original creators and contributors of the base model.

## Issues and Contributions

If you encounter any issues or have suggestions for improvements, please open an issue in this repository. Contributions are welcome!

---

Happy coding with Yi-Coder-9B-Chat GGUF!