Not-For-All-Audiences

Instructions to use Norquinal/OpenCAI-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Norquinal/OpenCAI-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Norquinal/OpenCAI-7B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Norquinal/OpenCAI-7B")
model = AutoModelForCausalLM.from_pretrained("Norquinal/OpenCAI-7B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Norquinal/OpenCAI-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Norquinal/OpenCAI-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Norquinal/OpenCAI-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Norquinal/OpenCAI-7B

SGLang

How to use Norquinal/OpenCAI-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Norquinal/OpenCAI-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Norquinal/OpenCAI-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Norquinal/OpenCAI-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Norquinal/OpenCAI-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Norquinal/OpenCAI-7B with Docker Model Runner:
```
docker model run hf.co/Norquinal/OpenCAI-7B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

OpenCAI

OpenCAI is a model fine-tuned from Mistral-7B and is an attempted open-source recreation of the style of roleplay found at C.AI. It was trained on 4800 samples of Discord roleplay interactions rather than C.AI outputs, as Discord is the likely origin of a majority of the material used to train C.AI's original model.

This model is primarily focused on chat and roleplay without any alignment. As such, it may output content that can be considered "unsafe" or "harmful." Please use the responsibly and to your best judgement.

Prompt Format

This model uses the Pygmalion-2/Metharme prompt format. The model has been trained on prompts using three different roles, which are denoted by the following tokens: <|system|>, <|user|> and <|model|>.

The <|system|> prompt can be used to inject out-of-channel information behind the scenes, while the <|user|> prompt should be used to indicate user input. The <|model|> token should then be used to indicate that the model should generate a response. These tokens can happen multiple times and be chained up to form a conversation history.

Example Prompt

<|system|>{system_prompt}
Characters:
   [char]: [description]
Summary: [summary of events]

Downloads last month: 9

Safetensors

Model size

7B params

Tensor type

F16

Model tree for Norquinal/OpenCAI-7B

Quantizations

5 models

Norquinal
/

OpenCAI-7B

OpenCAI

Prompt Format

Example Prompt

Model tree for Norquinal/OpenCAI-7B

Dataset used to train Norquinal/OpenCAI-7B