Instructions to use N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended")
model = AutoModelForCausalLM.from_pretrained("N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended

SGLang

How to use N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended with Docker Model Runner:
```
docker model run hf.co/N-Bot-Int/OpenElla-NovelWriter-Requiem-Ascended
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

📢 News / Changelog (click to collapse)

NEWS AND ANNOUNCEMENTS

(yes i write the announcement here, i figured people wont read on the main page so i'm announcing it here! - if you want to skip.. feel free to hide this!)

ANNOUNCEMENTS

New Models Will Be Delayed, Halted, or Canceled Until Further Notice
- Lightning.ai, one of the GPU hosts we use, has announced that they will no longer offer free credits starting the end of July. This means our lifeline for testing, dataset generation, and experimentation is OFFICIALLY CUT OFF until we find someone or a company to donate a cluster. NEW MODELS AIMED AT IMPROVEMENTS WILL BE DELAYED, CANCELED, OR PUSHED BACK as we try to do everything within Kaggle and Google Colab. PS: No, we are NOT rich in GPU... we're doing this for the community and our love of finetuning. This means future updates will no longer be "entirely" new architectures or models trained on new datasets — they'll probably be mixtures of our older AI models.
- We're running out of compute. Long story short: due to Lightning.ai removing their free tier, we're looking for alternatives to generate the datasets we need to train future AI models. We're also still trying to find our "signature model" — the one we can call entirely ours. You've seen it from Anthracite-org, where the moment you use their models you instantly know you've used Magnum because of its Opus-level length, or Stheno at RP, etc. We're currently trying to carve out that same kind of identity for ourselves, but it looks like that'll be delayed while we sort out our compute :(
We're Restructuring!
- N-Bot-Int has operated as a company for a while now — but, much like OpenAI, we're bleeding money... well, not exactly, since we've been running on free tiers. Still, that means calling ourselves a "company" no longer really serves a purpose, so N-Bot-Int is shifting from a company to a full-time laboratory! Expect more open-source releases from us! (Not models — as mentioned above, we don't have the compute for that :( )
Compliance
- It came to our attention that Meta took down the base model we used for OpenElla-Novelwriter v2. We're not usually the type of lab to overreact to a single incident, but watching that happen made something click — it wasn't just fear, it was a loss of trust in Meta and a need for real assurance going forward. That risk isn't hypothetical anymore; we've seen it happen firsthand. Given this, and given Llama 3.1's more restrictive licensing terms, we've decided that THIS WILL BE THE LAST LLAMA MODEL WE FINETUNE FOR THE FORESEEABLE FUTURE. Future models will explore QWEN or GEMMA instead — we're stepping away from Llama bases for now, both for our own peace of mind and for legal safety. THIS IS ALSO THE LAST TIME WE'LL USE AN ABLITERATED BASE, for safety reasons.
- Separately, we're also moving future model releases to AGPL-3.0 instead of Apache 2.0. This isn't about Meta or the takedown — it's about us not wanting companies to profit off our work without giving back to the community. This licensing format, actually started already, I just had to announce it incase other people aren't up to date
AI Refresh and Archival
- Most, if not all, of our Llama models (except recently updated ones) will soon be archived on an upcoming org that houses all of our archived models! Archived models are automatically set to Apache (or to the original base's license). This means OpenElla (our first model family), MiniMaid, and the rest will be archived soon!

THAT'S ALL THE NEWS WE HAVE FOR JULY!

We're sorry to say we'll no longer be releasing on a monthly schedule — running finetunes while keeping the lights on is NOT cheap, so we're no longer rushing for monthly releases until we find an alternative, sponsor, or cluster. We're currently in firefighting mode.

Support us!

Through Ko-fi: https://ko-fi.com/J3J61D8NHV If you have any questions, feel free to email us at: nexulon.botworkinteractives@gmail.com

OpenElla-NovelWriter-Requiem-Ascended is OFFICIALLY RELEASED!

Thanks Grok!

OpenElla-NovelWriter-Requiem-Ascended — quite possibly our best Llama model yet (yeah yeah, entirely subjective... let us have our OpenAI moment, just this once.)

Introducing OpenElla-NovelWriter Requiem Model! Requiem, based on the latin word (that links to our announcement), is our brand new ai model that showcases a new RP capabilities compared to our older generation of NovelWriters. Requiem Ascended is our first ai model that is finally sensitive to character cards, Has a unique blend of creativity freedom on all types of Roleplay, and Expanded capabilities trained on llama!
Compared to previous NovelWriters, Older novelwriters suffer from our blarbing sickness, where the ai model spat out the entire chat, without stopping and waiting for user's prompt, This final version of NovelWriter, named Requiem is the TYPE A of the family, A final type B is to be released to let YOU choose which of the final two are the best Requiem model!
Both Requiem Ascended and Requiem(TypeB) models are capable of RP, whilst also solving the Old NovelWriter family's issue of spatting out dialogues, and God modding Alongside it, the Requieum Ascended model has better stability when it comes to prompt following, better writing and dataset variety thanks to our DARE TIES strategy, of which we mixed all our ai models that learned from different varied dataset to form this Masterpiece!

READ MORE FOR MORE INFO 8 BILLIONS PARAMS MODEL

OpenElla-NovelWriter-Requiem-Ascended Procedure/Methodology:

OpenElla Requiem Ascended is our brand new ai model, it uses DARE TIES as a final methodology, however before forming the ai model, we trained the V3 of OpenElla-NovelWriter, taking the V2 and finetuning it with our new magnum synthetic dataset, this dataset contains a wide variety of data which we failed to properly balance, leading to V3 being biased to certain hemisphere of RP and developing a concerning bias to specific instances of "scenarios". This is why We failed to release V3 for final release which is named "OpenElla-NovelWriter-8B-Magnate"... Magnate were almost deleted however we decided to not waste the training credits we used(since this is the final time we'll get the credits from lightning.ai) and decided to combine them with all of the NovelWriter family. Below are the composition of each Requiem model:
- Requiem-Ascended: Trained with V3 + V2 + BASE, = V3.2 => V3.2 + V1 = RA
- Requiem: Trained with V3 + V2 + BASE = RE

The Base model we used, came from "akjindal53244/Llama-3.1-Storm-8B" as a stabilizer for this fusion, meanwhile both V3 and V2 were given a balanced weight and density to ensure the base model inherits the specialization of the two model and a final V1 is added for noise and enhancement. This is why we seperate the two family because we're not sure if some people would love the Ascended version or the Requiem, as both ai model has a different flavor of how they performs!

Feel free to decide which is the best for you!

Training Details

Finetuning Tool: MERGEKIT
Training Platform: Kaggle Free Tier with T4 x2!
OpenElla-NovelWriter-Requiem-Ascended is Our Brand New Powerful Model, If you ever encountered any issue, Want to commission us, or have any suggestions, please email us directly through nexulon.botworkinteractives@gmail.com we value any reports, suggestions to how we improve future Model, Once again feel free to finetune the model to your likings, However please consider Adding this Page for CREDITS
Please handle the AI with Care and ethical considerations, when FINETUNING this AI model, due to its UNCENSORED Nature.
We are not responsible for what this model generates. Use it responsibly and legally. You downloaded it, you own what you do with it.

What's Coming Next?

🔒 Requiem Base to be released soon! - Supporters on KO-FI receives the model early as usual!

Notices & Usage Tips

Use Llama 3 format — the model is based on Llama 3, Soooooo using Llama 3 works best!.
Calibrate per character card — every character is different, adjust your prompt, Model's settings(ie, temps, Top-K etc.) accordingly.
OUR SUGGESTIONS ESPECIALLY FOR KOBOLD USERS!
- Use DynaTemp for a better experience, drop the Min-Temp to 0.50, Max-temp to 1.50, on DRY set the MULT to 0.3 and Shift the Repeat Pen if you want to boost V3's influence......... somehow... dont ask why... but the ai model starts to act weird as you shift the repeat pen higher :|
- Format Llama 3 as mentioned above! sometimes the ai model includes [Author notes], it just depends on the character cards. Simply modify the character cards or include an author note to prevent or stop the ai from using the "[author notes] or OOC stuffs"
(Tho then again... we are bad at giving advices... just use whatever works to ya and comment what you've used so others can use the ai model xDDDDDD)

About

OpenElla-NovelWriter-Requiem-Ascended is
- Developed by: N-Bot-Int
- License: agpl 3.0
- Finetuned from model : N-Bot-Int/OpenElla-NovelWriter-merged, N-Bot-Int/OpenElla-NovelWriter-V2, N-Bot-Int/OpenElla-NovelWriter-Magnate, akjindal53244/Llama-3.1-Storm-8B
Detail card:
- Parameter
  - 8 Billion Parameters
  - (Please check your GPU Core, VRAM, CPU and RAM to see if you can comfortably run 8B models)
Finetuning tool:
- MERGEKIT
- Fine-tuned Using:
- Kaggle Free Tier with T4 x2