GOAT-70B-Storytelling model
GOAT-70B-Storytelling model trained by GOAT.AI lab as a core model for an autonomous story-writing agent.
GOAT-Storytelling-Agent
This agent facilitates the generation of high-quality, cohesive, and captivating narratives, including stories and books. It achieves this by utilizing inputs such as plot outlines, character profiles, their interrelationships, and other relevant details. Examples are provided below.
Model description
- Base Architecture: LLaMA 2 70B
- License: llama2
- Context window length: 4096 tokens
Training details
Training was performed on a GPU cluster of 64xH100s. FSDP ZeRO-3 sharding is employed for efficient training. We instruction finetune on a dataset of 18K examples for one epoch with batch size of 336, AdamW optimizer with learning rate 1e-5.
Learn more
- Blogpost: GOAT-Storytelling: Arbitrarily Long Story Writing Agent
- GitHub: here
- Generated examples: here
Uses
The main purpose of GOAT-70B-Storytelling is to generate books, novels, movie scripts and etc. as an agent in coping with our GOAT-Storytelling-Agent. It is specifically designed for storywriters.
Usage
Usage can be either self-hosted via transformers
or used with Spaces
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "GOAT-AI/GOAT-70B-Storytelling"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16
)
Currently, we support LLM endpoint generation, where you need to send a post request to the generation endpoint (we recommend using Text Generation Inference by HuggingFace).
Here is how you can utilize the model via GOAT-Storytelling-Agent:
from goat_storytelling_agent.storytelling_agent import StoryAgent
backend_uri = # Text generation endpoint
writer = StoryAgent(backend_uri, form='novel')
novel_scenes = writer.generate_story('treasure hunt in a jungle')
License
GOAT-70B-Storytelling model is based on Meta's LLaMA-2-70b-hf, and using own datasets.
GOAT-70B-Storytelling model weights are available under LLAMA-2 license.
Risks and Biases
GOAT-70B-Storytelling model can produce factually incorrect output and should not be relied on to deliver factually accurate information. Therefore, the GOAT-70B-Storytelling model could possibly generate wrong, biased, or otherwise offensive outputs.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 67.38 |
AI2 Reasoning Challenge (25-Shot) | 68.77 |
HellaSwag (10-Shot) | 87.74 |
MMLU (5-Shot) | 69.92 |
TruthfulQA (0-shot) | 53.53 |
Winogrande (5-shot) | 83.50 |
GSM8k (5-shot) | 40.79 |
- Downloads last month
- 741
Model tree for GOAT-AI/GOAT-70B-Storytelling
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard68.770
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard87.740
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard69.920
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard53.530
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard83.500
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard40.790