---
license: apache-2.0
base_model: Qwen/Qwen2-1.5B
metrics:
- accuracy
datasets:
- BEE-spoke-data/stepbasin-books
language:
- en
---

[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/pszemraj/long-generation-tests/runs/ethp25f9)
# Qwen2-1.5B-stepbasin-books

> [!IMPORTANT]  
> this was finetuned at 16384 context length

This is an experiment on long context text **generation** (i.e. 6k+ tokens generated) to evaluate if/when generation breaks down, etc. As such, all the data on which this model has been fine-tuned are full-length books.
## Details

This model is a fine-tuned version of [Qwen/Qwen2-1.5B](https://huggingface.co/Qwen/Qwen2-1.5B) on https://github.com/stepbasin/books/tree/master/books

It achieves the following results on the evaluation set:
- Loss: 2.8110
- Accuracy: 0.4298
- Num Input Tokens Seen: 44040192