---
license: cc-by-2.0
datasets:
- teknium/OpenHermes-2.5
language:
- en
tags:
- finance
- legal
- biology
- art
---

Behold, one of the first fine-tunes of Mistral's 7B 0.2 Base model. SatoshiN is trained on 4 epochs of a diverse custom data-set, combined with a single sanitization round of teknium/OpenHermes2.5.
It's a nice assistant that isn't afraid to ask questions, and gather additional information before providing a response to user prompts.

  I have found success using a variety of instruction-formats such as Alpaca, ChatML and Mistral. The custom training was performed on raw-text with the idea that it might acquire better generalization skills.

  Total model-size has increased from 7.24B to 7.35B after merging a .5GB LoRa via PEFT.

SatoshiN | Base-Model

Wikitext Perplexity: 6.27 | 5.4