Update README.md

README.md

@@ -6,4 +6,28 @@ language:
 - en
 tags:
 - text-generation-inference
----
+---
+
+# Simple Stories
+Simple Stories will be a series of small text generation models trained on the [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) dataset.
+
+The goal is to experiment with creating small language models that can perform highly specific tasks. In this case, the task is generating children's stories.
+
+## Model Details
+The model has 4M parameters (Safetensors seems to have inflated this to 13M, I will look into why in the future). This model has not been fine-tuned for instructions. It will simply spew out text when asked. I will be working on an instruct model in the coming days.
+
+The model is a decoder only transformer model with 4 decoder layers and 2 attention heads. The model was trained on only ~50MB of text and can already produce semi-coherent stories.
+
+The code used to train the model can be found on my [github](https://github.com/broskicodes/slms). For now, this is also the only way to train and obtain the tokenizer necessary for encoding and decoding text. Check it out if you are interested.
+
+## Sample
+Here is a short sample generated by the model.
+
+`Once upon a time, there was a little girl called Daisy. Daisy wanted to go to the park with her mommy. She packed some yummy food and chirpies and carried them . Daisy was so excited for her mommy to try. The puppy and Mommy brought a big spoon to make souping. Daisy loved swimming and jun ate until she was disappointed. They began to start playing in the garden. They gathered around and ate and boot into the bread . As Daisy got hungry on the grass, she found some magic. She read more to see what was Luckily, Daisy was very impressed. When the lady opened the pot, something tickling to another. It was a rare. Daisy was so happy that she gave the tunately. Daisy was no longer scared. She knew she had to tell Mommy at the store. She took her to the soup and opened the tasty hot chocolate. When Daisy gave it to Daisy and princessed around a special spoon every day.`
+
+No, the story doesn't fully make sense. But most of the words are valid English and the characters and overarching plot are consistent. This is progress :)
+
+## Going forward
+The direct next step is creating a instruct model for interacting with and generating custom stories. After that I will continue working to improve the base model by increasing the amount of data it is trained on and continueing to experiment with different hyperparameters.
+
+If you have any suggestions/questions, or you want to discuss anything about the model please reach out to me on twitter [@_broskitweets](https://twitter.com/_broskitweets).