Create README.md

README.md

@@ -0,0 +1,55 @@
+---
+license: apache-2.0
+library_name: peft
+tags:
+- trl
+- sft
+- generated_from_trainer
+base_model: TheBloke/Mistral-7B-Instruct-v0.1-GPTQ
+model-index:
+- name: mistral-finetuned-guided-meditations
+  results: []
+---
+
+# mistral-finetuned-guided-meditations
+
+This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.1-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GPTQ) specifically trained for generating guided meditations. The fine-tuning was conducted on the "jhana-guided-meditations-collection" dataset available on Hugging Face, utilizing the QLoRA fine-tuning approach.
+
+## Model description
+
+The model utilizes the LlamaTokenizer and is quantized for efficient load and execution. It is intended for generating mindful meditation scripts by understanding and generating contextually relevant content. This version has been optimized for better performance and lower resource utilization during inference.
+
+## Intended uses & limitations
+
+This model is intended for generating text related to guided meditations. It may not perform well on unrelated tasks or general-purpose language understanding due to its specialized training.
+
+## Training and evaluation data
+
+The model was trained on the "jhana-guided-meditations-collection" dataset, which consists of various guided meditation scripts. The data was preprocessed and tokenized using the LlamaTokenizer.
+
+## Training procedure
+
+### Training hyperparameters
+
+- Learning Rate: 0.0002
+- Batch Size: 8 for training, 8 for evaluation
+- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
+- Scheduler: Cosine learning rate scheduler
+- Training Steps: 250
+- Mixed Precision Training: Utilized Native AMP
+
+### Training results
+
+Training resulted in a model capable of generating coherent and contextually relevant meditation scripts, improving upon the base model's capabilities in this specific domain.
+
+### Framework versions
+
+- PEFT: 0.10.0
+- Transformers: 4.40.0.dev0
+- Pytorch: 2.2.2+cu121
+- Datasets: 2.18.0
+- Tokenizers: 0.15.2
+
+## Axolotl Fine-Tuning Details
+
+The model was fine-tuned using the Axolotl toolkit, with specific emphasis on low-resource environments. Key aspects of the fine-tuning process include utilizing QLoRA for efficient learning and adapting to the guided meditation domain, employing mixed precision training for enhanced performance, and custom tokenization to fit the unique structure of meditation scripts. The entire process emphasizes resource efficiency and model effectiveness in generating serene and contextually appropriate meditation guides.