Create README.md

README.md

@@ -0,0 +1,58 @@
+---
+license: other
+library_name: peft
+tags:
+- llama2
+---
+
+
+# Model Card for PIPPA ShareGPT Subset Lora 7b
+
+It is an experimental Lora focused on Roleplay that uses a subset of (PIPPA ShareGPT)[https://huggingface.co/datasets/kingbri/PIPPA-shareGPT]
+
+## Usage
+
+ShareGPT(Vicuna 1.1)
+```
+A chat.
+USER: {prompt}
+ASSISTANT:
+```
+
+## Bias, Risks, and Limitations
+
+This Lora is not intended for supplying factual information or advice in any form
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+1k of conversation from (PIPPA ShareGPT)[https://huggingface.co/datasets/kingbri/PIPPA-shareGPT]
+
+### Training Procedure 
+
+The version of this Lora uploaded on this repository was trained using a 8x RTX A6000 cluster in 8-bit with regular LoRA adapters and 32-bit AdamW optimizer.
+
+#### Training Hyperparameters
+
+Training using a fork of axolot with two paths [Patch 1](https://github.com/bdashore3/axolotl/commit/995557bdf3c6c8b3e839b224ef9513fc2b097f30) [Patch 2](https://github.com/bdashore3/axolotl/commit/8970280de2ea01e41c044406051922715f4086cb)
+
+- lora_r: 16
+- lora_alpha: 16
+- lora_dropout: 0.01
+- gradient_accumulation_steps: 6
+- micro_batch_size: 4
+- num_epochs: 3
+- learning_rate: 0.000065
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+Finetuning this model on 8xNVIDIA A6000 48GB in parallel takes about 30 minutes (7B)