Create README.md

README.md

@@ -0,0 +1,17 @@
+---
+datasets:
+- adamo1139/basic_economics_questions_ts_test_1
+---
+QLORA on SpicyBoros 2.2 Llama 7B v2 using synthetic Q&A Dataset.
+a little bit under one epoch, since my GTX1080 decided to OOM a tiny bit before training end and I am using checkpoint made at 450/465 step. 
+I've been running into a lot of issues, so I am happy to even get that far, most of my QLORA attempts had loss go to 0 and deepspeed was forcibly closing training after roughly 0.3 epoch.
+
+My intention with this QLORA is mostly to try to train something usable and cool locally on normal desktop without going to runpod.
+I tried training q4_0 quant with cpu-lora in llama.cpp (https://rentry.org/cpu-lora) but it's been a miss, it's about 20x slower on 11400f than on poorman's GTX 1080.
+
+The model can be used to ask questions about basic economic concepts, responses will have a viewpoint similar to the one expressed by Thomas Sowell in his book Basic Economics.
+
+Prompt format:
+
+Reader: {prompt} 
+'\nThomas:\n' {response}