Created Initial ReadMe

README.md

@@ -0,0 +1,55 @@
+---
+license: apache-2.0
+datasets:
+- Anthropic/hh-rlhf
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- text-generation-inference
+---
+# Model Card for OpenBezoar-HH-RLHF-SFT
+
+The OpenBezoar-HH-RLHF-SFT is an LLM that has been further instruction fine tuned version of [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT) model on a subset of [Anthropic's HH-RLHF Dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf).
+
+## Model Details
+
+- Base Model: [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT)
+- Dataset used for SFT: First 100K examples of the [HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset
+- Epochs: 1
+
+### Model Description
+
+Primary purpose of performing SFT on [OpenBezoar-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-SFT) is to minimize the distribution shift before applying Direct Preference Optimization (DPO) for human preferences alignment. For more information please refer to our paper.
+
+### Model Sources
+
+- **Repository:** [More Information Needed]
+- **Paper :** [More Information Needed]
+
+## Instruction Format
+
+We follow the typical format for instruction-based prompt templates, with a system prompt followed up by the user prompt. Both begins with a prefix and ends with two newline characters as described below. It is important to utilize this template in order to obtain best responses for instruction fine-tuning related tasks.
+```
+### System: {system}
+
+### Instruction: {instruction}
+
+### Response:
+```
+
+Notice that **no** end-of-sentence (eos) token is being appended.
+
+## Limitations
+
+- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
+- This model is not aligned to human preferences and therefore it may generate harmful and uncensored content.
+- Caution is urged against relying on this model for production or adjacent use-cases.
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]