Added the ReadMe

README.md

@@ -0,0 +1,56 @@
+---
+license: apache-2.0
+datasets:
+- Anthropic/hh-rlhf
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- text-generation-inference
+---
+# Model Card for OpenBezoar-HH-RLHF-DPO
+
+The OpenBezoar-HH-RLHF-DPO is an LLM that has been fine tuned for human preferences alignment using [Direct Preference Optimization (DPO)](https://arxiv.org/abs/2305.18290), on top of [OpenBezoar-HH-RLHF-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-HH-RLHF-SFT) model on a subset of [Anthropic's HH-RLHF Dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf).
+
+## Model Details
+
+- Base Model: [OpenBezoar-HH-RLHF-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-HH-RLHF-SFT) model on a subset of [Anthropic's HH-RLHF Dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf)
+- Dataset used for SFT: First 100K examples of the [HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset
+- Alignment Method: [Direct Preference Optimization (DPO)](https://arxiv.org/abs/2305.18290)
+- Epochs: 1
+
+### Model Description
+
+OpenBezoar-HH-RLHF-SFT is an LLM that is built upon the OpenLLaMA 3B v2 architecture. This model has been fine-tuned for human preferences alignment using [DPO](https://arxiv.org/abs/2305.18290). Alignment has been performed on top of the [OpenBezoar-HH-RLHF-SFT](https://huggingface.co/SurgeGlobal/OpenBezoar-HH-RLHF-SFT) model. For more information please refer to our paper.
+
+### Model Sources
+
+- **Repository:** [More Information Needed]
+- **Paper :** [More Information Needed]
+
+## Instruction Format
+
+We follow the typical format for instruction-based prompt templates, with a system prompt followed up by the user prompt. Both begins with a prefix and ends with two newline characters as described below. It is important to utilize this template in order to obtain best responses for instruction fine-tuning related tasks.
+```
+### System: {system}
+
+### Instruction: {instruction}
+
+### Response:
+```
+
+Notice that **no** end-of-sentence (eos) token is being appended.
+
+## Limitations
+
+- The model might not consistently show improved abilities to follow instructions, and it could respond inappropriately or get stuck in loops.
+- Although this model is aligned to human preferences and has been evaluated for performance, it is not guaranteed that it will **refrain** from generating harmful content exclusively.
+- Caution is urged against relying on this model for production or adjacent use-cases.
+
+## Citation
+
+If you find our work useful, please cite our paper as follows:
+
+```
+[More Information Needed]
+```