Update README.md

README.md

@@ -1,204 +1,143 @@
 ---
 library_name: peft
 base_model: cognitivecomputations/dolphin-2_6-phi-2
+license: apache-2.0
+datasets:
+- jondurbin/contextual-dpo-v0.1
+tags:
+- dpo
+- rag
 ---
 
-# Model Card for Model ID
+# Model Card for Phi-2-Contextual
+
+A lightweight model (2.8B) with enhanced RAG capabilities, with lower risk of hallucination.
+
+This is a DPO fine-tune of the Phi-2 architecture (in particular, dolphin-2_6-phi-2) over the dataset https://huggingface.co/datasets/jondurbin/contextual-dpo-v0.1.
+
+
+## Prompt format
+
+```
+BEGININPUT
+BEGINCONTEXT
+[key0: value0]
+[key1: value1]
+... other metdata ...
+ENDCONTEXT
+[insert your text blocks here]
+ENDINPUT
+[add as many other blocks, in the exact same format]
+BEGININSTRUCTION
+[insert your instruction(s).  The model was tuned with single questions, paragraph format, lists, etc.]
+ENDINSTRUCTION
+```
+with the following definitions:
+
+- `BEGININPUT` - denotes a new input block
+- `BEGINCONTEXT` - denotes the block of context (metadata key/value pairs) to associate with the current input block
+- `ENDCONTEXT` - denotes the end of the metadata block for the current input
+- [text] - Insert whatever text you want for the input block, as many paragraphs as can fit in the context.
+- `ENDINPUT` - denotes the end of the current input block
+- [repeat as many input blocks in this format as you want]
+- `BEGININSTRUCTION` - denotes the start of the list (or one) instruction(s) to respond to for all of the input blocks above.
+- [instruction(s)]
+- `ENDINSTRUCTION` - denotes the end of instruction set
+
+
+## Examples
+
+### Reference-based Q&A
+
+```
+BEGININPUT
+BEGINCONTEXT
+date: 2021-01-01
+url: https://en.wikipedia.org/wiki/Rottnest_Island
+ENDCONTEXT
+Rottnest Island (Nyungar: Wadjemup), often colloquially referred to as "Rotto", is a 19-square-kilometre (7.3 sq mi) island off the coast of Western Australia, located 18 kilometres (11 mi) west of Fremantle. A sandy, low-lying island formed on a base of aeolianite limestone, Rottnest is an A-class reserve, the highest level of protection afforded to public land.
+
+Together with Garden Island, Rottnest Island is a remnant of Pleistocene dune ridges.[7] Along with several other islands, Rottnest became separated from the mainland around 7,000 years ago, when sea levels rose; the traditional Noongar name for the island is Wadjemup, which means "place across the water where the spirits are".[8] Human artefacts have been found on the island dating back at least 30,000 years, but visitation and habitation of the island by the Noongar people appears to have ceased following its separation from the mainland.
+
+The island was first documented by Willem de Vlamingh in 1696, who called it 't Eylandt 't Rottenest ("Rats' Nest Island") after the quokka population. Following establishment of the Swan River Colony (now Perth) in 1829, the island was initially used by British settlers for agricultural purposes, and a permanent settlement was built in Thomson Bay. From 1838 to 1931, Rottnest Island was also used as a prison and forced labour camp for over 3,600 Aboriginal people, who were subjected to extremely harsh conditions on the island.[9][10] Other historical uses of the island include as a military site, and for internment camps housing enemy aliens. Many of the island's buildings date from the colonial period, often made from locally quarried limestone, and are now used as accommodation for holidays.
+ENDINPUT
+BEGININSTRUCTION
+In which years was the island used as a jail?
+ENDINSTRUCTION
+```
+
+produces
+
+```
+The island was used as a jail from 1838 to 1931.
+```
+
+### Structured output generation
+
+```
+BEGININPUT
+BEGINCONTEXT
+date: 2021-01-01
+url: https://web.site/123
+ENDCONTEXT
+The rock parrot (Neophema petrophila) is a species of grass parrot native to Australia. Described by John Gould in 1841, it is a small parrot 22 to 24 cm (8+3⁄4 to 9+1⁄2 in) long and weighing 50 to 60 g (1+3⁄4 to 2 oz) with predominantly olive-brown upperparts and more yellowish underparts. Its head is olive with light blue forecheeks and lores, and a dark blue frontal band line across the crown with lighter blue above and below. The sexes are similar in appearance, although the female tends to have a duller frontal band and less blue on the face. The preferred habitats are rocky islands and coastal dune areas from Lake Alexandrina in southeastern South Australia westwards across coastal South and Western Australia to Shark Bay.
+ENDINPUT
+BEGININSTRUCTION
+Generate a list with all the mentioned locations
+ENDINSTRUCTION
+```
+
+produces
+
+```
+The mentioned locations in the provided text are:
+
+1. Lake Alexandrina in southeastern South Australia
+2. Coastal dune areas extending from Lake Alexandrina to Shark Bay in Western Australia.
+```
+
+### Multiple contexts
+
+
+```
+BEGININPUT
+BEGINCONTEXT
+date: 2021-01-01
+url: https://wikipedia.com/Lego
+ENDCONTEXT
+Since 1963, Lego pieces have been manufactured from acrylonitrile butadiene styrene (ABS).[19][39] As of September 2008, Lego engineers use the NX CAD/CAM/CAE PLM software suite to model the elements. The software allows the parts to be optimised by way of mould flow and stress analysis. Prototype moulds are sometimes built before the design is committed to mass production. The ABS plastic is heated to 232 °C (450 °F) until it reaches a dough-like consistency. It is then injected into the moulds using forces of between 25 and 150 tonnes and takes approximately 15 seconds to cool. The moulds are permitted a tolerance of up to twenty micrometres to ensure the bricks remain connected.[42] Human inspectors check the output of the moulds to eliminate significant variations in colour or thickness. According to the Lego Group, about eighteen bricks out of every million fail to meet the standard required.[46]
+
+Lego factories recycle all but about 1 percent of their plastic waste from the manufacturing process. If the plastic cannot be re-used in Lego bricks, it is processed and sold on to industries that can make use of it.[47][48] Lego, in 2018, set a self-imposed 2030 deadline to find a more eco-friendly alternative to the ABS plastic.[49]
+ENDINPUT
+BEGININPUT
+BEGINCONTEXT
+date: 2021-01-02
+url: https://wikipedia.com/Rottnest
+ENDCONTEXT
+Rottnest Island (Nyungar: Wadjemup), often colloquially referred to as "Rotto", is a 19-square-kilometre (7.3 sq mi) island off the coast of Western Australia, located 18 kilometres (11 mi) west of Fremantle. A sandy, low-lying island formed on a base of aeolianite limestone, Rottnest is an A-class reserve, the highest level of protection afforded to public land.
+
+Together with Garden Island, Rottnest Island is a remnant of Pleistocene dune ridges.[7] Along with several other islands, Rottnest became separated from the mainland around 7,000 years ago, when sea levels rose; the traditional Noongar name for the island is Wadjemup, which means "place across the water where the spirits are".[8] Human artefacts have been found on the island dating back at least 30,000 years, but visitation and habitation of the island by the Noongar people appears to have ceased following its separation from the mainland.
+
+The island was first documented by Willem de Vlamingh in 1696, who called it 't Eylandt 't Rottenest ("Rats' Nest Island") after the quokka population. Following establishment of the Swan River Colony (now Perth) in 1829, the island was initially used by British settlers for agricultural purposes, and a permanent settlement was built in Thomson Bay. From 1838 to 1931, Rottnest Island was also used as a prison and forced labour camp for over 3,600 Aboriginal people, who were subjected to extremely harsh conditions on the island.[9][10] Other historical uses of the island include as a military site, and for internment camps housing enemy aliens. Many of the island's buildings date from the colonial period, often made from locally quarried limestone, and are now used as accommodation for holidays.
+ENDINPUT
+BEGININSTRUCTION
+What is threshold to ensure the pieces match?
+ENDINSTRUCTION
+```
+
+produces
+
+```
+The threshold to ensure the pieces match refers to the tolerance of up to twenty micrometres in the Lego brick manufacturing process. According to the provided context, this tolerance is set by the Lego Group and it helps to ensure that the bricks remain connected. 
+
+Reference(s):
+date: 2021-01-01
+url: https://wikipedia.com/Lego
+```
 
-<!-- Provide a quick summary of what the model is/does. -->
 
 
 
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure 
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-
-
 ### Framework versions
 
 - PEFT 0.7.1