TroyDoesAI commited on
Commit
12e43e4
1 Parent(s): 9d670de

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -6
README.md CHANGED
@@ -9,16 +9,11 @@ Experimenting with Dataset Quality to improve generations, TinyLlama is faster t
9
  Overview
10
  This model is meant to enhance adherence to provided context (e.g., for RAG applications) and reduce hallucinations, inspired by airoboros context-obedient question answer format.
11
 
12
- ---
13
- license: cc-by-4.0
14
- ---
15
-
16
- # Contextual DPO
17
-
18
  ## Overview
19
 
20
  The format for a contextual prompt is as follows:
21
  ```
 
22
  BEGININPUT
23
  BEGINCONTEXT
24
  [key0: value0]
@@ -34,6 +29,7 @@ ENDINSTRUCTION
34
  ```
35
 
36
  I know it's a bit verbose and annoying, but after much trial and error, using these explicit delimiters helps the model understand where to find the responses and how to associate specific sources with it.
 
37
  - `BEGININPUT` - denotes a new input block
38
  - `BEGINCONTEXT` - denotes the block of context (metadata key/value pairs) to associate with the current input block
39
  - `ENDCONTEXT` - denotes the end of the metadata block for the current input
@@ -46,6 +42,7 @@ I know it's a bit verbose and annoying, but after much trial and error, using th
46
 
47
  Here's a trivial, but important example to prove the point:
48
  ```
 
49
  BEGININPUT
50
  BEGINCONTEXT
51
  date: 2021-01-01
@@ -60,6 +57,7 @@ ENDINSTRUCTION
60
 
61
  And the expected response:
62
  ```
 
63
  Blueberries are now green.
64
  Source:
65
  date: 2021-01-01
 
9
  Overview
10
  This model is meant to enhance adherence to provided context (e.g., for RAG applications) and reduce hallucinations, inspired by airoboros context-obedient question answer format.
11
 
 
 
 
 
 
 
12
  ## Overview
13
 
14
  The format for a contextual prompt is as follows:
15
  ```
16
+ Contextual-Request:
17
  BEGININPUT
18
  BEGINCONTEXT
19
  [key0: value0]
 
29
  ```
30
 
31
  I know it's a bit verbose and annoying, but after much trial and error, using these explicit delimiters helps the model understand where to find the responses and how to associate specific sources with it.
32
+ - `Contextual-Request:` - denotes the type of request pattern the model is to follow for consistency
33
  - `BEGININPUT` - denotes a new input block
34
  - `BEGINCONTEXT` - denotes the block of context (metadata key/value pairs) to associate with the current input block
35
  - `ENDCONTEXT` - denotes the end of the metadata block for the current input
 
42
 
43
  Here's a trivial, but important example to prove the point:
44
  ```
45
+ Contextual-Request:
46
  BEGININPUT
47
  BEGINCONTEXT
48
  date: 2021-01-01
 
57
 
58
  And the expected response:
59
  ```
60
+ ### Contextual Response:
61
  Blueberries are now green.
62
  Source:
63
  date: 2021-01-01