Update README.md
Browse files
README.md
CHANGED
|
@@ -19,7 +19,7 @@ license: apache-2.0
|
|
| 19 |
- The process of building the dataset is as follows
|
| 20 |
* A. Extract important texts related to technology, such as technology trends and technology definitions, from research reports.
|
| 21 |
* B. Preprocess the extracted text
|
| 22 |
-
* C. Generate question and answer pairs (total 1.5 million) based on the extracted text by using ChatGPT API.
|
| 23 |
* D. Reformat the dataset in the form of (Instruction, Output, Source). ‘Instruction’ is the user's question, ‘Output’ is the answer, and ‘Source’ is the research report identification code that the answer is based on.
|
| 24 |
* E. Remove low-quality data by the data quality evaluation module. Use only high-quality Q&As for training. (1 million)
|
| 25 |
* ※ In KoRnDAlpaca v2 (planned for `23.10), in addition to Q&A, the instruction dataset will be added to generate long-form technology trends.
|
|
|
|
| 19 |
- The process of building the dataset is as follows
|
| 20 |
* A. Extract important texts related to technology, such as technology trends and technology definitions, from research reports.
|
| 21 |
* B. Preprocess the extracted text
|
| 22 |
+
* C. Generate question and answer pairs (total 1.5 million) based on the extracted text by using ChatGPT API(temporarily), which scheduled to be replaced with our own question&answer generation model(`23.11).
|
| 23 |
* D. Reformat the dataset in the form of (Instruction, Output, Source). ‘Instruction’ is the user's question, ‘Output’ is the answer, and ‘Source’ is the research report identification code that the answer is based on.
|
| 24 |
* E. Remove low-quality data by the data quality evaluation module. Use only high-quality Q&As for training. (1 million)
|
| 25 |
* ※ In KoRnDAlpaca v2 (planned for `23.10), in addition to Q&A, the instruction dataset will be added to generate long-form technology trends.
|