TianyiQ commited on
Commit
b70a6c9
·
verified ·
1 Parent(s): 3065ec8

Upload ./README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -111,9 +111,9 @@ ProgressGym-HistLlama3-8B-C014-instruct is one of the **36 historical language m
111
  | 1.8628 | 3.8636 | 255 | 2.2078 |
112
  | 1.8484 | 3.9394 | 260 | 2.2077 |
113
 
114
- Note that the training data volume for the continued pretraining stage is capped at 300MB. When the corresponding century's corpus exceeds this volume, the training data is randomly sampled to fit the volume.
115
 
116
- **ProgressGym-HistLlama3-8B-C014-instruct is an instruction-tuned language model.** It is tuned on [ProgressGym-TimelessQA](https://huggingface.co/datasets/PKU-Alignment/ProgressGym-TimelessQA), using the following hyperparameters:
117
  - learning_rate: 1.5e-05
118
  - train_batch_size: 8
119
  - eval_batch_size: 16
@@ -179,6 +179,7 @@ Note that the training data volume for the continued pretraining stage is capped
179
  - **[Leaderboard & Interactive Playground]** [PKU-Alignment/ProgressGym-LeaderBoard](https://huggingface.co/spaces/PKU-Alignment/ProgressGym-LeaderBoard)
180
  - **[Huggingface Data & Model Collection]** [PKU-Alignment/ProgressGym](https://huggingface.co/collections/PKU-Alignment/progressgym-666735fcf3e4efa276226eaa)
181
  - **[Github Codebase]** [PKU-Alignment/ProgressGym](https://github.com/PKU-Alignment/ProgressGym)
 
182
  - **[PyPI Package]** *(coming soon - [stay tuned](https://forms.gle/1TWFLL4ZCLeYTD5N6)!)*
183
 
184
  ## Citation
 
111
  | 1.8628 | 3.8636 | 255 | 2.2078 |
112
  | 1.8484 | 3.9394 | 260 | 2.2077 |
113
 
114
+ Note that the training data volume for the continued pretraining stage is capped at 3GB. When the corresponding century's corpus exceeds this volume, the training data is randomly sampled to fit the volume.
115
 
116
+ **ProgressGym-HistLlama3-8B-C014-instruct is an instruction-tuned language model.** It is tuned on [ProgressGym-TimelessQA](https://huggingface.co/datasets/PKU-Alignment/ProgressGym-TimelessQA), using the following hyperparameters. Note, however, that the snapshot at training step 10 is used for the final model, to minimize erosion of the value tendencies learned during continued pretraining; we qualitatively observe that this snapshot still possesses strong instruction-following capabilities.
117
  - learning_rate: 1.5e-05
118
  - train_batch_size: 8
119
  - eval_batch_size: 16
 
179
  - **[Leaderboard & Interactive Playground]** [PKU-Alignment/ProgressGym-LeaderBoard](https://huggingface.co/spaces/PKU-Alignment/ProgressGym-LeaderBoard)
180
  - **[Huggingface Data & Model Collection]** [PKU-Alignment/ProgressGym](https://huggingface.co/collections/PKU-Alignment/progressgym-666735fcf3e4efa276226eaa)
181
  - **[Github Codebase]** [PKU-Alignment/ProgressGym](https://github.com/PKU-Alignment/ProgressGym)
182
+ - **[Documentation]** [ProgressGym Documentation](https://pku-alignment.github.io/ProgressGym/)
183
  - **[PyPI Package]** *(coming soon - [stay tuned](https://forms.gle/1TWFLL4ZCLeYTD5N6)!)*
184
 
185
  ## Citation