ydeng9 commited on
Commit
88664b0
1 Parent(s): 9257b64

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -10
README.md CHANGED
@@ -7,6 +7,8 @@ language:
7
  base_model: mistralai/Mistral-7B-v0.1
8
  pipeline_tag: text-generation
9
  ---
 
 
10
  # zephyr-7b-sft-full-spin-iter1
11
 
12
  This model is a self-play fine-tuned model at iteration 1 from [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) using synthetic data based on on the [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) dataset.
@@ -23,16 +25,16 @@ This model is a self-play fine-tuned model at iteration 1 from [alignment-handbo
23
  ### Training hyperparameters
24
  The following hyperparameters were used during training:
25
 
26
- learning_rate: 5e-07
27
- train_batch_size: 8
28
- seed: 42
29
- distributed_type: multi-GPU
30
- num_devices: 8
31
- total_train_batch_size: 64
32
- optimizer: RMSProp
33
- lr_scheduler_type: linear
34
- lr_scheduler_warmup_ratio: 0.1
35
- num_epochs: 2.0
36
 
37
  ## Citation
38
  ```
 
7
  base_model: mistralai/Mistral-7B-v0.1
8
  pipeline_tag: text-generation
9
  ---
10
+ see our paper in https://arxiv.org/abs/2401.01335
11
+
12
  # zephyr-7b-sft-full-spin-iter1
13
 
14
  This model is a self-play fine-tuned model at iteration 1 from [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) using synthetic data based on on the [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) dataset.
 
25
  ### Training hyperparameters
26
  The following hyperparameters were used during training:
27
 
28
+ - learning_rate: 5e-07
29
+ - train_batch_size: 8
30
+ - seed: 42
31
+ - distributed_type: multi-GPU
32
+ - num_devices: 8
33
+ - total_train_batch_size: 64
34
+ - optimizer: RMSProp
35
+ - lr_scheduler_type: linear
36
+ - lr_scheduler_warmup_ratio: 0.1
37
+ - num_epochs: 2.0
38
 
39
  ## Citation
40
  ```