emozilla
/

scifi-fantasy-author-7b-8k_delta

Text Generation

text-generation-inference

Model card Files Files and versions Community

emozilla commited on May 11, 2023

Commit

54fbe61

•

1 Parent(s): 8dfcf0f

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -17,6 +17,6 @@ paricularly in the Science Fiction and Fantasy genres.
 The following hyperparameters were used
 |Batch Size|Epochs|Context length|Learning rate|Scheduler|Weight decay|Warmup ratio|
 |----------|------|--------------|-------------|---------|------------|------------|
-|       64 |    3 |         8192 |        2e-5 |  Cosine |         0. |       0.03 |
 The model reached a training loss of 2.008 and took approximately 8 hours on 8x A100 80 GB GPUs.

 The following hyperparameters were used
 |Batch Size|Epochs|Context length|Learning rate|Scheduler|Weight decay|Warmup ratio|
 |----------|------|--------------|-------------|---------|------------|------------|
+|      128 |    3 |         8192 |        2e-5 |  Cosine |         0. |       0.03 |
 The model reached a training loss of 2.008 and took approximately 8 hours on 8x A100 80 GB GPUs.