zhangirazerbayev commited on
Commit
1e4dd33
1 Parent(s): d782ab6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -15,7 +15,7 @@ datasets:
15
 
16
  # Model Description
17
  ProofGPT-v0.1 is a 1.3B parameter language model based on the GPT-NeoX architecture and trained on the [proof-pile](https://huggingface.co/datasets/hoskinson-center/proof-pile) (v1.1).
18
- The model is initialized with [pythia-1.3b](https://huggingface.co/EleutherAI/pythia-1.3b) weights. ProofGPT-v0.1's Weights & Biases training log is viewable [here](https://wandb.ai/zhangir-azerbayev/math-lm/groups/1.3B%20preliminary_227qly9c/workspace?workspace=user-zhangir-azerbayev).
19
 
20
  Detailed evaluations coming soon :)
21
 
 
15
 
16
  # Model Description
17
  ProofGPT-v0.1 is a 1.3B parameter language model based on the GPT-NeoX architecture and trained on the [proof-pile](https://huggingface.co/datasets/hoskinson-center/proof-pile) (v1.1).
18
+ We initiailized training with pythia-1.3b weights, a precursor to the [pythia-1.4b](https://huggingface.co/EleutherAI/pythia-1.3b) model that has roughly equivalent performance.
19
 
20
  Detailed evaluations coming soon :)
21