yuewang-sf commited on
Commit
e8086d7
1 Parent(s): 151881c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -46,7 +46,7 @@ Supported languages (9 in total) are as follows:
46
 
47
  ## Training procedure
48
 
49
- This checkpoint is initialized from off-the-shelf LLMs, i.e. its encoder is initialized from CodeGen-350M-mono and its decoder is initialized from CodeGen-16B-mono.
50
  It is trained on the unimodal code data at the first-stage pretraining, which includes a diverse set of pretraining tasks including _span denoising_ and two variants of _causal language modeling_.
51
  After that, it is further trained on the Python subset with the causal language modeling objective for another epoch to better adapt for Python code generation.
52
  Please refer to the paper for more details.
 
46
 
47
  ## Training procedure
48
 
49
+ This checkpoint is initialized from off-the-shelf LLMs, i.e. its encoder is initialized from [CodeGen-350M-mono](https://huggingface.co/Salesforce/codegen-350M-mono) and its decoder is initialized from [CodeGen-16B-mono](https://huggingface.co/Salesforce/codegen-16B-mono).
50
  It is trained on the unimodal code data at the first-stage pretraining, which includes a diverse set of pretraining tasks including _span denoising_ and two variants of _causal language modeling_.
51
  After that, it is further trained on the Python subset with the causal language modeling objective for another epoch to better adapt for Python code generation.
52
  Please refer to the paper for more details.