|
--- |
|
language: en |
|
thumbnail: |
|
widget: |
|
- text: "Old people with COVID-19 tends to suffer" |
|
--- |
|
|
|
# GPT-2 + bio/medrxiv files from CORD19: π¦ β β |
|
|
|
**GPT-2** fine-tuned on **biorxiv_medrxiv** files from [CORD-19](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge) dataset. |
|
|
|
|
|
## Datasets details: |
|
|
|
| Dataset | # Files | |
|
| ---------------------- | ----- | |
|
| biorxiv_medrxiv | 885 | |
|
|
|
|
|
## Model training: |
|
|
|
The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: |
|
|
|
```bash |
|
|
|
export TRAIN_FILE=/path/to/dataset/train.txt |
|
|
|
python run_language_modeling.py \\n --model_type gpt2 \\n --model_name_or_path gpt2 \\n --do_train \\n --train_data_file $TRAIN_FILE \\n --num_train_epochs 4 \\n --output_dir model_output \\n --overwrite_output_dir \\n --save_steps 2000 \\n --per_gpu_train_batch_size 3 |
|
``` |
|
|
|
## Model in action / Example of usage: β |
|
|
|
You can get the following script [here](https://github.com/huggingface/transformers/blob/master/examples/text-generation/run_generation.py) |
|
|
|
```bash |
|
python run_generation.py \\n --model_type gpt2 \\n --model_name_or_path mrm8488/GPT-2-finetuned-CORD19 \\n --length 200 |
|
``` |
|
```txt |
|
π΅π΄π¦ |
|
# Input: Old people with COVID-19 tends to suffer |
|
# Output: === GENERATED SEQUENCE 1 === |
|
Old people with COVID-19 tends to suffer more symptom onset time and death. It is well known that many people with COVID-19 have high homozygous ZIKV infection in the face of severe symptoms in both severe and severe cases. |
|
The origin of Wuhan Fever was investigated by Prof. Shen Jiang at the outbreak of Wuhan Fever [34]. As Huanan Province is the epicenter of this outbreak, Huanan, the epicenter of epidemic Wuhan Fever, is the most potential location for the direct transmission of infection (source: Zhongzhen et al., 2020). A negative risk ratio indicates more frequent underlying signs in the people in Huanan Province with COVID-19 patients. Further analysis of reported Huanan Fever onset data in the past two years indicated that the intensity of exposure is the key risk factor for developing MERS-CoV infection in this region, especially among children and elderly. To be continued to develop infected patients would be a very important area for |
|
``` |
|
|
|
![Model in action](https://media.giphy.com/media/TgUdO72Iwk9h7hhm7G/giphy.gif) |
|
|
|
|
|
|
|
> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) |
|
|
|
> Made with <span style="color: #e25555;">♥</span> in Spain |
|
|