Back to all models
text-generation mask_token:
Query this model
πŸ”₯ This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

⚑️ Upgrade your account to access the Inference API

Share Copied link to clipboard

Monthly model downloads

mrm8488/GPT-2-finetuned-CORD19 mrm8488/GPT-2-finetuned-CORD19
last 30 days



Contributed by

mrm8488 Manuel Romero
156 models

How to use this model directly from the πŸ€—/transformers library:

Copy to clipboard
from transformers import AutoTokenizer, AutoModelWithLMHead tokenizer = AutoTokenizer.from_pretrained("mrm8488/GPT-2-finetuned-CORD19") model = AutoModelWithLMHead.from_pretrained("mrm8488/GPT-2-finetuned-CORD19")

GPT-2 + CORD19 dataset : 🦠 ✍ βš•

GPT-2 fine-tuned on biorxiv_medrxiv, comm_use_subset and custom_license files from CORD-19 dataset.

Datasets details

Dataset # Files
biorxiv_medrxiv 885
comm_use_subset 9K
custom_license 20.6K

Model training

The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command:

export TRAIN_FILE=/path/to/dataset/train.txt

python \
    --model_type gpt2 \
    --model_name_or_path gpt2 \
    --do_train \
    --train_data_file $TRAIN_FILE \
    --num_train_epochs 4 \
    --output_dir model_output \
    --overwrite_output_dir \
    --save_steps 10000 \
    --per_gpu_train_batch_size 3
training loss

Model in action / Example of usage βœ’

You can get the following script here

python \
    --model_type gpt2 \
    --model_name_or_path mrm8488/GPT-2-finetuned-CORD19 \
    --length 200
# Input: the effects of COVID-19 on the lungs
# Output: === GENERATED SEQUENCE 1 ===
the effects of COVID-19 on the lungs are currently debated (86). The role of this virus in the pathogenesis of pneumonia and lung cancer is still debated. MERS-CoV is also known to cause acute respiratory distress syndrome (87) and is associated with increased expression of pulmonary fibrosis markers (88). Thus, early airway inflammation may play an important role in the pathogenesis of coronavirus pneumonia and may contribute to the severe disease and/or mortality observed in coronavirus patients.
Pneumonia is an acute, often fatal disease characterized by severe edema, leakage of oxygen and bronchiolar inflammation. Viruses include coronaviruses, and the role of oxygen depletion is complicated by lung injury and fibrosis in the lung, in addition to susceptibility to other lung diseases. The progression of the disease may be variable, depending on the lung injury, pathologic role, prognosis, and the immune status of the patient. Inflammatory responses to respiratory viruses cause various pathologies of the respiratory

Created by Manuel Romero/@mrm8488 | LinkedIn

Made with in Spain