Regarding fine tuning model on custom dataset
Hi
Thank you for sharing the finetuned longt5 model for SumPubMed dataset. I want to fine-tune this further on my own long math documents dataset. Could you please share how you fine-tuned the longt5 model, training set up, etc?
Did you use this script https://github.com/huggingface/transformers/blob/main/examples/pytorch/summarization/run_summarization.py?
Thanks!
Hello there!
I used something similar to this default script for fine-tuning on text summarization datasets.
https://github.com/huggingface/notebooks/blob/main/examples/summarization.ipynb
Thanks for sharing!
Hi, could you please share the learning rate used for fine-tuning with sumpubmed dataset? In the paper, it is mentioned 0.001 for other datasets.
I am fine-tuning on a new dataset, just wanted to know whether to start very slow (2e-5) like in Pegasus or high (0.001) like in Longt5? Also, what GPU configuration did you use for fine-tuning?