Custom dataset fine tuning

#5
by ChinmayBH - opened

Hello! Can anyone please tell me how we can use our own custom data to fine-tune Vilt-b32-finetuned-VQA? Answer will be really appreciable! Thank you !

I am also curious about this @ChinmayBH Did you figure it out?

I have the fine tuning notebook running but the training results are always very poor for me.
I was expecting to at least see the overfit. Should I expect this from the notebook as it exists?

Setup:
I am using the same dataset that the notebook is using. I wanted to get that working before trying a custom dataset.
I tried it without changes to the notebook and with a variety of settings for learning rate and epochs.
Running on Jupyter notebook. Torch 2.1.1_cu118

Example: dataset[0], the image looking through the net.
" what is this photo taken looking through? "

The type of result I get... Sometimes "net" might show up in the list but it is never the best answer.
0.338682621717453 yes
0.27205798029899597 no
0.036353662610054016 2
0.0306074358522892 skiing
0.02998974919319153 1

This comment has been hidden

Sign up or log in to comment