Can I get the bin file of JinaBertForMaskedLM?
#18
by
xaccelerator
- opened
Can I get the bin file of JinaBertForMaskedLM? I need do further pretrain on this model, I try to load JinaBertForMaskedLM from JinaBertModel, but the train loss down as slow as I train from a new initialized model.
hi @xaccelerator sorry we have no plan to open source the backbone yet, hopefully you understand.
hi @xaccelerator sorry we have no plan to open source the backbone yet, hopefully you understand.
okk,thanks
hi @xaccelerator sorry we have no plan to open source the backbone yet, hopefully you understand.
I try to freeze the param of JinaBertModel and only train the other param in JinaBertForMaskLM,after loss become stable I unfreeze the param and continue train,it works well!