Use BioGptModel to get features
#19
by
pete88b
- opened
We can't use BioGptForCausalLM to get the features as it returns logits for next token prediction. I'm pretty sure we want BioGptModel to give us last_hidden_state
Hi,
You can just load a BioGptModel from this checkpoint, it will just give you a warning that the head on top isn't initialized but you can use it for feature extraction:
from transformers import BioGptModel
model = BioGptModel.from_pretrained("microsoft/biogpt")
Hi Niels,
so my suggested change to the docs is good to merge (o:
and while i have your ear - thank you so much for notebooks like NER with BERT - really helpful