microsoft

/

Florence-2-large

like 695

Two changes:

• Fix Florence2Seq2SeqLMOutput
• Only set attention_mask device if inputs_embeds is not None

hi @Xenova in which case will inputs_embeds be None, no input_ids and pixel_values, so input is decoder_input_ids?

This occurs when encoder_outputs is already specified, meaning we don't need to prepare input_embeds again.

As seen here, you only use inputs_embeds in the LM forward method when encoder_outputs is None

when will this get merged ??
is there a ready to use fork ??

i keep getting
TypeError: Florence2Seq2SeqLMOutput.init() got an unexpected keyword argument 'loss'
when i try to finetune

hi @leoxiaobin , please merge this pr