microsoft

/

phi-2

like 3.13k

here's a modified version that returns attention and hidden states https://huggingface.co/wassname/phi-2-GPTQ_w_hidden_states/blob/main/configuration_phi.py

@wassname is there any plan to really change phi-2 ?
Because the following warning remains on the main page :
"Remark: In the generation function, our model currently does not support beam search (num_beams > 1). Furthermore, in the forward pass of the model, we currently do not support outputting hidden states or attention values, or using custom input embeddings."
I personally use a lot custom input embeddings and this makes phi unusable for many usecases in my opinion.

Hello @edmond and @wassname !

This will be updated once we integrate with the Phi implementation in HF.

Best regards,
Gustavo.

Thanks Gustov, much appreciated. Phi -2 is an awesome model for research as it fits on consumer gpu's even when doing strange experiments (VAE, Adaptors, Probing).

Amazing ! Im impatient

@wassname Do you know if the last commits include the changes we were hoping for ? Its hard for me to know without reading their code

It looks like they did fix it, thanks to whoever did that :)

https://huggingface.co/microsoft/phi-2/blob/b10c3eba545ad279e7208ee3a5d644566f001670/modeling_phi.py#L1351

Yeyy, glad to hear it, as Gemma still isnt so great compared to Phi-2 xD