tiiuae

/

falcon-40b

like 2.4k

Hi @Rocketknight1 @sgugger ,

Thanks for your work on Falcon support in transformers!

Could you please share if this model is going to be converted to the new in-library format (model_type="falcon") soon? If yes, when do you expect this to happen?

I'm asking because I'm working on Falcon support in Petals (https://github.com/bigscience-workshop/petals) and would like to understand (1) which format we need to support
and (2) if we can make a release with Falcon this week or should wait until the format changes.

I'd really appreciate you sharing your plans on this matter!

The model will be converted back to the new in-library format soon. The problem is that the library test-generation-inference added support for Falcon by checking its model_type and we need to wait for a new release of TGI that supports both model types (falcon and the current RefinedWeb) to avoid breaking many users deployment of Falcon through TGI.

Got it, thanks!