microsoft
/

phi-1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

phi-1 / modeling_mixformer_sequential.py

Commit History

Fixes flash-attn import with a try/except statement

9ed5987

gugarosa commited on Nov 1, 2023

Adds support for flash-attn rotary embedding and fused dense layers.

90c38d9

gugarosa commited on Nov 1, 2023

Adds support for MQA/GQA and attention mask during training / fine-tuning.

371fd51

gugarosa commited on Oct 30, 2023

Upload modeling_mixformer_sequential.py

633bca1

gugarosa commited on Sep 27, 2023

fix(phi-1): Checks length of `attention_mask`if it is passed as direct tensor.

1f890f7

gugarosa commited on Sep 26, 2023

Support for `attention_mask` in forward pass.

d22f35e

gugarosa commited on Sep 26, 2023

Upload MixFormerSequentialForCausalLM

44cca9f

suriyagunasekar commited on Sep 11, 2023

Upload MixFormerSequentialForCausalLM

0f4ae0e

suriyagunasekar commited on Sep 10, 2023