Fixes any potential overflow when calculating attention weights.

Delete modeling_mixformer_sequential.py

Delete configuration_mixformer_sequential.py

Upload pytorch_model.bin

Update to new model interface.

Improves type hinting on configuration arguments.

Fixes flash-attn import with a try/except statement

Adds support for flash-attn rotary embedding and fused dense layers.

Adds support for MQA/GQA and attention mask during training / fine-tuning.

Upload modeling_mixformer_sequential.py

Upload README.md

fix(phi-1): Checks length of `attention_mask`if it is passed as direct tensor.

Support for `attention_mask` in forward pass.

Update README.md

Upload tokenizer

Upload MixFormerSequentialForCausalLM

Update README.md

Update README.md

Update README.md

Update generation_config.json

Update generation_config.json

Update generation_config.json

Update README.md

Update README.md

Update README.md

Update README.md

Upload Research License.docx

Update README.md

Update README.md

Update README.md

Upload tokenizer

Upload MixFormerSequentialForCausalLM

Upload tokenizer

Upload MixFormerSequentialForCausalLM

initial commit