mosaicml-mpt-7b-chat-qlora / modeling_mpt.py

Commit History

apply gradient checkpointing to Attention blocks
1e53ac9
unverified

Alex Birch commited on

add support for AutoModelForCausalLM#from_pretrained()'s device_map='auto'. support gradient checkpointing, probably. add lots of type hints so I could understand what's going on. multiline long method signatures/calls (for easier comparison between checkpointed/non-checkpointed variants, and because these lines got even longer when I added type hints). make MPTForCausalLM#forward accept additional kwargs, since PeftModelForCausalLM#forward tries to send it an argument inputs_embeds=None, which it didn't like too much.
9f0a20b
unverified

Alex Birch commited on

Remove print about meta init (#2)
af38791

daking commited on

Add loss function and bool cast
c66b9f7

abhi-mosaic commited on

Upload folder using huggingface_hub
36b0251

sam-mosaic commited on