Commit History
Merge pull request #159 from AngainorDev/patch-1
8e568bb
unverified
add typehints
c7dee56
add new sharegpt, refactor prompt so it can be customized later, add exception if no data is processed
aac4b76
add check for attr
c9a149f
new validation for mpt w grad checkpoints
14668fa
Fix strict and Lint
b565ecf
match up gradient checkpointing when using lora w config
fe0b768
Fix set mem_id for inference and refactor
974dc00
Clean up landmark patching
a6190c8
Fix undefined LlamaForCausalLM and del try except
563b6d8
peft no longer needs device_map
cd0a6f6
Refactor landmark attention patch
919727b
fix formatting
958da70
Fix missing cfg.
a808bf9
unverified
Angainor Development
commited on
Merge pull request #182 from OpenAccess-AI-Collective/fix-llama-ref
0124825
unverified
address PR feedback
0c6f928
add streaming dataset support for pretraining datasets
eea2731
more gpt-neox long ctx fixes
ab5cd28
fix bettertransformers save, force it to skip after saving correctly in callback
1a82082
more tweaks to do pre-training with bettertransformers
1210dc8
experimental expansion of ctx len
488a67d
add validation/warning for bettertransformers and torch version
71a43f8
add support for opimum bettertransformers
1edc30c
fix for local variable 'LlamaForCausalLM' referenced before assignment
14163c1
Merge branch 'main' into patch-1
79e2a6f
unverified
Angainor Development
commited on
add support to extend context with xpos rope
a03a7d7
fix for max sequence len across different model types
7f09106
Fix backward compat for peft
aefb2fc
WIP: Rely on cfg.inference
813cfa4
unverified
Angainor Development
commited on
Fix grad checkpoint and outputs param
2a801b0
Fix patching via import instead of hijacking
e44c9e0
Feat: Add landmark attention
55b8542
Disable Wandb
f4df266
Bruno Cabral
commited on
Refactor out unmodified save_steps and eval_steps
2ef4634
Set to use cfg.seed or 42 for backward compat
2cfe9e9
Fix failing test
bfd27ba
Validate falcon with fsdp
babf0fd
Fix future deprecate prepare_model_for_int8_training
df9528f
Fix training over existing lora
193c73b
unverified
Angainor Development
commited on