Commit History
address PR feedback
0c6f928
winglian
commited on
add streaming dataset support for pretraining datasets
eea2731
winglian
commited on
more tweaks to do pre-training with bettertransformers
1210dc8
winglian
commited on
experimental expansion of ctx len
488a67d
winglian
commited on
add flash attn context for efficient training and attempt setting model to train mode:
8792199
winglian
commited on
add support for opimum bettertransformers
1edc30c
winglian
commited on
formatting for linter
f36e227
unverified
winglian
commited on
Add streaming inference & fix stopping at EOS
fec6bcc
Glavin001
commited on
Set matmul tf32
52765ac
Nanobit
commited on
new prompters, misc fixes for output dir missing using fsdp, and changing max seq len
4ac9e25
winglian
commited on
fix device map
74ebbf4
winglian
commited on
fix batch size calculation
5a631b3
winglian
commited on
Merge pull request #119 from NanoCode012/feat/update-inference
fac4600
unverified
Nanobit
commited on
Merge pull request #120 from OpenAccess-AI-Collective/model-from-path
c7021e1
unverified
winglian
commited on
black formatting
6fa40bf
winglian
commited on
add support for gradient accumulation steps
3aad5f3
winglian
commited on
fix up tokenizer config, isort fix
39a208c
winglian
commited on
Feat: Swap to GenerationConfig
988aeb9
Nanobit
commited on
Merge pull request #108 from OpenAccess-AI-Collective/docker-gptq
bbc5bc5
unverified
winglian
commited on
Fix security issue or ignore false positives
a1f9850
Nanobit
commited on
Apply isort then black
37293dc
Nanobit
commited on
Delete extract_lora.py
96e8378
Nanobit
commited on
Fix mypy typing
e9650d3
Nanobit
commited on
Lint finetune.py
82971e1
Nanobit
commited on
Lint and format
392dfd9
Nanobit
commited on
default to qlora support, make gptq specific image
6ef96f5
winglian
commited on
bnb fixes
21f17cc
winglian
commited on
refactor: fix previous refactors
56f9ca5
Nanobit
commited on
Refactor to use DictDefault instead
8bd7a49
Nanobit
commited on
Fix load error
93acb64
Nanobit
commited on
Convert attrdict to addict
bdfe7c9
Nanobit
commited on
move list not in list logic to fn
cc67862
winglian
commited on
load the tokenizer seperately from the model
32e6fe9
winglian
commited on
add logging and make sure model unloads to float16
a5bf838
winglian
commited on
remove un-needed code, add validation
1f5d83e
winglian
commited on
optionally be able to specify alpaca or chat style prompts
1d5ab84
winglian
commited on
add alpaca multiple choice instruct dataset support
b46bc02
winglian
commited on
reorder options so debug can happen in the same prepare step
f98e173
winglian
commited on
more fixes
bdbca8f
winglian
commited on
move filter to before saving so it doesn't happen everytime, update runpod manual script
0d28df0
winglian
commited on
Fix typo
52aada7
unverified
Nanobit
commited on
black formatting
2bc1a5b
winglian
commited on
Update finetune.py
915c56c
unverified
winglian
commited on
Don't save full model for lora
cd23959
unverified
Nanobit
commited on
Save adapter for lora
71a1f7f
unverified
Nanobit
commited on
setup runpod images
79deb35
winglian
commited on