add gptneox embeddings, fix phi2 inputs, also fix the casting (#1083) 78c5b19 unverified winglian commited on Jan 11
optimize calculation of cu_seqlens from position_ids (#1084) [skip ci] 90036eb unverified winglian commited on Jan 10
additional logging to get maximum token length of a sequence in the dataset (#1066) [skip ci] 2f2582e unverified winglian commited on Jan 10
update sharegpt conversations when chatml chat template is set (#1075) [skip ci] 0ce1a65 unverified winglian commited on Jan 10
fix: `train_on_inputs: true` ignored for sharegpt (#1045) [skip ci] 043c386 unverified Nanobit winglian commited on Jan 10
be more robust about checking embedding modules for lora finetunes (#1074) [skip ci] 0f10080 unverified winglian commited on Jan 10
swap the data collator for evals if not using sample packing (#1076) ead34c5 unverified winglian commited on Jan 10
Add: mlflow for experiment tracking (#1059) [skip ci] 090c24d unverified Johan Hansson winglian commited on Jan 9
Cosine learning rate schedule - minimum learning rate (#1062) 04b978b unverified ricdomolm winglian commited on Jan 9
Efficiently get the length of the tokenized docs (#1063) 81d3845 unverified ricdomolm winglian commited on Jan 8
streaming multipack for pretraining dataset (#959) 553c80f unverified jinwonkim93 jinwonkim93@github.com winglian commited on Jan 6
feat: always push checkpoint to hub if set (#1049) [skip ci] cbdbf9e unverified Nanobit commited on Jan 5
feature: better device mapping for large models (#918) bdfefaf unverified dg-kalle Karl-Johan Alm winglian commited on Jan 5
Added chatglm3 conversation type for training models like TinyLLama (#1036) 59b2d30 unverified xaviviro commited on Jan 4
bump transformers and update attention class map name (#1023) bcc78d8 unverified winglian commited on Jan 3
use recommended setting for use_reentrant w gradient checkpointing (#1021) 4d2e842 unverified winglian commited on Jan 2
Fix: bf16 support for inference (#981) 3678a6c unverified Tazik Shahjahan winglian commited on Dec 29, 2023
[WandB] Push axolotl config to top level wandb files (#1014) 4f4d638 unverified hamel commited on Dec 29, 2023
feat: remove need to add load_in* during merge (#1017) f6ecf14 unverified Nanobit commited on Dec 29, 2023
remove landmark attn and xpos rope implementations (#1010) 70b46ca unverified winglian commited on Dec 28, 2023
FEAT: add tagging support to axolotl (#1004) db9094d unverified Younes Belkada winglian commited on Dec 27, 2023
Feat: Warns to add to modules_to_save when adding tokens or switching special_tokens (#787) 1ffa386 unverified Nanobit commited on Dec 22, 2023
fix: switch to using the HuggingFace Transformers NEFT implementation (#941) ef24342 unverified dg-kalle commited on Dec 13, 2023
fix: remove excessive newlines in system prompt(s) for alpaca (#936) 450e04d unverified dg-kalle commited on Dec 13, 2023
new evals_per_epoch and saves_per_epoch to make things cleaner (#944) 5f79b82 unverified winglian commited on Dec 12, 2023
Respect sequence_len in config for `type: llama2_chat` (#926) f1de29d unverified hamel commited on Dec 12, 2023
Mixtral: More correct MoE, lower loss (#932) 86487c2 unverified casperhansen commited on Dec 10, 2023
winglian
commited on
