Commits · Dovakiins/qwerrwe

Pretrain multipack v2 (#1470)

5aa5097
unverified

winglian commited on Apr 2

fix pretraining_ on odd datasets (#1463)

586bd8d
unverified

monsoon-nlp commited on Apr 2

reduce verbosity of the special tokens (#1472)

0b10377
unverified

winglian commited on Apr 1

LISA (#1469)

0ddfb24
unverified

winglian

tmm1 commited on Apr 1

qwen2_moe support w multipack (#1455)

6086be8
unverified

winglian commited on Mar 29

fix some of the edge cases for Jamba (#1452)

05b398a
unverified

winglian commited on Mar 29

Support loading datasets saved via save_to_disk (#1432)

e634118
unverified

fozziethebeat commited on Mar 29

Jamba (#1451)

02af082
unverified

winglian commited on Mar 29

fix layer_replication arg to peft (#1446)

4155e99
unverified

winglian commited on Mar 27

support layer replication for peft and fix rslora integration (#1445)

25afd35
unverified

winglian commited on Mar 27

fix for accelerate env var for auto bf16, add new base image and expand torch_cuda_arch_list support (#1413)

da265dd
unverified

winglian commited on Mar 26

Fix falcon tokenization step (#1441) [skip ci]

bcdc9b1
unverified

Far El

winglian commited on Mar 26

make sure to capture non-null defaults from config validation (#1415)

601b77b
unverified

winglian commited on Mar 26

fix(dataset): normalize tokenizer config and change hash from tokenizer class to tokenizer path (#1298)

ff939d8
unverified

Nanobit commited on Mar 25

strip out hacky qlora-fsdp workarounds now that qlora-fsdp fixes are upstreamed (#1428)

2a1589f
unverified

winglian commited on Mar 21

support galore once upstreamed into transformers (#1409)

dd449c5
unverified

winglian commited on Mar 19

Feat: Add sharegpt multirole (#1137)

40a88e8
unverified

Nanobit commited on Mar 19

Add a config not to shuffle merged dataset (#1394) [skip ci]

43bdc5d
unverified

seungduk

winglian commited on Mar 19

fix(config): passing gradient_checkpoint_kwargs (#1412)

b1e3e1b
unverified

Nanobit commited on Mar 19

ORPO (#1419)

2ea70eb
unverified

winglian commited on Mar 18

beta support for multipack with gemmoe: (#1402)

8df7b88
unverified

winglian commited on Mar 14

Train parameters exclusively in specific ranges (#1390)

05bcc9e
unverified

seungduk commited on Mar 14

Update ChatTemplate enum to include alpaca and gemma (#1396)

0976781
unverified

chiragjn commited on Mar 13

chore: lint (#1389)

4326520
unverified

winglian commited on Mar 11

Add Glaive conversation format support (#1365)

b7d8a7d
unverified

Brian Fitzgerald

winglian commited on Mar 11

Fix pydantic configuration for the max_memory input (#1385) [skip ci]

0bc114d
unverified

dandm1

winglian commited on Mar 11

support for rslora (#1387) [skip ci]

7659c00
unverified

winglian commited on Mar 11

validation for fsdp and deepspeed (#1388) [skip ci]

3fd8093
unverified

winglian commited on Mar 11

FDSP + QLoRA (#1378)

9b6ee83
unverified

winglian commited on Mar 8

support for DoRA w/ PEFT (#1363)

0cfdb2c
unverified

winglian commited on Mar 6

lora+ support (#1352)

decb66e
unverified

winglian commited on Mar 5

Fix validation for early stopping (#1358)

b5b4492
unverified

chiragjn commited on Mar 4

fix for protected model_ namespace w pydantic (#1345)

6b3b271
unverified

winglian commited on Feb 28

Fix `use_mlflow` to be bool instead of str (#1344)

3a5a2d2
unverified

chiragjn commited on Feb 28

more fixes 20240228 (#1342) [skip ci]

0f985e1
unverified

winglian commited on Feb 28

add gemma instruct chat template (#1341)

c1a7b3d
unverified

winglian commited on Feb 27

more pydantic fixes (#1338)

3f69571
unverified

winglian commited on Feb 27

Support user-defined prompt processing strategies for dpo (#1248)

1e3d530
unverified

nopperl

winglian commited on Feb 26

add lion-pytorch optimizer (#1299) [skip ci]

1648279
unverified

Maxime

winglian commited on Feb 26

hotfix to exclude_unset from pydantic config when converting back to a dict (#1334)

269c543
unverified

winglian commited on Feb 26

hotfix for missing outputs params (#1333)

e7eed20
unverified

winglian commited on Feb 26

hotfix for lora rank (#1332)

cf00231
unverified

winglian commited on Feb 26

hotfix for capabilities loading (#1331)

7de912e
unverified

winglian commited on Feb 26

ADD: push checkpoints to mlflow artifact registry (#1295) [skip ci]

d756534
unverified

JohanWork

Nanobit

winglian commited on Feb 26

Pydantic 2.x cfg (#1239)

cc3cebf
unverified

winglian commited on Feb 26

make mlflow optional (#1317)

5894f0e
unverified

winglian commited on Feb 26

Validation always happens on first step (#1300)

e2786cc
unverified

LeonardoEmili commited on Feb 21

Add seq2seq eval benchmark callback (#1274)

5a5d474
unverified

LeonardoEmili commited on Feb 13

Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273)

8430db2
unverified

jinwonkim93 commited on Feb 13

Add MPS support (#1264)

fac2d98
unverified

Maxime

winglian commited on Feb 12

Commit History

Pretrain multipack v2 (#1470) 5aa5097 unverified

fix pretraining_ on odd datasets (#1463) 586bd8d unverified

reduce verbosity of the special tokens (#1472) 0b10377 unverified

LISA (#1469) 0ddfb24 unverified

qwen2_moe support w multipack (#1455) 6086be8 unverified

fix some of the edge cases for Jamba (#1452) 05b398a unverified

Support loading datasets saved via save_to_disk (#1432) e634118 unverified

Jamba (#1451) 02af082 unverified

fix layer_replication arg to peft (#1446) 4155e99 unverified

support layer replication for peft and fix rslora integration (#1445) 25afd35 unverified

fix for accelerate env var for auto bf16, add new base image and expand torch_cuda_arch_list support (#1413) da265dd unverified

Fix falcon tokenization step (#1441) [skip ci] bcdc9b1 unverified

make sure to capture non-null defaults from config validation (#1415) 601b77b unverified

fix(dataset): normalize tokenizer config and change hash from tokenizer class to tokenizer path (#1298) ff939d8 unverified

strip out hacky qlora-fsdp workarounds now that qlora-fsdp fixes are upstreamed (#1428) 2a1589f unverified

support galore once upstreamed into transformers (#1409) dd449c5 unverified

Feat: Add sharegpt multirole (#1137) 40a88e8 unverified

Add a config not to shuffle merged dataset (#1394) [skip ci] 43bdc5d unverified

fix(config): passing gradient_checkpoint_kwargs (#1412) b1e3e1b unverified

ORPO (#1419) 2ea70eb unverified

beta support for multipack with gemmoe: (#1402) 8df7b88 unverified

Train parameters exclusively in specific ranges (#1390) 05bcc9e unverified

Update ChatTemplate enum to include alpaca and gemma (#1396) 0976781 unverified

chore: lint (#1389) 4326520 unverified

Add Glaive conversation format support (#1365) b7d8a7d unverified

Fix pydantic configuration for the max_memory input (#1385) [skip ci] 0bc114d unverified

support for rslora (#1387) [skip ci] 7659c00 unverified

validation for fsdp and deepspeed (#1388) [skip ci] 3fd8093 unverified

FDSP + QLoRA (#1378) 9b6ee83 unverified

support for DoRA w/ PEFT (#1363) 0cfdb2c unverified

lora+ support (#1352) decb66e unverified

Fix validation for early stopping (#1358) b5b4492 unverified

fix for protected model_ namespace w pydantic (#1345) 6b3b271 unverified

Fix `use_mlflow` to be bool instead of str (#1344) 3a5a2d2 unverified

more fixes 20240228 (#1342) [skip ci] 0f985e1 unverified

add gemma instruct chat template (#1341) c1a7b3d unverified

more pydantic fixes (#1338) 3f69571 unverified

Support user-defined prompt processing strategies for dpo (#1248) 1e3d530 unverified

add lion-pytorch optimizer (#1299) [skip ci] 1648279 unverified

hotfix to exclude_unset from pydantic config when converting back to a dict (#1334) 269c543 unverified

hotfix for missing outputs params (#1333) e7eed20 unverified

hotfix for lora rank (#1332) cf00231 unverified

hotfix for capabilities loading (#1331) 7de912e unverified

ADD: push checkpoints to mlflow artifact registry (#1295) [skip ci] d756534 unverified

Pydantic 2.x cfg (#1239) cc3cebf unverified

make mlflow optional (#1317) 5894f0e unverified

Validation always happens on first step (#1300) e2786cc unverified

Add seq2seq eval benchmark callback (#1274) 5a5d474 unverified

Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273) 8430db2 unverified

Add MPS support (#1264) fac2d98 unverified

Pretrain multipack v2 (#1470)

5aa5097
unverified

fix pretraining_ on odd datasets (#1463)

586bd8d
unverified

reduce verbosity of the special tokens (#1472)

0b10377
unverified

LISA (#1469)

0ddfb24
unverified

qwen2_moe support w multipack (#1455)

6086be8
unverified

fix some of the edge cases for Jamba (#1452)

05b398a
unverified

Support loading datasets saved via save_to_disk (#1432)

e634118
unverified

Jamba (#1451)

02af082
unverified

fix layer_replication arg to peft (#1446)

4155e99
unverified

support layer replication for peft and fix rslora integration (#1445)

25afd35
unverified

fix for accelerate env var for auto bf16, add new base image and expand torch_cuda_arch_list support (#1413)

da265dd
unverified

Fix falcon tokenization step (#1441) [skip ci]

bcdc9b1
unverified

make sure to capture non-null defaults from config validation (#1415)

601b77b
unverified

fix(dataset): normalize tokenizer config and change hash from tokenizer class to tokenizer path (#1298)

ff939d8
unverified

strip out hacky qlora-fsdp workarounds now that qlora-fsdp fixes are upstreamed (#1428)

2a1589f
unverified

support galore once upstreamed into transformers (#1409)

dd449c5
unverified

Feat: Add sharegpt multirole (#1137)

40a88e8
unverified

Add a config not to shuffle merged dataset (#1394) [skip ci]

43bdc5d
unverified

fix(config): passing gradient_checkpoint_kwargs (#1412)

b1e3e1b
unverified

ORPO (#1419)

2ea70eb
unverified

beta support for multipack with gemmoe: (#1402)

8df7b88
unverified

Train parameters exclusively in specific ranges (#1390)

05bcc9e
unverified

Update ChatTemplate enum to include alpaca and gemma (#1396)

0976781
unverified

chore: lint (#1389)

4326520
unverified

Add Glaive conversation format support (#1365)

b7d8a7d
unverified

Fix pydantic configuration for the max_memory input (#1385) [skip ci]

0bc114d
unverified

support for rslora (#1387) [skip ci]

7659c00
unverified

validation for fsdp and deepspeed (#1388) [skip ci]

3fd8093
unverified

FDSP + QLoRA (#1378)

9b6ee83
unverified

support for DoRA w/ PEFT (#1363)

0cfdb2c
unverified

lora+ support (#1352)

decb66e
unverified

Fix validation for early stopping (#1358)

b5b4492
unverified

fix for protected model_ namespace w pydantic (#1345)

6b3b271
unverified

Fix `use_mlflow` to be bool instead of str (#1344)

3a5a2d2
unverified

more fixes 20240228 (#1342) [skip ci]

0f985e1
unverified

add gemma instruct chat template (#1341)

c1a7b3d
unverified

more pydantic fixes (#1338)

3f69571
unverified

Support user-defined prompt processing strategies for dpo (#1248)

1e3d530
unverified

add lion-pytorch optimizer (#1299) [skip ci]

1648279
unverified

hotfix to exclude_unset from pydantic config when converting back to a dict (#1334)

269c543
unverified

hotfix for missing outputs params (#1333)

e7eed20
unverified

hotfix for lora rank (#1332)

cf00231
unverified

hotfix for capabilities loading (#1331)

7de912e
unverified

ADD: push checkpoints to mlflow artifact registry (#1295) [skip ci]

d756534
unverified

Pydantic 2.x cfg (#1239)

cc3cebf
unverified

make mlflow optional (#1317)

5894f0e
unverified

Validation always happens on first step (#1300)

e2786cc
unverified

Add seq2seq eval benchmark callback (#1274)

5a5d474
unverified

Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273)

8430db2
unverified

Add MPS support (#1264)

fac2d98
unverified