Commits · jinaai/jina-bert-flash-implementation

feat: made from_bert work

851184a

Markus28 commited on Mar 6

feat: choose flash attention heuristically if not set explicitly

2e2b8d0

Markus28 commited on Mar 6

feat: select first LoRA upon initialization

fabeb13

Markus28 commited on Mar 6

feat: formatting and type hints

617fe56

Markus28 commited on Mar 6

support-multiple-task-ids (#5)

e151a8f
verified

Markus28

michael-guenther commited on Mar 6

fix: use proper initilization for embedding layer

850b9a2

Markus28 commited on Mar 6

fix: fixed typo

5c4e4bf

Markus28 commited on Mar 6

feat: added LoRA

8561a1f

Markus28 commited on Mar 6

feat: assert return_dict

326b1c4

Markus28 commited on Mar 5

fix: same assertions in other models

c1d92c9

Markus28 commited on Mar 5

fix: assert is None for other kwargs too

3f5615c

Markus28 commited on Mar 5

feat: added head_mask

599c64e

Markus28 commited on Mar 5

added classifier dropout

767b681

Markus28 commited on Mar 5

fix: formatting

ae4c28c

Markus28 commited on Mar 5

fix: formatting

f115a1d

Markus28 commited on Mar 5

feat: added further GLUE models

ec37ae5

Markus28 commited on Mar 5

feat: added BertForSequenceClassification

ba24fb1

Markus28 commited on Mar 5

fix: cast mask to bool

ca5f516

Markus28 commited on Mar 5

reference the flash attention GitHub

eec6c0e

Markus28 commited on Mar 5

fix: move flash components into top-level

5944ec8

Markus28 commited on Mar 5

feat: try to fix import error

4c4562b

Markus28 commited on Mar 5

feat: moved flash attention code into this repository

46df05d

Markus28 commited on Mar 5

add tokenizer

6343db7

michael-guenther commited on Mar 4

feat: added encode method

32458be

Markus28 commited on Mar 1

fix: try to skip initialization of task type embeddings

3b35eab

Markus28 commited on Mar 1

fix: try to skip initialization of task type embeddings

95ca1a8

Markus28 commited on Mar 1

feat: added option for QK normalization

463061d

Markus28 commited on Mar 1

fix: removed obscure config options

2e69073

Markus28 commited on Mar 1

feat: added small config

149d26f

Markus28 commited on Mar 1

feat: implement task type embeddings (#1)

8adf551
verified

Markus28 commited on Mar 1

feat: added back option not to use flash attention

d4d5621

Markus28 commited on Mar 1

feat: support gradient checkpointing

75d7a16

Markus28 commited on Feb 28

Added additional config options

5b58f09

Markus28 commited on Feb 27

removed unused imports

5e7b835

Markus28 commited on Feb 27

removed init from BertPretrainedModel

44fd417

Markus28 commited on Feb 27

added config_class and base_model_prefix

45b2292

Markus28 commited on Feb 27

Fixed typo

80472cb

Markus28 commited on Feb 27

Fixed typo

6fb6577

Markus28 commited on Feb 27

Try to subclass PretrainedModel

e209593

Markus28 commited on Feb 27

Try to subclass PretrainedModel

2b23340

Markus28 commited on Feb 27

strict=True for debugging

a0c289c

Markus28 commited on Feb 27

try to simplify checkpointing

4c68a4c

Markus28 commited on Feb 27

changed model_type

c35343d

Markus28 commited on Feb 23

feat: added dense_seq_output to config

75a4e4d

Markus28 commited on Feb 22

removed debugging

c2d8dc3

Markus28 commited on Feb 22

debugging

c4185ce

Markus28 commited on Feb 22

debugging

a1e1eff

Markus28 commited on Feb 22

debugging assertion

4d2995d

Markus28 commited on Feb 22

fix: fixed get_input_embeddings method

7e06371

Markus28 commited on Feb 22

feat: added get_input_embeddings method to BertForPreTraining

bb281f0

Markus28 commited on Feb 22

Commit History

feat: made from_bert work 851184a

feat: choose flash attention heuristically if not set explicitly 2e2b8d0

feat: select first LoRA upon initialization fabeb13

feat: formatting and type hints 617fe56

support-multiple-task-ids (#5) e151a8f verified

fix: use proper initilization for embedding layer 850b9a2

fix: fixed typo 5c4e4bf

feat: added LoRA 8561a1f

feat: assert return_dict 326b1c4

fix: same assertions in other models c1d92c9

fix: assert is None for other kwargs too 3f5615c

feat: added head_mask 599c64e

added classifier dropout 767b681

fix: formatting ae4c28c

fix: formatting f115a1d

feat: added further GLUE models ec37ae5

feat: added BertForSequenceClassification ba24fb1

fix: cast mask to bool ca5f516

reference the flash attention GitHub eec6c0e

fix: move flash components into top-level 5944ec8

feat: try to fix import error 4c4562b

feat: moved flash attention code into this repository 46df05d

add tokenizer 6343db7

feat: added encode method 32458be

fix: try to skip initialization of task type embeddings 3b35eab

fix: try to skip initialization of task type embeddings 95ca1a8

feat: added option for QK normalization 463061d

fix: removed obscure config options 2e69073

feat: added small config 149d26f

feat: implement task type embeddings (#1) 8adf551 verified

feat: added back option not to use flash attention d4d5621

feat: support gradient checkpointing 75d7a16

Added additional config options 5b58f09

removed unused imports 5e7b835

removed __init__ from BertPretrainedModel 44fd417

added config_class and base_model_prefix 45b2292

Fixed typo 80472cb

Fixed typo 6fb6577

Try to subclass PretrainedModel e209593

Try to subclass PretrainedModel 2b23340

strict=True for debugging a0c289c

try to simplify checkpointing 4c68a4c

changed model_type c35343d

feat: added dense_seq_output to config 75a4e4d

removed debugging c2d8dc3

debugging c4185ce

debugging a1e1eff

debugging assertion 4d2995d

fix: fixed get_input_embeddings method 7e06371

feat: added get_input_embeddings method to BertForPreTraining bb281f0

feat: made from_bert work

851184a

feat: choose flash attention heuristically if not set explicitly

2e2b8d0

feat: select first LoRA upon initialization

fabeb13

feat: formatting and type hints

617fe56

support-multiple-task-ids (#5)

e151a8f
verified

fix: use proper initilization for embedding layer

850b9a2

fix: fixed typo

5c4e4bf

feat: added LoRA

8561a1f

feat: assert return_dict

326b1c4

fix: same assertions in other models

c1d92c9

fix: assert is None for other kwargs too

3f5615c

feat: added head_mask

599c64e

added classifier dropout

767b681

fix: formatting

ae4c28c

fix: formatting

f115a1d

feat: added further GLUE models

ec37ae5

feat: added BertForSequenceClassification

ba24fb1

fix: cast mask to bool

ca5f516

reference the flash attention GitHub

eec6c0e

fix: move flash components into top-level

5944ec8

feat: try to fix import error

4c4562b

feat: moved flash attention code into this repository

46df05d

add tokenizer

6343db7

feat: added encode method

32458be

fix: try to skip initialization of task type embeddings

3b35eab

fix: try to skip initialization of task type embeddings

95ca1a8

feat: added option for QK normalization

463061d

fix: removed obscure config options

2e69073

feat: added small config

149d26f

feat: implement task type embeddings (#1)

8adf551
verified

feat: added back option not to use flash attention

d4d5621

feat: support gradient checkpointing

75d7a16

Added additional config options

5b58f09

removed unused imports

5e7b835

removed init from BertPretrainedModel

44fd417

added config_class and base_model_prefix

45b2292

Fixed typo

80472cb

Fixed typo

6fb6577

Try to subclass PretrainedModel

e209593

Try to subclass PretrainedModel

2b23340

strict=True for debugging

a0c289c

try to simplify checkpointing

4c68a4c

changed model_type

c35343d

feat: added dense_seq_output to config

75a4e4d

removed debugging

c2d8dc3

debugging

c4185ce

debugging

a1e1eff

debugging assertion

4d2995d

fix: fixed get_input_embeddings method

7e06371

feat: added get_input_embeddings method to BertForPreTraining

bb281f0