Hugging Face's logo

jinaai
/
jina-bert-flash-implementation

Transformers
bert
custom_code
Inference Endpoints
🇪🇺 Region: EU
Model card Files Community
18
jina-bert-flash-implementation

Commit History

reference the flash attention GitHub
eec6c0e

Markus28 commited on

fix: move flash components into top-level
5944ec8

Markus28 commited on

feat: try to fix import error
4c4562b

Markus28 commited on

feat: moved flash attention code into this repository
46df05d

Markus28 commited on

feat: added encode method
32458be

Markus28 commited on

fix: try to skip initialization of task type embeddings
3b35eab

Markus28 commited on

fix: try to skip initialization of task type embeddings
95ca1a8

Markus28 commited on

feat: added option for QK normalization
463061d

Markus28 commited on

fix: removed obscure config options
2e69073

Markus28 commited on

feat: added small config
149d26f

Markus28 commited on

feat: implement task type embeddings (#1)
8adf551
verified

Markus28 commited on

feat: added back option not to use flash attention
d4d5621

Markus28 commited on

feat: support gradient checkpointing
75d7a16

Markus28 commited on

Added additional config options
5b58f09

Markus28 commited on

removed unused imports
5e7b835

Markus28 commited on

removed __init__ from BertPretrainedModel
44fd417

Markus28 commited on

added config_class and base_model_prefix
45b2292

Markus28 commited on

Fixed typo
80472cb

Markus28 commited on

Fixed typo
6fb6577

Markus28 commited on

Try to subclass PretrainedModel
e209593

Markus28 commited on

Try to subclass PretrainedModel
2b23340

Markus28 commited on

strict=True for debugging
a0c289c

Markus28 commited on

try to simplify checkpointing
4c68a4c

Markus28 commited on

changed model_type
c35343d

Markus28 commited on

feat: added dense_seq_output to config
75a4e4d

Markus28 commited on

removed debugging
c2d8dc3

Markus28 commited on

debugging
c4185ce

Markus28 commited on

debugging
a1e1eff

Markus28 commited on

debugging assertion
4d2995d

Markus28 commited on

fix: fixed get_input_embeddings method
7e06371

Markus28 commited on

feat: added get_input_embeddings method to BertForPreTraining
bb281f0

Markus28 commited on

feat: fixed _from_config
18eed80

Markus28 commited on

feat: changed model_type
eeb05a3

Markus28 commited on

removed from_config
0ce78aa

Markus28 commited on

fix: try to get from_config to work
871fd36

Markus28 commited on

feat: added from_config, also pass additional kwargs from config to model
4164fd6

Markus28 commited on

feat: updated modeling_bert.py to allow MLM-only training
0f43653

Markus28 commited on

feat: reverted monkey patch
3160695

Markus28 commited on

feat: try to monkey-patch index_first_axis
ed92835

Markus28 commited on

feat: try to fix compilation
03d8e7c

Markus28 commited on

feat: added debug print
63832b9

Markus28 commited on

feat: updated .to() override to handle kwargs
e86d612

Markus28 commited on

Revert "feat updated debug pring"
a62c2ab

Markus28 commited on

feat updated debug pring
d21ee1b

Markus28 commited on

fix: try to fix .to(torch.float16) with ALiBi
adf376f

Markus28 commited on

Revert "feat: added back option to disable flash attention"
b7ee9c4

Markus28 commited on

feat: added back option to disable flash attention
a2c07ba

Markus28 commited on

fix: always use flash attention
bfc0b2d

Markus28 commited on