yurakuratov commited on
Commit
21343b9
1 Parent(s): ba0a672

docs: update usage example

Browse files
Files changed (1) hide show
  1. README.md +28 -4
README.md CHANGED
@@ -21,22 +21,46 @@ Paper: https://www.biorxiv.org/content/10.1101/2023.06.12.544594v1
21
 
22
  ## Examples
23
 
24
- ### Load pre-trained model
25
  ```python
26
  from transformers import AutoTokenizer, AutoModel
27
 
28
  tokenizer = AutoTokenizer.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
29
- model = AutoModel.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
 
 
 
 
 
 
 
30
  ```
31
 
32
- ### How to load the model to fine-tune it on classification task
33
  ```python
34
- from src.gena_lm.modeling_bert import BertForSequenceClassification
35
  from transformers import AutoTokenizer
36
 
37
  tokenizer = AutoTokenizer.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
38
  model = BertForSequenceClassification.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
39
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
41
  ## Model description
42
  GENA-LM (`gena-lm-bert-base-t2t`) model is trained in a masked language model (MLM) fashion, following the methods proposed in the BigBird paper by masking 15% of tokens. Model config for `gena-lm-bert-base-t2t` is similar to the bert-base:
 
21
 
22
  ## Examples
23
 
24
+ ### How to load pre-trained model for Masked Language Modeling
25
  ```python
26
  from transformers import AutoTokenizer, AutoModel
27
 
28
  tokenizer = AutoTokenizer.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
29
+ model = AutoModel.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t', trust_remote_code=True)
30
+
31
+ ```
32
+
33
+ ### How to load pre-trained model to fine-tune it on classification task
34
+ Get model class from GENA-LM repository:
35
+ ```bash
36
+ git clone https://github.com/AIRI-Institute/GENA_LM.git
37
  ```
38
 
 
39
  ```python
40
+ from GENA_LM.src.gena_lm.modeling_bert import BertForSequenceClassification
41
  from transformers import AutoTokenizer
42
 
43
  tokenizer = AutoTokenizer.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
44
  model = BertForSequenceClassification.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t')
45
  ```
46
+ or you can just download [modeling_bert.py](https://github.com/AIRI-Institute/GENA_LM/tree/main/src/gena_lm) and put it close to your code.
47
+
48
+ OR you can get model class from HuggingFace AutoModel:
49
+ ```python
50
+ from transformers import AutoTokenizer, AutoModel
51
+ model = AutoModel.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t', trust_remote_code=True)
52
+ gena_module_name = model.__class__.__module__
53
+ print(gena_module_name)
54
+ import importlib
55
+ # available class names:
56
+ # - BertModel, BertForPreTraining, BertForMaskedLM, BertForNextSentencePrediction,
57
+ # - BertForSequenceClassification, BertForMultipleChoice, BertForTokenClassification,
58
+ # - BertForQuestionAnswering
59
+ # check https://huggingface.co/docs/transformers/model_doc/bert
60
+ cls = getattr(importlib.import_module(gena_module_name), 'BertForSequenceClassification')
61
+ print(cls)
62
+ model = cls.from_pretrained('AIRI-Institute/gena-lm-bert-base-t2t', num_labels=2)
63
+ ```
64
 
65
  ## Model description
66
  GENA-LM (`gena-lm-bert-base-t2t`) model is trained in a masked language model (MLM) fashion, following the methods proposed in the BigBird paper by masking 15% of tokens. Model config for `gena-lm-bert-base-t2t` is similar to the bert-base: