Thomas Lemberger commited on
Commit
15ca687
1 Parent(s): 5a5c89b
Files changed (1) hide show
  1. README.md +83 -0
README.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - english
4
+ thumbnail:
5
+ tags:
6
+ - token classification
7
+ -
8
+ license:
9
+ datasets:
10
+ - EMBO/sd-nlp `PANELIZATION`
11
+ metrics:
12
+ -
13
+ ---
14
+
15
+ # sd-ner
16
+
17
+ ## Model description
18
+
19
+ This model is a [RoBERTa base model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang). It was then fine-tuned for token classification on the SourceData [sd-nlp](https://huggingface.co/datasets/EMBO/sd-nlp) dataset with the `PANELIZATION` task to perform 'parsing' or 'segmentation' of figure legends into fragments corresponding to sub-panels.
20
+
21
+ Figures are usually composite representations of results obtained with heterogenous experimental approaches and systems. Breaking figures into panels allows to identify more coherent descriptions of individual scientific experiments.
22
+
23
+ ## Intended uses & limitations
24
+
25
+ #### How to use
26
+
27
+ The intended use of this model is for 'parsing' figure legends into sub-fragments corresponding to individual panels as used in SourceData annotations (https://sourcedata.embo.org).
28
+
29
+ To have a quick check of the model:
30
+
31
+ ```python
32
+ from transformers import pipeline, RobertaTokenizerFast, RobertaForTokenClassification
33
+ example = """<s> Figure 1.Genetic screens reveal that ZBTB7A and ZBTB7B regulate ATXN1, and ZBTB7B shows more pronounced effect. (A) Western blot analysis of Atxn1 levels in cerebellar granule neurons (CGNs) after knockdown of Zbtb7a and/or Zbtb7b (n = 3).(B) Genetic interaction of wild‐type human ATXN1(30Q) with either ZBTB7A or ZBTB7B expressed in the Drosophila eyes. Co‐expression of ATXN1(30Q) with ZBTB7A or ZBTB7B severely disrupted the external Drosophila eye structure and increased ATXN1 levels. Scale bar: 100 μm. </s>"""
34
+ tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base', max_len=512)
35
+ model = RobertaForTokenClassification.from_pretrained('EMBO/sd-panels')
36
+ ner = pipeline('ner', model, tokenizer=tokenizer)
37
+ res = ner(example)
38
+ for r in res: print(r['word'], r['entity'])
39
+ ```
40
+
41
+ #### Limitations and bias
42
+
43
+ The model must be used with the `roberta-base` tokenizer.
44
+
45
+ ## Training data
46
+
47
+ The model was trained for token classification using the [EMBO/sd-nlp `PANELIZATION`](https://huggingface.co/datasets/EMBO/sd-nlp) dataset wich includes manually annotated examples.
48
+
49
+ ## Training procedure
50
+
51
+ The training was run on a NVIDIA DGX Station with 4XTesla V100 GPUs.
52
+
53
+ Training code is available at https://github.com/source-data/soda-roberta
54
+
55
+ - Command: `python -m tokcl.train PANELIZATION --num_train_epochs=6`
56
+ - Tokenizer vocab size: 50265
57
+ - Training data: EMBO/sd-nlp NER
58
+ - Training with 8234 examples.
59
+ - Evaluating on 2338 examples.
60
+ - Training on 2 features: `O`, `B-PANEL_START`
61
+ - Epochs: 6.0
62
+ - `per_device_train_batch_size`: 32
63
+ - `per_device_eval_batch_size`: 32
64
+ - `learning_rate`: 0.0001
65
+ - `weight_decay`: 0.0
66
+ - `adam_beta1`: 0.9
67
+ - `adam_beta2`: 0.999
68
+ - `adam_epsilon`: 1e-08
69
+ - `max_grad_norm`: 1.0
70
+
71
+ ## Eval results
72
+
73
+ Testing on 1131 examples from test set with `sklearn.metrics`:
74
+
75
+ ```
76
+ precision recall f1-score support
77
+
78
+ PANEL_START 0.92 0.95 0.94 3285
79
+
80
+ micro avg 0.92 0.95 0.94 3285
81
+ macro avg 0.92 0.95 0.94 3285
82
+ weighted avg 0.92 0.95 0.94 3285
83
+ ```