initial commit

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,157 @@
+---
+license: mit
+language:
+- fr
+metrics:
+- seqeval
+library_name: transformers
+pipeline_tag: token-classification
+tags:
+- medical
+- biomedical
+- medkit-lib
+widget:
+  - text: >-
+      La radiographie et la tomodensitométrie ont montré des micronodules diffus
+    example_title: example 1
+  - text: >-
+      Elle souffre d'asthme mais n'a pas besoin d'Allegra
+    example_title: example 2
+---
+
+
+# DrBERT-CASM2
+
+## Model description
+
+**DrBERT-CASM2** is a French Named Entity Recognition model that was fine-tuned from 
+[DrBERT](https://huggingface.co/Dr-BERT/DrBERT-4GB-CP-PubMedBERT): A PreTrained model in French for biomedical and clinical domains. 
+It has been trained  to detect the following type of entities: **problem**, **treatment** and **test** using the medkit Trainer.
+
+- **Fine-tuned using** medkit [GitHub Repo](https://github.com/TeamHeka/medkit)
+- **Developed by** @camila-ud, medkit, HeKA Research team
+- **Dataset source**
+
+  Annotated version from @aneuraz called 'corpusCasM2: A corpus of annotated clinical texts'
+  - The annotation was performed collaborativelly by the students of masters students from Université Paris Cité.
+
+  - The corpus contains documents from CAS:
+    ```
+    Natalia Grabar, Vincent Claveau, and Clément Dalloux. 2018. CAS: French Corpus with Clinical Cases.
+    In Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis,
+    pages 122–128, Brussels, Belgium. Association for Computational Linguistics.
+    ```
+# Intended uses & limitations
+
+## Limitations and bias
+
+This model was trained for **development and test phases**. 
+This model is limited by its training dataset, and it should be used with caution. 
+The results are not guaranteed, and the model should be used only in data exploration stages.
+The model may be able to detect entities in the early stages of the analysis of medical documents in French.
+
+The maximum token size was reduced to **128 tokens** to minimize training time.
+
+# How to use
+
+## Install medkit
+
+First of all, please install medkit with the following command:
+
+```
+pip install 'medkit-lib[optional]'
+```
+
+Please check the [documentation](https://medkit.readthedocs.io/en/latest/user_guide/install.html) for more info and examples.
+
+## Using the model
+
+```python
+from medkit.core.text import TextDocument
+from medkit.text.ner.hf_entity_matcher import HFEntityMatcher
+
+matcher = HFEntityMatcher(model="medkit/DrBERT-CASM2")
+
+test_doc = TextDocument("Elle souffre d'asthme mais n'a pas besoin d'Allegra")
+detected_entities = matcher.run([test_doc.raw_segment])
+
+# show information
+msg = "|".join(f"'{entity.label}':{entity.text}" for entity in detected_entities)
+print(f"Text: '{test_doc.text}'\n{msg}")
+```
+```
+Text: "Elle souffre d'asthme mais n'a pas besoin d'Allegra"
+'problem':asthme|'treatment':Allegra
+```
+
+# Training data 
+
+This model was fine-tuned on **CASM2**, an internal corpus with clinical cases (in french) annotated by master students. 
+The corpus contains more than 5000 medkit documents (~ phrases) with entities to detect. 
+
+**Number of documents (~ phrases) by split**
+
+| Split      | # medkit docs |
+| ---------- | ------------- |
+| Train      | 5824          |
+| Validation | 1457          |
+| Test       | 1821          |
+
+
+**Number of examples per entity type**
+
+| Split      | treatment | test | problem |
+| ---------- | --------- | ---- | ------- |
+| Train      | 3258      | 3990 | 6808    |
+| Validation | 842       | 1007 | 1745    |
+| Test       | 994       | 1289 | 2113    |
+
+## Training procedure
+
+This model was fine-tuned using the medkit trainer on CPU, it takes about 3h.
+
+# Model perfomances
+
+Model performances computes on CASM2 test dataset (using medkit seqeval evaluator)
+
+Entity|precision|recall|f1
+-|-|-|-
+treatment|0.7492|0.7666|0.7578
+test|0.7449|0.8240|0.7824
+problem|0.6884|0.7304|0.7088
+Overall|0.7188|0.7660|0.7416
+
+## How to evaluate using medkit
+```python
+from medkit.text.metrics.ner import SeqEvalEvaluator
+
+# load the matcher and get predicted entities by document
+matcher = HFEntityMatcher(model="medkit/DrBERT-CASM2")
+predicted_entities = [matcher.run([doc.raw_segment]) for doc in test_documents]
+
+evaluator  = SeqEvalEvaluator(tagging_scheme="iob2")
+evaluator.compute(test_documents,predicted_entities=predicted_entities)
+```
+You can use the tokenizer from HF to evaluate by tokens instead of characters 
+```python
+from transformers import AutoTokenizer
+
+tokenizer_drbert = AutoTokenizer.from_pretrained("medkit/DrBERT-CASM2", use_fast=True)
+
+evaluator  = SeqEvalEvaluator(tokenizer=tokenizer_drbert,tagging_scheme="iob2")
+evaluator.compute(test_documents,predicted_entities=predicted_entities)
+```
+
+# Citation
+
+```
+@online{medkit-lib,
+  author={HeKA Research Team},
+  title={medkit, A Python library for a learning health system},
+  url={https://pypi.org/project/medkit-lib/},
+  urldate = {2023-07-24}, 
+}
+```
+```
+HeKA Research Team, “medkit, a Python library for a learning health system.” https://pypi.org/project/medkit-lib/ (accessed Jul. 24, 2023).
+```

config.json

@@ -0,0 +1,43 @@
+{
+  "_name_or_path": "dcariasvi/DrBERT-CASM2",
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "O",
+    "1": "B-problem",
+    "2": "I-problem",
+    "3": "B-treatment",
+    "4": "I-treatment",
+    "5": "B-test",
+    "6": "I-test"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B-problem": 1,
+    "B-test": 5,
+    "B-treatment": 3,
+    "I-problem": 2,
+    "I-test": 6,
+    "I-treatment": 4,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.26.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

history.json

@@ -0,0 +1 @@
+[{"train": {"loss": 0.4125201638787985}, "eval": {"loss": 0.28105667685968394, "overall_precision": 0.6288554677542697, "overall_recall": 0.6964991530208922, "overall_f1-score": 0.6609511051574012, "overall_support": "3542", "overall_acc": 0.9027570584010166, "problem_precision": 0.6217303822937625, "problem_recall": 0.6821192052980133, "problem_f1-score": 0.6505263157894737, "problem_support": "1812", "test_precision": 0.6564245810055865, "test_recall": 0.7524012806830309, "test_f1-score": 0.7011437095972153, "test_support": "937", "treatment_precision": 0.610917537746806, "treatment_recall": 0.6633039092055486, "treatment_f1-score": 0.6360338573155987, "treatment_support": "793"}}, {"train": {"loss": 0.2762815960689264}, "eval": {"loss": 0.2743868621309166, "overall_precision": 0.6495202558635395, "overall_recall": 0.6880293619424054, "overall_f1-score": 0.6682204551686318, "overall_support": "3542", "overall_acc": 0.9052710094480358, "problem_precision": 0.633705475810739, "problem_recall": 0.6578366445916115, "problem_f1-score": 0.6455456268616301, "problem_support": "1812", "test_precision": 0.6851851851851852, "test_recall": 0.7502668089647813, "test_f1-score": 0.7162506367804381, "test_support": "937", "treatment_precision": 0.6414201183431952, "treatment_recall": 0.6834804539722572, "treatment_f1-score": 0.6617826617826619, "treatment_support": "793"}}, {"train": {"loss": 0.24771345957834906}, "eval": {"loss": 0.271967472341519, "overall_precision": 0.6427855711422845, "overall_recall": 0.7244494635798984, "overall_f1-score": 0.6811786567560393, "overall_support": "3542", "overall_acc": 0.9052710094480358, "problem_precision": 0.6335992023928215, "problem_recall": 0.7014348785871964, "problem_f1-score": 0.6657936092194865, "problem_support": "1812", "test_precision": 0.6627379873073436, "test_recall": 0.7801494130202775, "test_f1-score": 0.7166666666666668, "test_support": "937", "treatment_precision": 0.638731596828992, "treatment_recall": 0.7112232030264817, "treatment_f1-score": 0.6730310262529833, "treatment_support": "793"}}, {"train": {"loss": 0.22672327848620155}, "eval": {"loss": 0.2861445434745806, "overall_precision": 0.6255808266079727, "overall_recall": 0.7221908526256352, "overall_f1-score": 0.670423273489713, "overall_support": "3542", "overall_acc": 0.8994695839549146, "problem_precision": 0.6085686465433301, "problem_recall": 0.6898454746136865, "problem_f1-score": 0.6466632177961718, "problem_support": "1812", "test_precision": 0.666970802919708, "test_recall": 0.7801494130202775, "test_f1-score": 0.719134284308903, "test_support": "937", "treatment_precision": 0.6144834930777423, "treatment_recall": 0.7276166456494325, "treatment_f1-score": 0.6662817551963048, "treatment_support": "793"}}, {"train": {"loss": 0.2031083636096723}, "eval": {"loss": 0.2810970079027238, "overall_precision": 0.6511976047904192, "overall_recall": 0.7368718238283456, "overall_f1-score": 0.6913907284768213, "overall_support": "3542", "overall_acc": 0.9046356152273606, "problem_precision": 0.6431761786600496, "problem_recall": 0.7152317880794702, "problem_f1-score": 0.6772929187353018, "problem_support": "1812", "test_precision": 0.663963963963964, "test_recall": 0.7865528281750267, "test_f1-score": 0.7200781631656081, "test_support": "937", "treatment_precision": 0.6534541336353341, "treatment_recall": 0.7276166456494325, "treatment_f1-score": 0.68854415274463, "treatment_support": "793"}}]

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ebaffebd7050c6fb415bf3ca942d36bcb5feab62c817138c1bf57b2eca6a72e
+size 435615652

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3526e3c39fd476e1004854f34ad46f2088a51998aff0d7a25aa9e2f25c6c3146
+size 435655729

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 128,
+  "name_or_path": "Dr-BERT/DrBERT-4GB-CP-PubMedBERT",
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "special_tokens_map_file": null,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

trainer_config.yml

@@ -0,0 +1,9 @@
+learning_rate: 5.0e-06
+nb_training_epochs: 5
+dataloader_nb_workers: 0
+batch_size: 4
+seed: 0
+gradient_accumulation_steps: 1
+do_metrics_in_training: false
+metric_to_track_lr: loss
+log_step_interval: 100