Upload ITERForRelationExtraction

README.md

@@ -0,0 +1,104 @@
+---
+license: apache-2.0
+base_model:
+- microsoft/deberta-v3-large
+library_name: transformers
+tags:
+- relation extraction
+- nlp
+model-index:
+  - name: iter-conll04-deberta-large
+    results:
+      - task:
+          type: relation-extraction
+        dataset:
+          name: conll04
+          type: conll04
+        metrics:
+          - name: F1
+            type: f1
+            value: 77.461
+---
+
+
+# ITER: Iterative Transformer-based Entity Recognition and Relation Extraction
+
+This model checkpoint is part of the collection of models published alongside our paper ITER, 
+[accepted at EMNLP 2024](https://aclanthology.org/2024.findings-emnlp.655/).<br>
+To ease reproducibility and enable open research, our source code has been published on [GitHub](https://github.com/fleonce/iter).
+
+This model achieved an F1 score of `77.461` on dataset `conll04`
+
+### Using ITER in your code
+
+First, install ITER in your preferred environment:
+
+```text
+pip install git+https://github.com/fleonce/iter
+```
+
+To use our model, refer to the following code:
+```python
+from iter import ITERForRelationExtraction
+
+model = ITERForRelationExtraction.from_pretrained("fleonce/iter-conll04-deberta-large")
+tokenizer = model.tokenizer
+
+encodings = tokenizer(
+  "An art exhibit at the Hakawati Theatre in Arab east Jerusalem was a series of portraits of Palestinians killed in the rebellion .",
+  return_tensors="pt"
+)
+
+generation_output = model.generate(
+    encodings["input_ids"],
+    attention_mask=encodings["attention_mask"],
+)
+
+# entities
+print(generation_output.entities)
+
+# relations between entities
+print(generation_output.links)
+```
+
+### Checkpoints
+
+We publish checkpoints for the models performing best on the following datasets:
+
+- **ACE05**:
+  1. [fleonce/iter-ace05-deberta-large](https://huggingface.co/fleonce/iter-ace05-deberta-large)
+- **CoNLL04**:
+  1. [fleonce/iter-conll04-deberta-large](https://huggingface.co/fleonce/iter-conll04-deberta-large)
+- **ADE**:
+  1. [fleonce/iter-ade-deberta-large](https://huggingface.co/fleonce/iter-ade-deberta-large)
+- **SciERC**:
+  1. [fleonce/iter-scierc-deberta-large](https://huggingface.co/fleonce/iter-scierc-deberta-large)
+  2. [fleonce/iter-scierc-scideberta-full](https://huggingface.co/fleonce/iter-scierc-scideberta-full)
+- **CoNLL03**:
+  1. [fleonce/iter-conll03-deberta-large](https://huggingface.co/fleonce/iter-conll03-deberta-large)
+- **GENIA**:
+  1. [fleonce/iter-genia-deberta-large](https://huggingface.co/fleonce/iter-genia-deberta-large)
+
+
+### Reproducibility
+
+For each dataset, we selected the best performing checkpoint out of the 5 training runs we performed during training.
+This model was trained with the following hyperparameters:
+
+- Seed: `1`
+- Config: `conll04/small_lr`
+- PyTorch `2.3.0` with CUDA `11.8` and precision `torch.bfloat16`
+- GPU: `1 NVIDIA H100 SXM 80 GB GPU`
+
+Varying GPU and CUDA version as well as training precision did result in slightly different end results in our tests
+for reproducibility.
+
+To train this model, refer to the following command:
+```shell
+python3 train.py --dataset conll04/small_lr --transformer microsoft/deberta-v3-large --use_bfloat16 --seed 1
+```
+
+```text
+@inproceedings{citation}
+```
+

config.json

@@ -0,0 +1,72 @@
+{
+  "_name_or_path": "models/fleonce/iter-conll04-deberta-large",
+  "activation_fn": "gelu",
+  "architectures": [
+    "ITERForRelationExtraction"
+  ],
+  "d_ff": 4096,
+  "d_model": 1024,
+  "dataset": "conll04",
+  "dropout": 0.3,
+  "entity_types": [
+    "Loc",
+    "Org",
+    "Peop",
+    "Other"
+  ],
+  "features": 0,
+  "link_types": [
+    "Work_For",
+    "Kill",
+    "OrgBased_In",
+    "Live_In",
+    "Located_In"
+  ],
+  "max_length": 512,
+  "max_nest_depth": 1,
+  "model_type": "iter",
+  "num_links": 5,
+  "num_types": 4,
+  "threshold": 0.5,
+  "torch_dtype": "float32",
+  "transformer_config": {
+    "_name_or_path": "microsoft/deberta-v3-large",
+    "architectures": null,
+    "attention_probs_dropout_prob": 0.1,
+    "decoder_start_token_id": null,
+    "eos_token_id": null,
+    "hidden_act": "gelu",
+    "hidden_dropout_prob": 0.1,
+    "hidden_size": 1024,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "is_encoder_decoder": false,
+    "layer_norm_eps": 1e-07,
+    "max_length": 512,
+    "max_position_embeddings": 512,
+    "max_relative_positions": -1,
+    "model_type": "deberta-v2",
+    "norm_rel_ebd": "layer_norm",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 24,
+    "pooler_dropout": 0,
+    "pooler_hidden_act": "gelu",
+    "pooler_hidden_size": 1024,
+    "pos_att_type": [
+      "p2c",
+      "c2p"
+    ],
+    "position_biased_input": false,
+    "position_buckets": 256,
+    "relative_attention": true,
+    "share_att_key": true,
+    "task_specific_params": null,
+    "type_vocab_size": 0,
+    "vocab_size": 128100
+  },
+  "transformers_version": "4.37.0",
+  "use_bias": false,
+  "use_gate": true,
+  "use_mlp": true,
+  "use_scale": false
+}

generation_config.json

@@ -0,0 +1,5 @@
+{
+  "_from_model_config": true,
+  "max_length": 512,
+  "transformers_version": "4.37.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9106a242b0b024c5fe3d58abf2d6e790bf1649aaa44ae8c775a6e19e9c92f026
+size 1904033640