--- license: apache-2.0 tags: - generated_from_trainer datasets: - private base_model: t5-large model-index: - name: ner-news-t5-large results: [] --- # T5-Encoder(T5-large model) fine-tuned on very small dataset for token classification Simple experimental model that was trained in 3 epochs on very small dataset ## Usage ```python from transformers import AutoTokenizer, AutoModelForTokenClassification, NerPipeline model = AutoModelForTokenClassification.from_pretrained("imvladikon/t5-english-ner", trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained("imvladikon/t5-english-ner", trust_remote_code=True) pipe = NerPipeline(model=model, tokenizer=tokenizer, aggregation_strategy="max") print(pipe("London is the capital city of England and the United Kingdom")) """ [{'entity_group': 'LOCATION', 'score': 0.84536326, 'word': 'London', 'start': 0, 'end': 6}, {'entity_group': 'LOCATION', 'score': 0.8957489, 'word': 'England', 'start': 30, 'end': 37}, {'entity_group': 'LOCATION', 'score': 0.73186326, 'word': 'UnitedKingdom', 'start': 46, 'end': 60}] """ ``` ## Usage in spacy ```bash pip install spacy transformers git+https://github.com/explosion/spacy-huggingface-pipelines -q ``` ```python import spacy from spacy import displacy text = "My name is Sarah and I live in London" nlp = spacy.blank("en") nlp.add_pipe("hf_token_pipe", config={"model": "imvladikon/t5-english-ner", "kwargs": {"trust_remote_code":True}}) doc = nlp(text) print(doc.ents) # (Sarah, London) ``` This model is a fine-tuned version of [t5-large](https://huggingface.co/t5-large) on the private(en) dataset. It achieves the following results on the evaluation set: - Loss: 0.1956 - Commercial Item Precision: 0.0 - Commercial Item Recall: 0.0 - Commercial Item F1: 0.0 - Commercial Item Number: 1 - Date Precision: 0.8125 - Date Recall: 0.9286 - Date F1: 0.8667 - Date Number: 14 - Location Precision: 0.7143 - Location Recall: 0.75 - Location F1: 0.7317 - Location Number: 20 - Organization Precision: 0.8588 - Organization Recall: 0.9125 - Organization F1: 0.8848 - Organization Number: 80 - Other Precision: 0.3684 - Other Recall: 0.3333 - Other F1: 0.35 - Other Number: 21 - Person Precision: 0.8182 - Person Recall: 0.9310 - Person F1: 0.8710 - Person Number: 29 - Quantity Precision: 0.8 - Quantity Recall: 0.8571 - Quantity F1: 0.8276 - Quantity Number: 14 - Title Precision: 0.0 - Title Recall: 0.0 - Title F1: 0.0 - Title Number: 7 - Overall Precision: 0.75 - Overall Recall: 0.7903 - Overall F1: 0.7696 - Overall Accuracy: 0.9534 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 2 - eval_batch_size: 2 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 3.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | Commercial Item Precision | Commercial Item Recall | Commercial Item F1 | Commercial Item Number | Date Precision | Date Recall | Date F1 | Date Number | Location Precision | Location Recall | Location F1 | Location Number | Organization Precision | Organization Recall | Organization F1 | Organization Number | Other Precision | Other Recall | Other F1 | Other Number | Person Precision | Person Recall | Person F1 | Person Number | Quantity Precision | Quantity Recall | Quantity F1 | Quantity Number | Title Precision | Title Recall | Title F1 | Title Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:-------------------------:|:----------------------:|:------------------:|:----------------------:|:--------------:|:-----------:|:-------:|:-----------:|:------------------:|:---------------:|:-----------:|:---------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------:|:------------:|:--------:|:------------:|:----------------:|:-------------:|:---------:|:-------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:-----------------:|:--------------:|:----------:|:----------------:| | 0.8868 | 1.0 | 708 | 0.2725 | 0.0 | 0.0 | 0.0 | 1 | 0.8125 | 0.9286 | 0.8667 | 14 | 0.4167 | 0.75 | 0.5357 | 20 | 0.8272 | 0.8375 | 0.8323 | 80 | 1.0 | 0.0476 | 0.0909 | 21 | 0.8438 | 0.9310 | 0.8852 | 29 | 0.6667 | 0.7143 | 0.6897 | 14 | 0.0 | 0.0 | 0.0 | 7 | 0.7348 | 0.7151 | 0.7248 | 0.9446 | | 0.2984 | 2.0 | 1416 | 0.2121 | 0.0 | 0.0 | 0.0 | 1 | 0.8667 | 0.9286 | 0.8966 | 14 | 0.5 | 0.8 | 0.6154 | 20 | 0.8375 | 0.8375 | 0.8375 | 80 | 0.3077 | 0.1905 | 0.2353 | 21 | 0.8182 | 0.9310 | 0.8710 | 29 | 0.7333 | 0.7857 | 0.7586 | 14 | 0.0 | 0.0 | 0.0 | 7 | 0.7077 | 0.7419 | 0.7244 | 0.9481 | | 0.1729 | 3.0 | 2124 | 0.1956 | 0.0 | 0.0 | 0.0 | 1 | 0.8125 | 0.9286 | 0.8667 | 14 | 0.7143 | 0.75 | 0.7317 | 20 | 0.8588 | 0.9125 | 0.8848 | 80 | 0.3684 | 0.3333 | 0.35 | 21 | 0.8182 | 0.9310 | 0.8710 | 29 | 0.8 | 0.8571 | 0.8276 | 14 | 0.0 | 0.0 | 0.0 | 7 | 0.75 | 0.7903 | 0.7696 | 0.9534 | ### Framework versions - Transformers 4.21.1 - Pytorch 1.12.0+cu113 - Datasets 2.4.0 - Tokenizers 0.12.1 ## WANDB [training logs and reports](https://wandb.ai/imvladikon/huggingface/runs/uyl6ihl1)