2022-11-16 17:06:48,577 - __main__ - INFO - Label List:['O', 'B-PERSON', 'I-PERSON', 'B-NORP', 'I-NORP', 'B-FAC', 'I-FAC', 'B-ORG', 'I-ORG', 'B-GPE', 'I-GPE', 'B-LOC', 'I-LOC', 'B-PRODUCT', 'I-PRODUCT', 'B-DATE', 'I-DATE', 'B-TIME', 'I-TIME', 'B-PERCENT', 'I-PERCENT', 'B-MONEY', 'I-MONEY', 'B-QUANTITY', 'I-QUANTITY', 'B-ORDINAL', 'I-ORDINAL', 'B-CARDINAL', 'I-CARDINAL', 'B-EVENT', 'I-EVENT', 'B-WORK_OF_ART', 'I-WORK_OF_ART', 'B-LAW', 'I-LAW', 'B-LANGUAGE', 'I-LANGUAGE'] 2022-11-16 17:06:55,367 - __main__ - INFO - Dataset({ features: ['id', 'words', 'ner_tags'], num_rows: 75187 }) 2022-11-16 17:06:56,175 - __main__ - INFO - Dataset({ features: ['id', 'words', 'ner_tags'], num_rows: 9479 }) 2022-11-16 17:06:56,185 - transformers.tokenization_utils_base - INFO - loading file vocab.json 2022-11-16 17:06:56,185 - transformers.tokenization_utils_base - INFO - loading file merges.txt 2022-11-16 17:06:56,185 - transformers.tokenization_utils_base - INFO - loading file tokenizer.json 2022-11-16 17:06:56,185 - transformers.tokenization_utils_base - INFO - loading file added_tokens.json 2022-11-16 17:06:56,185 - transformers.tokenization_utils_base - INFO - loading file special_tokens_map.json 2022-11-16 17:06:56,185 - transformers.tokenization_utils_base - INFO - loading file tokenizer_config.json 2022-11-16 17:06:56,250 - __main__ - INFO - {'input_ids': [[1, 653, 761, 9, 3783, 17487, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 166, 32928, 9603, 47, 7, 1183, 10, 780, 5403, 9, 15581, 436, 479, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 15584, 3082, 3192, 23959, 15, 5, 2860, 3875, 9, 436, 4832, 41876, 38628, 9, 15643, 24610, 4743, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 15125, 6764, 15, 15643, 24610, 4743, 16, 5, 23001, 7, 5, 41184, 6304, 25132, 23909, 479, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 85, 16, 14092, 9, 10, 2270, 11235, 459, 2156, 5929, 1690, 523, 293, 2156, 10, 1307, 1062, 18185, 8, 30943, 9368, 2156, 8, 5, 2860, 2298, 2156, 566, 97, 383, 479, 2]], 'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]} 2022-11-16 17:06:56,251 - __main__ - INFO - ['[CLS]', 'ĠWhat', 'Ġkind', 'Ġof', 'Ġmemory', 'Ġ?', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]'] 2022-11-16 17:06:56,251 - __main__ - INFO - ['[CLS]', 'ĠWe', 'Ġrespectfully', 'Ġinvite', 'Ġyou', 'Ġto', 'Ġwatch', 'Ġa', 'Ġspecial', 'Ġedition', 'Ġof', 'ĠAcross', 'ĠChina', 'Ġ.', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]'] 2022-11-16 17:06:56,251 - __main__ - INFO - ['[CLS]', 'ĠWW', 'ĠII', 'ĠLand', 'marks', 'Ġon', 'Ġthe', 'ĠGreat', 'ĠEarth', 'Ġof', 'ĠChina', 'Ġ:', 'ĠEternal', 'ĠMemories', 'Ġof', 'ĠTai', 'hang', 'ĠMountain', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]'] 2022-11-16 17:06:56,251 - __main__ - INFO - ['[CLS]', 'ĠStanding', 'Ġtall', 'Ġon', 'ĠTai', 'hang', 'ĠMountain', 'Ġis', 'Ġthe', 'ĠMonument', 'Ġto', 'Ġthe', 'ĠHundred', 'ĠReg', 'iments', 'ĠOffensive', 'Ġ.', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]'] 2022-11-16 17:06:56,252 - __main__ - INFO - ['[CLS]', 'ĠIt', 'Ġis', 'Ġcomposed', 'Ġof', 'Ġa', 'Ġprimary', 'Ġste', 'le', 'Ġ,', 'Ġsecondary', 'Ġst', 'el', 'es', 'Ġ,', 'Ġa', 'Ġhuge', 'Ġround', 'Ġsculpture', 'Ġand', 'Ġbeacon', 'Ġtower', 'Ġ,', 'Ġand', 'Ġthe', 'ĠGreat', 'ĠWall', 'Ġ,', 'Ġamong', 'Ġother', 'Ġthings', 'Ġ.', '[SEP]'] 2022-11-16 17:06:56,252 - __main__ - INFO - ------------- 2022-11-16 17:06:56,252 - __main__ - INFO - ['[CLS]', 'ĠWe', 'Ġrespectfully', 'Ġinvite', 'Ġyou', 'Ġto', 'Ġwatch', 'Ġa', 'Ġspecial', 'Ġedition', 'Ġof', 'ĠAcross', 'ĠChina', 'Ġ.', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]'] 2022-11-16 17:06:56,252 - __main__ - INFO - [None, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None] 2022-11-16 17:07:05,682 - __main__ - INFO - {'id': [0, 1, 2, 3, 4], 'words': [['What', 'kind', 'of', 'memory', '?'], ['We', 'respectfully', 'invite', 'you', 'to', 'watch', 'a', 'special', 'edition', 'of', 'Across', 'China', '.'], ['WW', 'II', 'Landmarks', 'on', 'the', 'Great', 'Earth', 'of', 'China', ':', 'Eternal', 'Memories', 'of', 'Taihang', 'Mountain'], ['Standing', 'tall', 'on', 'Taihang', 'Mountain', 'is', 'the', 'Monument', 'to', 'the', 'Hundred', 'Regiments', 'Offensive', '.'], ['It', 'is', 'composed', 'of', 'a', 'primary', 'stele', ',', 'secondary', 'steles', ',', 'a', 'huge', 'round', 'sculpture', 'and', 'beacon', 'tower', ',', 'and', 'the', 'Great', 'Wall', ',', 'among', 'other', 'things', '.']], 'ner_tags': [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 8, 0], [31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], [0, 0, 0, 11, 12, 0, 31, 32, 32, 32, 32, 32, 32, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 32, 32, 0, 0, 0, 0, 0]], 'input_ids': [[1, 653, 761, 9, 3783, 17487, 2], [1, 166, 32928, 9603, 47, 7, 1183, 10, 780, 5403, 9, 15581, 436, 479, 2], [1, 15584, 3082, 3192, 23959, 15, 5, 2860, 3875, 9, 436, 4832, 41876, 38628, 9, 15643, 24610, 4743, 2], [1, 15125, 6764, 15, 15643, 24610, 4743, 16, 5, 23001, 7, 5, 41184, 6304, 25132, 23909, 479, 2], [1, 85, 16, 14092, 9, 10, 2270, 11235, 459, 2156, 5929, 1690, 523, 293, 2156, 10, 1307, 1062, 18185, 8, 30943, 9368, 2156, 8, 5, 2860, 2298, 2156, 566, 97, 383, 479, 2]], 'token_type_ids': [[0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], 'labels': [[-100, 0, 0, 0, 0, 0, -100], [-100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 8, 0, -100], [-100, 31, 32, 32, -100, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, -100, 32, -100], [-100, 0, 0, 0, 11, -100, 12, 0, 31, 32, 32, 32, 32, 32, -100, 32, 0, -100], [-100, 0, 0, 0, 0, 0, 0, 0, -100, 0, 0, 0, -100, -100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 32, 32, 0, 0, 0, 0, 0, -100]]} 2022-11-16 17:07:06,902 - transformers.configuration_utils - INFO - loading configuration file /content/NER-System/models/microsoft/deberta-base_1668615764.565312/checkpoint-14100/config.json 2022-11-16 17:07:06,904 - transformers.configuration_utils - INFO - Model config DebertaConfig { "_name_or_path": "/content/NER-System/models/microsoft/deberta-base_1668615764.565312/checkpoint-14100", "architectures": [ "DebertaForTokenClassification" ], "attention_probs_dropout_prob": 0.1, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "id2label": { "0": "O", "1": "B-PERSON", "2": "I-PERSON", "3": "B-NORP", "4": "I-NORP", "5": "B-FAC", "6": "I-FAC", "7": "B-ORG", "8": "I-ORG", "9": "B-GPE", "10": "I-GPE", "11": "B-LOC", "12": "I-LOC", "13": "B-PRODUCT", "14": "I-PRODUCT", "15": "B-DATE", "16": "I-DATE", "17": "B-TIME", "18": "I-TIME", "19": "B-PERCENT", "20": "I-PERCENT", "21": "B-MONEY", "22": "I-MONEY", "23": "B-QUANTITY", "24": "I-QUANTITY", "25": "B-ORDINAL", "26": "I-ORDINAL", "27": "B-CARDINAL", "28": "I-CARDINAL", "29": "B-EVENT", "30": "I-EVENT", "31": "B-WORK_OF_ART", "32": "I-WORK_OF_ART", "33": "B-LAW", "34": "I-LAW", "35": "B-LANGUAGE", "36": "I-LANGUAGE" }, "initializer_range": 0.02, "intermediate_size": 3072, "label2id": { "B-CARDINAL": 27, "B-DATE": 15, "B-EVENT": 29, "B-FAC": 5, "B-GPE": 9, "B-LANGUAGE": 35, "B-LAW": 33, "B-LOC": 11, "B-MONEY": 21, "B-NORP": 3, "B-ORDINAL": 25, "B-ORG": 7, "B-PERCENT": 19, "B-PERSON": 1, "B-PRODUCT": 13, "B-QUANTITY": 23, "B-TIME": 17, "B-WORK_OF_ART": 31, "I-CARDINAL": 28, "I-DATE": 16, "I-EVENT": 30, "I-FAC": 6, "I-GPE": 10, "I-LANGUAGE": 36, "I-LAW": 34, "I-LOC": 12, "I-MONEY": 22, "I-NORP": 4, "I-ORDINAL": 26, "I-ORG": 8, "I-PERCENT": 20, "I-PERSON": 2, "I-PRODUCT": 14, "I-QUANTITY": 24, "I-TIME": 18, "I-WORK_OF_ART": 32, "O": 0 }, "layer_norm_eps": 1e-07, "max_position_embeddings": 512, "max_relative_positions": -1, "model_type": "deberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "pooler_dropout": 0, "pooler_hidden_act": "gelu", "pooler_hidden_size": 768, "pos_att_type": [ "c2p", "p2c" ], "position_biased_input": false, "relative_attention": true, "torch_dtype": "float32", "transformers_version": "4.23.0", "type_vocab_size": 0, "vocab_size": 50265 } 2022-11-16 17:07:06,933 - transformers.modeling_utils - INFO - loading weights file /content/NER-System/models/microsoft/deberta-base_1668615764.565312/checkpoint-14100/pytorch_model.bin 2022-11-16 17:07:08,422 - transformers.modeling_utils - INFO - All model checkpoint weights were used when initializing DebertaForTokenClassification. 2022-11-16 17:07:08,422 - transformers.modeling_utils - INFO - All the weights of DebertaForTokenClassification were initialized from the model checkpoint at /content/NER-System/models/microsoft/deberta-base_1668615764.565312/checkpoint-14100. If your task is similar to the task the model of the checkpoint was trained on, you can already use DebertaForTokenClassification for predictions without further training. 2022-11-16 17:07:08,500 - __main__ - INFO - DebertaForTokenClassification( (deberta): DebertaModel( (embeddings): DebertaEmbeddings( (word_embeddings): Embedding(50265, 768, padding_idx=0) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) (encoder): DebertaEncoder( (layer): ModuleList( (0): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (1): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (2): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (3): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (4): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (5): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (6): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (7): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (8): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (9): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (10): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (11): DebertaLayer( (attention): DebertaAttention( (self): DisentangledSelfAttention( (in_proj): Linear(in_features=768, out_features=2304, bias=False) (pos_dropout): StableDropout() (pos_proj): Linear(in_features=768, out_features=768, bias=False) (pos_q_proj): Linear(in_features=768, out_features=768, bias=True) (dropout): StableDropout() ) (output): DebertaSelfOutput( (dense): Linear(in_features=768, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) (intermediate): DebertaIntermediate( (dense): Linear(in_features=768, out_features=3072, bias=True) (intermediate_act_fn): GELUActivation() ) (output): DebertaOutput( (dense): Linear(in_features=3072, out_features=768, bias=True) (LayerNorm): DebertaLayerNorm() (dropout): StableDropout() ) ) ) (rel_embeddings): Embedding(1024, 768) ) ) (dropout): Dropout(p=0.1, inplace=False) (classifier): Linear(in_features=768, out_features=37, bias=True) ) 2022-11-16 17:07:08,521 - __main__ - INFO - CONFIGS:{ "output_dir": "./eval_test1", "per_device_train_batch_size": 16, "per_device_eval_batch_size": 16, "save_total_limit": 2, "num_train_epochs": 3, "seed": 1, "load_best_model_at_end": true, "evaluation_strategy": "epoch", "save_strategy": "epoch", "learning_rate": 2e-05, "weight_decay": 0.01, "fp16": true, "logging_steps": 469.0 } 2022-11-16 17:07:08,522 - transformers.training_args - INFO - PyTorch: setting up devices 2022-11-16 17:07:08,557 - transformers.training_args - INFO - The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-). 2022-11-16 17:07:11,000 - transformers.trainer - INFO - Using cuda_amp half precision backend 2022-11-16 17:07:11,001 - __main__ - INFO - [[ MODEL EVALUATION ]] 2022-11-16 17:07:11,001 - transformers.trainer - INFO - The following columns in the evaluation set don't have a corresponding argument in `DebertaForTokenClassification.forward` and have been ignored: words, ner_tags, id. If words, ner_tags, id are not expected by `DebertaForTokenClassification.forward`, you can safely ignore this message. 2022-11-16 17:07:11,004 - transformers.trainer - INFO - ***** Running Evaluation ***** 2022-11-16 17:07:11,004 - transformers.trainer - INFO - Num examples = 9479 2022-11-16 17:07:11,004 - transformers.trainer - INFO - Batch size = 16 2022-11-16 17:07:11,007 - transformers.tokenization_utils_base - WARNING - You're using a DebertaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. 2022-11-16 17:07:33,916 - __main__ - INFO - {'eval_loss': 0.06115942820906639, 'eval_precision': 0.8953762782973517, 'eval_recall': 0.9100115483699032, 'eval_f1': 0.902634593356243, 'eval_accuracy': 0.9848035428915137, 'eval_runtime': 22.906, 'eval_samples_per_second': 413.822, 'eval_steps_per_second': 25.888, 'step': 0} 2022-11-16 17:07:33,916 - transformers.trainer - INFO - The following columns in the test set don't have a corresponding argument in `DebertaForTokenClassification.forward` and have been ignored: words, ner_tags, id. If words, ner_tags, id are not expected by `DebertaForTokenClassification.forward`, you can safely ignore this message. 2022-11-16 17:07:33,918 - transformers.trainer - INFO - ***** Running Prediction ***** 2022-11-16 17:07:33,918 - transformers.trainer - INFO - Num examples = 9479 2022-11-16 17:07:33,918 - transformers.trainer - INFO - Batch size = 16 2022-11-16 17:07:59,630 - __main__ - INFO - precision recall f1-score support CARDINAL 0.86 0.87 0.86 935 DATE 0.85 0.89 0.87 1602 EVENT 0.65 0.78 0.71 63 FAC 0.74 0.80 0.77 135 GPE 0.97 0.96 0.96 2240 LANGUAGE 0.83 0.68 0.75 22 LAW 0.71 0.68 0.69 40 LOC 0.74 0.77 0.76 179 MONEY 0.88 0.90 0.89 314 NORP 0.94 0.97 0.95 841 ORDINAL 0.79 0.87 0.83 195 ORG 0.92 0.92 0.92 1795 PERCENT 0.92 0.92 0.92 349 PERSON 0.95 0.95 0.95 1988 PRODUCT 0.65 0.76 0.70 76 QUANTITY 0.77 0.82 0.80 105 TIME 0.62 0.65 0.63 212 WORK_OF_ART 0.64 0.69 0.66 166 micro avg 0.90 0.91 0.90 11257 macro avg 0.80 0.83 0.81 11257 weighted avg 0.90 0.91 0.90 11257