Hugging Face's logo

djagatiya
/
ner-bert-base-cased-ontonotesv5-englishv4

Token Classification
Transformers
PyTorch
Safetensors
bert
Inference Endpoints
Model card Files Community
1
ner-bert-base-cased-ontonotesv5-englishv4 / eval.log
djagatiya's picture
djagatiya
model added.
b38757a
raw history blame contribute delete
No virus
29.9 kB
2022-07-03 16:16:30,561 - __main__ - INFO - Label List:['O', 'B-PERSON', 'I-PERSON', 'B-NORP', 'I-NORP', 'B-FAC', 'I-FAC', 'B-ORG', 'I-ORG', 'B-GPE', 'I-GPE', 'B-LOC', 'I-LOC', 'B-PRODUCT', 'I-PRODUCT', 'B-DATE', 'I-DATE', 'B-TIME', 'I-TIME', 'B-PERCENT', 'I-PERCENT', 'B-MONEY', 'I-MONEY', 'B-QUANTITY', 'I-QUANTITY', 'B-ORDINAL', 'I-ORDINAL', 'B-CARDINAL', 'I-CARDINAL', 'B-EVENT', 'I-EVENT', 'B-WORK_OF_ART', 'I-WORK_OF_ART', 'B-LAW', 'I-LAW', 'B-LANGUAGE', 'I-LANGUAGE']
2022-07-03 16:16:36,880 - __main__ - INFO - Dataset({
features: ['id', 'words', 'ner_tags'],
num_rows: 75187
})
2022-07-03 16:16:37,627 - __main__ - INFO - Dataset({
features: ['id', 'words', 'ner_tags'],
num_rows: 9479
})
2022-07-03 16:16:37,629 - transformers.tokenization_utils_base - INFO - Didn't find file models/bert-base-cased_1656837168.84538/checkpoint-14100/added_tokens.json. We won't load it.
2022-07-03 16:16:37,631 - transformers.tokenization_utils_base - INFO - loading file models/bert-base-cased_1656837168.84538/checkpoint-14100/vocab.txt
2022-07-03 16:16:37,631 - transformers.tokenization_utils_base - INFO - loading file models/bert-base-cased_1656837168.84538/checkpoint-14100/tokenizer.json
2022-07-03 16:16:37,631 - transformers.tokenization_utils_base - INFO - loading file None
2022-07-03 16:16:37,631 - transformers.tokenization_utils_base - INFO - loading file models/bert-base-cased_1656837168.84538/checkpoint-14100/special_tokens_map.json
2022-07-03 16:16:37,632 - transformers.tokenization_utils_base - INFO - loading file models/bert-base-cased_1656837168.84538/checkpoint-14100/tokenizer_config.json
2022-07-03 16:16:37,649 - __main__ - INFO - {'input_ids': [[101, 1327, 1912, 1104, 2962, 136, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 1284, 4161, 5834, 13967, 1128, 1106, 2824, 170, 1957, 2596, 1104, 14754, 1975, 119, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 160, 2924, 1563, 18405, 1116, 1113, 1103, 2038, 2746, 1104, 1975, 131, 21342, 19917, 1104, 16191, 17204, 3757, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 9996, 3543, 1113, 16191, 17204, 3757, 1110, 1103, 12267, 1106, 1103, 15090, 3391, 1116, 17354, 119, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 1135, 1110, 2766, 1104, 170, 2425, 188, 7854, 1162, 117, 3718, 188, 7854, 1279, 117, 170, 3321, 1668, 7115, 1105, 25973, 3590, 117, 1105, 1103, 2038, 6250, 117, 1621, 1168, 1614, 119, 102]], 'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
2022-07-03 16:16:37,649 - __main__ - INFO - ['[CLS]', 'What', 'kind', 'of', 'memory', '?', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']
2022-07-03 16:16:37,651 - __main__ - INFO - ['[CLS]', 'We', 'respect', '##fully', 'invite', 'you', 'to', 'watch', 'a', 'special', 'edition', 'of', 'Across', 'China', '.', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']
2022-07-03 16:16:37,651 - __main__ - INFO - ['[CLS]', 'W', '##W', 'II', 'Landmark', '##s', 'on', 'the', 'Great', 'Earth', 'of', 'China', ':', 'Eternal', 'Memories', 'of', 'Tai', '##hang', 'Mountain', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']
2022-07-03 16:16:37,652 - __main__ - INFO - ['[CLS]', 'Standing', 'tall', 'on', 'Tai', '##hang', 'Mountain', 'is', 'the', 'Monument', 'to', 'the', 'Hundred', 'Regiment', '##s', 'Offensive', '.', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']
2022-07-03 16:16:37,652 - __main__ - INFO - ['[CLS]', 'It', 'is', 'composed', 'of', 'a', 'primary', 's', '##tel', '##e', ',', 'secondary', 's', '##tel', '##es', ',', 'a', 'huge', 'round', 'sculpture', 'and', 'beacon', 'tower', ',', 'and', 'the', 'Great', 'Wall', ',', 'among', 'other', 'things', '.', '[SEP]']
2022-07-03 16:16:37,652 - __main__ - INFO - -------------
2022-07-03 16:16:37,653 - __main__ - INFO - ['[CLS]', 'We', 'respect', '##fully', 'invite', 'you', 'to', 'watch', 'a', 'special', 'edition', 'of', 'Across', 'China', '.', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']
2022-07-03 16:16:37,653 - __main__ - INFO - [None, 0, 1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]
2022-07-03 16:16:37,656 - datasets.fingerprint - WARNING - Parameter 'function'=<function tokenize_and_align_labels at 0x7f4e0440bee0> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.
2022-07-03 16:16:42,938 - __main__ - INFO - {'id': [0, 1, 2, 3, 4], 'words': [['What', 'kind', 'of', 'memory', '?'], ['We', 'respectfully', 'invite', 'you', 'to', 'watch', 'a', 'special', 'edition', 'of', 'Across', 'China', '.'], ['WW', 'II', 'Landmarks', 'on', 'the', 'Great', 'Earth', 'of', 'China', ':', 'Eternal', 'Memories', 'of', 'Taihang', 'Mountain'], ['Standing', 'tall', 'on', 'Taihang', 'Mountain', 'is', 'the', 'Monument', 'to', 'the', 'Hundred', 'Regiments', 'Offensive', '.'], ['It', 'is', 'composed', 'of', 'a', 'primary', 'stele', ',', 'secondary', 'steles', ',', 'a', 'huge', 'round', 'sculpture', 'and', 'beacon', 'tower', ',', 'and', 'the', 'Great', 'Wall', ',', 'among', 'other', 'things', '.']], 'ner_tags': [[0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 8, 0], [31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], [0, 0, 0, 11, 12, 0, 31, 32, 32, 32, 32, 32, 32, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 32, 32, 0, 0, 0, 0, 0]], 'input_ids': [[101, 1327, 1912, 1104, 2962, 136, 102], [101, 1284, 4161, 5834, 13967, 1128, 1106, 2824, 170, 1957, 2596, 1104, 14754, 1975, 119, 102], [101, 160, 2924, 1563, 18405, 1116, 1113, 1103, 2038, 2746, 1104, 1975, 131, 21342, 19917, 1104, 16191, 17204, 3757, 102], [101, 9996, 3543, 1113, 16191, 17204, 3757, 1110, 1103, 12267, 1106, 1103, 15090, 3391, 1116, 17354, 119, 102], [101, 1135, 1110, 2766, 1104, 170, 2425, 188, 7854, 1162, 117, 3718, 188, 7854, 1279, 117, 170, 3321, 1668, 7115, 1105, 25973, 3590, 117, 1105, 1103, 2038, 6250, 117, 1621, 1168, 1614, 119, 102]], 'token_type_ids': [[0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], 'labels': [[-100, 0, 0, 0, 0, 0, -100], [-100, 0, 0, -100, 0, 0, 0, 0, 0, 0, 0, 0, 7, 8, 0, -100], [-100, 31, -100, 32, 32, -100, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, -100, 32, -100], [-100, 0, 0, 0, 11, -100, 12, 0, 31, 32, 32, 32, 32, 32, -100, 32, 0, -100], [-100, 0, 0, 0, 0, 0, 0, 0, -100, -100, 0, 0, 0, -100, -100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 32, 32, 0, 0, 0, 0, 0, -100]]}
2022-07-03 16:16:45,238 - transformers.configuration_utils - INFO - loading configuration file models/bert-base-cased_1656837168.84538/checkpoint-14100/config.json
2022-07-03 16:16:45,241 - transformers.configuration_utils - INFO - Model config BertConfig {
"_name_or_path": "models/bert-base-cased_1656837168.84538/checkpoint-14100",
"architectures": [
"BertForTokenClassification"
],
"attention_probs_dropout_prob": 0.1,
"classifier_dropout": null,
"gradient_checkpointing": false,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"hidden_size": 768,
"id2label": {
"0": "O",
"1": "B-PERSON",
"2": "I-PERSON",
"3": "B-NORP",
"4": "I-NORP",
"5": "B-FAC",
"6": "I-FAC",
"7": "B-ORG",
"8": "I-ORG",
"9": "B-GPE",
"10": "I-GPE",
"11": "B-LOC",
"12": "I-LOC",
"13": "B-PRODUCT",
"14": "I-PRODUCT",
"15": "B-DATE",
"16": "I-DATE",
"17": "B-TIME",
"18": "I-TIME",
"19": "B-PERCENT",
"20": "I-PERCENT",
"21": "B-MONEY",
"22": "I-MONEY",
"23": "B-QUANTITY",
"24": "I-QUANTITY",
"25": "B-ORDINAL",
"26": "I-ORDINAL",
"27": "B-CARDINAL",
"28": "I-CARDINAL",
"29": "B-EVENT",
"30": "I-EVENT",
"31": "B-WORK_OF_ART",
"32": "I-WORK_OF_ART",
"33": "B-LAW",
"34": "I-LAW",
"35": "B-LANGUAGE",
"36": "I-LANGUAGE"
},
"initializer_range": 0.02,
"intermediate_size": 3072,
"label2id": {
"B-CARDINAL": 27,
"B-DATE": 15,
"B-EVENT": 29,
"B-FAC": 5,
"B-GPE": 9,
"B-LANGUAGE": 35,
"B-LAW": 33,
"B-LOC": 11,
"B-MONEY": 21,
"B-NORP": 3,
"B-ORDINAL": 25,
"B-ORG": 7,
"B-PERCENT": 19,
"B-PERSON": 1,
"B-PRODUCT": 13,
"B-QUANTITY": 23,
"B-TIME": 17,
"B-WORK_OF_ART": 31,
"I-CARDINAL": 28,
"I-DATE": 16,
"I-EVENT": 30,
"I-FAC": 6,
"I-GPE": 10,
"I-LANGUAGE": 36,
"I-LAW": 34,
"I-LOC": 12,
"I-MONEY": 22,
"I-NORP": 4,
"I-ORDINAL": 26,
"I-ORG": 8,
"I-PERCENT": 20,
"I-PERSON": 2,
"I-PRODUCT": 14,
"I-QUANTITY": 24,
"I-TIME": 18,
"I-WORK_OF_ART": 32,
"O": 0
},
"layer_norm_eps": 1e-12,
"max_position_embeddings": 512,
"model_type": "bert",
"num_attention_heads": 12,
"num_hidden_layers": 12,
"pad_token_id": 0,
"position_embedding_type": "absolute",
"torch_dtype": "float32",
"transformers_version": "4.20.0",
"type_vocab_size": 2,
"use_cache": true,
"vocab_size": 28996
}
2022-07-03 16:16:45,304 - transformers.modeling_utils - INFO - loading weights file models/bert-base-cased_1656837168.84538/checkpoint-14100/pytorch_model.bin
2022-07-03 16:16:46,439 - transformers.modeling_utils - INFO - All model checkpoint weights were used when initializing BertForTokenClassification.
2022-07-03 16:16:46,439 - transformers.modeling_utils - INFO - All the weights of BertForTokenClassification were initialized from the model checkpoint at models/bert-base-cased_1656837168.84538/checkpoint-14100.
If your task is similar to the task the model of the checkpoint was trained on, you can already use BertForTokenClassification for predictions without further training.
2022-07-03 16:16:46,442 - __main__ - INFO - BertForTokenClassification(
(bert): BertModel(
(embeddings): BertEmbeddings(
(word_embeddings): Embedding(28996, 768, padding_idx=0)
(position_embeddings): Embedding(512, 768)
(token_type_embeddings): Embedding(2, 768)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(encoder): BertEncoder(
(layer): ModuleList(
(0): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(1): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(2): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(3): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(4): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(5): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(6): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(7): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(8): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(9): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(10): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(11): BertLayer(
(attention): BertAttention(
(self): BertSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
)
)
)
(dropout): Dropout(p=0.1, inplace=False)
(classifier): Linear(in_features=768, out_features=37, bias=True)
)
2022-07-03 16:16:46,443 - __main__ - INFO - CONFIGS:{
"output_dir": "./models/finetuned-base-uncased_1656845190.560204",
"per_device_train_batch_size": 16,
"per_device_eval_batch_size": 16,
"save_total_limit": 2,
"num_train_epochs": 3,
"seed": 1,
"load_best_model_at_end": true,
"evaluation_strategy": "epoch",
"save_strategy": "epoch",
"learning_rate": 2e-05,
"weight_decay": 0.01,
"logging_steps": 469.0
}
2022-07-03 16:16:46,444 - transformers.training_args - INFO - PyTorch: setting up devices
2022-07-03 16:16:46,488 - transformers.training_args - INFO - The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
2022-07-03 16:16:46,494 - __main__ - INFO - [[ MODEL EVALUATION ]]
2022-07-03 16:16:46,494 - transformers.trainer - INFO - The following columns in the evaluation set don't have a corresponding argument in `BertForTokenClassification.forward` and have been ignored: ner_tags, id, words. If ner_tags, id, words are not expected by `BertForTokenClassification.forward`, you can safely ignore this message.
2022-07-03 16:16:46,497 - transformers.trainer - INFO - ***** Running Evaluation *****
2022-07-03 16:16:46,497 - transformers.trainer - INFO - Num examples = 9479
2022-07-03 16:16:46,498 - transformers.trainer - INFO - Batch size = 16
2022-07-03 16:25:59,032 - __main__ - INFO - {'eval_loss': 0.06829366087913513, 'eval_precision': 0.8785372224640836, 'eval_recall': 0.8963311717153771, 'eval_f1': 0.8873450004397152, 'eval_accuracy': 0.9835533880964035, 'eval_runtime': 552.5236, 'eval_samples_per_second': 17.156, 'eval_steps_per_second': 1.073, 'step': 0}
2022-07-03 16:25:59,032 - transformers.trainer - INFO - The following columns in the test set don't have a corresponding argument in `BertForTokenClassification.forward` and have been ignored: ner_tags, id, words. If ner_tags, id, words are not expected by `BertForTokenClassification.forward`, you can safely ignore this message.
2022-07-03 16:25:59,034 - transformers.trainer - INFO - ***** Running Prediction *****
2022-07-03 16:25:59,035 - transformers.trainer - INFO - Num examples = 9479
2022-07-03 16:25:59,035 - transformers.trainer - INFO - Batch size = 16
2022-07-03 16:34:58,579 - __main__ - INFO - precision recall f1-score support
CARDINAL 0.86 0.87 0.86 935
DATE 0.84 0.88 0.86 1602
EVENT 0.65 0.67 0.66 63
FAC 0.69 0.71 0.70 135
GPE 0.97 0.93 0.95 2240
LANGUAGE 0.76 0.73 0.74 22
LAW 0.54 0.55 0.54 40
LOC 0.73 0.80 0.76 179
MONEY 0.87 0.90 0.88 314
NORP 0.93 0.96 0.94 841
ORDINAL 0.80 0.87 0.83 195
ORG 0.88 0.90 0.89 1795
PERCENT 0.88 0.90 0.89 349
PERSON 0.94 0.95 0.94 1988
PRODUCT 0.62 0.76 0.69 76
QUANTITY 0.74 0.81 0.77 105
TIME 0.61 0.67 0.64 212
WORK_OF_ART 0.56 0.66 0.61 166
micro avg 0.88 0.90 0.89 11257
macro avg 0.77 0.81 0.79 11257
weighted avg 0.88 0.90 0.89 11257