Librarian Bot: Add base_model information to model (#3)

18eb2b5 5 months ago

No virus

12.6 kB

	---
	language:
	- de
	- en
	- es
	- fr
	- it
	- nl
	- pl
	- pt
	- ru
	- zh
	license: cc-by-nc-sa-4.0
	tags:
	- generated_from_trainer
	- ner
	- named-entity-recognition
	- span-marker
	datasets:
	- Babelscape/multinerd
	metrics:
	- precision
	- recall
	- f1
	pipeline_tag: token-classification
	widget:
	- text: Amelia Earthart flog mit ihrer einmotorigen Lockheed Vega 5B über den Atlantik
	nach Paris.
	example_title: German
	- text: Amelia Earhart flew her single engine Lockheed Vega 5B across the Atlantic
	to Paris.
	example_title: English
	- text: Amelia Earthart voló su Lockheed Vega 5B monomotor a través del Océano Atlántico
	hasta París.
	example_title: Spanish
	- text: Amelia Earthart a fait voler son monomoteur Lockheed Vega 5B à travers l'ocean
	Atlantique jusqu'à Paris.
	example_title: French
	- text: Amelia Earhart ha volato con il suo monomotore Lockheed Vega 5B attraverso
	l'Atlantico fino a Parigi.
	example_title: Italian
	- text: Amelia Earthart vloog met haar één-motorige Lockheed Vega 5B over de Atlantische
	Oceaan naar Parijs.
	example_title: Dutch
	- text: Amelia Earthart przeleciała swoim jednosilnikowym samolotem Lockheed Vega
	5B przez Ocean Atlantycki do Paryża.
	example_title: Polish
	- text: Amelia Earhart voou em seu monomotor Lockheed Vega 5B através do Atlântico
	para Paris.
	example_title: Portuguese
	- text: Амелия Эртхарт перелетела на своем одномоторном самолете Lockheed Vega 5B
	через Атлантический океан в Париж.
	example_title: Russian
	- text: Amelia Earthart flaug eins hreyfils Lockheed Vega 5B yfir Atlantshafið til
	Parísar.
	example_title: Icelandic
	- text: Η Amelia Earthart πέταξε το μονοκινητήριο Lockheed Vega 5B της πέρα από
	τον Ατλαντικό Ωκεανό στο Παρίσι.
	example_title: Greek
	- text: Amelia Earhartová přeletěla se svým jednomotorovým Lockheed Vega 5B přes Atlantik
	do Paříže.
	example_title: Czech
	- text: Amelia Earhart lensi yksimoottorisella Lockheed Vega 5B:llä Atlantin yli Pariisiin.
	example_title: Finnish
	- text: Amelia Earhart fløj med sin enmotoriske Lockheed Vega 5B over Atlanten til
	Paris.
	example_title: Danish
	- text: Amelia Earhart flög sin enmotoriga Lockheed Vega 5B över Atlanten till Paris.
	example_title: Swedish
	- text: Amelia Earhart fløy sin enmotoriske Lockheed Vega 5B over Atlanterhavet til
	Paris.
	example_title: Norwegian
	- text: Amelia Earhart și-a zburat cu un singur motor Lockheed Vega 5B peste Atlantic
	până la Paris.
	example_title: Romanian
	- text: Amelia Earhart menerbangkan mesin tunggal Lockheed Vega 5B melintasi Atlantik
	ke Paris.
	example_title: Indonesian
	- text: Амелія Эрхарт пераляцела на сваім аднаматорным Lockheed Vega 5B праз Атлантыку
	ў Парыж.
	example_title: Belarusian
	- text: Амелія Ергарт перелетіла на своєму одномоторному літаку Lockheed Vega 5B через
	Атлантику до Парижа.
	example_title: Ukrainian
	- text: Amelia Earhart preletjela je svojim jednomotornim zrakoplovom Lockheed Vega
	5B preko Atlantika do Pariza.
	example_title: Croatian
	- text: Amelia Earhart lendas oma ühemootoriga Lockheed Vega 5B üle Atlandi ookeani
	Pariisi .
	example_title: Estonian
	base_model: bert-base-multilingual-cased
	model-index:
	- name: span-marker-bert-base-multilingual-cased-multinerd
	results:
	- task:
	type: token-classification
	name: Named Entity Recognition
	dataset:
	name: MultiNERD
	type: Babelscape/multinerd
	split: test
	revision: 2814b78e7af4b5a1f1886fe7ad49632de4d9dd25
	metrics:
	- type: f1
	value: 0.927
	name: F1
	- type: precision
	value: 0.9281
	name: Precision
	- type: recall
	value: 0.9259
	name: Recall
	---

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment. -->

	# span-marker-bert-base-multilingual-cased-multinerd

	This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on an [Babelscape/multinerd](https://huggingface.co/datasets/Babelscape/multinerd) dataset.

	Is your data not (always) capitalized correctly? Then consider using the uncased variant of this model instead for better performance:
	[lxyuan/span-marker-bert-base-multilingual-uncased-multinerd](https://huggingface.co/lxyuan/span-marker-bert-base-multilingual-uncased-multinerd).

	This model achieves the following results on the evaluation set:
	- Loss: 0.0049
	- Overall Precision: 0.9242
	- Overall Recall: 0.9281
	- Overall F1: 0.9261
	- Overall Accuracy: 0.9852

	Test set results:
	- test_loss: 0.005226554349064827,
	- test_overall_accuracy: 0.9851129807294873,
	- test_overall_f1: 0.9270450073152169,
	- test_overall_precision: 0.9281906912835416,
	- test_overall_recall: 0.9259021481405626,
	- test_runtime: 2690.9722,
	- test_samples_per_second: 150.748,
	- test_steps_per_second: 4.711


	This is a replication of Tom's work. Everything remains unchanged,
	except that we extended the number of training epochs to 3 for a
	slightly longer training duration and set the gradient_accumulation_steps to 2.
	Please refer to the official [model page](https://huggingface.co/tomaarsen/span-marker-mbert-base-multinerd) to review their results and training script

	## Results:


	\| Language \| Precision \| Recall \| F1 \|
	\|--------------\|---------------\|------------\|------------\|
	\| all \| 92.42 \| 92.81 \| 92.61 \|
	\| de \| 95.03 \| 95.07 \| 95.05 \|
	\| en \| 95.00 \| 95.40 \| 95.20 \|
	\| es \| 92.05 \| 91.37 \| 91.71 \|
	\| fr \| 92.37 \| 91.41 \| 91.89 \|
	\| it \| 91.45 \| 93.15 \| 92.29 \|
	\| nl \| 93.85 \| 92.98 \| 93.41 \|
	\| pl \| 93.13 \| 92.66 \| 92.89 \|
	\| pt \| 93.60 \| 92.50 \| 93.05 \|
	\| ru \| 93.25 \| 93.32 \| 93.29 \|
	\| zh \| 89.47 \| 88.40 \| 88.93 \|

	- Special thanks to Tom for creating the evaluation script and generating the [results](https://huggingface.co/lxyuan/span-marker-bert-base-multilingual-cased-multinerd/discussions/1).


	## Label set

	\| Class \| Description \| Examples \|
	\|-------\|-------------\|----------\|
	\| PER (person) \| People \| Ray Charles, Jessica Alba, Leonardo DiCaprio, Roger Federer, Anna Massey. \|
	\| ORG (organization) \| Associations, companies, agencies, institutions, nationalities and religious or political groups \| University of Edinburgh, San Francisco Giants, Google, Democratic Party. \|
	\| LOC (location) \| Physical locations (e.g. mountains, bodies of water), geopolitical entities (e.g. cities, states), and facilities (e.g. bridges, buildings, airports). \| Rome, Lake Paiku, Chrysler Building, Mount Rushmore, Mississippi River. \|
	\| ANIM (animal) \| Breeds of dogs, cats and other animals, including their scientific names. \| Maine Coon, African Wild Dog, Great White Shark, New Zealand Bellbird. \|
	\| BIO (biological) \| Genus of fungus, bacteria and protoctists, families of viruses, and other biological entities. \| Herpes Simplex Virus, Escherichia Coli, Salmonella, Bacillus Anthracis. \|
	\| CEL (celestial) \| Planets, stars, asteroids, comets, nebulae, galaxies and other astronomical objects. \| Sun, Neptune, Asteroid 187 Lamberta, Proxima Centauri, V838 Monocerotis. \|
	\| DIS (disease) \| Physical, mental, infectious, non-infectious, deficiency, inherited, degenerative, social and self-inflicted diseases. \| Alzheimer’s Disease, Cystic Fibrosis, Dilated Cardiomyopathy, Arthritis. \|
	\| EVE (event) \| Sport events, battles, wars and other events. \| American Civil War, 2003 Wimbledon Championships, Cannes Film Festival. \|
	\| FOOD (food) \| Foods and drinks. \| Carbonara, Sangiovese, Cheddar Beer Fondue, Pizza Margherita. \|
	\| INST (instrument) \| Technological instruments, mechanical instruments, musical instruments, and other tools. \| Spitzer Space Telescope, Commodore 64, Skype, Apple Watch, Fender Stratocaster. \|
	\| MEDIA (media) \| Titles of films, books, magazines, songs and albums, fictional characters and languages. \| Forbes, American Psycho, Kiss Me Once, Twin Peaks, Disney Adventures. \|
	\| PLANT (plant) \| Types of trees, flowers, and other plants, including their scientific names. \| Salix, Quercus Petraea, Douglas Fir, Forsythia, Artemisia Maritima. \|
	\| MYTH (mythological) \| Mythological and religious entities. \| Apollo, Persephone, Aphrodite, Saint Peter, Pope Gregory I, Hercules. \|
	\| TIME (time) \| Specific and well-defined time intervals, such as eras, historical periods, centuries, years and important days. No months and days of the week. \| Renaissance, Middle Ages, Christmas, Great Depression, 17th Century, 2012. \|
	\| VEHI (vehicle) \| Cars, motorcycles and other vehicles. \| Ferrari Testarossa, Suzuki Jimny, Honda CR-X, Boeing 747, Fairey Fulmar. \|



	## Inference Example

	```python
	# install span_marker
	(env)$ pip install span_marker


	from span_marker import SpanMarkerModel

	model = SpanMarkerModel.from_pretrained("lxyuan/span-marker-bert-base-multilingual-cased-multinerd")

	description = "Singapore is renowned for its hawker centers offering dishes \
	like Hainanese chicken rice and laksa, while Malaysia boasts dishes such as \
	nasi lemak and rendang, reflecting its rich culinary heritage."

	entities = model.predict(description)

	entities
	>>>
	[
	{'span': 'Singapore', 'label': 'LOC', 'score': 0.999988317489624, 'char_start_index': 0, 'char_end_index': 9},
	{'span': 'Hainanese chicken rice', 'label': 'FOOD', 'score': 0.9894770383834839, 'char_start_index': 66, 'char_end_index': 88},
	{'span': 'laksa', 'label': 'FOOD', 'score': 0.9224908947944641, 'char_start_index': 93, 'char_end_index': 98},
	{'span': 'Malaysia', 'label': 'LOC', 'score': 0.9999839067459106, 'char_start_index': 106, 'char_end_index': 114}]

	# missed: nasi lemak as FOOD
	# missed: rendang as FOOD
	# :(
	```

	#### Quick test on Chinese
	```python
	from span_marker import SpanMarkerModel

	model = SpanMarkerModel.from_pretrained("lxyuan/span-marker-bert-base-multilingual-cased-multinerd")

	# translate to chinese
	description = "Singapore is renowned for its hawker centers offering dishes \
	like Hainanese chicken rice and laksa, while Malaysia boasts dishes such as \
	nasi lemak and rendang, reflecting its rich culinary heritage."

	zh_description = "新加坡因其小贩中心提供海南鸡饭和叻沙等菜肴而闻名, 而马来西亚则拥有椰浆饭和仁当等菜肴，反映了其丰富的烹饪传统."

	entities = model.predict(zh_description)

	entities
	>>>
	[
	{'span': '新加坡', 'label': 'LOC', 'score': 0.9282007813453674, 'char_start_index': 0, 'char_end_index': 3},
	{'span': '马来西亚', 'label': 'LOC', 'score': 0.7439665794372559, 'char_start_index': 27, 'char_end_index': 31}]

	# It only managed to capture two countries: Singapore and Malaysia.
	# All other entities were missed out.
	```


	## Training procedure

	One can reproduce the result running this [script](https://huggingface.co/tomaarsen/span-marker-mbert-base-multinerd/blob/main/train.py)

	### Training hyperparameters

	The following hyperparameters were used during training:
	- learning_rate: 5e-05
	- train_batch_size: 32
	- eval_batch_size: 32
	- seed: 42
	- gradient_accumulation_steps: 2
	- total_train_batch_size: 64
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: linear
	- lr_scheduler_warmup_ratio: 0.1
	- num_epochs: 3

	### Training results

	\| Training Loss \| Epoch \| Step \| Validation Loss \| Overall Precision \| Overall Recall \| Overall F1 \| Overall Accuracy \|
	\|:-------------:\|:-----:\|:------:\|:---------------:\|:-----------------:\|:--------------:\|:----------:\|:----------------:\|
	\| 0.0129 \| 1.0 \| 50436 \| 0.0042 \| 0.9226 \| 0.9169 \| 0.9197 \| 0.9837 \|
	\| 0.0027 \| 2.0 \| 100873 \| 0.0043 \| 0.9255 \| 0.9206 \| 0.9230 \| 0.9846 \|
	\| 0.0015 \| 3.0 \| 151308 \| 0.0049 \| 0.9242 \| 0.9281 \| 0.9261 \| 0.9852 \|


	### Framework versions

	- Transformers 4.30.2
	- Pytorch 2.0.1+cu117
	- Datasets 2.14.3
	- Tokenizers 0.13.3