Clear add labels to token classification example
Author: gwc4github
Pull request: no

I have spent a lot of time looking for a clear example on how to add labels to an existing model.  For example, I would like to train Bert to recognize addresses in addition to the B-PER, B-ORG, etc. labels.  So I think I would do the following
1. Add B-ADDRESS, B-CITY, B-STATE, etc. to a portion of a data set (like take a small subset of conll2003 or custom data.)
2. Add the labels to the id2label and label2id - BUT, where do I do this?  In the config object?  Is that all since the model does not expect the new labels?
3.  Set the label count variable (in config again?)
4. Train on the new datasets (conll2003 & the new data) using the config file?

So in addition to the questions above, I would  think that I could remove the head and do some transfer learning - meaning that I don't have to re-train with the conll2003 data.  I should be able to just add training with the new data so that I have Bert+conll2003+my new data but I am only training on my new data.  However, I don't see an example of this with HF either. 
Sorry if I am just missing it.  Here are some of the links I have looked at:
    https://huggingface.co/transformers/custom_datasets.html#token-classification-with-w-nut-emerging-entities
    https://huggingface.co/transformers/custom_datasets.html#fine-tuning-with-trainer
    https://discuss.huggingface.co/t/retrain-reuse-fine-tuned-models-on-different-set-of-labels/346/4      ** GOOD INFO but not complete
    https://github.com/huggingface/transformers/tree/master/examples/token-classification

@sgugger was in the thread above so he may be able to help?


Comments:
Have you looked at the [run_ner](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_ner.py) example and its corresponding [notebook](https://github.com/huggingface/notebooks/blob/master/examples/token_classification.ipynb)?

Thanks for getting back to me so quickly @sgugger.  
I am using what looks like the same exact version of run_ner with a different notebook.  However, I don't see anything in the notebook you provided or run_ner that adds new labels.  For example, if you look at the cell 8 of the notebook you linked, you see that it is only using the labels loaded with the model.  What if I wanted to find addresses or some other entity type?

Thanks for your help!
Gregg



I am confused, the labels are loaded from the dataset, not the model. If you have another dataset with other labels, the rest of the notebook will work the same way.

Sorry, 
What I want to do is load the Bert model for NER trained from Conll2003 and use transfer learning to add addition training with new, additional data tagged with additional labels.
In the end, I want to take advantage of the existing training and add my own; teaching it to recognize more entity types.

I have seen that some people seem to have done this but I haven't found the complete list of steps.  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.