Software Entity Recognition with Noise-robust Learning

We train a BERT model for the task software entity recognition (SER). The training data leverages WikiSER, a corpus of 1.7M sentences extracted from Wikipedia. The model uses self-regularization during the finetuning process, allowing it to be robust to texts in the software domain, including misannotations, different naming conventions, and others.

The model recognizes 12 fine-grained named entities: Algorithm, Application, Architecture, Data_Structure, Device, Error_Name, General_Concept, Language, Library, License, Operating_System, and Protocol.

Type	Examples
Algorithm	Auction algorithm, Collaborative filtering
Application	Adobe Acrobat, Microsoft Excel
Architecture	Graphics processing unit, Wishbone
Data_Structure	Array, Hash table, mXOR linked list
Device	Samsung Gear S2, iPad, Intel T5300
Error Name	Buffer overflow, Memory leak
General_Concept	Memory management, Nouvelle AI
Language	C++, Java, Python, Rust
Library	Beautiful Soup, FastAPI
License	Cryptix General License, MIT License
Operating_System	Linux, Ubuntu, Red Hat OS, MorphOS
Protocol	TLS, FTPS, HTTP 404

Model details

Paper: https://arxiv.org/abs/2308.10564

Code: https://github.com/taidnguyen/software_entity_recognition

Finetuned from model: bert-base-cased

Checkpoint for large version: https://huggingface.co/taidng/wikiser-bert-large

How to use

from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("taidng/wikiser-bert-base")
model = AutoModelForTokenClassification.from_pretrained("taidng/wikiser-bert-base")

nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "Windows XP was originally bundled with Internet Explorer 6."

ner_results = nlp(example)
print(ner_results)

Citation

@inproceedings{nguyen2023software,
  title={Software Entity Recognition with Noise-Robust Learning},
  author={Nguyen, Tai and Di, Yifeng and Lee, Joohan and Chen, Muhao and Zhang, Tianyi},
  booktitle={Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (ASE'23)},
  year={2023},
  organization={IEEE/ACM}
}