Combinatorial feature embedding based on CNN and LSTM for biomedical named entity recognition Abstract   With the rapid advancement of technology and the necessity of processing large amounts of data, biomedical Named Entity Recognition (NER) has become an essential technique for information extraction in the biomedical field. NER, which is a sequencelabeling task, has been performed using various traditional techniques including dictionary, rule, machine learning, and deep learningbased methods. However, as existing biomedical NER models are insufficient to handle new and unseen entity types from the growing biomedical data, the development of more effective and accurate biomedical NER models is being widely researched. Among biomedical NER models utilizing deep learning approaches, there have been only a few studies involving the design of highlevel features in the embedding layer. In this regard, herein, we propose a deep learning NER model that effectively represents biomedical word tokens through the design of a combinatorial feature embedding. The proposed model is based on Bidirectional Long ShortTerm Memory (biLSTM) with Conditional Random Field (CRF) and enhanced by integrating two different characterlevel representations extracted from a Convolutional Neural Network (CNN) and biLSTM. Additionally, an attention mechanism is applied to the model to focus on the relevant tokens in the sentence, which alleviates the longterm dependency problem of the LSTM model and allows effective recognition of entities. The proposed model was evaluated on two benchmark datasets, the JNLPBA and NCBIDisease, and a comparative analysis with the existing models is performed. The proposed model achieved a relatively higher performance with an F1score of 86.93% in case of NCBI Data, and a competitive performance for the JNLPBA with an F1score of 75.31%.

Improving Biomedical Information Extraction with Word Embeddings Trained on ClosedDomain Corpora Named Entity Recognition (NER) systems allow complex concepts extraction and text mining from Natural Language documents. Actually, NER systems based on Deep Learning (DL) approaches are able to reach stateoftheart performances when applied to general domain texts. On the other hand, the performances of these systems decrease when they are applied to texts that belong to specific domains, such as the biomedical one. In particular, Biomedical NER (BNER) is a crucial task for the automatic analysis of medical documents, such as Electronic Health Records (EHRs), in order to support the work of physicians and researchers. Thus, new approaches are required to boost BNER systems performances. In this paper we analyze the behaviour of a BNER DL architecture specifically devoted to Italian EHRs, focusing on the contribution of different Word Embeddings (WEs) models used as input text representation layer. The achieved results show the substantial contribution of WEs trained on a closed domain corpus exclusively formed by documents belonging to the biomedical domain. The resulting improvements are comparable with the ones obtained using the most recent and complex neural language models, such as ELMo or BERT, which have a much higher computational complexity if compared with classic WEs approach.