# Named Entity Recognition for Sensitive Data Discovery in Portuguese
## 5 Conclusions  
The main goal of this work was to develop a functional prototype of Named Entity Recognition for the Portuguese language. The focus of the developed prototype was the recognition of sensitive data in unstructured texts, according to all categories covered by GDPR. This prototype was validated, under the Portugal 2020 DataSense Project, through the tests of efficiency and performance. This validation was carried out with the project stakeholders, achieving an f1-score of 83.01% in the NER task.  
The work was developed using a hybrid approach, and several experiments were done in order to achieve the best results for each entity class. A rule-based model and morphological analysis were implemented, achieving the best results for entities with well-defined formats and that follow strict rules. Models based on lexicons were also implemented for a reduced set of entities, achieving an f1-score result of 62.36% for HAREM and 60.64% for SIGARRA. Although the global results when using lexicon-based models are lower than the current state-of-the-art, for TEMPO and VALOR entities the results were higher than those achieved with other methodologies, and they were a way of solving the PROFISSAO and MED entities for which there was no labelled data in Portuguese, but were necessary for the proposed prototype. These two approaches have, however, some limitations. They are restricted only to a fixed set of entities and well-defined rules. They are, nevertheless, the ones that have achieved the best results and a greater confidence in the results obtained. For the remaining classes of entities, different experiments were carried out, including the implementation of statistical machine learning models and the implementation of a Bidirectional-LSTM neural network. The two statistical models--CRF and RF--allowed us to conclude that the first one achieved better results than the second. With these two models, we were also able to understand that the HAREM corpus is not enough for training more complex models, due to its size and the reduced number of annotated entities. Finally, the third implemented model was the Bidirectional-LSTM, and ended up obtaining the best f1-score results in NER and thus, was used in the prototype. The use of different methodologies covered all sets of entities that represent sensitive data. We also conclude that it is possible for the Portuguese language to have valid results for named entities recognition tasks, and it can be used in real scenarios with a remarkable value in the Portuguese market.  
Figure 2: NER Component f1-score results by class, where the x-axis corresponds to the classes of entities and the y-axis to the f1-score results obtained.  
**Author Contributions:** This article was developed within the DataSense Project to which [PERSON] belongs as developer and [PERSON] as project manager. The results presented and the experiments performed were developed within the scope of [PERSON]'s Master's thesis and with the scientific help of its supervisor [PERSON] and [PERSON]. [PERSON]. The article was written by [PERSON] with the help of [PERSON], who currently continues the work presented here. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This work was supported in part by the DataSense Project. This project is being led by Link Consulting and received cofunding from the FEDER--Lisbon 2020, PT 2020, European Union's PT 2020 research, and innovation program under grant agreement cod POCI-01-0247-FEDER-038539.  
**Conflicts of Interest:** The authors declare no conflict of interest.