INFORMed PA: A NER for the Italian Public Administration Domain
Authors: Lucia C. Passaro, Alessandro Lenci, Anna Gabbolini (Università di Pisa, ETI3, 2017)
Topic: In-domain IR and IE

Abstract
In this paper, we illustrate the creation of a NER for the Public Administration (PA) domain. We discuss the creation of an annotated corpus with documents from the Italian Albo Pretorio Nazionale and provide results of the system evaluation.

Introduction
In the Public Administration (PA) domain, the rapid adoption of the new legislation about the governance transparency has been forcing Italian municipalities to produce their acts in a digital form and to make them available for both citizens and authorities. However, the acts delivered by PAs are typically in a free-text electronic format, which is not convenient for searching, decision-support, and data analysis. Therefore, the development of NLP tools to extract high-quality structured information, including Named Entities (NEs) such as Persons and Organizations, represents a key factor to enable the access to the wealth of information produced by PAs, and a crucial step in turning the keyword of ``transparency'' into reality. The potentialities of NLP tools can be exploited to mine the large document repositories produced by PA daily, with the aim of identifying trends in their activity, suggesting possible synergies to increase their efficiency, and raising ``red flags'' about suspicious behaviors, especially for their relationships with private companies. In this paper, we focus on Named Entity Recognition (NER) for PA. Several approaches have been proposed in literature including Rule-based, Machine Learning-based and Hybrid methods. Hand-made Rule-based NERs focus on extracting names using lots of human-made rules. In general, these systems consist of a set of patterns based on grammatical (e.g., part of speech), syntactic (e.g., word precedence) and orthographic features (e.g., capitalization) in combination with dictionaries AUTHOR. These approaches usually give good results, but require long development time by expert linguists. On the one hand, these systems have better results for restricted domains, being capable of detecting very complex entities, but, on the other one, they lack portability and robustness and do not necessarily adapt well to new domains and languages. Machine learning techniques, on the contrary, use a collection of annotated documents for training the classifiers. Therefore the development time moves from the definition of rules to the preparation of annotated corpora AUTHOR. The systems identify and classify nouns using machine learning algorithms such as Maximum Entropy AUTHOR, Support Vector Machines AUTHOR and Conditional Random Field AUTHOR. More recently, also deep learning architectures have been proposed for Named Entity Recognition AUTHOR. Finally, Hybrid NER systems, combine rule-based and machine learning-based methods, and make new methods using strongest points from each method AUTHOR. Existing general purpose Italian corpora annotated with NEs such as I-CAB AUTHOR are not optimal for training a NER for the domain of PA because of the gap between bureaucratic language and standard Italian, and also because of the lack of important classes such as act and normative references, that are very useful in PA-oriented applications. To tackle these problems, we decided to create a new corpus from scratch starting from: \item administrative documents belonging to the Italian Albo Pretorio; \item the CoLingLab NER, a general NER trained on I-CAB, from which we took the initial configuration of features. The corpus of PA documents written in Italian ``bureaucratese'', has the characteristics described in AUTHOR:\item Pseudo-technicisms or collateral technicisms (e.g., balneazione, fattispecie); \item Abstract nouns with -zione/-mento suffixes (e.g., stipulazione, espletamento), deverbal nouns, usually with zero suffix (e.g., subentro, scorporo, utilizzo) and denominal verbs (e.g., relazionare, disdettare); \item Archaic terms (e.g., allorché, suddetto) and latinisms (e.g. una tantum, pro capite); \item Forestierisms (e.g., governance, front office); \item Uncommon and formal terms (e.g., diniego for rifiuto); \item Stereotyped phrases (e.g., entro e non oltre, in riferimento all’oggetto); \item Abbreviations and acronyms. For the creation of a NER for PA, we decided to exploit the existing architecture employed for the project SEMPLICE) is a 2-year project funded by Regione Toscana in collaboration with IT companies to develop NLP-based tools for knowledge management, information extraction and opinion mining for local public administrations.} and in particular we adopted a statistical method based on the Stanford NER AUTHOR, a system implemented in Java and available for download under the GNU General Public License. This choice allowed us to easily compare the gain obtained by enriching the training corpus with PA documents and to speed up the development process. Moreover, using a Conditional Random Field (CRF) AUTHOR as learning algorithm made it possible for us to compare the PA model with other domain-adapted NERs AUTHOR.\\ This paper is structured as follows: In section CoLingLab, we present the CoLingLab NER and we show its performance on a sample of PA documents; in section paner we describe the adaptation of the system to PA texts and its performances (section performances). In section relations, we report on the annotation of relations that we performed on a sample of the corpus and finally discuss the results and ongoing work.

Conclusion
Named entities play an important role in administrative acts, especially in those - like the documents in the Albo Pretorio - describing the main actions taken by Municipalities. This kind of information is very useful to fullfil the obligations related to supervisory monitoring, disclosure, periodic self-assessment, and review of the government decisions. In this paper, we presented a NER for PA that shows a significant ability to identify the relevant entities, and in particular legislative reference and connected acts. It is important to stress the lexical and syntactic complexity of bureaucratic language represents a big challenge for NLP tools and methods. Such a complexity derives from the technical lexis of other domain-specific languages with which PA deals daily, such as education, environment, ICT technologies, public health and so on. In near feature we plan to explore the possibility of re-engineering our system to take advantage of new algorithms for entity extraction such as neural networks and in particular from character level word embeddings. Moreover, we will focus on the development of classifiers for Relation Extraction and Entity Linking.