# Wikidata Support in the Creation of Rich Semantic Metadata for Historical Archives
## 5 Evaluation and Discussion  
The aim of the evaluation we discuss in this section is to assess our mappings, in particular the complex ones (see Section 3.1) with respect to the quality of the suggestions for the users of the annotation platform. Obviously, the quantity (but probably also the quality) of the information we can extract from Wikidata strongly depends on the entity we are searching for. For the evaluation, we need entities, within the selected domain, for which we can get a significant amount of information, and, in particular, information enabling the system to use the complex mappings we aim at assessing. Thus, we selected four famous preeminent figures of the Italian history of the 20 th century for which the information we can extract from Wikidata satisfy our requirements: [PERSON], [PERSON], [PERSON], and [PERSON]. We prepared a questionnaire presenting the information extracted for each person and asking user opinions through questions including radio buttons and free-text comments (see Appendix A). Then, we submitted the questionnaire to a pool of potential users of the PRiSMHA platform, obtaining complete answers from 37 participants: 49% of them were between 40 and 59 years old, 24% were between 20 and 39 years old,, and 27% were between 60 and 79 years old,, 59.5% had a master degree, 32.5% had a PhD, and 8% a high school degree; 81% were workers, 11% were retired, and 8% were students. As we claimed above, our goal was a qualitative evaluation of the suggestions, i.e., of the information extracted from Wikidata on the basis of our mappings; therefore, a  
Figure 14: Part of the form related to the characterization of the 1978 Italian presidential election, with [PERSON] as candidate. quantitative analysis of the results makes little sense, since data are too small. However, we provide a short comment on quantitative results since they can still provide us with useful insights. As shown in Table 4, overall, the information extracted from Wikidata on the basis of our mappings seems to be useful: More than 80% of it was accepted by users (and the most part was accepted without changes). However, a significant number of answers indicate that the extracted information is not enough to characterize the person in focus (_missing info_ row in Table 4), with slightly different results for the four persons: 49% for [PERSON], 62% for [PERSON], 35% for [PERSON], and 65% for [PERSON]. For [PERSON], some users also claimed that there is too much information (_too much info_ row in Table 4): To understand this twofold claim, we need to look at user comments, which are discussed below. The core of our evaluation is user comments. In the following, we first present, and then discuss, the most relevant issues that emerged from them. All free-text questions (see Appendix A) were optional, and besides asking users for the reasons for their radio-button choices, they asked for explanations about missing or too much/too little information (see Table 4). Only the last question encouraged users to express a completely free comment. Some users explicitly wrote that the suggestions were clear and precise. In particular, a participant wrote: \"It seems to me that the system proposes clear and accurate information, easy to understand, also without a specific knowledge of the persons in focus or of the historical context they belong to. In some cases, the information is incomplete, but fortunately the proposal can be changed and integrated. I did not find really wrong or useless suggestions\". However, the most interesting comments are those containing some criticism. We discuss them in the following. 1. A lot of participants (54%) remarked that the information about [PERSON] lacks data about the very important fact that he was a Partisan in the Italian Resistenza, fighting against Fascism during World War II.