# A Crowdsourcing Recommendation Model for Image Annotations in Cultural Heritage Platforms
## 4 Crowdsourcing Recommendation System
### Data Annotation and Extension  
Since the extracted text data for users and images contain unstructured information, those texts are processed to identify representative keywords inside, which are then extended and used to represent the user and image features in the system. To accomplish this, the text is first annotated by building automatic queries to the DBpedia Spotlight (DB-SL) tool, which identifies DBpedia-named entities in the input text data, thus connecting the unstructured information fragments to the linked data cloud through the DBpedia framework. The DBpedia initiative is built around a very flexible, rich, and easy-to-use framework to collect different kinds of structured information published in Wikipedia, storing them in a hierarchical database according to a well-known data ontology, linking the information to various additional datasets on the Web, and finally allowing you to request complex queries against such data repository to enhance your original information. In any case, it is worth mentioning that this automatic identification of entities named in a given text is not an error-free process. As there is usually some ambiguity, different alternatives can exist for a given surface form, and the tool does not always make the right decision. In order to detect these errors, we carry out consistency checks over the results, trying to detect entities with property values excessively unrelated (sharing few of them) to the rest of the detected entities and searching among the alternatives for those ones more likely to fit in the set.  
In our work, we examined the _rdf:type_ property in the DBpedia ontology (_dbo_) domain, as the generality of these classes is useful for allowing the identification of the most relevant entities within the provided text. The specific goal of utilizing the DBpedia classes is to be able to enhance the annotations of users and images that feed the classification process, discovering related concepts between both datasets. In other words, identifying similar concepts to bridge the gap between the original annotations about images and users' profiles. That is, identifying the hierarchy of classes associated with the annotations of users and images will permit us to discover hidden relationships between different terminologies and similarities that may lead to related common concepts in the ontology, paving the way to suggest for annotation the instances of such associated categories. That is, if the user-extended information indicates an interest in some group of classes, it is reasonable to think that the instances classified in such categories could be appealing to him/her for image annotation. This task is performed through a process that extends the data of both users and images querying the DB-SL to identify the classes related to the original information, inspecting common features, and later feeding the extended similar concepts to the classifiers. For all extracted images and user data, all annotations and extractions were carried out using automatic SPARQL queries to the DBpedia and DB-SL.  
For instance, consider a typical image annotated with the description \"Felucca sailboats cruising on the river--Upper Egypt\", and suppose we have gathered a user post containing the text \"Luxor and Aswan Nile Cruise trip\". The JSON response of the DB-SL API returns resources and classes with some overlapping data for both image and user text (such as the \"Cruise_ship\" resource and the \"PopulatedPlace, Place, & Location\" classes), in addition to a newly discovered class \"River\" that is additionally retrieved from the user post and which had been already annotated as an important keyword from the text describing the image. In summary, using DBpedia ontology classes to extend the categories related to the original annotations may be very helpful to discover new similar categories between users and images, categories that may not be explicitly mentioned in the texts.  
Figure 4: A sample of the Egyptian images that are included in the dataset.