## 5 Analysis for Crawl Guidance and Enrichment
### Data Enrichment and Consolidation  
As the content analysis extracts structured data from unstructured resources, such as text and images, the generated data is highly heterogeneous. This is due to the data being generated by different components and during independent processing cycles. For instance, during one particular cycle, the text analysis component might detect an entity from the term \"Ireland\", while during later cycles, entities based on the term \"Republic of Ireland\" or the German term \"Irland\" might be extracted. These would all be classified as entities of type arco:Location and correctly stored in the ARCOMEM knowledge base as separate entities described according to the ARCOMEM data model. Data enrichment and consolidation in ARCOMEM follow two aims: (1) to enrich existing entities with related publicly-available knowledge; and (2) to identify data correlations within the Web archives created by ARCOMEM by aligning extracted entities with reference datasets. Both (1) and (2) exploit publicly-available data from the Linked Open Data (LOD) cloud, which offers a vast amount of data of both a domain-specific and domain-independent nature. To achieve the described aims, the enrichment approach first identifies correlating enrichments from reference datasets, which are associated with the respective entities, and, secondly, uses these shared enrichments to identify correlating entities in the ARCOMEM knowledge base. In particular, the current enrichment approach uses DBpedia [24] and Freebase [25] as reference datasets, though this approach does not depend on any particular dataset and is applicable to additional and more domain-specific datasets, e.g., event-specific ones. DBpedia and Freebase are particularly well-suited for the enrichment of Web crawls, due to their vast size, the availability of disambiguation techniques, which can utilise the variety of multilingual labels available in both datasets for individual data items and the level of inter-connectedness of both datasets, allowing the retrieval of a wealth of related information for particular items. For example, Freebase currently contains more than 22 million entities in more than 100 domains and more than 350 million facts about these entities. We distinct direct, as well as indirect correlation. To give an example for direct correlations based on our current enrichment implementation, for instance, the three entities mentioned above (\"Ireland\", \"Republic of Ireland\", \"Irland\"), all referencing the same real-world entity, are each associated with the same enrichments to the respective Freebase ([[http://www.freebase.com/view/en/ireland](http://www.freebase.com/view/en/ireland)]([http://www.freebase.com/view/en/ireland](http://www.freebase.com/view/en/ireland))) and DBpedia ([[http://dbpedia.org/resource/Ireland](http://dbpedia.org/resource/Ireland)]([http://dbpedia.org/resource/Ireland](http://dbpedia.org/resource/Ireland))) entries. Therefore, correlated ARCOMEM entities (and hence, the Web objects that these entities originate from) can be clustered directly by identifying joint enrichments of individual entities. In addition, the retrieved enrichments associate (interlink) the ARCOMEM data and Web objects with the vast knowledge, _i.e._, data graph, available in the LOD cloud, thus allowing one to retrieve additional related information for particular ARCOMEM entities. For instance, the DBpedia RDF description of Ireland ([[http://dbpedia.org/page/Ireland](http://dbpedia.org/page/Ireland)]([http://dbpedia.org/page/Ireland](http://dbpedia.org/page/Ireland))) provides additional data, facts and knowledge (for instance, a classification as island or country, geodata, the capital or population, a list of famous Irish people and similar information) in a structured and, therefore, machine-processable form. That knowledge is used to further enrich ARCOMEM entities and to create a rich and well-interlinked (RDF) graph of Web objects and related information.