Regions In a Linked Dataset For Change Detection

Linked Datasets (LDs) are constantly evolving and the applications using a Linked Dataset (LD) may face several issues such as outdated data or broken interlinks due to evolution of the dataset. To overcome these issues, the detection of changes in LDs during their evolution has proven crucial. As LDs evolve frequently, the change detection during the evolution should also be done at frequent intervals. However, due to limitation of available computational resources such as capacity to fetch data from LD and time to detect changes, the frequent change detection may not be possible with existing change detection techniques. This research proposes to explore the notion of prioritization of regions (subsets) in LDs for change detection with the aim of achieving optimal accuracy and efficient use of available computational resources. This will facilitate the detection of changes in an evolving LD at frequent intervals and will allow the applications to update their data closest to real-time data.

Related papers:

title: DSNotify - A solution for event detection and link maintenance in dynamic datasets
abstract: The dynamics of linked datasets may lead to broken links if data providers do not react to changes appropriately. Such broken links denote interrupted navigational paths between resources and may lead to unavailability of data. As a possible solution, we developed DSNotify, an event-detection framework that informs actors about various types of changes and allows them to maintain links to resources in distributed linked data sets. For representing changes we developed the DSNotify Eventset Vocabulary. Different from other vocabularies it applies a resource-centric perspective and preserves the timely order of changes. We further describe our reusable evaluation infrastructure, which can be extended for extracting change sets from arbitrary linked datasets.

title: RDF data evolution: efficient detection and semantic representation of changes
abstract: Many RDF data sources are constantly changing for both data and vocabulary (ontology) levels. Many integration tasks are impacted by these changes. In this context, it is important to develop approaches to detect and represent these changes. Many studies have focused on the detection , the representation and the management of changes at the ontology level. In this paper, we present an approach which allows to detect and represent elementary and complex changes that can be detected when we focus only on the data level. A first experiment was conducted on di↵er-ent versions of DBpedia.

title: Strategies for Efficiently Keeping Local Linked Open Data Caches Up-To-Date
abstract: Quite often, Linked Open Data (LOD) applications pre-fetch data from the Web and store local copies of it in a cache for faster access at runtime. Yet, recent investigations have shown that data published and interlinked on the LOD cloud is subject to frequent changes. As the data in the cloud changes, local copies of the data need to be updated. However, due to limitations of the available computational resources (e.g., network bandwidth for fetching data, computation time) LOD applications may not be able to permanently visit all of the LOD sources at brief intervals in order to check for changes. These limitations imply the need to prioritize which data sources should be considered first for retrieving their data and synchronizing the local copy with the original data. In order to make best use of the resources available, it is vital to choose a good scheduling strategy to know when to fetch data of which data source. In this paper, we investigate different strategies proposed in the literature and evaluate them on a large-scale LOD dataset that is obtained from the LOD cloud by weekly crawls over the course of three years. We investigate two different setups: (i) in the single step setup, we evaluate the quality of update strategies for a single and isolated update of a local data cache, while (ii) the iterative progression setup involves measuring the quality of the local data cache when considering iterative updates over a longer period of time. Our evaluation indicates the effectiveness of each strategy for updating local copies of LOD sources, i. e, we demonstrate for given limitations of bandwidth, the strategies’ performance in terms of data accuracy and freshness. The evaluation shows that the measures capturing change behavior of LOD sources over time are most suitable for conducting updates.

title: Concept drift and how to identify it
abstract: This paper studies concept drift over time. We first define the meaning of a concept in terms of intension, extension and label. Then we study concept drift over time using two theories: one based on concept identity and one based on concept morphing. A qualitative toolkit for analysing concept drift is proposed to detect concept shift and stability when concept identity is available, and concept split and strength of morphing chain if using the morphing theory. We apply our framework in four case-studies: a political vocabulary in SKOS, the DBpedia ontology in RDFS, the LKIF-Core ontology in OWL and a few biomedical ontologies in OBO. We describe ways of identifying interesting changes in the meaning of concept within given application contexts. These case-studies illustrate the feasibility of our framework in analysing concept drift in knowledge organisation schemas of varying expressiveness.

title: Observing Linked Data Dynamics
abstract: In this paper, we present the design and first results of the Dynamic Linked Data Observatory: a long-term experiment to monitor the two-hop neighbourhood of a core set of eighty thousand diverse Linked Data documents on a weekly basis. We present the methodology used for sampling the URIs to monitor, retrieving the documents, and further crawling part of the two-hop neighbourhood. Having now run this experiment for six months, we analyse the dynamics of the monitored documents over the data collected thus far. We look at the estimated lifespan of the core documents, how often they go on-line or off-line, how often they change; we further investigate domain-level trends. Next we look at changes within the RDF content of the core documents across the weekly snapshots, examining the elements (i.e., triples, subjects, predicates, objects, classes) that are most frequently added or removed. Thereafter, we look at how the links between dereferenceable documents evolves over time in the two-hop neighbourhood.

title: Interest-Based RDF Update Propagation
abstract: Many LOD datasets, such as DBpedia and LinkedGeoData, are voluminous and process large amounts of requests from diverse applications. Many data products and services rely on full or partial local LOD replications to ensure faster querying and processing. Given the evolving nature of the original and authoritative datasets, to ensure consistent and up-to-date replicas frequent replacements are required at a great cost. In this paper, we introduce an approach for interest-based RDF update propagation, which propagates only interesting parts of updates from the source to the target dataset. Effectively, this enables remote applications to 'subscribe' to relevant datasets and consistently reflect the necessary changes locally without the need to frequently replace the entire dataset or a relevant subset. Our approach is based on a formal definition for graph-pattern-based interest expressions that is used to filter interesting parts of updates from the source. We implement the approach in the iRap framework and perform a comprehensive evaluation based on DBpedia Live updates, to confirm the validity and value of our approach.

title: Dataset Dynamics Compendium: A Comparative Study
abstract: At the time of writing there exists no consensus about the approaches to detect, propagate and describe changes in resources and datasets of the Linked Open Data Web. This survey gives a comprehensive overview of the current technical solutions and a comparison of such based requirements we derived from use cases the community came up with. We give a detailed overview about the aspects of discovery, granularity level, and description of the changes, as well as the detection algorithms and notification mechanisms. Moreover, we present a high-level dataset dynamics stack that integrates the current technical solutions for dealing with changes in datasets of the Web of Data.

title: Towards Dataset Dynamics: Change Frequency of Linked Open Data Sources
abstract: Datasets in the LOD cloud are far from being static in their nature and how they are exposed. As resources are added and new links are set, applications consuming the data should be able to deal with these changes. In this paper we investigate how LOD datasets change and what sensible measures there are to accommodate dataset dynamics. We compare our findings with traditional, document-centric studies concerning the “freshness” of the document collections and propose metrics for LOD datasets.

title: From Changes to Dynamics: Dynamics Analysis of Linked Open Data Sources
abstract: The Linked Open Data (LOD) cloud changes frequently. Recent approaches focus mainly on quantifying the changes that occur in the LOD cloud by comparing two snapshots of a linked dataset captured at two different points in time. These change metrics are able to measure absolute changes between these two snapshots. However, they cannot determine the dynamics of a dataset over a period of time, i.e., the intensity of how the data evolved in this period. In this paper, we present a general framework to analyze the dynamics of linked datasets within a given time interval. We propose a function to measure the dynamics of a LOD dataset, which is defined as the aggregation of absolute, infinitesimal changes, provided by change metrics. Our method can be parametrized to incorporate and make use of existing change metrics. Furthermore, our framework enables the use of different decay functions within the dynamics computation for different weights on changes depending on when they occurred in the observed time interval. We apply our framework to conduct an investigation on the dynamics of selected LOD datasets. We apply our analysis on a large-scale LOD dataset that is obtained from the LOD cloud by weekly crawls over more than a year. Finally, we discuss the benefits and potential applications of our dynamics function in a real world scenario.

title: Keeping linked open data caches up-to-date by predicting the life-time of RDF triples
abstract: Many Linked Open Data applications require fresh copies of RDF data at their local repositories. Since RDF documents constantly change and those changes are not automatically propagated to the LOD applications, it is important to regularly visit the RDF documents to refresh the local copies and keep them up-to-date. For this purpose, crawling strategies determine which RDF documents should be preferentially fetched. Traditional crawling strategies rely only on how an RDF document has been modified in the past. In contrast, we predict on the triple level whether a change will occur in the future. We use the weekly snapshots of the DyLDO dataset as well as the monthly snapshots of the Wikidata dataset. First, we conduct an in-depth analysis of the life span of triples in RDF documents. Through the analysis, we identify which triples are stable and which are ephemeral. We introduce different features based on the triples and apply a simple but effective linear regression model. Second, we propose a novel crawling strategy based on the linear regression model. We conduct two experimental setups where we vary the amount of available bandwidth as well as iteratively observe the quality of the local copies over time. The results demonstrate that the novel crawling strategy outperforms the state of the art in both setups.

title: High-level change detection in RDF(S) KBs
abstract: With the increasing use of Web 2.0 to create, disseminate, and consume large volumes of data, more and more information is published and becomes available for potential data consumers, that is, applications/services, individual users and communities, outside their production site. The most representative example of this trend is Linked Open Data (LOD), a set of interlinked data and knowledge bases. The main challenge in this context is data governance within loosely coordinated organizations that are publishing added-value interlinked data on the Web, bringing together issues related to data management and data quality, in order to support the full lifecycle of data production, consumption, and management. In this article, we are interested in curation issues for RDF(S) data, which is the default data model for LOD. In particular, we are addressing change management for RDF(S) data maintained by large communities (scientists, librarians, etc.) which act as curators to ensure high quality of data. Such curated Knowledge Bases (KBs) are constantly evolving for various reasons, such as the inclusion of new experimental evidence or observations, or the correction of erroneous conceptualizations. Managing such changes poses several research problems, including the problem of detecting the changes (delta) between versions of the same KB developed and maintained by different groups of curators, a crucial task for assisting them in understanding the involved changes. This becomes all the more important as curated KBs are interconnected (through copying or referencing) and thus changes need to be propagated from one KB to another either within or across communities. This article addresses this problem by proposing a change language which allows the formulation of concise and intuitive deltas. The language is expressive enough to describe unambiguously any possible change encountered in curated KBs expressed in RDF(S), and can be efficiently and deterministically detected in an automated way. Moreover, we devise a change detection algorithm which is sound and complete with respect to the aforementioned language, and study appropriate semantics for executing the deltas expressed in our language in order to move backwards and forwards in a multiversion repository, using only the corresponding deltas. Finally, we evaluate through experiments the effectiveness and efficiency of our algorithms using real ontologies from the cultural, bioinformatics, and entertainment domains.

title: A Flexible Framework for Understanding the Dynamics of Evolving RDF Datasets
abstract: The dynamic nature of Web data gives rise to a multitude of problems related to the description and analysis of the evolution of RDF datasets, which are important to a large number of users and domains, such as, the curators of biological information where changes are constant and interrelated. In this paper, we propose a framework that enables identifying, analysing and understanding these dynamics. Our approach is flexible enough to capture the peculiarities and needs of different applications on dynamic data, while being formally robust due to the satisfaction of the completeness and unambiguity properties. In addition, our framework allows the persistent representation of the detected changes between versions, in a manner that enables easy and efficient navigation among versions, automated processing and analysis of changes, cross-snapshot queries spanning across different versions, as well as queries involving both changes and data. Our work is evaluated using real Linked Open Data, and exhibits good scalability properties.

title: Repairing broken RDF links in the web of data
abstract: In the web of data, linked datasets are changed over time. These changes include updating on features and address of entities. The address change in RDF entities causes their corresponding links to be broken. Broken link is one of the major obstacles that the web of data is facing. Most approaches to solve this problem attempt to fix broken links at the destination point. These approaches have two major problems: a single point of failure; and reliance on the destination data source. In this paper, we introduce a method for fixing broken links which is based on the source point of links, and discover the new address of the detached entity. To this end, we introduce two datasets, which we call 'superior' and 'inferior'. Through these datasets, our method creates an exclusive graph structure for each entity that needs to be observed over time. This graph is used to identify and discover the new address of the detached entity. Afterward, the most similar entity, which is candidate for the detached entity, is deduced and suggested by the algorithm. The proposed model is evaluated with DBpedia dataset within the domain of 'person' entities. The result shows that most of the broken links, which had referred to a 'person' entity in DBpedia, had been fixed correctly.

title: Evaluating Query and Storage Strategies for RDF Archives
abstract: There is an emerging demand on efficiently archiving and (temporal) querying different versions of evolving semantic Web data. As novel archiving systems are starting to address this challenge, foundations/standards for benchmarking RDF archives are needed to evaluate its storage space efficiency and the performance of different retrieval operations. To this end, we provide theoretical foundations on the design of data and queries to evaluate emerging RDF archiving systems. Then, we instantiate these foundations along a concrete set of queries on the basis of a real-world evolving dataset. Finally, we perform an empirical evaluation of various current archiving techniques and querying strategies on this data. Our work comprises -- to the best of our knowledge -- the first benchmark for querying evolving RDF data archives.