title: Reconstructing Data Provenance from Log Files

authors: Tan, Yu Shyang

subjects: Data Provenance

abstract: Data provenance describes the derivation history of data, capturing details such as the entities involved and the relationships between entities. Knowledge of data provenance can be used to address issues, such as data quality assurance, data audit and system security. However, current computer systems are usually not equipped with means to acquire data provenance. Modifying underlying systems or introducing new monitoring software for provenance logging may be too invasive for production systems. As a result, data provenance may not always be available.

This thesis investigates the completeness and correctness of data provenance reconstructed from log files with respect to the actual derivation history. To accomplish this, we designed and tested a solution that first extracts and models information from log files into provenance relations then reconstructs the data provenance from those relations. The reconstructed output is then evaluated against the ground truth provenance. The thesis also details the methodology used for constructing a dataset for provenance reconstruction research.

Experimental results revealed data provenance that completely captures the ground truth can be reconstructed from system-layer log files. However, the outputs are susceptible to errors generated during event logging and errors induced by program dependencies. Results also show that usage of log files of different granularities collected from the system can help resolve logging errors described. Experiments with removing suspected program dependencies using approaches such as blacklisting and clustering have shown that the number of errors can be reduced by a factor of one hundred. Conclusions drawn from this research contribute towards the work on using reconstruction as an alternative approach for acquiring data provenance from computer systems.

text: Reconstructing Data Provenance from Log Files Data provenance describes the derivation history of data, capturing details such as the entities involved and the relationships between entities. Knowledge of data provenance can be used to address issues, such as data quality assurance, data audit and system security. However, current computer systems are usually not equipped with means to acquire data provenance. Modifying underlying systems or introducing new monitoring software for provenance logging may be too invasive for production systems. As a result, data provenance may not always be available.

This thesis investigates the completeness and correctness of data provenance reconstructed from log files with respect to the actual derivation history. To accomplish this, we designed and tested a solution that first extracts and models information from log files into provenance relations then reconstructs the data provenance from those relations. The reconstructed output is then evaluated against the ground truth provenance. The thesis also details the methodology used for constructing a dataset for provenance reconstruction research.

Experimental results revealed data provenance that completely captures the ground truth can be reconstructed from system-layer log files. However, the outputs are susceptible to errors generated during event logging and errors induced by program dependencies. Results also show that usage of log files of different granularities collected from the system can help resolve logging errors described. Experiments with removing suspected program dependencies using approaches such as blacklisting and clustering have shown that the number of errors can be reduced by a factor of one hundred. Conclusions drawn from this research contribute towards the work on using reconstruction as an alternative approach for acquiring data provenance from computer systems.

year: 2017

title: Seasonal abundance of small cladocerans in Lake Mangakaware, Waikato, New Zealand

authors: Greenwood, Tracey L.

subjects: biology

abstract: The seasonal changes in the dynamics and life histories of the Cladocera in Lake Mangakaware, North Island, New Zealand, were studied over 19 months by sampling at weekly or 2-weekly intervals. Lake Mangakaware is a 13.3 ha polymictic lake with high nutrient status, low Secchi disc transparencies, and an unstable thermal regime. The four planktonic cladoceran species (Bosmina longirostris, B. meridionalis, Ceriodaphnia pulchella, and C. dubia) exhibited disjunct population maxima. Only B. longirostris was perennially present. All species exhibited low fecundities and low lipid content, indicating that food resources were limited and that competitive interactions and resistance to starvation were probably important in determining species success. Increases in body size in cooler seasons were unrelated to clutch size, giving further support for the view that available food was limited. These results are consistent with previous experimental findings that subtle differences in life history can determine seasonal success and the outcome of competition between similar species.

text: Seasonal abundance of small cladocerans in Lake Mangakaware, Waikato, New Zealand The seasonal changes in the dynamics and life histories of the Cladocera in Lake Mangakaware, North Island, New Zealand, were studied over 19 months by sampling at weekly or 2-weekly intervals. Lake Mangakaware is a 13.3 ha polymictic lake with high nutrient status, low Secchi disc transparencies, and an unstable thermal regime. The four planktonic cladoceran species (Bosmina longirostris, B. meridionalis, Ceriodaphnia pulchella, and C. dubia) exhibited disjunct population maxima. Only B. longirostris was perennially present. All species exhibited low fecundities and low lipid content, indicating that food resources were limited and that competitive interactions and resistance to starvation were probably important in determining species success. Increases in body size in cooler seasons were unrelated to clutch size, giving further support for the view that available food was limited. These results are consistent with previous experimental findings that subtle differences in life history can determine seasonal success and the outcome of competition between similar species.

year: 1999-03

title: Breaking the browsing barrier for historic search of newspaper texts

authors: Keegan, Te Taka Adrian Gregory

subjects: computer science

abstract: Traditionally, to find information in a newspaper collection it has been necessary to painstakingly browse through the individual issues hoping that relevant words will catch your eye. Researchers of New Zealand’s encounter history received a boost when, in 1996, the Alexander Turnbull Library produced a collection on microfiche totaling almost 18,000 pages and covering a printing period from 1842 to 1933. While having all the newspapers in a single collection was a large step forward, browsing or searching for information in this microfiche collection was still time consuming.
By incorporating the collection into a digital library with an Internet interface, and by enabling full-text search, we have broken this browsing barrier. Making the information available this way is even more significant as the majority of the collection is written in the Māori language; we have created a Māori language resource that is sorely needed by education and research institutes, and one that provides quick and accurate access to the previously obscure source. The process involved in developing this unique digital library collection, the advantages of traditional newspaper stored in this medium, and the possibilities that we intend to investigate in the future, will be discussed in this paper.

text: Breaking the browsing barrier for historic search of newspaper texts Traditionally, to find information in a newspaper collection it has been necessary to painstakingly browse through the individual issues hoping that relevant words will catch your eye. Researchers of New Zealand’s encounter history received a boost when, in 1996, the Alexander Turnbull Library produced a collection on microfiche totaling almost 18,000 pages and covering a printing period from 1842 to 1933. While having all the newspapers in a single collection was a large step forward, browsing or searching for information in this microfiche collection was still time consuming.
By incorporating the collection into a digital library with an Internet interface, and by enabling full-text search, we have broken this browsing barrier. Making the information available this way is even more significant as the majority of the collection is written in the Māori language; we have created a Māori language resource that is sorely needed by education and research institutes, and one that provides quick and accurate access to the previously obscure source. The process involved in developing this unique digital library collection, the advantages of traditional newspaper stored in this medium, and the possibilities that we intend to investigate in the future, will be discussed in this paper.

year: 2001