An adaptive provenance collection architecture in scientific workflow systems

I! LIST! OF! FIGURES! VII! LIST! OF! TABLES! X! LIST! OF! ABBREVIATIONS! XI! DECLARATION! XII! ACKNOWLEDGEMENTS! XIII! 1! INTRODUCTION! 1! 1.1! SCIENTIFIC! WORKFLOW! MANAGEMENT! SYSTEM! 3! 1.2! CASE! STUDY! OF! PROVENANCE! IN! A! WORKFLOW! SYSTEM! 5! 1.3! PROVENANCE! COLLECTION! MECHANISMS! IN! WORKFLOW! SYSTEMS! 8! 1.3.1! Adaptive,provenance,collection,in,workflow,system,. ,9! 1.4! THESIS! CONTRIBUTION! 12! 1.5! THESIS! OUTLINE! 12! 1.5.1! An,“Aspect;oriented”,Thesis,Outline,;,the,Concerns,of,the,Thesis,. ,14! 2! LITERATURE! REVIEW! 17! 2.1! PROVENANCE! 18! 2.1.1! Provenance,Concepts,. ,18! 2.1.2! Provenance,Systems,in,eScience,. ,22! 2.1.3! Architectural,layers,of,provenance,systems,. ,26! 2.2! SCIENTIFIC! WORKFLOW! MANAGEMENT! SYSTEM! 27! 2.2.1! Reference,Model,for,SWfMS,. ,28! 2.3! PRINCIPLES! OF! WORKFLOW:! EXECUTION! AND! CONTROL! 30! 2.3.1! Workflow,scheduling,. ,33! 2.3.2! Workflow,Engine,. ,36! 2.3.3! Workflow,Controller,. ,36! 2.4! TOWARDS! ADAPTIVE! PROVENANCE! IN! SCIENTIFIC! WORKFLOW! 38! 2.4.1! Reflective,Architecture,. ,39! 2.4.2! Aspect;Oriented,Programming,Architecture,. ,56! 2.5! SUMMARY! 60! iv 3! MODELS! OF! COMPUTATION! IN! SCIENTIFIC! WORKFLOW! SYSTEMS! 63! 3.1! SCIENTIFIC! WORKFLOW! SYSTEMS! WITH! UNDERLYING! MODEL! OF! COMPUTATION! 64! 3.1.1! Case,Study:,A,Simple,Dataflow,experiment,. ,65! 3.2! INFLUENCES! OF! DATAFLOW! MODEL! ON! IMPORTANT! WORKFLOW! SYSTEMS! 68! 3.2.1! The,MoC,in,PtolemyII,and,Kepler,. ,68! 3.2.2! Provenance,in,the,Kepler,workflow,system,. ,71! 3.3! THE! PROCESS! NETWORK! MODEL! AS! A! FOUNDATION! OF! WORKFLOW! 74! 3.4! PROCESS! NETWORK! APPLICATION! 77! 3.4.1! A,simple,Producer,and,Consumer,process,network,in,PNA,. ,83! 3.4.2! Implementing,a,Process,Network,case,study,. ,84! 3.5! SUMMARY! 85! 4! MODEL! OF! PROVENANCE! 87! 4.1! INTRODUCTION! 87! 4.2! A! GENERALIZED! NOTION! OF! MODEL! OF! PROVENANCE! 89! 4.2.1! A,Simple,MoP,. ,90! 4.3! REVIEWING! MODEL! OF! PROVENANCE! 95! 4.3.1! Anand’s,MoP,. ,96! 4.3.2! COMAD,MoP,. ,98! 4.3.3! Muniswamy;Reddy’s,MoP,. ,98! 4.4! OPEN! PROVENANCE! MODEL! MOP! 101! 4.4.1! The,Open,Provenance,Model,for,Workflows,MoP,. ,105! 4.4.2! Interoperability,of,OPM,in,workflow,systems,. ,106! 4.5! A! MECHANISM! FOR! MULTIPLEFGRANULARITY! PROVENANCE! 109! 4.5.1! A,design,for,Multiple;granularity,MoP,. ,111! 4.5.2! Discussion,of,multiple;granularity,provenance,. ,126! 4.6! SUMMARY! 131! 5! MECHANISMS! FOR! PROVENANCE! COLLECTION! 134! 5.1! INTRODUCTION! 134! 5.1.1! Provenance,collection,phases,in,Workflow,lifecycle,. ,136! 5.1.2! Workflow,Orientation,. ,136! 5.1.3! Level,of,abstraction,. ,137! 5.1.4! Prospective,and,Retrospective,provenance,information,. ,138! 5.1.5! Granularity,of,Provenance,Information,. ,138! 5.1. 6! Accessibility,of,detailed,information,. ,139! v 5.1. 7! Model,of,Provenance,(MoP),. ,140! 5.1. 8! Architectural,layers,of,provenance,systems,. ,140! 5.1. 9! Coupling,strategy,. ,143! 5.1.10! Storing,,accessing,and,querying,provenance,infrastructure,. ,144! 5.1.11! Provenance,representation,techniques,. ,144! 5.1.12! Types,of,instrumentation,. ,146! 5.2! REVIEW! OF! PROVENANCE! COLLECTION! WORKS! IN! WORKFLOW! SYSTEMS! 148! 5.2.1! Kepler,. ,150! 5.2.2! Matrioshka,. ,151! 5.2.3! Provenance;Aware,Storage,System,. ,153! 5.2.4! SPADE,. ,154! 5.2.5! Karma,. ,155! 5.2. 6! Pegasus,. ,158! 5.2. 7! VIEW,.