Pathomx: an interactive workflow-based tool for the analysis of metabolomic data Background: Metabolomics is a systems approach to the analysis of cellular processes through small-molecule metabolite profiling. Standardisation of sample handling and acquisition approaches has contributed to reproducibility. However, the development of robust methods for the analysis of metabolomic data is a work-in-progress. The tools that do exist are often not well integrated, requiring manual data handling and custom scripting on a case-by-case basis. Furthermore, existing tools often require experience with programming environments such as MATLAB® or R to use, limiting accessibility. Here we present Pathomx, a workflow-based tool for the processing, analysis and visualisation of metabolomic and associated data in an intuitive and extensible environment. Results: The core application provides a workflow editor, IPython kernel and a HumanCyc™-derived database of metabolites, proteins and genes. Toolkits provide reusable tools that may be linked together to create complex workflows. Pathomx is released with a base set of plugins for the import, processing and visualisation of data. The IPython backend provides integration with existing platforms including MATLAB® and R, allowing data to be seamlessly transferred. Pathomx is supplied with a series of demonstration workflows and datasets. To demonstrate the use of the software we here present an analysis of 1D and 2D 1 H NMR metabolomic data from a model system of mammalian cell growth under hypoxic conditions.Pathomx is a useful addition to the analysis toolbox. The intuitive interface lowers the barrier to entry for non-experts, while scriptable tools and integration with existing tools supports complex analysis. We welcome contributions from the community. Pathomx is a workflow-based tool for the exploration of metabolomic data. It supports a complete processing workflow through data import, processing, analysis and visualisation. It is open source and features a plugin system that can be readily extended with new features and integrates readily with existing tools. Workflow construction requires no prior programming knowledge but can utilise it where available. The resulting workflows can be shared and re-used or exported as standalone Python scripts. Plugin development is supported through a simple, well-documented Python-based API. We welcome contributions of plugins and workflows from the community. Metabolomics is a systems approach to the analysis of cellular processes through small-molecule metabolite profiles of a cell, tissue, organ or organism that results from the combined action of proteome, transcriptome and genome [1] . Metabolomics can be split broadly into targeted and untargeted approaches. Targeted metabolomics uses focused study of known pathways, reactions or metabolites in in vitro cell models and has been used to gain insight into metabolic requirements and vulnerabilities of cancer cells [2] . Untargeted metabolomics is a hypothesis-forming approach in which datasets derived from biological fluids are queried using multivariate analysis techniques, with the goal of identifying biomarkers or metabolic changes that can inform future study. This approach has been successfully employed for the identification of novel disease markers [3] .. The standardisation of sample handling and data acquisition has contributed to improved reproducibility in metabolomics [4] . Data analysis methods in contrast are less well defined. Existing tools commonly build on mathematical environments, such as MATLAB® or R and require a level of familiarity not usually available in those from non-mathematical backgrounds. The difficulties moving data between these environments and associated packages is a hindrance to an integrated workflow. In our own group we have used this type of hybrid platform, combining MATLAB®-based NMRLab and MetaboLab [5] for processing and PLS Toolbox (Eigenvector Research, Wenatchee WA USA) for multivariate analysis, with Chenomx (Edmonton, Alberta, Canada) and the Human Metabolome Database [6] for metabolite identification. It is our experience that the complexity of the analysis workflow acts as a significant barrier to the use of metabolomics by non-experts, hinders discovery and slows throughput.. These issues are not unique to metabolomic analysis and the preceding decade has seen work to address them within the bioinformatics field. Scientific workflow tools have emerged in recent years as a powerful and flexible approach to the analysis of large datasets [7] . Automation of workflows can contribute to the reproducibility of analysis and reduction in error, while simultaneously increasing throughput. The major workflow analysis platforms in current use are Taverna [8] and Galaxy [9] , which have established themselves as key tools in the bioinformaticians' toolkit. Both share a common approach of stepwise workflow-construction paired with server-based batch processing, yet differ on the level of abstraction of their components. Taverna is a low-level workflow creator, offering construction of complex functions from discrete algorithmic steps and with a particular focus on remote service integration. Galaxy in contrast offers high-level components that perform common bioinformatics tasks wholesale, with a focus on local-service integration and the need for no programming experience. Both platforms have been developed with a focus on genomic and transcriptomics analysis and lack support for the analysis of metabolomic data. The batch-based processing paradigm also limits application to the steps of analysis that can be fully automated while the latter stages of metabolomic data analysis are typically more exploratory, with iterative application of multivariate techniques, interrogation of biological databases, and pathway visualisation for interpretation of the data. Tools are already available to aid in the various stages of metabolomic data analysis, with MetaboAnalyst [10] , a webbased metabolomic analysis pipeline, being of particular note. It includes modules for enrichment, pathway and time-series analysis, and has a particular focus on usability with the complete pipeline configurable through a simple web-based interface. However, this simplicity does come at the cost of the adaptability and automation that workflow analysis can offer. Further, the inability to adapt or extend analysis modules means that complete analysis of a dataset will often require other tools.. Recognising the benefits that workflow-based analysis could offer to metabolomics analysis while hoping to overcome the limitations of batch-based processing, we developed Pathomx: a workflow-based tool for data analysis. The software is designed to be adaptable, intuitive and to integrate well with existing tools and pipelines, acting as the essential glue in the metabolomics toolbox. Pathomx is an open source and cross-platform analysis tool. It is developed in Python (v2.7; Python Software Foundation) with a graphical user interface (GUI) based on Qt (v5.1; Digia) and graphing powered by Matplotlib (v1.1.1) [11] . The processing kernel is based on IPython (v3.0.0).. In Pathomx nomenclature plugins provide tools that are then used for construction of workflows. The software ships with a base set of tools for data import, processing, analysis, visualisation and export based on the NumPy (v1.7.1), SciPy (v0.12.0) [12] , Pandas (v0.14.1), SkiKit-Learn (v0.15.1) [13] and NMRGlue (v.0.4) [14] Python packages. Many of the algorithms in the default toolkit have subsequently been released as standalone Python packages to allow use outside Pathomx. These include biocyc (v0.1.0) a Python BioCyc API, gpml2svg (v0.3.0) a GPML renderer, icoshift (v0.6.0) a Python implementation of the Icoshift algorithm [15] , metaboviz (v0.0.3) a metabolic pathway drawing package utilising the pydot (v1.0.28) interface to Graphviz (v2.12) [16] and pathminer (v0.0.2) a metabolic pathway mining algorithm. The functionality described in this paper relates to the base plugins provided with Pathomx 3.0.. Data analysis workflows are constructed using a drag-and-drop interface. Dragging a tool from the toolkit creates a new tool in the workflow. Selecting the tool allows configuration options to be changed, data sources to be configured and the tool code to be run. Inputs can also be managed directly from the workflow editor by dragging the output of one tool into the appropriate input of another. Recalculation and regeneration of figures is dynamic and the current run-state is visualised within the editor (blue = complete, red = error, green = active). The default toolkit makes extensive use of Pandas DataFrames and standard structures to allow tools to communicate easily. Tools can make use of parallel processing to allow efficient execution of complex workflows on large datasets on a standard modern desktop machine. Errors are flagged with both descriptive text and kernel backtraces for debugging purposes. Source code is available for all tools and can be modified using the inline editor to tweak behaviour. Resulting workflows can be exported as standalone scripts to run independently of the Pathomx environment. Figures can be exported as high-resolution TIF files for publication. Interfaces to both MATLAB® and R are available through the IPython backend allowing data to be passed between environments as required. Data may be imported from a number of other tools and public databases, including Metabolights [17] and Gene Expression Omnibus (GEO) [18] .. Pathomx includes a subset of the HumanCyc™ Homo sapiens pathway data available under license from SRI International [19] . Database cross-referencing is supported for KEGG [20] , HMDB [6] and other databases, generated from BioCyc annotations and the MNXref database [21] .

O O O O O O O B O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O B O O O O O O O O B O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B B O O O O O O O O O B O O O O B O O O O O O O O O O O O O B O O O B B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Distributed Data Replenishment Abstract-We propose a distributed data replenishment mechanism for some distributed peer-to-peer-based storage systems that automates the process of maintaining a sufficient level of data redundancy to ensure the availability of data in presence of peer departures and failures. The dynamics of peers entering and leaving the network are modeled as a stochastic process. A novel analytical time-backward technique is proposed to bound the expected time for a piece of data to remain in P2P systems. Both theoretical and simulation results are in agreement, indicating that the data replenishment via random linear network coding (RLNC) outperforms other popular strategies. Specifically, we show that the expected time for a piece of data to remain in a P2P system, the longer the better, is exponential in the number of peers used to store the data for the RLNC-based strategy, while they are quadratic for other strategies. R ECENT development of Peer-to-Peer (P2P) networks opens a new possibility for building large scale distributed systems over the Internet. Typically in such systems, data are replicated across multiple nodes (peers) at different network locations such that network failures in some parts of the Internet will not prevent a user from accessing the data stored in other parts of the Internet. To that end, many recent research efforts have been focused on using P2P platforms to build reliable, large scale distributed systems for Internet services [1] , [2] , [3] , [4] . In this paper, we investigate some theoretical underpinnings and examine the simulated performance for a class of large scale distributed systems based on a randomized P2P approach via coding techniques.. In a nutshell, a distributed system over the Internet is an overlay network of storage and computing nodes, linked together in such a way to allow computational, storage, and bandwidth resources to be shared. Popular P2P networks such as BitTorrent [5] and KaZaA [6] , for example, are distributed systems that enable their users to share data and bandwidth. Since the overlay nodes are located geographically apart, each node has a different network access, and data are replicated across multiple nodes, these systems are less susceptible to the bottleneck failures. However, if not properly designed, they will incur substantial communication/coordination overheads among nodes. For a distributed storage system, one of the main challenges is to design efficient coordination mechanisms among nodes in order to maintain the integrity and availability of data in the system while minimizing the communication overheads.. Data replenishment. Distributed storage system research have been focused on indexing, maintaining, and retrieving data correctly and efficiently. In this paper, we will not discuss various aspects of indexing and retrieving data. These topics have been well investigated in [1] , [2] , [7] , [8] , [9] , [10] , [11] , [12] , [13] . Rather, we will focus on theoretical analysis of scalable methods for maintaining data in a highly volatile environment such as P2P networks. Specifically, in a P2P network, the data are stored on a peer's hard drive. Consequently, when a peer departs the network, so does the data it carries. Therefore, it is preferable to employ some form of data replenishment mechanism which ensures that at any time, the requested data are available at one or multiple peers collectively. Furthermore, the data replenishment mechanism should be simple and preferably distributed for it to be effective in highly dynamic environments. Data replenishment mechanism is the focus of this paper.. Approach overview. Traditionally in a distributed storage system, a file is replicated in its entirety at one or multiple locations. However, for the same overall storage redundancy, a more robust approach is to break up a single file into many pieces, code these pieces properly, then disperse them to multiple nodes in a network [14] , [15] , [16] , [17] . A user recovers the file by downloading its many pieces simultaneously from different locations. In this paper, we consider the following variant of the setup described in [16] , [18] , [19] . In this setup, a file to be stored, is first broken up into multiple pieces or packets, coded using either Reed-Solomon, repetition, or random linear network codes (RLNC), then dispersed to a number of peers in the network. Now, any peer can depart the network along with its data. If a new peer joins, it can be recruited to help replenish the missing data.. There are many ways to replenish the missing data. We will show that the data replenishment via RLNC is much more efficient than the strategies using repetition and traditional channel code. We do not consider other coding schemes such as turbo and convolution codes since they are perhaps more suitable for communication applications. We note that the concept of data replenishment is very much similar to data repair as termed in [18] . Thus, our work is not new in the sense of using RLNC for distributed data storage. In [16] , [17] , [18] , [19] , the authors examined the fundamental tradeoff between the replenishment bandwidth and storage capacity in a static setting. Our work on the other hand, does not consider this fundamental tradeoff. Instead, we focus on the analysis of different data replenishment techniques and their effects on data recoverability over time. Specifically, we show that RLNC-based storage technique in P2P environments will, on average, result in higher data availability over time than other techniques.. The outline of our paper is as follows: In Section 2, we provide a brief review on the recent advances in distributed storage with an emphasis on network coding. In Section 3, we describe different replenishment strategies for a synchronous network model. In Section 4, we model the evolution of a piece of data through time for different replenishment schemes as discrete stochastic processes. These processes however are intractable, therefore we propose a time-backward model based on which, we are able determine an approximate closed-form expression for the elapsed time that a piece of data are expected to remain in the system. We call this expected time the absorption time. Our analytical results show that using the data replenishment via RLNC, the absorption time is exponential in the number of peers used to store the data. This is much more robust than other data protection strategies based on repetition or channel coding techniques whose absorption times are quadratic in the number of peers. As an extension, in Section 8, we present an analysis for an asynchronous model which describes peer arrivals and departures as Poisson processes. Our results indicate that the performance of the asynchronous model depends critically on the peer arrival and departure rates. Another way to view this is that if the elements are drawn from IR, then a randomly drawn row will definitely be independent from any other M equations because jIRj is infinite. We will make this approximation to model the replenishment process as follows:. For simplicity, we will focus on the following simple scenario. A file is broken up into K parts, then N codewords are generated by linearly combining these three parts (codewords) at random. Now, at every time step, a new peer is chosen uniformly at random to depart. A new peer joins. Two distinct remaining peers ðM ¼ 2Þ are then uniformly chosen at random to have their codewords copied to the new peer. The new peer then generates its new codeword by linearly combining these two codewords with random coefficients. In general, if M ! K, it will almost always be possible to recover the file, independent of the number of replenishments. This is because we can almost always get K linearly independent codewords, unless with a small probability, the generated codeword happens to be linearly dependent on some M 0 < K codewords. We use the diagram in Fig. 1 to visually depict how the dependencies among the codewords progress in discrete time steps. The meanings of the solid and nonsolid circles will become clear shortly when we discuss the time-backward A circle in the current time step that is connected to two circles in the previous time step, represents a codeword that is a linear combination of two codewords. Solid circles are parent codewords that are part of the linear combinations in the current codewords, while nonsolid circles are not part of the linear combinations of the current codewords.. 1. It is straightforward to show that the lower bound for this probability. process. For now, at time step t ¼ 0, there are seven codewords. Any of these codewords can be represented as a linearly combination of any three other codewords due to the initial mixing. At time step t ¼ 1, the codeword 6 is replaced by a random linear combination of codewords 5 and 7. At this stage, the file can be recovered using any triplet of codewords except (5, 6, 7) since these three codewords are not linearly independent. At t ¼ 2, codeword 2 is replaced by a random linear combination of codewords 1 and 3. As such, one cannot use triplets (5, 6, 7) or (1, 2, 3 ) to recover the file at this time. The process repeats, and eventually, all codewords will be some linear combinations of some two codewords, and the file will no longer be recoverable.. A. Our contribution is a modeling technique that produces an approximate but closed-form solution for the expected number of time steps to get from any state to any other, including the state in which the file is no longer recoverable. Furthermore, we can bound the error on this approximate time by a factor of 2. The key to this modeling technique is to consider a more tractable time-backward model in which the replenishments are performed backward in time. In conclusion, we suggest that, to maintain data for as long as possible in a distributed setting with limited peer communication and storage, it is better to mix the data as proposed in the RLNC strategy. We show that the average number of replenishments before a file is no longer recoverable is exponential in the number of peers used store the data distributedly for RLNC-based strategy and quadratic for other traditional strategies. We also propose a novel time-backward technique to approximate the mean absorption time. We believe this technique is general as such, it can be applied to other problems such as gene population in which, it is intractable to directly model an exponentially large number of states using Markov chain representations. Yevgeniy Kovchegov received the BA degree in mathematics from NYU in 1997, the MS degree in mathematics from Stanford in 2000, and the PhD degree in mathematics from Stanford in 2002. After spending three years at UCLA Math Department as a visiting faculty, he joined Oregon State University (OSU) Department of Mathematics as a tenure-track assistant professor. In 2011, he was promoted to the rank of associate professor with indefinite tenure. He works in the field of probability and stochastic processes. His research interests include the mathematical models of statistical mechanics, interacting particle systems, quantum computation, self-similarity, reinforced processes, and percolation. His work is centered around the following topics: mixing times; extending the probabilistic coupling method; orthogonal polynomials in stochastic processes, probability and statistics; quantum walks and quantum computation; occupation times; hitting times; chaos and fractals; applications of probability theory in computer and wireless networks, network coding, economics, chemistry, and environmental sciences.. Viet Le received the BE degree from the University of Technology Sydney and the MS degree from Oregon State University in 2006 and 2010, respectively. He was a recipient of the Australian Development Scholarship, and is currently with IBM Internet Security Systems group. He is interested in software development, networking, and applying theories into practice. His current and past projects span from video streaming, data warehousing, to intrusion prevention systems.. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib. In their seminal paper, Ahlswede et al. showed that it is possible to maximize the transmission rate from a source to multiple receivers (multicast capacity) by allowing the intermediate nodes to perform coding, i.e., mixing data from different flows [20] . Since then, network coding has found its way to many applications, ranging from efficient wireless communication and networking to distributed storage systems. Our work is motivated by the recent advances in network coding for distributed storage systems.. In [21] , Dimakis et al. provided a survey on recent network coding techniques for distributed storage. Much research in this area have been focused on 1) the fundamental tradeoff between the repair bandwidth and the storage capacity of nodes and 2) techniques for constructing capacity-achieving network codes. In a distributed storage system, if a node fails, a new node is recruited and attempts to reconstruct the missing data from the failed node by downloading the data from other nodes. The codes for this type of setting is called regenerating codes, and the amount of downloaded data required for reconstructing the missing data is called the repair bandwidth. For example, suppose a 1 MB portion of a file is lost, then to repair the file, the exact missing portion is needed to repair the file. So 1 MB is the repaired bandwidth. This scenario is only possible if there is the original copy of the file exists. In many distributed settings, there is no central server to keep the original copy, so it might not be possible. However, if more redundancy of the file is used in the beginning, then it is possible to reconstruct any missing portion from the remaining parts. For example, if RS (7, 4) is used, i.e., for every four original parts (1 MB each), we store three redundancy parts (parity). Furthermore, suppose there are seven peers each stores one part. Then, for any three missing parts (peers), one can reconstruct the entire file from the remaining four parts (peers). Even for one missing part, using RS(7, 4) one still needs to using all four remaining parts to regenerate the missing part. The repair bandwidth in this case is 4 MBs. It has been shown that the repair bandwidth using network coding can be substantially smaller than that of using traditional Maximum Distance Separable (MDS) codes provided that the storage capacities of nodes in the systems are sufficiently large [17] . In [22] , Wu and Dimakis showed techniques that can reduce the repair bandwidth in the case where there is only a single node failure. Wu et al. also proposed techniques for constructing MDS codes that achieve the fundamental bounds on the minimum repair bandwidth and storage capacity [23] , [24] .. Li et al. also considered joint design of regenerating codes and network topology for efficiently utilizing network links and reducing repair bandwidth [25] . Their design called RCTREE, is able to produce efficient code regeneration, even in dynamic environments where nodes enter and depart the network frequently. In [26] , Duminuco and Biersack studied computational, communication, and storage costs of a real implementation of random linear regenerating codes in peer-to-peer systems. They concluded that with a small increase in storage cost and computation, a significant reduction of the communication cost can be achieved.. Regenerating codes also finds security applications in distributed storage systems. In [27] , Pawar et al. derived the fundamental bounds on the storage capacity that guarantee against eavesdropping and adversarial attacks.. We want to point out a few differences between this work and the existing works. First, all the existing works focus on the constructing regenerating codes optimally. This requires detail information (which packets each node has) exchange between the newly joined node and the nodes it connects to for repairing the data. In contrast, our work is focused on a randomized approach in which a newly joined node connects to a few nodes at random and download data from these nodes. This method is suboptimal but is scalable since the information exchange between nodes is kept to minimal. Second, in many existing works, the authors examined the fundamental tradeoff between the repair bandwidth and storage capacity in a static setting, while our work considers the dynamics of data replenishment of different techniques and their effects on data recoverability over time. Specifically, we show that RLNC-based storage technique in P2P environments will, on average, result higher data availability over time than other techniques.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O B I O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning flexible sensori-motor mappings in a complex network Abstract Given the complex structure of the brain, how can synaptic plasticity explain the learning and forgetting of associations when these are continuously changing? We address this question by studying different reinforcement learning rules in a multilayer network in order to reproduce monkey behavior in a visuomotor association task. Our model can only reproduce the learning performance of the monkey if the synaptic modifications depend on the pre-and postsynaptic activity, and if the intrinsic level of stochasticity is low. This favored learning rule is based on reward modulated Hebbian synaptic plasticity and shows the interesting feature that the learning performance does not substantially degrade when adding layers to the network, even for a complex problem. Experimental work of Asaad et al. (1998) has revealed mechanisms which might be involved in a visuomotor learning task performed by monkeys. In this task, the monkeys are trained to associate visual stimuli (pictures) with delayed saccadic movements, left or right, with associations being reversed from time to time. In Fusi et al. (2007) , the authors hypothesize that learning and forgetting associations happens purely by synaptic modification mechanisms, on multiple time scales. To support this hypothesis, they reproduce the experimental results of Asaad et al. with a simplified onelayer network, and make further predictions which are also experimentally verified.. The objective of this paper is to investigate whether there are learning prescriptions which allow us to extend the results of Fusi et al. (2007) to the case of a multilayer network. One major limitation of the analysis of Fusi et al. (2007) was the assumption that the plastic input to the decision network was generated by a single layer of synaptic inputs. In fact, many algorithms result in a loss of performance as the number of hidden layers is increased, although this problem is often not explicitly addressed Fahlman and Lebiere (1991) , Bak and Chialvo (2001) , Hinton and Salakhutdinov (2006) ; Bengio and LeCun (2007) . We therefore ask: what are the appropriate synaptic mechanisms which allow us to reproduce the learning performance observed in this experiment within a complex network?. To address this question we consider three reinforcement learning algorithms: a novel form of reward-modulated adaptive Hebbian learning (RAH), together with the previously described node perturbation (NP) (Werfel et al. 2005 ) and the associative reward penalty (ARP) algorithm (Mazzoni et al. 1991; Barto 1985 Barto , 1989 Barto and Jordan 1989) . We investigate the performance of these algorithms on the monkey association paradigm (Asaad et al. 1998; Fusi et al. 2007) and different network architectures.. We find that both RAH and ARP can reproduce the monkey data, but not NP. In fact, adding a few layers slows down learning, much less in the case of RAH and ARP and much more in the case of NP. Nevertheless, the problem of learning with many layers is present in more general architectures (Fahlman and Lebiere 1991; Bak and Chialvo 2001; Hinton and Salakhutdinov 2006; Bengio and LeCun 2007) and though there are solutions for particular cases (Hinton and Salakhutdinov 2006; Bengio and LeCun 2007) , no global solution exists. Our simulations imply that the less noise driven a rule is, the better its performance in a deep network structure. We modeled the learning of visuomotor associations by monkeys (Asaad et al. 1998 ) using a multilayer neural network. Three reinforcement learning algorithms have been used, one novel and two from the literature. The simulation results showed that:. (1) Learning in the case of punishment must be faster than in the case of reward in order to recreate outcomes comparable to the monkey data. This is achieved by either a very low learning rate for reward as compared with punishment, or by the stop-learning criterion in case of reward. (2) In a multilayer feedforward network, the learning speed of monkeys can be reached by the reward-modulated Hebbian-type synaptic plasticity with a stop-learning criterion (i.e., by our RAH algorithm, or by ARP with an appropriately steep activation function). (3) The high stochasticity used by reinforcement learning rules to quickly learn in the case of a single hidden layer may prevent the learning of simple associations within a reasonable speed when adding more hidden layers. When the level of stochasticity is low, or if stochasticity is absent, adding a few other hidden layers does not affect the speed of learning. The experiment of Asaad et al. (1998) provides us with interesting information on how the monkey behaves in a rapidly changing environment. The monkey is not able to follow an optimal game strategy, but instead takes 30 trials to learn the task. Every time the associations are reversed, it completely forgets the old associations and learns the new ones from the beginning. This behavior can be attributed to the difficulty of the task; the two noninverting associations confuse the monkey, as they create uncertainty about the nature of the mistakes. Fusi et al. (2007) have proposed a simple associative model for reproducing this behavior, tested on a single-layer network. The model however implies that learning could take place through a multilayered network. Thus, we tested the model, implementing it in a multilayer network using one novel (RAH) and two well-known (NP and ARP) algorithms. We showed that, for the considered network architecture, only the Hebbian-type algorithms (RAH and ARP) allow us to reproduce the monkey behavior within reasonable accuracy. In contrast, the performance of NP significantly decreases as the number of association reversals increases.. Our simulations require that synaptic modifications in case of an error must be larger than in case of a successful trial to reproduce the experimental data. This imbalance in the synaptic changes provides us with an explanation for the strong reset observed after a mistake; strong synaptic changes in unsuccessful trials cause the synapses to rapidly forget their past memories and learn the new associations. The absence of pattern interference can be simultaneously achieved, as long as sparse coding in the input layer is used.. The imbalance between the synaptic changes in rewarded and nonrewarded trials may also cause undesired forgetfulness of past skills with each mistake. The sparseness in the representation of the patterns can also limit this problem. If sparseness is maintained through the structure, for different stimuli there are different learning pathways with little overlapping. Nevertheless, the problem of memory preservation while learning new features (stability-plasticity dilemma) is common to all algorithms but appears more prominent when synapses are bounded. There is no global solution to this issue, but various suggestions have been made such as smart stochastic selection of synaptic updates (Fusi and Senn 2006) , the cascade model (Fusi et al. 2005) , or modularity (Calabretta and Parisi 2005; Calabretta et al. 1998) .. In general, the Hebbian-type algorithms (RAH and ARP) seem to generalize relatively well in large associative networks of input patterns with sparse representation. We investigated their behavior on a simple linearly separable problem, without association reversals, and saw that both rules retain their performance when adding layers, in contrast to the NP. In addition, we saw that this conclusion still holds for RAH (and ARP) for a more complex associative problem, since in the limit of low neuronal stochasticity, the two rules become identical. A similar type of reward-based Hebbian synaptic plasticity which depends on the pre-and postsynaptic mean firing rates was used to learn the inputs to a recurrently connected decision network (Fusi et al. 2007; Soltani et al. 2006a,b) , but in these cases the effective potentiation and depression rates were constant (corresponding to a fixed value for P 11 i j ).. Despite the promising simulation results, a formal proof that reward-modulated Hebbian learning follows the expected reward gradient is lacking. In contrast, NP is gradient ascending (Werfel et al. 2005) , and this property can also be proven for implementations of NP in spiking neurons and arbitrary network architectures (Fiete and Seung 2006) . Hence, although for the selected problems and architectures RAH surpasses NP in learning speed and performance, NP will always raise the expected reward for sufficiently small learning rates, unless the dynamics is already in a local reward maximum. While with increasing number of layers within the network the local gradient property seems to loose its relevance, it remains an open question why synaptic modifications in Hebbian or anti-Hebbian directions seem to gain importance.

B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O B O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O


Grouped variable importance with random forests and application to multiple functional data analysis In this paper, we study the selection of grouped variables using the random forests algorithm. We first propose a new importance measure adapted for groups of variables. Theoretical insights of this criterion are given for additive regression models. The second contribution of this paper is an original method for selecting functional variables based on the grouped variable importance measure. Using a wavelet basis, we propose to regroup all of the wavelet coefficients for a given functional variable and use a wrapper selection algorithm with these groups. Various other groupings which take advantage of the frequency and time localisation of the wavelet basis are proposed. An extensive simulation study is performed to illustrate the use of the grouped importance measure in this context. The method is applied to a real life problem coming from aviation safety. In high dimensional setting, the identification of the most relevant variables has been the subject of much research during the last two decades (Guyon and Elisseeff; . For linear regression, the Lasso method (Tibshirani; is widely used. Many variable selection procedures have also been proposed for non linear methods. In the context of random forests 2001) , it has been shown that the permutation importance measure introduced by Breiman, is an efficient tool for selecting variables (Díaz-Uriarte and Alvarez de Andrés; Genuer et al.; Gregorutti et al.; 2014) .. In many situations, as in medical studies and genetics, groups of variables can be clearly identified and it is of interest to select groups of variables rather than to select variables individually (He and Yu; . Indeed, the interpretation of the model may be improved as well as the prediction accuracy by grouping the variables according to an a priori knowledge on the data. In the end, grouping the variables can be seen as a solution to stabilize variable selection methods. In linear settings, and more particularly for the linear regression, the group Lasso has been developed to deal with groups of variables, see for instance (Yuan and Lin; . Group variable selection have also been proposed for kernel methods (Zhang et al.; and neural networks (Chakraborty and Pal; . As far as we know, this problem has not been studied for the random forests algorithm introduced by Breiman (2001) . In this paper, we adapt the permutation importance measure for groups of variables in order to select groups of variables in the context of random forests.. The first contribution of this paper is a theoretical analysis of the grouped variable importance measure. Generally speaking, the grouped variable importance does not reduce to the sum of the individual importances and it can be hardly related to these last. However, in more particular models such as additive regression models, we derive exact decompositions of the grouped variable importance measure.. The second contribution of this work is an original method for selecting functional variables based on the grouped variable importance measure. Functional Data Analysis (FDA) is a field in statistics that analyzes data indexed by time (Ramsay and Silverman; Ferraty and Vieu; Ferraty; 2011) . One standard approach in FDA consists in projecting the functional variables on a finite dimensional space spanned by a functional basis. Classical bases in this context are Splines, wavelets, Karhunen-Loève expansion for instance. Most of the papers about regression and classification methods for functional data consider only one functional predictor. Some references are Cardot et al. (1999 Cardot et al. ( , 2003 ; ; Cai and Hall (2006) for linear regression methods, Amato et al. (2006) ; Araki et al. (2009) for logistic regression methods, Biau et al. (2005) ; Fromont and Tuleau (2006) for k-NN algorithms and Villa (2006, 2008) for SVM classification. The multivariate FDA problem, where p functional variables are observed, has been less studied. Recently, Matsui and Konishi (2011) ; Fan and James (2013) have proposed answers to the linear regression problem with Lasso-like penalties. The logistic regression has been studied by Matsui (2014) . Classification based on multivariate functional variables has also been considered using CART algorithm and SVM (Yang et al.; Yoon and Shahabi; .. We propose a new approach of multivariate FDA using random forest and the grouped variable importance measure. Indeed, various groups of basis coefficients can be proposed for a given functional decomposition. For instance, one can choose to regroup all the coefficients of a given functional variable. In this case, the selection of a group of coefficients corresponds to the selection of a functional variable. Various other groupings are proposed for a wavelet decomposition. For a given family of groups, we adapt the recursive feature elimination algorithm 2002) which is particularly efficient when the predictors are strongly correlated (Gregorutti et al.; 2014) . In the context of random forests, this backward-like selection algorithm is guided by the grouped variable importance. Note that by regrouping the coefficients, the computational cost of the algorithm is drastically reduced compared to a backward strategy that would eliminate only one coefficient at each step.. An extensive simulation study illustrates the applications of the grouped importance measure for FDA. The method is finally applied to a real life problem coming from aviation safety. The aim of this study is to explain and predict landing distances. We select the most relevant flight parameters regarding the risk of long landings, which is a major issue for airlines. In order to speed up the algorithm, the dimension of the flight data is reduced in a preprocessing step. In Appendix B, we propose a modified version of the well-known shrinkage method Donoho and Johnstone (1994) that simultaneously shrinks to zero the coefficients of the observed curves of a functional variable.. The group permutation importance measure is introduced in Section 2. Section 3 deals with multivariate FDA using random forests and the grouped variable importance measure. The application to flight data analysis is developed in Section 4. Experiments 1a-b and 2a-b illustrate the results of Corollary 1 (see Figure 11 and 12). Indeed the regression function of both cases satisfies the additive property (2.3) for these experiments. In Experiments 1a and 2a, the variables of the group W are independent and the grouped variable importance is nothing more than the sum of the individual importances in this case. In Experiments 1b and 2b, the variables of the group W are positively correlated. In these situations, the grouped variable importance is larger than the sum of the individual importances, which agrees with Equation 2.4. Note that the grouped variable importance increases with p, which is natural because the amount of information for predicting Y increases with the group size in these models. On the other hand, it was shown in Gregorutti et al. (2014) that individual importances decrease with correlation between the predictors. Indeed we observe that the sum of the individual importances decreases with p in the correlated cases. The regression function of Experiments 3 does not satisfy the additive form (2.3). Although the variables in the group are independent, the grouped variable importance is not equal to the sum of the individual importances (Figure 13 ). In a general setting, it appears that these two quantities differ.. We now comment the results of the sparse Experiments 1c and 1d ( Figure 14) . It is clear that I(W) = I(X 1 ) = j=1...p I(X j ) for these two experiments (see Proposition 1 for instance). Regarding Experiment 1c, the boxplots of the estimated valuesÎ(W) and j=1...pÎ (X j ) agree with this equality. On the other hand, in Experiment 1d, we observe that j=1...pÎ (X j ) is significantly higher thanÎ(W). Indeed, it has been noticed by Nicodemus et al. (2010) that the individual importances of predictors that are not associated with the outcome tend to be overestimated by the random forests, when there is correlation between predictors. In contrast, the estimatorÎ(W) seems to correctly estimate I(W) even for large p. Indeed, for both experiments 1c and 1d, the importance Î (W) is unchanged when the size of the group p varies from 2 to 16. Note that for variable selection, we may prefer to consider the normalized importanceÎ nor to select in priority small group of variables.

O O O O B I O O O O B I I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Time-critical on-demand data broadcast: algorithms, analysis, and performance evaluation Abstract-On-demand broadcast is an effective wireless data dissemination technique to enhance system scalability and deal with dynamic user access patterns. With the rapid growth of time-critical information services in emerging applications, there is an increasing need for the system to support timely data dissemination. This paper investigates online scheduling algorithms for timecritical on-demand data broadcast. We propose a novel scheduling algorithm called SIN-that takes the urgency and number of outstanding requests into consideration. An efficient implementation of SIN-is presented. We also analyze the theoretical bound of request drop rate when the request arrival rate rises toward infinity. Trace-driven experiments show that SIN-significantly outperforms existing algorithms over a wide range of workloads and approaches the analytical bound at high request rates. T HE ever-growing popularity of the Internet and the resultant slow responses perceived by users have given rise to vast research efforts on improving the performance of Web accesses. As the system scale and user base continue to grow, there is an increasing demand for information providers to be capable of concurrently delivering a large amount of information to a huge number of users, especially in popular events such as elections and Olympics games. As a result, innovative delivery technologies, including satellite communications (e.g., StarBand [26] and DIRECWAY [27] ), cable networks, and wireless networks (e.g., 2.5G and 3G), have been developed and deployed to provide shared broadband Internet accesses.. Different from traditional networks, a distinguished feature of these new technologies is that they naturally support broadcast. In contrast to unicast, where a data item of interest to multiple clients must be sent individually to each client, broadcast satisfies all outstanding requests for the same item by a single transmission. This leads to a more efficient use of shared bandwidth, hence improving the system throughput and user-perceived response time [19] , [30] . In general, there are two data broadcast approaches [8] , [20] :. 1 Push-based broadcast computes the broadcast program based on historical access statistics; on-demand broadcast schedules broadcast items on the fly based on current outstanding requests. While push-based broadcast is useful for certain applications (e.g., a small set of data items with stable access patterns), on-demand broadcast is more widely used for dynamic, large-scale data dissemination like that in the Internet.. With the rapid growth of time-critical information services and business-oriented applications, there is an increasing demand to support quality of service (QoS) in content distribution [15] , [24] , [28] . In many situations, user requests are associated with time constraints as a measure of QoS. These constraints can be imposed either by the users or the applications. For example, in wireless financial services, many users are interested in the up-to-minute (or even "second") stock quotes in order to react to dynamic and rapid market developments. As another example, in wireless location-based services [18] , the queried information (e.g., the local theaters) is valid only within a local area. When the mobile user moves away from the area, the information becomes invalid. In addition, a service level agreement (SLA) between a content/service provider and its users usually specifies the desired performance for Web requests, e.g., the response time of requests for CNN.com should not exceed 5 seconds [24] . In all the above cases, a deadline is associated with each request beyond which the serving of the request is useless (or less useful).. This paper focuses on on-demand data broadcast with time constraints, which we shall refer to as time-critical ondemand broadcast. A key issue in the design of an ondemand data broadcast system is the scheduling algorithm used to select and broadcast requested items from outstanding requests. While there has been significant work on developing on-demand broadcast scheduling algorithms (e.g., [1] , [2] , [10] , [29] ), none of them has considered the time constraints associated with requests. On the other hand, although some time-critical scheduling algorithms have been proposed for unicast-based real-time systems and push-based broadcast systems (e.g., [11] , [14] ), they are not applicable or not effective to on-demand broadcast systems (see Section 2 for details). The objective of this paper is to develop new scheduling algorithms for timecritical on-demand broadcast.. The main contributions of this paper are three-fold:. . This is the first effort, to the best of our knowledge, to take into account the time constraints in the design of on-demand broadcast scheduling algorithms. We propose a low-complexity scheduling algorithm, called SIN-, for time-critical on-demand broadcast. . We analyze the theoretical bound of request drop rate when the request arrival rate rises toward infinity. The analytical results are used as a yardstick in the experimental evaluation. They can also be employed to facilitate bandwidth provisioning in on-demand data broadcast systems. . We conduct trace-driven simulation experiments to study the performance of the proposed scheduling algorithm over a wide range of workloads and show that SIN-significantly outperforms existing algorithms and approaches the analytical bound at high request rates. The rest of this paper is organized as follows: Section 2 summarizes the related work. The system model is described in Section 3. Section 4 presents the proposed scheduling algorithm, SIN-, and discusses its implementation. Section 5 analyzes the theoretical bound of request drop rate when the request rate rises toward infinity. Section 6 experimentally compares the performance of the proposed algorithm against existing algorithms and the analytical bound. Finally, Section 7 concludes the paper with a brief discussion on future work. On-demand data broadcast has attracted a lot of attention due to its scalability to user population and adaptiveness to dynamic user access patterns. The scheduling problem for time-critical on-demand broadcast is becoming increasingly important with the rapid growth of time-critical information services in emerging applications. A new scheduling algorithm called SIN-that integrates the urgency and productivity of broadcast items has been proposed in this paper. An efficient implementation of SIN-has been presented to reduce the scheduling complexity. Moreover, an analytical model has been developed to investigate the theoretical bound of request drop rate when the request arrival rate rises towards infinity. The analytical bound has been used as a yardstick in our experimental evaluation. It can also be employed to facilitate bandwidth provisioning in on-demand data broadcast systems.. Trace-driven simulation experiments have shown that the proposed SIN-algorithm considerably outperforms existing algorithms over a wide range of workloads and approaches the analytical bound when the request arrival rate is high. In conclusion, SIN-is an effective yet simple scheduling algorithm and can be used in practical timecritical on-demand broadcast systems such as satellitebased content distribution networks and wireless Internet.. The work reported in this paper is a first step to timecritical on-demand broadcast scheduling. There are many research issues that deserve further work. This paper assumed the data to be broadcast is available on the broadcasting server; in practice, the data to be broadcast could reside on some remote servers, for which we plan to develop push-based caching techniques to speed up data retrieval and facilitate broadcast scheduling. We also plan to combine scheduling with indexing to improve energy efficiency for mobile clients. Another direction for future research is to investigate soft deadlines where the revenue of broadcasting an item is represented by a function of latency. He is a coeditor of the book Web Content Delivery (Springer). His research interests include Web and Internet (particularly caching, replication and content delivery), mobile and pervasive computing (especially data management and delivery), wireless sensor networks, and distributed systems. He is a member of the IEEE.. Wang-Chien Lee received the BS degree from the Information Science Department, National Chiao Tung University, Taiwan, the MS degree from the Computer Science Department, Indiana University, and the PhD degree from the Computer and Information Science Department, Ohio State University. He is currently an associate professor of computer science and engineering at Pennsylvania State University. Prior to joining the faculty of Penn State, he spent five plus years as a principal member of the technical staff at Verizon/GTE Laboratories, Inc. Dr. Lee's primary research interests lie in the areas of pervasive and mobile computing, data management, Internet technologies, wireless networks, and security. He has worked on building infrastructures to facilitate query processing in various wireless networks and pervasive computing systems (such as wireless broadcast systems, peer-to-peer systems, and sensor networks). He has served as a guest editor for several journal special issues on mobile database-related topics, including IEEE Transactions on Computers, IEEE Personal Communications Magazine, ACM MONET, and ACM WINET. He was a program committee cochair for the First International Conference on Mobile Data Access (MDA '99) and the International Workshop on Pervasive Computing (PC 2000) . He is a member of the IEEE, the IEEE Computer Society, and the ACM. There has been much work on developing on-demand broadcast scheduling algorithms. Acharya and Muthukrishnan [1] introduced a new performance metric called stretch for variable-size data items and investigated several scheduling algorithms. Aksoy and Franklin [2] proposed a low-overhead and scalable scheduling algorithm called RÂW. Datta et al. [9] took into consideration the energy saving issue in the design of on-demand broadcast systems. Liberatore [22] studied the scheduling algorithms for requests asking for a list of dependent items. Edmonds and Pruhs [10] proposed two constant approximation algorithms for scheduling variable-size data items. Hu and Chen [13] investigated dynamic traffic-aware scheduling algorithms. However, none of these algorithms has considered the time constraints of requests in making scheduling decisions.. Most existing studies on broadcast scheduling with time constraints investigated only periodic push-based broadcast [14] , which is fundamentally different from on-demand broadcast in system architecture. Xuan et al. [31] evaluated several alternative system designs for time-constrained data accesses and showed that on-demand broadcast with the earliest deadline first (EDF) scheduling algorithm performs well. They concentrated on the system design aspect of ondemand broadcast, which is complementary to the focus of this paper on the algorithmic aspect of time-critical ondemand broadcast.. A closely related area is task scheduling in real-time systems [7] , [17] , [23] . One of the most classical scheduling algorithms is EDF [23] . It offers optimal performance in lightloaded systems in terms of deadline missing rate. However, when task service times are not available, EDF performs poorly because it may favor the tasks whose remaining lifetimes are shorter than their service times. As a result, these tasks would not only miss their deadlines but also waste system resources to prevent more tasks from meeting their deadlines. Various techniques have been proposed to handle this "domino effect" [11] . Buttazzo et al. [7] studied valuedeadline task scheduling in real-time systems. In this approach, each task is characterized by an importance value and the scheduling algorithm aims to maximize the cumulative value of the tasks that meet their deadlines. Unfortunately, these existing scheduling algorithms are designed for a unicast environment where a newly arrived task cannot be combined with any existing tasks. In contrast, this paper focuses on a broadcast environment where a new request can be merged with existing outstanding requests if the same item is requested. These requests would be satisfied by a single broadcast of the item. This fundamental difference in system model gives rise to the development of new scheduling algorithms.. Other related work includes time-constrained transaction processing over push-based broadcast data [21] , [25] , where multiple versions of data items are broadcast to reduce the transaction abort rate and user-perceived response time. Schedules that minimize response time for push-based broadcast have been extensively studied in the algorithmic community (e.g., [5] and [16] ). These studies complement our work in different aspects.. In summary, existing time-critical scheduling algorithms are confined to push-based broadcast and unicast systems only. Meanwhile, existing on-demand broadcast scheduling algorithms ignore the time constraints associated with requests. To the best of our knowledge, there has been no study on the scheduling algorithms for time-critical ondemand broadcast. A trace-driven simulator has been developed to evaluate the performance of proposed SIN-algorithm. The simulator models the architecture shown in Fig. 1 . Real traces collected from the World Cup '98 Website [32] were used to simulate the requests made by the clients. Similar performance trends were observed for different daily traces. Due to space limitations, we shall report only the experimental results of one trace (i.e., the day-38 trace) in this paper. The day-38 trace contains more than 7 million requests for 4,923 distinct Web objects (e.g., HTML pages and images). 3 The average request rate is 83 per second. The access counts of different objects sorted in descending order are shown in Fig. 4 . It can be seen that the access pattern follows a Zipf-like distribution, which is consistent with the observation made in the literature [6] . To simulate different levels of workloads, we changed the time scale of the trace by introducing a request rate scaling factor f. The interarrival time between two consecutive requests in our experiments was set to the actual time logged in the trace divided by f. It is obvious that the higher the value of f, the heavier the workload. The data transmission rate of the broadcast channel is described in the number of objects that can be transmitted per second. 3. Interested readers are referred to [4] for more details of the WorldCup98 Web server traces.. The default data transmission rate was set at 10 objects/ sec (i.e., 0.1 second taken to broadcast an object). To model the time constraints of requests, each request was assigned a relative deadline randomly generated based on a specified distribution (i.e., exponential, uniform, 4 or fixed distribution) with the designated mean value. The workload parameters used in our experiments are summarized in Table 2 . Each simulation run started with an empty service queue. The first 1,000,000 requests were considered the start-up period and performance statistics were collected for the subsequent 2,000,000 requests.. In the following, we first investigate the setting of parameter in the SIN-algorithm and examine its performance against the analytical bound at high request rates. Next, we compare SIN-with the state-of-the-art scheduling algorithms under various workloads. Finally, we evaluate the scheduling complexity of SIN-.

O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O


Multi-agent human-machine dialogue: issues in dialogue management and referring expression semantics This paper presents an implemented model of dialogue management for simple dialogues involving multiple speakers. In our model, the user is one speaker, and the system 'plays' a number of other speakers. We present a number of principles governing dialogue management in such cases, which relate to turn-taking and the identification of the addressees of utterances. We also consider how to extend a syntactic and semantic treatment of first- and second-person personal pronouns, and of addressee terms, in order to deal with the multi-speaker scenario. We give some examples of our current system, and conclude by outlining some extensions of the system to include disagreements, interruptions, and private communication between subgroups of dialogue participants. Human-machine dialogue systems typically support dialogue between two agents: the human user is one agent, and the system plays the part of the other. In this scenario, the user and the system take turns at being the speaker, and when one of them is the speaker, the other is the addressee (the agent being spoken to).. However, in real life dialogue, there are frequently more than two participants. Automated dialogue systems can be configured in various ways to operate in a multi-speaker scenario. Firstly, a system can simulate each dialogue participant as a separate autonomous agent (e.g. Padilha and Carletta [1] ).Secondly, a system can play the part of a single agent in a context where there are several human speakers (Wang, [2] ). Finally, the system could support a dialogue between a single human user and several agents, all of which are played by the system. Here the agents can either be genuinely autonomous, or they can act in the service of a shared plan, delivering lines given to them by a central controller.. To extend a dialogue system to deal with multi-speaker interactions, whichever of the above scenarios is envisaged, a number of things must be supplied. At the dialogue level, we need a theory of turn-taking, to decide when to make an utterance, and who the addressees of other speakers' utterances are. At the level of sentence syntax and semantics, we need to pay special attention to constructions which are used to refer to dialogue participants (especially personal pronouns) and which are used to control turn-taking (especially terms of address).. We have already built a two-speaker dialogue system, which incorporates full sentence parsing and generation using a declarative grammar, and a range of standard dialogue management techniques (de Jager et al [3] ; Bayard et al, [4] ). This paper describes how we are extending this system to a multi-speaker environment, focussing on the additional syntactic constructions and dialogue management principles which are required, and on the interactions between these.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O