Inductive transfer with context-sensitive neural networks Abstract Context-sensitive Multiple Task Learning, or csMTL, is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks. Motivated by problems with the application of MTL networks to machine lifelong learning systems, csMTL encoding of multiple task examples was developed and found to improve predictive performance. As evidence, the csMTL method is tested on seven task domains and shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks. We argue that the reason for this performance improvement is a reduction in the number of effective free parameters in the csMTL network brought about by the shared output node and weight update constraints due to the context inputs. An examination of IDT and SVM models developed from csMTL encoded data provides initial evidence that this improvement is not shared across all machine learning models. Multiple task learning (MTL) is well recognized as a method of inductive transfer that is able to develop superior performing hypotheses from impoverished training sets. Research on MTL has occurred using traditional machine learning methods (Bakker and Heskes 2003; Caruana 1997; Baxter 1997; Heskes 2000; Thrun and Pratt 1997; Ben-David and Schuller 2003) , statistical regression methods (Greene 2002; Zellner 1962; Breiman and Friedman 1998) , Bayesian methods involving constraints such as hyper priors (Allenby and Rossi 1999; Arora et al. 1998; Bakker and Heskes 2003) , and most recently kernel methods such as support vector machines (SVMs; Jebara 2004; Allenby and Rossi 2005) . All of these Editor: Risto Miikkulainen D.L. Silver ( ) · R. Poirier · D. Currie Jodrey School of Computer Science, Acadia University, Wolfville, NS, Canada B4P 2R6 e-mail: danny.silver@acadiau.ca approaches rely upon the development of multiple hypotheses under a constraint or regularization that characterizes a similarity or relatedness between the tasks.. Multiple task learning (MTL) neural networks are one of the better documented methods of inductive transfer of task knowledge (Caruana 1997; Silver and Mercer 1996) . An MTL network has an output for each task and develops internal representations that are shared by all tasks. Inductive transfer occurs as a function of sharing these representations. We have investigated the use of MTL networks for the purpose of developing a machine lifelong learning system capable of retaining learned knowledge for use in future learning. A key finding has been problems introduced by multiple outputs of MTL systems, such as the build-up of redundant outputs from different sets of training examples for the same task (O'Quinn et al. 2005; Silver and Poirier 2005) . Perhaps, more importantly, MTL transfer is dependent upon an elusive measure of task relatedness for selecting the more related tasks for optimal inductive transfer (Caruana 1997; Thrun 1996) . Lacking a solution to these problems, we have investigated methods of MTL that do not require multiple outputs nor depend upon a method of measuring task relatedness.. This paper presents context-sensitive multiple task learning, or csMTL, as a method of inductive transfer that uses a standard back-propagation single-output neural network and additional contextual inputs for learning multiple tasks. The goal of this paper is not to introduce a new learning algorithm, but rather to study the effects of the encoding method on existing algorithms, particularly, artificial neural networks. Our contribution is to demonstrate the strengths and weaknesses of this approach in contrast to existing methods.. Toward this end, we show that the number of free parameters of csMTL networks are lower than their MTL counterparts and that the shared output weights force an additional constraint over a csMTL network's hypothesis space. Experiments on two synthetic and five real-world domains demonstrate that csMTL networks develop models that often perform better than MTL networks. Most notably, we show empirically that csMTL hypotheses for a primary task outperform MTL hypotheses when the examples from the secondary tasks are drawn from the same function as the primary task.. Potentially, the csMTL approach allows multiple tasks to be learned by any single task learning (STL) method. In the later portion of the paper we investigate this possibility. Specifically, we show that Inductive Decision Trees and Support Vector Machines (with linear and radial basis kernels) do not benefit from the introduction of context attributes when learning multiple tasks. The resulting models are equivalent to STL models developed only from the primary task examples. This suggests that neural networks have an ability to capture regularities in the examples across the various tasks and associate these regularities with the context inputs in a way that, at least, these two other methods cannot. Our long-term goal is to develop an ML3 methodology and related system, based on csMTL, which is capable of sequential knowledge retention and inductive transfer. The system is meant to satisfy a number of ML3 requirements including the effective and efficient consolidation of task knowledge into a long-term network using task rehearsal (Silver and Mercer 2002) , and effective and efficient inductive transfer during new learning. With this is longterm goal in mind, our short-term directions for future research include:. • Performing a detailed analysis of the contribution of shared output node representation on the reduction of number of necessary training examples. We believe that this does not reduce the number of free parameters, per se, but rather reduces the effective size of hypothesis space. Both mathematical analyses and experimentation will be necessary in order to confirm or deny this belief.. • Examining the conditions under which csMTL networks will produce less accurate hypotheses than MTL networks. The reduction in the number of free parameters, and the effective modification to the functional form of the model leads us to believe this may occur. In order to best apply csMTL, it is important to have an understanding of the conditions under which it is best used.. • Investigating more optimal methods of ensuring that small numbers of examples for the primary task under csMTL are not overwhelmed by large numbers of examples for each of the secondary tasks. In this research we simply duplicate the primary task examples to equal the number for each secondary task. More optimal methods are likely possible.. • Experimenting with other machine learning methods to determine which methods, and more importantly, which attributes of methods lead to better results given multi-task data structured as per csMTL.. • Exploring if Abu-Mostafa's theory of hints can be applied directly, or with some extensions, to explain the effects of csMTL learning, and even of multi-task learning models in general, when the multiple tasks are not the same task.. • Investigating the use of real-valued context inputs from the environment as a grounded source of task relatedness.. • Investigating domains of tasks that have multiple outputs per task, such as image transformation tasks. The IDT method demonstrated no significant difference in performance between a single task model, and a model given additional examples for related tasks using contextual attributes. This shows that the IDT method does not experience inductive transfer through the use of contextual attributes with data from related tasks. An analysis of several of the trees that were developed indicate that the reason for this is that IDTs tend to create decision boundaries using the context input attributes that lead to separate representations for each of the tasks.. Similarly, the SVM method, using linear or radial basis kernels, neither improved nor degraded from the use of contextual attributes with additional data from other tasks. The SVM method with the given kernels does not appear to directly experience inductive transfer through the use of contextual attributes in the same way as the neural networks. This may be because with SVM, the radial basis kernel,. produces higher values for the support vectors which belong to the same task as the example being tested. Due to the difference in the contextual components of the input vectors, x − x 2 will be greater by 2 for support vectors from different tasks than for support vectors from the same task. This effect will result in lower values for K(x, x ) for support vectors from different tasks, which in at least some situations may limit the degree of inductive transfer from other tasks.. Based on these results, we conjecture that the performance improvement in csMTL over MTL networks is not due to the transformation of the data from a multiple output format to a single output format using contextual inputs, but rather is due to the manner in which a neural network develops a model given this transformation of the data. As shown in Sect. 4, the back-propagation algorithm places functional constraints on the parameters of the model, which reduces the number of examples available to find an accurate hypothesis. The MTL and csMTL models perform statistically better than the STL models on all four domains of tasks. The csMTL models perform statistically better than MTL models on the Logic, Dermatology and Glass domains but at the same level of accuracy on the CoverType domain. MTL, which takes advantage of shared representation and knowledge transfer from the secondary tasks, produces better models than STL. csMTL, constrained by the common output node and biased by the context inputs is able to build the most accurate hypotheses for three of the domains.. . . . Figure 8 shows no significant difference between the csMTL and STL methods, regardless of the number of primary task examples. Both methods benefit from an increase in the number of training examples, but IDTs do not develop better models from related task examples when they are encoded using additional context inputs. 6.2 csMTL and SVM models Support Vector Machines, or SVMs, bear some resemblance to neural networks in that they develop a multi-layered functional model. The lower layer projects the inputs into a feature space, and computes a value based upon a weighted combination of these features in order to perform classification (Boser et al. 1992; Smola and Schoelkopf 1998) .. However, there are also significant differences between the two models. The training in SVMs is an optimization problem which does not depend upon the initial values of free parameters. Also, the combination of kernel mapping functions as well as the selection of a varying set of support points and parameters allows for very complex decision boundaries.. This combination of similarities and differences between SVMs and neural networks make SVMs an appealing machine learning method to study with multiple tasks examples encoded using context attributes.. The mean accuracy for the best STL SVM model was 71.4% with a 3.3% standard deviation, occurring for a SVM model trained with a radial basis kernel using a gamma of 0.8 and a constraint violation cost of 1. The mean accuracy for the best csMTL SVM model built using primary and secondary task data with contextual inputs was 72.4% with a 3.9% standard deviation, occurring for a SVM model trained with a radial basis kernel using a gamma of 0.1 and a constraint violation cost of 1.. The results indicate that there is no significant difference between STL and csMTL SVM models that use linear or radial basis kernels. In future work we intend to explore other kernels that may support csMTL encoded examples. The STL and MTL networks were configured with the appropriate number of input and output nodes, respectively (see Table 1 ). The csMTL networks had additional context inputs and only 1 output. The number of hidden nodes for all methods was varied from 5 to 200. The learning rate for each method was optimized between 0.01 and 0.001 through preliminary testing. The momentum term was fixed at 0.9 for all runs.. The objective is to learn the primary task of each domain (e.g. T 0 for the Logic domain) using an impoverished training set of examples (20 for the Logic domain) as shown in Table 1. The secondary tasks of each domain have sufficient training examples to develop models with accuracies greater than .75 using a STL network. Tuning sets with small numbers of examples for the primary task (6 for the Logic domain) are used to prevent the neural network from over-fitting.. It is important to note, that in the case of csMTL, the training examples for the primary task are duplicated to ensure an equal number of training examples as that of each secondary task. In the case of the Logic domain, the 20 training examples for the primary task were duplicated nine times to make a total of 200 training examples, the same number as each of the secondary tasks. This is necessary to ensure a fair update to the weights for all tasks being learned by the csMTL network which has a single output node. The duplication of primary examples does not help with MTL which has a separate output for each task. With duplicated examples, MTL develops hypotheses that quickly over-fit to the training data and have less generalization accuracy. An independent test set of examples is used to determine hypothesis performance (see Table 1 ). A network output of 0.5 or greater is considered a positive classification. The mean accuracies reported are from 5 repeated trials. For each repeated trial, the examples in the data sets were randomly sampled from the available data. Figure 3 shows the mean accuracy of models for the four tasks developed under the three methods as the number of hidden nodes increases. The p-values from a difference of means T-test over the models for each task is shown in Table 2 . All graphs show that the performance of the MTL and csMTL models become statistically stable at 20 hidden nodes or more. The STL models plateau at or before 100 hidden nodes. Therefore, all methods are robust given sufficient representation and early stopping using a validation set to prevent over-fitting.. STL, MTL, and csMTL networks were configured for the Logic domain as described in Sect. 5.2, but with a fix number of 20 hidden nodes, sufficient for learning under all methods.. All three networks are presented with training sets of size 10 to 200 for the primary task, increasing the number of examples by 10. For MTL and csMTL, the five secondary tasks had training sets of 200 examples, which is sufficient to build accurate hypotheses for this domain. For csMTL, the training examples for the primary task are duplicated as many times as necessary to ensure 200 training examples for each run-the same number as that of each secondary task. As discussed in Sect. 5.2, this is to ensure a fair update of network weights for each task being learned by the csMTL network. Thus, for csMTL, variation in the "number of training examples" means a variation in the number of unique examples in the training set.. A tuning set of 20 examples for the primary task is used during training, and an independent test set of size 1000 is used to measure accuracy. Mean test set accuracies from three repeated studies each with random draws of examples are compared. Figure 4 shows the mean accuracy of models developed under the three methods as the number of primary task examples increases. The results show that csMTL hypotheses are consistently better than STL models, and beyond 30 training examples they are consistently better than MTL models. This further suggests that the csMTL method is able to selectively transfer knowledge from the more related secondary tasks, without an explicit measure of relatedness.. The primary tasks for the synthetic Logic domain and the real-world Glass, Dermatology and CoverType domains were used to generate four new domains called Logic, Glass, Derm and Cover, each consisting of six tasks. The training examples for each task of a domain were drawn from the original primary task for that domain; 20 examples per task for the Logic domain, 10 for Glass, 20 for Derm and 30 for Cover. For each domain, the six sets of training data were also combined into a large training set call STL-AllData. The STLAllData training set allowed us to develop the best possible primary task model for each of the new domains using STL.. A csMTL network was configured for each of the domains as described in Sect. 5.3 with one output node, a layer of hidden nodes sufficient for learning all tasks (20 for the Dermatology, Glass domains and 30 for the Covertype domain) and a layer of input nodes equal to the number of primary inputs plus the number of tasks in the domain. For the Logic domain the number of hidden nodes for all methods was varied from 2 to 200. The STL networks used were identical to the csMTL networks less the context inputs. The MTL networks were the same as the STL networks but with one output for each task. Independent test sets for the primary task of each domain (see Table 1 ) were used to measure model accuracies. Mean accuracies from three repeated studies were computed to compare model performance.. As an additional study, the examples for one of the equivalent secondary tasks was replaced by the same number of examples of another task from each domain. In the case of the synthetic Logic domain, we chose replacement tasks that were known to be least related to the primary task. For the real-world domains, secondary tasks and examples were randomly chosen. The intention was to observe the effect on the MTL and csMTL methods as the diversity of tasks increased. The replacement of an equivalent task was repeated three times until there were only two secondary tasks left with examples drawn from the same function as the primary task. As before, independent test sets for the primary task of each domain were used to compute mean accuracies so as to measure method performance. Figure 6 shows how the performance of the MTL and csMTL methods vary as the number of less related secondary tasks increases in each domain. Clearly, the MTL method is more affected by the variety of related tasks as compared to the csMTL method. The results suggest that selective inductive bias is occurring in the csMTL network more effectively than in the MTL network. The results on the Cover domain indicate that the last two secondary tasks actually provided beneficial inductive bias to the primary task.. A csMTL network was configured for each domain with one output node, a layer of hidden nodes sufficient for learning all tasks (30 for the Band, 20 for the Logic, 10 for the fMRI, 30 for the Covertype and 20 for the Dermatology, Glass and HD domains) and a layer of input nodes equal to the number of primary inputs plus the number of tasks in the domain. The STL networks used were identical to the csMTL networks less the context inputs. The MTL networks were the same as the STL networks but with one output for each task.. For all five domains, the objective is to learn the primary task using an impoverished training set of examples as shown in Table 1 . Each of the other tasks of the domain have 35 Fig. 7 csMTL compared to STL and previous MTL methods. Shown is the mean test set accuracy for primary task hypotheses for each of seven domains of tasks or more training examples that have been demonstrated to develop models with accuracies greater than .75 using a STL network. A small tuning set of examples for the primary task of each domain is used to prevent the neural network from over-fitting to the impoverished training sets.. As described in Sect. 5.2, in the case of csMTL, the training examples for the primary task are duplicated as many times as necessary to ensure an equal number of training examples to that of each secondary task. This ensures a fair update to the weights for all tasks being learned by the csMTL network. As discussed in Sect. 5.2, this duplication of primary examples does not help MTL in the same manner.. An independent test set is used to determine hypothesis performance. See Table 1 for details on the number of examples in each of these data sets. A network output of 0.5 or greater is considered a positive classification. The mean accuracies reported are from repeated trials (10 for the Band, fMRI, Heart Disease and Glass domains, 30 for the Logic, Covertype, and Dermatology domains). For each repeated trial, the examples in the data sets were randomly sampled from the available data ensuring a balance of positive and negative examples. Figure 7 and Table 3 show the results for the seven domains. They compare the mean test set accuracy of the csMTL hypotheses developed for the primary tasks to hypotheses developed with no inductive transfer under STL and with transfer under standard MTL. The MTL and csMTL results demonstrate the advantage of knowledge transfer with mean accuracies that are significantly better than STL models for all domains. Furthermore, csMTL performs significantly better than MTL on all domains except Dermatology and Heart Disease. The results suggest that the csMTL method is at least as good as standard MTL. It is able to transfer knowledge from shared internal representation to a primary task when training on examples of a mixture of tasks that are more or less related. . The objective is to compare csMTL with STL IDTs using the Logic domain. The same primary task and data used in Sect. 5.3 was used for this experiment. Similarly, we examined the effect of varying the number of primary task examples from 10 to 100. There were 200 examples for each of the secondary tasks in the csMTL scenario. A set of 1000 primary task examples is used to test prediction accuracy of the tree for task T 0 . The trees were built using the J48 algorithm of open source software WEKA 3 (Witten and Frank 2005) . Preliminary experiments were conducted varying the branching confidence level from 0.10 to 0.40 and the minimum number of instances per leaf from 2 to 30. A confidence value of 0.25 and minimum number of instances per leaf of 15 were found to be the best choices for the two methods.. This experiment compares the mean accuracy of STL and csMTL SVM models for the T 0 task of the Logic domain. The experiment was developed in the R statistical programming language using the SVM implementation in the e1071 library, which is based upon libSVM (Chang and Lin 2001) .. Each simulation consisted of selecting 20 examples at random from a pool of 1020 examples, under the constraint that 10 selected examples were of each class. These 20 examples were used as the training set for T 0 , and the remaining 1000 examples were retained as the independent test set. For each simulation, two SVMs were trained. The first was a single task SVM using only the 20 training examples. The second was an SVM which used 200 examples from each of 5 other related tasks marked using additional contextual attributes, and the 20 training examples were duplicated another 9 times to provide an equal degree of weighting with the other tasks. The prediction accuracy of each SVM was determined using the 1000 examples remaining in the test set for T 0 .. In order to select appropriate values for the kernel type and parameters for the STL and csMTL SVMs, 50 repetitions were performed for:. • linear kernels with constraint violation costs of 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, and 1 • radial basis kernels with combinations of gamma equal 0. 1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 , and 1, with constraint violation costs of 0.001, 0.005, 0.01, 0.05 , 0.1, 0.5, and 1. The best mean and standard deviations of prediction accuracy are reported for each method.

B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Towards Characterizing the Non-Locality of Entangled Quantum States The behavior of entangled quantum systems can generally not be explained as being determined by shared classical randomness. In the first part of this paper, we propose a simple game for n players demonstrating this non-local property of quantum mechanics: While, on the one hand, it is immediately clear that classical players will lose the game with substantial probability, it can, on the other hand, always be won by players sharing an entangled quantum state. The simplicity of the classical analysis of our game contrasts the often quite involved analysis of previously proposed examples of this type.In the second part, aiming at a quantitative characterization of the non-locality of npartite quantum states, we consider a general class of n-player games, where the amount of communication between certain (randomly chosen) groups of players is measured. Comparing the classical communication needed for both classical players and quantum players (initially sharing a given quantum state) to win such a game, a new type of separation results is obtained. In particular, we show that in order to simulate two separated qubits of an npartite GHZ state at least Ω(log log n) bits of information are required. The classical outcomes of measurements performed on an entangled quantum state can generally not be explained by local classical randomness. This non-local property of quantum mechanics is demonstrated by the pseudo-telepathy game proposed in Section 2: A simple task, which obviously cannot be accomplished by separated classical players, is solvable by players sharing quantum entanglement. The non-locality of an n-partite quantum state is often characterized by the amount of communication needed by n separated classical systems for simulating the outcomes of local measurements performed on the respective parts of the state. There are clearly several ways to quantify this communication, each revealing a different aspect of the non-local information contained in the state. Contrary to the approach of Cleve and Buhrman [8] , we consider the information exchanged between certain groups of systems instead of counting the overall communication. This leads to an alternative quantification of non-local information and, consequently, to new separation results. They cannot directly be compared with the results based on Yao's model (as for instance [5] ), but rather unveil another facet of the nature of entanglement as well as the gap between quantum and classical correlation.. The results obtained in Section 3 are formulated in terms of communication complexity with respect to certain games. The difference between the amount of classical communication needed for classical and quantum players, respectively, to win such games directly lead to lower bounds for the communication needed to simulate quantum states. For instance, the separation stated by Lemma 2 implies that for the classical simulation of two separated two-dimensional quantum systems sharing a GHZ state with n − 2 other systems, at least Ω(log log n) bits of additional information are necessary.. It is one of the goals of this paper to shed some light on the nature of quantum entanglement, a phenomenon which is not yet completely understood. While separation results, as the ones presented here, can be seen as lower bounds for the amount of non-local information contained in entangled quantum states, some work has been done to determine the maximal communication being necessary for an exact simulation of such states by classical systems (see, e.g., [3] ). It is, however, still an open problem to find the most accurate way to characterize entanglement between quantum systems in terms of classical communication.. where e i is the bitstring of length n which has a bit 1 at the ith position and zeros at all other positions. Furthermore, for all A ∈ A, let r A be an arbitrary element of A. Defineb (i) (for i = 1, . . . , n) as the bitstring which is identical tob (i) except that the bits at positions r A (for all A ∈ A) are omitted. Since the stringsb (i) have length n + 1 + l, the stringsb (i) obviously have length l ′ = n + 1 + l − |A| ≤ n − √ n + l + 2.. If the stringsb (i) (for i = 1, . . . , n) form a set of n linearly independent vectors, then l ′ ≥ n, i.e., n − √ n + l + 2 ≥ n which again implies (9) . It thus remains to be shown that the bitstringsb (1) , . . . ,b (n) are indeed linearly independent. Assume by contradiction that there is a nonempty set I ⊆ {1, . . . , n} such that. To show that this leads to a contradiction, we will distinguish three cases.. (a) |I| ≡ 0 (mod 4): From condition (10) the sum i∈Ib (i) is nonzero. Since the last n + 1 bits ofb (i) correspond tob (i) , this obviously contradicts equation (13) .. (b) |I| ≡ 0 (mod 4) and I ∩Ā = ∅: Let r be an element of the intersection I ∩Ā. By definition, there is exactly one bitstringb (i) with i ∈ I having a bit 1 at the rth position, namelyb (r) . Note that the bit ofb (i) at position r corresponds to a bit ofb (i) at some position r ′ (in the construction ofb (i) fromb (i) only bits with an index in sets A with A ∈ A are omitted). Consequently, the r ′ th bit of the sum in (13) is 1.. (c) |I| ≡ 0 (mod 4) and I ∩Ā = ∅: Since I is nonempty, there exists a set A ∈ A such that I ∩ A = ∅. Assume that |I ∩ A| > 1, i.e. there are at least two different indices r 1 and r 2 in I ∩ A. Consequently, the sum i∈Ib (i) has a bit 1 at position r 1 and r 2 . By the construction of the stringsb (i) , at least one of these bits corresponds to a bit in the sum in (13) which can thus not be zero.. It remains to be shown that |I ∩ A| > 1. Using the fact thatb (i) +b (i) = 0 (over GF(2)) we have i∈A/I∪I/Ab. where the last equality follows from A ∈ A and condition (11) as well as from assumption (13) . With condition (10) this implies that |A/I ∪ I/A| ≡ 0 (mod 4) .. On the other hand, using (12) and |I| ≡ 0 (mod 4), |A/I ∪ I/A| = |A| + |I| − 2|A ∩ I| ≡ −2|A ∩ I| (mod 4) .. Together with (14) we conclude that |A ∩ I| must be even and thus, since the set A ∩ I is nonempty, we have |A ∩ I| > 1.

O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Temporal index sharding for space-time efficiency in archive search Time-travel queries that couple temporal constraints with keyword queries are useful in searching large-scale archives of time-evolving content such as the web archives or wikis. Typical approaches for efficient evaluation of these queries involve slicing either the entire collection [20] or individual index lists [10] along the time-axis. Both these methods are not satisfactory since they sacrifice compactness of index for processing efficiency making them either too big or, otherwise, too slow.We present a novel index organization scheme that shards each index list with almost zero increase in index size but still minimizes the cost of reading index entries during query processing. Based on the optimal sharding thus obtained, we develop a practically efficient sharding that takes into account the different costs of random and sequential accesses. Our algorithm merges shards from the optimal solution to allow for a few extra sequential accesses while gaining significantly by reducing the number of random accesses. We empirically establish the effectiveness of our sharding scheme with experiments over the revision history of the English Wikipedia between 2001-2005 (≈ 700 GB) and an archive of U.K. governmental web sites (≈ 400 GB). Our results demonstrate the feasibility of faster time-travel query processing with no space overhead. Due to the ubiquitous access and ease of location using search engines, much of the human-generated content is available electronically over the Web. This content is constantly evolving with additions, deletions and modifications occurring at very high rates [5, 12, 13, 19] . Search engines continuously crawl and index the Web to keep up with its latest state.. There is a growing awareness, however, that this singleminded pursuit for the latest state of the Web results in losing access to the history of content which could be important in a number of advanced applications. As an answer to this, efforts such as the Internet Archive [3] and European Archive [2] store regular crawls of large portions of the Web for retrospective analysis. Effective querying over these collections requires the use of so-called time-travel queries that combine temporal constraints with standard keyword queries. These queries aim to return relevance ordered lists of documents (or, more precisely, document versions) that were "alive" during the specified time interval [10] . For example, on an archive of U.K. government websites, one may pose a query such as inheritance tax @ [2000] [2001] [2002] to locate documents about inheritance taxation details prevalent during the years [2000] [2001] [2002] . Note that the same query without a temporal constraint would tend to generate results only about latest taxation details and not necessarily from the required time.. Efficient evaluation of these time-travel queries over archives of large collections (such as the Web) is a challenge that we address in this paper. Earlier efforts combined the ideas of index organization in temporal databases with inverted indexes [7, 10, 20] , which essentially meant that the timeenriched inverted indexes are sliced along the time-axis (or partitioned vertically). Each resulting partition of the index list contains index entries of only those document versions that have a valid-time interval overlapping with the time interval of the partition. Answering a time-travel query in such a temporally sliced index involves choosing the correct -much smaller-index partition (or partitions) to evaluate the query.. Though attractive, such an index organization suffers from an index-size blowup incurred during the slicing process, since index entries whose valid-time interval spans across the slicing boundary are replicated. While a careful choice of the slicing boundaries can help to reduce the index blowup [10] in order to achieve acceptable levels of efficiency, 2 to 3 times index size increase is necessary. This work presents a novel method of index organization based on sharding for solving time-travel queries. Previous approaches traded off space and efficiency of query processing resulting in an index size blowup. Also, there was no single partitioning scheme for a given query time interval granularity. Unlike this, we look at minimizing the I/O during query processing without incurring any overhead in space and are independent of the query workload. We also propose a cost model, which takes into account different I/O costs of the local storage media where the index is stored. The resulting sharding balances sequential read costs and random access costs for efficient query processing. We carried out extensive experiments and show that with no space overhead we consistently perform better than the state-of-art sliced indexes, sometimes even by a factor of 2. Temporal information associated with documents has recently seen increasing attention in information retrieval. One of the earliest known efforts in this direction is by Anick and Flynn [8] who developed a framework for versioning the complete index for historical queries. Recently, Alonso et al. [6] give an overview of relevant research directions. The work on time-travel text search by Berberich et al. [10] is closest to our work in this paper. It introduced the notion of time-travel queries, limiting its discussion to time-point queries. To support this functionality efficiently, index lists from an inverted index are temporally partitioned, in our terminology into slices, providing guarantees on either the space used by the index or query performance. Index entries whose valid-time interval overlaps with multiple of the determined temporal slices are judiciously replicated and put into multiple index lists, thus increasing the overall size of the index. The work by Herscovici et al. [16] focuses on exploiting the redundancy commonly seen in versioned documents to compress the inverted index. Similarly, He et al. [14, 15] consider the problem of efficiently storing inverted indexes on disk using compression; these are orthogonal to our work and could be combined with our sharding techniques.. Research in temporal databases has taken a broader perspective beyond text documents and targeted general class of time-annotated data. Index structures tailored to such data like the Multi-Version B-Tree [9] or LHAM [18] are re- lated to our work, since they also, implicitly or explicitly, rely on a temporal partitioning and replication of data. It is therefore conceivable to apply our proposed techniques in conjunction with one of these index structures.

O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


P Systems and Topology: Some Suggestions for Research Summary. Lately, some studies linked the computational power of abstract computing systems based on multiset rewriting to Petri nets and the computation power of these nets to their topology. In turn, the computational power of these abstract computing devices can be understood just looking at their topology, that is, information flow.This line of research is very promising for several aspects:its results are valid for a broad range of systems based on multiset rewriting; it allows to know the computational power of abstract computing devices without tedious proofs based on simulations; it links computational power to topology and, in this way, it opens a broad range of questions.In this note we summarize the known result on this topic and we list a few suggestions for research together with the relevance of possible outcomes. Imagine that a computing device based on multiset rewriting is defined. Let us call this computing device S 1 . This could be a P system with symport/antiport, or with catalysts, of with conformons, or it could be a definition of Diophantine equations, or some model of Brane calculi of whatever else you like.. What would you do in order to know the computing power of such a system? Probably you would try to simulate with S 1 another computing system, say S 2 with known computational power. If this is possible, than you can say that S 1 can compute at least as much as S 2 can compute. Then you would probably try to simulate S 1 with S 2 , if this is possible, then you know that the two systems have the same computational power. This is the standard way to know the computational power of a computing system.. There is another way to know the computational power of S 1 . This new way is based on the fact that a computing system has a way to store information and a way to manipulate it. This other way looks at how such a system stores information and how it manipulates it and it deducts (in between other things) the computing power of S 1 .. From a point of view this is not a new idea: this is a very well establish concept in formal grammars. If somebody gives you a formal grammar S 1 and asks what it can compute, probably you would not try to simulate with S 1 another grammar S 2 of known computing power. Instead, you would simply look at the productions of S 1 and from this (because of the Chomsky hierarchy) you would be able to deduct what it can be generated by S 1 .. The approach that we are going to consider in this paper is about this: it shows how to know the computing power of a formal system based on multiset rewriting without running simulations but simply looking at the kind of operations that the formal system can perform.. The suggestions for research present in Section 7 have been in part inspired by conversations taking place during the 7 th Brainstorming Week on Membrane Computing (February 2 -6, 2009, Seville, Spain). This note is not self contained, it has been written having in a mind readers knowledgeable in P systems and with a strong interest in Petri nets. Citation indicate where the used but not defined concepts can be found.

B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O


Clustering with Bregman divergences A wide variety of distortion functions are used for clustering, e.g., squared Euclidean distance, Mahalanobis distance and relative entropy. In this paper, we propose and analyze parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences. The proposed algorithms unify centroid-based parametric clustering approaches, such as classical kmeans and information-theoretic clustering, which arise by special choices of the Bregman divergence. The algorithms maintain the simplicity and scalability of the classical kmeans algorithm, while generalizing the basic idea to a very large class of clustering loss functions. There are two main contributions in this paper. First, we pose the hard clustering problem in terms of minimizing the loss in Bregman information, a quantity motivated by rate-distortion theory, and present an algorithm to minimize this loss. Secondly, we show an explicit bijection between Bregman divergences and exponential families. The bijection enables the development of an alternative interpretation of an efficient EM scheme for learning models involving mixtures of exponential distributions. This leads to a simple soft clustering algorithm for all Bregman divergences. Data clustering is a fundamental "unsupervised" learning procedure that has been extensively studied across varied disciplines over several decades [14] . It has produced several parametric clustering methods which partition the data into a pre-specified number of partitions with a cluster representative corresponding to every cluster, such that a well-defined cost function involving the data and the representatives is minimized. For hard clustering, wherein the partitions are disjoint, the most well-known and widely used algorithm of this type is the iterative relocation scheme of Euclidean kmeans [14] . The popularity of this algorithm stems from its simplicity and scalability. The corresponding soft 1 clustering algorithm obtained by applying EM [9] to a mixture model of Gaussians with identical, isotropic covariances, is also popular and can be scaled to large data sets [6] .. Underlying both hard and soft Euclidean kmeans is a Gaussian "noise" model, which corresponds to a squared-Euclidean distortion function [15] . This dis-tortion function is also implicit in several other scalable techniques in the data mining literature. However, in many data mining applications, this distortion function is not a good match with the data, and consequently kmeans performs poorly as compared to other approaches [25] . In fact, in such situations kmeans often becomes a convenient strawman to show the superiority of a competing technique! This has also led to the search for more appropriate distance functions for specific applications [1, 25] .. Is it possible to devise an algorithm which has the simplicity and scalability of kmeans but can cater to a much larger class of distortion functions? A hint towards an affirmative answer to this question is provided by the Linde-Buzo-Gray (LBG) algorithm [17] based on the Itakura-Saito distance, which has been used in the signal-processing community for clustering speech data. The more recent information theoretic clustering algorithm [10] for clustering probability distributions also has a flavor similar to kmeans. This algorithm uses the KL-divergence as the distortion function and is wellsuited for various clustering tasks in the analysis of highdimensional text data.. Our question can now be posed as: what class of distortion functions admit an iterative relocation scheme where a global objective function based on the distortion with cluster centroids is progressively decreased? In this paper, we give a precise answer to this question: we show that such a scheme works for arbitrary Bregman divergences. In fact, it can be shown [4] that such a scheme only works for Bregman divergences. The scope of this result is vast since Bregman divergences include a large number of useful loss functions such as square loss, KL divergence, logistic loss, Mahalanobis distance, Itakura-Saito distance, hinge loss, etc.. We pose the hard clustering problem as one of obtaining an optimal quantization in terms of minimizing the loss in Bregman information, a quantity motivated by rate-distortion theory. A simple analysis then yields a version of the loss function that readily suggests a natural algorithm to solve the clustering problem for arbitrary Bregman divergences. Partitional hard clustering to minimize the loss in mutual information, a topic of recent study [10] , is seen to be a special case of our approach. Thus, this paper unifies several parametric partitional clustering approaches.. Further, we present a fundamental theoretical result by showing that there exists a bijection between Bregman divergences and exponential families. Since generative model-based soft clustering algorithms typically use mixtures of exponential distributions to model data, we revisit EM for mixture model estimation for this class of problems. We show that, with proper representation, the bijection gives an alternative interpretation of a well known efficient EM scheme [22] applicable in this case. The scheme simplifies the computationally intensive maximization-step of the EM algorithm, resulting in a general soft-clustering algorithm for all members of the exponential family, e.g., Poisson, Bernoulli, Binomial and Multinomial models. Both hard and soft clustering versions have essentially the same scalability as kmeans. Moreover they can be readily adapted to mixed data types, where different distortion functions within the family of Bregman divergences are appropriate for different subsets of features. This makes our theory and techniques suitable for a much wider class of data mining applications.. The remainder of this article is organized as follows. We introduce the concept of Bregman information to motivate the Bregman hard clustering problem and propose an algorithm to solve this clustering problem in section 2. In section 3, we establish a connection between exponential families and Bregman divergences and use it develop a soft Bregman clustering algorithm in section 4. In section 5, we present some experimental results that illustrate the usefulness of the Bregman clustering algorithm. In section 6, we discuss related work. Finally, in section 7, we present concluding remarks.. A word about the notation: bold faced variables, e.g., x, µ, etc., represent vectors, sets are represented by calligraphic upper-case alphabets, e.g., X , Y, etc., and enumerated as {x i } n i=1 where x i are the elements of the set. R, R ++ and R d denote the set of reals, the set of positive reals and the d-dimensional real vector space respectively. x denotes the L 2 norm. Probability density functions are denoted by lower case alphabets, e.g., p, q, etc. Probability measure on a set is denoted by ν. If a random variable X is distributed as p, we denote this by X ∼ p. Expectation of functions of a random variable X ∼ p are denoted by E p [·] when the random variable is clear from the context. The inverse of a function f is denoted by f −1 . In this paper, we presented hard and soft clustering algorithms minimizing loss functions involving Bregman divergences. This analysis presents a unified view of an entire class of parametric clustering algorithms. First, in the hard-clustering framework, we show that a kmeans type iterative relocation scheme solves the Bregman hard-clustering problem for all Bregman divergences. Further, from a related result, we see that Bregman divergences are the only distortion functions for which such a centroid-based clustering scheme is possible. Secondly, using insights from existing literature, we show that there is a bijection between Bregman divergences and exponential families. This result is useful in developing an alternative interpretation of the EM algorithm for learning mixtures of exponential distributions, eventually resulting in a set of Bregman softclustering algorithms.. As discussed in the paper, special cases of this analysis have been discovered and widely used by researchers in applications ranging from speech coding to text clustering. There are four salient features of this framework that make these results particularly useful for datamining applications. First, the computational complexity of the entire class of Bregman clustering algorithms is linear in the data-points. Hence, the algorithms are extremely scalable and appropriate for large-scale datamining tasks. Secondly, the modularity of the proposed class of algorithms is evident from the fact that only one component in the proposed schemes, viz the Bregman divergence used in the assignment step, needs to be changed to get an algorithm for a new loss function. This simplifies the implementation and application of this class of algorithms to various data types. Thirdly, the algorithms discussed are applicable to mixed data types that are commonly encountered in data-mining. Since the linear combination of convex functions with non-negative coefficients is always convex [23] , one can have different convex functions appropriately chosen for different sets of features. The Bregman divergence corresponding to any linear combination of the component convex functions can now be used to cluster the data. This vastly increases the scope of the proposed techniques. Finally, because of the similarity of Bregman hard clustering to kmeans, existing techniques that employ kmeans in various settings such as data stream clustering, privacy preserving clustering, etc., can be easily extended to the general framework of Bregman clustering. This work is largely inspired by three broad and overlapping ideas. First, considering the clustering problem from an information theoretic viewpoint is very insightful. Such considerations occur in several techniques, from classical vector quantization to information theoretic clustering [10] and the information bottleneck method [26] . In particular, the information theoretic clustering [10] approach solved the problem of distributional clustering with a formulation involving loss in Shannon's mutual information. In this paper, we have significantly generalized that work by proposing techniques for obtaining optimal quantizations by minimizing loss in Bregman information corresponding to arbitrary Bregman divergences. Second, our soft clustering approach is based on the relationship between Bregman divergences and exponential distributions and the suitability of Bregman divergences as distortion or loss functions for data drawn from exponential distributions. It has been previously shown [3] that the KL-divergence, which is the most natural distance measure for this parameter space, between two members p θ and pθ of an exponential family, is always a Bregman divergence. In particular, it is the Bregman divergence D ψ (θ,θ) corresponding to the cumulant function ψ of the exponential family. In our work, we extend this concept to say that the Bregman divergence of the Legendre conjugate of the cumulant function is, in some sense, a natural distance function for the data drawn according to that exponential family.. The third broad idea is that many learning algorithms can be viewed as solutions for minimizing loss functions based on Bregman divergences. Elegant techniques for the design of algorithms and the analysis of relative loss bounds in the online learning setting extensively use this framework [3] . In the unsupervised learning setting, use of this framework typically involves development of alternate minimization procedures [8] . For example, [21, 27] analyze and develop iterative alternate projection procedures for solving unsupervised optimization problems involving objective functions based on Bregman divergences under various kinds of constraints. Further, [7] develops a generalization of PCA for exponential families using loss functions based on the corresponding Bregman divergences and proposes alternate minimization schemes for solving the problem.. There has also been work on learning algorithms that involve minimizing loss functions based on distortion measures that are somewhat different from Bregman divergences. For example, [19] presents the convexkmeans clustering for distortion measures that are always non-negative and convex in the second argument, using the notion of a generalized centroid. Bregman divergences, on the other hand, are not necessarily convex in the second argument and also, the minimizer of the Bregman clustering loss function always happens to be the actual centroid, i.e., the expectation of the set. There are a number of experimental results in existing literature [17, 10, 20, 16] that illustrate the usefulness of Bregman divergences and the Bregman clustering algorithms in specific domains. The classical kmeans algorithm, which is a special case of the Bregman hard clustering algorithm for the squared Euclidean distance has been successfully applied to a large number of domains where a Gaussian distribution assumption is valid. Besides this, there are at least two other domains where special cases of Bregman clustering methods have been shown to provide good results.. The first is the text-clustering domain where the information-theoretic clustering algorithm [10] and the EM algorithm based on the Naive-Bayes model [20] , which are, respectively, special cases of the Bregman hard and soft clustering algorithms for KL-divergence have been shown to provide superior results to other existing algorithms on large real datasets such as the 20-Newsgroups, Reuters and Dmoz datasets. This is to be expected as text documents can be effectively modeled using multinomial distributions whose corresponding Bregman divergence is just the KL-divergence between the word distributions. Recently, [16] showed that a convex combination of KL-divergence and squared Euclidean distance seems to give even better performance on this domain. We note that [16] essentially uses a Bregman divergence derived from a convex function that is convex combination of two convex functions [23] , and hence, is a special case of the proposed Bregman clustering framework.. Speech coding is another domain where a special case of the Bregman clustering algorithm based on the Itakura-Saito distance, namely the Linde-Buzo-Gray (LBG) algorithm [17] , has been widely and successfully applied. Again, this is to be expected as speech power spectra tend to follow exponential family density of the form p(x) = λe −λx , whose corresponding Bregman divergence is the Itakura-Saito distance.. Since special cases of Bregman clustering algorithms have already been known to provide substantial improvements over other existing methods in certain domains, we do not experimentally re-evaluate the Bregman clustering algorithms against other methods. Instead, we only focus on showing that the quality of the clustering depends on the match between the data characteristics and the choice of Bregman divergence used for clustering.. In order to do this, we performed two experiments using datasets of increasing level of difficulty. For our first experiment, we created three 1-dimensional datasets of 100 samples each, based on mixture models of Gaussian, Poisson and Binomial distributions respectively. All the mixture models had three components with equal priors centered at 10, 20 and 40 respectively. The standard deviation, σ of the Gaussian densities was set to 5 and the number of trials N of the Binomial distribution was set to 100 so as to make the three models somewhat similar to each other. The datasets were then each clustered using three versions of the Bregman hard clustering algorithm corresponding to the Bregman divergences obtained from the Gaussian (kmeans), Poisson and Binomial distributions respectively. The quality of the clustering was measured in terms of the normalized mutual information 6 [24] between the predicted clusters and the original clusters and the results were averaged over 10 trials. Table 4 shows the normalized mutual information values for the different divergences and datasets. From the table, we can see that clustering quality depends on the choice of Bregman divergence and is significantly better when the appropriate Bregman divergence is used. The second experiment involved a similar kind of comparison of clustering algorithms for multidimensional datasets drawn from multivariate Gaussian, Binomial and Poisson distributions respectively. The datasets were sampled from mixture models with 15 overlapping components and had 2000 10-dimensional samples each. The results of the Bregman clustering algorithms shown in table 5 lead to the same conclusion as before, i.e., the choice of the Bregman divergence used for clustering is crucial for obtaining good quality.

B O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O B O O O O O O O O O O O O B I O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O B O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O


Effective keyword-based selection of relational databases The wide popularity of free-and-easy keyword based searches over World Wide Web has fueled the demand for incorporating keyword-based search over structured databases. However, most of the current research work focuses on keywordbased searching over a single structured data source. With the growing interest in distributed databases and service oriented architecture over the Internet, it is important to extend such a capability over multiple structured data sources. One of the most important problems for enabling such a query facility is to be able to select the most useful data sources relevant to the keyword query. Traditional database summary techniques used for selecting unstructured data sources developed in IR literature are inadequate for our problem, as they do not capture the structure of the data sources. In this paper, we study the database selection problem for relational data sources, and propose a method that effectively summarizes the relationships between keywords in a relational database based on its structure. We develop effective ranking methods based on the keyword relationship summaries in order to select the most useful databases for a given keyword query. We have implemented our system on PlanetLab. In that environment we use extensive experiments with real datasets to demonstrate the effectiveness of our proposed summarization method. In this paper, we introduce a novel summary technique for relational databases for enabling keyword-based selection of distributed data sources. Different from traditional summary used in IR for text databases that mainly relies on keyword frequency information, our summary exploits the structure of the relational database and contains pairs of keywords with scores indicating the strength of their relationship, which are measured based on the references between tuples. We also propose an estimation method for ranking the usefulness of databases in answering a keyword query with our KR-summary. Our experimental results with real dataset demonstrate the effectiveness of our proposed summary approach. Further, our evaluation of the distributed indexing mechanism for the KR-summaries implemented over PlanetLab [7] confirms its feasibility, scalability and efficiency over a real distributed environment. Indeed, unlike traditional data integration techniques, our free-and-easy keyword based selection method requires no human intervention, hereby enabling scalability over a large network of distributed relational data sources.. Our summary technique can be extended in two directions. First, we can further incorporate IR-based weighting methods and weightings based on link structure into the KR-summary. For example, the D matrix could include the weighting of each keyword in the tuples it appears in, and the T matrix could use real numbers to indicate the importance of different links, instead of binary values. Second, we can exploit some sampling-based methods for constructing the KR-summary more efficiently and making it more compact. We discuss the related work to our approach in this section. In order to evaluate the performance of our proposed approach for the selection of relational databases, we have implemented the system over PlanetLab [7] , which is a testbed for large-scale distributed systems. All codes are written in Java. We built the distributed index for KR-summaries over a Chord-based [28] P2P network as described in Section 3.2. We selected sixteen physical nodes on Planetlab in various areas, and each physical node is used to simulate several to hundreds of Chord nodes, where each Chord node shares one relational database.. We use real life DBLP 1 dataset to generate 82 relational databases by dividing the whole dataset according to different bibliography types such as inproceedings, articles, books, etc.. Figure 8 shows the schema of the databases storing inproceedings papers. The schemas of other bibliography types are similar. There is no overlap between different generated databases. The average number of tuples per database is 46735, and the average number of distinct keywords extracted from each database is 19817, after removing stop words and stemming. The numbers of tuples of different databases are similar. Keyword queries are composed of randomly selected keywords from the databases. We tested with a set of 112 queries in total, which consists of 30 2-keyword queries, 34 3-keyword queries, 21 4-keyword queries, and 27 5-keyword queries.. We use MySQL 2 to store all the databases and generate KR-summaries for them.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Balanced and 1-balanced graph constructions There are several density functions for graphs which have found use in various applications.In this paper, we examine two of them, the first being given by b(G) = |E(G)|/|V (G)|, and the other being given by has been useful in designing networks with reduced vulnerability to attack and in studying the World Wide Web, and a similar function is useful in the study of rigidity. First we give a new characterization of balanced graphs. Then we introduce a graph construction which generalizes the Cartesian product of graphs to produce what we call a generalized Cartesian product. We show that generalized Cartesian product derived from a tree and 1-balanced graphs are 1-balanced, and we use this to prove that the generalized Cartesian products derived from 1-balanced graphs are 1-balanced. We follow the notation of Diestel [3] for graphs, with the major exceptions that we use K n for the complete graph on n vertices and we use b(G) for the function (G) := |E(G)| |V (G)| . Graphs considered in this paper are loopless, but multiple edges are allowed. If a graph has an edge, it is called non-trivial. In this paper we look at two density functions, both related to the average degree of a graph. The first of these is b(G) = |E(G)| |V (G)| for a graph G. Graph G is said to be balanced if for all non-trivial subgraphs H of G,

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Syndrome Measurement Strategies for the [[7,1,3]] Code Abstract Quantum error correction (QEC) entails the encoding of quantum information into a QEC code space, measuring error syndromes to properly locate and identify errors, and, if necessary, applying a proper recovery operation. Here we compare three syndrome measurement protocols for the [[7,1,3]] QEC code: Shor states, Steane states, and one ancilla qubit by simulating the implementation of 50 logical gates with the syndrome measurements interspersed between the gates at different intervals. We then compare the fidelities for the different syndrome measurement types. Our simulations show that the optimal syndrome measurement strategy is generally not to apply syndrome measurements after every gate but depends on the details of the error environment. Our simulations also allow a quantum computer programmer to weigh computational accuracy versus resource consumption (time and number of qubits) for a particular error environment. In addition, we show that applying syndrome measurements that are unnecessary from the standpoint of quantum fault tolerance may be helpful in achieving better accuracy or in lowering resource consumption. Finally, our simulations demonstrate that the single-qubit non-fault tolerant syndrome measurement strategy achieves comparable fidelity to those that are fault tolerant. In conclusion we have explored different syndrome measurement techniques for the [ [7, 1, 3] ] QEC code in different error environments. The three SM types explored are those using Shor states and Steane states, both of which are fault tolerant, and using a single qubit ancilla, which is not fault tolerant. We explore two variations of both the Steane state and single-qubit SM methods: the SM are repeated or not repeated. The simulations implement 50 logical gates within the [ [7, 1, 3] ] QEC encoding with SM applied at various intervals using the different types of SM. The results demonstrate that the choice of SM leading to the highest fidelity, and the number of times SM should be applied during the 50 gates that will optimize fidelity, will strongly depend on the error environment. In a depolarizing environment and when bit-flip and σ y errors dominate, the SM type leading to the higest fidelity is the Steane states. Even so, the number of times the SM should be applied to give this highest fidelity is different for the three environments. When phase-flip errors are dominant the Steane state SM give the lowest fidelity. The Shor state and single-qubit SM always give similar fidelities with the single-qubit SM fidelity being slightly higher.. The different SM methods utilize different amount of resources and our results allow us to properly weigh resource consumption, time and the number of qubits, versus accuracy as measured via fidelity. While a given SM may provide the highest overall fidelity the cost in resources may prove overburdensome. For example, when performing the 50 gates in a depolarizing environment the highest fidelity is achieved using Steane ancilla applied every five logical gates (q = 10) and repeating the SM at a total cost of 560 qubits. Applying repeating one-qubit SM just once after all 50 gates costs only 2 qubits with a sacrifice in fidelity of about p/10.. Furthermore, we note that despite the fact that the single-qubit SM are not fault tolerant, they generally achieve a fidelity that is relatively close to the other SM, at least over 50 gates. Whether this condition will hold over larger numbers of gates will be explored elsewhere but it will likely depend strongly on the details of the error environment and how often SM are applied. Nevertheless, these simulations suggest that at least for small computations high accuracy may be achievable without strictly following all the tenets of QFT.. I would like to thank G. Gilbert for insightful comments. This research is supported under MITRE Innovation Program Grant 51MSR662.

O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An empirical re-sampling method on intrinsic mode function to deal with speed variation in machine fault diagnostics Order tracking has proved to be effective in dealing with the effects of speed variation in the analysis of rotating machinery vibration signals. To implement traditional order tracking in practice requires rotational speed information. However, it may be difficult in some cases to mount an appropriate monitoring device to obtain reliable speed information. In this paper, a novel empirical re-sampling of intrinsic mode functions obtained from empirical mode decomposition is explored, so that the approximation of order tracking effects without rotational speed is possible. At the same time, the newly introduced intrinsic cycle concept in the intrinsic mode function simplifies linking of the resultant spectra to signal variations, and is therefore beneficial for condition monitoring of rotating machines. In the paper the rationale behind the technique is first explained. Secondly, the effectiveness of the technique is demonstrated on a dynamic gear simulation model. Lastly, the technique is applied to experimental data from a gearbox test rig. Both the simulation and experimental studies corroborate the usefulness of the proposed technique. Rotating machinery vibration signals often suffer from the effects of variation of the rotational speed. Order tracking techniques were therefore developed to deal with these influences. In recent years, various classes of order tracking techniques have been developed and applied. These include computed order tracking which transforms time domain signals into the angle domain, Vold-Kalman filter order tracking which extracts each individual order or orders from the time domain, etc. To implement traditional order tracking one needs to capture vibration and rotational speed signals simultaneously. The vibration signals are usually measured by accelerometers while the rotational speed signals are captured by tachometers. However, direct measurement of speed may sometimes be difficult due to the complexities of the machine, e.g. inaccessible gearboxes inside a machine. Thus, the analyst sometimes has to compromise on accuracy for simplification of the system. Furthermore, reliable order tracking (such as Vold-Kalman filter order tracking) requires experience and sufficient mathematical background to support the analysis. These factors may also impede the advancement of order tracking in practice. Simplified signal processing methods to achieve order tracking effects in practical application on real rotating machines might be very beneficial. Thus, it is reasonable to explore the possibilities of using other signal processing methods without the involvement of rotational speed to approximate the effects of order tracking on rotating machine vibration signals.. Vibration signals from rotating machinery may in general be non-stationary and nonlinear. In the literature, a number of techniques have been proposed to deal with such signals. These include techniques to deal with the effects of frequency variation (such as order tracking [1] ), modeling techniques for diagnosis (for example cyclostationary modeling [2] ) and statistics based methods to detect and characterize non-stationarity (for example spectral kurtosis [3] ), as well as time-frequency domain methods to capture the variation of signals with the time (for example the wavelet transform). Various techniques have been developed, but few can deal with both non-stationary and nonlinear signals properly.. In 1998 Huang and his colleagues [4] developed a method that can deal with both non-stationary and nonlinear signals and is known as empirical mode decomposition (EMD). It is an adaptive and unsupervised method with which complicated signals can be separated into a collection of intrinsic mode functions (IMFs) in the time domain. IMFs have been proved to have the ability of capturing useful vibration signals for rotating machine fault diagnosis (see for example [5] , [6] and [7] ). However, due to the possible frequency variation of IMFs, signal processing techniques that are dependent on Fourier analysis will also be influenced by smearing of the spectra. Specifically for rotating machine vibration signals, the speed variation effects would therefore still remain in the IMFs. Thus, order tracking analysis of IMFs will naturally enhance the diagnostic ability. Since there is no need for rotational speed information in the EMD process, this does however provide possibilities to simplify the order tacking analysis based upon it.. In this paper, a novel way of reconstructing IMFs from EMD to approximate the effect of computed order tracking for rotating machine vibration signals is proposed. In stead of using traditional speed information to achieve the order tracking, an empirical re-sampling method is used on the IMFs which approximates the order tracking effects. Avoiding the need for speed information largely simplifies the practice of signal measurement and the method offers an alternative condition monitoring tool. Besides, the proposed method uses IMFs in stead of complicated raw signals. The simple data structure of IMFs offers additional advantages over the traditional signal processing method in the frequency domain, in that more direct connection between the frequency spectra and signal variations can be obtained. This will be further explained in the next paragraph. In the following, the first part of the paper explains the rationale behind the technique. Subsequently a simplified gear mesh model is used to demonstrate its ability in simulation studies. Lastly, real experimental gearbox data is studied and demonstrates the ability of the technique in condition monitoring. In this paper, a novel method based on re-sampling of an intrinsic mode function from the empirical mode decomposition method is developed. It is called the intrinsic cycle re-sampling method. The re-sampling process takes advantage of an intrinsic mode function of symmetric, oscillating and zero mean nature to reconstruct an intrinsic mode function into a re-sampled intrinsic mode function which suppress frequency variations of the signal. In this way an approximation of order tracking is obtained without knowledge of the rotational speed. At the same time, it is known that intrinsic mode functions may include non-stationary and nonlinear signals which contain ample machine condition information. Therefore, inherent to the intrinsic cycle re-sampling method developed here, is its ability in condition monitoring. The method further brings advantages for condition monitoring, in that the empirical intrinsic mode function offer specific characteristics for trending. Two parameters are suggested to link signal variations to the final intrinsic cycle re-sampling spectrum map, namely main frequency component and sidebands variations. The simulation and experimental studies both demonstrate that through tracking the changes of these two parameters, intrinsic cycle re-sampling method provides direct and clear connections to signal variations so as to provide a good alternative method for condition monitoring.

O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O