Analyzing correspondence between sound objects and body motion Links between music and body motion can be studied through experiments called sound-tracing. One of the main challenges in such research is to develop robust analysis techniques that are able to deal with the multidimensional data that musical sound and body motion present. The article evaluates four different analysis methods applied to an experiment in which participants moved their hands following perceptual features of short sound objects. Motion capture data has been analyzed and correlated with a set of quantitative sound features using four different methods: (a) a pattern recognition classifier, (b) t-tests, (c) Spearman's ρ correlation, and (d) canonical correlation. This article shows how the analysis methods complement each other, and that applying several analysis techniques to the same data set can broaden the knowledge gained from the experiment. Body motion is an integral part of the experience of music. On the one hand, body motion can create sound, such as a sound-producing action on a musical instrument. As humans, we all have some ecological knowledge of how sound-producing actions relate to sound [Clarke 2005 ]. For instance, most of us immediately know what sound to expect when we see two cymbals that are about to crash. On the other hand, sound can lead to body motion. We experience this when music makes us tap a foot or bop our head, and music can influence basic human motion patterns, such as walking [Styns et al. 2007 ]. As such, Leman [2008] has suggested that the human body functions as a "mediator" between physical. The research leading to these results has received funding from the European Union Seventh Framework Programme under grant agreement no. 257906, Engineering Proprioception in Computer Systems (EPiCS), and the Norwegian Research Council, project no. 183180 Sensing Music-Related Actions (SMA). Authors' addresses: K. Nymoen and J. Torresen, University of Oslo, Department of Informatics, Postboks 1080 Blindern, 0316 Oslo, Norway; R. I. Godøy and A. R. Jensenius, University of Oslo, Department of Musicology, Postboks 1017 Blindern, 0316 Oslo, Norway. Correspondence email: krisny@ifi.uio.no. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or permissions@acm.org. c 2013 ACM 1544-3558/2013/05-ART9 $15.00 DOI: http://dx.doi.org/10. 1145/2465780.2465783 sound waves and the mind, structuring our perception of musical sound. Music-related motion (like dance) then becomes a type of corporeal articulation of our cognition of music. Further, it has been suggested that mental images of motion play an essential role in our involvement with music, and that most features in a musical soundscape could potentially afford gestural responses by listeners [Godøy 2006 [Godøy , 2010a .. It has been shown that our perception of phenomena is often done through a combination of modalities-an effect known as cross-modality [Stein and Meredith 1993; Vroomen and de Gedler 2000] . While research on cross-modality to a large extent has been concerned with interactions between vision, audition and tactile perception, motion sensations have also been suggested as important to our perception-for instance in Liberman and Mattingly's [1985] motor theory of speech perception. This theory claims that speech is understood not only as sound, but also through our own experience with gestures that produce phonemes. Although originally conceived as a special phenomenon for speech perception, Galantucci et al. [2006] have presented convincing arguments for why this theory should be extended to perception in general.. The evidence for an interaction between motor sensations and other modalities was strengthened with the discovery of mirror neurons in the premotor cortex of macaques [Pellegrino et al. 1992] . These neurons were shown to activate not only when the macaque performed an action, but also when seeing the action performed by someone else, and even when only hearing the sound of the action [Kohler et al. 2002] .. Based on considerations like the ones presented previously, it has been suggested that research on relationships between sound and body motion can help us learn more about processes in the brain [Zatorre 2005 ]. Furthermore, this research can contribute to more efficient solutions for so-called query-by-gesture in music information retrieval databases [Leman 2008; Godøy and Jensenius 2009] . Moreover, research on correspondences between musical sound and body motion can facilitate development of new musical interfaces, where body motion is the main input source [van Nort 2009; Nymoen et al. 2011b] .. One of our contributions to research on sound and motion has been studies of so-called sound-tracing Nymoen et al. 2010 Nymoen et al. , 2011a Nymoen et al. , 2012 . The point of sound-tracing is to observe and collect data of listener's spontaneous gestural renderings of sound, that is, to capture listeners' immediate associations of sound and motion. In these studies, participants have been asked to render the perceptual qualities of short sound objects through bodily motion, either in two dimensions by drawing with a pen on a digital tablet, or three dimensions by free-air hand gestures. The underlying assumption and motivation of sound-tracing experiments is that they may reveal salient sound-motion links in perception. A main challenge with this type of research is to develop techniques for capturing, storing, analyzing, and visualising both action and sound in a coherent and consistent manner. This article addresses the analysis of such data, evaluating strengths and weaknesses of using standard correlation measures, hypothesis tests, and pattern recognition classifiers as analysis tools. We have shown that different techniques for analyzing sound-tracings have different strengths and weaknesses. The choice of method depends on the aim of the research, but when sound-motion links at both meso and micro timescales are of interest, it may be appropriate to apply multiple analysis methods. Our analysis showed that people performed similar actions to sounds with rising tendencies, and also to sounds with falling tendencies, whether the rise or fall was in pitch or in spectral centroid. These similarities existed even when the sounds were easy to tell apart. As such, we can say that the sounds in our experiment were more detailed than the motion responses.. Although this article has shed light upon some of the possible strategies for quantitative analysis of sound-tracings, there is still a need for defining methods that can analyze correspondences between input data and response data that are time-varying and multidimensional, and that ideally also allows proper testing of the significance of the results. We find the challenge of analyzing multidimensional data in music and motion intriguing and hope that the challenge will inspire readers to contribute to future research in this area.. The experiment in the present article did not consider the spatial aspect of sound. Our lab has recently acquired a 32-channel speaker setup, and we plan to pursue research on links between sound spatialization and body motion in the future. In future work we will also apply the results of our soundtracing studies in choosing algorithms for actively controlling music in mobile music systems, and to define mappings in new musical interfaces.. Furthermore, we plan to continue our experiments on cross-modal analysis, specifically by having a different set of subjects match the sounds that were used in this experiment to the recorded motion data. After presenting the four different approaches for analyzing the recorded motion data, we will now evaluate our results and the analysis methods used. We will start by looking at the results themselves in Section 8.1, and then continue by evaluating the utility of the analysis techniques in Section 8.2. The overall results from the pattern recognition analysis serve as a good point of departure for our discussion. This analysis confirmed that there were certain similarities between the sound-tracings that different subjects performed to the same sound. At a first glance, the precision of the classifier might not have been very impressive, with an average classification accuracy of 22.5%. However, if we would compare this accuracy to a random selection (which would give an accuracy of 5.6%), it is evident that the classifier indeed must have identified some similarities between the motion recordings. Furthermore, we could clearly see certain clusters in the output of the classifier. Most of the sound-tracings were predicted by the classifier to be one of the sound classes that were similar to the correct sound. This suggests that the extracted motion features reflected salient aspects of how we relate motion to sound.. The most apparent correspondence we observed between a sound feature and motion feature was the positive correlation between vertical position and pitch. This was not surprising, as the same has been reported in previous sound-tracing experiments by ourselves and other researchers (e.g., Eitan and Granot [2006] , Huron et al. [2009] , and Nymoen et al. [2011a] . This relationship may be explained through learned metaphors where pitch is represented in a vertical dimension, such as the vertical ordering of notes in a musical score, or the typical cartoon scenario where the bad coyote falls of a cliff, accompanied by a descending glissando.. It should be noted that another typical metaphor for pitch is the size metaphor, where high pitch would be related to small size [Huron et al. 2009 ]. Hence, we could have expected a negative correlation between hand distance and pitch in our experiment. A high number of components in the CCA were correlated to both hand distance and pitch, and we also saw this relationship through high absolute values in the Spearman correlation. However, there was no agreement among the participants as to whether rising pitch should entail moving hands apart or together, and as such we cannot say that this is a strong relationship. Also, a t-test showed no significant difference between HandDivergenceMean for "falling" sounds and "rising" sounds.. The SVM classifier predicted several of the sounds with stable pitch but falling or rising spectral centroid (Sounds 7 and 8) to be one of the "steady" sounds, indicating that in addition to the soundtracings of "falling" and "rising" sounds, where vertical position is a pertinent feature, these soundtracings also had similarities with the sounds in the "steady" class. The same indication was previously found through an analysis of variance, where Sounds 7 and 8 caused significantly less vertical motion [Nymoen et al. 2012] .. The classifier was able to distinguish quite well between sound-tracings of impulsive and nonimpulsive sounds. Most of the sound-tracings to impulsive sounds had a high acceleration peak approximately synchronized with the sound onset. The significant difference in OnsetAcceleration between impulsive and non-impulsive sounds suggests that this feature was the most pertinent to distinguish between these groups. The data set that has been analyzed in this article was recorded in autumn 2010. This section will present the experimental setup, as well as the sound and motion features that have been used in the analysis process. Having applied four different analysis techniques to the data set, we have observed conceptual differences between the methods, and also learned more about the strengths and weaknesses of each of them.

O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Collision Detection for Deformable Objects Abstract For many years, collision detection has been of major interest in computer graphics. Numerous approaches have been investigated to detect interfering objects in applications such as robotics, computational biology, games, surgery simulation, and cloth simulation. While many of the original collision detection methods primarily address the problem of rigid bodies, recent approaches have started focusing on deformable objects.. Deformable collision detection is an essential component in interactive physically-based simulation and animation which is a rapidly growing research area with an increasing number of interesting applications. Fig. 1 illustrates one of the major applications for deformable collision detection, namely cloth simulation. In cloth simulation, approaches to dynamically deforming clothes have to be combined with efficient algorithms that handle self-collisions of cloth as well as the interaction of cloth with animated avatars. Such applications require collision detection algo- Surgery simulation illustrated in Fig. 2 is a second major application area for deformable collision detection. In such environments, collisions among deformable organs have to be detected and resolved. Furthermore, collisions between surgical tools and deformable tissue have to be processed. In the case of topological changes due to cutting, self-collisions of tissue can occur and have to be handled. Since interactive behavior of surgery simulation environments is essential, efficient algorithms for deformable collision detection are required. In addition to cloth and surgery simulation, deformable collision detection methods are useful in environments with animated objects. Fig. 3 illustrates a sequence of two animated objects. In this example, specific collision detection algorithms for deformable objects can be used to detect interferences of both objects at interactive rates. Furthermore, the intersection volume of both objects can be computed if their surfaces are closed. In BVH approaches, the efficiency of the basic BV has to be investigated very carefully. This is due to the fact, that deforming objects require frequent updates of the hierarchy. So far, it has been shown that AABBs should be preferred to other BVs, such as OBBs. Although OBBs approximate objects tighter than AABBs, AABBs can be updated or refit very efficiently. Additionally, 4-ary or 8-ary trees have shown a better overall performance compared to binary trees.. Although deformable modeling environments require frequent updates of BVHs, BVHs are nevertheless well-suited for animations or interactive applications, since updating or refitting of these hierarchies can be done very efficiently. If the deformations are known in advance, this can even be done in an output-sensitive manner. Furthermore, BVHs can be employed to detect self-collisions while applying additional heuristics to accelerate this process. Also, BVHs work with triangles and tetrahedrons as object primitives, which allows for a more sophisticated collision response compared to a pure vertex-based response.. Though the two discussed stochastic methods are principally different, they both have common characteristics. The most important one is the possibility to balance the quality of the collision detection or the collision ratio against computation time. It has been shown that stochastic approaches are applicable for real-time applications. However, it has to be considered, that no exact or physically-correct simulation is possible.. The first approach works on BVHs and can therefore extend many of the methods described in section 2. The second one is independent of any hierarchy and is directly employed to pairs of primitives which simplifies the collision response scheme.. Distance fields have been employed to detect collisions and even self-collisions in non-interactive applications. Although efficient algorithms for computing distance fields have been proposed recently, this generation is still not fast enough for interactive applications, where distance fields have to be updated during run-time due to deforming geometry. However, distance fields provide a highly robust collision detection, since they divide space strictly into inside and outside.. For interactive applications, distance fields can be used to represent all rigid objects contained within the environment. Since distance fields yield not only the penetration depth but also normals needed for collision response at interactive rates, collision detection between deformable objects and the rigid objects is carried out very efficiently. In order to decrease storage requirements or generation time, it is possibly to reduce the resolution of the distance field, which results in lowered accuracy. Thus, distance field approaches can balance performance and accuracy. Lev66] , the computation of molecular atom interaction is accelerated with a neighborhood search based on a uniform 3D grid. Another related approach is presented in [Rab76] , where a hash map is used to represent the 3D grid.. Spatial subdivision is a simple and fast technique to accelerate collision detection in case of moving and deforming objects. Spatial subdivision can be used to detect collisions and self-collisions. Algorithms based on spatial subdivision are independent of topology changes of objects. They are not restricted to triangles as basic object primitive, but also work with other object primitives if an appropriate intersection test is implemented.. The main difficulty in spatial subdivision is the choice of the data structure that is used to represent the 3D space. This data structure has to be flexible and efficient with respect to computational time and memory. In [THM * 03], a hash table is used which has shown to be very efficient in physicallybased simulation environments with dynamically changing spatial distributions of object primitives.. In contrast to other collision detection methods, image-space techniques do not require time-consuming pre-processing. This makes them especially appropriate for dynamically deforming objects. They can be used to detect collisions and self-collisions. Image-space techniques usually work with triangulated surfaces. However, they could also be used for other object primitives as long as these primitives can be rendered. Topology changes of objects do not cause any problems.. Since image-space techniques work with discretized representations of objects, they do not provide exact collision information. The accuracy of the collision detection depends on the discretization error. Thus, accuracy and performance can be balanced in a certain range by changing the resolution of the rendering process.. Image-space techniques can be accelerated with graphics hardware. However, due to buffer read-back delays and the limited flexibility of programmable graphics hardware, it is not always guaranteed that implementations on graphics hardware are faster than software solutions in all cases (see [HTG04] ). As a rule of thumb, graphics hardware should only be used for geometrically complex environments.. While image-space techniques efficiently detect collisions, they are limited in providing information that can be used for collision response in physically-based simulation environments. In many approaches, further post-processing of the provided result is required to compute or to approximate information such as the penetration depth of colliding objects. In this paper, a variety of deformable collision detection approaches has been presented. The discussed approaches are based on bounding volume hierarchies, distance fields, or spatial partitioning. Further, image-space techniques and stochastic methods have been described. All presented methods address problems which especially occur in deformable simulation environments.. Approaches based on bounding volume hierarchies have shown to be very efficient. In these methods, the basic bounding volume and the strategy for generating and updating the hierarchy have to be chosen very carefully in order to handle frequent update requests in simulation environments with deformable objects. Stochastic methods are a promising approach to time-critical applications, since they allow for balancing performance and accuracy.. Distance fields are especially appropriate for collision detection between rigid and deformable objects. In this case, pre-computed distance fields do not only detect collisions, but also provide the penetration depth which is essential for a physically-correct collision response. Collision detection based on spatial subdivision has also shown to be very efficient in deformable simulation environments. In these ap- proaches, research is mainly focused on efficient data structures.. Further interesting approaches to deformable collision detection are based on image-space techniques. These algorithms are commonly accelerated with graphics hardware. This makes them especially appropriate for environments with geometrically complex objects and promising results have been presented. While all approaches provide promising solutions to certain aspects in deformable collision detection, there exist no general or optimal approach. Further, it is not the intention of this survey to provide a consistent comparison of all approaches. This is not possible due to several reasons. First, the approaches require different input data, e. g., some approaches only work with closed surface meshes. Second, the approaches provide different collision information. They detect interfering surfaces or estimate penetration depth information. Some algorithms provide accurate collision in-formation, stochastic or image-space methods only compute approximate collision information. Some approaches are optimized for two-body collision tests, other methods, such as spatial subdivision, can handle n-body environments. Further, there exist very specialized solutions that are restricted to certain applications. Therefore, all approaches have their benefits, drawbacks, and restrictions in terms of a specific application.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Towards a goal-driven approach to action selection in self-adaptive software Self-adaptive software is a closed-loop system, since it continuously monitors its context (i.e. environment) and/or self (i.e. software entities) in order to adapt itself properly to changes. We believe that representing adaptation goals explicitly and tracing them at run time are helpful in decision-making for adaptation. While goal-driven models are used in requirements engineering, they have not been utilized systematically yet for run-time adaptation. To address this research gap, this article focuses on the deciding process in selfadaptive software, and proposes the Goal-Action-Attribute Model (GAAM). An action selection mechanism, based on cooperative decision-making, is also proposed, which uses GAAM to select the appropriate adaptation action(s). The emphasis is on building a light-weight and scalable run-time model which needs less design and tuning effort comparing with a typical rule-based approach. The GAAM and action selection mechanism are evaluated using a set of experiments on a simulated multi-tier enterprise application, and two sample ordinal and cardinal preference lists. The evaluation is accomplished based on a systematic design of experiment and a detailed statistical analysis with ANOVA in order to investigate several research questions. The findings are promising, considering the obtained results, and other impacts of the approach on engineering self-adaptive software. Although, one case study is not enough to generalize the findings, and the proposed mechanism does not always outperform a typical rule-based approach, less effort, scalability and flexibility of GAAM are remarkable. Regarding the scale of existing software applications, the dynamic environments in which they function, as well as variable system requirements, software operation management is often costly, Among the four adaptation processes, this article deals mostly with the deciding process. Of course, tracing goals for identifying their satisfaction or denial status is also related to the analyzing process. Among different forms of decision-making denoted by Roy [43] -selecting, sorting, ranking and description-the deciding process in self-adaptive software is closer to the selecting format. The deciding process basically addresses an action selection problem in order to select a proper action from a finite set of alternatives. A novel model, called Goal-Action-Attribute Model (GAAM), is proposed for tackling the action selection problem.. Goal-based approaches to adaptation are not widely addressed in many research efforts in the self-adaptive software area. The well-established goal models in Requirements Engineering (RE) are designed for development time. They aim at analyzing different design decisions for software engineers instead of focusing on run-time evaluation of goals for automated decision-making. GAAM is inspired by the available goal models in RE, but the emphasis is building a run-time model and preparing a decision-making mechanism to reason upon for adaptation. Therefore, although GAAM has some similarities with these goal models, it has at least two differences: first, monitored attributes activate adaptation goals in GAAM, while in design-time goal models, such as SIG in NFR framework [10] , operationalizations and design decisions change statuses of goals. Second, in design-time goal models, the objective is to analyze the impact of operationalizations on goals (i.e. specify their satisfaction levels), while in GAAM the objective is to select an action to change the current status.. Two points are notable about the evaluation of GAAM and the action selection problem. First, one of the shortcomings that software engineering for self-adaptive applications face is the lack of case studies. Some people use systems which are not open source or accessible by the others, and other cases are only very limited scenarios for a single quality aspect in toy applications. Although, we had some limited experiments on real applications, we decided to evaluate GAAM and its action selection mechanism with a simulated model. Second, many case studies on self-adaptive software do not deal with application-level adaptation. The actions in those works change something in middleware, server, network, or even operating system. Therefore, in this article adaptation actions are limited to application level. This does not mean that GAAM cannot include other actions. There is no assumption about the type of actions and we selected the application-level adaptation only because it has not been addressed extensively so far.. The rest of this article is organized as follows. Section 2 specifies the problem statement, and Section 3 elaborates GAAM and proposes a goal-based action selection mechanism for self-adaptive software. For evaluation, the proposed model is applied to an experimental model of a news web application in Section 4. Section 5 reviews the obtained results and the statistical analysis. Section 6 discusses notable related work. Section 7 draws some conclusions, and finally Section 8 provides several ideas for future enhancements. This article aims at providing a goal-based model and decision-making mechanism to address runtime action selection in self-adaptive software systems. For this purpose, this work deals with how to keep goals in the adaptation manager and involve them in decision-making at run-time. To trace goals, attributes are monitored continuously, and to change the adaptable software actions need to be selected by activated goals. The proposed goal-based model, GAAM, addresses the desired deciding process, discussed in the problem statement (Section 2), in the following ways:. • Explicit goals -Rule-based methods implicitly utilize the goals to adapt the system. In fact, developers indirectly consider the goals in defining and tuning the rule set. However, GAAM and the proposed goal-based action selection method explicitly represents the goals, trace them at run-time, and incorporate them in the deciding process. This characteristic leads to better traceability for administrators, and to more trust for stakeholders due to revealing why adaptation changes occur.. • Multiple objective decision-making -The goal-based approach discussed in this article intrinsically supports multiple goals. However, handling these goals in a rule-based method is not an easy task. This can result in a large set of rules with precisely adjusted priorities to address all the desired goals, while in the proposed approach each goal is defined with its own utility for the system's adaptation actions. These goals are later coordinated in the action selection mechanism.. • Flexibility -Modifying rules at run-time is not always safe, and the system's behavior is not predictable. In the goal-based action selection approach, this issue can be handled easier comparing with common rule-based mechanisms. Goals, actions, and attributes can be added or removed from GAAM, although we cannot make the claim that the system behavior is immune from any kind of change. The easiest way of changing GAAM is altering goal preferences. As noted before, these preferences can be generated with a dynamic algorithm.. The proposed solution for action selection problem has been validated by a set of experiments on a simulated model. The model has been built using a queue-server network for a news web application, but it can be used for other enterprise applications as well. Three research questions (RQ1 to RQ3, in Section 4), have been set to be investigated in the empirical evaluation. These questions address effectiveness of adaptation mechanisms (RQ1), comparing ordinal and cardinal utility for modeling preferences (RQ2), and contrasting goal-based and rule-based adaptation mechanisms (RQ3) in the conducted experiments.. Obtained results suggest that adaptation impacts positively on the evaluation function formulated based on the satisfaction levels of goals in the experiments. Thus, regarding the RQ1, adaptation improves goal satisfaction during the application operation. Both goal-based mechanisms, using ordinal and cardinal utility functions, perform well in the experiments. The interesting point is that the ordinal utility function in the conducted experiments generated acceptable results, with respect to the effort spent to define and tune the preferences. We cannot conclude that only by considering these results the ordinal utility can always be as well as cardinal utility function. But at least regarding the more effort required to define and tune cardinal preferences, ordinal preferences are easier to elicit and might lead to results as good as the cardinal case. Therefore, in response to RQ2, although using ordinal and cardinal utilities do not show a statistical difference, ordinal utility can be the initial choice for defining preferences. Thus, the answer to this question is more a design guideline not a utility function selection, due to investigating the question in a single case study. A similar argument can be applied to comparing the goal-based and the rule-based methods for RQ3. While they are not significantly different in terms of statistical measures, formulating and refining rules, as well as resolving their conflicts are not always straight-forward. Hence, the less effort for designing and tuning a goal-based deciding process is a plus point in selecting an appropriate approach. Of course representing explicit goals and a more clear multi-objective approach toward decision-making are notable for the goal-based approach. In future, several modifications can be applied to the layers of GAAM and the action selection mechanism. The goal layer of GAAM can use a more complex structure. Goal hierarchies and their relations (e.g. logical and/or) can be utilized in tracing goals, if higher goals in the hierarchy are connected to actions. In this case high level goals may be activated and be involved in the deciding process. This can lead to a change in the action selection mechanism. The activation function can generate a dissatisfaction level. This level indicates how much a goal's vote will affect the outcome of action selection mechanism. This might be presented as a kind of fuzzy inference as well.. Moreover, the relationships between actions in the action layer can be added to GAAM. For example, the precondition for some actions may be to first perform other actions. Therefore, if those secondary actions are desired by goals, there will be another source of stimulating actions. By considering these relationships, those secondary actions would be selected in the next adaptation rounds. This issue can be related to adding memory to GAAM, not only for goals, but also for actions. One idea is to define an energy level for goals and actions, as suggested by Maes [29] .. The focused problem in this article aims at selecting a single action that is primitive or composite. However, the problem can be generalized to plan a reactive course of actions or contingency plan. Automated AI planning can be utilized, but it is not often a simple problem to solve, especially in the presence of multiple goals. Another possible way to extend this work is to combine the goal-based and rule-based methods in order to use both the reactive and deliberative decision-making approaches. This idea is inspired from Gat's three layer architecture [13] and discussed in this context by Jeff Magee and Jeff Kramer [25] .. Another direction for extending this work is that this article only considers Borda count and intensity voting for selecting adaptation actions. However, other voting schemes from the literature can also be investigated; for example the schemes discussed in [31] . For cardinal preferences, Nash voting [38] can be used, and for the ordinal case, approval voting [4] is one of the options. Also, two-phase voting can also be implemented to select actions in the first phase with different voting schemes, and to aggregate the results with a secondary voting scheme, such as majority voting.. Although GAAM and the proposed action selection mechanism do not have any assumption about the adaptable software, in evaluating them mission-critical systems and particularly multitier enterprise applications have been targeted. The proposed model needs to be evaluated with other systems to identify the challenges. We have already started some experiments on web-based applications and service-based communication systems. The results were promising and the design and tuning of GAAM did not have a remarkable difference comparing with the reported experience in this article. Researchers in the self-adaptive software domain subscribe to the view that deciding is a vital process (among others pointed out by Oreizy et al. [39] ), and its realization is still a challenge, as noted by Salehie et al. [46] and McKinley et al. [35] . Existing solutions for self-adaptive software, and more broadly for autonomic systems, have still not addressed all of the requirements of a desired deciding process (discussed before). Maes enumerates several specific requirements in [30] . Finding good enough actions, minimizing back and forth switching between actions that contribute to distinct goals, and never getting stuck in a loop or deadlock situation to satisfy an unattainable goal, are some of these requirements. However, no remarkable set of criteria and metrics exist for verifying that a solution complies with these requirements. Gjorven et al. discusses applicability of some quality factors for adaptation [14] , but there are notable difficulties for evaluating most of the quality aspects of adaptation.. In decision-making approaches, it is common to build a structure to relate goals, attributes, and actions, such as discussed in multi-attribute utility theory [23] . In software engineering, goals are mostly defined and utilized in requirements specification, such as i* [56] and Soft-goal Interdependency Graph (SIG) [10] . SIG is an AND-OR graph which decomposes and relates goals using AND and OR relationships. However, i* models goals as a structure of connected nodes without explicitly representing a logical relationship between the parent and offsprings. Both models were used in representing non-functional goals. For example, Non-Functional Requirements framework (NFR) benefits from SIG [10] . SIG decomposes non-functional goals and design solutions (called operationalizations). Design decisions stimulate the model from operationalization nodes, and the reasoning propagates the impacts though goals. By reviewing the goals' satisfaction degree, designers evaluate their decisions.. The goal-driven solutions for self-adaptive software can be divided into two broad categories, which are not necessarily orthogonal, namely: development-time and run-time approaches. The idea of goaldriven models has been extensively used for decision-making in the development phase, especially for design decisions. In the self-adaptive software domain, Subramanian et al. adopted the NFR Framework to build a knowledge-based system for developing adaptable software architectures based on requirements specifications [55] . Brown et al. also employ the KAOS goal model to represent the adaptation semantics [6] . Among other works in this category, [27] , [37] and [16] are also noteworthy.. In the run-time category, based on the literature review, few research efforts utilize a goal-based model in the decision-making process at run-time. For example, Kramer et al. discuss an architecturebased adaptation approach, which utilizes a goal management layer [25] . This layer generates a reactive plan to satisfy the goals. Salehie et al. have also worked on an adaptation approach based on quality goals, in order to trace and satisfy these goals at run-time [48] . In another work, Salehie et al. employed a weighted voting scheme for goal-based decision-making [49] . A notable point about a goal-driven deciding process is that it may not be possible or efficient to use automated planning, in this case, reactive planning. For example, Srivastava et al. [53] argue that planning is an appropriate option for self-healing, but is generally not effective for self-optimizing. Therefore, having a goal-based decisionmaking mechanism does not necessarily mean that a planning-based approach is employed.. The focused problem in this article is also similar to the Action Selection Problem (ASP) in autonomous agents [29] and robotics, especially in mobile robots [42] . Similarities are due to dynamism and uncertainty in the environment, as well as online decision-making. However, the problem in software is generally more complex than in robotics. Software systems, especially largescale distributed applications, have much more attributes and effectors, and are often more complex than mobile robots. Some researchers believe that the complexity of software in comparison with most systems in mature engineering disciplines can be attributed to the nature of software systems: the notable difference is that their behaviors do not obey descriptive physical laws [8] .. The focus of this article is on the run-time category. Due to the similarity of the focused problem with the ASP in robotics and autonomous agents, several ideas from these disciplines have been adopted and employed. In particular, the idea of behavior-based robotics [1] seems promising for this purpose. The majority of existing architectures for the adaptation manager in self-adaptive software takes benefit from the Sensor-Plan-Act (SPA) model, that is used extensively in building traditional robotic systems. In these systems, events are collected, analyzed, and fused to update the domain model (i.e. world model). The system then plans its strategy in the new situation. However, the idea of behavior-based robotics is to use distributed specialized task-achieving modules, called behaviors, and to apply command fusion instead of sensor fusion [1] . In this way, there is no need to develop, maintain and extend a coherent monolithic model of the adaptable software and its context. Gat argues that this is one of the main problems of the Sense-Plan-Act schema used in most traditional robots [13] . The successful experiences in building behavior-based robots motivated us to apply the idea to self-adaptive software. Behaviors in robotics are mapped and extended to goals in this domain.. A goal-driven action selection mechanism can be realized in two general competitive and cooperative forms. In the former, the goals are competing with each other in order to select the next action, whereas in the latter, the preferences of goals are combined or fused to determine what to do next. Arkin discusses variant forms of cooperative and competitive mechanisms in behavior-based robotics [1]. In the cooperative category, Arkin names superpositioning (vector addition) as the most straightforward method, if this is feasible. In the competitive methods, arbitration is a way to select one goal (winnertakes-all), e.g., based on predefined priorities. The subsumption architecture basically employs this method [5] . A less autocratic method has been utilized by Maes in an activation network [29] . Maes argues that the lack of explicit goals and goal-handling capability in autonomous agents lead to significant limitations in their operation [29] . Notably, when an agent does not have goals, every situation has to carry complete information for deciding about the next most appropriate action. Maes proposes an activation network for actions, in order to facilitate dynamic action selection based on stimulated goals or actions.. The Distributed Architecture for Mobile Navigation (DAMN) uses a more democratic way by adopting a voting mechanism [42] . In fact, this method, and generally the voting-based mechanisms, can arguably be placed into the competitive category by Arkin. The social choice methods and voting games are well-known in cooperative game theory, in order to combine decisions made by agents [11] . DAMN, introduced by Rosenblatt, realizes the first level of behaviors in the subsumption architecture [42] . DAMN uses a voting mechanism for command fusion, regarding the safety behaviors for the turn and speed of the mobile robot. The beauty of DAMN's design is that the deliberative and reactive components of the architecture can operate at the same level, and it is scalable due to its lack of hierarchy.. A valid question is that what differences exist between GAAM and the Belief-Desire-Intention (BDI) model for intelligent agents [41] . Beliefs represent the system knowledge about the situation (self and context). Desires refer to the objectives and can be instantiated by goals, and intentions are what an agent decided to do and may include plans. The BDI model has some limitations that are discussed in [40] . The three elements of BDI would be roughly mapped to the three key entities of GAAM. The point is that GAAM does not suffer from some limitations of BDI; for instance in representing goals explicitly. Moreover, the relationships between entities in GAAM are nit similar to BDI. Nevertheless, BDI and its extensions in multi-agent systems are well-established and built on a strong body of knowledge, that could be useful in designing self-adaptive software systems.

O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O


Indulgent algorithms (preliminary version) Informally, an indulgent algorithm is a distributed algorithm that tolerates unreliable failure detection: the algorithm is indulgent towards its failure detector. This paper formally characterises such algorithms and states some of their interesting features. We show that indulgent algorithms are inherently safe and uniform. We also state impossibility results for indulgent solutions to divergent problems like consensus, and failure-sensitive problems like non-blocking atomic commit and terminating reliable broadcast. Indulgent algorithms. The notion of partial failures is a fundamental characteristic of a distributed system: some of the processes might fail whereas others might keep executing their algorithm. A usual metric to evaluate the reliability of a system is its ability to mask partial failures. Reliable distributed systems are designed to provide continuous service despite the failures of some of their processes. This ability typically relies on some failure detection mechanism that provides hints about which processes are correct and which are not. Distributed algorithms usually differ on the assumptions made about the reliability of that mechanism. Some algorithms assume failure detectors that accurately detect crashes. For example, the state machine replication algorithm of [16] , the election algorithm of [13] , and the nonblocking atomic commit algorithm of [17] , all assume that any correct process pi accurately detects when any other process pj has failed. Other algorithms do make weaker assumptions about failure detectors. For instance, none of the consensus algorithms of [2, 6, 10, 14] , or the replication algorithms of [11, 12, 5] , excludes the possibility of false failure detections. In a sense, those algorithms are indulgent (towards their failure detector). They can cope with false failure detections. As we pointed out in the introduction, characterising indulgent algorithms go through defining what it means for a failure detector to be unreliable. We defined three classes of unreliable failure detectors and our definitions differ significantly from the original failure detector class definitions, introduced by Chandra and Toueg in [2] .. In [2], the failure detector classes were defined according to desirable completeness and accuracy properties: completeness measures the extent to which a failure detector suspects the crash of faulty processes, while accuracy restricts the mistakes made about false failure detections. In contrast, our unreliable failure detector classes were defined according to undesirable properties that capture the intuition of unreliable failure detection. We do not claim to have exhaustively captured the notion of unreliable failure detection through our three failure detector classes. In fact, exploring the space of meaningful definitions for the notion of unreliable failure detector is, we believe, an interesting open issue.

O O O B O O O B O O O B O O O B B O B O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O B B O O O O O O O O O O O O B B O O O B I O O O O O O O O O O O O O O B I O O O O O O O


Robust exponential stability criteria of uncertain stochastic systems with time-varying delays This article investigates the problem of delay-dependent exponential stability in mean square for continuous-time linear stochastic systems with structured uncertainties and time-varying delays. By applying descriptor model transformation of the systems, a new type of Lyapunov-Krasovskii functional is constructed, and by introducing some free weighting matrices, some new delay-dependent and delay-independent stability criteria are derived respectively in terms of an LMI algorithm. The new stability criteria are less conservative than existing results. Numerical examples demonstrate that the new criteria are effective and are an improvement over existing results. During the last few decades, many authors have investigated the robust stability analysis of dynamic systems with delays. In general, delay-dependent stability criteria make use of the information on the size of delays, they are less conservative than delayindependent ones, especially for small delays. Therefore, much interest has been focused on the delay-dependent stability analysis of delay systems (see, e.g. Haussman 1978; Ichikawa 1982; Verriest and Florchinger 1995; Li and De Souza 1997; Mao, Koroleva, and Rodkina 1998; Moon, Park, Kwon, and Lee 2001; Yue and Won 2001; Lu and Tsai 2003; He, Wu, She, and Liu 2004; Chen, Guan, and Lu 2005; Liu and Zhang 2005; Lu, Su, and Tsai 2005) . However, most of these results have focused on the deterministic systems with delays (see, e.g. Li and De Souza 1997; Moon et al. 2001; He et al. 2004; Liu and Zhang 2005) , few authors (Haussman 1978; Ichikawa 1982; Verriest and Florchinger 1995; Mao et al. 1998; Yue and Won 2001; Lu and Tsai 2003; Lu et al. 2005 ) have discussed the delay-dependent stability for stochastic systems with delays. Verriest and Florchinger (1995) presented the stability of a linear stochastic differential equation via Riccati equations. Based on the LMI approach, Yue and Won (2001) , Lu and Tsai (2003) , Chen et al. (2005) and Lu et al. (2005) gave the delay-dependent robust stability criteria of uncertain stochastic systems. However, the criteria given in Yue and Won (2001) involved the parameterised model transformation to determine the stability of the systems. Lu and Tsai (2003) and Lu et al. (2005) used some inequality constraint. Chen et al. (2005) used a descriptor integral inequality constraint and the criteria given in Chen et al. (2005) and Lu et al. (2005) with matrix constraint P I, ( 4 0 is a scalar, P is the Lyapunov matrix). These results show considerable conservativeness.. This article presents some new delay-dependent stability criteria for continuous-time linear stochastic systems with structured uncertainties and time-varying delays. First, by extending the descriptor system, an approach was introduced in Fridman and Shaked (2002) for deterministic systems with delays to the stochastic systems case. We present the stochastic systems in the equivalent descriptor stochastic systems. Based on the equivalent descriptor form representation, we construct a new type of Lyapunov-Krasovskii functional. Second, we extend the idea proposed in He et al. (2004) to the stochastic systems case. In the derivative of the Lyapunov-Krasovskii functional, the terms q(t) and g(t) as state vectors are retained, and in order to exclude those constraint conditions in Yue and Won (2001) , Lu and Tsai (2003) , Chen et al. (2005) and Lu et al. (2005) , some free weighting matrices are introduced. In consequence, the stability criteria of systems do not concern any product terms of the Lyapunov-Krasovskii functional matrices with the systems matrices; moreover, the influences of the terms xðtÞ, xðt À hðtÞÞ, R t tÀhðtÞ qðsÞds and R t tÀhðtÞ gðsÞds in Newton-Leibniz formula are expressed in terms of free weighting matrices. Finally, based on an LMI algorithm, we obtain some less conservative delaydependent stability criteria for uncertain linear stochastic systems. The results include the delayindependent/rate-dependent and delay-dependent/ rate-independent stability criteria. Furthermore, the methods can be easily extended to the uncertain nonlinear stochastic systems with delays case and the stochastic systems with polytopic-type uncertainties and delays case. Numerical simulation examples show that the results are effective and are an improvement over existing criteria. For convenience, we adopt the following notations: Tr(A)(A T ) denotes the trace (transpose) of the matrix A and A ! 0(A 4 0) means that A is a positive semidefinite (positive definite) matrix. L 2 fi 0 ð½À, 0; R n Þ represents the family of R n -valued stochastic processes (s), À s 0, such that (s) is fi 0 -measurable for every second and R 0 À E ðsÞ 2 ds 5 1. E{Á} denotes the mathematical expectation operator with respect to the given probability measure P. This article presents some new stability criteria for a class of uncertain stochastic systems with time-varying delays. New techniques are developed to make the criteria less conservative. First, based on the equivalent descriptor form of the original system, a new type of Lyapunov-Krasovskii functional is constructed.

O O O O O O B I O B I O O O O O O O B I I I I O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Guided Local Search for the Three-Dimensional Bin-Packing Problem The three-dimensional bin packing problem is the problem of orthogonally packing a set of boxes into a minimum number of three-dimensional bins. In this paper we present a heuristic algorithm based on Guided Local Search (GLS). Starting with an upper bound on the number of bins obtained by a greedy heuristic, the presented algorithm iteratively decreases the number of bins, each time searching for a feasible packing of the boxes using GLS. The process terminates when a given time limit has been reached or the upper bound matches a precomputed lower bound. The algorithm can also be applied to two-dimensional bin packing problems by having a constant depth for all boxes and bins. Computational experiments are reported for two-and three-dimensional instances with up to 200 boxes, and the results are compared with those obtained by heuristics and exact methods from the literature. The three-dimensional bin packing problem asks for an orthogonal packing of a set of boxes into a minimum number of three-dimensional bins. The only restriction imposed on the solution is that the boxes have xed orientation. The problem has several industrial applications in cutting and loading contexts, and since rotation of the boxes is not allowed, the model can be used for solving several scheduling problems.. In this paper we present a new local search heuristic based on the Guided Local Search (GLS) method 20, 23] which has its origin in constraint satisfaction applications, but also has proven to be a very powerful metaheuristic for solving hard combinatorial problems. GLS uses memory to guide the search to promising regions of the solution space; this is done by augmenting the cost function with a penalty term that penalizes \bad" features of previously visited solutions.. Starting with an upper bound on the number of bins obtained by a greedy heuristic, the GLS algorithm iteratively tightens the upper bound by removing one bin from a feasible solution. That is, for a given number of bins, GLS constructs a feasible packing of the boxes. When such a packing has been found, the number of bins is decreased by one, and the procedure iterates until a given time limit has been reached or the upper bound matches the lower bound | in our case the L 2 bound from Martello The GLS algorithm assigns coordinates (including bin numbers) to the boxes. Translation of boxes along coordinate axes within one bin or moving boxes from one bin to another de nes the neighbourhood of the local search algorithm. The objective value of a given solution is the total volume of the pairwise overlap between boxes. Searching for a feasible solution is therefore equivalent to minimizing the objective function since an objective value of zero indicates a feasible packing.. To speed up the local search an implementation of Fast Local Search (FLS) has been used 20, 23] . Since the neighbourhood is fairly large in the current setting, FLS drastically speeds up the search for a local minimum. This is done by shadowing less promising parts of the neighbourhood, that is, temporarily xing box positions.. The GLS algorithm has been evaluated on instances with up to 200 boxes of varying types showing very promising results compared to previous approaches: Within a xed time limit, it generally nds better solutions than the exact algorithm by Martello, Pisinger and Vigo 16] , and this also applies when comparing to other heuristic algorithms as Lodi, Martello and Vigo 14, 15] . Furthermore, it is easily generalized to problems in which rotations are allowed and/or when the items to be packed have irregular shape.. The present paper is organized as follows: In Section 2 we de ne the considered problem more formally, and describe some lower bounds. Section 3 gives an overview of previous heuristics based on local search. The concept of Guided Local Search and our new algorithm is presented in Section 4. The paper is concluded with extensive empirical results in Section 5. Our experiments have shown that Guided Local Search can be applied to bin packing problems in two-and three-dimensions with success. Since the concept of GLS is still relatively new, it is important to determine the classes of problems for which it is suitable. However, a more important achievement is that the presented algorithm applies local search directly on the packing problems. Most other successful local search heuristics for cutting and packing somehow make use of a construction algorithm inside the search. Thus the local search is restricted to a higher level of the search, like assigning items to bins, or determining an appropriate packing order. But the construction algorithm may become a bottleneck for the search, since one will never nd better solutions than the construction algorithm is able to produce. Another bene t of working directly on the packing problem is, that we are able to handle items and bins of a general form as long as the overlap between items can be determined e ciently.. Dowsland 10] used a similar objective function without signi cant success. Using simulated annealing the investigated neighbourhood became too large, and thus the convergence towards good solutions was slow. It is thus interesting how GLS and FLS are able to focus the search on interesting parts of the solution space.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning

Latent

Tree

Graphical

Models

We

study

the

problem

of

learning

a

latent

tree

graphical

model

where

samples

are

available

only

from

a

subset

of

variables.

We

propose

two

consistent

and

computationally

efficient

algorithms

for

learning

minimal

latent

trees,

that

is,

trees

without

any

redundant

hidden

nodes.

Unlike

many

existing

methods,

the

observed

nodes

(or

variables)

are

not

constrained

to

be

leaf

nodes.

Our

first

algorithm,

recursive

grouping,

builds

the

latent

tree

recursively

by

identifying

sibling

groups

using

so-called

information

distances.

One

of

the

main

contributions

of

this

work

is

our

second

algorithm,

which

we

refer

to

as

CLGrouping.

CLGrouping

starts

with

a

pre-processing

procedure

in

which

a

tree

over

the

observed

variables

is

constructed.

This

global

step

groups

the

observed

nodes

that

are

likely

to

be

close

to

each

other

in

the

true

latent

tree,

thereby

guiding

subsequent

recursive

grouping

(or

equivalent

procedures)

on

much

smaller

subsets

of

variables.

This

results

in

more

accurate

and

efficient

learning

of

latent

trees.

We

also

present

regularized

versions

of

our

algorithms

that

learn

latent

tree

approximations

of

arbitrary

distributions.

We

compare

the

proposed

algorithms

to

other

methods

by

performing

extensive

numerical

experiments

on

various

latent

tree

graphical

models

such

as

hidden

Markov

models

and

star

graphs.

In

addition,

we

demonstrate

the

applicability

of

our

methods

on

real-world

datasets

by

modeling

the

dependency

structure

of

monthly

stock

returns

in

the

S&P

index

and

of

the

words

in

the

newsgroups

dataset.

The

inclusion

of

latent

variables

in

modeling

complex

phenomena

and

data

is

a

wellrecognized

and

a

valuable

construct

in

a

variety

of

applications,

including

bioinformatics

and

computer

vision,

and

the

investigation

of

machine-learning

methods

for

models

with

latent

variables

is

a

substantial

and

continuing

direction

of

research..

There

are

three

challenging

problems

in

learning

a

model

with

latent

variables:

learning

the

number

of

latent

variables;

inferring

the

structure

of

how

these

latent

variables

relate

to

each

other

and

to

the

observed

variables;

and

estimating

the

parameters

characterizing

those

relationships.

Issues

that

one

must

consider

in

developing

a

new

learning

algorithm

include

developing

tractable

methods;

incorporating

the

tradeoff

between

the

fidelity

to

the

given

data

and

generalizability;

deriving

theoretical

results

on

the

performance

of

such

algorithms;

and

studying

applications

that

provide

clear

motivation

and

contexts

for

the

models

so

learned..

One

class

of

models

that

has

received

considerable

attention

in

the

literature

is

the

class

of

latent

tree

models,

i.e.,

graphical

models

Markov

on

trees,

in

which

variables

at

some

nodes

represent

the

original

(observed)

variables

of

interest

while

others

represent

the

latent

variables.

The

appeal

of

such

models

for

computational

tractability

is

clear:

with

a

tree-structured

model

describing

the

statistical

relationships,

inference

-processing

noisy

observations

of

some

or

all

of

the

original

variables

to

compute

the

estimates

of

all

variables

-is

straightforward

and

scalable.

Although

the

class

of

tree-structured

models,

with

or

without

latent

variables,

is

a

constrained

one,

there

are

interesting

applications

that

provide

strong

motivation

for

the

work

presented

here.

In

particular,

a

very

active

avenue

of

research

in

computer

vision

is

the

use

of

context

-e.g.,

the

nature

of

a

scene

to

aid

the

reliable

recognition

of

objects

(and

at

the

same

time

to

allow

the

recognition

of

particular

objects

to

assist

in

recognizing

the

scene).

For

example,

if

one

knows

that

an

image

is

that

of

an

office,

then

one

might

expect

to

find

a

desk,

a

monitor

on

that

desk,

and

perhaps

a

computer

mouse.

Hence

if

one

builds

a

model

with

a

latent

variable

representing

that

context

("office")

and

uses

simple,

noisy

detectors

for

different

object

types,

one

would

expect

that

the

detection

of

a

desk

would

support

the

likelihood

that

one

is

looking

at

an

office

and

through

that

enhance

the

reliability

of

detecting

smaller

objects

(monitors,

keyboards,

mice,

etc.)

Work

along

these

lines,

including

by

some

of

the

authors

of

this

paper

(Parikh

and

Chen,

Choi

et

al.,

show

the

promise

of

using

tree-based

models

of

context.

This

paper

considers

the

problem

of

learning

tree-structured

latent

models.

If

all

variables

are

observed

in

the

tree

under

consideration,

then

the

well-known

algorithm

of

Chow

and

Liu

provides

a

tractable

algorithm

for

performing

maximum

likelihood

(ML)

estimation

of

the

tree

structure.

However,

ML

estimation

of

latent

tree

models

is

NP-hard

(Roch,

This

has

motivated

a

number

of

investigations

of

other

tractable

methods

for

learning

such

trees

as

well

as

theoretical

guarantees

on

performance.

Our

work

represents

a

contribution

to

this

area

of

investigation..

There

are

three

main

contributions

in

our

paper.

Firstly,

by

adopting

a

statistical

distance-based

framework,

we

develop

two

new

algorithms,

recursive

grouping

and

CLGrouping,

for

the

learning

of

latent

trees,

which

applies

equally

well

to

discrete

and

Gaussian

models.

Secondly,

we

provide

consistency

guarantees

(both

structural

and

parametric)

as

well

as

very

favorable

computational

and

sample

complexity

characterizations

for

both

of

our

algorithms.

Thirdly,

through

extensive

numerical

experiments

on

both

synthetic

and

real-world

data,

we

demonstrate

the

superiority

of

our

approach

for

a

wide

variety

of

models

ranging

from

ones

with

very

large

tree

diameters

(e.g.,

hidden

Markov

models

(HMMs))

to

star

models

and

complete

trees.

Our

first

algorithm,

which

we

refer

to

as

recursive

grouping,

constructs

a

latent

tree

in

a

bottom-up

fashion,

grouping

nodes

into

sibling

groups

that

share

the

same

parent

node,

recursively

at

each

level

of

the

resulting

hierarchy

(and

allowing

for

some

of

the

observed

variables

to

play

roles

at

arbitrary

levels

in

the

resulting

hierarchy).

Our

second

algorithm,

CLGrouping

first

implements

a

global

construction

step,

namely

producing

the

Chow-Liu

tree

for

the

observed

variables

without

any

hidden

nodes.

This

global

step

then

provides

guidance

for

groups

of

observed

nodes

that

are

likely

to

be

topologically

close

to

each

other

in

the

latent

tree,

thereby

guiding

subsequent

recursive

grouping

or

neighborjoining

(Saitou

and

Nei,

computations.

Each

of

these

algorithms

is

consistent

and

has

excellent

sample

and

computational

complexity.

As

Pearl

points

out,

the

identification

of

latent

tree

models

has

some

built-in

ambiguity,

as

there

is

an

entire

equivalence

class

of

models

in

the

sense

that

when

all

latent

variables

are

marginalized

out,

each

model

in

this

class

yields

the

same

joint

distribution

over

the

observed

variables.

For

example,

we

can

take

any

such

latent

model

and

add

another

hidden

variable

as

a

leaf

node

connected

to

only

one

other

(hidden

or

observed)

node.

Hence,

much

as

one

finds

in

fields

such

as

state

space

dynamic

systems

(e.g.,

Luenberger

Section

there

is

a

notion

of

minimality

that

is

required

here,

and

our

results

are

stated

in

terms

of

consistent

learning

of

such

minimal

latent

models.

In

this

paper,

we

proposed

algorithms

to

learn

a

latent

tree

model

from

the

information

distances

of

observed

variables.

Our

first

algorithm,

recursive

grouping,

identifies

sibling

and

parent-child

relationships

and

introduces

hidden

nodes

recursively.

Our

second

algorithm,

CLGrouping,

first

learns

the

Chow-Liu

tree

among

observed

variables

and

then

applies

latent-tree-learning

subroutines

such

as

recursive

grouping

or

neighbor

joining

locally

to

each

internal

node

in

the

Chow-Liu

tree

and

its

neighbors.

These

algorithms

are

structurally

consistent

(and

risk

consistent

as

well

in

the

case

of

Gaussian

and

discrete

symmetric

distributions),

and

have

sample

complexity

logarithmic

in

the

number

of

observed

variables.

Using

simulations

with

synthetic

datasets,

we

showed

that

RG

performs

well

when

the

number

of

hidden

variables

is

small,

and

that

CLGrouping

performs

significantly

better

than

other

algorithms

when

there

are

many

hidden

variables

in

the

latent

tree.

Using

both

Gaussian

and

discrete

real-world

data,

we

compared

the

performances

of

our

algorithms

to

other

EM-based

approaches

and

the

neighbor-joining

method,

and

our

algorithm

show

superior

results

in

both

accuracy

(measured

by

KL-divergence

and

graph

distance)

and

computational

efficiency.

In

addition,

we

introduced

regularized

CLGrouping,

which

is

useful

in

learning

a

latent

tree

approximation

with

a

given

number

of

hidden

nodes.

The

MATLAB

implementation

of

our

algorithms

can

be

downloaded

from

the

project

webpage

Log-likelihood

which

is

a

contradiction.

If

i

is

not

a

leaf

node

in

T

p

then

there

exist

a

node

u

i,

j

such

that

(i,

u)

E

p

Let

k

u

if

u

V

otherwise,

let

k

be

an

observed

node

in

the

subtree

away

from

i

and

j

(see

Figure

(b)).

Then,.

which

is

again

a

contradiction.

Therefore,

(i,

j)

E

p

and

i

is

a

leaf

node.

The

relevant

literature

on

learning

latent

models

is

vast

and

in

this

section,

we

summarize

the

main

lines

of

research

in

this

area..

The

classical

latent

cluster

models

(LCM)

consider

multivariate

distributions

in

which

there

exists

only

one

latent

variable

and

each

state

of

the

variable

corresponds

to

a

cluster

in

the

data

(Lazarsfeld

and

Henry,

Hierarchical

latent

class

(HLC)

models

(Zhang

and

Kočka,

Zhang,

Chen

et

al.,

generalize

these

models

by

allowing

multiple

latent

variables.

HLC

allows

latent

variables

to

have

different

number

of

states,

but

assume

that

all

observed

nodes

are

at

the

leaves

of

the

tree.

Their

learning

algorithm

is

based

on

a

greedy

approach

of

making

one

local

move

at

a

time

(e.g.,

introducing

one

hidden

node,

or

replacing

an

edge),

which

is

computationally

expensive

and

does

not

have

consistency

guarantees.

Another

greedy

learning

algorithm

called

BIN

(Harmeling

and

Williams,

is

computationally

more

efficient,

but

enforces

that

each

internal

node

is

hidden

and

has

three

neighboring

nodes.

In

contrast,

we

fix

the

number

of

states

in

each

hidden

node,

but

allow

observed

nodes

to

be

internal

nodes.

Our

algorithms

are

guaranteed

to

recover

the

correct

structure

when

certain

(mild)

conditions

are

met..

Many

authors

also

propose

reconstructing

latent

trees

using

the

expectation

maximization

(EM)

algorithm

(Elidan

and

Friedman,

Kemp

and

Tenenbaum,

However,

as

with

all

other

EM-based

methods,

these

approaches

depend

on

the

initialization

and

suffer

from

the

possibility

of

being

trapped

in

local

optima

and

thus

no

consistency

guarantees

can

be

provided.

At

each

iteration,

a

large

number

of

candidate

structures

need

to

be

evaluated,

these

methods

assume

that

all

observed

nodes

are

the

leaves

of

the

tree

and

to

reduce

the

number

of

candidate

structures.

Algorithms

have

been

proposed

(Hsu

et

al.,

with

sample

complexity

guarantees

for

learning

HMMs

under

the

condition

that

the

joint

distribution

of

the

observed

variables

generated

by

distinct

hidden

states

are

distinct..

The

reconstruction

of

latent

trees

has

been

studied

extensively

by

the

phylogenetic

community

where

sequences

of

extant

species

are

available

and

the

unknown

phylogenetic

tree

is

to

be

inferred

from

these

sequences.

See

Durbin

et

al.

for

a

thorough

overview.

Efficient

algorithms

with

provable

performance

guarantees

are

available

(Erdős

et

al.,

Daskalakis

et

al.,

However,

the

works

in

this

area

mostly

assume

that

only

the

leaves

are

observed

and

each

internal

node

(which

is

hidden)

has

the

same

degree

except

for

the

root.

The

most

popular

algorithm

for

constructing

phylogenetic

trees

is

the

neighbor-joining

(NJ)

method

by

Saitou

and

Nei

Like

recursive

grouping,

the

input

to

the

algorithm

is

a

set

of

statistical

distances

between

observed

variables.

The

algorithm

proceeds

by

recursively

pairing

two

nodes

that

are

the

closest

neighbors

in

the

true

latent

tree

and

introducing

a

hidden

node

as

the

parent

of

the

two

nodes.

For

more

details

on

NJ,

the

reader

is

referred

to

Durbin

et

al.

Section

Another

popular

class

of

reconstruction

methods

used

in

the

phylogenetic

community

is

the

family

of

quartet-based

distance

methods

(Bandelth

and

Dress,

Erdős

et

al.,

Jiang

et

al.,

Quartet-based

methods

first

construct

a

set

of

quartets

for

all

subsets

of

four

observed

nodes.

Subsequently,

these

quartets

are

then

combined

to

form

a

latent

tree.

However,

when

we

only

have

access

to

the

samples

at

the

observed

nodes,

then

it

is

not

straightforward

to

construct

a

latent

tree

from

a

set

of

quartets

since

the

quartets

may

be

not

be

consistent.

In

fact,

it

is

known

that

the

problem

of

determining

a

latent

tree

that

agrees

with

the

maximum

number

of

quartets

is

NP-hard

(Steel,

but

many

heuristics

have

been

proposed

(Farris,

Sattath

and

Tversky,

Also,

when

only

samples

are

available,

quartet-based

methods

are

usually

much

less

accurate

than

NJ

(St.

John

et

al.,

so

we

only

compare

our

proposed

algorithms

to

NJ.

For

further

comparisons

between

(the

sample

complexity

and

other

aspects

of)

quartet

methods

and

NJ,

the

reader

is

referred

to

Csűrös

and

St.

John

et

al.

Another

distance-based

algorithm

was

proposed

in

Pearl

Section

This

algorithm

is

very

similar

in

spirit

to

quartet-based

methods

but

instead

of

finding

quartets

for

all

subsets

of

four

observed

nodes,

it

finds

just

enough

quartets

to

determine

the

location

of

each

observed

node

in

the

tree.

Although

the

algorithm

is

consistent,

it

performs

poorly

when

only

the

samples

of

observed

nodes

are

available

(Pearl,

Section

The

learning

of

phylogenetic

trees

is

related

to

the

emerging

field

of

network

tomography

(Castro

et

al.,

in

which

one

seeks

to

learn

characteristics

(such

as

structure)

from

data

which

are

only

available

at

the

end

points

(e.g.,

sources

and

sinks)

of

the

network.

However,

again

observations

are

only

available

at

the

leaf

nodes

and

usually

the

objective

is

to

estimate

the

delay

distributions

corresponding

to

nodes

linked

by

an

edge

(Tsang

et

al.,

Bhamidi

et

al.,

The

modeling

of

the

delay

distributions

is

different

from

the

learning

of

latent

tree

graphical

models

discussed

in

this

paper..

A

quartet

is

simply

an

unrooted

binary

tree

on

a

set

of

four

observed

nodes.

The

term

consistent

here

is

not

the

same

as

the

estimation-theoretic

one.

Here,

we

say

that

a

set

of

quartets

is

consistent

if

there

exists

a

latent

tree

such

that

all

quartets

agree

with

the

tree.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O