Departure from linear mechanical behaviour of a helical spring We model small deviations from linear mechanical behaviour of a loaded vertically suspended helical spring. A Taylor expansion of the elasticity equations governing the axial extension of the spring is used to determine the relative magnitudes of linear and (quadratic and cubic) nonlinear terms in the force-extension relationship. This relationship is the basis for the derivation of a model for the static extension of a loaded spring, and a wave equation that models small amplitude oscillation. The models account for the natural decline in pitch angle down a suspended spring, and provide accurate fits to measurements of static extension and periods of oscillation that are not adequately represented by equations based on Hooke's law. The static and dynamic data yield consistent estimates of the spring rate. We consider models of the static extension and extensional oscillation of a vertically suspended, loaded, helical spring. Our initial interest is to fit experimental data (static extension and period of oscillation) obtained for steel, wire laboratory springs which typically have a spring index greater than 10 and spring constants in the range 1-10 N m −1 . Brass disks, which we refer to as the ''bob'', provide the load. We observe that damping of the motion during oscillation leads to a reduction in amplitude of about 0.25% per oscillation, with the result that any increase in the period of oscillation due to this weak damping is an order of magnitude less than the uncertainty in the measurement of the period. As a consequence we may confine ourselves to models for undamped oscillation.. A system such as this exhibits almost ideal mechanical behaviour, and is commonly used in introductory texts on mechanics to illustrate Hooke's law and simple harmonic motion, and in undergraduate calculus texts as an application of linear second order differential equations. A point-mass equation for the motion of the bob is adequate for the purpose of introducing readers to the key qualitative characteristics of the motion. This elementary model is based on the assumptions that the force-extension relationship for the spring is linear, and that the kinetic energy associated with motion of the spring is negligible relative to that of the bob.. A first approximation, if the kinetic energy of the spring is to be properly accounted for, is to treat the spring as an elastic fibre [1] , and the motion as a longitudinal wave in a fibre with one free end at which an additional mass is attached. Over the past four decades considerable discussion has appeared in the physics education literature of more detailed aspects of the system represented by the resulting one-dimensional wave equation for a Hookean spring. Cushing [2] gives a thorough account of the essential features of this literature. Analyses of a similar nature have appeared in the mechanics [3] and the mathematics literature [4] .. Sayre and De Forest [5] applied perturbation and series approximation methods to the elasticity equations for a uniform helical spring to model deviation from linear static extension with an expression that effectively corresponds to quadratic and cubic terms in an extension-load model.. More sophisticated models of mechanical response and wave propagation in springs, from an engineering and mechanical design perspective, is comprehensively surveyed by Pearson [6] , and more recently Lee [7] . These analyses assume uniform helical geometry. Lin and Pisano [8] develop more generalized models for helical springs with variable pitch angle and radius.. In a recent paper [9] we assumed a quadratic force-extension relationship to develop extension and oscillation models. This earlier work has shortcomings that we have overcome (as pointed out in Section 4).. In the current paper we establish a suitable approximation for the force-extension relationship for a spring using a Taylor series expansion of the elasticity equations that govern the geometry of a helical spring under the action of an axial applied force. Sufficient accuracy is obtained by truncating the series at the cubic term yielding a force-extension relationship of the form. A crucial point about our model is that the higher order spring constants (k 2 and k 3 ) are not additional independent parameters. They have values determined explicitly by k 1 , the Poisson's ratio for the wire, and the geometric constants of the helix. Perhaps of some surprise to many, the quadratic term dominates the departure from (linear) elastic behaviour, although the smaller cubic term is not insignificant.. We obtain a series approximation for the inverse of this cubic force-extension relationship to express the static extension as a function of the bob's mass. A wave equation for the oscillating system is derived from the second law of motion for elements along the spring, and we obtain an eigenvalue equation which yields the frequencies of oscillation as a perturbation correction to the equation that applies to a Hooke's law spring. The measurements are over a range of bob masses that result in extension of the spring to just over twice the natural length. The natural decrease in pitch angle down a suspended spring, due to the decrease in the spring's mass supported at each point down the spring, is accounted for by using the mass per unit length of the spring as the fundamental length variable.. The reason for the nearly linear force-extension relationship in wire springs is that the deformation in the wire, as a spring stretches, is largely torsional, and, since laboratory springs have several hundred turns, a twist of only a few degrees per turn results in extension of the spring to several times the natural length. We may assume therefore, for the purpose of modelling the system, that the material of the spring exhibits a strictly elastic behaviour.. Thus we consider an experiment within the range often described as being ''within the elastic limits'' of the spring. Sufficiently precise measurement, using standard laboratory equipment, of static extension, or period of oscillation will often reveal a slight departure from linear mechanical behaviour. This nonlinear behaviour of the spring is a consequence of the geometry of the helix, rather than inelastic behaviour of the material itself, and is encapsulated in the elasticity equations.. We obtain the series expansion for the elasticity equations for the spring in Section 2.1, and equations governing static extension and oscillation are developed in Sections 2.2 and 2.3 respectively. The measurements are outlined in Section 3 in and Section 4 contains the results obtained by fitting the models to the data, and concluding remarks are made in Section 5. An Appendix contains additional detail associated with the solution of the wave equation.. Throughout the paper we use the term linear spring to refer to a (hypothetical) spring that obeys Hooke's law, and nonlinear spring to reflect non-Hookean behaviour. We have developed nonlinear expressions that model the mechanical behaviour of a vertically suspended loaded helical spring, operating within the elastic limits of the material. In the absence of applied forces the spring is assumed to form a uniform helix, but both the static extension model and the dynamic model incorporate the natural variation in pitch angle of the spring when it is suspended.. The slight departure from linear mechanical behaviour of the spring arises from the geometry of the helix. Evaluation of terms in a Taylor expansion of the elasticity equations reveal that the departure from elastic behaviour is largely quadratic in nature, and not cubic as is often anticipated, and furthermore that the departure from Hooke's law is due mainly to bending in the wire.. The models are successful in explaining the departure from linear behaviour observed in the static extension and the periods of oscillation data for a helical, wire laboratory spring. Independent analyses of the static and dynamic data yield consistent estimates of the spring rate.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On the Cauchy problem for a quantum kinetic equation linked to the Compton effect The Cauchy problem is studied for an homogeneous quantum kinetic equation describing the Compton effect. Since the collision kernel commonly used in physics is highly singular, numerical simulations are performed for related collision kernels to get a preliminary insight into the behavior of the solutions. Some of the numerical results are then given a theoretical explanation. Global existence of a solution to the Cauchy problem is proven when the L 1 initial data are a.e. smaller than the Planck distribution function, and non-existence of solutions to the Cauchy problem is proven when the L 1 initial data are a.e. bigger than the Planck distribution function. The Compton effect describes the change in wavelength of X-rays and gamma rays due to scattering by electrons. Its discovery by Arthur H. Compton in 1922 strengthened the case for quantum mechanics in an epoch when it was still much debated. Indeed it supported the notion that not only radiation, but matter as well, had both wave and corpuscular properties. From a modelization point of view, it represents one of the few known cases in quantum kinetic theory where the collision kernel can be derived precisely.. Among the physics papers in the area we mention the following pioneering ones. G. Cooper developed the Fokker-Planck equation for the Compton scattering in a plasma without recourse to a non-relativistic approximation [1] . Dreicer presented a simple kinetic theory which includes the interactions between electrons and photons and may thus describe relaxation phenomena [2] . Kompaneets studied the thermal equilibrium between quanta and electrons [3] . Zel'dovich and Levich studied the process of equilibrium in a system consisting of radiation and totally ionized plasma [4] .. The mathematical perspective was developed in a.o. the following articles. Entropy increase and comparison principles for the Fokker-Planck equation describing the radiation distribution in a homogeneous plasma were derived by Caflish and Levermore in [5] . There the entropy function was used to find the equilibrium distributions for the scattering alone and for scattering with emission and absorption. In [6] solutions to the Fokker-Planck equation developing singularities in finite time, independently of the total initial number of photons, were analyzed. Escobedo and Mischler obtained existence results for a quantum kinetic equation with a simplified regular and bounded kernel [7] . They studied the asymptotic behavior of the solutions, and showed that the photon distribution function may condensate at energy zero asymptotically in time. Numerical methods were developed in [8] for the same quantum kinetic equation, as well as for the Kompaneets equation. Chane-Yook and Nouri kept the physical kernel in the same quantum kinetic equation and derived a local existence in time theorem for the Cauchy problem in the case of small initial data [9] . In the context of parabolic problems, Brandle, Groisman and Rossi proposed numerical schemes for the approximation of solutions with possible blow-ups [10] . Adaptive methods were performed in [11] in order to reproduce the asymptotic behaviour of the solutions.. In this paper, global existence in time for the Cauchy problem is considered for the model with physical kernel. Section 2 recalls the model and some of its properties. Section 3 describes earlier results for the model. Section 4 presents numerical simulations for various collision kernels, from the constant one considered in [7] to the physical one. This is done with a scheme that preserves the main physical properties of the model. Different initial data are considered, essentially distinguishing the case where the initial datum corresponds to a total number of photons bigger than the Planck distribution one. Section 5 gives a theoretical explanation of some numerical simulations, in the particular cases where the initial datum is an L 1 function almost everywhere smaller (resp. bigger) than the Planck distribution function. Global existence of a solution to the Cauchy problem is derived when the initial datum is almost everywhere smaller than the Planck distribution function. For an initial datum a.e. bigger than the Planck distribution function, non-existence of any solution to the Cauchy problem is proven. This is a limitation of the possible initial data for the Cauchy problem. The quantum kinetic equation linked to the Compton effect studied in this paper presents a strong singularity at energy zero. Numerical simulations, with more and more singular kernels up to the physical one, point out the real singular part of the collision kernel. Moreover, they show that the problem behaves quite differently depending on the value of the initial total number of photons M(F i ) compared to the Planck one M 0 . In particular, blow-up immediately occurs with the physical kernel if M(F i ) > M 0 . Theoretical results stress these features for L 1 initial data that are almost everywhere smaller (resp. bigger) than the Planck distribution function.

O O O O O O B I I O O O B I O O O O O O O O B I I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On-line anomaly detection and resilience in classifier ensembles Detection of anomalies is a broad field of study, which is applied in different areas such as data monitoring, navigation, and pattern recognition. In this paper we propose two measures to detect anomalous behaviors in an ensemble of classifiers by monitoring their decisions; one based on Mahalanobis distance and another based on information theory. These approaches are useful when an ensemble of classifiers is used and a decision is made by ordinary classifier fusion methods, while each classifier is devoted to monitor part of the environment. Upon detection of anomalous classifiers we propose a strategy that attempts to minimize adverse effects of faulty classifiers by excluding them from the ensemble. We applied this method to an artificial dataset and sensor-based human activity datasets, with different sensor configurations and two types of noise (additive and rotational on inertial sensors). We compared our method with two other well-known approaches, generalized likelihood ratio (GLR) and One-Class Support Vector Machine (OCSVM), which detect anomalies at data/feature level.We found that our method is comparable with GLR and OCSVM. The advantages of our method compared to them is that it avoids monitoring raw data or features and only takes into account the decisions that are made by their classifiers, therefore it is independent of sensor modality and nature of anomaly. On the other hand, we found that OCSVM is very sensitive to the chosen parameters and furthermore in different types of anomalies it may react differently. In this paper we discuss the application domains which benefit from our method. The field of activity recognition has gained an increasing level of interest driven by applications in health monitoring and assistance, manufacturing and entertainment. In addition, given the advances in portable sensing technologies and wireless communication many of these systems rely on the fusion of the information from (heterogeneous) sensor networks. In this framework, the detection of anomalous (e.g. faulty or misbehaving) nodes constitute an important aspect for the design of robust, resilient systems. Besides activity recognition, anomaly detection (AD) is an important issue in the fields of control systems, navigation, and time series analysis. Generally, an anomalous pattern is one that is not desired or not expected. Therefore it often decreases the system performance or generates abnormal behavior in the data stream. It can be due to sensor failure, signal degradation, environmental fluctuations, etc. In most fields, such as health monitoring, a desirable characteristic is to process the data stream online and get a realtime anomaly detection to take an appropriate counteraction.. As mentioned above, there is a tendency toward the use of large number of sensors and with different sensor modalities to have more information about the observed environment. In a pattern recognition system, there are different levels to fuse sensor information; data, feature, or classifier. Generally, the goal of data fusion is to achieve more reliable data. Feature fusion concatenates different features from all the sensors before classification (Fu et al., 2008) . Classifier fusion is applicable when an ensemble of classifiers is used and each classifier is assigned to different subset of sensors, and finally, the decision is made by a combination of the classifier decisions (Ruta and Gabrys, 2000) . This architecture allows for a decentralized classification system where each classifier decides about a separate data stream. Therefore, if a particular stream is faulty or misbehaving we can remove the corresponding channel easily from the fusion in order to continue the classification, keeping the system performance as high as possible. A good example of such systems is human activity recognition (HAR) with on-body sensors mounted on specific limbs. A good practice is to devote a distinct classifier to each of the sensors . In these systems, one inevitable anomaly is the rotation or sliding of the sensors. If these anomalies can be detected, it is easier to take an appropriate counter-action for the misbehaving sensor, without the need of reconfiguring the whole system (Sagha et al., 2011a; Chavarriaga et al., 2011) . This desirable characteristic is the core feature of Opportunistic activity recognition systems where elements in the network may appear, disappear or change during life time .. Given a classifier ensemble, the anomaly detection process can be applied at different levels; raw signal or feature level, or at classifier/fusion level. Hereafter, we call both raw and feature level as low level and the detection process as Low Level Anomaly Detection (LoLAD), while for the later case we name the process as Fusion Level Anomaly Detection (FuLAD). The former case is the most commonly applied (Chandola et al., 2009) , however, it is often not applicable as different sensor modalities may be available -requiring to design specific AD for each modality -as well as the energy and computational cost in the case of wireless sensor networks. Therefore in this paper we introduce an anomaly detection mechanism at the level of classifier fusion, based on the consistency of the classifier decisions. Applying the method on two human activity datasets, we show that, upon detection of anomalous classifiers, an adaptation strategy such as removing them from the fusion chain, leads to a graceful performance degradation.. The structure of the paper is as follows; the next section summarizes related work on anomaly detection and resilience, then we describe the proposed method in Section 3. The description of the experiments and the results are presented in Sections 4 and 5, respectively. Finally, we discuss about the use of the method and its pros and cons in Section 6, followed by a conclusion. We propose a method for detecting anomalies in classifier ensembles. The detection process take place at the classifier fusion level (FuLAD), in contrast to the most common approach of doing it at the data/feature level (LoLAD). The proposed approach is independent of the sensor modality and the type of noise or anomalies it has to deal with. In this work, we thoroughly tested the method on datasets from activity recognition with on-body sensors, but it can be applied in any other application. Although there are many alternative approaches that consider fault and change detection at raw data or feature level, this work is the first one which explores them at fusion level.. We propose two different approaches: one is based on the Mahalanobis distance which compares the individual classifier decisions and the fused output, while the second approach is based on the mutual information between classifiers in the ensemble. They spot the faulty classifiers based on the behavior of the classifiers from the training set. Upon detection of anomalous classifiers, compensatory strategies (such as removing them from the ensemble) can be applied to improve the performance. Besides testing on synthetic data, we applied our method to two activity recognition datasets, allowing us to test it in realistic conditions using different sensor modalities and configurations. The results show the feasibility of change and anomaly detection at the fusion level and the improvement of the classification for all the cases. Although the proposed method does not build explicit models for each sensor type or modality in the ensemble, its performance is comparable to that achieved with LoLAD. The IT measure can detect anomalies just as well as the feature level method, even when only two classifiers remain healthy. Whereas the DB measure works satisfactorily until half of the classifiers are affected.. Alternatively, the distance measures presented here can be used to quantify the confidence of each classifier in the ensemble at runtime. This measure can then be used to dynamically reconfigure the ensemble, taking into account different specifications including classification performance, as well as communication costs or energy consumption (Kurz et al., 2011b) . Nowadays, multimodal sensor networks are increasingly used to gather information about the environment and improve context and activity recognition. Classifier ensembles can provide an efficient and flexible approach to achieve this. However, sensors are always prone to noise and anomalies that may degrade the system performance. Therefore, the ability of detect such anomalies may allow to dynamically adapt the system, bringing more robustness to the recognition chain.. Anomaly detection can be performed at the data/feature or fusion level. This choice depends on several factors, including the application requirements and the available technology. Complementing existing methods for low level anomaly detection, in this paper we assess the possibility of detecting anomalies at the fusion level with a comparable performance. The proposed method relies on a distance measure between the expected system behavior (based on training data), and the data collected at run time. We provide two metrics, one based on the Mahalanobis distance and the other one based on information theoretical measures. We characterize the proposed methods using both an artificially generated dataset and real data from two scenarios of human activity recognition using wearable sensors. The final results show the possibility of recognizing the anomalous behavior at the fusion level, resulting in graceful degradation of the classification performance.. Regarding the proposed measures, the IT approach has an advantage over the DB, as it does not take the fused decision into account. In that sense, it could be more suitable for dynamic classifier ensembles -where sensors (and their corresponding trained classifiers (Kurz et al., 2011a; Calatroni et al., 2011) ) -may be added or removed from the network on runtime. Having said so, in this approach the length of the window should be carefully chosen in order to have a reasonable estimate of the mutual information.. An interesting characteristic of FuLAD is that nodes in the ensemble will be removed only if the corresponding classifier is affected by the anomaly. For example, if an accelerometer is slightly rotated, the classifier decisions may not change, therefore it may still be included in the fusion. This property is not provided by LoLAD methods since do not take into account information about classification.. The conventional approaches of LoLAD create a model of data and detect anomalies when the new data does not follow the model. Even when the best model is chosen or the parameters are perfectly estimated, it may not cover all the possible anomalies. In contrast, FuLAD is independent of the type of anomaly. There is no need to investigate all the possible anomalies of each sensor. Only monitoring the classifier decisions could bring information if the sensor is affected or not. Moreover, in the case of multimodal setup, LoLAD requires the development of methods and parameters for each modality. In contrast, FuLAD is independent of the modality of the sensors used, because the working space is composed of the classifier outputs, independent of their input domain.. Another characteristic of FuLAD is that it puts the burden of the computation of anomaly detection on a central node (fusion node). This has consequences in the system design with respect to the amount of computational power at each node and the communication requirements (i.e., need to transfer classification labels or raw data). Noteworthy, this property, inherited from the classifier ensemble approach, allows for more flexibility in the case of dynamically changing, self-adaptive recognition systems ). We can handle anomalies in two manners: through detection and isolation, or by anomaly resilience and adaptation. In the former case, the goal is to detect whether there is an anomaly in the data or not (detection) and which part of the system is affected (isolation). While for the latter one, the system is designed to be tolerant against anomalies, or to be able to take suitable counteraction whenever an anomaly is detected. For example, by changing the network structure in wireless sensor networks (WSN) or adapting parameters to the new data trend. We propose a method to automatically detect anomalies in a classifier ensemble at the fusion level, as shown in Fig. 1 . The classifier ensemble architecture is more convenient in the sensor networks deployed for pattern recognition in the sense that different modalities could be covered with different classifiers and also the ensemble could be easily reconfigured to attain different criteria such as cost and accuracy (Chavarriaga et al., 2011) .. The rationale behind the method is to find a model of classifier coherence from the training set. Then, at run time, looking at a sequence of classifiers' outputs, we determine how much each classifier deviates from this model. Whenever this deviation exceeds a defined threshold the classifier is considered as anomalous. Finally, to make the system resilient, we remove the anomalous classifiers and reconfigure the ensemble.. To compute the deviation, we present two measures. One is based on the Mahalanobis distance, Section 3.1, and the other is based on an information theoretic approach, Section 3.2. Later, we will discuss the pros and cons of each approach. The first was originally proposed by Sagha et al. (2011b) and here we provide further characterization of it. Moreover, in order to compare the performance of the fusion level and feature level methods, in Sections 3.3 and 3.4 we describe two commonly used anomaly detection methods: GLR and OCSVM. The performance of the proposed methods has been evaluated on one artificially generated dataset and two real datasets of human activities. The activity datasets contain several sensor modalities, and different ensemble configurations were tested based on the sensor modalities or location.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Memory Efficient Software Synthesis with Mixed Coding Style from Dataflow Graphs Abstract-This paper presents a set of techniques to reduce the code and data sizes for software synthesis from graphical digital signal-processing programs based on the synchronous dataflow model. By sharing the kernel code among multiple instances of a block with a shared function, we can further reduce the code size below the previous results based on inline coding style. A systematic approach also is devised to give up the single appearance schedule for reducing the data buffer requirement. The proposed techniques have been evaluated with two real-life examples to prove their significance. Minimizing the memory requirement is very important to synthesize code for embedded systems, specially for an on-chip design. Critical constraints on the memory size have made assembly programming by hand still a popular way of software development for embedded systems in spite of low productivity. Growing complexity of embedded systems, fast design turnaround time, limited development budget, and short lifecycle of products, however, will make the use of high-level software design methodology mandatory: high-level language compiler or automatic code generation from block diagram specification. In this paper, we aim to reduce the code and data sizes for software synthesis from graphical digital signal-processing (DSP) programs based on the synchronous dataflow (SDF) [1] model, which can be used as a block diagram specification model. In an SDF graph, each node contains a kernel (code fragment) of a host language tailored to an implementation engine while the dataflow graph itself is a coordination language among function modules. Fig. 1(a) is an example SDF graph, and each arc is annotated with the number of tokens produced or consumed by an activation of its source or destination node. Each arc is assigned to a data buffer whose size can be constrained to the maximum number of tokens accumulated during a chosen execution of the graph. These data buffers compose the state of the SDF graph. Numerous DSP design environments, including a number of commercial tools, support SDF or closely related models ( [2] - [4] ) for both simulation and code generation.. A key property of the SDF model is that static schedules can be constructed at compile time, thus removing the run-time overhead of dynamic scheduling. Fig. 1(b) shows three valid schedules. Among valid schedules for an SDF graph, if every block appears exactly once in the schedule such as 61 and 62 in Fig. 1 , the schedule is called a single appearance(SA) schedule. An SA-schedule that has no nested loop is called a flat SA-schedule. 61 of Fig. 1 is a flat SA-schedule, while 62 is not.. Software synthesis from an SDF graph includes determining a feasible schedule and a coding style for each dataflow node, both of Manuscript received August 31, 1999. This work was supported by the academic fund of Ministry of Education, Republic of Korea, through the InterUniversity Semiconductor Research Center, Seoul National University, under ISRC-98-E-2103.. W. Sung is with the CAP Laboratory in Seoul National University, Korea (e-mail: yong@iris.snu.ac.kr).. S. Ha is with the Department of Computer Engineering, Seoul National University, Korea (e-mail: sha@iris.snu.ac.kr).. Publisher Item Identifier S 1063-8210(00)09510-X. which affect the memory requirements of the generated software for code and data. One of the main scheduling objectives for software synthesis is to minimize the total (sum of code and data) memory requirements. Once the schedule is determined, codes are generated according to the scheduled sequence. Since nodes are prepared in libraries, the kernel inside a node is assumed already optimized and treated as a unit. Two popular coding styles are inlining and function. The former generates an inline code for each node at the scheduled position, while the latter calls a function that contains the kernel. After a schedule is performed, the code for each dataflow node is generated depending on the coding style of the node. Fig. 1 (c) shows three programs based on the same schedule 61 with various coding styles: inline, switch, and function. Among multiple valid schedules for an SDF graph, there exist tradeoffs between code memory size and data buffer memory requirement, depending on the coding style. Since a single appearance schedule guarantees the minimum code size for inline code generation, a group of researchers focused on finding a single appearance schedule that minimizes the buffer memory requirement [5] . Bhattacharyya et al.developed an algorithm to find out a looped SA-schedule with minimum buffer requirement among multiple looped SA-schedules, ignoring buffer sharing possibilities. Ritz et al. used an integer linear programming formulation to minimize the data memory [4] . Both works ( [5] , [4] ) stick to single appearance schedules and do not exploit code-sharing optimization, in which multiple nodes share the same kernel in the generated code. There is a recent work on mixing coding styles for memory-efficient code synthesis [6] . However, they also did not consider the possibility of code sharing. Most previous approaches first assume a coding style for all dataflow nodes and search for an optimal schedule afterwards. Also, they try to minimize the code size first and the data size later. Even though they produce good results for a set of applications, they could not produce good codes for some applications that we will demonstrate. In this paper, we propose a pair of optimization techniques to overcome their limitations by mixing the coding style of different dataflow nodes. The first technique is to reduce the code size by sharing the kernel among multiple instances of the same block; which requires function style code generation instead of inlining. The second technique is to give up single appearance schedule, an important schedule class for the minimum code size, for data memory minimization. By applying 1063-8210/00$10.00 © 2000 IEEE these two techniques, we could reduce the memory requirements of two important examples by 13.6% and 8.1% over the best results from SA-schedule [5] at little expense of increased run-time overhead.. In the context of the design of complex integrated circuits, the current challenge is the design-error free generation of large systems from behavioral specifications, allowing one to reuse previously designed parts. High-level synthesis (HLS) tools are now available, but their constant evolution and the concurrent change of the design libraries are so rapid that neither the programs nor the libraries can be considered provably correct. As a consequence, the results of HLS must undergo extensive verification before being fed to the logic design step [1] . Yet, due to the complexity of the circuits, and to their abstract specification, their exhaustive simulation is out of reach, and the current technology of automatic formal verification tools is no longer applicable.. Let us briefly recall the principles of today's verification tools. At the bit and word level, circuits have a fixed word length, fixed-width datapath, and a finite and known number of memory elements. Binary decision diagrams [2] and their enhancements [3] , [4] efficiently represent the set of states reachable from the initial state by repetitively applying the "next state" transition relation. The functional correctness of the circuit needs only be established on the reachable states. "Model checkers" compute the truth of properties, expressed as temporal logic formulas, on the reachable states of a design description. These tools in reality perform either an explicit or a symbolic enumeration of the reachable states. As a consequence, their applicability is limited to circuits with fixed structure, and with a number of states that in practice cannot exceed 2 100 (i.e., 100 bits of memory). These tools can no longer be applied automatically on designs with datapaths and on abstract specifications: the designer must proceed to manual abstractions (such as datapath width reduction) [5] and verification decomposition [6] , the validity of which are not supported by automatic software. Other approaches, also requiring expert manual guidance, gave interesting results combining symbolic simulation and theorem proving [7] , [8] , but their wide use is hindered by the lack of automatic translation from conventional design languages to the input format of theorem provers. In this paper, we have presented a pair of optimization techniques to jointly minimize the code and data memory requirement. Before applying the proposed optimization techniques, we carefully analyze the gains and overheads. Selective application of the optimization techniques shows significant improvements in memory requirement for both code and data in an important class of applications.. Beyond what we achieved in this paper, there are more works to be studied in the future. At first our techniques should be extended to deal with the case where no SA-schedule exists. Considering the possibility of buffer sharing is another topic. Moreover, it may be better to devise a new scheduling policy to minimize the code and data memory considering the proposed optimization possiblities.

O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Agent-based trust model involving multiple qualities A key limitation of current Web services standards is the inability to differentiate service instances at runtime using (nonfunctional) qualities of services (QoS). Such differentiation is necessary to allow for runtime selection and binding to service instances in a manner that continually adapts selected services to the service consumer's preferences and needs. Quality values are volatile, depend on the trust associated with instances and service providers, and also depend on the needs of service consumers. We propose a multiagent framework where agents consider the consumers' QoS preferences, determine trust levels to associate to service instances and providers, and automatically select service instances on a consumer's behalf. The service agents use a trust model that is centered on a shared conceptualization for QoS (ontology) and a QoS preference model that considers consumer's tradeoffs among qualities as well as relationships between qualities. We evaluate our approach via simulations on simplified but realistic service instances and service consumers. Our results show that using these considerations for QoS, service agents are able to determine over time the 'best' service selection for a consumer. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Web services standards, while essential for automated access to services on the Web, are mostly geared towards design-time automation and usage. Limiting ourselves to design-time automation is not appropriate for software agents doing automatic runtime service selection and binding. Service descriptions via WSDL are necessary but not sufficient to automate service selection and binding-they principally lack a means to represent nonfunctional service attributes. To achieve true runtime integration we need a means to collect nonfunctional information about services and use that information to assign dynamic trust levels to the service providers and implementations.. As in real-life situations, the agents in our system need to be able to make decisions using partial information, limited computational resources, and bounded time. As such, an agent's decision process is to derive a trust value for each service instance. The service instance with the highest trust value is selected.. Our primary contribution is a trust model based on a shared conceptualization of qualities of service (QoS). Our trust model takes into account a consumer's subtle preferences for qualities when determining the trust value to assign to service instances. Specifically, we consider a consumer's preference for tradeoffs among qualities and relationship between qualities. We show the effectiveness of our approach using simulation experiments.. The remainder of this paper is organized as follows. Section 2 reviews the conceptual background underlying our approach, including assumptions and framework. Section 3 formulates the service selection problem using trust. Sections 4 and 5 give two solutions to this problem. Section 6 discusses our empirical evaluation and shows our results. Section 7 highlights some related work. Section 8 discuses some directions. Solutions to the dynamic service selection problem are crucial for realizing the potential of Web services. After a mathematical formulation of the problem, we proposed an initial solution built on a trust model of services that directly take into account the qualities advertised by service providers and the quality preferences of consumers. Considering more detailed consumer preferences for qualities, we presented an updated trust model where the preferences for qualities take into account the shape of the consumer's utility curve for a single quality, the consumer's preferences for tradeoffs among pairs of qualities, and the relationships between pairs of qualities.. An area of future work is to complement our global view of trust with one that is local to each agent. Essentially, each agent would keep track of its past interactions with service providers and service instances and would can use the resulting computed trust value to complement the global view.. A natural follow-on to this preference model is to consider the consumer's utility for multiple (greater than two) qualities, the tradeoffs between multiple qualities, and making explicit the relationships between multiple qualities. A promising approach is to apply multi-objective utility and decision theory, as studied by Keeney and Raiffa [5] . Finally, our current model for trust does not consider how truthful providers are in their QoS advertisements and does not include a model to capture transitive trust. These could be added to our model by including an honesty notion that compares a provider's advertisements to its reputation for qualities and by introducing endorsements between providers and a consumer's list of trusted providers.. Finally, an analytical model that complements our problem formulation and empirical evaluation would give insight into the system and also help derive mechanisms to give incentives discouraging service consumers from being untruthful and allow them to share. Service selection and binding approaches fall into two primary categories: design-time and runtime. In design-time selection and binding, the application designer or architect use service registries coupled with service descriptions to select and test binding to a service. Nonfunctional characteristics are considered during trial and error tests of the selected services. Newer techniques using richer semantic descriptions of services can help in the discovery of service interfaces. OWL-S [10] is an example of a rich service ontology used for semantic service discovery.. The other category, of most concern to our work, is runtime service selection and binding. In this case, the service interface is already discovered. At runtime, the service implementations are discovered, selected, and bound to-all based on QoS requirements, models and metrics, and middleware. Key models are those of the W3C [11] and UML [1] . Brokering and middleware approaches that use QoS include works by Ran [9] and by Wohlstadter et al. [12] .. Using QoS for service selection is also proposed by Ran [9] , Kalepu et al. [4] and Zeng et al. [14] ; however, while these work fail to address selection adequately for open environments because they do not consider trust, reputation, and a decentralized multiagent architecture as we are proposing. Wohlstadter et al. propose a policy language for advertising the QoS needs of clients and to allow the middleware to match servers to clients [12] . However, their work does not address a complete conceptualization of nonfunctional attributes for Web services. Further, Wohlstadter et al.'s matchmaking techniques lack support to enable dynamic evolution of the QoS exposed by the services and does not consider relationships that exist between qualities.. The literature on trust neither addresses QoS directly nor the applicability of trust for dynamic and autonomic service selection. Huynh et al. give a framework for trust determination in open systems, but do not use QoS conceptualizations to determine the needs of the truster [3] . Zacharia and Maes give a general model for reputations that we extend into dynamic service selection [13] .. Finally, our agents are decision-theoretic in design. Their decision-making algorithm, used to make selection decisions, applies the works by Keeney and Raiffa [5] in decision making with multiple objectives, extending it in terms of ontologies and computing with quality relationships. The trust assigned to a service instance is, in essence, a predictor of quality values for future usage of the service instance. It is an aggregate value that corresponds to historical levels of quality of the service instance and to how well the provider's quality advertisements match the quality needs of a consumer. Our approach to selection extends the framework and trust model described previously [6, 7] .. Since we are mainly concerned with agents that automate service selection decisions on behalf of their consumers, we make certain simplifying assumptions about the environment and the agents participating in it.

O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O


Formalisations and applications of BPMN We present two formalisations of the Business Process Modelling Notation (BPMN). In particular, we introduce a semantic model for BPMN in the process algebra CSP; we then study an augmentation of this model in which we introduce relative timing information, allowing one to specify timing constraints on concurrent activities. By exploiting CSP refinement, we are able to show some relationships between the timed and the untimed models. We then describe a novel empirical studies model, and the transformation to BPMN, allowing one to apply our formal semantics for analysing different kind of workflows. To provide a better facility for describing behaviour specification about a BPMN diagram, we also present a pattern-based approach using which a workflow designer could specify properties which could otherwise be difficult to express. Our approach is specifically designed to allow behavioural properties of BPMN diagrams to be mechanically verified via automatic model-checking as provided by the FDR tool. We use two examples to illustrate our approach. Modelling of business processes and workflows is an important area in software engineering. Business Process Modelling Notation (BPMN) allows developers to take a process-oriented approach to modelling of systems, but the notation specification [19] does not contain a precise semantics and this means BPMN diagrams are ambiguous and cannot be verified for behavioural correctness. In this paper we describe two formalisations of the notation and also how to apply the notation to workflows that are normally beyond its scope.. Specifically the following results are presented in this paper:. • An untimed process semantics for BPMN in the process algebra CSP, allowing formal reasoning via CSP's process refinements and verification via model checking.. • A relative timed semantics for BPMN in CSP, giving a behavioural model for specifying timing information to BPMN diagrams. By choosing CSP as the common semantic domain, behavioural properties can be preserved between the untimed and timed semantic models.. • A empirical studies model and the bi-directional transformation to BPMN, allowing one to apply the formal semantics and corresponding tool support for analysing workflows describing empirical studies e.g. clinical trials.. • A small specification language PL, giving an alternative pattern-based approach to behavioural specification for BPMN. The correctness of BPMN diagrams against a PL specification may be verified via model checking. This paper presents a formalisation of BPMN and describes some of its applications through examples. In particular, we introduce a process semantics in CSP and describe how this may be applied to reasoning as well as the refinement of BPMN diagrams. A timed model is then introduced, which augments our untimed model with relative timing, and using these two models we discuss the notion of compatibility and its relationship between the models. We also present a pattern-based approach to constructing property specifications for BPMN, which complements our formal semantics. In applications of BPMN we use a well-documented example of ticket reservation systems to illustrate the applicability of our approach, as well as investigating the use of BPMN in modelling empirical studies. The following are some results about the timed model. We say a diagram is timed if it contains timing information and untimed otherwise; every timed diagram is a timed variant of another untimed diagram, i.e. an untimed diagram augmented with timing information. Below is an intuitive property about timed variation. Results described in Sections 2 to 4 are based on the work reported in [26, 29] ; Section 5 is based on [27] ; Section 7 is based on [28] ; and parts of Section 6 are based on [30] . The restrictive disjunctive normal form and the more complete treatment of the bounded existence pattern, both in Section 6, are previously unpublished.

O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O B I O O O O B O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Ubiquitous Home: Retrieval of Experiences in a Home Environment Automated capture and retrieval of experiences at home is interesting due to the wide variety and personal significance of such experiences. We present a system for retrieval and summarization of continuously captured multimedia data from Ubiquitous Home, a two-room house consisting of a large number of cameras and microphones. Data from pressure based sensors on the floor are analyzed to segment footsteps of different persons. Video and audio handover are implemented to retrieve continuous video streams corresponding to moving persons. An adaptive algorithm based on the rate of footsteps summarizes these video streams. A novel method for audio segmentation using multiple microphones is used for video retrieval based on sounds with high accuracy. An experiment, in which a family lived in this house for twelve days, was conducted. The system was evaluated by the residents who used the system for retrieving their own experiences; we report and discuss the results. This research combines the work from the two main research areas of Ubiquitous Environments and Multimedia Retrieval. Ubiquitous environments are equipped with a large number of sensors of different types, enabling acquisition of data regarding the events that take place in them. They are sometimes referred to as smart environments , if they are able to recognize and respond to the actions of the humans in the environments.. The current research on smart and ubiquitous environments can be divided in to two major categories. One aims at providing services to the people in the environment by detecting and recognizing their actions. Such environments serve as information appliances; examples are the Aware Home Project [2] and Ambient Intelligence Project [3] . The other aims at storing and retrieval of media, in different levels from photos to experiences. This type of research has become possible due to the recent developments in storage technologies facilitating recording large amounts of data. Some of the projects, such as CHIL [4] , attempt to combine both directions by supporting user interaction real-time and using retrieval for long term support.. A discussion of the state of the art of multimedia retrieval can be found in [5] . Most of the existing research deals with a previously edited single video stream with specific content. For such data, the common approach is content analysis, making use of domain knowledge where applicable. However, the use of context data where available, can improve the performance greatly [6] . Life-log video captured by a wearable camera has been indexed and retrieved successfully by using supplementary context information such as location, motion, and time [7] .. There are several ongoing projects that work on multimedia retrieval for ubiquitous environments. The Ubiquitous Sensor Room [8] is an environment that captures data from both wearable and ubiquitous sensors to retrieve video diaries related to experiences of each person in the room. Jaimes et al. [9] utilize graphical representations of important memory cues for interactive video retrieval from a ubiquitous environment. The Sensing Room [10] is a ubiquitous sensing environment equipped with cameras, floor sensors and RFID sensors for long-term analysis of daily human behavior. Video and sensor data are segmented into 10-minute intervals and the activity in the room during each segment is recognized using a Hidden Markov Model. Matsuoka et al. [11] attempt to understand and support daily activity in a house, using a single camera installed in each room and sensors attached to the floor, furniture and household appliances.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The Belfast Induced Natural Emotion Database Abstract-For many years psychological research on facial expression of emotion has relied heavily on a recognition paradigm based on posed static photographs. There is growing evidence that there may be fundamental differences between the expressions depicted in such stimuli and the emotional expressions present in everyday life. Affective computing, with its pragmatic emphasis on realism, needs examples of natural emotion. This paper describes a unique database containing recordings of mild to moderate emotionally colored responses to a series of laboratory-based emotion induction tasks. The recordings are accompanied by information on selfreport of emotion and intensity, continuous trace-style ratings of valence and intensity, the sex of the participant, the sex of the experimenter, the active or passive nature of the induction task, and it gives researchers the opportunity to compare expressions from people from more than one culture. T HEORIES about the capacity of humans to recognize emotions from the facial behavior of others are based largely on the study of highly selected still photographs of posed expressions-an experimental paradigm that has been called the "standard method" [1] . Theories of emotional expression are, of course, attempts to explain natural, not posed human behavior. Despite this fact, there has been relatively little research on natural dynamic examples of emotion (see [2] for a review). Although the lack of natural evidence in support of theories of emotional expression has been criticized [1] , the reluctance to use examples of natural expression when studying emotion is at least partly understandable. When using posed photographs the researcher typically presents participants with examples of prototypical facial expressions based on the Facial Action Coding System (FACS) developed by Ekman and Friesen [3] . Thus, the recognition of the emotion can be judged against the "correct" expression adopted by the poser. The use of examples of natural emotion is much more problematic. In this case, the researcher is using a spontaneous facial expression with no means of knowing objectively what emotion is being experienced by the encoder or the extent to which the facial expression reflects that emotion. The problem can be conceived as a tradeoff between ecological validity and reliability, with the decision over which approach to favor being influenced by the nature of the research question and its sensitivity to each of these factors. If natural everyday emotional expressions showed only quantitative differences from the posed examples (perhaps occurring as a weaker version of the prototypical forms) then perhaps the overreliance on these examples would be of little importance, but recent research suggests that there may be fundamental differences between the two. A number of research studies have indicated that even actors do not show the prototypical patterns of facial behavior when attempting to convey emotion. It has been reported [4] that Hollywood actors only exhibit the prototypical patterns specified by Ekman and Friesen when portraying happiness, and more recent research [5] found none of the prototypical patterns of action units when experienced actors simulated strong emotions. Additionally, it has been suggested [6] that in speech, there may be important qualitative differences between posed and natural examples of emotion expression.. Recently, the overreliance on static images has also been questioned. Evidence is mounting which demonstrates that important information is contained in the dynamic unfolding of facial expressions over time [7] , [8] and that expressions of different emotional states may have specific temporal patterns as well as morphological ones. For example, it has been shown [8] that when dynamic expressions were presented to participants, anger is recognized best when played at a medium speed, sadness is best recognized when it unfolds slowly, and happiness and surprise are recognized most accurately when the expressions change/develop quickly.. This mismatch between theory and reality is of crucial importance for researchers in the field of affective computing. Arguably, it might be possible to construct systems that can deliver believable emotional performances based on the posed prototypical patterns-after all, humans appear to have the capacity to be emotionally engaged by cartoon characters, line drawings, and even inanimate objects. However, if the prototypical patterns do not reflect natural facial behavior, then automatic systems trained on such patterns could not conceivably detect with any degree of accuracy the everyday emotions expressed by real people.. The obvious answer would be to train perceptual systems on examples of naturally expressed emotion. However, it has been pointed out [9] , [10] that there is a shortage of useful databases that can serve this purpose.. In addition, the currently existing databases fall short in many necessary aspects. The Belfast Naturalistic Database has low levels of quality in many of the recordings and suffers from copyright issues, limiting its broad dissemination. Often databases have sporadic and sparse periods of emotion (AMI [11] , Canal9 [12] ) rather than targeting emotional reactions with specific tasks. Many databases do not combine audio and visual components (CohnKanade [13] , MMI-Facial Expression Database [14] , [15] ). Others are aimed at conversational interactions introducing speech as a complicating factor in the expression of emotion (AMI [11] , SEMAINE [16] ). The extent to which databases are annotated is also an issue; extensive annotation greatly adds to the utility of a database.. Ideally, a database of emotional expression would contain high-quality examples of a wide range of spontaneous natural emotion accompanied by a wealth of information about situational variables, individual characteristics of the actor, the emotion being experienced, and its intensity. Normally, the more natural and spontaneous the emotional expression, the harder it is to capture, the less information is available about the actor, and the harder it is to reproduce the eliciting circumstances. The more information you collect about the actor and the more you exert control over the eliciting circumstances, the less natural and spontaneous is the emotional expression you are likely to capture. This incompatibility between ecological validity and control means that, for the foreseeable future, useful databases are likely to compromise between spontaneity and rigid control. We argue that exerting control over the context results in relatively standardized tasks that allow us to collect information about the participant's emotional state. Yet, allowing the participant to react to that context in their own way results in tasks that are engaging enough to yield interesting and useful examples of natural emotional expression. In addition to high quality audio-visual recordings of facial expressions of people in a range of emotional situations, the database described in this paper offers information on self-report of emotion and intensity, continuous trace style ratings of valence and intensity, the sex of the participant, the sex of the experimenter, the active or passive nature of the induction task, and it gives researchers the opportunity to compare expressions from people from more than one culture. As far as we know, this is a unique set of attributes. Development of effective emotion-oriented computer systems is dependent on access to examples of natural emotional behavior. The recordings comprising this database were collected initially to facilitate psychological investigation of sex differences, cultural differences, and individual differences in the expression and perception of emotion. However, we also appreciate the value of this database to researchers in the field of affective computing. This unique collection of mild to moderate natural emotional behavior with a range of useful annotation should prove an invaluable resource for those working on the perception and recognition of emotion. The eliciting situations offer a compromise between entirely natural situations where the encoder is unaware of being observed (with the inevitable practical and ethical problems this entails) and situations in which the experimenter exerts complete control over encoder facial expressions by prescribing muscle configurations. In the eliciting procedures described here, the situations are fixed but encoders are free to respond as they see fit. Of course, in any laboratory-based experimental setting, the extent to which an individual's reactions are entirely "spontaneous" is open to question. We have tried to create sufficiently engaging tasks that would retain an encoder's attention. Inevitably, the extent to which the encoder was led to believe the focus of the research was on something other than the facial expression of emotion differed from task to task. There may be ways to investigate the degree of conscious control that encoders exhibit in their behavioral responses to the different tasks but there is probably no way to ensure that behavior in a laboratory setting exactly mirrors that in a more natural setting. However, we should remember that even in entirely natural social interactions, the extent to which an individual consciously manages the impression they give varies from person to person, from situation to situation, and probably from moment to moment.. These eliciting situations, where the tasks are presented to encoders in the same order, were designed to illuminate individual variations in encoder responses to the situations. Each individual's emotional appraisal of a situation varies for a wide variety of reasons and our recent research has begun to ask what proportion of that variance can be attributed to predictors such as sex, personality, or cultural background. Such variation is, of course, an important facet of human behavior and we have barely begun to explore how much such variation will need to be addressed in artificial systems designed to encode or decode human emotional signals. Questions about the relative importance of universals in the expression and perception of human emotion are empirical questions and it is vital that affective computing does not make premature assumptions about the answers. It is important that ongoing research in this field has access to examples of relatively natural emotional behavior performed by a range of people of differing genders, cultures, and personalities. The database we describe in this paper at least makes a start by providing some of the required variation in emotional responses. However, the database is by no means a finished product and should be regarded rather as a "work in progress.". We are continuing to collect recordings of emotional behavior as part of our ongoing psychological research on different aspects of emotion expression and perception and we intend to expand this database as these recordings and their annotations become available.. Ian Sneddon received the BSc degree in psychology in 1974 and the PhD degree in animal social behavior, both from St. Andrews University. He is currently a senior lecturer in the School of Psychology, Queen's University Belfast. Moving from ethological study of animal behavior to the detailed observation of human behavior, he is a certified FACS coder. His research interests include the expression and perception of emotion, emotion induction techniques and ethical issues, cross-cultural differences in emotion perception/expression. He was on the board of management of the HUMAINE network of excellence and is a member of the SEMAINE project.. Margaret McRorie received the BA degree in psychology from Queen's University Belfast in 1997 and the PhD degree in 2001. She was appointed as Individual Differences lecturer in the Psychology Department at Queen's in 2002. Her research within the area of Individual Differences has continued, and she is currently focusing on individual differences issues within the area of emotion research. She is a certified FACS coder, and is specifically interested in exploring individual characteristics in the encoding and decoding of emotion. Her other research interests include emotion induction techniques, and crosscultural differences in expressing and perceiving emotion. She is a member of the SEMAINE project.. Gary McKeown is a cognitive psychologist in the School of Psychology, Queen's University Belfast. His PhD degree explored mechanisms of implicit learning in the control of complex systems. His research interests include communication, with interest in risk perception and decision making in environmental and health settings. This led to an interest in emotion and, in particular, the interrelationship of cognition and emotion. Recent research has focused on emotion, social signal processing, and cross-cultural emotion perception. He is a member of the SEMAINE project.. Jennifer Hanratty received the BSc degree in psychology in 2003 from Queen's University Belfast (QUB). She returned to QUB in 2006 and was awarded the PhD degree on individual and situational differences in emotional expression in 2010. She is currently working as a research fellow on the SEMAINE project. Her research interests include natural emotional expression and perception, emotion regulation, and personality processes related to emotion.. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.

O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O