Facial Expression Recognition by Supervised Independent Component Analysis Using MAP Estimation SUMMARY Permutation ambiguity of the classical Independent Component Analysis (ICA) may cause problems in feature extraction for pattern classification. Especially when only a small subset of components is derived from data, these components may not be most distinctive for classification, because ICA is an unsupervised method. We include a selective prior for demixing coefficients into the classical ICA to alleviate the problem. Since the prior is constructed upon the classification information from the training data, we refer to the proposed ICA model with a selective prior as a supervised ICA (sICA). We formulated the learning rule for sICA by taking a Maximum a Posteriori (MAP) scheme and further derived a fixed point algorithm for learning the de-mixing matrix. We investigate the performance of sICA in facial expression recognition from the aspects of both correct rate of recognition and robustness even with few independent components. key words: facial expression recognition, supervised independent component analysis, fixed-point algorithm Various methods have been proposed for automatic recognition of facial expression in the past several decades, which could be roughly classified into three categories: 1) Appearance-based method, represented by eigenfaces, fisherfaces and other methods using machine-learning techniques, such as neural networks and Support Vector Machine (SVM); 2) Model-based methods, including graph matching, optical-flow-based method and others; and 3) Hybrids of appearance based and model-based methods, such as Active Appearance Model (AAM). Detailed review of these methods could be found in two surveys in Refs. [1] [2] . Appearancebased methods are superior to model-based methods in system complexity and performance reproducibility. Further, appearance-based methods allow efficient characterization of a low-dimensional subspace within the overall space of raw image measurement, which deepen our understanding of facial expressions from their manifolds in subspace, and provide a statistical framework for the theoretical analysis of system performance. ICA, a powerful technique for blind source separation, was applied to facial expression recognition by Bartlett et al. for feature extraction. [3] They argued that facial expression consists of those features standing for minor, non-rigid, local variations of faces [3] . Struc- tural information for those local variations are related to higher-order statistics, which could be well extracted by ICA. [5] The efficiency of ICA in extracting features for facial expression recognition has been verified by many previous works. [4] [6] [7] The major purpose of the present work is to improve the performance of ICA in facial expression recognition. In the classical ICA, the derived independent components are in random order, i.e., permutation ambiguity, where the original order provides no information on the significance of components in discrimination. [8] As a result, the derived independent components may not be most distinctive for the classification task, especially when only a small subset of components is derived. Feature selection must be performed along with the feature extraction. The selection can be applied before, during or after ICA. In Ref. [4] , Best Individual Feature (BIF) selection was adopted, where features were chosen according to some defined criteria individually. Methods by means of Sequential Forward Selection (SFS) and Sequential Floating Forward Selection (SFFS) were also proposed. [9] Since the selection is performed after ICA, the features are limited to those chosen from the set of independent components obtained. To create a candidate set with enough representative features for discrimination, a large number of independent components should be learned, which may be expensive in computational cost. It is meaningful to search for a way to affect the selection of features before or during ICA. GEMC [10] makes a selection before ICA by heuristically replacing PCA with a discriminant analysis as the pre-processing to ICA, which still lacks a mathematical explanation. ICA in a local facial residue space is also proposed for face recognition, which can be regarded as using the pre-specified residue space to limit the selection of independent components before applying ICA. [11] We propose an approach to implement the feature selection during the learning of independent components. A constraint ICA has been proposed for the analysis of EEG signals, where all components should be sparse and close to a supplied reference signal by including a correlation term. [12] In our case, we try to design a method to let those components with higher degrees of class separation emerge easier than others. The classical ICA in Ref. [13] was shown to be derivable under the scheme of Maximum Log-Likelihood (MLL) estimation. [14] Instead of using the uniform prior for de-mixing coefficients in MLL, we take the Maximum a Posteriori (MAP) estimation. A prior defined by the degree of class separation is introduced to the de-mixing coefficients, which in turn increases the probability of the corresponding independent component being significant in classification. In Ref. [15] , a neural-network based supervised ICA has been proposed, where a three-layered feed-forward network minimizes the dependency of hidden outputs and also minimizes errors between output signals and their targets. Compared to the neural-network based version, our method uses the MAP estimation, which provides a base for further derivations of various approximated learning algorithms. Especially, a fast fixed-point version has been derived in the present work, which needs fewer iterations than feed-forward neural network with fixed learning rate, and is more robust in convergence.. In Section 2, we formulate sICA and derive the learning rule in the fixed-point form. The algorithm for facial expression recognition is also given. In Section 3, numerical experiments are made and the performance of our proposed algorithm is investigated by making a comparison with ICA. We also discuss the influence of the introduced selective prior. Finally, we summarize the present paper and explain our future work. In the present paper, we have proposed a supervised version of ICA for facial expression recognition by performing feature selection along with the learning of ICA. In order to decrease the computational burden of ICA, we try to obtain distinctive bases by only estimating a small subset of independent components. A selective prior defined on the scatter matrix has been introduced to the classical ICA, and MAP estimation is applied to derive the learning rule in a fixed-point algorithm. We performed numerical experiments to investigate the influence of new prior term. We also compared the results of sICA with the results of classical ICA. Our method shows better performance than ICA in that: a) recognition rate under a small number of independent components is largely improved, which is meaningful in dealing with a large training dataset, and b) sICA provides more robust performance than ICA in cases with few components. Experimental results on JAFFE and Cohn-Kanade Databases show that a high recognition rate of about 85% has been obtained by our method, when nearly half of all images are randomly chosen as training data. Further, sICA outperforms MDA by over 5 points on average, and also outperforms PCA in our experiments. There are still some problems left for us to study, such as the decision of optimal λ. Investigation on various priors is also a part of our future work.

B I I O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O B I I B I I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Facial expression recognition in dynamic sequences: An integrated approach Automatic facial expression analysis aims to analyse human facial expressions and classify them into discrete categories. Methods based on existing work are reliant on extracting information from video sequences and employ either some form of subjective thresholding of dynamic information or attempt to identify the particular individual frames in which the expected behaviour occurs. These methods are inefficient as they require either additional subjective information, tedious manual work or fail to take advantage of the information contained in the dynamic signature from facial movements for the task of expression recognition.In this paper, a novel framework is proposed for automatic facial expression analysis which extracts salient information from video sequences but does not rely on any subjective preprocessing or additional user-supplied information to select frames with peak expressions. The experimental framework demonstrates that the proposed method outperforms static expression recognition systems in terms of recognition rate. The approach does not rely on action units (AUs), and therefore, eliminates errors which are otherwise propagated to the final result due to incorrect initial identification of AUs. The proposed framework explores a parametric space of over 300 dimensions and is tested with six state-of-the-art machine learning techniques. Such robust and extensive experimentation provides an important foundation for the assessment of the performance for future work. A further contribution of the paper is offered in the form of a user study. This was conducted in order to investigate the correlation between human cognitive systems and the proposed framework for the understanding of human emotion classification and the reliability of public databases. HighlightsExtraction of dynamic signals via a parametric space to improve the automatic facial expression recognition rate.An objective comparison with systems utilizing static apex expression recognition.The use of a visualisation technique for the analysis and initial understanding of facial feature data.An intuitive user study to investigate the correlation between human perception and machine vision. Facial expression analysis has long been a research area of great interest. Indeed, work beginning as early as the nineteenth century [1] demonstrated that the analysis of facial expressions was of significance. The work in [2] was the first to formalise six different expressions that contained distinctive facial content. These six expressions were summarised as typical emotional displays of: happiness, sadness, fear, disgust, surprise and anger, and are now commonly known as the basic emotions. Until recently, the task of facial expression analysis has been a topic of research primarily associated with the field of psychology and much on the subject has been published in this area. However, interest broadened with the publication of the work in [3] which presented a preliminary investigation of the task of automatic facial expression analysis from a sequence of images. More recently, automatic facial expression analysis has attracted much attention particularly in the field of computer science. Some of the reasons for this are due to the advancements in related research sub-areas such as face detection [4] , tracking and recognition [5] , as well as new developments in the area of machine learning such as feature extraction, and supervised learning [6, 7] .. Much of the recent work on facial expression analysis tended to focus on ways of capturing the 'moment' or the point in time-series data (termed: static expression recognition) at which a particular facial expression begins to occur and when it ends. Previous approaches have mainly concentrated on attempting to capture expressions through either action units (AU) [8, 9] or from discrete frame extraction techniques [10] . All of these methods require either manual selection in order to determine where the particular behaviour occurs or the subjective imposition of thresholds. This means that any classification is highly dependent on the subjective information in the form of a threshold or other human-derived knowledge.. The approach proposed in this paper is formulated in order to tackle the aforementioned problems and to improve the performance of facial expression recognition by exploring dynamic signals. It offers a number of advantages over existing approaches: (a) the system does not require manual specification of the frame which shows peak expression; (b) the system uses the dynamic information of the facial features extracted from video sequences and outperforms techniques based on static images; and (c) it does not rely on the voting from groups of frames, where errors made earlier in the process are propagated leading to incorrect classification(s).. In addition to these advantages, a novel experimental evaluation presented in this paper offers a number of different perspectives for the task of facial expression analysis. For the learning of the expressions, six state-of-the-art machine learning methods are employed. Furthermore, an investigation of those sequences which are consistently mis-classified by the automatic methods is presented. This then forms the basis for a user study, which along with the use of visualisation tools offer an insight into the consistency of human perception and machine vision.. In summary, the contributions of the work are highlighted as follows:. • A novel automatic framework for the recognition of facial expressions using the dynamics of the sequences. Specific contributions include -The use of a group-wise registration algorithm to improve the robustness of tracking performance; -Construction of a parametric space of over 300 dimensions to represent the dynamics of facial expressions; -The use of six state-of-the-art machine learning methods for the automatic recognition task; -An objective comparison between the proposed system (which utilises dynamic information) and systems which utilise static apex images.. • Investigation of the correlation between human perception and machine vision for human emotion recognition.. -The use of a visualisation technique for the analysis and initial understanding of facial feature data, and also for identifying outliers and noise in the data; 2. -An intuitive user study to investigate the correlation between human perception and machine vision on facial expression recognition, and to assess the quality of a public dataset.. The remainder of this paper is structured as follows. Section. 2 presents the background material for automatic facial expression analysis and provides an overview of current approaches. Section. 3 describes the proposed approach (salient facial point tracking and feature extraction methods, and construction of dynamic signal parametric space) along with the automatic learning methods. Section. 4 details the evaluation framework that is employed as well as the experimental setup and user survey. Finally, Section. 5 concludes the paper along with some suggestions for further development. Facial expression analysis and recognition has become one of the most active research topics in recent decades due to its potential contribution to future human-computer interaction analysis. In this paper, a data-driven approach was proposed in order to exploit the dynamic information in video sequences for automatic expression recognition. This was achieved by generating a facial landmark tracking framework and building a parametric space in which to capture the dynamic signal information from both the geometric and texture features. A comprehensive range of machine learning methods were then employed for the task of facial expression recognition. A robust approach such as this to the evaluation step not been presented previously in the literature.. The evaluation aspect of the work was further developed by including a framework for classification accuracy comparison, feature visualisation, and also by offering a novel correlation analysis of human perception and machine vision through the use of a user study. This multifaceted evaluation provides an intuitive way to guide future work on facial expression analysis and in particular recognition. In the evaluation, both automatic classifiers and human participants were able to classify the expressions of happiness and surprise easily, but encountered difficulty in identifying the other basic expressions. However, the dataset used in this paper is relatively small, and some of the expressions are poorly represented (e.g. disgust and fear), whilst others are well represented. This will make it difficult for classifier learners to learn a given concept well. One way to address this is to either balance the dataset, or acquire more data. Other aspects that could be useful perhaps are the use of more contextual information such as audio data and body movements could in order to achieve better performance and a better understanding of human emotions.. Topics for future work include: the investigation of the correlation between the automatic learning methods, the integration of contextual information for expression recognition, and the investigation of the applicability of the work to other forms of video media (skype, video conferencing, streamed data, etc.). A system for automatic facial analysis may include many different aspects. Two of the most common are: (i) the automatic detection and classification of facial expressions -an area where much work has been carried out in the past [11, 12] , (ii) realistic facial expression synthesis in computer graphics [13] , which is useful for studying the perception of expressions and also realistic computer animation; and (iii) expression analysis, important for affect recognition [14] .. Typical facial expression recognition systems aim to classify an input facial image or video sequence into one of the six basic emotions mentioned previously. Facial expressions are formed through the movement of facial muscles, resulting in dynamic facial features such as the deformation of eyebrows, eyes, mouth and skin. Such changes can be captured and used in order to classify a given facial expression. In broad terms, there two approaches a) Facial Action Unit (AU) based techniques and b) content-based (non-AU) techniques; summarised in Section 2.1 and 2.2 respectively.

O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A declarative debugger of incorrect answers for constraint functional-logic programs Debugging is one of the essential parts of the software development cycle. However, the usual debugging techniques used in imperative languages such as the step by step execution often are not suitable for debugging declarative programming languages. We present here a graphical debugging environment for constraint lazy functionallogic programs based on declarative debugging. The debugger displays the computation tree associated with a computation which has produced an incorrect answer, and navigates it with the assistance of the user until the error, an incorrect program rule, is found out. The debugger supports programs including equality and disequality constraints. Categories and Subject DescriptorsKeywords Functional-Logic Languages, Declarative DebuggingThe lack of auxiliary tools such as debuggers has been pointed out in [12] as a possible impediment to the success of functional languages like Haskell [8] . The same arguments can be applied as well to the case of functional-logic languages [6] . However, implementing debuggers for these languages is not an easy task. The approach followed traditionally in imperative programming, based on the step by step execution of the program, is not suitable in declarative languages, particularly in the case of the lazy functional and functional-logic languages, where features like higher order, polymorphism or lazy evaluation makes the computation more difficult to observe and understand.Declarative debugging was first proposed by E. Y. Shapiro [11] as an alternative in the field of logic programming, and has been * Work partially supported by the Spanish CYCIT (project TIC2002-01167 'MELODIAS') Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. later applied in functional and functional-logic programming as well. A declarative debugging session starts when the user observes an initial symptom, i.e. an unexpected behavior of the program which is regarded as wrong. The process can be divided into two phases:1. The declarative debugger generates a suitable computation tree (CT in short) representing the behavior of the program during the computation. Each node of the tree represents the result of some subcomputation. The value at the root of the tree corresponds to the result of the overall computation, while the results at the children of each node N must correspond to the subcomputations necessary to obtain the result at N . Moreover, each node must have some associated fragment of code, which is responsible for the computation result stored at the node. 2. The debugger navigates the CT trying to locate a node whose result is incorrect but with correct results at all its children nodes. Such a node is called a buggy node, and corresponds to a fragment of code which has produced an erroneous result from correct inputs, and is therefore incorrect. In order to check the correctness of the nodes, the debugger relies on an external oracle, usually the user.Here we present DDT , a graphical declarative debugger of incorrect answers for the constraint lazy functional-logic programming language T OY [1]. The prototype is based on a previous version described in [5] , the main novelty of the new version being the possibility of debugging programs including equality and disequality constraints. The system T OY, including the debugger, can be downloaded from: http://toy.sourceforge.net.In the next section the programs considered in our setting are presented, together with a small example of incorrect program which will be used in the rest of the paper. The section 3 presents informally the computation trees handled by the debugger, while section 4 provide some details about the implementation of the program transformation employed by the tool. A short debugging session for our example program is shown in section 5, which also introduces some of the features of the graphical interface. The efficiency of the system is discussed in section 6. The paper ends in section 7 with some conclusions and some proposals for future work.A T OY program can include declarations of data, function types, type alias, infix operator declarations, and defining rules for functions symbols. Two important syntactic categories in this setting are expressions and patterns. The possible forms of an expression 8 We have presented a graphical environment for finding incorrect program rules in constraint lazy functional-logic programs. The debugger is equipped with a graphical user interface and has been implemented for the language T OY, although the same approach can be used for debugging similar languages such as Curry [7] . In fact, a textual version of the debugger, which uses a top-down strategy for detecting buggy nodes, is already part of the Curry system of the Münster University [2] . The debugger currently supports programs with equality and disequality constraints, but the implementation is almost independent of the constraint domain. The only change that must be done in order to apply the debugger to other constraint domains is the redefinition in each case of the primitive getConstraintStore, which is used by the debugger for obtaining the constraint store at the end of the computation. We plan to extend the debugger to programs including arithmetic constraints and also for programs allowing constraints over finite domains.. A different task is to extend the debugger for dealing with missing answers, which currently are not admitted by the prototype. A missing answer is obtained when the set of computed answers for a given goal does not cover some expected answer. In this case the computation tree is different and more complex, since it must represent all the computations occurred while solving the goal.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Conditional random fields for activity recognition in smart environments One of the most common functions of smart environments is to monitor and assist older adults with their activities of daily living. Activity recognition is a key component in this application. It is essentially a temporal classification problem which has been modeled in the past by naïve Bayes classifiers and hidden Markov models (HMMs). In this paper, we describe the use of another probabilistic model: Conditional Random Fields (CRFs), which is currently gaining popularity for its remarkable performance for activity recognition. Our focus is on using CRFs to recognize activities performed by an inhabitant in a smart home environment and our goal is to validate the claim of its higher or similar performance by comparing CRFs with HMMs using data collected in a real smart home. There is growing interest in development of smart environments which are capable to reason about inhabitants in order to provide health monitoring and health assistance. This technology can be used to monitor the activities that inhabitants perform in their everyday settings and to remotely assess their functional wellbeing. The need for development of such technologies is underscored by the aging of the population, the cost of formal health care, and the importance that individuals place on remaining independent in their own homes. One of the most important steps toward monitoring the functional health of smart environment residents is to recognize the activities they usually perform in the environment. Therefore, the aim of activity recognition in a smart home setting is to recognize Activities of Daily Living (ADLs) [1] that residents perform in their homes. Activity recognition is essentially a sequence classification problem. Observation data sequences obtained by various sensors are annotated with corresponding activity labels and are used to train different prediction models. Over the past few years, there has been an upsurge of innovative sensor technologies and recognition algorithms for this area from different research groups. The Easyliving project uses the images from two sets of color stereo cameras to track multiple people moving through a space. The eWatch project [2] proposes a new system based on eWatch for activity recognition and monitoring. The Georgia Tech Aware Home [3] identifies people based on pressure sensors embedded into the smart floor in strategic locations. The CASAS project [4] designed and assessed several algorithms that built probabilistic models of activities and used them to recognize activities in complex situations.. Models used for the purpose of activity recognition can be classified as probabilistic [5, 6] , logic based [7] or hand-crafted [8] . Probabilistic models are the most popular approaches for this task, because sensor readings are usually noisy and activities are commonly performed in a non-deterministic way. Several probabilistic models have been proposed to model the sequence classification problem of activity recognition and among these, hidden Markov models (HMMs) [9] and more recently Conditional Random Fields (CRFs) [10] are the most common ones. HMMs have been used frequently for activity recognition [11, 12] , while CRFs have been proposed originally for the context of natural language processing. CRFs are expanding to other areas including activity recognition [13] , although they have typically been dependent on costly accelerometer and RFID sensors.. Studying CRFs 1 and their application to the field of activity recognition in our smart home is the main focus of this paper. In addition, we compare HMMs and CRFs for the purpose of activity recognition in smart home environment.. The remaining of this paper is organized as follows: In section 2, we give a detailed description of the experimental setup, an effective procedure to collect and annotate data. In sections 3 and 4, we discuss HMMs and CRFs respectively. The more emphasis would be on CRF portion. In section 5, we present the experimental results. Finally in section 6, we conclude the paper and present the future work. 1 In this paper we only consider linear-chain CRFs.. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. In this paper we use conditional random fields for activity recognition, a classical problem in the area of smart environments. We also compare its performance, for recognizing ADLs, to the performance of HMM, another popular probabilistic approach for activity recognition. With our results it can be concluded that CRFs work quite well for data streams. So, it is quite suitable for our projects as the raw sensor data that we collect are in the form of data streams.. In future we want to use CRFs for assessing the completeness of activities. This could be done by recognizing individual steps of an activity that could be compared with statistical inputs from psychologist to determine how well it was completed. Another prospective area in which CRFs could have good application is online activity recognition. Although attention has been focused in this area, a little success has been achieved. The reason for this is the incapability of popular machine learning and data mining techniques to handle data streams. As CRFs have the capability to handle this kind of data, it would be a good approach to solve the problem of online activity recognition. In this section, we describe how data is collected in our smart home testbed, how it is annotated and which features are extracted from collected data.

B I I O B I O B I O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O


Complexity of Gaussian-radial-basis networks approximating smooth functions Complexity of Gaussian radial-basis-function networks, with varying widths, is investigated. Upper bounds on rates of decrease of approximation errors with increasing number of hidden units are derived. Bounds are in terms of norms measuring smoothness (Bessel and Sobolev norms) multiplied by explicitly given functions a(r, d) of the number of variables d and degree of smoothness r. Estimates are proven using suitable integral representations in the form of networks with continua of hidden units computing scaled Gaussians and translated Bessel potentials. Consequences on tractability of approximation by Gaussian-radial-basis function networks are discussed. -norm of the weight function.. ; the rest follows from Proposition 4.2.. 2. Composing estimates of variations with respect to sets of translated Bessel potentials and Gaussians, we get an upper bound on rates of approximation by networks with n Gaussian RBF units for functions from Bessel spaces. . Then the statements follow from the upper bounds on f G βr given in Propositions 4.1 and 4.2, resp. 2

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Rethinking security properties, threat models, and the design space in sensor networks: A case study in SCADA systems In recent years we have witnessed the emergence and establishment of research in sensor network security. The majority of the literature has focused on discovering numerous vulnerabilities and attacks against sensor networks, along with suggestions for corresponding countermeasures. However, there has been little guidance for understanding the holistic nature of sensor network security for practical deployments. In this paper, we discuss these concerns and propose a taxonomy composed of the security properties of the sensor network, the threat model, and the security design space. In particular, we try to understand the application-layer goals of a sensor network, and provide a guide to research challenges that need to be addressed in order to prioritize our defenses against threats to applicationlayer goals. In this paper, we presented a taxonomy with the aim to provide a holistic view of the security of sensor networks. We believe this research direction will provide a better understanding of the security issues and will help the network designer decide on the most effective security mechanisms under resource constraints. However, there are many research challenges that need to be addressed first, such as, developing a systematic analysis of the threat model and its relation to the security countermeasures, the precise definitions of security metrics, and the detailed study of real world deployment scenarios.

O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


What is beautiful is usable An experiment was conducted to test the relationships between users' perceptions of a computerized system's beauty and usability. The experiment used a computerized application as a surrogate for an Automated Teller Machine (ATM). Perceptions were elicited before and after the participants used the system. Pre-experimental measures indicate strong correlations between system's perceived aesthetics and perceived usability. Post-experimental measures indicated that the strong correlation remained intact. A multivariate analysis of covariance revealed that the degree of system's aesthetics affected the post-use perceptions of both aesthetics and usability, whereas the degree of actual usability had no such effect. The results resemble those found by social psychologists regarding the effect of physical attractiveness on the valuation of other personality attributes. The ®ndings stress the importance of studying the aesthetic aspect of human±computer interaction (HCI) design and its relationships to other design dimensions. q The tension between form and function has long been at the crossroad of artifact design [1±3] . Whereas emphasis on function stresses the importance of the artifact's usability and usefulness, accentuating the artifact's form serves more the aesthetic, and perhaps social, needs of designers and customers. Until the ®rst quarter of this century, the design of commodities and mass production artifacts were quite devoid of aesthetic considerations. Petroski [3] credits two industrial design pioneers, Loewy and Dreyfuss, with the introduction of aesthetic considerations to mass production and with the development of industrial design as an explicit marketing instrument. Evidently, aesthetics considerations gained importance quickly. About half a century later, Norman [4, 5] laments the appropriation of modern design by designers who place aesthetics ahead of usability. Similar sentiments concerning designers' priorities can be found in various areas of artifact design.. For example, Tufte [6] denounces the increasing use of ªchartjunkºÐthe application of unnecessary decoration and ink elements that hinder the ef®cient reading of visual displays.. Perhaps in a backlash to recent tendencies by the computer industry to oversell glitz and fashion in its products or because of its origins in disciplines that emphasize ef®ciency, the ®eld of human-computer interaction (HCI) appears to stress the prominence of usability over aesthetics. There is little doubt that, in general, the criterion of aesthetic design is an integral part of effective interaction design [7] . Yet, readers of HCI textbooks can hardly ®nd any reference to aesthetic considerations in design. Contrast this with the plethora of publications that deal with the concept of usability and its application and evaluation. The claim for the prominence of usability in the ®eld of HCI goes sometimes as far as stating that ªmeasurement of usability de®nes the success or lack thereof in a GUI designº [8, p. 60] .. In a sense the concepts of aesthetics and usability represent two orthogonal dimensions of HCI. Whereas aesthetics usually refers often to non-quanti®able, subjective, and affectbased experience of system use, usability is commonly measured by relatively objective means and sets ef®ciency as its foremost criterion [8] .. The near neglect of the aesthetic aspect of HCI is unfortunate for several reasons. First, it reveals a gap between the practice of much of the computer industry and the research foci of HCI in terms of the attention given to HCI design criteria. Second, it seems to ignore important needs of computer users, who, like consumers of other commodities [9, 10] are likely to value aesthetics and fashionable designs in addition to usability. Third, previous research suggests that aesthetic perceptions of an interface are highly correlated with perceptions of the interface's ease of use [11, 12] . Therefore, it appears that users do not perceive these two design dimensions as independent. Thus, our purpose in this study is to experimentally explore users' perceptions of usability and aesthetics before and after using an application, in order to tease out how they effect users' evaluation of the quality of their interaction with the application. This study demonstrated once again the tight relationships between users' initial perceptions of interface aesthetics and their perceptions of the system's usability. Moreover, we showed that these relations endure even after actual use of the system. We believe that these results shed new light on the role of aesthetics in HCI design and its effects on how users experience their interaction with computerized systems. The results of this study are commensurate with social psychology ®ndings that people associate a person's physical attractiveness with other personal attributes. Similarly, research in the areas of marketing and consumer behavior indicate that aesthetic features of the shopping environment are perceived as related to other, seemingly independent attributes (e.g. functional) of that environment. Obviously, more research is needed to assess the contingencies and boundaries of the aesthetics-usability relationships. Most importantly, these relationships should be studied during a longer time frame than we were able to do. Yet, we believe that there is suf®cient evidence already to justify the elevation of the issue of aesthetic design from its current standing at the cellars of HCI research. This study corroborates the results of earlier studies [11, 12] that found strong correlations between users' perception of an interface aesthetics and their perception of the usability of the entire system. We also demonstrated that users are capable of distinguishing between various properties of the system. For example, in our study users did not associate the interface's attribute of amount of information with the interface's beauty or ease of use. This ®nding reinforces the claim that the aesthetics-usability association is a genuine phenomenon and not the result of an evaluation method bias [12] . Our study was not designed to explore the process that leads to this association, thus further studies are needed to shed more light on the cognitive and/or affective processes that lead users to associate interface aesthetics with other system attributes.. Perhaps the most interesting ®nding of this study, though, relates to the question of what most strongly determine post-use perceptions of usability, aesthetics and overall satisfaction with the system. An intuitive answer to this question would probably be that aesthetics affects perceptions of aesthetics, actual usability affects perceptions of usability, and that some combination of the two affects satisfaction with the system. The ®nding that the participants who interacted with an aesthetic ATM perceived the system as more aesthetic than participants who interacted with less aesthetic ATMs is therefore expected. However, most surprising is the fact that post-experimental perceptions of system usability were affected by the interface's aesthetics and not by the actual usability of the system.. While overall evaluations of aesthetics and usability were higher when using the more aesthetic interface, we also found an interesting ®nding regarding changes between the pre-and the post-experimental evaluations. Users in the medium-and low-aesthetics groups tended to evaluate the ATM's usability and especially its aesthetics more favorably after they used it relative to their initial evaluations. These results might be explained in terms of a ªmere exposureº effect [43] , which suggests that our evaluations of an object improve after we are repeatedly exposed to that object. However, as Judd and Brauer [44] indicate, there seem to be several processes that operate during repeated exposures to an object, some of which lead to even lower evaluations of object that were initially evaluated negatively. Perhaps, these results re¯ect the process of users' adapting to a system with which they had to interact. In a sense it is the HCI version of ªlove the one you're withº. Clearly, more research is needed on this issue.. This research was motivated primarily by the tension between function and form in HCI. The results suggesting that interface aesthetics has a major effect on a priori perceptions of ease of use, and perhaps more importantly on post facto evaluations of usability may come as an uneasy surprise to those versed in the ®eld of HCI. The view of usability and aesthetics as interrelated attributes questions the unequivocal message expressed by the HCI literature in favor of the former over the latter [20, 21] .. The design of information systems in general and of the user interface speci®cally is often compared to architectural design. In noting the potential similarity between these two design disciplines, Hooper [45, p. 13] wrote that architectural analyses emphasize the facades of buildings for several reasons. Among those reasons are that the facade is the introduction to the building: ªThis is what most people experience directlyº. In addition, the facade can serve as a ªmembrane between the inside and the outside, and¼ its purpose is to articulate the relationship between the two.º However, Hooper later dismisses the viability of the ªinterface as facadeº approach. Our study demonstrates that regardless of its desirability to interface designers, users might relate to this analogy. The facade of an information system is what users experience ®rst and it is what cues users about the inside. Moreover, the facade taints how the user perceives further interactions with the system.. The implications for interface design are clear. The fact that users perceive aesthetically appealing interfaces as indicative of usable systems calls for an integrative approach to interface design which will take simultaneous account of the two seemingly unrelated properties. Intuitively, it would seem that the importance of aesthetics design is relevant mainly to systems that are used voluntarily. There is a higher likelihood of purchasing or using a system that is perceived to be more usable (as well as more aesthetic). However, the results of this study might be relevant even in situations where users are compelled to use a system. For one, as shown here, there are strong correlations between users' satisfaction from using the system and their perceptions of its aesthetics and usability (which by itself is affected by perceived aesthetics). User satisfaction is important even for involuntary system use. Second, the advantage of aesthetic interfaces might transcend the mere (yet important) perceptions of the system's quality to the domain of actual performance. As demonstrated by Isen [46] , positive affect is likely to improve decision making and creativity. Thus, the positive affect created by aesthetically appealing interfaces may be instrumental in improving users' performance as well.. We should note that this study has certain limitations, as is the case with any exploration of new research venues. The post-use evaluations of the ATM's attributes in our experiment were obtained after a relatively short (though quite intense) time period. The results thus, are only indicative of a short-term in¯uence of aesthetic perceptions. Clearly, more research is needed to establish its long-term in¯uence. In addition, this study presents only bottom-line data (that is, only the end results of the human±computer interaction). Thus, the process underlying the relationships between aesthetics, perceived usability and other system attributes can only be speculated. Better understanding of these relationships requires the use of more process-oriented data. It should also be pointed out that the relative homogeneous nature of the participants in this study (all were engineering students) restricts the generalizability of the experimental results. Obviously, generalization could have been broader if participants had more heterogeneous backgrounds. However, given that engineering students with broad computer experience are likely to represent a relatively rational approach towards information technology, the current results may look even more striking. In a sense, one would be hard pressed to ®nd a different group of students who could better distinguish between the appearance and the behavior of an artifact. On the other hand, it might be that more mature participants (e.g. professionals) would have perceived the aesthetics-usability relations differently. Thus, it is recommended that future studies indeed sample from such populations.. In addition to further exploration of the nature of relationships between interface aesthetics and users' perceptions of other system attributes, one of the most intriguing questions is: What makes an aesthetic interface? A few general guidelines exist already in the ®eld of HCI [21] and in in¯uential works on esthetical preferences in general [47] . Yet, as Martindale et al. [48] demonstrated, these guidelines and other commonly held beliefs do not always withstand empirical scrutiny. Again, work in the area of marketing and consumer behavior [49] may provide initial basis for such research. Much effort has been put in recent decades into studying, and then advocating, the effective design of HCI. One of the central sub-disciplines that emerged in the ®eld during the early 1980s was that of usability engineering [13, 14] . With a few exceptions (e.g. [15] , who emphasized the user's experience), usability engineering has traditionally emphasized objective performance criteria, such as time to learn, error rate and time to complete a task [8] . Recently, as usability became one of HCI's most popular concepts [16, 17] , less stringent and more subjective criteria such as user satisfaction have also been widely recognized as appropriate measures of good HCI design [18, 19] .. There is a pervasive claim for the prominence of usability criterion over aesthetics in HCI. Even when classical elements of aestheticsÐsuch as screen design and graphicsÐ are dealt with [20±22], they are mostly analyzed in terms of their effects on human information processing rather than on human affect and experience. Whenever considerations of aesthetics and usability contradict, the unequivocal recommendation is to give priority to the latter [20, 21] . This sentiment is expressed today with regard to the design of Web sites as well. For example, in a guide to Web site usability, Spool et al. [23] argue that ª¼ no one surfs the online employee policy manual just for kicksº (p. 4). They conclude ªwe found no evidence that graphic design helps users retrieve information on a web siteº (p. 83, italics added). Unfortunately, this approach ignores the affective aspect of users' behavior [24] and its relevance to HCI. It also ignores the potential relationships that exist between usability and aesthetics and their combined effect on users' evaluation of computer systems. For example, Jordan [10] found that usability and aesthetics (as well as several other factors) are both instrumental in creating pleasurable electronic products.. Because the aesthetics aspect was by large overlooked in HCI research, it is not surprising that only a handful of studies to date have investigated the relations between users' perceptions of aesthetics and usability. Kurosu and Kashimura [11] explored the relationships between a priori perceptions of the ease of use of an automatic teller machine (ATM)Ðwhich they termed ªapparent usabilityºÐand other variables. These variables included factors believed by HCI professionals to enhance usability (termed ªinherent usabilityº by Kurosu and Kashimura). Another factor included in Kurosu and Kashimura's study was the perceived beauty of the interface. The study, conducted in Japan, found surprisingly high relationships between users' judgements of the interface's aesthetics and its apparent usability r 0:59: (In fact, this correlation was higher than all but one of the inherent usability factors.) In a recent study, Tractinsky [12] corroborated Kurosu and Kashimura's ®ndings in a different culture (Israel), and removed doubts about potential method bias as an alternative explanation to the high correlation between apparent usability and perceptions of interface aesthetics.. The mechanism that links affective and cognitive evaluations of user interfaces is not clear. We speculate that the strong correlations found between perceived usability and aesthetics resemble ®ndings in the social psychology literature about the relationships between physical attractiveness and socially desirable characteristics. This conjecture is in line with the recent evidence suggesting that individuals experience media similar to their experience of social environments and interactions. Reeves and Nass [25] presented the results of an extensive research project, which found that ªindividuals' interactions with computers, television, and new media are fundamentally social and natural, just like in real lifeº (p. 5, original emphasis). The social phenomenon of inferring personality attributes from physical attractiveness was demonstrated by Dion et al. [26] . In a paper titled ªWhat is beautiful is good,º Dion et al. found that people who are physically attractive are assumed (by other people) to possess more socially desirable personality traits than persons who are unattractive. Researchers suggested two mechanisms by which the ªbeautiful is goodº phenomenon can be explained. The ®rst mechanism suggests that the carry over from the physical appearance to other traits may re¯ect a stereotyping approach, which associates beauty with other personal attributes [26, 27] . The other explanation involves the halo effect. That is, because physical beauty is the most obvious and accessible personal characteristic accessible to others [26] , it is perceived early in the interaction and then tends to color later perceptions and inferences about other personal characteristics.. Recently, issues of aesthetics, and more generally, affective properties of the shopping environment became the foci of research in the ®elds of marketing and consumer behavior.. The concept of ªretail personalityº was introduced by Martineau [28] , who illustrated how a store's functional and psychological characteristics are capable of de®ning its image in a consumer's mind. Russell and Pratt [29] suggested that a store's affective quality (which is based to a large extent on its physical characteristics, including aesthetics) must be considered in any complete description of the shopping environment. Donovan and Rossiter [30] found that feelings generated by a retail store environment relate to shopping behavior while Bloch [31, p. 16] concluded that the ªphysical form or design of a product is an unquestioned determinant of its marketplace successº. Edell and Burke [32] demonstrated that feelings generated by advertisements (especially by the nonverbal elements of the ad) may subsequently in¯uence cognitive processing of the ad's content. A study by Darden and Babin [9] found that not only are store characteristics capable of enticing affective response in consumers, but that a store's functional (i.e. more objective) and affective (i.e. subjective) qualities are interrelated in the consumers' eyes. Similar to the ªbeauty is goodº phenomenon in social psychology, consumer researchers suspect that a halo effect is responsible for a carry over of ®rst impressions of products or shopping environments to consumers' evaluations of other attributes of these products or environments [33] . In addition, this literature suggests that aesthetics may affect perceptions of products by inducing affective response which, in turn, in¯uence evaluations of other product attributes and of the product in general. In Fazio et al., [34, p. 212 ] terms, ªaffect is preattentively`extracted' and in¯uences subsequent perception.º.. To conclude, three different processes may induce positive relationships between interface aesthetics and perceived usability: (a) A popular stereotyping which might associate successful design on one (noticeable) design dimension with successful design of other, less implicit design dimensions. (b) A halo effect may cause carry over of an aesthetic (or not aesthetic) design to perceptions of other design features. (c) An affective response to the design's aesthetics may improve users' mood and their overall evaluations of the system.

O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O B O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B O O O B O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O B O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O


Improved solutions for the traveling purchaser problem Scope and Purpose ÐThe traveling purchaser problem (TPP) is a generalization of the well-known traveling salesman problem (TSP), which has many real-world applications. Examples include the purchase of the required raw materials for the manufacturing factories in which the total cost has to be minimized, and the scheduling of a set of jobs over some machines with dierent set-up and job processing costs in which the total cost for completing the jobs has to be minimized. The TPP has been shown to be computationally intractable. Therefore, many heuristic solution procedures have been proposed to solve the TPP approximately. The purpose of this paper is to consider some variations of the existing solution procedures to improve the solutions. Computational experiments and performance comparisons among the existing algorithms as well as the proposed variations are provided, and the results are analyzed.AbstractÐThe traveling purchaser problem (TPP) is an interesting generalization of the well-known traveling salesman problem (TSP), in which a list of commodity items have to be purchased at some markets selling various commodities with dierent prices, and the total travel and purchase costs must be minimized. Applications include the purchase of raw materials for the manufacturing factories in which the total cost has to be minimized, and the scheduling of jobs over some machines with dierent set-up and job processing costs in which the total costs for completing the jobs has to be minimized. The TPP has been shown to be computationally intractable. Therefore, many heuristic solution procedures, including the Search algorithm, the Generalized-Savings algorithm, the Tour-Reduction algorithm, and the Commodity-Adding algorithm have been proposed to solve the TPP approximately. In this paper, we consider some variations of these algorithms to improve the solutions. The proposed variations are compared with the existing solution procedures. The results indicate that the proposed variations signi®cantly improve the existing solutions. # There are numerous generalizations of the traveling salesman problem (TSP). Examples include the time-constrained traveling salesman problem (TCTSP), the stochastic traveling salesman problem (STSP), the multiple traveling salesman problem (MTSP), the vehicle routing problem (VRP), and many others. The traveling purchaser problem (TPP) is another interesting generalization of the TSP, which has many applications. One example is the purchase of parts and raw materials for the manufacturing factories in which the total cost has to be minimized. Since the required parts and raw materials may be carried by various warehouses located in dierent areas with dierent prices in this case, the purchaser has to select the warehouses to be visited, and plan the tour for these selected warehouses in order to minimize the sum of the purchase and the travel costs. Another example is the scheduling of a set of jobs over some machines. Each machine has a dierent set-up cost and a dierent job processing cost, and the engineer has to determine which machines, and in what orders the jobs should be processed on these machines so that the total cost to complete the jobs is minimized.. The TPP was originally proposed by Ramesh [6] . The problem may be stated as follows. We are given a network with a set of markets V = {v 1 , v 2 , F F F, v n } with a special market v 1 called the domicile, and a set of items P = {1, 2, F F F, m} to be purchased. We are also given a travel cost matrix D = [d(v i , v j )], 1 i, j n, between any two markets; and C = [c(v i , k)], 2 i n, 1 k m, the purchase costs of the items at the various markets. Then, the objective of the TPP is to ®nd a tour, which starts at the domicile v 1 , goes through a subset of the n markets and returns to v 1 once all m items are purchased at chosen markets, and the total travel and purchase cost is minimized. It is assumed that: (1) each item is available in at least one market; (2) the traveling purchaser may pass through a market or the domicile v 1 any number of times without purchasing an item there; (3) the traveling purchaser may purchase as many items as there are available at each market; and (4) no items are available at the domicile v 1 .. The TPP has been shown to be computationally intractable, and many heuristic solution procedures have been proposed to solve the TPP approximately, including the Search algorithm [6] , the Generalized-Savings algorithm [2] , the Tour-Reduction algorithm [3] and the Commodity-Adding algorithm [4] . In this paper, we consider several variations of the four existing algorithms to improve the solutions. The proposed variations are compared with the existing solution procedures. The results indicate that the proposed variations signi®cantly improve the original versions of the four existing solution procedures. Extensive computational experiments and comparisons are provided. The traveling purchaser problem (TPP) is an interesting generalization of the well-known traveling salesman problem (TSP), which has many applications. The TPP has been shown to be computationally intractable. Therefore, many heuristic solution procedures have been proposed to solve the TPP approximately, including the Search algorithm, the GeneralizedSavings algorithm, the Tour-Reduction algorithm, and the Commodity-Adding algorithm.. In this paper, we considered several variations of the four existing algorithms to improve the solutions. Those variations include the Next-Bloc and the Next-Neighbor search algorithms, the Parameter-Selection and the Tie-Selection Generalized-Savings algorithms, the AdjustedCheapest and the Nearest-Cheapest Tour-Reduction algorithms, and the Random-Order and the Sequence-Order Commodity-Adding algorithms. We experimented with the proposed variations on many problems which were randomly generated. The results indicated that the proposed variations signi®cantly improved the original algorithms. In particular, the improvements made by the Next-Neighbor Search, the Parameter-Selection Generalized-Savings, the Nearest-Cheapest Tour-Reduction, and the Random-Order Commodity-Adding algorithms with TSP resequencing, the market-drop, and the market-exchange improvement procedures, were remarkable. We also compared the four best variations. The results indicated that the Random-Order Commodity-Adding algorithm (variation D10) signi®cantly outperformed the other three best variations as well as the four original algorithms.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Compression Model for DNA Multiple Sequence Alignment Blocks Abstract-A particularly voluminous dataset in molecular genomics, known as whole genome alignments, has gained considerable importance over the last years. In this paper, we propose a compression modeling approach for the multiple sequence alignment (MSA) blocks, which make up most of these datasets. Our method is based on a mixture of finite-context models. Contrarily to other recent approaches, it addresses both the DNA bases and gap symbols at once, better exploring the existing correlations. For comparison with previous methods, our algorithm was tested in the multiz28way dataset. On average, it attained 0.94 bits per symbol, approximately 7% better than the previous best, for a similar computational complexity. We also tested the model in the most recent dataset, multiz46way. In this dataset, that contains alignments of 46 different species, our compression model achieved an average of 0.72 bits per MSA block symbol. O VER the past years, impressive progress has been made in DNA sequencing. The genomes of many species, some of them very large, are now available and are increasing in number and size at a fast pace. This progress caused a significant growth in the amount of data available in public databases, including those related to whole genome alignments. Recently, Hanus et al. [1] proposed a modeling strategy for whole genome alignment data, relying on well-established statistical evolutionary models for representing the DNA bases and on a context-based modeling approach for representing the alignment gaps. This resulted in a sophisticated method capable of attaining coding performances much better than the popular Lempel-Ziv (LZ)-based coding techniques. The algorithm assumes independence between the insertion/deletion mutational process and the evolutionary nucleotide substitution process. Based on this assumption, the positions of the gaps and the nucleotides in the alignment are encoded independently. The nucleotides are compressed using appropriate modeling of genome evolution. The gap positions, resulting from the insertion/deletion mutational process, are encoded using prediction techniques inspired by lossless binary image compression methods.. In this paper, we propose an alternative approach for modeling and compressing whole genome alignment data. Instead of separating the data into two different sources (DNA bases and gaps), handled independently, we build a model that addresses both the DNA bases and the alignment gaps at once. This new modeling strategy allows exploring additional data correlations not considered by the Hanus et al. approach, such as intercolumn dependences and base/gap relations, resulting in further coding gains.. Whole genome alignments are important datasets that provide the opportunity to study and analyze the evolutionary process of several species (vertebrate and mammalian, for example). In the next section, we explain in more detail this particular genomic dataset. The University of California at Santa Cruz comparative genomics group and the Ensemble Compara project [2] both provide genome alignment datasets, requiring hundreds of gigabytes of storage, that are growing in size in every new release. In this paper, to allow a direct comparison with the results attained by the method proposed by Hanus et al. [1] , we also use the multiz28way [3] dataset for assessing the performance of the compression models (this set includes alignments of 28 vertebrate genome sequences). Besides this dataset, we also provide compression results regarding the most recent dataset available, multiz46way that contains alignments of 46 different species.. The compression of DNA sequences has proved to be a difficult problem, even when using high complexity models. Some of the best compression algorithms require an average of at least 1.7 bits per symbol (bps) [4] - [7] , for a maximum of . In the whole genome alignment case, we consider a five letter alphabet, , that includes each of the four bases, denine, ytosine, uanine, and hymine, and where symbol " " represents a missing entry (i.e., an alignment gap). An uniformly distributed and independent sequence drawn from this alphabet requires to be stored. The modeling method that we propose, using a mixture of finite-context models, attains rates around 0.94 bps on the multiz28way dataset, about 7% less than the method reported in [1] . This is considerably less than the 1.7 bps average value that is usually attained in single sequences because, even having a larger alphabet, the correlation among the data is also larger, as explained in the following sections. Recent advances in DNA sequencing led to a fast growth in the amount of genomic data available in public databases. This paper deals with one particularly voluminous dataset in molecular genomics, known as whole genome alignments. These datasets contain genomes of different species aligned using an extra symbol representing missing entries. In this paper, we presented a modeling approach specifically designed for compressing whole genome alignments. The proposed algorithm is based on a mixture of finite-context models and arithmetic coding. We use context templates similar to those used in image coding, but also some designed taking into account the strong correlation that exists in each column of the MSA blocks. Despite the ACM and CM being relatively different, they explore the column correlation of the MSA blocks in a similar way. The SCM is an order-0 model that explores the statistical information of each column independently, by using the symbol frequencies of the current column. In Section V, we presented the results for several combinations of these models. We concluded that using the context templates T4, T9, T10, and the ACM, we could attain the best compression results. Using this mixture, the proposed method outperforms the LZ and MSAc methods, achieving compression rates close to 0.94 bps for the multiz28way dataset and about 0.72 bps for the multiz46way dataset. We also concluded that the model with the best performance is associated with the T9 template configuration. We used two datasets in our simulations: the multiz28way, also used by Hanus et al. [1] , and the most recent dataset, with 46 species, multiz46way. As mentioned previously, our algorithm is based on a mixture of several models that are combined in order to compute the probabilities that are communicated to the arithmetic encoder. We have performed several simulations for understanding which models should be included in the mixture. Fig. 9 depicts the results obtained using several combinations. We show only the best combinations, due to the large number of all possible arrangements. T4, T9, and T10 denote the models based on the templates presented in Fig. 4 . SCM, CM, and ACM correspond to the SCM, CM, and ACM, respectively. The results indicate that the SCM generates worse results, when compared to the CM and ACM models. The combination with CM and ACM provides similar results. However, the mixture T4+T9+T10+ACM seems to be the one that provides the best compression results and, for this reason, it was chosen.. In Section IV-B, we have explained how the mixture of models is performed. Because this mixture depends on a parameter , we investigated how the performance is related to the value of . Fig. 10 displays this relation, where it can be observed that the providing the best results is between 0.95 and 0.96. For , the global compression results is about 0.93930 bps. On the other hand, for , the compression is about 0.93934 bps. Since the variation is minimal, we decided to use 0.95 as the default value.. Table II presents the overall compression ratio for different alignment files and for the whole multiz28way and multiz46way datasets. The "FcmMxMSA" column contains the results attained by the compression approach proposed in this paper, including the additional information required by the decoder for recovering the MSA blocks, such as the size of each block. Table II contains also the results attained by the method proposed by Hanus et al. [1] (only for the multiz28way, because the available software, MSAc, could not handle the multiz46way dataset), and by LZ compression.. As can be seen, on average, the proposed method requires less than 1 bps for compressing the MSA blocks of the multiz28way dataset, outperforming the MSAc and LZ methods. The compression ratio is even higher for the multiz46way dataset, with an average of 0.72 bps to store the MSA blocks. However, as can be seen by the compression value provided by LZ, this dataset is in fact more compressible. It should be noted that the parameters of the model have been tuned using only the multiz28way dataset. Nevertheless, and looking at the compression gains in relation to LZ, we verify that the model remains effective for the multiz46way dataset, i.e., it does not reveal overfitting. The "LB" column provides the number of bits per symbol that would be required if we could select the best model to encode each symbol (but without considering the side information indicating the model used). Therefore, this value can be seen as a lower bound on the number of bits per symbol that can be Fig. 10 . Relation between the average number of bits per symbol for the multiz28way dataset and the parameter . This evaluation was performed using the combination T4+T9+T10+ACM.. attained by these models. For a certain set of models, the gap between this value and the value effectively attained depends only on the quality of the probability mixing procedure. Fig. 11 displays two charts showing the contribution of each symbol to the final number of bits per symbol attained in each of the two datasets (in this case, without considering the overhead associated with the additional information, such as the size of the blocks). The left chart shows the average bits per symbol required by the proposed algorithm for compressing each symbol. The chart on the right-hand side shows the relative frequency of each symbol. After analyzing the information presented in Fig. 11 , it is clear that symbols " " and " " are the ones having the worst compression results. On the other hand, the gap symbol (" ") is the one with the best compression ratio. Regarding the right-hand side graphic of Fig. 11 , we conclude that the gap symbol is the most frequent in the datasets. On the contrary, the less frequent symbols are the " " and " " [24] .. These observations suggest the reason why we obtained better compression results in the multiz46way dataset. In this dataset, the relative frequency of the gap symbol is about 8% higher than in the multiz28way dataset (with more species involved, there is a higher probability of having alignment gaps). Also, the gap symbols are easier to compress (they tend to appear in larger clusters, which facilitates the compression). Combined, these two aspects have a decisive influence in the final compression ratio.. In order to understand more clearly the performance of each model, we have collected statistical information of each one during the compression of the multiz28way dataset. The goal was to count how many times a certain model was the best one in the mixture. For each symbol, the best model is the one that alone would generate better compression results than others. These results are shown in Fig. 12 . As can be seen, on average, the model associated with the context template T9 was of the time the best one, whereas the model with context T10 was the best of the time. Both the ACM and the model associated with context template T4 have been the best model, each of them, of the time. Finally, some considerations regarding the computational complexity of the methods. Our method took about 22.5 h to compress the multiz28way dataset. This measure takes into account the compression of the MSA blocks and the extra information contained in the files. The extra information was compressed using the LZ method. The MSAc method took almost 23 h to compress the same dataset, i.e., both approaches are comparable in terms of computational complexity. The multiz46way dataset, which contains about twice the data of the multiz28way dataset, took about 48 h to compress using our approach, i.e., about twice the time, suggesting that the computational complexity of the technique is linear in relation to the size of the dataset. Moreover, since the algorithm addresses both DNA bases and gaps as image pixels, without distinguishing between them, its complexity does not depend on the real probability model of the data. Fig. 12 . Performance of the four models: using template T4, T9, T10, and ACM. The performance is based on the number of times that, during compression, each model was the best one. The best model in the mixture is the one that, if considered alone and for each symbol, could generate the best compression results among the four. To our knowledge, the only algorithm proposed so far for the compression of whole genome alignments was introduced by Hanus et al. [1] . Their method is based on well-established statistical evolutionary models and on prediction techniques used for lossless binary image compression.. In their proposed evolutionary-based compression scheme, the nucleotides in a MSA block are compressed using the predictions obtained from a nucleotide substitution model, whereas the gaps are encoded independently using techniques from lossless binary image compression. According to them, encoding the nucleotides and the gaps separately is justified by the independence of the two underlying mutational processes and should not introduce an inherent loss to the achievable compression rate. In this section, we will describe briefly the algorithm proposed in [1] .

O O O O O O O O O O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O