Optimizing the Design of a Wind Farm Collection Network In this article we study a network design problem that arises in the exploitation of wind energy. We formulate this problem as a mixed integer programming problem, relate this formulation to other problems in combinatorial optimization, strengthen the formulation and propose cutting planes, and finally present experimental results for real-world instances of the collection network design problem. The design of networks plays an important role in the planning of large systems, especially in the fields of telecommunications, transportation, and energy. In particular it has become necessary to design networks for collecting the energy produced through eco-friendly means, such as wind turbines or solar panels. In this article we study a problem proposed by a Canadian company involved in the design of wind farm collection networks. In Section 2 we give a precise definition of this problem and in Section 3 a formulation of our problem as a mixed integer program. In Section 4 we survey the literature on problems that are similar to ours. In Section 5 we discuss ways of strengthening our model and introduce classes of cutting planes. Finally we present experimental results in Section 6 and our conclusions in Section 7. In this article we have formulated the design of a wind farm collection system as a mixed integer programming problem. We have shown that tightening the mixed integer programming formulation and including some cutting planes at the root node of the branch-and-bound tree enabled one to solve all the instances provided by the industrial partner (except one). There is still much work to do, however. For let D i be a minimum-capacity standard cut for S i

O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I I O O O O O O O B I I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Hybrid tabu search and a truncated branch-and-bound for the unrelated parallel machine scheduling problem We consider the problem of scheduling a number of jobs on a number of unrelated parallel machines in order to minimize the makespan. We develop three heuristic approaches, i.e., a genetic algorithm, a tabu search algorithm and a hybridization of these heuristics with a truncated branch-and-bound procedure. This hybridization is made in order to accelerate the search process to near-optimal solutions. The branchand-bound procedure will check whether the solutions obtained by the meta-heuristics can be scheduled within a tight upper bound. We compare the performances of these heuristics on a standard data set available in the literature. Moreover, the influence of the different heuristic parameters is examined as well. The computational experiments reveal that the hybrid heuristics are able to compete with the best known results from the literature. The parallel machine scheduling problem consists of scheduling a set of jobs N on a set of parallel machines M without interruption. Each job j (index j = 1, ..., n) becomes available for processing at time zero and has to be processed by exactly one out of the m (index i = 1, ..., m) parallel machines. Every machine can process only one job at a time and once a job is started on a certain machine, no preemption is permitted. In general, various classes of parallel machine scheduling problems have been defined to minimize the maximal completion time (C i ) of the machines or the makespan (C max = max i C i ), depending on how the processing times of the jobs are defined. In the case where the processing time p j of a job j is independent of the machine, the problem is known as the identical parallel machine scheduling problem. The problem can be represented by P ||C max using the classification scheme of Graham et al. (1979) . In the case where each machine i has a different speed s i for processing the jobs, it is referred to as the uniform parallel machine scheduling problem (U ||C max ) and the processing time of a job j on machine i is derived as follows: p ij = p j /s i . When the processing time p ij depends on the machine on which the job j is scheduled, but no relationship exists between the machine speeds, the problem is known as the unrelated parallel machine scheduling problem (R||C max ). In this paper, we consider the unrelated parallel machines scheduling (UPMS) problem with the objective of minimizing the maximal completion time of the machines, which is known to be NP-hard (Garey and Johnson, 1979) .. The literature on (identical) parallel machine scheduling is very extended. However, the unrelated parallel machine scheduling problem has been much less studied. In the following paragraphs, an overview is given of the most relevant papers that focus on the unrelated parallel machine problem with a makespan objective. With respect to exact approaches, we mention the work of van de Velde (1993) and Martello et al. (1997) . van de Velde (1993) proposes an exact algorithm using a branch and bound technique and an iterated local search meta-heuristic, which are both based on a dual approach. The paper of Martello et al. (1997) discusses effective lower bounds for the problem based on Lagrangian relaxation and presents some approximate algorithms. Some efficient heuristic approaches are discussed in Glass et al. (1994) , Piersma and Van Dijk (1996) , Srivastava (1998) , Sourd (2001) , Mokotoff and Chrétienne (2002) , Mokotoff and Jimeno (2002) , Ghirardi and Potts (2005) , Monien and Woclaw (2006) , Guo et al. (2007) , Fanjul-Peyro and Ruiz (2010) , Fanjul-Peyro and Ruiz (2011) and Lin et al. (2011) . Glass et al. (1994) perform a comparison between a genetic algorithm, a simulated annealing approach and a tabu search algorithm. Another effective tabu search algorithm is presented in the paper of Srivastava (1998) . Guo et al. (2007) develop hybrid heuristic methods based on simulated annealing, tabu search and squeaky wheel optimization. Piersma and Van Dijk (1996) propose a local search technique with an efficient neighborhood search. Similarly, Sourd (2001) presents two heuristic algorithms based on a large neighborhood search. Monien and Woclaw (2006) present an experimental study on the unsplittable Truemper algorithm and Lin et al. (2011) discuss a two-phase scheduling heuristic with two improvement methods. Mokotoff and Chrétienne (2002) develop a cutting plane algorithm that deals with the polyhedral structure of this UPMS problem. This algorithm is further refined in Mokotoff and Jimeno (2002) , where heuristic algorithms based on partial enumeration are presented. This approach was considered to be the most effective approach for R||C max , together with the recovering beam search of Ghirardi and Potts (2005) . However, Fanjul-Peyro and Ruiz (2010) present a set of iterated greedy local search based heuristics that outperform the aforementioned state-of-the-art procedures. In addition, Fanjul-Peyro and Ruiz (2011) propose meta-heuristics based on a size reduction of the original job-machine assignment problem with which they challenge the well-performing methods of Fanjul-Peyro and Ruiz (2010) . They prove that they can obtain better results by running their algorithms in parallel mode having several physical cores or processors available and keeping the best result at the end.. The contribution of this paper can be summarized as follows. First, we develop three meta-heuristic approaches to solve the UPMS problem. These meta-heuristics are improved by the inclusion of a local search algorithm that takes problem specific information into account. Second, we present a hybridization of these meta-heuristics with a novel truncated branch-and-bound procedure in order to accelerate the search process to nearoptimal solutions. As such, the meta-heuristics are guided in their search process to skip non-promising areas of the solution space. Third, we compare the performances of these heuristics on a standard data set available in the literature. We carefully examine the influence of the different heuristic parameters, such as the backtrack limit and the upper bound strategy, to select the best combination for the hybrid algorithms and check the influence of the problem parameters on the performance of these hybrid meta-heuristics.. The outline of this paper can be summarized as follows. Section 2 gives a brief overview of the stand-alone genetic algorithm, tabu search procedure and the local improvement technique developed for the problem under study. Section 3 describes the principles and the implementation of the truncated branch-and-bound procedure. In Section 4, the computational experiments are discussed, which show the promising results of the hybrid heuristics in solving the unrelated parallel machine scheduling problem. Section 5 summarizes the main conclusions and contributions.. 2 Meta-heuristic solution procedures This paper considered the problem of scheduling a number of jobs on a number of unrelated parallel machines in order to minimize the makespan. We have developed a genetic algorithm and a tabu search algorithm, which were hybridized with a truncated branchand-bound procedure in order to accelerate the search process to near-optimal solutions. Both meta-heuristics were further improved by means of an effective local search algo- rithm and were combined in order to take advantage of both heuristics. The truncated branch-and-bound procedure checked the possibility of scheduling the solutions obtained by the meta-heuristics within an upper bound set on the makespan value. Each individual job-machine assignment was successively examined by the branch-and-bound and altered if necessary. In doing so, the meta-heuristics were stimulated to skip non-promising areas of the solution space. We compared the performances of these heuristics on a standard data set available in the literature. We carefully examined the influence of the different heuristic parameters, such as the backtrack limit and the upper bound strategy, to select the best combination for the hybrid algorithms. These experiments showed that each heuristic performed best with a different upper bound strategy and backtrack limit.. Our computational experiments revealed that the tabu search heuristic outperformed the genetic algorithm, but that hybridization with the truncated branch-and-bound procedure was able to compete with and sometimes outperform the best known results from the literature. For the job correlated and machine correlated processing time instances as well as for the bigger instances, our heuristics were even able to find better solutions than the CPLEX solutions.. An interesting field of future research could be to develop an exact branch-and-bound procedure, relying on the knowledge obtained by the truncated branch-and-bound procedure, and to benchmark this procedure against our hybrid procedures and the methods described in the literature. Moreover, the extension of our procedures to more complex problems, such as the inclusion of setup times is definitely an interesting future research topic.

O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


From Regular to Strictly Locally Testable Languages A classical result (often credited to Y. Medvedev) states that every language recognized by a finite automaton is the homomorphic image of a local language, over a much larger so-called local alphabet, namely the alphabet of the edges of the transition graph. Local languages are characterized by the value k = 2 of the sliding window width in the McNaughton and Papert's infinite hierarchy of strictly locally testable languages (k-slt). We generalize Medvedev's result in a new direction, studying the relationship between the width and the alphabetic ratio telling how much larger the local alphabet is. We prove that every regular language is the image of a k-slt language on an alphabet of doubled size, where the width logarithmically depends on the automaton size, and we exhibit regular languages for which any smaller alphabetic ratio is insufficient. More generally, we express the trade-off between alphabetic ratio and width as a mathematical relation derived from a careful encoding of the states. At last we mention some directions for theoretical development and application. A classical result [13] , often credited to Y. Medvedev [12] , states that every regular language is the homomorphic image of a local language over a larger alphabet called local. In a local language the sentences are characterized by three sets: the initial letters, the final letters and the set of factors of length k = 2. Parameter k is the width of the simplest sliding window device introduced by McNaughton and Papert [11] . The result simply derives from the fact that the set of paths in an edge-labelled graph is a local language over the alphabet of the edges. Considering a finite automaton for the regular language, the local language of accepting paths can be naturally projected on the original language.. Our work originates from two observations. First, in the classic result the alphabet of the local language is larger than the source alphabet, by a multiplicative factor, to be called the alphabetic ratio, in the order of the square of the number of states. The simplicity of sliding window machines and languages is very attractive, but the huge size of the local alphabet in Medvedev theorem makes their application impractical.. Then a natural question concerns the local alphabet in the classical result: how small can the alphabetic ratio be? A small alphabet may, for instance, allow to encode messages from a regular language into an slt language, to be transmitted over a communication channel, so that a more economical sliding window receiver can be used instead of a general finite state machine.. Second, the local languages are a member of McNaughton and Papert's [11] infinite hierarchy of k-strictly locally testable, for short k-slt, languages. Then, by considering k-slt, instead of just 2-slt i.e., local languages, we raise a more general question: what is the minimum alphabetic ratio such that, for some finite parameter k, every regular language is the alphabetic homomorphism of a k-slt language? In that case, how big does the width parameter k need to be? More precisely, our main result, which generalizes Medvedev theorem, expresses the trade-off between two parameters: the alphabetic ratio and the width.. We spend a few lines to show the early but enduring interest for subfamilies of regular languages characterized by some form of local testability, without entering into details.. At the basis of formal language theory, the classical theorem of N. Chomsky and M.P. Schutzenberger characterizes context-free languages by a homomorphism applied to the intersection of a Dyck language and a 2-slt one. Several similar characterizations for other language families have later been proved. In mathematics, the slt languages have been applied in the theory of semigroups by A. De Luca and A. Restivo [1] . In linguistics, a persistent idea is that natural languages can be modeled, at various levels, by locally testable properties. For instance, the psychologist W. Wickelgren [14] made the observation that the set of English words are essentially a 3-slt (finite) language, and several brain scientists (in particular V. Braitenberg [3] ) have suggested that sequences of finite length, such as the factors occurring in a locally testable language, can be easily stored and recognized by certain neural circuits (in particular the synfire chains of M. Abeles) that have been observed in the cortex. In computational linguistics locally testable definitions have proved to be useful at various levels of finite-state models. Many persons (e.g. [7] ) working on language learning models have been attracted by the efficiency of learning algorithms for various types of locally testable languages. Contemporary comparative work on the aural pattern recognition cababilities of humans and animals [10] have called attention to the subregular hierarchies induced by local testability. In mathematical biology, in his seminal article on language theory and DNA [9] , T. Head shows that certain splicing languages are precisely the slt languages.. The paper is organized as follows. After the basic definitions in Section 2, we introduce in Section 3 a new classification of regular languages based on their homomorphic characterization via a k-slt language over an alphabet of size m. In Section 3 we prove a lower bound on the alphabetic ratio. In Section 4 we state and demonstrate a generalization of Medvedev theorem, including a mathematical analysis of the relationship between language complexity, alphabetic ratio, and width. The Conclusion presents an open problem and mentions conceivable developments and applications of the main result. We have generalized Medvedev's homomorphic characterization of regular languages: instead of using as generator a local language over a large alphabet, which depends on the complexity of the regular language, we can use a strictly locally testable language over a smaller alphabet that does not depend on complexity, but just on the source alphabet. We have proved that the smallest alphabet one can use in the generator is the double of the alphabet of the regular language; thus, for instance, four symbols suffice to homomorphically generate any regular binary language.. In the main proof we have offered a specific and fairly optimized construction of the strictly locally testable language, for which we have derived the relationship between the width, the alphabetic ratio, and the complexity of the regular language. In our opinion, the construction should be of its own interest, as a new technique for simulating a NFA by means of a larger, yet strictly locally testable, machine. Our encoding is asymptotically optimal with respect to language complexity, and remains very close to the theoretical optimum for finite values of complexity. But it is an open technical question whether a different construction would yield better values for the alphabetic ratio and the width parameter.. Applications and developments of our result are conceivable in areas where a language characterizationà la Medvedev has been found valuable, as in the next ones. Picture languages. A main family of 2-dimensional languages, the tiling systems [8] , is defined by a 2-dimensional Medvedev characterization. Does our result extend to 2D languages? Context-free languages. Combining our result with the Chomsky-Schutzenberger theorem it should be possible to obtain non-erasing homomorphic characterizations using a small alphabet. Consensual languages [5] . This generalization of finite-state machines motivated by modelling tightly connected concurrent computations uses homomorphism between words as its core mechanism. Information transmission for reducing the receiver cost was already mentioned in the introduction.

O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A note on functional linear regression This work focuses on the problem of linear regression with functional covariate and scalar response. We compare the performance of two linear (parametric) and one nonparametric (kernel) regression estimators via a Monte Carlo simulation study and the analysis of two real data sets. For one of the linear estimators, studied by Cai and Hall (2006) , a new cut-off method is proposed, which improves over the thresholding procedure used by these authors. We consider the linear regression problem with functional auxiliary variable X defined on an interval T = [0, T ] and scalar response Y , that is, we assume that the following model holds. where a ∈ R, b ∈ L 2 (T ) and has zero mean and finite variance. Note that it is possible to express model (1) as Y = a + b, X , where , denotes the L 2 (T ) inner product. From now on it is assumed that a = 0 and T = [0, 1]. We observe n random, independent copies of (X, Y ): (X i , Y i ), for i = 1, . . . , n. This sample is only recorded on an equispaced grid t 1 , t 2 , . . . , t N of [0, T ] whose internodal space is w = T/N. Our aim is to estimate the regression function m(x) := T bx, using the information contained in the sample. This specific problem has been tackled, for instance, by Cardot, Ferraty and Sarda (2003) , Ramsay and Silverman (2005) (Chapter 15) and Cai and Hall (2006) .. As pointed out in Ramsay and Silverman (2005) , minimization of the residual sum of squares yields a regression estimator which adapts perfectly to the sample points but is not very informative. Cai and Hall (2006) express this idea stating that this is an infinite-dimensional problem. This is why it is necessary an intermediate step of smoothing or regularization, which reduces the dimension of parameter b. A standard approach is to expand b and X i using an orthonormal basis. with b j = b, φ j and c ij = X i , φ j . The system {φ j } ∞ j=1 can be, for example, the Fourier basis (see Ramsay y Silverman 2005) or the eigenfunctions of the covariance operator of X (see Cai y Hall 2006) . Using the expansions in (2) and Parseval's identity, the residual sum of squares can be reexpressed as. The regularization step in this case reduces to truncating all the series at, say, the J-th term and using the approximations b. The value of J, with 1 ≤ J ≤ n, is called by Cai and Hall (2006) the frequency cut-off and, in a sense, it may be considered a resolution level. In Section 2 we will specify in detail how to carry out the estimation of the "slope" b and the regression function m, first when {φ j } ∞ j=1 is the Fourier basis and secondly when the series expansion is done in terms of the covariance eigenfunctions. The last part of Section 2 introduces the nonparametric kernel estimator of m. Our aim is to compare the performance of the three resulting regression estimators under the parametric linear model given in (1). This is done in Section 3 via a Monte Carlo study and the analysis of two real data sets. To our knowledge this is the first simulation study comparing the performance of the two parametric, linear regression estimators. The kernel estimator is included in the study in order to quantify the reduction of the error attained by incorporating all the available information on the model. In this work we have considered the linear regression model for functional auxiliary variable X and scalar response Y . We have compared the performance of three regression estimators under this model via a Monte Carlo study and also via the analysis of two real datasets. Two of the estimators considered are parametric, linear, and are characterized by the regularization of X in terms of the trigonometric basis and the covariance eigenfunctions respectively. The choice of the cut-off in these expansions has been addressed and, in the case of the second linear estimator, we have proposed a cut-off selection procedure which improves over previous existent ones. In future work it would be interesting to consider a roughness penalty approach for this linear estimator based on the covariance eigenfunctions expansion. The third regression estimator included in the analysis has been a nonparametric, kernel one. The Monte Carlo study shows that, under the hypothesis of linearity, the parametric estimator based on the eigenfunction expansion outperforms the others. However, the examples with real data suggest that this estimator is highly dependent on the linearity assumption and that the parametric estimator based on trigonometric expansion with roughness penalty is more robust to misspecification of the model.

O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O


Characterization of Fashion Themes Using Fuzzy Techniques for Designing New Human Centered Products Fabric selection plays an important role in fashion garment design. Designers often use both physical and normalized linguistic criteria for fabric selection. Perception and preference of consumers in their specific sociocultural context, expressed by fashion themes or emotional linguistic criteria, affect greatly new fashion product design. Modeling the relationship between linguistic design criteria and fashion themes of a brand image perceived by consumers becomes thus significant. For setting up this model, we first use fuzzy relations and correlation techniques to select the most relevant linguistic design criteria of fabric hand for each specific fashion theme. The selected criteria can then effectively reduce the complexity of the model and interpret consumer perception of fabrics. Finally, we use a weighted aggregation operator to predict the similarity degree between any new product and fashion themes. Compared with other models, the proposed method is more robust and easier to be interpreted with real data collected for design of senior T-shirt fabrics. Traditionally, designers select an appropriate fabric material based on their knowledge and experience for fashion product development. Automation of the fabric selection becomes an interesting research area for garment industry. In a garment company, key features on garment products can generally be characterized using three information sources: 1) physical measures by appropriate devices (numerical data), 2) normalized basic sensory evaluation data (normalized and neutral linguistic data), and 3) consumer perception on garment products (emotional linguistic data). The understanding of the relations between these information sources permits to identify consumer's behaviours on fashion themes and integrate them into the development of new garment products.. In a general context, the relationship between a textile product design and a consumer perception on garment products can be described in Fig. 1 . The textile design criteria include physical properties such as shear stiffness, bending rigidity, thickness, and sensory aspects such as fabric hand evaluation, and visual evaluation. The physical properties are characterized by instrumental measures while the sensory linguistic criteria are determined by classical descriptive sensory evaluation methods 1, 2 at different levels (fabric hand, vision, and olfaction).. However, the perception of consumers on garments products is not limited to the previous sensory linguistic criteria. It considers two levels: (i) Basic perception on products, corresponding to normalized sensory criteria of design. It is independent of the social and cultural context of products and consumers. (ii) Consumer perception in a social and cultural context, 3 including a number of complex concepts such as comfort and well-being. In a specific context, each complex concept is defined by the marketing department according to its brand image, and it can be further decomposed into a set of ambiances or relatively concrete indicators. For example, "well-being" in garments is decomposed into "professional", "sportive", "protection", and "leisure". Much research has been done for modelling the relationship between physical criteria and sensory descriptors to predict tactile basic perception according to mechanical properties. 4, 5, 6, 7, 8 However, consumer perception in a specific socio-cultural context such as fashion themes is not taken into account in these methods. In practice, fabrics are generally selected by garment designers according to a number of physical and basic sensory criteria such as fabric hand and fabric appearance. The results of selected fabrics do not necessarily correspond to the desired fashion themes or the required brand image of garment products. Hence, the characterization of the relationship between basic sensory criteria and fashion themes is particularly significant for the fashion garment design. The relevancy of sensory criteria to a given fashion theme is in fact usually considered as one of the most important elements for improving the brand image of garment products. In a new garment design, the conformity of perception between garments and their brand image enhances consumers' loyalty on the brand and reduces the risk and cost of design for the company.. In the existing literature, modelling of a complex relationship or system is often performed using artificial intelligent 9,10,11 from a set of representative learning data. However, in practice, these methods are not efficient in most of industrial applications, in which the quantity of learning data is often limited due to the restrictions of experimental cost and time. In this paper, we propose a new method to model the relationship between basic sensory criteria and fashion themes using fuzzy modeling techniques 12, 13, 14, 15 and aggregation techniques. 16, 17 It permits designers to find the most relevant basic sensory criteria at different levels (fabric hand, style, and color, etc.) for a given fashion theme in order to produce the most appropriate prototype meeting specific requirements of consumers. We focus on the relationship between fabric hand criteria and fashion themes. First, sensory evaluation is organized to obtain similarity degrees given by a number of consumers between each fashion theme and prototypes. Then, these similarity degrees are aggregated as fuzzy sets, characterizing the evaluation score of all consumers. A round composition of fuzzy relation is applied to calculate the similarity between sensory criteria and fashion themes. A ranking list of these criteria for each theme is obtained. Then, an overall correlation coefficient and a procedure are proposed to remove the irrelevant sensory criteria. Finally, the predicted similarity degree between sensory criteria and fashion themes is modelled by a weighted aggregation operator. This model can effectively help designers to predict the fashion themes of garment products from specific fabric samples and select the most appropriate fabrics according to consumer requirements on fashion themes. This paper presents an original method for predicting a fashion theme or emotional linguistic attribute from basic sensory linguistic data. This method uses both fuzzy relation and correlation computations for selecting a small set of relevant sensory descriptors and combines the relevant descriptors using a weighted aggregation operator to predict the similarity degree between new fabric and fashion themes. It helps designers to predict the consumer perception from known fabrics and design new fashion oriented products enhancing the brand image of the company. Compared with other prediction methods such as linear regression, the proposed method is more robust and interpretable owing to its capacity of treating uncertainty. This work can also be extended to support other fashion oriented product design.

O O O O O O O O O O O O O B I O O O O O B I I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Amplify-forward relaying for multiple-antenna multiple relay networks under individual power constraint at each relay This article considers the design of an optimal beamforming weight matrix of multiple-antenna multiple-relay networks. It is assumed that each relay utilizes the amplify and forward strategy, i.e., it multiplies the received signal vector by a matrix, dubbed the relay weight matrix, and forwards the resulting vector to the destination. Furthermore, we assume that the source and the destination have the same number of antennas and that each transmit antenna is virtually paired to a different destination antenna. The relay weight matrices are concurrently designed to optimize the mean square error (MSE) criterion at the destination, assuming each relay node is subject to a power constraint. Accordingly, it is demonstrated that this problem can be cast as a convex optimization problem in which the individual power constraints are tackled by employing the method of Lagrange multipliers in two stages. First, the relay gain matrix is computed analytically in terms of Lagrange dual variables, thereby converting the original problem into a scalar optimization problem. Then, these scalar variables are computed numerically. The proposed scheme is evaluated through simulation with various numbers of relays and antennas to obtain MSE and bit error rate (BER) metrics and it is shown that the resulting MSE and BER achieved through using the proposed method outperforms that of MMSE-MMSE method introduced by Oyman et.al., which is regarded as the best known method for the underlying problem. It is well established that in most cases relaying techniques provide considerable advantages over direct transmission, provided that the source and relay cooperate efficiently. The choice of relay function is especially important as it directly affects the potential capacity benefits of node cooperation [1] [2] [3] [4] [5] . In this regard, two relaying methods, amplify-forward (AF) [6, 7] and estimate-forward [8, 9] , are extensively addressed in the literature. As the names imply, the former just amplifies the received signal but the latter estimates the signal with errors and then forwards it to the destination.. It has been shown that increasing the number of relays has the advantage of increasing the diversity gain and flexibility of the network; however, it renders some new issues to arise [10] . For instance, the relaying algorithm and power allocation across relays should be addressed is such cases. Relay selection [11, 12] and power allocation [13, 14] are two well-known methods when dealing with the power management issues.. The capacity and reliability of the relay channel can be further improved by using multiple antennas at each node. The use of relays together with using multiple antennas has made it a versatile technique to be used in emerging wireless technologies [15] [16] [17] [18] [19] [20] . Relaying strategies for the multi-antenna multiple-relay (MAMR) networks is more challenging than single-antenna networks, since in addition to scaling and phase operations, matrix operations should also employed at the relays.. AF MIMO relay systems have drawn considerable attention in the literature due to their simplicity and ease of implementation. In this regard, a plethora of works are devoted to finding a proper relaying strategy for AF MAMR networks. In [21] , the idea of linear distributed multi-antenna relay beamforming is introduced where each relay performs a linear reception and transmission in addition to output power normalization. In this article, K single antenna transmitted independent data streams to their respected single antenna receivers. The linear operations suggested in this article are matched filter, zero forcing, and minimum mean square error (MMSE). They are briefly called MF-MF, ZF-ZF, and MMSE-MMSE schemes, respectively. In [22] , a method based on QR decomposition is suggested which works better than the ZF-ZF scheme. Combinations of various schemes are also considered in [22] . For example in ZF-QR scheme, relays perform ZF algorithm in reception and QR algorithm (channel triangulation) in transmission.. In [23] , the so-called incremental cooperative beamforming is introduced and it is shown that it can achieve the network capacity in the asymptotic case of large K with a gap no more than O(1/log(K)). However, this method is not suited when few relays are incorporated since this method only works properly when the number of relays tends to infinity.. In [24] , a wireless sensor network that is composed of some multi-antenna sensors aimed to transmit a noisy measurement vector parameter to the fusion centre is formulated as a MAMR network. Moreover, it is assumed that the second hop associated with the resulting MAMR network has a diagonal channel matrix and the destination noise is small enough to be ignored. The current manuscript is actually an extension of [24] since neither the channel matrices need to be diagonal nor the destination noise is restricted to be zero.. In [25] , it is shown that an MAMR network with singleantenna source and destination can be transformed to a single-antenna multiple relay (SAMR) network by performing maximal ratio combining at reception and transmission for each relay nodes. This enables the network beamforming introduced in [14] to be readily employed.. In [26] , by using ZF-ZF scheme, an MAMR network with M single-antenna source-destination pairs is transformed to M SAMR networks to which network-beamforming proposed in [14] is applied.. In [27] , the relay gain matrices are obtained by maximizing the MSE at destination restricting the received power at the destination. In [28] , a linear relaying scheme for an MAMR network fulfilling the target SNRs on different independent substreams transmitted from each source antennas is proposed and the powerefficient relaying strategy is derived in closed form. In [29] , a nearly optimal relaying scheme is proposed to maximize the mutual information between the source and the destination under total relay power constraint.. In this article, the problem of MAMR network with multiple antennas at source and destination with individual relays power constraints is formulated as a convex optimization problem. The optimum relay gain matrices are obtained by solving the optimization problem using Lagrange dual variables method. This relays gain matrices are obtained in terms of K scalar variables where K is the number of relays. Then those variables are computed numerically. As noted before, the articles that investigate this configuration either suggest the relay gain matrix heuristically or concern another constraint such as a limited power constraint at the destination, the destination quality of service or the sum power of relays. In our opinion, the limited power for each relay is a more realistic assumption, because each relay in the network has its own power supply and unused power for each relay cannot be used by other relays. In the same manner as [26] [27] [28] [29] , complete CSI is considered to be available for optimum relay design. The optimization can be performed at the destination, and then the processing results are fed back to the relays. Although the closed form formula is not obtained but a parametric relation form of the relay gain matrices are derived. These parameters can be calculated either numerically or heuristically. A simpler form of the relay gain matrices is derived for the two relay case. The initial works on this issue are first addressed in [30] while the optimal solution is not fully treated there. Figure 1 illustrates a typical MAMR relay network system in which there are M single-antenna sources, trying to send independent data streams through K multiantenna relays to their affiliated single-antenna destinations. In fact, the aim is to send independent data streams from each source antenna to the corresponding single-antenna destination. Thus, each single-antenna destination can merely apply a simple scaling to its received signal and the integral part of the interference cancellation process must be performed at multiantenna relays. A relay network with multiple relay each having multiple antennas is considered. The relay matrices are found by solving an optimization problem. In this problem, the MSE at the destination is minimized and the individual relay output power considered as constraint. The Lagrange dual problem is then obtained to compute the Lagrange dual variables numerically. Solving Lagrange dual problem (22) is simpler than the primary problem (9) . This is because, solving Lagrange dual problem requires the calculation of K scalar unknown variables but in primary problem case, K unknown N × N matrices needs to be computed. So, the dimension of the problem decreases N × N times.. Two numerical methods based upon SINR and BER are introduced to obtain the optimum value of η that is employed for the actual simulation of the proposed optimum scheme.. The system with the proposed optimum, MMSE-MMSE and ZF-ZF schemes, is simulated and the average BER as well as MSE at destination are obtained. The results show that the proposed optimum scheme outperforms MSE-MSE and ZF-ZF schemes by a good margin. Indeed, analytical computation of Lagrange dual variables and considering normalization parameter η as the optimization problem variable can be considered for future investigations.

O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Linear Gaussian affine term structure models with unobservable factors: Calibration and yield forecasting This paper provides a significant numerical evidence for out-of-sample forecasting ability of linear Gaussian interest rate models with unobservable underlying factors. We calibrate one, two and three factor linear Gaussian models using the Kalman filter on two different bond yield data sets and compare their out-of-sample forecasting performance. One step ahead as well as four step ahead out-of-sample forecasts are analyzed based on the weekly data.When evaluating the one step ahead forecasts, it is shown that a one factor model may be adequate when only the short-dated or only the long-dated yields are considered, but two and three factor models performs significantly better when the entire yield spectrum is considered. Furthermore, the results demonstrate that the predictive ability of multi-factor models remains intact far ahead out-of-sample, with accurate predictions available up to one year after the last calibration for one data set and up to three months after the last calibration for the second, more volatile data set. The experimental 1 corresponding author. Email: paresh.date@brunel.ac.uk. Phone: +44 1895 265613, Fax: +44 1895 269732 1 data denotes two different periods with different yield volatilities, and the stability of model parameters after calibration in both the cases is deemed to be both significant and practically useful. When it comes to four step ahead predictions, the quality of forecasts deteriorates for all models, as can be expected, but the advantage of using a multi-factor model as compared to a one factor model is still significant.In addition to the empirical study above, we also suggest a nonlinear filter based on linear programming for improving the term structure matching at a given point in time. This method, when used in place of a Kalman filter update, improves the term structure fit significantly with a minimal added computational overhead. The improvement achieved with the proposed method is illustrated for out-of-sample data for both the data sets.This method can be used to model a parameterized yield curve consistently with the underlying short rate dynamics. Exponential affine term structure models is one of the oldest and the most widely studied class of dynamic interest rate models. The main advantage of these models is the fact that the yields can be expressed as affine functions of the short rate. The exponential affine term structure models are often classified into three categories:. • Gaussian affine models. The single factor linear model proposed in Vasicek (1977) is a Gaussian affine model and was the first model for which closed-form formulae for bond prices were obtained. All the state variables in these types of models have constant volatilities. A multi-factor Gaussian affine model is discussed in Babbs and Nowman (1999) . Extensions of the Gaussian affine models to match the current term structure are discussed in Hull and White (1990) , Hull and White (1993) and Hull and White (1994) . The Gaussian models have a high degree of tractability and a variety of products can be priced in closed-form with these types of models. Recently, closed-form formulae for swaption pricing under a multi-factor Gaussian affine model have been reported in Schrager and Pelsser (2006) .. • CIR affine models. Models of this type were first proposed in Cox et al. (1985) and were extended to multi-factor case in Beaglehole and Tenny (1991) . All the state variables in these models have CIR-type square root volatilities. Unlike the Gaussian models, the interest rate is guaranteed to remain non-negative provided it starts from a nonnegative value.. • A three-factor affine family. This family represents the models that mix Gaussian and CIR type state variables; see Balduzzi et al. (1996) , Rhee (1999) and Longstaff and Schwartz (1992) for examples.. A general framework for multi-factor affine term structure models was proposed in Duffie and Kan (1996) . An empirical comparison of several different short rate models appears in Chan et al. (1992) .. In this paper, we model the behavior of government bond yields by using linear Gaussian term structure models. Cross-sectional as well as time-series data of gilt yields is used for calibration, i.e. each discrete measurement in time series consists of a cross-section of gilt yields. The short rate is assumed to be an affine function of unobservable state variables. Each yield is assumed to be the sum of the theoretical yield for the corresponding time to maturity and a zero mean stochastic disturbance. Since the theoretical yield is affine in the short rate, this set-up gives a linear state space system. The Kalman filter can then be used to calibrate the model using noisy yield measurements.. There are many accounts of Kalman filtering-based calibration and forecasting of a time series, including detailed treatments in Harvey (1989) and in Durbin and Koopman (2002) . A brief overview of Kalman filteringbased calibration for interest rate models appears in (James and Webber (2000) , chapter 18). In empirical research reports concerning this issue in detail, Babbs and Nowman (1999) and Lund (1997) estimate the generalized Vasicek models using Kalman filtering. Other filtering applications in the area of interest rate modelling are reported in Rossi (2004), Jegadeesh and Pennacchi (1996) , Ball and Torous (1996) , Duan and Simonato (1995) and Gravelle and Morley (2005) . The work presented here follows Babbs and Nowman (1999) in that we use linear Gaussian transition equations with constant prices of risk.. In contrast with the previously cited research, the emphasis of this paper is short and medium term out-of-sample forecasting of future yields. While a lot of empirical research has been carried out to predict the term structure and to test the expectations hypothesis (see, e.g. Lanne (2000) and references therein), relatively little empirical work has been done in testing the out-ofsample, short term forecasting ability of an affine term structure model with unobservable factors. We provide a significant numerical evidence of predictive ability of a simple linear Gaussian model with unobservable state variables. A forecasting model may be useful in a variety of situations, e.g. in predicting downside risk of future performance of a gilt portfolio or in generating scenarios for a stochastic programming based optimization. It is shown in Reisman and Zohar (2004) that using re-balancing based on a short term prediction of the term structure can significantly improve the returns from a bond portfolio. For multi-factor models, we demonstrate that the predictive ability remains unimpaired for a long period after calibration.. Besides this empirical work based on the application of existing theoretical results, a new linear programming-based heuristic is suggested for estimating the unobservable states. This heuristic filter is aimed at improving the yield curve matching without increasing the calibration effort or the model complexity. This provides a very useful alternative to Nelson-Siegel type static yield curve models since it is consistent, by definition, to the assumed interest rate dynamics 2 . The rest of the paper is organized as follows. Section 2 provides a brief overview of linear Gaussian affine term structure model used in this work.. Section 3 presents a similarly brief overview of Kalman filtering-based calibration. Section 4 offers a detailed empirical analysis for calibration of a one, 2 The consistency of a parameterized yield curve is defined in section 5.. two and three factor linear Gaussian models and yield prediction using all these models. Section 5 suggests a new linear programming heuristic for state estimation in a linear Gaussian model to improve the matching of the initial term structure. This heuristic is shown to out-perform the Kalman state update in term structure matching at a fairly small added computational cost.. Finally, section 6 summarizes the contributions of this work. An extensive empirical study were carried out for Kalman filtering-based calibration of linear Gaussian interest rate models using UK gilt yields data and US treasury data. The following conclusions can be drawn from the evidence presented:. • A multi-factor model outperforms a one factor model in short term yield forecasting both in-sample and out-of-sample when the entire yield curve is considered. When only closely spaced (in logarithmic terms) yields are considered, even a one factor model is seen to be adequate for one-step ahead prediction.. • A three factor model performs better than a two factor model for UK gilt data and performs marginally worse for US treasury data. While this is a very limited evidence, it appears that a more parsimonious two factor model may actually be preferable in a volatile environment as compared to a three factor model. The short-term predictive ability of a linear Gaussian model is reasonably stable, with the mean relative error remaining at or below 2.47% for UK gilt data up to one year after calibration and at or below 3.50% for US treasury data up to three months after calibration, in the two factor case.. • In multi-step ahead prediction with four time-steps, the predictive ability of models deteriorates, as can be expected, although the performance of the multi-factor models may still be satisfactory from a portfolio management point of view. As far as one factor model is concerned, the error in multi-step ahead prediction is not much worse than the error in one-step ahead prediction.. • A valid criticism of Gaussian model is that they allow negative values of interest rate with positive probability. In forecasting terms, this did not pose a problem, since none of the conditional expectations computed (around 10000 in-sample and out-of-sample yield predictions in all). were close to zero. This may obviously be a problem in markets with very low interest rates (e.g. in modelling Japanese bonds at the time of writing).. • Further, a new linear programming state update heuristic was suggested to improve fit to a given term structure, while still operating in

O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


CompareSVM: supervised, Support Vector Machine (SVM) inference of gene regularity networks Background: Predication of gene regularity network (GRN) from expression data is a challenging task. There are many methods that have been developed to address this challenge ranging from supervised to unsupervised methods. Most promising methods are based on support vector machine (SVM). There is a need for comprehensive analysis on prediction accuracy of supervised method SVM using different kernels on different biological experimental conditions and network size.We developed a tool (CompareSVM) based on SVM to compare different kernel methods for inference of GRN. Using CompareSVM, we investigated and evaluated different SVM kernel methods on simulated datasets of microarray of different sizes in detail. The results obtained from CompareSVM showed that accuracy of inference method depends upon the nature of experimental condition and size of the network. Conclusions: For network with nodes (<200) and average (over all sizes of networks), SVM Gaussian kernel outperform on knockout, knockdown, and multifactorial datasets compared to all the other inference methods. For network with large number of nodes (~500), choice of inference method depend upon nature of experimental condition. CompareSVM is available at http://bis.zju.edu.cn/CompareSVM/. To summarize, CompareSVM can be used to infer GRN with high accuracy (AUC) for networks (<200) with SVM Gaussian kernel for biological datasets (knockout, knockdown, multifactorial and all). For large network, choice of algorithm depends upon the type of biological condition. There had been variation in prediction accuracy in all inference methods, therefore; prediction should be limited for simple network. Future work is needed for the development of semi-supervised methods that have the ability to predict target of TF which have no prior known targets. We have developed a tool based on SVM to infer GRN, and used it to compare four widely used kernels to evaluate prediction accuracy on three biological conditions and combination of all. Our results are in agreement with already published report [2] , but as the size of nodes exceeds 200, the unsupervised method (CLR) slightly outperforms the supervised method (SVM) in multifactorial experimental condition. We have also observed that the large number of repeats is required on network of different sizes to accurately estimate the prediction accuracy of these methods.. The most important observation from this evaluation is that there is no one universal method suitable for inference of GRN for all biological conditions. The suitability of inference method depends up size and type of expression profiles of microarray data. On average, the unsupervised methods achieve low accuracy with exception of multifactorial dataset. Although, Pearson correlation is Figure 4 Prediction accuracy (AUC) average overall network sizes of unsupervised and supervised methods on knockout, knockdown, multifactorial and average data generated by GeneNetWeaver and extracted from E. coli. For each network of size: 10, 30, 50, 100, 150, 200 and 500. 10 networks were generated for each size and experimental condition.. comparatively accurate enough and even does not require parameter optimization. Unsupervised methods are suitable for simple and small problem but are fast and do not require any prior training for inference of GRN.. The limitation of CompareSVM due to its supervised nature approach is inability to predict target of TF which have no prior known targets. The performance of the CompareSVM depends upon the list of known target genes; therefore, TF with incomplete list of interaction will produce poor local models. This may direct the research into hybrid model which are based on supervised and semi-supervised methods which can address this challenge. We evaluated the prediction accuracy of unsupervised method (CLR) using estimation (Pearson, Spearman) with comparison to supervised approaches (SVM using four different kernels that have been mentioned in the methods section) using microarray simulated data. The prediction accuracy was measured by the area under the Receiver Operator Characteristics curve (AUC) for prediction methods (supervised and unsupervised) using three different experimental conditions (knockout, knockdown and multifactorial) and also average of three experimental conditions (all). Networks of node size 10, 30, 50, 100, 150, 200 and 500 were extracted from E. coli and expressed data were simulated by GeneNetWeaver. Gene-NetWeaver generated number of nodes equal to number of experiments. 10 networks were simulated and generated by GeneNetWeaver for a given size of network and experimental condition, therefore each evaluation was tested ten times. The large standard deviation was observed in predication accuracy of all methods and experimental conditions. For small network, the accuracy varies from random networks to close perfect prediction.. In unsupervised method CLR, the correlation method (Spearman, Kendall) performed poorly for knockdown experimental condition. In knockout experimental condition, same pattern was observed, but as the network size approached to 500, the predication accuracy approached to 70% as shown in Figure 3 . In multifactor experimental condition, it performed exceptionally well and slightly better than supervised methods. But its prediction performance was not suited for network with small number of nodes even in multifactorial experimental condition as shown in Figure 3 .. CLR, the correlation method Pearson has more dramatic impact on prediction accuracy on knockout experimental condition compared to Spearman as changes in the expression values are not monotonically distributed, but both performed poorly on the knockdown experimental condition with comparison to SVM. Pearson performed very well as the size of network increased as well as on the network with few nodes compared to Spearman correlation method. Although, supervised methods had higher prediction accuracy for small network, but as the size of network increased in multi-factorial experimental condition, the CLR (Pearson, spearman) performed better as shown in Figure 3 .. In SVM, the three (linear, Gaussian and poly) out of four kernel methods outperformed the unsupervised methods with the exception of multifactorial experimental condition. Although, for network size up to 200 nodes, performance was better than unsupervised methods in multifactorial condition as well. In all experimental conditions, the Gaussian kernel was most consistent and had least standard derivation. Polynomial kernel achieved highest accuracy in knockout experiments as network size approached to 500 nodes and in knockdown experimental condition, linear kernel marginally better than polynomial kernel in prediction accuracy for large networks as shown in Figure 3 . In multifactorial condition, these three kernels (linear, Gaussian and poly) performed well for networks of small size up to 200 nodes and slightly underperformed as network size approached 500 with compared to unsupervised methods. Sigmoid kernel had poor performance in all experimental conditions and its performance was not better than random guess.. Gaussian kernel possibly is the best option for prediction of GRN from microarray data as it has high accuracy and less standard derivation on small datasets compared to all other inference methods. It also has overall best performance for all biological conditions (Figure 4) .. Linear kernel is a fast and does not require any additional parameter to optimize the prediction with contract to all other kernel methods. Even with its simplicity compare to other complex kernels, it still performs almost same as other complex kernel methods and ranked 2 nd in overall prediction accuracy. It slightly outperformed the Gaussian kernel as the size of node reaches 500 in three biological experimental conditions. Our analysis indicates that suitability of a method for predication of GRN depends on biological condition and size of network. For small networks (<200), all biological conditions can be inferred by Gaussian kernel with high prediction accuracy ( Figure 4 ). As the size of network approaches to 500, the CLR (Pearson, Spearman) outperforms the all other methods in multifactorial condition, but SVM (poly nominal kernel) is best suited for knockout data and SVM (linear kernel) for knockdown. Gaussian overall performed better in all experimental conditions and sizes of the network as shown in Figure 4 . The structure and topology of gene regularity network (GRN) is essential to understand the mechanism that how gene transcription factor (TF) regulates genes expression and consequent cellular behaviors such as development, differentiation and response to stimuli. The deregulation of these networks results in change the genes expression and leads to implication in ontogenesis and tumor progression [1] . Technologies like high through output sequencing and microarray offer a great deal of information about individual genes, but the reconstruction of GRN based on genome wide data still remains a big challenge. The prior biological knowledge along with genomics and post genomics data has given rise to supervised techniques to solve this challenge.. Many computational methods have been developed to infer GRN, mostly using unsupervised approaches.. Recently, there has been surged in supervised approaches due to identification of large number of transcription factors and their targets which enabled to have sufficient data to train supervised models. The most recent and largest study published by Maetschke SR [2] has clearly shown that the supervised approaches outperform unsupervised and semi-supervised approaches for inference of GRN. They compared the prediction accuracy of 17 unsupervised methods with support vector machine (SVM) and reported it outperform unsupervised methods in different experimental conditions except in knockout experiment by Z-score method. Similar study to evaluate supervised inference of GRN was done by Mordelet and Vert [3] . They compared supervised techniques to context likelihood to relatedness (CLR), algorithms for the reconstruction of accurate cellar networks (ARACNE), relevance networks (RN) and a Bayesian networks (BN) on an E. coli benchmark data set by Faith et al. [4] .. Any algorithm for supervised learning can be used in principal for interference of GRN. We have used state of art SVM algorithm for inference of GRN for different biological conditions and network size. Ben-Hur and Noble [5] provided a very simple method where a local model is used to estimate the prediction of interacting partners of each protein in the network. This in turn is used for all local models and then combine together to predict edges throughout the network. We used this concept to estimate local model for each TF based on their expression profiles and the genes regulated by TF from other genes. All models were than combined to rank candidate regulatory relationship between TF's and all genes in the genome [3] .. In this article, we investigated and compared prediction accuracy of SVM using 4 widely used kernel functions on wide range of networks and experimental data types (multifactorial, knockout, knockdown and all) with comparison to unsupervised methods. The aim of our study was to identify a suitable algorithm for inference of GRN with respect to each experimental condition and network size. We have also developed a simple tool and given the name "CompareSVM" (Supervised inference of Regularity Networks and comparison) for inference and comparison of GRNs (see Additional file 1 to install software and Additional file 2 for instructions how to install and run the software). Although many supervised algorithms have been developed, but we used SVM as it has been successfully applied to inference of GRN. Compar-eSVM unlike unsupervised techniques requires a list of known regulation relationship between TF and target genes in addition to gene expression data. This is standard method in the field of machine learning for supervised techniques, the known/prior knowledge is used to train a classifier to be able to predict the unseen data. This limitation is of not much concern due to huge surge in the regulation databases for many species and specially availability of well documented model species.. GeneNetWeaver is an open source tool for in silico benchmark generation and profiling of network inference. It uses in vivo microarray compendia along with synthetic data to access the performance of network inference methods. It enables us to simulate and generate dataset for gene knockdown, knockout and multifactorial microarray expression profiles for E. coli. Knockdown experiment refers to the technique in which expression of one or more genes is reduced. Knockout experiment is a genetic technique in which one of the gene expression is completely made inactive and multifactorial experiment refers to the technique where small number of genes expression values are perturbed by a small random number.. We have tested CompareSVM on E. coli data extracted by GeneNetWeaver of different nodes (number of genes) ranging from 10 to 500 (see Additional files 3 and 4). Our aim was to identify which inference method worked better for a given experimental condition profiles.. We also investigated the accuracy of this method as the size of the network is increased for a certain biological condition profiles. We have identified that for small networks of all biological condition, SVM (Linear, Gaussian, Polynomial) outperform unsupervised inference methods, whereas, for large network SVM ( Linear, Gaussian, Polynomial) also perform better with the expectation of multifactorial experimental condition.

B O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On quantile quantile plots for generalized linear models The distributional assumption for a generalized linear model is often checked by plotting the ordered deviance residuals against the quantiles of a standard normal distribution.Such plots can be difficult to interpret, because even when the model is correct, the plot often deviates substantially from a straight line. To rectify this problem García Ben and Yohai (2004, J. Comput. Graph. Stat. 13: 36-47) proposed plotting the deviance residuals against their theoretical quantiles, under the assumption that the model is correct. Such plots are closer to a straight line, when the model is correct, making them much more useful for model checking. However the quantile computation proposed in García Ben and Yohai is, in general, relatively complicated to implement and computationally expensive, so that general purpose software for these plots is only available for the Poisson and binary cases in the R package robust. As an alternative the theoretical quantiles can efficiently and simply be estimated by repeatedly simulating new response data from the fitted model and computing the corresponding residuals. This method also provides reference bands for judging the significance of departures of QQ-plots from ideal straight line form. A second alternative is to estimate the quantiles using quantiles of the response variable distribution 1 according to the estimated model. This latter alternative generally has lower computational cost than the first, but does not yield QQ-plot reference bands. In simulations the quantiles produced by the new methods give results indistinguishable from the original García Ben and Yohai quantile computations, but the scaling of computational cost with sample size is much improved so that a 500 fold reduction in computation time was observed at sample size 50000. Application of the methods to generalized linear models fitted to prostate cancer incidence data suggest that they are particularly useful in large dataset cases that might otherwise be incorrectly viewed as zero-inflated. The new approaches are simple enough to implement for any exponential family distribution and for several alternative types of residual, and this has been done for all the families available for use with generalized linear models in the basic distribution of R. Consider a Generalized Linear Model (GLM) for n response variable observations y i , each with expectation µ i , g(µ i ) = X i β, y i ∼ EF(µ i , φ), where X i is the ith row of a model matrix, dependent on known covariates; β is a vector of coefficients to be estimated; φ is a scale parameter; and EF(µ i , φ) is some exponential family distribution dependent on µ i and a known or unknown scale parameter φ. β is estimated by maximum likelihood or maximum penalized likelihood estimation (for example if the model is a generalized additive model, or if some elements of β are to be treated as random effects), while φ can be estimated independently, typically using estimates based on either the model deviance or the Pearson statistic.. After estimation, all the information available for model checking is contained in the residuals (although there is little of it in residuals for a binary response; e.g. Cox and Snell (1989, page 73) . The raw residuals are r i = y i −μ i , whereμ i is the model estimate of µ i . Because the distribution of these depends in a complicated way on the fitted model they are difficult to use for model checking unless the response is Gaussian. Therefore it is usual to standardize the residuals, so that they will have constant variance and near constant distribution, if the model is correct. Two common standardizations are those used to produce Pearson and deviance residuals.. Pearson residuals utilize the fact that for any exponential family distribution, there exists a known function, V such that var(y i ) = V (µ i )φ. In consequence the Pearson residuals, often have a distribution that is close to normal. The latter fact prompts the plotting of sorted deviance residuals against the quantiles of a standard normal, for model checking purposes.. However, there are many applications of GLMs for which such plots show substantial deviation from a straight line, even when the model is correct (e.g. García Ben and Yohai, 2004) .. Modelling a response consisting of low counts is the most obvious example. García Ben and Yohai (2004) propose avoiding the problems with normal QQ plots of the deviance residuals, by computing the empirical cumulative distribution function,F D of the deviance residuals, conditional on the fitted model. They then generate n quantiles d * i = F −1 D ((i − 0.5)/n) against which the sorted deviance residuals d i should be plotted. This should always yield a plot that is 'close' to a straight line, if the model is correct.. The García Ben and Yohai (2004) are never worse than normal QQ plots of the deviance residuals, and offer a substantial improvement in situations in which normal QQ plots are curved even when the model is correct. However the García Ben and Yohai method could usefully be improved in two respects. Firstly, the method by which they compute the quantiles is moderately complicated to implement, and is relatively computationally expensive for a checking method.. Specifically, in general, each evaluation ofF D requires n evaluations of the quantile function and cumulative distribution function for the exponential family used in the model. i.e. each evaluation has O(n) computational cost. In the absence of analytic shortcuts, accurate computation of the d * i will require tabulatingF D at O(n) points. Hence in general the computational cost of the d * i is O(n 2 ). Only the Poisson and binary cases seem to have been implemented in the R package robust (Wang, et al, 2010) , and it is a relatively daunting task to implement all the other distributions routinely used with GLMs.. The second issue with the García Ben and Yohai (2004) QQ plots is that for count data there can be substantial random deviations from the ideal straight line, corresponding to discrepancies between the observed and expected number of observations of each count. This is easiest to see for binary data, where any deviation between the number of 1s observed and expected will cause some positive residuals to be assigned to negative quantiles, or vice-versa. Since these random discrepancies can sometimes be quite large, it would be useful for the plots to be accompanied by reference bands, indicating deviations that are larger than expected.. The remainder of this note shows how to approximate the d * i simply in O(kn log(n)) computer time and how to compute reference bands, where k is a constant of order 10-100. The methods will be applicable to raw, Pearson or deviance residuals.

O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B


Impending Brain Informatics Research from Web Intelligence Perspective Web Intelligence (WI)-based portal techniques (e.g. the wisdom Web, data mining, multi-agent, and data/knowledge grids) will provide a new powerful platform for Brain Sciences. New understanding and discovery of the human intelligence models in Brain Sciences (e.g. cognitive science, neuroscience, brain informatics) will yield new WI research and development. In this paper, we briefly investigate three high-impact research issues as well as present a case study, to demonstrate the potentials of Brain Informatics (BI) research from WI perspective. Brain Informatics (BI) is a new interdisciplinary field to study human information processing mechanism systematically from both macro and micro points of view by cooperatively using experimental cognitive neuroscience and WI-centric advanced information technology. In particular, it attempts to understand human intelligence in depth, towards a holistic view at a long-term, global field of vision to understand the principle, models, and mechanisms of human multi-perception, language, memory, reasoning, decision-making, planning, problem solving, learning, discovery, and creativity. 30 On one hand, although brain sciences have been studied from different disciplines such as cognitive science and neuroscience, BI represents a potentially revolutionary shift in the way that research is undertaken. It attempts to capture new forms of collaborative and interdisciplinary work. In this vision, new kinds of BI methods and global research communities will emerge, through infrastructure on the Wisdom Web and Knowledge Grids that enable high speed and distributed, largescale analysis and computations, and radically new ways of sharing data/knowledge repositories. On the other hand, some of these lessons in cognitive science and neuroscience are applicable to novel technological developments in BI, yet others may need to be enhanced or transformed in order to manage and account for the complex and possibly more innovative practices of sharing data/knowledge that are made technically possible by the Wisdom Web and Knowledge Grids. As two related emerging fields of research, BI and WI mutually support each other. Their synergy will yield profound advances in the analysis and understanding of data, knowledge, intelligence and wisdom, as well as their relationships, organization, and creation process. When BI meets WI, it is possible to have a unified and holistic framework for the study of machine intelligence, human intelligence, and social intelligence.. Brain Informatics emphasizes on a systematic approach for investigating human information processing mechanisms, including measuring, collecting, modeling, transforming, managing, and mining multiple human brain data obtained from various cognitive experiments by using fMRI and EEG. Multi-aspect analysis in multiple human brain data sources is an important methodology in BI. The proposed methodology attempts to change the perspective of cognitive/brain scientists from a single type of experimental data analysis towards a holistic view at a longterm, global field of vision to understand the principle, models, and mechanisms of human information processing. New generations of WI research and development need to understand multi-nature of intelligence in depth. The recently designed instrumentation (fMRI etc.) and advanced IT are causing an impending revolution in both WI and BI, making it possible for us to understand and develop humanlevel WI.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O