Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces We propose a novel method of dimensionality reduction for supervised learning problems. Given a regression or classification problem in which we wish to predict a response variable Y from an explanatory variable X, we treat the problem of dimensionality reduction as that of finding a low-dimensional "effective subspace" for X which retains the statistical relationship between X and Y . We show that this problem can be formulated in terms of conditional independence. To turn this formulation into an optimization problem we establish a general nonparametric characterization of conditional independence using covariance operators on reproducing kernel Hilbert spaces. This characterization allows us to derive a contrast function for estimation of the effective subspace. Unlike many conventional methods for dimensionality reduction in supervised learning, the proposed method requires neither assumptions on the marginal distribution of X, nor a parametric model of the conditional distribution of Y . We present experiments that compare the performance of the method with conventional methods. Many statistical learning problems involve some form of dimensionality reduction, either explicitly or implicitly. The goal may be one of feature selection, in which we aim to find linear or nonlinear combinations of the original set of variables, or one of variable selection, in which we wish to select a subset of variables from the original set. The setting may be unsupervised learning, in which a set of observations of a random vector X are available, or supervised learning, in which desired responses or labels Y are also available. Developing methods for dimensionality reduction requires being clear on the goal and the setting, as methods developed for one combination of goal and setting are not generally appropriate for another. There are additional motivations for dimensionality reduction that it is also helpful to specify, including: providing a simplified explanation of a phenomenon for a human (possibly as part of a visualization algorithm), suppressing noise so as to make a better prediction or decision, or reducing the computational burden. These various motivations are often complementary.. In this paper we study dimensionality reduction in the setting of supervised learning. Thus, we consider problems in which our data consist of observations of (X, Y ) pairs, where X is an m-dimensional explanatory variable and where Y is an -dimensional response. The variable Y may be either continuous or discrete. We refer to these problems generically as "regression" problems, which indicates our focus on the conditional probability density function p Y |X (y | x). In particular, our framework includes discriminative approaches to classification problems, where Y is a discrete label.. We wish to solve a problem of feature selection in which the features are linear combinations of the components of X. In particular, our modeling framework posits that there is an r-dimensional subspace S ⊂ R m such that. for all x and y, where Π S is the orthogonal projection of R m onto S. The subspace S is called the effective subspace for regression. Based on a set of observations of (X, Y ) pairs, we wish to recover a matrix whose columns span the effective subspace. The effective subspace can help to provide explanation of the statistical relation between X and Y , by isolating the feature vectors that capture that relation. Also, finding such a space can suppress noise, to the extent that the orthogonal direction to the effective subspace is noisy vis-a-vis the prediction of Y .. We approach the problem as a semiparametric statistical problem; in particular, we make no assumptions regarding the conditional distribution p Y |Π S X (y | Π S x), nor do we make any assumptions regarding the marginal distribution p X (x). That is, we wish to estimate a finite-dimensional parameter (a matrix whose columns span the effective subspace), while treating the distributions p Y |Π S X (y | Π S x) and p X (x) nonparametrically.. Having found an effective subspace, we may then proceed to build a parametric or nonparametric regression model on that subspace. Thus our approach is an explicit dimensionality reduction method for supervised learning that does not require any particular form of regression model, and can be used as a preprocessor for any supervised learner. This can be compared to the use of methods such as principal components analysis (PCA) in regression, which also make no assumption regarding the subsequent regression model, but fail to make use of the response variable Y .. There are a variety of related approaches in the literature, but most of them involve making specific assumptions regarding the conditional distribution p Y |Π S X (y | Π S x), the marginal distribution p X (x), or both. For example, classical two-layer neural networks involve a linear transformation in the first "layer," followed by a specific nonlinear function and a second layer (Bishop, 1995) . Thus, neural networks can be seen as attempting to estimate an effective subspace based on specific assumptions about the regressor p Y |Π S X (y | Π S x). Similar comments apply to projection pursuit regression (Friedman and Stuetzle, 1981) , ACE (Breiman and Friedman, 1985) and additive models (Hastie and Tibshirani, 1986 ), all of which provide a methodology for dimensionality reduction in which an additive model E[Y | X] = g 1 (β T 1 X) + · · · + g K (β T K X) is assumed for the regressor. Bayesian approaches to variable selection include the Automatic Relevance Determination (ARD) method proposed by Neal (1996) for neural networks. Vivarelli and Williams (1999) have adapted this approach to feature selection in the setting of Gaussian process regression. These methods also depend on a specific parametric model, on which a Bayesian prior distribution is assumed.. Canonical correlation analysis (CCA) and partial least squares (PLS, Höskuldsson, 1988 , Helland, 1988 are classical multivariate statistical methods that can be used for dimensionality reduction in regression (Fung et al., 2002, Nguyen and Rocke, 2002) . These methods are based on a linearity assumption for the regressor, however, and thus are quite strongly parametric.. The line of research that is closest to our work has its origin in a technique known as sliced inverse regression (SIR, Li, 1991) . SIR is a semiparametric method for finding effective subspaces in regression. The basic idea is that the range of the response variable Y is partitioned into a set of "slices," and the sample means of the observations X are computed within each slice. This can be viewed as a rough approximation to the inverse regression of X on Y . For univariate Y the method is particularly easy to implement. Noting that the inverse regression must lie in the effective subspace if the forward regression lies in such a subspace, principal component analysis is then used on the sample means to find the effective subspace. Li (1991) has shown that this approach can find effective subspaces, but only under strong assumptions on the marginal distribution p X (x)-in particular, the marginal distribution must be elliptically symmetric.. Further developments in the wake of SIR include principal Hessian directions (pHd, Li, 1992) , and sliced average variance estimation (SAVE, Weisberg, 1991, Cook and Yin, 2001 ). These are all semiparametric methods in that they make no assumptions about the regressor (see also Cook, 1998) . However, they again place strong restrictions on the probability distribution of the explanatory variables. If these assumptions do not hold, there is no guarantee of finding the effective subspace.. There are also related nonparametric approaches that estimate the derivative of the regressor to achieve dimensionality reduction, based on the fact that the derivative of the conditional expectation E[y | B T x] with respect to x belongs to the effective subspace (Samarov, 1993 , Hristache et al., 2001 . However, nonparametric estimation of derivatives is quite challenging in high-dimensional spaces.. There are also dimensionality reduction methods with a semiparametric flavor in the area of classification, notably the work of Torkkola (2003) , who has proposed using nonparametric estimation of the mutual information between X and Y , and subsequent maximization of this estimate of mutual information with respect to a matrix representing the effective subspace.. In this paper we present a novel semiparametric method for dimensionality reduction that we refer to as Kernel Dimensionality Reduction (KDR). KDR is based on the estimation and optimization of a particular class of operators on reproducing kernel Hilbert spaces (Aronszajn, 1950) . Although our use of reproducing kernel Hilbert spaces is related to their role in algorithms such as the support vector machine and kernel PCA (Boser et al., 1992 , Vapnik et al., 1997 , Schölkopf et al., 1998 , where the kernel function allows linear op-erations in function spaces to be performed in a computationally-efficient manner, our work differs in that it cannot be viewed as a "kernelization" of an underlying linear algorithm. Rather, we use reproducing kernel Hilbert spaces to provide characterizations of general notions of independence, and we use these characterizations to design objective functions to be optimized. We build on earlier work by Bach and Jordan (2002) , who showed how to use reproducing kernel Hilbert spaces to characterize marginal independence between pairs of variables, and thereby design an objective function for independent component analysis. In the current paper, we extend this line of work, showing how to characterize conditional independence using reproducing kernel Hilbert spaces. We achieve this by expressing conditional independence in terms of covariance operators on reproducing kernel Hilbert spaces.. How does conditional independence relate to our dimensionality reduction problem? Recall that our problem is to find a projection Π S of X onto a subspace S such that the conditional probability of Y given X is equal to the conditional probability of Y given Π S X. This is equivalent to finding a projection Π S which makes Y and (I − Π S )X conditionally independent given Π S X. Thus we can turn the dimensionality reduction problem into an optimization problem by expressing it in terms of covariance operators.. In a presence of a finite sample, we need to estimate the covariance operator so as to obtain a sample-based objective function that we can optimize. We derive a natural plugin estimate of the covariance operator, and find that the resulting estimate is identical to the kernel generalized variance that has been described earlier by Bach and Jordan (2002) in the setting of independent component analysis. In that setting, the goal is to measure departures from independence, and the minimization of the kernel generalized variance can be viewed as a surrogate for minimizing a certain mutual information. In the dimensionality reduction setting, on the other hand, the goal is to measure conditional independence, and minimizing the kernel generalized variance can be viewed as a surrogate for maximizing a certain mutual information. Not surprisingly, the derivation that leads to the kernel generalized variance that we present here is quite different from the one presented in the earlier work on kernel ICA. Moreover, the argument that we present here can be viewed as providing a rigorous foundation for other, more heuristic, ways in which the kernel generalized variance has been used, including the model selection algorithms for graphical models presented by Bach and Jordan (2003b) .. The paper is organized as follows. In Section 2, we introduce the problem of dimensionality reduction for supervised learning, and describe its relation with conditional independence and mutual information. Section 3 derives the objective function for estimation of the effective subspace for regression, and describes the KDR method. All of the mathematical details needed for the results in Section 3 are presented in the Appendix, which also provides a general introduction to covariance operators in reproducing kernel Hilbert spaces. In Section 4, we present a series of experiments that test the effectiveness of our method, comparing it with several conventional methods. Section 5 describes an extension of KDR to the problem of variable selection. Section 6 presents our conclusions. We have presented KDR, a new kernel-based approach to dimensionality reduction for regression and classification. One of the most notable aspects of this method is its generality-. .2727 .2736 .2758 Table 6 : Variable selection using the proposed kernel method.. we do not impose any strong assumptions on either the conditional or the marginal distribution. This allows the method to be applicable to a wide range of problems, and gives it a significant practical advantage over existing methods such as CCA, PPR, SIR, and pHd. These methods all impose significant restrictions on the conditional probability, the marginal distribution, or the dimensionality of the effective subspaces. Our experiments have shown that the KDR method can provide many of the desired effects of dimensionality reduction: it provides data visualization capabilities, it can successfully select important explanatory variables in regression, and it can yield classification performance that is better than the performance achieved with the full-dimensional covariate space. We have also discussed the extension of the KDR method to variable selection. Experiments with classical data sets has shown an accordance with the previous results on these data sets and suggest that further study of this application of KDR is warranted.. The theoretical basis of KDR lies in the nonparametric characterization of conditional independence that we have presented in this paper. Extending earlier work on the kernelbased characterization of independence in ICA (Bach and Jordan, 2002), we have shown that conditional independence can be characterized in terms of covariance operators on a reproducing kernel Hilbert space. While our focus has been on the problem of dimensionality reduction, it is also worth noting that there are many other possible applications of this characterization. In particular, conditional independence plays an important role in the structural definition of probabilistic graphical models, and our results may have applications to model selection and inference in graphical models.. There are several statistical problems which need to be addressed in further research on KDR. First, a basic analysis of the statistical consistency of the KDR-based estimatorthe convergence of the estimator to the true subspace when such a space really exists-is needed. We expect that, to prove consistency, we will need to impose a condition on the rate of decrease of the regularization coefficient ε as the sample size n goes to infinity. Second, and most significantly, we need rigorous methods for choosing the dimensionality of the effective subspace. If the goal is that of achieving high predictive performance after dimensionality reduction, we can use one of many existing methods (e.g., cross-validation, penalty-based methods) to assess the expected generalization as a function of dimensionality. Note in particular that by using KDR as a method to select an estimator given a fixed dimensionality, we have substantially reduced the number of hypotheses being considered, and expect to find ourselves in a regime in which methods such as cross-validation are likely to be effective. It is also worth noting, however, that the goals of dimensionality reduction are not always simply that of prediction; in particular, the search for small sets of explanatory variables will need to be guided by other principles. Finally, asymptotic analysis may provide useful guidance for selecting the dimensionality; an example of such an analysis that we believe can be adopted for KDR has been presented by Li (1991) for the SIR method.. Proof Obviously, the operator is unique, if it exists. From Riesz's representation theorem (see Reed and Simon, 1980, Theorem II.4 , for example), the existence of Σ Y X f ∈ H 2 for a fixed f can be proved by showing that the right hand side of Equation (16) is a bounded linear functional on H 2 . The linearity is obvious, and the boundedness is shown by. For the second last inequality, k(·, x) 2 H = k(x, x) is used. The linearity of the map Σ Y X is given by the uniqueness part of Riesz's representation theorem.. From Equation (17), Σ Y X is bounded, and by definition, we see Σ * Y X = Σ XY , where A * denotes the adjoint of A. If the two RKHS are the same, the operator Σ XX is called the covariance operator. A covariance operator Σ XX is bounded, self-adjoint, and trace-class.. In an RKHS, conditional expectations can be expressed by cross-covariance operators, in a manner analogous to finite-dimensional Gaussian random variables.. Theorem 2 Let (H 1 , k 1 ) and (H 2 , k 2 ) be RKHS on measurable spaces Ω 1 and Ω 2 , respectively, with k 1 and k 2 measurable, and (X, Y ) be a random vector on Ω 1 × Ω 2 . Assume that. are finite, and for all g ∈ H 2 the conditional expectation. where Σ XX and Σ XY are the covariance and cross-covariance operator.. This completes the proof.. XX be the right inverse of Σ XX on (KerΣ XX ) ⊥ . Under the same assumptions as Theorem 2, we have. for all f ∈ (KerΣ XX ) ⊥ and g ∈ H 2 . In particular, if KerΣ XX = 0, we have. Proof Note that the productΣ. Y Y with a bounded operator V (Baker, 1973, Theorem 1) , and the second equation holds for any self-adjoint operator. Take f = Σ XX h ∈ RangeΣ XX . Then, Theorem 2 yields. This completes the proof.. The assumption E Y |X [g(Y ) | X = ·] ∈ H 1 in Theorem 2 can be simplified so that it can be checked without reference to a specific g.

B I O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Approximate Inference for Medical Diagnosis Computer-based diagnostic decision support systems (DSS) will play an increasingly important role in health care. Due to the inherent probabilistic nature of medical diagnosis, a DSS should preferably be based on a probabilistic model. In particular Bayesian networks provide a powerful and conceptually transparent formalism for probabilistic modeling. A drawback is that Bayesian networks become intractable for exact computation if a large medical domain would be modeled in detail. This has obstructed the development of a useful system for internal medicine. Advances in approximation techniques, e.g. using variational methods with tractable structures, have opened new possibilities to deal with the computational problem. However, the only way to assess the usefulness of these methods for a DSS in practice is by actually building such a system and evaluating it by users. In the coming years, we aim to build a DSS for anaemia based on a detailed probabilistic model, and equipped with approximate methods to study the practical feasibility and the usefulness of this approach in medical practice.In this paper, we will sketch how variational techniques with tractable structures can be used in a typical model for medical diagnosis. We provide numerical results on arti cial problems. In addition, we describe our approach to develop the Bayesian network for the DSS, and show some preliminary results. Health care will change profoundly with the introduction of clinical diagnostic decision support systems (DSSs), preferably integrated with electronic patient data and together with on-line computer communication 1]. However, such systems have not yet entered daily clinical practice for a variety of reasons 2]. Some of the reasons are the lack of adequate computer infrastructure in hospitals, the lack of standardization of terminology and the poor performance of current medical diagnostic systems. In this paper we focus on this last issue.. Diagnostic reasoning in the medical domain is a typical example of reasoning with uncertainty. This uncertainty has di erent sources: missing patient information, uncertainty in medical tests results or observations, and the uncertainty about the physiological processes involved. A model on which a DSS is based should be able to deal with these uncertainties. The di erent systems that have been developed so far use a variety of modeling approaches which can be roughly divided into two categories: The large systems, that attempt to cover the whole of internal medicine use a rule-based approach with some rather heuristic method to quantify uncertainty. These methods perform poorly in practice 3, 4] . The main reasons are that the modeling of the relations between diseases and ndings is at a very course level. Therefore, the diagnoses suggested by these systems are too super cial for clinical use. Secondly, the diagnostic process requires reasoning from causes to e ects (diseases ! nding) and vise versa at the same time. The rule based approach, together with the heuristics for uncertainty, is not well suited for such bidirectional reasoning.. For smaller systems, the probabilistic approach is typically used. The probabilistic approach has the important advantage of mathematical consistency and correctness. In particular Bayesian networks (see e.g. 5, 6, 7] ) provide a powerful and conceptual transparent formalism for probabilistic modeling. In addition, they allow for easy integration of domain knowledge and learning from data. Systems that are based on detailed modeling have been restricted to a relatively small domain 8, 9] . The reason for this restriction is that Bayesian networks will become intractable for exact computation if a large medical domain would be modeled in detail.. To proceed one has to rely on approximate computations. Recently, variational methods for approximationare becoming increasingly popular 10, 11, 12] . An advantage of variational methods techniques is that they provide bounds on the quantity of interest in contrast to stochastic sampling methods which may yield unreliable results due to nite sampling times. Until now, variational approximations have been less widely applied than Monte Carlo methods, arguably since their use is not so straightforward. We argue that variational methods are indeed applicable to large, detailed Bayesian networks for medical diagnosis constructed by human experts.. Although the formalism of Bayesian networks is very powerful, the construction of networks for medical diagnosis is not straightforward. A learning approach depends crucially on the availability of high quality patient data. In particular, rare disorders should be well covered. In general, unfortunately, this is rather exception than rule 13]. Therefore, to reach a successful diagnostic DSS requires explicit modeling e ort by human experts. The existing medical literature is not su cient to de ne the probabilistic model. Not all probabilistic relations between variables have been documented. But it provides a useful starting point for model design. Once a minimal performance is thus obtained, the model can be improved by learning from patient data.. The paper is organised as follows. In section 2 we discuss the modular structure of a broad and detailed probabilistic model for medical diagnosis constructed by human experts. In section 3 we show how variational methods can provide approximations for the conditional probabilities in these networks. This is illustrated by results on the Lauritzen's chest clinic model and a random intractable network. In section 4 we describe our approach to model medical diagnosis, and we give a short description of our system for megaloblastic anaemia, called Promedas. We conclude with a discussion in section 5. The development of a DSS for comprehensive medical diagnosis in internal medicine represents a great challenge for AI. A broad and detailed probabilistic network is intractable for exact inference in this context. It is currently unknown, whether variational or other approximate methods are su ciently powerful to provide a practical solution. The \quality of approximation" is to a large extent a user de ned (medical) issue, since (1) comparison with exact inference is not possible due to the size of the networks and (2) errors in the approximation will be judged as acceptable not just on their numerical values but more importantly on their medical implications. The only way to assess the usefulness of approximate methods for modeling medical domains is by actually building such a system and evaluating it by users. The Promedas model must be extended to several 100 variables in order to be able to address this issue properly.. Our results on the Asia problem with evidence (not reported here), as well as the results reported here, show that the factorized variational approximation is qualitatively correct in the sense that it correctly estimates whether probabilities are high or low. However, the numerical errors can be rather large. The results of variational approximations using structure gives a signi cant improvement. Our results seem to indicate that this improvment is independent of the problem size. The choice of optimal structure for Q is a topic for further investigation.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Large vocabulary continuous speech recognition of an inflected language using stems and endings In this article, we focus on creating a large vocabulary speech recognition system for the Slovenian language. Currently, state-of-the-art recognition systems are able to use vocabularies with sizes of 20,000 to 100,000 words. These systems have mostly been developed for English, which belongs to a group of uninflectional languages. Slovenian, as a Slavic language, belongs to a group of inflectional languages. Its rich morphology presents a major problem in large vocabulary speech recognition. Compared to English, the Slovenian language requires a vocabulary approx. ten times greater for the same degree of text coverage. Consequently, the difference in vocabulary size causes a high degree of OOV (out-of-vocabulary words). Therefore OOV words have a direct impact on recognizer efficiency. The characteristics of inflectional languages have been considered when developing a new search algorithm with a method for restricting the correct order of sub-word units, and to use separate language models based on sub-words. This search algorithm combines the properties of sub-wordbased models (reduced OOV) and word-based models (the length of context). The algorithm also enables better search-space limitation for sub word models. Using sub-word models, we increase recognizer accuracy and achieve a comparable search space to that of a standard word-based recognizer. Our methods were evaluated in experiments on a SNABI speech database. The natural development of language has caused its variability and ambiguity. The result is about 6 000 known languages today. They differ to a great extend in word formation rules. Homonym disambiguation is another big challenge today. From the speech recognition point of view, it would be logical to classify all languages by their common sources, the results of which could also be seen in various multilingual recognition experiments. The reason for such presumptions is the fact that similar languages share the same or, at least, similar grammatical and phonetical attributes as their sources. In general, development in speech recognition is moving towards the customisation of recognizers for use with different language groups. This article refers to large-vocabulary speech recognition with emphasis on inflectional languages, among which is the Slovenian language. Its rich morphology, therefore, represents a major problem in large vocabulary speech recognition, which is reflected in a high degree of OOV words and a much more varied word order, in comparison to the English language. A common word order in English would be SVO (subject, verb, object) structure, whereas this is not the case in Slavic languages; the only exceptions being Macedonian and Bulgarian. However, freer word order reduces the efficiency of the statistical language modeling commonly used in large vocabulary speech recognition. Besides the Upper Sorbian language, Slovenian is the only language to include additional word forms for dual, which causes an even greater number of words.. Another feature of the Slovenian language is the category of verbal aspect and palatalization, where the next sound causes a change in the previous sound, creating even more new word forms. The afore-mentioned features of inflectional languages prevent straightforward usage of the state of the art recognition systems technology developed for English.. This article is divided into six chapters. Sub-word modeling is covered in the following chapter. The main advantage of sub-word-based models compared to word-based is a smaller OOV ratio. The core part of this chapter describes the morphological structure of the Slovenian language. The third chapter discusses those problems encountered in the recognition of different sub-word units. A mathematical formula is presented for recognition process with word and sub-word units. The definition of a novel search algorithm follows, for the recognition of inflectional languages. We treat it from the point of acoustical and language modeling. We also present different improvements in the proposed search algorithm. The fourth chapter describes the experimental system setup and captures speech recognition results with different search algorithms and with different vocabulary units. Recognition error, recognition speed, and the size of search space, expressed as the average number of active instances are compared. Discussion on improvements in recognition results are presented in chapter five when using a new search algorithm with extended context in speech recognition of the Slovenian language. The last chapter provides a summary of the presented works, achievements, and ideas for future work. In this article we presented a new search algorithm with sub-word models, which restricts search space by using sub-word units in the correct order, limiting the number of endings for an individual stem, using separate sub-word language models (extended context), and combining the trees of stems, which were derived from the same tree of endings, into one common tree. The result was the smallest search space amongst all search algorithms, using sub-word models and identical search space, compared with standard word-based search algorithm. Using higher order sub-word-based language models (trigram), did not contribute so much to the performance of new search algorithm, because the problem of free word order arise. The essential feature of sub-word-based language model is the capability of modeling dependencies within a word. In general sub-word-based language models are less constrained and lead to increases in word-based perplexity. Therefore, such models would still have to be combined with ones that could produce probabilities for larger units (i.e. words, classes of words). One promising way for the future would be to combine those models, which capture different dependencies in language.. This recognition system is designed to be extendable to other inflectional languages and, with minor modifications, also used for other languages which include inflectional morphology. However, in recognition using sub-word models, some problems remain: How to preserve acoustic separability with shorter units. One of the possible solutions might be the integration of new knowledge sources from the field of speech understanding, or incorporating higher-level linguistic information (semantic and grammatical analysis) into the decoding process. Table 1 : An example of different word forms for the word "nesti" (to carry).

B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multivariate image analysis in biomedicine In recent years, multivariate imaging techniques are developed and applied in biomedical research in an increasing degree. In research projets and in clinical studies as well m-dimensional multivariate images (MVI) are recorded and stored to databases for a subsequent analysis. The complexity of the m-dimensional data and the growing number of high throughput applications call for new strategies for the application of image processing and data mining to support the direct interactive analysis by human experts. This article provides an overview of proposed approaches for MVI analysis in biomedicine. After summarizing the biomedical MVI techniques the two level framework for MVI analysis is illustrated. Following this framework, the state-of-the-art solutions from the fields of image processing and data mining are reviewed and discussed. Motivations for MVI data mining in biology and medicine are characterized, followed by an overview of graphical and auditory approaches for interactive data exploration. The paper concludes with summarizing open problems in MVI analysis and remarks upon the future development of biomedical MVI analysis. In recent years we observe an increasing number of biomedical imaging applications that associate a number of m signal values to pixel or voxel coordinates p in a two-or three-dimensional array (i.e., p = (x, y) or = (x, y, z), respectively). In the result stack of m intensity images, m locally corresponding signal values s 1 , . . ., s m are associated to a pixel (or voxel) p and can be interpreted as a point s (p) = (s 1 , . . ., s m ) in an m-dimensional space. Without loss of generality, we assume that the signals are subject to a post-imaging normalization step and mapped to an appropriate scale s i 2 [0;1]. This mapping procedure often includes the application of a log function to enhance weak signals and/or a rescaling of the vector components s i to have zero mean and unit variance. In this article, such imaging approaches are referred to as biomedical multivariate imaging, i.e., techniques for recording multivariate images or volumes.. In microbiology, researchers analyze MVIs to corelate different molecular parameters locally in cells and tissue specimens. The local identification of molecules is enabled by ongoing advances in the development, understanding, and optimization of monoclonal antibody markers (mAb). In addition, advanced staining techniques, multiband imaging, and color decomposition [1] allow simultaneous visualization of macromolecules in the cells (e.g., [2] [3] [4] ). This simultaneous identification of molecules by selective visualization, enables quantitative studies on a large scale, resulting in a new perspective on optical microscopy as a tool for quantitative studies in microbiological research which can be interpreted as a renaissance of optical microscopy [5] [6] [7] . The imaging approaches are applied to cell samples from different specimens, in certain stress conditions, or with different pharmaceutical treatment to get more comprehensive descriptions of spatial and temporal molecular dynamics in the cell [8] . But not the molecular visualization capacities have been advanced only, the implementation of highly standardized protocols through the application of modern robotics and personal computers pave the way to a new level of experiment automation [9] . Thus, cellular parameters can be simultaneously and locally identified in highthroughput screening (HTS) approaches.. Another group of MVI techniques is based on combining different imaging modalities by merging registered images from different devices, which is usually referred to as intermodular imaging. In microbiology for instance, microscopy and magnet resonance imaging (MRI) can be combined to get new insight into cell structure [10] .. However, multivariate imaging is developed and applied much more frequently in the field of medical imaging. In this article, the term medical imaging comprises imaging techniques applied to macrobiological structures in the context of medical diagnosis. The imaging techniques are ultrasonic imaging, X-ray imaging, computer tomography (CT), single photon emission CT (SPECT), positron emission tomography (PET), and MRI [11] [12] [13] [14] . Although a few multivariate imaging systems are already in clinical use, MVI based diagnosis is still more regarded a field of research than a standard diagnostic imaging technique.. The approaches can be subdivided into intramodular and intermodular ones. Intramodular imaging applies m different parameterizations of one imaging technique to record a set of m images. One favorable advantage of this approach is, that the m images usually have identical spacial resolution and the registration of the images is feasible. One example of intramodular multivariate imaging is multispectral microscopy imaging, which has been successfully applied in skin lesion classification [15] . Another group of intramodular imaging approaches is based on applying MRI with different protocols. In so called multispectral MRI, a T1-, T2-, and proton density weighted image is recorded for brain studies. In another brain study context, so called dynamic functional MRI (fMRI) and diffusion tensor imaging [16] is applied. In the latter case, a number of diffusion weighted MRI plus one reference image is recorded to visualize the microscopic water motion in the brain. In cancer diagnosis, dynamic contrast enhanced MRI (DCE MRI) [17, 18] is applied to visualize vascular features of suspicious lesions. A time series of MRI volumes is recorded, and the tissue parameters are characterized by the signal dynamics, which are affected by an injected paramagnetic contrast agent. Combinations of multispectral and contrast enhanced MRI is applied for multiple sclerosis analysis [19] and breast lesion diagnosis [20] .. The second group of intermodular approaches use different imaging techniques, usually based on different physical effects. The recordings usually differ in spacial resolution, signal scales and are sometimes recorded with a considerable time lag in between. Thus, the coregistration of the images becomes a difficult scientific problem. Usually, this problem is solved by tuning the imaging parameters to adjust the spacial resolutions and by applying registration algorithms. The first is done by tuning the imaging sequence in MRI or applying special filters in CT or PET. Several intermodular imaging approaches have been reported, among them are most recently PET/CT [21] or CT/MRI/PET [22] .. Although open source software is available for examining multimodal image data [23] , the growing number of research projects and diagnosis systems that employ multivariate images (MVI) and the data complexity results in a call for new approaches for interpretation, which has not been answered yet, as to the best of the authorÕs knowledge and as also reported in [21] .. This article reviews the methods that have been proposed for MVI analysis. The review is oriented along a two level analysis framework which is outlined in the following. The first level of analysis is called the exploration level or E-level (see Fig. 1 ). The image content, i.e., the pixel or region parameters s are displayed and a user perceives displays from different MVI (or different regions in one MVI). Through interactive browsing of image regions of interest (ROI), knowledge is discovered on the same level intention-driven as by serendipity. Since all m images cannot be displayed in an instant, the images have to be processed to adapt the content of the images to the perceptual skills of the human user. To this end, visualization algorithms are needed that account for the statistical features in the MVI.. In the second level, quantitative image parameters u ðiÞ 2 R d of ROIs are extracted from the image and analyzed full-automatically (see Fig. 1 ). The extraction step can be realized full-automatically or with some user-interaction. In both ways, the extraction process needs to be highly standardized since reproducibility of this step is crucial. The extracted data are stored to a database. The ROI parameters u (i) are computed from the signal values s (p) and describe biological parameters quantitatively. In a basic set up, each pixel p is considered as a ROI:. Afterwards, statistical data mining methods are applied to find and prove hidden structures and regularities in the data, e.g., clusters. The entire level is referred to as the statistical level or S-level. Usually, the user provides some background information and ideas about the data structure, so serendipity-driven discoveries are more exceptional. Examples for labels are sample treatment parameters, the gender or age of a patient, or a TNM-class label 1 for instance. In this article, all kinds of such labels will be referred to as meta-labels y (j) . The index j usually identifies an index of a ROI or an entire MVI. The S-level needs data mining methods, that can be applied to sets of extracted parameter vectors to localize interesting ROIs, detect interesting patterns u (i) or components u k , k 2 {1, . . ., d}. The complete two level framework for MVI analysis is illustrated in Fig. 1 . To realize the E-and the S-level components, methods from the fields of image processing and data mining have been proposed, which are summarized in the following sections. The development and application of multivariate imaging in biomedicine is a rapid evolving field. The analysis of large sets of MVIs will become fundamental for diagnostic practice and scientific research, from the micro-to the macrobiological level. Reviewing proposed methods along a two level analysis framework, we are surprised to observe, that computer science is not really well prepared to provide suitable methods to for MVI data. It seems apparent, that progress in the field would be accelerated, if researchers and engineers working in biomedical MVI analysis would organize a MVI analysis community to identify fundamental problems across single particular applications and propose terms for system performance and accuracy. Particularly benchmark problems and data sets need to be installed to help researchers evaluating, developing, comparing, and discussing their methods, which holds for all aspects of data mining, from image processing to multimodal display. More than this, it would be a prerequisite for the application of more complex models of data structure analysis. A big step forward would be some KDD (knowledge discovery in databases) or visualization competition (like this yearÕs IEEE Visualization contest 6 ) with a primary focus on biomedical images or MVIs. Additionally, display techniques need to be analyzed on a more integrated, multimodal level, and analyzed on a psychophysical level. Still, discussions of integrated systems of multimodal display, like in [98] are exceptional.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O


Timeliness-Accuracy Balanced Collection of Dynamic Context Data Abstract-In the future, we are likely to see a tremendous need for context-aware applications which adapt to available context information such as physical surroundings, network, or system conditions. We aim to provide a fundamental support for these applications-a real-time context information collection service. This service delivers the right context information to the right user at the right time. The complexity of providing the real-time context information service arises from 1) the dynamically changing status of information sources, 2) the diverse user requirements in terms of data accuracy and service latency, and 3) constantly changing system conditions. In this paper, we take into consideration these dynamics and focus on addressing the trade-offs between timeliness, accuracy, and cost for information collection in distributed real-time environments. We propose a middleware-based approach to enable a judicious composition of services for accuracy-aware scheduling and cost-aware database maintenance. Specifically, we characterize the problem in terms of Quality-of-Service Satisfaction (QoSSat), Quality-of-Data Satisfaction (QoDSat), and Cost. We propose a middleware framework for the real-time information collection process, where the information mediator coordinates and facilitates communication between information sources and consumers. We design a family of algorithms for real-time request scheduling, request servicing, and directory service maintenance to be implemented at the mediator to support QoSSat and QoDSat. Our studies indicate that the composition of our proposed scheduling algorithm and directory service maintenance policy can improve the overall efficiency of the system. We also observe that the proposed policies perform very well as the system scales in the number of information sources and consumer requests. R ECENT advances in mobile computing, communications, and embedded systems are likely to enable the deployment of large-scale ubiquitous computing environments. We expect that real-time context information services will gain importance in the future. Examples of applications that require real-time access to distributed information include network management, stock trading, air traffic control, security/surveillance, medical alerts, and patient tracking. To facilitate such real-time applications, an information collection architecture that can seamlessly provide real-time access to dynamically changing data regardless of diverse user requirements and changing system conditions is a must.. Real-time information collection in a dynamic environment presents the system designer with interesting challenges. First, information sources provide a continuous stream of data that can dynamically vary over time. This information may need to be captured and stored rapidly and accurately. Second, users requiring access to this dynamically changing data present variable user requirements in terms of the accuracy of data and timeliness of the service. Furthermore, network and service providers would like to ensure effective utilization of underlying computation, communication, and storage resources. Ideally, applications would like to obtain accurate state information in a real-time manner with the least cost. The underlying middleware architecture must deal with the issue of balancing these competing goals of timeliness, accuracy, and cost-effectiveness. Since many applications are willing to tolerate information imprecision and bounded delivery latencies, our strategy is to exploit these accuracy and latency margins to ensure that most applications receive information at the desired levels of quality and timeliness while minimizing resource consumption.. Given the data intensive nature of the system, an information repository, i.e., directory service (DS), is a must for the system to function efficiently. Maintaining an effective DS, which reflects the data changes as closely as possible/necessary, is not a trivial task in dynamic environments. In addition to processing source updates, the system must also respond to user requests in a timely fashion. Processing source updates at the expense of user requests will mean that fewer user requests will finish on time, while delaying source updates in favor of user requests will mean that the DS will not be representative of the state of the external environment. In addition to timeliness constraints, user requests may also provide accuracy constraints. Since information in the DS may not be up to date, there is a possible mismatch between the DS accuracy and user accuracy constraints. We are faced with a cost-accuracy trade-off since an accurate DS makes it easier to satisfy more user requests with less overhead in a more timely fashion, but could also introduce a high DS maintenance cost.. Contributions of this Paper. Existing and ongoing research in data management addresses the trade-offs between timeliness and data freshness, as well as the accuracy and cost trade-offs in stream-based environments. However, increasingly many applications need more finegrained interactions with the system in the format of multiple individual information retrieval requests rather than transactions. In addition, both user requests and data source updates may have explicit time constraints and data accuracy requirements. In this context, the problem of balancing the trade-offs between 1) timeliness, 2) accuracy, and 3) cost simultaneously has not been addressed to the best of our knowledge. The mere application of previously proposed techniques in the area of real-time scheduling or dynamic data management is insufficient to address the issue at hand; a simplistic composition of the proposed techniques cannot address the cost/accuracy/timeliness trade-offs appropriately either. Therefore, we develop a middleware-based approach to address the challenges in enabling a judicious composition of these techniques. Specific contributions of this paper are as follows:. . We characterize the problem of cost-effectively providing timeliness and accuracy in terms of Quality-of-Service Satisfaction (QoSSat), Quality-ofData Satisfaction (QoDSat), and Cost. . We propose a middleware framework for the realtime information collection process and design a family of algorithms to support QoSSat and QoDSat. We design a Timeliness and Accuracy Balanced Scheduling (TABS) algorithm to support timely response to both user requests and source updates while ensuring that accuracy constraints of requests are satisfied. Furthermore, we present a Minimized Cost DS maintenance algorithm (MC) that balances the distribution of source update and consumer retrieval requests to maintain a constant frequency ratio. Finally, request servicing algorithms are designed to determine the process flow for each single request by analyzing the possibility of meeting its timeliness and accuracy constraints. . Through extensive experiments, we study the interaction between scheduling, request servicing, and DS maintenance and explore dynamic adaptations and a judicious composition of the policies that address timeliness/accuracy/cost trade-offs under varying conditions. The rest of this paper is organized as follows: Section 2 formulates the accuracy-driven real-time information collection problem. Section 3 presents a middleware architecture for real-time information collection. Section 4 addresses a solution via the algorithms for real-time request scheduling, request servicing, and DS maintenance. Section 5 presents experimental results. Section 6 discusses related work. We conclude in Section 7 with future research directions. A key concern of our work is to exploit the accuracy and latency margins to ensure that most applications receive information at the desired levels of accuracy and timeliness while minimizing the consumption of resources (storage, network bandwidth, etc.) in distributed environments. In this paper, we have shown, using the proposed real-time information collection architecture, how an application's tolerance of information imprecision can be taken into account to improve system timeliness. We have also indicated that the trade-off between timeliness and accuracy can be addressed by scheduling requests from both sources and consumers in a uniform manner (i.e., not favoring either type). At the same time, we observe that a well-maintained DS (in concert with the scheduling algorithm) is essential for balanced QoSSat and QoDSat under different system load and input characteristics. The eventual goal of our work is to develop effective tools for context-aware applications in highly dynamic and distributed environments. Middleware techniques for achieving the competing goals of timeliness, accuracy, and costeffectiveness, such as those described in this paper, are key to delivering the right information to the right user at the right time in pervasive computing environments. This paper forms the basis for future research in the direction of bringing together real-time systems and data management techniques through composable middleware systems.. Qi Han received the MS degree in computer science from the Huazhong University of Science and Technology, China, and the PhD degree in computer science from the University of California, Irvine. She is an assistant professor in the Department of Mathematical and Computer Sciences, Colorado School of Mines. Her research interests include distributed systems, middleware, mobile and pervasive computing, systems support for sensor applications, and dynamic data management. She is specifically interested in developing adaptive middleware techniques for next generation distributed systems. She is a member of the IEEE and ACM.. Nalini Venkatasubramanian received the MS and PhD degrees in computer science from the University of Illinois, Urbana-Champaign. She is an associate professor in the School of Information and Computer Science, University of California, Irvine. Her research interests include distributed and parallel systems, middleware, mobile environments, multimedia systems/applications, and formal reasoning of distributed systems. She is specifically interested in developing safe and flexible middleware technology for highly dynamic environments. She was a member of the technical staff at HewlettPackard Laboratories in Palo Alto, California, for several years, where she worked on large scale distributed systems and interactive multimedia applications. She has also worked on various database management systems and on programming languages/compilers for highperformance machines. She is a member of the IEEE and ACM.. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib. In this section, we compare our work to related research in temporal and real-time databases and dynamic data management.. Temporal and Real-Time Databases. After researchers identified the need to incorporate both temporal and realtime capabilities [13] , [14] to ensure temporal data consistency while providing time-constrained transaction processing, several projects (such as STRIP [3] , ARCS [4] , and QMF [5] ) address the trade-off between transaction timeliness and data freshness. Our work differs from the aforementioned prior research in several ways. First, a regular transaction is a sequence of database operations, whereas the incoming requests in our case (consumer or source update requests) are special transactions with only one single operation. Realtime transaction processing techniques focus on the optimized execution of a group of operations to meet transaction deadlines; in our case, the deadlines are on the fine-grained operations themselves. This implies that we must explore trade-offs between fine-grained operations, and traditional real-time transaction processing techniques will not suffice. Second, data is represented approximately in our system in order to reduce communication overhead for maintaining the repository. The approximation also implies that source update requests can have a direct impact on the data accuracy, which can indirectly affect timeliness. Our objective is to maintain a separation of concerns and isolate tasks within the real-time path flow (scheduling, request servicing) from tasks that maintain accuracy and minimize cost (DS maintenance). At the same time, we would like to allow adequate interaction between these components to balance the cost/accuracy/time trade-offs.. Dynamic Data Management. Approximate data caching approaches [15] , [1] , [12] , [16] , [17] , [18] have been proposed to address the trade-off between the accuracy of cached data and the overhead involved in maintaining the cache. In our work, we address time constraints in addition to precision requirements imposed on user queries; this further complicates the DS maintenance problem. In addition, our work considers system performance in terms of overall deadline satisfaction ratio, rather than for each individual query. Therefore, a scheduling mechanism is needed.. Data streaming has been a vibrant research field in recent years for both traditional Internet environments (e.g., OpenCQ [19] , Niagara [20] , Telegraph [21] , STREAM [22] , and Aurora [23] ) and emerging ad hoc sensor networks (e.g., COUGAR [24] and Quasar [25] ). Researchers in this area have addressed a whole gamut of issues, such as system architectures, concepts/semantics of continuous queries, QoS specifications for fresh information delivery, system scalability, etc. Our work targets a similar context where data is fast changing and update intensive, but we provide support for collecting and accessing the data in real-time. In addition, we take advantage of an application's tolerance toward data imprecision and address the three factors-request timeliness, data accuracy, and maintenance overhead.. The real-time data collection problem bears some similarity to dynamic Web content caching [26] , [27] , [28] , [29] , [30] at a higher level. Since changes to underlying data might affect more than one Web page, the key issues in dynamic Web content caching are to determine what dynamic pages should be cached and when a cached page has become obsolete. Although both timeliness and accuracy matter here, Web requests do not have explicit time constraints associated, while the data retrieval requests in our scenario may arrive into the system with userspecified deadlines. In addition, in the dynamic Web content caching problem, it is the Web pages that are being cached; on the other hand, we store the dynamic data (more specifically, numerical data such as stock prices, network measurement data, etc.). We can therefore use range-based approximation to represent the dynamic data and then exploit applications' tolerance toward data imprecision to decrease data collection overhead.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Processing first-order queries under limited access patterns We study the problem of answering queries over sources with limited access patterns. Given a first-order query Q, the problem is to decide whether there is an equivalent query which can be executed observing the access patterns restrictions. If so, we say that Q is feasible. We define feasible for first-order queries-previous definitions handled only some existential cases-and characterize the complexity of many first-order query classes. For each of them, we show that deciding feasibility is as hard as deciding containment. Since feasibility is undecidable in many cases and hard to decide in some others, we also define an approximation to it which can be computed in NP for any first-order query and in P for unions of conjunctive queries with negation. Finally, we outline a practical overall strategy for processing first-order queries under limited access patterns. We study the problem of answering queries over sources with limited access patterns, i.e., where the source relations cannot be accessed in arbitrary ways, but only when certain combinations of attribute values are given. Consider a source relation book (isbn,author ,publisher ). The access patterns for book may prescribe that at least one of isbn or author must be given as input in order to access the relation, whereas only providing a value for publisher is not sufficient. This is specified by giving the access patterns book ioo , book oio , where 'i' denotes a required input slot and 'o' denotes an output slot for which no value is required. 1 Given a query Q over a schema with limited access patterns, we are interested in the following questions:. A query plan is an executable query together with an annotation that satisfies the given access patterns and induced input requirements. For our Q(x) above, a query plan is:. Note that the subgoal pub io (x, y) requires x to be bound. This is achieved by annotating the head of the new rule for E(x) with the pattern E i , indicating that x is an input parameter of that rule. The occurrence ¬E i (x) in the rule body is thus annotated accordingly, which is indeed valid since we can obtain a binding for x from the preceding subgoal coa io (knuth, x).. The examples above involve universal queries (see Π1 in the preliminaries). We show that deciding feasiblity is Π P 2complete for this class of queries. However, if we define the relations pub and coa above as conjunctive views over the book relation (e.g., pub(a, p) ← book(i, a, p)), we obtain query expressions of the form ∀x∃ȳ F , i.e., Π2 formulas. We show that in general feasibility is undecidable for Π2 formulas.. In order to support the most efficient query planning, it is important to handle each query expression according to its form. For example, if a formula is not in DNF, we can first convert it to DNF, then process it by an algorithm that handles DNF formulas. However, conversion to DNF may cause an exponential length increase. In particular, converting positive existential formulas to unions of conjunctive queries gives an upper bound of NEXP for deciding feasibility, whereas we obtain an upper bound of Π P 2 which matches our lower bound. This is the motivation for studying a large number of query classes: by taking into account the form of the query, a query planner can choose an appropriate optimal strategy. We have characterized the complexity of query feasibility in the presence of access patterns for a large number of classes.. Our results extend previous ones and indicate that feasibility is just as hard as containment. However, we had to show this laboriously case by case, since there is no obvious uniform reduction. From a theoretician's point of view, we would like to find one, but we conjecture it does not exist. Also we are interested in investigating the problem of feasibility in the presence of integrity constraints.. Query processing in the presence of limited access patterns is even more important from a practicioner's point of view.. Traditionally it is encountered in data integration, specifically over web sources. With the emergence of web services [WSD03] as a simple means for distributed computation and data access, the need to understand executability and feasibility of queries is further increased, since it is a fundamental problem, e.g., in web service composition [TKL03] and planning for distributed scientific workflows [DBG + 03, LAG03, HBCS03]. Clearly, our results belong to the realm of database theory since we characterize the inherent complexity of query feasibility -a prerequesite for rewriting queries into equivalent executable ones. However, we would like to emphasize that our work was motivated and is driven by a number of very practical engineering problems: In the Biomedical Informatics Research Network [BIR01], we are developing a database mediator system for federating heterogeneous brain data [GLM03, LGM03] . The current prototype takes a query against a global-as-view definition and unfolds it into a Σ1DNF plan. We have used the notions of answerable part and orderable, as well as novel runtime techniques [NL04] in the prototype system. Our theoretical investigations have solved several of the pending algorithmic issues and we can now proceed with the extension of the mediator planner. In the BIRN project, as in many other projects pertaining to scientific data management, sources with limited query capabilities are ubiquitous. See Example 3 for an application example in the context of scientific workflows (a.k.a. "analysis pipelines"). There has been a considerable amount of work in the area of query processing with limited query capabilities [Ull88, RSU95, LRO96, FLMS99, DGL00, MLF00, VP00, LC01b, Li03]. However, the complexity of query feasibility under access pattern restrictions has been largely open until now. The work by Li et al. [LC01b, Li03] is most closely related to ours. They show that for conjunctive queries (CQ) and conjunctive queries with union (UCQ), the notion of feasibility is closely related to that of minimal queries (and thus to containment). In particular, they show that (i) for CQ, deciding feasibility 3 is NP-complete, (ii) for Q ∈ UCQ, Q = Q1 ∨ · · · ∨ Qn is feasible iff each minimal equivalent Q i of Qi is orderable, and (iii) feasibility is undecidable for datalog. They also provide an algorithm that in some cases can compute complete answers to CQ queries which are not feasible (the completeness depends on the given database instance).. In [NL04] we extend these results to conjunctive queries with negation (CQ ¬ ) and unions of conjunctive queries with negation (UCQ ¬ ) and show that for both classes deciding feasibility is Π P 2 -complete. A uniform treatment for UCQ ¬ and its subclasses CQ, CQ ¬ , and UCQ is obtained by the notion of the answerable part ans(Q) of Q, which for those classes is shown to be the minimal feasible query containing Q (see Theorem 5 below). In [NL04] we also extend previous work [Li03] on the runtime handling of query answers and provide an elegant and efficient way to compute underestimates and overestimates of answers to infeasible queries, often allowing to quantify the degree of completeness at runtime.. In the context of data integration, several works have studied how to retrieve additional answers, e.g., by a "domain enumeration approach" that uses bindings from relations not necessarily mentioned in the query [DL97, LC01a] . It is also common to consider source capabilities expressed as conjunctive views with access patterns. An executable query plan in this setting is one which (i) expresses the query using the views only and (ii) observes the access patterns. [RSU95] show that for conjunctive queries and views finding an equivalent executable plan (i.e., deciding feasibility) is NP-complete. [DGL00] give a polynomial time algorithm for computing a maximally-contained (not necessarily equivalent) plan. The maximally-contained plan may be recursive (a datalog program) even if the query and source views are conjunctive. Summarizing, these works seek to improve "answerability" of infeasible queries including maximallycontained rewritings, while we characterize the complexity of query feasibility, i.e., equivalent rewritings. In particular, for unions of conjunctive queries with negation (UCQ ¬ ), our notation of answerable part ans(Q) provides a minimal executable and minimal feasible query containing Q.. As we will see, the notion of executability is related to that of safety. The latter can be see as a decidable syntactic approximation of domain independence (which is undecidable); see [GT91] for different classes of safe queries (e.g., evaluable, allowed, and range-restricted ). Similarly, our syntactic notions of executability and orderability can be seen as approximations of the semantic notion of feasibility.

O O B O O B O O O O O O O B O O O O B O O O O B O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O B I O O O O O O B O O O O O B O O B O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O B O O O O O O O O O O O B O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O B O B O O B B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O B O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O B O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O


Exploring network topology evolution through evolutionary computations We present an evolutionary methodology that explores the evolution of network topology when a uniform growth of the network traffic is considered. The network redesign problem is formulated as an optimization problem, subject to a set of design and performance constraints, while minimizing the redesign cost by maintaining as many as possible of the network devices that constitute the original topology. The experimental results for a 3-level network redesign problem (consisting of 65 client nodes) demonstrate the value of the search technique within the genetic algorithms in finding good solutions with respect to redesign cost and time. Networks are continually being refined and upgraded as needs change and technology evolves. In specific, the network traffic has been growing, partially due to streaming multimedia applications. Network traffic represents both the client-to-client and client-server communications. Therefore, a static traffic matrix, which was used to design a network topology, might not be valid to represent the current network workload a few months after the initial design.. The network topology redesign problem is similar to network topology design, which includes determining network topologies along with the choice of network technology, such as ATM, Ethernet, and Router, that enable all clients to communicate and access servers efficiently, while minimizing the network hardware cost. However, the network redesign software should maintain as many as possible of the original network devices as long as the redesign criteria, for example the uniform traffic growth, are satisfied. Furthermore, the cost of replacing old network devices with new network devices should be minimized. Maintaining the original devices creates a legacy problem that may complicate the network design problem, which is already known as an NPcomplete problem [1] .. We encoded a network as a tree data structure rather than the classical genetic encoding schemes such as a binary string, which requires a very long binary vector to list all inputs and design decisions. The tree structure eliminates the processing time to encode and decode the Boolean information within the network chromosome. However, using an object-oriented tree forces us to devise novel ways to implement the evolutionary mutation and crossover operators. We will improve the redesign process by considering various redesign factors, such as relocating clients, incrementing traffic in non-uniform, and splitting/merging group tasks or site tasks. Weiss and Zeyer [5] describe a case-based reasoning system for the redesign of local area networks (LAN). Their technique takes as inputs a redesign problem and a description of the LAN. Then, a case is selected on the basis of the information included in the redesign problem, where a case contains a corresponding solution (list of actions) for the described redesign problem. The work of Weiss and Zeyer was based on Klein's dissertation [2] , where he presented a framework for conflict resolution in a cooperative design. With a number of design actions (whether the actions are applied in serial or in parallel), the possibility of having conflict or creating a new conflict is great. Klein proposed a conflict resolution expert system to (1) detect the occurrence of conflict, and (2) recommend a solution for the detected conflict. This work can resolve any conflict that may occur due to the removal of a bottleneck during the network redesign process. There is no explicit redesign objective in mind when the redesign process was initiated, and hence no optimization.. An early effort by Ravikumar, Pandit, and Mishra [4] describes applying a number of classical techniques, such as the KernighanLin (KL) algorithm for partitioning and the Steiner tree for routing, for the LAN redesign problem. The KL algorithm was used to find the best possible location to add/move clients between the segments of a LAN; A Steiner tree was used to connect clients within the segment, obtaining a minimal cable cost.

O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I B O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O