Dynamic programming based algorithms for set multicover and multiset multicover problems Given a universe N containing n elements and a collection of multisets or sets over N, the multiset multicover (MSMC) problem or the set multicover (SMC) problem is to cover all elements at least a number of times as specified in their coverage requirements with the minimum number of multisets or sets. In this paper, we give various exact algorithms for these two problems with or without constraints on the number of times a multiset or set may be chosen. In this paper, we consider exact algorithms for multiset multicover (MSMC) and set multicover (SMC) problems [27, 23, 14] . The MSMC problem is a generalization of set cover in which we have multisets instead of sets and an element may be covered multiple times. Formally, we are given a universe N = {1, . . . , n}, a positive integral coverage requirement vector B = (b i ) for the elements i ∈ N and a collection of multisets F ms over N. Any one of these multisets, S ∈ F ms , may contain a number of copies of each element. The objective of the MSMC problem is to determine the minimum number of multisets such that each element i is covered at least b i times. It is similar for the SMC problem, which deals with a collection of sets F mc instead of multisets F ms . We use F sc for the collection of sets in the set cover (SC) problem. For both the MSMC $ This is an extended version with improved results of a preliminary version which appears in Proceedings of ISAAC 2009. and the SMC problems, in order to satisfy the minimization requirement, each multiset or set can be chosen multiple times. Mathematically, both problems can be posed as special cases of the covering integer programs (CIP) problem [23, 25, 24, 18, 19] as defined by Definition 1. [25] ). Let Z + denote the nonnegative integers. The CIP problem P = (A, B, D, E) seeks to minimize E T · x subject to Ax ≥ B, x ∈ Z R + and x ≤ D, where A ij , b i ∈ B and e j ∈ E for i = 1, . . . , n, j = 1, . . . , R are all nonnegative integers. Ax ≥ B and x ≤ D, where d j ∈ D is some positive integer, are called the covering constraints and the multiplicity constraints, respectively. Now, if we take the n rows and the R columns of the matrix A as the n elements in the universe N and the R sets or multisets over N = {1, . . . , n}, respectively, then the set or multiset j would contain A ij copies of element i. In the CIP problem, if we set A ij ∈ [0, c], e j = 1, b i ∈ [1, b] and remove the multiplicity constraints, we have the Multiset Multicover problem (MSMC). Here the constant values c and b mean the maximum number of times any element appears in any multiset and the maximum coverage requirement, respectively. For the MSMC problem, if we keep the multiplicity constraints x j ≤ d j , where max 1≤j≤R (d j ) = d, we have the MSMC with multiplicity constraints problem. Similarly, if we further require A ij ∈ {0, 1} and based on whether we remove the multiplicity constraints or not, we have the Set Multicover problem (SMC) and the SMC with multiplicity constraint problem. Comparing with the dynamic programming based exact algorithm for the MSMC without multiplicity constraints problem in [13] , although the DMCM algorithm (Algorithm 2) can greatly reduce the space complexity, the time complexity is still considerably high. It should be possible to devise a more efficient algorithm which uses O((b + c + 1) n ) time. It should also be interesting to design an exact multiset multicover algorithm with O * ((b + 1) n ) time; that is, the time is independent of the number of times that any element appears in a multiset.. We must emphasize that counting the number of transformed set covers for multiset multicover, i.e., the c k (F ms ) value, is different from directly counting the number of multiset multicovers, i.e., the c k (F ms ) value. Although there is now an exact algorithm for calculating c k (F ms ) [15] , it requires exponential space. So it is worthwhile to try to devise polynomial space efficient algorithms for computing c k (F ms ).. It would be worthwhile also to apply our results to some practical scenarios. For example, the minimum length wireless link scheduling with non-unit traffic demands problem [16] can be cast as a set multicover problem. Similarly, the minimum cost cell planning problem in 4G cellular networks [2] can be interpreted as a multiset multicover problem.. Finally, besides the multicovering problems studied in this paper, similar to the set partition problem studied in [5] , exact algorithms for the partition versions of set or multiset multicover problems, i.e., we need to cover all elements exactly the number of times as specified in their coverage requirements with the minimum number of sets or multisets, would be interesting to design. A natural application of this kind of partitioning algorithm is the graph multicoloring problem [4] . A straightforward method for solving the graph multicoloring problem is to first transform it into a graph coloring problem. This is realized by simply replacing each vertex v with b(v) coloring demands as a clique of b(v) copies of the vertex v, where each copy of v is adjacent to all the copies of the neighbors of v in the original graph. Then by applying the fastest exact graph coloring algorithm proposed in [5] , we can exactly solve the graph multicoloring problem in O * (2 bn ) time and O * (2 bn ) space since there are at most bn vertices in the transformed graph. This appears to be a tremendously time-and space-consuming method. We are currently working on more efficient exact algorithms for the graph multicoloring problem, including the non-preemption based version, i.e., the colors assigned to each node need to be contiguous, as well as the preemption based version. In addition, we are also working on exact algorithms for the sum multicoloring problem [4] and various machine scheduling problems [17] which can be cast as either a graph multicoloring problem or a sum multicoloring problem [4] .

B I O O O B I O B I O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


State complexity of union and intersection of square and reversal on k regular languages In the paper, we continue our study on the state complexity of combined operations on regular languages. We study the state complexities ofWe obtain the exact bounds for these combined operations and show that the state complexities of are lower than the corresponding mathematical compositions. State complexity of finite automata which is the number of states of finite automata, is an important, ongoing topic in formal languages and automata theory. Nowadays, finite automata of very large sizes are widely used in software engineering, programming languages, natural language and speech processing, and other practical areas. These applications make the research on state complexity essential and well-motivated.. The earliest research on state complexity dates back to the 1950s [20] . However, most results were obtained after 1990 with the help of powerful computers and software for experiments, e.g. Grail+ [29] . Existing literature includes studies of the state complexity of individual operations, such as catenation, union, intersection, star, reversal, shuffle, power, proportional removal, cyclic shift, etc [1, 4, 5, 6, 11, 13, 14, 15, 19, 25, 26, 27] .. However, in practice, it is often the case that the operation to be performed on finite automata is not just a single individual operation, but a combination of several individual operations in some specific order. This motivated the study of state complexity of combined operations which started in 2007 [23] . In [23] , the state complexities of (L 1 ∪ L 2 ) * and (L 1 ∩ L 2 ) * are investigated, and it is pointed out that the mathematical composition of the state complexities of the component individual operations of a combined operation cannot be directly used as the state complexity of the combined operation. Indeed, the state complexity of the combined operation can be much lower than its corresponding mathematical composition. For example, let L 1 and L 2 be two regular languages accepted by m-and n-state deterministic finite automata (DFAs), respectively. The state complexity of L * 1 is known to be 3 4 2 m and the state complexity of. n − 2 n−1 [18, 27] . Then the mathematical composition of these two state complexities for the combined operation (. From this example, we can see that although the mathematical composition of the state complexities of component individual operations does serve as an upper bound of the state complexity of the combined operation, this upper bound usually cannot be reached. Recently, it has also been shown that there does not exist a general algorithm to compute the state complexities of combined operations even if all the state complexities of individual operations are known [24] . Thus, the state complexity of each combined operation should be investigated separately.. A number of results on the state complexity of combined operations have been obtained in the past four years. Most of these results are concerned with the combined operations that consist of two different individual operations, [2, 3, 8, 10, 16, 17, 23] . Besides these basic combined operations, only a few combined operations composed of arbitrarily many individual operations have [6, 7, 9] . Clearly, combined operations with arbitrarily many individual operations are more general than basic combined operations because the latter can be viewed as the special cases of the former. Therefore, combined operations with arbitrarily many individual operations should be the emphasis of the study of state complexity of combined operations.. In this paper, we study the state complexities of four particular combined. , where L i is a regular language accepted by an n i -state DFA, 1 ≤ i ≤ k. We show that the state complexities of. for n i ≥ 3 and k ≥ 2, the same as the mathematical compositions of the state complexities of their component operations.. , we prove that their state complexities are both. for n i ≥ 3 and k ≥ 2. In contrast to the other two combined operations, in this case the state complexities of these two combined operations are lower than the mathematical compositions of the state complexities of their component operations.. In the next section, we introduce the basic definitions and notations used in the paper. In Sections 3 and 4, we investigate the state complexities of. are shown. In Section 6, we conclude the paper. In this paper, we studied the state complexities of union and intersection of squares of k regular languages, and union and intersection of reversals of k regular languages. We obtained the state complexities of the four particular combined operations. , where L i is a regular language accepted by an n i -state DFA, n i ≥ 3, 1 ≤ i ≤ k, and k ≥ 2. The state complexities of the first two combined operations are equal. They are also exactly the same as the mathematical compositions of the state complexities of their component individual operations. The state complexities of the latter two combined operations are also equal, but lower than the corresponding mathematical compositions.. In this paper, all the results are proved with increasing alphabets. In the worst-case example for. i , an alphabet of the size 2k was used.. The witness DFA for. is over a 3k-letter alphabet. It is interesting to study whether the sizes of these alphabets can be reduced. However, it is impossible to design a worst-case example for arbitrary k ≥ 2 and n i ≥ 3 with a fixed alphabet. Note that there are a limited number of different DFAs with a fixed number of states if the alphabet is fixed. Thus, when k is large enough, some of the operand DFAs with the same number of states may be indeed the same according to pigeonhole principle. Therefore, the study of state complexity of operations on k operand languages uses increasing alphabets in general.. Another possible future topic could be the state complexities of these combined operations on a smaller, fixed alphabet when k is also fixed. We expect more results on the state complexities of combined operations with arbitrarily many individual operations and operand languages.

B I O O O O O O O O O O B I O O O O O O O O O B I O B I O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O B I O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O


Control of the Effects of Regularization on Variational Optic Flow Computations We consider a variational model for the determination of the optic-flow in a general setting of non-smooth domains. This problem is ill-posed and its solution with PDE techniques includes a regularization procedure. The goal of this paper is to study a method to solve the optic flow problem and to control the effects of the regularization by allowing, locally and adaptively the choice of its parameters. The regularization in our approach is not controlled by a single parameter but by a function of the space variable. This results in a dynamical selection of the variational model which evolves with the variations of this function. Such method consists of new adaptive finite element discretization and an a posteriori strategy for the control of the regularization in order to achieve a trade-off between the data and the smoothness terms in the energy functional. We perform the convergence analysis and the a posteriori analysis, and we prove that the error indicators provide, as, a byproduct, a confidence measure which shows the effects of regularization and serves to compute sparse solutions. We perform several numerical experiments, to show the efficiency and the reliability of the method in the computations of optic flow, with high accuracy and of low density. The computation of the displacement field between the subsequent frames of an image sequence is a classical problem in image processing. This displacement field is called the optic flow and is used in many applications (medical imaging, video processing, . . .). Moreover, it belongs to a family of correspondence problems where one seeks a smooth mapping that maps the features in one image to another one. Its reliable computation constitutes a very challenging task in computer vision. Most of the methods to compute the optic flow are based on constancy assumptions on image features such as the gray levels, the brightness, . . . . In general, these assumptions are not sufficient for its determination and they allow only the estimation of the component normal to the edges of the image, this is the so-called aperture problem. It shows that the estimation of the optic flow is in fact an ill-posed problem, we refer the reader to the reviews in [12, 14, 37] and the references therein.. The optic flow problems have received much attention in the image analysis community (see [1-3, 10, 13-15, 18, 26-29, 31, 33, 34, 37] and the references therein). The variational methods are some of the most efficient approaches for solving such problems [3, 4, 14-16, 25, 38, 47-50] . They consist in searching the solutions which are minimizers of suitable energy functionals. Their success is mainly due to the fact that the functional to minimize expresses the data constraints (constancy assumptions) and regularization terms (smoothness) to cope with the aperture problem and to recover a fully dense vector fields (defined in the entire domain). A major drawback of the variational methods is the reliability of the computed fields in all locations of the computational domain. While obtaining a dense vector fields is one of the advantages of such methods (no need for extra interpolation procedure), it is difficult to distinguish which parts of the estimated fields are due only to the filling-in effect of the regularization. This reliability is of great importance in many applications which use the optic flow techniques (for instance, in medical imaging, the optic flow allows to measure some physical quantities which are difficult to measure directly [39] ).. To circumvent this difficulty, some confidence measures were proposed earlier in the literature to assess the reliability and to sparsify the estimated optic flow. The most widely used confidence measure is based on the magnitude of the image gradients [8, 40] , but is not efficient (see the discussion in [13] ). Recently, a local energy-based measure was introduced in [13] , it seems to give better results than the previous one, but its derivation as well as its performances follow from heuristic considerations rather than by a well founded mathematical arguments. In particular, the main question on the reliability of the computations is clearly linked to a balance between the two competing terms of the energy (data constraints versus regularization) and not to the total energy. It turns out that building such confidence measures should be considered in a more general setting on how to control the regularization process (enforcing its effects only when and where it is necessary).. Most methods for the choice of the regularization parameters obey to the following paradigm [20, 21, 41, 42, 44] : Optimize, then discretize. Thus, the value of the parameter (usually a real number) is fixed thanks to some a priori estimates linking the noise magnitude and such value. This method is not robust with respect to the variations of the discretization parameters. In our case, to deal with a real valued function instead of a single parameter, we invert the paradigm: we discretize, then we optimize. To this end we use a sophisticated approach based on a posteriori estimates which allow to measure both the discretization error and the model error [5, 11] .. In this paper, starting with a variational model we propose a finite element method for computing the optic flow which includes an implicit confidence measure. Our strategy is based on locally adaptive choice of the regularization parameters which allows us to select and modify dynamically the initial model. The adaptation process is completely a posteriori and uses only the measure on the estimated optic flow, hence it differs from other approaches like the image-driven or the flow-driven regularization methods [1, 3] (where the models are given a priori) as well as from algorithmic approaches based on the multigrid method [32, 43] (where there is one single parameter of regularization and the number of cycles is fixed a priori). Moreover, even if the procedure of selecting locally the parameters is nonlinear (i.e. belongs to the nonlinear approximation theory), the variational model remains linear which is an advantage when comparing to flow-driven models [4] .. This new approach of the control of the regularization increases the efficiency of most existing variational models currently used to solve problems of image analysis and of computer vision. In fact, it should be considered as a tool for selecting suited models by adjusting the influence of each term in the energy functional where various constraints of the considered problem are expressed. The selection is automatic and almost no hand tuning is required, which constitutes a promising method, in particular to design or to simplify some nonlinear models. Moreover, the method introduced in this paper, benefit from each improvement of the model (the energy functional) with respect to the data.. The adaptivity process which allows the local choice of the parameters is performed with a rigorous analysis of the residual error estimators. These are error indicators which are equivalent to the discretization error in an appropriate norm [9, 35, 45] . We use a modified energy norm depending on the regularization parameters used as scaling factors in the error indicators. Our goal is not only to provide an efficient method to solve optic flow problems but also to show how this error indicators constitute a natural confidence measure which yield a low density flow fields (i.e. achieve the goal of sparsification of the computed vector fields). In addition we use the error indicators to improve the computations results and to also build the "best" mesh for the optic flow. The method is, from this point of view, self contained and no extra confidence measures, nor sparsification process are needed. The method should also work with the finite difference method but our choice is related to the fact that the analysis and the adaptive process are more usual in the framework of finite element methods. In addition, this choice is enforced by the fact that well suited meshes for optic flow are usually unstructured. The cost of the interpolation on structured meshes where the images are given is low compared to cost of the computations in such grids (which are very large, particularly for high resolution images). Moreover, in many applications, the optic flow computations are used to estimate some physical data (wavefront, velocity of the blood in vessels or in the heart [39] , . . .). This data serve in other simulations in various finite element codes.. There exists a wide variety of variational models for optic flow determination [12, 46] , linear, nonlinear, discontinuitypreserving,. . . . In order to apply our program and to show the reliability of the method, we will consider a linear model introduced in [14, 15] called the CLG (Combined LocalGlobal) approach which is representative of variational linear optic flow methods. In a second part of this work (a forthcoming paper) we will present the method for some nonlinear models.. The paper is organized as follows: In Sect. 2 we state the variational optic flow model and we establish its principal properties. Section 3 is devoted to the finite element dis-cretization and the well posedness. In Sect. 4, we perform the a posteriori analysis and define the error indicators. In Sect. 5, we discuss implementation issues. Numerical experiments are presented in Sect. 6 . In Appendix A, we establish the convergence of the discrete solutions and Appendix B is devoted to the (rather) technical derivation of the a posteriori error estimates which justify mathematically our approach of the control of regularization. We have proposed a new a posteriori strategy to control the effects of the regularization of ill-posed problems. It allows locally adaptive choice of the regularization parameters which is very promising for an efficient and automatic model selection. This approach applied to the optical flow estimation yields a low density field with good accuracy. The method is not a new model to compute the optic flow but should be seen as a tool to enhance the existing variational methods [46] , allowing the dynamical selection of the model by adaptive changes in the energy functional. This approach is at the beginning, and many improvements should be worked out, for example to go towards real time computations, . . . . It should also be used with the more sophisticated models proposed currently in the literature for its full validation. We restricted ourselves to simple models in or- der to focus on the main ideas but much remains to be done for improvements. Finally note that in the image analysis problems, nonlinear regularization methods, such as BVregularization, are more suitable to preserve edges [36] . The use of our method in this last case is under consideration. We believe, that the combination of our method and existing regularization techniques, image-driven, flow-driven, and multigrid methods, is a challenging and promising direction of research to solve ill-posed problems.

O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Secrecy in Multiagent Systems We introduce a general framework for reasoning about secrecy requirements in multiagent systems. Our definitions extend earlier definitions of secrecy and nondeducibility given by Shannon and Sutherland. Roughly speaking, one agent maintains secrecy with respect to another if the second agent cannot rule out any possibilities for the behavior or state of the first agent. We show that the framework can handle probability and nondeterminism in a clean way, is useful for reasoning about asynchronous systems as well as synchronous systems, and suggests generalizations of secrecy that may be useful for dealing with issues such as resource-bounded reasoning. We also show that a number of well-known attempts to characterize the absence of information flow are special cases of our definitions of secrecy. In the past two decades there have been many attempts to define what it means for a system to be perfectly secure, in the sense that one group of agents is unable to deduce anything at all about the behavior of another group. More generally, many articles in computer science have, in a variety of different settings, defined properties of secrecy and have discussed techniques for achieving these properties. In the computer-security literature, early definitions of "perfect security" were based on two different intuitions. Noninterference [Goguen and Meseguer 1982] attempted to capture the intuition that an agent at a high security level is unable to interfere with an agent at a lower security level, while nondeducibility [Sutherland 1986 ] attempted to capture the intuition that an agent at a low security level is unable, to deduce anything about the state of agents at a higher security level. Other definitions have involved a notion of "information flow," and taken a system to be secure if it is impossible for information to flow from a high-level user to a low-level user. With these basic ideas in mind, definitions of security have been provided for a wide variety of system models, including semantic models that encode all possible input/output behaviors of a computing system and language-based models that deal with process algebras and with more traditional constructs such as imperative programming languages. (Focardi and Gorrieri [2001] provide a classification of security properties expressed using process algebras; Sabelfeld and Myers [2003] give a survey of language-based techniques.). Sutherland's definition of nondeducibility was based on a simple idea: a system can be described as a set of "worlds" that encode the local states of users, and security is maintained if high-level and low-level states are independent in the sense that a low-level user can never totally rule out any high-level state based on his own local state. As we shall see in Section 4, nondeducibility is closely related to Shannon's [1949] probabilistic definition of secrecy in the context of cryptography, which requires high-level and lowlevel events to be probabilistically independent. This can be shown to imply that the low agent's posterior probability of a high event should be the same as his prior probability of that event before he began interacting with the system. (Nondeducibility is also closely related to Cohen's [1977] definition of strong dependency. Cohen's work is concerned with information transmission over channels, which are represented as functions that transform inputs into outputs, whereas Sutherland's work is concerned with possible worlds that represent states of a system, but their definitions are essentially identical.). Definitions of noninterference based on Goguen and Meseguer's early work (see, for example, McCullough [1987] and McLean [1994] ) are quite different in flavor from the definitions of Shannon and Sutherland. Typically, they represent the system as a set of input/output traces, and deem the system secure if the set of traces is closed under operations that add or remove high-level events. Variants of this idea have been proposed to deal with issues such as verification, system composition, timing attacks, and so on. Although these definitions have been useful for solving a variety of technical problems, the complexity of some of this work has, in our view, obscured the simplicity of earlier definitions based on the notion of independence. We claim that the basic principle of nondeducibility captures notions of secrecy in an elegant and useful way.. In this article we define secrecy in terms of an agent's knowledge, using the "runs-and-systems" framework [Fagin et al. 1995] . The runs-and-systems · 5: 3 framework generalizes the standard input/output trace models that have been used in many definitions of noninterference. Trace-based approaches focus only on input and output values, which makes them poorly suited to modeling systems where agents have richer states than sequences of input and output events. For example, they are insufficient for modeling properties such as deadlock [Focardi and Gorrieri 2001] and for describing semisynchronous systems (where agents know the time to within some tolerance ǫ). The added generality of the runs-and-systems approach lets us deal with these issues in a straightforward way.. Many frameworks for reasoning about secrecy and information flow have assumed, often implicitly, a very coarse notion of uncertainty. Either an agent knows, with certainty, that some fact is true, or she does not; a definition of secrecy (with respect to some agent) amounts to a characterization of which facts the agent must not know, or which facts she must think are possible. Indeed, this is precisely the intuition that we make precise in Section 3.2. In the literature, such definitions are called possibilistic, because they consider only what agents consider possible or impossible. In practice, however, such a coarse-grained notion of uncertainty is simply too weak; it is easy to concoct examples where one agent has possibilistic secrecy, but where intuition suggests that secrecy is not maintained. (See the introduction to Section 4 for such an example.) We extend our definitions of secrecy to incorporate probability, a much more fine-grained notion of uncertainty. Just as Shannon's definitions of secrecy can be viewed as a probabilistic strengthening of Sutherland's definition of nondeducibility, our definitions of probabilistic secrecy generalize the possibilistic definitions we give. In fact, there is a sense in which they are the same definitions, except with a different measure of uncertainty. (This intuition can be made precise using plausibility measures. See Halpern and O'Neill [2005, Section 5 ] for details.). Our approach has an additional advantage: it enables us to provide syntactic characterizations of secrecy, using a logic that includes modal operators for reasoning about knowledge and probability. We discuss what it means for a fact to "depend on" the state of an agent and show that secrecy can be characterized as the requirement that low-level agents never know any fact that depends on the state of a high-level agent. (In the probabilistic case, the requirement is that low-level agents must think that any such fact is equally likely at all points of the system.) This knowledge-based characterization lets us make precise the connection between secrecy (of the high-level agent with respect to the low-level agent) and the notion of a secret, that is, a fact about the system that an agent is not allowed to know. This syntactic approach also opens the door to natural generalizations of information-flow properties that require secrecy for only some facts, as well as allowing us to consider notions of secrecy based on more computational notions of knowledge, which may be more appropriate for resource-bounded agents.. As we show in Section 5, our approach provides insight into a number of other definitions of secrecy and noninterference that have been proposed in the literature. We illustrate this point by considering separability [McLean 1994] , generalized noninterference [McLean 1994] , nondeducibility on strategies [Wittbold and Johnson 1990] , and probabilistic noninterference [Gray and Syverson 1998 ]. One of our goals in this section, obviously, is to convince the reader that our definitions are in fact as general as we claim they are. More importantly, we hope that providing a unified framework for comparing definitions of secrecy will facilitate the cross-fertilization of ideas.. We believe that it is important to provide broad, general definitions of secrecy and noninterference for a general class of multiagent systemsdefinitions that emphasize the underlying unity of the notions we are trying to capture. Our work is intended to do just that. Although our definitions should be appropriate for a wide variety of settings, to motivate our various definitions we discuss example programs written in a simple imperative language with input and output operators that allow high-level users and lowlevel users to interact with the system at runtime. (The language is similar to that considered by O'Neill et al. [2006] , who discuss secrecy-based properties for interactive, imperative programming languages and prove that such properties can be enforced using a well-known type system.) We hope the examples will make it clear that the definitions apply equally well in other settings.. The rest of the article is organized as follows. Section 2 reviews the multiagent systems framework and the definition of knowledge in multiagent systems. In Section 3 we define secrecy and relate it to Sutherland's notion of nondeducibility. We also consider syntactic definitions of secrecy using a logic of knowledge. Section 4 considers probabilistic secrecy. In Section 5 we compare our definitions with others that have been given in the security literature. We conclude in Section 6. Most proofs are deferred to the appendix. We have defined general notions of secrecy for systems where multiple agents interact over time, and have given syntactic characterizations of our definitions that connect them to logics of knowledge and probability. We have applied our definitions to the problem of characterizing the absence of information · 5: 33 flow, and have shown how our definitions can be viewed as a generalization of a variety of information-flow definitions that have been proposed in the past.. We are not the first to attempt to provide a general framework for analyzing secrecy; see, for example, Focardi and Gorrieri [2001] , Mantel [2000] , McLean [1994] , Ryan and Schneider [1999] , and Zakinthinos and Lee [1997] for some other attempts. However, we believe that our definitions are more closely related to the intuitions that people in the field have had, because those definitions have often been expressed in terms of the knowledge of the agents who interact with a system.. Our definitions of probabilistic secrecy (and their plausibilistic generalizations) demonstrate the underlying simplicity and unity of our definitions. Likewise, our results on the symmetry of secrecy illustrate the close connection between notions of secrecy and independence. The definitions and results that we have presented, and their underlying intuitions of knowledge and independence, do not depend on the particular system representation that we describe here, so they should be broadly applicable. Indeed, one major theme of our work is the importance of having good system models. Given the right system model and the right measure of uncertainty, a reasonable definition of secrecy-often some form of run-based f -secrecy-usually follows quite easily. By providing a general, straightforward way to model systems, the runs-andsystems framework provides a useful foundation for appropriate definitions of security.. Although we have discussed secrecy largely with respect to the kinds of input and output systems that have been popular with the theoretical security community, our definitions of secrecy apply in other contexts, such as protocol analysis, semantics for imperative programming languages, and database theory. Chor, Goldreich, Kushilevitz, and Sudan [1998] , for example, consider the situation where a user wants to query a replicated database for some specific database item, but wants a guarantee that no one will be able to determine, based on his query, which item he wants. It is not hard to show that the definition of privacy given by Chor et al. is a special case of secrecy in an adversarial system with a cell corresponding to each possible item choice.. There are several possible directions for future work. One is the verification of secrecy properties. Because we have provided syntactic characterizations of several secrecy properties in terms of knowledge and local propositions, it would seem that model-checking techniques could be applied directly. (van der Meyden [1998] gives some recent results on model-checking in the runs-andsystems framework.) However, verifying a secrecy property requires verifying an infinite set of formulas, and developing techniques to do this efficiently would seem to require some nontrivial advances to the state of the art in model checking. Of course, to the extent that we are interested in more limited forms of secrecy, where an agent is restricted from knowing a small set of formulas, knowledge-based model-checking techniques may be immediately applicable. At any rate, we emphasize that it is our goal in this article to provide general techniques for the specification, rather than the verification, of secrecy.. Another direction for future work is a careful consideration of how secrecy definitions can be weakened to make them more useful in practice. Here we briefly consider some of the issues involved: -Declassification: Not all facts can be kept secret in a real-world computer system. The canonical example is password checking, where a system is forced to release information when it tells an attacker that a password is invalid. Declassification for information-flow properties has been addressed by, for example, Myers et al. [2004] . It would be interesting to compare their approach to our syntactic approach to secrecy, keeping in mind that our syntactic definitions can be easily weakened simply by removing facts from the set of facts that an agent is required to think are possible. -Computational secrecy: Our definitions of secrecy are most appropriate for attackers with unlimited computational power, since agents "know" any fact that follows logically from their local state, given the constraints of the system. Such an assumption is unreasonable for most cryptographic systems, where secrecy depends on the inability of attackers to solve difficult computational problems. The process-algebraic approach advocated by Mitchell et al. [2006] and the work on probabilistic algorithmic knowledge of Halpern and Pucella [2003b] may help to shed light on how definitions of secrecy can be weakened to account for agents with computational limitations. -Quantitative secrecy: Our definitions of probabilistic secrecy require independence: an agent's posterior probability distribution on the possible local states of a secret agent must be exactly the same as his prior distribution. This requirement can be weakened using the information-theoretic notions of entropy and mutual information. Rather than requiring that no information flow from one user to another, we can quantitatively bound the mutual information between their respective local states. Information-theoretic approaches to secrecy have been discussed by Wittbold and Johnson [1990] , and more recently by Clark et al. [2002] , Lowe [2002] , and Di Pierro et al. [2002] . -Statistical privacy: In some systems, such as databases that release aggregate statistical information about individuals, our definitions of secrecy are much too strong because they rule out the release of any useful information. Formal definitions of secrecy and privacy for such systems have recently been proposed by Evfimievski et al. [2003] and by Chawla et al. [2005] . These definitions seek to limit the information that an attacker can learn about a user whose personal information is stored in the database. It would be interesting to cast those definitions as weakenings of secrecy.. These weakenings of secrecy are all conceptually different, but it seems highly likely that there are relations and connections among them. We hope that our work will help to clarify some of the issues involved. We are certainly not the first to discuss formal definitions of secrecy: many definitions have been proposed over the last two decades. One reason for this is that researchers have sought an "ideal" definition of security that has a variety of useful properties (such as verifiability and composability). While we certainly agree that verifiability and composability are important properties, we believe that the intuition behind secrecy should be isolated from stronger properties that happen to imply secrecy-especially when we have to worry about subtle issues such as probability and nondeterminism.. In this section we consider how our definitions relate to other informationflow conditions. We show in particular how they can capture work that has been done in the synchronous setting, the asynchronous setting, and the probabilistic setting. Because there are literally dozens of articles that have, in one way or another, defined notions of secrecy or confidentiality, this section is in no way meant to be comprehensive or representative. Rather, we have chosen examples that inspired our definitions, or examples for which our definitions give some insight. In light of our earlier comments, we also focus on definitions that have tried to capture the essence of secrecy rather than notions that have been more concerned with issues like composability and verification.. One important strand of literature to which we do not compare our work directly here is the work on defining information flow and noninterference using process algebras related to CCS and CSP Gorrieri 1994, 2001; Ryan and Schneider 1999; Ryan et al. 2001 ]. Although we believe that the intuitions behind many of these definitions are closely related to our notions of secrecy, a careful discussion of this issue would take us too far afield.. One possibility for future work is a careful consideration of how processes can be translated to the runs-and-systems framework in a way that captures their semantics, and then shows how some of the process-algebraic definitions can be recast as examples of secrecy. In Halpern and O'Neill [2003] we give one instance of such a translation: we show how definitions of anonymity given using CSP by Schneider and Sidiropoulos [1996] can be captured in the runsand-systems framework.

B O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Tracer spectrum: a visualisation method for distributed evolutionary computation Abstract We present a novel visualisation method for island-based evolutionary algorithms based on the concept of tracers as adopted in medicine and molecular biology to follow a biochemical process. For example, a radioisotope or dye can be used to replace a stable component of a biological compound, and the signal from the radioisotope can be monitored as it passes through the body to measure the compound's distribution and elimination from the system. In a similar fashion we attach a tracer dye to individuals in each island, where each individual in any one island is marked with the same colour, and each island then has its own unique colour signal. We can then monitor how individuals undergoing migration events are distributed throughout the entire island ecosystem, thereby allowing the user to visually monitor takeover times and the resulting loss of diversity. This is achieved by visualising each island as a spectrum of the tracer dye associated with each individual. Experiments adopting different rates of migration and network connectivity confirm earlier research which predicts that island models are extremely sensitive to the size and frequency of migrations. Distributed forms of evolutionary computation are growing in popularity amongst practitioners, and a recent special issue of the Genetic Programming and Evolvable Machines journal was dedicated to parallel and distributed evolutionary algorithms [17, 18] . In the past 10 years there have been a few studies which have attempted a detailed analysis of the behaviour of these algorithms. This is an unenviable challenge due to the added complexity brought about by additional issues beyond those addressed by standard theoretical analysis, such as the topology of population connections, migration rates and sizes, and the homogeneity versus heterogeneity of the different populations. Notable studies that undertake such an analysis include Cantú-Paz [1] , Fernández et al. [6] and Skolicki [13, 14] , which analyse the impact of migration rates and sizes, population sizing and network topology.. We present a visualisation tool which can be used to analyse the behaviour of distributed evolutionary algorithms in real-time. Such a tool can present the possibility of facilitating the choice of particular parameter settings and alerting the user to undesirable situations that may arise, such as loss of diversity, and can also be used to verify existing and future theoretical analysis of these distributed systems.. The remainder of this letter is structured as follows. A brief background to visualisation in evolutionary computation and distributed forms of evolutionary computation is provided with an emphasis on the island model in Sect. 2, which is followed by an introduction to the Tracer Spectrum visualisation approach in Sect. 3. Finally some conclusions and future work are outlined in Sect. 5. A novel visualisation tool, Tracer Spectrum, for distributed evolutionary algorithms is introduced. This is a valuable contribution to the visualisation toolkit, which to date is lacking specific visualisation tools to monitor distributed EC systems. Initial use of the tool provides visual evidence to support recent findings with respect to the importance of the frequency and size of migration events in an island-based framework. The tracer spectrum tool can be used to validate theoretical developments in distributed evolutionary computation, and as a tool for the practitioner to monitor the flow of individuals through their distributed ecosystem and their corresponding relaxation time (i.e., convergence to an equilibrium), facilitating the detection of undesirable behaviour such as loss of diversity. Tracer spectrum also has a role to play as an educational tool to demonstrate distributed evolutionary computation, and can help to build intuition in researchers and practitioners. A number of extensions and variations on the basic tracer spectrum idea were proposed, and it is hoped that these may provide some inspiration for future research directions in this area. Fig. 8 An illustration of how island origin can be traced using the fixed dye method for GP trees. Each node is marked with the origin island tracer colour. It is then possible to trace the origin of each node in a population back to its island of origin

O O O B O O B I I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Enhancements of Trapdoor Permutations Abstract. We take a closer look at several enhancements of the notion of trapdoor permutations. Specifically, we consider the notions of enhanced trapdoor permutation . These enhancements arose in the study of Oblivious Transfer and NIZK, but they address natural concerns that may arise also in other applications of trapdoor permutations. We clarify why these enhancements are needed in such applications, and show that they actually suffice for these needs. This article surveys and studies two enhancements of the notion of trapdoor permutations (TDP). Our exposition clarifies how these enhancements of TDP emerge in two central applications, and shows that these enhancements actually suffice for the corresponding applications. As is often the case when studying known definitions, we find it useful to start with a review of the historical roots of the various definitions.

O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Determining the Automorphism Group of the Linear Ordering Polytope In this paper we explore the combinatorial automorphism group of the linear ordering polytope P n LO for each n > 1. We establish that this group is isomorphic to Z 2 Sym(n + 1) if n > 2 (and to Z 2 if n = 2). Doing so, we provide a simple and uni ed interpretation of all the automorphisms. The linear ordering polytope is a familiar object from polyhedral combinatorics. It is de ned as the convex hull of the 0/1{vectors encoding linear orders (or total orders) on a given base set. Exploiting results on the facial structure of this family of polytopes and using advanced techniques in linear programming, e cient algorithms could be designed to solve real-world instances of some hard combinatorial optimization problems. For example, the triangulation problem for input-output tables can be formulated as a linear program on the linear ordering polytope. This problem asks, given a matrix of n n positive real numbers, to permute simultaneously the rows and columns of this matrix in order to maximize the sum of the entries above the main diagonal. Using results on the linear ordering polytope, Reinelt 13 ] was able to solve triangulation problems up to n = 60.. In our case, the base set is the set n = f1; 2; : : : ; ng. Its members are called elements. We denote by A n (or simply A when no confusion can occur) the set of arcs on the base set: A n = f(i; j) j i; j 2 n; with i 6 = jg. Here, a linear order (on the base set) is a complete, antisymmetric and transitive subset of A. Each linear order is encoded by means of its`characteristic vector' in the real vector space R A . This vector space has dimension jAj = n(n ? 1) and its canonical basis is fe (i;j) j (i; j) 2 Ag. Moreover, if x 2 R A then x (i;j) is the coordinate of x associated to the arc (i; j). Given a subset R of A (that is, an antire exive relation R on the base set), the characteristic vector (R) of R is de ned by (R) = P (i;j)2R e (i;j) . In order to keep concise notations, we use upper case letters for relations and lower case letters for their characteristic vectors: if R A, then r is the same thing as (R). As stated above, the linear ordering polytope P n LO is the convex hull in R A of characteristic vectors of all linear orders on the base set: , and 488 602 996 facets were found for n = 8 (see 2] { this description is conjectured to be complete). Wide families of facet-de ning inequalities are known as for example: the k-fence inequalities, the k-wheel inequalities, the Z k inequalities, the M obius ladder inequalities 9,13], the -critical fence inequalities 10], the (m; k)-fence inequalities 1], the t-reinforced k-fence inequalities, the augmented t-reinforced k-fence inequalities 11], the Paley inequalities 8] and the r-extended M obius ladders inequalities 7] . The computational complexity of some of the corresponding separation problems is studied in 13, 12] .. Recently, Bolotashvili, Girlich and Kovalev 1] used symmetries of the linear ordering polytope to obtain new facets. Symmetries of a polytope are usually called automorphisms. They can be of di erent kinds 15]: e.g. vectorial, a ne, euclidian. In this paper, we focus on the most general kind of automorphisms: combinatorial automorphisms, which are simply de ned as permutations of the vertex set mapping each face to a face. More precisely, identifying any face with its set of vertices, we de ne a combinatorial automorphism of a polytope P as a permutation of the set Vert(P) (the vertex set of P) such that (F) = f (v) j v 2 Fg is a face of P for each face F of P. A ne (resp. euclidian) automorphisms of a polytope P in R d are permutations of its vertices induced by a ne permutations (resp. isometries) of R d mapping P to itself. Clearly, every euclidian automorphism is an a ne automorphism and every a ne automorphism is a combinatorial automorphism.. Subsequently, we will only consider combinatorial automorphisms (they will be called simply`automorphisms'). The automorphisms of a polytope P form the group Aut(P) (for an excellent introduction to group theory, we recommend Fraleigh's book 6]). Knowing the full automorphism group of a polytope helps studying its facial structure: automorphisms can create links between families of facets that seem unrelated at rst sight, and sometimes`old' facets can be recycled to produce new ones. For instance, Doignon and Regenwetter 3, 4] were able to produce further facets of the approval voting polytope using somè strange' automorphisms.. In this paper, we show that all automorphisms of P n LO can be obtained by composing the obvious ones (which were known to Reinelt 13] ) with the automorphisms found by Bolotashvili, Girlich and Kovalev 1] (de nitions are recalled in the next paragraph). Moreover, we provide a simple and uni ed interpretation for the automorphisms by extending the base set (see Section 3). It turns out that every automorphism of the linear ordering polytope is an euclidian automorphism.. We now de ne the`relabeling',`duality' (called arc reversal in 13]) and`exchange' automorphisms. Each permutation of the base set yields a linear permutation R of R A de ned by R (e (i;j) ) = e ( (i); (j)) , which induces the relabeling automorphism R (no ambiguity should arise from the use of the same notation). The duality automorphism D simply maps the vertex corresponding to a linear order L to the vertex corresponding to its dual linear order L ?1 = f(y; x) j (x; y) 2 Lg. It is also induced by a linear permutation D of R A , de ned by D(e (i;j) ) = e (j;i) . For each xed element r, the a ne permutation E r de ned by E r (x) (i;r) = x (r;i) ; E r (x) (r;j) = x (j;r) ; E r (x) (i;j) = x (i;j) + x (j;r) + x (r;i) ? 1 (for all distinct elements i, j with i; j 6 = r), induces the exchange automorphism E r . These were recently introduced by Bolotashvili, Girlich and Kovalev 1]. As will be shown later, the relabeling, duality and exchange automorphisms span the whole automorphism group of the linear ordering polytope and this group is isomorphic to Z 2 Sym(n + 1) when n > 2. If X is any set, Sym(X) denotes the symmetric group on X { the group of all permutations of the set X { and if k is an integer greater than zero, then Sym(k) = Sym(f1; 2; : : : ; kg). Note that every relabeling automorphism is a composition of exchange automorphisms, so duality and exchange automorphisms also span Aut(P n LO ). As P 2 LO is a closed segment, its automorphism group is isomorphic to Z 2 . From now on, we assume n > 2.

O O B I O O B I I O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O


Information-theoretic approaches to branching in search Deciding what to branch on at each node is a key element of search algorithms. We present four families of methods for selecting what question to branch on. They are all information-theoretically motivated to reduce uncertainty in remaining subproblems. In the first family, a good variable to branch on is selected based on lookahead. In real-world procurement optimization, this entropic branching method outperforms default CPLEX and strong branching. The second family combines this idea with strong branching. The third family does not use lookahead, but instead exploits features of the underlying structure of the problem. Experiments show that this family significantly outperforms the state-of-the-art branching strategy when the problem includes indicator variables as the key driver of complexity. The fourth family is about branching using carefully constructed linear inequality constraints over sets of variables. Search is a fundamental technique for problem solving in AI and operations research (OR). At a node of the search tree, the search algorithm poses a question, and then tries out the different answers (which correspond to the branches emanating from that node). Many different ways of deciding which question to pose (branching strategies) have been studied. In this paper we introduce a new paradigm for developing branching strategies, employing information theory * This was work was funded by, and conducted at, CombineNet, Inc., Fifteen 27th St., Pittsburgh, PA 15222.. as the principle that guides the development. In the context of solving integer programs, we develop four high-level families of strategies, and show that some of those families significantly improve performance over existing strategies. All four strategies aim to reduce the uncertainty in the current subtree.. The rest of this section introduces the necessary background material. Each of the following four sections introduces one high-level family of new strategies. We introduced a new paradigm for branch selection in search based on an information-theoretic approach. In the beginning of a search, there is the most uncertainty about the optimal solution. When the search is complete, there is zero uncertainty. Using this observation, we developed four families of methods for selecting what question to branch on at a search node so as to reduce the amount of uncertainty in the search process. All four of our families of methods are information-theoretically motivated to reduce remaining entropy.

O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O


An unconditional study of computational zero knowledge We prove a number of general theorems about CZK, the class of problems possessing computational zero knowledge proofs. Our results are unconditional, in contrast to most previous works on CZK which rely on the assumption that one-way functions exist.We establish several new characterizations of CZK, and use these characterizations to prove results such as:1. Honest-verifier CZK equals general CZK.Public-coin CZK equals private-coin CZK."monotone formula closure"). CZK with imperfect completeness equals

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Belief Propagation for Continuous State Spaces: Stochastic Message-Passing with Quantitative Guarantees The sum-product or belief propagation (BP) algorithm is a widely used messagepassing technique for computing approximate marginals in graphical models. We introduce a new technique, called stochastic orthogonal series message-passing (SOSMP), for computing the BP fixed point in models with continuous random variables. It is based on a deterministic approximation of the messages via orthogonal series expansion, and a stochastic approximation via Monte Carlo estimates of the integral updates of the basis coefficients. We prove that the SOSMP iterates converge to a δ-neighborhood of the unique BP fixed point for any tree-structured graph, and for any graphs with cycles in which the BP updates satisfy a contractivity condition. In addition, we demonstrate how to choose the number of basis coefficients as a function of the desired approximation accuracy δ and smoothness of the compatiblity functions. We illustrate our theory with both simulated examples and in application to optical flow estimation. Graphical models provide a parsimonious yet flexible framework for describing probabilistic dependencies among large numbers of random variables. They have proven useful in a variety of application domains, including computational biology, computer vision and image processing, data compression, and natural language processing, among others. In all of these applications, a central computational challenge is the marginalization problem, by which we mean the problem of computing marginal distributions over some subset of the variables. Naively approached, such marginalization problems become intractable for all but toy problems, since they entail performing summation or integration over high-dimensional spaces. The sum-product algorithm, also known as belief propagation (BP), is a form of dynamic programming that can be used to compute exact marginals much more efficiently for graphical models without cycles, known as trees. It is an iterative algorithm in which nodes in the graph perform a local summation/integration operation, and then relay results to their neighbors in the form of messages. Although it is guaranteed to be exact on trees, it is also commonly applied to graphs with cycles, in which context it is often known as loopy BP. For more details on graphical models and belief propagation, we refer the readers to the papers [14, 27, 2, 15, 28] .. In many applications of graphical models, we encounter random variables that take on continuous values (as opposed to discrete). For instance, in computer vision, the problem of optical flow calculation is most readily formulated in terms of estimating a vector field in R 2 . Other applications involving continuous random variables include tracking problems in sensor networks, vehicle localization, image geotagging, and protein folding in computational biology. With certain exceptions (such as multivariate Gaussian problems), the marginalization problem is very challenging for continuous random variables: in particular, the messages correspond to functions, so that they are expensive to compute and transmit, in which case belief propagation may be limited to small-scale problems. Motivated by this challenge, researchers have proposed different techniques to reduce complexity of BP in different applications [3, 25, 8, 12, 7, 13, 23, 19] . For instance, various types of quantization schemes [7, 13] have used to reduce the effective state space and consequently the complexity. In another line of work, researchers have proposed stochastic methods inspired by particle filtering [3, 25, 8, 12] . These techniques are typically based on approximating the messages as weighted particles [8, 12] , or mixture of Gaussians [25] . Other researchers [23] have proposed the use of kernel methods to simultaneously estimate parameters and compute approximate marginals in a simultaneous manner.. In this paper, we present a low-complexity alternative to belief propagation with continous variables. Our method, which we refer to as stochastic orthogonal series message-passing (SOSMP), is applicable to general graphical models, and is equipped with various theoretical guarantees. As suggested by its name, the algorithm is based on combining two ingredients: orthogonal series approximation of the messages, and the use of stochastic updates for efficiency. In this way, the SOSMP updates lead to a randomized algorithm with substantial reductions in communication and computational complexity. Our main contributions are to analyze the convergence properties of the SOSMP algorithm, and to provide rigorous bounds on the overall error as a function of the associated computational complexity. In particular, for tree-structured graphs, we estabish almost sure convergence, and provide an explicit inverse polynomial convergence rate (Theorem 1). For loopy graphical models on which the usual BP updates are contractive, we also establish similar convergence rates (Theorem 2). Our general theory provides quantitative upper bounds on the number of iterations required to compute a δ-accurate approximation to the BP message fixed point, as we illustrate in the case of kernel-based potential functions (Theorem 3).. The reminder of the paper is organized as follows. We begin in Section 2, with the necessary background on the graphical models as well as the belief propagation algorithm. Section 3 is devoted to a precise description of SOSMP algorithm. In Section 4, we state our main theoretical results and develop some of their corollaries. In order to demonstrate the algorithm's effectiveness and confirm theoretical predictions, we provide some experimental results, on both synthetic and real data, in Section 5. In Section 6, we provide the proofs of our main results, with some of the technical aspects deferred to the appendices. Belief propagation is a widely used message-passing algorithm for computing (approximate) marginals in graphical models. In this paper, we have presented and analyzed the SOSMP algorithm for running BP in models with continuous variables. It is based on two forms of approximation: a deterministic approximation that involves projecting messages onto the span of r basis functions, and a stochastic approximation that involves approximating integrals by Monte Carlo estimates. These approximations, while leading to an algorithm with substantially reduced complexity, are also controlled: we provide upper bounds on the convergence of the stochastic error, showing that it goes to zero as O(log t/t) with the number of iterations, and also control on the deterministic error. For graphs with relatively smooth potential functions, as reflected in the decay rate of their basis coefficients, we provided a quantitative bound on the total number of basic arithmetic operations required to compute the BP fixed point to within δ-accuracy. We illustrated our theoretical predictions using experiments on simulated graphical models, as well as in a real-world instance of optical flow estimation.. Our work leaves open a number of interesting questions. First, although we have focused exclusively on models with pairwise interactions, it should be possible to develop forms of SOSMP for higher-order factor graphs. Second, the bulk of our analysis was performed under a type of contractivity condition, as has been used in past work [26, 11, 17, 21] on convergence of the standard BP updates. However, we suspect that this condition might be somewhat relaxed, and doing so would demonstrate applicability of the SOSMP algorithm to a larger class of graphical models.. Summing over indices j = 1, 2, . . . , r and recalling the expansion (40), we find that We begin by providing some background on graphical models and the belief propagation (or sum-product) algorithm.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O


Combined model for text entry rate development We combine the power law of learning and theoretical upper limit predictions to describe the development of text entry rates from users' first contact to asymptotic expert usage. The combined model makes comparing text entry methods easier. We present the rationale for the model and two candidate implementations. The first is a simple regression model with a reasonable fit to the data. The second fits measured data better, but is more complicated. The merits of text entry methods such as handwriting or touch-typing depend on the entry speed achievable. Slower methods are often justified if users lack skill in a faster method. Thus, two critical characteristics of text entry methods are expert performance and the practice required to achieve it.. A model based on the power law of learning [1] is built using text entry rate data measured in a longitudinal experiment. The model is of the form R N = aN x where N is the session number and R N is the text entry rate predicted for session N. a and x are regression constants. Models for upper limit or expert performance include those for stylus operated soft keyboards [5] , two-thumb text entry on physical keyboards [3] , and unistroke writing time [2] . We described models combining the traditional power law regression model and upper limit prediction model of text entry rate. We then proposed two models that satisfy these requirements. The models have both descriptive and predictive uses in comparing text entry methods. We are still examining models that fit the measured data better than model 1 but are simpler than model 2. The assessment of the theoretical foundations of the models is also work in progress. However, even our current imperfect models do a better job at describing the whole learning process than either of the component models.. Furthermore, they provide a clear graphical representation of a text entry method that facilitates comparisons with other methods. The method with the greatest area below the modeled curve during the period of interest is the best. A good method starts high, has a high upper limit, and reaches the upper limit fast. A not-so-good method exhibits a combination of the opposite symptoms. Figure 4 is an example of this kind of comparison. The QWERTY soft keyboard layout is initially faster, but after session 11 the OPTI layout is faster and according to the models it will remain so. In the examples above we used data from only one experiment. While other data sets produce similar results, we have not yet tested our models against all available data.

O O O B I O O O O O B I O B O O O O O O O O O O B I O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O B O O B O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O B I O O O O O O O O O O O O O O O O B I O B O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I B O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O B


Some exact tests for manifest properties of latent trait models Item response theory is one of the modern test theories with applications in educational and psychological testing. Recent developments made it possible to characterize some desired properties in terms of a collection of manifest ones, so that hypothesis tests on these traits can, in principle, be performed. But the existing test methodology is based on asymptotic approximation, which is impractical in most applications since the required sample sizes are often unrealistically huge. To overcome this problem, a class of tests is proposed for making exact statistical inference about four manifest properties: covariances given the sum are non-positive (CSN), manifest monotonicity (MM), conditional association (CA), and vanishing conditional dependence (VCD). One major advantage is that these exact tests do not require large sample sizes. As a result, tests for CSN and MM can be routinely performed in empirical studies. For testing CA and VCD, the exact methods are still imprac- * Corresponding author at: Department of Quantitative Economics University of Amsterdam, Roetersstraat 11, 1018 WB Amsterdam, The Netherlands, Tel.: +31-20-5254244; fax: +31-20-5254349.Email addresses: j.g.degooijer@uva.nl (Jan G. De Gooijer), ayuan@howard.edu (Ao Yuan) August 26, 2012 tical in most applications, due to the unusually large number of parameters to be tested. However, exact methods are still derived for them as an exploration toward practicality. Some numerical examples with applications of the exact tests for CSN and MM are provided.Preprint submitted to Computational Statistics & Data Analysis Item response theory (IRT), as opposed to the classical test theory, is a modern theory of standardized tests that are commonly used in educational and psychological measurement settings. In psychometrics, it describes the application of mathematical models to data from questionnaires and tests as a basis for measuring abilities, attitudes, or other variables. Items may be questions that have incorrect and correct responses, statements to indicate level of agreement, or patient symptoms scores, etc.. IRT makes it possible in principle to analyse a collection of test items assigned to many subjects or examinees. Using various (non)parametric methods, the goal is to estimate a property (parameter) such as an examinee's ability, attitude, intelligence or strength of some traits. The properties are not directly observable. Once the parameter estimates are obtained, statistical tests are usually conducted to assess the extent to which the parameters predict item responses given the model used. IRT generally has greater flexibility and provides more sophisticated information. It can perform many tasks that cannot be realized by using classical test theory. Some basic references to the historical literature in this field include Birnbaum (1968) , Lord and Novick (1968) , Fisher (1974) , Cressie and Holland (1983) , Joag-Dev and Proschan (1983) , Holland and Rosenbaum (1986) , Rosenbaum (1987) , Stout (1987) , Stout (1990) , van der Linden and Hambleton (1997) .. IRT models can be divided into two families: unidimensional and multidimensional. The unidimensional model assumes that the response data are unidimensional in the reference population, i.e. the item response probabilities are a function of a single underlying property. However, because of the greatly increased complexity, the majority of IRT research and applications utilize a unidimensional model. Another commonly used condition is monotonicity, i.e. the item response characteristic curves are nondecreasing functions. In this context the works by Junker (1991) , Junker (1993) , and Junker and Ellis (1997) are worth mentioning. Their main results include an asymptotic characterization of monotone unidimensional property for dichotomously-scored items in terms of a collection of physically meaningful manifest properties. This is useful because manifest properties are amenable to conventional hypothesis testing. Recently, Yuan and Clarke (2001) developed asymptotic test methods for four manifest properties: covariances given the sum are non-positive (CSN), manifest monotonicity (MM), conditional association (CA), and vanishing conditional dependence (VCD) (see Section 2 for a brief introduction). An IRT model can have none or some of the manifest properties mentioned above. However, since the desired properties are characterized by a (usually large) collection of statistics, the asymptotic validity requires unrealistically huge sample sizes. As a rule of 2 thumb, asymptotic methods will be valid if the data sample size n ≥ 30m 2 , where m is the number of unknown parameters in the problem. In practice, data sample sizes are often much smaller than that required by valid asymptotic methods. For example, in investigating some of the manifest properties based on performances of students of some grade in a given region, the data size is often in the low hundreds or less. We will see latter (Sections 2 and 4) that the required sample sizes, using asymptotic methods, are in the thousands, tens of thousands and more for the above four manifest properties. So the asymptotic tests are apparently impractical. The objective of the current paper is to construct exact tests of certain properties, for datasets with relatively small sample sizes, so that many realistic studies can be carried out in practice such as the above mentioned example.. The concept of exact test, originally proposed by Fisher (1935) In IRT inference, with data typically in the form of a table with binary entries, the null hypotheses are often composite. But for some hypotheses, testing can be performed on those specified on the boundary of the parameter set. As a result some kind of conditional independence can be achieved which gives rise to parameter-free exact tests. These simpler hypothesis tests are also tests for the original ones with the same significance level. We elaborate on the form of the exact tests in subsequent sections. In particular, exact tests for CSN and MM can be routinely performed in empirical studies. For CA and VCD, the exact methods are still impractical in most applications, due to the unusually large number of parameters to be tested. But we still derive exact computational methods for them as an exploration toward practicality. First, in Section 2, we provide key definitions, and notations. Next, in Section 3, we give four exact tests for four different manifest conditions. The finite-sample performance of two exact test statistics is considered in Section 4 by simulation for several unidimensional IRT models. This is followed by a small illustration of the tests to an empirical item response dataset. Finally, we provide some concluding remarks in Section 5.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O