Inapproximability of the kidney exchange problem Abstract. To overcome the shortage of kidneys available for transplantation, several countries have started various programmes of kidney exchanges between incompatible patient-donor pairs. This situation can be modelled as a cooperative game in which the players are the patient-donor pairs and their preferences are derived in the first step from the suitability of the donated kidney and in the second step from the length of the obtained cycle of exchanges. Although the core of such a cooperative game is always nonempty and one core solution can be found by the famous Top Trading Cycles algorithm, many questions concerning the structure of the core are NP-hard. In this paper we show that the problem of finding a core permutation that maximizes the number of patients who obtain a suitable kidney is not approximable within n 1−ε for any ε > 0 unless P = N P . The most effective treatment for kidney failure that is currently known is transplantation. However, the number of patients grows very fast and the supply of cadaveric kidneys is insufficient (for the most recent statistical data see e.g. [16] ). Moreover, very often the kidney from a willing living donor, genetic or emotional relative of a patient, is not suitable for immunological reasons.. Therefore in several countries various kidney exchange programmes have been established to systematically search for incompatible patient-donor pairs and possible kidney exchanges between them: in Romania [9] , the Netherlands [8] , USA [13] ; other countries have at least changed their legislation to enable such exchanges [6] . All practical experiences as well as simulation studies (for example [12, 13, 14] ) show that kidney exchanges really help to increase the number of transplantations.. Most transplantation centres organize just exchanges between two pairs [8, 11, 14] , however, allowing longer cycles might lead to better results [12, 13] . Moreover, it is also important to try to balance the desire of an individual patient for the best possible outcome for him with the need for a socially optimal solution and this is just the place for an application of game theory models and methods. Therefore we follow the approach started in [10] and developed in [3] , in that we represent kidney exchange as a cooperative game in which patient-donor pairs seek cyclic exchanges of kidneys. Preferences of players take into account not only the suitability of the received kidney, but also the length of the obtained cycle, as it is an accepted rule that all operations on a cycle should be performed simultaneously [13] and with longer cycles logistics becomes more complicated.. In this paper we concentrate on the core of the considered kidney exchange game. With no indifferences, it is always nonempty and the famous Top Trading Cycles algorithm, originally proposed by Gale in [15] for housing markets, can be used to find a core permutation for each kidney exchange game instance [2] . On the other hand, in [1] it was proved that in the case with indifferences it is NP-complete to decide whether the core is nonempty. However, the case with strict preferences is also not so simple. The core may contain many pemutations and an efficient description of its structure is unlikely to exist, as in [4] and [7] several decision problems concerning the structure of the core were proved to be NP-complete.. In this paper we concentrate on the problem of maximizing the number of players covered by a core permutation, as this is the number of patients for whom a suitable donor has been found. We give a very pessimistic result showing that unless P = N P , this problem is not approximable within n 1−ε for any ε > 0.

B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Soft faults and the importance of stresses in memory testing Abstract: Memory testing in general, and DRAM testing in particular, has become greatly dependent on the modification of stresses (timing, temperature and voltages) Memory testing has seen a continued growth in complexity in the past decade, due to the need to detect many newly observed subtle types of faulty behavior. The extent of this increase in test complexity is most apparently reflected in the extensive employment of stress (ST) modifications, such as timing, temperature and voltages, to increase the effectiveness of test patterns. STs today are considered as an integral part of any modern industrial memory test, used either to ensure a higher fault coverage of a given test, or to target specific failure mechanisms not detected at nominal operational conditions [Falter00] .. From a theoretical point of view, STs have always been considered necessary in the context of validating the specifications, where proper functionality of the memory is examined across the operational range specified in the data sheets [vdGoor98] . It has not yet been theoretically justified, however, why it would be necessary sometimes to apply far more stressful STs than the specifications allow, STs that bring the memory very close to total operational failure.. This paper defines a new class of memory faults, soft faults, based on a special classification of memory faults that identifies specific fault related voltage ranges within a faulty memory cell. The new fault class identifies the underlying fundamental reasons for using STs in memory testing, and makes it possible to analyze the types of STs needed for a specific defect.. Section 2 introduces a classification of memory faults and defines the concept of soft faults. To appreciate the importance of soft faults, Section 3 derives their relative probability in DRAMs and compares it with that in SRAMs. Section 4 discusses ways for detecting soft faults and describes the way they are related to STs. The concepts presented in the paper are then validated, using defect injection and electrical simulation of a memory model in Section 5. Finally, Section 6 ends with the conclusions. This paper presented a new classification of DRAM faults, that identifies the new class of soft faults. It is shown that soft faults are the reason that makes stress application a necessary part of memory testing. The importance of soft faults is validated by calculating their probability for a specific defect. In addition, ways to detect soft faults in DRAMs have been presented and evaluated using a simulation-based analysis of a number of DRAM defects.

B I O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O B I O O O O B I O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O


Sliced and Radon Wasserstein Barycenters of Measures Abstract This article details two approaches to compute barycenters of measures using 1-D Wasserstein distances along radial projections of the input measures. The first method makes use of the Radon transform of the measures, and the second is the solution of a convex optimization problem over the space of measures. We show several properties of these barycenters and explain their relationship. We show numerical approximation schemes based on a discrete Radon transform and on the resolution of a non-convex optimization problem. We explore the respective merits and drawbacks of each approach on applications to two image processing problems: color transfer and texture mixing. We introduce two novel different definitions of barycenters of multi-dimensional measures based on one-dimensional optimal transport. We show that these Radon and Sliced Wasserstein Barycenters enjoy the same invariance properties as the usual Wasserstein barycenter. They both minimize variational problems, which are almost identical, up to the lack of surjectivity of the Radon transform. We estimate this deviation to be negligible on a set of examples. We introduce Lagrangian and Eulerian discretization schemes, which enable the approximation of these barycenters with fast algorithms. The computational time is orders of magnitude faster than the Wasserstein barycenter counterpart for two input measures. Furthermore, they can be applied to more than two input densities. We show on several numerical examples that, while these barycenters exhibit significant geometrical differences with respect to the Wasserstein barycenter, they appear to be very well suited to several applications in image processing and computer graphics.. where we have introduced the following change of variable. which proves (7). Property (8) is proved similarly. Properties (9) and (11) directly follow from (8) . Computational Optimal transport. There is a vast literature on the numerical computation and approximation of the opti-

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O


Two-machine flowshop scheduling with job class setups to minimize total flowtime This paper studies the two-machine flowshop scheduling problem with job class setups to minimize the total flowtime. The jobs are classified into classes, and a setup is required on a machine if it switches processing of jobs from one class to another class, but no setup is required if the jobs are from the same class. For some special cases, we derive a number of properties of the optimal solution, based on which we design heuristics and branch-and-bound algorithms to solve these problems. Computational results show that these algorithms are effective in yielding near-optimal or optimal solutions to the tested problems. In many manufacturing settings classes of jobs are processed on one or more machines. On each machine, a setup is required at the beginning of each batch, where a batch is a maximal set of consecutively processed jobs from the same class. A machine can only process one job at a time, and cannot perform any processing while undergoing a setup. For the objective of minimizing the total flowtime, a schedule defines how batches are formed and specifies the processing order of the batches and that of the jobs within the batches.. The single-machine job class scheduling problem to minimize the (weighted) total flowtime has been widely studied by many researchers. For the case of two job classes, Gupta (1984) and Potts (1991) proposed a polynomial time algorithm and a dynamic programming algorithm, respectively. However, Gupta's algorithm is not optimal. For the case of multiple job classes, Gupta (1988) , and Ahn and Hyun (1990) proposed different heuristics. Mason and Anderson (1991) developed a branch-and-bound algorithm for the problem to minimize the mean and weighted flowtime. Crauwels et al. (1998) , too, proposed a branch-and-bound algorithm, which is superior to that of Mason and Anderson (1991) by using the derived lower bounds from a Lagrangian relaxation of the machine capacity constraints. Crauwels et al. (1997) also developed several local search heuristics, whose performance is superior to the method in Ahn and Hyun (1990) . Reviews on this topic have been presented by Potts and Van Wassenhove (1992) , Webster and Baker (1995) , and Potts and Kovalyov (2000) .. In contrast to the existence of many significant research results on the single-machine job class scheduling problem to minimize the (weighted) total flowtime, there have been few attempts to study the problem involving two or more machines (see, for example, Cheng et al., 2000) . The two-machine flowshop job class scheduling problem is evidently NP-hard because if all the setup times of the job classes are zero, it becomes the classical two-machine flowshop scheduling problem, which is NP-hard (Gonzalez and Sahni, 1978) . For the two-machine flowshop scheduling problem to minimize the mean flowtime, Woo and Yim (1998) proposed an efficient heuristic algorithm. Ho and Gupta (1995) proposed polynomial time algorithms under the dominant machine situation. When each job belongs to a different job class, the problem becomes the two-machine flowshop scheduling problem with sequence-independent setup times, for which Allahverdi (2000) developed a branch-and-bound algorithm and a heuristic. For the general case, it is highly unlikely that a polynomial algorithm can be found to solve the problem. In this paper we study several special cases of the two-machine flowshop job class scheduling problem and develop efficient algorithms for them. We solve two special cases of the problem and develop several heuristics and branch-and-bound algorithms for the other cases. The efficiency and effectiveness of the heuristics and branch-and-bound algorithms are numerically evaluated. In this paper we have considered several special cases of the two-machine flowshop scheduling with job class setups and derived some optimal solution properties for these problems to minimize the total flowtime. We have discussed the use of these optimal properties to develop heuristic algorithms. We have also developed branch-and-bound algorithms for cases 3.3, 3.4 and 3.5, in which the best solution of two heuristic algorithms is used as an initial upper bound, and the derived lower bounds and optimal properties are used to reduce the size of the search tree. We have conducted computational experiments to test these algorithms and the results demonstrate that all of the algorithms are very efficient and effective in solving the special cases under study.

O B I O B I I O O B I O O O O O B I O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O B O B I O O O O O O O O O O O O O O O B I O B I I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Stable Roommates Spanner We introduce a new geometric spanner whose construction is based on a generalization of the known Stable Roommates problem. The Stable Roommates Spanner combines the most desirable properties of geometric spanners: a natural definition, small degree, linear number of edges, strong (1 + ε)-spanner for every > 0, and an efficient construction algorithm. It is an improvement over the well-known Yao graph and Θ-graph and their variants. We show how to construct such a spanner for a set of points in the plane in O(n log 10 n) expected time. We introduce a variant of the Stable Roommates Spanner called the Stable Roommates Θ-Spanner which we can generalize to higher dimensions and construct more efficiently in O(n log d n) time. This variant possesses all the properties of the Stable Roommates Spanner except that it is no longer a strong spanner. Given a weighted graph G = (V, E) and a real number t ≥ 1, a t-spanner of G is a spanning subgraph G with the property that for every edge {x, y} in G, there exists a path between x and y in G whose weight is no more than t times the weight of the edge {x, y}. Thus, shortest-path distances in G approximate shortest-path distances in the underlying graph G and the parameter t represents the approximation ratio. Every path that approximates the shortest-path distance in the underlying graph G within a factor t is called a t-spanning path. The smallest t for which G is a t-spanner of G is called the spanning ratio or the stretch factor of the graph G .. Spanners have been studied in many different settings. The various settings depend on the type of underlying graph G, on the way weights are assigned to edges in G, on the specific value of the spanning ratio t, and on the function used to measure the weight of a shortest path. In this article, the setting is a weighted geometric graph whose vertex set is a set of points in R 2 and whose edge set consists of line segments joining two vertices. The edges are weighted by the Euclidean distance between their endpoints.. There is a vast literature on different methods for constructing t-spanners with various properties in this geometric setting (see [8] for a comprehensive survey of the area). Aside from trying to build a spanner that has small spanning ratio, additional properties of the spanners are desirable. Typical goals in this area include the construction of spanners that also have stretch factor approaching 1, few edges, bounded degree, high connectivity, and low weight, to name a few. Notice that some of these properties actually oppose each other. For example, reducing the number of edges or bounding the degree of a spanner tends to increase its stretch factor and decrease its connectivity. Therefore, one needs to balance the various properties depending on the goals one wishes to achieve.. We introduce a new approach for building spanners inspired by the Stable Roommates problem. The Stable Roommates problem consists of finding a stable perfect matching for a complete weighted graph on vertex set X. More precisely, the problem can be viewed in the following way from which its name arises. Given a set X of 2n people, each person ranks all the others with a unique number between 1 and 2n − 1 in order of preference. Let r x (y) denote the rank given to y by x. A perfect matching M is a set of n pairs {x, y} ⊂ X where each element of X appears in exactly one pair. For a vertex x, let m(x) denote the vertex it is matched to in M (i.e. {x, m(x)} ∈ M). A perfect matching M is stable provided the following holds: ∀ x∈X y∈X ((r x (y) < r x (m(x)))∧(r y (x) < r y (m(y)))), where ∧ denotes the AND operator. In other words, there are no two persons who would both rather be matched with each other over their current matched roommates.. A geometric representation of this problem has already been introduced in [1] , where a participant is represented as a point in a metric space, and the preference list of a point x is a sorted list of the other points by nondecreasing order of their distances from x. In this setting, the Stable Roommates problem was solved in O(n log n) time. Our geometric problem is slightly different. We allow a point to have more than one roommate as long as it shares a separate room with each roommate (represented by a cone). This generalization will be much clearer when we specify the exact context in the next section.. In [11] , Yao devised a method for constructing a spanner. The graph, known as the Yao graph Y k , is defined on a point set P ⊂ R 2 as follows. At each point p ∈ P, let k equally-separated rays with apex p define k cones of equal angle. For each cone with apex p, add the shortest edge {p, q} (where q is the closest point to p in the given cone) to Y k . Do this for each point in P and the resulting graph is the Yao graph. Bose et al. have proved in [2] that for k ≥ 7, Y k is a t-spanner. 2 cos(2π/k)−1 . It has also been shown that Y 4 and Y 6 are t-spanners for a constant t in [2] and [4] respectively. It is still open whether Y 5 is a t-spanner for constant t. A spanner of P is a strong t-spanner if for any pair of points p, q ∈ P, a t-spanning path between p and q that contains only edges whose length is at most |pq| exists, where |pq| denote the Euclidean distance between p and q. The unit disk graph of a point set P UDG(P) is a spanning subgraph of the complete Euclidean obtained by pruning the edges whose length is greater than one unit. The intersection of a strong spanner of P with UDG(P) is also a spanner of UDG(P). The Y k is a strong spanner, it contains a minimum spanning tree and linear number of edges, where treating t as a constant; however its degree is not bounded. In [10] Keil and Gutwin [7] introduced a variant of the Y k , known as the Θ-graph, where Θ refers to the angle of the cones. The Θ-graph is identical to the Yao graph except for how the shortest edge is selected in a given cone. Instead of using the Euclidean distance, all the points in a given cone with apex p are projected onto the bisector of the cone, and the point q whose projection is closest to p is selected. Generally, the bisector can be replaced with any ray emanating from the apex and contained in the cone. Θ-graphs are t-spanners for t = 1 1−2 sin(π/k) [9] and possess almost all the properties of Yao graphs except they are not strong spanners and not necessarily contain a minimum spanning tree. The main advantage of the Θ-graph is that distance computation via these projections allow for more efficient computation.. Combining ideas from both the Yao graph, such as defining k > 8 cones of equal angle around each input point, and the Stable Roommates problem, we introduce the Stable Roommates spanner of a point set P ⊂ R 2 , denoted SRS k (P). It is a spanner with the following properties:. 1. It has a spanning ratio at most 1/(cos(θ) − sin(θ)) where θ is the angle of a cone.. 2. It is a strong spanner.. 3. It has a maximum degree at most k.. 4. It contains a minimum spanning tree as a subgraph.. Although we prove SRS k (P) is a t-spanner for 1/(cos(θ) − sin(θ)) due to the simplicity of the proof, one can verify it is also a t-spanner for 1+ √ 2−2 cos θ 2 cos θ−1 , which is the best currently known upper bound on the stretch factor of the Y k [2] . Note that besides possessing the good properties of Y k , the spanner SRS k (P) also has a constant bounded degree as opposed to Y k . It also has a natural definition and can be constructed by a simple and intuitive algorithm in near quadratic time. We also show how to compute the Stable Roommates spanner in O(n log 10 n) expected time where n is the cardinality of the point set. By replacing the approach taken in the Yao graph with the one of the Θ-graph in the above construction, we get a variant which we call the Stable Roommates Θ-spanner. This variant possess the same properties except that it is no longer a strong spanner nor does it necessarily contain a minimum spanning tree as a subgraph. However, it can be generalized to higher dimensions and constructed in O(n log d n) time, which improves the construction time of the Stable Roommates spanner in the plane by a factor of O(log 8 n). Given any t-spanner of a point set, Gudmundsson et. al [6] show how to compute a subgraph that is a (1 + )t-spanner for every > 0 with weight at most O(1) × the weight of a minimum spanning tree. Therefore, all of the above spanners can be made to have low weight.. The paper is organized as follows. In the next section we first describe the Stable Roommates spanner, followed with an analysis of all its properties such as degree and stretch factor. Then, in Section 3 we present an O(n log 10 n) expected time algorithm for constructing the Stable Roommates Spanners. Finally, in Section 4 we show a deterministic construction with generalization to higher dimensions and lower running time of O(n log d n) for the variant of the SRS k , called the Stable Roommates Θ-spanner.

B I O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O


Software mutational robustness Abstract Neutral landscapes and mutational robustness are believed to be important enablers of evolvability in biology. We apply these concepts to software, defining mutational robustness to be the fraction of random mutations to program code that leave a program's behavior unchanged. Test cases are used to measure program behavior and mutation operators are taken from earlier work on genetic programming. Although software is often viewed as brittle, with small changes leading to catastrophic changes in behavior, our results show surprising robustness in the face of random software mutations. The paper describes empirical studies of the mutational robustness of 22 programs, including 14 production software projects, the Siemens benchmarks, and four specially constructed programs. We find that over 30 % of random mutations are neutral with respect to their test suite. The results hold across all classes of programs, for mutations at both the source code and assembly instruction levels, across various programming languages, and bear only a -013-9195-8 limited relation to test suite coverage. We conclude that mutational robustness is an inherent property of software, and that neutral variants (i.e., those that pass the test suite) often fulfill the program's original purpose or specification. Based on these results, we conjecture that neutral mutations can be leveraged as a mechanism for generating software diversity. We demonstrate this idea by generating a population of neutral program variants and showing that the variants automatically repair latent bugs. Neutral landscapes also provide a partial explanation for recent results that use evolutionary computation to automatically repair software bugs. The ability of biological organisms to maintain functionality across a wide range of environments and to adapt to new environments is unmatched by engineered systems. Understanding the intertwined mechanisms and evolutionary drivers that have led to the robustness and evolvability of biological systems is an important subfield of evolutionary biology. In this paper we focus on neutral landscapes and mutational robustness, applying these concepts to software.. Today's software arose through fifty years of continued use, appropriation, and refinement by software developers. The tools, design patterns and codes that we have today are those that have proven useful and were robust to software developer's edits, hacks and accidents, and those that survived the economic pressures of the marketplace. We hypothesize that these evolutionary pressures have caused software to acquire mutational robustness resembling that of natural systems. Mutational robustness in biological systems is believed to be intimately related to the capacity for unsupervised evolution and adaptation. We posit that software mutational robustness points to the potential for powerful methods of unsupervised software enhancement and evolution.. Robustness is important in software engineering, especially as it relates to reliability, availability or dependability. Here we focus on genetic robustness, defining software mutational robustness in terms of changes to computer code. In this context, we define a neutral mutation to be a random change applied to a program representation (source code, abstract syntax tree, assembly, binary, etc.) such that the mutated program's behavior remains unchanged on its regression test suite. 1 Thus, software fitness is assessed by the program's performance on a set of test cases. In Sect. 6.1 we discuss the use of test suites to assess software fitness. In Sect. 3.1 we present a formal definition of software mutational robustness as the fraction of mutations to a program which do not change its correctness as assessed by a set of regression tests.. Software mutational robustness measures the fraction of software mutants that are neutral. Neutral variants are equivalent to the original program with respect to the test suite. They may or may not be semantically equivalent (compute the same function) to the original program, they may or may not have the same nonfunctional properties (run-time, memory consumption, etc.), and they may or may not satisfy the specification (required behavior) of the original designers. Empirically, we find that the program's test suite is an acceptable proxy for the program specification. We find many neutral variants that are both semantically distinct from the original program and still satisfy the original program's specification or intended behavior.. This result can be understood more easily when one considers that there are an infinite number of ways to encode any algorithm in software. For example, consider this fragment of a recursive quick-sort implementation:. Swapping the order of the last two statements to changes the run-time behavior of the program without changing the output, giving an alternate implementation of the specification. We find that neutral mutations are prevalent in software and contribute to evolvability, as discussed in Sect. 4.4. Our mutation operators (delete, copy, and swap) are described in detail in Sect. 3. They are notable because they do not create new code de novo. Delete and copy are both plausible analogs of genetic operations on DNA, and all three are edit operations that are routinely performed by programmers. They are also related to operators commonly used in the genetic programming community, although we note that we do not have an explicit terminal set as typical in genetic programming. In effect, our terminal set corresponds to all of the statements contained in the program being studied.. We are interested in the extent to which mutational robustness enables software evolvability by which we mean the use of automated methods for software development and maintenance. In particular, there is increasing interest in automatic program repair, and many of the more promising approaches rely on unsound program transformations (Sect. 2.4.1). These may involve both source-level edits (e.g., [16, 36, 92, 93] ) and modification to program state at run-time (e.g., [66] ). Mutational robustness may help explain why program transformations, such as swapping two statements [92] or clamping an integer value [66] , can produce acceptable program behavior.. The primary contributions of the paper include:. 1. The empirical measurement of software mutational robustness in a large collection of off-the-shelf software, demonstrating that mutational robustness is prevalent. We find largely uniform mutational robustness scores with an average value of 36.8 % and a minimum across all software instances of 21.2 %. We evaluate this claim using 22 programs involving over 150,000 lines of code and 23,151 tests.. 2. An application of software mutational robustness to repair unknown software defects proactively. As an illustration, we seeded bugs into 11 programs, generated populations 5,000 of neutral variants using mutation, and studied the behavior of the variants on the seeded bugs using test cases withheld during neutral variant generation. In eight of the programs, the neutral population contained at least one variant that ''repaired'' the latent bug, passing the withheld test case. 3. An alternate interpretation of the software engineering technique known as mutation testing to include neutral mutations and software robustness. We discuss the relation between software functionality, software test suites and specifications, demonstrating the value of neutral mutations, both for proactively repairing unknown bugs and as a likely enabler of automated software evolution techniques.. In the remainder of the paper, we review related work in Biology and Software Engineering in Sect. 2. We then describe our software representations and mutation operators in Sect. 3. The experimental design and experimental results are given in Sect. 4. We present a practical application of software mutational robustness in Sect. 5 . Finally, we analyze our results and discuss potential threats to validity and implications in Sects. 6 and 7. The previous sections described experimental results, using three simple mutation operators, which show that software is surprisingly robust to random mutations. For the programs we tested, 37 % of the mutations had no effect on software functionality, as measured by the programs test suites. Software mutational robustness, or neutrality, is observed even in programs that are is completely correct according to their specifications. Just as neutrality is believed to enhance evolvability in naturally evolving populations, so may software neutrality enable and explain the evolvability of software, either through automated means (e.g., [92] ) or by humans.. Software robustness is potentially useful for enhancing the resilience of software systems. We demonstrate this idea by describing a method that increases software diversity, automatically generating software variants that are immune to as yet undiscovered bugs. The insights into software described here suggest several opportunities for the Software Engineering community, including the following: Creating system diversity, for example, to protect against security exploits; incorporating machine learning methods into software development and maintenance; improving program performance; or developing error-tolerant computations.. We postulate that the presence of mutational robustness in software is not an effect of intentional design, but is rather an effect of software's provenance through natural selection-even though the agents of selection, mutation and reproduction are human engineers. In this way, mutational robustness can be viewed as a property arising through inadvertent selection in both natural and engineered systems. Further study of software as an evolved system may yield new insights into those aspects of evolution that are specific to biological systems and those which are general across other complex evolved, and even engineered, systems. Because software is fundamentally easier to instrument and observe than naturally occurring populations, studying software robustness may lead in the future to an increased understanding of the role of neutrality in natural evolution. The results presented here contradict the prevailing folk wisdom that software is a precise and intentionally engineered mechanism, which is brittle to small perturbations. We find software to be inherently robust to random mutations, malleable within extensive neutral landscapes, and evolvable by combining the mutation operators described here with crossover and selection [50, 51, 92] .. This new view of software suggests a number of exciting areas for future work. These include further analysis of the extent, origins and mechanics of software mutational robustness, novel practical applications to software engineering, and new parallels between software applications and evolved biological organisms. We next discuss threats to the validity of our findings, and then explore each of these three areas of future work in turn. The previous work most closely related to software mutational robustness includes work on neutral theories in biology, investigations of the effect of neutrality in fitness landscapes in evolutionary computation and the field of mutation testing in software engineering. In the following three subsections we highlight some of the most relevant aspects of these fields.

O B I O B I O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O


Generalizing preference elicitation in combinatorial auctions Combinatorial auctions where agents can bid on bundles of items are desirable because they allow the agents to express complementarity and substitutability between the items. However, expressing one's preferences can require bidding on all bundles. Selective incremental preference elicitation by the auctioneer was recently proposed to address this problem [4], but the idea was not evaluated. In this paper we show that automated elicitation is extremely beneficial: as the number of items for sale increases, the amount of information elicited is a vanishing fraction of the information collected in traditional "direct revelation mechanisms" where bidders reveal all their valuation information. The elicitors also maintain the benefit as the number of agents increases-except rank lattice based elicitors which we show ineffective. We also develop elicitors that combine different query types, and we present a new query type that takes the incremental nature of elicitation to a new level by allowing agents to give approximate answers that are refined only on an as-needed basis. We show that determining VCG payments requires very little additional elicitation. Finally, we show that elicitation can be easily adapted to combinatorial reverse auctions, where the benefits are similar to those in auctions, except that the elicitation ratio improves as the number of agents increases. In the process, we present methods for evaluating different types of elicitation policies. Combinatorial auctions, where agents can submit bids on bundles of items, are economically efficient mechanisms for selling k items to n bidders, and are attractive when the bidders' valuations on bundles exhibit complementarity (a bundle of items is worth more than the sum of its parts) and/or substitutability (a bundle is worth less than the sum of its parts). Determining the winners in such auctions is a complex optimization problem that has recently received considerable attention (e.g., [1, 6, 11, 15, 19, 20] ).. An equally important problem, which has received much less attention, is that of bidding. There are 2 k − 1 bundles, and each agent may need to bid on all of them to fully express its preferences. This can be undesirable for any of several reasons: determining one's valuation for any given bundle can be computationally intractable [9, 13, 18] ; there is a huge number of bundles to evaluate; communicating the bids can incur prohibitive overhead (e.g., network traffic); and agents may prefer not to reveal all of their valuation information due to reasons of privacy or long-term competitiveness [16] . Appropriate bidding languages [6, 8, 11, 17, 19] can solve the communication overhead in some cases (when the bidder's utility function is compressible). However, they still require the agents to completely determine and transmit their valuation functions and as such do not solve all the issues. So in practice, when the number of items for sale is even moderate, the bidders cannot bid on all bundles. Instead, they may bid on bundles which they will not win, and they may fail to bid on bundles they would have won. The former problem leads to wasted effort, and the latter problem leads to reduced economic efficiency of the resulting allocation of items to bidders.. Selective incremental preference elicitation by the auctioneer was recently proposed to address these problems [4] , but the idea was not evaluated. In this paper we examine several different query types and elicitation policies for this purpose. We also study the complexity of not only determining the optimal allocation of items to bidders but also the bidders' Vickrey-Clarke-Groves (VCG) payments [3, 7, 21] . In addition to combinatorial auctions we study combinatorial reverse auctions (procurement auctions). Our experiments show that only a vanishing fraction of the bidders' preference information needs to be elicited in order to determine the provably optimal (economically efficient) allocation-and the VCG payments. In all of the elicitation algorithms of this paper, as the number of items for sale increases, the amount of informa- tion elicited is a vanishing fraction of the information collected in traditional "direct revelation mechanisms" where bidders reveal all their valuation information! The elicitation schemes also maintain their benefit as the number of agents increases. Rank lattice based elicitors are the exception: we showed that they scale poorly in agents and we explained why. By using the VCG pricing scheme, each agent is motivated to answer the queries truthfully, even if the agents are allowed to pass on queries and answer queries that were not asked. We showed that determining the VCG payments requires very little additional preference elicitation beyond what is needed to determine the optimal allocation.. We adapted preference elicitation to combinatorial reverse auctions and showed that, there too, as the number of items for sale increases, the fraction of preference information that is revealed approaches zero. Unlike in auctions, the elicitation ratio improves as the number of agents increases.. We experimented with several query types. Using a combination of value queries to get exact values, and order queries which are easier to answer, we can reduce the amount of exact valuation the bidders need to do. More work needs to be done to reduce the overall amount of agent-side computation. Our bound-approximation queries take the incremental nature of elicitation to a new level. The agents are only asked for rough bounds on valuations first, and more refined approximations are elicited only on an as-needed basis. A related approach would be to propose a bound, and ask whether the agent's valuation is above or below the bound. This suggest a relationship between preference elicitation and ascending combinatorial auctions where the auction proceeds in rounds, and in each round the bidders react to price feedback from the auctioneer by revealing demand (e.g., [2, 14, 22] ). As future research, we plan to explore this connection more deeply. We also desire to design new, increasingly effective preference elicitation algorithms. While the idea and some algorithms for preference elicitation in combinatorial auctions have been presented previously [4] , they have not been validated. To evaluate the usefulness of the idea, we conducted a host of experiments. We present the results in the rest of the paper. Each plot shows how many queries were needed to find an optimal allocation and prove that it is optimal (that no other allocation is better). In each plot, each point represents an average over 50 runs, where each run is on a different problem instance (different draw of valuations for the agents). Each algorithm was tested on the same set of problem instances. The plots show results for the instance sizes on which all of the 50 instances took less than 2 minutes to solve on a 2.8 GHz Linux machine.. Unfortunately, real data for combinatorial auctions are not publicly available. 4 Therefore, as in all of the other academic work on combinatorial auctions so far, we used randomly generated data. We first considered using existing benchmark distributions. However, the existing problem generators output instances with sparse bids, that is, each agent bids on a relatively small number of bundles. This is the case for the CATS suite of economically-motivated random problem instances [10] as well as for the other prior benchmarks [1, 6, 19] . In such cases, the communication is a non-issue, which undermines the purpose of elicitation. In addition, the instances generated by many of the earlier benchmarks do not honor the free disposal constraints (because for an agent, the value of a bundle can be less than that of a sub-bundle). In many real settings, each bidder has a nonzero valuation for every bundle. For example in spectrum auctions, each bidder has positive value for every bundle because each item is of positive value to every bidder (at least due to renting and reselling possibilities). In other settings, there may exist worthless items for some bidders. Even in such cases, under the free disposal assumption, the bidders have positive valuations for almost all bundles-except bundles that only contain worthless items (because, at worst, they can throw away the extra items in any bundle for free).. To capture these considerations, we developed a new benchmark instance generator. In each problem instance we generate, each bidder has a nonzero valuation for almost every bundle, and all valuations honor free disposal. Specifically, the generator assigns, for each agent in turn, integer valuations using the following routine. We impose an arbitrary maximum bid value MAXBID = 10 7 in order to avoid integer arithmetic overflow issues, while at the same time allowing a wide range of values to be expressed. Valuations generated with this routine exhibit both complementarity and substitutability.

O B I O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An adaptive fuzzy observer-based approach for chaotic synchronization This paper presents an adaptive fuzzy observer design to synchronize chaotic systems. The chaotic system is expressed in the form of Takagi-Sugeno fuzzy model (T-S fuzzy system), which considers the effect of model mismatches. Based on this model, an adaptive fuzzy observer is developed to deal with the synchronization of nonidentical chaotic systems. In contrast to the framework of parallel distributed compensation for T-S fuzzy system, the proposed method does not rely on the existence of common matrix P which is imposed in stability conditions. The computer simulation examines the performance of two well-known chaotic systems, Lorenz system and Chua circuit. The results show that the proposed approach can not only attain synchronization but also is robust to parameter perturbations in the drive system. Since the pioneering works of [5, 18] nonlinear systems with the particular properties of broadband, noise-like, and difficult to predict. With these properties the secure communication can perform transmitting messages, masked by chaotic signals or modulated in chaotic systems. The idea of secure communication is to encrypt a plain text at the transmitter and decrypt the cipher text at the receiver. The transmission channels are public in general. That means anyone is able to detect or obtain the transmitted data. Therefore, it is advisable to mask or modulate the information within a chaotic signal and retrieve it from the received signal. For these purposes, the principle of synchronization of chaotic system is utilized.. Synchronization of chaotic system has attracted much interest in study for its potential applications in many areas such as secure communication, chemical reactions, biological systems, and information processing [3] . The primary scheme of chaos synchronization is the drive-driven systems addressed by Pecora and Carroll [13] , where the output of drive system is to control the response of driven system such that coherent response of both systems can be attained. From the control point of view, synchronization can be viewed as designing proper observer [11] . The driving system usually transmits some of its states to the driven system at the receiving end. Then, based on the limited signals developing control of the observer is to recover all states of the drive system. Accordingly, the state variables of the receiver (driven) system have synchronous responses with the original (drive) system.. There are various methodologies for chaos synchronization studied in the past decade [3] . Many research works are developed on the basis of exact mathematic model of the system and noise-free condition. However, for practical application such perfection seems to be unrealistic. It is very common that the perturbations of parameters are inevitable due to device aging or varying of the operating environment. Besides, noise or disturbance does exist physically. The robust control strategy usually deals with these problems and provides an effective solution to synchronization of chaotic system. Several adaptive synchronization methods have been reported recently. For examples, Liao and Tsai [9] addressed an adaptive observer to estimate the unknown parameter and disturbance of a chaotic system with output feedback term. Chen and Lu [4] derived an adaptive algorithm to a specific uncertain unified chaotic model. Feki [7] designed complete adaptive observer-based response system to synchronize chaos with parameter uncertainties. Having been employed in the continuous-time chaotic systems, adaptive control is also applicable to synchronization of discrete-time chaos. The principle of parameter identification was cited in [6] . Based on this principle, the slave system parameters were tuned by both synchronization and optimization algorithms. Synchronization of model-unknown discrete-time generalized Henon map was discussed in [19] .. The above research works are essentially based on the explicit mathematic model to analyze and design the synchronization of continuous-time or discrete-time chaotic system. In this work, the synchronization by fuzzy control methodology is addressed. The fuzzy model of chaotic system can be obtained by model construction [15] or linearization method [14] which is adopted in this paper. Generally, the nonlinear system is linearized around some operating points to produce cer-tain local linear dynamics. These local models are then smoothly aggregated by the fuzzy inferences, which give the T-S fuzzy model. In fact a wide class of chaotic systems can be represented by T-S fuzzy models with one premise variable in fuzzy rules [8] .. As to the controller and observer design of T-S fuzzy system, much research works are based on the concept of parallel distributed compensation [16, 17] to derive the sufficient conditions of stability and observability and yield fruitful results. According to the approach, a common matrix P is required for each fuzzy local system, which is to guarantee the stabilization of the global fuzzy system (in the sense of Lyapunov). Generally, the exploration of the common matrix for synthesis of controller or observer can be recast as convex problem that can be solved efficiently by linear matrix inequality (LMI) optimization techniques. In spite of LMIÕs advantages, the existence of the solution, which satisfies the sufficient conditions, is still not guaranteed. Specifically as the number of fuzzy rule becomes large or too many constraints are imposed, the solution could be infeasible [10] .. As opposed to LMIÕs method, we explore an adaptive fuzzy observer scheme to deal with the synchronization problem. In each fuzzy subspace, the local dynamics is regarded as the nominal model and the effect of other rules stand for the uncertainty. Accordingly, the T-S fuzzy model can be treated as the linear uncertain system. Further, by appropriately selecting the observer gain and deriving an updating law for estimation of the uncertainty, the synchronization and stability of the global system can be guaranteed by the Lyapunov approach. Finally, the computer simulations demonstrate the effectiveness and robustness of proposed scheme. In this paper we presents the adaptive fuzzy observer for synchronization of chaotic systems. In contrast to the earlier studies [7, 9] , the chaotic system is specifically expressed in T-S fuzzy model. Based on the concept of system linearization, the adaptive algorithms are developed to the design of observer. It is demonstrated that if the chaotic system satisfies the hypotheses then synchronization can be guaranteed by the proposed approach. Computer simulations have verified that the adaptive fuzzy observer has well performance both in synchronization of chaotic system and control robustness.

O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O


Morphosys: an integrated reconfigurable system for data-parallel and computation-intensive applications Abstract: This paper introduces MorphoSys, a reconfigurable computing system developed to investigate the effectiveness of combining reconfigurable hardware with general-purpose processors for word-level, computation-intensive applications. MorphoSys is a coarse-grain, integrated reconfigurable system-on-chip targeted at high-throughput and data-parallel applications. It comprises of a reconfigurable array of processing cells, a modified RISC processor core and an efficient memory interface unit. This paper describes the MorphoSys architecture, including the reconfigurable processing array, the control processor and data and configuration memories. The suitability of MorphoSys for the target application domain is then illustrated with examples such as video compression, data encryption and target recognition. Performance evaluation of these applications indicates improvements of up to an order of magnitude on MorphoSys in comparison with other systems. Reconfigurable systems are computing systems that combine a reconfigurable hardware processing unit with a software-programmable processor. These systems allow customization of the reconfigurable processing unit, in order to meet the specific computational requirements of different applications. This paper has presented a new reconfigurable architecture, MorphoSys. Its performance has been evaluated for many of the target applications with impressive results that validate this architectural model.. Work on the physical implementation of MorphoSys on a custom-designed chip is in progress.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Approximation algorithms for minimizing the total weighted tardiness on a single machine Given a single machine and a set of jobs with due dates, the classical N P-hard problem of scheduling to minimize total tardiness is a well-understood one. Lawler gave an FPTAS for it some twenty years ago. If the jobs have positive weights the problem of minimizing total weighted tardiness seems to be considerably more intricate. In this paper, we give some of the first approximation algorithms for it. We examine first the weighted problem with a fixed number of due dates and we design a pseudopolynomial algorithm for it. We show how to transform the pseudopolynomial algorithm to an FPTAS for the case where the weights are polynomially bounded.For the case with an arbitrary number of due dates and polynomially bounded processing times, we provide a quasipolynomial algorithm which produces a schedule whose value has an additive error proportional to the weighted sum of the due dates. We also investigate the performance of algorithms for minimizing the related total weighted late work objective. We study the problem of scheduling jobs on a single machine to minimize the total weighted tardiness. We are given a set of n jobs. Job j, 1 ≤ j ≤ n, becomes available at time 0, has to be processed without interruption for an integer time p j , has a due date d j , and has a positive weight w j . For a given sequencing of the jobs the tardiness T j of job j is defined as max{0, C j − d j }, where C j is the completion time of the job. The objective is to find a processing order of the jobs which minimizes n j=1 w j T j . In the 3-field notation used in scheduling the problem is denoted 1| | j w j T j .. According to Congram et al., 1| | j w j T j is an "N P-hard archetypal machine scheduling problem" whose exact solution appears very difficult even on very small inputs [2] . We proceed to review what is known on the complexity of the problem. In the case of one machine it has long been known that an optimal preemptive schedule has the same total weighted tardiness as an optimal nonpreemptive schedule [13] . Early on the problem was shown to be N P-hard in the ordinary sense by Lenstra et al. [12] when the jobs have only two distinct due dates by a reduction from the knapsack problem. It was shown to be strongly N P-hard for an arbitrary number of due dates in [9] . Much later Yuan [15] showed that the problem remains N P-hard even for the case where all the jobs have a common due date. Lawler and Moore [11] have presented a pseudopolynomial solution for the case when all jobs have a single common due date. From the approximation point of view there is very little known. The only case that seems to be better understood is the usually easier case of agreeable weights: in that case p j < p i implies w j ≥ w i . Lawler gave a pseudopolynomial algorithm for the agreeable-weighted case [9] . In 1982 he showed how to modify that algorithm to obtain a fully polynomial-time approximation scheme for the case of unit weights [10] . A fully polynomial-time approximation scheme (FPTAS) for a minimization problem is an algorithm which for any ε > 0 runs in time polynomial in 1/ε and the size of the input and outputs a solution of cost at most (1 + ε) times the optimum. After the first publication of this work [7] , the paper by Cheng, Ng, Yuan and Liu appeared [1] . There it is shown that the schedule which minimizes max j w j T j yields an (n − 1)-approximation for 1| | j w j T j . Interestingly, the complexity of the unit weight case, 1| | j T j , was an open problem for many years until Du and Leung showed it is N P-hard [3] .. In this paper we make progress on the problem of minimizing the total weighted tardiness by examining first the case where the number of distinct due dates is fixed. Our main contribution is a pseudopolynomial algorithm whose complexity depends on the total processing time. This implies that the problem is in P when the processing times are polynomially bounded. We then show how to modify the pseudopolynomial algorithm in two steps: first so that its complexity depends on any upper bound on the maximum tardiness of an optimal schedule and second, so that it yields an FPTAS when the maximum job weight is bounded by a polynomial in n. Our main approach is based on viewing the problem as having to pack the jobs into a finite number of bins where the cost of each job depends on which bin it is assigned to and some jobs may be split between two bins. Hopefully some of the ideas we introduce could be of use for further study of approximating the long-open general case with an arbitrary number of due dates.. For the general case with an arbitrary number of distinct due dates we give a result that may be of interest when the due dates are concentrated around small values. Under the assumption that the maximum processing time is bounded by a polynomial in n, we provide a quasipolynomial algorithm which produces a schedule whose value has an additive error equal to δ j w j d j for any fixed δ > 0. 1| | j w j T j falls into the class of N P-hard problems where the optimum value can be zero. This renders the notion of a multiplicative approximation somewhat problematic. Additive guarantees may be of particular interest in this setting. We obtain the latter result by combining a partition of the time horizon into geometrically increasing intervals with a shift of the due dates to the interval endpoints.. The total weighted late work is an objective function which is conceptually related to the total weighted tardiness. The late work V j of job j in a given schedule is defined as min{T j , p j }. In other words, the late work is the amount of processing performed on job j after its due date d j . Hence V j ≤ T j , with equality in the case where the start time of job j is on or before d j . The problem of minimizing the total weighted late work, denoted by 1| |Σ j w j V j , is a well-studied problem which is N P-hard even with unit weights [14] . We formalize the relation between the two objective functions by showing that an optimal schedule for 1| | j w j V j achieves an O(P ) approximation with respect to the total weighted tardiness, where P . = n i=1 p i is the total processing time. We find it interesting that the guarantee is independent of the weights. We survey the known. Figure 2: Placement of straddlers algorithmic results on 1| | j w j V j in Section 4.2.. The outline of this paper is as follows. In Section 2 we present two pseudopolynomial algorithms. A first relatively simple one is presented in Section 2.2. The more involved one in Section 2.4 is shown to be amenable to a transformation to an FPTAS in Section 3. The quasipolynomial algorithm is presented in Section 4.1. In Section 4.2 we examine schedules that minimize the total late work from the point of view of total weighted tardiness. We conclude with open questions in Section 5. This paper is an extensively revised and augmented version of [7] .. 2 Pseudopolynomial algorithms

B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O


Parallel watershed transformation algorithms for image segmentation The watershed transformation is a mid-level operation used in morphological image segmentation. Techniques applied on large images, which must often complete fast, are usually computationally expensive and complex entailing ecient parallel algorithms. Two distributed approaches of the watershed transformation are introduced in this paper. The algorithms survey in a Single Program Multiple Data (SPMD) model both local and global connectivity properties of the morphological gradient of a gray-scale image to label connected components. The sequentiality of the serial algorithm is broken in the parallel versions by exploiting the ordering relation between two neighboring pixels successively incorporated in the same region. Thus, a path is traced, for every unlabeled pixel, down to its region of inclusion (whose label is then propagated backwards); in the second algorithm, regions grow independently around their seeds. In both cases only pixels which satisfy the ordering relation are incorporated in any region. This way, not only dierent regions are explored in a parallel fashion, but also dierent parts of the same region, when the latter extends to neighboring subdomains, are treated likewise. Running time and relative speedup evaluated on a Cray T3D parallel computer are used to appreciate the performance of both algorithms. Ó The watershed transformation applied to a gray-scale image detects and labels objects, which are connected components of similar gray-level. The classical fashion to describe the construction of watersheds identi®es an image with a topographical surface. Starting by piercing holes in the regional minima (connected plateaus of constant altitude from which it is impossible to reach a location of lower altitude without having to climb) of the surface, and then slowly sinking it into a lake, water will progressively engulf the basins enclosing various minima. To prevent water from intermingling at the border between dierent basins, a hinder is set up. Once the surface is completely covered by water, the set of obstacles depicts the watershed image. Various de®nitions of watersheds have been proposed in the literature (see Refs. [2, 8, 9, 21, 23] ) for both digital and continuous spaces.. The watershed transformation is prevalently used in industrial, biomedical, and computer vision applications. As concrete examples, we mention here the contribution of watersheds to an automatic system of analysis of images acquired during oil exploration in Ref. [22] , road trac analysis [4] , and also to segmentation of electrophoresis gels [9] , a moving heart (in nuclear medicine) [6] , and 3D holographic images [3] .. Initial eorts in the area of parallel implementation of watersheds came from the work in Refs. [2, 5, 8, 9] since for large images the complexity of the analysis entails fast parallel algorithms. Previous experiments of parallelization of watersheds [11, 16] using the classical sequential algorithm based on an ordered queue [2, 9] resulted in not too ecient implementations. The reason is the highly sequential nature of the approach itself. By parallelizing it, the global ordered queue is divided into several local ordered queues. Therefore, if pixels in one queue in the global ordered queue are disseminated in dierent local ordered queues, a strong synchronization between the processors maintaining those particular local queues is required. In addition, due to the domain decomposition, repeated relabelings of the same pixels are performed to appropriately label parts of catchment basins which do not lie on the same subdomain as the regional minima from which the basins were generated.. Parallel realizations of watersheds based on image integration and sequential raster and antiraster scannings proved to be scalable, but still computationally expensive because of the repeated scannings [12, 16, 17] .. Other parallel implementation of watersheds can be found in Refs. [1, 10, 18] . Due to the recursive nature of the watershed transformation, its parallelization is not a trivial task. In this paper, we aim to exploit a local property of the watershed ooding in order to break the sequentiality of the classical immersion simulation and increase the data locality. As in Ref. [8] , a predecessor±successor¯ooding relation between two consecutive neighboring pixels incorporated into the same catchment basin is established. Further on, following these¯ooding connections between pixels, shortest paths between non-minima and minima, to which region the non-minima belong, are tracked and labeled. For this purpose, two SPMD algorithms are presented in this paper. Although the approaches are dierent, the goal remains the same, namely, to split the morphological gradient of a gray-scale image, regarded as a topographic surface, into geodesic in¯uence zones (see Ref. [2] ).. Both algorithms set labels in separate stages for minima and non-minima, and dierences occur in the latter stage. In the ®rst algorithm, based on rainfalling simulation, every non-minimum pixel on a path a drop of water would slide toward a minimum is labeled, when the drop starts falling from that particular non-minimum point in the topographical relief. Equivalently, one can start and``walk'' downward following a steepest slope line (a path fp 0 Y F F F Y p s g, where for any pair p i Y p i1 , p i1 is the lowest altitude neighboring pixel of p i , or the next closer pixel to a lower gray-level pixel, in the case of a plateau) toward a minimum pixel, and propagate the label of the latter backward along the whole path (see Fig. 1) ; hence, the namè`r ainfalling simulation''. Alternatively, in the second algorithm, water springs from regional minima and immerses higher neighborhoods which have not been already¯ooded (see Fig. 2 ). This bottom±up method is called``hillclimbing simulation''. However, neither of the methods constructs dams between neighboring areas of labels (0-width watersheds).. In every stage, local connectivity is ®rst exploited to compute labels within each subimage, and then the results are merged to set labels for the whole image. Both methods are parallel by nature, solving a multiple origins and multiple destinations (non-minima/minima) connectivity problem [7] . Preliminary results of this work have been published in [13±15] .. The paper is organized as follows. In Section 2, common de®nitions used in the paper are introduced. Section 3 presents two parallel watershed algorithms; the ®rst is based on rainfalling simulation, and the second on hillclimbing simulation. Section 4 summarizes some key insights into both implementations and their complexity along with experimental results. Finally, the paper is concluded in Section 5. Two parallel watershed algorithms were presented here to label connected components for image segmentation. Both algorithms start by detecting ®rst the seeds of the regions, the regional minima of the gradient image. The image is then lowercomplete transformed and represented as an acyclic directed graph or forest. Pixels are vertices and there is an arc between any two nodes if their lower-complete values satisfy the given ordering relation. Two methods of propagating the labels of minima, which are leaves, are proposed and compared. One is rainfalling simulation, in which every non-minimum pixel parses arcs downward to a leaf to get labeled; and the other is hillclimbing simulation, in which labels are propagated bottom±up along steepest slope lines.. The aim was to compare the two approaches from the eciency viewpoint and results. It can be observed that both algorithms are scalable, that is, the total execution time drops down, but not linear, with an increasing number of processors. However, increasing relative speedup is achieved until 64 and 128 processors. As expected, the hillclimbing based method is more time consuming than the rainfalling approach, but it scales better than the latter. Finally, the results of the segmentation are similar to those of the serial watershed algorithms which have been shown to produce good results. The parallel algorithms introduced in this paper do however suer from the major shortcoming of traditional serial watershed algorithms, that is oversegmentation. A parallel design solution to this problem can be found in Ref. [19] .

O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Parsing with probabilistic strictly locally testable tree languages Abstract-Probabilistic k-testable models (usually known as k-gram models in the case of strings) can be easily identified from samples and allow for smoothing techniques to deal with unseen events during pattern classification. In this paper, we introduce the family of stochastic k-testable tree languages and describe how these models can approximate any stochastic rational tree language. The model is applied to the task of learning a probabilistic k-testable model from a sample of parsed sentences. In particular, a parser for a natural language grammar that incorporates smoothing is shown. S TOCHASTIC models based on k-grams predict the probability of the next symbol in a sequence as a function of the k À 1 previous symbols and have been widely used in natural language processing [1] , [2] , [3] . From a theoretical (albeit not historical) point of view, k-gram models can be regarded as a probabilistic extension of strictly locally testable string languages [4] . Strictly locally testable models or k-testable models, 1 a kind of model that classifies strings by looking at substrings of length at most k, are easy to learn from samples and can be dealt with using efficient and simple algorithms [5] , [6] , [7] . It is also straightforward to incorporate probabilities into the model, although data sparseness often leads to the assignment of null probabilities to many strings. To alleviate this, there is a number of well-known techniques [8] , [9] to smooth the distribution.. When hierarchical relations are established among the data components, trees become a more adequate representation than strings of symbols. This is the case, for example, in natural language parsing [10] , [11] , [12] , [13] or structured text processing [14] , [15] . Context-free grammars [16] provide a traditional formalism that handles structural information. This kind of grammars can be easily written and updated by humans, although it is difficult to learn them automatically. For instance, it is hard to find the appropriate degree of generalization unless some information about the size of the target grammar is available [17] .. The class of parse trees generated by a context-free grammar can be characterized as a rational tree language [18] , [19] and any rational tree language can be recognized by a bottom-up (also called frontier-to-root or ascending) finite-state tree automaton. 2 The class of k-testable tree languages [21] is a proper subclass of the class of rational tree languages, where the effect of events that have occurred beyond a certain depth window are ignored when processing a tree. Then, k-testable tree models can be regarded as a special case of automata whose states are simply subtrees truncated to a certain depth. If valence (that is, number of children) remains bounded, the number of states in the automaton remains finite (and observable).. Stochastic models that assign a probability to a tree can be useful to select the best parse tree for a sentence and resolve structural ambiguity. For instance, in many natural-language processing applications, obtaining the correct syntactical structure for a sentence is an important intermediate step before assigning an interpretation to it. But, ambiguous parses are very common in real natural-language sentences (e.g., those longer than 15 words; the fact that many ambiguous parse examples in books sound a bit awkward is due to the fact that they involve short sentences [22] , [23, p. 411] ). Choosing the correct parse for a given sentence is a crucial task if one wants to interpret the meaning of the sentence, due to the principle of compositionality [24, p. 358] , which states, informally, that the interpretation of a sentence is obtained by composing the meanings of its constituents according to the groupings defined by the parse tree. 3 As will be introduced in this paper, a class of probabilistic models analogous to the k-gram models can be defined for tree languages. In this probabilistic extension, the automaton behaves as a top-down generating device, but the computation is still performed bottom-up: The state that generates a subtree can be recognized by a finite-state computation over the generated subtree or, more precisely, over those nodes lying within a depth k in the subtree.. Our notation is introduced in Section 2 and some known properties of strictly locally testable tree languages are presented in Section 3. In Section 4, we show how to obtain from a given, general, tree automaton the k-testable automaton that minimizes the relative entropy or KullbackLeibler divergence [27] to the original one. In Section 5, we justify how the best model from a stochastic sample is obtained. The application of these models to describe the syntactical structure of sentences is presented in Section 6, together with some techniques to smooth the distributions. Experiments using one of these models, called child-annotated model for short, for structural disambiguation are presented in Section 7 and compared with other family-annotated models. Finally, conclusions are given in Section 8. We have introduced a class of probabilistic models (the strictly k-testable class) that describes tree languages and can be easily estimated from treebank samples. We have found an efficient procedure to compute the probabilities of the k-testable model that approximates any given stochastic rational tree language. The relative entropy between the k-testable model and the original one can also be efficiently computed and, as expected, the divergence becomes smaller as k increases. These strictly locally testable models can be updated incrementally and are adequate to apply backing-off techniques [9] or to be used as starting point by more complex learning algorithms [34] , [46] .. Fixed-depth probabilistic models may be seen as a special PCFG in which nonterminals are specialized for each possible offspring (up to a certain depth) and, so, are the expansion probabilities of rules. This may help model linguistic phenomena such as the differential distribution of determinants in noun phrases acting like a subject or like an object. We have introduced a procedure to build k-testable grammars smoothed by using a rule-based backing-off approach. In particular, we have compared the parsing performance of these models with that of unannotated PCFG and of parentannotated PCFG [38] . As future work, we plan to study the use of statistical confidence criteria to eliminate unnecessary annotations by merging states, therefore, reducing the number of parameters to be estimated. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O