Chemical-Reaction-Inspired Metaheuristic for Optimization Abstract-We encounter optimization problems in our daily lives and in various research domains. Some of them are so hard that we can, at best, approximate the best solutions with (meta-) heuristic methods. However, the huge number of optimization problems and the small number of generally acknowledged methods mean that more metaheuristics are needed to fill the gap. We propose a new metaheuristic, called chemical reaction optimization (CRO), to solve optimization problems. It mimics the interactions of molecules in a chemical reaction to reach a low energy stable state. We tested the performance of CRO with three nondeterministic polynomial-time hard combinatorial optimization problems. Two of them were traditional benchmark problems and the other was a real-world problem. Simulation results showed that CRO is very competitive with the few existing successful metaheuristics, having outperformed them in some cases, and CRO achieved the best performance in the realworld problem. Moreover, with the No-Free-Lunch theorem, CRO must have equal performance as the others on average, but it can outperform all other metaheuristics when matched to the right problem type. Therefore, it provides a new approach for solving optimization problems. CRO may potentially solve those problems which may not be solvable with the few generally acknowledged approaches. O PTIMIZATION is prevalent in almost every field of science and engineering, ranging from profit maximization in economics to signal interference minimization in electrical engineering. In our daily lives, we also encounter various optimization problems, such as finding the quickest route from one place to another, at minimum cost, and minimizing the construction costs of building facilities in a city, while, at the same time, avoiding congestion of human flow among such facilities. Optimization refers to the study of problems in which one seeks to optimize (either minimize or maximize) the result by systematically choosing the values of the variables in feasible regions. We normally define an optimization problem with several components: an objective function f, a vector of variables X = {x 1 , x 2 , . . . , x n }, and a vector of constraints C = {c 1 , c 1 , . . . , c m } which limit the values assigned to X, where n and m correspond to the problem dimensions and the total number of constraints, respectively. We define a solution s as the set of values assigned to X confined by C, and the solution space S as the set of all possible solutions. For minimization problems, our goal is to find the minimum solution s * ∈ S where f (s * ) ≤ f (s) for all s. We can write . where R, E, and I represent the real number set, the index set for equalities, and the index set for inequalities, respectively. Equation (1) represents the generic form for every type of optimization. Without loss of generality, we consider minimization problems throughout this paper. 1 What we need to do is to search the solution space and pick out solution points sequentially. Then one evaluates the objective function value of each solution point. An optimization method (i.e., algorithm) tells us which point should be picked from the current solution. We can get one, or multiple, points in an instance, depending on how the algorithm operates.. We can formulate many problems into this generic form, i.e., (1) , and then apply the existing methods to obtain the optimal solutions, with the help of the computer. However, in computation complexity theory [1] , there is a class of problems, namely, nondeterministic polynomial-time hard (NP-hard) problems, with no known algorithms in finding the optimal solutions in polynomial time, unless P = NP. In other words, for such problems, the computational effort required to obtain the best solutions grows exponentially with the problem size. They are normally not solvable by any optimization algorithms in a reasonable amount of time or we cannot guarantee that the computed results are of high quality. Most of the time, the formulated problems are of huge dimensions and examining every possible solution (i.e., the brute-force method) becomes impossible. It may take several years of CPU time to obtain the solutions, in spite of using the most powerful supercomputer. We often cannot tolerate such long computational time and sacrifice optimality for near-optimal solutions if the processing time is limited. Thus, we always adopt approximate algorithms, which can compute "good" solutions efficiently, to tackle the NP-hard problems. In quantum mechanics and statistical mechanics, we can model chemical reactions and molecular interactions with potential energy surface (PES) (Fig. 1) , which is subject to the Born-Oppenheimer separation of nuclear and electronic motion [2] . Fig. 1 depicts the potential energy (PE) changes of the atom arrangements in a chemical system. The z-axis represents the PE while the x and y-axes capture the molecular structures of the chemical substances, which correspond to the atomic positions and every possible orientation of all the involved atomic nuclei. PES can be a two, three, or multi-dimensional (hyper)surface, depending on how complicated the chemical system is. In any chemical reaction, the initial species (i.e., reactants) change to the products by the formation and destruction of chemical bonds. Before the formation of products, the reactants normally change to a series of intermediate species. These small chemical changes are called elementary steps. During each step, the chemicals are in the transition states. Fig. 1 shows a simple example of a chemical reaction involving three elementary steps. The solid line gives the reaction pathway from the reactants to products, via several transition states and intermediate species.. There is a rule of thumb for this natural tendency-"Every reacting system seeks to achieve a minimum of free energy" 2 [3] . That means chemical reactions tend to release energy, and thus, products generally have less energy than reactants. In terms of stability, the lower the energy of the substance, the more stable it is. Therefore, products are always more stable than reactants.. It is not difficult to discover the correspondence between optimization and chemical reaction. Both of them aim to seek the global minimum (but with respect to different objectives) and the process evolves in a stepwise fashion. With this discovery, we develop a chemical-reaction-inspired metaheuristic for solving optimization problems by mimicking what happens to molecules in chemical reactions. We name it chemical reaction 2 Free energy is also known as Gibbs free energy, which indicates the amount of energy needed for a system to do useful work at constant temperature and pressure. In a chemical reaction, the reactants have higher free energy, and thus, they can do useful work (i.e., react with the others). At equilibrium (i.e., final stage of the reaction), the products have a minimum of free energy so they can no longer react. optimization (CRO). It is a multidisciplinary design which loosely couples computation with chemistry.. The rest of this paper is organized as follows. Section II briefly describes some basic concepts of optimization and related work. In Section III, we give the design framework of CRO and show how the concept of chemical reaction is implemented in our algorithm. We show the workability of CRO with evaluations using computer simulations in Section IV. We conclude this paper and give some potential future work in Section V. CRO is a chemical-reaction-inspired metaheuristic. It mimics the interactions of molecules in a chemical reaction, and guides the transformation of molecules along the PES toward the more stable state by redistributing the energies among molecules and by interchanging the energies from one form to another. It tries to locate the global minima of the objective function. It is an inter-disciplinary research problem, in which we exploit the characteristics of chemical reactions in solving optimization problems. As shown in the simulation section, we can apply it to solve well-known NP-hard problems, QAP and RCPSP, and a practical NP-hard problem CAP. We show that CRO is effective in solving optimization problems, and among the few successful metaheuristics. With the NFL theorem, CRO must have equal performance as the others on the average but it can outperform all other metaheuristics when matched to the right problem type. As the "spectrum" of optimization problems is huge, we will never be satisfied with the small number of generally acknowledged methods, such as SA, GA and ACO. As no single metaheuristic can work equally well over a wide range of problems, the large number of optimization problems and the small number of satisfactory metaheuristics mean that we need more metaheuristics to fill the gap. CRO provides a new approach for solving optimization problems and belongs to the few acknowledged successful metaheuristics.. We decide to demonstrate the fundamental version of CRO in this paper. In the future, we can develop even better versions of CRO through hybridation with the others (e.g., CRO + SA, CRO + ACO, etc.) or through the incorporation of greedy approaches. It is possible to give good results to a broader range of problems sets and find the global optima of some "unsolved" problems, based on this fundamental version of CRO. Metaheuristics are collections of ideas aiming to solve general computational problems. A metaheuristic is usually in the form of a procedure framework which instructs computers how to search for solutions in the solution space. Each metaheuristic consists of several building blocks and control parameters for fine tuning. We can replace these components and/or change the parameter values in order to suit our purposes. From this, we can see metaheuristics contain a high degree of flexibility. Most of the metaheuristics involve randomization in the calculation, and thus, the outputs may vary in different runs of the computation. Since exact optimal solutions are not guaranteed, they belong to the group of approximate algorithms. We adopt them to solve NP-hard optimization problems because they can locate good solutions efficiently most of the time.. This paper is motivated by other efforts to apply natural phenomena to metaheuristics. Among the most famous ones are simulated annealing (SA) [4] , genetic algorithm (GA) [5] - [7] , and ant colony optimization (ACO) [8] , [9] . Other proposed metaheuristics include particle swarm optimization (inspired by the social behavior of bird flocking) [10] , bees algorithm (inspired by the behavior of honey bees in collecting nectar) [11] , harmony algorithm (inspired by the improvisation process of musicians) [12] , etc. There are non-nature-inspired metaheuristics also, like tabu search (TS) [13] . We will briefly introduce SA, GA, ACO, and TS in the following sections.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O


Complete subdivision algorithms, II: isotopic meshing of singular algebraic curves Given a real function f (X, Y ), a box region B and ε > 0, we want to compute an ε-isotopic polygonal approximation to the curve C : f (X, Y ) = 0 within B. We focus on subdivision algorithms because of their adaptive complexity. Plantinga & Vegter (2004) gave a numerical subdivision algorithm that is exact when the curve C is non-singular. They used a computational model that relies only on function evaluation and interval arithmetic. We generalize their algorithm to any (possibly non-simply connected) region B that does not contain singularities of C. With this generalization as subroutine, we provide a method to detect isolated algebraic singularities and their branching degree. This appears to be the first complete numerical method to treat implicit algebraic curves with isolated singularities. Given ε > 0, a box region B and a real function f : R 2 → R, we want to compute a polygonal approximation P to the implicit curve C : f = 0 restricted to B. The approximation P must be (1) "topologically correct" and (2) "ε-close" to C ∩ B. We use the standard interpretation of requirement (2) , that d(P, C ∩ B) ≤ ε where d(·, ·) is the Hausdorff distance on compact sets. In recent years, it has become accepted [2] to interpret requirement (1) to mean P is isotopic to C ∩ B, which we denote by P ≈ C ∩ B. This means we not only require that P and C ∩ B to be homeomorphic, but they must be embedded in R 2 "in the same way". E.g., if C ∩ B consists of two ovals, these can be embedded in R 2 as two ovals exterior to each other, or as two nested ovals. Isotopy, but not homeomorphism, requires P to respect this distinction. In this paper, we mainly focus on topological correctness, since achieving ε-closeness is not an issue for our subdivision approach (but cf. [2, pp. 213-4 This paper presents the first complete numerical subdivision algorithm for meshing an implicit algebraic curve that has only isolated singularities. This solves an open problem in the exact numerical approaches to meshing in 2-D [2, p. 187]. We pose three challenges: (a) An worst case complexity bound for our procedure is possible. But this may not the best way to measure adaptive algorithms. We would like to provide are adaptive bounds, similar to the integral analysis in [5] . (b) In 3-D, a square-free integer polynomial f (X, Y, Z) could have a 1-dimensional singularities. We pose the problem of designing a purely numerical subdivision algorithm to handle 1-dimensional singularities. (c) The practical implementation of an adaptive algorithm handling singularities, even based on our outline, must handle many important details. Computational experience is invaluable for future research into singularity computation.

O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O B I I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O


Fast, flexible, and highly resilient genuine fifo and causal multicast algorithms We study the fifo and causal multicast problem, two groupcommunication abstractions that deliver messages in an order consistent with their context. With fifo multicast, the context of a message m at a process p is all messages that were previously multicast by m's sender and addressed to p. Causal multicast extends the notion of context to all messages that are causally linked to m by a chain of multicast and delivery events.We propose multicast algorithms for systems composed of a set of disjoint groups of processes: server racks or data centers. These algorithms offer several desirable properties: (i) the protocols are latency-optimal, (ii) to deliver a message m only m's sender and addressees communicate, (iii) messages can be addressed to any subset of groups, and (iv) these algorithms are highly resilient: an arbitrary number of process failures is tolerated and we only require the network to be quasi-reliable, i.e., a message m is guaranteed to be received only if the sender and receiver of m are always up. To the best of our knowledge, these are the first multicast protocols to offer all of these properties at the same time. Developing dependable distributed applications is not easy. The complexity stems from the asynchrony and unreliability of typical distributed systems: processes execute at different speeds and may abruptly stop executing their code (i.e., crash). Moreover, messages may be arbitrarily delayed, received out-of-order, and even lost, if the sender or receiver is faulty. To ease the development of distributed systems, several group-communication abstractions have been proposed [4, 8] . Two common abstractions are atomic broadcast and atomic multicast. While in the former messages are addressed to all system members, in the latter messages are addressed to subsets of the system members (i.e, groups).. Broadcast and multicast abstractions ensure similar reliability guarantees-agreement on the set of messages delivered-but offer various message ordering properties. Two of these properties, fifo and causal order, are of special interest: they ensure that a message m is not delivered at a process p that does not know m's context, where the notion of context is defined differently for each order property. With fifo order, the context of m at p is the messages that were previously broadcast (or multicast) by m's sender and addressed to p. Causal order extends the notion of context to all messages that causally precede m, i.e., messages that are causally linked to m through a chain of broadcast (or multicast) and delivery events. Fifo and causal order help the programming of distributed applications in various domains such as global snapshot construction [2] and fair resource allocation [10] . Causal multicast may also serve as a building block to implement atomic multicast [16] .. Fifo and causal broadcast protocols have been largely studied in the literature. In this paper, we propose fifo and causal multicast protocols for systems composed of a set of disjoint groups (e.g., server racks or data centers), each containing several processes. In particular, we show that mechanisms devised for fifo and causal broadcast protocols are not applicable to multicast protocols. As our main contribution, we propose fifo and causal multicast algorithms that offer several desirable properties. To the best of our knowledge, these algorithms are the first to be simultaneously fast, genuine, flexible, and highly resilient, in a precise sense, as we now explain.. First, they are fast: messages can be delivered in two communication steps; and we further show that this is optimal. Second, these protocols are genuine [7] : (i) to deliver a message m only the sender and the addressees of m participate in the protocol. Third, the algorithms are flexible in the sense that a process p may multicast messages to groups p does not belong to, that is, groups are "open". Finally, our algorithms are highly resilient: they tolerate an arbitrary number of process failures, and can cope with quasi-reliable links which guarantee that if both the sender and receiver of a message m are correct, i.e., they do not crash, then m is eventually received. This is in contrast to several multicast protocols [12, 13, 14, 15, 9] , which rely on reliable links-message delivery is guaranteed as long as the receiver is correct, regardless of the correctness of the sender. Reliable links are not a realistic assumption: to send a message m, the machine Mp hosting process p typically inserts m into one (or more) local buffer before m is sent over the wire. Hence, even though p thinks that m was successfully sent, m may still be lost in case Mp crashes before m hits the wire.. As discussed in [3] , devising open group multicast protocols that tolerate quasi-reliable links introduce difficulties as we explain next. Consider some process p that multicasts a message m1 to some group g2. Later, p multicasts a message m2 to groups g1 and g2 and crashes. Message m2 is received by processes in g1, and since m2 is the first message multicast from p to g1, m2 is delivered by processes in g1. On the contrary, all messages sent from p to members of g2 are lost. Note that this can happen because p crashes and links are quasi-reliable.. From the reliability guarantees of multicast, correct processes in g2 must eventually deliver m2. However, if they do so, the ordering guarantees of fifo and causal multicast will be violated: members of g2 cannot deliver m1 before m2 since m1 was lost. If messages were broadcast, then m1 would also be addressed to g1, and thus, g1 could help g2 by forwarding m1 to g2. With multicast however, g1 does not even know about the existence of m1, since m1 was not addressed to g1. In this paper, we propose a mechanism to cope with this problem despite an arbitrary number of process failures and, in contrast to [3] , the resulting fifo and causal multicast algorithms are latency-optimal and as latency-efficient as their broadcast counterparts.. The rest of the paper is structured as follows. In Section 2 we discuss the related work. Section 3 presents the system model and some definitions. Sections 4 and 5 respectively provide fifo and causal multicast algorithms; Section 6 shows their latency-optimality. We conclude the paper in Section 7. The correctness proofs of the algorithms can be found in [17] . In this paper, we proposed fifo and causal multicast algorithms that are open-group. These protocols tolerate an arbitrary number of process failures, tolerate quasi-reliable networks, and we showed that they are latency-optimal.. As future work, we intend to study the minimum message complexity of these two problems and characterize how the amount of information about process failures affects this complexity. Fifo and causal broadcast were originally specified as part of the Isis system [4] . In [8] , fifo broadcast is implemented by reliably broadcasting messages along with a sequence number and by delivering messages in the sequence number order.. The first implementation of causal broadcast uses a simple strategy [4] : the causal history of delivered messages is piggybacked on each message to be broadcast. The amount of information contained in messages is thus unbounded. In [14] , causal order is ensured differently: messages carry control information in the form of a matrix of counters, where each entry (p, q) denotes the number of messages that were multicast from process p to q in the causal history. This control information is used to know when messages can be safely delivered. This algorithm does not tolerate process failures. A fault-tolerant algorithm that ensures causal order using a similar technique appears in [8] . Although [8] specifies both causal broadcast and multicast, the algorithm given considers the broadcast case only.. In [5] , processes may belong to several groups at the same time but messages sent from a process p cannot be multicast to groups p is not a member of. Using the terminology of [6] , the protocol in [5] is closed-group. In this algorithm, each message carries a vector of counters, and this for every group in the system. Messages may be large if the number of groups is high. In contrast, [13] only requires processes to append a vector of counters to messages, where the size of the vector is equal to the number of groups. However, this protocol is not fault-tolerant. In [15] , causal separators are used to reduce the amount of control information needed in systems that span several network domains. In [3] , the authors propose a more general approach that is topologyoblivious.. The necessary conditions on how much information should be stored at processes and appended to messages to ensure causal order are presented in [9] . This paper also provides an information-optimal algorithm that does not need any a priori knowledge of the communication network. The algorithm in [12] does not append any information on messages but only considers the unicast case and postpones the sending of messages until after all the previous messages sent were acknowledged.. In this paper, we present fault-tolerant and latency-optimal fifo and causal multicast protocols, respectively denoted as A fifo and A causal . To the best of our knowledge, these are the first algorithms that are at the same time open group, latency-optimal, and tolerate quasi-reliable networks. Table 1 provides a comparison of the algorithms. The last four columns respectively indicate: the best-case latency of the algorithms, measured in the number of communication delays; whether an algorithm A allows messages to be multicast from a process p to groups p does not belong to, in which case we say that A is an open group algorithm, or not, in which case we say that A is a closed-group algorithm; and the process as well as network failure resilience.

O O O O O O B I I I O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Quasi-hamiltonian paths in semicomplete multipartite digraphs A quasi-hamiltonian path in a semicomplete multipartite digraph D is a path which visits each maximal independent set (also called a partite set) of D at least once. This is a generalization of a hamiltonian path in a tournament.In this paper we investigate the complexity of finding a quasi-hamiltonian path, in a given semicomplete multipartite digraph, from a prescribed vertex x to a prescribed vertex y as well as the complexity of finding a quasi-hamiltonian path whose end vertices belong to a given set of two vertices {x, y}. While both of these problems are polynomially solvable for semicomplete digraphs (here all maximal independent sets have size one), we prove that the first problem is NP-complete and describe a polynomial algorithm for the latter problem. Notation not introduced below or in the next section follows [? ] .. We always denote by V (A) the set of vertices (arcs) of a digraph and by n (m) its cardinality.. A digraph D = (V, A) is semicomplete multipartite if whenever there is an arc between u and v but no arc between u and w, there is an arc between v and w. Equivalently, there is a partition V 1 , V 2 , . . . , V c of V into independent sets so that every vertex in V i shares an arc with every vertex in V j for 1 ≤ i < j ≤ c. We call V 1 , V 2 , . . . , V c the partite sets of D. A multipartite tournament, or MT in short, is a semicomplete multipartite digraph, or SMD in short, without cycles of length two. An (x, y)-path ([x, y]-path) in a digraph D is a path which starts in x and ends in y (has end vertices in {x, y}).. A digraph D is k-strong if it has at least k + 1 vertices and D − X is strongly connected for every X ⊂ V with |X| ≤ k − 1. A digraph is hamiltonian-connected (weakly hamiltonian-connected) if it contains an (x, y)-hamiltonian ([x, y]-hamiltonian) path for every choice of distinct vertices. Thomassen [? ] proved that every 4-strong semicomplete digraph D is hamiltonian connected and gave an infinite class of 3-strong tournaments which are not hamiltonian-connected.. In [? ] a polynomial algorithm for testing whether a given semicomplete digraph contains an (x, y)-hamiltonian path was given. The algorithm relies on the results in [? ] as well as additional technical results. Interestingly, the algorithm cannot be modified to solve the problem of finding the longest (x, y)-path in a semicomplete digraph and the complexity of this problem is still open. Conjecture 1.1.. [? ] There exists a polynomial algorithm for finding the longest (x, y)-path in a semicomplete digraph.. In [? ] Thomassen also gave a complete characterization of weakly hamiltonian connected tournaments. That characterization implies a polynomial algorithm for deciding whether a given tournament contains an [x, y]-hamiltonian path. From this we can also derive a polynomial algorithm for finding the longest [x, y]-path in a tournament (and in fact, in a semicomplete digraph).. For multipartite tournaments it is an open problem whether the existence of an [x, y]-hamiltonian path can be decided in polynomial time and the problem is most likely quite difficult 2 (see [? ] for an algorithm for this problem in the case of extended tournaments). This makes it interesting to study problems for multipartite tournaments which contain the above mentioned hamiltonian path problems for tournaments as special cases.. Let D = (V, A) be a semicomplete multipartite digraph with partite sets V 1 , V 2 , . . . , V c . A quasihamiltonian path in D is a path P such that V (P ) ∩ V i = ∅ for 1 ≤ i ≤ c (so for semicomplete digraphs a quasi-hamiltonian path is a hamiltonian path). A digraph is quasi-hamiltonian-connected (weakly quasi-hamiltonian-connected) if it contains an (x, y)-quasi-hamiltonian ([x, y]-quasihamiltonian) path for every choice of distinct vertices. The following result generalizes Thomassen's well known result [? ] that every 4-strong tournament is hamiltonian-connected Theorem 1.2.. [? ] Every 4-strong multipartite tournament D is quasi-hamiltonian-connected, that is, it contains a quasi-hamiltonian path starting in x and ending in y for every choice of distinct vertices of D.. In [? ] Guo et al. introduced the following notion, which generalizes weak hamiltonian-connectivity in semicomplete digraphs. A semicomplete multipartite digraph D with partite sets V 1 , V 2 , . . . , V c is weakly quasi-hamiltonian set-connected if every pair of distinct partite sets V i , V j , 1 ≤ i < j ≤ c contains vertices x ∈ V i , y ∈ V j such that D has an [x, y]-quasi-hamiltonian path. The authors of [? ] gave a complete characterization of weakly quasi-hamiltonian set-connected multipartite tournaments which generalizes Thomassen's work on weakly hamiltonian connected tournaments in [? ] . An easy corollary of their work is the following.. There is a polynomial algorithm for deciding whether a multipartite tournament D with given distinct partite sets V i , V j contains an [x, y]-quasi-hamiltonian path for some x ∈ V i , y ∈ V j .. It is mentioned in [? ] that they chose that way of generalizing weak hamiltonian-connectedness of tournaments to multipartite tournaments, because the straightforward generalization to weak quasihamiltonian paths would not lead to a nice generalization of Thomassen's characterization for the existence [x, y]-hamiltonian paths in tournaments.. In this paper we consider the complexity of finding quasi-hamiltonian (x, y)-and [x, y]-paths in semicomplete multipartite digraphs. We prove that, somewhat surprisingly, it is NP-complete to decide whether a semicomplete multipartite digraph contains an (x, y)-quasi-hamiltonian path. In doing so, we also identify a problem which is NP-complete for tournaments and thus add to the short list 3 of (natural) problems which are NP-complete already for tournaments. We prove that for the subclass of extended semicomplete digraphs a polynomial algorithm does exist by reducing the problem to the (x, y)-hamiltonian path problem for semicomplete digraphs.. Our main contribution is a polynomial algorithm for weak quasi-hamiltonicity, namely we prove that there exists a polynomial algorithm for deciding whether there exists an [x, y]-quasi-hamiltonian path in a given semicomplete multipartite digraph. Since our main objective was to prove that this problem is indeed polynomial and in order to keep the presentation reasonably simple, we have chosen to present an algorithm whose correctness is fairly easy to verify rather than going for an algorithm with a much better complexity.. We conclude the paper with a number of open problems and some remarks.

O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An HMM-based approach for off-line unconstrained handwritten word modeling and recognition AbstractÐThis paper describes a hidden Markov model-based approach designed to recognize off-line unconstrained handwritten words for large vocabularies. After preprocessing, a word image is segmented into letters or pseudoletters and represented by two feature sequences of equal length, each consisting of an alternating sequence of shape-symbols and segmentationsymbols, which are both explicitly modeled. The word model is made up of the concatenation of appropriate letter models consisting of elementary HMMs and an HMM-based interpolation technique is used to optimally combine the two feature sets. Two rejection mechanisms are considered depending on whether or not the word image is guaranteed to belong to the lexicon. Experiments carried out on real-life data show that the proposed approach can be successfully used for handwritten word recognition. HANDWRITING is one of the easiest and most natural ways of communication between humans and computers. However, early investigations in automatic handwriting recognition were limited by the memory and power of the computers available at that time which did not permit the design of real-time systems. Thanks to the recent progress in electronics and to the latest generation of computers, these problems have been overcome; therefore, since the beginning of the 1980s, there has been a dramatic increase of research in this field. According to the way handwriting data are generated, two classes are distinguished: If the data provided to the system correspond to the pixels of a static image obtained with a scanner or a CCD camera after the writing is completed, then we are in the off-line recognition case. If the data correspond to the sequence of pixels (defined by their coordinates) drawn by the user on a digitized tablet and transmitted to the system during the writing, then we are in the on-line recognition case. Off-line and on-line systems are also distinguished by the applications they are devoted to. The former are dedicated to bank check processing, mail sorting, commercial forms-reading, etc., while the latter are mainly dedicated to pen computing industry and security domains such as signature verification and author authentication. Off-line handwriting recognition is a more difficult task, because the temporal information, such as the number and the order of the strokes and the pressure, is not available as in the on-line case. On the other hand, off-line systems can achieve huge economic benefits even with low recognition rates, while on-line systems must achieve high recognition rates to be used in a commercial system. In the remainder of this paper, we shall talk about off-line handwriting recognition.. Despite the impressive progress achieved in handwriting recognition, the results are still far from human performance. This is a reason why researchers have limited their studies to particular problems and applications. In this context, isolated character recognition can be seen as a less complicated task where satisfactory solutions are already available. In word recognition tasks, the application specifies the lexicon of possible words. For small lexicons, as in bank check processing, most approaches are global, where a word is considered as an indivisible entity [1] , [2] , [3] . For large lexicons, as in postal applications [4] , [5] , [6] , the segmentation of words into basic units such as letters is required. Owing to the difficulty of this operation, most successful approaches are segmentation-recognition methods in which words are first loosely segmented into letters or pieces of letters, and a dynamic programming technique is used in recognition to choose the definitive segmentation [7] , [8] . Although these methods are less robust when the segmentation process fails to split a pair of letters (or more), they have many advantages over global ones. Indeed, for a given learning database, it is more reliable to train a small set of letters than whole words. Furthermore, unlike analytic approaches, global approaches are possible only for lexicon-driven problems and do not satisfy the portability criterion since, for each new application, the set of the lexicon words must be trained.. During the last decade, hidden Markov models (HMMs), which can be thought of as a generalization of dynamic programming techniques [9] , have become the predominant approach to automatic speech recognition [9] , [10] , [11] . These stochastic models have been shown to be well-adapted to summarize variability phenomena involved in time-varying signals. The success of HMMs in speech recognition has recently led many researchers to apply them to handwriting recognition by representing each word image as a sequence of observations. According to the way this representation is carried out, two approaches can be distinguished: implicit segmentation [4] , [12] , which leads to a speech-like representation of the handwritten word image, and explicit segmentation [5] , [6] , which requires a segmentation algorithm to split words into letters or pseudoletters.. In this paper, we propose an explicit segmentation-based HMM approach to recognize unconstrained handwritten words (uppercase, cursive and mixed). This system uses three sets of features: The first two are related to the shape of the segmented units, while the features of the third set describe segmentation points between these units. The first set is based on global features, such as loops, ascenders, and descenders, and the second set is based on features obtained by the analysis of the bidimensional contour transition histogram of each segment. Finally, segmentation features correspond to either spaces, possibly occurring between letters or words, or the vertical position of segmentation points that split connected letters. Given that the two sets of shape-features are separately extracted from the image, we represent each word by two feature sequences of equal length, each consisting of an alternating sequence of shape-symbols and segmentation-symbols. In the problem we are dealing with, we consider a vocabulary which is large but dynamically limited. For example, in city name recognition, the contextual knowledge brought by the postal code identity can be used to reduce the lexicon of possible city names to a small size. Since the entire vocabulary of words is large, it is more realistic to model basic units, such as letters, rather than whole words. Indeed, this modeling needs only a reasonable number of models to train (and to store). Then, each word (or word sequence) model can be dynamically built by concatenating letter models. This modeling is also more appropriate for available learning databases, which often do not contain all the possible words to be recognized. Our system also contains a mechanism to reject unreliable decisions.. This paper is organized as follows. Section 2 describes the fundamentals of hidden Markov models. Section 3 details the steps of preprocessing, segmentation, and feature extraction. Section 4 deals with the application of HMMs to handwritten word recognition in a dynamic vocabulary. Section 5 presents the experiments performed to validate the approach. Section 6 concerns the rejection mechanism considered by our system. Finally, Section 7 gives some concluding remarks and perspectives. In this paper, we presented a complete system for recognizing unconstrained handwritten words. We consider that our approach achieves good performance given that the data correspond to real word images and were not filtered; however, it is hard to compare with other approaches since we are not using the same databases. The main strength of the proposed system lies in its training phase, which does not require any manual segmentation of the data to train the character models. We used character HMMs to model explicitly both segmented shapes and associated segmentation points, leading to a better discrimination between characters. By building the word model as a sequence of character models, each consisting of a pair of associated upper-case and lower-case HMMs, the writing style is implicitly detected during recognition. We proved also that rejection can be efficiently implemented using HMMs. We believe that our system can still be improved, mainly in feature extraction and recognition phases. As mentioned in Recognition rate (RR), confusion rate (CR) and reliability (RL) as a function of rejection rate when the probability that the word image belongs to the lexicon is in HXS.. Section 3.3, we use simple features to describe the segments and we rely on the context to discriminate the words. This may not be sufficient for small words which only differ in one or two letters (e.g., Reims versus Rennes, Nantes versus Mantes, etc.). In this case, low-level features describing the pixel distributions in the segments are necessary to eliminate the ambiguity between such words. An elegant method could be a hierarchical data representation which provides more details as the feature sequence length gets smaller and the ambiguity between the dynamic lexicon candidates (to be computed dynamically) gets higher. This could be an optimal compromise between performance and speed requirements in commercial systems. Markovian modeling could be improved by replacing discrete HMMs by semicontinuous HMMs in order to avoid the inherent loss of information when producing our feature sets, particularly histogram-based and segmentation-based features; or by considering hybrid Markovian/neural models to overcome the lack of discrimination power in conventional HMMs. Finally, our model architecture could be optimized by adding in the character model (Fig. 6 ) a null transition from state 6 to state 4 (t TR ), generating in this way a loop that can model the oversegmentation of a letter into any number of graphemes. This would be particularly interesting for the design of generic models that can accommodate an important writing style change possibly occurring with the use of new sources of handwriting data. Experiments were carried out on unconstrained handwritten French city name images manually localized on real mail envelopes which were scanned at a resolution of 200 dpi. The sizes of the learning, validation and test sets were 11,410, 3,724, and 4,313, respectively. To simulate the address recognition task, we assume that the city names in competition are independent for a given list of corresponding postal code hypotheses. This is only partly true, since confusable codes may belong to the same geographic area and, in some cases, the cities in a given area tend to share a common element (e.g., BOULOGNE SUR SEINE, postal code 92100, and NEUILLY SUR SEINE, postal code 92200). Nevertheless, under this assumption, for each image in the test set, we choose x À I city names from a vocabulary of 9,313 city names. The prior probabilities were assumed to be equal so that the average perplexity was maximum and equal to x. Indeed, for large vocabularies, rw is very difficult to estimate due to the lack of sufficient training samples. Moreover, we cannot rely on bigram or trigram statistics in this case because there is no reason for the size of a city to be correlated to its ASCII name. For example, PARIS is a big city, but PALIS is not. The best way for a reliable estimation of rw is, actually, to use city information, such as the population size, the postal code (in France, a zip code ending with 000 corresponds to a big city), the economic power, etc. To simulate the case where one, two, or three digits in the postal code are ambiguous (one, two, or three ambiguous digits give rise to 10, 100, or 1,000 possible postal codes), the values chosen for x were 10, 100, and 1,000. Recognition was carried out using the logarithmic version of the Viterbi algorithm, while, for training, we used the Baum-Welch algorithm. Since segment data were not labeled, the model parameters were initialized with random values. Training experiments carried out with uniform and various random starts showed essentially the same performances. The recognition rates obtained using the models w I , w P , and w Q (defined in Section 4.5) are reported in Table 1 and some well-recognized French city name examples are given in Fig. 11 .. From the above results, it is clear that HMMs enable the system to achieve good performance even though the features used are not very discriminative at the grapheme level (a grapheme consists of a letter, left part, or right part of a letter) for which the recognition rate is only around 23 percent. Actually, it is the redundancy brought by the ordered association of features to describe a word that is discriminative and this discrimination is as high as the word length. The use of the segmentation features and preprocessing has also significantly contributed to recognition accuracy [20] . The similarity of the results obtained with models w I and w P is not surprising since the advantages of w I are the drawbacks of w P and vice versa (Section 4.5). The interpolation technique yielded a significant improvement, especially if we keep in mind that only the models associated with nonfrequent characters in the training set took advantage from it. Confusions in our system come from many sources, mainly from words with several overlapping characters (Fig. 12a) , poor images (Fig. 12b) , images with underline or with noise or remnant strokes above them (Fig. 12c) , or the lack of examples to reliably estimate some model parameters.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B B O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multi-label ensemble based on variable pairwise constraint projection Multi-label classification has attracted an increasing amount of attention in recent years. To this end, many algorithms have been developed to classify multi-label data in an effective manner. However, they usually do not consider the pairwise relations indicated by sample labels, which actually play important roles in multi-label classification. Inspired by this, we naturally extend the traditional pairwise constraints to the multi-label scenario via a flexible thresholding scheme. Moreover, to improve the generalization ability of the classifier, we adopt a boosting-like strategy to construct a multi-label ensemble from a group of base classifiers. To achieve these goals, this paper presents a novel multi-label classification framework named Variable Pairwise Constraint projection for Multi-label Ensemble (VPCME). Specifically, we take advantage of the variable pairwise constraint projection to learn a lower-dimensional data representation, which preserves the correlations between samples and labels. Thereafter, the base classifiers are trained in the new data space. For the boosting-like strategy, we employ both the variable pairwise constraints and the bootstrap steps to diversify the base classifiers. Empirical studies have shown the superiority of the proposed method in comparison with other approaches. Traditional supervised learning deals with the data sample associated with only single label, which indicates its class membership. However, in a broad range of real-world applications, one sample is usually related to multiple class labels simultaneously, and such data has appeared in a large volume of various domains, e.g., scene or sentiment classification, video annotation and functional genomics [4, 10, 19, 25, 43] . To address this problem, multi-label learning has received a growing interest in the last decade [12, 30, 38] . Compared to single-label learning, multi-label learning is more challenging as each sample contains more than one label, e.g., a document concerning health might also be related to other topics, such as education, entertainment and government [8, 9, 21] . Typical multi-label learning methods include multi-label neural networks, multi-label transfer learning and multi-label kernel learning [13, 16, 34, 46] , which are fundamentally derived from the corresponding single-label learning approaches. For more details, we refer the readers to [35] .. In this paper, we study supervised multi-label classification. During the past few years, many multi-label classification methods have been proposed in a number of works [4, 9, 11, 27, 28] , whereas they often neglect the pairwise relations suggested by sample labels. Such relations can indicate whether a given sample pair has similar label sets or not, which are of vital importance in multi-label classification. Besides, the pairwise label constraints have been proved to be effective in single-label. The main contributions of this work are highlighted as follows. l We present a novel multi-label classification framework to construct a multi-label ensemble.. It takes advantage of the variable pairwise constraint projection to obtain a well preserved low-dimensional data representation, where the base classifiers are learned. l A boosting-like strategy is utilized to obtain a group of diversified base classifiers, whose diversities are essentially encouraged by both the variable pairwise constraints and the bootstrap steps. The attained base classifiers are combined to construct a multi-label ensemble for final classification. l Extensive experiments were conducted on various real-life datasets. Results demonstrate that our framework enjoys significant advantages compared to other methods.. The remainder of the paper is structured as follows. We briefly review the multi-label classification and ensemble learning in Section 2. We introduce our multi-label classification framework as well as some rigorous analysis in Section 3. Experimental results on a broad range of real-world multi-label datasets are reported in Section 4. Finally, we provide the concluding remarks and discuss future work in Section 5. In this paper, we introduce a novel multi-label classification framework called Variable Pairwise Constraint projection for Multi-label Ensemble (VPCME) to construct a multi-label ensemble for handling multi-label data. This framework involves two inherent components, i.e., the variable pairwise constraint projection and the boosting-like strategy. In detail, we employ the variable pairwise constraint projection to obtain a well preserved lower-dimensional data space, where the base classifiers are learned. Besides, we make use of a boosting-like strategy to improve the generalization ability of the classifier. For the boosting-like strategy, both the variable pairwise constraints and the bootstrap steps are exploited to diversify a group of base classifiers. In this work, the majority voting is adopted to decide the estimated label set for each test data. We conducted extensive interesting experiments over a range of multi-label datasets. Results show that our proposed approach performs better than other competing methods.. Nevertheless, there still remain some problems to be explored in future. For example, it is sensible to develop one principle way to select the optimal threshold for variable pairwise constraints. Due to the high dimensions of many real-world data, it is practically important to study the joint learning of multi-label feature selection and multi-label ensemble so as to select the most informative feature subsets. Another interesting problem is to explore the way to speed up the multi-label ensemble approach such that the computational costs can be greatly reduced. In this section, we report the results of four experimental groups in Section 4.3.2 respectively as well as some analysis. Table 3 (a-e) and Table 4 report the results of Group1, Fig. 1 depicts the results of Group2, the results of Group3 are tabulated in Table 5 Table 3 shows the results of different multi-label algorithms on several data sets. Records are tabulated in terms of averaged mean values as well as standard deviations over 20 test runs. To examine whether the results are statistically significant, paired t-tests were carried out at 1% significance level. Our work focuses on multi-label classification and the proposed framework is closely related to ensemble learning. So in this section, we provide a brief description about multi-label classification and ensemble learning. This section show empirical studies on a broad range of real-world multi-label datasets. First, we. give a brief description about the evaluation metrics and data sets. Second, experimental setup and performance comparisons are presented. Third, we report the results as well as some analysis.

O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An Action Compiler Targeting Standard ML We present an action compiler that can be used in connection with an action semantics based compiler generator. Our action compiler produces code with faster execution times than code produced by other action compilers, and for some non-trivial test examples it is only a factor two slower than the code produced by the Gnu C Compiler. Targeting Standard ML makes the description of the code generation simple and easy to implement. The action compiler has been tested on a description of the Core of Standard ML and a subset of C. Automatically generating a compiler from a formal description of a language does not always lead to efficient compilers. A formalism that supports easy construction of readable, complete, and reusable descriptions of most programming languages and at the same time has tool support for automatically generating efficient compilers seems to be non-existing. One formalism that tries to satisfy these requirements to a language description formalism and allows automatic generation of efficient compilers is Action Semantics (AS) [12, 16] . By efficient compilers we mean compilers that produce fast code, and not compilers that run fast or produce small code. An AS based compiler generator produces a front end which maps each program in the described language to an action. The front end is then connected to an action compiler, and the result is a compiler for the described language. Previous results [20, 21] have shown that it is possible to generate compilers that produce code that is less than ten times slower than the code generated by handwritten compilers, and in some cases even as fast as only two times slower. Some restrictions have been put on the actions handled by the compiler to achieve this result, and often the implementation of the code generator in the action compiler is very complicated.. We present an action compiler that produces more efficient code than previous action compilers, and on some examples only a factor two slower than the code produced by the Gnu C compiler. The code generator translates actions to Standard ML (SML) [15] in a straight forward way. The SML code is then compiled to an executable using the MLton 2 compiler. An action compiler annotates and transforms the action in several steps. Our action compiler performs type inference (Section 3) and code generation (Section 4), but no optimizations on the action as seen in previous results. Instead we generate code that can easily be optimized by MLton.. It is an advantage to be familiar with AS and SML, but not a prerequisite, when reading the paper. We will shortly introduce action semantics in the following section. In Section 2 we present the Action Environment which serves as a front end generator in our compiler generator. Type inference of actions is an essential part of generating efficient code from actions, and the subject of Section 3.. The main contribution of this paper, namely the rules for translating actions into SML, is described in Section 4. Before evaluating the action compiler in Section 6 we take a look at previous work on compiling actions in Section 5. In Section 7 the limitations of our action compiler are discussed. Section 8 concludes. We have presented an action compiler that, compared to previous results, is a small improvement with respect to the efficiency of the generated code. Our main contribution is the simplicity of the code generation where we use SML as target language.. Future work includes investigating how to generate code that is easier for the SML compiler to optimize. Especially the way data is represented in the generated code needs improvement. Relaxing the restrictions put on actions would also improve the system. Improving the type inference algorithm such that it accepts a bigger set of actions, would allow more natural descriptions of languages, but here we are also limited by the target language (SML) being strongly typed.. It would be interesting to see our compiler tested on the full Standard ML language or another realistic language, instead of just a sublanguage. Previous work has also only been tested on small languages; so far it has not been investigated how well action compilers scale to handle realistic programming languages. We think that at the moment van der Storm's compiler is the compiler that has the best chance of handling actions originating from a realistic language description because it can handle data that can be described using ATerms [3] .. Using another target language is also worth investigating. There are compilers for Scheme and OCaml that on some examples produce faster code than MLton does on similar SML programs. . fail −→ fn (t, b) => raise FAIL (27). choose-nat −→ fn (t, b) => random () The Actress system [7] showed how to compile actions into C code. The compilation involved several action optimizations where the most important one was binding elimination. The system has been tested on a specification of a small imperative language called Specimen, and the running time of the generated C code for some programs has been compared to running times for implementations of the same programs in Pascal. This comparison shows that the generated C code is between a factor 5 to 28 slower than the compiled Pascal code. The rules describing the code generation are complicated because they use a set of variables to pass data between actions and must keep track of which variables are used and have been used by sub-actions.. Peter Ørbaek's OASIS [21] generated SPARC assembler code. This system applied several optimizations known from handwritten compilers, like constant propagation and tail recursion detection. In a comparison between programs compiled with a generated compiler for an imperative language HypoPL and equivalent programs written in C and compiled with GCC 2.4.3 (with full optimization), Ørbaek showed that the code from the generated compiler was between 1.5 to 4 times slower. Due to the low level of the target language, the code generation is complicated 4 . Continuing the work done by Brown, Moura and Watt on the Actress system, Kent D. Lee developed the Genesis system [13] . The systems have many similarities with respect to type inference and action transformations, but instead of generating C code, Genesis generates Java bytecode. One advantage of this is the portability of the generated code. As with the OASIS system, the low level target language makes code generation complicated, and special transformations of actions are needed. Lee does not present any evaluation of the generated code.. A somewhat different approach has been demonstrated by Bondorf and Palsberg in [2] . By writing an action interpreter in Scheme and applying the Similix partial evaluator, they were able to generate an action compiler that generates Scheme code. The advantage of this approach is that it is easier to write an action interpreter than an action compiler, and the hard work is done by Similix. Should AN change it is also easier to update an action interpreter than an action compiler. Their evaluation of the generated scheme code shows that it is almost 100 times slower than code generated by a hand written compiler.. Recently Tijs van der Storm [5] has shown a simpler approach to compiling actions to C and Java. Comparing it with Actress and Genesis the compiler is simpler because it does not perform any type inference or optimizations. Instead of translating an action combinator to a sequence of statements, it translates it to a function that calls the functions representing the sub-actions. Because his compiler does not perform type inference, the code produced is not easily optimized by the C (or Java) compiler which is reflected in his test results. Van der Storm has only documented a test where he uses an action calculating fibonacci numbers (see Section 5 in [5] ). The best result achieved when calculating the 20th fibonacci number is a running time of 0.8 seconds.. To compare we have achieved a running time of 0.5 seconds for calculating the 33rd fibonacci number on slower hardware (Intel Pentium III 1 Ghz) than the hardware used by van der Storm (AMD XP 1800+). We were not able run his action compiler to better compare the two compilers.. There is a huge selection of compiler generators employing other formalisms than AS available. We shall mention two systems here that also seems to be popular outside academia, contrary to the AS based systems.. The Eli system [9] is based on attribute grammars. In addition to using attribute grammars the user can specify part of the compiler by "analogy" which means that the system has a large library of constructs used in common programming languages, so if the user wants scope rules similar to the ones used by Algol 60, he should just include the right module in the specification instead of writing it from scratch. The user can also specify part of the compiler by "solution" which means that he can write arbitrary fragments of C code that solves a problem. There are no examples in the literature of using Eli for implementing compilers for functional or object oriented languages, but a large set of real world imperative languages (Algol 60, C, Pascal) have been implemented completely or partly. In [14] the compiler for a Pascal-like language generated by Eli is compared with GCC, and the results show that the Eli generated compiler produces code that is approximately 35 % slower than the code produced by GCC.. In Gentle [18] the specification of a compiler is done in a logic programming language which is used in all parts of the specification. The specification language resembles Prolog but is more restricted and therefore the unification algorithm could be optimized. In [19] Vollmer reports that Gentle generates very efficient compilers with respect to compilation time, and that user experience shows that developing compilers in Gentle saves time compared to hand-coding compilers.

O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient model-based leak detection in boiler steam-water systems A model-based least-squares algorithm with a time-varying forgetting factor is developed for leak detection in boiler steam-water systems. The algorithm has been tested using real industrial data from Syncrude Canada, and has proven to be effective in detection of boiler tube or steam leaks; proper implementation of the algorithm would lead to early leak warning, which is important in maintaining safe operation of the plant. # Boilers are important components in power, chemical and oil refinery industries; they transform water into steam for power generation or other industrial applications. A common boiler fault is the tube leak in the riser and downcomer sections due to aging and thermal stress. Early detection of such faults in operation are important: It helps in reducing possible damage to equipments and productivity loss caused by (otherwise) unscheduled boiler shut-down; it also ensures safety for operators.. Boiler tube leaks are a special type of faults in process systems, which belong to the class of gross error problems. Most methods for gross error estimation use statistical analysis and system identification methods (Kao, Thamhane & Mah, 1992; Chemielewaki, Manousiouthakia, Tilton & Felix, 2000; Crowe, 1988) . For example, Narasimhan and Mah (1987) derived a generalized likelihood ratio method for identifying gross errors in steady state chemical processes; Chemielewaki et al. proposed a method for tank leak and instrument bias detection, in which a Kalman filter was used in the detection scheme.. However, since boilers are complex systems involving various disturbances, nonlinearities, and time-varying characteristics in the transient responses, the above steady-state based methods would not provide satisfactory performance in our leak detection problems. In this research, an effective model-based leak detection scheme is presented, taking into account boiler dynamics; the algorithm is based on least squares estimation with a time-varying forgetting factor. Application of the proposed scheme to industrial case studies shows advantages of the method.. Briefly, the paper is organized as follows. Section 2 briefly introduces an analog boiler model. Section 3 presents the proposed leak detection algorithm, together with some discussion of the implementation issues. Section 4 collects results in three case studies involving Syncrude industrial boilers. Finally, Section 5 offers some concluding remarks. In this work, we provided an effective and efficient solution to a leak detection problem involving industrial boilers. The solution was a model-based least squares algorithm with a time-varying forgetting factor. The algorithm has been tested using industrial data from Syncrude Canada, all resulting in accurate and early detection of boiler leaks. Thus we conclude that implementation of the proposed algorithm would lead to significant improvement in safety and maintenance of industrial boilers.. Boiler model parameters in the form of Eq. (2) 10 000 10 000

O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Improving learning accuracy of fuzzy decision trees by hybrid neural networks Abstract-Although the induction of fuzzy decision tree (FDT) has been a very popular learning methodology due to its advantage of comprehensibility, it is often criticized to result in poor learning accuracy. Thus, one fundamental problem is how to improve the learning accuracy while the comprehensibility is kept. This paper focuses on this problem and proposes using a hybrid neural network (HNN) to refine the FDT. This HNN, designed according to the generated FDT and trained by an algorithm derived in this paper, results in a FDT with parameters, called weighted FDT. The weighted FDT is equivalent to a set of fuzzy production rules with local weights (LWs) and global weights (GWs) introduced in our previous work. Moreover, the weighted FDT, in which the reasoning mechanism incorporates the trained LWs and GWs, significantly improves the FDTs learning accuracy while keeping the FDTs comprehensibility. The improvements are verified on several selected databases. Furthermore, a brief comparison of our method with two benchmark learning algorithms, namely, fuzzy ID3 and traditional backpropagation, is made. The synergy between FDT induction and HNN training offers new insight into the construction of hybrid intelligent systems with higher learning accuracy.Index Terms-Approximate reasoning, fuzzy decision trees, hybrid neural networks, knowledge acquisition, learning, local and global weights. The inclusion of some knowledge parameters such as LW and GW is necessary for enhancing the representation power of FDTs. This paper proposes refining these knowledge parameters in a hybrid neural network to improve the learning performance of FDTs, instead of increasing the numbers of linguistic terms or the complexity (the nodes) of FDTs to improve that. The main advantages of the proposed method are as follows:. 1) the learning performance can be improved by refining these parameters without much computational effort. 2) since each knowledge parameter used in FDTs has the clear meaning, the comprehensibility of FDTs can be kept. 3) to determine these knowledge parameters, the training of HNN can replace the task of consultation with domain specialists to a great extent. The synergy between fuzzy decision tree induction and hybrid neural network offers new insight into the construction of hybrid intelligent systems with better learning accuracy.

O B O O B I I O B I I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I I B I I B I B B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O


Preliminary forensic analysis of the Xbox One Video game consoles can no longer be viewed as just gaming consoles but rather as full multimedia machines, capable of desktop computer-like performance. The past has shown that game consoles have been used in criminal activities such as extortion, identity theft, and child pornography, but with their ever-increasing capabilities, the likelihood of the expansion of criminal activities conducted on or over the consoles increases. This research aimed to take the initial step of understanding the Xbox One, the most powerful Microsoft console to date. We report the outcome of conducting a forensic examination of the Xbox One, and we provide our Xbox One data set of hard drive images and unique files so that the forensic community may expand upon our work. The Xbox One was found to have increased security measures over its predecessor (Xbox 360). The encryption of the data and the new file types introduced made it difficult to discern potential digital evidence. While these added security features caused great difficulty in forensically acquiring digital forensic artifacts, some important and interesting digital evidence was gathered using open-source tools. We were able to find digital evidence such as times that the user initially set up the console, and times when the system was restored or shutdown. We were also able to determine what games and applications had been downloaded along with when the games were played. Finally, through our network forensic experiments, we were able to determine that various applications had different levels of security and that game traffic was encrypted. According to Forbes, as of January 2014, Microsoft had sold approximately 3.4 million Xbox One units since its release on November 22, 2013. In recent years, Microsoft's Xbox systems have maintained top gaming console sales amongst its competitors.. Gaming systems are now comparable to desktop computers such that they are just as powerful, have networking capabilities, they contain high-powered graphics processors and a large amount of storage.. As the capabilities of these gaming consoles increase, so does the potential for them to be used in illicit activities. Criminal investigators have historically sought to gather evidence from PCs, mobile phones, PDAs, and other mobile devices; however, they may overlook these gaming consoles even though they can potentially contain valuable forensic artifacts that can be used for evidentiary purposes. It is imperative to provide analysis of the Xbox One to provide investigators an understanding of the proprietary system with hopes of retrieving the evidence it may hold.. This research serves as an initial examination of the Xbox One and its file system, allowing investigators some information on how to investigate an Xbox One. This includes understanding the file structure of the system, what forensically valuable information is available on the console, and where that information may be located. More research is needed with regard to the Xbox One, its file system, and the encryption methods used. Understanding the file types (and potentially, sub-types) found within the Xbox One, such as .xvd and .xvi, is necessary to further examine the console. Although many of these files seem, on the basis of their entropy scores, to be encrypted (or at least compressed with an unknown scheme), some files were not, and these should therefore be priorities for reverse engineering.. It may also prove valuable to look at an Xbox One hard disk drive when it is first shipped. The hard drive employed in this experimental work had been used prior to selecting it. Although it was restored to factory settings, it may prove useful to see what, if anything, is dissimilar between a factory restored drive versus a newly shipped drive.. A more thorough examination of the Xbox Live network is needed. There are many other applications and games that need to be tested. Additionally, there is a mobile phone application known as Smart Glass that allows the user to interact with the Xbox through their phone. It would be of interest to examine this relationship to see what data can be found in the transmissions between the phone and Xbox One.. A database could be made containing all of the downloaded material available for the Xbox One, along with hash values and the file sizes of that material. This would allow for quick identification of different applications, games, or other downloaded material, thereby making investigations involving this device more efficient. This research provides the initial foundation for understanding how an Xbox One can be examined in a forensically sound manner. Even without modifications, the Xbox One is a very powerful computing device and should not be seen as just a gaming console; it is marketed for its multimedia capabilities just as much as it is for its video games. As with the previous generations of video game consoles, crime will undoubtedly take place involving the Xbox One, both over Xbox Live and locally, thus making it essential that investigators understand how to analyze the system.. The complexity of the Xbox One seems greater than its predecessors. It appears to make heavy usage of encryption (at least after connection to Xbox Live), and its new file types made it extremely difficult to discern any information. We were not able to determine exactly when or how the user used applications, watching television did not seem to log any data on the system, and signing in to the user's profile could not be determined from the metadata alone. The data contained in the MFT along with information ascertained from this research can be of great importance to investigators. We were able to retrospectively link files to the games and applications that were installed on the Xbox One, we could see when the console was last shutoff, we could see when the system was restored to factory settings, and were even able to determine the first time the user ever used the system. The metadata found in the MFT can be used by investigators to develop timelines, patterns of use, and corroborate a story. We also noted that the network traffic that was captured varied based on what specific applications were being used. Each application had different levels of security associated with them, for example we were not able to see the calls or messages made with Skype but we were able to view the exact game clip that was watched during the use of the Game DVR application. The video games themselves were secure. We were able to discern which game was being played but the actions that took place during play were not discovered.

O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O B I O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O