Integrating Guidance into Relational Reinforcement Learning Abstract. Reinforcement learning, and Q-learning in particular, encounter two major problems when dealing with large state spaces. First, learning the Q-function in tabular form may be infeasible because of the excessive amount of memory needed to store the table, and because the Q-function only converges after each state has been visited multiple times. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. The first problem is often solved by learning a generalization of the encountered examples (e.g., using a neural net or decision tree). Relational reinforcement learning (RRL) is such an approach; it makes Q-learning feasible in structural domains by incorporating a relational learner into Q-learning. The problem of sparse rewards has not been addressed for RRL. This paper presents a solution based on the use of "reasonable policies" to provide guidance. Different types of policies and different strategies to supply guidance through these policies are discussed and evaluated experimentally in several relational domains to show the merits of the approach. In reinforcement learning (for an excellent introduction see the book by Sutton and Barto, 1998) , an agent tries to learn a policy, i.e., how to select an action in a given state of the environment, so that it maximizes the total amount of reward it receives when interacting with the environment.. Q-learning (Watkins, 1989 ) is a form of reinforcement learning where the optimal policy is learned implicitly in the form of a Q-function, which takes a state-action pair as input and outputs the quality of the action in that state. The optimal action in a given state is then the action with the largest Q-value.. One of the main limitations of standard Q-learning is related to the number of different state-action pairs that may exist. The Q-function can in principle be represented as a table with one entry for each state-action pair. When states and actions are characterized by parameters, the number of such pairs grows combinatorially in the number of parameters and thus can easily become very large, making it infeasible to represent the Q-function in tabular form, let alone learn it accurately (convergence of the Q-function only happens after each state-action pair has been visited many times). This problem is typically solved by integrating into the Q-learning algorithm an inductive learner, which learns a function that generalizes over given state-action pairs. Thus reasonable estimates of the Q-value of a stateaction pair can be made without ever having visited it. Examples include neural networks (Rumelhart and McClelland, 1986) , nearest neighbor methods (Smart and Kaelbling, 2000) and regression trees (Chapman and Kaelbling, 1991) .. A relational learner is employed by Džeroski, De Raedt, and Blockeel (1998) , hence the name "relational reinforcement learning" or RRL. This relational learner uses first order representations for states and actions, and uses a first order regression algorithm that maps these structural descriptions onto real numbers. The use of first order representations gives RRL a broader application domain than classical Q-learning approaches. Examples of such relatively complex applications that will be described in more detail further in this paper, include learning to solve simple planning tasks in a blocks world, or learning to play certain computer games (Tetris, Digger) .. In structural domains, the state space is typically very large, and although a relational learner can provide the right level of abstraction to learn in such a domain, the problem remains that rewards may be distributed very sparsely in this state space. Using random exploration through the search space, rewards may simply never be encountered. In some of the application domains mentioned above this prohibits RRL from finding a good solution.. While plenty of exploration strategies exist (Wiering, 1999) , few deal with the problems of exploration at the start of the learning process. It is exactly this problem that we are faced with in our RRL setting. There is, however, an approach which has been followed with success, and which consists of guiding the Q-learner with examples of "reasonable" strategies, provided by a teacher (Smart and Kaelbling, 2000) . Thus a mix between the classical unsupervised Q-learning and (supervised) behavioral cloning is obtained. It is the suitability of this approach in the context of RRL that we explore in this paper. In this paper, we address the problem of integrating guidance and experimentation in reinforcement learning, and in particular relational reinforcement learning (RRL). The use of a more expressive representation in RRL allows for larger and more complex learning tasks to be addressed, where the problem of finding rewards that are sparsely distributed is more severe. We show that providing guidance to the reinforcement learning agent does help improve performance in such cases. Guidance in our case takes the form of traces of execution of a "reasonable policy" that provides sufficiently dense rewards.. We demonstrate the utility of guidance through experiments in three domains: the blocks world and two computer games (Tetris and Digger). Each of these three application domains is characterized by a huge state space and hard to reach rewards. We investigate the effect of using guidance in a number of settings, characterized along several dimensions. The dimensions include the mode of providing guidance, the form of the guidance policy used, and the generalization engine used within relational reinforcement learning.. We investigate two modes of using guidance: providing all guidance at the start and spreading guidance, i.e., providing some guided episodes followed by several exploration episodes, and repeating this. A variation on the latter mode is active learning, where the agent asks for guided traces starting from initial states that it selects itself rather than receiving guided traces from randomly chosen initial states. Three different forms of guidance policies are considered: hand-coded-possibly optimal-policies, policies already learned by RRL in a previous experiment and the policy of a human performing the task at hand. We use two different generalization engines within RRL, induction of relational regression trees (TG) and relational instance-based regression (RIB).. Overall, the use of guidance in addition to experimentation improves performance over using experimentation only, for all considered combinations of the dimensions mentioned above. We observed improvements in terms of the overall performance level achieved, the convergence speed, or both. The improvements result from using the best of both worlds: guidance provides perfect or reasonably good information about the optimal action to take in a narrow range of situations, while experimentation can obtain imperfect information about the optimal action to take in a wide range of situations. The actual magnitudes of performance improvement does depend on the considered combination of the mode of providing guidance and generalization engine.. While both guidance at the start and spread guidance improve the performance of RRL, spread guidance often yields higher, but more importantly never lower performance. This is especially the case if regression engine is vulnerable to over-generalization such as the relational regression trees (TG-trees). Providing all the guidance up front doesn't quite work well in this case for several reasons. Namely, making a split on "perfect" guidance generated examples only distinguishes between the regions of equal state-values, but not the actions that move you between them. This can be corrected by splits further down the tree, but this requires lots of extra examples, and therefore more learning episodes. This problem is aggravated by the fact that after making a split, the guidance received so far is lost. These problems do not appear when instance-based regression (RIB) is used as a generalization engine as the RIB-algorithm is designed to remember high yielding examples and the bias towards action difference of the used distance prevents the over-generalization that occurs with TG.. Active learning with spread guidance helps improve performance in the later stages of learning, by enabling fine tuning of the learned Q-function by focusing on problematic regions of the state space. This often results in a significant reduction of the cases where RRL exhibits non-optimal behavior. Experiments show that a sufficiently high level of performance has to be reached by G-RRL for the active guidance to have any effect. If performance is too low to allow fine-tuning, active guidance does not improve on normal guidance.. Guidance from different sources can be successfully used within the proposed approach. Whether we used traces generated by hand-programmed policies, policies learned by RRL or a human performing the task at hand, we always obtained higher performance as compared to using no guidance. The quality of the policy used to generate guidance does matter, as illustrated by the experiment on the Tetris domain, where using a better policy to provide guidance yielded significantly better performance. In this section, we summarize the experimental results from the previous sections and draw some general conclusions. This takes the form of several questions that were addressed by the experiments and the answers obtained from the experimental results.. -Does guidance improve the performance of RRL for problems with large state spaces and sparse rewards? The answer to this question is an equivocal yes. Guidance always improved the performance of RRL, in terms of the level of performance achieved, the speed of convergence, or both. This proved true across a range of experimental setups, where the problem addressed was varied, as well as the guidance policies, the mode of providing guidance and the generalization engine used within RRL. The only exception is that RRL-TG may converge more slowly towards the optimal policy or even achieve a lower performance level if all guidance is supplied at the start. -Does more guidance mean better results? Providing more guided traces can improve the performance of RRL, in particular when they are provided to RRL-RIB or we are talking about a relatively small number of guided traces, which are provided at the start. Providing RRL-TG with a large number of guided traces at the start can even reduce performance due to overgeneralization problems. -What mode of providing guidance works best? This depends on the generalization engine used. For RRL-TG, guidance should be spread, i.e., guided traces should be interleaved with exploration traces. For RRL-RIB, providing all the guidance at the start works best. If a relatively high level of performance is achieved, it can be further improved by using active guidance. -Which generalization engine works better within RRL? RRL-TG is more efficient and achieves higher levels of performance, provided a large number of episodes are available and guidance is spread. RRL-RIB converges faster and achieves better performance levels when a smaller number of episodes is available and all evidence is given at the start. -How do the different guidance policies affect the performance of G-RRL? When the guidance policy yields high rewards or is even optimal, G-RRL often, but not always, approaches the performance of the guidance policies. When the guidance policy is clearly not optimal, G-RRL may perform much better than the guidance policy. In such cases, improving the guidance policy is likely to result in improved performance in G-RRL. However, it is crucial not to provide too much good guidance to RRL-TG at the start, due to overgeneralization problems. In such cases, guidance from suboptimal policies may avoid overgeneralization and result in better performance. The idea of incorporating guidance in automated learning of control is not new. In Chambers and Michie (1969) three kinds of cooperative learning are discussed. In the first, the learning system just accepts the offered advice. In the second, the expert has the option of not offering any advice. In the third, some criterion decides whether the learner has enough experience to override the human decision. Roughly speaking, the first corresponds to behavioral cloning, the second to reinforcement learning and the third to guided reinforcement learning. The link between this work and behavioral cloning (Bain and Sammut, 1995; Urbancic, Bratko, and Sammut, 1996) is not very hard to make. If we would supply the TG generalization algorithm with low Q-values for unseen state-action pairs, RRL would learn to imitate the behavior of the supplied traces. Because of this similarity of the techniques, it is not surprising that we run into similar problems as one encounters in behavioral cloning. Scheffer, Greiner, and Darken (1997) discusses some of these problems. The differences between learning by experimentation and learning with "perfect guidance" (behavioral cloning) and the problems and benefits of both approaches are highlighted. Behavioral cloning seems to have the advantage, as it sees precisely the optimal actions to take. However, this is all that it is given. Learning by experimentation, on the other hand, receives imperfect information about a wide range of state-action pairs. While some of the problems Scheffer mentions are solved by the combination of the two approaches as we suggest in this paper, other problems resurface in our experiments. Scheffer states that learning from guidance will experience difficulties when confronted with memory constraints so that it can not simply memorize the ideal sequence of actions but has to store associations instead. This is very closely related to the problems our generalization engine has when it is supplied with only perfect state-action pairs.. Wang combines observation and practice in the OBSERVER learning system in Wang (1995) which learns STRIPS-like planning operators (Fikes and Nilsson, 1971) . The system starts with learning from "perfect guidance" and improves on the planning operators (preand postconditions) through practice. There is no reinforcement learning involved.. Lin's work on reinforcement learning, planning and teaching (Lin, 1992) and the work of Smart and Kaelbling on reinforcement learning in continuous state-spaces (Smart and Kaelbling, 2000) is closely related to ours in terms of combining guidance and experimentation. Lin uses a neural network approach for generalization and uses a human strategy to teach the agent. The reinforcement learning agent is then allowed to replay each teaching episode to increase the amount of information gained from a single lesson. However, the number of times that one lesson can be replayed has to be restricted to prevent over-learning. This behavior is strongly related to the over-generalization behavior of TG when only perfect guidance is presented.. Smart's work deals with continuous state-spaces uses a nearest neighbor approach for generalization and use example training runs to bootstrap the Q-function approximation. The use of nearest neighbor and convex hulls successfully prevents overgeneralization. It is not clear how to translate the convex hull approach to the relational setting.. Another technique that is based on the same principles as our approach is used by Dixon, Malak, and Khosla (2000) . They incorporate prior knowledge into the reinforcement learning agent by building an off-policy exploration module in which they include the prior knowledge. They use artificial neural networks as a generalization engine.. Other approaches to speed up reinforcement learning by supplying it with non-optimal strategies include the work of Shapiro, Langley, and Shachter (2001) . There the authors embed hierarchical reinforcement learning within an agent architecture. The agent is supplied with a "reasonable policy" and learns the best options for this policy through experience. This approach is complementary to and can be combined with our work on G-RRL.. Recently, there has been a large increase in the work on policy generation for relational domains. Most closely related to the approach used in the RRL system is the work on relational Markov decision processes. Kersting and De Raedt (2003) introduce Logical Markov Decision Processes as a compact representation of relational MDPs. By the use of abstract states (i.e. a conjunction of first order literals) and abstract actions that are defined in a STRIPS-like manner (Fikes and Nilsson, 1971 ) they greatly reduce the number of possible (state, action) pairs and thus the number of Q-values that need to be learned. Independent of this, Morales (2003) introduced rQ-learning, i.e. Q-learning in R-Space. R-Space consists of r-states and r-actions which are very comparable to the abstract states and abstract actions in the work of Kersting and De Raedt. The rQ-learning algorithms tries to calculate the Q-values of the (r-state, r-action) pairs. Van Otterlo (2004) defines the CARCASS representation that consists of pairs of abstract states and the set of abstract actions that can be performed in the given abstract state. While this is again comparable to the two previously discussed approaches, van Otterlo not only defines a Q-learning algorithm for his representation but also suggests learning a model of the relational MDP defined by the CARCASS to allow prioritized sweeping to be used as a solution method. Fern, Yoon, and Givan (2003) use an approximate variant of policy iteration to handle large state spaces (Fern, Yoon, and Givan, 2003) . A policy language bias is used to enable the learning system to build a policy from a sampled set of Q-values. Just like in standard policy iteration (Sutton and Barto, 1998) , approximate policy iteration interleaves policy evaluation and policy improvement steps. Yoon, Fern, and Givan (2002) present an approach for translating abstract policies from small relational MDPs to larger problems. Our experiments are aimed at evaluating the influence of guidance on the performance of relational reinforcement learning. The guidance added to exploration should have a twofold effect on learning performance. In terms of learning speed, we expect the guidance to help the Q-learner to discover higher yielding policies earlier in the learning experiment.. Through the early discovery of important states and actions and the early availability of these state-action pairs to the generalization engine, we also expect that it is possible for the Q-learner to reach a higher level of performance-i.e., a higher average reward-for a given amount of learning experience.. To test these effects, we compare RRL (without guidance) with G-RRL (with guidance) in the following setup: first we run RRL in it's natural form, giving it the possibility to train for a certain number of episodes; at regular time intervals we extract the learned policy from RRL and test it on a number of randomly generated test problems. To compare with G-RRL, we substitute some of the exploration with guided traces. These traces are generated by either a hand-coded policy, a previously learned policy or a human controller. In between these traces, G-RRL is allowed to explore the state-space further on its own. Note that in the performance graphs, the traces presented to G-RRL will count as episodes.. We conduct experiments in the three domains described in Section 4: the blocks world and two computer games (Tetris and Digger). Each of these three application domains is characterized by a huge state space and hard to reach rewards. In these domains, we investigate the effects of using guidance in a number of settings, characterized along several dimensions. The dimensions include the strategy/mode of providing guidance (as described in Section 5.3) and the generalization engine used within relational reinforcement learning (as described in Section 3).. We use several types of policies for guidance. In the blocks world, we use optimal (handcoded) policies and policies where optimal actions are interleaved with random actions (we call the latter "half-optimal"). In the Tetris game, we use performance traces from a human player, as well as hand-coded inoptimal policies. Finally, in the Digger policies, we use a policy learned by relational reinforcement learning to provide traces for guidance, thus bootstrapping the RRL process.

O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O


Asymptotic solvers for second-order differential equation systems with multiple frequencies In this paper, an asymptotic expansion is constructed to solve second-order differential equation systems with highly oscillatory forcing terms involving multiple frequencies. An asymptotic expansion is derived in inverse of powers of the oscillatory parameter and its truncation results in a very effective method of dicretizing the differential equation system in question. Numerical experiments illustrate the effectiveness of the asymptotic method in contrast to the standard Runge-Kutta method. The behaviour of signals comprising several non-commensurate frequencies is very important in the design and analysis of electronic circuits. Nonlinearities in circuits can result in such signals giving rise to distortion and resulting in degradation of performance. Hence, the accurate and efficient simulation of circuit behaviour in the presence of such signals is essential. It is to address this issue that the current paper is directed. As an example of nonlinear circuits, the Van der Pol oscillator shall be considered. It has numerous applications in science and engineering, for example, from describing the action potentials of biological neurons [5, 9] to the modelling of resonant tunneling diode circuits [12] . A coupled Van der Pol-Duffing system shall also be considered. Such coupled systems have applications in secure communications, [6, 10] . In addition, these coupled systems can be realised using analog circuitry [1] .. Consider a second-order differential equation system of the form y (t) + f (y(t))y (t) + g(y(t)) = F ω (t), t ≥ 0, (1.1). where f :. where every entry f jv (y) : C d → R and g j (y) : C d → R is an analytic scalar function for j, v = 1, 2, · · · , d. The initial conditions are y(0) = y 0 ∈ C d and y (0) = y 0 ∈ C d , and the forcing term F ω (t) is. in which a 1 , · · · , a M ∈ R + → C d are analytic functions. Note that we have assumed that there is a finite set of frequencies ω 1 , · · · , ω M ∈ R \ {0} in the forcing term. At least some of these frequencies are large which results in a highly oscillatory solution and one which is very expensive to obtain with classical discretization methods. Furthermore, assume that the functions f (y(t)) and g(y(t)) are analytic to ensure the existence and uniqueness of the solution y(t).. When d = 1, ω 2m−1 = mω, ω 2m = −mω, m = 0, 1, · · · , M 2 , ω 1, the above multiple frequency case reduces to a single frequency case y (t) + f (y(t))y (t) + g(y(t)) = ∞ k=−∞ b k (t)e ikωt , t ≥ 0, which has been already analysed in [3] .

O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient Opportunistic Routing in Utility-Based Ad Hoc Networks Abstract-Due to resource scarcity, a paramount concern in ad hoc networks is utilizing limited resources efficiently. The self-organized nature of ad hoc networks makes the network utility-based approach an efficient way to allocate limited resources. However, the effect of link instability has not yet been adequately addressed in literature. To efficiently address the routing problem in ad hoc networks, we integrate the cost and stability into a network utility metric, and adopt the metric to evaluate the routing optimality in a unified, opportunistic routing model. Based on this model, an efficient algorithm is designed, both centralized and distributed implementations are presented, and extensive simulations on NS-2 are conducted to verify our results. In this paper, we studied the routing problem in ad hoc networks by considering resource scarcity, and the unstable nature of mobile nodes. We used a model that is different from existing resource-efficient routing, and adopted a new metric called expected maximum utility to assess the routing optimality. By studying the relationship between link cost and link stability, we successfully combined these two different metrics, and designed an algorithm which integrates the opportunistic routing scheme to find the efficient routing solution. In the future, we will explore the effect of signal strength on stability, and analyze the relationship between retransmission and opportunism. Various existing routing protocols [1] , [7] pursue the minimum hop count, or minimum cost as their optimization goal. As discussed in [2] , and [4] , these metrics are not necessarily ideal because they do not take link stability into account. Another routing algorithm [8] models the link stability as the longevity of the link, and identifies the best route as the route with the longest lifetime. One similar scheme [9] adopts the "best worst link" concept in which the least stable link (the bottleneck link) over any route is identified, and the route with the best "bottleneck" is selected. Although these schemes consider the link stability issue, they fail to differentiate between two routes with the same bottleneck link, but different link characteristics. Expected transmission count [2] , and MintRoute [4] adopt a path delivery ratio by measuring the hop-by-hop link delivery ratio along the path. Expected transmission count weighs links with a metric called minimum expected transmission number, which is equal to the link cost divided by the link delivery ratio. However, this metric requires the unrealistic assumption that retransmission is unlimited.. Opportunistic routing (OR) [1] , [10] , [11] is mainly proposed to address the unreliable communication in multi-hop wireless networks by utilizing the broadcast advantages of wireless communication. Relay selection, and prioritization are two key issues of OR protocol design. Some works [10] , [12] , [13] apply the OR scheme to geographic routing, and select nodes that are geographically closer to the destination to form a relay set. Other OR protocols [1] , [14] , [15] select any node with smaller cost to the destination as a relay. In the model proposed in this paper, relays do not need to be "closer" to the destination. Some previous works [16] - [20] focus on link layer coordination schemes to ensure that multiple receivers of a packet agree on a relay in a distributed fashion. These schemes select the relay set based on some heuristic methods. Although one recent work [14] demonstrates the performance benefit by integrating network coding, this result is based on a strong assumption of unlimited retransmissions. Most existing works [1] , [21] prioritized the relays based on the expected number of transmissions along the best path alone. Until recently, other works [14] , [22] proposed metrics capable of capturing the expected number of any-path transmissions needed to successfully deliver a packet between two nodes under the OR framework.. Recently, there is a trend in integrating network coding techniques into opportunistic routing [14] , [23] . In [14] , the authors adopt the random linear coding scheme to allow intermediate nodes to code the received packets so that no MAC layer acknowledge scheduling is needed. However, controlling of outgoing packets is a big challenge, and the authors [14] fail to provide a convincing solution with an in-depth analysis.. Numerous works adopt utility as the optimization objective. Li & Xue & Nahrstedt [24] present a price-based scheme to effectively allocate resources among multiple multi-hop flows. Their approach maximizes the aggregated utility of flows (the network utility), while maintaining basic fairness among multiple flows. Qiu & Marbach [25] propose a market-based approach to efficiently allocate bandwidth in wireless ad hoc networks. Although there are numerous existing works applying the network utility related approaches, as far as we know none of them combine stability with link cost to design an optimization model that will maximize the expected utility. In our previous work [26] , we proposed the basic expected network utility-based model. In this paper, we incorporate an opportunistic routing scheme, and design an efficient routing scheme. Zhou et al. [27] propose a utility-based routing model in wireless ad hoc networks. Similar to our work, their work also uses utility to reflect the QoS requirement, but they use delay as the QoS measurement, which is estimated through the number of hops. In their work, a trade-off exists between energy cost, and transmission delay. The objective of their routing model is to maximize the surplus, equal to utility minus cost.

O B I O O B I I O O O O O O O O B I I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O B O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


User interactions in social networks and their implications Social networks are popular platforms for interaction, communication and collaboration between friends. Researchers have recently proposed an emerging class of applications that leverage relationships from social networks to improve security and performance in applications such as email, web browsing and overlay routing. While these applications often cite social network connectivity statistics to support their designs, researchers in psychology and sociology have repeatedly cast doubt on the practice of inferring meaningful relationships from social network connections alone. This leads to the question: Are social links valid indicators of real user interaction? If not, then how can we quantify these factors to form a more accurate model for evaluating sociallyenhanced applications? In this paper, we address this question through a detailed study of user interactions in the Facebook social network. We propose the use of interaction graphs to impart meaning to online social links by quantifying user interactions. We analyze interaction graphs derived from Facebook user traces and show that they exhibit significantly lower levels of the "small-world" properties shown in their social graph counterparts. This means that these graphs have fewer "supernodes" with extremely high degree, and overall network diameter increases significantly as a result. To quantify the impact of our observations, we use both types of graphs to validate two well-known socialbased applications (RE [Garriss 2006] and SybilGuard ). The results reveal new insights into both systems, and confirm our hypothesis that studies of social applications should use real indicators of user interactions in lieu of social graphs. Social networks are popular infrastructures for communication, interaction, and information sharing on the Internet. Popular social networks such as MySpace and Facebook provide communication, storage and social applications for hundreds of millions of users. Users join, establish social links to friends, and leverage their social links to share content, organize events, and search for specific users or shared resources. These social networks provide platforms for organizing events, user to user communication, and are among the Internet's most popular destinations.. Recent work has seen the emergence of a class of sociallyenhanced applications that leverage relationships from social networks to improve security and performance of network applications, including spam email mitigation [Garriss 2006 ], Internet search [Mislove 2006] , and defense against Sybil attacks ]. In each case, meaningful, interactive relationships with friends are critical to improving trust and reliability in the system. Unfortunately, these applications assume that all online social links denote a uniform level of real-world interpersonal association, an assumption disproven by social science. Specifically, social psychologists have long observed the prevalence of low-interaction social relationships such as Milgram's "Familiar Stranger" [Milgram 1977 ]. Recent research on social computing shows that users of social networks often use public display of connections to represent status and identity [Donath 2004 ], further supporting the hypothesis that social links often connect acquaintances with no level of mutual trust or shared interests. This paper aims to answer the question: Are social links valid indicators of real user interaction? To do this, we gathered extensive data from crawls of the Facebook social network, including social and interaction statistics on more than 10 million users. We show that interaction activity on Facebook is significantly skewed towards a small portion of each user's social links. This finding casts doubt on the assumption that all social links imply equally meaningful friend relationships.. We introduce the interaction graph as a more accurate representation of meaningful peer connectivity on social networks. Analysis of interaction graphs derived from our Facebook data reveal different characteristics than the corresponding social graph. Most notably, interaction graphs exhibit an absence of small-world clustering. We also observe much lower average node degrees in the interaction graph as compared to the Facebook social graph. This confirms the intuition that human interactions are limited by constraints such as time, and brings into question the practice of evaluating social networks in distributed systems directly using social connectivity graphs.. Our study concludes with experiments to evaluate the effects of interaction graphs on two well known social applications. The performance of RE [Garriss 2006 ] improves with the use of interaction graphs, as the streamlined link structure helps control spam proliferation. In the case of Sybilguard ], the system becomes less able to effectively classify nodes once its assumptions about graph structure are violated. These experiments strongly suggest that social-based applications should be designed with interactions graphs in mind, so that they reflect real user activity rather than social linkage alone. For our experiments, we implement the SybilGuard algorithm on both our social graph and interaction graphs and measure the percentage of paths that successfully intersect as w increases. For each graph and each value of w we chose 25000 random pairs of nodes to perform intersection tests on.. The reduction of highly connected super nodes in the interaction graph means that random walks (and random routes) are less likely to connect. Figure 18 shows that for the Facebook social graph, the probability for all paths to intersect approaches 100% at w = 1200. For interaction graphs, the percentage of intersecting paths never reaches 100% since a large fraction of random walks never intersect. SybilGuard, as a result, is less effective on a graph that models user trust (interaction graph) than on a normal social graph. A major factor affecting the performance of the SybilGuard algorithm is the prevelence of self-loops in the random walks. Any walk that returns to the origin point before going w steps is useless for the purposes of performing intersection tests. Table 2 shows the total number of self-loops encountered during all experimental runs on each graph. The drop in efficacy observed in Figure 18 is directly correlated to the increase in self-looping from 3.8% on the social graph to an upwards of 20% on interactions graphs. The body of research geared towards real-world social webs and physical networks has only recently begun to be leveraged to understand online social networks. One of the original papers to study the emerging social network phenomena focused on the Club Nexus website of Stanford University [Adamic 2003 ]. More recently traces from CyWorld, MySpace and Orkut have been profiled [Ahn 2007] , as have YouTube, Flickr, LiveJournal, and (again) Orkut [Mislove 2007 ]. Yet another study focused on profiling social network evolution on Flickr and Yahoo! 360 [Kumar 2006 ]. Finally, a recent measurement study analyzed the growth of Flickr social network using a three month crawl data [Mislove 2008 ]. These studies confirm that online social networks obey power-law scaling characteristics [Barabasi 1999 ] and exhibit high clustering coefficients, firmly establishing them as small-world networks [Amaral 2000] .. Recent studies analyzed the online communication patterns among the users in a large IM trace [Leskovec 2008] , and in an online social network [Chun 2008 ]. The IM study [Leskovec 2008 ] also reported a relatively higher value of average path length for the graph formed from user interactions. However, the IM interaction graph is more resilient to node removal than the interaction graphs in Facebook, as indicated by our assortativity values. Like our study, the CyWorld interaction study [Chun 2008] showed that CyWorld user interactions are bi-directional. User interaction behavior differs significantly from our study, however. CyWorld users with less than 200 friends interact only with a small subset of friends and users with more than 200 friends interact evenly. In addition, both activity and social graphs are similar in CyWorld and exhibit multi-scaling behavior.. This multi-scaling is unique to CyWorld; all other social networks analyzed so far, including Facebook, exhibit simple power-law connectivity scaling [Mislove 2007 , Ahn 2007 , Leskovec 2008 ].

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multiobjective genetic algorithms for multiscaling excited state direct dynamics in photochemistry This paper studies the effectiveness of multiobjective genetic and evolutionary algorithms in multiscaling excited state direct dynamics in photochemistry via rapid reparameterization of semiempirical methods. Using a very limited set of ab initio and experimental data, semiempirical parameters are reoptimized to provide globally accurate potential energy surfaces, thereby eliminating the need for fullfledged ab initio dynamics simulations, which are very expensive. Through reoptimization of the semiempirical methods, excited-state energetics are predicted accurately, while retaining accurate ground-state predictions. The results show that the multiobjective evolutionary algorithm consistently yields solutions that are significantly better-up to 230% lower error in the energy and 86.5% lower error in the energy-gradient-than those reported in the literature. Multiple high-quality parameter sets are obtained that are verified with quantum dynamical calculations, which show near-ideal behavior on critical and untested excited state geometries. The results demonstrate that the reparameterization strategy via evolutionary algorithms is a promising way to extend direct dynamics simulations of photochemistry to multi-picosecond time scales. Many phenomena in science and engineering are inherently multiscale and in the recent years there has been growing interest in developing effective modeling and simulation methods to explain or predict their behavior. In essence, there is a significant premium on cost-effective modeling techniques that can simulate physical, chemical or biological phenomena across multiple scales in both time and space, even at the price of losing information at intermediate scales.. One multiscaling approach is to apply modeling methods of a single scale and couple them by transferring key information from the finer scale to a coarser scale. An important and often daunting task in this multiscaling approach is the development of proper coupling methods and evolutionary algorithms can potentially play an important role [23, 24] .. One such area of multiscaling where evolutionary algorithms are very useful is in modeling excited-state dynamics in photochemistry. Photochemical reactions, as well as many spectroscopic measurements, involve electronic excited states of molecules and their concomitant structural changes. Such excited-state reactions are fundamental in many biological (for example, photosynthesis and vision) and technological (for example, solar cells and LED displays) settings. These reactions and the associated dynamics are energetically subtle and require highly accurate descriptions of the relevant interatomic forces. Thus, reliable predictions are costly even for small molecular reactions but rapidly become near impossible for reactions in complex environments, such as in solvents (for example, water), in solid cages (for example, zeolites), or with proteins.. The ab initio multiple spawning (AIMS) methods, which simultaneously solve both the electronic and nuclear Schrödinger equations [4, 5] , while very flexible and accurate, can be computationally expensive, especially for large molecules. Hence, having substantially faster semiempirical potentials that accurately reproduce higher-level quantum chemistry results would make it possible to address critical biological processes and technologically useful chemical reactions. However, the semiempirical methods [11, 12, 25] -which neglect many two-electron integrals of ab initio methods-while significantly less expensive than AIMS, often yield erroneous energetics and unphysical dynamics.. Therefore, in order to obtain globally accurate energetics, the parameter sets-which replace the non-neglected twoelectron integrals of ab initio methods-have to be reoptimized for different classes of molecules using a very limited set of ab initio and experimental data [19, 28] . The reparameterization strategy is a promising way to extend direct dynamics simulations of photochemistry to multi-picosecond time scales. However, the reoptimization problem is massively multimodal and involves multiple objectives such as minimizing the difference between calculated and predicted energies, gradients of energies, and stationary point geometries. Current methods, mostly based on a staged fixedweight single-objective optimization, fall quite short of yielding globally correct PESs and thus can produce unphysical dynamics. Evolutionary algorithms, on the other hand, are robust search methods that simultaneously optimize multiple objectives, and hence, are particularly suited for rapid reparameterization of semiempirical parameters. Therefore, the purpose of this paper is to use multiobjective evolutionary algorithms for rapid reparameterization of semiempirical methods to obtain globally-correct excited-state dynamics.. This paper is organized as follows. In the next section, we provide a brief introduction to reparameterization of semiempirical methods. We then describe the multiobjective evolutionary algorithm used for reparameterization in section 3 followed by results and discussion in section 4. We outline future work followed by summary and conclusions in sections 5 and 6 respectively. We have taken the first step towards multiscaling excited state direct dynamics of photochemistry. Our main goal is to obtain high-quality reoptimized semiempirical parameters that are transferable to other complex molecules containing similar properties as ethylene and benzene and our limited tests so far have shown that this is indeed the case. We are now in the process of systematically testing the transferability of the reoptimized parameter sets of ethylene and benzene to other molecules.. In this study we used an fixed evolutionary operators that do not adapt linkage, and the evolutionary algorithm might not be scalable in reparameterizing semiempirical methods for other complex molecules. One class of evolutionary algorithms that might be particularly effective-especially for complex molecules-might be the so called competent genetic algorithms [14, 20] .. The multiple reoptimized parameter sets of semiempirical methods all yield globally correct potential energy surfaces. Therefore, the interactions between the parameters contain important information about the molecules. Understanding these key interactions are central to developing a robust multiscaling method. We have used traditional clustering techniques, the model-building process of Bayesian optimization algorithm [20, 1] , and symbolic regression via genetic programming to understand the parameter interactions. In this study, we investigated the use of genetic and evolutionary algorithms in multiscaling simulations of excited state dynamics in photochemistry. Specifically, we used multiobjective genetic algorithms to rapidly reparameterize semiempirical methods to produce globally accurate potential energy surfaces based on very limited ab initio and/or experimental data. Such reparameterization not only eliminates the need for a full-fledged ab initio dynamics simulation, which is prohibitively expensive for large molecules, but also eliminates drawbacks of semiempirical methods that use standard parameter sets and can yield unphysical dynamics. The results show that the evolutionary approach provides significantly better results-with up to 230% lower error in the energy and 86.5% lower error in the energy gradient-than those reported in literature. Furthermore, it also provides a large number of parameter sets, all of which yield globally accurate PESs and physical dynamics.. In the spirit of semiempirical methods, it is reasonable to expect transferability to ethylene-and benzene-containing molecules, such as stilbene and phenylacetylene dendrimers. Our initial tests on transferability shows that this is indeed the case-that is, parameter sets optimized for ethylene are applicable to benzene, and vice versa. This is an ultimate goal of this work, since it would allow direct simulation of photoinduced cis-trans isomerization in molecules such as stilbene and azobenzene, as well as energy transfer in dendrimeric molecules. Furthermore, this opens up the possibility of accurate simulations of photochemistry in complex environments such as proteins and condensed phases. The energy difference between twisted and pyramidalized geometry. The points on the best non-dominated set-especially those with low error in energy and energy-gradient-are near ideal value (shown by dashed lines) for energy difference between D 2d S1 and pyramidalized S1 configuration of 0.9 eV. Furthermore, the unphysical points with error in energy less than 2.0 eV, the energy difference between the D 2d S1 and pyramidalized S1 configurations is greater than -0.034 eV, which is well within the error bars of the ab initio methods. The best non-dominated front after 100 generations for benzene compared to the published results [28] . NSGA-II results are for population size n = 800, and are averaged over 10 independent runs. The results obtained through GAs are significantly better-226% lower in error in energy, and 32.5% lower in error in energy gradient-than existing reparameterized sets. We demonstrate the effectiveness of using multiobjective genetic algorithm in rapid reparameterization of semiempirical methods for ethylene and benzene. We begin with estimating population-sizing and run-duration requirements and then compare the performance of the evolutionary approach in predicting globally accurate PESs-specifically on critical and untested excited states-with previously published results.. Since the fitness calculations for ethylene are reasonably fast-about 2 seconds per evaluation on a 1.7 GHz AMD Athlon XP workstation-we first verify the populationsizing and run-duration requirements using limited number of NSGA-II runs. In order to verify population-sizing requirements, we ran 5 independent runs of NSGA-II with a population size of 2000 for 200 generations and used the best non-dominated set out of those 5 runs as an approximation of the true Pareto-optimal front, which contains 61 distinct solutions. Using the population-size model for niching [18] , we compute that the population size required to maintain at least 1 copy of each of the Pareto-optimal points with a probability of 0.98 to be 750. To verify this estimate we ran 10 independent runs of NSGA-II with population sizes between 50-800 with a fixed number of function evaluations of 80,000 for each run. The performance of NSGA-II with different population sizes are shown in figure 2. As shown in figure 2 , while NSGA-II with population sizes below 800 are unable to converge to the approximate Pareto-optimal front, NSGA-II with a population size of 800 discovers almost all the Pareto-optimal points.. We now look at the convergence rate of NSGA-II and the run-duration requirements for reparameterization. Specifically, we considered 10 independent runs of NSGA-II with a population size of 800 and looked at the evolution of the best non-dominated front at different generations of the evolutionary process as shown in figure 3 . The results show that reasonably good quality solutions start appearing as early as 10 th generation and the solution quality improves at a steady pace till about 25 generations and gradually up to about 100 generations. We found that after about 100 generations the improvement in solution quality was minimal.. Based on population-sizing and run-duration requirements in the remainder of the results we used a population size of 800 and ran NSGA-II for 100 generations. Moreover, the number of decision variables (semiempirical parameters) remain the same with different molecules involving carbon and hydrogen and the population-sizing and run-duration estimates should hold for reparameterization for those molecules as well. However, we note that the evaluation time increases with the complexity of the molecule under consideration.. We begin with comparing the solution qualities provided by the best non-dominated front of NSGA-II over the current published results of Owens [19] for ethylene in figure 4 . As shown in the figure, the solutions obtained through the genetic algorithm is significantly superior, both in terms of error in energy and energy-gradient, than those previously reported. Specifically, we obtain solutions that are 226% lower error in the energy and 32.5% lower error in the energy gradient. Additionally, one of the best points reported in [19] actually yields an inaccurate potential energy surface. In contrast, all 45 distinct solutions in the best non-dominated set with error in energy lower than 1.2 eV yield globally accurate PESs. All the unphysical points obtained through evolutionary approach have an error in energy greater than 1.23 eV.. We now consider solutions obtained through the GA and that of Owens with error in energy less than 2 eV, and evaluate their results on energetic calculations for a set of ethylidene geometries for which they were not reoptimized. Before comparing the results of GA with those of Owens, we provide certain salient properties of cis-trans isomerization of ethylene. The ground state for ethylene is a planar structure as shown in figure 5 . When it is excited, the carbon-carbon bond twists 90. • and decreases in the energy gap from 7.8 eV to 2.5 eV. The twisted geometry, however is not an excited state minimum but a saddle point with respect to pyramidalization of one of the carbon atoms. However, as shown in figure 5 , the PM3 and AM1 parameter sets incorrectly indicate that the twisted geometry is excited state minimum.. Therefore, we consider the above two important energetics, the results of which are shown in figure 6:. • Energy differences between planar ethylene (ground state, S0 minimized D 2h ) and twisted geometry (S1 minimized D 2d ), ideal value for which is 2.28 eV as calculated by ab initio methods [3] . If the energy difference between the planar and twisted geometry is less than zero, than the excited state minimum would be the planar structure, which is erroneous. In other words, for good parameter sets, the energy difference between the planar and twisted geometry should be greater than zero, preferably around 2.28 eV.. • Energy differences between the twisted geometry (S1 minimized D 2d ) and pyramidalized structure, ideal value for which is 0.88 eV as calculated by ab initio methods [3] . As shown in figure 5 the standard semiempirical parameter sets do not capture this feature, and therefore, this energetics is one of the critical phenomena in determining the quality of the reoptimized parameter sets. If the energy difference between the twisted geometry and the pyramidalized structure is less than zero, then the excited state minimum would be the twisted geometry (as predicted by standard parameter sets) which is inconsistent with ab initio and experimental results. Therefore, for good parameter sets, the energy difference between the twisted and pyramidalized geometries must be greater than zero, preferably around 0.88 eV. The energy differences between planar and twisted geometry, and twisted geometry and pyramidalized structure, for both the best non-dominated set are shown in figure 6 along with the corresponding solutions. As shown in figure 6 , the best non-dominated solutions with error in energies lower than 0.8 eV yield near ideal energies for both excited-state transitions. Indeed, during the evolutionary process, we find about 1,247 distinct parameter sets other than the best nondominated ones that demonstrate near-ideal energetics. In essence, the genetic-algorithm reoptimized parameter sets correctly identify the lowest-energy excited state as the pyramidalized structure as opposed to standard semiempirical parameter sets and some of the previously reported reparameterized sets. In contrast, the energetics of the solutions reported in [19] deviate significantly from the ideal values. It should be noted that for the unphysical points in both solutions sets, the energy difference between the twisted and pyramidalized geometries was greater than -0.034 eV, which is well within the error bars of the ab initio methods.. To verify the effectiveness of NSGA-II, we tested reparameterization on benzene which is a more complex than ethylene. The results for benzene reoptimization are shown in figure 7 . Similar to the results obtained for ethylene, we observe that the GA provides significant improvement-46% lower error in the energy and 86.5% lower error in the energy gradient-over previously reported results [28] . Furthermore, 75 out of 82 distinct best non-dominated solutions with error in energy less than 8 eV yield physically accurate dynamics. Similar to ethylene, while the standard semiempirical parameter sets yield unphysical dynamics, the genetic-algorithm reoptimized parameter sets yield results consistent with experiments and ab initio computations. For example, the newly optimized parameter sets predict an S2 lifetime of 100 fs, in agreement with experiment [21] . We have also conducted dynamics simulations using a limited set of most distinct and best non-dominated parameter sets for both ethylene and benzene, results of which are beyond the scope of this paper. In essence, the dynamics simulations indicate that for each of the reoptimized parameter sets, the results are in accord with ab initio and experimental results. The results demonstrate the effectiveness of evolutionary search in providing multiple parameter sets for semiempirical methods that yield globally correct PESs even on untested and potentially critical reaction paths.

B I I O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O


Searching for Nonlinear Feedback Shift Registers with Parallel Computing Abstract. Nonlinear feedback shift registers (NLFSRs) are used to construct pseudorandom generators for stream ciphers. Their theory is not so complete as that of linear feedback shift registers (LFSRs). In general, it is not known how to construct all NLFSRs with maximum period. The direct method is to search for such registers with suitable properties. Advanced technology of parallel computing has been applied both in software and hardware to search for maximum period NLFSRs having a fairly simple algebraic normal form. Feedback shift register (FSR) sequences have been widely used in many areas of communication theory, as key stream generators in stream ciphers cryptosystems, pseudorandom number generators in many cryptographic primitives, in the design of correlators for spread spectrum communication systems, global positioning systems or radar systems, and as testing vectors in hardware design. Golomb's book [6] is a pioneering one that discusses this type of sequences. A modern treatment of the subject is contained in the book of Golomb and Gong [7] .. The theory of linear feedback shift registers (LFSRs) is quite well understood. In particular, it is known how to construct LFSRs with maximum period; they correspond to primitive polynomials over the binary field F 2 . The order n of FSR is the number of its cells and an FSR can generate a binary sequence of period up to 2 n . The main drawback of primitive LFSRs is that their linear complexity is equal to their order. In recent years, nonlinear feedback shift registers (NLFSRs) have received much attention in designing numerous cryptographic algorithms such as stream ciphers and lightweight block ciphers to provide security in communication systems. In most cases, NLFSRs have much greater linear complexity than LFSRs of the same period. However, there are no general methods of designing maximum period NLFSRs. The construction of a special class of NLFSRs with maximum period has been given by Mykkeltveit et al. [11] . Recently, maximum period NLFSRs of order up to n = 64 have been constructed in the paper of Mandal and Gong [9] , but these NLFSRs have very complicated algebraic normal form (ANF ). Dubrova [3] has given an example of a Galois shift register of order n = 100 generating a sequence with maximum period but this sequence does not have the de Bruijn property; i.e., some patterns of n-bits appear more than once in the sequence.. In this article the approach from the papers of Gammel, Goettfert, and Kniffler [5] and Chan, Games, and Rushanan [2] is followed. NLFSRs having a simple algebraic normal form and maximum period are directly found by exhaustive searching. In particular, the conjecture posed in [2] on the existence of maximum period NLFSRs whose feedback functions have linear terms and one quadratic term has been experimentally investigated. Like the notion of linear m-sequence, these NLFSRs generate quadratic m-sequences. The conjecture has been verified up to order n = 29. Quadratic m-sequences of this type up to order n = 21 have been classified according to the weight (number of terms) of primitive polynomials involved. These experimental investigations support the Chan, Games and Rushanan conjecture of the existence of a special type of quadratic m-sequences for each order n.. Our experiments have applied parallel computing. NLFSRs generating quadratic m-sequences have been found by using three servers with 54 CPU cores. The investigations from the previous paper [12] have also been continued using the Field Programmable Gate Arrays to find maximum period NLFSRs. Nonlinear feedback shift registers have been applied in [13] to construct modified alternating step generators. In the paper, we have searched for NLFSRs generating modified de Bruijn sequences. Methods of parallel computing have been applied both in software and hardware to speed the search up. The software has been used to look for special form quadratic m-sequences. An enumeration of all quadratic m-sequences up to order n = 21 generated by feedback functions with the term x i + x i x j and linear terms defined by primitive polynomials has been provided and classified by the weights of these polynomials. The conjecture of Chan, Games and Rushanan [2] has been verified numerically up to order n = 29 for NLFSRs whose feedback functions have the special algebraic normal form.. Hardware search of NLFSRs is a continuation of the previous work [12] . The feedback functions of the registers of orders n = 29, 30 and 31 we found are expressed in terms of negations of some arguments; this gives simpler formulae than ANFs and it is suitable for implementation.. • n = 29, x 0 + x 1 + x 3 + x 4 + x 13 + x 16 + x 18 + x 19 + x 20 + x 22 + x 23 + x 27 + x 6 x 10 x 19 x 20 x 21 x 24 x 26 x 27 + x 6 x 10 x 19 x 20 x 21 x 24 x 26 x 27 • n = 30, x 0 + x 4 + x 7 + x 8 + x 9 + x 10 + x 11 + x 12 + x 13 + x 14 + x 15 + x 22 + x 27 + x 28 + x 1 x 3 x 7 x 9 x 11 x 13 x 15 x 19 x 20 x 24 x 25 x 27 + x 1 x 3 x 7 x 9 x 11 x 13 x 15 x 19 x 20 x 24 x 25 x 27 • n = 31, x 0 + x 2 + x 6 + x 7 + x 8 + x 9 + x 10 + x 14 + x 15 + x 16 + x 20 + x 21 + x 26 + x 29 + x 4 x 5 x 7 x 9 x 12 x 19 x 21 x 22 x 24 x 25 x 26 x 29 + x 4 x 5 x 7 x 9 x 12 x 19 x 21 x 22 x 24 x 25 x 26 x 29 where x i = x i + 1 Table 1 . The number of the special form quadratic m-sequences of orders 4 ≤ n ≤ 12 n = 4 n = 5 n = 6 n = 7 n = 8 n = 9 n = 10 n = 11 n = 12

O O B I I I O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Incremental Satisfiability and Implication for UTVPI Constraints Abstract. Unit two-variable-per-inequality (UTVPI) constraints form one of the largest class of integer constraints which are polynomial time solvable (unless P=NP). There is considerable interest in their use for constraint solving, abstract interpretation, spatial databases, and theorem proving. In this paper we develop a new incremental algorithm for UTVPI constraint satisfaction and implication checking that requires O(m + n log n + p) time and O(n + m + p) space to incrementally check satisfiability of m UTVPI constraints on n variables and check implication of p UTVPI constraints. The unit two-variable-per-inequality (UTVPI) constraints form one of the largest class of integer constraints which are polynomial time solvable (unless P=NP). There is considerable interest in their use for constraint solving [7, 6] , abstract interpretation [9] , spatial databases [11] and theorem proving [8] . In this paper we develop new incremental algorithms for UTVPI constraint satisfaction and implication.. A UTVPI constraint has the form ax+by ≤ d where x, y are integer variables, d ∈ Z and a, b ∈ {−1, 0, 1}. For example x+y ≤ 2, x−y ≤ −1, 0 ≤ −1 and x ≤ 2 are UTVPI constraints. UTVPI constraint solving is based on transitive closure: A constraint ax−y ≤ d 1 and y +bz ≤ d 2 implies the constraint ax+bz ≤ d 1 +d 2 . We can determine all the UTVPI consequences of a set of UTVPI constraints by transitive closure, but we need to tighten some constraints. The transitive closure procedure can generate constraints of the form x + x ≤ d and −x − x ≤ d, which need to be tightened to x ≤ d 2 and −x ≤ d 2 respectively. Jaffar et al. [7] and Harvey et al. [6] present incremental consistency checking algorithms for adding a UTVPI constraint c to a set φ of UTVPI constraints. They are based on maintaining the transitive and tight closure of the set of UTVPI constraints φ involving n variables. Both algorithms require O(n 2 ) time and O(n 2 ) space for an incremental satisfaction check. Both algorithms can also be used to incrementally check implication of UTVPI constraints by φ ∪ {c}. These algorithms require O(n 2 + p) time and O(n 2 + p) space for an incremental implication checking, where p is the number of constraints we need to check for implication. In order to (non-incrementally) check satisfiability of m UTVPI constraints on n variables these approaches require O(n 2 m) time, and to check implication they require O(n 2 m + p) time. An improvement on the complexity of (non-incremental) satisfiability for UTVPI constraints was devised by Lahiri and Musuvathi [8] . They define a non-incremental satisfiability algorithm requiring O(nm) time and O(n + m) space. The key behind their approach is to map UTVPI constraints to difference constraints (also called separation theory constraints) of the form x − y ≤ d, where x and y are integer variables and d ∈ Z.. The difference constraints are a well studied class of constraints because of their connection to shortest path problems. We can consider the constraint x − y ≤ d as a directed edge x → y with weight d. Satisfiability of difference constraints corresponds to the problem of negative weight cycle detection, and implication of difference constraints corresponds to finding shortest paths (see e.g. [3] for details).. The mapping of UTVPI to difference constraints by Lahiri and Musuvathi [8] is a relaxation of the problem. The relaxed problem is solved by a negative (weight) cycle detection algorithm but it guarantees only the satisfiability in Q for the UTVPI problem. In order to check satisfiability in Z they need to construct an auxiliary graph and check for certain paths in this graph.. In this paper we first extend Lahiri and Musuvathi's algorithm [8] to check satisfaction incrementally in O(n log n + m) and O(n + m) space. Then we show how to build an incremental satisfiability and implication algorithm using the relaxation of Lahiri and Musuvathi and incremental approaches to implication for difference constraints of Cotton and Maler [3] , which can incrementally check implication in O(n log n + m + p) time and O(n + m + p) space. We have presented new incremental algorithms for UTVPI constraint satisfaction and implication checking which improve upon the previous asymptotic complexity, and perform better in practice for sparse constraint systems. We can easily adapt the algorithms herein to provide non-incremental implication checking in O(n 2 log n + nm + p) time and O(n + m + p) space, and generate all implied constraints in O(n 2 log n + nm) time and O(n + m + p) space, where p is the number of implied constraints generated.

O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


SASI:

a

generic

texture

descriptor

for

image

retrieval

In

this

paper,

a

generic

texture

descriptor,

namely,

Statistical

Analysis

of

Structural

Information

(SASI)

is

introduced

as

a

representation

of

texture.

SASI

is

based

on

statistics

of

clique

autocorrelation

coefficients,

calculated

over

structuring

windows.

SASI

defines

a

set

of

clique

windows

to

extract

and

measure

various

structural

properties

of

texture

by

using

a

spatial

multi-resolution

method.

Experimental

results,

performed

on

various

image

databases,

indicate

that

SASI

is

more

successful

then

the

Gabor

Filter

descriptors

in

capturing

small

granularities

and

discontinuities

such

as

sharp

corners

and

abrupt

changes.

Due

to

the

flexibility

in

designing

the

clique

windows,

SASI

reaches

higher

average

retrieval

rates

compared

to

Gabor

Filter

descriptors.

However,

the

price

of

this

performance

is

increased

computational

complexity.

In

recent

years,

textural

information

has

been

widely

used

as

a

visual

primitive

in

many

image

processing

applications

The

potential

areas

include

industrial

and

biomedical

surface

inspection,

ground

classification

and

segmentation

of

satellite

or

aerial

imagery,

document

analysis,

scene

analysis,

texture

synthesis

for

computer

graphics

and

animation,

biometric

person

authentication,

content-based

image

retrieval

and

modelbased

image

coding

Although

the

above

application

areas

necessitate

the

utilization

of

texture

analysis,

only

a

limited

number

of

successful

interpretations

of

texture

exist

so

far.

The

success

of

a

texture

descriptor

heavily

depends

on

the

data

type

and

the

application

area.

A

major

problem

in

representing

texture

is

that

the

textures

in

the

real

world

are

often

quite

His

address:

Sermaye

Piyasasi

Kurulu,

Doc.Dr.

Bahriye

Ucok

Besevler/Ankara/TURKEY

E-mail

address:

carkaci@ceng.metu.edu.tr

in

the

design

of

the

Gabor

Filter

Banks.

Otherwise,

the

implementation

of

a

complete

Gabor

expansion

would

entail

a

generally

impractical

number

of

filters.

Also,

in

a

digital

world,

it

is

not

always

possible

to

cope

with

all

sizes

of

analog

Gabor

Filters,

which

may

cause

problems,

especially,

with

the

textures

that

consist

of

small

texels

or

sharp

corners.

Another

limitation

of

the

Gabor

descriptor

is

the

restriction

of

the

filtering

area,

which

must

fit

in

a

rectangle,

unless

some

pre-processing

is

done..

In

this

study,

we

explore

a

generic

texture

descriptor,

which

overcomes

the

above

mentioned

difficulties

and

works

well

on

a

wide

range

of

textures.

The

SASI

descriptor,

proposed

in

this

paper,

is

based

on

second

order

statistics

of

clique

autocorrelation

coefficients,

which

are

the

autocorrelation

coefficients

over

a

set

of

moving

windows.

The

clique

windows

of

various

size

and

shape,

which

are

defined

by

a

neighborhood

system,

are

used

as

a

tool

for

describing

the

characteristics

of

textures

in

different

granularity.

The

order

of

the

neighborhood

system

controls

the

structure

of

the

clique

windows.

Because

of

the

flexibility

in

the

definition

of

clique

windows,

SASI

can

cope

with

a

broad

class

of

textures,

which

may

consist

of

discontinuities

or

small

primitives..

SASI

is

tested

on

Brodatz

Album

CUReT

PhoTex

and

VisTex

image

databases.

The

experiments

are

also

performed

on

a

combined

database

obtained

by

joining

all

of

the

images

in

these

databases.

It

is

observed

that

SASI

improves

the

retrieval

rates

compared

to

Gabor

Filters.

However

the

price

of

this

improvement

is

the

increased

computational

complexity..

The

paper

is

organized

as

follows.

In

Section

we

introduce

SASI

descriptor

behind

a

series

of

definition..

Experimental

results

are

given

in

Section

Section

concludes

with

discussions

on

the

strengths

and

weaknesses

of

SASI

compared

to

Gabor

Filter

descriptors.

Two

sets

of

experiments

are

done

to

show

the

power

of

SASI.

First,

SASI

descriptor

is

analyzed

in

detail

and

compared

to

Gabor

Filter

descriptor.

Latter,

SASI

and

Gabor

Filter

descriptor

are

tested

on

the

image

retrieval

problem

by

using

four

different

image

databases,

namely

Brodatz

Album,

CUReT

PhoTex

and

VisTex

The

experiments

are

also

performed

on

a

database

generated

by

joining

all

the

images

of

these

four

databases..

Brodatz

Album

contains

pictures

with

size

and

gray

values

after

digitizing,

showing

a

variety

of

textures,

collected

for

artistic

purposes

It

is

a

de

facto

standard

set

of

images

for

texture

retrieval

problem.

Due

to

its

popularity

and

comparable

studies

exist

in

the

literature

a

more

comparative

analysis

is

provided

on

the

Brodatz

Album

in

the

following

sections.

Further

details

can

be

found

at

O O O O B O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B B O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O