Effective typestate verification in the presence of aliasing This article addresses the challenge of sound typestate verification, with acceptable precision, for real-world Java programs. We present a novel framework for verification of typestate properties, including several new techniques to precisely treat aliases without undue performance costs. In particular, we present a flow-sensitive, context-sensitive, integrated verifier that utilizes a parametric abstract domain combining typestate and aliasing information. To scale to real programs without compromising precision, we present a staged verification system in which faster verifiers run as early stages which reduce the workload for later, more precise, stages.We have evaluated our framework on a number of real Java programs, checking correct API usage for various Java standard libraries. The results show that our approach scales to hundreds of thousands of lines of code, and verifies correctness for 93% of the potential points of failure. In the software development life cyle, early defect identification increases productivity by reducing development costs and improving software quality and reliability. Development organizations employ a variety of tools to identify defects early, ranging from testing and dynamic analysis to program verification techniques. One class of partial program verification techniques addresses checking if programs satisfy specified safety properties [Dwyer and Clarke 1994; Naumovich et al. 1999; Corbett et al. 2000; DeLine and Fähndrich 2001; Ball and Rajamani 2001; Foster et al. 2002; Flanagan et al. 2002; Ashcraft and Engler 2002; Ramalingam et al. 2002; Das et al. 2002; Field et al. 2003] ). Defects related to these safety properties can lead to run-time failures, and may be difficult to cover with normal testing practices.. Typestate [Strom and Yemini 1986] is an elegant framework for specifying a class of temporal safety properties. In the typestate model, a finite-state automaton encodes legal program state changes, and transitions to automaton error states indicate violations of safety properties. Typestates can encode correct usage rules for many common libraries and application programming interfaces (APIs) [Whaley et al. 2002; Alur et al. 2005] . For example, typestate can express the property that a Java program should not read data from java.net.Socket until the socket is connected.. This article addresses the challenge of typestate verification, with acceptable precision, for real-world Java programs. We focus on sound verification 1 ; if the verifier reports no problem, then the program is guaranteed to satisfy the desired properties. However, if the verifier reports potential problems, they may or may not indicate actual program errors. Imprecise analysis can lead a verifier to produce "false positives": reported problems that do not indicate an actual error. Users will quickly reject a verifier that produces too many false positives.. While the most sophisticated and precise analyses can reduce false positives, such analyses typically do not scale to real programs. Real programs typically rely on large and complex supporting libraries, which the analyzer must process in order to reason about program behavior. This paper presents several new typestate verification techniques, ranging from the simple but imprecise, to the fairly precise but somewhat expensive. We also present a staged typestate verification approach, which exploits verifiers with varying cost/precision trade-offs. Early stages employ the efficient but imprecise analyses; subsequent stages employ progressively more expensive and precise techniques. Each progressively more precise stage focuses on verifying only "parts" of the program that previous stages failed to verify.. The key technical challenge facing typestate verification for Java concerns pointer aliasing. Since all structured data in Java is heap-allocated, almost all interesting operations involve pointer dereferencing. Further, Java libraries encourage layers of encapsulation around data, which leads to multiple levels of pointer dereferencing. In order to prove that a program manipulates an object correctly, the verifier must cut through the tangle of alias relationships by which the program manipulates the object of interest.. Researchers have developed a variety of efficient flow-insensitive mayalias (pointer) analysis techniques [Das 2000; Heintze and Tardieu 2001; Steensgaard 1996 ] that scale to fairly large programs. These analyses produce a statically bounded (abstract) representation of the program's runtime heap and indicate which abstract objects each pointer-valued expression in the program may denote. Unfortunately, these scalable analyses have a serious disadvantage when used for verification. They require the verifiers to model any operation performed through a pointer dereference conservatively as an operation that may or may not be performed on the possible target abstract objects identified by the pointer analysis-this is popularly known as a "weak update" as opposed to a "strong update" [Chase et al. 1990] .. To support strong updates and more precise alias analysis, we present a framework to check typestate properties by solving a flow-sensitive, contextsensitive dataflow problem on a combined domain of typestate and pointer information. As is well known [Cousot and Cousot 1979] , a combined domain allows a more precise solution than could be obtained by solving each domain separately. Furthermore, the combined domain allows the framework to concentrate computational effort on alias analysis only where it matters to the typestate property. This concentration allows more precise alias analysis than would be practical if applied to the whole program. We have presented a practical approach to typestate checking, exploiting staged analysis to realize precise alias analysis at reasonable cost. Results show that the techniques work well across a suite of programs, and indicate that efficient, precise typestate verification may prove valuable in the software development lifecycle. Overall, the results show that our combination of techniques is relatively successful and efficient at verifying these typestate properties. The various techniques complement each other, contributing to the effectiveness of the staged verifier.. Since our goal in this paper is the successful verification of typestate properties, we have deliberately chosen a set of mature benchmarks. For our experiments, we assume that typestate violations in these benchmarks are all false alarms. We have examined, by hand, many of the warnings which our most precise verifier does not eliminate.. The specJVM98 code's use of PrintStream accounts for 241 of the 365 warnings reported. These are all false positives, stemming from a few lines of code in the specJVM98 harness. This program stores a PrintStream object in a static field Context.out, and uses the object ubiquitously throughout the various benchmarks. The particular idiom by which the program caches the PrintStream object in a static field defeats our focus heuristics, leading to a loss of precision.. Of the remaining 124 warnings, 53 arise from the Vector and Stack properties. Most of these warnings appear to represent a failure of the typestate property to capture all legal behavior, as opposed to solver limitations. For example, our typestate property for Vector does not account for the return value from Vector.size(). Many times, application code accesses a Vector via statements guarded by a test that size > 0. This pattern accounts for many of the false positives for the Stack and Vector rules. For proper treatment, these APIs require at least range-check analysis, as commonly applied to array-bounds checking [Gupta 1993 ].. The remaining warnings appear to arise from a combination of analysis approximations and typestate property limitations.. We expect that in the near future we can improve precision by a) accesspath tracking for objects that are not typestate objects, but are likely to point to them, and b) increasing the scope of focus by exploiting inexpensive local alias reasoning. We suspect that substantial improvements in alias precision are within reach, without undue performance compromise.. In many cases, programmers deduce from application logic that a particular iterator must have a next element, or a particular collection must not be empty. The typestate property for a single object does not allow for application logic which ensures, via some back door, that an object occupies a particular typestate. Designing efficient, effective analysis for more general specifications remains a difficult problem. The rightmost cluster of bars shows the total number of warnings across all runs. Overall, The FI verifier verifies correctness for 30% of PPFs. The LocalFocus verifier verifies correctness for 64% of PPFs. The Base verifier verifies correctness for 68% of PPFs. The Unique verifier verifies correctness for 72% of PPFs. The APMust verifier verifies correctness for 85% of PPFs. The APFocus verifier verifies correctness for 93% of PPFs. Table IV shows detailed results for verification warnings produced by the most precise (APFocus) solver. By construction, the Staged verifier has the same precision as APFocus. Section 6.7 discusses the sources of many false positives. Table IV 6.6 Performance Figure 7 reports the running times of the various verifiers across the benchmarks. The results show the expected relative costs of the various verifiers.. 6.6.1 Impact of Staging. The Staged verifier improves performance compared to the APFocus verifier on 9 of the 10 codes where typestate checking takes more than 30 seconds. On these 10 codes, staging improves performance by up to 85% (tvla), with a median of 34%. Staging hurts performance by 40% on l2j; on this code, many PPFs survive early verification stages, and the cost/precision tradeoffs of the various solvers do not pay off. 6.6.2 Impact of Sparsification. We evaluated the sparsification of Section 6.2 across all runs of the staged verifier. With sparsification, 70% of supergraphs have fewer than 3500 nodes, 95% have fewer than 25,000 nodes, and 100% have fewer than 40,000 nodes. The corresponding numbers without sparsification are drastically higher: roughly 80% of unpruned supergraphs have more than 125,000 nodes, and 20% have over 290,000 nodes. Overall, sparsification reduces median supergraph size by roughly a factor of 50. We would expect a corresponding reduction in space and running time, if we could run the unpruned verifiers without running out of memory.. 6.6.3 Impact of Initial Pointer Analysis. The precision of the preceding flow-insensitive pointer analysis significantly impacts performance and precision. A more accurate pointer analysis allows better sparsification, more effective live analysis and improved disambiguation overall. We ran many of the analyses with a context-insensitive Andersen-style pointer analysis, without the custom context-sensitivity policies described earlier. Many benchmarks timed out on several rules; we conclude that adequate precision in the preceding pointer analysis is vital.. Our context-sensitivity policy employs object-sensitivity for types from the standard libraries typically relevant to these typestate properties (namely collections and I/O streams). Some benchmarks defeat this object-sensitivity policy by using application-level collections or streams. For example, TVLA uses a library of application-level collections, and specJVM98 uses a reporting library of custom I/O streams. To handle these cases more effectively, we need to infer a pointer-analysis context-sensitivity policy for application classes that match typestate properties. Iterative refinement techniques [Plevyak and Chien 1994; Guyer and Lin 2003 ] may apply to this problem. Many existing verification frameworks [Das et al. 2002; Ball et al. 2001; Corbett et al. 2000 ] use a two-phased approach, performing points-to analysis as a preceding phase, followed by typestate checking. This approach only supports weak updates as discussed in Section 3.3.. The current version of ESP [Dor et al. 2004 ] uses an integrated approach, recording must and may alias information in a flow-sensitive manner. They observe that the may set becomes polluted and expensive to maintain, and even hint toward maintaining a must-not set as a possible future solution. In contrast, our approach adds must-not and also introduces the notions of uniqueness and focus, and uses staging to achieve increased scalability and precision. DeLine and Fähndrich [2004] present a type system for typestate properties for objects. Their system guarantees that a program that typechecks has no typestate violations, and provides a modular, sound checker for object-oriented programs. To handle aliasing, they employ the adoption and focus operations to a linear type system, as described in Fähndrich and DeLine [2002] . With these operations, the type checker can assume must-alias properties for a limited program scope, and thus apply strong updates allowing typestate transitions. Our approach can prove correctness of a more general class of programs, since a context-sensitive analysis can accept programs for which an expression cannot be assigned a unique type at a given program point. Furthermore, our focus operation generates facts that can flow across arbitrary program scopes, in contrast to the limited program scope handled by Fähndrich and DeLine [2002] . On the other hand, our approach is non-modular and thus more expensive. Degen et al. [2007] present a type and effect system for a language called Java(X). Interestingly, this type system combines typestate type annotations with a notion of droppability, which force the programmer to retain pointers to objects in particular typestates, which could be used to protect against certain types of resource leaks. Aiken et al. [2003] point out the importance of strong updates to avoid imprecision in the context of typestate verification (and other analyses). They present an inference algorithm for inferring restricted and confined pointers, which they use to enable strong updates. We believe that the focusing technique we exploit, inspired by Sagiv et al. [2002] , can sometimes achieve a similar effect without explicitly inferring restricted and confined pointers, and sometimes enable strong updates even when the pointers are not restricted/confined. Further, the uniqueness technique we use provides a somewhat orthogonal, cheap, technique for enabling strong updates. Field et al. [2003] present algorithms based on abstractions that integrate alias and typestate information, but restricted to shallow programs, with only single-level pointers to typestate objects.. The parametric shape analysis presented in Sagiv et al. [2002] has served as the basis for very precise verification algorithms, where the verification is integrated with heap analysis [Yahav and Ramalingam 2004] . These algorithms, however, do not scale well. We plan to extend our staged verifier by adding such precise verifiers as a last stage.. Counter-example guided refinement Henzinger et al. 2002] based approaches have had impressive results in certain domains. But they have so far been less successful in dealing with complex heap manipulation, partly because these approaches attempt to automatically derive appropriate heap analyses. Our staged verifier has a "refinement" flavor, but restricted to a fixed set of manually crafted verifiers.. Aliasing of our combined domain resembles previous approaches to flowsensitive, context-sensitive access-path-based pointer analysis [Landi and Ryder 1992; Choi et al. 1993 ]. Emami, Ghiya, and Hendren [Emami et al. 1994] presented a domain that combined may and must points-to information. Our IFDS-based solvers memoize function summaries, similar to Wilson and Lam's partial transfer functions [Wilson and Lam 1995] . Our domain differs from these previous works since a) it tracks must and must-not paths, but not may, and b) Java's strong typing avoids complications arising from pointers to stack locations.. Iterative refinement techniques [Plevyak and Chien 1994; Guyer and Lin 2003 ] perform pointer analysis in multiple passes, with a client-independent first pass, followed by subsequent passes using context-sensitivity policies driven by client feedback. In future work we plan to integrate these techniques into our framework, where each typestate solver provides feedback for the next stage's pointer analysis. Hackett and Rugina [2005] exploit a staged analysis to obtain a relatively scalable interprocedural shape analysis. This approach uses a scalable imprecise pointer-analysis to decompose the heap into a collection of independent locations. The abstraction used in Hackett and Rugina [2005] is similar in spirit to our APMust abstraction, but the properties they target are more ambitious, namely proving the absence of memory errors.. Recently, Shoham et al. [2007] presented a static analysis for client-side mining of temporal API specifications. Their approach is based on the heap abstractions described in this paper, but additionally use quotient-based abstractions for creating bounded descriptions of (unbounded) sequences of events. Yorsh et al. [2007] present a modular typestate analysis based on the abstractions introduced in this paper. Their approach is based on symbolic summarization and requires a decision procedure. We believe that a combination of our approaches has to be used in order to achieve a truly scalable typestate verification in the presence of aliasing. The preliminary flow-insensitive pointer analysis provides a mostly contextinsensitive field-sensitive Andersen's analysis [Andersen 1994 ], enhanced with a selective object sensitivity policy [Milanova et al. 2005] . The pointer analysis relies on an SSA register-transfer language representation of each method, which gives a measure of flow-sensitivity for points-to sets of local variables [Hasti and Horwitz 1998 ]. The pointer analysis names each contextsensitive allocation site as an instance key, and builds the call graph on-the-fly.. The pointer analysis uses a mostly context-insensitive policy; most methods are represented by exactly one context. However, some methods are cloned to disambiguate pointer flow in specific contexts. Most significantly, Java collection classes and I/O stream containers are treated with unlimited depth (up to recursion) object-sensitivity [Milanova et al. 2005] . That is, for each allocation site of a Java collection or I/O stream, the analysis creates a context (clone) for each method of the class, representing said method when invoked on an object from that particular allocation site. Furthermore, the analysis recursively applies object-sensitive cloning to all objects allocated in such methods. As a result, the contents of Java collections from different allocation sites are fully disambiguated, eliminating a major source of pointer analysis pollution.. Additionally, the pointer analysis adds one-level of call-string context to calls to a few library factory methods, including arraycopy, and clone statements. These methods tend to badly pollute pointer flow precision if handled without context-sensitivity.. The analysis deals with reflection by tracking objects to casts, as in Fink et al. [2004] and Livshits et al. [2005] . When an object is created by a reflective call (e.g., Class.newInstance or Constructor.newInstance), the analysis assumes (unsoundly) that the object will be cast to a declared type before being accessed. The analysis tracks these flows, and infers the type of object created by newInstance based on the declared type of relevant casts. While technically unsound, we believe that this approximation is accurate for the vast majority of reflective factory methods in Java programs. Our current implementation does not resolve calls via Method.invoke nor field accesses through java.lang.reflect.Field. In general, precise but conservative treatment of reflection remains a difficult challenge, beyond the scope of this work.. The system uses a substantial library of models of native code behavior for the standard libraries. We have built up these native method models over years of development of the analysis infrastructure in various products. We have no guarantee that our native method models are complete or adequate. A systematic way to model native methods or deal soundly with unmodelled methods remains a challenge, beyond the scope of this work.. For these experiments, we configure the analysis to ignore some system libraries such as java.awt and javax.swing. In our current implementation, if these libraries are included in the analysis scope, the treatment of reflection causes the computed call graph to blow up immensely, beyond our ability to scale the typestate verification. We believe that these libraries generally do not have side effects that affect the typestate properties considered here, but clearly this omission is a potential source of unsoundness. Of course, when dealing with the full Java language, many other sources of unsoundness also apply, including concurrency, the potential for dynamic class loading of unanticipated code, and unmodelled native methods. Based on all these factors, we believe that excluding the GUI libraries is a reasonable choice for both a realistic tool and for this empirical evaluation. Fully sound precise verification of any property for the full Java language remains a difficult and unsolved problem.. The flow-sensitive combined typestate and alias analysis builds on a general Reps-Horwitz-Sagiv (RHS) IFDS tabulation solver implementation [Reps et al. 1995] . We have enhanced the standard IFDS solver in straightforward ways to handle Java's exceptional control-flow and polymorphic dispatch without undue precision loss. The experiments evaluate the following verification algorithms:. FI: flow-insensitive analysis (Section 2.2.1) LocalFocus: the intraprocedural analysis (Section 2.2.2) Base: the base analysis (Section 3.3) Unique: the analysis using the unique reasoning (Section 4) APMust: the integrated analysis without focus (Section 5) APFocus: the integrated analysis with focus (Section 5.) Staged: a staged analysis consisting of three stages: LocalFocus, Unique, and APFocus.. Note that each verifier performs the FI analysis as a first step, since it is extremely fast and can prune the workload based on the "verification scope" passed from the previous stage. The experiments use an access-path depth limit of 2, and unlimited access-path set width. All experiments ran on an IBM Intellistation Z pro with two 3.06 GHz Intel Xeon CPUs and 3.62 GB of RAM, running Windows XP. The analysis implementation, consisting of roughly 200,000 lines of Java code, ran on the IBM J2RE 1.4.2 for Windows, with a max heap of 800MB. Figure 6 shows the percentage of warnings, as a percentage of total number of statements that the callgraph indicates might transition to err (points of potential failure (PPF)). The number shown above each bar in the figure is the total number of PPFs.

O B B O O B O B O O O O B O B B B O O O O O O O O O O O B O B O B O O O O O O O O O O O B O O O O O O O O O O O O O B O O O B O B O O B O O O O O O O O O O B O O O O O O O O O O O O B O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O B O B B O O O O O O O O O O B B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O B B O O O O O O O O O O B O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O B O O O O O O O O O B O B B O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O B O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O B O O O O O B B I O O O O O O O O O B O O O O O O O B O O O O O O B B I O O O O O O O O O O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O B O O O O O O O O B O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O B O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O B O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O B B O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O


Investigating the role of body shape on the perception of emotion In order to analyze the emotional content of motions portrayed by different characters, we created real and virtual replicas of an actor exhibiting six basic emotions: sadness, happiness, surprise, fear, anger, and disgust. In addition to the video of the real actor, his actions were applied to five virtual body shapes: a low-and high-resolution virtual counterpart, a cartoon-like character, a wooden mannequin, and a zombie-like character (Figures 1 and 2) . In a point light condition, we also tested whether the absence of a body affected the perceived emotion of the movements. Participants were asked to rate the actions based on a list of 41 more complex emotions. We found that the perception of emotional actions is highly robust and to the most part independent of the character's body, so long as form is present. When motion alone is present, emotions were generally perceived as less intense than in the cases where form was present. Virtual characters in animated movies and games can be very expressive and have the ability to convey complex emotions. However, it has been suggested that the more anthropomorphic a humanoid robot or virtual character becomes, the more eerie it is perceived to be [Mori 1970] . In this article, we focus on one potential factor that could affect this "uncanny valley" theory, by examining how emotional actions on real and virtual characters are perceived. We ask the following question: Are real and virtual characters moving with the same biological motion perceived to be exhibiting the same emotions?. Evidence from neuroscience studies suggest that different neural networks are activated when viewing real or virtual stimuli [Perani et al. 2001] . Therefore, we chose as our stimuli both real and virtual versions of humanoids acting out six basic emotions: sadness, happiness, surprise, fear, anger, and disgust. Our "real" stimulus is a video of an actor whose motion was captured while acting out the six emotions. We created a highly realistic virtual counterpart along with a lower resolution version (by reducing the number of polygons in the model), which were both animated using the original captured motion, as depicted in the video. In Experiment 1, we tested how emotions are perceived on these real and virtual humans. Furthermore, we explored how emotions were perceived when the same actions were applied to anthropomorphic but nonhuman characters. A cartoon-like figure was chosen in order to determine if his friendly appearance would lead him to be perceived as exhibiting more positive emotions, whereas we felt that a zombie-like character would be perceived in a more negative light. A wooden mannequin model was chosen as a completely neutral character that we felt may be perceived to be exhibiting less intense emotions than the others. In Experiment 2, we tested if emotion perception was the same when motion alone was presented (i.e., for a point-light figure) .. In all cases, we wished to examine the effects of body representation and motion and did not want to confound this with expressive facial or hand movements. Hence, the faces and hands were blurred in all videos where they were present.. In Experiment 1, the six emotions depicted on the human and nonhuman figures were shown to groups of participants who were asked to rate them according to a list of complex emotions. We found that body representation had little or no effect on people's perception of the emotional content of the scene, which leads us to conclude that relative to real footage, captured body motion is as effective at depicting emotional body language and people's perception of emotional motion is very robust and unaffected by a character's physical appearance. In Experiment 2, a further group of participants rated six movies depicting point-light figures. We found that emotions were perceived as less intense when motion alone was present. Descriptions of the static bodies were first assessed, but minimal insight was gained. The three male characters were simply described as being male, with little reference to the motion capture sensors attached. We found that the zombie was often described as disgusting, rotting, or decomposing. The toon character was often described as cartoon-like, but few people described him as cute. The mannequin was mainly described as a wooden person.. We first analyzed the data separately for each of the six displayed basic emotions. A two factor ANOVA was conducted on the data where conditions were body type (six) and rated emotion (41). As expected, for all displayed emotions, there was a main effect of rated emotion where the list of emotions were rated differently to each other ( p < 0.0001 in all cases). For displayed emotion anger, there was no main effect of body type, which implies that the average ratings for each of the bodies were the same. Also, there was no interaction between body type and rated emotion, which implies that ratings for each of the 41 emotions were not statistically different across body types. Similar effects were found for displayed emotions surprise, disgust, and fear. No main effect of body type was found for displayed emotions happiness or sadness. However, interactions between body type and rated emotion did occur for both happiness (F 200,2240 = 1.235, p < 0.02) and sadness (F 200,2240 = 1.29, p < 0.004). Posthoc analysis using Newman-Keuls pairwise comparisons showed that some of the bodies were rated (Table II) . However, these differences only account for less than 1% of all the one-to-one comparisons possible. Therefore, our main finding is that participants were very robust in their ratings, regardless of the appearance of the character. In order to test whether or not there was a difference between the real and virtual representations, we collapsed all of the ratings for the five virtual characters and compared them to the real data using two-factor ANOVAs for each of the displayed emotions, as before. We found no main effect of body type for any of the displayed basic emotions, nor any interaction between body type and rated emotion for anger, happiness, sadness, or fear. However, we did find an interaction between rated emotion and body for surprise (F 40,760 = 2.22, p < 0.00003). Post-hoc analysis showed that there were differences in 2 out of the 41 rated emotions. First, the average intensity rating for surprise was higher for the real character than for the virtual characters ( p < 0.0009), where the average intensities were 8.7 and 6.8 out of 10, respectively. Also, for rated emotion interest ( p < 0.0025), where the average intensities were 4.9 and 7, respectively. This implied that the real actor conveyed these emotions more intensely than the virtual. A common side-effect of using motion capture data is that limb contacts are difficult to maintain, due to the fact that the original actor may have been a different size to the virtual. For the surprise emotion, the real actor touched his face with both hands to indicate being shocked. In the virtual versions, the hands did not quite reach the face at times, and this may have accounted for the difference in ratings here.. An interaction between body type and rated emotion was also found for disgust, where differences occurred for 2 of the 41 rated emotions. Participants rated the real as having a higher intensity of disgust than the virtual ( p < 0.04), with average ratings of 5.1 and 3.2. There was also a difference for happiness where the virtual was rated as having a higher intensity than the real ( p < 0.01), with average values of 0.45 and 3.2. For this motion, there was some hand-hand contact but these were maintained for each of the virtual characters, so this small difference in ratings for real and virtual may have been due to some other factor such as different neural responses for real and virtual characters (as in Perani et al. [2001] ). It is also interesting to note that the shape of the disgust curve shows the most divergence. Disgust is widely regarded as the most difficult emotion to accurately identify [Atkinson et al. 2004 ].. We compared the results of the NF figure to the collapsed ratings for the six other body types tested in Experiment 1 (i.e., those with form present F) using two-factor ANOVAs for each of the displayed emotions.. For displayed emotions happiness, disgust, and surprise a main effect of body was found ( p < 0.05 in all cases), where the emotions displayed by NF were rated as having lower intensity than F. There was also an interaction between body and rated emotion for disgust and surprise ( p < 0.00001 in both cases). For disgust, this was due to lower intensity ratings for disgust and annoyance for NF than F, and a higher intensity rating for happiness. The body/rated emotion interaction for surprise was due to surprise, interest, hope, anticipation, and enthusiasm being rated as having lower intensities for NF than F. There was no main effect of body for fear or sadness, however there was an interaction between body and rated emotion ( p < 0.02 in both cases). For fear, this was due to fear and panic having lower intensity ratings for NF than for F. For sadness, the interaction was due to shame having a lower intensity rating for NF. For displayed emotion anger there was no main effect of body, nor an interaction. Figure 4 shows that the ratings for NF were lower than for F, but appear to be following the same trend. These results would indicate that emotions were portrayed with less intensity by the figure with no form than by those where form was present. From examination of the graphs in Figure 4 , an alternative explanation could be that, rather than reducing perceived intensity, there may have been a shift in the ratings of emotions categorically to other emotions. It has been shown that body movements and static postures are sufficient to convey at least the basic emotions. The responses to experiments, which aim to recognize emotions from body motions, are highly consistent [Wallbott 1998; Coulson 2004; Atkinson et al. 2004; Crane and Gross 2007] .. Much research has been conducted on the perception of human motion and emotions. It is widely accepted that people can recognize human motion from even a very small set of cues [Johansson 1973 ]. Atkinson et al. [2004] showed that a small number of cues are enough to recognize every emotion with an above chance rate for point-light animations. They confirmed that emotions can be recognized from body motion, even when static form is minimized by use of point-lights. Moreover, they found that the rated intensity of the motion depends more on the motion than on the shape, since point-light motions and real footage were equally rated. Roether et al. [2008] found lateral asymmetry of full-body emotion expression, where the left body side was perceived as having higher emotional expressiveness than the right side. Pasch and Poppe [2007] compare the interpretation of static body postures for a realistic virtual character and a mannequin similar to the wooden mannequin in our study. They show that realism has an effect on the perception of body postures representing the basic emotions, with very different results for each emotion. However, there is no consistent result that would show which character is more qualified to convey emotions.. Evidence from the fields of psychology and neuroscience has shown that different neural networks are activated when presented with real and virtual stimuli. For example, Perani et al. [2001] demonstrated this when participants viewed sequences of a real hand and virtual reproduction. They also found a limited effect due to the degree of realism of the reproduction. Han et al. [2005] analyzed the differences in brain activity when participants viewed cartoons or movie clips. Findings from this study suggest that the human brain functions in a different way when interacting with real people in everyday life than with artificial characters or static social stimuli. More recently, Mar et al. [2007] conducted a study using a set of stimuli derived from the film Waking Life. Participants saw identical biological motion for animated and live-action footage. Using BOLD (blood oxygenation level dependent) signals as a measure of neural activation, a higher level of activity was found in visual processing areas when presented with real stimuli. Chaminade et al. [2007] investigated how the appearance of computer animated characters influenced the perception of their actions. They found that the perceived biological nature of a motion decreased with characters' anthropomorphism.. Other studies have shown that people can respond socially to human and nonhuman entities [Reeves and Naas 1996; Slater and Steed 2002] and that they can engage with virtual humans whether or not they look human [Nowak and Biocca 2003] . Computer animation researchers have also analyzed the interactions between motion and the model it is applied to. For example, Hodgins et al. [1998] and McDonnell et al. [2005] performed perceptual experiments, the results of which indicated that a viewer's perception of motion characteristics is affected by the geometric model used for rendering. In McDonnell et al. [2007] , we showed that sex perception of virtual characters was influenced by the body shape and also by the motion that was applied to it.

O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O


An Efficient Quantum Algorithm for the Hidden Subgroup Problem in Nil-2 Groups In this paper we show that the hidden subgroup problem in nil-2 groups, that is in groups of nilpotency class at most 2, can be solved efficiently by a quantum procedure. The algorithm is an extension of our earlier method for extraspecial groups in Ivanyos et al. (Proceedings of the 24th Symposium on Theoretical Aspects of Computer Science (STACS), vol. 4393, pp. 586-597, 2007), but it has several additional features. It contains a powerful classical reduction for the hidden subgroup problem in nilpotent groups of constant nilpotency class to the specific case where the group is a p-group of exponent p and the subgroup is either trivial or cyclic. This reduction might also be useful for dealing with groups of higher nilpotency class. The quantum part of the algorithm uses well chosen group actions based on some automorphisms of nil-2 groups. The right choice of the actions requires the solution of a system of quadratic and linear equations. The existence of a solution is guaranteed by the Chevalley-Warning theorem, and we prove that it can also be found efficiently. Efficient solutions to some cases of the hidden subgroup problem (HSP), a paradigmatic group theoretical problem, constitute probably the most notable successes of quantum computing. The problem consists in finding a subgroup H in a finite group G hidden by some function which is constant on each coset of H and is distinct on different cosets. The hiding function can be accessed by an oracle, and in the overall complexity of an algorithm, a query counts as a single computational step. To be efficient, an algorithm has to be polylogarithmic in the order of G. While classically no algorithms of polynomial query complexity are possible for the HSP, it can be solved efficiently in abelian groups by a quantum algorithm. A detailed description of the so called standard algorithm can be found for example in [19] . The main quantum tool of this algorithm is Fourier sampling, based on the efficiently implementable Fourier transform in abelian groups. Factorization and discrete logarithm [14, 28] are special cases of this solution.. After the settling of the abelian case, substantial research was devoted to the HSP in some finite non-abelian groups. Beside being the natural generalization of the abelian case, the interest of this problem is enhanced by the fact that important algorithmic problems, such as graph isomorphism, can be cast in this framework. The standard algorithm has been extended to some non-abelian groups by Rötteler and Beth [25] , Hallgren et al. [9] , Grigni et al. [7] and Moore et al. [21] . For the Heisenberg group, Bacon, Childs and van Dam [1] used the pretty good measurement to reduce the HSP to some matrix sum problem that they could solve classically. Ivanyos et al. [12] and Friedl et al. [5] have efficiently reduced the HSP in some non-abelian groups to HSP instances in abelian groups using classical and quantum group theoretical tools, but not the non-abelian Fourier transform. This latter approach was used recently by Ivanyos et al. [13] for extraspecial groups.. The so far unknown complexity of two special cases of the HSP would be of particular interest. The first one is the hidden subgroup problem in the symmetric group because it contains as special instance the graph isomorphism problem. Recently, Moore, Russell and Sniady [22] have shown that no algorithm based one a particular approach can solve the graph isomorphism problem efficiently. The other one is the hidden subgroup problem in the dihedral group because of its relation to certain lattice problems investigated by Regev [26] .. In this work we extend the family of groups where the HSP is efficiently solvable by a quantum algorithm to nilpotent groups of nilpotency class at most 2 (nil-2 groups, in short). These are groups whose lower (and upper) central series are of length at most 2. Equivalently, a group is nil-2 group if the derived subgroup is contained in the center. Nilpotent groups form a rich subclass of solvable groups, they contain for example all (finite) p-groups. Extraspecial groups are, in particular, nil-2 groups. Our main result is: Theorem 1 Let G be a nil-2 group, and let f be an oracle that hides the subgroup H of G. Then there is an efficient quantum procedure which finds H .. The overall structure of the algorithm presented here is closely related to the algorithm in [13] for extraspecial groups, but has also several additional features. The quantum part of the algorithm is restricted to specific nil-2 groups, which are also p-groups and are of exponent p (that is, in which every element has order 1 or p). It consists essentially in the creation of a quantum hiding procedure (a natural quantum generalization of a hiding function) for the subgroup H G of G, where G stands for the derived (a.k.a. commutator) subgroup of G. The procedure uses certain automorphisms of the groups to define some appropriate group actions, and is analogous to what has been done in [13] for extraspecial p-groups of exponent p.. While dealing with extraspecial p-groups of exponent p basically solves the HSP for all extraspecial groups (the case of remaining groups, of exponent p 2 , easily reduces to groups of exponent p), this is far from being true for nil-2 groups. Indeed, one of the main new features of the current algorithm is a classical reduction of the HSP in nil-2 groups to the HSP in p-groups of exponent p and nilpotency class at most 2, where moreover the hidden subgroup is either trivial or of cardinality p. In fact, our result is much more general: we prove an analogous reduction in nil-k groups for any constant k. We believe that this general reduction might be useful for designing efficient quantum algorithms for the HSP in groups of higher nilpotency class.. Our second main novel feature concerns the quantum hiding procedure for H G . While in extraspecial groups it was reduced to the efficient solvability of a single quadratic and a single linear equation modulo p, here we look for a nontrivial solution of a homogeneous system of d quadratic and d linear equations, where d can be any integer. The reason for this is that while in extraspecial groups the derived subgroup is one dimensional, in nil-2 groups we have no a priori bound on its dimension. If the number of variables exceeds the global degree of the system then the solvability itself is an immediate consequence of the Chevalley-Warning theorem [3, 30] : the number of solutions is divisible by p and therefore there is always a nontrivial one. (See [20] for the complexity theory aspects of search problems where existence of a solution is guaranteed by mathematical theorems.) Our result is that if the number of variables is sufficiently large, more precisely is of (d 3 ), then we can also find a nontrivial solution in polynomial time.. The structure of the paper is the following. In Sect. 2 we briefly describe the extension of the standard algorithm for quantum hiding procedures, and then we discuss some basic properties of nilpotent groups, in particular p-groups of exponent p and of nilpotency class at most 2. Section 3 contains the description of the classical reduction of the HSP in groups of constant nilpotency class to instances where the group is also of exponent p, and the subgroup is either trivial or cyclic of order p. Section 4 gives the description of the quantum algorithm for the special instances: we reduce finding H to the design of an efficient hiding procedure for H G , and prove the existence of such a procedure. Finally Sect. 5 shows how to solve the system of quadratic and linear equations occurring in the design of the hiding procedure.. Even if the hidden subgroup problem is hard for the symmetric group and also for general solvable groups, it may happen that there is an efficient solution in nilpotent groups. The works [1, 13] and this paper can be considered as the first steps in investigating the complexity of the HSP for this family of groups.

O O O O O O B I I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Investigating whether hyperNEAT produces modular neural networks HyperNEAT represents a class of neuroevolutionary algorithms that captures some of the power of natural development with a computationally efficient high-level abstraction of development. This class of algorithms is intended to provide many of the desirable properties produced in biological phenotypes by natural developmental processes, such as regularity, modularity and hierarchy. While it has been previously shown that HyperNEAT produces regular artificial neural network (ANN) phenotypes, in this paper we investigated the open question of whether HyperNEAT can produce modular ANNs. We conducted such research on problems where modularity should be beneficial, and found that HyperNEAT failed to generate modular ANNs. We then imposed modularity on HyperNEAT's phenotypes and its performance improved, demonstrating that modularity increases performance on this problem. We next tested two techniques to encourage modularity in HyperNEAT, but did not observe an increase in either modularity or performance. Finally, we conducted tests on a simpler problem that requires modularity and found that HyperNEAT was able to rapidly produce modular solutions that solved the problem. We therefore present the first documented case of HyperNEAT producing a modular phenotype, but our inability to encourage modularity on harder problems where modularity would have been beneficial suggests that more work is needed to increase the likelihood that HyperNEAT and similar algorithms produce modular ANNs in response to challenging, decomposable problems.

O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O B O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O B O B O O O O O O O O B O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O


Structural Modelling with Sparse Kernels A widely acknowledged drawback of many statistical modelling techniques, commonly used in machine learning, is that the resulting model is extremely difficult to interpret. A number of new concepts and algorithms have been introduced by researchers to address this problem. They focus primarily on determining which inputs are relevant in predicting the output. This work describes a transparent, advanced non-linear modelling approach that enables the constructed predictive models to be visualised, allowing model validation and assisting in interpretation. The technique combines the representational advantage of a sparse ANOVA decomposition, with the good generalisation ability of a kernel machine. It achieves this by employing two forms of regularisation: a 1-norm based structural regulariser to enforce transparency, and a 2-norm based regulariser to control smoothness. The resulting model structure can be visualised showing the overall effects of different inputs, their interactions, and the strength of the interactions. The robustness of the technique is illustrated using a range of both artifical and "real world" datasets. The performance is compared to other modelling techniques, and it is shown to exhibit competitive generalisation performance together with improved interpretability. The problem of empirical data modelling is germane to many applications. In empirical data modelling a process of induction is used to build up a model of a system from examples. Ultimately the quantity and quality of the observations will govern the performance of this model. However, the choice of modelling approach will also influence the performance of the model. By its observational nature, data obtained is finite and sampled; typically this sampling is non-uniform and due to the high dimensional nature of the problem, the data will form only a sparse distribution in the input space. Consequently, the problem is nearly always ill-posed (Poggio, Torre, & Koch, 1985) in the sense of Hadamard (1923) . To address the ill-posed nature of the problem it is necessary to convert the problem to one that is well-posed. For a problem to be well-posed, a unique solution must exist that varies continuously with the data. Conversion to a well-posed problem is typically achieved with some form of capacity control, which aims to balance the fitting of the data with constraints on the model flexibility, producing a robust model that generalises successfully. Previous approaches to restoring the well posedness have included regularisation methods (Tikhonov, & Arsenin, 1977) . In this paper, the method chosen is based around kernel methods due to their rigorous formulation and good generalisation ability for small sample sizes. Girosi (1997) and Smola (1998) have shown that kernel methods can be placed in a regularisation framework, which guarantees their well posedness; Support Vector Machines (SVMs) (Vapnik, 1995) and Gaussian Processes (GPs) (Rasmussen, 1996) are examples. For tutorial introductions to SVMs see Burges (1998) , Cristianini and Shawe-Taylor (2000) or Gunn (1998) . Given a dataset, D = {(x 1 , y 1 ), . . . , (x l , y l ) | x i ∈ R d , y i ∈ R}, the model is a weighted linear summation of kernels,. where these kernels are 'centred' on the data points. Consequently, the solution is opaque due to the large number of terms that will typically exist in this expansion. Furthermore, the multivariate basis functions can be difficult to interpret. The number of terms in the expansion can be reduced in some circumstances by enabling a proportion of the kernel multipliers to become zero. This can be achieved using a loss function that has a 'dead-zone', such as an -Insensitive loss function (Vapnik, 1995) . Whilst a predictive model may be the ultimate goal of modelling, it is often desirable and sometimes even essential to be able to interpret the final model structure. This is especially true in medical domains, where black-box models, such as traditional neural networks, bagged descision trees as well as kernel methods, are viewed with great suspicion (Wyatt, 1995; Plate, 1999) . In situations where model interpretation is important, many researchers revert to using simpler, but more interpretable modelling methods, for example logistic regression. As Plate observes (Plate, 1999) there is a danger in using such simple models, since they typically suffer from the problem of model mismatch, and hence they may fail to discover an important relationship in the data because they lack the flexibility to model it. In this work we introduce interpretability, or transparency, by producing a parsimonious model, which has a sparse structural representation, but is flexible enough to avoid problems of model mismatch. The transparency is beneficial in that it enables the model to be validated and interpreted. Features that aid transparency are input selection and ways of decomposing the model into smaller more interpretable pieces that can be easily visualised. To address this issue we introduce a modified kernel model of the form,. where the kernel is replaced by a weighted, c j , linear sum of kernels, K j . Transparency can then be introduced by a careful choice of the additive kernels, K j and by making their weighting coefficients, c j , sparse. In this paper we focus on the integration of an ANOVA (ANalysis Of Variance) representation to provide a transparent approach to modelling. ANOVA kernels (Stitson et al, 1999) have previously been used with SVMs, with promising performance. However, the difference here is to develop a technique that will select a sparse ANOVA kernel producing strong transparency. The ANOVA representation is motivated by the decomposition of a function into additive components, with the goal of representing the function by a subset of the terms from this expansion. A function may be decomposed into. where d is the number of inputs, f 0 represents the bias and the other terms represent the univariate, bivariate, etc., components. The notation x i denotes the scalar value of input i. The basis functions are semi-local and are similar to the approaches used by Friedman (1991) in the Multivariate Adaptive Regression Splines (MARS) technique and in the Adaptive Spline Modelling of Observational Data (ASMOD) technique (Kavli & Weyer, 1995) . The additive representation is advantageous when the higher order terms can be ignored, so that the resulting model is represented by a small subset of the ANOVA terms, which may be easily visualised. This produces a transparent model, in contrast to the majority of neural network models, providing the modeller with structural knowledge that can be used for both validation and model interpretation. Due to the curse of dimensionality (Bellman, 1961) , an exhaustive search of the possible model structures is demanding. Even in the highly restrictive scenario, that the solution is a weighted linear combination of fixed basis functions, the parameter space has size 2 d . Extension to flexible basis functions, which is required for typical modelling, will only compound this dimension. Accordingly, greedy methods are typically used. ASMOD employs an evolutionary strategy to search the model space using a forward selection/backward elimination algorithm to select suitable refinements to a model. The MARS algorithm employs a recursive partitioning procedure to search the model space for an appropriate model. The drawback with both approaches is that they can become entrapped by local minima, due to the greedy nature of their search algorithms. A problem with deploying additive models in advanced flexible non-linear modelling methods is that they cannot provide a transparent model if the phenomenon being modelled contains high dimensional interactions. One possibility is to enforce transparency by constraining the order of possible interactions (e.g. restriction to univariate and bivariate terms only), providing a coarse, but interpretable structure, at the expense of structural integrity.. The aim of this work is to produce transparent models that generalise well, using a global approach to the modelling problem. This paper introduces a new SUpport vector Parsimonious ANOVA (SUPANOVA) technique to realise this goal. It will be shown that the technique is attractive since it can employ a wide range of loss functions (Smola, Schölkopf, & Müller, 1998) , can produce interpretable models, and it is solved by breaking the problem down into simple convex optimisation problems, which can be implemented using readily available mathematical programming optimisers (Mészáros, 1998) . The structure of the paper is as follows. The next section provides an overview of transparent modelling techniques: Section 4 introduces sparse additive kernel modelling, with a particular example, the SUPANOVA technique developed in Section 4, Section 5 describes the datasets which were used for evaluation, and the associated results. The paper ends with a discussion as to the applicability of interpretable modelling methods. An approach to modelling with an emphasis on good generalisation and model interpretation has been described. Model interpretation is achieved through a interactive model representation providing input selection and enhanced visualisation. To obtain a accurate representation a large hypothesis space must be considered, and consequently a technique is warranted that has excellent capacity control. In this respect, kernel methods have been extended to allow the incorporation of a parsimonious kernels. An example if this, the SUPANOVA technique, has been described that decomposes the problem into three simple convex optimisation problems, which can be solved efficiently. With application to four datasets we have shown the the additive structure of the parsimonious SUPANOVA technique can aid in the understanding of complex relationships that can exist in data generated from physical systems. Four modelling problems were used to assess the performance of the SUPANOVA approach.. In each experiment, 90% of the data was used for training and validating the model and 10% of the data was used for estimating the generalisation performance. The SUPANOVA algorithm was executed multiple times for each problem, using both an -Insensitive and a quadratic loss function, with the whole data set "randomly" partitioned into the training and test sets. The capacity control parameter λ α was determined using 8-fold cross validation, combined with an automatic search procedure, which locates a local minimum of the validation error.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Computing the Crosscap Number of a Knot Using Integer Programming and Normal Surfaces The crosscap number of a knot is an invariant describing the nonorientable surface of smallest genus that the knot bounds. Unlike knot genus (its orientable counterpart), crosscap numbers are difficult to compute and no general algorithm is known. We present three methods for computing crosscap number that offer varying trade-offs between precision and speed: (i) an algorithm based on Hilbert basis enumeration and (ii) an algorithm based on exact integer programming, both of which either compute the solution precisely or reduce it to two possible values, and (iii) a fast but limited precision integer programming algorithm that bounds the solution from above.The first two algorithms advance the theoretical state-of-the-art, but remain intractable for practical use. The third algorithm is fast and effective, which we show in a practical setting by making significant improvements to the current knowledge of crosscap numbers in knot tables. Our integer programming framework is general, with the potential for further applications in computational geometry and topology.Burton, B. A. and Ozlen, M. 2012. Computing the crosscap number of a knot using integer programming and normal surfaces. Knot invariants lie at the heart of computational knot theory. Nevertheless, computing invariants can be challenging: algorithms often require complex implementations and exponential running times, and for some invariants no general algorithm is known.. In this article we focus on invariants of knots in R 3 that relate to 2-dimensional surfaces: knot genus and crosscap number. In essence, these measure the simplest embedded orientable and nonorientable surfaces, respectively, that have a single boundary curve following the knot.

O O B I O O O O B I O B I O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O B I O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O


Simultaneous multilingual search for translingual information retrieval We consider the problem of translingual information retrieval, where monolingual searchers issue queries in a different language than the document language(s) and the results must be returned in the language they know, the query language. We present a framework for translingual IR that integrates document translation and query translation into the retrieval model. The corpus is represented as an aligned, jointly indexed "pseudo-parallel" corpus, where each document contains the text of the document along with its translation into the query language. The queries are formulated as multilingual structured queries, where each query term and its translations into the document language(s) are treated as synonym sets. This model leverages simultaneous search in multiple languages against jointly indexed documents to improve the accuracy of results over search using document translation or query translation alone. For query translation, we compared a statistical machine translation (SMT) approach to a dictionarybased approach. We found that using a Wikipedia-derived dictionary for named entities combined with an SMT-based dictionary worked better than SMT alone. Simultaneous multilingual search also has other important features suited to translingual search, since it can provide an indication of poor document translation when a match with the source document is found. We show how close integration of CLIR and SMT allows us to improve result translation in addition to IR results. Cross-Lingual Information Retrieval (CLIR) typically focuses on the task of retrieving documents in language f (or a set of M languages F) that are relevant to a query in language e. However, in many real-life scenarios, such as intelligence work, international business, or tourism, users may not be able to read language f and therefore need results in the query language, e. This requires searching documents in another language, but returning results in the query language; we call this task translingual information retrieval. In CLIR, result translation has usually been considered a separate, post-processing step that should be evaluated separately. In contrast, we show that by viewing CLIR and result translation as parts of the same translingual IR task, we can leverage an integrated, simultaneous search over source and translation to improve the accuracy of search results, and, at the same time, use search results to improve document translations.. Many CLIR systems have used a query translation approach, in which the documents are indexed in their source language(s), the query is translated into each of the F document languages, and the retrieval is done completely in languages F. Another approach to CLIR is to do retrieval in the query language e by indexing the document translations in language e and searching using the original query. Both query translation (QT) and document translation (DT) attempt to map the query and the documents into a common language, but they use different translation strategies. In DT, each document is a large, coherent context with full, grammatical sentences, whereas a query may be short and nongrammatical, with little or no context. On the other hand, once a document is translated, any mistakes or deletions in the translation cannot be remedied, whereas translating a query allows for more flexibility in incorporating multiple possible translations using synonyms and related terms.. In this paper, we present a framework for translingual IR that integrates document translation and query translation in a novel way that is particularly suited for translingual applications. Our approach uses representations in the query language and the document languages simultaneously. We take advantage of having a corpus that has been automatically translated to develop a hybrid model that integrates QT and DT into the indexing and searching, rather than as a post-processing step. We define a pseudo-parallel document as a single document containing both the source (in f ∈ F) and machine translation (in e) of the document. The query is translated into each f ∈ F, and then a multilingual structured query is created, which represents the original query terms as well as their translations into all languages. Finally, the multilingual structured query is run over the pseudo-parallel indexed corpus to retrieve the results. In this Simultaneous Multilingual IR (SMLIR) model, we simultaneously search each document in both the document language, f ∈ F, and the query language e, thus allowing the relative advantages of the QT and DT approaches to complement each other. Our comparison of SMLIR against another state-of-the-art hybrid approach using both query and document translation shows that this advantage yields better performance for SMLIR.. We use the SMILR framework to experiment with different knowledge sources for query translation. We present a novel approach using Wikipedia to obtain both name translations and name variants, including both spelling variations and slang. Given the large difference between the English, Arabic and Chinese languages, differences in name spellings pose significant problems that variants help to address. Our experiments demonstrate that using Wikipedia for query translation yields better results than using statistical machine translation (SMT) alone for query term translation.. The SMLIR approach has additional benefits in a translingual setting. When returning documents in translation, a relevant document that is unreadable to the user is just as useless as an irrelevant document. As part of our system, we show how to use SMLIR to detect translation errors in the documents at query time and subsequently correct them. This approach to automatic postediting allows us to use query-time information to improve the translated responses, and therefore present the user with a more readable document. Both approaches seek to improve translingual IR: SMLIR improves result relevance, and through post-editing improves the translation quality of returned results. Our novel approach for translingual IR features the use of multilingual structured queries which are issued over a pseudoparallel indexed corpus to retrieve results. This hybrid model integrates QT and DT into the indexing and searching, rather than as a post-processing step, and thus allows proper treatment of multilingual documents. Our experimental results show that this approach significantly outperforms a previous hybrid approach, which merges the results of separate queries issued over separately indexed source and English documents. Our experiments evaluated results for English queries and Chinese documents, but our implementation of SMLIR currently includes three languages, English, Chinese and Arabic, demonstrating the ability to seamlessly integrate multiple languages into one framework.. We also experimented with different approaches to query translation. We introduced a method to generate name variants from Wikipedia, an approach that is critical to capturing the variety of name translations between languages as different as Chinese, English and Arabic. Our experimental results show that query translation based on Wikipedia with expansion using name variants outperforms query translation using only a probabilistic phrase table from statistical machine translation. While we expected query translation using a combination of Wikipedia for names and probabilistic translation for topic terms to outperform either approach alone, our results show that using Wikipedia for names alone is not significantly different from the combined approach.. Our research was motivated by a need to provide monolingual speakers the ability to query a multilingual corpus in their own language and receive documents in that same language. The SMLIR framework meets this need and furthermore, provides clues about the quality of document translation. We showed how the framework allows us to detect mis-translated names when query names match against the document but not the translation. Our preliminary method for post-editing names, correcting the translation, shows that we can dramatically improve the quality of translated names in the relevant documents. This is important in Figure 5 . Merged outperforms SMLIR on queries with event/topic arguments, but does poorly on queries with only name arguments.. the context of translingual IR, since it is difficult to tell whether a document is relevant when it is poorly translated.. We are currently working on improving our techniques for postediting of names, incorporating better methods for handling morphological differences and extending it to handle both Chinese and Arabic names. Another direction for future work is the inclusion of name transliteration in query translation. Finally, we will also explore how we can modify our implementation to better exploit a combination of probabilistic term translation with Wikipedia translation of names. Many CLIR systems use some form of QT to search documents in languages other than the query language. Since a term frequently has more than one translation in the target language, previous research has focused on methods for dealing with this kind of ambiguity. Pirkola's [98] approach features "structured queries" which use the term frequencies of all possible translations of a query term; document frequencies are computed based on all documents which contain any of the possible translations. This method has been extended in various ways, addressing computational issues in computing frequencies (e.g., [Kwok 00; Oard and Ertrunc 02]) and augmenting the query translation with translation probabilities that are used as weights for each translated term [Darwish and Oard 03] . Language modeling has also been used as a basis for weighting term translations appropriately (e.g., [Xu and Weischedel 00; Lavrenko and Croft 01; Kraaij 04]. Our research critically differs from all of these approaches in our joint use of query and document translation.. Recent research examines the joint use of query and document term translation [Wang and Oard 06]. Wang and Oard use bidirectional term alignments derived using Giza++ [Och and Ney 00] to translate both query terms and document terms. A key characteristic of their work is the mapping of translated terms to language specific synsets from WordNet [Miller et al. 90 ] for English and other languages. When a WordNet for a specific language does not exist, they compute the synsets for that language automatically from the Giza++ word alignments. This approach thus attempts to capture matches between query terms and document terms based on meaning. They experiment with a number of different methods for matching query-language synsets against document-language synsets. While closer to the work we report here, our use of full document translation rather than term translation alone allows us to carry out simultaneous search in document and query languages. We also demonstrate different ways of doing the query translation using a combination of statistical phrase tables and Wikipedia.. Early work by Oard [98] compares CLIR search using QT and DT separately against more standard approaches of dictionary based look-up for query term translation, and finds the use of DT promising. The approaches of McCarley [99] and Chen and Gey [04] for the joint use of QT and DT are perhaps most similar to our own. Chen and Gey do an "approximate" fast document translation by replacing each word in a document with the single most likely translation and subsequently build a query language index. McCarley uses a full machine translation system to translate his corpus, as we do in our system. Then, Chen and Gey as well as McCarley translate the query using either a 1-best machine translation or the 1-best translation from a statistical translation lexicon, and then do pseudo-relevance feedback for query expansion. In both systems, a document-language search is done with the query translation over the indexed source documents (QT), and then a second query-language search is done with the original query over the separately indexed translations (DT). Finally, the scores from the QT and DT runs are merged to get a score for the hybrid system, and the result documents are reranked. McCarley merges results using arithmetic mean; Chen and Gey sum the scores, which is rank equivalent to mean. Both find that their hybrid systems outperform the QT and DT systems; McCarley's hybrid system even outperforms his monolingual system, where human translations are used to search the source documents.. Rather than build two indexes and run two separate searches, we build a single index where each document is indexed bilingually, as both the original document and its translation into the query language. Then we create a single multilingual query which combines the original query with the query translation(s) into the document language(s). Since we are not merging output from different systems, no parameter tuning is required to determine whether to weight QT or DT higher or how to merge them. This approach is similar to the approach taken by [Nie and Jin 03] for multilingual information retrieval, where documents in multiple languages were combined in a multilingual index that could be searched with a single multilingual query which was constructed via query translation. In their system, the multilingual approach outperformed the "separate indexing then merging" approach. However, their goal was different than ours, since the documents were already multilingual and the purpose was to return relevant documents in multiple languages.. We also explore various improvements to query translation, beyond the single-best machine translation used by [McCarley] and [Chen and Gey] . We derive synonym and translation dictionaries from Wikipedia. Since Wikipedia is created and edited by humans, we hypothesize that it will be better at translating than machine translation. In particular, Wikipedia contains the translations of many named entities, which may be mistranslated by statistical machine translation. [Ferrandez et al. 07 ] demonstrate that Wikipedia is an excellent source of named entity translations for cross-lingual question answering. For their Spanish-English cross-lingual question-answering application, a full 59% of the named entities should not be translated. Wikipedia helps them detect which ones to translate, as well as providing translations for many of the named entities. In contrast, in our case, we are dealing with languages that use non-overlapping alphabets (English to Chinese and Arabic) and thus all names must be translated. Furthermore, given the differences, name misspellings abound. Our use of re-directs to build a set of name variants addresses this problem. Our research on simultaneous multilingual IR is done as part of the DARPA GALE (Global Autonomous Language Exploitation) Here we describe the nature of GALE queries, the use of Indri and our method for query translation using translation dictionaries and synonym dictionaries. Then we describe how to represent a query in the SMLIR framework, and the settings we used to experiment with query translation. For a country to be involved in an event/topic, there must exist an official state action regarding the event.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O