Debugging Incoherent Terminologies Abstract In this paper we study the diagnosis and repair of incoherent terminologies. We define a number of new nonstandard reasoning services to explain incoherence through pinpointing, and we present algorithms for all of these services. For one of the core tasks of debugging, the calculation of minimal unsatisfiability preserving subterminologies, we developed two different algorithms, one implementing a bottom-up approach using support of an external description logic reasoner, the other implementing a specialized tableau-based calculus. Both algorithms have been prototypically implemented. We study the effectiveness of our algorithms in two ways: we present a realistic case study where we diagnose a terminology used in a practical application, and we perform controlled benchmark experiments to get a better understanding of the computational properties of our algorithms in particular and the debugging problem in general. Ontologies play a crucial role in the Semantic Web (SW), as they allow information to be shared in a semantically unambiguous way and domain knowledge to be reused (possibly created by external sources). As a result, however, SW technology is highly dependent on the quality and correctness of these ontologies. Two general strategies for quality assurance are predominant, the first based on developing more and more sophisticated ontology modeling tools, and the second based on logical reasoning. In this paper we focus on the latter. With the advent of expressive ontology languages such as OWL and its close relation to description logics (DLs), state-of-the-art DL reasoners can efficiently detect inconsistencies even in very large ontologies. The practical problem remains what to do if an ontology has been detected to be locally incorrect.. Inconsistent ontologies can be dealt with in two main ways. One is to simply "live with" the inconsistency and to apply a nonstandard reasoning method to obtain meaningful answers in the presence of inconsistencies. Such an approach is taken in [11] . An alternative approach is to resolve (or "debug") the error whenever an inconsistency is encountered. In this paper we focus on this debugging process and on the terminological part of ontologies (and talk about debugging of terminologies). We introduce the formal foundations for debugging and diagnosis of logically incorrect terminologies, more precisely the notions of minimal unsatisfiability-preserving sub-TBoxes (abbreviated MUPS) and minimal incoherence-preserving sub-TBoxes (MIPS) as the smallest subsets of axioms of an incoherent terminology preserving unsatisfiability of a particular (respectively, of at least one) unsatisfiable concept.. Our approach to diagnosing incoherent terminologies is based on traditional model-based diagnosis, which has been studied for many years in the AI community [20] . Here the aim is to find minimal fixes, in our case minimal subsets of a terminology that need to be repaired or removed to render a terminology logically correct and therefore usable again. We will see that in Reiter's terminology, MIPS and MUPS correspond to minimal conflict sets.. We will describe two algorithms for debugging: a bottom-up method using the support of an external reasoner, and an implementation of a specialized top-down algorithm. The former is based on the systematic enumerations of terminologies of increasing size based on selection functions on axioms; the latter is based on Boolean minimization of labels in a labeled tableau calculus. Both methods have been implemented as prototypes. The prototype for the informed bottom-up approach is called DION (Debugger of Inconsistent ONtologies); 1 the prototype of the specialized top-down method is called MUPSter.. We provide a detailed evaluation of our methods. This is done in two ways. First, we present a case study where we apply our algorithms to two medical terminologies that are used in a real application in the Academic Medical Center Amsterdam for the admission of patients to intensive care units. Second, we perform a set of controlled benchmark experiments to get a better understanding of the computational properties of the debugging problem and our algorithms for solving it.. The combined results of the case study and the controlled experiments show that, on the one hand, debugging is useful in practice but that, on the other hand, we cannot guarantee that our tools will always find explanations in a reasonable time. The most important criteria will turn out to be the size and complexity of the definitions and the number of modeling errors. In this paper we have presented a formal characterization for debugging and diagnosis, and we defined two algorithms for computing MIPS, which are directly useful and provide the basis for the calculation of diagnoses. We also provided an evaluation of these algorithms, done in two ways. First, we studied the effectiveness of our proposal in a realistic setting on two real-world terminologies. Second, we conducted a controlled experimental analysis to get a better understanding of the computational properties of the debugging problem and our algorithms for solving it.. The results of these evaluations are mixed. In general, we can conclude that our proposed notions for debugging are useful in practice. Our experience shows that a number of heuristics and additional measures, such as the weight of MIPS or the pinpoints, are crucial in practical applications.. In terms of computational behavior our results are less positive: for each of the described methods there are relatively simple cases, where debugging fails in reasonable time. For example, the DION tool, which performs slightly better on average in our controlled experiments, fails to compute MIPS for a number of realworld terminologies, where MUPSter comes up with a solution within minutes. On the other hand, the more complex reasoning becomes, the worse MUPSter's performance is.. Our experiments yielded a better understanding of the properties of incoherent terminologies that influence the debugging time, most notably the complexity of the definitions and the number of logical modeling errors. Several open questions remain, mostly related to the combination of properties. This means that an estimation of the runtime of the debuggers on the basis of the structure of the incoherent terminology is still very difficult.. An interesting result is the comparison of the two methods: top-down and bottomup, which both have their merits in different cases. This might be an argument to extend the MUPSter approach to more expressive languages and nonrestricted ontologies.. We note as a final remark that since this article was written, extensions of our methods and alternative implementations have been published (most notably [17] and [12] ). Availability of these tools will offer the opportunity for more extensive benchmarking, on more expressive terminologies and ontologies. The results of the experiments with the adapted DL benchmark set differ from the results described in the previous section, as they show high failure rate of the MUPSter tool to solve the more difficult cases, but very good computational behavior of DION. More concretely, we note that MUPSter fails in all of the nine different classes to calculate even the third problem, whereas DION fails only in a single case (k_branch_p) to terminate in time.. Analysis The MUPSter tool fails to debug terminologies that are based on complex unsatisfiability problems, such as the ones used in the DL98 evaluation. These problems have been purpose-built to point to computational difficulties in satisfiability checking; that is, they explicitly exploit structural properties of formulas and provers. More concretely, experience showed that the DL systems of the 1990s failed in these cases because they used naive tableau algorithms and the complexity of the test set forced their calculations to be "lost in the search space.". We conclude that basic optimizations can significantly improve the performance of MUPSterbecause techniques such as lemmatizing and caching can avoid the mentioned computational traps. The price in this case is a loss of theoretical correctness. More precisely, the terminologies returned by our algorithm might not be minimal any more. We doubt whether this is a problem in practice, however, since minimality is a nice but not strictly required feature of the MIPS.. Since DION uses optimized reasoners that can nowadays solve problems like those of the DL98 test set within milliseconds, it solved the problems easily. This success also has to do with the structure of the test set: since we created the TBoxes from concepts in a deterministic way, each axiom is related to the part of the concept it was created from. But this syntactic structure corresponds one-to-one to DION's search strategy: in DION we basically start with the definition on an unsatisfiable concept and include the next following axioms in our test procedure on the basis of a syntactic relation (occurrence of the defined name). Each calculation of MIPS therefore mirrors the construction of the TBox from the unsatisfiable concept.. To counter this process, we decided to add a number of irrelevant axioms to the test set. We arbitrarily chose the first TBox k_path_p and added 100, 200, 400, and 800 random axioms. 23 Figure 7 summarizes the runtimes of this experiment. Although the problem is in principle exponential in the size of the TBox, DION shows linear behavior in this experiment. This result also corresponds to the increasing runtime RacerPro needs for checking coherence. The explanation is relatively simple: the strategy underlying DION controls the inherent complexity of the search process in a nice way. Before presenting our own work, we briefly discuss the most important related work in the literature. In our earlier conference publications [23] we were the first to propose the framework for debugging and diagnosing of terminologies that is defined in Section 2. There we also coined the term pinpointing as a means of reducing a logically incorrect terminology to a smaller one, from which a modeling error could be more easily detected by a human expert. In [22] we grounded notions of MIPS and MUPS in the well-established theory of model-based diagnosis [20] .. The area of debugging inconsistent ontologies has received much attention since the publication of [23] . In this section we do not aim to give a general literature survey, but discuss some of the most influential pieces of work, in particular the work by the MINDSWAP group and the work based on belief revision. The results from the previous section are encouraging: for the practical cases they were developed for, the MUPS, MIPS, diagnoses, and the like indeed helped in some cases. What remains to be investigated is whether we have effective algorithms to calculate those debugging notions in general, whether a particular method will be more appropriate than another, and how well our methods will scale. In the next section we answer these questions by performing some controlled experiments, in which we run our tools MUPSter and DION against a number of benchmarks.

B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Joule expansion imaging techniques on microlectronic devices Abstract-We have studied the electrically induced off-plane surface displacement on two microelectronic devices using Scanning Joule Expansion Microscopy (SJEM). We present the experimental method and surface displacement results. We show that they can be successfully compared with surface displacement images obtained using an optical interferometry method. We also present thermal images using Scanning Thermal Microscopy (SThM) technique to underline that SJEM is more adapted to higher frequency measurements, which should improve the spatial resolution. We have presented surface displacement images obtained on two electrical devices using two different techniques: an optical interferometric one and a Scanning Joule Expansion Microscopy one.. In addition, we have presented SThM images to compare them with SJEM images. We have underlined that the main drawback of SThM measurements regarding SJEM ones is the geometry and the low cut-off frequency of the tip that demands low frequency measurements and limits the spatial resolution. We have presented the performances of the three methods in terms of bandwidth, spatial resolution and sensitivity on the same device.. We are now studying the influence of the working frequency on the spatial resolution and we plan to work under vacuum to improve it.

B I O O O O O O O O O O O O O B O O B I O O B I O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On probabilistic inference by weighted model counting A recent and effective approach to probabilistic inference calls for reducing the problem to one of weighted model counting (WMC) on a propositional knowledge base. Specifically, the approach calls for encoding the probabilistic model, typically a Bayesian network, as a propositional knowledge base in conjunctive normal form (CNF) with weights associated to each model according to the network parameters. Given this CNF, computing the probability of some evidence becomes a matter of summing the weights of all CNF models consistent with the evidence. A number of variations on this approach have appeared in the literature recently, that vary across three orthogonal dimensions. The first dimension concerns the specific encoding used to convert a Bayesian network into a CNF. The second dimensions relates to whether weighted model counting is performed using a search algorithm on the CNF, or by compiling the CNF into a structure that renders WMC a polytime operation in the size of the compiled structure. The third dimension deals with the specific properties of network parameters (local structure) which are captured in the CNF encoding. In this paper, we discuss recent work in this area across the above three dimensions, and demonstrate empirically its practical importance in significantly expanding the reach of exact probabilistic inference. We restrict our discussion to exact inference and model counting, even though other proposals have been extended for approximate inference and approximate model counting.Key words: Bayesian Networks, Exact Inference, Weighted Model Counting, Compilation Some of the work discussed in this paper is based on [1] [2] [3] . Standard algorithms [4] [5] [6] [7] [8] for exact inference on Bayesian networks exploit only topological structure and are known to be Θ(n2 w ), where n is the number of network variables, and w is the treewidth of the network. That is, they are both worst case and best case bounded exponentially by treewidth. In recent years, the types of problems considered in probabilistic reasoning have often yielded networks with large treewidths, calling into question the applicability of exact inference methods, and shifting interest more towards approximate inference algorithms. It has long been believed though that a network exhibiting high treewidth may not necessarily be difficult for exact inference, if it also exhibits a certain amount of local structure in the form of determinism [9] and context-specific independence (CSI) [10] . A typical algorithm that can exploit both topological and local structure is O(n2 w ). That is, only its worst case is exponential in treewidth (e.g., when no local structure exists), but where there is sufficient local structure, its performance can be significantly better.. A number of approaches have been proposed in the literature for exploiting local structure, e.g., [11, 12, 10, [13] [14] [15] 1, 16, 9, 17] . Our aim in this paper is to survey and provide further results on a particular class of these approaches, which works by reducing the problem of probabilistic inference into one of weighted model counting (WMC) on a propositional knowledge base. In particular, the approach commences by encoding the Bayesian network into a knowledge base in conjunctive normal form (CNF), and then assigns weights to the CNF variables based on the network probabilities. These assignments of weights to variables induce a weight for each CNF model, allowing one to represent the probability of some evidence as the sum of weights for models consistent with the evidence. A number of variations on this model counting approach have appeared in the literature recently, and they vary across three orthogonal dimensions. The first dimension concerns the specific encoding used to convert a Bayesian network into a CNF. The second dimension relates to whether WMC is performed using a search algorithm on the CNF [18, 19, 16] , or by compiling the CNF into a structure that renders WMC a polytime operation in the size of the compiled structure [15, 20] . 1 The third dimension deals with the specific properties of network parameters (local structure) which are captured in the CNF encoding.. Probabilistic inference by WMC can be powerful for several reasons. First, the encoding of Bayesian networks into logical knowledge bases presents opportunities to effectively reveal local structure in the form of determinism and context-specific independence. Second, it provides a very natural framework for exploiting the computational power of evidence [2] . Finally, it can leverage highly refined techniques for solving satisfiability, which form the basis for both search and knowledge compilation approaches to WMC. Such techniques are especially effective, as we shall see, on encodings of Bayesian networks that contain large amounts of determinism. In recent years, these advantages have lead to several breakthroughs in exact probabilistic inference that have challenged common perceptions of what exact inference can do. In this paper, we present a survey of these works by placing them along the three dimensions discussed earlier, and then demonstrate empirically their practical importance in significantly expanding the reach of exact inference.. The remainder of this paper is organized as follows. In Section 2, we provide background material. The paper then addresses each of the three orthogonal dimensions that differentiate approaches that utilize WMC: encodings in Section 3, search vs. knowledge compilation in Section 4, and local structure in Section 5. We briefly discuss what happens to WMC techniques in the absence of local structure in Section 6. The paper ends with conclusions in Section 7. We provided in this paper a survey and some new results on a class of approaches for exact probabilistic inference, which reduces the problem to one of weighted model counting (WMC) on a CNF encoding of a Bayesian network. If the network exhibits sufficient local structure, and if this structure is captured during the encoding process, then WMC techniques can efficiently handle networks that have very high treewidths (i.e., ones that are outside the scope of algorithms that exploit only topological structure). The advantages of WMC approaches include a declarative method for encoding local structure, an ability to leverage highly refined techniques from satisfiability solving, and a powerful way for exploiting available evidence. Recent literature on WMC approaches provides different ways of encoding a Bayesian network, different ways of performing WMC (e.g., search vs compilation), and different degrees of utilizing local structure. Our survey of recent WMC approaches has been made systematic by placing these approaches across these three dimensions. The relationship between probabilistic inference and WMC was highlighted in [23] , where it was shown that, like model counting, exact probabilistic inference is #P-complete (however [23] did not address weights directly). This connection was further discussed in [24] , which shows, among other things, that a DPLL-style [25] SAT solver can be adapted to solve both Bayesian network inference and model counting by adding memoization.. An approach to probabilistic inference based on compiling a CNF encoding of a Bayesian network and then performing polytime WMC (in the size of the compiled representation) was introduced in [15] , although this work did not use the term weighted model counting. By making use of determinism, this approach was shown in [15, 22] to perform exact inference very efficiently on many networks having large amounts of determinism, in spite of very large treewidth. The encoding was later enhanced to utilize other types of local structure [1] , allowing it to be effective on a larger class of networks (e.g., ones having lesser amounts of determinism). The ability of WMC to capitalize on available evidence while computing marginals was introduced in [2] . Here, WMC by compilation was shown to efficiently solve many problems with evidence, where they could not be compiled without evidence and where they could not be solved by algorithms that exploit only topological structure, with or without evidence, even after applying classical evidence-exploitation techniques. Methods for increasing the decomposability of the generated CNF by applying structured resolution were described in [3] . The increase in decomposability allowed many networks to be compiled for the first time and significantly decreased the time and space required to compile networks that were accessible previously. Many of the algorithms for this compilation approach are implemented in a publicly available tool called Ace, 3 which uses the c2d compiler [26] . 4 A system described in [16] leverages a state-of-the-art model counter called Cachet 5 to perform inference in a Bayesian network. This work uses a search algorithm rather than compilation and, like [15] , performs WMC on a CNF constructed from a Bayesian network. Among the novel aspects of this work is a new encoding of the network into CNF, which is smaller than the encoding described in [15] in many cases. This search-based algorithm has also been able to significantly outperform algorithms that exploit only topological structure on many networks with large amounts of determinism.. Classical approaches to probabilistic inference, based on conditioning and variable elimination, have also been extended to exploit local structure. For example, a class of conditioning algorithms [8, 27, 12] proceed in a way that is very similar to that of DPLL-style SAT solvers and model counters that use component analysis. A clear connection between these types of algorithms was made in [12] , which also describes how to incorporate additional advances from SAT solvers, including nogood learning and unit propagation, into an algorithm called value elimination. In this approach, search is performed on instantiations of network variables, and the logical techniques, many of which are also used by WMC algorithms, are utilized to capitalize on determinism and context-specific independence. The approach allows for dynamic ordering of variables, has the same guarantees as algorithms that exploit only topological structure, outperforms such algorithms in many circumstances, and is capable of trading time for space, similar to recursive conditioning [8] . In a sense, this class of conditioning approaches is quite similar in spirit to the WMC approaches we discuss in this paper, as they are both based on top-down, recursive conditioning and decomposition techniques. Yet, the WMC approaches we discuss have empirically appeared to be more scalable in general, as they operate directly on CNF representations, putting them at a greater advantage when exploiting satisfiability-based techniques.. Another class of approaches for exploiting local structure is based on variable elimination algorithms (and the closely related jointree algorithm). For example, the technique of zero-compression can be used to exploit determinism in the jointree framework [9] , yet this technique requires one to initially perform inference on the jointree before it is zero-compressed. Hence, the technique initially takes time and space exponential in the network treewidth, potentially realizing savings in future runs, but making many of our data sets inaccessible to this method. More sophisticated extensions to variable elimination algorithms have also been proposed recently, including [11] , [14] , and [17] , which propose non-tabular representations of factors, to avoid the exponential complexity in treewidth. Significantly, [17] is a compilation approach that is able to compile many networks in much less time than WMC compilation techniques described later, while matching WMC online performance. Yet, all of the networks on which these approaches (and zero-compression) have reported are solvable by methods that are exponential in network treewidth.. A third, more specialized, class of algorithms for exploiting local structure include Quickscore [13] and superlink. 6 These algorithms have demonstrated exponential improvements over classical algorithms by solving problems that have very large treewidths. The Quickscore algorithm applies only to a particular class of networks (two-level, noisy-or networks), and superlink includes domain-specific techniques for genetic linkage networks. WMC techniques were shown in [2] to be more efficient than Quickscore, and in many cases significantly more efficient than superlink. We repeat some of these empirical comparisons in Section 5.3. Another example of an algorithm for exploiting local structure, which is able to handle networks with very high treewidth, is value elimination [12] . As mentioned earlier, this algorithm is closest to the WMC algorithms we discuss in this paper. However, the WMC approach based on Cachet was shown to dominate value elimination on a wide class of networks [16] .. Any method that is capable of approximating model counting can potentially be applied to approximate answers to probabilistic queries. One such a method, called ApproxCount, was proposed in [28] and is based on work from [29] . This method samples from the solution space of a CNF near-uniformly, by combining random walk with Metropolis moves. An advantage of the approach is that it can provide an approximation in cases where exact inference is not practical. Another advantage is that the time required can be adjusted by trading accuracy. Although this is an interesting area, we will only be covering exact probabilistic inference and exact model counting in this paper.

O O O O B I I O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I B I B I I B O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Exploring bimanual camera control and object manipulation in 3D graphics interfaces We explore the use of the non-dominant hand to control a virtual camera while the dominant hand performs other tasks in a virtual 3D scene. 'Iwo experiments and an informal study are presented which evaluate this interaction style by comparing it to the status-quo unimanual interaction. In the first experiment, we find that for a target selection task, performance using the bimanual technique was 20% faster. Experiment 2 compared performance in a more complicated object docking task. Performance advantages are shown, however, only after practice. Free-form 3D painting was explored in the user study. In both experiments and in the user study participants strongly preferred the bimanual technique. The results also indicate that user preferences concerning bimanual interaction may be driven by factors other than simple time-motion performance advantages.Bimanual input, 3D interfaces, camera control, interaction techniques, empirical evaluation INTRODUCTION Several user interface researchers over the past decade, having recognized that in the physical world people often use both hands to cooperatively perform many tasks, have explored the possibility of using both hands simultaneously in the computer interface. In an early study, Buxton and Myers [4] showed that in a compound task, a one-handed interface (i.e. the status-quo) was inferior to a two-handed interface which split the compound task into two subtasks that could be performed in parallel by both hands. Kabbash, Buxton, and Sellen [I31 came to a similar conclusion, however, they also showed that two hands could be worse than one if an inappropriate interaction technique is employed, particularly when cognitive load is increased. Several user interface researchers over the past decade, having recognized that in the physical world people often use both hands to cooperatively perform many tasks, have explored the possibility of using both hands simultaneously in the computer interface. In an early study, Buxton and Myers [4] showed that in a compound task, a one-handed interface (i.e. the status-quo) was inferior to a two-handed interface which split the compound task into two subtasks that could be performed in parallel by both hands. Kabbash, Buxton, and Sellen [I31 came to a similar conclusion, however, they also showed that two hands could be worse than one if an inappropriate interaction technique is employed, particularly when cognitive load is increased.. Building partly on this empirical work, several researchers have demonstrated systems with compelling bimanual interfaces for both 2D [2, 151 and 3D [5,9, 17, 19 , 231 applicaPermission to make digital or hard copies ofall or part ofthis work for personal or classroom use is granted without fee provided that topics are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redisttiiwte to lists. requires prior specific permission and/or a fee.. CHI '99 Pittsburgh PA USA Copyright ACM 1999 0-201-48559-l/99/05.35.00 2Aliaslwavefront 210 King Street East Toronto, Ontario Canada M5A 157 { ravin I gordo} @aw.sgi.com tions. However, apart from some fundamental work by Hinckley [9, 10, 111 , little formal evaluation of bimanual3D interfaces has been carried out.. In this paper, we describe and evaluate a bimanual interaction technique for desktop 3D graphics applications which not only increases the input control bandwidth but also enhances user perception of the virtual 3D scene. Essentially, we propose using the non-dominant hand to operate the virtual camera controls typically found in 3D graphics applications, thus freeing the dominant hand to perform other manipulative tasks in the 3D scene. Other researchers [5, 9, 19 , 231 have demonstrated camera operations using the non-dominant hand but have either done so in concert with the dominant hand (i.e., both hands are used to specify camera parameters) [5, 231 or attempted to directly mimic the real world [9, 171, using higher (~2) degree-of-freedom input devices more suited to virtual reality applications. We focus our attention on mouse and keyboard based desktop 3D environments which form the basis of current commercial 3D graphics applications for modeling, design, and animation.. In order to motivate our work, we first briefly review the various depth cues used in 3D displays, followed by a discussion of a current theoretical model of bimanual interaction. We then discuss how one of the most powerful 3D depth cues can be enhanced by following the principles of this bimanual interaction model. Our proposed bimanual interaction technique is then evaluated for a range of typical 3D tasks. Our experiments and informal study have shown that having the non-dominant hand operate a subset of possible camera controls in 3D graphics interfaces can be beneficial over a range of tasks. The results of Experiment 2, however, caution that when the interaction style deviates from Guiard's KC model and both hands begin to operafe in a <symmetric manner, temporal benefits may not be immediately apparent. Of particular interest is the strong preference shown by participants for the two-handed technique regardless of their temporal performance in the task. Because subjective preferences cannot be quantified as reliably as, say, time-motion performance, less weight tends to be placed on such data. While there is a possibility that some of this subjective data suffers from the "good participant" effect (where participants will rate highly experimental conditions which they perceive are favoured by the experimenter, even if the favoured conditions are not explicitly revealed to the participants), we believe, however, that the subjective preference data is in some ways more valuable than quantitative data.. The creative people (artists, modelers, animators, designers) who use 3D graphics applications want interfaces that "feel right", and don't necessarily place much importance on speed advantages. If speed is everything, then one could argue that command line interfaces which experts can often operate much faster than GUIs would still dominate the industry. Clearly, GUIs predominate for reasons other than speed efficiency. As discussed in the introduction, there is a large perceptual component to many 3D graphics tasks and frequent epistemic actions are required to gain a good perceptual understanding of the scene. We believe that this translates into the user getting a better or faster understanding or evaluation of the results of their pragmatic actions. Non-dominant hand operation of camera controls, in addition to speed advantages in some tasks, reduces the cost of epistemic actions and provides the user with a greater sense of engagement with the 3D scene -a step in making 3D graphics interfaces "feel right". Depth Cues in Virtual 3D Scenes 3D graphics applications typically utilize a variety of depth cues to enhance user's perception of the virtual 3D scene. These cues, whose origins can be traced to the human visual perception literature, include perspective, occlusion or interposition, light and shadows, relative size, textual gradient, proximity-luminance covariance, relative motion gradient, retinal binocular disparity, and motion parallax (see [8, 221 for a review).. In 3D graphics, perspective is one of the most commonly employed cues, as evident in the ubiquitous wireframe "groundplane" present in most 3D applications. Also important are occlusion cues which are implemented via hidden line and surface removal techniques. Lights and shadows are less frequently used in interactive 3D graphics because of the high computational cost involved, although this is changing with ever faster graphics engines. Stereopsis, which results from retinal binocular disparity, is a strong depth cue and has been investigated extensively in the virtual reality domain, but is not commonly used in desktop 3D graphics because of the need for expensive and cumbersome viewing apparatus. Also, based on an extensive review of the role of the various depth cues in 3D perception and 3D display design, Wickens et. al. [21, 22] concluded that while stereopsis, motion, and occlusion are all salient cues, motion (e.g., the kinetic depth effect [3, 61 generated when the user's view of the scene is continuously varied by manipulating the virtual camera) is particularly important in creating a sense of three-dimensionality since stereopsis may provide no benefit when motion cues are present. Other evidence [3, 6] also demonstrate and emphasize the importance of motion cues. . Task and Stimuli The 3D object docking task required participants to select an object in one corner of the virtual 3D scene and place it inside a target object located at the diagonally opposite corner.. As shown in Figure 3 (colours have been changed to accommodate greyscale printing), the scene consisted of two objects and a groundplane (identical to the one used in experiment 1) in the middle of the virtual scene. The object to be manipulated was a blue coloured sphere. The target was a purple cube with translucent faces. Colours and transparency effects were chosen to ensure that participants were not hindered in their task by insufficient visual cues. The manipulated object was two thirds the size of the target object.. Target: Purple cube with translucent faces Object: Opaque blue sphere As in Experiment 1, we compared task performance using a one-handed (1H) vs. a two-handed (2H) technique. In the 1H technique, participants used their dominant hand to operate a mouse which controlled both the selection cursor and the camera. Clicking and dragging on the object selected and moved the object; clicking and dragging anywhere else in the scene moved the camera in the appropriate direction. When selected, the object could be moved in two dimensions at a time, always parallel to the plane of the screen (i.e., in the screen's x-y plane). In order to move the object along the z-axis in the virtual scene, the camera ideally has to move 90 degrees such that the virtual scene's zaxis became parallel to the screen's x or y axis. This "screen space" or "image plane" style of object movement is commonly employed in 3D graphics applications which use the 2 degree-of-freedom mouse as the primary input device. It works reasonable well, but as discussed in the introduction, requires constant switching between camera control and object manipulation in order to move an object in 3D space.. In the 2H technique, participants selected and manipulated the object with the dominant hand mouse, while the nondominant hand operated a second mouse which controlled the camera. In this case, both the camera and the object could be manipulated simultaneously. As a result, it becomes possible to move the object into the target in a single movement if the non-dominant hand controlling the camera can coordinate its movements with the dominant hand controlling the object (one way of visualizing this movement is to think of the camera being move'd such that the target is being brought closer to the viewer, while the object is also being moved such that it is also being brought closer to the viewer. At some point in the middle, the object and target will meet). Of course, an alternate strategy is to simply move the camera first, followed by the object, and keep alternating between the two until the task is completed. This is similar to the strategy that has to be used in the 1H technique, except that no explicit switching of modes from camera control to object manipulation is required in the 2H technique since each task is assigned to a different hand.. The camera control metaphor was identical to that used in Experiment 1.. When the object was within the target's boundaries, the target turned bright green. Participants released the dominant hand left mouse button while the object was within the target to indicate completion of a trial.

O O O B O B B O B B B O O O B O O O O O B O O O O O O O O O O O O O B O O O O O B O O O O O O B O O O O O O O O B O O O O O O O O O O O B B O O O O O O O O O O B O O O O B O B B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O B B O B B O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O B B O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O B O O O O B O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B B O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B B B O O O O O O B O B B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O B B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O B O O O B B O B O O O B O B O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O B O O O O O O B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O B O B B O O O O O B O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O B B O O O O O O O O B O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O


Systematic pattern selection using pattern language grammars and design space analysis AbstractSoftware patterns provide reusable solutions to recurring design problems in a particular Software patterns capture reusable design knowledge and expertise that provides proven solutions to recurring software design problems that arise in particular contexts and domains [45] . In recent years, patterns have become part of the mainstream of software development. Different types of software patterns have been published. The most popular software patterns are probably design patterns (see [21] or the proceedings of the PLoP conference series). Design patterns can be applied very broadly because they focus on everyday design problems. In addition to design patterns, the patterns community has created patterns for software architecture [12, 46, 49] , analysis [20] , and even non-IT topics such as organizational or pedagogical patterns.. There are many other kinds of patterns, and some are specific for a particular domain. In this article, we focus on patterns for software architecture and software design.. Consider the relevant patterns in a design situation have been documented in different pattern texts and possibly by different pattern authors. In this and similar situations, systematically applying software patterns requires a certain amount of expertise from the software architect or designer, because she/he has to well understand how a pattern's solution fits into the overall architecture and how other patterns can be applied. to resolve new open issues that might arise as a consequence of applying the pattern. To resolve these problems, today many pattern authors document patterns as part of larger pattern languages [5] , containing rich pattern interdependencies and extensive examples or case studies (see for instance [46, 32, 49, 50, 53] ).. The purpose is to guide the architect or designer step by step to reach the domain-specific goal of the pattern language. Once an initial solution has been envisioned or designed using the pattern language, the patterns can be applied in an incremental refinement process that generates step by step a coherent "whole". Pattern languages are used for evolving whole architectures through this process of piecemeal growth (see [2] ).. Patterns use a description format that supports the concept of piecemeal growth. In his original pattern concept, Christopher Alexander explains the role of patterns in a piecemeal growth process like this [5] : Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice.. A pattern following this concept must capture the possible variations of the same problem/solution pair. In other words, a pattern covers a whole problem space and an associated solution space, not only a single problem and solution 1 . This has severe consequences for the pattern description format: patterns are described with a consistent structure, but the pattern descriptions uses a fairly informal, narrative style. In addition, each pattern author uses a slightly different structure, according to the author's preferences and what fits best 1 Below we use the term "design space" for this problem and solution space.. 2 to the (technical) subject matter. Emphasis is on readability, understanding the language as a whole, and documenting all relevant variants and related patterns of a pattern in a domain.. In contrast to template-like descriptions of a solution to a problem, each pattern describes a great number of variants and possible relationships to other patterns, leading to a web of patterns that can be applied step by step. The individual paths through this web of patterns are called pattern sequences [3, 4] . Because this way a relevant domain can be captured quite completely and for each pattern all relevant links to other related patterns are documented, this description format enables piecemeal growth of a software architecture. This is a great strength of the pattern approach.. But at the same time, this strength is also a weakness because patterns require a certain expertise from the architects both for the relevant patterns and pattern languages and for the domain in which the patterns should be applied. Questions arise like: Which pattern should I choose first? Which variant of the pattern works best? Which pattern should be applied next? Usually the answers to these questions are not simple.. In each particular design situation the architect must consider functional features as well as quality goals, such as performance, maintainability, reusability, etc. These quality goals are usually competing. That is, no optimal (i.e. maximal) solution can be found but only a compromise.. The main goal of this article is to supplement the pattern-based approach and the body of existing pattern literature 2 by providing an approach to better support the selection of patterns and systematic design decisions based on patterns. We propose a two step approach:. • Firstly, we formally document the grammar of a pattern language and annotate it with the effects on quality goals. From the pattern language grammar we can derive the pattern sequences of a pattern language. During the selection of sequence steps, we can use the quality goal annotations to get an approximate overview of the design decision's effects on quality goals.. • Secondly, we analyze in detail the more complex or difficult design decisions in the pattern language that can be found in the annotated pattern language grammar. This is done by documenting the design space [36, 35] for each of these design decisions. A design space explains the considerations and options of a particular design decision in depth.. The pattern language grammar and the design spaces have to be documented once for a pattern language (or a set of related patterns). The initial documentation of the pattern language grammar and design spaces can be provided by the pattern language author (as in the case study provided in this article), or it can be done during an architectural design process (as done in some of the projects discussed in Section 7). The 2 Please note that many existing patterns in the literature are not documented as part of a pattern language, or miss important links to other, related patterns, for instance, because these other patterns were written later in time. We provide a practical approach which can be used to connect these patterns to existing pattern languages.. 3 pattern language grammar and the design spaces can subsequently be applied to guide the pattern selection in many other concrete design situations. They are therefore also a means to document and reuse the design knowledge and design rationale from earlier design experiences based on a pattern language. Finally, as pointed out in Section 7, a pattern language grammar and design spaces documentation can also be used as a way to communicate design decisions to technical and non-technical stakeholders of a software system.. 2 Patterns, pattern languages, and pattern selection We presented an approach to systematize the selection process of software patterns in pattern languages.. The approach is feasible because the documentation of the pattern language grammar and the design spaces has to take place only once (for instance performed by the pattern language author or during the first design with a pattern language), and then the pattern language grammar and the design spaces can be applied for multiple systems (of course they can be refined if information is missing). In contrast to former approaches (see the discussion in Section 8), our approach is capable to use patterns from many pattern sources, deal with architectural patterns (not only GoF design patterns), and provide domain-specific design decisions based on 34 quality goals and forces. The options in the selection process are first reduced to pattern sequences using a formal pattern language grammar, and then to a detailed view on a single design decision in a QOC design space. Thus, we significantly reduce the complexity of the pattern selection process -from the selection among the whole patterns and potential relationships in a number of pattern catalogs or pattern languages to an ordered, systematic sequence of documented design decisions. Throughout the decision process, our approach explicitly considers the quality goals (forces) of the patterns and their variants.. It is important to note that our approach is focused on the abstraction level of patterns and pattern languages (as in the examples in this article). Thus it should not be directly applied to concrete system designs without using the pattern language abstraction. That is, it should not be used for organizing concrete system requirements, because this would entail the danger that other possible design solutions than those documented as patterns are not considered, or that patterns as solutions are applied "blindly" leading to pattern overkill and abuse. It would be interesting to investigate in how far our approach can be extended with requirements engineering approaches, such as goal-oriented requirements engineering [33] , to systematically map concrete requirements to pattern forces.. As discussed in Section 7, our approach can also be used for other goals than only pattern selection. We mentioned projects where our approach helped in the communication with (non-technical) stakeholder, for understanding pattern material, and for technology comparison/evaluation. We envision further application areas for the approach: for instance, the pattern language grammars and design spaces can potentially be used. as an input for model-driven tools. As a future work, we want to examine these and other further application areas for the approach in more depth. We also plan to investigate the use of the approach for pattern language composition.

O B I O O O O O B I I O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Relations among conditional probabilities We describe a Gröbner basis of relations among conditional probabilities in a discrete probability space, with any set of conditioned-upon events. They may be specialized to the partially-observed random variable case, the purely conditional case, and other special cases. We also investigate the connection to generalized permutohedra and describe a "conditional probability simplex."

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O


Integrating generations with advanced reference counting garbage collectors We propose the use of generations with modern reference counting. A reference counting collector is well suited to collect the old generation, containing a large fraction of live objects that are modified infrequently. Such a collector can be combined with a tracing collector to collect the young generation, typically containing a small fraction of live objects. We have designed such a collector appropriate for run on a multiprocessor. As our building blocks, we used the sliding-views on-the-fly collectors.We have implemented the new collector on the Jikes Research Java virtual machine (Jikes RVM) and compared it against the concurrent reference counting collector supplied with the Jikes RVM package. Our measurements demonstrate short pause times, retaining those of the original on-the-fly collectors and a gain in application throughput time. It turns out that a modern reference counting collector may benefit from the use of generations. Automatic memory management is well acknowledged as an important tool for fast development of large reliable software. However, the garbage collection process has an important impact on the overall runtime performance. See, for example, measurements in [24, 9] demonstrating how collector modifications in a production JVM influence the overall throughput of the JVM on standard benchmarks, and the discussion on the impact of garbage collection on performance in [21] . Thus, clever design of efficient memory management and garbage collection is an important goal in today's technology.. In the context of modern SMP platforms and modern applications there is a growing need for a memory manager that is especially targeted at large server configurations: 64-bit sharedmemory multiprocessors running multithreaded applications on a multi-gigabyte heap. Such applications include, for example, web application servers, which must provide relatively fast responses to client requests and scale to support thousands of clients.. Tracing garbage collectors [33, 13] traverse the live objects in the heap, starting from root pointers directly available to the application and tracing their descendants reachable via heap references. The traced objects are either marked or copied to a different area. In the first case, all unmarked objects may be reclaimed in a second phase denoted sweep. Such a collector is called a mark and sweep collector. In the latter case, after all live objects have been copied to another area, the previously active area may be reclaimed as a whole. Such a collector is called a copying collector. The performance bottle-neck for tracing collectors is the traversal of all live objects in the heap. (The sweep phase is usually run using auxiliary bitmaps that make it very efficient [12] .) Tracing collectors cannot scale to very large heap sizes (with large spaces of live objects) since they must visit all the live objects in the heap.. Reference counting is a most intuitive method for automatic storage management, known since the sixties [14] . The main idea is to keep for each object a count of the number of references to the object. When this number becomes zero for an object o, we know that o can be reclaimed. With the spread of the 64-bit architectures and the increase in usage of multigigabyte heaps, reference counting seems very promising to future garbage collected systems. While tracing collectors's performance depend on the space occupied by live objects, reference counting is different. The amount of work required for reference counting is proportional to the amount of work done by the user program between collections plus the amount of space that is actually reclaimed. This method seems appropriate for large heaps.. Many well studied garbage collection algorithms are not suitable for multiprocessors. In particular, many collectors run on a single thread after all program threads have all been stopped. This causes bad processor utilization, and hinders scalability. In order to make better use of a multiprocessor, concurrent collectors have been presented and studied. A concurrent collector is a collector that does most of its collection work concurrently with the program without stopping the program threads. Most concurrent collectors need to stop all program threads at some point during the collection, in order to initiate and/or finish the collection, but the time the mutators must be halted is short. Using the notation of [24] , on-the-fly collectors never stop all the program threads simultaneously. Instead, each thread cooperates with the collector at its own pace through a mechanism called (soft) handshakes. Such collectors are Cycle collection. A major disadvantage of reference counting is that it does not collect cycles. In order to reclaim cyclic structures, we also use the on-the-fly mark-and-sweep collector of [4] occasionally to collect the full heap and reclaim garbage cycles. We propose the use of generations with modern reference counting. A reference counting collector is well suited to collect the old generation, containing a large fraction of live objects that are modified infrequently. Such a collector can be combined with a tracing collector to collect the young generation, typically containing a small fraction of live objects. We have designed such a collector appropriate for run on a multiprocessor. As our building blocks, we used the sliding-views on-the-fly collectors.. We have implemented the new collector on the Jikes Research Java virtual machine (Jikes RVM) and compared it against the concurrent reference counting collector supplied with the Jikes RVM package. Our measurements demonstrate short pause times, retaining those of the original on-the-fly collectors and a gain in application throughput time. It turns out that a modern reference counting collector may benefit from the use of generations.. key words: Runtime systems, Memory management, Garbage collection, Generational Garbage Collection. We have presented a design and implementation of a generational reference counting collector. Our building blocks were the sliding views garbage collectors and the resulting collector is on- the-fly with pause times of several milliseconds. We have used reference counting for the full collection and tracing for the young generation. This choice of collectors matches the typical high percentage of dead objects in the young generation and low percentage of dead objects in the old generation. The collector was implemented on the Jikes RVM and was run on a 4-way IBM Netfinity server. Measurements against the Jikes RVM concurrent collector and against the original algorithm of Levanoni and Petrank show a large improvement in throughput and the same low pause times. We have taken measurements on a 4-way IBM Netfinity 8500R server with a 550MHz Intel Pentium III Xeon processor and 2GB of physical memory. The benchmarks we used were the SPECjvm98 benchmark suite and the SPECjbb2000 benchmark. These benchmarks are described in detail in SPEC's Web site [42] . We tested our new collectors against the Jikes RVM concurrent collector distributed with the Jikes RVM package reported by Bacon et al [6] . The Jikes RVM concurrent collector is an advanced on-the-fly pure reference-counting collector and it is also an on-the-fly collector in which the mutators are only very loosely synchronized with the collector, allowing very low pause times. We also compared the generational collector against the Levanoni-Petrank reference counting collector, from which the generational collector was constructed. Finally, we discuss the generational behavior of the different benchmarks (7.5) and the tuning of parameters (7.6), namely we discuss the chosen size of the young generation. We stress that the measurements are somewhat different from the conference version as explained in Section 6.4 above. Generational collection. Generational garbage collection was introduced by Lieberman and Hewitt [32] , and the first published implementation was by Ungar [45] .. Concurrent collection. Concurrent collectors have been presented and studied in many previous papers (see for example, [7, 43, 44, 20, 3, 16, 11, 23, 22, 37, 40, 36, 9] ). On-thefly collectors have been proposed in [20, 23, 22, 25, 24, 6, 31, 4] .. Historically, the study of concurrent reference counting for modern multithreaded environments and multiprocessor platforms has not been as extensive and thorough as the study of concurrent and parallel tracing collectors. However, recently, we have seen several studies and implementations of modern reference counting algorithms on modern platforms [39, 6, 31] building on and improving on previous work. Bacon et al. [6] and Levanoni and Petrank [31] following DeTreville [16] have presented on-the-fly reference counting algorithms that overcome the concurrency problems of reference counting. Levanoni and Petrank have completely eliminated the need for synchronization operations in the write barrier. In addition, the algorithm of Levanoni and Petrank drastically reduces the number of counter updates (for common benchmarks).. Generational reference counting. Concurrently with this work [5] , Blackburn and McKinley [10] have designed and implemented a generational combination of reference counting for the old generation and copying collector for the young generation. In contrast to this work, they did not deal with concurrency, but suggested a novel control of pause times: each collection runs a full young generation collection and part of the old generation collection, as affordable with the pause time constraint. This yielded short controllable pause times. Also, by not dealing with concurrent mutations, some of the mechanisms used here are not required and a throughput improvements may be gained. Our collector, by using concurrency, achieves an order of magnitude shorter pause times when run on a multiprocessor.. Generational collection without moving objects. Usually, on-the-fly garbage collectors do not move objects ‡ . Demers, et al. [15] presented a generational collector that does not move objects. Their motivation was to adapt generations for conservative garbage collection. Here we exploit their ideas: instead of partitioning the heap physically and keeping the young objects in a separate area we partition the heap logically. For each object, we keep one bit indicating if it is young or old.. Further background on garbage collection can be found in [29] 3. An overview of reference counting algorithms. The traditional method of reference counting was first developed for Lisp by Collins [14] . The idea is to keep a reference count field for each object telling how many references exist to the object. Whenever a pointer is updated the system invokes a write barrier that keeps the reference counts updated. In particular, if the pointer is modified from pointing to O 1 to pointing to O 2 then the write barrier decrements the count of O 1 and increments the count of O 2 . When the counter of an object is decreased to zero, it is reclaimed. The reference counts of all its children are then decremented as well and the reclamation may continue recursively. This method was later used in SmallTalk-80 [27] , the AWK [1] and Perl [46] programs. Improvements to the naive algorithm were suggested in several subsequent papers. Weizenbaum [47] studied ameliorating the delay introduced by recursive deletion. Several works [41, 50] use a single bit for each reference counter with a mechanism to handle overflows. They built on the fact that most objects are singly-referenced, except for the duration of short transitions.. Deutsch and Bobrow [18] noted that most of the overhead on counter updates originates from the frequent updates of local references (in stack and registers). They suggested using the write barrier only for pointers on the heap. Now, when a reference count decreases to zero, the object can not be reclaimed since it may still be reachable from local references. Instead of reclaiming an object whose reference count is decremented to zero, such an object is recorded in a table denoted the Zero Count Table, ZCT. To collect objects, a garbage collection is invoked. During the collection one can reclaim all objects in the ZCT that still have zero heap reference count and are not accessible from local references. This method is called deferred reference counting and it yields a great saving in the write barrier overhead. It is used in most modern reference counting collectors. In particular, this method was later adapted for Modula-2+ [16] . Further study on reducing work for local variables can be found in [8] and [38] .. Reference counting seemed to have an intrinsic problem with multithreading. First, the updates on the counters may be executed concurrently by more than one thread and so the updates must be atomic. Using a synchronization operation in the write barrier seems to be too costly. Since write barriers are run very often (with each pointer update), then even a compare-and-swap operation is too costly for the overall program throughput. An even more problematic issue is the fact that, if the user program allows races, then the collector may fail to operate correctly. For example, suppose Thread T 1 executes O.next ← B while T 2 concurrently executes O.next ← C. Suppose also, that before these operations started concurrently, O.next pointed to the object A. Using a naive write barrier, both threads start by reading the old value of O.next (so that they can later decrement its reference count). They both read A as the referenced object. Then, both perform the assignment. Due to the race, only one assignment prevails. Finally, both T 1 and T 2 decrement the counter of the old object and increment the counter of the new one. Thus, both will reduce the reference count of A and they will increment the counts of B and C. Note that the counters are now wrong. First, A's count has been decremented twice (which is wrong since only one pointer was moved) and second, the counters of both B and C were incremented while only one of them obtained a new reference.. The first system to work with reference counting in a multithreaded environment was presented by DeTreville [16] . He described a concurrent multiprocessor reference counting collector for Modula-2+. DeTreville's algorithm adopted Deutsch and Bobrow's ideas of deferred reference counting and added the idea of a local transaction log. Each thread records its modifications in a local transaction log that requires no synchronization. During the collection, the collector uses all transaction logs to actually update the counters. This eliminates the races on the reference counts since only the collector modifies them, but leaves the problem with program races. To solve that, DeTreville used a single central lock for each update operation (on heap references). This implies that only a single update can occur simultaneously in the system, placing a hard bound on its scalability.. Plakal and Fischer in [39] proposed a collection method based on reference counting for architectures that support explicit multi-threading on the processor level. Bacon et al. [6] provided a solution to the entire garbage collection problem with a pure reference counting approach, showing that this is feasible. Their contributions include a novel on-the-fly cycle collector, an improvement upon Deutsch and Bobrow's Deferred Reference Counting algorithm that does not require use of Zero Count Tables, and an algorithm for concurrent collection by introducing epochs that eliminate the requirement for a single shared buffer for recording information on required increments and decrements of reference counts. In terms of synchronization, they have reduced the need for the central lock and suggested one compare-and-swap instead. This is an improvement over the naive approach which would require at least three compare-and-swap's.. Concurrently with Bacon et. al., Levanoni and Petrank [31] presented a reference counting collector that completely eliminated the need for a synchronization operation in the write barrier. Levanoni and Petrank also build on Deutsch and Bobrow and on DeTreville, using deferred reference counting with local transaction buffers. However, they suggest a carefully designed write barrier and a careful analysis showing that program races can also be overcome without synchronization. Using their write barrier, a program race would result in multiple copies of transaction records rather than in foiling the counts. Their collector can easily overcome multiple records in the transaction buffers.. In addition to the improvement in synchronization, Levanoni and Petrank also presented a significant reduction in the overhead on transaction logging and counter updates. Consider a pointer slot that, between two garbage collections is assigned the values o 0 , o 1 , o 2 , . . . , o n . All previous reference counting collectors execute 2n updates of reference counts for these assignments:. However, only two are required: RC(o 0 )--and RC(o n )++. Furthermore, in modern collectors in which the program threads log updates in a transaction buffer, only one transaction needs to be logged, whereas other collectors log n transactions. Furthermore, records for new objects do not need to be taken, since objects are created with null pointers. As most modifications are performed on new objects (in typical benchmarks), measurements have shown that this improvement reduces the number of logs and counter updates by a factor of 100-1000 for standard Java benchmarks.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O