Logistic Regression, AdaBoost and Bregman Distances Abstract. We give a unified account of boosting and logistic regression in which each learning problem is cast in terms of optimization of Bregman distances. The striking similarity of the two problems in this framework allows us to design and analyze algorithms for both simultaneously, and to easily adapt algorithms designed for one problem to the other. For both problems, we give new algorithms and explain their potential advantages over existing methods. These algorithms are iterative and can be divided into two types based on whether the parameters are updated sequentially (one at a time) or in parallel (all at once). We also describe a parameterized family of algorithms that includes both a sequential-and a parallel-update algorithm as special cases, thus showing how the sequential and parallel approaches can themselves be unified. For all of the algorithms, we give convergence proofs using a general formalization of the auxiliary-function proof technique. As one of our sequential-update algorithms is equivalent to AdaBoost, this provides the first general proof of convergence for AdaBoost. We show that all of our algorithms generalize easily to the multiclass case, and we contrast the new algorithms with the iterative scaling algorithm. We conclude with a few experimental results with synthetic data that highlight the behavior of the old and newly proposed algorithms in different settings. We give a unified account of boosting and logistic regression in which we show that both learning problems can be cast in terms of optimization of Bregman distances. In our framework, the two problems become very similar, the only real difference being in the choice of Bregman distance: unnormalized relative entropy for boosting, and binary relative entropy for logistic regression.. The similarity of the two problems in our framework allows us to design and analyze algorithms for both simultaneously. We are now able to borrow methods from the maximumentropy literature for logistic regression and apply them to the exponential loss used by AdaBoost, especially convergence-proof techniques. Conversely, we can now easily adapt boosting methods to the problem of minimizing the logistic loss used in logistic regression.. The result is a family of new algorithms for both problems together with convergence proofs for the new algorithms as well as AdaBoost.. For both AdaBoost and logistic regression, we attempt to choose the parameters or weights associated with a given family of functions called features or, in the boosting literature, weak hypotheses. AdaBoost works by sequentially updating these parameters one by one. That is, on each of a series of iterations, a single feature (weak hypothesis) is chosen and the parameter associated with that single feature is adjusted. In contrast, methods for logistic regression, most notably iterative scaling (Darroch & Ratcliff, 1972; Della Pietra, Della Pietra, & Lafferty, 1997) , update all parameters in parallel on each iteration.. Our first new algorithm is a method for optimizing the exponential loss using parallel updates. It seems plausible that a parallel-update method will often converge faster than a sequential-update method, provided that the number of features is not so large as to make parallel updates infeasible. A few experiments described at the end of this paper suggest that this is the case.. Our second algorithm is a parallel-update method for the logistic loss. Although parallelupdate algorithms are well known for this function, the updates that we derive are new. Because of the unified treatment we give to the exponential and logistic loss functions, we are able to present and prove the convergence of the algorithms for these two losses simultaneously. The same is true for the other algorithms presented in this paper as well.. We next describe and analyze sequential-update algorithms for the two loss functions. For exponential loss, this algorithm is equivalent to the AdaBoost algorithm of Freund and Schapire (1997) . By viewing the algorithm in our framework, we are able to prove that AdaBoost correctly converges to the minimum of the exponential loss function. This is a new result: Although Kivinen and Warmuth (1999) and Mason et al. (1999) have given convergence proofs for AdaBoost, their proofs depend on assumptions about the given minimization problem which may not hold in all cases. Our proof holds in general without such assumptions.. Our unified view leads directly to a sequential-update algorithm for logistic regression that is only a minor modification of AdaBoost and which is very similar to the algorithm proposed by Duffy and Helmbold (1999) . Like AdaBoost, this algorithm can be used in conjunction with any classification algorithm, usually called the weak learning algorithm, that can accept a distribution over examples and return a weak hypothesis with low error rate with respect to the distribution. However, this new algorithm provably minimizes the logistic loss rather than the arguably less natural exponential loss used by AdaBoost.. A potentially important advantage of the new algorithm for logistic regression is that the weights that it places on examples are bounded in [0, 1] . This suggests that it may be possible to use the new algorithm in a setting in which the boosting algorithm selects examples to present to the weak learning algorithm by filtering a stream of examples (such as a very large dataset). As pointed out by Watanabe (1999) and Domingo and Watanabe (2000) , this is not possible with AdaBoost since its weights may become extremely large. They provide a modification of AdaBoost for this purpose in which the weights are truncated at 1. We speculate that our new algorithm may lead to a viable and mathematically cleaner alternative.. We next describe a parameterized family of iterative algorithms that includes both parallel-and sequential-update algorithms as well as a whole range of algorithms between these two extremes. The convergence proof that we give holds for this entire family of algorithms.. Although most of this paper considers only the binary case in which there are just two possible labels associated with each example, it turns out that the multiclass case requires no additional work. That is, all of the algorithms and convergence proofs that we give for the binary case turn out to be directly applicable to the multiclass case without modification.. For comparison, we also describe the generalized iterative scaling algorithm of Darroch and Ratcliff (1972) . In rederiving this procedure in our setting, we are able to relax one of the main assumptions usually required by this algorithm.. The paper is organized as follows: Section 2 describes the boosting and logistic regression models as they are usually formulated. Section 3 gives background on optimization using Bregman distances, and Section 4 then describes how boosting and logistic regression can be cast within this framework. Section 5 gives our parallel-update algorithms and proofs of their convergence, while Section 6 gives the sequential-update algorithms and convergence proofs. The parameterized family of iterative algorithms is described in Section 7. The extension to multiclass problems is given in Section 8. In Section 9, we contrast our methods with the iterative scaling algorithm. In Section 10, we discuss various notions of convergence of AdaBoost and relate our results to previous work on boosting. In Section 11, we give some initial experiments that demonstrate the qualitative behavior of the various algorithms in different settings. Our work is based directly on the general setting of Lafferty, Della Pietra, and Della Pietra (1997) in which one attempts to solve optimization problems based on general Bregman distances. They gave a method for deriving and analyzing parallel-update algorithms in this setting through the use of auxiliary functions. All of our algorithms and convergence proofs are based on this method.. Our work builds on several previous papers which have compared boosting approaches to logistic regression. Friedman, Hastie, and Tibshirani (2000) first noted the similarity between the boosting and logistic regression loss functions, and derived the sequential-update algorithm LogitBoost for the logistic loss. However, unlike our algorithm, theirs requires that the weak learner solve least-squares problems rather than classification problems. Duffy and Helmbold (1999) gave conditions under which a loss function gives a boosting algorithm. They showed that minimizing logistic loss does lead to a boosting algorithm in the PAC sense. This suggests that the logistic loss algorithm of Section 6 of this paper, which is close to theirs, may turn out also to have the PAC boosting property. We leave this as an open problem. Lafferty (1999) went further in studying the relationship between logistic regression and the exponential loss through the use of a family of Bregman distances. However, the setting described in his paper apparently cannot be extended to precisely include the exponential loss. The use of Bregman distances that we describe has important differences leading to a natural treatment of the exponential loss and a new view of logistic regression.. Our work builds heavily on that of Kivinen and Warmuth (1999) who, along with Lafferty, were the first to make a connection between AdaBoost and information geometry. They showed that the update used by AdaBoost is a form of "entropy projection." However, the Bregman distance that they used differed slightly from the one that we have chosen (normalized relative entropy rather than unnormalized relative entropy) so that AdaBoost's fit in this model was not quite complete; in particular, their convergence proof depended on an assumption that does not hold in general.. 2 Kivinen and Warmuth also described updates for general Bregman distances including, as one of their examples, the Bregman distance that we use to capture logistic regression.. Cesa-Bianchi, Krogh, and Warmuth (1994) describe an algorithm for a closely related problem to ours: minimization of a relative entropy subject to linear constraints. In related work, Littlestone, Long, and Warmuth (1995) describe algorithms where convergence properties are analyzed through a method that is similar to the auxiliary function techniques. A variety of work in the online learning literature, such as the work by Littlestone, Long, and Warmuth (1995) and the work by Warmuth (1997, 2001 ) on exponentiated gradient methods, also use Bregman divergences, and techniques that are related to the auxiliary function method. In this section we discuss various notions of convergence of AdaBoost, relating the work in this paper to previous work on boosting, and in particular to previous work on the convergence properties of AdaBoost. The algorithms in this paper define a sequence of parameter settings λ 1 , λ 2 , . . . . There are various functions of the parameter settings, for which sequences are therefore also defined and for which convergence properties may be of interest. For instance, one can investigate convergence in value, i.e., convergence of the exponential loss function, as defined in Eq. (14); convergence of either the unnormalized distributions q t or the normalized distributions q t /( i q t i ), over the training examples; and convergence in parameters, that is, convergence of λ t .. In this paper, we have shown that AdaBoost, and the other algorithms proposed, converge to the infimum of the exponential loss function. We have also shown that the unnormalized distribution converges to the distribution q as defined in Theorem 1. The normalized distribution converges, provided that q = 0. In the case q = 0 the limit of q t /( i q t i ) is clearly not well defined. Kivinen and Warmuth (1999) show that the normalized distribution converges in the case that q = 0. They also show that the resulting normalized distribution is the solution to. Here P m is the simplex over the m training examples (i.e., the space of possible normalized distributions); D R (q q 0 ) is the relative entropy between distributions q and q 0 ; and q 0 is the uniform distribution over the training examples, q 0 = (1/m)1. This paper has discussed the properties of the unnormalized distribution: it is interesting that Kivinen and Warmuth's results imply analogous relations for the normalized distribution.. We should note that we have implicitly assumed in the algorithms that the weak learner can make use of an unnormalized distribution, rather than the normalized distribution over training examples that is usually used by boosting algorithms. We think this is a minor point though: indeed, there is nothing to prevent the normalized distribution being given to the weak learner instead (the algorithms would not change, and the normalized distribution is well defined unless q i = 0, in which case the algorithm has already converged). In our view, the use of the unnormalized rather than the normalized distribution is a minor change, although the use of the normalized distribution is perhaps more intuitive (for instance, the "edge" of a weak learner is defined with respect to the normalized distribution).. Finally, the convergence of the parameter values λ t is problematic. In the case that q = 0, some of the parameter values must diverge to +∞ or −∞. In fact, the parameter values can diverge even if q = 0: all that is needed is that one or more of the components of q be equal to zero. Even if q is on the interior of , there is no guarantee of convergence of the parameter values, for if the constraints are not linearly independent, there may be several parameter values which give the optimal point. Thus, the parameters may diverge under our assumptions, or even under the assumption that q = 0. This is problematic, as the values for λ are used to define the final hypothesis that is applied to test data examples. Variants of our sequential-update algorithms fit into the general family of "arcing" algorithms presented by Breiman (1999 Breiman ( , 1997a , as well as Mason et al.'s "AnyBoost" family of algorithms . The information-geometric view that we take also shows that some of the algorithms we study, including AdaBoost, fit into a family of algorithms described in 1967 by Bregman (1967) , and elaborated upon by Censor and Lent (1981) , for satisfying a set of constraints. In this section, we briefly describe some experiments using synthetic data. We stress that these experiments are preliminary and are only intended to suggest the possibility of these algorithms' having practical value. More systematic experiments are clearly needed using both real-world and synthetic data, and comparing the new algorithms to other commonly used procedures.. In our experiments, we generated random data and classified it using a very noisy hyperplane. More specifically, in the 2-class case, we first generated a random hyperplane in 100-dimensional space represented by a vector w ∈ R 100 (chosen uniformly at random from the unit sphere). We then chose 1000 points x ∈ R 100 . In the case of real-valued features, each point was normally distributed x ∼ N (0, I). In the case of Boolean features, each point x was chosen uniformly at random from the Boolean hypercube {−1, +1} 100 . We next assigned a label y to each point depending on whether it fell above or below the chosen hyperplane, i.e., y = sign(w · x). After each label was chosen, we perturbed each point x.. In the case of real-valued features, we did this by adding a random amount ε to x where ε ∼ N (0, 0.8 I). For Boolean features, we flipped each coordinate of x independently with probability 0.05. Note that both of these forms of perturbation have the effect of causing the labels of points near the separating hyperplane to be more noisy than points that are farther from it. The features were identified with coordinates of x.. For real-valued features, we also conducted a similar experiment involving ten classes rather than two. In this case, we generated ten random hyperplanes w 1 , . . . , w 10 , each chosen uniformly at random from the unit sphere, and classified each point x by arg max y w y · x (prior to perturbing x).. Finally, in some of the experiments, we limited each weight vector to depend on just 4 of the 100 possible features.. In the first set of experiments, we tested the algorithms to see how effective they are at minimizing the logistic loss on the training data. (We did not run corresponding experiments for exponential loss since typically we are not interested in minimizing exponential loss per se, but rather in using it as a proxy for some other quantity that we do want to minimize, such as the classification error rate.) We ran the parallel-update algorithm of Section 5 (denoted "par" in the figures), as well as the sequential-update algorithm that is a special case of the parameterized family described in Section 7 (denoted "seq"). Finally, we ran the iterative scaling algorithm described in Section 9 ("i.s."). (We did not run the sequential-update algorithm of Section 6 since, in preliminary experiments, it seemed to consistently perform worse than the sequential-update algorithm of Section 7).. As noted in Section 9, GIS requires that all features be nonnegative. Given features that do not satisfy this constraint, one can subtract a constant c j from each feature h j without changing the model in Eq. (34) The new features define an identical model to that of the old features because the result of the change is that the denominator and numerator in Eq. (34) are both multiplied by the same constant, exp(− j λ j c j ).. A slightly less obvious approach is to choose a feature transformation. where c j (x) = min h j (x, ) .. Like the former approach, this causes h j to be nonnegative without affecting the model of Eq. (34) (both denominator and numerator of Eq. (34) are now multiplied by exp(− j λ j c j (x))). Note that, in either case, the constants (c j or c j (x)) are of no consequence during testing and so can be ignored once training is complete. In a preliminary version of this paper, 3 we did experiments using only the former approach and found that GIS performed uniformly and considerably worse than any of the other algorithms tested. After the publication of that version, we tried the latter method of making the features nonnegative and obtained much better performance. All of the experiments in the current paper, therefore, use this latter approach.. The results of the first set of experiments are shown in figure 4. Each plot of this figure shows the logistic loss on the training set for each of the three methods as a function of the number of iterations. (The loss has been normalized to be 1 when λ = 0.) Each plot corresponds to a different variation on generating the data, as described above. When there are only a small number of relevant features, the sequential-update algorithms seems to have a clear advantage, but when there are many relevant features, none of the methods seems to be best across-the-board. Of course, all methods eventually converge to the same level of loss.. In the second experiment, we tested how effective the new competitors of AdaBoost are at minimizing the test misclassification error. For this experiment, we used the same Figure 4 . The training logistic loss on data generated by a noisy hyperplanes by various methods.. parallel-and sequential-updatealgorithms (denoted "par" and "seq"), and in both cases, we used variants based on exponential loss ("exp") and logistic loss ("log"). Figure 5 shows a plot of the classification error on a separate test set of 2000 examples. When there are few relevant features, all of the methods overfit on this data, perhaps because Figure 5 . The test misclassification error on data generated by noisy hyperplanes.. of the high-level of noise. With many relevant features, there is not a very large difference in the performance of the exponential and logistic variants of the algorithms, but the parallelupdate variants clearly do much better early on; they seem to "go right to the solution," exactly the kind of behavior we would hope for in such an algorithm.

B I B O B I O O O O O O O B O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O B O B I O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O B I O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Clock-skew-optimization methodology for substrate-noise reduction with supply-current folding 3) On the average, the error of BELT compared to VG4 is 5.2%; while on the average, the VG4 delay is almost a factor of three higher than that predicted by ELT. 4) Compared to SPICE-based VG4, the error of BELT is 9% on the average.These experiments illustrate that our estimation technique is sufficiently accurate for design planning, while ignoring the blockages is prohibitively costly.Finally, we note how efficient the estimation technique is. The total runtime in seconds for running the 13 test cases mentioned above was 0.24, 23.0, and 29.0 for BELT, VG1, and VG4, respectively. In other words, BELT is about 100 times faster than running an actual bufferinsertion algorithm. For medium sized nets such as n786 and n869, the runtime of VG4 plus the SPICE simulation is over 10 000 times slower than the runtime of BELT.We presented closed-form formulas for estimating the achievable buffered delay when buffering restrictions exist in the layout. We demonstrated that adding blockages to the layout could cause significant error in estimation techniques that ignore the blockage terrain. We also showed that our technique is a lower bound, has an error of less than one percent for two-pin nets, and has only a few percent error for multisink nets. Abstract-In a synchronous clock distribution network with negligible skews, digital circuits switch simultaneously on the clock edge; therefore, they generate a lot of substrate noise due to the resulting sharp peaks on the supply current. A solution is to split a large design in different clock regions and introduce intentional clock skews between them, while taking the timing constraints into account. In this paper, the authors present a complete design flow to optimize the clock tree for less substrate-noise generation in large digital systems. It proposes a technique to assign combinatorial cells and flip-flops to the clock regions. It also takes into account the impact of unintentional clock skew such as jitter on the computed skews in order to assure a robust design. During the optimization, it uses compressed supply-current profiles to improve the CPU time. Experimental results show more than a factor-of-2 reduction in substrate-noise generation from large digital circuits of which the skews are optimized.Index Terms-Clock skew, deep submicrometer, ground bounce, low-noise digital design, mixed analog-digital ICs, power modeling and estimation, substrate noise. Substrate noise is a major obstacle for mixed-signal integration, where it is mainly caused by switching-induced ground bounce in the digital domain [1] . In large digital circuits, high peaks on the supply current create power-supply noise in the supply network. The part of this noise on the V SS rail is the so-called ground bounce. Decreasing the peak and the slope of the supply current caused by the switchings of digital circuits will reduce the ground-bounce generation and, therefore, the substrate noise. Such decrease can be obtained by introducing different skews to a clock network driving a synchronous digital system, creating different clock regions (Fig. 1) .. The use of intentional clock skew for noise reduction has been previously reported for reducing the peak current [2] and the ground bounce [3] , [4] . In prior approaches, the skew of every individual flipflop was optimized (fine-grained skewing) without considering the communication power penalty between many clock regions due to increasing number of switchings and due to extra delay buffers for fixing the hold-time violations (Fig. 2) . The number of switchings can be higher if the timing difference between clock regions is substantially large, i.e., large skew values are needed for reducing the ground bounce. In this case, multiple-timed cells toggle more than necessary, therefore, cause an increase in power. Power penalty can also be severe in designs that require large skew values when we implement the skew of a particular flip-flop by means of delay units. Fine-grained skewing may also require many iterations during the skew-optimization process since the fidelity of the results cannot be guaranteed if the switching behavior of the circuit changes as a result of the new skew values. Limiting the number of clock regions significantly reduces CPU and memory requirements during the skew-optimization process, since, for a large digital system, the required number of clock regions is substantially less than the total number of the flip-flops. For all these reasons, we need a clustering algorithm that groups the flip-flops into clock regions before the optimization. For each clock region, a small amount of skew spread can still be allowed in order to minimize the number of clock buffers driving the flip-flops [5] . This could be an extension of our approach. Up to now, none of the previous skewoptimization approaches [2] , [3] gives a minimum (required) value for the number of clock regions, which is set by the relation between the major resonance frequency of the circuit and the rise/fall time of the supply current. In this paper, we consider this effect to design the optimal clock network [6] . In fact, we will show that after a required number of clock regions, no further reduction of the ground bounce is achieved. We also use representative supply-current profiles to optimize the clock skews since the use of the total transient data is not feasible for the clock-skew optimization [6] , [7] .. The cost function used for the optimization of the clock skews also yields more accurate results as compared to previous work, which uses some mathematical functions based on a triangular approximation of the supply current [2] , [6] . For synchronous systems, the spectrum of the substrate noise has peaks at multiples of the digital clock frequency. These peaks are shown to be the dominant components of the total supply current [7] , [8] . As a result, the optimization should be performed at each clock harmonic on the constraint space formed by the skews. This gives more optimal results. Additional constraints such as performance/race reliabilities of the clock are also introduced in the optimization in order to have a clock tree tolerant to process variations, which have also been addressed [9] .. Two techniques to reduce digital substrate-noise generation have been demonstrated in [7] : clock frequency modulation and intentional clock skews. The experimental validation of the use of intentional clock skews in order to compare its effectiveness with the use of other low-noise digital-design techniques such as decoupling has been presented in [8] . With respect to frequency modulation, in [7] we focus on the frequency-spectrum analysis of the supply current in an effort to find the optimum settings for the modulating waveform. In this paper, we particularly describe the clock-skew-optimization methodology where we highlight the following of our contributions: folding the supply current, determining the required number of clock regions, clustering the clock regions, and optimizing the clock skews.. Digital signal processing (DSP) and multimedia applications usually spend most of their time executing a small number of code segments with well-defined characteristics, called kernels. To accelerate the execution of such kernels, various high-performance data paths have been proposed [1] - [11] . Research activities in high-level synthesis (HLS) [2] - [4] and application-specific instruction processors (ASIPs) [5] - [11] have proven that the use of complex resources instead of primitive ones [like a single arithmetic logic unit (ALU)] improves the performance of the data path. In these works, complex operations (instructions in the ASIP domain) are used at the behavioral level instead of groups of primitive recourses. At the architectural level, the complex operations are implemented by optimal customdesigned hardware units, called templates or clusters. A template may be a specialized hardware unit or a group of chained units. Special hardware is usually used for operations that commonly appear (e.g., multiply-add). Chaining is the removal of the intermediate registers between the primitive units, improving the total delay of the combined units. The templates can be either obtained by an existing library [2] - [5] or can be extracted by the kernel's data flow graph (DFG) [6] - [11] . Although performance is improved with the usage of templates, the area of the produced data path is larger than the area of the corresponding data path implemented by primitive resources [2] - [11] . However, the area-time product, which indicates the efficiency of the synthesized data path [2] , is decreased over a data path implemented by primitive resources. We presented closed-form formulas for estimating the achievable buffered delay when buffering restrictions exist in the layout. We demonstrated that adding blockages to the layout could cause significant error in estimation techniques that ignore the blockage terrain. We also showed that our technique is a lower bound, has an error of less than one percent for two-pin nets, and has only a few percent error for multisink nets. In this paper, we have introduced an efficient clock-skewoptimization methodology for reducing the ground bounce (therefore, substrate noise) in large digital circuits. The methodology splits the digital system in different clock regions and optimizes the clock skews between them to reduce the substrate-noise generation. The required number of clock regions is computed based on the elimination of the major resonance frequency determined by the on-chip circuit capacitance and the supply-line parasitics. The run-time of the optimization is improved by using supply-current profiles that can be used as periodic pulses for the representation of the total supply current. Additional constraints such as performance and race reliabilities have been introduced into the optimization in order to have a resulting clock tree tolerant to process variations.. Experimental results show around a factor-of-2 reduction in the rms value of the generated ground-bounce voltage. The effectiveness of the methodology has also been verified with measurements on a fabricated mixed-signal chip. The supply-current shaping by the use of intentional clock skews has been shown to be very effective for reducing the substrate-noise generation if timing constraints allow shaping.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I I B I I B I I I B I B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fair resource allocation with guaranteed statistical QoS for multimedia traffic in wideband CDMA cellular network Abstract-A dynamic fair resource allocation scheme is proposed to efficiently support real-time and non-real-time multimedia traffic with guaranteed statistical quality of service (QoS) in the uplink of a wideband code-division multiple access (CDMA) cellular network. The scheme uses the generalized processor sharing (GPS) fair service discipline to allocate uplink channel resources, taking into account the characteristics of channel fading and intercell interference. In specific, the resource allocated to each traffic flow is proportional to an assigned weighting factor. For real-time traffic, the assigned weighting factor is a constant in order to guarantee the traffic statistical delay bound requirement; for non-real-time traffic, the assigned weighting factor can be adjusted dynamically according to fading channel states and the traffic statistical fairness bound requirement. Compared with the conventional static-weight scheme, the proposed dynamic-weight scheme achieves capacity gain. A flexible trade-off between the GPS fairness and efficient resource utilization can also be achieved. Analysis and simulation results demonstrate that the proposed scheme enhances radio resource utilization and guarantees statistical QoS under different fairness bound requirements. F UTURE wireless communication networks are expected to support multimedia traffic, such as video, voice, and data, with a variety of quality of service (QoS) requirements and to make efficient use of the radio resources. One promising approach to efficient QoS support is dynamic resource allocation, which takes into account the variation of traffic load and channel conditions [1] , [2] , [3] . The radio resources should be allocated in a fair manner so that each user can be guaranteed the agreed upon QoS, even though other users may be greedy in demanding bandwidth. How to ensure fairness while efficiently supporting QoS for multimedia traffic in wireless networks is an important problem which has recently attracted more and more attention [4] , [5] , [6] . The works in [4] , [5] , [6] extend the existing fair scheduling schemes developed for wireline networks, such as generalized processor sharing (GPS) [7] , to time-division multiple access (TDMA)-based and hybrid time-division/code-division multiple access (TD/CDMA)-based wireless networks. These schemes are implemented using a conventional time-scheduling approach, requiring high complexity due to the intensive computation for the virtual-time of each packet [8] . On the other hand, the time-scheduling approach is not fully suited to CDMAbased wireless networks since the radio resources in such networks are mainly related to the spreading bandwidth, transmission power, and channel rates [9] . Fair resource allocation schemes need to be redesigned in order to improve radio resource utilization and achieve fairness with low complexity in such CDMA-based wireless networks.. An important issue in wireless cellular networks is the effect of multipath channel fading on the link capacity. The effect of multipath fading on the uplink capacity of conventional power-controlled CDMA cellular networks has been investigated in [10] , [11] . Even with perfect closeloop power control, multipath channel fading may cause a significant reduction in the uplink capacity because the mobile stations (MSs) in deep fading attempt to transmit signals with very high power which causes a substantial increase in intercell interference. The variability of the capacity is also increased due to an enlarged variation of the intercell interference. The reduction and variability of uplink capacity can cause inefficiency of fair resource allocation for QoS support. One approach to improve the efficiency is to apply combined rate and power control, taking into account the channel fading conditions [12] . A rate scheduler, which performs rate control, allocates more bandwidth to the MSs with good channels and less bandwidth to others in bad channel conditions. Such an approach is referred to as opportunistic scheduling [13] . The opportunistic scheduling has the drawbacks of:. 1. the lack of short-term fairness (although the longterm asymptotical fairness can be achieved [14] ), 2. not supporting the GPS service discipline and, therefore, being difficult to support QoS for multimedia traffic, and 3. implementation complexity for the CDMA uplink.. In [15] , a rate-scheduling approach based on GPS is applied to the CDMA downlinks. Given the limit of the total downlink transmission power, the rate-scheduling scheme dynamically allocates the downlink power and rates according to weights assigned to the users. The user weights are optimized for each scheduling period to guarantee the required minimum channel rates, adapting to the time-varying channel condition, at the cost of high complexity. However, the short-term fairness cannot be guaranteed due to the frequent adjustment of the weights. In [16] , a low-complexity code-division GPS (CDGPS) scheme for dynamic fair scheduling is proposed. The scheme makes use of the adaptive feature in the wideband CDMA physical layer to efficiently support QoS for multimedia traffic and uses a fixed weight assignment to guarantee GPS fairness. A low-complexity GPS-based bandwidth scheduling scheme similar to the CDGPS is also proposed in [17] , where a multicarrier CDMA system is considered. However, both schemes are designed without considering the effect of multipath fading on link capacity in the cellular network.. In this paper, a dynamic fair resource allocation scheme is proposed to efficiently support both real-time and nonreal-time (multimedia) traffic with guaranteed statistical quality of service (QoS) in the uplink of a wideband codedivision multiple access (CDMA) cellular network. The scheme uses the generalized processor sharing (GPS) fair service discipline to allocate uplink channel resources, taking into account the characteristics of channel fading and intercell interference. In specific, the resource allocated to each traffic flow is proportional to an assigned weighting factor. For real-time traffic, the assigned weighting factor is a constant in order to guarantee the traffic statistical delay bound requirement (i.e., the probability of exceeding a maximal delay requirement is smaller than a prescribed value); for non-real-time traffic, the assigned weighting factor can be adjusted dynamically according to fading channel states and the traffic statistical fairness bound requirement (i.e., the probability to achieve the GPS fairness is larger than a prescribed value). Analysis and simulation results are given to demonstrate that the proposed fair resource allocation scheme can 1. guarantee statistical QoS delay bound for real-time traffic and a statistical fairness bound for non-realtime traffic, 2. improve the uplink capacity, and 3. achieve efficient radio resource utilization. The rest of this paper is organized as follows: Section 2 describes the system model. In Section 3, dynamic fair resource allocation schemes for real-time and non-real-time traffic are presented. Section 4 gives an analysis of the capacity gain achieved by the proposed scheme. Section 5 presents simulation results, followed by concluding remarks in Section 6. An efficient dynamic fair resource allocation scheme has been proposed for supporting multimedia traffic in the uplink of wideband CDMA cellular networks with QoS satisfaction. The analysis and simulation results show that, in a multipath fading environment, the proposed scheme can reduce the intercell interference, increase the network capacity, guarantee a statistical delay bound for real-time traffic and a statistical fairness bound for non-real-time users. The proposed scheme allows for a flexible trade-off between the generalized processor sharing (GPS) fairness and efficiency in resource allocation and is an effective way to maximize the radio resource utilization under the fairness and QoS constraints.

O B I O O O O O B I O O O O O O O O B I O O O O O O O O O B I O O O B I I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O B I I O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O B I O O O O O O O O O O O O O O O O O


Exploratory evaluations of a computer game supporting cognitive behavioural therapy for adolescents The need to provide effective mental health treatments for adolescents has been described as a global public health challenge [27]. In this paper we discuss the exploratory evaluations of the first adolescent intervention to fully integrate a computer game implementing Cognitive Behavioural Therapy. Three distinct studies are presented: a detailed evaluation in which therapists independent of the design team used the game with 6 adolescents experiencing clinical anxiety disorders; a study in which a member of the design team used the game with 15 adolescents; and finally a study assessing the acceptability of the game and intervention with 216 practicing therapists. Findings are presented within the context of a framework for the design and evaluation of complex health interventions. The paper provides an in-depth insight into the use of therapeutic games to support adolescent interventions and provides stronger evidence than previously available for both their effectiveness and acceptability to stakeholders. Mental disorders are health conditions defined by the experiencing of severe and distressing psychological symptoms, to the extent that normal functioning is seriously impaired. The Global Burden of Disease Study has found that mental health difficulties are currently the leading cause of disability in developed countries [34] . Large scale international studies also conclude that many people experiencing difficulties do not receive appropriate specialist treatment [35] . The human and economic costs of this situation are substantial and calls have been made for innovative new approaches to help in addressing this challenge [30] . Young people have been identified as particularly vulnerable [5, 27] . Patel et al. [27] [8] . Writing in 1997 and reflecting on 25 years of the successful development of CBT, Aaron Beck the chief originator of this approach, indicated that one of the greatest challenges for CBT in its second 25 years was its translation into therapeutic programmes suitable for children and adolescents [3] . The meta-cognitive tasks in CBT, requiring people to think about their thinking, are often challenging for adults and do not come naturally to younger people. Research suggests that effective CBT programmes for younger people should be tailored to their developmental needs, use content which is young person focused, emphasise elements of fun and make the process experiential [2, 5, 8] .. Recent years have witnessed a significant increase in research on the use of technology to support psychological or talk-based interventions [12, 24] . Much of this work has focused on computerised CBT (cCBT) interventions targeting mild disorders in adults. Systems targeting younger groups, and groups experiencing more severe difficulties, have received significantly less attention, and our understanding of the requirements of such systems remains limited. Coyle and Doherty [11] recommend that systems targeting adolescents seek to provide an engaging, client-centred experience. In the context of face-to-face interventions, which many adolescents find difficult, it is suggested that technologies such as computer games can be used as a 'third party in the room', helping to make the Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. therapeutic process less difficult for adolescents by taking some of the emphasis off direct face-to-face conversations.. The primary contribution of this paper is to present the exploratory evaluations of gNats Island, a computer game developed to support face-to-face CBT interventions for adolescents experiencing medium to severe mental health difficulties. The game is designed to fully integrate with a six-week clinical intervention. As such, it constitutes a significant alteration to the conduct of a treatment. Our research is reported within the overall context of a framework for the design and evaluation of complex health interventions [7] . No similar evaluation has been reported to date. A secondary contribution is to demonstrate the use of mixed methods to evaluate a new computer supported mental health intervention. Three distinct studies of gNats Island are presented. Taken together they provide strong initial evidence of the potential of therapeutic games to support adolescent interventions. They also provide an indepth insight into the use of technology to support face-toface treatments with young people experiencing moderate to severe mental health difficulties. Alongside those interested in mental health technologies, this work will be of interest to researchers investigating health technologies more generally. It is also relevant to the broader community interested in serious games, behaviour change technologies and design for younger users. We begin by providing a brief introduction to the central theories of CBT. Research on the use of technology to support adolescent mental health interventions is clearly in its early stages. In this paper we have reported on the first fully integrated use of a computer game to support a CBT intervention. We conclude this paper by considering several important lessons which can be drawn from the studies reported. Taken together the studies provide an in-depth insight into the use of technology to support a face-to-face intervention for adolescents experiencing moderate to severe difficulties. Definitive evidence on the clinical effectiveness of the gNats Island intervention will require larger scale clinical trials. However building on previous research, this paper provides good initial evidence of the potential of therapeutic games to assist in adolescent treatments. Unlike previous studies of games in face-to-face settings it provides quantitative evidence to support the observations of clinicians. We have also presented strong evidence that a game-based approach can be acceptable to both young people and mental health professionals.. Technology in this case has served several purposes. By reshaping the dynamics of the therapeutic interaction in an age appropriate manner, it helped to create a context in which adolescents could engage more easily with the therapeutic process and with the clinician. The game provides an overall structure and active process. By using elements of fun and concrete representations of difficult concepts, it seeks to make the CBT process more suited to the developmental needs of adolescents. While potential improvements have been identified, particularly in the case of younger adolescents, client engagement was good in both clinical studies and feedback from clinicians and adolescents suggests that the game did help in creating the fun and experiential process recommended in adolescent interventions.. Many of the lessons from this paper can be generalised to other mental health technologies. While gNats Island does change the dynamics of the therapeutic interaction, it does so in a manner supportive of the client-therapist relationship. This was critical to clinicians' acceptance of the game and we recommend that this relationship be carefully considered when designing any new system in this area. Designers should also consider the protocols surrounding use of technologies (e.g. how to manage expectations when introducing new technology), consider developing supplementary materials for both clients and their parents, provide paper-based options and options at follow up sessions similar to those used in the intervention.. Based on the experience of the clinician in our second clinical study, we believe that gNats Island and other mental health technologies will be most effective when clinicians have the option to use it flexibly. Used flexibly, technology can help to provide a structure around which clinicians can tailor interventions to best suit the needs of their clients, including the option of a manualised approach.. To support flexibility, systems should be modular, be easy to pick up and put down and easy to integrate with other therapeutic activities.. From the perspective of large-scale definitive evaluations, which require strict protocol, flexibility of use creates challenges. The exploratory nature of the evaluation in this paper allowed us to undertake three complementary studies, each providing a different view on the use and usefulness of gNats Island. They provided a valuable and flexible context in which to address both HCI and clinical issues. Future work on gNats Island will therefore involve both formal and exploratory studies. In our formal study the game will be used in a manualised manner. Exploratory studies will be used to further examine the flexible use of the game, and to investigate new design ideas. Many of the professionals who participated in the professional survey have volunteered to participate in these studies. As this field progresses there will also be a need for long-term implementation studies, examining the use of systems such as gNats Island in day-to-day clinical practice. This study has provided initial evidence that gNats Island is acceptable to mental health professionals, with each clinician indicating they would like to continue using the game -albeit with the option to use it flexibly. The majority of adolescents also reported enjoying the game and would recommend it to a friend. With the exception of Derek (neutral clinical impact) case studies suggest that the intervention had a positive impact on adolescents. Unlike previous studies of games in face-to-face interventions, this study has provided quantitative evidence supporting clinicians' observations regarding the impact of the game. It is important however at this point to note that, while the CBCL scores indicated improvements in anxiety symptoms for many of the adolescents, clients self reported YSR symptom scores were less conclusive. In most cases YSR measures varied widely across baseline collection points, thus preventing firm conclusions on this measure.. One of the most important factors observed in this study was the change in dynamics supported by the game. Adolescents attending mental health services -particularly those who are reluctant to attend, e.g. Carla -can often react confrontationally or not at all to direct dialogue with a clinician. With younger children clinicians will often use play materials (e.g. puppets or building blocks) to ease the difficulties associated with clinical settings. This study provides evidence that technologies such as computer games can act as an age-appropriate mediating factor in adolescent interventions. Unlike the situation in which play is used with younger children -where games provide the clinician with a means of observing the child's behaviourthe game in this case helps to actively engage adolescents in a cognitive bahavioural intervention process. It is important that, while the game altered the dynamics of the clienttherapist relationship, it did so in a manner that clinicians felt supported and even enhanced this critical relationship. Support for the client-therapist relationship was a critical factor in clinicians acceptance this new technology.. Alongside providing outcome measures, quantitative data collected in this study has served a second purpose. Session rating scores highlighted points in the intervention at which some adolescents experienced difficulties. For example, the scores of some adolescents (e.g. Amy) were found to dip in session 4. Our second clinical study provides a more indepth insight into this and other important issues identified in the first study -e.g. the flexible use of gNats Island.. This final study provides further evidence that gNats Island is perceived as both acceptable and useful by a broad range of mental health professionals. As in CS1, flexibility in use emerged as an important consideration. In many ways this reflects current clinical practice. 61.6% of participants in this study reported currently using manualised (noncomputer based) treatments. However many reported doing so in a flexible manner.

O B O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Verifying pointer safety for programs with unknown calls We study the automated verification of pointer safety for heap-manipulating imperative programs with unknown procedure calls. Given a Hoare-style partial correctness specification S = {Pre} C {Post} in separation logic, where the program C contains calls to some unknown procedure U , we infer a specification SU for the unknown procedure U from the calling contexts. We show that the problem of verifying the program C against the specification S can be safely reduced to the problem of proving that the procedure U (once its code is available) meets the derived specification SU . The expected specification SU for the unknown procedure U is automatically calculated using an abduction-based shape analysis. We have also implemented a prototype system to validate the viability of our approach. Preliminary results show that the specifications derived by our tool fully capture the behaviors of the unknown code in many cases. While automated verification of memory safety remains a big challenge [14, 21] , especially for substantial system software such as the Linux distribution and device drivers, significant advances have been seen recently on the automated verification of pointer safety for such software [6, 22] . The abduction-based compositional shape analysis [6] is able to calculate the pre/post specifications for procedures in a bottom-up approach, based on the call-dependency graph. It can verify the pointer safety of a large portion of the Linux kernel and many device drivers manipulating shared mutable data structures. One issue that has not been dealt within their work is unknown procedure calls. In their SpaceInvader tool, unknown calls are currently ignored and replaced by the empty statement skip during the verification. This may lead to imprecise or unsound results in general. Our work here is to investigate this issue carefully and provide a better solution to the verification of pointer safety of programs with unknown calls.. Automated program verifiers usually require to have access to the entire given program, which in practice may not be completely available for various reasons. For instance, some programs (e.g. in C) may contain unknown calls that correspond to function pointers, some programs (e.g. in OO) may contain calls to interface methods whose actual implementations may not be available statically, or some programs may have calls to library procedures whose code is not available during verification. Other possible scenarios can be found in remote procedure calls such as COM/DCOM [20] , mobile code and software upgrading, where program fragments may be unavailable at verification time. To deal with the verification of programs with unknown procedure calls, current automated program verifiers either • ignore the unknown procedure calls, e.g. replacing them by skip [6] , which can be unsound in general; or • assume that the program and the unknown procedure have disjoint memory footprints so that the unknown call can be safely ignored due to the hypothetical frame rule [18] ; however, this assumption does not hold in many cases; or • use specification mining [1] to discover possible specifications for the (unknown part of the) program, which is performed dynamically by observing the execution traces and is not likely to be exhaustive for all possible program behaviors; or • take into account all possible implementations for the unknown procedure [10, 12] .. In general, there can be too many such candidates, making the verification almost impossible at compile time; or • simply stop at the first unknown procedure call and provide an incomplete verification, which is obviously undesirable.. Approach and contributions. We propose a different approach in this paper to the verification of programs with unknown procedure calls. Given a specification S = {Pre} C {Post} for the program C containing calls to an unknown procedure U, our solution is to proceed with the verification for the known fragments of C, and at the same time infer a specification S U that is expected for the unknown procedure U based on the calling context(s). The problem of verifying the program C against the specification S can now be safely reduced to the problem of verifying the procedure U against the inferred specification S U , provided that the verification of the known fragments does not cause any problems. The inferred specification is subject to a later verification when an implementation or a specification for the unknown procedure becomes available (e.g. at loading time in Java). The intuition of our method to infer the unknown procedure's specification can be divided into two steps. The first step is to analyze the code before the unknown procedure call to discover its precondition. The second is to analyze the code after the unknown call in order to discover its postcondition. For the second step, one might suggest to use a deductive backwards analysis, starting from the postcondition of the program being verified, to derive an expected postcondition for the unknown procedure. We cannot follow this suggestion due to a technical challenge: backwards reasoning over the separation domain is simply too costly to implement ( [6] ). Therefore, we exploit an abductive forwards reasoning [6, 11] to derive the unknown procedure's postcondition, and in this way the verification can be accomplished.. Our paper makes the following technical contributions: • We propose a novel framework in separation logic for the verification of pointer safety for programs with unknown calls.. • The top-down feature of our approach can potentially benefit the general software development process. Given the specification for the caller procedure, it can be used to infer the specification for the callee procedures. This is a potentially beneficial complement for current bottom-up approaches ( [13] ).. • We have enhanced the abduction mechanism, resulting in an improved algorithm for the verification use. Compared with bi-abductive analysis [6] we have added additional reasoning rules (e.g. for the list tail matching) such that the use of inaccurate reasoning rules (like the "missing" rule) is avoided in many cases. We have also introduced a more practical partial order to judge the quality of different solutions for abduction.. • We have successfully incorporated the call-by-reference mechanism into the forward analyses in the separation domain.. • We have built a prototype system to test the viability and performance of our approach, and we have conducted some initial experimental studies to evaluate the precision of the results and the scalability of our system. Preliminary results show that our tool can derive expressive specifications which fully capture the behaviors of the unknown code in many cases. Outline. Section 2 employs a motivating example to informally illustrate our main approach. Section 3 presents the programming language and the abstract domain for our analysis. Section 4 introduces our abductive reasoning. Section 5 defines two abstract semantics used in our verification. Section 6 depicts our verification algorithms. Experimental results are shown in Section 7, followed by some concluding remarks. It is a challenging problem to automatically verify (even pointer safety of) heapmanipulating imperative programs with unknown procedure calls. We propose a novel approach to this problem, which infers expected specifications for unknown procedures from their calling contexts during the verification process. The program is proven correct subject to the condition that the invoked unknown procedures meet the inferred specifications. We employ a forward shape analysis with separation logic and an enhanced abductive reasoning mechanism to synthesize both pre-and postconditions of the unknown procedure. As a proof of concept, we have also implemented a prototype system to test the viability of the proposed approach.. There are two possible future directions. One is to explore a more general solution for unknown calls in sequence as discussed, e.g., it might be possible for us to invent some heuristics to strengthen the postcondition for the first unknown call, so that the precondition of the second can be strengthened accordingly, to achieve more reasonable specifications for both. Another direction is to extend this method to an abstract domain combining separation and numerical information [7] , so that more general properties, such as memory safety and functional correctness, can be specified and verified. We envisage that, with the combined domain, the abstract semantics and analysis algorithms will remain conceivably the same, but the abduction will be redefined to discover the antiframes for the newly introduced numerical features.

O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O


Generalized likelihood ratio tests for complex fMRI data: a Simulation study Abstract-Statistical tests developed for the analysis of (intrinsically complex valued) functional magnetic resonance time series, are generally applied to the data's magnitude components. However, during the past five years, new tests were developed that incorporate the complex nature of fMRI data. In particular, a generalized likelihood ratio test (GLRT) was proposed based on a constant phase model [19] . In this work, we evaluate the sensitivity of GLRTs for complex data to small misspecifications of the phase model by means of simulation experiments. It is argued that, in practical situations, GLRTs based on magnitude data are likely to perform better compared to GLRTs based on complex data in terms of detection rate and constant false alarm rate properties.Index Terms-fMRI, generalized likelihood ratio test, magnitude data, statistical parametric maps. M AGNETIC resonance imaging is not only capable of providing excellent anatomical information. It also offers the ability to visualize functional activity in the human brain, by means of so-called functional magnetic resonance imaging (fMRI). fMRI is a technique for determining which parts of the brain are activated by different types of stimuli. From a succession of rapidly acquired images that reflect localized changes in cerebral blood flow and oxygenation, fMRI can provide detailed images of localized brain activity induced by sensory, motor, or cognitive tasks, by processing each pixel's time series. Since activation-related signal changes are of the order of 1% to 10%, the construction of such activation maps requires sophisticated statistical tests.. In the past, many techniques were proposed for the construction of activation maps, often referred to as statistical parametric maps (SPMs). Initial detection methods were based on simple subtraction of images acquired during a resting state from images acquired during activation [1] , [31] . In order to improve the detection rate, activation detection methods using the -score, correlation coefficients, analysis of variance, or Student-test statistics were employed [20] , [23] . Those tests all fit into the framework of the so-called general linear model (GLM) test Manuscript received November 17, 2004 ; revised January 6, 2005. The Associate Editor responsible for coordinating the review of this paper and recommending its publication was P. M. Thompson. Asterisk indicates corresponding author.. *J. Sijbers is with the University of Antwerp, Vision Lab, CMI, Groenenborgerlaan 171, U316, B-2020 Antwerpen, Belgium and also with the Fund for Scientific Research-Flanders, Belgium (F.W.O.), Egmontstraat 5 1000 Brussels, Belgium (e-mail: jan.sijbers@ua.ac.be).. A. J. den Dekker is with the Delft University of Technology, Delft Center for Systems and Control, 2628CD Delft, The Netherlands (e-mail: a.j.dendekker@dcsc.tudelft.nl).. Digital Object Identifier 10.1109/TMI.2005.844075 [5] . A main advancement of the GLM test was the incorporation of a hemodynamic response function. Furthermore, more flexible and general variants of the general linear model were developed such as tests based on independent component analysis [16] , [27] , multiresolution analysis [4] , and likelihood ratio tests [19] , [22] . Although magnetic resonance data are intrinsically complex valued, statistical tests are generally applied to magnitude fMRI images, since only the signal amplitude is assumed to be related to neural activation. Indeed, most of the tests available for fMRI data analysis were specifically developed for magnitude (i.e., single-valued) times series. Only recently, tests were developed for complex valued data [3] , [19] , [21] . In this respect, Nan and Nowak showed how to construct a generalized likelihood ratio test (GLRT) for complex fMRI data with constant phase values [19] . Thereby, it was shown that, under the assumption of a constant phase model, the GLRT has a significantly higher detection rate compared to tests based on magnitude data, especially in low signal-to-noise (SNR) regions.. The assumption of constant phases, however, is crucial and may be too restrictive. In this respect, it has previously been reported that phase traces may exhibit activation-dependent modulations, especially for voxels with large venous blood fractions (i.e., voxels with high blood volume fractions show a time-dependency related to the paradigm) [10] , [17] . On the other hand, since in most fMRI studies, a voxel contains a large ensemble of vessels with various orientations and sizes, the assumption of a constant phase value may still be appropriate. However, there may also be other causes for nonconstant phases as a function of time. For example, due to physiological processes or instabilities of the MR imaging system, the phase may show a slowly varying drift [12] . Similar phase behavior was found in our experimental fMRI data sets: though most data sets show constant phase values as a function of time, some of those data sets clearly revealed linear phase drifts (e.g., see Fig. 1 ).. In this paper, we take a closer look at the feasibility of the constant phase assumption underlying the GLRT proposed by Nan and Nowak [19] . Thereby, we will try to answer questions like "What if the true phase values are not truly constant but, for example, are more appropriately described by a linear phase model?" and "Would it be helpful to construct a GLRT based on a more complex phase model, or should we rather apply a GLRT based on magnitude data?".. In order to simplify the discussion (as was also done by Nan and Nowak), the noise disturbing the fMRI time series is assumed to be Gaussian white noise [13] , [15] , [19] . The white noise model may be an oversimplification of the noises inherent in fMRI (especially if physiological noise is dominant Fig. 1 . Left: phase time courses of an experimental fMRI data set. The paradigm was four "dummy" images followed by a 12 rest-12 stimulus period, repeated three times (i.e., 76 images in total). Right: the magnitude traces corresponding to the phase traces on the left (the offset of each magnitude trace was adjusted for matching with the phase trace). No significant activation was detected in the magnitude traces, except for the middle trace (offset 200). For both plots, the task paradigm is shown in the background (gray denotes stimulus).. over thermal noise). However, if the data are observed to be correlated, they can be whitened based on an estimated autocorrelation structure [2] , [19] , [29] - [30] . Moreover, valuable methods exist to eliminate, for example, physiological noise from the fMRI time courses, prior to activation detection [3] , [7] , [28] . Hence, the conclusions of this work can easily be extended to more realistic noise models that incorporate colored noise.. The organization of this paper is as follows. Section II briefly reviews the general theory for the construction of a GLRT. In Section III, the GLRTs for magnitude as well as for complex data with a constant, linear, or random phase model are derived. Next, in Section IV, simulation experiments are discussed that were set up to test the performance of the GLRTs. Finally, conclusions are drawn in Section V. Most statistical tests for detecting activation from functional fMRI data are applied to the magnitude components of intrinsically complex valued fMRI data. However, previous work has reported the observation that the phase components of an fMRI time course are virtually constant. Based on this observation, a generalized likelihood ratio test (GLRT) for complex valued fMRI data with constant phases was developed, showing significantly higher detection rates compared to standard tests in low SNR regions [19] .. However, phase values of complex fMRI time series may not be constant but for example linearly varying or related to the paradigm. Therefore, in this work, we tested the performance of the GLRT based on constant phases if a slight nonconstant behavior was introduced to the phase components. Simulation results show that, even with a small deviation from a constant baseline, the detection rate of this GLRT drops drastically, and, even worse, the GLRT looses its constant false alarm rate (CFAR) property. Hence, we conclude that a GLRT based on complex data and constant phases is very vulnerable to a misspecification of the phase model underlying the complex fMRI data.. Furthermore, a GLRT for complex data with a linear phase model was constructed. Such a test covers a much wider range of phase models, but requires the estimation of an additional parameter. It was shown that, whenever the underlying phase model was correct, this GLRT showed a slight improvement of detection rate compared to a GLRT based on magnitude data and was observed to have the CFAR property. In case the underlying phases could not be described by a linear phase model, the performance of this GLRT was observed to drop drastically as well.. In conclusion, we believe that in practice it is safer to use a GLRT for magnitude fMRI data than a GLRT for complex fMRI data, unless one is absolutely confident that the underlying phase values are constant.

O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I I I B I B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O B I I I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O