Scalable statistical bug isolation We present a statistical debugging algorithm that isolates bugs in programs containing multiple undiagnosed bugs. Earlier statistical algorithms that focus solely on identifying predictors that correlate with program failure perform poorly when there are multiple bugs. Our new technique separates the effects of different bugs and identifies predictors that are associated with individual bugs. These predictors reveal both the circumstances under which bugs occur as well as the frequencies of failure modes, making it easier to prioritize debugging efforts. Our algorithm is validated using several case studies, including examples in which the algorithm identified previously unknown, significant crashing bugs in widely used systems. This paper is about statistical debugging, a dynamic analysis for identifying the causes of software failures (i.e., bugs). Instrumented programs monitor their own behavior and produce feedback reports. The instrumentation examines program behavior during execution by sampling, so complete information is never available about any single run. However, monitoring is also lightweight and therefore practical to deploy in a production testing environment or to large user communities, making it possible to gather information about many runs. The collected data can then be analyzed for interesting trends across all of the monitored executions.. In our approach, instrumentation consists of predicates tested at particular program points; we defer discussing which predicates are chosen for instrumentation to Section 2. A given program point may have many predicates that are sampled independently during program execution when that program point is reached (i.e., each predicate associated with a program point may or may not be tested each time the program point is reached). A feedback report R consists of one bit indicating whether a run of the program succeeded or failed, as well as a bit vector with one bit for each predicate P. If P is observed to be true at least once during run R then R(P) = 1, otherwise R(P) = 0.. Let B denote a bug (i.e., something that causes incorrect behavior in a program). We use B to denote a bug profile, i.e., a set of failing runs (feedback reports) that share B as the cause of failure. The union of all bug profiles is exactly the set of failing runs, but note that B i ∩ B j = / 0 in general; more than one bug can occur in some runs.. A predicate P is a bug predictor (or simply a predictor) of bug B if whenever R(P) = 1 then it is statistically likely that R ∈ B (see Section 3.1). Statistical debugging selects a small subset S of the set of all instrumented predicates P such that S has predictors of all bugs. We also rank the predictors in S from the most to least important. The set S and associated metrics (see Section 4) are then available to engineers to help in finding and fixing the most serious bugs.. In previous work, we focused on techniques for lightweight instrumentation and sampling of program executions, but we also studied two preliminary algorithms for statistical debugging and presented experimental results on medium-size applications with a single bug [10, 16] . The most general technique we studied is regularized logistic regression, a standard statistical procedure that tries to select a set of predicates that best predict the outcome of every run. As we worked to apply these methods to much larger programs under realistic conditions, we discovered a number of serious scalability problems:. • For large applications the set P numbers in the hundreds of thousands of predicates, many of which are, or are very nearly, logically redundant. In our experience, this redundancy in P causes regularized logistic regression to choose highly redundant lists of predictors S. Redundancy is already evident in prior work [10] but becomes a much more serious problem for larger programs.. • A separate difficulty is the prevalence of predicates predicting multiple bugs. For example, for many Unix programs a bug is more likely to be encountered when many command line flags are given, because the more options that are given non-default settings the more likely unusual code paths are to be exercised. Thus, predicates implying a long command line may rank near the top, even though such predicates are useless for isolating the cause of individual bugs.. • Finally, different bugs occur at rates that differ by orders of magnitude. In reality, we do not know which failure is caused by which bug, so we are forced to lump all the bugs together and try to learn a binary classifier. Thus, predictors for all but the most common bugs have relatively little influence over the global optimum and tend to be ranked very low or not included in S at all.. These problems with regularized logistic regression persist in many variations we have investigated, but analysis of this body of experimental work yielded some key technical insights. In addition to the bug predictors we wish to find among the instrumented predicates, there are several other kinds of predicates. First, nearly all predicates (often 98% or 99%) are not predictive of anything. These non-predictors are best identified and discarded as quickly as possible. Among the remaining predicates that can predict failure in some way, there are some bug predictors. There are also super-bug predictors: predicates that, as described above, predict failures due to a variety of bugs. And there are sub-bug predictors: predicates that characterize a subset of the instances of a specific bug; these are often special cases of more general problems. We give the concepts of super-and sub-bug predictors more precise technical treatment in Section 3.3.. The difficulty in identifying the best bug predictors lies in not being misled by the sub-or super-bug predictors and not being overwhelmed by the sheer number of predicates to sift through. This paper makes a number of contributions on these problems:. • We present a new algorithm for isolating multiple bugs in complex applications (Section 3) that offers significant improvements over previous work. It scales much more gracefully in all the dimensions discussed above and for each selected predicate P it naturally yields information that shows both how important (in number of explained program failures) and how accurate a predictor P is.. • We validate the algorithm by a variety of experiments. We show improved results for previously reported experiments [10] . In a controlled experiment we show that the algorithm is able to find a number of known bugs in a complex application. Lastly, we use the algorithm to discover previously unknown serious crashing bugs in two large and widely used open source applications.. • We show that relatively few runs are sufficient to isolate all of the bugs described in this paper, demonstrating that our approach is feasible for in-house automatic testing as well as for deployment to end users (see Section 4.3).. • We report on the effectiveness of the current industry practice of collecting stack traces from failing runs. We find that across all of our experiments, in about half the cases the stack is useful in isolating the cause of a bug; in the other half the stack contains essentially no information about the bug's cause.. • Finally, we show that, in principle, it is possible for our approach to help isolate any kind of failure, not just program crashes. All that is required is a way to label each run as either "successful" or "unsuccessful.". With respect to this last point, perhaps the greatest strength of our system is its ability to automatically identify the cause of many different kinds of bugs, including new classes of bugs that we did not anticipate in building the tool. By relying only on the distinction between good and bad executions, our analysis does not require a specification of the program properties to be analyzed. Thus, statistical debugging provides a complementary approach to static analyses, which generally do require specification of the properties to check. Statistical debugging can identify bugs beyond the reach of current static analysis techniques and even new classes of bugs that may be amenable to static analysis if anyone thought to check for them. One of the bugs we found, in the RHYTHMBOX open source music player, provides a good illustration of the potential for positive interaction with static analysis. A strong predictor of failure detected by our algorithm revealed a previously unrecognized unsafe usage pattern of a library's API. A simple syntactic static analysis subsequently showed more than one hundred instances of the same unsafe pattern throughout RHYTHMBOX.. The rest of the paper is organized as follows. After providing background in Section 2, we discuss our algorithm in Section 3. The experimental results are presented in Section 4, including the advantages over our previous approach based on regularized logistic regression. Section 5 considers variations and extensions of the basic statistical debugging algorithm. We discuss related work in Section 6 and offer our conclusions in Section 7. In this section we briefly survey related work. There is currently a great deal of interest in applying static analysis to improve software quality. While we firmly believe in the use of static analysis to find and prevent bugs, our dynamic approach has advantages as well. A dynamic analysis can observe actual run-time values, which is often better than either making a very conservative static assumption about run-time values for the sake of soundness or allowing some very simple bugs to escape undetected. Another advantage of dynamic analysis, especially one that mines actual user executions for its data, is the ability to assign an accurate importance to each bug. Additionally, as we have shown, a dynamic analysis that does not require an explicit specification of the properties to check can find clues to a very wide range of errors, including classes of errors not considered in the design of the analysis.. The Daikon project [5] monitors instrumented applications to discover likely program invariants. It collects extensive trace information at run time and mines traces offline to accept or reject any of a wide variety of hypothesized candidate predicates. The DIDUCE project [7] tests a more restricted set of predicates within the client program, and attempts to relate state changes in candidate predicates to manifestation of bugs. Both projects assume complete monitoring, such as within a controlled test environment. Our goal is to use lightweight partial monitoring, suitable for either testing or deployment to end users.. Software tomography as realized through the GAMMA system [1, 12] shares our goal of low-overhead distributed monitoring of deployed code. GAMMA collects code coverage data to support a variety of code evolution tasks. Our instrumentation exposes a broader family of data-and control-dependent predicates on program behavior and uses randomized sparse sampling to control overhead. Our predicates do, however, also give coverage information: the sum of all predicate counters at a site reveals the relative coverage of that site.. Efforts to directly apply statistical modeling principles to debugging have met with mixed results. Early work in this area by Burnell and Horvitz [2] uses program slicing in conjunction with Bayesian belief networks to filter and rank the possible causes for a given bug. Empirical evaluation shows that the slicing component alone finds 65% of bug causes, while the probabilistic model correctly identifies another 10%. This additional payoff may seem small in light of the effort, measured in man-years, required to distill experts' often tacit knowledge into a formal belief network. However, the approach does illustrate one strategy for integrating information about program structure into the statistical modeling process.. In more recent work, Podgurski et al. [13] apply statistical feature selection, clustering, and multivariate visualization techniques to the task of classifying software failure reports. The intent is to bucket each report into an equivalence group believed to share the same underlying cause. Features are derived offline from finegrained execution traces without sampling; this approach reduces the noise level of the data but greatly restricts the instrumentation schemes that are practical to deploy outside of a controlled testing environment. As in our own earlier work, Podgurski uses logistic regression to select features that are highly predictive of failure. Clustering tends to identify small, tight groups of runs that do share a single cause but that are not always maximal. That is, one cause may be split across several clusters. This problem is similar to covering a bug profile with sub-bug predictors.. In contrast, current industrial practice uses stack traces to cluster failure reports into equivalence classes. Two crash reports showing the same stack trace, or perhaps only the same top-of-stack function, are presumed to be two reports of the same failure. This heuristic works to the extent that a single cause corresponds to a single point of failure, but our experience with MOSS, RHYTHM-BOX, and EXIF suggests that this assumption may not often hold. In MOSS, we find that only bugs #2 and #5 have truly unique "signature" stacks: a crash location that is present if and only if the corresponding bug was actually triggered. These bugs are also our most deterministic. Bugs #4 and #6 also have nearly unique stack signatures. The remaining bugs are much less consistent: each stack signature is observed after a variety of different bugs, and each triggered bug causes failure in a variety of different stack states. RHYTHMBOX and EXIF bugs caused crashes so long after the bad behavior that stacks were of limited use or no use at all.. Studies that attempt real-world deployment of monitored software must address a host of practical engineering concerns, from distribution to installation to user support to data collection and warehousing. Elbaum and Hardojo [4] have reported on a limited deployment of instrumented Pine binaries. Their experiences have helped to guide our own design of a wide public deployment of applications with sampled instrumentation, presently underway [11] .. For some highly available systems, even a single failure must be avoided. Once the behaviors that predict imminent failure are known, automatic corrective measures may be able to prevent the failure from occurring at all. The Software Dependability Framework (SDF) [6] uses multivariate state estimation techniques to model and thereby predict impending system failures. Instrumentation is assumed to be complete and is typically domain-specific. Our algorithm could also be used to identify early warning predicates that predict impending failure in actual use. This section describes ideas and terminology needed to present our algorithm. The ideal program monitoring system would gather complete execution traces and provide them to an engineer (or, more likely, a tool) to mine for the causes of bugs. However, complete tracing of program behavior is simply impractical; no end user or tester would accept the required performance overhead or network bandwidth.. Instead, we use a combination of sparse random sampling, which controls performance overhead, and client-side summarization of the data, which limits storage and transmission costs. We briefly discuss both aspects.. Random sampling is added to a program via a source-to-source transformation. Our sampling transformation is general: any collection of statements within (or added to) a program may be designated as an instrumentation site and thereby sampled instead of run unconditionally. That is, each time instrumentation code is reached, a coin flip decides whether the instrumentation is executed or not. Coin flipping is simulated in a statistically fair manner equivalent to a Bernoulli process: each potential sample is taken or skipped randomly and independently as the program runs. We have found that a sampling rate of 1 /100 in most applications 1 keeps the performance overhead of instrumentation low, often unmeasurable.. Orthogonal to the sampling transformation is the decision about what instrumentation to introduce and how to concisely summarize the resulting data. Useful instrumentation captures behaviors likely to be of interest when hunting for bugs. At present our system offers the following instrumentation schemes for C programs:. branches: At each conditional we track two predicates indicating whether the true or false branches were ever taken. This applies to if statements as well as implicit conditionals such as loop tests and short-circuiting logical operators. returns: In C, the sign of a function's return value is often used to signal success or failure. At each scalar-returning function call site, we track six predicates: whether the returned value is ever < 0, ≤ 0, > 0, ≥ 0, = 0, or = 0. scalar-pairs: Many bugs concern boundary issues in the relationship between a variable and another variable or constant. At each scalar assignment x = ..., identify each same-typed in-scope variable y i and each constant expression c j . For each y i and each c j , we track six relationships to the new value of x: <, ≤,>,≥, =, =. Each (x, y i ) or (x, c j ) pair is treated as a distinct instrumentation site-in general, a single assignment statement is associated with multiple distinct instrumentation sites.. All predicates at an instrumentation site are sampled jointly. To be more precise, an observation is a single dynamic check of all predicates at a single instrumentation site. We write "P observed" when the site containing P has been sampled, regardless of whether P was actually true or false. We write "P observed to be true" or merely "P" when the site containing P has been sampled and P was actually found to be true. For example, sampling a single negative return value means that all six returns predicates (< 0, ≤ 0, > 0, ≥ 0, = 0, and = 0) are "observed." However only three of them (< 0, ≤ 0, and = 0) are "observed to be true." 2 These are natural properties to check and provide good coverage of a program's scalar values and control flow. This set is by no means complete, however; in particular, we believe it would be useful to have predicates on heap structures as well (see Section 4). In this section we present the results of applying the algorithm described in Section 3 in five case studies. Table 2 shows summary statistics for each of the experiments. In each study we ran the programs on about 32,000 random inputs. The number of instrumentation sites varies with the size of the program, as does the number of predicates those instrumentation sites yield. Our algorithm is very effective in reducing the number of predicates the user must examine. For example, in the case of RHYTHMBOX an initial set of 857,384 predicates is reduced to 537 by the Increase(P) > 0 test, a reduction of 99.9%. The elimination algorithm then yields 15 predicates, a further reduction of 97%. The other case studies show a similar reduction in the number of predicates by 3-4 orders of magnitude.. The results we discuss are all on sampled data. Sampling creates additional challenges that must be faced by our algorithm. Assume P 1 and P 2 are equivalent bug predictors and both are sampled at a rate of 1 /100 and both are reached once per run. Then even though P 1 and P 2 are equivalent, they will be observed in nearly disjoint sets of runs and treated as close to independent by the elimination algorithm.. To address this problem, we set the sampling rates of predicates to be inversely proportional to their frequency of execution. Based on a training set of 1,000 executions, we set the sampling rate of each predicate so as to obtain an expected 100 samples of each predicate in subsequent program executions. On the low end, the sampling rate is clamped to a minimum of 1 /100; if the site is expected to be reached fewer than 100 times the sampling rate is set at 1.0. Thus, rarely executed code has a much higher sampling rate than very frequently executed code. (A similar strategy has been pursued for similar reasons in related work [3] .) We have validated this approach by comparing the results for each experiment with results obtained with no sampling at all (i.e., the sampling rate of all predicates set to 100%). The results are identical except for the RHYTHMBOX and MOSS experiments, where we judge the differences to be minor: sometimes a different but logically equivalent predicate is chosen, the ranking of predictors of different bugs is slightly different, or one or the other version has a few extra, weak predictors at the tail end of the list.

O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The complexity of obstruction-free implementations Abstract. Obstruction-free implementations of concurrent objects are optimized for the common case where there is no step contention, and were recently advocated as a solution to the costs associated with synchronization without locks. In this article, we study this claim and this goes through precisely defining the notions of obstruction-freedom and step contention. We consider several classes of Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 ( obstruction-free implementations, present corresponding generic object implementations, and prove lower bounds on their complexity. Viewed collectively, our results establish that the worst-case operation time complexity of obstruction-free implementations is high, even in the absence of step contention. We also show that lock-based implementations are not subject to some of the time-complexity lower bounds we present. Major chip manufacturers are changing their focus from improving the speed of individual processors to increasing parallel processing capabilities. With multi-core and multi-processor systems becoming commonplace, most computer programs will be concurrent, in some form or another. A major challenge for the theory of distributed computing is to devise efficient algorithms to cope with concurrent accesses to shared data objects. The operations of these objects would typically be implemented in software, out of more elementary primitives exported by base objects.. Traditional locking-based techniques are known to scale poorly and may induce deadlock and fault-tolerance issues. In contrast, implementations of shared objects that do not use locks require processes to coordinate without relying on mutual exclusion, thus avoiding the inherent problems of locking. The safety property typically required from these implementations is linearizability [Herlihy 1991; Herlihy and Wing 1990] ; roughly, every operation on the object should appear to take effect instantaneously between its invocation and response.. Algorithms that avoid the use of locks are often, however, inefficient. Not surprisingly, the source of the inefficiency stems from handling contended situations. In fact, it is often argued [Herlihy et al. 2003a [Herlihy et al. , 2003b ] that, although contention should indeed be expected and handled appropriately, it is rare or can be made so through the use of appropriate mechanisms for managing contention. It is thus tempting to figure out the extent to which shared object implementations can exploit this rarity. In short, this would consist in devising shared object implementations that plan for the worst (high contention) and hope for the best (no contention). A natural way to explore this approach is to look for implementations that always preserve safety (linearizability) but have their termination depend on the absence of contention, that is, ensuring (or expediting) termination when there is no contention. This article studies the inherent complexity of such implementations.. In these implementations, fast termination is expected in the absence of step contention. When there is step contention, termination might not even be ensured. An operation may return control to a higher-level entity, called a client, which might decide if and when to retry the operation and may elect to use more sophisticated base objects. The term, obstruction-free [Herlihy et al. 2003a] , has been used to coin. The Complexity of r ] has interval contention 5, point contention 4, and step contention 3; operation [i , r ] has interval and point contention 4, and step contention 1 ([i , r ] is step-contention free). Square brackets denote invocations and responses, while solid intervals denote steps on base objects. these implementations. Informally, an implementation is called obstruction-free, "if it guarantees progress for every thread that eventually executes in isolation. Even though other threads may be in the midst of executing operations, . . . " [Herlihy et al. 2003a, Page 522] .. Nothing was known on the inherent complexity of such implementations and, in fact, we are not aware of any attempt to precisely define them.. The first contribution of this article is to precisely characterize obstruction-free implementations and this, itself, goes first through formally capturing the very notion of contention. We provide the first formal definition of the concept of step contention, according to which processes should only be considered contending on the implemented shared object if they concurrently apply primitive operations on its base objects. The absence of step contention allows scenarios where other processes have pending operations on the same implemented object but are not accessing the base objects of the implementation. This is fundamentally different from alternative contention metrics: point contention [Attiya and Fouren 2003] and interval contention [Afek et al. 2002] , both counting also failed or swapped-out processes. (See the scenario presented in Figure 1 .) Unlike interval contention or point contention, a process encounters step contention on an implemented object only if other processes concurrently apply primitive operations on its base objects. Clearly, implementations that would guarantee termination only in the absence of interval (or point) contention might not be robust: failed (or swapped-out) processes can be used as an excuse for other processes to never terminate. An implementation might not tolerate a single process failure while being obstruction-free with respect to interval contention for instance. In short, obstruction-freedom with respect to point or interval contention do not preclude the use of locking. Our goal is to study the inherent complexity of implementations that are robust (in particular, those who guarantee global progress and thus preclude locking), yet that exploit the rarity of step contention.. If a client application is not willing to wait indefinitely for operation completion, it may be beneficial for an obstruction-free implementation to return a fail indication to the client in the face of step contention. By doing that, the implementation enables the client to choose whether to re-invoke the same operation or to invoke another one. We show however, by reduction to the (wait-free) consensus impossibility [Fischer et al. 1985; Loui and Abu-Amara 1987] , that there is an inherent uncertainty as to whether the operation could have had an effect on the object or not (Theorem 1).. When the implementation uses primitives that cannot solve consensus, for example, reads and writes, this implies that the implementation cannot be guaranteed to return either a legal response or a fail indication. 1 Several classes of obstruction-free implementations can be considered, and these differ based on how step contention is handled once detected and what base objects are available. We precisely define each class, we present corresponding generic object implementations and we give lower bounds on their complexity.. To measure the performance of obstruction-free implementations, we study their step contention-free complexity: this is the number of steps taken by a process running alone, until it returns a value. By focusing on the cost of the implementations in executions where no process encounters step contention, we capture here the complexity of obstruction-free implementations in situations where obstructionfreedom is expected to be especially beneficial.. We first study obstruction-free implementations that are restricted to use only overwriting or trivial primitives (e.g., write, swap and read). We present a generic obstruction-free object implementation which has a linear step contention-free complexity and uses a linear number of read/write objects (Theorem 10). We show that these complexities are asymptotically optimal, by noticing that they are subject to the lower bound of Jayanti et al. [2000] (Observation 1). This implies that obstruction-free implementations of perturbable objects, for example, fetch&add, modulo-b counter (for b ≥ 2n), b-valued compare&swap (for b ≥ n), and LL/SC bits, using overwriting or trivial primitives must have (n) step contention-free complexity and must use (n) base objects.. We then study a class of obstruction-free implementations that are restricted to use simple primitives when there is no step contention, but might use more powerful primitives when there is. More specifically, these are wait-free linearizable object implementations that use only (cheap) overwriting or trivial primitives when there is no step contention, but may fall back on more powerful (and expensive) primitives like compare&swap, when step contention occurs. Such implementations are also called solo-fast in the sense that termination is faster when there is no step contention. We give a solo-fast implementation of consensus, which we believe is interesting in its own right, and derive a generic solo-fast implementation of any object type (Theorem 2). This generic implementation has a linear step contention-free complexity. We prove that such implementations must indeed have non-constant step and space complexity, that is, they are also inherently expensive. Specifically, an n-process solo-fast implementation of a perturbable object, using only overwriting or trivial primitives in step-contention free executions, must have (n) space complexity (Theorem 4) and (log n) step contention-free complexity (Theorem 5).. We finally consider obstruction-free implementations that use powerful primitives even when there is no step contention. We prove that, for any obstruction-free implementation of binary consensus, there exist executions where no process is aware of step contention, in which either some process accesses √ n distinct base. The Complexity of Obstruction-Free Implementations 24:5 objects while performing a single operation, or some process incurs √ n memory stalls [Dwork et al. 1997 ] (Theorem 6). The stalls metric captures the fact that when process p applies a nontrivial primitive to base object r right after k distinct processes other than p apply nontrivial primitives to r , then p suffers a delay of length proportional to k. This result holds also for implementations of any perturbable object (Theorem 7). Since these results are obtained in executions where none of the processes is aware of step contention, they hold for any obstructionfree implementation of these objects, regardless of how processes behave if they become aware of step contention. We also prove that, in any obstruction-free implementation of a perturbable object in which processes are not allowed to fail their operations, the number of memory stalls incurred by some process that is unaware of step contention is (n) (Theorem 9). We show that lock-based implementations of perturbable objects are not subject to these bounds, thus establishing a separation between the worst-case operation time-complexity of obstruction-free and lock-based implementations.. To summarize, the contributions of this article are the following: We define the notion of step contention and use it to define obstruction-free implementations (Section 2), and show that a obstruction-free consensus implementation from registers cannot return just legal values or fail (Section 3). We then study the complexity of different classes of obstruction-free implementations that use strong synchronization primitives only in the presence of step contention (Section 4) as well those that can use these primitives even in the absence of step contention (Section 5). This article studies obstruction-free implementations by defining their specification, presenting generic object implementations, and proving lower bounds on their complexity. Our lower bounds concentrate on the cost in uncontended executions (which are argued to be the most frequent in practice); some of them do not restrict the behavior of the processes in contended situations where processes might be using locks, randomization or other expensive mechanisms. By measuring the complexity of the weakest form of implementations that do not use locks known to date, our results capture the seemingly inherent cost of avoiding locking.. We have established tight bounds on the cost of generic obstruction-free implementations. There might be more efficient obstruction-free solutions for specific problems. For obstruction-free consensus, for example, an ( √ n) lower bound on the number of registers (or historyless objects) was proved by Fich et al. [1998] . There is however a gap since the upper bound is O(n); moreover, these results do not bound the step contention-free complexity of obstruction-free consensus. It would be interesting to close the gap between these upper bounds and the logarithmic lower bound we prove for the step contention-free complexity of any generic solo-fast implementation. Luchangco et al. [2003] presented a generic object implementation that uses only constant number of reads and writes when an operation runs in the absence of point contention. Unfortunately, in their implementation this also means lack of pending operations; moreover, an operation invoked after a prefix with point contention may have to apply C&S primitives, even if it does not encounter any point contention itself. In this article, we gave a generic solo-fast implementation. Our solo-fast implementation performs O(n) reads and writes, even in the absence of step contention. By employing adaptive collect [Afek et al. 1999; Attiya and Fouren 2003 ], the step complexity can be made to depend only on the point contention, so it is constant when there is no point contention. By employing adaptive collect for unbounded concurrency [Gafni et al. 2001] , it can be made independent of the number of processes. Aguilera et al. [2006] proposed the concept of abortable objects that are, at first glance, similar to our obstruction-free objects. Abortable objects are live: operations on them return either matching responses or a predefined ⊥ value. To prevent trivial implementations and ensure some level of progress in face of failures, they satisfy an additional nontriviality requirement that ⊥ can only be returned in the presence of interval contention, and in the eventual absence of step contention, repeated invocations may return ⊥ only finitely many times. As a result, a suspended operation may prevent a concurrent operation from making progress only for a bounded period of time. Aguilera et al. [2006] argue that if ⊥ is allowed to be output only in the presence of step contention, then the resulting implementations may not preserve correctness under composition: by replacing base object of a correct (linearizable, live and nontrivial) implementation with its correct implementation from some finer base objects, we may not obtain a correct implementation from the finer base objects. As we show in the appendix, composability can be achieved at the expense of a slightly stronger definition of obstruction-freedom based on step contention. We propose a definition of a live object that is only allowed to return ⊥ in the presence of step contention with an additional guarantee that an operation may only take effect within the interval of one of its occurrences. We also provide a generic implementation of such objects from read-write objects.

O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A methodology for quality-based mashup of data sources The concept of mashup is gaining tremendous popularity and its application can be seen in a large number of domains. Enterprises using and relying upon mashup have improved their mass collaboration and personalization. In order for mashup technology to be widely accepted and widely used, we need a methodology by which can make use of the quality of the input to the mashup process as a governing principle to carry out mashup. This paper reviews the concept of mashup in different domains and proposes a conceptual solution framework for providing quality based mashup process. Over the past few years, the adaptation of new web-related technologies has been going on at a rapid rate. The emergence of Web 2.0, the application and integration of Web 2.0 based technologies has opened up new and innovative ways of collaboration and research challenges that needs to be address. Among the suite of Web 2.0 technologies is mashup, which is defined as a combination of openly published APIs (Application Programming Interface), content (eg. data tables), and application components (eg. Web services) to extract information from several different sources, which are combined, and revisualized in other forms (eg., twitter with Google Maps to view images on the maps) [14] .. Mashup can be used by people from who are normal internet users (for data mashup) to professionals or application developers. Additionally, the application of mashup can be seen in different domains. A case in point is the VARIAZIONI project, where in semantic content enriching is carried out on a collaborative basis, which would be useful for non-technical users who can easily catalogue the existing digital assets, comprising multimedia master classes audio/video concerts, images, scanned documents, etc and create new contents from the existing contents of other institutions [1] . Mashup architectures extending SOA model are have been used [2] to facilitate service composition in the form of mashup component model and to help developers leverage to create their own composite services. Jin et al [3] have provided some directions for building both simple as well as sophisticated mashup-based applications.. As the number of individuals and organizations utilizing the mashup technology is increasing the crucial issues is to develop a methodology for quality based mashup process. In this paper we present a conceptual overview of a methodology for mashup which is governed or driven by the quality of the underlying data source. In the context of this paper, we refer to mashup of various data sources into a larger information set. However, this method can be equally applicable to services as well. This paper is divided into four sections. In Section 2 we summarize and discuss the general approaches and applications of mashup. In Section 3 we will present a conceptual solution framework to enable quality based mash up processes. Finally, Section 4 concludes the paper and sets the ground for future work. In this paper we carried out a review of the existing literature on mashup. We discussed and pointed out that mashup has different applications in different domains. Additionally, we discussed that in the current literature there is no methodology by which the quality of the inputs to the mashup process can be determined.. Once the quality of the inputs to the mashup process has been determined the user can subsequently choose those inputs that are of high quality, thereby resulting in a higher quality output.. We proposed a conceptual solution that can address this issue. We intend to continue working on this issue along many directions. One direction would be to develop the precise mathematical measures and metrics for determining and accessing quality. Another direction would be to find out how the QBE can be best organized and hosted and how the quality information in QBE can be organized and stored. Another additional direction that we intend to work on is to develop a fuzzy-mechanism for search results retrieval during the user query process. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.. iiWAS2008, November 24-26, 2008 Vancea et al [10] explore the application of mashup technology in databases. Due to the powerful features of the database systems, the authors propose a web mashup process that is supported by integration and synchronization at the database level. This is an interesting aspect because traditionally the source of the contents used for mashup process was from feeds or API-based web services, which have limitations such as working with complex data structures and different APIs for each web server that needs different implementations for mashup users. The authors proposed a generic proxy mechanism for data integration and synchronization to support database-driven web mashup architecture [10] . Nakano et al [11] have proposed a method for creating web services from web applications. They proposed an effective method of creating wrappers that makes Web applications usable as Web services. This method extracts important segments, such as search results of a hotel search application, from an HTML document generated by the web application and generates extraction rules for the wrappers.

O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O B O O O O O O B O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O B O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O B O O O O O B O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Bounds for the MSE performance of constant modulus estimators The constant modulus (CM) criterion has become popular in the design of blind linear estimators of sub-Gaussian independent and identically distributed (i.i.d.) processes transmitted through unknown linear channels in the presence of unknown additive interference. In this paper, we present an upper bound for the conditionally unbiased mean-squared error (UMSE) of CM-minimizing estimators that depends only on the source kurtoses and the UMSE of Wiener estimators. Further analysis reveals that the extra UMSE of CM estimators can be upper-bounded by approximately the square of the Wiener (i.e., minimum) UMSE. Since our results hold for vector-valued finite-impulse response/infinite-impulse response (FIR/IIR) linear channels, vector-valued FIR/IIR estimators with a possibly constrained number of adjustable parameters, and multiple interferers with arbitrary distribution, they confirm the longstanding conjecture regarding the general mean-square error (MSE) robustness of CM estimators. In this paper we have derived, for the general multisource linear model of Fig. 1 , two closed-form bounding expressions for the UMSE of CM-minimizing estimators. The first bound is based on the kurtosis of the MMSE estimates, while the second is based on the UMSE of the MMSE estimators. Analysis of the second bound shows that the extra UMSE of CM-minimizing estimators is upper-bounded by approximately the square of the minimum UMSE. Thus the CM-minimizing estimator generates nearly MMSE estimates when the minimum MSE is small. Numerical simulations suggest that the bounds are tight (with respect to the performance of CM-minimizing estimators designed by gradient descent).. This work confirms the long-standing conjecture (see, e.g., [4] and [5] ) that the MSE performance of the CM-minimizing estimator is robust to general linear channels and general (multisource) additive interference. As such, our results supersede previous work demonstrating the MSE robustness of CM-minimizing estimators in special cases (e.g., when only AWGN is present, when the channel does not provide adequate diversity, or when the estimator has an insufficient number of adjustable parameters). In this section, we give more detailed information on the linear system model and the MSE, unbiased MSE, and CM criteria. The following notation is used throughout: denotes transpose, conjugate, Hermitian, and Moore-Penrose pseudo-inverse. Likewise, denotes expectation, the -norm defined by the -norm defined by for positive semidefinite , the identity matrix, the -transform, and the field of nonnegative real numbers. In general, we use boldface lowercase type to denote vector quantities and boldface uppercase type to denote matrix quantities.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fast distributed algorithms for (weakly) connected dominating sets and linear-size skeletons Motivated by routing issues in ad hoc networks, we present polylogarithmic-time distributed algorithms for two problems. Given a network, we first show how to compute connected and weakly connected dominating sets whose size is at most O(log ) times the optimum, being the maximum degree of the input network. This is bestpossible if NP DTIME[n O(log log n) ] and if the processors are required to run in polynomial-time. We then show how to construct dominating sets that have the above properties, as well as the "low stretch" property that any two adjacent nodes in the network have their dominators at a distance of at most O(log n) in the output network. (Given a dominating set S, a dominator of a vertex u is any v ∈ S such that the distance between u and v is at most one.) We also show our time bounds to be essentially optimal. We present fast distributed algorithms for computing connected and weakly connected dominating sets with good "low stretch" properties, in a given distributed network. Our motivation comes from the study of ad hoc wireless networks. A crucial way in which these differ from current cellular networks is that they do not have a separate routing infrastructure such as a system of base-stations; the mobiles have to conduct their own communication through routing. In these networks it is necessary to set up a so-called backbone, i.e., a set of vertices and links among them that is in charge of routing. In the specialized literature there is a general consensus that the backbone should be a dominating set, i.e., each vertex is either in the backbone or next to some vertex in it. To quote Rajaraman [15] , "The most basic clustering that has been studied in the context of ad hoc networks is based on dominating sets." Moreover, the following additional features are considered to be appealing: (a) the backbone should be "small" and (b) it should be connected or weakly connected. Computing small connected dominating sets has been the focus of many papers. A good starting point is the comparative analysis of [5] with its comprehensive bibliography.. A dominating set D is connected if the subgraph induced by it is connected, while D is weakly connected if the graph induced by the stars of vertices of D is connected (the star of vertex u is the neighborhood of u including u itself). An equivalent definition is the following: connect every two vertices of D that are at distance 1 or 2. If the resulting graph (with vertex set D) is connected, then D is weakly connected. While connectivity appears to be a natural requirement, several authors have argued that the right notion to apply in the wireless context is weak connectivity (see for instance [6] ). As a reasonable first approximation for many situations of interest, wireless networks can be modeled as message-passing, synchronous networks: vertices are processors, edges are communication links and the network is synchronous. Communication proceeds in synchronous rounds: in each round, every vertex sends messages to its neighbors, receives messages from its neighbors, and does some local computation. In this model, the running time is the number of communication rounds. In this fashion, communication is charged for while local computation is not. This simplification is justified since the former is orders of magnitude greater than the latter. This is the standard model assumed in this paper. What does it mean to "compute" a subset S of the vertices, such as a (weakly) connected dominating set, in this model? What we aim for is an efficient distributed algorithm, at the end of which each vertex knows whether it is a member of S or not. Henceforth, we denote by n the number of vertices of the network and by its maximum degree.. There are several papers describing distributed algorithms for computing (weakly) connected dominating sets in our model. These algorithms fall into two categories. The first category consists of algorithms that are fast, but provide no guarantee that the dominating set they return will be small. The algorithm of [18] falls into this category. The second category consists of algorithms that are guaranteed to compute a "smallest possible" dominating set, but require polynomial number of rounds in the number of vertices of the network (see [17] for an analysis of several known dominating set algorithms). By "smallest possible" we mean an O(log )-approximation; this is the best one can hope for, assuming that each vertex performs a polynomial-time-bounded computation during every communication round and that NP ⊆ DTIME[n O(log log n) ] [7, 13, 16] . (Logarithms in this paper are to the base two, unless specified otherwise.) Algorithms in this category are typically distributed implementations of sequential approximation algorithms such as the ones in [8] . There are also quite nice distributed algorithms for computing "best possible" dominating sets in polylogarithmically many rounds ( [9] and, with some modifications, [14] ). But these dominating sets are, in general, neither connected nor weakly connected.. In this paper, we present several randomized distributed algorithms that compute the "smallest possible" connected and weakly connected dominating set in polylogarithmically (in n) many rounds. As above, by "smallest possible" we mean dominating sets of size O(log ) · Opt. In fact, our guarantee is in terms of the smallest, not necessarily (weakly) connected, dominating set; since the smallest dominating set has size at most that of a smallest (weakly) connected dominating set, this suffices.. Our results are based on a technique that might be of independent interest. We show that given a graph G that is connected, G can be sparsified by deleting all but a linear number of edges. Crucially, the resulting graph stays connected. This is interesting in view of two facts. First, as shown by Proposition 1, there is no deterministic or randomized algorithm running in o(n) rounds that is capable of removing even one edge and preserving connectivity. In particular, this implies that there are no o(n)-round protocols for computing spanning or Steiner trees. Such trees are the tool of choice for connecting up a dominating set D while at the same time introducing only linearly many (in the size of D) new vertices. Second, the result is also interesting because real networks sometimes are bound to have a superlinear number of edges. Indeed, the following model has been proposed as a cost-effective way to set up sensor networks [19] . The sensor networks are distributed essentially at random over the area of interest that can be assumed to be the unit square; each chip is a radio transmitter that sets up a connection to the k nearest chips. It is known that if one wants the network to be connected then k must be (log n) [19] . Therefore the resulting network will have a superlinear number of edges and the same might very well hold for the graphs induced by dominating sets in the graph. Furthermore, Proposition 11 shows the running time of O(log n) for our sparsification, to be best-possible.. In addition to domination, (weak) connectivity and small size, there is yet another feature of the backbone that would be attractive. Since the vertices in the dominating set take care of routing, it is desirable that dominators of "nearby" vertices are "relatively nearby". In particular, we show that our above results can be strengthened as follows. In Section 3, we present a polylogarithmic-time randomized distributed algorithm to construct a connected dominating set S of size at most O(log ) times the optimum, with the following additional "low stretch" property: for any pair of nodes u and v in G, the distance (in the subgraph induced by S) between their respective dominators is at most O(log n) times the distance between u and v in G. To our knowledge, this is the first result of this kind in this distributed setting.. Thus we present fast distributed algorithms for constructing dominating sets with good size, connectivity, and stretch properties; our running times are also shown to be essentially best-possible. We remark that in the case of the unit-disk graph, which is sometimes considered to be a good model of ad hoc networks, our algorithms deliver essentially a 28-approximation rather than an O(log n)-approximation (to be precise, the approximated solution is of size at most 28 · Opt + 7). Finally, one of our algorithms (Algorithm 2 in §2.2) admits a straightforward asynchronous implementation. Recent experimental work has shown that the algorithms described in this paper perform quite well in practice for ad hoc networks [5] .

O B I O O O B I O O O O O B O O B I I O O O B I O O O O O B O O O O O O O O O O B I O B O O O O O O O O O O O O O O B B O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O


Exploitation of unlabeled sequences in hidden markov models Abstract-This paper presents a method for effectively using unlabeled sequential data in the learning of hidden Markov models (HMMs). With the conventional approach, class labels for unlabeled data are assigned deterministically by HMMs learned from labeled data. Such labeling often becomes unreliable when the number of labeled data is small. We propose an extended Baum-Welch (EBW) algorithm in which the labeling is undertaken probabilistically and iteratively so that the labeled and unlabeled data likelihoods are improved. Unlike the conventional approach, the EBW algorithm guarantees convergence to a local maximum of the likelihood. Experimental results on gesture data and speech data show that when labeled training data are scarce, by using unlabeled data, the EBW algorithm improves the classification performance of HMMs more robustly than the conventional naive labeling (NL) approach. O NE major problem in designing classifiers is the scarcity of training data. Usually, a classifier is trained on pairs of observed feature vectors and their class labels. Such a framework is called supervised learning. In most cases, class labels are manually assigned by experts. Therefore, it is expensive and time consuming to collect large amounts of labeled data. Because of this labeling cost, data is often scarce in practice. Consequently, the designed classifier becomes unreliable and its generalization performance becomes poor [1] , especially in nonlinear models.. To overcome this problem in supervised learning, a new learning scheme called semisupervised learning has been proposed in which unlabeled data are also used to train classifiers. Since unlabeled data can be easily collected without labeling efforts, semisupervised learning has attracted classifier designers and has been studied in various applications for both static data [2] , [3] , [4] , [5] , [6] and sequential data [7] , [8] . It was reported that the classifiers learned from both labeled and unlabeled data could achieve better classification performance than those learned from small amounts of labeled data.. In this paper, we focus on semisupervised learning for hidden Markov models (HMMs). HMMs are stochastic state transition models that have been extensively used in two types of applications. The first one is concerned with classification of sequences in speech recognition (e.g., [9] ), in gesture recognition (e.g., [10] ), in computational biology (e.g., [11] ), etc. In these tasks, given a sequence, HMMs assign a class label to the entire sequence. The second type deals with the determination of state sequences given observation sequences. Examples of this type of application include part of speech tagging in natural language processing (e.g., [12] ) and named entity extraction in information extraction (e.g., [13] ). In the second type of application, the term "labeled data" means the observed sequences with the state sequence information associated with them, while the term "unlabeled data" means the sequences without the state sequence information [14] , [15] , [16] . That is, the second one is concerned with "partially hidden data" which are not "unlabeled data" in the sense used for the semisupervised learning for static data. Such "partially hidden data" of the second application type can be processed by the standard learning framework of HMMs and, in this paper, we investigate the "unlabeled data" in the first type of application, the classification of sequences.. For the first applications, a simple semisupervised learning scheme has been used, which we refer to as the naive labeling (NL) approach [7] , [8] . With the NL approach, HMMs are first trained solely on given labeled data. Then, pseudoclass labels are deterministically assigned to unlabeled data by classifying them using the trained HMMs. The HMMs are retrained with these newly labeled data.. The NL approach appears to be a method for classifier adaptation under the assumption that the initial model is to some extent reliable. In the above two studies where the quantities of initial labeled data were relatively large, the NL approach could improve the HMMs. This applies to the case when training HMMs used in speech recognition systems for adaptation. However, when trained on small amounts of labeled data, initial models become unreliable and, therefore, the pseudolabels also become unreliable. As a result, the addition of unlabeled data with such unreliable labels may not improve the generalization performance of the HMM.. To overcome this problem associated with the NL approach, in this paper, we present a new semisupervised learning approach that can use unlabeled data for training HMMs more effectively than the NL approach. In our approach, as with [3] , [4] , [5] , [6] , the class labels are treated as missing information and the pseudoclass labels are probabilistically assigned to unlabeled data so that the joint likelihood function for both labeled and unlabeled data is maximized. To handle unlabeled data, we introduce extended tied-mixture HMMs (ETM-HMMs) as a mixture of tied-mixture HMMs (TM-HMMs) [17] , [18] . For training ETM-HMMs, we derive an extended Baum-Welch (EBW) algorithm. Unlike the NL approach, the proposed algorithm theoretically guarantees convergence to a local maximum of the likelihood.. The EBW algorithm can be regarded as an extension of the conventional labeling approach for static data based on the EM algorithm [19] to the one for sequential data. Although the usefulness of static unlabeled data has been claimed, the usefulness of sequential unlabeled data in such approach has not been shown. We presented an outline of our approach in a previous paper [20] . In the present paper, we formally explain the EBW algorithm and empirically compare it with the NL approach.. The rest of the paper is organized as follows: After the formal definition of labeled and unlabeled data in Section 2, the conventional NL approach is explained in Section 3. Section 4 briefly reviews TM-HMMs and introduces ETMHMMs. Next, the EBW algorithm is presented in Section 5. Section 6 provides some experimental results using gesture and speech data in which the effect of unlabeled data in our method is evaluated and compared with the NL approach. Section 7 concludes the paper. In this paper, we proposed the EBW algorithm to enable the learning of HMMs from both labeled and unlabeled sequential data. Conventionally, in HMM learning, unlabeled sequences have been used heuristically by the NL approach without the guarantee of convergence. In contrast, in the EBW algorithm, the parameter reestimation formulae have been formally derived in the framework of the EM algorithm. We also found that our method utilized unlabeled sequences more effectively than the NL approach in terms of classification performance. Two experimental results on gesture data and speech data showed that the EBW algorithm reduced the classification errors in most cases in contrast to the NL approach.. Although when the initial labeled data were scarce, our method could compensate for the insufficiency of labeled training data by using unlabeled data, when the initial labeled data were sufficient, the EBW algorithm sometimes had a detrimental effect on the classification performance. This is a limitation of our approach and the situations in which unlabeled sequences do not help should be studied further. The reason why adding unlabeled data could not monotonically improve the performance can probably be explained based on the analysis in [28] .. In future work, we can apply our method for adaptation where unseen test data whose properties are different from those of training data can be regarded as unlabeled data. Furthermore, there exists a more sophisticated NL approach called cotraining [29] , in which feature vectors need to be separated into two feature sets, each of which is capable of learning a classifier. When we have such a redundant data set, it is interesting to compare the NL approach, the EBW algorithm, and cotraining.

O O O O O B I I O O O O O O O O O B I O O O O B I I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Extending record typing to type parametric modules with sharing We extend term unification techniques used to type extensible records in order to solve the two main typing problems for modules in Standard ML: matching and sharing. We obtain a type system for modules baaed only on well known unification problems, modulo some equational theories we define. Our formalization is simple and has the elegance of polymorphic type disciplines based on unification.It can be seen as a synthesis of previous work on module and record typing. Building programs from modules can be safely done only if modules can be connected coherently.. The difficulty of module coherence checking depends on the complexity and power of the module system considered. The SML module system [5, 7] is, among the modular programming languages with decidable checking, the most expressive existing one. SML modules allow multiple but consistent views of the same module (sharing), and parametrization of a module (functors) by a module specification (signatures), and they can be typed statically.. Moreover, the module language itself is largely independent of the SML core language, and therefore provides a starting point to build module systems for many different programming languages.. Previous work on the static semantics of SML modules had been based on special-purpose term algebras to represent modules and on specific and complex static semantic notions [3, 7, 6 ], or does not address sharing nor multiple views of modules [4] . imposed on inferred types in order to reject defective signatures.. In our system, these conditions are ensured directly by the unification process.. We adapt the algebra of extensible record types proposed by R6my [8, 10] to encode the types of modular objects, and extend its equational theory to perform unification on the terms obtained.. The two main problems in module checking -matching a structure against a signature; finding types for signatures containing sharing constraints -are then expressed as unification problems modulo certain equational theories. We show that both problems have a principal solution whenever they have a solution.. We then reformulate the module checking problem as a type system using these results. For this type system, we easily obtain a result on principal signatures, the equivalent of the principal type result for the ML core language. Finally, we. show that principal signatures are no more than principal solutions for the problem of typing a signature by unification.. We start by presenting an overview of the typing problems related to SML modules. In section 3 we present the module language.. In section 4 we discuss the analogies between record types and modules and present R&my's extensible records and our language of module types.. In the next two sections we give our solutions for the mat thing and the sharing problems.. In section 7 we present a type system for modules and results on principality, consistency and defective signatures.

O O O O O B O O B O O O B O O O O O B O B O O O O O O O O O O O B B O B O O O B I O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O B O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O B O B O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O