Lower bounds for online makespan minimization on a small number of related machines Abstract In online makespan minimization, the jobs characterized by their processing time arrive one-by-one and each has to be assigned to one of the m uniformly In this article we prove new lower bounds for online makespan minimization for a small number of uniformly related machines. The competitive ratio (as a function of the number of machines) for this fundamental problem is upper bounded by a constant but there has been little progress on determining its exact value, as well as the values for a fixed m.. The instance of this problem consists of a sequence of machines with possibly different speeds and a sequence of jobs specified by their processing times. A schedule assigns each job to one of the machines; the time needed to process a job is equal to its processing time divided by the speed of the machine where it is assigned. The objective is to minimize the makespan (also called the length of the schedule, or the maximal completion time). Usually a schedule also needs to specify the timing of each job (its starting and completion times) so that the jobs on each machine do not overlap.. Due to the simplicity of the problem we consider, this is not necessary and it is sufficient to specify the assignment to the machines, silently assuming that each job is started as soon as all the previous jobs on its machine are processed. Instead of calculating the completion times individually for each job, we can calculate the completion time of each machine as the total processing time of the jobs allocated to it divided by the speed of the machine; the makespan is then the maximum of the completion times over all machines.. In the online version of the problem, jobs appear online one-by-one. When a job appears, an online algorithm has to make an irrevocable decision and assign the job to a machine. This decision is permanent and made without the knowledge of the future jobs; the algorithm is not even aware of whether any future jobs exist or not. An online algorithm is R-competitive if for each instance it produces a schedule with makespan at most R times the optimal makespan.. We prove lower bounds of 2.141391 for m = 4 and 2.314595 for m = 5. Our lower bound is based on an instance where the processing times are a geometric sequence, similarly as in previous works (Berman et al 2000; Epstein and Sgall 2000; Ebenlendr and Sgall 2012) . The speeds are chosen so that any online algo-rithm can use only the two fastest machines, called the active machines. The bound is then obtained by carefully analyzing the possible patterns of scheduling the jobs on these machines.. Generalizing this to larger values of m, we use computer search for elimination of possible patterns and give instances with up to 5 active machines.. Our and previous bounds are summarized in Ta We have shown new lower bounds for online makespan scheduling on a small number of uniformly related machines. In contrast to the recent asymptotic bound from (Ebenlendr and Sgall 2012) , for small m we are able to take an advantage of the combinatorial structure of the problem and of the fact that even for the optimal online algorithm (pattern), the completion times of the active machines will be uneven. The gaps between the current lower and upper bounds are still significant and existing methods are limited to using geometric sequences of processing times for the lower bounds and doubling techniques for the algorithms. Finding new techniques seems necessary for further progress. It is known that for two and three machines, the tight bounds are 1.618 and 2 respectively (see the next paragraph). Two other lower bounds for a small number of machines were 2.2880 for m = 6 and 2.4380 for m = 9. by Berman et al (2000) . That paper is not interested in the exact number of machines, as its focus is on the overall bounds, and consequently their choice of the speeds of machines is not optimal. Their lower bounds can be modified using our Lemma 1 to work for m = 5. and m = 7, respectively; see an explanation at the end of Section 2. For an arbitrary (large) number of machines, the current lower bound is 2.5648 (Ebenlendr and Sgall 2012) , the only improvement of the mentioned lower bound of 2.4380 for m = 9.. Naturally, the lower bounds need to be compared to the existing algorithms. For a small number of machines the best currently known algorithm is the greedy List Scheduling (LS). Here List Scheduling is defined so that the next job is always scheduled so that it will finish as early as possible. Its competitive ratio for m = 2 is exactly φ ≈ 1.618, the golden ratio, and for m ≥ 3 it is at most 1+ (m − 1)/2; this bound is tight for 3 ≤ m ≤ 6 (Cho and Sahni 1980) . Moreover, for m = 2, 3 it can be checked easily that there is no better deterministic algorithm. For m = 2 it is even possible to give the exact optimal ratio for any speed combination and it is always achieved by greedy List Scheduling (Epstein et al 2001) . Already for three machines, it is not known exactly for which speed combinations List Scheduling is optimal, even though we know it is optimal in the worst case. Some recent progress is reported in (Cai and Yang 2011; Han et al 2011) . Another special case when some partial results about optimality of List Scheduling are known is the case when m − 1 machines have the same speed, see, e.g., Han et al (2011); Musitelli and Nicoletti (2011) .. For an arbitrary (large) number of machines, the greedy algorithm is far from optimal: its competitive ratio is Θ(log m) (Aspnes et al 1997) . The first constantcompetitive algorithm for non-preemptive scheduling on related machines was developed in (Aspnes et al 1997) . The currently best algorithms are 3 + √ 8 ≈ 5.828-competitive deterministic and 4.311-competitive randomized ones (Berman et al 2000) . For an alternative very nice presentation see (Bar-Noy et al 2000) . All these algorithms use doubling, i.e., strategies that work with some estimate of the optimal makespan and when it turns out that the estimate is too low, it is multiplied by 2 or some other constant. While this is a standard technique for obtaining a constant competitive ratio, it would be surprising if it led to optimal algorithms. Very little is known for randomized algorithms in addition to the 4.311-competitive algorithm mentioned above. For an arbitrary m, the current lower bound is 2, see (Epstein and Sgall 2000) . For two machines, we know that the best randomized algorithm has competitive ratio between 1.5 and 1.527 (Epstein et al 2001) .. We are not aware of any other studies of randomized algorithms for online makespan minimization on related machines.. The problem of non-preemptive scheduling can be formulated in the language of online load balancing as the case where the jobs are permanent and the load is their only parameter corresponding to our processing time. Consequently, there are many results on load balancing that extend the basic results on online scheduling in a different direction, see e.g. (Azar 1998) .. It is interesting to compare our results to the related problem of preemptive scheduling on uniformly related machines. In that problem, each job can be divided into several pieces that may be scheduled on different machines so that the corresponding time slots are non-overlapping; the time slots may also be nonconsecutive, so that at some times a partially completed job is not being processed at all. Contrary to our (nonpreemptive) problem, with preemption it is necessary to specify the schedule completely, including the timing information. In the online version, for each job we have to specify its schedule completely before the next job is revealed. Interestingly, for this preemptive problem, it is possible to provide an optimal online algorithm for any combination of the speeds and its competitive ratio is between 2.1163 and e ≈ 2.7182, see (Ebenlendr et al 2009; Ebenlendr 2011) . Similar results seem to be out of reach for non-preemptive scheduling, as the combinatorial structure is much more difficult and the value of the optimum is NP-hard to compute, while for the preemptive scheduling it is efficiently computable, in fact. given by an easy formula.. The state of the art should be also compared with makespan minimization on identical machines (i.e., the special case with all the speeds equal). There the competitive ratio of List Scheduling is known to be equal to 2−1/m (Graham 1966) , and it is optimal for m = 2 and m = 3 (Faigle et al 1989) . Better algorithms are known for m = 4 (Chen et al 1994) and larger (Galambos and Woeginger 1993) . For an arbitrary m, the optimal competitive ratio is between 1.88 (Rudin III 2001) and 1.92 (Fleischer and Wahl 2000) .

O O O O B O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A unified model for metasearch, pooling, and system evaluation We present a unified model which, given the ranked lists of documents returned by multiple retrieval systems in response to a given query, simultaneously solves the problems of (1) fusing the ranked lists of documents in order to obtain a high-quality combined list (metasearch); (2) generating document collections likely to contain large fractions of relevant documents (pooling); and (3) accurately evaluating the underlying retrieval systems with small numbers of relevance judgments (efficient system assessment). Our approach is based on the Hedge algorithm for on-line learning. In effect, our proposed system "learns" which documents are likely to be relevant from a sequence of on-line relevance judgments. In experiments using TREC data, our methodology is shown to outperform standard methods for metasearch, pooling, and system evaluation, often remarkably so. We have shown that the Hedge algorithm for on-line learning can be adapted to simultaneously solve the problems of metasearch, pooling, and system evaluation, in a manner which is both efficient and highly effective. In the absence of relevance judgments, Hedge produces metasearch lists whose quality equals or exceeds that of benchmark techniques such as CombMNZ and Condorcet. While in the presence of relevance judgments, the performance of Hedge increases rapidly and dramatically.. When applied to the problems of pooling and system evaluation, Hedge identifies relevant documents very quickly, and these documents form an excellent and efficient pool for evaluating the quality of retrieval systems.. We note two possible extensions to this work. First, in all our experiments the tunable parameter β was set to 0.1. We were guided by results of Freund and Schapire in choosing this value, but we have not attempted to optimize it for our purposes, nor have we investigated its affect on our results. Our results may very well improve with a better choice of β. Second, the pools need not be generated one document at a time. At any round of the algorithm, one could output any prefix of the metasearch list as candidate documents for inclusion in the pool. We suspect that performance will degrade as larger prefixes are added to the pool with batch feedback, and this remains work to be explored.

O O B O B B O O B O O O O B O O O O O O O O O O B B O B O O O B O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O B O O O O O O B B O O O O B O O O B O O O O O O O B O O B O B O O O O O O O O O O O O O B O O B O O O O O B O B O O O O O O O B B O O B O O O O O O O O O B O O B O O O O O O O O O B B O O B O O O O O O O O O O O O O O B O O O B O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O B O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Application of the topological gradient method to color image restoration The goal of this note is to generalize the topological gradient method, applied to restoration and classification problems for grey-level images, to color images. We illustrate our approach with numerical tests.Application de la méthode du gradient topologique au traitement d'images couleur. Le but de cette note est de généraliser l'approche du gradient topologique, appliquée au problème de la restauration et de la classification d'images en niveaux de gris,à des images couleur. Quelques tests numériques sont présentés pour illustrer cette approche. The goal of this note is to propose a new method for color image restoration and classification problems. This method is based on the topological gradient approach that has been introduced for topological optimization purpose [6, 1] . The basic idea is to adapt the topological gradient approach applied to diffusive image restoration [4, 5] and grey level classification problem [3] . In fact, color or multispectral images can be represented or modeled in various ways [7] . In this note we shall focus on the RGB model in which a color image I is usually represented by a 3D vector. For each pixel (x, y), the vector I(x, y) represents the intensity of the three colors: Red, Green, Blue. Since each component can be treated as a grey level image, then the topological gradient approach can easily be used.. Let v be a given noisy color image defined as a vectoriel function v : (x, y) ∈ Ω ⊂ R 2 −→ v(x, y) = (v 1 , v 2 , v 3 ), where Ω represents the support of the image, and v k is the value of the pixel (x, y) in each color subspace (red, green, blue). Let u = (u 1 , u 2 , u 3 ) be the restored image. To find u, we solve the following PDE problem in each subspace (see [4, 5] for more details about this equation). where n denotes the outward unit normal to ∂Ω, and c k is a positive coefficient. In topological optimization approach, c k takes only two values: c k 0 in the smooth part of the image and a small value ε k > 0 on edges. Inspired by the work of G. Aubert et al. [2] in which the authors propose a classification model coupled with a restoration process, we propose to use the topological gradient approach applied to color image restoration for regularized classification problem. Since the classification problem consists in finding a partition of Ω into a family of open sets {Ω i } i=1,..,n , the constant c k 0 in the restoration process, which controls the regularity of the restored image and then the length of the interfaces, will be considered as a regularity term in the classification process.. The structure of this note is the following. We review in section 2 the topological gradient approach for image restoration and generalize it to color images. The classification problem and its generalization to color image is presented in section 3. Some numerical tests are presented and discussed in both sections. In this note, we have considered the topological gradient approach for color image restoration and classification problems. The model we proposed in this note is an extension to color images of the method introduced in [4] and [3] . The numerical tests show that our method can successfully remove the noise and preserve the global features of a color image. The numerical classification results are also satisfactory. As some coupling between the red, green and blue channels is important and should be considered, we propose in a future work to explore other color spaces (such as 2D and 3D histograms on HSV and CB spaces) and some coupling methods such as the Di Zenzo gradient [8] .

O O O B I O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O


Segmentation According to Natural Examples: Learning Static Segmentation from Motion Segmentation Abstract-The Segmentation According to Natural Examples (SANE) algorithm learns to segment objects in static images from video training data. SANE uses background subtraction to find the segmentation of moving objects in videos. This provides object segmentation information for each video frame. The collection of frames and segmentations forms a training set that SANE uses to learn the image and shape properties of the observed motion boundaries. When presented with new static images, the trained model infers segmentations similar to the observed motion segmentations. SANE is a general method for learning environment-specific segmentation models. Because it can automatically generate training data from video, it can adapt to a new environment and new objects with relative ease, an advantage over untrained segmentation methods or those that require human-labeled training data. By using the local shape information in the training data, it outperforms a trained local boundary detector. Its performance is competitive with a trained top-down segmentation algorithm that uses global shape. The shape information it learns from one class of objects can assist the segmentation of other classes. A picture is worth a thousand words, but a pixel is nearly worthless. Its brightness and color might be due to an object surface, lighting conditions, or instrument sensitivity. Its value may be corrupted by sensor noise. It provides no depth information. There are few useful vision tasks that can be performed with a single pixel. Therefore, optimally grouping pixels is a primary challenge in computer vision. This paper describes Segmentation According to Natural Examples (SANE), an algorithm that learns to segment objects in still images by training on videos of moving objects.. Many image segmentation algorithms have been developed in the last 40 years, but the endeavor has always been troubled by what David Marr called the "almost ... philosophical problems" of specifying a useful region definition [20] . Image properties that might indicate a significant image boundary in one environment might be an unimportant local variation in another. If the segmentation is a preprocessing step, the regions useful for one algorithm might be useless to another. Learning approaches based on human-labeled boundaries have been explored [3] , [14] , [22] , but acquiring the training data is expensive and the utility of these subjectively defined segmentations is unclear.. SANE addresses these concerns. It learns a model of region boundaries by observing object motion in videos, and then applies this model to segment new static images. It produces segmentations with closed boundaries; uses an objective, useful, region definition; and can be trained to segment new environments with relatively low cost.. Motion provides a concrete, general-purpose region definition for segmentation. Defining a region as a collection of elements that undergo coherent motion corresponds well to intuition. All the visually distinct elements of a car can be considered a single region because they frequently move together, but a pile of leaves is composed of many elements that move independently on a windy day. This motion definition is clearly useful; for example, a robot manipulator must know whether a collection of pixels represents a single entity or something that will disintegrate at the slightest touch.. Apart from their utility, motion-defined regions are desirable because their test and training example sets can be automatically generated. A video camera on a sidewalk can record hundreds of moving cars, people, or animals every hour, and software can use their motion to separate them from their surroundings. Thus, motion-defined regions combine utility and practicality.. SANE uses background subtraction to find the segmentations of moving objects in videos. This provides segmentation information for each video frame. The collection of frames and segmentations forms a training set that SANE uses to learn the image and shape properties that correspond to the observed motion boundaries. Then, when presented with new static images, the model infers segmentations similar to the observed motion segmentations.. SANE contributes a framework that uses video motion to generate training data for new segmentation environments, and learning and inference algorithms that translate the data's shape and image information into a useful still-image segmentation algorithm. Because it does not require human-labeled training data, SANE can adapt to a new environment and new objects with relative ease. The large, automatically labeled training sets also allow SANE to use simple learning algorithms that rely on counting and nonparametric probability density estimates. Comparisons to the well-known normalized cuts segmentation algorithm [35] and to the Martin et al. [22] learned edge detector reveal performance advantages due to SANE's adaptability and shape modeling. SANE's output is competitive with the output of the top-down LOCUS segmentation algorithm [45] , and it learns useful shape information that generalizes well to the segmentation of novel object classes and environments. Future extensions to SANE might include the use of more sophisticated local features, such as those used in the Martin edge detectors [22] and extending the model to use four segmentation labels, which would allow for the production of arbitrary segmentations, even separating overlapping objects. More accurate inference and learning algorithms might also improve the results.. The current implementation of SANE requires a few minutes to segment images composed of a few thousand pixels, but can take up to a few hours for images containing tens of thousands of pixels. However, these running times are dominated by the construction of the MRFs, rather than by inference, and they could be greatly reduced by software optimizations.. Currently, SANE can learn a scale-robust model by being exposed to example objects of different scales such as differently sized people or cars. This could be enhanced by changing the training procedure so the images are presented at a number of different scales, or by modifying the model to account for scale more directly, perhaps in the context of the multiresolution approach. The related issue of image resolution might be more difficult to tackle-the relatively poor performance of the Martin edge detectors suggest that different features are useful for segmentation on high and low-resolution images. SANE acquires its training data through background subtraction, but the Stauffer and Migdal algorithms cannot handle videos with moving backgrounds, or separate multiple moving objects whose paths intersect. In order to learn under these conditions, more sophisticated background subtraction or motion segmentation algorithms would be required, particularly those able to use optical flow.. Further work might integrate SANE with other algorithms, such as manipulation or object recognition, in which segmentation can be optimized against other objective criteria, such as its utility in assisting the performance of another task. The concept of using manipulation to provide segmentation knowledge is similar to the recent work by Fitzpatrick [7] , but with the goal of learning better segmentation algorithms rather than focusing on detecting specific objects and their boundaries. Extra sources of segmentation information might allow a future SANE algorithm to distinguish between objects and shadows and to learn to segment a larger class of objects. The results have demonstrated that the SANE algorithm provides advantages over a prominent nonlearned image segmentation algorithm and over a trained local boundary detector. SANE's output is competitive with a learningbased, top-down segmentation algorithm despite using local neighborhood shape information instead of a global class-specific model. Its ability to learn the visual features appropriate to segmentation in a particular visual environment and to combine those features with a strong learned shape model had clear performance benefits. It demonstrates that simple boundary detection features and local neighborhood shape information can be combined into a powerful segmentation algorithm. The automatically labeled training data made the use of simple and powerful nonparametric density estimators practical.. Furthermore, experiments on generality demonstrate that the shape information learned is not specific to a particular object class or environment, and that it is responsible for much of SANE's performance. This validates the utility and practicality of SANE's shape model, the central structure of the model. Most importantly, this learning was done with selflabeled training data, optimizing against an objective and useful segmentation ground truth. Therefore, it is practical to train SANE to be a useful component of a deployed vision system in a new environment.

B O O O O O O B O B B O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O


The SAT-based Approach to Separation Logic Abstract. The SAT-based approach to the decision problem for expressive, decidable, quantifierfree first-order theories has been investigated with remarkable results at least since 1993. One such theory, successfully employed in the formal verification of complex, infinite state systems, is Separation Logic (SL), which combines Boolean logic with arithmetic constraints of the form x j y ( c, where ( is e, <, >, Q, =, or m. The SAT-based approach to SL was first proposed and implemented in 1999: the results in terms of performance were good, and since then a number of other systems for SL have appeared. In this paper we focus on the problem of building efficient SAT-based decision procedures for SL. We present the basic procedure and four optimizations that improve dramatically its effectiveness in most cases: (a) IS 2 preprocessing, (b) early pruning, (c) model reduction, and (d) best reason detection. For each technique we give an example of how it might improve the performance. Furthermore, for the first three techniques, we give a pseudo-code representation and formally state the soundness and completeness of the resulting optimized procedure. We also show how it is possible to check the satisfiability of valuations involving constraints of the form x j y < c using the BellmanYFord algorithm. Lastly, we present an extensive comparative experimental analysis, showing that our solver TSAT++, built along the lines described in this paper, is currently the state of the art on various classes of problems, including randomly generated, hand-made, and real-world instances. The SAT-based approach to satisfiability problems beyond propositional logic dates back to at least the early 1990s (Armando and Giunchiglia, 1993) , when it was noted that, under some suitable conditions, the problem of determining the satisfiability of any decidable, quantifier-free first-order theory can be reduced to Boolean search coupled with a satisfiability procedure (i.e., procedure capable of deciding whether any given set of literals in satisfiable or not w.r.t. the given theory). In more detail, the SAT-based approach to the satisfiability problem of a formula 0 in a theory T amounts to using:. Y a SAT solver to generate a valuation m entailing 0 in propositional logic, and till a satisfiable m is found (in which case also 0 is satisfiable), or a set of valuations whose disjunction is logically equivalent to 0 has been generated and tested (in which case 0 is unsatisfiable). Over the years, the SAT-based approach has been applied to more theories and even to different problems, such as propositional modal logics (Giunchiglia and Sebastiani, 1996; Giunchiglia et al., 2002) , conformant planning (Castellini et al., 2003) , and combination of expressive theories (Stump et al., 2002) , with remarkable results. As the research proceeded, it became clear that the approach could harvest the technological improvements achieved in propositional satisfiability. See (Armando et al., 2005b) for a unifying perspective on the SAT-based approach.. Many verification and scheduling problems involve arithmetic constraints of the form x j y ( c, where x and y are variables ranging over the reals or the integers and ( is e, <, >, Q, =, or m. These constraints are called separation terms by Pratt (Pratt, 1977) , and Separation Logic (from now on, SL) is the name now used to denote the logic allowing for arbitrary Boolean combination of separation terms.. j SL is also called Bdifference logic[ by some authors (see, e.g., Cotton et al., (2004) ) and can be seen as a generalization of a well-known framework for temporal reasoning, the Temporal Constraint Network, introduced by Decther, Meiri and Pearl (Dechter et al., 1989) . SL is the logic we focus on in this paper.. The first application of the SAT-based approach to a significant fragment of SL was given in Armando et al. (1999) . In this case, as well as with modal logics and conformant planning, excellent results were obtained. Since then, a number of other systems for SL have appeared (see, e.g., Oddi and Cesta, 2000; Audemard et al., 2002; Strichman et al., 2002; Armando et al., 2005a; Cotton et al., 2004) .. In this paper we focus on the problem of building efficient SAT-based decision procedures for SL. To this end, we present the basic procedure and four optimizations that improve dramatically its effectiveness in most cases: (a) IS 2 preprocessing, (b) early pruning, (c) model reduction, and (d) best reason detection. Optimizations (a) and (b) were first proposed in Armando et al. (1999) , whereas (c) and (d) have been presented for the first time in Armando et al. (2005a) . For each technique we give an example of how it might improve performance. Furthermore, for the first three techniques, we give a pseudo-code representation and formally state the soundness and completeness of the corresponding procedure. We also show how it is possible to check the satisfiability of valuations involving constraints of the form x j y < c using the well-known BellmanYFord algorithm (from now on, BF).. We then present an extensive comparative experimental analysis, showing that our solver TSAT++, built along the theoretical lines of the approach, is j Unfortunately, the name Separation Logic is also used to denote an extension of Hoare logic. Strichman et al. (2002) is the first reference we are aware of where the name is resumed from Pratt's paper.. currently the state of the art on various classes of problems, including randomly generated, hand-made, and real-world instances.. The paper is structured as follows. Section 2 is about SL and presents its syntax, semantics, and some other formal properties of SL; Section 3 introduces the basic SAT-based procedure for SL, while the optimizations are presented in Section 4; in Section 5 we present a satisfiability algorithm for valuations based on BF; in Section 6 we describe the actual implementation of our system and present a thorough experimental evaluation; in Section 7 we outline the related work; lastly, in Section 8 we have the conclusions. In this paper we have focused on the problem of building efficient SAT-based decision procedures for SL. We have presented the basic procedure from Armando et al. (1999) along with some key optimizations. We have also shown how it is possible to check the satisfiability of valuations involving constraints of the form x j y < c using BF. An extensive comparative experimental analysis shows that our solver TSAT++, built along the lines described in this paper, is currently the state of the art on various classes of problems, including randomly generated, hand-made, and real-world instances. We believe that the techniques described in this paper can be fruitfully extended to other (more expressive) logics than SL.. The benchmark problems used for the experiments presented in this paper and the executable of TSAT++ are publicly available at the URL http://www.ai.dist. unige.it/Tsat. Several systems tailored for SL, employing different approaches and techniques, have been built and tested over the years. We now give an overview of them, highlighting the pros and cons of each one and chronologically reviewing the techniques introduced by each one. SK (Stergiou and Koubarakis, 1998 ); Stergiou j Following a suggestion by Offer Strichman, we have also tried SEP with an option that disables the use of a specialized data structure called Bconjunction matrix^ (Strichman et al., 2002) . This can have a dramatic impact on SEP: some problems that are solved with conjunction matrix within the time limit are not solved without, and vice versa.. and Koubarakis, 2000) . The procedure SK has been the first dealing with a significant fragment of SL. Its main features are the combined usage of forwardchecking, back-jumping, and the minimum remaining value heuristic (MRV). Forward-checking works by checking whether the valuation built so far entails either a literal or its negation, form each literal not yet in the valuation. This actually reduces the search space, at the price of performing a potentially large number of useless satisfiability. SK is also able to detect conflict sets and to improve on backtracking via a technique similar to back-jumping. MRV is used to choose literals that appear in disjunctions with the smallest number of unassigned disjuncts: if there is a unit clause, the literal in it will be selected by MRV and then propagated, thus mimicking unit propagation.. The main difference between SK and SAT-based procedures lies in the way valuations propositionally entailing the input formula are searched. In fact, SK is based on syntatic branching: given a disjunction l ¦ l 0 , first l is added to the current valuation, and, upon failure, l 0 is considered. As explained below, this type of search may lead to the exploration of search space already explored.. Tsat (Armando et al., 1999) . Tsat was the first application of the SAT-based approach to SL. The system employs a branching schema now known as semantic branching. Unlike syntactic branching, semantic branching selects a not yet assigned literal l, and considers in turn the case in which l is true and the case in which l is false. Notice that in the second case, the conjunction of l with (l ¦ l 0 ) forces the assignment of l 0 by unit propagation: as already observed in D'Agostino (1992), syntactic branching may lead to redundant exploration of parts of the search space, which semantic branching avoids. The following example, adapted from Armando et al. (1999) , clearly illustrates this issue. EXAMPLE 16. Let 0 be a formula including the following clauses:. Let 0(i, j) denote the jth disjunct of the ith disjunction displayed in 0; for example, 0(1, 2) is x 7 j x 8 e 20. Assume that the dots stand for further (possibly many) unspecified clauses such that no satisfiable extension of the valuation {0(1, 1), 0(2, 1)} exists.. Consider the behavior of syntactic versus semantic branching when {0(1, 1), 0(2, 1)} is the valuation built so far. Since no satisfiable extension of it exists, after some search, failure is necessarily detected; both procedures backtrack and remove 0(2, 1) from the current valuation. Now syntactic branching goes on with the valuation {0(1, 1), 0(2, 2)}, whereas semantic branching proceeds with {0(1, 1), K0(2, 1)}, which leads immediately, via unit propagation, to {0(1, 1), K0(2, 1), 0(2, 2)}.. Working with the latter valuation rather than with the former may lead to considerable savings: assume that both procedures extend the valuation with 0(3, 1); since {0(1, 1), K0(2, 1), 0(2, 2) 0(3, 1)} is unsatisfiable, semantic branching immediately backtracks and considers 0(3, 2), whereas syntactic branching may waste a big amount of resources in the vain attempt of finding a satisfiable extension of {0(1, 1), 0(2, 2), 0(3, 1)}.. Semantic branching was shown in Armando et al. (1999) to dramatically improve the performance with respect to SK, up to one order of magnitude on randomly generated binary DTPs.. In Tsat, also IS 2 was introduced, gaining to the system another order of magnitude in performanceVthis despite the fact that, to enumerate valuations, Tsat adapted a rather simple SAT solver, due to Böhm (Böhm and Speckenmeyer 1996) , which did not employ any modern optimization such as back-jumping and learning. Satisfiability checking used lp_solve v2.2 (Berkelaar, 1997) , which provided a free implementation of the Simplex method.. CSPi (Oddi and Cesta, 2000) . CSPi features an essentially CSP-based solution schema, implementing an efficient incremental procedure for forward-checking. Semantic branching is used, showing results that are better than Tsat on small instances, and comparable on bigger ones. Notice that performance, up to (Oddi and Cesta, 2000) , was measured in terms of how many calls to the satisfiability check function were done, rather than CPU time.. MathSAT (Audemard et al., 2002) . MathSAT uses SIM (Giunchiglia et al. 2001 ) as enumerator and a hierarchical satisfiability checker employing Y in this order Y equality reasoning, BF for SL-constraints, the Simplex method for full linear arithmetic, and inequalities reasoning. The simplest solver is chosen onthe-fly, thereby obtaining both expressivity and efficiency at the same time.. MathSAT also introduces a number of optimizations, among which are preprocessing based upon syntactic equivalence, enhanced early pruning, that is, early pruning conditioned upon a heuristic function, and back-jumping/learning based upon reason detection. Also, a form of model reduction is used, based upon triggering. On randomly generated binary DTPs, MathSAT improves the performance over Tsat in terms of CPU time. However, the gap between the two solvers decreases as the number of variables increases.. Epilitis (Tsamardinos and Pollack, 2003) . Epilitis is, so far, the last CSPbased system. Epilitis is restricted to binary DTPs. It uses semantic branching, incremental forward checking, a MRV heuristics, and size-bounded learning of size n (Bayardo and Miranker, 1996) . This means that conflict clauses are retrieved and stored only if they contain less than n literals (in practice, n = 10 is used). Once stored, a clause is never forgotten. On randomly generated binary DTPs, Epilitis shows significantly better performance than Tsat in terms of CPU time, of up to one order of magnitude.. SEP (Strichman et al., 2002) . SEP is a back-end to the UCLID verification tool , employing the so-called eager variant of the SAT-based approach. Given a formula 0, rather than enumerating valuations and checking them for satisfiability, SEP builds a propositional formula 0 0 whose satisfying valuations are ensured to correspond to satisfiable valuations of 0. The current version of SEP uses Chaff to find valuations satisfying 0 0 . To the best of our knowledge, SEP is so far the only solver using the eager SAT-based approach to SL. SEP suffers from the fact that the size of 0 0 can be exponential in the size of 0. On the other hand, as reported in Strichman et al. (2002) , if SEP can get past the encoding phase, the problem is easy to solve for Chaff.

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Preserving correctness during business process model configuration Abstract. A configurable process model captures a family of related process models in a single artifact. Such models are intended to be configured to fit the requirements of specific organizations or projects, leading to individualized process models that are subsequently used for domain analysis or solution design. This article proposes a formal foundation for individualizing configurable process models incrementally, while preserving correctness, both with respect to syntax and behavioral semantics. Specifically, assuming the configurable process model is behaviorally sound, the individualized process models are guaranteed to be sound. The theory is first developed in the context of Petri nets and then extended to a process modeling notation widely used in practice, namely Event-driven Process Chains. The design of business process models is labor-intensive, especially when such models are required to be detailed enough to support the development of software systems. To avoid the effort of creating process models from scratch, several consortia and vendors have defined so-called reference process models. These models capture proven practices and recurrent business operations in a given domain. They are designed in a generic manner and are intended to be individualized to fit the requirements of specific organizations or IT projects. Commercial process modeling tools come with standardized libraries of reference process models such as the IT Infrastructure Library (ITIL) 1 or the Supply Chain Operations Reference (SCOR) model [Ste01] . Also, the SAP Reference Model [CK97] incorporates a collection of process models corresponding to common business operations supported by SAP's Enterprise Resource Planning (ERP) system.. Reference process models in commercial use lack an explicit representation of configuration alternatives and decisions. As a result, their individualization is entirely manual [RA07] . Analysts take the reference models merely as a source of inspiration, but ultimately, they design their own model on the basis of the reference model, with little guidance as to which model elements need to be removed, added or modified to address a given requirement. To address this shortcoming, we introduced the concept of configurable process models [RA07] in previous work. A configurable process model represents multiple variants of a business process model in an integrated manner. In line with methods from the field of software product lines [PBL05] , these alternatives are captured as variation points. That means, instead of having to add or remove model elements manually, the fact that a task in a reference process model may or may not appear in an individualized model is captured by attaching a variation point to that task allowing users to select or deselect it. Individualized models are obtained from configurable models by interpreting the values for each variation point. In separate work we also reported on experiences from using configurable process models in practice [LHRS08, GWJV + 09]. These studies showed that configurable process modeling provides an intuitive mechanism to combine different process variants in a single model and is preferred to a traditional process modeling paradigm.. While configurable process models provide guidance to analysts during individualization, they do not guarantee that the individualized models are correct, whether syntactically or semantically. For example, if a model element or an entire path in a reference process model is removed during configuration, the remaining model elements need to be re-connected to maintain syntactic correctness. Also, the configuration of variation points attached to parallel splits, decision points and synchronization points in a configurable process model may lead to the introduction of deadlocks. And if the individualized process model contains such semantic errors, it needs to be manually fixed.. The contribution of this article is a formal framework for configuring reference process models in a correctness-preserving manner. The framework includes a technique to derive propositional logic constraints that, if satisfied by a configuration step, guarantee the syntactic correctness of the resulting model. We prove that for a large class of process models, these constraints also ensure that semantic correctness is preserved. The framework supports staged configuration [CHE04] . In other words, it allows correctness to be checked at each intermediate step of the configuration procedure. Whenever a value is assigned to a variation point, the current set of constraints is evaluated. If the constraints are satisfied, the configuration step is applied. If on the other hand the constraints are violated, we compute a reduced propositional logic formula, from which we can identify additional variation points that need to be configured simultaneously in order to preserve correctness (e.g. if an edge in the process model is removed, all nodes in a path starting with that edge need to be removed). The set of constraints is incrementally updated after each step of the configuration procedure.. The proposal is intended as a foundation for analyzing properties of configurable process models, particularly with respect to correctness. Accordingly, we initially adopt a Petri net-based representation of process models, thus abstracting from the specificities of process modeling notations used in practice such as UML Activity Diagrams (UML ADs), Event-driven Process Chains (EPCs) or the Business Process Modeling Notation (BPMN). We use a class of Petri nets, namely workflow nets, which are specifically designed to represent business processes [Aal97] . Workflow nets come with a notion of behavioral correctness known as soundness, which ensures the absence of deadlocks and improper completion. In this article, we enhance workflow nets with the notion of variation point, leading to the concept of a configurable Workflow net. We then define a notion of configuration step over such nets and we show how to derive correctness-preserving constraints for such steps. A core result of the article is that, for workflow nets that satisfy the "free-choice" property [DE95] , if the outcome of a configuration step starting from a sound Workflow net is a Workflow net, then this latter Workflow net is sound. This means that for this class of nets, configuration steps that preserve syntactic correctness also preserve behavioral correctness.. Having established a formal foundation for process model configuration, we apply it to Configurable Eventdriven Process Chains (C-EPCs) [RA07] -a configurable version of the EPC notation. We reuse previous results to link C-EPCs to Petri nets, and we show how the notion of configuration step defined on Petri nets can be adapted to fit the specificities of C-EPCs.. The article is structured as follows. Section 2 introduces workflow nets and the notion of soundness while Section 3 introduces the notion of configurable Workflow net and configuration step. Section 4 discusses the derivation of constraints that guarantee the preservation of syntactic correctness, and proves that these constraints also guarantee soundness for free-choice nets. Section 5 shows how the framework can be applied to the individualization of configurable EPCs. The article concludes with a section on related work, a summary, and an outlook on open issues.. Preserving correctness during business process model configuration 461 We have exposed a formal framework for staged correctness-preserving configuration of reference process models. Assuming the initial configurable process model is correct, the framework guarantees that the individualized process models are also correct at each stage of the configuration procedure. This is achieved by capturing the syntactic correctness constraints as a propositional logic formula. This formula, in conjunction with another formula capturing the domain constraints, is used to check the correctness-preservation of each configuration step. If a configuration step violates the constraints, a formula is derived to suggest ways of making the configuration step correctness-preserving. A cornerstone of the framework is a proof that, for free-choice process models, the enforcement of these syntactic constraints also ensures the preservation of semantic correctness.. Having established this formal framework using Petri nets, we have illustrated its application to a practical process modeling notation, namely EPCs. This application is achieved by reusing a previous formalization of EPCs and defining a mapping from configuration operators defined for C-EPCs into corresponding operators defined for Petri nets. This mapping then allows us to check the correctness-preservation of configuration steps applied to C-EPCs by reasoning on the induced Petri nets. In order to achieve a simple mapping, we restricted our study to C-EPCs without OR connectors. While this limitation does not affect the expressive power of the (C-)EPC notation, it is still worth lifting given that OR connectors can add convenience by allowing modelers to represent certain constructions in a more compact way. Extending the proposal presented here in order to directly support the OR connector is a direction for future work.. In previous work [LLS + 07] we developed a toolset, namely Synergia, for questionnaire-driven configuration of C-EPC process models. Based on the framework exposed in this article, we have extended this toolset to incrementally check the syntactic and semantic correctness of a C-EPC at each step of its configuration. The toolset can be downloaded from the Process Configuration web-site. 8 The results presented in this article focus on the control-flow perspective of process modeling. We envisage extending the formal framework presented here to cover other dimensions than control-flow, such as data-flow between tasks in a process model and allocation of tasks to resources. Initial results in this direction are exposed in [LDH + 08]. Another direction for future work consists in investigating techniques for automating the construction of configurable process models. A possible starting point is to collect a number of related process models from different (preferably successful) process design projects, and to merge them together. But how this merger can be facilitated is still an open question. For example, since process models are usually represented as graphs, algorithms from the field of graph matching could prove useful [Bun00] .. Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. Variability modeling has been widely studied in the field of Software Product Line Engineering (SPLE) [PBL05] . Techniques developed in this field enable the configuration of software artifacts based on models that relate these artifacts to domain concepts (e.g. parameters, options or features). The techniques differ in the way domain models are captured and related to software artifacts, and also in the way they capture constraints. The Adele Configuration Manager [EC94] and the Cosmic Configurable Middleware [TGN04] use first-order logic to capture constraints. In contrast, we use propositional logic, for which we can apply efficient techniques to discard incorrect configuration steps or to suggest ways of repairing them. Batory [Bat05] presents a Feature-Oriented Domain Analysis (FODA) technique in which constraints are captured in propositional logic. The respective tool uses a SAT solver to determine if a configuration is valid. A similar approach is adopted in [AC04] . Our work is inspired by these approaches but it is targeted at business process model configuration. Thus, we deal with graph-oriented models (hence, structural correctness needs special attention) and we are concerned with ensuring absence of deadlocks or livelocks and other behavioral properties.. The literature exhibits a heterogeneous set of methods for capturing variability in process models. The approach in [BDK07] is based upon the principle of model projection. Since a reference process model typically contains information for multiple application scenarios, it is possible to create a projection for a specific scenario (e.g. a class of users) by fading out those process branches that are not relevant to the scenario in question. Plain EPCs are used to represent process models. Another proposal for capturing variability in process models is defined in [SP06] , which relies on stereotype annotations to accommodate variability in a so-called variant-rich process model. Accordingly, process activities can be annotated to become variation points and variants can be attached to these variation points in the form of other stereotyped activities. Although stereotypes are an extensibility mechanism of UML, in this proposal they are applied to both UML ADs and BPMN models. A subset of these stereotypes also appears in [RK08] . Specifically, in this approach the focus is on two types of variation points: optional and alternative ones. An optional variation point allows the selection of at most one variant among the available ones. An alternative variation point allows the selection of exactly one variant. This idea of annotating model elements to represent variability has also been investigated in [CA05] . In this approach, any control-flow element of an UML AD can be annotated using presence conditions (PCs) and meta-expressions (MEs). PCs indicate if the model element they refer to should be present or be removed. MEs are used to compute attributes of model elements relevant to the UML notation (e.g. the name of an activity).. Although there exists a variety of approaches, none of these deal with the matter of preserving model correctness during configuration. The only ones that get closer to providing some sort of support in this respect are [BDK07] and [CA05] . These approaches prompt users with a list of syntactic issues detected during process individualization. However these issues need to be manually fixed.. Our previous work includes the definition of variability mechanisms for existing process modeling languages such as EPCs [RA07] , YAWL, SAP WebFlow and BPEL [GAJVL08] . In [LLS + 07] we proposed a framework which ensures domain conformance (but not syntactic or behavioral correctness) by linking configurable process models to domain models expressed as questionnaires. Finally, the use of the hiding and blocking operators for variation points is sketched in [GAJV07] .. In this article we outlined a technique to derive propositional logic constraints from process models. Similar techniques have been used for analyzing Petri nets [AIN04] and process graphs [SOS05] . However, the constraints we derive are specifically aimed at checking that a configuration step preserves the structural properties of workflow nets.. Links between EPCs and Petri nets have been widely discussed in literature. The mapping from EPCs to Boolean nets, a variant of colored Petri nets, in [LSW98] , is the first significant formal contribution in this area. The semantics is however restricted to EPCs in which OR connectors appear only at the boundaries of singleentry, single-exit regions. Another attempt to attach a formal to C-EPCs in a more general way can be found in [NR02] , but it was later proven to be flawed in the sense that a fixed point is not guaranteed [Kin06] . Again, the problem lies with the formalization of OR connectors, and specifically the OR-join connector. This finding has triggered work on defining OR-joins based on timers, history or context [HOS05, MA07, HOS + 08]. While these formalizations implement the OR-join in different ways, they essentially agree on the semantics of XOR and AND connectors. The difficulties in reaching a consensus semantics of the OR-join in EPCs underpins our decision to leave this operator out of the scope of this article, and to adopt the semantics of EPCs without OR-joins proposed in [Aal99] . Petri nets are a formal model of concurrent systems [Mur89] . Petri nets benefit from a rich body of theoretical results, analysis techniques and tools. They have been extensively applied to the formal verification of business process models [VBA01] . These features make Petri nets suitable for establishing a formal foundation for business process model configuration. In addition, mappings exist between process modeling languages used in practice (e.g. UML ADs, EPC, BPMN, BPEL) and Petri nets. These mappings provide a basis for extending the results outlined in this article to concrete process modeling notations.. We use a class of Petri nets, namely workflow nets, specifically designed for business process modeling. Workflow nets have a single starting point and ending point, which captures the intuition that business processes are instantiated, and each process instance progresses independently through a series of activities until completion. A desirable property is that an instance of a Workflow net always completes properly. This is captured by the notion of soundness. To make the article self-contained, we provide an introduction to workflow nets and soundness.

O O O O O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O B I I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Fair coalitions for power-aware routing in wireless networks Abstract-Several power aware routing schemes have been developed under the assumption that nodes are willing to sacrifice their power reserves in the interest of the network as a whole. But, in several applications of practical utility, nodes are organized in groups, and as a result a node is willing to sacrifice in the interest of other nodes in its group but not necessarily for nodes outside its group. Such groups arise naturally as sets of nodes associated with a single owner or task. We consider the premise that groups will share resources with other groups only if each group experiences a reduction in power consumption. When this is the case the groups may form a coalition in which they route each other's packets. We demonstrate that sharing between groups has different properties from sharing between individuals and investigate fair mutually-beneficial sharing between groups. In particular, we propose a pareto-efficient condition for group sharing based on max-min fairness called fair coalition routing. We propose distributed algorithms for computing the fair coalition routing.Using these algorithms we demonstrate that fair coalition routing allows different groups to mutually benefecially share their resources. Wireless networks typically consist of nodes that must discharge increasingly complex computing and communication functionalities despite constraints on power, bandwidth, size and memory. Significant progress has been made to improve hardware to address these needs and much is being done to develop software that uses techniques like poweroptimizing algorithms. Comparatively less has been done to exploit sharing amongst nodes as a way to address these challenges. This is unfortunate, since sharing can yield great benefits. A variety of challenges impede progress: (a) determining which resources can be shared, (b) deciding when to share resources, as sharing would evidently involve a cost, (c) deciding with whom to share resources, and (d) determining how to share resources.. Oftentimes, groups of nodes rather than individual nodes are basic entities in the sharing mechanism. The resource expenditure of the group as a whole is more important than that of a single node or the entire network. Groups are often formed on the basis of membership in an organization or a shared task. For example, employees of an organization A may carry computers that belong to A. When these devices form an ad hoc network, they may share resources with other devices with the objective of minimizing the total resource consumed by the devices in A, rather than that of all devices in the network. Thus, the devices belonging to an organization form a natural group. Wearable computers involved in one distributed computation may form a group. In a sensor network, different groups would consist of. The research was supported in part by NSF EIA00-88028, NSF ANI01-06984, ONR N00014-02-1-0715 and NCR-0238340.. sensors that measure different attributes such as temperature, pressure etc. In both the above cases, the resource consumed by groups is more important than that consumed by individual nodes as the distributed computation can be performed and the attributes can be measured even when some members fail. The research in this case must investigate issues pertinent to sharing of resources from the perspective of groups.. A group is an intermingled set of nodes having a purpose in common. We do not consider the motivation behind the group formation, but investigate the sharing of resources among different groups. The critical resource we focus on is power. Nodes in wireless networks are powered by battery, and size limitations compel the usage of low lifetime batteries. This calls for judicious consumption of power. Normally, communication consumes higher power than other operations. Nodes share power by routing each others packets, and it is well-known that multihop routing substantially decreases the overall power consumption of the network [25] . We address the research challenges that arise when nodes decide to route each others packets with the sole objective of reducing the power consumption of their groups. We first enumerate these challenges.. The nodes in a group share power by routing each other's packets to common destinations. Groups are said to form coalitions when they route each other's packets. The first challenge is to determine which groups would form coalitions. Presumably, a precondition for forming coalitions among groups is that each group communicates the same amount of information to the chosen destinations while consuming less power after the coalition is formed. Whether or not the precondition is satisfied depends on the routing in the coalition, and the number of possible routes can be an exponential function of the number of nodes in the groups. There need not even exist a routing that reduces the power consumption of each group. Fig.1(a) and (b) show that if each group consists of a single node, then groups do not mutually benefit from the coalition; but this no longer holds if the groups consist of two or more nodes ( Fig.1(c) ). The challenge then is to answer whether there exists at least one joint routing that makes the coalition mutually beneficial. The next challenge is to decide the joint routing when the coalition is formed. We will show in Section III-C that the routing that minimizes the total power consumption of all groups is not the right choice, as it may increase the power consumption of some groups despite minimizing the power consumption of the network as a whole. The benefit of a group due to the coalition operation is the decrease in its power consumption after it joins the coalition. We need to determine a routing that shares the benefit equitably. A simplistic approach is to insist that the groups each get Fig. 1 . In (a) and (b), we show two different routings where node a constitutes group A and node b constitutes group B. Both groups need to send traffic to the access point(AP). In (a) the farther node a routes its traffic to b and b sends to AP. So the routing in (a), reduces the power cost of a but increases that for b. In (b) each node routes directly to AP and there is no reduction in power costs for both groups. In (c) nodes a 1 and a 2 constitute group A and b 1 constitutes group B.. Here a 1 can send its traffic through b 1 and b 1 can in turn send through a 2 . This could result in a decrease in the total power for group A and B as against the case when the groups route to AP independently. the same benefit, but this can be wasteful if one group can gain benefit without harming the others. A max-min fair [1] routing uses the following strategy for a pair of groups: determine the greatest minimum benefit to be gained by either of the two groups when sharing and maximize the benefit of the other group so long as the changes do not reduce this minimum. This strategy can be generalized to multiple groups. The challenge now is to compute a maxmin fair power aware coalition routing.. We survey the relevant literature in Section II. We provide a mathematical framework for a coalition of two groups in Section III. This section presents several interesting properties of coalition routings. For example, a max-min fair power aware coalition routing exhibits important characteristics that do not hold for max-min fair allocation of other resources such as bandwidth. We show that the maxmin fair coalition routing is guaranteed to attain the desired minimum benefits for each group should the coalition be feasible. We present a polynomial complexity algorithm for computing the fair coalition routing in Section IV. This algorithm needs solving a linear program at a central processor, which requires the knowledge of the global topology. We present a distributed computing scheme which allows the routing to be computed via simple iterative computations and message exchanges at each participating node in Section V. All proofs can be found in the technical report [9] .. II. RELATED WORK The existing research on efficient utilization of power in wireless networks can be classified into the following broad categories. The first maximizes the lifetime of any given node through optimum battery discharge strategy [6] , [17] . The second varies the transmission power levels of nodes so as to control the network topology as desired [8] , [14] , [23] . The third reduces the power consumption by optimizing several parameters at the MAC layer [11] , [22] . The last maximizes the lifetime of the network by balancing the power consumption of different nodes [4] , [15] . Another prevalent approach is to route in accordance with a power based metric rather than a distance metric [25] . However the common feature of the existing research is that the basic entity is a node. The performance of the network is either quantified in terms of the aggregate performance of the nodes or that of the bottleneck node. However, in our case the basic entity is a group rather than a single node, and the operations are coalitions. The performance objective we consider is fairness and the issues significantly differ on account of the choice of the basic entity. We are concerned about the performance of each group rather than the network as a whole. Relaying and caching strategies have been proposed [21] , [18] for node cooperation where a node decides to relay the requests of other nodes. The algorithm in [21] propels the network towards a pareto optimal operating point. Our research is complementary in the sense that we assume that a group of nodes decide to route the packets of other groups based on the interest of the group as a whole. We present an algorithm that obtains a specific pareto optimal objective, the max-min fair operating point. We have studied the problem of forming coalitions between groups of nodes with the intent of saving power. We found that an application of max-min fair techniques to this problem yields an efficient and balanced approach which we call fair coalition routing. We developed theory and algorithms for fair coalition routing. We have carried out a range of simulations that demonstrate that fair coalition routing is practical and beneficial in common cases. We generalize the framework and the computation algorithms for a coalition among multiple groups in [9] . The coalition routing algorithms presented in this paper provide foundations for developing operational protocols. Design of such protocols would require deployment of mechanisms to enforce group routings e.g., security checks. We discuss some of these issues in [9] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Simple Learning Algorithms for Decision Trees and Multivariate Polynomials There were two techniques in the literature for learning multivariate polynomials and decision trees. These are learning decision trees under the uniform distribution via the Fourier Spectrum Kushilevitz, Mansour 93 and Jackson 94] and learning decision trees and multivariate polynomials under any distribution via Lattice Theory Bshouty 94, Schapire and Sellie 93]. These two approaches are used for proving the learnability of many other interesting classes such as CDNF (poly size DNF and CNF) under any distribution and DNF and AC 0 under the uniform distribution. In this paper we develop a new approach. Learning decision trees and multivariate polynomials via multivariate interpolation. This is an algebraic approach that is a modi cation of the approaches used in Roth, Benedek 89 and Bshouty 95] and is based on the divide and conquer technique for learning Bshouty 95]. This new approach yelds simple learning algorithms for multivariate polynomials and decision trees over elds under any constant bounded product distribution. The output hypothesis is a (single) multivariate polynomial that is an -approximation of the target under any constant bounded product distribution. Previous approaches do not achieve this property.The new approach also solves learnability of many other problems that cannot be solved using previous approaches. Problems include PAC-learning with membership queries of j-disjoint DNF (conjunction of any j terms is 0) under any constant bounded product distribution, PAC-learning with membership queries of multivariate polynomial with bounded degree over any eld under the constant bounded product distribution. Learning multivariate polynomials with bounded term size from membership queries only. This in particular gives a learning algorithm for O(log n)-depth decision tree from membership queries only and a new learning algorithm of any multivariate polynomial over su ciently large elds from membership queries only. We also show that all the results we have for learning from membership queries only are the best that can be achieved. We also prove learnability of Xor of decision trees and sum of decision trees with leaves that contain values from some eld. Two techniques were used in the literature for PAC-learning decision tree with membership queries. Kushilevitz and Mansour gave in KM93] a technique for learning decision trees under the uniform distribution via the Fourier Spectrum. Jackson in J94] extended the result to learning DNF under the uniform distribution. The output hypothesis is a majority of parities. Bshouty gave in Bs94] a technique for learning decision trees under any distribution via the Monotone Theory. Schapire and Sellie gave in SS93] a Lattice based algorithm for learning multivariate polynomials over the binary eld under any distribution. In the former the output hypothesis for the decision tree is depth 3 formulas. Jackson J95] generalizes his DNF learning algorithm from uniform distribution to any xed constant bounded product distribution. This is a distribution where there is a constant 0 < c < 1=2 that is independent of the number of variables n such that for any variable x i the probability that x i = 1 is bounded between c and 1 ? c.. Both techniques, the Fourier Spectrum and the Lattice based algorithms give also learnability of many other classes such as learning decision trees over parities (nodes contains parities) under constant bounded product distributions and learning CDNF (poly size DNF that has poly size CNF) under any distribution.. In this paper we use a new approach to learning decision trees and multivariate polynomials via multivariate interpolation. This new approach leads to simple learning algorithms for decision trees over constant bounded product distributions that output hypotheses that is a multivariate polynomial (parity of terms). The algorithm we develop gives a single hypothesis that approximate the target against any constant bounded product distribution. In fact the hypothesis is a good hypothesis under any distribution that supports small terms. This is a distribution D that satis es Pr D T = 1] = 1=!(poly(n)) for any term T of size !(log n) where n is the number of variables.. The new approach also solves learnability of many other problems, some of which cannot be solved using the previous known approaches. These problems include 1. PAC-learning with membership queries of multivariate polynomials over the binary eld with nonmonotone terms under any distribution that support small terms.. In particular disjoint DNF (the conjunction of every two terms is 0), j-disjoint DNF (the conjunction of any j terms is 0) for constant j, decision trees and any polynomial number of Xor of them are PAC-learnable with membership queries under any distribution that supports small terms. The output hypotheses of the learning algorithm is a multivariate polynomial.. It is known that multivariate polynomials (with monotone terms) are PAC-learnable with membership queries under any distribution SS93]. Our contribution is to show the same when the terms are not nesasary monotone. It is also known that any DNF is PAC-learnable with membership queries under constant bounded product distribution J94]. In J94] the output hypothesis is a majority of parities. Our contribution for j-disjoint DNF is to use an output hypothesis that is a parity of terms and to show that the output hypothesis is an approximation of the target against any constant bounded distribution. Also, the learnability of a Xor of these classes cannot be achieved using previous known approaches.. We also have an extension of (1.) to any eld. We show 2. PAC-learning with membership queries of decision trees with leaves from some eld F and any polynomial sum of them under any distribution that support small terms.. We also study the learnability of multivariate polynomials from membership queries only. We show . This is a linear system of equations and can be solved as follows. is not zero will be proven in the next section.. 3 Simple Algorithm for the Boolean Domain. Let MUL(n; t) be the set of all multivariate polynomials over the binary eld over n variables with t terms where each term is of size O(log n). This class includes decision trees of depth O(log n) and j-disjoint O(log n)-DNF.. We rst show how to zero-test elements in MUL(n; t). Let f 2 MUL(n; t). Choose a term T = x i1 x ik of maximal size in f. Randomly and uniformly choose values from f0; 1g for the variables not in T. The projection will not be zero because the term T will stay alive in the projection. Since the projection is a nonzero function with k = O(log n) variables there is at least one assignment for x i1 ; : : :; x ik that gives value 1 for the function. This show that for a random and uniform assignment a, f(a) = 1 with probability at least 1=2 k = 1=poly(n). So to zero test a function in f 2 MUL(n; t) randomly and uniformly choose polynomial number of assignments a i . If f(a i ) is zero for all the assignments then with probability close to 1 we have f 0. We now show how to reduce zero-test to learning. Let f 0 = f. Since we can zero-test we can nd the minimal i 1 such that f 0 j x1 0;:::;:::;xi 1 0 0. This implies that f x1 0;:::;:::;xi 1 ?1 0 = x i1 f 1 (x i1+1 ; : : :; x n ) for some multivariate polynomial f 1 2 MUL F (n; k; t; d). Now do the same for f 1 = f x1 0;:::;:::;xi 1 ?1 0;xi 1 1 : At some stage we get f 0 = x i1 x i2 x ik f k . If f k 1 then x i1 x ik is a term in f. Now de nê f = f + x i1 x ik . This removes this term from f. Then do the same forf to get a new term. This way we collect all the terms of f. Now let MUL be the set of all boolean multivariate polynomials with polynomial number of nonmonotone terms. This class includes the class of decision trees and jdisjoint DNF.. Let f 2 MUL. To PAC-learn f we randomly choose an assignment a and de ne f 0 = f(x + a). This make sure that (large) terms in f of size k will have in average k=2 positive literals. We now take a zero-restriction. That is we substitute x i 0 in f 0 with probability 1=2. This make sure that with high probability the projection f 00 is in MUL(n; poly(n)). We then use the previous algorithm to learn f 00 . The terms of f 00 are terms in f 0 . We continue to take zero-restrictions and collect small terms of f 0 until we get a good approximation of f 0 . A complete analysis of this algorithm is given in section 6.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O