Inter-core prefetching for multicore processors using migrating helper threads Multicore processors have become ubiquitous in today's systems, but exploiting the parallelism they offer remains difficult, especially for legacy application and applications with large serial components. The challenge, then, is to develop techniques that allow multiple cores to work in concert to accelerate a single thread. This paper describes inter-core prefetching, a technique to exploit multiple cores to accelerate a single thread. Inter-core prefetching extends existing work on helper threads for SMT machines to multicore machines.Inter-core prefetching uses one compute thread and one or more prefetching threads. The prefetching threads execute on cores that would otherwise be idle, prefetching the data that the compute thread will need. The compute thread then migrates between cores, following the path of the prefetch threads, and finds the data already waiting for it. Inter-core prefetching works with existing hardware and existing instruction set architectures. Using a range of state-ofthe-art multiprocessors, this paper characterizes the potential benefits of the technique with microbenchmarks and then measures its impact on a range of memory intensive applications. The results show that inter-core prefetching improves performance by an average of 31 to 63%, depending on the architecture, and speeds up some applications by as much as 2.8×. It also demonstrates that inter-core prefetching reduces energy consumption by between 11 and 26% on average. Although multi-core processors have become ubiquitous over the past decade, leveraging the parallelism they offer to increase application performance remains challenging. There are several reasons for this, including the prevalence of non-parallelized legacy code, the difficulty of parallel programming, and the inherently serial nature of many programs and algorithms. Even in applications that are Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ASPLOS '11, March 5-11, 2011 , Newport Beach, California, USA. Copyright c 2011 ACM 978-1-4503-0266-1/11/03. . . $10.00 amenable to parallelization, the applicability of multicore processors has its bounds: as manufacturers supply an increasing number of cores, more and more applications will discover their scalability limits. Furthermore, Amdahl's law dictates that the more hardware parallelism is available, the more critical sequential performance becomes.. For many applications, then, the only way to get parallel speedup is to exploit non-traditional parallelism -to use multiple cores to accelerate the execution of a single thread. For singlethreaded legacy codes, this allows the multicore processor to increase performance without altering the program. Non-traditional parallelism allows programs with limited parallelism to profitably use several cores per thread, increasing the application's scalability.. Previous work [5, 8, 9, 17, 18, 20, 22, 33] has demonstrated the use of "helper threads" which run concurrently in separate contexts of a simultaneous multithreading (SMT) processor and significantly improve single thread performance. Helper thread prefetching has advantages over traditional prefetching mechanisms -it can follow more complex patterns than either hardware prefetchers or in-code software prefetchers, and it does not stop when the main thread stalls, since the prefetch threads and the main thread are not coupled together. However, SMT prefetching has its limitations. The helper thread competes with the main thread for pipeline resources, cache bandwidth, TLB entries, and even cache space. The helper thread cannot target more distant levels of the cache hierarchy without thrashing in the L1 cache. In addition, core parallelism is typically more abundant than thread parallelism -even on a 4-core Nehalem, only 1 SMT thread is available for prefetching. Perhaps most importantly, not all multicores support SMT.. This paper describes and evaluates helper threads that run on separate cores of a multicore and/or multi-socket computer system. Multiple threads running on separate cores can significantly improve overall performance by aggressively prefetching data into the cache of one core while the main thread executes on another core. We call this technique inter-core prefetching. When the prefetcher has prefetched a cache's worth of data, it moves on to another core and continues prefetching. Meanwhile, when the main thread arrives at the point where it will access the prefetched data, it migrates to the core that the prefetcher recently vacated. It arrives and finds most of the data it will need is waiting for it, and memory accesses that would have been misses to main memory become cache hits.. Inter-core prefetching can target both distant shared caches and caches private to the core. Inter-core prefetching is especially attractive because it works with currently available hardware and system software. Thus, it would be easy to incorporate in a compiler or runtime system. Previous approaches have required fundamental changes to the microarchitecture or were limited to prefetching into shared caches. This paper describes inter-core prefetching and our implementation under Linux. We characterize the potential impact of intercore prefetching on a range of currently-available multicore processors running focused microbenchmarks and then demonstrate its impact on a wide range of memory-intensive applications. Our results show that inter-core prefetching improves performance by an average of 31 to 63%, depending on the architecture, and speeds up some applications by as much as 2.8×. We also demonstrate that inter-core prefetching reduces energy consumption by between 11 and 26% on average. Finally, we show that because of the separation of resources, inter-core prefetching is more effective than SMT thread prefetching.. The remainder of this paper is organized as follows: Section 2 places inter-core prefetching in context with other work on nontraditional parallelism. Section 3 describes the inter-core prefetching technique in detail. Sections 4 and 5 contain our methodology and results, and Section 6 concludes. This paper describes inter-core prefetching, a technique that allows multiple cores to cooperatively execute a single thread. Intercore prefetching distills key portions of the original program into prefetching threads that run concurrently with the main thread but on a different core. Via lightweight migration, the main thread follows the prefetching threads from core to core and finds prefetched data waiting for it in its caches. Our results show that inter-core prefetching can speed up applications by between 31 and 63%, depending on the architecture, and that it works across several existing multi-core architectures. Our results also show that, although it uses multiple cores, it can reduce energy consumption by up to 50%. Inter-core prefetching demonstrates that it is possible to effectively apply helper thread-based techniques developed for multithreaded processors to multi-core processors, and even overcome several limitations of multithreaded prefetchers. We evaluate inter-core prefetching in four stages. First, we use simple microbenchmarks to measure its potential and understand some of its tradeoffs. Then, we evaluate the technique's applicationlevel impact using the workloads described in Section 4. Next, we evaluate inter-core prefetching's effect on power and energy consumption. Finally, we compare inter-core prefetching's performance across cores to a version of the technique that targets SMT contexts, and also with data spreading, another software-only cache optimization. Prefetching is an important technique to speed up memoryintensive applications. There has been significant work on both hardware prefetchers [6, 15] and software controlled prefetchers [2, 7, 25] . The introduction of multithreaded and multicore architectures introduced new opportunities for prefetchers. Multithreaded architectures are a natural target, because threads share the entire cache hierarchy.. Helper thread prefetchers [5, 8, 9, 14, 17, 18, [20] [21] [22] 31 , 33] accelerate computation by executing prefetch code in another SMT context. Normally, these techniques predict future load addresses by doing some computation in the helper thread and prefetch the corresponding data to the nearest level of shared cache. In many cases, the prefetch code comes from the main thread, allowing those threads to follow arbitrarily complex prefetch patterns.. There are several ways to generate helper threads. The work in [9] uses hand-built helper threads while, Kim and Yeung [18] describe compiler techniques to generate the reduced version statically with the help of profiling. Zhang, et al. [32] and Lu, et al. [21] describe the generation of helper threads in dynamic compilation systems. Collins, et al. [8] generate the helper threads completely in hardware.. Prior research has also targeted prefetching across cores. In Slipstream Processors [14] , a reduced version of the program runs ahead of the main program in another core. Multiple specialized hardware pipes transport information (including loaded values) from one to the other. Lu, et al. [21] demonstrate helper thread prefetching on a CMP, but that work is limited to prefetching into a cache that is shared by the main core and the prefetch core. Brown, et al. [3] propose changes to CMP hardware and coherence to enable a thread on one core to effectively prefetch for a thread on a separate core. In contrast to these approaches, inter-core prefetching enables cross-core prefetching into private caches with no new hardware support. Since it requires no special hardware support, inter-core prefetching can work between sockets in a single system. It also works across CMPs from multiple vendors.. Gummaraju, et al. [12] implement a compilation framework that maps a program written for stream processors to general purpose processors. In their system, there are three threads -one for execution, one for data prefetching, and one that manages execution and prefetching. They target SMT threads.. Other research attempts to leverage the hardware parallelism of multicores to accelerate serial execution in other ways. Speculative multithreading [19, 23, 28, 30] uses multiple hardware contexts or cores to execute different parts of the serial execution in parallel and then does runtime verification of correctness. Mitosis [28] additionally uses helper threads to precompute dependent data. Prescient instruction prefetch [1] is another helper thread-based technique that improves performance by prefetching instructions instead of data.. Other work aggregates cache capacity in multicore/multiprocessor systems to hide memory latency. Cooperative Caching [4] proposes a hardware technique to aggregate private caches. The technique stores cache lines evicted from one private cache to another private cache and ensures maximum utilization of the cache space by keeping a single copy of the same cache line and performing cache-to-cache transfers as needed. Michaud [24] proposes a hardware mechanism to migrate threads and distribute data over the aggregate cache space.. Data Spreading [16] is a software-only mechanism to aggregate the space of multiple private caches. The technique transforms a program to leverage the aggregate cache capacity via compilerdirected migrations. They show that data spreading can provide both performance and power benefits, if the working set fits in the aggregate cache space. However, data spreading does not provide benefit if the working set is too large or the program does not have a repetitive access pattern. Inter-core prefetching removes these two limitations by prefetching data into the next cache. Because of this, inter-core prefetching's performance is much less sensitive to the actual cache size or access pattern.. Using separate threads to access memory and perform computation is similar in spirit to decoupled access/execute architectures [29] , but inter-core prefetching accomplishes this without specialized hardware. Decoupled access/execute architectures use a queue to pass data from a memory thread to a compute thread. The queue requires that the two threads be tightly coupled, and that the memory thread correctly compute the result of all branches. As a result, there is rarely enough "slack" between the threads to allow the memory thread to issue memory requests far enough in advance to fully hide their latency. The Explicitly-Decoupled architecture [11] uses the queue only for passing branch outcomes and shares the entire cache hierarchy to get the advantage of prefetching. In our scheme, the private cache of a helper core serves as the "queue", but because inter-core prefetching places no ordering constraints on the accesses, it can completely decouple the threads.. Unlike helper thread prefetching, runahead execution [10, 26] does not require extra hardware contexts or cores. During the long latency load operation, instead of blocking, it assumes a dummy value and switches to the runahead mode to continue execution. When the original load operation is satisfied, it switches back to the normal execution. This improves memory level parallelism but still places all of the burden of prefetching, demand misses, and execution on a single core. This section describes the processors and applications we use to experimentally evaluate inter-core prefetching. The next section presents the results of those experiments.

O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Optimising the flow of experiments to a robot scientist with multi-objective evolutionary algorithms A Robot Scientist is a physically implemented system that applies artificial intelligence to autonomously discover new knowledge through cycles of scientific experimentation. Additionally, our Robot Scientist is able to execute experiments that have been requested by human biologists. There arises a multi-objective problem in the selection of batches of trials to be run together on the robot hardware. We describe the use of the jMetal framework to assess the suitability of a number of multi-objective metaheuristics to optimise the flow of experiments run on a Robot Scientist. Experiments are selected in batches, chosen in order to maximise the information gain and minimise the use of resources. The evolutionary multi-objective algorithms evaluated here perform well in finding solutions to this problem, either finding a long, fairly efficient Pareto optimal front, or a shorter, highly efficient Pareto optimal front. A Robot Scientist is a physically implemented robotic system that applies techniques from artificial intelligence to execute cycles of automatic scientific experimentation. Our Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. GECCO'07, Each of the evolutionary multi-objective optimisation methods applied to the problem returned Pareto fronts that contained individuals that represent efficient batches of trials.. The particle swarm method, OMOPSO, and the MOGA PAES returned the most efficient batches, but covered only a fraction of the front. It is notable that both of these approaches experienced particular difficulties with the lower part of the Pareto front (few trials, few plates and less exhausted queue time). This is the area of the front that is most rugged: small changes in the number of trials lead to proportionally large jumps in the number of plates required for example. Both OMOPSO and PAES have aggressive strategies for adding solutions to the archive. This extra selection pressure is likely to be responsible for both the higher performance of these metaheuristics on the experiment flow optimisation problem, and for the lower diversity in the resulting populations. It may be necessary to accommodate a higher number of dominated solutions in the parent population than are retained by these approaches, in order to explore this area of the search space.. Both SPEA2 and NSGA-II also returned similar fronts and both manage to explore the rugged territory in the lower portion of the front. In SPEA2, diversification pressure results from the use of a fitness measure that combines both a dominance measure and a k-nearest neighbour distance measure. The truncation operator also exploits the k-nearest neighbour metric in order to preserve both the diversity of the population, and the solutions that lie at the ends of the front. SPEA2 also admits dominated solutions to the archive whenever the set of non-dominated solutions is smaller than the archive size. The preservation of dominated solutions may be instrumental in the ability of SPEA2 to explore the most rugged areas of the Pareto front. NSGA-II also promotes diversity through the use of a crowding operator.. AbYSS adopts the crowding distance metric from NSGA-II for selecting the individuals that will be copied to the external archive, and the density estimation measure from SPEA2 is used to determine which individuals should be in the reference set. As such it is unsurprising that the diversity of the populations returned by AbYSS should approach that of both NSGA-II and SPEA2.. However, AbYSS is still foiled by the rugged, lower end of the front. This is perhaps the result of maintaining two reference sets: one that privileges diversity and another that privileges dominance. It may be that the segregation of unfit but diverse solutions prevents the development of better solutions in the lower portion of the front, by reducing the opportunity for "diverse" individuals to mate with fitter individuals. Further work is required to determine if this is the case.. AbYSS manages to improve on the Pareto optimality of the front returned by both NSGA-II and SPEA2. The scatter search improvement method, which here, as in PAES, takes the form of a (1+1) evolution strategy, appears to exert a notable pressure towards more optimal solutions by systematically searching the area around each point.. The decision as to which of these metaheuristics best serves our needs for the experiment flow optimisation problem depends on the use to which the population of individuals will be put. A Robot Scientist must select a batch of trials from among the population of batches that allows it to make the best use of the resources available. From the resulting Pareto front, a Robot Scientist must choose which batch of experiments to run together. This may be done by selecting the batch that is most efficient on one of the given objective dimensions from among those Pareto optimal individuals in the front. The dimension chosen depends on the availability of resources or the number of trials queued.. We do not know, a priori, what the optimal batches for a given set of trials would be. We do know that an individual batch that contains more trials, eliminates more queuing time and requires fewer plates is preferred over a batch that is weaker in any one or more of those dimensions. To that end, we prefer a front that contains more efficient individuals over one that maintains diversity. OMOPSO and PAES perform better in this respect, and OMOPSO runs slightly faster than PAES.. We have, at this stage, carried out only qualitative evaluation of the performance of these metaheuristics on the experiment flow optimisation problem. Unary quantitative comparisons such as Success Counting and Inverted Generational Distance (see [9] for examples) are not possible as the true Pareto front of this real world problem is not known. Future work will include binary quantitative comparisons, such as Two Set Coverage and Two Set Hypervolume (examples also available in [9] ) on the most promising metaheuristics.. Future work will also look at the effect of tailoring the metaheuristics by using different crossover and mutation operators. It is likely that the metaheuristics examined perform differently when using different operators, and that gains and losses in performance may not be uniform across all metaheuristics. We will also attempt to discover the effect of "tuning" the parameters of the metaheuristics and operators. Finally we will examine the suitability of the evolutionary optimisation approach to solving a problem that also includes hard constraints, such as the limits on the numbers of nutrients that can be dispensed simultaneously. We will investigate recent advances in constraint handling for multi-objective evolutionary algorithms, such as the ones described in [5] .. This paper demonstrates that evolutionary multi-objective optimisation algorithms are an appropriate method of solving the experiment flow optimisation problem. We have also demonstrated that there are differences in the performance of these algorithms on the experiment flow optimisation problem, and that an aggressive approach is needed, even at the expense of diversity, in order to find highly efficient batches of trials. At this stage, we restrict ourselves to a qualitative analysis of the performance of these metaheuristics. Figures 4 to 7 show the comparative performance of the metaheuristics we have applied to the experiment flow optimisation problem. In future work we intend to carry out quantitative comparisons of these metaheuristics, via binary quantitative comparisons, such as Two Set Coverage and Two Set Hypervolume (examples of these methods are available in [9] ).. Each set of points is the set of results found in 20 runs over the set of 100 trials. The more closely these points coalesce around a line, the more reliably the metaheuristic returns This section presents the results for the metaheuristics evaluated. Section 7 discusses the implications of these results in depth.. Figures 4 and 5 show the behaviour of the three MOGAs: PAES, NSGA-II and SPEA2. SPEA2 and NSGA-II achieve similar results. The fronts that they find are near-identical, with SPEA2 covering more of the Pareto front. PAES covers much less of the Pareto front. However, the fronts that PAES identifies are superior to those uncovered by either NSGA-II or SPEA2. Figures 6 and 7 show the performance of the the hybrid scatter search metaheuristic, AbYSS, and the particle swarm optimiser, OMOPSO, with SPEA2 shown for the purpose of comparison. AbBYSS finds a similar front to both NSGA-II and SPEA2. However, it truncates the front, and thus does not return values for the lower boundaries of the Pareto front. Nevertheless, it does result in a somewhat better front than either NSGA-II or SPEA. OMOPSO ( Figure 7 ) results in a population that covers less of the front than those populations identified by SPEA2 and NSGA-II, but it identifies superior points to either of these methods, and to AbYSS. The fronts that OMOPSO returns are similar to those returned by PAES.. Mean runtimes were calculated for the five metaheuristics. The PSO OMOPSO was run over 250 iterations rather than the 25,000 generations that were run for the MOGAs and AbYSS. Performance at this number of iterations was similar to the performance of the other metaheuristics. Runtime for each metaheuristic is, on average, less than two minutes, with AbYSS and OMOPSO having mean runtimes under one minute. For a batch process, all of these runtimes are acceptable.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Finite volume methods for unidirectional dispersive wave models Abstract. We extend the framework of the finite volume method to dispersive unidirectional water wave propagation in one space dimension. In particular we consider a KdV-BBM type equation. Explicit and IMEX Runge-Kutta type methods are used for time discretizations. The fully discrete schemes are validated by direct comparisons to analytic solutions. Invariants conservation properties are also studied. Main applications include important nonlinear phenomena such as dispersive shock wave formation, solitary waves and their various interactions. Water wave modeling is a complicated process and usually leads to models which are hard to analyze mathematically as well as to solve numerically. Under certain simplifying assumptions approximate models are obtained, e.g. the KdV equation [28] , the BBM equation [4] and Boussinesq systems [11, 40, 8] . All these models assume the wave to be weakly nonlinear and weakly dispersive, propagating mainly in one space direction. These approximate models consider mainly unidirectional or bidirectional wave propagation on flat or complex bathymetries.. In this paper we study the application of some finite volume schemes to a scalar nonlinear dispersive partial differential equation modeling unidirectional wave propagation. Specifically, we consider the KdV-BBM equation in its general form: u t + α u x + β uu x − γ u xxt + δ u xxx = 0, (1.1) for x ∈ R, t > 0, where α, β, γ, δ are positive real numbers, [4] . The finite volume method is well known for its accuracy, efficiency, robustness and excellent local conservative properties. Most often this method is employed to approximate solutions to hyperbolic conservation laws. The system of Nonlinear Shallow Water Equations (NSWE) is a classical example of the successful application of modern finite volume schemes to water wave problems. A wide range of numerical methods have been employed to compute approximate solutions to dispersive wave equations of KdV-BBM type : finite difference schemes [10, 50] , finite element methods [9, 33, 2] and spectral methods [38, 39, 15, 35] . Recently discontinuous Galerkin schemes have also been employed to dispersive wave equations [51, 30, 18] , (the list is far from being exhaustive). However, the application of finite volume or hybrid FV/FD methods remain most infrequent for this type of problems. To our knowledge, only a few recent works are in this direction [3, 17, 6, 47, 42, 12] .. In order to apply the finite volume method to the KdV-BBM equation (1.1), we rewrite it in a conservative form, including a nontrivial evolution operator, an advective and a dispersive flux functions. In the finite volume literature there exist several ways to approximate these fluxes. For the advective part we test three different numerical fluxes each one representing a particular family of finite volume method:. • average flux (m-scheme),. • central flux, (KT-scheme) as a representative of central schemes, [34, 29] , • characteristic flux (CF-scheme), as a representative of upwind schemes and linearized Riemann solvers, [20, 22] .. The dispersive term is discretized using simply the average flux, while high order approximations are used for the BBM term (γu xxt ). The central flux and the characteristic flux are widely used in the case of conservation laws. On the other hand the average flux, known to be unstable for conservation laws, performs equally well. The evaluation of the numerical flux functions require approximate values of the solution at the cell interfaces. The order of the approximation determines the space accuracy of the underlying finite volume scheme. We consider first order, taking simply piecewise constant approximations, as well as high order schemes. The high order accuracy is achieved through application of various reconstruction techniques such as TVD [46] , UNO [26] and WENO [31] .. The time discretization of (1.1) is based on Runge-Kutta methods. The stability of the resulting system of ode's depends on the interplay between the BBM term (γ u xxt ) and the KdV type dispersive term (δ u xxx ). An explicit discretization of the ode system is sufficient when these terms are of the same order. Thus, Strong Stability Preserving Runge-Kutta (SSP-RK) methods, which preserve the TVD property of the finite volume scheme, [44, 24] are used for the explicit discretization.. However, when γ ≪ δ the resulting semidiscrete system of ode's is highly stiff and therefore implicit methods with strong stability characteristics are preferable. To balance the high computational cost of fully implicit methods and stability considerations we rely on Implicit-Explicit Runge-Kutta (IMEX) methods, [1] . Indeed IMEX RK methods turned out to be well suited for the time discretization of the KdV-BBM equation (1.1) exhibiting excellent stability behavior.. The validated numerical method is applied to study the KdV-BBM equation (1.1) in a systematic way through a series of numerical experiments. In particular, we focus on the following issues:. • accuracy of the finite volume method for solitary wave propagation and invariants conservation • dispersive shock formation (we underline that the finite element as well as spectral methods break down for this experiment while the finite volume method provides robust and accurate results) • interactions of solitary waves (overtaking collisions) The paper is organized as follows. In Section 2 the governing equation (1.1) is presented briefly along with its basic properties. In Section 3 the finite volume discretization as well as fully discrete schemes are presented in details. In Section 4 we validate the discretization procedure by comparisons with analytical solution. Several important test cases are also presented. The main scope of the present article is to extend the framework of finite volume methods to scalar unidirectional dispersive models. We chose the celebrated BBM-KdV equation (2.1) as an important representative model arising in the water wave theory and having all main features of dispersive wave equations.. The BBM-KdV equation can be also viewed as a dispersive perturbation of the inviscid Burgers equation. Consequently, our method relies on classical finite volume schemes which discretize the advection operator. Then, a special treatment was proposed for the KdVdispersion term, while the BBM-dispersion required an elliptic operator inversion per each time step, hence, providing a physical regularization to numerical solutions. We propose and implement also several methods to obtain high order accurate schemes.. The proposed discretization procedure is validated by comparisons with an analytical solitary wave solution. The order of convergence is measured as well as invariant preservation is studied extensively. The numerical method is applied to several important test cases such as a solitary wave propagation and a dispersive shock formation. We make also use of proposed higher order extensions to study the overtaking solitary waves collision for the KdV-BBM equation.. The extension to more realistic bi-directional wave propagation models such as Boussinesq type equations [40, 36, 32, 16] .

O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O


Regularized Interpolation for Noisy Images Abstract-Interpolation is the means by which a continuously defined model is fit to discrete data samples. When the data samples are exempt of noise, it seems desirable to build the model by fitting them exactly. In medical imaging, where quality is of paramount importance, this ideal situation unfortunately does not occur. In this paper, we propose a scheme that improves on the quality by specifying a tradeoff between fidelity to the data and robustness to the noise. We resort to variational principles, which allow us to impose smoothness constraints on the model for tackling noisy data. Based on shift-, rotation-, and scale-invariant requirements on the model, we show that the -norm of an appropriate vector derivative is the most suitable choice of regularization for this purpose. In addition to Tikhonov-like quadratic regularization, this includes edge-preserving total-variation-like (TV) regularization. We give algorithms to recover the continuously defined model from noisy samples and also provide a data-driven scheme to determine the optimal amount of regularization. We validate our method with numerical examples where we demonstrate its superiority over an exact fit as well as the benefit of TV-like nonquadratic regularization over Tikhonov-like quadratic regularization.

O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O B


Improving the effectiveness of collaborative recommendation with ontology-based user profiles Collaborative recommendation is effective at representing a user's overall interests and tastes, and finding peer users that can provide good recommendations. However, it remains a challenge to make collaborative recommendation sensitive to a user's specific context and to the changing shape of user interests over time. Our approach to building context-sensitive collaborative recommendation is a hybrid one that incorporates semantic knowledge in the form of a domain ontology. User profiles are defined relative to the ontology, giving rise to an ontological user profile. In this paper, we describe how ontological user profiles are learned, incrementally updated, and used for collaborative recommendation. Using book rating data, we demonstrate that this recommendation algorithm offers improved coverage, diversity, personalization, and cold-start performance while at the same time enhancing recommendation accuracy. Recommender systems [2] have become essential tools in assisting users to find what they want in increasingly complex information spaces. Collaborative recommenders predict a target user's interest in particular items based on the opinions or preferences of other users [21] . Typical collaborative recommenders rely on profiles of users represented as Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. flat vectors of ratings or preference scores. Thus, the same collection of user preferences across all items or resources is used as the basis for generating recommendations regardless of the user's current information context or task-related needs.. Consider the following example. Suppose Steve buys and rates mystery-detective fiction novels for his own entertainment ("Da Vinci Code"), books on computer science topics ("Python Programming") for work-related purposes, and children's books ("Green Eggs and Ham") for his daughter. It makes little sense to represent Steve's interest in books in a single representation that aggregates all of these disparate interests without some acknowledgment that they represent different sorts of needs and contexts. To recommend effectively to a user with diverse and changing interests, three aspects of the recommendation context must be addressed: the short-term user activity, representing immediate user interests; long-term user profiles, representing established preferences; and existing ontologies that provide an explicit representation of the domain of interest. With this knowledge, a system can leverage a variety of sources of evidence to provide the best personalized experience for the user, including both the semantic evidence associated with the user's individual interaction, as well as social knowledge derived collaboratively from peer users.. In this paper, we present an approach to collaborative recommendation that effectively incorporates semantic knowledge from ontologies with collaborative user preference information. Our novel approach provides improved recommendation diversity as well as robustness with the coldstart problem. The salient feature of our framework is the notion of ontological user profiles which are instances of a pre-existing domain ontology (e.g. book ontology) with numerical annotations associated with concepts derived from users' past behavior and preferences. Rather than being associated with single atomic entities like individual books, users' choices and preferences are associated with relevant concepts in the ontology. So, the fact that Steve buys computer books such as "Python Programming" can be readily distinguished from his interest in children's books because they occupy disparate places in the book ontology.. We present an algorithm based on spreading activation to incrementally update these user profiles, as a result of ongoing user interaction, in a way that takes into account relationships among concepts in the ontology as well as the collaborative evidence derived from the ontological profiles of similar users. Our approach to recommendation generation is an extension of the standard user-based collaborative framework. In our case, user similarities are computed based on their interest scores across ontology concepts, instead of their ratings on individual items.. When considering the performance of recommender systems, there is a tradeoff between accuracy and diversity, because high accuracy may often be obtained by safely recommending to users only the most popular items [1, 24] . Our approach not only outperforms traditional collaborative filtering in prediction accuracy but also offers improvements in coverage, recommendation diversity, personalization, and cold-start performance.. The rest of this paper is organized as follows. We begin by presenting some related work and background. Next, we discuss our motivation for augmenting collaborative filtering by incorporating domain knowledge in the form of an ontology. We present our specific algorithms for ontologybased collaborative recommendation. After a discussion of the datasets and a description of the experimental methodology, we present the results of our evaluation. Finally, we share our conclusions and outlook. Research in the area of recommender systems focuses on collaboration-based, content-based, and hybrid approaches. Widely used collaborative filtering methods can be divided into two main categories including memory-based (user-based) and model-based algorithms [18] . User-based techniques [10] generally model the user as a vector of item ratings and compare these vectors using a correlation or similarity measurement. Model-based techniques use a variety of methods to build compact representations of the rating data to reduce noise [12] .. Content-based filtering methods [3, 15, 16] have also been used in the context of recommending books and Web pages, where content descriptors are available. Rather than using simple feature vector models, our work differs from existing approaches by taking advantage of the deeper semantic knowledge in an existing ontology.. Many recommender systems suffer from the cold-start problem of handling new items or new users. With collaborative techniques, items that are new to the system cannot be recommended until they have been rated by several users. Similarly, generating recommendations for new users that have rated few items is very challenging. In addition, data sparsity and scalability are among the most common issues for user-based collaborative filtering models [20] . Hybrid recommenders [5] combine semantic or content-knowledge with collaborative filtering to deal with these shortcomings. Knowledge-based recommender systems use knowledge about users and products to pursue a knowledge-based approach to generating a recommendation, reasoning about what products meet the user's requirements [4] . Our work can be described as a knowledge-based collaborative hybrid.. The availability of large product taxonomies such as Amazon.com and the Open Directory Project has allowed researchers to incorporate semantic information into recommender systems [14] . In order to address rating sparsity, Ziegler et al. [25] classify products by topics based on taxonomic information. Cho and Kim [7] have utilized a product taxonomy to overcome scalability issues. In [19] , spreading activation techniques are used to find related concepts in the ontology given an initial set of concepts and corresponding initial activation values.. In our approach, the hierarchical structure of an ontology is used explicitly and automatically in the learning and incremental updating of user profiles. There has been little work in the area of ontological user modeling and even less in the application of such models to Web personalization [9] . Our research follows the lead of other systems [8] that use ontologies to mediate information access, but these systems have generally not incorporated user modeling.

O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Economical active feature-value acquisition through Expected Utility estimation In many classification tasks training data have missing feature values that can be acquired at a cost. For building accurate predictive models, acquiring all missing values is often prohibitively expensive or unnecessary, while acquiring a random subset of feature values may not be most effective. The goal of active feature-value acquisition is to incrementally select feature values that are most cost-effective for improving the model's accuracy. We present two policies, Sampled Expected Utility and Expected Utility-ES, that acquire feature values for inducing a classification model based on an estimation of the expected improvement in model accuracy per unit cost. A comparison of the two policies to each other and to alternative policies demonstrate that Sampled Expected Utility is preferable as it effectively reduces the cost of producing a model of a desired accuracy and exhibits a consistent performance across domains.Algorithms In many predictive modeling problems, feature values for training data are missing, but can be acquired at a cost. Often the cost of acquiring the missing information varies according to the nature of the information or of the particular instance for which information is missing. Consider, for example, patient data used to induce a model to predict whether or not a treatment will be effective for a given patient. Some patient data may have missing demographic Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. information that can be obtained at a low cost. In contrast, acquiring diagnostic test results from different health-care providers can be significantly more expensive and time-consuming. Various solutions are available for learning models from incomplete data, such as imputation methods [8] , and learners that ignore missing feature values such as the Naive Bayes classifier. However, these solutions almost always undermine model performance as compared to that of a model induced from complete information. Since obtaining all missing values may be prohibitively expensive, it is desirable to identify what information would be most cost-effective to acquire. In this paper we address this generalized version of the active feature-value acquisition (AFA) task for classifier induction [10] : given a model built on incomplete training data, select feature values that would be most cost-effective to acquire for improving the model's accuracy. The problem of feature-value acquisition is different from traditional active learning [2] in which class labels rather than feature values are missing and are costly to acquire.. Unlike prior work [9] , we study AFA in a setting where the total cost to be spent on acquisitions is not determined a priori, but rather can be determined on-line based on the model's performance as learning progresses. This setting is motivated by the inherent uncertainty regarding the tradeoff between costs and improvement in model accuracy. An incremental spending strategy enables a decision maker to re-evaluate the desirability of further expenditures by incrementally exploring the performance curve resulting from a series of acquisition decisions. For example, one may choose not to acquire additional information if the current model accuracy is satisfactory, or if additional information is unlikely to provide a significant improvement in the model. We propose a general setting for AFA that specifies an incremental acquisition schedule. Given the current model, an AFA strategy identifies feature-value acquisitions that are estimated to be most cost-effective with respect to model accuracy.. We present a solution to the AFA task that ranks alternative featurevalue acquisitions based on an estimation of the expected improvement in model performance per unit cost. Our approach is general, i.e., it can be applied to select acquisitions for any learner, and to attempt to improve any performance metric. Experimental results on decision tree induction to improve classification accuracy demonstrate that our method does consistently result in significantly improved model accuracy per unit cost compared to random feature-value acquisition. The method is particularly advantageous in challenging tasks for which there is a significant variance across potential acquisitions with respect to their contribution to learning per unit cost. In this paper, we propose an expected utility approach to active feature-value acquisition, that obtains feature values based on the estimated expected improvement in model accuracy per unit cost. We demonstrate how this computationally intensive method can be made significantly faster, without much loss in performance, by constraining the search to a sub-sample of potential feature-value acquisitions. Experiments with uniform feature costs show that this Sampled Expected Utility approach consistently builds more accurate models than random sampling for the same number of featurevalue acquisitions, and exhibits consistent performances across domains as compared to policies employing an instance-based ranking of features. Additional experiments on artificial datasets with different cost structures demonstrate that for the same cost, Sampled Expected Utility builds more accurate classifiers than the costagnostic random feature acquisition approach. Its performance is also more consistent than that of a simple cost-sensitive method which acquires feature values in order of increasing cost. Our results are presented in Figure 1 . For all datasets, Sampled Expected Utility builds more accurate models than random sampling for any given number of feature acquisitions. These results demonstrate that the estimation of the expected improvement in the current model's accuracy enables effective ranking of potential queries. Consequently, Sampled Expected Utility selects queries that on average are more informative for the learner than an average query selected at random. The differences in performance between these two systems on all datasets is significant, as defined above. Since Sampled Expected Utility was proposed in order to reduce the computational costs of our original Expected Utility approach, we also examined the performance and computational time of the exhaustive Expected Utility algorithm for vote. We computed the average time it took to select queries in each iteration for each of the methods. These timing results are summarized in Table 2 . The results show that constraining the search in Expected Utility by random sampling (or Error Sampling) can significantly reduce the selection time (by two orders of magnitude in this case) without a significant loss in accuracy.. While Error Sampling can rank acquisitions of complete instances effectively, it does not consider the value of individual feature values. Despite this, we observed that Error Sampling performs quite well. In particular, it often performs significantly better than random sampling and it sometimes performs better than Sampled Expected Utility. However, the performance of Error Sampling in this general setting of AFA is inconsistent, as it may perform significantly worse than random selection, as is seen on the lymph dataset. These results suggest that the use of Error Sampling in our current AFA setting is a promising direction for future work, but is dependent on improving the Error Sampling strategy such that Error Sampling consistently performs better than random selection. Note that, in the instance-completion setting of AFA for which Error Sampling was originally designed, it always performs better than random [10] . In summary, Expected Utility-ES often exhibits superior performance with respect to Sampled Expected Utility and random selection. However, it is susceptible to the inconsistent performance of Error Sampling and thus may potentially perform worse than random sampling. On the other hand, Sampled Expected Utility exhibits consistent improvements over random sampling on all datasets. To the best of our knowledge, the methods we propose here are the first approaches designed for the general problem of incrementally ranking and selecting feature values for inducing any classifier under a general acquisition cost structure. In this section, we discuss alternate settings for the AFA task.. Lizotte et al. [9] study AFA in the budgeted learning scenario, in which the total cost to be spent towards acquisitions is determined a priori and the task is to identify the best set of acquisitions for this cost. In contrast, our setting aims to enable the user to stop the acquisition process at any time, and as such the order in which acquisitions are made is important. Given this criterion, we attempt to select the next acquisition that will result in the most accurate model per unit cost. Lizotte et al. also assume that feature values are independent given the class, and as such consider queries of the form "Give me the value of feature j for any instance in class k." However, our approach evaluates feature-value acquisitions of specific instances, which allows us to 1) incorporate feature-value costs that vary per instance; and 2) to better estimate the expected value of an acquisition by capturing improvements from better modeling of feature interactions. Note that a set of features may exhibit different interactions for different instances, in which case evaluating potential acquisitions for individual instances is critical.. In this paper, we explored the use of the Error Sampling policy designed for the instance-completion setting, in which all missing feature values are acquired for a selected training instance [17, 10] . Sampled Expected Utility selects individual features, and hence can be also employed in the instance-completion setting, e.g., by selecting the instance with the highest sum of utilities of individual feature-value acquisitions.. Some work on cost sensitive learning [15] has addressed the issue of inducing economical classifiers when there are costs associated with obtaining feature values. However, most of this work assumes that the training data are complete and focuses on learning classifiers that minimize the cost of classifying incomplete test instances. An exception, CS-ID3 [13] , also attempts to minimize the cost of acquiring features during training; however, it processes examples incrementally and can only request additional information for the current training instance. CS-ID3 uses a simple greedy strategy that requests the value of the cheapest unknown feature when the existing hypothesis is unable to correctly classify the current instance. It does not actively select the most useful information to acquire from a pool of incomplete training examples. The LAC* algorithm [5] also addresses the issue of economical feature acquisition during both training and testing; however, it also adopts a strategy that does not actively select the most informative data to collect during training. Rather, LAC* simply requests complete information on a random sample of instances in repeated exploration phases that are intermixed with exploitation phases that use the current learned classifier to economically classify instances.. Traditional active learning [2, 4] assumes access to unlabeled instances with complete feature data and attempts to select the most useful examples for which to acquire class labels. Active featurevalue acquisition is a complementary problem that assumes labeled data with incomplete feature data and attempts to select the most useful additional feature values to acquire. We begin by evaluating our proposed approaches on four datasets from the UCI repository [1] , the details of which are presented in Table 1 . For the sake of simplicity, we selected datasets that have only nominal features. In the future work section, we describe how we can extend our approach to handle numeric features. None of the UCI datasets provide feature acquisition costs -in our initial experiments we simply assume all costs are equal. Later, we present additional experiments with different cost structures. We compare all the proposed methods to random feature acquisition, which selects queries uniformly at random to provide a representative sample of missing values. For the Sampled Expected Utility and Expected Utility-ES we set the exploration parameter α to 10. Given the computational complexity of Expected Utility it is not feasible to run the exhaustive Expected Utility approach on all datasets. However, we did run Expected Utility on the vote dataset. For all methods, as a base learner we used J48 decisiontree induction, which is the Weka [16] implementation of C4.5 [11] . Laplace smoothing was used with J48 to improve class probability estimates.. The performance of each acquisition scheme was averaged over 10 runs of 10-fold cross-validation. In each fold of cross-validation, we generated learning curves in the following fashion. Initially, the learner is given a random sample of feature values, i.e. the instance-feature matrix is partially filled. The remaining instancefeature pairs are used to initialize the pool of available queries. At each iteration, the system selects a batch of queries, and the values for these features are acquired. This process is repeated until a desired number of feature values is acquired. Classification accuracy is measured after each batch acquisition in order to generate a learning curve. One system (A) is considered to be significantly better than another system (B) if the average accuracy across the points on the learning curve of A is higher than that of B according to a paired t-test (p < 0.05). As in [10] , the test data contains only complete instances, since we want to approximate the true generalization accuracy of the constructed model given complete data for a test instance. For each dataset, we selected the initial random sample size to be such that the induced model performed at least better than majority class prediction. The batch size for the queries was selected based on the difficulty of the dataset. For problems that were harder to learn, we acquired a larger number of feature-values and consequently used larger batch sizes.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Risk

Assessment

for

One-Counter

Threads

Abstract

Threads

as

contained

in

a

thread

algebra

are

used

for

the

modeling

of

sequential

program

behavior.

A

thread

that

may

use

a

counter

to

control

its

execution

is

called

a

'one-counter

thread'.

In

this

paper

the

decidability

of

risk

assessment

(a

certain

form

of

action

forecasting)

for

one-counter

threads

is

proved.

This

relates

to

Cohen's

impossibility

result

on

virus

detection

(Comput.

Secur.

[22][23][24][25][26][27][28][29][30][31][32][33][34][35]

Our

decidability

result

follows

from

a

general

property

of

the

traces

of

one-counter

threads:

if

a

state

is

reachable

from

some

initial

state,

then

it

is

also

reachable

along

a

path

in

which

all

counter

values

stay

below

a

fixed

bound

that

depends

only

on

the

initial

and

final

counter

value.

A

further

consequence

is

that

the

reachability

of

a

state

is

decidable.

These

properties

are

based

on

a

result

for

ω-one

counter

machines

by

Rosier

and

Yen

(SIAM

J.

Comput.

Threads

as

contained

in

a

thread

algebra

are

used

for

the

modeling

of

sequential

program

behavior.

Threads

may

make

use

of

services

such

as

a

counter

or

a

stack

to

control

their

execution.

A

regular

thread

using

a

counter

is

called

a

'one-counter

thread'.

One-counter

threads

form

a

proper

subclass

of

the

pushdown

threads

as

studied

in

i.e.

regular

threads

that

may

use

a

stack

over

a

finite

sort.

In

that

paper

it

is

shown

that

equality

of

pushdown

threads

is

decidable,

while

their

inclusion

is

not.

The

latter

undecidability

result

even

holds

for

one-counter

threads

(of

course,

a

counter

is

similar

to

a

stack

over

a

one-element

data

type)..

In

this

paper

the

decidability

of

a

certain

form

of

risk

assessment

for

one-counter

threads

is

proved.

We

assume

that

risks

are

modeled

by

the

execution

of

an

action

risk,

and

the

question

is

to

forecast

the

execution

of

risk

given

the

specification

of

the

thread

under

execution.

For

pushdown

threads

this

type

of

forecasting

is

decidable:

rename

the

action(s)

to

be

forecast

and

decide

whether

the

thread

thus

obtained

equals

the

original

one.

Forecasting

becomes

much

more

complicated

if

a

thread

may

contain

test

actions

that

yield

a

reply

according

to

the

result

of

this

type

of

forecasting.

For

example,

assume

that

a

thread

may

contain

a

test

action

ok

that

yields

true

if

its

true-branch

does

not

execute

risk,

and

false

otherwise.

Then

a

current

test

action

ok

in

the

code

to

be

inspected

may

yield

true

because

a

future

one

will

yield

false.

The

reply

to

these

ok

actions

can

be

modeled

using

a

risk

assessment

service.

For

regular

threads,

such

a

service

has

a

decidable

reply

function

while

for

pushdown

threads

this

is

still

an

open

question.

In

this

paper

we

show

that

the

associated

service

is

decidable

for

one-counter

threads,

thereby

answering

one

of

the

open

questions

in

Our

modeling

of

risk

assessment-that

is,

the

existence

of

a

test

action

ok

and

a

risk

assessment

service

as

described

above-was

introduced

in

This

modeling

was

inspired

by

Cohen's

well-known

impossibility

result

on

virus

detection

first

described

in

Essentially,

Cohen's

result

establishes

the

non-existence

of

a

test

that

decides

whether

or

not

a

code

fragment

is

virus-free.

This

point

will

be

elaborated

on

in

our

conclusions..

In

technical

terms

the

decidability

result

of

the

present

paper

is

based

on

a

reachability

result

for

so-called

ω-one

counter

machines,

proved

by

Rosier

and

Yen

in

This

result

implies

a

general

property

of

the

traces

of

one-counter

threads:

if

a

state

is

reachable

from

some

initial

state,

then

it

is

also

reachable

along

a

path

in

which

all

counter

values

stay

below

a

fixed

bound

that

depends

only

on

the

initial

and

final

counter

value..

The

paper

is

structured

as

follows.

In

the

next

section

we

introduce

the

basics

of

thread

algebra

and

services.

Then,

in

Sect.

we

discuss

one-counter

threads

and

the

above-mentioned

reachability

result.

In

Sect.

we

recall

that

risk

assessment

for

regular

threads

is

decidable

and

come

up

with

a

new

short

proof

of

this

fact.

Furthermore,

we

use

our

proof

technique

to

show

the

decidability

of

risk

assessment

for

one-counter

threads.

In

Sect.

we

prove

some

further

reachability

results

for

onecounter

threads.

Finally,

in

Sect.

we

end

with

some

conclusions.

Risk

assessment

as

investigated

in

this

paper

has

two

interesting

characteristics.

First,

it

offers

an

alternative

to

Cohen's

seminal

on

the

impossibility

of

virus

detection

The

crucial

twist

in

our

approach

is

that

we

use

a

test

that

only

establishes

whether

its

true-branch

is

risk-free

and

that

does

not

evaluate

its

false-branch,

thus

resisting

arguments

that

are

based

on

self-reference

(this

form

of

risk

assessment

was

first

proposed

in

Secondly,

exploiting

a

result

of

Rosier

and

Yen

we

extended

the

class

of

threads

for

which

risk

assessment

is

decidable

to

the

one-counter

threads..

We

expect

that

risk

assessment

is

also

decidable

for

pushdown

threads

(i.e.,

regular

threads

that

may

use

a

stack),

but

our

proof

does

not

generalize

to

this

case

(some

partial

results

can

be

found

in

The

essential

problem

is

that

control

decisions

depending

on

a

stack

over

at

least

two

elements

can

occur

at

any

stack

contents

(e.g.,

a

test

on

the

identity

of

the

top-element),

while

control

decisions

in

the

case

of

a

counter

can

take

place

only

if

the

counter

has

value

So,

the

decidability

of

a

service

for

risk

assessment

of

pushdown

threads

(or

any

similar

form

of

action-forecasting)

is

still

an

open

question..

Finally,

apart

from

the

decidability

of

risk

assessment,

the

one-counter

threads

form

a

noteworthy

class

of

threads.

We

conclude

with

a

summary

of

some

typical

properties

of

this

class

of

threads:.

Equivalence

is

decidable

(shown

in

Inclusion

is

undecidable

(see

also

Reachability

is

decidable

(Theorem

in

the

present

paper),

and

Reachability

is

preserved

under

bounded

counter

values

and

polarized

action

labels

(Theorem

in

the

present

paper).

B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O


Category Level Object Segmentation by Combining Bag-of-Words Models with Dirichlet Processes and Random Fields Abstract This paper addresses the problem of accurately segmenting instances of object classes in images without any human interaction. Our model combines a bag-of-words recognition component with spatial regularization based on a random field and a Dirichlet process mixture. Bag-ofwords models successfully predict the presence of an object within an image; however, they can not accurately locate object boundaries. Random Fields take into account the spatial layout of images and provide local spatial regularization. Yet, as they use local coupling between image labels, they fail to capture larger scale structures needed for object recognition. These components are combined with a Dirichlet process mixture. It models images as a composition of regions, each representing a single object instance. Gibbs sampling is used for parameter estimations and object segmentation.Our model successfully segments object category instances, despite cluttered backgrounds and large variations in appearance and viewpoints. The strengths and limitations of our model are shown through extensive experimental evaluations. First, we evaluate the result of two methods to build visual vocabularies. Second, we show how to combine strong labeling (segmented images) with weak labeling (images annotated with bounding boxes), in order to limit the labeling effort needed to learn the model. Third, we study the effect of different initializations. We present results on four image databases, including the challenging PASCAL VOC 2007 data set on which we obtain state-of-the art results. After several decades of research, image segmentation still remains an open problem. Many different approaches have been investigated, combining various image properties such as color, texture, edges, motion, etc. Initially, these methods worked in an unsupervised way: without exploiting a database of manually segmented images to automatically learn parameters for optimal performance. Also, many of the methods operate in a 'bottom-up' way, generating the image segmentation by a process of aggregating local image information, and usually failing to capture high level image information. However, image segmentation is deeply related to image understanding, requiring long-range dependencies to resolve ambiguities that arise at a small scale.. The problem we address in this paper is that of accurately segmenting instances of object classes in images, without giving any prior information on object identities, orientations, positions and scales. This is also known as 'figureground segmentation'. Note that this differs from 'image segmentation' or 'scene segmentation', which correspond to the situation where everything in the image has to be segmented. In object segmentation only several objects of interest have to be segmented. Fig. 1 Examples of object category segmentation obtained by our method without user interaction. Input images (columns 1 and 4), object category masks (columns 2 and 5) and object category segmentation (columns 3 and 6) We assume that the objects belong to known categories, and these categories are defined by sets of training images which are used to learn object appearance models. These training images play a fundamental role because object models built from these images allow object recognition, which we couple with the segmentation process. In particular, we are interested in segmenting object categories that demonstrate large intra-class appearance variations. In Fig. 1 we show several typical images with corresponding segmentation masks produced by our method. Starting from cluttered images including objects of interest, the method is able to recognize and localize objects, and to automatically produce segmentation masks that can be used to extract objects without manual effort. The major contribution of our approach is an instance-based modeling of the scene. More precisely, the object recognition is enhanced by a mechanism which allows to distinguish and model the different instances belonging to a particular class. The number of instances is automatically estimated and controls the number of regions produced by our segmentation.. The model presented in this paper combines three complementary components: (a) a random field (RF) component which ensures short-range spatial contiguity of the segmentation by aligning segment boundaries with low-level image boundaries, (b) a Dirichlet process component that ensures mid-range spatial contiguity by modeling the image as a composition of blobs, each of which corresponds to a single object, and (c) a bag-of-words object recognition component which allows strong intra-class appearance and imaging variations. Although the combination of RFs with a recognition component based on visual words has been explored before, the main contribution of the model presented in this paper is the addition of a Dirichlet process to achieve higher quality segmentation and instance-level segmentation. This paper extends (Larlus and Jurie 2008) with additional experiments and an evaluation of vocabulary construction methods.. In the remainder we first review related work in Sect. 2. Then, in Sect. 3 we present our model and the estimation of its parameters. Visual vocabulary construction for bagof-words methods based on decision trees is described in Sect. 4. We present our experimental results in Sect. 5, and conclude with a discussion in Sect. 6. We conclude this paper with a discussion of our model, and indicate extensions to overcome some of its limitations.. Segmentation is commonly considered as an isolated problem: a given image has to be segmented in some 'meaningful' manner, without any supplementary information. Where 'meaningful' is often understood as segmenting at the level of objects, or their constituent parts. Much early work on segmentation tried to solve the task at a local level; clearly such methods can not resolve ambiguities in the local image features. Semantic grouping is required within the segmentation process, and category-level recognition can provide the necessary cues for this. Similarly, recognition requires accurate segmentation to avoid distraction from background clutter and occluding objects. Our model couples these two processes, and the parameters of its category appearance models are estimated from manually segmented images. The estimated category models can then be applied to segment new instances of the same categories in other images.. Robust category-level recognition requires dealing with intra-class variations and imaging conditions such as occlusions, illumination changes, view point and scale variations. Our choice of patch descriptors ensures some level of invariance to illumination changes. Where some methods rely on rigid shape models for recognition, ours relies on a bagof-words representation which are intrinsically robust to occlusions and non-rigid deformations. Our blob model does not impose hard constraints between object parts, but does implement accumulation of evidence on the object position and size to guide the assignment from patches to objects. The Dirichlet process over patch-to-blob assignments in our model is interesting because it introduces dependencies at an automatically adapted scale, which is determined by the size and number of the blobs. We can imagine using multi-scale patches which would probably improve the recognition ability of the model but increase its complexity.. Our experiments show the benefit of using a supplementary object category detector, which operates at a level of bounding boxes, to improve results when segmenting many object categories simultaneously. Note that the segmentation that our model returns is richer than what could be obtained using a simple combination between a detector and a colorbased segmentation method, as our model separates different object instances and handles multiple categories per image. Note that even if the final goal is to predict a class label per pixel, and not to identify all different instances of each category, it can be beneficial to separately model the different instances. This is because it allows the modeling of instance specific appearance models, for color in our case, which can improve the segmentation accuracy. Furthermore, we experimentally find that we can successfully combine annotations in the form of pixel-level segmentation and bounding boxes, the latter being much easier to produce. Adding images annotated with bounding boxes leads to improved segmentation results.. In future work we want to further study the interplay between the instance specific and category level appearance models. In the current work, low-level image cues are used by either the instance level or category level model, whereas in principle the features used by both models do not need to be disjoint. In particular it is interesting whether we could learn which features are useful at the instance level and which are useful at the category level. Furthermore, it is worthwhile to improve the capacity of the model to distinguish multiple instances of the same category which are very close to each other. Some form of geometric information should be included in the category level appearance models to resolve such ambiguities and improve segmentation results.

O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O