Deterministic versus stochastic seasonal fractional integration and structural breaks This paper considers a general model which allows for both deterministic and stochastic forms of seasonality, including fractional (stationary and nonstationary) orders of integration, and also incorporating endogenously determined structural breaks. Monte Carlo analysis shows that the suggested procedure performs well even in small samples, accurately capturing the seasonal properties of the series, and correctly detecting the break date. As an illustration, the model is estimated for four different US series (output, consumption, imports and exports). The results suggest that the seasonal patterns of these variables have changed over time: specifically, in the second subsample the systematic component of seasonality becomes insignificant, whilst the degree of persistence increases.JEL Classification: C22 This paper analyses seasonality in the presence of structural breaks. Modelling seasonality is still a hotly debated topic in the time series literature. Hylleberg (1986) classifies seasonal models in three categories. The first includes purely deterministic seasonal models, which are characterised by seasonal dummy variables of the form: , ... , 2 , 1 ,. where y t is the observed time series data with s observations per year, D it is a seasonal dummy adopting a value 1 if t belongs to the i th period of the year and 0 otherwise, and u t is a white noise. The definition of the seasonal dummy simply allows for the mean of the series to vary by season, and therefore it raises no statistically interesting issues. The reason for using models like (1) is that the factor that might produce the seasonal variation can be readily identified.. A second type of seasonality is the one defined in terms of a seasonal stochastic stationary process, where y t is specified as , ... , 2 , 1 ,. (2) and φ (L s ) and θ(L s ) are seasonal AR and MA polynomials with all roots lying outside the unit circle. Finally, if the seasonal component is changing across time, seasonal differencing is usually adopted. In such a case, the process is said to contain seasonal unit roots, and the model is expressed as. Many test statistics have been developed in recent years for testing seasonal unit roots: Dickey, Hasza and Fuller (1984) , Hylleberg, Engle, Granger and Yoo (1990) , Tam and Reimsel (1997) , etc. Nevertheless, all these types of seasonality can coexist in a single framework, which, in its more general form, can be written as:. where the difference between stationary and nonstationary seasonality comes from d being equal to 0 and 1 respectively. However, the value d in (4) is not necessarily an integer number. If d is allowed to be any real number, the process is said to be seasonally fractionally integrated, with much greater flexibility in the dynamic behaviour of the series (see Gil-Alana, 2005) .. The notion of a fractional Gaussian noise with seasonality was suggested by Jonas (1981) and extended in a Bayesian framework by Carlin et al. (1985) and Carlin and Dempster (1989) . Porter-Hudak (1990) applied a seasonally fractionally integrated model to quarterly US monetary aggregates, and concluded that a fractional ARMA model was more appropriate than the usual ARIMA specification for these series. Other recent empirical papers on seasonal fractional integration are those of Gil-Alana and Robinson (2001) and Gil-Alana (2002) .. The present study focuses on the model given by equation (4), extended to incorporate endogenously determined structural breaks. Note that fractional integration (at the zero frequency) has been recently related to structural breaks (see, e.g. Granger and Hyung, 1999; Gourieroux and Jasiak, 2001; Diebold and Inoue, 2001; etc.) , and thus we should expect a similar relationship in the presence of seasonality. The outline of the paper is as follows: Section 2 describes the procedure for estimating the parameters in the model. In Section 3 we carry out several Monte Carlo experiments to examine the finite sample behaviour of the seasonal procedure we implement. An empirical application is carried out in Section 4, while Section 5 contains some concluding comments. This paper considers a general model which allows for both deterministic and stochastic forms of seasonality, including fractional (stationary and nonstationary) orders of integration, and also incorporating endogenously determined structural breaks. Monte Carlo analysis shows that the suggested procedure performs well even in small samples, accurately capturing the seasonal properties of the series, and correctly detecting the break date. As an illustration, the model is estimated for four different US series (output, consumption, imports and exports). The results can be summarised as follows.. First, we find evidence of a structural break in all the series, with the seasonal pattern This paper can be extended in several directions. First, other deterministic linear or even non-linear models can be included in the regression models (4) and (5), and the estimation can be carried out adopting the same procedure described here. Second, confidence intervals directly based on our procedure can be obtained using bootstrapping methods, although these are highly computationally intensive, especially if the sample size is large. Another possible extension is to consider different degrees of seasonal integration at each of the frequencies for each subsample. Note that the polynomial (1 -L 4 ) can be decomposed into (1 -L)(1 + L)(1 + L 2 ), and therefore using the polynomial (1 -L 4 ) d implies that the same order of integration d is imposed at all frequencies. However, a problem with this approach is that it is even more computationally intensive, given the greater number of values required in the grid- 0.15, 0.59) -0.07 (-0.37, 0.12) -0.11 (-0.34, 0.19)

O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


DYLEMA: Using walking robots for landmine detection and location Detection and removal of antipersonnel landmines is an important worldwide concern. A huge number of landmines has been deployed over the last twenty years, and demining will take several more decades, even if no more mines were deployed in future. An adequate mine-clearance rate can only be achieved by using new technologies such as improved sensors, efficient manipulators and mobile robots. This paper presents some basic ideas on the configuration of a mobile system for detecting and locating antipersonnel landmines efficiently and effectively. This paper describes the main features of the overall system, which consists of a sensor head that can detect certain landmine types, a manipulator to move the sensor head over large areas, a locating system based on a global-positioning system, a remote supervisor computer and a legged robot used as the subsystems' carrier. The whole system has been configured to work in a semi-autonomous mode with a view also to robot mobility and energy efficiency. Detection and location of antipersonnel landmines is being done at present mainly by human operators handling manual equipment. The robotisation of this activity can give so many benefits to the human community in many countries. There is worldwide interest in eradicating deployed landmines, and solutions are coming from new, emerging engineering fields.. New kinds of sensors are required to detect landmines efficiently, but current sensors can be carried by mobile robots. Legged locomotion offers some important advantages for moving on natural terrain and appears to be a good solution for carrying mine sensors efficiently over infested fields.. Some preliminary work has been done to study the potential of using walking robots for demining. This paper addresses the development of a walking robot endowed with a manipulator able to scan areas with a sensor head based on a metal detector. It also presents the potential advantages of walking robots and a new system for landmine detection. This paper introduces the main system and provides some details of the configuration of the walking robot and the manipulator, and it outlines the hardware and software architecture as well.. The system has to be completed with the tools it needs for forming databases of potential alarms and providing the operator with adequate images and graphs. The incorporation of new sensors, detectors, and software for signature analysis will be addressed in the second step of this project.

B O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Demand-driven structural testing with dynamic instrumentation Producing reliable and robust software has become one of the most important software development concerns in recent years. Testing is a process by which software quality can be assured through the collection of information. While testing can improve software reliability, current tools typically are inflexible and have high overheads, making it challenging to test large software projects. In this paper, we describe a new scalable and flexible framework for testing programs with a novel demand-driven approach based on execution paths to implement test coverage. This technique uses dynamic instrumentation on the binary code that can be inserted and removed on-the-fly to keep performance and memory overheads low. We describe and evaluate implementations of the framework for branch, node and def-use testing of Java programs. Experimental results for branch testing show that our approach has, on average, a 1.6 speed up over static instrumentation and also uses less memory. In the last several years, the importance of producing high quality and robust software has become paramount [15] . Testing is an important process to support quality assurance by gathering information about the behavior of the software being developed or modified. It is, in general, extremely labor and resource intensive, accounting for 50-60% of the total cost of software development [17] . Given the importance of testing, it is imperative that there are appropriate testing tools and frameworks. In order to adequately test software, a number of different testing techniques must be performed. One class of testing techniques used extensively is structural testing in which properties of the software code are used to ensure a certain code coverage.Structural testing techniques include branch testing, node testing, path testing, and def-use testing [6, 7, 8, 17, 19] .. Typically, a testing tool targets one type of structural test, and the software unit is the program, file or particular methods. In order to apply various structural testing techniques, different tools must be used. If a tool for a particular type of structural testing is not available, the tester would need to either implement it or not use that testing technique. The tester would also be constrained by the region of code to be tested, as determined by the tool implementor. For example, it may not be possible for the tester to focus on a particular region of code, such as a series of loops, complicated conditionals, or particular variables if def-use testing is desired. The user may want to have higher coverage on frequently executed regions of code. Users may want to define their own way of testing. For example, all branches should be covered 10 times rather than once in all loops.. In structural testing, instrumentation is placed at certain code points (probes). Whenever such a program point is reached, code that performs the function for the test (payload) is executed. The probes in def-use testing are dictated by the definitions and uses of variables and the payload is to mark that a definition or use in a defuse pair has been covered. Thus for each type of structural testing, there is a testing "plan". A test plan is a "recipe" that describes where probes should be placed and what should be done when a probe is reached.. In most tools, the instrumentation is placed in the binary code before execution and remains in the code until execution terminates. This type of instrumentation can be expensive in both time and space. The instrumentation causes code growth and thus instrumenting a complete program may not be possible. Also, even though coverage may only require one instantiation of a code element, the instrumentation stays in the code, causing unnecessary time overhead.. In this paper, we describe a testing framework that addresses both flexibility and scalability for structural testing. Our approach enables testers to use different testing strategies, including custom testing, in an efficient and automatic way. The key ideas in our approach are a test planner that generates a plan from a test specification and an instrumenter that (1) inserts instrumentation when needed in a demand-driven fashion as the program executes and (2) deletes the instrumentation when no longer needed, according to a test plan. The approach is path specific and uses the actual execution paths of an application to drive the instrumentation and testing. The granularity of the instrumentation is flexible and includes statement level and structure level (e.g., loops, methods, program).. To ensure flexibility, we developed a specification language from which a test plan can be automatically generated by a test planner. The test specification describes what tests to apply and under what conditions to apply them. The specification language has both a visual representation and textual form. The visual language is expressed through a graphical user interface (GUI). The GUI is also able to display test results and present them to the user with a test analyzer, highlighting relevant parts of the application with the test results.. We implemented the test framework-the GUI, test planner, dynamic instrumenter, and test analyzer-and incorporated them into the Eclipse integrated development environment [5] and the IBM Jikes Java Research Virtual Machine [2] . Our prototype tool, called Jazz, can perform branch, node and def-use coverage testing over multiple code regions in a Java program, as desired by the tester. The prototype demonstrates the feasibility and practicality of our approach. Our results are very encouraging, with both very low run-time overhead and memory usage. This paper makes several contributions, including:. • A novel and low cost approach for instrumenting a program along an execution path to perform different types of tests; • A new framework for generating structural software testing tools that use dynamic instrumentation; • A technique that enables dynamic insertion and removal of test instrumentation on demand; and • An implementation and experimental evaluation of a tool that implements our approach for testing Java programs. In the next section, we give an overview of our framework including a user scenario. In Section 3, we discuss the test planner and the dynamic instrumenter. Section 4 describes particular test planners, and experimental results are presented in Section 5. The paper concludes with related work and a summary. This paper addresses the need for scalable and flexible testing tools by developing a framework where test specifications are automatically converted to an implementation of the specifications. The framework is flexible in that both standard and custom structural tests can be incorporated into tools created through the framework. Different tests, code regions, code granularities, and coverages can all be incorporated. The framework also generates tools that are scalable because the instrumentation is dynamically inserted on demand as the program executes. Instrumentation is also deleted at the time it is no longer needed. Experimental results indicate savings in both time and memory over statically placing instrumentation before program execution. There are a number of commercial tools that perform coverage testing on Java programs, including JCover [11] , Clover [4] and IBM's Rational PurifyPlus [10] . Of these tools, only Clover does both branch coverage and statement coverage; none do def-use coverage. Unlike our framework, these tools statically instrument a program with probes that remain for the entire execution of the tested program. Our demand-driven approach does not modify the program source code or class files. Instead, it operates on binary code, enabling the use of dynamic instrumentation. Our framework also avoids unnecessary overhead due to static instrumentation by removing instrumentation as soon as it is not needed.. Tikir and Hollingsworth [21] use a dynamic technique for node coverage with Dyninst [9] . As in Jazz, the Dyninst tool dynamically inserts instrumentation on method invocations for node coverage. Unlike our approach however, instrumentation is only removed via a garbage collection process. Instead of removing instrumentation as soon as possible, a separate thread periodically removes the instrumentation. However, this instrumentation remains until collected, even when it is not needed. In comparison to their coverage tool, Jazz works well. They report slowdowns of 1.001 to 2.37 (average 1.36) for C programs, while Jazz's slowdowns are 0.99 to 1.04 (average 1.03) for Java programs. Although it is difficult to directly compare these results, the demand-driven technique has better performance because instrumentation is inserted on paths and removed immediately rather than periodically.. Path profiles can be used to compute code coverage [1] . Path profiling transforms a CFG into a directed acyclic graph (DAG) and assigns values to the nodes so that each unique path from the entry to the exit of the DAG produces a unique sum. However, the instrumentation needed path profiling cannot be removed. Because the results presented in [1] do not include the overhead for edge labeling, instrumentation insertion, or path regeneration, a performance comparison is difficult.. The concept of fast breakpoints was pioneered by Kessler [12] but these breakpoints were not applied in a general manner to dynamically instrument programs for structural testing. Dynamic instrumentation systems like PIN [18] , Dyninst [9] and Paradyn [13] used a technique similar to ours to instrument a program. Like our framework, Dyninst is intended to be general, with a language for specifying instrumentation [9] . However, their instrumentation techniques were not designed to support test development. Using SPECjvm98 benchmarks [20] , we performed experiments to measure Jazz's performance and memory needs. The experiments were run on a unloaded 2.4 GHz Pentium IV with 1 GB of memory and RedHat Linux 7.3. All results are averages over three program runs. The test specification for the experiments covers all loaded methods. For def-use testing, the specification selects all variables and all def-use pairs. The test inputs are the data sets provided in SPECjvm98.

O O B O B I O O O B B O O O O O O O B I O O O O B O O B O O B O O O O O O O O O O B O O B B O B O O O O O O O O O O O O O B O O O B O O O O O O O B O B O O O O O O O O O O O O O B O O O B I O O O B O O O O O O O O O B O O O O O O O O O O O B O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O B O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O O B B O O O O O O O B O O O O B O O O O O O O B O O O O O B O O O O O B B O O O O O O B O B O O O B O B O B O O B O O O O O O O O B O O O O O O O O O B O O O B O O O O O O O O O O B O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O B O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O B O O O O O B O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B O O O O O O O B O O O O O O B O O O O O O B O O O B B O O O B O B O O B O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O B B B O O B I O O O O O O O O O O O O O O O O O B O O B O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O B B O O O B O O O O O O O O B O O O O B O O O O O O O O O O O O O O B O O O B O O B O B O O O O O O O O B O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O B B O O O O O O O O O O O O O O O O O B O O O O O B O O B O O O B O O B O B O B O B O O O O O O O O O O O B O O O O O O O B O B O O O O O O O O B O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B B I O O O O O O O O O O O O O O O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O


Hardware support for accurate per-task energy metering in multicore systems Accurately determining the energy consumed by each task in a system will become of prominent importance in future multicore-based systems because it offers several benefits, including (i) better application energy/performance optimizations, (ii) improved energy-aware task scheduling, and (iii) energy-aware billing in data centers. Unfortunately, existing methods for energy metering in multicores fail to provide accurate energy estimates for each task when several tasks run simultaneously. This article makes a case for accurate Per-Task Energy Metering (PTEM) based on tracking the resource utilization and occupancy of each task. Different hardware implementations with different trade-offs between energy prediction accuracy and hardware-implementation complexity are proposed. Our evaluation shows that the energy consumed in a multicore by each task can be accurately measured. For a 32-core, 2-way, simultaneous multithreaded core setup, PTEM reduces the average accuracy error from more than 12% when our hardware support is not used to less than 4% when it is used. The maximum observed error for any task in the workload we used reduces from 58% down to 9% when our hardware support is used. Energy is one of the most-if not the most-expensive resources in a computingsystem. For instance, the electricity demand in data centers, where many computers work nonstop dissipating large amounts of power, shows the fastest growth among all sectors. In fact, current facilities consume several megawatts, enough to power small towns [Belady and Malone 2006] . Koomey [2011] estimates that worldwide energy consumption attributable to servers and data centers was more than 200 billion kilowatt hours (kWh) annually in 2010, and recent studies estimate the corresponding electrical cost to be $30 billion [Raghavendra et al. 2008] . Energy already accounts for 20% of the total cost of ownership in a large-scale computing facility [Hamilton 2009 ]. This cost doubles if we add the cost for the cooling infrastructure, implying that the total energy-related cost is already in the same order of magnitude as hardware-related cost (servers). Additionally, although server cost has remained almost constant over successive generations, energy cost is expected to rise [Barroso 2005 ].. Energy demand is also an issue for home computers. A typical desktop computer may use in the order of 100 to 200 Watts (W; the particular figure depends on the type of computer and peripherals), whereas laptops fit in a lower range (60-100W). The energy cost of running a computer can be computed as In this article, we look at the challenges and opportunities associated with accurate per-task energy metering. As shown in this work, existing approaches based on an even distribution of energy across tasks are highly inaccurate. Therefore, we propose (i) a fair reference approach to distribute energy across tasks and (ii) an affordable implementation, PTEM, that tracks task activity and resource utilization at very low cost (below 0.3% energy overhead and 0.85% area overhead).. PTEM is shown to provide highly accurate per-task energy estimates with an average error of 3.1% for SMT multicore configurations and 2.1% for single-threaded multicore configurations. We further discuss the required changes, at both the hardware and the software levels, to provide such an accurate, yet implementable, per-task energy metering mechanism.. Finally, we show how to use PTEM in the context of parallel applications and show a case study where PTEM provides the required information to minimize the energy consumption of servers and cloud computing facilities. We use an enhanced version of SMTSim [Tullsen et al. 1998 ] extended with power models analogous to those of Wattch [Brooks et al. 2000] . Wattch-like power models are built on top of the CACTI 6.5 simulation tool [Muralimanohar and Balasubramonian 2009] . CACTI is a flexible tool modeling delay, energy (dynamic and leakage), and area of cache memories and SRAM-based arrays. Power models for functional units have been updated to use modern designs.. We assume a clustered multicore architecture where each cluster consists of a set of cores, with each core having private data and instruction first-level caches, plus a shared on-chip second-level cache accessed through a shared bus. We refer to such cache as LLC. Details about the configuration can be found in Table II . All clusters are connected to memory through a shared bus (see Figure 2) . Several studies show that hierarchical bus configurations scale quite easily to large systems and provide a good area-performance trade-off, while retaining many of the advantageous features of simpler bus arrangements [Salminen et al. 2007 ]. In the same line, other studies show that bus-based networks can significantly lower energy consumption and simplify network protocol design and verification with no loss in performance [Udipi et al. 2010] .. We consider a processor configuration with clusters consisting of 4 cores, with those cores being two-threaded SMT (4C2S). We first perform an intracluster evaluation for this 4C2S setup to facilitate understanding of PTEM behavior. Then, for the full processor evaluation, we consider setups of four and eight clusters, with two-way SMT 4-core (4C2S) clusters. Thus, 32-and 64-threaded setups are considered, respectively.. Benchmarks. In addition to traces from a real HPC application, detailed in Section 9, we use traces collected from the whole SPEC CPU2006 benchmark suite using the reference input set. Each trace contains 100 million instructions, selected using the SimPoint methodology [Sherwood et al. 2001] . Using these benchmarks, we generate different workloads with different numbers of benchmarks. Running all N-task combinations is increasingly time consuming, as the number of combinations is too high. We classify benchmarks into two groups depending on their memory behavior. Benchmarks in the memory group (denoted MEM) are those presenting an LLC miss rate higher than 1%-that is, mcf , milc, lbm, libquantum, soplex, gcc, bwaves, and omnetpp. The rest of the benchmarks are CPU (ILP) bounded and are denoted ILP. From these two groups, we generate three workload types denoted I, M, and X, depending on whether all benchmarks in a cluster belong to group ILP, MEM, or a combination of both.. We generate eight workloads per group and processor setup. Benchmarks in each workload are randomly picked out from all benchmarks of the corresponding type. In the case of X, half of the benchmarks belong to ILP and the other half to MEM. We do not put any constraint on whether benchmarks can repeat in a particular workload, because the random selection of benchmarks is always performed out of the corresponding (original) group of benchmarks.. Metrics. In order to evaluate the accuracy of PTEM, as a reference, we always use the estimates that we would obtain with the idealized model. In each experiment, Fig. 3 . Per-task LLC cache energy breakdown and access/occupancy rates when executing mc f and gcc in a single-threaded 2-core configuration.. we measure the off estimation or prediction error of each model with respect to the idealized model, which is computed as follows:

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Collaborative Knowledge Semantic Graph Image Search In this paper, we propose a Collaborative Knowledge Semantic Graphs Image Search (CKSGIS) system. It provides a novel way to conduct image search by utilizing the collaborative nature in Wikipedia and by performing network analysis to form semantic graphs for search-term expansion. The collaborative article editing process used by Wikipedia's contributors is formalized as bipartite graphs that are folded into networks between terms. When a user types in a search term, CKSGIS automatically retrieves an interactive semantic graph of related terms that allow users to easily find related images not limited to a specific search term. Interactive semantic graph then serve as an interface to retrieve images through existing commercial search engines. This method significantly saves users' time by avoiding multiple search keywords that are usually required in generic search engines. It benefits both naïve users who do not possess a large vocabulary and professionals who look for images on a regular basis. In our experiments, 85% of the participants favored CKSGIS system rather than commercial search engines. As of December 2007, Google Images is providing keywordbased queries to search more than one billion images on the Internet. While most commercial image search engines mainly rely on hyperlinks and metadata to index images [1] , researchers have been investigating the concept detection approach in order to understand image content directly and annotate keyword terms to images [2] . Keyword-based query is more natural to general users. However, it shares the same obstacles as in information retrievalwhat keywords should I enter in the search box to find out the information I really need? Users experienced finding the right term. Content-based image search tried to circumvent this problem. However, many terms are difficult to portray visually. In most occasions, we do not know what to 'ask' and thus spend lots of time to try various search terms again and again until we find the things we want.. With this trend in mind, we propose to utilize the collaborative knowledge of human beings on Wikipedia to find out semantic graphs of search terms. We propose to use an important hidden linkage of terms inside human (i.e., article editors) to consider the relatedness of terms as well as how close terms are. Our hypothesis is that whoever contributed to two articles, there is a large likelihood that these two terms are somehow related so that an 'expert' knows both of them. We thus draw bipartite graph between authors and terms and collapses it into one-dimensional graphs with weighted links. Since relationships somewhat captured human being knowledge behind scenes, the constructed semantic graphs look useful and can serve as an important basis for query expansion on image searches. The rest of this paper is organized as follows. We first present the framework of the CKSGIS in Section 2. In Section 3, we describe the image search interface. Experiment results of the new system are reported in Section 4. Finally, conclusions and direction of future work are provided in Section 5. A controlled user study based on 30 students show that more than 80% of the participants preferred our semantic relatedness results to those provided by traditional image search interface. 87% said that the obtained relatedness helps them to learn more about the collection. 73% found CKSGIS is more flexible, and 73% found that, it is easier to use. These results indicate that our semantic graph based approach is a promising method for users to use in their daily life search tasks.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Can we predict the failure point of a loaded composite material? As a model of composite material, the fiber bundle model has been chosen -where a bundle of fibers is subjected to external load and fibers have distributed thresholds. For different loading conditions, such a system shows few precursors which indicate that the complete failure is imminent. When external load is increased quasi-staticallybursts (number of failing fibers) of different sizes are produced. The burst statistics shows a robust crossover behavior near the failure point, around which the average burst size seems to diverge. If the load is increased by discrete steps, susceptibility and relaxation time diverge as failure point is approached. When the bundle is overloaded (external load is more than critical load) the rate of breaking shows a minimum at half way to the collapse point. The pattern and statistics of energy emission bursts show characteristic difference for below-critical and over-critical load levels. Prediction of the failure point is a major challenge in various scenarios of fracture and breakdown [1, 2, 3] -ranging from fracturing in nano-materials to onset of earthquakes. A material body can tolerate certain level of load or force on it and beyond that level it collapses. If the load is increased continuously -when does the collapse point come ? Is there any precursor which signals that the complete failure is imminent ?. Fiber bundle model (FBM) has been proved [4, 5, 6, 7] to capture the essentials of failures in composite materials. FBM contains a large number of fibers with statistically distributed strength thresholds. It has simple geometry and clear-cut rules for how stress caused by a failed fibers is redistributed on intact fibers. Most importantly, this model can be solved analytically to an extent (For reviews, see [7] ) that is not possible for any other model of material-failure. The statistical properties of FBM is well studied [8, 9, 12] , and the failure dynamics at a constant load has been formulated [11] through recursion relations which in turn explore the phase transition and associated critical behavior in this model.. In this article we discuss how we can predict the failure point of a loaded FBM from the available precur- sors. The term precursor usually means [13] some prior indications of an upcoming incident and in current context such incident is the complete failure (collapse) of a fiber bundle under external load. We consider equalload-sharing model, in which the load previously carried by a failed fiber is shared equally by all the remaining intact fibers [7] . The bundle consisting of N elastic fibers, clamped at both ends (Fig. 1) . All the fibers obey Hooke's law with force constant set to unity for simplicity. Each fiber i is associated with a breakdown threshold x i for its elongation. When the length exceeds x i the fiber breaks immediately, and does not contribute to the strength of the bundle thereafter. The individual thresholds x i are assumed to be independent random variables with the same cumulative distribution function P(x) and a corresponding density function p(x):. We analyse three different loading cases: quasi-static loading, load increment by equal steps and overloaded situation. For prediction purpose it is important that precursors can be seen in a single system. Therefore, throughout this article we will present and discuss results that can be seen in a single bundle containing large number of fibers. For simplicity, we consider the uniform distribution of fiber thresholds: P(x) = x for 0 ≤ x ≤ 1.

O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient pattern matching over event streams Pattern matching over event streams is increasingly being employed in many areas including financial services, RFIDbased inventory management, click stream analysis, and electronic health systems. While regular expression matching is well studied, pattern matching over streams presents two new challenges: Languages for pattern matching over streams are significantly richer than languages for regular expression matching. Furthermore, efficient evaluation of these pattern queries over streams requires new algorithms and optimizations: the conventional wisdom for stream query processing (i.e., using selection-join-aggregation) is inadequate.In this paper, we present a formal evaluation model that offers precise semantics for this new class of queries and a query evaluation framework permitting optimizations in a principled way. We further analyze the runtime complexity of query evaluation using this model and develop a suite of techniques that improve runtime efficiency by exploiting sharing in storage and processing. Our experimental results provide insights into the various factors on runtime performance and demonstrate the significant performance gains of our sharing techniques. In this paper, we studied the evaluation and optimization of pattern queries over event streams. We rigorously defined a query evaluation model, the NFA b automata, analyzed its expressibility, and provided compilation techniques for building query plans based on this model. We also analyzed the runtime complexity and developed sharing techniques for efficient runtime evaluation. Our system could process tens of thousands of events per second for fairly expensive queries and offers much higher throughput for cheaper queries. Our sharing techniques also produced remarkable performance benefits, ranging from 40% to 110%, over a spectrum of query workloads. We plan to continue our research in a few directions. We will extend our system to handle out-of-order events by augmenting the NFA b model with techniques including invalidation and re-computation: The NFA b proceeds as before and, when receiving an out-of-order event, invalidates some of the existing runs affected by the event and recomputes from valid intermediate steps. In addition, our current implementation of queries with negation and composition is strictly based on the query semantics. We will explore new opportunities for optimization that are particularly suitable for these queries. Finally, we will study robust pattern matching over uncertain events that are produced from a variety of sensor networks. Much related work has been covered in previous sections. We discuss broader areas of related work below.. Event languages for active databases [14, 13, 6, 23, 22, 36] offer temporal operators including sequencing and Kleene closure, but do not support complex predicates to compare events. As we showed in this paper, such predicates are crucial in pattern definition. Those languages also lack efficient implementation over high-volume streams.. Traditional pub/sub systems [2, 12] offer predicate-based filtering of individual events. Our system significantly extends them with the ability to match complex patterns across multiple events. Cayuga [10, 11] supports patterns with Kleene closure and event selection strategies including partition contiguity and skip till next match, but does not allow output of complete matches. In comparison, our system supports more event selection strategies and output of complete matches both in the evaluation model and in runtime optimization. The Cayuga implementation focuses on multi-query optimization, which is directly applicable when our system is extended to handle multiple queries.. Sequence databases [27, 26] offer SQL extensions for sequence data processing. SEQUIN [27] uses joins to specify sequence operations and thus cannot express Kleene closure. SQL-TS [26] adds new constructs SQL to handle Kleene closure, but restricts pattern matching to only contiguous tuples in each relevant partition. Its optimization based on predicate containment can be integrated into our system for workloads that exhibit such containment relationships.. Many recent event systems [16, 25, 31, 34] offer relatively simple event languages and stream-based processing. These systems lack important constructs for pattern matching such as Kleene closure and choices of event selection strategies. In particular, our work significantly extends prior work [34] with Kleene closure and event selection strategies, two features that fundamentally complicate event pattern matching, a formal rich model NFA b for evaluation and theoretical analysis, and a suite of sharing techniques. The evaluation framework of [34] is further extended to handle out-of-order events [20] . SASE+ [15] provides a rich language for pattern matching but lacks implementation details.. There have also been theoretical studies on the underlying model of complex event processing. CEDR [5] proposes a new temporal model that captures the duration of events and analyzes the consistency of event processing for outof-order streams. A related study [33] designs a temporal model for such events that has a bounded representation of timestamps and offers associativity of the sequencing operator. These results can be applied to guide the extension of our system to handle events with duration. In this section, we provide background on event pattern languages, which offers a technical context for the discussion in the subsequent sections.. Recently there have been a number of pattern language proposals including SQL-TS [26] , Cayuga [10, 11] , SASE+ [34, 15] , and CEDR [5] .. 1 Despite their syntactic variations, these languages share many features for pattern matching over event streams. Below we survey the key features of pattern matching using the SASE+ language since it is shown to be richer than most other languages [15] . This language uses a simple event model: An event stream is an infinite sequence of events, and each event represents an occurrence of interest at a point in time. An event contains the name of its event type (defined in a schema) and a set of attribute values. Each event also has a special attribute capturing its occurrence time. Events are assumed to arrive in order of the occurrence time.

O B I O B I B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Scalable target detection for large robot teams In this paper, we present an asynchronous display method, coined image queue, which allows operators to search through a large amount of data gathered by autonomous robot teams. We discuss and investigate the advantages of an asynchronous display for foraging tasks with emphasis on Urban Search and Rescue. The image queue approach mines video data to present the operator with a relevant and comprehensive view of the environment in order to identify targets of interest such as injured victims. It fills the gap for comprehensive and scalable displays to obtain a network-centric perspective for UGVs. We compared the image queue to a traditional synchronous display with live video feeds and found that the image queue reduces errors and operator's workload. Furthermore, it disentangles target detection from concurrent system operations and enables a call center approach to target detection. With such an approach we can scale up to very large multi-robot systems gathering huge amounts of data that is then distributed to multiple operators. Many applications such as interplanetary construction, search and rescue in dangerous environments, or cooperating uninhabited aerial vehicles have been proposed for multi-robot systems (MrS). Controlling these robot teams has been a primary concern of many HRI researchers. These efforts have included theoretical and applied development of the Neglect Tolerance model and Fan-out model to characterize the control of independently operating robots [1, 2] , predefined rules to coordinate cooperating robots as in Playbook™ [3] and Machinetta [4] , and techniques for influencing teams obeying biologically inspired control laws [5, 6, 7] . While our efforts to increase span of control over unmanned vehicle (UV) teams appear to be making progress, the asymmetry between what we can command and what we can comprehend is growing. Automation can reduce excessive demands for human input, but throttling the information being collected and returned is fraught with danger. A human is frequently included in the loop of a MrS expressly to monitor and interpret video being gathered by UVs. This can be a difficult task for even a single camera [8] and begins exceeding operator capability before reaching ten cameras [9, 10] . With increasing autonomy of robot teams and plans for biologically inspired swarms of much greater size the problem of absorbing and benefiting from their product seems even more pressing than learning how to command them.. One approach to this problem is the Picture-in-Picture display (PiP), a specialized solution for integrating unmanned aerial vehicle (UAV) camera video [11, 12] . In a PiP display current video, scaled and transformed to reflect the position and orientation of the UAV and camera, is projected onto a map of the area being surveyed. As the cameras move about, the areas of the map falling under their view are refreshed. By presenting camera views in context, the PiP display eliminates many of the mental transformations and confusions necessary to interpret video being viewed from a remote camera [13] . Because the operator can attend to the information being gathered by the team of UAVs rather than the video from any particular UAV, the PiP display provides a network-centric [14] view of the collected information and has been shown to lead to enhanced situation awareness [11] . Another favorable property of the PiP display is that the ease of use should be independent of team size because the size and resolution of the map remains constant. As additional UAVs are added, regions of the map will be updated more frequently reducing uncertainty but without imposing additional load on the operator. In fact, as the update rate is increased the operator will no longer have to predict or extrapolate and can simply observe making the task even easier.. The integration of unmanned ground vehicles (UGVs) into a network-centric view and reducing the difficulty of monitoring their video is a more challenging problem because there is no convenient correlate of an aerially viewed map to provide a common representation. UGV camera views typically provide a projection of a 3D area onto a horizontal plane, the camera plane of the UGV. The resulting image captures the space between a UGV and obstructions such as walls or vegetation. Even though the view may be geo referenced, how informative it is will depend on the degree to which it is relatively unobstructed. Differences in geometry also make UGV views much less informative. Because Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. HRI '11, March 6-9, 2011 , Lausanne, Switzerland. Copyright 2011 ACM 978-1-4503-0561-7/11/03...$10.00. a UAV views the scene below it from a relatively long constant distance, objects and surroundings, obeying the inverse square law, have spatial relations that appear relatively constant. For a UGV, by contrast, objects will remain within camera view for long periods changing drastically in size and appearance as the UGV approaches and moves away from them. In this case views need to be chosen carefully for informativeness rather than simply transformed to fit a map.. While PiP views are inherently spatially organized and present collected data in its entirety this may not necessarily work well for UGVs. Because UGV cameras collect many drastically different views of the same scene there is no simple algorithmic way to fuse all these views simultaneously into a single fully informative display. Simple projections onto a map as for PiP are impossible because such projections would involve a complete 3d reconstruction of the environment. Even if this were achieved, with advances in 3d mapping and 3d reconstruction from images, viewing such a 3d reconstruction will also not provide a simultaneous perspective on the entire environment. Once simultaneity is lost, the user is no longer guaranteed to be able to see new events on screen as they occur. But an asynchronous display of data may suffice for static environments such as presumed for Urban Search and Rescue (USAR) with immobilized victims and may prove valuable also without a 3d reconstruction.. The problem addressed in this paper is the design of an asynchronous, scalable, and comprehensive display, without requiring a 3d reconstruction, to enable operators to detect relevant targets in environments that are being explored by large teams of UGVs. We will present one particular design for such a display and test it in the context of USAR with large robot teams with some degree of autonomy and supervised by a single operator. The purpose of our experiment was to examine the impact of our asynchronous image queue display on overall performance. It presents information to subjects asynchronously but ordered by a quality metric that relates to the utility of the information. This stands in contrast to the video stream that presents information as it becomes available. Either method can have advantages and disadvantages. Especially navigation tasks benefits from immersion and situation awareness. In fact, in the streaming video condition we observed more teleoperation instances while participants for the image queue avoided teleoperating the robots and relied more heavily on the autonomy. Consequently, one would expect them to benefit from further improvements to the autonomy for which there is ample room. As autonomy improves, we ultimately see the need for navigation reduced to situations in which the operator has to assist robots to fix unexpected errors. Furthermore, image queue participants have no need to teleoperate a robot in contrast to streaming video participants when they encounter a victim in the video feed. Most importantly, they do not need to stop the robot in order to precisely locate the victim. In essence, we have decoupled the navigation and error recovery tasks from the victim detection tasks, allowing the latter to be completed entirely asynchronously without any penalties for performance in terms of the number of victims. Even more so, by decoupling it we reduced the number of errors that occur for marking victims, false positives and negatives alike, while at the same time reducing overall workload. We can hence conclude that the added situation awareness from teleoperating robots while searching for victims provides little to no benefit for such a system with many robots. On the other hand, the reduction in errors for image queue is particularly significant because avoiding missed targets is crucial to most foraging tasks. Especially for USAR thoroughness and correctness are two of the most important performance metrics.. Participants in the streaming video condition were confronted with a bank of videos much like a security guard monitoring many surveillance cameras. Informal observation of participants suggests that the continued monitoring of video feeds synchronously during system operation puts a greater effort on operators. Not only does it require continuous attention but participants switch between tasks more often (whenever a victim appears in a video feed). This leads to an impression of time pressure that may well contribute to the increased number of errors. The significant difference of individual dimension of workload, temporal demand and effort, supports this observation.. The overall significance of the successful decoupling of detection from other tasks goes further than the mere reduction of errors, workload, and teleoperation instances. It allows the design of a system that treats target detection tasks as notifications for a call center. Regardless of the origin of the notification our experiment indicates that we could assign an image from the image queue to any operator and hence scale our operations in terms of the number of operators. The small amount of contextual information in form of a mini map and some additional imagery from the database seems to suffice and does not jeopardize performance. As autonomous exploration continues to improve and scales to very large teams of autonomous robots, we can now ensure that our current methods for target detection by human operators scale with it.. There is, however, one problem with this approach. Errors that originate from a complex interaction between tasks will be difficult to detect by human operators. Effectively, operators rely more on the autonomy and system design and sacrifice the ability to have a complete overview of the system and its functions. Such a sacrifice is often made when scaling large and complex systems but its impact on foraging tasks is not yet well understood. We conjecture that it is in fact less dramatic than a failure in a complex plant monitoring operation which can lead to precisely the type of disasters that our robot teams are supposed to find victims in. At this point we can well imagine two operators supervising 24 UVs and responding to UV originated requests such as verifying/marking targets as well as other types of alarms that may require them to regain situation awareness beyond what the image queue requires, such as getting stuck UVs out of trouble ( Figure 11 ). As further work we intend to directly address scalability with a call center approach and investigate its effects on even larger teams with multiple operators. Another useful extension of this work to moving targets is also planned. Evidently, an asynchronous display method will face particular challenges when dealing with dynamic environments and targets and will require more sophisticated techniques. Data were analyzed using a repeated measures ANOVA comparing streaming video with the image queue condition. Overall, in both conditions participants were successful in searching through the environment. On average participants in the streaming video condition found 9.10 victims while those in the image queue condition found 8.51 (Figure 7 ) without a significant difference between conditions (F 1,28 = .733, p = .387). The area explored ( Figure 7 ) for both conditions did also not differ significantly (F 1,28 = 2.147, p = .154). Every mark a participant made for a victim was compared to ground truth to determine whether there was in fact a victim at the location. A mark made further than 2 meters away from any victim or multiple marks for one victim were counted as false positives. Victims that were missed, but present in the video feed, and not marked were counted as false negatives.. While we initially tried assigning marks to the nearest robot it led to assigning some marks at distances that did not seem credible. Using longer distances also made anomalies such as assigning a mark to a victim in another room more likely. The fact that the data show low rates of false positives (marking a victim beyond the 2 m radius) of between 1 (image queue) and 2.4 (streaming video) while successful markings were 8.5 and 9.1 suggests that the 2 m criterion is doing well at associating most marks with victims. Visual inspection confirms this with most false positive marks falling well outside the 2 m radii rather than nearby.. For both types of errors the data showed a significant advantage for the image queue condition (Figure 8 ). There were significantly fewer false positives (F 1,28 = 13.032, p = .001) as well as fewer false negatives (F 1,28 = 5.526, p = .026) with an average in the image queue condition of 7.48 while participants in the streaming video condition missed 9.34 victims on average (Figure 8 ). In addition we monitored the number of instances that participants initiated a teleoperation session with an individual robot. The repeated measures ANOVA shows a significant difference for teleoperation instances between the streaming video and image queue condition ( Figure 9 ). Participants in the streaming video condition teleoperated on average 21.24 times while participants in the streaming video conditon teleoperated on average 4.97 times (F 1,28 = 150.719, p < .001). The full scale NASA-TLX workload measure also revealed a significant difference, which is unlike the earlier studies [28] where no advantage was found for panorama pictures GUI. A significant advantage in workload (F 1,28 = 7.347, p = .001) was observed favoring the image queue condition (Figure 9 ). Examining individual dimension of workload comparing streaming video and image queue we found significant differences for temporal demand (F 1,28 = 6.503, p = .016) and effort (F 1,28 = 4.576, p = .040) (Figure 10 ).

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


In lieu of swap: Analyzing compressed RAM in Mac OS X and Linux The forensics community is increasingly embracing the use of memory analysis to enhance traditional storage-based forensics techniques, because memory analysis yields a wealth of information not available on non-volatile storage. Memory analysis involves capture of a system's physical memory so that the live state of a system can be investigated, including executing and terminated processes, application data, network connections, and more. One aspect of memory analysis that remains elusive is investigation of the system's swap file, which is a backing store for the operating system's virtual memory system. Swap files are a potentially interesting source of forensic evidence, but traditionally, most swap file analysis has consisted of string searches and scans for small binary structures, which may in some cases be revelatory, but are also fraught with provenance issues. Unfortunately, more sophisticated swap file analysis is complicated by the difficulty of capturing mutually consistent memory dumps and swap files, the increasing use of swap file encryption, and other issues. Fortunately, compressed RAM facilities, such as those in Mac OS X Mavericks and recent versions of the Linux kernel, attempt to reduce or eliminate swapping to disk through compression. The storage of compressed pages in RAM both increases performance and offers an opportunity to gather digital evidence which in the past would have been swapped out. This paper discusses the difficulty of analyzing swap files in more detail, the compressed RAM facilities in Mac OS X and Linux, and our new tools for analysis of compressed RAM. These tools are integrated into the open-source Volatility framework. Traditionally, digital forensics has focused primarily on non-volatile storage devices and involved preservation, imaging, recovery, and analysis of files stored on hard drives, removable media, etc. That investigative model typically embraced a "pull the plug and image" strategy, which involved powering down forensic targets without regard for their live state and making copies of non-volatile storage devices for analysis. This resulted in loss of a significant amount of potentially actionable digital evidence, including information about currently executing processes, live network connections, data in the clipboard, volatile malware, and other OS and application data structures. Increasingly, the forensics community has become aware of the potential for live forensics and memory analysis to enhance the investigative process, yielding evidence not available on non-volatile storage. Live forensics typically involves a survey of a running machine "on-the-spot", using a set of statically compiled binaries which are executed on the target to glean information about its state and available evidence. These tools are often traditional systems administration tools, which list running processes, monitor filesystem activity, capture network traffic, monitor changes to the Windows registry, and attempt to detect malware, such as keystroke loggers. Memory analysis typically involves capture of a system's physical memory (e.g., a RAM dump, acquired via a combination of software and/or hardware) for later investigation, offline, using memory analysis tools. Live forensics and memory analysis are similar in that they both potentially offer a wealth of data to a forensic investigator that would be otherwise unavailable. Similarly, they are both potentially invasive, disturbing the state of a running system to varying degrees, but memory analysis strives to minimize this disruption by requiring only that a memory dumping utility be executed on the system, rather than a number of evidence-gathering applications. Because of recent research advances in memory analysis, much of the live system state observable with live forensics can now be recreated in the lab from a physical memory dump. One aspect of memory analysis that remains elusive is investigation of the system's swap file, which is a backing store for the operating system's virtual memory system. Virtual memory is discussed in greater detail in Section Memory analysis for modern virtual memory systems, but briefly, the swap file is typically stored on disk and contains the contents of physical memory pages that have been swapped out due to high memory pressure, essentially, a shortage of RAM induced by running large numbers of or particularly memory-hungry applications. The swap file can therefore contain actionable evidence, but because the swap file can be large and is stored on slow, non-volatile media, capturing a mutually consistent copy of both RAM and the swap file while a system continues to execute is very challenging. 1 There are additional challenges in swap file analysis, which are discussed in detail in Section Swap files as a source of evidence, but a new virtual memory component emerging in modern operating systems, called compressed RAM or compressed swap, offers an opportunity to gather digital evidence which in the past would have been swapped to disk. After providing some additional background in the following sections on virtual memory systems and memory analysis, we discuss our newly developed plugins for the Volatility framework, which automatically identify and decompress compressed memory regions in both Mac OS X Mavericks and Linux, making this data available for analysis. We also discuss the results of a series of experiments, which offer insight into the quantity and quality of the additional evidence made available by our plugins. Whereas swap was previously difficult to integrate into forensics analysis, the new implementations of in-memory, compressed swap in Mac OS X and Linux make much of the data available to investigators using standard memory acquisition techniques. In this paper we documented the implementations of compressed swap on both Mac OS X Mavericks and Linux and described our new open source tools for decompressing and analyzing compressed memory regions. The tools, which are integrated into the Volatility framework, allow extraction of all compressed pages at once and for per-process extraction. The complete extraction of all pages can be very useful when performing searches for content like personally identifiable information (PII), financial information, or for malware with Yara signatures. The per-process extraction ensures that as much information as can be found in memory will be included in the process memory dumps. Our experiments suggest that in many circumstances, processing compressed memory will make the difference between the investigator being able to properly analyze a process (or not), as a substantial amount of a process's address space may be compressed. We also illustrated a number of useful forensics artifacts discovered in compressed pages, in routine testing, reinforcing our position that analysis of compressed swap is necessary. We expect that compressed RAM will be integrated into other operating systems, such as Microsoft Windows, in the future, based on the positive reviews associated with the Mac and Linux implementations. Closed source operating systems will present new challenges, as modeling of substantial portions of the virtual memory system is necessary for decompressing memory. Supporting analysis of compressed swap on mobile devices is also part of our future work. For example, Android 4.4 optionally enables frontswap, and handling compressed swap on ARM-based devices will require updates to Volatility's core functionality (e.g., the page table entry walking code will require updates to support ARM).. Another aspect of our future work is creation of publicly available memory samples with compression enabled. When we began this project, there were no publicly available memory samples that we could use. To support There has been a substantial amount of high-quality research in memory analysis since the 2005 DFRWS memory analysis challenge provided the catalyst for work in this area. Volatility (The volatility framework) has emerged as a primary framework for integrating new results in memory analysis for Linux, Windows, Mac, and Android systems, but a number of other research efforts have contributed substantially to the analysis capabilities that investigators now have at their disposal (Sylve et al., 2012; Carbone et al., 2009; Dolan-Gavitt et al., 2009; Schuster, 2006; Carrier, 2006; Adelstein, 2006; Case et al., 2008 Case et al., , 2010a Case et al., , 2010b Van Baar et al., 2008; Dolan-Gavitt, 2007; Vidas, 2007) .. Little formal work exists on swap file analysis, due primarily (we suspect) to many of the problems we pointed out in Section Swap files as a source of evidence. As we noted there, some of our observations regarding the difficulties related to swap file analysis are due to Kornblum (Kornblum, March 2007) and Walters (Petroni et al., 2006) . One recent research effort that targets swap file analysis in conjunction with traditional memory analysis is the work by Javaid et al., which analyzes process heap behavior to detect malware and uses virtual machine introspection to incorporate live swap file analysis (Javaid et al., 2012) .. The Linux and Mac compressed swap facilities discussed in this paper are new and to our knowledge, we are the first researchers to evaluate the forensic value of these facilities and to present tools capable of analyzing compressed swap.

O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A multi-collection latent topic model for federated search Abstract Collection selection is a crucial function, central to the effectiveness and efficiency of a federated information retrieval system. A variety of solutions have been proposed for collection selection adapting proven techniques used in centralised retrieval. This paper defines a new approach to collection selection that models the topical distribution in each collection. We describe an extended version of latent Dirichlet allocation that uses a hierarchical hyperprior to enable the different topical distributions found in each collection to be modelled. Under the model, resources are ranked based on the topical relationship between query and collection. By modelling collections in a low dimensional topic space, we can implicitly smooth their term-based characterisation with appropriate terms from topically related samples, thereby dealing with the problem of missing vocabulary within the samples. An important advantage of adopting this hierarchical model over current approaches is that the model generalises well to unseen documents given small samples of each collection. The latent structure of each collection can therefore be estimated well despite imperfect information for each collection such as sampled documents obtained through query-based sampling. Experiments demonstrate that this new, fully integrated topical model is more robust than current state of the art collection selection algorithms. In this paper we have introduced a topic modelling approach to collection selection based on a hierarchical latent Dirichlet allocation (MCTM) model with document groupings. We have shown with extensive experiments that the topic model performs comparably with state-of-the-art collection selection approaches and that it outperforms a non-hierarchical topic modelling approach, LDA, when applied to this problem.. We note that while the MCTM model contains a number of parameters (four smoothing parameters and a topics count), good or optimal values can be chosen for these parameters based on model fit, by maximising the likelihood of the sampled documents. Thus the parameter values are selected in a way that is independent of the collection selection problem itself. Since the model parameters can be chosen to best fit the data and not to maximise collection selection performance on training data, we can consider the MCTM based resource ranking algorithm to be an unsupervised learning technique. Thus our approach is somewhat different from that of central-rank-based collection selection (CRCS) and other collection selection algorithms, where the parameters of the algorithm need to be tuned using a set of test queries and relevance judgements for best performance on a new set of resources. This tuning requires considerable effort in generating representative sets of queries, and labelling relevant documents. Although estimating the MCTM is computationally intensive in comparison to other models, this is an off-line task and parameter estimation can be parallelized (Asuncion et al. 2008) . It is important to note, however, that at retrieval time the model is comparable to existing collection selection models.. Topic modelling approaches such as LDA can deal to a certain extent with problems of synonymy and polysemy due to the fact that topics are defined and discovered by the co-occurrence of words across documents. Synonymy and polysemy are particularly important problems for federated search and collection selection in particular because of the small samples of documents that are often used to represent large collections. Thus LDA and MCTM may offer a principled way of dealing with this problem. One of the biggest problems with query-based sampling (QBS) of uncooperative collections is missing vocabulary in the sample. The MCTM model may be capable of dealing with this problem by ''inferring'' the presence in each sample of additional vocabulary terms from high density topics, since the term distributions for those topics are estimated across the samples from the different collections.. The usefulness of the MCTM model is not limited to collection selection. Once a model has been learnt using samples of each collection, the model can be used for a number of different purposes, including predicting the source of a document, assigning new documents to collections, determining the similarity between collections (based on their topic representation), visualizing the contents of collections (e.g. using the topical equivalent of a ''tag cloud''), and facilitating navigation through different collections.. Future work includes investigating more complicated collection models which allow for correlation between topics within individual documents, such as correlated topic models (Blei and Lafferty 2007) or Pachinko allocation (Li and McCallum 2006) , to see if collection ''aware'' versions of these models can be developed and adapted to the collection selection problem. A second interesting direction would be to investigate hyperparameter estimation (Wallach 2008) as well as non parametric topic modelling techniques based on Dirichlet processes (Teh et al. 2006) , where the optimal number of topics for the model is discovered during model estimation. In general we observed that no single technique dominates performance across the different testbeds. This indicates that there is a certain amount of variance in the performance of state-of-the-art approaches depending on the test corpus. 7 Moreover, the fact that MCTM performed well (albeit not always the best) across the different collections provides strong indication of the quality of the technique. For example, although the approach did not provide statistically significant improvements in comparison to CRCS(e), the variability (or uncertainty) of performance was found to be smaller when using MCTM.. The results indicated that MCTM consistently outperforms the non-hierarchical LDA approach. Showing that the additional modelling complexity results in a better estimation for the likelihood of a query given the collection P(q|c). The results also indicated a trend that MCTM improved at larger cut-off values. In other words, MCTM performed better as more servers were selected. This suggests that using information from topically related collections does improve the representation of collections. CRCS(e) and ReDDE-LM, by Fig. 12 Standardised relative P@10 value at ten collections selected. The top plot reports the standardised scores for each technique and the bottom plot is the mean performance score across all collections. The 95% confidence interval for the mean is also presented 7 The variance in selection performance across different test collections has been observed previously by researchers in DIR including French et al. (1998). design, perform well at lower cut-off values. This is because if a collection sample already contains one or more relevant documents for the query, these documents are more likely to be ranked high in the centralised sample index resulting in high precision. However, if a collection representation does not contain a relevant document then that collection is less likely to be ranked highly. The results would indicate that the MCTM model could be addressing this limitation by implicitly smoothing collection representations with information from other topically related collections.. Finally, it is possible that a combination of different techniques (in particular a mixture of topic modelling and the simple language modelling used in ReDDE-LM) might result in more robust performance across different testbeds. A fusion approach to collection selection may result in significant gains combining a mixture of different strategies.

O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O