Substructure based approach to finite element model updating A substructure-based finite element model updating technique is proposed in this paper. A few eigenmodes of the independent substructures and their associated derivative matrices are assembled into a reduced eigenequation to recover the eigensolutions and eigensensitivities of the global structure. Consequently, only the concerned substructures and the reduced eigenequation are re-analyzed in the optimization process, thus reducing the computational load of the traditional model updating methods which perform on the global structure. Applications of the proposed substructure-based model updating to a frame structure and a practical bridge demonstrate that the present method is computationally effective and efficient. Accurate finite element (FE) models are frequently required in a large number of applications, such as optimization design, damage identification, structural control and structural health monitoring [1] .. Due to uncertainties in the geometry, material properties and boundary conditions, the dynamic responses of a structure predicted by a highly idealized numerical model usually differ from the measured responses. For example, Brownjohn et al. [2] reported that the differences between the experimental and numerical modal frequencies of a curved cable-stayed bridge exceeded 10% for most modes and even reached 40% in some cases. In another study [3] , 18% difference was found between the analytical and measured frequencies. Jaishi and Ren [4] observed differences of up to 20% in the natural frequencies predicted by an FE model and those measured in a steel arch bridge, and reported that the Modal Assurance Criteria (MAC) values could be as low as 62%. Similarly, Zivanovic et al. [5] found that the natural frequencies of a footbridge predicted by an FE model in the design before updating differed from their experimental counterparts by 29.8%. Therefore, an effective model updating method is necessary to obtain a more accurate FE model that can be used for other purposes such as prediction of response and damage identification.. Model updating methods are usually classified into two categories: one-step methods and iterative methods [6] . The former directly reconstruct the stiffness and mass matrices of the analytical model, while the latter modify the physical parameters of the FE model repeatedly to minimize the discrepancy between the analytical modal properties (frequencies and mode shapes) and the measurement counterparts. The iterative methods are becoming more popular, since they allow for the physical meaning of the predicted parameters to be reflected, and the symmetry, positive-definiteness and sparseness in the updated matrices to be preserved.. Most iterative model updating methods employ optimization techniques, which require the eigensolutions and associated sensitivity matrices of the analytical model to be calculated in each iteration [7] . As the analytical model of a practical structure in civil engineering usually comprises a large number of degrees of freedom (DOFs) and contains many uncertain parameters that need to be updated, extracting the eigensolutions and associated eigensensitivities from the large-size system matrices is very time-consuming.. The substructuring method is preferable to cope with large scale structures. In general, the individual substructures are analyzed independently to obtain designated solutions, which are subsequently assembled to recover the properties of the global structure by constraining the interfaces of the adjacent substructures [8] . The substructuring method is advantageous mainly in three aspects. First, as the global structure is replaced by smaller and more manageable substructures, it is much easier and quicker to analyze the small system matrices. Second, the separated substructures are analyzed independently. When one or more substructures are modified, only the modified substructures need to be reanalyzed while the others remain unchanged [9] . This property can be promising when applied to model updating or damage identification field. When the uncertain parameters or the damage areas are localized within parts of a structure, only one or more substructures containing those parts are re-analyzed during model updating or damage identification, and the other substructures are untouched [10, 11] . The substructuring method will be more efficient when some identical substructures exist or when the substructuring method is incorporated with parallel computation. Finally, the substructuring method is helpful to be combined with the model reduction technique in calculating the eigensolutions and eigensensitivities [12, 13] .. Kron [14] first proposed a substructuring method to study the eigensolutions of large scale systems in a piecewise manner, and it has been developed by the authors in terms of efficiency [15, 16] . This paper attempts to extend the substructuring method to calculate the eigensolutions and eigensensitivities for the sensitivity-based model updating process. The eigensolutions and eigensensitivities of the global structure are recovered from a few eigenmodes and their associated derivatives of the independent substructures. In particular, eigensensitivity with respect to an elemental parameter of the global structure is calculated from the derivative matrices of one substructure that contains the element. The derivatives of other substructures to the elemental parameter are zero. This can save a large amount of computational effort in calculation of the eigensolutions and eigensensitivities which dominate the model updating process. The effectiveness and efficiency of the proposed method are illustrated through a numerical frame structure and a practical bridge. This paper has proposed a substructuring method to calculate the eigensolutions and eigensensitivities for the model updating purposes. The eigensolutions of the global structure are calculated from some lowest modes of the substructures. Calculation of the eigensensitivities with respect to an elemental parameter requires analysis of the sole substructure that contains the element.. Since the model updating process involves frequent calculation of the eigensolutions and eigensensitivities, this substructure-based model updating method is advantageous in improving the computational efficiency.. The effectiveness of the substructure-based model updating method was illustrated by a frame structure. Although the substructuring method introduces some slight errors in calculation of the eigensolutions and eigensensitivities, the effect of the errors on the model updating results is negligible when proper master modes are retained in the substructures. Application to a practical bridge demonstrates that the proposed substructure-based model updating is efficient to be applied to update large-scale structures with a large number of design parameters. The accuracy of the proposed substructuring method is improved by including more master modes in the substructures. Fig. 1 . The frame structure. Fig. 2 . Location and severity of discrepancies identified using the substructuring method. Tables Table 1. The assumed stiffness reduction in the elements. Table 2 . The frequencies and mode shapes of the frame using the proposed substructuring method. Table 3 . The frequencies and mode shapes of the frame structure before and after updating (case 2). Table 4 . The frequencies and mode shapes of the frame structure before and after updating (case 3). Table 5 . The information of division formation with 11 substructures. Table 6 . Comparison of the computation time and the number of iterations. Table 7 . The frequencies and mode shapes of the bridge before and after updating.

O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Topic (query) selection for IR evaluation The need for evaluating large amounts of topics (queries) makes IR evaluation an uneasy task. In this paper, we study a topic selection problem for IR evaluation. The selection criterion is based on the overall difficulty of the chosen set, as well as the uncertainty of the final IR metric applied to the systems. Our preliminary experiments demonstrate that our approach helps to identify a set of topics that provides confident estimates of systems' performance while keeping the requirement of the query difficulty. Given the tremendous amount of data on the Web and the huge volumes of queries that users issue to search engines, it is a big challenge to evaluate how well a search system can perform well to satisfy users' information needs. In order to tackle the challenge, we propose to select a set of queries from a large query pool so that a system's performance on the selected set can serve as an accurate indicator of the system's performance on the whole pool of queries.. The Cranfield evaluation methodology, which has been adopted in evaluation initiatives such as the Text REtrieval Conference (TREC), uses standard test collections consisting of documents, topics, and relevance judgments. An IR system produces a ranked list of documents for each topic, a metric such as average precision (AP) is calculated on the ranked results and indicates how well this system did on this topic. The mean of such a metric over a number of topics (e.g. mean average precision or MAP), is generally used to measure the system's overall performance.. The robustness and reliability of this methodology has been well studied (e.g. [1, 6, 5] ). The sources of error include limited pool depths, incomplete relevance information, topic set size, averaging across topics with different numbers of relevant documents etc. Given such shortcomings, when we use a set of topics and relevance judgements to evaluate a system, there will be some uncertainty in our estimate. Motivated by the Modern Portfolio Theory of finance [2] , we attempt to account for, quantify and then minimize the uncertainty involved in running such an evaluation.. In this context, we consider topic difficulty, which potentially convolutes many factors: (i) individual systems find particular topics difficult (ii) topics are likely to be difficult for all systems if they do not have corresponding content in the collection (iii) ambiguous or wide topics can be difficult even if the collection contains rele- vant content. An estimate of a topic's intrinsic difficulty (i.e., (ii) and (iii) above) can be obtained by looking across systems' performance on that topic.. We first propose our approach in Section 2, present experimental results in Section 3, and conclude in Section 4. This paper considered the standard IR evaluation task of calculating the relative effectiveness of a group of systems based on a set of topics. The outcome of such a measurement has some inherent uncertainty due to properties of individual queries/topics in the set, as well as interactions between them. Motivated by the Modern Portfolio Theory, we propose a novel mean-variance based topic selection approach for minimizing the uncertainty in the measured effectiveness. Given a restriction on the size of the subset, our approach identifies those topics that reduce the uncertainty of systems' effectiveness, under the condition that the overall difficulty of the set matches predefined preferences.

O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Incremental integration testing of concurrent programs We present a method for selecting test sequences for concurrent programs from labeled transitions systems (LTS). A common approach to selecting test sequences from a set of LTSs is to derive a global LTS, called the reachability graph, and then force deterministic program executions according to paths selected from the graph. However, using a reachability graph for test path selection introduces a state explosion problem. To overcome this problem, a reduced graph can be generated using incremental reachability analysis, which consists of repeatedly generating a reachability graph for a subset of LTSs, reducing this graph, and using the reduced graph in place of the original LTSs. Unfortunately, existing incremental reachability analysis techniques generate reduced graphs with insufficient information for deterministic testing. We present an incremental approach to testing concurrent programs. Incremental testing consists of incremental reachability analysis for test path selection and deterministic testing for test execution. We define a new type of reachability graph for incremental analysis, called an annotated labeled transition system (ALTS). An ALTS is an LTS annotated with information necessary for deterministic testing. We propose practical coverage criteria for selecting tests paths from an ALTS and present an ALTS reduction algorithm. The results of several case studies are reported. To select test sequences Select a set of paths from reachability graph. For each selected paths, derive one or more inputs and force deterministic executions according to the pathdeterministic testing.

O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Mechanisms for store-wait-free multiprocessors Abstract Store misses pose a significant performance challenge on sharedmemory multiprocessors [7] . In uniprocessors, store miss latency can be hidden with small, simple store buffers that allow other accesses to proceed past store misses [3] . 1 Unfortunately, in multiprocessor systems, application correctness often depends on the precise order of accesses to distinct memory locations (e.g., acquiring a lock before accessing a shared variable) [18] . To enable programmers to reason about the correctness of shared memory accesses, multiprocessor system designers have developed elaborate programming models, called memory consistency models, which establish minimal ordering guarantees that hardware must enforce [1] . To implement these guarantees, conventional processors often stall other accesses while in-flight store misses complete.. Today, programmers must choose from a spectrum of consistency models that trade increasing programming complexity for hiding more store-related stalls. At one extreme, strictly-ordered consistency models provide the most intuitive programming interface, but current implementations expose substantial store miss latency. At the other extreme, relaxed consistency models allow unrelated accesses to proceed past store misses, but require programmers to explicitly annotate accesses where precise ordering is required. Ideally, we would like to provide programmers with the simplest programming models and yet, as in uniprocessors, never stall as a result of store misses [16] . We call such designs store-wait-free multiprocessors.. Store delays arise from two sources: (1) insufficient store buffer capacity during store bursts, and (2) ordering constraints that stall execution until all in-flight stores complete. Under strictly-ordered consistency models, ordering stalls dominate, as loads may not retire if a store miss is outstanding. Even in aggressive implementations that hide some stalls via prefetching and speculative load execution [10] , substantial stalls remain [7, 11, 13, 26, 27] , accounting for nearly a third of execution time in commercial applications [25] . In models that relax load-to-store order [7] , the key bottleneck shifts to capacity stalls. Conventional store buffer capacity is limited because every load must associatively search the buffer for a matching store, and this search is on the load's critical path.. Fully-relaxed models alleviate store buffer capacity pressure by allowing stores to drain in any order, but still incur substantial ordering delays at synchronization instructions [7, 13, 25, 36] .. To bridge the performance gap among consistency models, researchers have proposed systems that speculatively relax ordering constraints and/or retire instructions past synchronization operations [12, 13, 16, 21, 23, 24, 27, 36] . If a memory race exposes an ordering violation, these systems roll back execution to the violating instruction. Speculation improves performance because races are rare in well-behaved concurrent applications, and hence most ordering delays are dynamically unnecessary. Unfortunately, existing designs address only specific classes of synchronization stalls [21, 23, 24, 36] , provide insufficient speculation depth [27] , replace all stores with read-modify-write operations [12, 13] , and/or implement impractical fine-grained rollback mechanisms that recover precisely to the violating instruction [12, 13] .. We propose two mechanisms to address limited store buffer capacity and relax ordering constraints with practical hardware. Together, these mechanisms enable store-wait-free implementations of any consistency model.. We eliminate store-buffer-capacity stalls through the scalable store buffer (SSB). The SSB eliminates the non-scalable associative search of conventional store buffers by forwarding processor-visible/speculative values to loads directly from the L1 cache while maintaining globally-visible/committed values in L2. The SSB maintains store order in a FIFO structure that does not require an associative search, but enables a properly-ordered commit to L2.. We eliminate ordering-related stalls through atomic sequence ordering (ASO). Rather than enforce ordering constraints on each memory access, ASO enforces ordering at coarse granularity over access sequences that appear atomic to other processors. If a data race exposes an ordering violation, ASO recovers to the start of the nearest atomic sequence. Because it uses coarse-grain rollback, ASO can be implemented with efficient checkpoint-based rollback mechanisms [2, 8, 20] . ASO leverages the SSB to provide sufficient speculation depth and recover correct memory state using the ordered FIFO in the event of a violation.. With these mechanisms, we construct store-wait-free implementations of sequential consistency (SC) and SPARC total store order (TSO). We compare these to a conventional implementation of SPARC relaxed memory order (RMO). Using cycle-accurate fullsystem simulation of scientific and commercial multiprocessor applications, we demonstrate: • Substantial store stalls under all consistency models. In even the most aggressive conventional implementations, we observe an average of 31% of execution time on store-related stalls under SC, 16% under TSO, and 14% under RMO.. • Scalable buffering bridges the TSO-RMO gap. We demonstrate that the SSB provides store buffer capacity exceeding application demands, closing the performance gap between TSO and RMO. The SSB improves performance by as much as 45% over conventional TSO.. • Store-wait-free implementations. ASO further eliminates dynamically unnecessary ordering constraints, including synchronization stalls. Our mechanisms enable SC or TSO guarantees while outperforming RMO by 6% to 9%.. The remainder of this paper is organized as follows. In Section 2, we provide motivation and high-level descriptions of our mechanisms. We present hardware details in Section 3 and evaluate our designs in Section 4. In Section 5, we discuss related work. We conclude in Section 6. In this paper, we have shown that store-related capacity and ordering stalls degrade the performance of commercial applications under all memory consistency models, including relaxed models. We propose two mechanisms to address these stalls, the scalable store buffer to eliminate capacity stalls, and atomic sequence ordering to eliminate ordering stalls. With these mechanisms, systems can provide the intuitive programming interface of stronglyordered consistency models with performance exceeding fullyrelaxed models. Furthermore, the underlying hardware mechanisms required (e.g., register checkpoints, tracking of speculative reads, speculative write data within the cache hierarchy) overlap strongly with the mechanisms required to implement emerging programming models like transactional memory. By implementing these underlying mechanisms in future designs, systems architects can provide a migration path to new programming models while enabling store-wait-free multiprocessing for legacy code. Adve and Gharachorloo present an overview of the differences between memory consistency models in [1] . Several prior studies investigate the performance gap between consistency models in scientific [11, 13, 26, 27] and commercial [7, 25] applications. Our results corroborate these studies; even under fully-relaxed consistency models, programs incur significant delays because of ordering constraints at synchronization operations.. Chou, Spracklen, and Abraham [7] introduce the epoch model for analyzing store misses in commercial applications. They evaluate several optimizations to reduce store stalls, including the store miss accelerator to reduce capacity misses, and hardware scouting to improve store prefetching effectiveness. These mechanisms can be combined with ASO and the SSB. Finally, they demonstrate that speculation techniques (from [23] ) can bridge the performance gap among consistency models, but their analysis assumes speculation always succeeds and does not evaluate a hardware implementation. We evaluate our mechanisms using cycle-accurate full-system simulation of a shared-memory multiprocessor using FLEXUS [37] . FLEXUS models the SPARC v9 ISA and can execute unmodified commercial applications and operating systems. FLEXUS extends the Virtutech Simics functional simulator with models of an out-oforder processor core, cache hierarchy, protocol controllers and interconnect. We simulate a 16-node directory-based shared-memory multiprocessor system running Solaris 8. We implement a lowoccupancy directory-based NACK-free cache-coherence protocol. Our system performs speculative load execution and store prefetching (at execute under SC, at retirement under other consis- tency models) as described in [7, 10] . Hence, our base system is similar to the system described in [25] . We configure our processor model to approximate the hardware resources of the Intel Core 2 microarchitecture. We list other relevant parameters in Table 1 (left). We drive the web servers using a separate client system (client activity is not included in timing results). Finally, we include two scientific applications as a frame of reference for our commercial application results.. We measure performance using the SimFlex multiprocessor sampling methodology [37] . The SimFlex methodology extends the SMARTS [38] statistical sampling framework to multiprocessor simulation. Our samples are drawn over an interval of from 10s to 30s of simulated time for OLTP and web server applications, over the complete query execution for DSS, and over a single iteration for scientific applications. We launch measurements from checkpoints with warmed caches and branch predictors, then warm queue and interconnect state for 100,000 cycles prior to measuring 50,000 cycles. We use the aggregate number of user instructions committed per cycle (i.e., committed user instructions summed over the 16 processors divided by total elapsed cycles) as our performance metric, which is proportional to overall system throughput [37] .. FLEXUS fully supports execution under the SC, TSO and RMO consistency models. Unfortunately, the commercial applications we study include code that requires TSO for correctness. Hence, while we can simulate all applications directly under SC and TSO, and scientific applications under RMO, our commercial applications lack the necessary memory barriers to run under RMO.. To work around this limitation, we dynamically insert memory barrier instructions when simulating our commercial applications under RMO. These barriers stall retirement until all preceding memory operations complete, but allow loads within the instruction window to execute speculatively. Because it is difficult to automatically identify all locations where barriers are required, we err on the side of inserting too few barriers, thus overestimating RMO performance (conservatively underestimating speedups from ASO). We introduce a barrier after each RMW operation (i.e., lock acquire). However, we do not introduce memory barriers at lock release because it is difficult to reliably identify the releasing store. As these barriers are not sufficient to guarantee correctness, we separately track TSO-consistent memory values and confirm that all loads return values that occur under TSO. In most cases, ordering differences between TSO and RMO do not affect the value returned by a load. If a memory ordering difference causes a load value mismatch, the affected load and its dependent instructions are re-executed to assure correct program execution. These mismatches are rare and have a negligible performance impact.

O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O B O O B O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B O O O B O O O B O O O O O O B O O O B O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O B O O O O B O O O O B O O O O O B O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O B I I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O B O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O


Multimodal group action clustering in meetings We address the problem of clustering multimodal group actions in meetings using a two-layer HMM framework. Meetings are structured as sequences of group actions. Our approach aims at creating one cluster for each group action, where the number of group actions and the action boundaries are unknown a priori. In our framework, the first layer models typical actions of individuals in meetings using supervised HMM learning and low-level audio-visual features. A number of options that explicitly model certain aspects of the data (e.g., asynchrony) were considered. The second layer models the group actions using unsupervised HMM learning. The two layers are linked by a set of probabilitybased features produced by the individual action layer as input to the group action layer. The methodology was assessed on a set of multimodal turn-taking group actions, using a public five-hour meeting corpus. The results show that the use of multiple modalities and the layered framework are advantageous, compared to various baseline methods. The automatic analysis of meetings has recently attracted attention in a number of fields, including audio and speech processing, computer vision, human-computer interaction, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. and information retrieval [23, 16, 5, 13, 9, 24, 7] . Analyzing meetings poses a diversity of technical challenges, and opens doors to a number of relevant applications. On one hand, meetings constitute an important case study of human interaction. Understanding people interaction has been a long-term goal in social psychology [15] , so a computational framework to analyze group behavior could be useful to facilitate analysis performed by psychologists in organizations (e.g., for training of staff on issues like interpersonal communication and teamwork management). On the other hand, meetings can be seen as raw, unlabeled data, possibly generated in large amounts, for which automatic analysis could add value for browsing and retrieval purposes, e.g., to structure a single meeting into a sequence of high-level items, or to discover recurrent patterns in a large meeting collection.. Meetings are characterized by their multimodal and group nature [11, 15] . Regarding the first factor, single modalities [23, 16, 9, 24] have been used for various tasks, but there are few works that model individual and group behavior in conversational settings using multiple modalities (as captured by a network of cameras and microphones) [3, 13, 14] , despite the experimental evidence supporting this approach. For the second factor, a meeting can be seen as proceeding through phases, where a group disseminates information, discusses, and makes decisions [15] . A simple model can thus be used to define a meeting as a continuous sequence of group actions (i.e., involving multiple simultaneous participants) chosen from one or more pre-defined action dictionaries, which is well suited for supervised learning [13, 7] , as long as the action dictionaries are well defined. This implies that the actions comprising each dictionary should be mutually exclusive, exhaustive, and unambiguous to human observers, at least to a degree for which manually labeled data for supervised learning can be reliably generated.. In reality however, meetings are not restricted to predefined action sets. Furthermore, high-level group actions in meetings can be ambiguous (and expensive) to label. Roughly speaking, the degree of ambiguity correlates with the actions' level of semantic meaning. Basic actions like writing or speaking can be clearly identified, group actions like discussions are more ambiguous, and high-level actions like information sharing might be very difficult to label reliably, which could seriously challenge supervised methods.. In this view, modeling high-level group actions with unsupervised approaches, which find "action structure" in either individual meetings or whole collections, without the need for labeled data or previous knowledge of the actions, become very attractive options [26, 28] , especially given the vast amount of data that is generated in many real cases. Given adequate features, clustering an individual meeting could partition it into action-consistent segments. Clustering an entire collection could further find action-consistent clusters across meetings. Additionally, unsupervised methods could naturally deal with variations (e.g. in the number of participants) that would otherwise need to be modeled explicitly in supervised methods.. In this paper, we present a layered probabilistic framework for group action clustering in meetings, as an alternative to fully supervised methodologies. Through the definition of an adequate set of individual actions, we decompose the group action clustering problem into two layers. The first one performs supervised learning to recognize individual actions of participants using low-level audio-visual (AV) features. Supervision at this level can be especially convenient because individual actions are often well-defined and thus can be reliably labeled. Individual actions constitute the link between low-level AV features and high-level group actions. The second layer models group actions in an unsupervised way, using the output of the first layer as observations, and producing a temporal segmentation of a meeting into group action segments. Both layers use HMM-based approaches for action recognition and clustering, respectively. Our framework is extensible: with minor modifications, it can be used to cluster group actions in either individual meetings or in an entire meeting collection. We apply the methodology to a publicly available meeting corpus, for a set of eight group actions based on multimodal turn-taking patterns, and illustrate its validity with respect to a number of baseline methods. In our view, our methodology constitutes an attractive option for analysis of high-level group actions in meetings, due to its potential to deal with actions that would otherwise be difficult to pre-define and/or expensive to label.. The rest of the paper is organized as follows. Section 2 reviews related work. Section 3 introduces our approach. Section 4 presents experiments and discussion. Concluding remarks are provided in Section 5. In this paper, meetings were defined as sequences of multimodal group actions. We addressed the problem of clustering group actions, proposing a layered HMM framework to decompose the group action clustering problem into two subproblems. The first layer maps low-level AV features into probability-based, individual-action features. The second layer groups such features into clusters, which correspond reasonably well to group actions. Experiments on a public meeting corpus demonstrate the effectiveness of our framework. For future work, we will consider the use of semi-supervised approaches to assign semantic labels to the clustering outputs, and the extension of our approach to other dictionaries of group actions. Learning-based approaches for the automatic interpretation of human activities in videos have been used for the past ten years. Most works have focused on supervised learning methods, defining models for a handful of activities in a particular domain, and using statistical models for recognition. Individual action [22] , and interaction recognition [20, 10] have been predominately investigated using visual features, although some work on the speech community can be categorized as interaction recognition [9, 24] . To our knowledge however, little work has been conducted on recognition of group-based, multimodal actions from multiple audio-visual streams captured by cameras and microphones [3, 13, 14] . [3] described automatic discovery of "influence" in a lounge room where people played interactive debating games. Our previous work studied various sequence models to recognize turn-taking patterns in a formal meeting room scenario, where people discuss around a table and use a white-board and a projector screen [13, 14] .. Most of the existing work has used Hidden Markov Models (HMMs) and extensions (see [18] for a recent review of models). The basic HMM works well for temporally correlated sequential data, but it is challenged by a large number of parameters, and the risk of over-fitting when learned from limited data [19] . This situation might occur in the case of multimodal group actions where, large vectors of audio-visual features from all participants are concatenated to define the observation space [13, 14] .. The above problem has been recently addressed with hierarchical representations [19, 7, 25] . In [19] , (supervised) layered HMMs were proposed to model multimodal singleperson office activities at various time granularities. The lowest layer captured video, audio, keyboard and mouse activity features; the middle layer classifies AV features into basic events; the highest layer uses outputs of previous layers to recognize higher-level office activities. In [7] , two methods for meeting structuring from audio-only were presented, using multilevel Dynamic Bayesian Networks (DBNs). In [25] , an approach for unsupervised discovery of multilevel video structures using hierarchical HMMs was proposed, in the context of sports videos. In this model, the higher-level structure elements usually correspond to semantic events, while the lower-level states represents variations occurring within the same event. However, in both [25, 7] , the lowlevel actions have no obvious interpretation, and the number of low-level actions is a model parameter learned during training, or set by hand, which makes the structure of the models difficult to interpret. Recently, we proposed the use of layered HMMs in a fully supervised setting to recognize both individual and group actions in meetings [27] . Although promising, the method might find problems of scalability to large dictionaries of group actions, given the need for labeled training data at the group action level.. Different from supervised methods for activity recognition, unsupervised data-driven approaches find action-based clusters from the data, without a priori knowledge of the action dictionaries [26, 28] . In [26] , a normalized-cut approach was used to cluster single-person actions like running, walking, etc., using features at different temporal scales, and a distribution-based distance measure to compute similarity between video segments. One limitation of such approach was the lack of a sound mechanism to detect the number of clusters. Recently, an unsupervised technique was proposed to detect unusual human activity in a surveillance setting, using analysis of co-occurrence between video clips and motion/color features of moving objects, without the need to build models for usual activities [28] . The two approaches relied only on visual information.. Unlike previous work, our work combines supervised HMM recognition and unsupervised HMM clustering in a stratified framework, to model multimodal group actions in meetings. The layered structure in our approach, that explicitly considers different semantic levels (individual and group) coincides with the structure of meetings as modeled in social psychology [15] . The distinct treatment for each layer (supervised vs. unsupervised) tries to respond to the different nature of each of the action types.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Empirical analysis of factors affecting confirmation bias levels of software engineers Abstract Confirmation bias is defined as the tendency of people to seek evidence that verifies a hypothesis rather than seeking evidence to falsify it. Due to the confirmation bias, defects may be introduced in a software product during requirements analysis, design, implementation and/or testing phases. For instance, testers may exhibit confirmatory behavior in the form of a tendency to make the code run rather than employing a strategic approach to make it fail. As a result, most of the defects that have been introduced in the earlier phases of software development may be overlooked leading to an increase in software defect density. In this paper, we quantify confirmation bias levels in terms of a single derived metric. However, the main focus of this paper is the analysis of factors affecting confirmation bias levels of software engineers. Identification of these factors can guide project managers to circumvent negative effects of confirmation bias, as well as providing guidance for the recruitment and effective allocation of software engineers. In this empirical study, we observed low confirmation bias levels among participants with logical reasoning and hypothesis testing skills. In cognitive psychology, confirmation bias is defined as the tendency of people to seek evidence that could verify their hypotheses rather than seeking evidence that could falsify them. The term confirmation bias was first used by Wason in his rule discovery task (Wason 1960) and later in his selection task (Wason 1968) .. One can observe the effects of confirmation bias during any phase of the software development process. Due to confirmation bias, conditions which have the potential to make the software fail may be overlooked during requirements analysis and design phases of SDLC. This, in turn, leads to an increase in software defect density. In addition to the defects which are introduced during early phases of the SDLC, significant number of defects may also be introduced and/or overlooked during the implementation phase. Due to confirmation bias, developers might prefer only the unit tests to make their code run rather than unit tests, which aim to make their code fail. We argue that testers with high confirmation bias levels will employ testing strategies, which do not include adequate attempts to fail the code. As a result, most of the defects that have propagated from earlier phases of the SDLC (e.g., requirements analysis, design and implementation phases) may be overlooked leading to a significant increase in the post-release defect density.. To the best of our knowledge, Stacy and MacMillian are the two pioneers who recognized the potential effects of confirmation bias on software engineering (Stacy and MacMillan 1995) . There is also empirical evidence showing the existence of confirmation bias among testers. According to the results obtained by Teasley et al. in both naturalistic and laboratory studies of test case selection, testers are four times more likely to choose positive tests (i.e., tests that make the code run) than negative tests (i.e., tests that break the code).. In our previous empirical studies, we found a positive correlation between developers' confirmation biases and software defect density Bener 2010, 2013) . We also discovered that more post-release defects are overlooked in the modules that are tested by software engineers with high confirmation bias (Calikli and Bener 2010) . In another research study, we compared confirmation bias values of developer groups from three different projects and two different companies . We defined developer groups as the set of developers who contribute to the development of a common set of source code files. Developer groups were identified for each project by mining log files that were obtained from version management systems. Two of the project groups, Telecom1 and Telecom2, belonged to a large-scale telecommunication company and the third project group ERP belonged to an independent software vendor specialized in Enterprise-Resource Planning. Developer groups of Telecom2 and ERP project groups had confirmation bias values that were much closer to ideal values compared to those of developer groups belonging to project group Telecom1. As a result of the interviews we conducted with developers and projects managers, we discovered that negative tests were part of testing routine of developers in project groups Telecom2 and ERP besides positive tests, while only positive tests were conducted by developers in project group ERP.. In our previous studies, we defined a methodology to quantify confirmation bias levels that resulted in the formation of a confirmation bias metrics set Bener 2010a, b, 2013; Calikli et al. 2010a, b; ) . Having found a correlation between confirmation biases of developers and software defect density, we built predictive models to identify defect prone parts of software. The performance of our prediction model, which we built by using confirmation bias metrics, was comparable with the performance of the models, which we built by using product (i.e., static code) and process (i.e., churn) metrics .. In order to circumvent negative effects of confirmation bias as well as providing guidance to recruit new software engineers and to allocate existing ones to positions where they can perform most effectively, it is crucial to identify factors having significant impact on confirmation bias. In this paper, we present a methodology to quantify confirmation bias levels in terms of a single derived metric and investigate the factors which affect confirmation bias levels of software engineers.. The rest of the paper is organized as follows: In Sect. 2, we explain our research goal and the corresponding hypotheses. Our methodology to quantify/measure confirmation bias levels of software engineers is explained in Sect. 3. We give the details of our empirical analysis in Sect. 4. Results of our empirical study and discussions are presented in Sect. 5. Section 6 addresses threats to validity. Finally, we conclude and mention future work in Sect. 7. The overall aim of this research is to explore factors which affect confirmation biases of software engineers. Empirical investigation of factors affecting confirmation bias requires quantification/measurement of confirmation bias levels. For this purpose, we defined a methodology to define and extract confirmation bias metrics. During our previous studies (Calikli and Bener 2010a, b; Calikli et al. 2010a, b) , we conducted the initial form of the methodology to measure/quantify confirmation bias and the content of confirmation bias metric set was at an immature level. Hence, we were unable to define a single derived metric to measure confirmation bias levels of software engineers. In this paper, in addition to performing our empirical investigation by using a more extensive dataset, we were able to perform multi-way ANOVA test where we managed to represent the dependent variable ''confirmation bias level'' as a single value. As a result of our empirical analyses, we were able to strengthen the following claims which originate from our previous studies:. -Confirmation bias levels of individuals who have been trained in logical reasoning and mathematical proof techniques are significantly lower. In other words, given a statement, such individuals show tendency to refute that statement rather than immediately accepting its correctness. -A significant effect of experience in software development/testing has not been observed. This implies that training in organizations is focused on tasks rather than personal skills. Considering that the percentage of people with low confirmation bias is very low in the population (Wason 1960 (Wason , 1968 Evans et al. 1993) , an organization should find ways to improve basic logical reasoning and strategic hypothesis testing skills of their software engineers.. As future work, our field studies will also include observational techniques (e.g., thinkaloud protocols, participant observation and observation synchronized shadowing). In this way, we aim to strengthen our hypothesis regarding the connection between confirmation bias and defect rates through unit testing. We also aim to conduct laboratory experiments, where we can obtain detailed information about developers' unit testing activities in a controlled environment. Finally, we intend to investigate the relationship between levels of motivation and confirmation bias as well as the relationship between personality and confirmation bias.

O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I


Identifying the addressee in human-human-robot interactions based on head pose and speech In this work we investigate the power of acoustic and visual cues, and their combination, to identify the addressee in a human-human-robot interaction. Based on eighteen audiovisual recordings of two human beings and a (simulated) robot we discriminate the interaction of the two humans from the interaction of one human with the robot. The paper compares the result of three approaches. The first approach uses purely acoustic cues to find the addressees. Low level, feature based cues as well as higher-level cues are examined. In the second approach we test whether the human's head pose is a suitable cue. Our results show that visually estimated head pose is a more reliable cue for the identification of the addressee in the human-human-robot interaction. In the third approach we combine the acoustic and visual cues which results in significant improvements. Building human-friendly robots which are able to interact and cooperate with humans has been an active research field Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. in recent years [2, 1] . A major challenge in this field is to develop robots that can interact and cooperate with humans by understanding human communication modalities such as speech, gaze and gestures.. In this work we address the problem of automatically determining when a robot was addressed by human and when not. This is an important problem, when robots should eventually become companions in our daily lives. A household robot for example should know whether a person in the room is talking to him (the robot) or whether this person is talking to someone else in the room.. In this work we investigate to what extent the addressee of an utterance can be determined in a situation where people interact with each other and a robot intermittently. Rather than deciding on the basis of information of the visual and auditory channels separately, we also combine the information from both channels and compare the results with each other. For this purpose, we recorded eighteen multi-party interactions with a simulated robot and analyzed the power of head pose and acoustic cues to discriminate between the addressees of the speakers.. The remainder of this paper is organized as follows: In Section 2 we review some related work. Section 3 describes the data collection setup. In Section 4 we investigate how well the addresses of an utterance can be determined based on the visually estimated head pose of a person. Section 5 describes our experiments on identifying the addressee based on acoustic cues and analyzing the speaker's speech. In Section 6 we present experiment results for audio-visual determination of the addressee. Section 7 concludes the paper. In this work we investigated the power of acoustic and visual cues to identify the addressee in multiparty communication between two humans and a simulated robot.. First, we investigated the correlation between the addressee of a speaker and the user's head orientation: We found that looking towards another person is a very reliable indicator that the other person was addressed. In fact in 99.5% of the cases in our data when a person looked towards the other person while speaking, the person was really addressing the other person.. Looking at the robot, however, could not be used as such a clear indicator: Here, in 65% of the cases when a person A) Occurrence of "Robot" R(X) B) Parseability Z(X) C) Utterance length S(X) looked at the robot while talking, the robot also was addressed. In the remaining 35% of the cases, however, the person was indeed talking to the other human while looking at the robot, indicating that visual cues by itself might not be sufficient. We then investigated how well the addressee can be determined based on visually estimated head pose of the speaker. We employed a neural network based approach to estimate a person's head pose and then use a probabilistic model to find the most likely visual target of the speaker. With this approach and automatically learned priors and classconditional distributions for a person's head pose, we could correctly identify the visual target in 93% of the frames on four recordings. Looking towards the robot could be detected with precision of 0.74 and a recall of 0.85.. By using the estimated visual target to determine the acoustic addressee of the speaker, the correct addressee could be identified 89% of time. Speech commands towards the robot were (visually) detected with a precision of 0.6 and a recall of 0.8.. We also investigated the power of using cues that are automatically derived from the speaker's speech to distinguish between conversations between two humans and commands directed to the robot.. On a German pilot study we investigated the usefulness of various features that were derived from the hypothesis of a speech recognizer. These features included sentence length, the number of imperatives, the perplexities on different language models as well as the parseability of a sentence by a grammar for commands. Best classification results where obtained using a multi-layer perceptron as classifier.. A similar set of speech-related features was then also used to discriminate the addressees on our English data set that was collected for this study. On this data, 49% of the utterances could be correctly classified solely based on speech related features. Commands towards the robot were detected with a recall rate of 0.91 and a precision of 0.19.. Finally, we combined the purely speech-based and head pose based approaches for discriminating the addressees. This resulted in significant improvements, despite the comparably poor results of the speech-based discrimination approach: By using the weighted sum of the acoustic and visual posterior probabilities for the addressees, correct classification rate of the addressees increased from 0.9 (visually estimated) to 0.92. Furthermore, the precision of the detection of commands towards the robot was improved from 0.57 (visually) to 0.65 (combined), while keeping a recall rate of 0.81. A body of literature suggest that gaze, head pose and body orientation play an import role during social interaction and in particular are used and perceived as a signal of attention during human interaction [3, 9, 5, 7] .. The relation between gaze and speech in multi-party communication between several people recently has been investigated by Vertegaal et al. [12] . They found that subjects looked about three times more at individuals they spoke to.. Stiefelhagen and Zhu [11] have investigated the relation of eye gaze and head orientation in multi-party interaction. They concluded that head pose is a reliable cue to determine at whom someone looked in small meetings.. Other researches have investigated how people use speech and gaze when interacting with attentive objects in a smart environment. Maglio et al [8] have for instance shown that people tend to look towards objects with which they interact by speech. In their study they found that subjects nearly always looked at the addressed device before making a request.. Bakx et al. [4] have analyzed facial orientation during multi-party interaction with a multi-modal information booth. They found that users were nearly always looking at the screen of the information kiosk when interacting with the system. However, when the user was talking to a friend next to the system, the user was still looking towards the information system in 57% of the time, thereby limiting the discriminative power of facial orientation to find the addressee. Bakx et al. also analyzed using the utterance length of the speaker for discriminating between addressees. They concluded that by combining the acoustic feature with facial orientation, some improvement in detecting the correct addressee can be achieved. The data collection setup mimics the interaction between two humans and a robot. One person -acting as the hostintroduces another person -acting as his/her guest-to the new household toy, a robot. Our experiments focus on the recordings of the host, since the goal of this work is to determine if the host addresses the robot or the guest. Therefore, the guest was played by the same person throughout the collection sessions, only the hosts differ for each session and were randomly selected. Subjects included under-graduate students, graduate students and one professor. The main selection criteria was that they are native speakers of English, since all recordings were done in English. Apart from taking part in this data collection, the selected subjects were not involved in this project.. In order to provoke a challenging scenario which includes robot commands directed to the robot and also conversation about the robot, the hosts were given instructions about the task and also an example dialog beforehand. In those instructions the hosts were asked to imagine that they had recently purchased a new household robot which can do many things, such as bring drinks, adjust the lights, vacuum the room, etc. and that they now expect a guest to whom they want to introduce their robot, discuss pros and cons of robots, and give the robot some commands in order to show the capabilities. Figure 1 shows the data collection arrangement. The robot consists of a construction using a Canon VCC-1 camera to simulate the eyes, and a Sony distance microphone to simulate the robots ears. The distance between humans and robot is about 4 meters. Since we expected the far-distance speech recognition performance to deteriorate, we additionally recorded close talking speech using a Sennheiser lapel microphone.. We did audio and video recordings of 18 sessions, each of roughly 10 minutes length. The audio data were fully transcribed and tagged on the turn level to indicate whether the host addresses the robot or the guest ([command] ,[no command]). For training and tuning of speech recognition components, we divided the data into 8 sessions for training, and 5 sessions for development. For the training of the video components, we manually labeled the first 2.5 minutes of the video recordings of four out of these 8 training sessions. For evaluating the speech parts, we used another set of 5 sessions. Table 1 shows the amount of audio and video

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Unboundedness of thresholding and quantization for bandlimited signals It is well-known that the Shannon sampling series is locally uniformly convergent for all signals in the Paley-Wiener space PW 1 p . An interesting question is how this good local approximation behavior is affected if the samples are disturbed by the non-linear threshold operator. This operator, which sets to zero all samples with absolute value smaller than some threshold, arises in the modeling of many applications, e.g., in wireless sensor networks. Moreover, it constitutes an essential part of a large class of quantizers, and consequently is important for all digital signal processing applications that involve conversion between analog and digital domains. In this paper, the approximation behavior of the Shannon sampling series that only uses the samples with absolute value larger than or equal to some threshold is analyzed. It is shown that there exists a signal in PW 1 p such that the local approximation error increases unboundedly as the threshold tends to zero. Moreover, for a fixed threshold, the local approximation error can grow arbitrarily large on the set of signals whose norm is bounded by one. With this, we generalize results of Butzer et al. that were given in the paper ''On quantization, truncation and jitter errors in the sampling theorem and its generalizations, '' Signal Processing (2) 1980 [1]. We conclude the paper with a discussion about the differences in the reconstruction behavior between the sampling series which is truncated in the domain of the sampled signal, i.e., time-domain truncation, and the sampling series which is truncated in the range of the sampled signal. A well known fact [2] [3] [4] about the convergence behavior of the Shannon sampling series for signals in PW This theorem plays a fundamental role in applications because it establishes the uniform convergence on compact subsets of R for a large class of signals, namely PW 1 p , which is the largest space within the scale of PaleyWiener spaces. Unfortunately, it is not possible to extend the theorem in such a way that the uniform convergence holds on all of R for the space PW classical and quantum computation and the Shannon sampling theorem is one important step in his argumentation.. The principle of digital signal processing relies on the fact that certain bandlimited signals can be perfectly reconstructed from their samples. However, this is only true if the sample values are known exactly. For various reasons this is not always the case in applications. For example, in digital signal processing applications the samples are not known exactly because the inevitable quantization process in analog to digital conversion has limited resolution only [6, 7] . Due to its high practical importance, the analysis of the quantization error has gained a lot of attention in research [6] . A deterministic analysis of the quantization process is difficult because of the non-linear nature of the quantization operator. This fact was recognized for example in [8] , where the authors write ''Deceptively simple in its description and construction, the uniform quantizer has proved to be surprisingly difficult to analyze, precisely because of its inherent nonlinearity.'' This fact is also the reason why the quantization error is often treated probabilistically, and modeled as additive white noise [9, 10] . However, it turned out that this noise model is not always satisfactory, because it can lead to false predictions [11, 12] . In contrast, the deterministic analysis is difficult, but reveals some properties of the quantization process which cannot be analyzed with the additive noise description of the quantization error. Only a few papers conduct a deterministic analysis of the quantization process [3, 1, 13] .. In this paper, we provide for the space PW 1 p the first rigorous deterministic analysis of the pointwise behavior of the Shannon sampling series, where the samples are disturbed by the threshold operator, which sets to zero all sample values with absolute value smaller than some threshold d40. This operator constitutes an essential part of many quantization schemes, and thus the results obtained here are equally relevant for a large class of quantization operators. Of course there are also applications where the threshold operator is important on its own. Wireless sensor networks are one possible application where the threshold operator is directly involved. In wireless sensor networks the sensors sample some bandlimited signal in space or time and then transmit the samples to the receiver [14, 15] . Then, using these samples, the receiver tries to reconstruct the signal perfectly, or at least approximately if a perfect reconstruction is not possible. In order to save energy, it is common to let the sensors transmit only if the absolute value of the signal exceeds some threshold d40. In this case, the receiver has to reconstruct the signal by using only the samples with absolute value larger than or equal to the threshold d. Truncation is a very important operator, not because it is an integral part in the quantization process. In Section 3 we have briefly given the interpretation of In the following discussion we will compare the reconstruction behavior of the Shannon sampling series for both types of truncation and point out the differences. First, we contrast the global behavior of S N f and A d f . For the truncation in the domain of the signal, we have the well-known result [22] that. i.e., for fixed N, the peak value of S N f is bounded above, and it follows that. For the truncation controlled in the range we do not have such a behavior. As was shown in [16] , for all 0 o do1=3,. Hence, for fixed threshold d, 0o do1=3, the peak value of . Next, we will discuss the asymptotic speed of divergence. From (23) it follows, using some additional arguments, involving the Banach-Steinhaus Theorem and density arguments, that. p , which shows that the peak value of S N f does asymptotically grow slower than logðNÞ. It is natural to ask whether a similar result is also true for the truncation that is controlled in the range of the sampled signal. The answer to this question is open. From a practical point of view, the mere signal reconstruction is often not enough and the interest is rather in the output of a stable linear time-invariant (LTI) system. In [21] the approximation of stable LTI systems by sampling series with samples that are disturbed by the threshold operator was analyzed. It was shown that if T : PW . where h T ¼ T sinc is the impulse response of the stable LTI system T. For a precise definition of a stable LTI system, see for example [21] . From this result, the following question arises. Theorem 5, which gives a positive answer to this question for the special case where the system T is the ideal low-pass filter, may be an indication that this question can be answered in the affirmative for general stable LTI systems.. The analysis of thresholding and quantization is difficult because of the non-linear nature of both operations. This is why stochastic approaches are often used to linearize the problem. The proofs in this paper show that the findings could not have been derived with a stochastic model. In the proof of Theorem 5, we furthermore gave an explicit procedure for the construction of a divergence creating signal f 1 . This leads to the following question. In this paper we treated the problems for signals in PW 1 p , i.e., deterministic signals. For certain applications it is desirable to have results for stochastic processes also. In [23] , the mean-square convergence behavior of the Shannon sampling series was analyzed for bandlimited continuous-time wide-sense stationary stochastic processes.. It would be interesting to study the approximation of such stochastic processes if the samples are additionally disturbed by the threshold operator. In this analysis the threshold operator would still be treated deterministically.. Finally, we come to the question if we can further strengthen the divergence statements. For the global behavior of the Shannon sampling series without thresholding we have the following result. There exists a signal f 1 2 PW 1 p such that lim sup N-1 JS N f 1 J 1 ¼ 1 [22] . In this statement we have a ''lim sup'', just like in Theorems 3 and 5. For stochastic processes, a divergence result was given in [23] , where a ''lim'' is used.

O O O O B O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Leakage efficient chip-level dual-Vdd assignment with time slack allocation for FPGA power reduction To reduce power, Vdd programmability has been proposed recently to select Vdd-level for interconnects and to powergate unused interconnects. However, Vdd-level converters used in the Vdd-programmable method consume a large amount of leakage. In this paper, we develop chip-level dual-Vdd assignment algorithms to guarantee that no low-Vdd interconnect switch drives high-Vdd interconnect switches. This removes the need of Vdd-level converters and reduces interconnect leakage and interconnect device area by 91.78% and 25.48%, respectively. The assignment algorithms include power sensitivity based heuristics with implicit time slack allocation and a linear programming (LP) based method with explicit time slack allocation. Both first allocate time slack to interconnects with higher transition density and assign low-Vdd to them for more power reduction. Compared to the aforementioned Vdd-programmable method using Vdd-level converters, the LP based algorithm reduces interconnect power by 65.13% without performance loss for the MCNC benchmark circuits. Compared to the LP based algorithm, the sensitivity based heuristics can obtain slightly smaller power reduction but run 4X faster. FPGA power modeling and reduction has become an active research recently. [1, 2] present power evaluation frameworks and study power characteristics for parameterized FPGA architectures. [3] proposes configuration inversion method to reduce leakage power of multiplexers. [4] studies the interaction between a suite of power-aware FPGA CAD algorithms. In addition, low power FPGA circuits and architectures have been proposed, including region-based powergating [5] and Vdd-programmability for FPGA logic blocks and interconnects [6, 7, 8, 9, 10] .. In this paper, we are aiming at improving the Vdd programmable interconnects proposed in [8] , where a Vdd-level converter is inserted in front of each interconnect switch to avoid excessive leakage. However, the fine-grained level converter insertion introduces large leakage overhead. For example, the leakage overhead of level converters in routing channels is 36% of total power for circuit s38584 in 100nm technology. In this paper, we use the Vdd-programmable interconnect same as that in [8] , but remove the level converters in routing channels by developing novel CAD algorithms. Experimental results show that compared to [8] , we reduce interconnect leakage power and area by 91.78% and 25.48% respectively, and reduce total interconnect power by up to 65.13% without performance loss.. The rest part of the paper is organized as follows. Section 2 presents modeling and problem formulation. Section 3 presents both power sensitivity based heuristics and a linear programming (LP) based algorithm. Section 4 presents the experimental results and Section 5 concludes this paper. Proofs of Theorems are included in the technical report [11] . We have developed chip-level dual-Vdd assignment algorithms to guarantee that no VddL switch drives VddH switches. This removes the need of Vdd-level converters and reduces interconnect leakage and interconnect device area by 91.78% and 25.48% respectively compared to [8] . We have presented two simple yet practical power sensitivity based heuristics, tree based heuristic and segment based heuristic, which implicitly allocate time slack first to interconnects with higher transition density and assign VddL to them for more power reduction. We have also presented a linear programming (LP) based algorithm in which time slack is first explicitly allocated to each routing tree by formulating the problem as an LP problem to maximize a lower bound of power reduction, and then the Vdd-level assignment is solved optimally within each routing tree given the allocated time slack. We have conducted the experiments on MCNC benchmark set and compared the power reduction by the 4 The runtime includes single-Vdd placement and routing by VPR and generating the interface files between VPR and fpgaEva-LP2. three algorithms. Compared to [8] , the LP based algorithm obtains the best power reduction and reduces interconnect power by 65.13% without performance loss. The tree based heuristic and segment based heuristic reduce interconnect power by 58.03% and 64.19%, respectively. Compared to the LP based algorithm, the segment based heuristic has slightly smaller power reduction, but runs 4X faster and is effective for large circuits. It has been shown that the fine-grained Vdd-level converter insertion introduces large leakage overhead. If CAD algorithms can guarantee that no VddL interconnect switch drives VddH switches, no level converter is needed. In this paper, we propose two ways to avoid using level converters 1 . In the first approach, we enforce that there is only one Vdd-level within each routing tree, namely, tree based assignment. In the second approach, we can have different Vdd-levels within a routing tree, but no VddL switch drives VddH switches, namely, segment based assignment. To make the presentation simple, we summarize the notations frequently used in this paper in Table 1 . They will be explained in detail when first used.

O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O B I B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O