Learning hybrid neuro-fuzzy classifier models from data: to combine or not to combine? To combine or not to combine? This very important question is examined in this paper in the context of a hybrid neuro-fuzzy pattern classiÿer design process. A general fuzzy min-max neural network with its basic learning procedure is used within ÿve di erent algorithm-independent learning schemes. Various versions of cross-validation and resampling techniques, leading to generation of a single classiÿer or a multiple classiÿer system, are scrutinised and compared. The classiÿcation performance on unseen data, commonly used as a criterion for comparing di erent competing designs, is augmented by further four criteria attempting to capture various additional characteristics of classiÿer generation schemes. These include: the ability to estimate the true classiÿcation error rate, the classiÿer transparency, the computational complexity of the learning scheme and the potential for adaptation to changing environments and new classes of data. One of the main questions examined is whether and when to use a single classiÿer or a combination of a number of component classiÿers within a multiple classiÿer system. With an increasing computer power available at a ordable prices and availability of vast amount of data there is an increasing need for robust methods and systems, which can take advantage of all available information. Automatic model building directly from data with a minimal or no human supervision is already absolutely crucial in order to stay competitive and maximally exploit the data in quickly changing business environments. However, the methodology for ensuring that created models (i.e. classiÿers, predictors, etc.) are as good as possible should be in place before using them with conÿdence.. No human supervision in model building also implies that one should use powerful enough techniques which can learn the data to any degree of accuracy. There are currently a lot of methods from soft computing, machine learning, and statistics domains which, in principal, satisfy this requirement. In the pattern recognition domain the examples include the nearest-neighbour classiÿers [19, 21] , decision trees [19] , neural networks with su cient number of hidden nodes [4, 14, 19] , fuzzy if-then rules systems which are built directly from data [1] [2] [3] [16] [17] [18] , neuro-fuzzy techniques based on hyperbox fuzzy sets [8] [9] [10] [11] [12] [13] , Bayesian networks or logical rule bases. From the statistical point of view most of these methods could be classiÿed as non-parametric models. The main challenge in such cases is to design a model building strategy which would guard against overÿtting of the training data or, in other words, would lead to a good generalisation performance.. Over the last 10 years, there has been a great amount of interest in the combination of the learning capability and computational e ciency of neural networks with the fuzzy sets ability to cope with uncertain or ambiguous data. This has led to a development of various hybrid neuro-fuzzy techniques [1] [2] [3] [16] [17] [18] , including a general fuzzy min-max (GFMM) neural network for clustering and classiÿcation [8] [9] [10] [11] [12] [13] . The development of the GFMM originated from our investigation into uncertain information processing in the context of the decision support for the operational control of industrial processes. This generic pattern recognition method based on hyperbox fuzzy sets combines supervised and unsupervised learning within a single learning algorithm, can grow to meet the demands of the problem, has the ability to incorporate new information without a need for complete retraining, learns on-line and has the ability to process inputs in the form of real (conÿdence) intervals. It has been successfully applied to a very challenging problem of leakage detection and identiÿcation in water distribution systems where a hierarchical system based on the GFMM has been used [12] .. Since proposing the original GFMM neural network a number of extensions have been proposed in an attempt to produce a exible pattern recognition framework which could accommodate various problems when generating models directly from high-dimensional real-world data. These extensions included a development of agglomerative learning algorithms for GFMM resulting in a generation of hierarchical structures [11] , various approaches to dealing with missing data [10] and the use of statistical resampling techniques in the process of generating classiÿers with good generalisation performance through: data editing techniques [8] , combining multiple copies of the GFMM classiÿer at the decision and model levels [9] , and estimating the parameters controlling the complexity of the ÿnal GFMM classiÿer [11] .. Though commonly the main objective of the classiÿer design process is constructing a classiÿer with as good a performance as possible, in many pattern classiÿcation applications there are some additional aspects which are regarded as equally important. For instance, it may be a part of the requirements that the classiÿer model is transparent and has the ability to provide an easily interpretable explanation of the suggested classiÿcation decision to a non-technical user. On the other hand, there may be applications where the speed of generation of the model is of primary concern or there are restrictions on the size of the classiÿcation model that can be stored. Yet another example could be an application where due to the non-stationary environment the emphasis should be put on the potential adaptability of the classiÿer model while in operation.. Bearing this in mind, in this paper we will concentrate on the analysis of various algorithm independent classiÿer model generation approaches for designing a GFMM classiÿer [8] [9] [10] [11] [12] [13] or a GFMM-based multiple classiÿer system. Various model generation approaches, which could be used together with the base GFMM learning algorithms, will be assessed and discussed in the context of the following four criteria: (a) the ability to estimate the performance of the model on unseen data; (b) the generated model's power to explain the suggested decisions which can be interpreted by the user; (c) the computational complexity involved in the classiÿer model building process; and (d) the potential for adaptation of the classiÿer model to changing environments and new classes of data.. Speciÿcally, the above criteria will be applied to ÿve GFMM classiÿer model building schemes including generating classiÿers on the basis of the full training data set, using a k-fold and multiple 2-fold cross-validation with various pruning approaches and combining multiple copies of the GFMM classiÿer.. Though in terms of pure classiÿcation performance the ensemble=combination methods [5] [6] [7] 9, 20] have been frequently shown to o er high classiÿcation performance gains in comparison to individual classiÿers, when they are considered in the context of other criteria like the ones mentioned earlier the choice of the classiÿer generation scheme is no longer so clear. In this sense one of the main questions investigated will be that of whether to use a single model or a combination of a number of components forming the ÿnal classiÿer. As it will be illustrated each of the discussed model generation approaches has some advantages and disadvantages which make them more suitable for certain applications.. The remaining of this paper is organised as follows. The next section presents a summary of the GFMM neural network with deÿnitions of hyperbox fuzzy sets and associated fuzzy membership function. It also provides a description of the base learning algorithms which can be used to place and adjust hyperboxes in the input space. In the following section di erent classiÿer generation schemes utilising statistical resampling techniques are discussed. This is followed by some simulation results illustrating their properties. And ÿnally, conclusions are presented in the ÿnal section. Five di erent GFMM classiÿer model generation approaches have been presented and discussed. Depending on a classiÿcation problem each of the approaches can be shown to have some advantages over another. If the speed of model generation and quick adaptability to the changing environment are of primary concern then the base learning algorithm can be used without any hyperbox pruning procedures. On the other hand, if the model building can be carried out o -line, the approaches based on multiple cross-validation either for estimating parameters controlling the complexity of the model or for building an ensemble of classiÿers are likely to provide a better classiÿcation performance and the true error estimation. If the explanation of the classiÿcation decisions is to be provided, all the model generation schemes resulting in a single GFMM model can be used. This is with the exception of the ensemble of GFMM classiÿers in which case the single GFMM model transparency is lost. While currently the advantages of the resampling techniques can be fully realised only during the initial model building process, an evolving validation set accounting for new data could be used for periodical model updating and hyperbox pruning. The frequency of such model validation would be dependent on the dynamics of a speciÿc pattern classiÿcation problem.

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Spatial stress and strain distributions of viscoelastic layers in oscillatory shear One of the standard experimental probes of a viscoelastic material is to measure the response of a layer trapped between parallel surfaces, imposing either periodic stress or strain at one boundary and measuring the other. The relative phase between stress and strain yields solid-like and liquid-like properties, called the storage and loss moduli, respectively, which are then captured over a range of imposed frequencies. Rarely are the full spatial distributions of shear and normal stresses considered, primarily because they cannot be measured except at boundaries and the information was not deemed of particular interest in theoretical studies. Likewise, strain distributions throughout the layer were traditionally ignored except in a classical protocol of Ferry, Adler and Sawyer, based on snapshots of standing shear waves. Recent investigations of thin lung mucus layers exposed to oscillatory stress (breathing) and strain (coordinated cilia), however, suggest that the wide range of healthy conditions and environmental or disease assaults lead to conditions that are quite disparate from the "surface loading" and "gap loading" conditions that characterize classical rheometers. In this article, we extend our previous linear and nonlinear models of boundary stresses in controlled oscillatory strain to the entire layer. To illustrate non-intuitive heterogeneous responses, we characterize experimental conditions and material parameter ranges where the maximum stresses migrate into the channel interior. We first recall the formulation developed in [7, 11, 12] , which is a generalization of the Ferry shear wave model [4] [5] [6] (also considered by Schrag et al. [15, 16] for linear viscoelastic experiments) to finite depth layers and nonlinear constitutive laws. We briefly summarize the key elements from these references in order to describe the present focus on heterogeneous stress distributions in a viscoelastic layer being driven by oscillatory shear strain or stress. The equations of motion for an incompressible fluid are, 1. where T is the total stress tensor, v is the fluid velocity, and ρ is the fluid density. The total stress tensor is decomposed as T = − pI + τ where p is the pressure [2] and τ incorporates both the viscous stress and "extra stresses" arising from elastic properties of the material of interest. Our baseline model is the upper convected Maxwell (UCM) model (Eqs. (3) and (4) with a = 0), which possesses the nonlinearity common to all nonlinear differential constitutive models, and which reduces in the linear limit to a simple viscoelastic fluid with a viscous parameter (η 0 ) and a relaxation time (λ 0 ). We consider a single stress relaxation mode here, but any finite sum can be similarly studied. We will also consider the Giesekus model which adds a quadratic nonlinearity (in the stress tensor) to the UCM model:. where the scalar parameter a is the Giesekus mobility parameter, D is the rate-of-strain tensor, D = 1/2(∇ v + ∇ v T ), and the symbol over τ is the upper convected derivative [1, 10] : We have explored the response of a viscoelastic layer in oscillatory shear, focusing on the spatial distribution of shear and normal stresses in the layer. We use the upper convected Maxwell model and the Giesekus model to explore nonlinear contributions to an underlying linear viscoelastic stress response structure in the layer. The linear stress response structure is exactly solvable in closed form, and the shear stress formula explicitly reveals non-monotone distributions of stress in the shear gap, as well as parameter regimes where the maximum shear stress arises in the interior of the layer. The upper convective derivative, for the frequency-locked response, has the identical velocity and shear stress distributions, coupled with an explicit normal stress distribution suppressed by the assumption of linearity. Finally, the Giesekus quadratic stress nonlinearity is shown by numerical simulations of a coupled system of four quasi-linear partial differential equations to modify the analytical features of the UCM model, while preserving the fundamental features of non-monotonicity of stress envelopes in the shear gap and the possibility of maximum stresses arising in the layer interior. Such behavior implies that localized shear thinning of viscoelastic layers in oscillatory strain driving conditions can arise anywhere in the layer, and in particular not always at the driven interface. The implications of these phenomena for transport of mucus layers in lung pathways is a subject of future interest.

O O O O O O B O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O B I I O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A tour of tempo: a program specializer for the C language Specialization is an automatic approach to customizing a program with respect to configuration values. In this paper, we present a survey of Tempo, a specializer for the C language. Tempo offers specialization at both compile time and run time, and both program and data specialization. To control the specialization process, Tempo provides the program developer with a declarative language to describe specialization opportunities for a given program. ::: ::: The functionalities and features of Tempo have been driven by the needs of practical applications. Tempo has been successfully applied to a variety of realistic programs in areas such as operating systems and networking. We give an overview of the design of Tempo and of its use in specializing realistic applications.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B


A Programming Perspective of the Membrane Systems We present an operational semantics of the membrane systems, using an appropriate notion of configurations and sets of inference rules corresponding to the three stages of an evolution step in membrane systems: maximal parallel rewriting step, parallel communication of objects through membranes, and parallel membrane dissolving. We define various arithmetical operations over multisets in the framework of membrane systems, indicating their complexity and presenting the membrane systems which implement the arithmetic operations. Finally we discuss and compare various sequential and parallel software simulators of the membrane systems, emphasizing their specific features. Keywords: membrane systems, operational semantics, arithmetical operations over multisets.Membrane systems represent a computational model inspired by cell compartments and molecular membranes. Essentially, such a system is composed of various compartments, each compartment with a different task, and all of them working simultaneously to accomplish a more general task of the whole system. A detailed description of the membrane systems (also called P systems) can be found in [17] . A membrane system consists of a hierarchy of membranes that do not intersect, with a distinguishable membrane, called the skin membrane, surrounding them all. The membranes produce a delimitation between regions. For each membrane there is a unique associated region. Regions contain multisets of objects, evolution rules and possibly other membranes. Only rules in a region delimited by a membrane act on the objects in that region. The multiset of objects from a region corresponds to the "chemicals swimming in the solution in the cell compartment", while the rules correspond to the "chemical reactions possible in the same compartment". Graphically, a membrane structure is represented by a Venn diagram in which two sets can be either disjoint, or one is a subset of the other. More details (concepts, results) and several variants of membrane systems are presented in [17] .A P system consists of several membranes that do not intersect, and a skin membrane, surrounding them all. The membranes delimit regions, and contain multisets of objects, as well as evolution rules. Each membrane has a unique associated region. The space outside the skin membrane is called the outer region (or the environment). Because of the one-to-one correspondence between the membranes and the regions, we usually use the word membrane instead of region. Only rules in a region delimited by a membrane act on the objects in that region. Moreover, the rules must contain target indications, specifying the membrane where objects are sent after applying the rule. The objects can either remain in the same region, or pass through membranes in two directions: they can be sent out of the membrane which delimits a region from outside, or can be sent in one of the membranes which delimit a region from inside, precisely identified by its label. The membranes can also be dissolved. When such an action takes place, all the objects of the dissolved membrane remain free in the membrane placed immediately outside, but the evolution rules of the dissolved membranes are lost. The skin membrane is never dissolved. The application of evolution rules is done in parallel, and it is eventually regulated by priority relationships between rules. A P system of degree m is a structure Π = (O, µ, w 1 , . . . , w m , (R 1 , ρ 1 ) ,. . . , (R m , ρ m ), i o ), where:(i) O is an alphabet of objects, and µ is a membrane structure;

O O O O O B I O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I I I O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Effect of team diversity on software project performance Purpose -This research seeks to investigate the relationship between knowledge diversity (KD) in software teams and project performance. Previous research has shown that member diversity affects team performance; most of that work, however, has focused on diversity in personal or social attributes, such as gender or social category. Current research targets at the knowledge level aim to facilitate the implementation of knowledge management in organizations. Design/methodology/approach -A research framework was developed based on conflict theory and empirically tested on software teams in Taiwan. Findings -It was found that KD increases task conflict, which in turn has significant positive effects on team performance and that value diversity (VD) increases relationship conflict, which in turn negatively affects team performance. Research limitations/implications -The findings indicate that task conflict can enhance team performance, while relationship conflict can reduce team performance. Therefore, it is important to maintain healthy relationships among team members. Practical implications -This research concludes that KD is beneficial and that VD is harmful to project outcome in software development. It is, therefore, useful for managers to form teams whose members encompass a broad knowledge base. Originality/value -This paper proposes a novel way to measure knowledge and VD in teams and reports the effects of these attributes on team performance. The work also shows that a proper level of task conflict in a software team is necessary for achieving high performance. The performance of software development teams is an important topic in the information systems (IS) domain. As evident by Moore's law, the information industry has prospered greatly due to rapid price reductions for computer hardware (Kelly, 1998) . The enhancement and advancement of function also has played an important role in expediting this progress. However, the success rate of software projects is much lower than desired. For example, 18 percent of all projects fail and 53 percent are challenged (Standish Group, 2004) . These projects were completed years behind schedule, exceeded their budget by millions of dollars, and failed to meet their users' needs even if completed. Thus, software development is a high-risk enterprise (Karlsen et al., 2006; Faisal et al., 2006) . A long-standing key question that has intrigued the minds of researchers concerns the problem in managing software projects efficiently while promoting team performance (Robey and Smith, 1993) .. In search of factors for successful team performance, researchers have examined various personality characteristics of team members. As software development is a labor-and knowledge-intensive task, teamwork in software projects has been long acknowledged as a crucial criterion for the successful design and deployment of software projects (Jiang et al., 2003; Gottschalk and Solli-Sather, 2007) . Every software project will inevitably face the issue of team composition. It has interested researchers whether bringing diversity in team composition would promote successful teamwork and further lead a project towards fulfillment of its mission, vision, and values. To quantify successful team performance, researchers have studied the personality characteristics, interpersonal relationships and interactions among team members (Barki and Hartwick, 2001; Gottschalk and Karlsen, 2005) . Previous research, however, has not shown any major consistent effects of team member diversity on work performance. Byrne's (1971) similarity-attraction theory suggested that similarity in interaction, value, and demographics are favored virtues in team composition as they help maintain effective work environments. In contrast, some diversity theorists and group researchers (Cox et al., 1991; Jehn, 1995; Guzzo and Dickson, 1996) have proclaimed the benefits of diversity in workgroups. Current findings about the effect of diversity on team performance are mixed (Williams and O'Reilly, 1998) . Therefore, further research is required to identify the factors underlying the relationship between team diversity and software team performance.. In this study, we derived and tested a new theoretical model to explore what and how diversity affects the performance of software development teams. More specific, we address the following research questions in this paper:. RQ1. What kinds of relationships exist between the composition of software teams and performance?. RQ2. How does team diversity affect the performance of software development projects?. This research provides a model to explain the effect of diversity on software team performance and an empirical study to test this model. We report two major findings. First, knowledge diversity (KD) significantly increases task conflict, and task conflict positively affects team performance. Project leaders can leverage the knowledge differences of members in order to achieve higher performance. Second, VD increases relationship conflict, which negatively affects performance. Hence, the diversity of values among team members should be minimized. Moreover, interpersonal relationships must be managed carefully in situations where team members have very different values. These findings will help decision makers manage software projects by selecting appropriate team members and effectively managing diversity in workgroups for project success. In conclusion, this paper contributes to software team composition by increasing our theoretical and empirical understanding of how team diversity affects its performance. We have presented a framework to exploring the relationship among diversity, conflict, and performance. We differentiated three types of diversity and demonstrated their respective impact on software team performance. The results show that different forms of diversity give rise to different forms of conflict, which in turn affect the perceived team performance. KD positively affects team performance through its impact on task and relationship conflicts. VD may harm team performance through relationship conflict. SD affects both task conflict and relationship conflict, but in opposite directions, to generate a mixed effect on team performance. The results of this study provide direction for creating and managing diverse teams to enhance team performance. Project leaders can enhance team performance by leveraging the knowledge differences of members and by managing inter-group conflicts carefully if the team members have very different values. Our study helps promote effective management of diversity and conflicts in workgroups and delineates the critical importance of these two factors for project success. The purpose of this study was to explore the differential impact of three team compositional factors (KD, SD, and VD) and two intervening variables (task conflict and relationship conflict) on software team performance. We have developed and empirically assessed our model. Major empirical findings and their possible interpretations are discussed below.. Finding 1. KD affects team performance through the effect of both task and relationship conflicts. As indicated by the significant positive paths from KD to task conflict, and from task conflict to team performance in Figure 2 , our results indicate that software teams with higher KD will engender more task conflict and that teams with more task conflict are more likely to produce better team performance. A second path by way of relationship conflict also indicates that an increase in KD will reduce relationship conflicts, which will, in turn, increase team performance. Both paths suggest that managers of a software team may want to assemble a team whose members have diverse knowledge backgrounds to stimulate a higher degree of overall performance. This finding is consistent with the observation of Kankanhalli et al. (2007) .. Finding 2. The relation between SD and performance is mixed.. The effect of SD on team performance is mixed, which is in agreement with the previous study of Jehn (1999) . Our findings show that SD positively influences both task conflict and relationship conflict. A possible explanation is that the indicators used to measure social category may have a mixed effect. For example, age as a variable of SD may also correlate with the tenure of members. As shown previously, tenure can lead to task conflict (Jehn, 1995) and relationship conflict (Jehn et al., 1997) . Furthermore, Ely (1994) argued that the proportion of males and females is the key effect of gender on performance. These measurement problems may cause the mixed effect we see for SD on team performance.. Finding 3. VD negatively affects software team performance.. VD among team members will increase relationship conflict, which negatively affects team performance. Hence, VD negatively affects team performance. This implies that a project manager should carefully choose members with similar value judgment.. Finding 4. KD may reduce the relationship conflict among team members.. While most of our hypotheses received support, we did have an unexpected finding. That is, the impact of KD on relationship conflict is negative, a denial of our positive hypothesis. A possible explanation is that team members with heterogeneous educational backgrounds may have better information exchange and communication relationships due to different viewpoints (Nonaka and Takeuchi, 1995) , which could facilitate cooperation among team members.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Detecting Phenotype-Specific Interactions between Biological Processes from Microarray Data and Annotations Abstract-High throughput technologies enable researchers to measure expression levels on a genomic scale. However, the correct and efficient biological interpretation of such voluminous data remains a challenging problem. Many tools have been developed for the analysis of GO terms that are over-or under-represented in a list of differentially expressed genes. However, a previously unexplored aspect is the identification of changes in the way various biological processes interact in a given condition with respect to a reference. Here, we present a novel approach that aims at identifying such interactions between biological processes that are significantly different in a given phenotype with respect to normal. The proposed technique uses vector-space representation, SVD-based dimensionality reduction, differential weighting, and bootstrapping to asses the significance of the interactions under the multiple and complex dependencies expected between the biological processes. We illustrate our approach on two real data sets involving breast and lung cancer. More than 88 percent of the interactions found by our approach were deemed to be correct by an extensive manual review of literature. An interesting subset of such interactions is discussed in detail and shown to have the potential to open new avenues for research in lung and breast cancer. H IGH-THROUGHPUT technologies such as microarrays allow researchers to measure the expression of thousands of genes simultaneously [1] , [2] . Researchers, however, still face the challenge of translating such expression measurements into a better understanding of the underlying biological phenomenon. Existing annotations and gene ontologies such as GO [3] are essential in this step. An approach based on the identification of the biological processes that are significantly over-or underrepresented in the set of DE genes is widely used for this purpose [4] , [5] , [6] . As of 2005, there were at least 15 such tools focusing on this type of ontological analysis [7] . Since then, even more tools have been developed. A more recent review [8] included not fewer than 68 tools in this area. Efforts have been made to improve this approach in various ways such as by including the entire list of DE genes rather than a subset of DE genes (e.g., GSEA [9] and related), taking into consideration the structure of the GO DAG [10] , [11] , taking into consideration sets of GO terms at a time [12] , etc. A previously unexplored aspect is the identification of those relationships between biological processes that are significantly perturbed in a given condition. Here, we will present a first attempt toward this goal.. The technique described here aims to identify nontrivial relationships between biological processes represented by GO terms that are statistically significant in a given condition. The primary goal of this technique is to suggest novel hypotheses and directions for further exploration of the condition under study. A nontrivial relationship is a relationship that cannot be inferred from the directed acyclic graph (DAG) structure of the GO. Any relationships that can be directly inferred from the GO structure (e.g., a relationship between a given GO term and any of its distant ancestors or descendants) are not considered specific to any given condition and hence, are discarded by our technique.. A relationship between two biological processes, as predicted by our method, can also: 1) reveal a biological interdependence or interaction among GO terms that cannot be inferred easily from existing literature and that may be specific to the given condition; 2) suggest novel gene networks or possible interconnections in a gene network formed by the genes annotated with predicted associated terms; and 3) be used, in further studies, to predict novel gene annotation when one of the functions in the relationship is already known to be annotated with a gene and the other is not. As such, these functional relationships have the potential to broaden the knowledge base of research and enhance our understanding of biological phenomenon under study. In this paper, we presented a method that analyzes expression data obtained from a microarray experiment together with all existing gene annotations to find statistically significant interactions between specific biological processes involved in the given biological condition. Applying the proposed method to two independent data sets yielded a number of interesting statistically significant interactions specific to the given conditions. Most of these interactions are well supported by independent studies from the literature. For the purpose of evaluation, we manually surveyed the literature and assigned each predicted biological processes pair to one of the three categories: 1) supported by literature and considered trivial, 2) supported by literature and considered nontrivial, and 3) yet unknown. Predicted pairs assigned to categories 1 and 2 were considered correct or true positive, while the pairs assigned to category 3 were assumed to be false positive. More than 88 percent of the associations predicted by any of the four schemes were found to be correct and valid. A subset of such interactions was discussed in details and shown to have the potential to open a number of new avenues for research in lung and breast cancer. Overrepresentation studies, in general, ignore the biological fact that most complex functions in cells are carried out through the concerted activities of many gene products. Current vocabularies such as that provided by GO are classified mostly based on individual gene functions as opposed to the functionality of a set of interacting genes. Antonov and Mewes proposed a method to find overrepresentation of complex functions annotated to a set of genes of interest, wherein a gene may belong simultaneously to two or more functional categories [12] , [68] . They defined the complex functional categories as algebraic combination of base categories, such as union, intersection, or difference of the gene sets corresponding to the base categories. Nam et al. analyzed the differentially expressed gene sets using composite GO annotations using a parametric (Z-) test [69] .. A recently developed tool, ToppCluster [70] , finds association between functional categories by computing the significance scores from the p-values of overrepresented categories. Euclidean-based distance between significance scores is used to perform a 2D hierarchical clustering and to produce either a heatmap or a Cytoscape-based network, which is then used to find association between the functional categories or annotated genes. Another tool, GENECODIS [71] , finds significant concurrent annotations in gene lists by finding the significance of co-occurrence of annotations with at least three genes. The hypergeometric and chi-squared tests are employed to compute the significance level of their prediction. Unlike ToppCluster, our proposed method does not rely on other overrepresentation techniques that are based on single annotations, and unlike GENECODIS, our technique can find significant functional categories with no common input gene. In addition, these studies find groups of functional categories associated with some list of differentially expressed genes, while the method described here finds biological process pairs, whose association has changed in the studied phenomenon.. The novelty of our approach can be demonstrated by the following example. Suppose, a DE gene g a is annotated with a biological process A and, under the same biological condition, another DE gene g b is annotated with a biological process B. The biological process A might have been shown to interact in a specific condition, in another research, with the biological process B. However, current analysis tools may fail to provide such information due to the nature and/or structure of the ontology and the database. As such, the current methods and techniques, oblivious of any interaction between biological processes, may only provide the two GO categories as over-or underrepresented, individually or collectively as a complex category and not reveal further insight about the nature or strength of association among functions in the complex category.

O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Deno: A Decentralized, Peer-to-Peer Object-Replication System for Weakly Connected Environments Abstract-This paper presents the design, implementation, and evaluation of the replication framework of Deno, a decentralized, peer-to-peer object-replication system targeted for weakly connected environments. Deno uses weighted voting for availability and pair-wise, epidemic information flow for flexibility. This combination allows the protocols to operate with less than full connectivity, to easily adapt to changes in group membership, and to make few assumptions about the underlying network topology. We present two versions of Deno's protocol that differ in the consistency levels they support. We also propose security extensions to handle a class of malicious actions that involve misrepresentation of protocol information. Deno has been implemented and runs on top of Linux and Win32 platforms. We use the Deno prototype to characterize the performance of the Deno protocols and extensions. Our study reveals several interesting results that provide fundamental insight into the benefits of decentralization and the mechanics of epidemic protocols. T HIS paper describes the design, implementation, and performance of Deno [10] , [13] , [14] , [25] , a system that support object replication in a transactional framework for weakly connected environments. Deno's system model is illustrated in Fig. 1 . One or more clients connect to each peer server, which communicates through pair-wise information exchanges. The servers are not necessarily ever fully connected. Deno's application domain include asynchronous groupware applications (e.g., Lotus Notes [24] ), traditional file and database systems, and distributed middleware systems that require asynchronous consensus support.. Deno's underlying protocols are based on an asynchronous protocol called bounded weighted voting [25] . Asynchronous solutions for managing replicated data [7] , [21] , [24] , [26] have a number of advantages over traditional synchronous replication protocols in large-scale and weakly connected environments. They can operate with less than full connectivity, easily adapt to frequent changes in group membership, and make few demands on the underlying network topology. However, this functionality comes at a price: Asynchronous solutions are generally either slow or require reconciliation or have low availability because they rely on primary-copy schemes [33] .. The protocols retain the advantages of current asynchronous protocols, but generally perform better, have fewer connectivity requirements, and achieve higher availability. No server ever needs to have complete knowledge of group membership and a given server only needs to be in intermittent contact with at least one other server to take full part in the voting and commitment process. As such, the protocol is highly suited for environments with weak connectivity.. The protocol's strengths result from a combination of weighted voting and epidemic information flow [17] , a process where information flows pair-wise through the system (much like a disease passing from one host to the next). The protocol is completely decentralized. There is no primary server that owns an item or serializes the updates to that item (as in Bayou [34] ). Any server can create new object replicas and servers need only be able to communicate with a minimum of one other server at a time in order to make progress. Instead of synchronously assembling quorums, which has been extensively addressed by previous work (e.g., [20] , [23] , [35] ), votes are cast and disseminated among servers asynchronously through pairwise propagation. Servers commit or abort transactions locally and all servers eventually reach the same decisions.. The use of voting allows the system to have higher availability than primary-copy protocols [1] , [4] , [20] , [30] , [35] . The use of weighted voting allows implementations to improve performance by adapting currency (a.k.a. weight) distributions to site availabilities, update activity, or other relevant characteristics [9] . Each server has a specific amount of currency and the total currency in the system is fixed at a known value. The advantage of a static total is that servers can determine when a plurality or majority of the votes has been accumulated without complete knowledge of group membership. This last attribute is key in dynamic, wide-area environments because it allows the protocol to operate in a completely decentralized fashion, eliminating performance bottlenecks and single points of failure.. The use of epidemic protocols divorces protocol requirements from communication requirements. First, an epidemic algorithm only requires protocol information to move throughout the system eventually. The lack of hard deadlines and connectivity requirements is ideally suited to weakly connected environments, where individual nodes are routinely disconnected. Second, epidemic protocols remove reliance on network topology. Synchronization partners in epidemic protocols can be chosen randomly, eliminating the single point of failures that occur with more structured communication patterns such as spanning trees.. The key contributions of this paper can be summarized as follows: First, we present decentralized, peer-to-peer replication protocols that combine weighted voting and asynchronous, epidemic information flow. This combination enables our protocols to operate with less than full connectivity, easily adapt to changes in group membership, and make few assumptions about the underlying network topology. We describe in detail the consistency levels provided by the protocols and outline formal correctness proofs.. Second, we discuss an extension of our replication protocols to provide security against a specific class of internal security threats that involve misrepresentation of protocol-specific voting information. To handle such threats, we propose a vote validation technique that requires cryptographic primitives and modifications to the update commit criteria. A unique aspect of our solution is that it not only enables a trade off between performance and the degree of tolerance to malicious servers, but also allows for individual servers to support nonuniform degrees of tolerance without adversely affecting the performance of the rest of the system.. Third, we present the design of the Deno prototype system that implements the proposed protocols. The basic Deno architecture has been implemented and runs on top of Linux and Win32 platforms. We conduct a detailed performance study of our protocols and other comparable epidemic protocols that appeared in the literature using the Deno prototype. Our study provides fundamental insight into the benefits of decentralization and mechanics of epidemic protocols. One particularly interesting result of our evaluation is that the presumed performance advantage of the centralized approach over a decentralized voting approach is not significant with asynchronous, epidemic information flow.. The rest of this paper is structured as follows: Section 2 describes the basic Deno system model and the decentralized replication protocols employed by Deno in detail. Section 3 briefly addresses security issues and describes a security extension based on vote validation. Section 4 summarizes other Deno features that promote weaklyconnected operation. Section 5 presents our prototypebased performance study that characterizes the performance of the replication protocols and their extensions as well as other competitive epidemic protocols. Section 6 briefly describes related work and Section 7 summarizes and concludes the paper. Finally, the Appendix provides correctness proofs for the proposed Deno protocols as well as pseudocode for basic Deno operations. We presented the design, implementation, and evaluation of Deno, a highly available object-replication system designed for weakly connected environments. Deno's consistency protocols are based on an asynchronous weighted-voting approach implemented through epidemic information flow. Our voting approach achieves higher availability than primary-copy approaches [34] , and higher availability and performance than ROWA approaches [3] .. Our base protocol ensures weakly consistent executions where update transactions are serializable and queries always access transactionally consistent database states. Our extended protocol provides strong consistency and globally serializable executions by providing a unique global commit order on all update transactions. Both protocols allow queries to be executed and committed entirely locally and without blocking. Furthermore, neither protocol suffers from local or global deadlocks.. We also classified and addressed a specific class of internal security attacks that involve vote misrepresentation. We proposed a secure version of our protocols that handles such malicious actions using a combination of cryptographic primitives, modifications to the update commit criteria, and explicit validation of votes. The proposed protocol is parameterized and can be tuned to trade off degrees of tolerance to malicious insiders and commit performance, thereby allowing individual servers to set arbitrary degrees of tolerance based on their individual requirements and resources.. Our performance study based on the Deno prototype revealed several interesting results. First, the presumed performance advantage of the primary-copy approach over a uniform voting approach is not as significant with asynchronous epidemic protocols. The reason is that epidemic voting protocols allow servers to independently arrive at the same conclusions, whereas primary-copy schemes require all commit information to emanate from a single, distinguished server. Second, our extended protocol performs nearly as well as the base protocol, while providing significantly stronger semantics. The result is increased functionality at essentially little cost in performance. Third, protecting against internal threats comes at a cost, but the marginal cost for protecting against larger cliques of malicious insiders is generally low. Finally, speculative update propagation and voting provides a considerable performance advantage for protocols that use pair-wise communication and this advantage is magnified when application-specific commutativity information is used to decrease the rate of transaction aborts. The problem of consistent access to replicated data has long been studied in many contexts and a wide variety of solutions have been proposed (e.g., [2] , [5] , [16] , [19] , [33] , [35] ). Because of the intrinsic shortcomings of traditional synchronous replication solutions [20] , [33] , [35] in mobile and weakly connected environments [21] , asynchronous replication protocols have recently gained a lot of attention (e.g., [7] , [24] , [26] ). Asynchronous approaches commonly allow servers to execute updates locally without any synchronization with other servers and propagating updates afterward as separate activities. Due to space limitations, we restrict our attention to asynchronous update-anywhere approaches that utilize the epidemic model [3] , [17] , [24] , [29] , [34] .. Many epidemic systems take an optimistic approach and use reconciliation-based protocols (e.g., Ficus [29] , Lotus Notes [24] ) that are only viable in nontransactional singleitem domains such as file systems. These approaches only ensure that all copies of a single item eventually converge to the same value and therefore are not safe for environments requiring transactional semantics.. Bayou [34] takes a more pessimistic approach and ensures that all committed updates are serialized in the same order at all servers using a primary-copy scheme. More recently, Agrawal et al. [3] described a pessimistic ROWA [5] approach that ensures strong consistency and serializability. Our protocols differ from these protocols primarily in using a novel combination of weighted-voting and epidemic information flow to improve availability and performance.. Holliday et al. [22] also proposed an epidemic quorumbased approach that provides serializability as our extended protocol. Holliday's work assumes a more traditional replicated database environment and static currencies, whereas our emphasis is on making progress under incomplete system information in dynamic environments.. Despite the growing need for secure protocols for managing replicated data, this topic is yet to be addressed for the decentralized, asynchronous environments that we target. Security research in synchronous centralized systems mostly targeted group communication protocols. Ensemble [32] addresses only external security threats providing secure authentication, integrity, and privacy, whereas Rampant [31] is designed to handle Byzantine attacks. These systems are commonly based on primary-copy models to coordinate replica management and require much stronger connectivity and reliable multicast primitives. Secure election protocols (e.g., [15] , [18] ) provide voter privacy and rely on a small number of central facilities for counting votes, but are impractical under weak-connectivity as restrictions in connectivity and disconnections make reliance on central authorities untenable.. Castro and Liskov [8] described a practical replication algorithm for tolerating Byzantine attacks in asynchronous environments where update commitment is centralized and coordinated via a primary-copy server. To the best of our knowledge, the only proposal that addresses system security using asynchronous epidemic information flow is that of Malkhi et al. [27] . Malkhi et al. provided an analytical treatment of epidemic-style update diffusion algorithms that are tolerant of Byzantine faults. This work makes strong assumptions about the number of malicious servers and where and how updates are created and initially received. No previous work, including Malkhi et al.'s [27] , has addressed update commitment in decentralized, epidemic systems and databases.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Reverted indexing for feedback and expansion Traditional interactive information retrieval systems function by creating inverted lists, or term indexes. For every term in the vocabulary, a list is created that contains the documents in which that term occurs and its relative frequency within each document. Retrieval algorithms then use these term frequencies alongside other collection statistics to identify the matching documents for a query. In this paper, we turn the process around: instead of indexing documents, we index query result sets. First, queries are run through a chosen retrieval system. For each query, the resulting document IDs are treated as terms and the score or rank of the document is used as the frequency statistic. An index of documents retrieved by basis queries is created. We call this index a reverted index. With reverted indexes, standard retrieval algorithms can retrieve the matching queries (as results) for a set of documents (used as queries). These recovered queries can then be used to identify additional documents, or to aid the user in query formulation, selection, and feedback. In ad hoc information retrieval, users describe their information needs by queries to search a document collection. Query terms are often used with an inverted index to rank documents by estimated relevance. In query expansion based on relevance feedback, users explicitly identify relevant documents to help refine a search iteratively. In this paper, we present and evaluate a general approach to indexing a collection for exploration using document-based queries, which we call Reverted Indexing. Reverted indexing is a variant of inverted indexing founded on the retrievability of documents by queries [5] . While traditional inverted indexing associates terms with documents in which they occur, reverted indexing relates documents with the queries that retrieve them.. To build the reverted index, we first assemble a large set of queries, and refer to its elements as basis queries to distinguish them from the user queries that represent users' information needs. Basis queries can be compiled from keywords extracted from query logs or from the documents comprising the collection, or by using other conjunctive mechanisms to form more complex queries (e.g. using metadata or facets). Once a set of basis queries is determined, we use a standard ranking function to order documents by their relevance to each basis query. For each document we construct the vector with elements corresponding to the basis queries and values determined by that document's estimated relevance to that basis query. This reverted index thus associates each document with the basis queries that retrieve it, as in Figure 1 (b). This processing is performed off-line and does not require substantial computation for simple ranking functions.. The process is analogous to standard term-based inverted indexing. A conventional inverted indexes is a set of lists, one for each term. A given term's list contains elements for each document in which that term occurs, as in Figure 1 (a). Each element is weighted according to the term's importance within the corresponding document.. In sum, the basic conceptual foundation for conventional inverted indexes is term occurrence, while reverted indexes are founded on document retrievability. In the next section, we review related representations for document collections. We then describe our indexing scheme in detail and present query expansion experiments that demonstrate improved performance over well-regarded methods. We also show that our method reduces online computational costs, and conclude the paper with a discussion of our results and potential future extensions. This paper introduces a framework for ad hoc retrieval of basis queries (as results) using document-ids (as queries). A basis query set and a base retrieval function combine to construct the reverted index. By choosing specific reverted retrieval functions and a reverted query language, we use the reverted index for retrieval focusing on the problem of automated query expansion. There are two major directions for future work. One is to invent better methods for constructing and querying the reverted index; the other is to create applications and interactions for the ad hoc retrieved basis queries.. For reverted index construction, extensions include automatically extracting bi-grams and longer n-grams, adding facets and other metadata to the basis queries, and incorporating effective queries mined from large scale search logs. When creating the reverted index using base retrieval functions, one is not limited to traditional IR models; any function that can produce a ranking can be used. For example, to bring this approach more in line with Craswell et al. [9] , one could use relative click rates as a ranking function. A reverted index would then allow users to construct ad hoc multi-docid queries to retrieve the most likely basis queries to have been responsible for producing clicks on those multiple documents.. Another interesting idea is to construct the index by running the same set of basis queries across multiple retrieval algorithms or (web) search engines, and storing that algorithmic choice as part of the basis query. When doing a reverted query, one would then be able to retrieve or discover not only the best basis queries, but also the algorithm or engine to use to produce those results. Integration of multiple media types (images and image queries, music and music queries) is also possible; a reverted song ID query could retrieve textual, audio, and image basis queries.. For reverted index querying, one interesting approach relates to synonyms. If a user's information need can be decomposed into multiple aspects, then relevant documents identified as belonging to each aspect could be treated as synonyms. For example, if dw and dx are relevant to one aspect, and dy and dz are relevant to another, the reverted query could be [syn(dw dx) syn(dy dz)]. This conflation has an effect on global and local statistics and could produce better results. Boolean constructs for integrating non-relevant documents may be useful as well, e.g. if dw and dy are rel-evant, and dx and dz are non-relevant, a possible ad hoc reverted query might be: [(dw ∧ ¬ dx) ∨ (dy ∧ ¬ dz)]. In this paper, we described reverted indexes as a representation for document collections. Using the retrievability of documents, reverted indexes complement term-based inverted indexes. Index construction involves issuing a broad set of basis queries to establish retrievability within the collection. Document identifiers are then used as generalized ad hoc queries against the reverted index to retrieve "relevant" basis queries. Because the reverted index is wholly analogous to standard inverted indexing, most Information Retrieval techniques for ranking and retrieval (models, query languages and operators, et cetera) are directly applicable.. Our experiments demonstrate high performance query expansion using reverted indexing in combination with proven ranking techniques. Results show that this approach outperforms two strong, established baselines for query expansion for on test collections with consistent and significant improvements over a range of judgment depths and types. Also, the computational costs of using reverted indexing are substantially lower at retrieval time relative to the baselines. In this section, we summarize related work and discuss differences with our proposed framework. A primary source of inspiration was the work of Azzopardi et al. [5] . Adopting a document-centric view of information retrieval, they ask what portion of a document collection is retrieved by a large set of queries at a rank less than or equal to k. Building on the notion of retrievability, we index the set of documents retrieved by basis queries. We then apply established informa- tion retrieval algorithms to the reverted index to determine the basis queries most characteristic of a given document. The reverted index also identifies documents that are likely to not be retrievable in the sense used by Azzopardi et al. [5] .. Robertson [18] explored symmetries in information retrieval within the classic vector space framework, examining the concept of duality in representing documents by terms, and vice versa. From this perspective, both queries and documents are treated as "bags of terms." In reverted indexing, on the other hand, basis queries are never processed according to their constituent terms, but rather treated as a atomic units distinct from other basis queries with which they may share common terms.. Craswell et al. [9] use a web-scale search engine to collect a large set of queries and clicked results. They combine these sets into a bi-partite graph and use random walks on the graph to infer likely end points from a single starting point. Knowing what documents many internet users clicked after issuing a query makes it possible to go backwards to find the most common queries related to a given document. Our work differs in key ways. First, by relating queries and clicks in a bipartite graph, Craswell et al. [9] is constrained to a single document starting point, i.e. you can initiate a random walk from only one document at a time. In contrast, reverted indexing can process a group of documents as a set. Furthermore, documents can be combined using boolean operators or other query constructs within reverted indexing. Craswell et al. [9] associate documents only with queries that users have issued. Our framework can use any automatically-extracted basis queries (n-grams, conjunctions, and such) that can be handled by the base ranking algorithm. Reverted indexing is broadly applicable to both large web collections with abundant user data, and to smaller enterprise or personal collections without available clickthrough statistics.. Query expansion is an established approach to improving users' search experience. Traditionally, query expansion is performed at search time using explicit relevance feedback (e.g. [19] ) though other variants propose implicit relevance feedback (e.g. [17, 11] ). Billerbeck et al. [6] present methods for query expansion based on processing user logs to determine which previously issued queries returned a given document. They define "document surrogates" comprised of the terms in issued queries for which a given document appeared among the top 19 results. The number of queries associated with any given document is limited to 39 to retain queries with maximum statistical similarity. Three variations on their general approach permute the representations used in the first and second rounds of retrieval. Either the full document collection or the collection of surrogates is used to perform retrieval or select expansion terms.. In federated information retrieval (FIR), query-based sampling [7] has been proposed to aid in database selection and query expansion. For this, a set of queries is assembled and executed and the set of returned documents are used to estimate collection word frequency statistics, typically to then produce a traditional inverted index. In this manner, multiple document collections are indexed more efficiently using partial information. Ogilvie and Callan [13] applied the approach to query expansion using blind (pseudo) relevance feedback [11] in the FIR setting to examine sampling density and performance tradeoffs with poor results. More recently, Shokouhi, et al., [20] extended this approach by experimenting with database selection and local and global expansion to show improvements in FIR.. Puppin et al. [15, 16] use query logs to partition a webscale document collection using a query-document matrix. They find the top 100 documents for each query from a training set (derived from a query log), and then discover query-document clusters. The text terms of the query set from each document cluster comprises a "document surrogate". A secondary inverted index is constructed from these proxy documents. In use, a query is first evaluated against the secondary index, and well-matching proxy documents are then used to select which primary subcollection index to search. Thus Puppin et al.'s two-tier term→document inverted indexes are close in spirit to Billerbeck et al. [6] , who create proxy documents by augmenting documents with terms derived from queries that retrieved them. In Billerbeck et al., surrogates are query-text proxies for individual documents, while Puppin et al.'s surrogates are query-text proxies for document clusters. In both cases, the entity being indexed is a text document.. We share with both Billerbeck et al. and Puppin et al. the notion of retrievability, but draw a strong distinction in our work relative to what is being indexed. Instead of indexing document proxies composed of query text, we index results sets directly. Our document proxies are constructed out of document identifiers (docids) from query results sets, a process that we call "reverted indexing". This allows us to query the reverted index using document ids to identify queries that were effective at retrieving the given documents. These retrieved queries can be used in a variety of ways: to expand the original query, to browse the collection, etc. Furthermore, reverted indexing should not be confused with direct files: whereas a direct file contains only the terms of the corresponding document, a reverted index contains queries that retrieved the document. These queries may be simple terms, conjunctions, or more complex expressions available in the underlying search engine. We are also not required to mine query logs for our set of system (basis) queries, Q. Billerbeck et al. and Puppin et al., note that 25% of the documents in their collection have no associated queries, and thus no surrogates. This result is consistent with the experiments in [5] . Automatically generating a large set of basis queries is likely to associate more documents in the collection with at least one basis query. Additionally, the surrogates don't preserve each query as a distinct unit, in contrast to our approach (but similar to [18] ). Because logged queries sharing terms are collapsed into terms within a surrogate, the richness (and size) of the representation of documents by surrogates is reduced. Normalizations for the distinctiveness of a system query (akin to inverse document frequency) will in turn be skewed. Also, the estimated relevance and ranking information associated with the logged queries is not retained in the document surrogates. We preserve this information and use it for normalizations that have proven beneficial in standard information retrieval settings. We expect them in turn to improve performance within our framework. We evaluate our approach using a common scenario: query expansion. Such work has a long history, from Rocchio in the early 1970s [19] to today. Query expansion typically proceeds in the following manner (see also (1) User issues query, system returns ranked documents (2) User judges the top n documents, and marks 1 ≤ k ≤ n of them as relevant (3) System selects m terms from these k relevant documents and assigns a weight to each term (4) System runs the expanded, weighted query against the collection and returns as-yet unseen results to the user Variations on these steps interchange system and user involvement. For example, in Step (2) if the user were to make document relevance judgments to a depth in the ranked list of n, marking k ≤ n documents as relevant, this is called relevance feedback (RF). If the system instead automatically assumes that all n documents are relevant, this is called pseudo-relevance feedback (PRF). In Step (4) if the user rather than the system were to choose which of the top m terms to add back to the query, this is called manual query expansion. In this paper we focus on automatic query expansion, which means that the system is wholly responsible for constructing and issuing the expanded query based on the (relevant or pseudo-relevant) documents supplied to it.. To compare several approaches to automatic query expansion algorithms, we fix steps (1), (2) , and (4) across all conditions and only vary Step (3). In Step (1) the topic title-only query is issued using the PL2 retrieval model and the system returns a ranked list of documents. For RF, in. Step (2), NIST relevance judgments are used to simulate user judgment on the top n documents returned by Step (1). (Note that we are applying n judgments of relevance rather than trying to identify n relevant documents.) For PRF, all n documents are simply assumed relevant. Documents identified in. Step (2) are fed to the various automated query expansion algorithms of Step (3). In. Step (4), PL2 is again used to retrieve documents using the expanded and weighted queries from Step (3).. After an initial comparison of several well-regarded algorithms, we chose a Divergence from Randomness model, PL2, for Steps (1) and (4) as it exhibits high precision at low recall, a necessary condition for obtaining many highranked relevant documents for RF or PRF. In both conditions, we use the PL2 algorithm with the same modelspecific parameters on the same standard index. The only difference comes at Step (3), how the expansion terms are selected and weighted. For the control condition, we use two established expansion algorithms, Kullback-Leibler Divergence (KL) and Bose-Einstein (Bo1), for expansion term selection and weighting, as implemented in the Terrier open source retrieval platform [1] . For comparison, we use PL2 ranking on our reverted indexes to select and weight expansion terms as in Section3.2. The number of expansion terms is m = 500 for all conditions, but this choice was based solely on a parameter sweep over the baseline KL and Bo1 expansion algorithms. While fifty expansion terms is a more typical setting, we found that baseline effectiveness continued to improve as more terms were added. We chose to compare against the strongest possible baseline. For our experiments we use two primary TREC collections: (a) Topics 51-200 from TREC 1-3 on Disks 1 and 2, and (b) Topics 301-450 from TREC 6-8 on Disks 4 and 5. We label each of these collections TREC123 and TREC678, respectively.

B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O