A roadmap to robot motion planning software development ABSTRACT: PhD programs and graduate studies in robotics usually include motion planning among its main subjects. Students that focus their research in this subject find themselves trapped in the necessity of programming an environment where to test and validate their theoretic contributions. The programming of this robot motion planning environment is a big challenge. It requires on the one hand good programming skills involving the use of software development tools, programming paradigms, or the knowledge of computational complexity and efficiency issues. On the other hand it requires coping with different related issues like the modeling of objects, computational geometry problems and graphical representations and interfaces. The mastering of all these techniques is good for the curricula of roboticists with a motion planning profile. Nevertheless, the time and effort devoted to this end must remain reasonable. Within this framework, the aim of this paper is to provide the students with a roadmap to help them in the development of the software tools needed to test and validate their robot motion planners. One of the main tasks in robotics is to plan collisionfree paths for a robot from a start to a goal configuration among the obstacles in the workspace. This problem is known as robot motion planning [1À3]. The basic approach is a computationally hard geometric problem that can get much harder if other issues are to be taken into account like sensor uncertainties, differential constraints, or the existence of dynamic obstacles. Robot motion planners must cope with the modeling of objects and their location, with collision detection and other computational geometry problems, with graph representations and search algorithms, and with graphic rendering and other graphical user interface issues. Therefore, the development of a robot motion planner is not an easy Correspondence to J. Rosell (jan.rosell@upc.edu). This paper has introduced a framework for the implementation of a motion planner under an opensource multi-platform philosophy. First, some general development tools are suggested to aid the generation of a software project. They include the programming paradigm (OOP), the programming language (Cþþ), a control version system (OpenSVN), software design tools (StarUML), a multi-platform project generation tool (CMake) and a documentation tool (Doxygen). Then, graphics issues are discusses and Qt is proposed as a graphical user interface library and Coin3D as the library for the graphical representation of 3D worlds. The framework finally includes proposals for the computational geometry needs: collision detection software (PQP) and motion planning tools like sampling (random, Halton, Sdk), nearest neighbor computation (ANN), and graph representation and searching (Boost graph library).. As a didactic guide towards the final goal of implementing a motion planner, the paper ends providing some kick-off examples that illustrate: how to load a scene (that represents the workspace and the robot) described by an XML file and an Inventor file; how to use the collision detection package PQP together with the visualization in Coin3D; and how to saple a configuration space using both random and deterministic sampling sequences (Halton and Sdk). These examples aim at facilitating the students' start-up.. The framework is currently being used by the authors as a teaching aid in a post-graduate motion planning course (some practical exercises to create a basic motion planner based on a probabilistic roadmap can be downloaded together with the kick-off examples). The framework has also been used as the base of the Kautham Planner, a motion planner based on deterministic sampling and harmonic functions developed by the authors [21] .

O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Multiresolution Clinical Decision Support System Based on Fractal Model Design for Classification of Histological Brain Tumours Abstract-Tissue texture is known to exhibit a heterogeneous or non-stationary nature; therefore using a single resolution approach for optimum classification might not suffice. A clinical decision support system that exploits the subbands' textural fractal characteristics for best bases selection of meningioma brain histopathological image classification is proposed. Each subband is analysed using its fractal dimension instead of energy, which has the advantage of being less sensitive to image intensity and abrupt changes in tissue texture. The most significant subband that best identifies texture discontinuities will be chosen for further decomposition, and its fractal characteristics would represent the optimal feature vector for classification. The performance was tested using the support vector machine (SVM), Bayesian and k-nearest neighbour (kNN) classifiers and a leave-one-patient-out method was employed for validation. Our method outperformed the classical energy based selection approaches, achieving for SVM, Bayesian and kNN classifiers an overall classification accuracy of 94.12%, 92.50% and 79.70%, as compared to 86.31%, 83.19% and 51.63% for the co-occurrence matrix, and 76.01%, 73.50% and 50.69% for the energy texture signatures; respectively. These results indicate the potential usefulness as a decision support system that could complement radiologists' diagnostic capability to discriminate higher order statistical textural information; for which it would be otherwise difficult via ordinary human vision.Key words: Texture analysis, fractal dimension, brain tumours, tissue classification.Meningioma tumours usually occur in adults, with a marked female bias represented by a one to three man to women ratio [1] . It also accounts for 27% of all primary brain tumours, making it the most common tumour of that type [2] . An automated meningioma diagnosis system is essential in improving reproducibility by overcoming subjective diagnosis due to variability associated with expert's evaluation.That is, when differences become minor between tumour subtypes of the same grade this might trigger for an increase in intra-observer variability, i.e. pathologist not being able to give the same reading of the same image at more than one occasion, and inter-observer variability, i. variation between different pathologists; and thereby increasing uncertainty that may impact patient outcome. While the dataset could be acquired at fixed magnification and microscope settings, and with the same staining protocol, the variability in the reported diagnosis may still occur [3, 4] . This could be attributed to the non-homogeneous nature of the diseases, namely, not all samples referring to a certain tumour subtype look identical; raising the issue of misclassification. The automated diagnosis system could also assist in overcoming other diagnosis variability-related subjective factors such as, preconception, expectations, relying on diligence, and fatigue, which could cause differences in image perception. Meningiomas can have three grades numbered from I till III, over here we are more concerned with classifying different subtypes within the same grade, which is considered a more challenging task compared to grade classification, as histopathological features tend to become easier to differentiate by the naked eye in the latter compared to the former case.The main concern in the texture analysis problem is two-fold: to capture distinctive characteristics that will maximise the difference between the various image regions, and the selection of the best pattern classifier that can give the optimal performance. Some examples of the different methods used for cytopathological diagnosis include immunocytochemistry, cytophotometry, flow cytometry, and microarray-based comparative genomic hybridisation. However, nuclear texture or chromatin granularity is also considered an important measure in cytopathological diagnosis for its capability of characterising disease or abnormality in cell structures, based on the assumption that different tumour grades tend to form different shapes and behaviours and hence reflect on the tissue texture general appearance; and thus could be easily characterised and captured by a machine learning algorithm. Moreover, approaching the problem for a texture analysis perspective can provide the capability of discriminating higher order statistical textural information for which it would be otherwise difficult via ordinary human vision.However, histopathological tissue texture is known to be heterogeneous, and a varying degree of texture heterogeneity exists. The non-stationary nature of this kind of texture weakens the ability for an effective automated classification from a monoresolution viewpoint, and image pre-processing prior to feature Dealing with the heterogeneous nature of tissue texture in a multiresolution manner while simultaneously exploiting its fractal dimension characteristics can enhance the capability of discriminating higher order statistical textural features, for which it would be otherwise difficult via ordinary human vision; and hence can reduce intra-and inter-observer variability and improve the currently followed clinical protocol in histopathological diagnosis for patient management.. A clinical decision support system that integrates a robust feature extraction method -that can deal with tissue heterogeneity -with the most appropriate classifier model design -that would improve the quality of the extracted features -for an optimised meningioma texture classification was the main concern of this work. The approach that employed the FD for wavelet tree-structured decomposition when integrated with three different classifiers demonstrated its capability to distinguish grade I histopathological meningioma images with an improved accuracy as compared to energy-based decompositions. The SVM classifier along with the BBS FD approach, which proved to be more 35 statistically significant, relies on revealing texture structure complexity that would better characterise the information situated in the middle and high frequency bands. Also the appropriate decomposition levels would be detected when no more significant difference among the subbands exist, saving unnecessary computations. The classification performance using the SVM classifier with up to ten levels of resolution using the FD features for best basis selection (BBS FD ) is shown in Table III , where a threshold value for the FD features was not used to stop the WP decomposition. The best classification accuracy of 91.19% was achieved at the eighth level of decomposition. Alternatively, using the appropriate threshold for D m and L m as discussed in the subband optimisation section, the decomposition should terminate when there is no significant difference between the FD signatures for each subtype separately -highlighted in boldgiving an improved overall accuracy of 91.94% but with a significant save in computational time; while the best classification accuracies using the NBC and kNN were 91.01% and 76.50%. The overall classification accuracy degrades by 8.75% for SVM, 3.12% for NBC, and 9.37% for kNN if the input histopathological images were not pre-processed as in section III.B, that is, converting images to grey level instead of selecting the appropriate colour channel relative to the cell nuclei, and without applying the morphological gradient operation for reduction of mechanical distortion. Also, the geometric deformation experiments on the dataset showed a slight reduction of 0.7%, 1.5%, and 0.9% in classification performance for the SVM, NBC, and kNN classifiers, respectively. Moreover, ensembles with bagging were applied for improving reported accuracies for the best performing classifier. Each individual SVM is trained independently using a bootstrap technique then aggregated to make a collective decision via majority voting. The accuracy of the classifier with best performance, namely SVM, and after applying the feature reduction technique in section VI-B gave a 94.12% (a 2.1% improvement in overall classification accuracy as compared to previous work [20] ); also the NBC and kNN had a 1.6% and 4.85% improvement, respectively. 25 The selected subbands for the best bases selection process spanned by the BBS FD method for multiple tested images were found to be: fibroblastic (W The three subband decomposition approaches were also run at up to ten levels of resolution, and the corresponding classification accuracy is determined at each level. The BBS FD fractal approach outperformed the others, where the BBS CM and BBS E approaches achieved a maximum overall classification accuracy of 86.31%, 76.01% for SVM, and 83.19%, 73.50% for NBC, and 51.63%, 50.69% for kNN; respectively. Also using Wilcoxon signed-rank test, there was significance between the BBS FD proposed technique and the BBS E and BBS CM techniques (p < 0.05). Moreover, Coifman and Wickerhauser energy-based best bases selection from a library of WPs approach with the BBS FD WP tree-structured approach resulted in an increase in classification error of 5.32% for SVM, 6.26% for NBC, and 8.94% for kNN. Table IV shows the confusion matrix for the best feature extraction method that demonstrated the highest classification accuracy (BBS FD classified using SVM). For comparison, the confusion matrices 26 for the NBC and kNN are shown in Table V and VI. Also in order to assess the generalisation capacity of the SVM classifier, the number of support vectors (nSV) for each of the three different feature extraction methods is shown in Table VIII . The total nSV show that the BBS FD method had the least numbers for all WP decomposition levels.. Different types of texture images ranging from fine to coarse for the purpose of non-tissue texture classification were used in this paper to investigate the efficacy of the BBS FD technique. Eight different texture images having a size of 256 × 256 with 8-bit grey levels were selected from the Brodatz album.. Each image that defines a separate class was divided into 32 x 32 size image segments with 50% overlapping, and a NBC was applied for classification. A holdout validation approach was used to test the accuracy, by reserving nearly one third of the image segments (64 samples) for training and the rest The corresponding kNN classifier best classifications without feature reduction for k = 1, 3, 5, 7, and 9, using the BBS FD method were 74.13%, 76.01%, 75.37%, 75.69%, and 75.69%, respectively. All were achieved in the 5 th decomposition level, except for k = 7 which occurred in the second level; therefore, this justifies the selection of a k value of 3 for the kNN meningioma classification to ensure best performance.. Regarding classifiers behaviour illustrated in Fig. 9 , the BBS FD classification accuracies using SVM and kNN remain fairly stable throughout all decomposition levels -improves in deeper levels of resolution -and well above compared to the other two methods. Although the BBS FD classification accuracies using NBC from level 2 to 4 remains higher than the other two methods, it starts to degrade afterwards, scoring accuracies equal or lower to that of the energy approach BBS E .. Furthermore, the statistical measures of the classification tests in Fig. 10 and Fig. 11 show a very good sensitivity for the BBS FD with none of the four meningioma subtypes scoring less than 90% (the highest the psammomatous with 93.75%), the BBS CM had all the subtypes greater than 80% but less than 90% except for the psammomatous subtype with 92.50%, while the BBS E showed the least performance, especially for the fibroblastic subtype with 62.50%. Concerning measuring the negatives that are correctly identified, or known as specificity, again a very good performance for BBS FD with all subtypes above 97% (the worst was the meningothelial with 97.10%), the BBS CM and BBS E had a lower performance having the worst specificity of 93.75% for the transitional for the former and 90.83% for the meningothelial for the latter.

O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Adaptive bolus chasing computed tomography angiography: Control scheme and experimental results In this paper, a new adaptive bolus-chasing control scheme is proposed to synchronize the bolus peak in a patient's vascular system and the imaging aperture of a computed tomography (CT) scanner. The proposed control scheme is theoretically evaluated and experimentally tested on a modified Siemens SOMATOM Volume Zoom CT scanner. The first set of experimental results are reported on bolus-chasing CT angiography using realistic bolus dynamics, real-time CT imaging and adaptive table control with physical vasculature phantoms. The data demonstrate that the proposed control approach tracks the bolus propagation well, and clearly outperforms the constant-speed scheme that is the current clinical standard. With the advent of the multi-slice CT scanner, CT Angiography (CTA) has become an important investigative tool for vascular diseases such as aneurysms and atherosclerosis [1] [2] [3] [4] . It produces cross-sectional images of arteries throughout the body in a fast and minimally invasive way. To better define the vasculature from its surrounding soft tissue, a dose of contrast medium (a bolus) is usually injected into a vein through an IV (intravenous) tube. During the scanning, the patient lies on the CT table. The table is translated into the CT gantry, and the part of the patient's body which is under the x-ray aperture is scanned at that time. In the meantime, the contrast bolus propagates along the blood vessel. Consequently, synchronization of the CT imaging aperture and the contrast bolus peak would result in CT images with a higher Contrast to Noise Ratio (CNR).. The problem, however, is that the contrast bolus dynamics are highly nonlinear, complicated, and influenced by many factors, e.g., patient weight, vasculature diseases, and injection patterns. In current practice, the CT table moves at a preset constant speed, and is very likely to miss the bolus peak. To compensate for this problem, a large amount of contrast medium is injected, which is harmful to the patient's kidney.. To overcome the above problem, the timing bolus [5] [6] [7] and ROI (Region Of Interest) threshold triggering [8] methods have been proposed for a long time. Both methods endeavor to provide a "correct" time to start the CT table. The former method uses a small amount of the test contrast bolus to obtain the bolus arrival time at the ROI, which is then used to start the CT table for the normal scan. However, the arrival time for the test bolus and the main bolus may not be consistent because of different injection volumes and complicated bolus dynamics. The latter method presets a threshold for a selected ROI, and starts to move the CT table when the bolus density inside the ROI reaches this threshold. This is also questionable because the optimal threshold is not known a priori. When the threshold is set too low, the CT table starts too early, and when it is set too high, the CT table does not start at all. A fundamental problem for both methods is that they move the CT table at a constant velocity which does not match the varying bolus velocity during the propagation.. Another class of approaches to obtain the high-CNR CT images tries to optimize the bolus geometry by means of varying the injection rate and duration [8] [9] [10] [11] [12] [13] [14] [15] . It is expected that if a bolus keeps its maximum density for a longer time at a position, then it is easier to synchronize the CT imaging aperture and the bolus peak. As a result, many injection protocols were reported. The multiple-injection method [15] injects the bolus at varying rates; the inverse method [16] obtains the injection protocol (input) from the desired output (the optimized bolus geometry) by taking the system as a LTI (Linear Time-Invariant) system. These approaches need a test bolus and/or a LTI system assumption, which is far away from the reality. In addition, the geometry optimization method may end up with a larger amount of contrast dose, which is not good for patients.. To make it clear, in the following texts "bolus chasing" means to track the bolus throughout the scan by means of varying the table speed, not just to chase the bolus arrival. The bolus chasing techniques were studied in Digital Subtraction Angiography (DSA) [17, 18] and Magnetic Resonance Angiography (MRA) [19] [20] [21] [22] . Preferable results have been reported in these studies. However, there are few studies on bolus chasing CTA [23] . The reason for this is that CTA has a much smaller field of view (z direction) compared to DSA or MRA, and the CT image reconstruction algorithm for varying pitch is complicated.. Recently, we have developed the CT image reconstruction algorithms for varying pitch [24] and investigated the bolus characteristic from hundreds of clinical bolus datasets [26] . We have also developed an adaptive bolus chasing control schemes and numerically tested it on clinical bolus datasets [25] . To the best of our knowledge, no experimental result has ever been reported in the literature on bolus chasing CTA. In this paper, we report the experimental results of the control schemes on a Siemens SOMATOM Volume Zoom CT scanner. Although the experiments are conducted on phantoms, results have successfully shown that the proposed control scheme works very well.. The layout of the paper is as follows. Bolus propagation model and problem statement are introduced in Section II. Section III describes the experimental setup. Section IV presents the approach to the problem. Then in Section V, the experimental results are given. Conclusions are provided in Section VI.

B I I B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Stochastic margin-based structure learning of Bayesian network classifiers The margin criterion for parameter learning in graphical models gained significant impact over the last years. We use the maximum margin score for discriminatively optimizing the structure of Bayesian network classifiers. Furthermore, greedy hill-climbing and simulated annealing search heuristics are applied to determine the classifier structures. In the experiments, we demonstrate the advantages of maximum margin optimized Bayesian network structures in terms of classification performance compared to traditionally used discriminative structure learning methods. Stochastic simulated annealing requires less score evaluations than greedy heuristics. Additionally, we compare generative and discriminative parameter learning on both generatively and discriminatively structured Bayesian network classifiers. Margin-optimized Bayesian network classifiers achieve similar classification performance as support vector machines. Moreover, missing feature values during classification can be handled by discriminatively optimized Bayesian network classifiers, a case where purely discriminative classifiers usually require mechanisms to complete unknown feature values in the data first. Generative probabilistic classifiers optimize the joint probability distribution of the features X and the corresponding class labels C using maximum likelihood (ML) estimation. The class label is usually predicted using the maximum a posteriori estimate of the class posteriors PðC9XÞ obtained by applying Bayes rule. Discriminative probabilistic classifiers such as logistic regression model PðC9XÞ directly. Discriminative classifiers may lead to better classification performance, particularly when the class conditional distributions poorly approximate the true distribution [1] .. Basically, in Bayesian network classifiers both parameters and structure can be learned either generatively or discriminatively [2] . Discriminative learning requires objective functions such as classification rate (CR), conditional log-likelihood (CL), or the margin (as we propose to use in this paper), that optimize the model for a particular inference scenario, e.g. for a classification task. We are particularly interested in learning the discriminative structure 1 of a generative. Bayesian network classifier that factorize as PðC,XÞ ¼ PðX9CÞPðCÞ.. Learning the graph structure of a Bayesian network classifier is hard. Optimally learning various forms of constrained Bayesian network structures is NP-hard [3] even in the generative sense. Recently, approaches for finding the globally optimal generative Bayesian network structure have been proposed. These methods are based on dynamic programming [4, 5] , branch-and-bound techniques [6, 7] , or search over various variable orderings [8] . The experiments of these exact methods are restricted to $ 50 variable nodes. Alternatively, approximate methods such as stochastic search or greedy heuristics are used, which can handle cases with many more variables.. Discriminative structure learning is not less difficult because of the non-decomposability 2 of the scores. Discriminative structure learning methods -relevant for learning Bayesian network classifiers -are usually approximate methods based on local search heuristics. In [9] , a greedy hill-climbing heuristic is used to learn a classifier structure using the CR score. Particularly, at each iteration one edge is added to the structure which complies with the restrictions of the network topology and the acyclicity constraints of a Bayesian network. In a similar algorithm, the CL has been applied for discriminative structure learning [10] . Recently, we introduced a computationally efficient order-based greedy search heuristic for finding discriminative structures [2]. Our order-based structure learning is based on the observations in [11] and shows similarities to the K2 heuristic [12] . However, we proposed to use a discriminative scoring metric (i.e. CR) and suggest approaches for establishing the variable ordering based on conditional mutual information [13] . One of the most successful discriminative classifiers, namely the support vector machine (SVM), finds a decision boundary which maximizes the margin between samples of distinct classes resulting in good generalization properties [14] of the classifier. Recently, the margin criterion has been applied to learn the parameters of probabilistic models. Taskar et al. [15] observed that undirected graphical models can be efficiently trained to maximize the margin. More recently, Guo et al. [16] introduced the maximization of the margin for parameter learning based on convex relaxation to Bayesian networks. We proposed to use a conjugate gradient algorithm for maximum margin optimization of the parameters and show its advantages with respect to computational requirements [17] . Further generative and discriminative parameter learning methods for Bayesian network classifiers are summarized in [2, 17] and references therein.. In this paper, we use the maximum margin (MM) criterion for discriminative structure learning. We use greedy hill-climbing (HC) and stochastic search heuristics such as simulated annealing (SA) [18, 19] for learning discriminative classifier structures. SA is less prone to get stuck in local optima. We empirically evaluate our margin-based discriminative structure learning heuristics on two handwritten digit recognition tasks, one spam e-mail, and one remote sensing data set. We use naive Bayes (NB) as well as generatively and discriminatively optimized tree augmented naive Bayes (TAN) [20] structures. Furthermore, we experimentally compare both discriminative and generative parameter learning on both discriminative and generatively structured Bayesian network classifiers. Maximum margin structure learning outperforms recently proposed generative and discriminative structure learning approaches. SA heuristics mostly lead to better performing structures at a lower number of score evaluations (CR or MM) compared to HC methods. Discriminative parameter learning produces a significantly better classification performance than ML parameter learning on the same classifier structure. This is especially valid for cases where the structure of the underlying model is not optimized for classification [21] . We introduced the MM score for structure learning in [22] using the HC heuristic. The benefit of the MM score over other discriminative scores (i.e. CR) remained open in [22] since the HC heuristic might get trapped in local optimal solutions. This makes the reported performance gain of the MM score during structure learning ambiguouseither MM is useful, or the HC heuristic using CR gets stuck in lowperforming local optimal solutions. For this reason we use SA which partially alleviates this problem. Recently, we also used the MM score for exact structure learning of Bayesian network classifiers [23] . This method is capable to find the global optimal solution. It is based on branch-and-bound techniques within a linear programming framework which offers the advantage of an any-time solution, i.e. premature termination of the algorithm returns the currently best solution together with a worst-case sub-optimality bound. Empirically it is shown that MM optimized structures compete with SVMs and outperform generatively learned network structures. Unfortunately, experiments are limited to rather small-scale data sets from the UCI repository [24] . To overcome these limitations, we use approximate methods for structure learning in this paper.. The paper is organized as follows: In Section 2, we introduce Bayesian network classifiers as well as NB and TAN structures. In Section 3, we present the non-decomposable discriminative scores CL, CR, and MM. Additionally, we discuss techniques for making the determination of these discriminative scores computationally competitive. Section 4 introduces different structure learning heuristics. Particular focus is on SA which is rarely used for discriminative learning of Bayesian network structures. In Section 5, we present experimental results. Section 6 concludes the paper.

O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O


Exploring the spectrum of dynamical regimes and timescales in spontaneous cortical activity Abstract Rhythms at slow (\1 Hz) frequency of alternating Up and Down states occur during slow-wave sleep states, under deep anaesthesia and in cortical slices of mammals maintained in vitro. Such spontaneous oscillations result from the interplay between network reverberations nonlinearly sustained by a strong synaptic coupling and a fatigue mechanism inhibiting the neurons firing in an activity-dependent manner. Varying pharmacologically the excitability level of brain slices we exploit the network dynamics underlying slow rhythms, uncovering an intrinsic anticorrelation between Up and Down state durations. Besides, a non-monotonic change of Down state duration is also observed, which shrinks the distribution of the accessible frequencies of the slow rhythms. Attractor dynamics with activity-dependent self-inhibition predicts a similar trend even when the system excitability is reduced, because of a stability loss of Up and Down states. Hence, such cortical rhythms tend to display a maximal size of the distribution of Up/Down frequencies, envisaging the location of the system dynamics on a critical boundary of the parameter space. This would be an optimal solution for the system in order to display a wide spectrum of dynamical regimes and timescales. Our brain needs to continuously collapse into discrete representations the enormous variety of sensory information and potentially associated actions. This is one of the reasons why attractor dynamics has long been proposed as a plausible and appropriate substrate for brain functions (Hopfield 1982; Amit 1989) . Computational modeling has proven the versatility of such theoretical framework, accurately describing the evolution of the firing activity recorded during the performance of a wide range of cognitive functions (Zipser et al. 1993; Amit and Brunel 1997; Koulakov et al. 2002; Wang 2002; Machens et al. 2005; Deco and Rolls 2005) . In vitro preparations show attractor dynamics as an emergent property of coordinated cell assemblies capable, through synaptic reverberation, to sustain metastable high-firing activity patterns (Cossart et al. 2003; Shu et al. 2003; Lau and Bi 2005; Sasaki et al. 2007 ). Some of these preparations spontaneously display slow oscillatory patterns of spiking activity in which Up states at high firing rates alternate rather regularly with almost quiescent Down states (Sanchez-Vives and McCormick 2000; Shu et al. 2003 ). Yet, attractor dynamics together with a self-inhibition modeling an activity-dependent fatigue mechanism, provide a reliable theoretical framework capable to quantitatively describe the observed features of this phenomenon (Latham et al. 2000; Compte et al. 2003 Slow rhythms in cortical slices largely share electrophysiological properties with the in vivo recordings during slow wave sleep and ketamine anaesthesia, originally characterized in the neocortex of the cat (Steriade et al. 1993) . Furthermore, Up states of spontaneous slow rhythms from the auditory and somatosensory cortices of rats in vivo, show stereotyped time course which conserve their temporal structure in the activity patterns produced in response to sensorial stimuli (Luczak et al. 2009; Harris et al. 2010) . Hence, the study of the neurobiological substrate of slow Up/Down oscillations and the characterization of the underlying network dynamics is directly related to the question of which neuronal machinery is available for information processing in the brain.. Here we explore the footprints leaved by the nonlinear dynamics of the cortical networks during Up/Down oscillations aiming at strengthening the hypothesis that the nervous tissue, and in particular the cerebral cortex, is an excitable medium with a potentially high computational power (Langton 1990; Beggs 2008) . In particular, we investigate the relationship between network excitability and the timescales observable in vitro like Up and Down state durations. We report a wide spectrum of such timescales whose distribution largely match the one predicted by the theoretical framework of attractor dynamics and activity-dependent self-inhibition. The resulting working regime of such cortical networks appears to be supported by a balanced interplay between intrinsic fluctuation of the spiking activity and an eroded landscape of force field driving the system to wonder with timescales longer than the ones available at microscopic level (Tsuda 2002; Durstewitz and Deco 2008; Braun and Mattia 2010) . Slow Up/Down oscillations are a network phenomenon which emerges in isolated cortical tissue (Sanchez-Vives and McCormick 2000; Cossart et al. 2003) from the interplay between a strong synaptic self-excitation sustaining the reverberation of local spikes, and an activitydependent self-inhibition associated to a fatigue mechanism (Latham et al. 2000; van Vreeswijk and Hansel 2001; Compte et al. 2003; Holcman and Tsodyks 2006; Curto et al. 2009; Ngo et al. 2010) . Mean-field theory of IF neuronal networks provides a quantitatively reliable description of the dynamical regimes and timescales of such relaxation oscillators (Gigante et al. 2007 Matching simulations of IF neuron networks, mean-field theory and in vitro experiments, we further found a wide spectrum of average Up and Down state durations with a non-trivial distribution. Its shape highlights a theoretically expected anticorrelation of the permanence times, together with an asymmetry in the maximum state durations: Down states are one order magnitude longer than Up states. This asymmetry and the wide range of timescales seem naturally emerging features of the proposed theoretical description, provided that the usually observed heterogeneity in the excitability of the cortical tissue is taken into account. Network parameters for in silico experiments have been sampled from random uniform distributions, but only the simulated networks displaying an alternation of firing rates between Up and Down states are taken into account. The resulting cloud of cell assemblies (the circles in the bifurcation diagram of Fig. 5c, d ) are distributed closely to the bifurcation boundaries delimiting the corner where relaxation oscillations become unstable and asynchronous firing regimes take place. Hence, networks with Up/Down oscillations are those on the verge of losing stability. This is reminiscent of criticality, which emerges when nonlinear systems are placed at ''the edge of chaos'' (Langton 1990) , where long range activity correlations in space and time are expected to be maximal. The wide spectrum of dynamical scales are the basis for complex computations and information processing in neuronal networks (Bertschinger and Natschläger 2004; Beggs 2008) , and here we speculate that the emergence of slow Up/Down oscillations may reflect how cortical networks spontaneously settle in order to make available a high computational power.. Inspecting the behaviour of single brain slices when their excitability is enhanced through the progressive blockade of the inhibitory synaptic transmission, we observed changes in the state durations which depended on the permanence times under control conditions. As a result, the wide spectrum of timescales is shrunk, yielding an increase of the slow oscillation frequency and a shortening of Up state durations. From a mean-field perspective, this excitability enhancement implies a multivariate change of the key parameters C rec , DI and g. If the cloud of the networks in the parameter space is shifted far from the critical boundaries delimiting the stability of the relaxation oscillations, shorter state durations can be obtained in agreement with experimental observations. Hence, even when only a single network parameter like synaptic feedback is modulated, a kind of homeostatic process may yield the whole system to adapt other parameters accordingly.. Symmetrically, an excitability loss can be modelled shifting the network cloud in the opposite direction of the bifurcation diagram. Critical boundaries will be crossed, and for different reasons yet another compression of the distribution of the state durations is expected. Indeed, the loss of excitability dampens the gain function U, which reflects in a smaller gap between the firing rates in the Up and Down states. Under these conditions, the energy landscape has shallower valleys which eventually melt in a single attractor well associated to a stable asynchronous state. The excitability dampening ruins the force field (Tsuda 2002; Durstewitz and Deco 2008) allowing the system to have a wondering only mildly constrained by what remains of the barriers and the valleys of the energy landscape. The permanence times in the almost overlapped Up and Down states are consequently shortened. Such transition from relaxation oscillations to stationary asynchronous states well represents the change in the dynamical regimes of the activity recorded in vivo when the brain shifts from deep sleep states to resting wakefulness (Destexhe et al. 2007; Curto et al. 2009; Deco et al. 2010) .. In summary, cortical networks seem to have slow Up/ Down oscillations when set in a region of the parameter space maximizing the range of available timescales, which corresponds to a subspace close to the stability boundary of the oscillatory activity regimes. Finally, it should not be neglected that the theoretical framework adopted here is well suited to describe not only spontaneous slow oscillations but also other physiological and pathological brain states like epilepsy, identifiable as different dynamical regimes in the parameter space.

O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O B I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O


Increasing threshold search for best-valued agents This paper investigates search techniques for multi-agent settings in which the most suitable agent, according to given criteria, needs to be found. In particular, it considers the case where the searching agent incurs a cost for learning the value of an agent and the goal is to minimize the expected overall cost of search by iteratively increasing the extent of search. This kind of search is applicable to various domains, including auctions, first responders, and sensor networks. Using an innovative transformation of the extents-based sequence to a probability-based one, the optimal sequence is proved to consist of either a single search iteration or an infinite sequence of increasing search extents. This leads to a simplified characterization of the the optimal search sequence from which it can be derived. This method is also highly useful for legacy economic-search applications, where all agents are considered suitable candidates and the goal is to optimize the search process as a whole. The effectiveness of the method for both best-valued search and economic search is demonstrated numerically using a synthetic environment. Consider a government agency seeking the cheapest contractor to provide a service. It can issue a call for bids and select the lowest bidder. Depending on the costs associated with this process, this may be inefficient from the social welfare perspective (i.e., minimize the overall expense of all parties involved). For example, if there are costs for preparing, submitting, and processing the bids, then requesting all bids may be more costly than necessary. Instead, the agency can publish a threshold on the maximum allowable bid (e.g., a reservation price) and request that contractors submit their bids only if they are below this threshold, potentially reducing the number of bids and their associated costs. However, it must repeat this process with greater reservation prices until at least one bid is received, incurring additional costs for reissuing the call for bids. In many cases, the right sequence of reservation prices results in an overall saving in cost, thereby improving social welfare.. This paper provides a thorough analysis of the problem of deriving the optimal threshold-based search sequence in settings similar to the one above. In the general model, an Copyright c 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. agent needs to find the agent with the best value (either the lowest or the highest, depending on the application) in its environment. The agent is acquainted with the distribution of agents' values; however, the process of learning the value of any specific agent incurs a cost (Kephart and Greenwald 2002; Choi and Liu 2000) . To guarantee search completeness, all agents must be probed, but not all agents must be compelled to reply. As part of the probing, the agent can publish a threshold, compelling a response only from agents whose values are above or below it, and repeat the process until at least one agent responds. Since the process of probing all agents is also a costly act, the optimal sequence should trade off the expected decrease in the number of replying agents with the increase in the number of search rounds.. There are many multi-agent systems (MAS) that can benefit from such a search technique, here called increasing threshold search. For example, a volunteer ambulance corps dispatcher needs to find the closest volunteer to an emergency. She must page the volunteers and request that they call back to learn their locations. Instead of requesting that all volunteers call back, she can request that only volunteers within a certain distance of the emergency call back, and repeat the request with greater distances until at least one volunteer calls. Similarly, in a data-centric sensor network (Intanagonwiwat et al. 2003) , the sink may only need the highest sensor reading. Requesting that all sensors send their data significantly depletes the sensors' power supply. Rather, the sink can request only readings above a certain threshold and iteratively decrease the threshold until at least one reply is received.. Search by iteratively increasing the search extent is a common approach to minimizing search costs, such as depthfirst iterative-deepening search in artificial intelligence (Korf 1985) and expanding ring search in networking (Chang and Liu 2004) . Given a cost structure for the individual search rounds, the overall cost of this type of search depends on the sequence of search extents applied. Most research in optimizing such sequences is in expanding ring search as it applies to networks (Hassan and Jha 2004; Chang and Liu 2004; Cheng and Heinzelman 2005) . Although this problem has been extensively analyzed, the results are not applicable to the problem analyzed in this paper. In the expanding ring search problem, the cost of each round As illustrated throughout the paper, increasing threshold search is applicable to a wide variety of MAS settings, yet it has not received adequate attention in the literature. Most research of techniques utilizing increasing search extents has focused on other models. As discussed in Section 4, the strategies most studied in those domains are very different from the optimal strategy to our problem, both in structure and quality of the solution obtained. By correlating the reservation values to the respective probabilities in our proof of Theorem 1, we reveal that the essence of the optimal solution is captured by a single reservation probability. Upon calculating the optimal reservation probability using the equations supplied, the searcher can easily derive the optimal sequence of search extents.. One important result is that the optimal reservation probability and corresponding expected cost of search are distribution independent. This substantially simplifies calculations. Furthermore, the solution to one problem instance can be used to derive the optimal search sequence for any other instance that only differs in its distribution of values by merely applying a simple transformation.. As evident in Section 5, increasing threshold search can also be useful in economic search settings where the searcher is not constrained to finding the best-valued agent. This has many implications in the evolving research area known also as search theory (McMillan and Rothschild 1994) . Among the various possible extensions to existing economic search models are threshold-based enhancements to sequential search models with finite decision horizon and an integrated preliminary increasing threshold search for refining the population on which sequential search takes place.. Other possible extensions of this work include competition and cooperation models for two or more searchers when operating in settings where search is costly and one or more of them is capable of using increasing threshold search. Consider a searcher agent using a strategy S = [r 1 , . . . , r m = x max ]. (In order to guarantee search completeness when using a finite sequence, the following should hold: r m = x max .) If the agent has to start the i th search round, then there is necessarily no agent found below r i−1 . The a priori probability of such a scenario is (1 − F (r i−1 )) N . Furthermore, upon reaching the i th round, the searcher agent can update its beliefs concerning the PDF of the values of the N agents, as it knows that these are necessarily in the interval. Similarly, the CDF of any of the agents' values in round i,. The expected cost of the i th round is thus:. as it takes into account the cost of initiating the new search round and the learning costs for each possible number of agent values j (1 ≤ j ≤ N ). The expected cost of using strategy S is the sum of the expected cost of each of the m search rounds weighted by the probability of reaching that round:. where r 0 ≡ x min . The probability of starting the i th search round can alternatively be formulated as the product of the probability that no agent was found in each of the i − 1 previous rounds, expressed as. For the specific case in which the reservation values are chosen from a finite set {x 1 , x 2 , ..., x m }, the optimal strategy can be derived with the following dynamic programming formulation:. where C(x i ) is the cost of continuing the search if a search up to value x i failed to yield any bids. For the general case in which the interval [x min , x max ] is continuous and the process is not constrained by a finite number of rounds, the optimal search strategy must be derived with different methodology since, as we prove in Theorem 1, the optimal search sequence is either a single search round in which the value of all agents is learned or an infinite sequence of reservation values. Theorem 1. The optimal sequence of reservation values is either. for some P and ∀i, j > 0.. Proof. Assume the finite sequence S 1 = [r 1 , . . . , r m ] is the optimal strategy. We use S 2 = [r 2 , . . . , r m ] to denote the optimal strategy to be used if no agent is found in the first search round and denote its expected cost from that point on by V (S 2 ). Using S 2 , we construct strategy S 1 = [r 2 , . . . , r m ] to be applied from the first round, where F 1 (r i ) = F 2 (r i ) ∀1 < i ≤ m. The new strategy S 1 has an expected cost V (S 1 ), which equals V (S 2 ) according to (5). Since S 1 is the optimal strategy, V (S 1 ) ≤ V (S 1 ). Now consider a new strategy S 2 = [r 1 , . . . , r m ] to be applied from the second round on, where F 2 (r i ) = F 1 (r i ) ∀1 ≤ i ≤ m. We denote the expected cost of S 2 from that point on by V (S 2 ). According to (5), we obtain V (S 2 ) = V (S 1 ). Since S 2 is the optimal strategy from the second round, then. The same logic can be applied to any search round j ≤ m, resulting in V (S j ) = V (S 1 ). In particular, V (S 1 ) = V (S m ). However, the cost onwards when reaching the last (m th ) round, V (S m ), equals α + β(N ) since r m = x max ; thus, we necessarily find all agents. Therefore, we obtain V (S 1 ) = α + β(N ), which is equivalent to S 1 = [r 1 = x max ]. Any optimal strategy that has an expected cost less than α + β(N ) must consist of an infinite sequence and satisfy F i (r i ) = F i+1 (r i+1 ) = P , ∀i > 0 and some P .. The immediate implication of Theorem 1 is that the optimal search strategy can be expressed as a single value 0 < P ≤ 1, denoted the reservation probability. This is the key to deriving the optimal sequence of reservation values. As outlined below, the searcher only needs to derive P and then set the reservation value r i in each round i such that F i (r i ) = P . Additionally, the derivation of P is distribution independent, such that the same value can be used for problem instances that only differ by their CDF.. First we show how to derive P . Since the optimal sequence is infinite and the expected cost from each round onwards is stationary, the expected cost of using P is:. Consequently:. The value P = P * that minimizes V (P ) according to (8) is the optimal reservation probability. This can be solved using numerical approximation. Then, based on (2), the corresponding reservation value to be used in each round can be calculated by solving for r i in the equation P = F (ri)−F (ri−1) 1−F (ri−1) , i.e.,. We consider the special case where the cost of learning the values of j agents is linear in j, i.e., β(j) = cj. This setting is highly applicable as, in many cases, agents are evaluated individually and independent of one another. Substituting β(j) = cj, the expression N j=1 (j N j P j (1 − P ) N −j ) in the numerator of (8) is the mean of a binomially distributed random variable, which equals N P . Therefore,. This result enables the proof of Proposition 1, which highlights the nature of the trade-off by which P * is set. Figure 1 : Performance of the proposed method compared to expanding ring search Proposition 1. When β(j) is linear in j, the reservation probability that minimizes V (P ), P = P * , satisfies c = (1 − P * ) N −1 V (P * ).. Proof. Differentiating (10) with respect to P and setting it to zero obtains:. Notice that V (P )(1 − (1 − P ) N ) = α + cN P according to (10). Substituting the latter expression in (11), we observe that the value P * which satisfies the equation is given by. The explanation of Proposition 1 requires understanding the trade-off associated with any increase in P . By increasing P , we increase the chance of finding each of the agents. Each agent found due to the increased chance will incur a cost c. On the benefit side, if the agent found due to the increase is the only agent found during that round, then the increase has actually saved us the expected cost associated with continuing the search V (P ). The probability that the latter case holds is (1 − P ) N −1 (i.e., when all other agents are characterized with a value above F i (P )). Otherwise, the search just ends. Since the incurred cost c is fixed, and the expected benefit (1 − P ) N −1 V (P ) decreases as P increases, the optimal P value is the one for which (1 − P ) N −1 V (P ) = c, i.e., when the additional benefit due to the potential saving is offset by the cost incurred by finding that agent. The expected costs of the strategies were calculated under various synthetic settings. Figure 1 depicts the performance (measured as the expected cost of search) of the three expanding ring-based methods and the increasing threshold search as a function of the number of agents N in the environment. The performance is evaluated in three settings that differ in their search costs. The distribution of values used in all three settings is Gaussian, with µ = 50 and σ = 12.5, normalized over the interval (0,100). The costs of search used in the different settings are: (A) α = 1, β(i) = i; (B) α = 1, β(i) = (i) 2 ; and (C) α = 100, β(i) = i. As expected, according to Theorem 1, the performance of the increasing threshold search generally dominates the methods inspired by expanding ring search. Also, as expected, none of the expanding ring-based methods generally dominates any of the other methods. Similarly, the differences in performance between the increasing threshold search technique and the expanding ring search techniques are setting dependent. In some settings, an expanding ring-based method can result in performance close to the one achieved with increasing threshold search (e.g., two-step technique in setting (C)) while it can perform significantly worse in others (e.g., in settings (B) and (A)). Finally, we observe that the number of agents in the environment is a significant factor affecting performance of all methods. Nevertheless, the effect of this factor on the performance of the increasing threshold search is significantly smaller than on the performance of the expanding ring search methods.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


DisCons: a novel tool to quantify and classify evolutionary conservation of intrinsic protein disorder Background: Analyzing the amino acid sequence of an intrinsically disordered protein (IDP) in an evolutionary context can yield novel insights on the functional role of disordered regions and sequence element(s). However, in the case of many IDPs, the lack of evolutionary conservation of the primary sequence can hamper the study of functionality, because the conservation of their disorder profile and ensuing function(s) may not appear in a traditional analysis of the evolutionary history of the protein.Results: Here we present DisCons (Disorder Conservation), a novel pipelined tool that combines the quantification of sequence-and disorder conservation to classify disordered residue positions. According to this scheme, the most interesting categories (for functional purposes) are constrained disordered residues and flexible disordered residues. The former residues show conservation of both the sequence and the property of disorder and are associated mainly with specific binding functionalities (e.g., short, linear motifs, SLiMs), whereas the latter class correspond to segments where disorder as a feature is important for function as opposed to the identity of the underlying sequence (e.g., entropic chains and linkers). DisCons therefore helps with elucidating the function(s) arising from the disordered state by analyzing individual proteins as well as large-scale proteomics datasets. Conclusions: DisCons is an openly accessible sequence analysis tool that identifies and highlights structurally disordered segments of proteins where the conformational flexibility is conserved across homologs, and therefore potentially functional. The tool is freely available both as a web application and as stand-alone source code hosted at http://pedb.vib.be/discons. DisCons is a novel and freely available online and downloadable tool that combines the quantitative description of the position-specific evolutionary conservation of the amino acid sequence with predictions of the conservation of its disordered/flexible state, providing meaningful information on the evolutionary context of a disordered protein segment. Furthermore, DisCons uses this combined information to classify each disordered position into one of three categories, namely: constrained, flexible and non-conserved. These classes have been suggested to correlate with distinct functions that arise from the disordered state; therefore DisCons may provide information orthogonal to those obtained by other methods, which potentially enhances the reliability of the identification of functionally relevant disordered segments within proteins. We demonstrated that DisCons can be used to investigate both sequence-and disorder conservation in a functionally meaningful manner by bench-marking our procedure on MoRF and SLiM datasets, which are known to be conserved functional units enriched in structural disorder. It is important to emphasize that the success of calculation with DisCons strongly depends on the quality of the underlying multiple sequence alignment; therefore it is Figure 5 Example of a flexible disordered linker. The intrinsically disordered protein Sic1 binds to the CDC4 subunit of an SCF ubiquitin ligase forming a fuzzy complex (data for the structural ensemble obtained from the Protein Ensemble Database, PED). Two conformations of the same Sic1 fragment from the ensemble of this complex are displayed on panels A and B. Below, the DisCons profile on panel C corresponds to the same fragment, and shows the segment that is flexible disordered between two constrained disordered regions (1st MoRF and 2nd MoRF) that bind to Cdc4. This former segment corresponds to a flexible linker (shown in orange in panels A and B), which is found between two MoRFs (1st and 2nd on panels A and B) that bind to the same binding pocket in the fuzzy complex. The flexible conservation of the linker indicates that the sequence is of less importance than its conformational flexibility.. advised to review and optimize each MSA to maximize the information of the output. Taken this into consideration, DisCons can be used as an online or stand-alone tool for quantifying the conservation of both sequence and structural disorder by analyzing large-scale protein datasets and individual proteins. As such, DisCons might provide an additional layer of information for the investigation of protein disorder, and could serve to enhance the performance of prediction software such as MoRFPred [13] , or provide descriptive information for disorder related databases such as D2P2 [32] , MobiDB [33] or PED [31] . Intrinsically disordered proteins (IDPs) and intrinsically disordered regions (IDRs) within structured proteins are defined by the lack of a stable tertiary structure and a corresponding high degree of flexibility under physiological conditions [1] . The importance of conformational flexibility is reflected in the observation that IDPs and proteins with IDRs are often involved in essential cellular processes, such as cell-cycle regulation, transcription, and translation [2] [3] [4] . Additionally, they often play major roles in pathologies associated with aggregation and misfolding [5, 6] , making them attractive potential drug targets [7] . Genes encoding such amino acid sequences are under reduced selective pressure, which is manifest in a higher sequence diversity compared to genes of structured proteins/domains [8] . Whereas the functionality of a protein segment is often approached by investigating the evolutionary history of its primary sequence [9] , this is often difficult to achieve with IDP/IDR sequences, due to their generally high sequence diversity [10] .. On the other hand, combining the information derived from analyzing the conservation of both sequence and disorder can be much more useful, and this idea has recently been suggested to partition disordered residue positions into three separate groups of potentially different functional attributes: i) 'constrained' , if both features (amino acid sequence and the property of disorder) are conserved; ii) 'flexible' , if only disorder is conserved; and finally, iii) 'non-conserved' positions where disorder is not conserved. These specific evolutionary behaviors have been shown to correlate with distinct disorder-related functional categories [11] . In general, segments of constrained disorder are often associated with protein binding and molecular recognition, whereas flexible disorder is prevalent in linker segments acting as entropic chains. Nonconserved disorder has not been associated with specific protein function so far.. Here, we present DisCons, a novel web application and downloadable, stand-alone source code that offers a description of the conservation of both the amino acid sequence and of the feature of structural disorder, and performs the classification of disordered positions into these three categories. Thus, DisCons provides an additional (integrative) layer of information that together with other sequence-based tools, such as the PAML software package [12] , MoRFpred [13] and Anchor [14] , should facilitate the effective identification of functionally important disordered regions in proteins. Both the web application and the downloadable version of DisCons hosted at the website are freely available without registration. The DisCons website is divided into four functional sections that are accessible both through the menu and via the options shown on the welcome page. These sections correspond to the three running mode interfaces: 'quick', 'advanced' and 'from alignment'. The fourth functional section is 'help', which offers the complete documentation of the server and source code, in addition to a user guide.. The 'quick' running mode requires a single protein sequence (in FASTA format) as input, or alternately, a UniProt [15] accession ID. In this mode, the default parameters are used through all the calculation steps of the DisCons workflow. Although this calculation is the easiest to set up, experienced users might prefer to use the 'advanced' tool enabling a better understanding of the results leading to more fine-tuned functional interpretations.. The 'advanced' mode also accepts a single protein sequence or UniProt ID in a manner similar to the 'quick' calculation, but in this mode users can manually set all the parameters of the underlying calculation, allowing for a detailed optimization of the protocol pipeline, and a better overall command of the final results.. Finally, the 'from alignment' mode is best suited if the user already has a custom made, reliable multiple sequence alignment that can be used for the calculations. The main advantage of this mode is speed, since the need for running a BLAST search and constructing the multiple sequence alignment with MAFFT (which are the most time-consuming of all the steps) is circumvented. Therefore, this mode is significantly faster than the others which start from a single sequence (although even in 'quick' and 'advanced' modes, the approximate time for generating the results is 34 seconds for a~2400 residue long protein). By default, the stand-alone source code is also running 'from alignment'; however if the necessary dependencies, namely BLAST+ [16] and MAFFT [17] are available locally in the user's computer, the full pipeline can be utilized in a straightforward manner.. Depending on the running mode, the workflow of the calculations has a different starting point (Figure 1) . In 'quick' and 'advanced' modes, the procedure starts with a BLASTP or PSI-BLAST search [16] to collect sequences similar to the query sequence. In 'advanced' mode, the search dataset (Swiss-Prot [15] (used by default) or PDB [18] ) and the BLAST threshold values can be specified. Next, a multiple sequence alignment (MSA) is created from the set of identified homologous sequences using MAFFT [19] . This is the most crucial part of the procedure, since aligning disordered regions is non-trivial due to the potential diversity of related sequences. Because an incorrect alignment will compromise the subsequent calculations, it is advised either to use the 'advanced' mode to fine-tune the alignment procedure or to use a reliable, user-defined multiple alignment in the 'from alignment' mode.. In the next step (which is the starting point when running in the 'from alignment' mode), the MSA is used to construct an aligned disorder profile by running IUPred (default) [20] , VSL2 [21] , ESpritz [22] or FoldIndex [23] on each of the aligned sequences. The disorder scores are first transformed to a binary scale (1 = disordered, 0 = ordered; residues with a disorder score of 0.5 or greater are considered as disordered) for each sequence. Details about the transformation procedure for the three different disorder predictors are given in the help section of the website. Next the fraction of disordered residues at every position in the MSA across the different sequences is calculated, thereby effectively quantifying the position-wise conservation of disorder.. Next, sequence conservation scores for each position in the alignment are calculated using the algorithm developed by Capra et al. [9] . In 'advanced' and 'from alignment' modes, a number of parameters such as the algorithm of choice or the background distribution can be specified to make the calculation more robust.. Finally, positions in the MSA are scored by both their sequence-and disorder conservation, ranging from 0 (very diverse) to 9 (highly conserved) ( Figure 1 ). Based on these pairs of scores, each position will fall into one of four distinct categories. As suggested by Bellay et al. [11] , positions with a higher degree of disorder can be 'constrained' ('C'), if both the sequence and disorder conservation scores are 5 or greater; 'flexible' ('F'), if the sequence conservation is lower than 5 but the disorder conservation is 5 or greater; or 'non-conserved' ('N'), if the disorder conservation is lower than 5, but higher than 0. Positions with a disorder conservation score of 0 are completely lacking disorder and therefore are considered as 'structured' ('S'). Thus, regions of constrained disorder show strong conservation both at the amino acid level, and also of the disorder feature, while flexible disordered regions are variable in terms of amino acid sequence, but retain a significant level of disorder in evolution. Lastly, non-conserved disordered regions lack disorder as a conserved feature, and are generally thought not to be associated with functions [11] .. On the results page, the position-specific conservation profile is provided on the output screen, and the fractions of residues falling into each of these distinct categories are also displayed in a tabular format at the bottom of the page, effectively quantifying the conservation of disorder in the query sequence (a part of such an output is shown in Figure 1 ). The sequences of consecutive 'constrained' disordered regions are also recorded and are available for download in FASTA format along with the profiles and fractions in text format using the links that are provided. Such segments of consecutive stretches of constrained disorder are most likely to correspond to functionally important IDRs such as linear motifs or MoRFs, as we describe below. . The procedure starts either from a query sequence, followed by a BLASTP or PSI-BLAST search and the aligning of the retrieved sequences, or from a user-provided multiple sequence alignment. Next, position-specific sequence-and disorder conservation scores are calculated and finally these pairwise scores are combined to create the (aligned) DisCons profile. The result of one such classification is displayed at the bottom of the figure.

O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O