Segmentation of Bone Structure in X-ray Images using Convolutional Neural Network Abstract-The segmentation process represents a first step necessary for any automatic method of extracting information from an image. In the case of X-ray images, through segmentation we can differentiate the bone tissue from the rest of the image. There are nowadays several segmentation techniques, but in general, they all require the human intervention in the segmentation process. Consequently, this article proposes a new segmentation method for the X-ray images using a Convolutional Neural Network (CNN). In present, the convolutional networks are the best techniques for image segmentation. This fact is demonstrated by their wide usage in all the fields, including the medical one. As the X-ray images have large dimensions, for reducing the training time, the method proposed by the present article selects only certain areas (maximum interest areas) from the entire image. The neural network is used as pixel classifier thus causing the label of each pixel (bone or none-bone) from a raw pixel values in a square area. We will also present the method through which the network final configuration was chosen and we will make a comparative analysis with other 3 CNN configurations. The network chosen by us obtained the best results for all the evaluation metrics used, i.e. warping error, rand error and pixel error. It is relatively easy for a human to recognize some objects out of an image. Thus, starting from small age, we manage to accurately identify certain common objects from a picture. The identification of specific objects from the field we are working in can be performed only after a process of acquiring good knowledge and experience over time. In the same way, the present article aims at identifying (segmentation) some bone tissues from an X-ray.. If the entire process is desired to be automatically performed by a computer, then we need a model trained to make the difference between a bone tissue and the rest of the image. Deep neural networks are among the best models that are recommended for this type of tasks due to their method of managing to imitate the human recognition process in the visual cortex.. Their mode of operation consists in extracting some features characteristic for each image and creating some feature maps at the level of each physical layer of the network. The method of creating each feature map consists in convolving the input image with a linear filter, adding a bias term and applying a non-linear function. Thus, we create a sequence of features that converge to a final layer (the output layer). On the final layer a 2-type classification process is performed: bone tissue and others. As the neural network must segment an X-ray image, the 2 classes are sufficient because we are interested only in the one identifying the bone tissue.. This architecture was presented for the first time by Fukushima [1] (1980) by using a network architecture (Neocognitron) that further led to the development of several similar architectures. This network architecture was later developed by LeCun [2] and Simard [3] under the name of Convolutional Neural Networks (CNN).. As we mentioned above, the mode of operation of these networks consists in realizing some feature maps that are extracted from an image with the help of specialized filters. This operating method is used by other similar techniques from computer vision field: local receptive fields for predictor units based on semi-linear predictability minimization [4] or Gabor filters [5] . The main problem of these methods is that the parameters of the filters are calculated in the initial phase and remain constant until the end of the analysis process.. Contrary to this method, the use of CNN makes possible the random initialization of filters whose final values are determined only after the neural network training process. This procedure presents a double advantage. First, the filters are calibrated on the dataset that must be recognized and will be used for training the network. Second, the minimum intervention of the user (the network can calculate itself the necessary parameters).. A major improvement for the CNN architectures consisted in introducing max pooling [6] [7] layers. The main role of this layer is to extract only the best features existing at a certain time period and to eliminate the others. This method is highly connected to the capacity of generalization/specialization of the network. By selecting a feature and eliminating the others we increase the capacity of generalization; and by selecting the best feature from a great number of features, we manage to keep the level of specialization constant.. The use of this type of network for image recognition is very common and so successful that in the field of handwriting recognition, the human recognition capacity was surpassed by the neural network. [8] Moreover, this type of network was used for segmenting the neural membranes and the results obtained were promising. [9] During time, this technique was used for face recognition [10] , field in which it recorded the best results.. A step forward in using this type of neural networks is Segmentation of bone structure in X-ray images using convolutional neural network. Cosmin Cernazanu-Glavan 1 building some Convolutional Neural Network Committees. This technique consists in training several neural networks with different configurations on the same dataset. There are several techniques for combining their results, but Meier et al. [11] showed that the arithmetic mean for determining a single set of outputs remains the best technique. We must add that this technique was not approached in this paper because of the longer period of training time (several days) required for a single network. In present, there are several techniques for X-ray image segmentation, but in general they all require the human intervention during one or several stages of the procedure. In what follows, we will present some of the most important techniques used, their number being much higher. We must add that we have chosen the techniques with the most promising results and with a high degree of automation.. Consequently, we can point out the segmentation technique proposed by Hu et all [12] consisting in two key procedures: a fitting weight matrix built to reduce the effect of subjective factors entered by the user in order to obtain the initial rough contour of cervical vertebra and a curve fitting method based on rotating and overlapping parabolic curves to derive the final segments of cervical vertebra.. An interesting approach is presented in [13] . It aims at determining some salient regions. Thus, the proposed algorithm uses a hexagonal representation of the image pixels and a hyper graph structure to process this hierarchical structure.. Another method developed by Qu et all [14] uses the PCA based particle filtering for defining a segmentation algorithm used in noisy medical images such as X-ray angiographic and ultrasound images where the image quality is very poor. It exploits the prior clinical knowledge of object's shape information through a PCA model.. For automated medical image segmentation, a new fuzzy level set algorithm [15] uses fuzzy clustering as the initial level set function. The fuzzy level set method is derived from the classical Hamilton-Jacobi functional and it is enhanced with locally regularized evolution. Even if the author mentions it is an automated method, the human intervention is necessary in the initial stage of the algorithm.. In [16] too, it is required a priori global knowledge of the geometric structure of each hip. Then, a search is performed using a shape context model. The shape model consists of the integration of two statistical models: the first contains knowledge about the edges extracted from radiograph and the second describes the local image structure around each model point. The method proposed in [17] creates a new ontology for knowledge-based approaches of automated systems used in segmentation of radiographic images. The author aims at creating an automated system which could help the doctors in making some decisions.. Using two specialized artificial neural networks trained with statistical information for the segmentation task, in [18] is proposed a new method of X-ray segmentation. Thus, the first network sections the image in two regions (white pixels and non-white pixels), and the second network sections the interest areas only from the white-pixels region. Although it is an automated method, its utility is restrained as it can be used only for simple tasks.. In contrast to the methods presented above, our approach aims at eliminating the human intervention from the segmentation process. It is desirable to obtain a complete automation of the segmentation process irrespective of the type of information contained in an X-ray image. For a successful solution, we considered as a best option the use of a convolutional neural network that could perform an automatic segmentation. Our goal consists in identifying the bone tissue and delimitating it from the rest of the image. Thus, input data for other automatic methods of diagnose and analysis are supplied.. We performed a pixel segmentation method that determined if the pixel is/is not part of the bone tissue. This can be achieved by cutting up the area surrounding the pixel (this is situated in the centre of the area) and applying a convolution operator for that area.. The network presented in this article uses an area of 128x128 pixels for classifying a single pixel. The classification of all the pixels in an image is done by scanning the entire image with the determined area (128 x 128 pixels). We start from the top left corner and we move the area to the right. When we reach to the right edge, we pass to another row and the area is moved to the left, and the process is being repeated.. The process ends when the area reaches the right bottom corner of the image, thus being covered all the pixels from the image. Each pixel from the image was classified as belonging/not belonging to the bone tissue. The main strength of our approach consists in using a convolutional neural network as pixel classifier for the segmentation process of some X-ray images. The network analyzes each pixel from the image and tries to classify them in the 2 classes {bone, non-bone}. In this way, we try to separate the bone tissue area from the rest of the image.. The use of CNN makes our method a complete automatic one in contrast to other segmentation methods of X-ray images. We determined a CNN configuration and its performances were compared to other three networks of the same type, but with different configurations. For all the three considered metrics, i.e. pixel error, warping error and rand error, our network obtained the best results in contrast to other configurations. For ensuring a minimum training time of the network, we used only the interest areas from an image.. The last series of experiments focused on extracting an accurate contour of the bones. This method allows CNN to provide an exact delimitation of a specific bone and it is a completely different way of using the network for the X-ray images. The results were very good allowing the use of this type of neural network in order to obtain any contour of the bones from an X-ray image. [25] The remarkable results obtained by the CNN open interesting perspectives for applying it to other medical images, too. In this regard, there have been performed experiments of segmenting some images from physical areas of the human body, others than the ones used in the images from the training set. The results obtained have not been satisfactory and they require running the experiments on a new generation of graphic cards.. As a final conclusion, for obtaining maximum performances, it is recommended to use the convolutional neural network for images that belong to the same category (in this case -X-rays for the chest area). During the experiments performed for obtaining a wide view upon the performances recorded by the network, we Table I ).

O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Statistical timing analysis for intra-die process variations with spatial correlations Piocess variatiuns have become a critical issue in performafrce verification of high-performance designs: We present a new, stofistical timing analysis method that accounts for inter-and intra-die process variations and their spatial correlations. Since staristical timing analysis has an exponential run time complexiw, wepropose a method whereby a sratistica1,bound on theprobability distribution function of the exact circuit delay is computed with linear run time. First, we develop o model.for representing inter-and intra-die variations and their spatial correlations. Using this model, we then show how gate delays and arrival times can be represented as a sum of components, such that the correlation information berween arrival times and gate delays is preserved. We then show how arrival rimes are propagated and merged in the circuit to obtain an arvival time distribution that is an upper bound on the distribution of the exact circuit delay. Weprove the correctness of the bound and also show how the houndcan be improved hypropagating multiple arrival times. The proposed algorithms were implemented and tested on a set of benchmark circuits under several process variation scenorioios. The results were compared wifh Monte Carlo simulation and show an accuracy of 3.32% on average over all test CaSeS. Static timing analysis has become an indispensable part of performance verification. Static timing analysis has the advantage that it does not require input vectors and has a run time that is linear with the size of the circuit. A number of methods have been proposed to increase the accuracy of static timing analysis through improved delay models and analysis techniques. In recent technologies, the variability of circuit delay due to process variations has become a significant concern. As process geometries continue to shrink, the ability to control critical device parameters is becoming increasingly difficult, and significant variations in device length, doping concentrations, and oxide thicknesses have resulted.. Traditionally, process variations have been modeled in static timing analysis (STA) using so-called case analysis. In this methodology, best-case, nominal and worst-case SPICE parameters sets are constructed and the timing analysis is performed several times, each time using one case file. Each execution of static timing analysis is therefore deterministic, meaning that the analysis uses deterministic delays for the gates and any statistical variation in the underlying silicon is hidden, While this approach has been successfully used in the past to model die-to-die variations, it is not able to accurately model variations within a single die. With the continual scaling of feature sizes, the ability to control critical device parameters on a single die has become increasingly difficult. Using a worst-case analysis for these so-called intra-die variations therefore leads to very pessimistic analysis results since it assumes that all devices on a die have worst-case characteristics, ignoring their inherent statistical variation. The emerging dominance of intra-die variations therefore poses a major obstacle for deterministic STA, giving rise to the need for statistical timing analysis approaches.. Permission to make digital or hard copies of all or part of this work for personal or cIas5room use is granted wilhoul fee provided that copies are not made or distributed for profit oc commercial advantage and that copies hear this notice and the full cililion on the first page. To copy otherwise, to republish, l o post on servers or to redistribute to lists, requires prior specific permission and/or a Tee.. In general, process variations can be divided into inter-die variations and intra-die variations. Inter-die variations are variations that occur from one die to the next, meaning that the same device on a chip has different features among different die of a wafer, from wafer to wafer, and from wafer lot to wafer lot. Intra-die variations are vanations in device features that are present within a single chip, meaning that a device feature varies between different locations on the same die. Intra-die variation result from equipnient limitations or statistical effects in the fabrication process, such as statistical variations in the doping concentrations.. Intra-die variations often exhibit spatial correlations, where devices that are close to each other have a higher probability of being alike than devices that are placed far apart. This has been reported especially for gate length variations [I] . Intra-die variations can also have a deterministic component due to topologically dependencies of device processing, such as CMP effects and optical proximity effects [Z] . In some cases, such topological dependencies can be directly accounted for in the analysis [3][4], whereas in other cases, such variations are treated as random.. Statistical timing analysis is similar to deterministic timing analysis in that arrival times are propagated through the circuit from primary inputs to primary output. In statistical timing analysis, however, the gate delays and arrival times are represented with random variables. The difticulty of statistical timing analysis results from the correlations that arise among the arrival times in the circuit and between the arrival times and gate delays. These correlations must be taken into account when arrival times are propagated in the circuit, leading to an exponential run time complexity and making statistical timing analysis a challenging problem.. A number of statistical timing analysis approaches have been proposed in recent years [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] . In [13] thecorrespondence between deterministic timing analysis and statistical timing analysis was first shown. However, the proposed method does not address the correlation between the arrival times. In [14], a novel mcthod using discretized probability distributions is proposed. However, the run time of the method is exponential and the proposed approaches to reduce the run time have an uncleni impact on the accuracy. In [ 151, a novel method using statistical bounds is proposed with gate delays restricted tu Gaussian distributions. However, to obtain a high quality bound, it is necessary to enumerate all paths in the circuit, leading to exponential run time complexity. In [16], a path based statistical delay computation is presented using an accurate delay model. However, the analysis is performed one path at a time and the number of critical and near-cntical paths in a circuit can be very large. In [17], a new circuit optimization method was therefore proposed that reduces the number of near critical paths in a circuit, thereby improving the statistical delay ofthe circuit. Finally, in [IS] a method using statistical hounds is presented that addresses the arrival time correlations due to path reconvergence. However, the method does not address arrival time correlations due to spatial correlations between the gate delays.. In this paper, we therefore propose a new statistical timing analysis approach to model the impact of process variations on circuit delay. We model both inter-and intra-die process variations and account for spatial correlations of the gate delays. In our analysis, we focus on gate length variability since it has been shown to have a dominant impact on gate delay [I] . However, our analysis can be easily extended to other process variations as well. We fust present a model for inter-and intra-die gate length variation and their spatial correlations. Gate delays and arrival times are represented as a sum of random variables, and presewe the spatial correlation infonnatioo.. The correlation between the arrival times complicates the computation of the maximum arrivals times, as required during arrival time propagation. Since the exact computation of the maximum arrival time requires exponential run time, we propose a method that produces an upper bound on the exact anival time in linear run time.. We prove the correctness of the proposed bound in the presence of spatially correlated gate delays. The obtained bound is itself a random variable with a probability distribution function, allowing for the computation of useful statistical quantities such as confidence points. In order to improve the proposed bound, we propose a method whereby multiple arrival times are propagated in the circuit at the expense of additional mn time. We implemented the proposed methods and tested them on benchmark circuits. We demonstrate that using the proposed methods, the statistical delay of a circuit can be computed with high accuracy.. The remainder ofthis paper is organized as follows. In Section 2, we present our model of process variations and our modeling assumptions. In Section 3 we present our approach for statistical timing analysis. In Section 4, we present the heuristic method for improving the quality of the bound by propagating multiple arrival times. In Section 5. we present our results and in Section 6 we draw our conclusions. In this paper, we have proposed a new statistical timing analysis algorithm. The method has a linear run time and computes an upper bound on the distribution of the exact circuit delay. We first, proposed a model for inter-and intra-die process variations that accounts for spatial correlations. We then presented an efficient method for propagating arrival times in the circuit, which is linear in run time, and computes an upper bound on the distribution function of the exact circuit delay. We proved the correctness of the bound and showed how the bound is improved by propagating multiple arrival times at each node, using a heuristic method for selecting propagated arrival times. We tested the proposed methods on a number of synthesized benchmark circuits and demonstrated the accuracy and efficiency of the approach. The statistical hound computation, as well as the proposed refinement method were implemented and tested on the synthesized version of ISCAS85 [20] benchmark circuits. Delay sensitivities were calculated for the standard cell library which used a 180 nm nominal device length. We used 3 levels of intra-die variation to model spatial correlation, as shown in Figure 3 . Accordingly, each gate k was randomly allocated a location on a 4x4 grid, which determined the random variables'associated with that gate along the hierarchy. Process variability information was used for different scenarios having a total standard deviation of 10%,14% and 15% from L",,,. The computed hounds were compared with Monte Carlo simulation and worst case analysis. Monte Carlo simulation was performed for 10,OW samples. The worst case analysis assumes the total variation to be inter-die variation and computes the 99% confidence point for the circuit delay CDF by setting ALinrer at its 99% point. For each gate length random variable, a Gaussian delay distribution uuncated at the 3 sigma point, was used. Table 1 shows the results for the hound computation and refinement using multiple arrival time propagation. A total standard deviation of 14% was divided among inter-die (5.7%). intra-die with spatial correlation (8.06%) and random intra-die variation (10%). For each circuit, the total number of nodesledges (column 2) is shown. The 99% confidence points for worst case analysis (column 3), for single and multiple anival times (column 4 & 5 ) and for Monte Carlo (column 6 ) is shown. The % error between the Monte Carlo results and our approach (column 7) was 2.98% on an average. Although we only report the 99% points in Table I , the computed hounds are CDFs and allow the computation of other confidence points. Column 8 shows the runtime of our algorithm for 100 arrival times. For most circuits, the run time is very small with the maximum being 300 seconds. Table 2 shows comparisons between 99% confidence points obtained by our algorithm using LOO arrival times and Monte Carlo simulation for two different variation scenarios. In (Column 2, 3 & 4) a total standard deviation of 10% was equally divided among inter-die. intra-die and random variations. The average error for all the circuits was 2.35%. The runtimes were small, not exceeding 300 seconds. In (Column 5,6 & 7) a total standard deviation of 15% was again equally divided among the three components. Average error was 4.63% for all circuits and maximum runtime was 280 seconds. Figure 3 shows the CDFs for the proposed upper hounds with

O B B O O B O O B O O O O O O O O O O O O B O O O O O O B B B O O O O O B O O O B O O O B B O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B B O O O O O O B I O O O O B O O O O O O B O O B B O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B B O O B O O B O O O O O O O O B O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B B O O O O B O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O B B O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B B O O O O B B O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O B B O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B B O O O O O O B O O O O B O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O B B O O B O O O B O O O B O B O O O B O O B O B O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O B O O O O O O O B O O O O O B O O B O O O O O O O O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O B B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B B O B O O O O O O O O O O O O O O O O O B O O O O B O O O B O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O B B B O O O O O O O O O O O O O O O O O O O O O O B O B B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B B I O O O O O O O O O B O O O B O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O


Modeling process asymmetries with Laplace moving average Many records in environmental science exhibit asymmetries: for example in shallow water and with variable bathymetry, the sea wave time series shows front-back asymmetries and different shapes for crests and troughs. In such situation, numerical models are available but are highly CPU-time consuming. A stochastic process aimed at modeling such asymmetries has already been proposed, the Laplace Moving Average process. The objective of this study is to propose a new estimator of the defining function in a non-parametric approach. Results based on a comprehensive numerical study will be shown in order to evaluate the performances of the proposed method. Marine coastal systems are subject to loadings due to sea waves, and long time series of the loadings are often needed to carry out a study of the performances of the system, or to assess the extreme behavior. During its lifetime, this system is likely to encounters various sea states, and thus one needs to be able to simulate long and numerous series with realistic characteristics. Many such systems are located near the shore, in shallow water and with variable bathymetry. In this context, the waves are known to be non-linear, and show high asymmetries. Two kind of asymmetries can arise in this context: the top-down (vertical) asymmetry and the front-back (horizontal) asymmetry. The former describes the different behavior of troughs compared to crests, as for waves the peaks are sharp compared to the bottom which are flatter. The latter asymmetry, referred to front-back asymmetry is linked to the time irreversibility: for example, the front steepness of crests is higher than the back one. Many other situations in environmental science or in economics data also lead to such asymmetries.. The aforementioned criterion are known to be impossible to reproduce with a Gaussian process, and models with such asymmetries must be constructed. Recently, such model has been proposed ( [1, 2] ), along with a description of some of its properties ( [3, 4] ) and an estimation procedure of some of its characteristics ( [5] ). But, as we will explain in the sequel, no procedure is available to retrieve the front-back asymmetries. The goal of the study is thus to propose a new method for estimating the kernel, which is an unknown function that rules the behavior of the process, and then to study the behavior of this estimator both on simulated and realistic dataset.. The paper is organized as follows: the first section introduces the model and some of its characteristics, and in particular the high-order spectrum properties that will be used in the estimation procedure, which is the aim of the second part: this section will be devoted to the introduction of the new estimator and to the comparison with the classical approach. In a third section, we will present a simulation study to assess the performances of the new estimation procedure. A comparison with a realistic dataset will also be carried out in the fourth and last section. In this article we proposed a method to model time series that exhibits both horizontal and vertical asymmetries. The model is based on a novel versatile tool, called LMA, which involves four parameters for the marginal distribution and a function called kernel. A new estimator based on splines has been proposed to estimate this function. Thanks to an inclusive numerical study, the performances of the new model have been checked and it shows great improvement compared to the previous estimator available. Some future work is also left on the construction of a parametric model on the phase of the transfer function to ease the estimation procedure.. We also tested the proposed methodology on a time series of sea waves. We checked that both horizontal and vertical asymmetries are well respected, along with dynamical parameters such as the skewness of the Hilbert transform or the crests and troughs slopes. The procedure shows great enhancement compared to existing methods concerning LMA but it still shows some discrepancies, that seems to come from the non-linearity of the data, which was found from a test based on the bispectrum. Hence, some improvements are still needed to overcome this, by using a second-order model to enhance the model.

O O B O B I I O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Comparison

of

PI

controllers

designed

for

the

delay

model

of

TCP/AQM

networks

One

of

the

major

problems

of

communication

networks

is

congestion.

In

order

to

address

this

problem

in

TCP/IP

networks,

Active

Queue

Management

(AQM)

scheme

is

recommended.

AQM

aims

to

minimize

the

congestion

by

regulating

the

average

queue

size

at

the

routers.

To

improve

upon

AQM,

recently,

several

feedback

control

approaches

were

proposed.

Among

these

approaches,

PI

controllers

are

gaining

attention

because

of

their

simplicity

and

ease

of

implementation.

In

this

paper,

by

utilizing

the

fluid-flow

model

of

TCP

networks,

we

study

the

PI

controllers

designed

for

TCP/AQM.

We

compare

these

controllers

by

first

analyzing

their

robustness

and

fragility.

Then,

we

implement

these

controllers

in

platform

and

conduct

simulation

experiments

to

compare

their

performances

in

terms

of

queue

length.

Taken

together,

our

results

provide

a

guideline

for

choosing

a

PI

controller

for

AQM

given

specific

performance

requirements.

Routers

in

a

network

transmit

incoming

packets

to

the

destinations

over

links

which

have

finite

bandwidth.

Thus,

links

can

get

congested

if

the

amount

of

incoming

packets

exceeds

the

link

capacity.

When

there

are

congested

links

in

a

network,

the

buffers

of

the

routers

might

overflow

and,

consequently,

new

incoming

packets

might

be

lost.

To

address

the

congestion

problem

in

TCP/

IP

networks,

queue

management

and

scheduling

algorithms

are

required

at

the

routers.

The

traditional

queue

management

technique

at

a

router,

known

as

tail

drop,

sets

a

maximum

queue

length

in

terms

of

packets

and

accepts

packets

for

the

queue

until

it

overflows,

then

drops

subsequent

incoming

packets

until

the

queue

decreases.

Tail

drop

has

some

drawbacks

such

as

flow

synchronization,

link

under-utilization,

and

long

end-to-end

delay

In

order

to

overcome

these

drawbacks,

Active

Queue

Management

(AQM)

scheme

is

recommended

in

The

well-known

AQM

scheme

is

Random

Early

Detection

(RED),

which

drops

packets

with

a

probability

that

depends

on

the

average

queue

length.

Since

RED

drops

packets

by

detecting

the

congestion,

it

significantly

improves

the

link

utilization

compared

to

tail

drop

scheme.

In

addition,

the

flow-synchronization

is

eliminated

and

the

effects

of

burst

traffic

are

attenuated

However,

tuning

RED

parameters

is

a

difficult

task;

if

these

parameters

are

not

chosen

carefully,

the

performance

of

RED

can

degrade,

and,

the

system

may

become

unstable.

The

stability

of

RED

is

investigated

in

by

studying

the

maximum

value

of

the

packet

marking

probability

that

does

not

cause

instability.

As

shown

in

TCP/RED

system

becomes

unstable

if

the

round-trip

delay

and

link

capacity

increase

significantly,

and/or

the

number

of

TCP

sessions

decreases

drastically..

In

order

to

obtain

better

performance

compared

to

RED,

by

using

the

linearized

fluid-flow

model

of

TCP

proposed

in

several

feedback

control

based

advanced

AQM

controllers

are

proposed

in

the

literature

e.g.,

and

references

therein.

In

an

H

AQM

controller

was

constructed

by

solving

two-block

H

minimization

problem

to

regulate

the

queue

length

against

the

variations

of

the

plant

parameters.

By

using

the

l-synthesis

approach

in

an

H

AQM

controller

was

designed

considering

delay-free

part.

In

by

designing

a

robust

observer,

an

H

state

feedback

controller

was

designed

to

solve

the

same

problem.

In

an

H

state

feedback

was

designed

in

order

to

solve

the

problem

considering

also

the

disturbances

on

the

available

bandwidth.

However,

it

appears

that

these

proposed

controllers

are

not

easy

to

-see

front

matter

Ó

Elsevier

B.V.

All

rights

reserved.

implement

in

real

networks

due

to

their

computational

complexities..

In

a

PI

AQM

controller

design

was

proposed

by

using

the

small-gain

theorem.

It

was

shown

there

that

PI

controllers

provide

good

responses

in

achieving

AQM

performance

requirements.

Based

on

this

and

their

ease

of

implementation

in

real

networks,

several

PI

AQM

controller

designs

have

been

proposed

following

those

works,

see

for

instance,

Note

that,

from

the

practical

implementation

of

a

controller,

it

is

required

to

keep

the

stability

of

the

closed-loop

system

under

round-off

errors

during

implementation.

A

controller

for

which

the

closed-loop

system

can

be

destabilized

by

small

perturbations

in

the

controller

coefficients

is

said

to

be

fragile

(see,

e.g.,

There

are

only

a

few

studies

addressing

the

fragility

problem

of

AQM

controllers.

To

the

best

of

the

authors'

knowledge,

the

first

study

was

performed

in

where

a

method

to

compute

the

largest

available

intervals

for

the

PI

controllers

parameters

have

been

developed.

Recently

in

using

the

complete

characterization

of

the

set

of

all

stabilizing

PI

controllers

and

its

corresponding

geometric

properties,

a

new

method

for

tuning

the

parameters

of

PI/AQM

controllers

has

been

proposed.

Such

an

approach

allows

us

to

design

a

PI

controller

stabilizing

the

network

against

perturbations

in

the

network

parameters.

In

addition,

since

the

approach

gives

a

simple

procedure

to

determine

the

controller

coefficients

providing

the

maximum

parametric

stability

margin

in

the

controller's

gains

space,

the

designed

PI

controller

stabilizes

the

network

also

against

the

perturbations

on

the

coefficients

of

the

controllers..

Although

PI

controllers

are

widely

used

in

many

control

applications

including

complicated

systems

(see

e.g.,

there

does

not

exist

a

generally

accepted

tuning

methodology.

In

addition,

determining

the

PI

parameters

is

a

difficult

task

for

many

applications

As

pointed

out

in

several

surveys

(see

and

references

therein),

a

high

percentage

of

PI

controllers

have

poor

performances

in

many

applications,

due

to

bad

controller

tuning.

Many

tuning

methods

do

not

consider

some

restrictions

such

as

unmodelled

dynamics,

non-linearities,

and

presence

of

delay.

Another

reason

for

the

performance

degradation

of

the

PI-controllers

is

the

uncertainties

in

the

controller

components

due

to

the

aging

problem.

This

means

that

fragility

of

the

controller

should

be

taken

into

account..

In

this

paper,

we

compare

several

PI

controllers

designed

for

TCP/AQM

considering

some

performance

requirements

with

arising

problems

in

practice

such

as

fragility

and

robustness.

Some

of

these

PI

controllers

are

currently

available

for

TCP/AQM

given

in

and

the

other

PI

controllers

are

designed

in

the

paper

by

utilizing

the

approaches

in

The

designed

PI

controllers

are

based

on

considering

the

transfer

function

of

the

linearized

model

of

TCP

as

an

integrating

system

or

a

second

order

system

with

delay.

In

order

to

compare

the

robustness

and

fragility

of

the

controllers,

the

stability

region

of

all

stabilizing

PI

AQM

controllers

for

the

considered

network

presented

in

is

utilized.

For

a

performance

comparison,

the

controllers

are

implemented

in

and

validated

under

different

realistic

scenarios

considering

various

performance

metrics..

It

is

worth

mentioning

that

there

also

are

propositions

of

PD

and

PID

controllers

for

AQM

schemes,

see

for

instance,

However,

to

the

best

of

the

authors'

knowledge,

the

boundary

of

the

stability

region

in

the

controller's

parameters

space

of

such

controllers

is

not

completely

known

and,

therefore,

an

appropriate

comparison

of

robustness

and

fragility

issues

can

not

be

made

as

we

performed

here

for

PI

AQM

controllers..

The

remainder

of

the

paper

is

organized

as

follows.

The

mathematical

model

of

the

TCP

fluid-flow

model

is

given

in

Section

Various

PI

controller

design

methods

for

AQM

schemes

are

summarized

in

Section

Section

provides

a

theoretical

analysis

of

these

PI

controllers

as

well

as

simulation

results

comparing

their

performance

in

platform.

Concluding

remarks

are

presented

in

Section

In

this

paper,

a

comparison

of

various

PI

controllers

designed

for

the

delay-model

of

TCP/AQM

is

performed.

The

compared

PI

controllers

are

the

ones

currently

available

for

TCP/AQM

proposed

in

and

the

designed

ones

for

TCP/AQM

in

the

current

paper

using

the

approaches

given

in

It

should

be

noted

that,

it

is

possible

to

find

different

PI

controller

design

approaches

for

TCP/

AQM

than

the

presented

ones

in

the

paper.

However,

we

believe

that

the

chosen

PI

controller

design

approaches

are

more

appropriate

considering

the

linear

model

of

TCP/AQM

among

the

others

in

the

literature.

The

comparison

in

the

paper

is

based

on

the

fragility,

robustness,

and

performance

issues

of

the

controllers.

For

a

comprehensive

and

realistic

comparison,

we

implemented

the

controllers

in

for

different

traffic

scenarios

and

validated

the

results

considering

some

AQM

performance

objectives..

Our

analysis

showed

that

the

PI

controllers

of

are

less

fragile

compared

to

the

controllers

designed

by

the

approach

of

Zeigler-Nichols,

and

On

the

other

hand,

the

controllers

proposed

by

are

more

robust

compared

with

the

rest

because

their

proportional

or

integral

gain

margins

are

relatively

large.

Additionally

we

note

that

the

controller

designed

by

the

methods

of

have

also

good

robustness

property

in

the

sense

that

they

limit

the

sensitivity

function.

The

simulation

results

indicate

that,

in

general,

robust

controllers

have

better

link

utilization

and

provide

low

jitter,

while

the

controllers,

which

may

not

suffer

''fragility''

problem,

provide

smaller

packet

dropping..

Note

that

the

approaches

of

are

for

the

simplified

model

of

the

system

K

which

may

result

in

conservativeness.

In

addition,

since

the

approach

of

is

based

on

the

small-gain

theorem,

like

another

conservativeness

arises.

The

controllers

proposed

by

can

also

be

considered

conservative

because

they

are

designed

using

a

first

order

approximation

of

the

transfer

function.

The

approach

of

however,

gives

the

set

of

all

stabilizing

controllers,

and

allows

us

to

measure

the

parametric

stability

margins..

In

conclusion,

we

have

identified

which

PI

controllers

to

choose

for

different

performance

and

robustness

metrics

in

AQM.

As

in

all

other

application

areas,

these

controllers

form

a

baseline

for

an

initial

design;

depending

on

the

performance

requirements

of

a

particular

network,

PI

controllers

can

be

modified

for

further

improvements.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A

Distributed

Mincut/Maxflow

Algorithm

Combining

Path

Augmentation

and

Push-Relabel

We

propose

a

novel

distributed

algorithm

for

the

minimum

cut

problem.

Motivated

by

applications

like

volumetric

segmentation

in

computer

vision,

we

aim

at

solving

large

sparse

problems.

When

the

problem

does

not

fully

fit

in

the

memory,

we

need

to

either

process

it

by

parts,

looking

at

one

part

at

a

time,

or

distribute

across

several

computers.

Many

MINCUT/MAXFLOW

algorithms

are

designed

for

the

shared

memory

architecture

and

do

not

scale

to

this

setting.

We

consider

algorithms

that

work

on

disjoint

regions

of

the

problem

and

exchange

messages

between

the

regions.

We

show

that

the

region

push-relabel

algorithm

of

Delong

and

Boykov

(A

scalable

graph-cut

algorithm

for

N-D

grids,

in

CVPR,

uses

Θ(n

rounds

of

message

exchange,

where

n

is

the

number

of

vertices.

Our

new

algorithm

performs

path

augmentations

inside

the

regions

and

push-relabel

style

updates

between

the

regions.

It

uses

asymptotically

less

message

exchanges,

O(B

where

B

is

the

set

of

boundary

vertices.

The

sequential

and

parallel

versions

of

our

algorithm

are

competitive

with

the

state-ofthe-art

in

the

shared

memory

model.

By

achieving

a

lower

amount

of

message

exchanges

(even

asymptotically

lower

in

our

synthetic

experiments),

they

suit

better

for

solving

large

problems

using

a

disk

storage

or

a

distributed

system.

Minimum

s-t

cut

(MINCUT)

is

a

classical

combinatorial

problem

with

applications

in

many

areas

of

science

and

engineering.

This

research

was

motivated

by

the

wide

use

of

MINCUT/MAXFLOW

problems

in

computer

vision,

where

large

sparse

instances

need

to

be

solved.

We

start

by

a

more

detailed

overview

of

models

and

optimization

techniques

in

vision,

where

the

MINCUT

problem

is

employed

and

give

examples

of

our

test

problems.

We

developed

a

new

algorithm

for

MINCUT

problem

on

sparse

graphs,

which

combines

augmenting

paths

and

pushrelabel

approaches.

We

proved

the

worst

case

complexity

guarantee

of

O(|B|

sweeps

for

the

sequential

and

parallel

variants

of

the

algorithm

(S/P-ARD).

While

there

are

many

algorithms

in

the

literature

with

complexities

in

terms

of

elementary

arithmetic

operations

better

than

we

could

possibly

prove,

we

showed

that

our

algorithms

are

fast

and

competitive

in

practice,

even

in

the

shared

memory

model.

We

proposed

an

improved

algorithm

for

local

problem

reduction

(Sect.

and

determined

that

most

of

our

test

instances

are

difficult

enough

in

the

sense

that

very

few

vertices

can

be

decided

optimally

by

looking

at

individual

regions.

The

result

that

S/P-ARD

solves

test

problem

in

few

tens

of

sweeps

is

thus

non-trivial.

We

also

gave

a

novel

parallel

version

of

the

region

push-relabel

algorithm

of

Delong

and

Boykov

and

a

number

of

auxiliary

results,

relating

our

approach

to

the

state-of-the..

Both

in

theory

and

practice

(randomized

test),

S-ARD

has

a

better

asymptote

in

the

number

of

sweeps

than

the

push-relabel

variant.

Experiments

on

real

instances

showed

that

when

run

on

a

single

CPU

and

the

whole

problem

fits

into

the

memory,

S-ARD

is

comparable

in

speed

with

the

non-distributed

BK

implementation,

and

is

even

significantly

faster

in

some

cases.

When

only

a

single

region

is

loaded

into

memory

at

a

time,

S-ARD

uses

much

fewer

disk

I/O

than

S-PRD.

We

also

demonstrated

that

the

running

time

and

the

number

of

sweeps

are

very

stable

with

respect

to

the

partition

of

the

problem

into

up

to

regions.

In

the

parallel

mode,

using

CPUs,

P-ARD

achieves

a

relative

speedup

of

about

times

over

S-ARD

and

uses

just

slightly

larger

number

of

sweeps.

P-ARD

compares

favorably

to

other

parallel

algorithms,

being

a

robust

method

suitable

for

a

use

in

a

distributed

system.

Our

algorithms

are

implemented

for

generic

graphs.

Clearly,

it

is

possible

to

specialize

the

implementation

for

grid

graphs,

which

would

reduce

the

memory

consumption

and

might

reduce

the

computation

time

as

well..

A

practically

useful

mode

could

be

actually

a

combination

of

a

parallel

and

sequential

processing,

when

several

regions

are

loaded

into

the

memory

at

once

and

processed

in

parallel.

There

are

several

particularly

interesting

combinations

of

algorithm

parallelization

and

hardware,

which

may

be

exploited:

parallel

on

several

CPUs,

parallel

on

several

network

computers,

sequential,

using

Solid

State

Drive,

sequential,

using

GPU

for

solving

region

discharge..

There

is

the

following

simple

way

how

to

allow

region

overlaps

in

our

framework.

Consider

a

sequential

algorithm,

which

is

allowed

to

keep

regions

in

memory

at

a

time.

It

can

then

load

pairs

of

regions

and

alternate

between

the

regions

in

a

pair

until

both

are

discharged.

With

PRD,

this

is

efficiently

equivalent

to

discharging

twice

larger

regions

with

a

overlap

and

may

significantly

decrease

the

number

of

sweeps

required.

In

the

case

of

a

grid,

it

would

take

times

more

regions

to

allow

overlaps

in

all

dimensions.

However,

to

meet

the

same

memory

limit,

the

regions

have

to

be

times

smaller.

It

has

to

be

verified

experimentally

whether

it

is

beneficial.

In

fact,

the

RPR

implementation

of

Delong

and

Boykov

uses

exactly

this

strategy:

a

dynamic

region

is

composed

out

of

a

number

of

smaller

blocks

and

blocks

are

discharged

until

the

whole

region

is

not

discharged.

It

is

likely

that

with

this

approach

we

could

further

reduce

the

disk

I/O

in

the

case

of

the

streaming

solver.

We

plan

to

provide

also

a

distributed

MPIbased

implementation

as

well

as

address

the

question

of

dynamic

updates

of

the

problem

and

implement

a

more

efficient

input-output

interface

for

the

use

in

real

applications.

The

memory-efficient

representation

of

graphs

having

repetitive

structure

is

also

possible.

The

results

are

summarized

in

Table

The

time

reported

is

the

wall

clock

time

passed

in

the

calculation

phase,

not

including

any

time

for

graph

construction.

The

number

of

sweeps

for

DD

has

the

same

meaning

as

for

P-ARD/PRD,

it

is

the

number

of

times

all

regions

are

synchronously

processed.

RPR

however

is

asynchronous

and

uses

dynamic

regions.

For

it,

we

define

sweeps

Comparing

to

Table

we

see

that

P-ARD

on

CPUs

is

about

times

faster

than

S-ARD.

The

speed-up

over

BK

varies

from

on

to

more

than

on

gargoyle..

We

see

that

DD

gets

lucky

some

times

and

solves

the

problem

really

quickly,

but

often

it

fails

to

terminate.

We

also

observe

that

our

variant

of

P-PRD

(based

on

highest

first

selections

rule)

is

a

relatively

slow,

but

robust

distributed

method.

RPR,

which

is

based

on

LIFO

selection

rule,

is

competitive

on

the

segmentation

problems

but

is

slow

on

other

problems,

despite

its

compile-time

optimization

for

the

particular

graph

structure.

It

is

also

uses

relatively

higher

number

of

blocks,

The

version

we

tested

always

returned

the

correct

flow

value

but

often

a

wrong

(non-optimal)

cut.

Additionally,

for

connected

it

failed

to

terminated

within

hour.

A

variant

of

path

augmentation

algorithm

was

shown

by

Boykov

and

Kolmogorov

to

have

the

best

performance

on

computer

vision

problems

among

sequential

solvers.

There

were

several

proposals

how

to

parallelize

it.

Partially

distributed

implementation

(Liu

and

Sun

augments

paths

within

disjoint

regions

first

and

then

merges

regions

hierarchically.

In

the

end,

it

still

requires

finding

augmenting

paths

in

the

whole

problem.

Therefore,

it

cannot

be

used

to

solve

a

large

problem

by

distributing

it

over

several

computers

or

by

using

a

limited

memory

and

a

disk

storage.

For

the

shared

memory

model

Liu

and

Sun

reported

a

near-linear

speed-up

with

up

to

CPUs

for

and

segmentation

problems.

Strandmark

and

Kahl

obtained

a

distributed

algorithm

using

a

dual

decomposition

approach.

The

subproblems

are

MINCUT

instances

on

the

parts

of

the

graph

(regions)

and

the

master

problem

is

solved

using

the

subgradient

method.

This

approach

requires

solving

MINCUT

subproblems

with

real

valued

capacities

and

does

not

have

a

polynomial

bound

on

the

number

of

iterations.

The

integer

algorithm

proposed

by

Strandmark

and

Kahl

is

not

guaranteed

to

terminate.

Our

experiments

(Sect.

showed

that

it

did

not

terminate

on

some

of

the

instances

in

sweeps.

In

Sect.

we

relate

dual

variables

in

this

method

to

flows..

The

push-relabel

algorithm

(Goldberg

and

Tarjan

performs

many

local

atomic

operations,

which

makes

it

a

good

choice

for

a

parallel

or

distributed

implementation.

A

distributed

version

(Goldberg

runs

in

O(n

time

using

O(n)

processors

and

O(n

m)

messages,

where

n

is

the

number

of

vertices

and

m

is

the

number

of

edges

in

the

problem.

However,

for

a

good

practical

performance

it

is

crucial

to

implement

gap

relabel

and

global

relabel

heuristics

(Cherkassky

and

Goldberg

The

global

relabel

heuristic

can

be

parallelized

(Anderson

and

Setubal

but

it

is

difficult

to

distribute.

We

should

note,

however,

that

the

global

relabel

heuristic

was

not

essential

in

the

experiments

with

computer

vision

problems

we

made

(Sect.

Delong

and

Boykov

proposed

a

coarser

granulation

of

push-relabel

operations,

associating

a

subset

of

vertices

(a

region)

to

each

processor.

Push

and

relabel

operations

inside

a

region

are

decoupled

from

the

rest

of

the

graph.

This

allows

to

process

several

non-interacting

regions

in

parallel

or

run

in

a

limited

memory,

processing

few

regions

at

a

time.

The

gap

and

relabel

heuristics,

restricted

to

the

regions

(Delong

and

Boykov

are

powerful

and

distributed

at

the

same

time.

Our

work

was

largely

motivated

by

Delong

and

Boykov

and

the

notice

that

their

approach

might

be

extendible

to

augmenting

path

algorithms..

However,

our

first

attempt

to

prioritize

augmentation

to

the

boundary

vertices

by

the

shortest

distance

to

the

sink

did

not

lead

to

a

correct

algorithm.

In

this

section,

we

first

discuss

heuristics

for

improving

the

distance

labeling

(making

it

closer

to

the

true

distance

at

a

cheap

cost)

commonly

used

in

the

push-relabel

framework.

They

are

essential

for

the

practical

performance

of

the

algorithms.

We

then

describe

our

base

implementations

of

S-ARD/S-PRD

and

the

solvers

they

rely

on.

In

the

next

section,

we

describe

an

efficient

implementation

of

ARD,

which

is

more

sophisticated

but

has

a

much

better

practical

performance.

All

of

the

labeling

heuristics

can

only

increase

the

labels

and

preserve

validity

of

the

labeling.

Therefore,

they

do

not

break

theoretical

properties

of

the

respective

algorithms.

All

experiments

were

conducted

on

a

system

with

Intel

Core

Quad

Hz,

GB

memory,

Windows

XP

bit

and

Microsoft

VC

compiler.

Our

implementation

and

instructions

needed

to

reproduce

the

experiments

can

be

found

at

We

conducted

series

of

experiments:.

-Synthetic

experiments,

where

we

observe

general

dependencies

of

the

algorithms,

with

some

statistical

significance,

i.e.

not

being

biased

to

a

particular

problem

instance.

It

also

serves

as

an

empirical

validation,

as

thousands

of

instances

are

solved.

Here,

the

basic

implementation

of

S-ARD

was

used.

-Sequential

competition.

We

study

sequential

versions

of

the

algorithms,

running

them

on

real

vision

instances.

Only

a

single

core

of

the

CPU

is

utilized.

We

fix

the

region

partition

and

study

how

much

disk

I/O

it

would

take

to

solve

each

problem

when

only

one

region

can

be

loaded

in

the

memory

at

a

time.

In

this

and

the

next

experiment

we

used

the

efficient

implementation

of

ARD.

Note,

in

the

preceding

publication

(Shekhovtsov

and

Hlavac

we

reported

worse

results

with

the

earlier

implementation.

-Parallel

competition.

Parallel

algorithms

are

tested

on

the

instances

which

can

fully

fit

in

GB

of

memory.

All

cores

of

the

CPU

are

allowed

to

be

used.

We

compare

our

algorithms

with

two

other

state-of-the-art

distributed

implementations.

O B O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O


On the relationship between the traceability properties of Reed-Solomon codes Abstract. Fingerprinting codes are used to prevent dishonest users (traitors) from redistributing digital contents. In this context, codes with the traceability (TA) property and codes with the identifiable parent property (IPP) allow the unambiguous identification of traitors. The existence conditions for IPP codes are less strict than those for TA codes. In contrast, IPP codes do not have an efficient decoding algorithm in the general case. Other codes that have been widely studied but possess weaker identification capabilities are separating codes. It is a well-known result that a TA code is an IPP code, and an IPP code is a separating code. The converse is in general false. However, it has been conjectured that for Reed-Solomon codes all three properties are equivalent. In this paper we investigate this equivalence, providing a positive answer when the number of traitors divides the size of the ground field. Distributing digital contents is an activity which is prone to an undesirable attack: unauthorized redistributions performed by dishonest users. To fight against this, the distributor can apply the fingerprinting technique. This technique consists in making each copy of the content unique by embedding a mark before delivering it. The embedding process must satisfy several properties. First, the process must be robust. This means that it should not be possible to remove or degrade the mark once it is embedded without rendering the content unusable. Second, the marked content must not differ substantially from the original content and must retain the same functionality.. The distributor assigns a unique mark to each user, and delivers the marked content correspondingly. In other words, the users will receive different objects that will have identical appearance from their perspective. Due to the uniqueness of the objects, they are discouraged from redistributing their own copy. However, several dishonest users, called traitors, may form a coalition, compare their copies and create a pirated copy. This is known as a collusion attack. The underlying idea is that the mark contained in the pirated copy hides the identity of the traitors. What is worse, that mark could coincide with that of a user outside the coalition and frame that innocent user. A traitor tracing scheme should be prudently designed so that this situation never happens ideally, or it happens with arbitrarily small probability. A stronger traitor tracing scheme would provide, besides that, a mechanism to identify at least one of the traitors. This goal is achieved if the set of marks chosen by the distributor constitutes a code with tracing properties. Such codes are also known as fingerprinting codes and the marks are called fingerprints.. Codes with tracing properties can be classified according to their capabilities in terms of protecting innocent users and identifying traitors. These properties are not equivalent in the general case. However, in this paper we show the equivalence of some of these properties for numerous families of Reed-Solomon codes.. The paper is organized as follows. In the next section we introduce the topic and the notation, and present some previous results. In Section 3 we present the main results of the paper, showing the equivalence of some tracing properties for Reed-Solomon codes, when the coalition size divides the size of the ground field. Next, in Section 4 we provide an illustrative example and a table summarizing the results. Finally we present the conclusions.

O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On the Hardness of Offline Multi-objective Optimization It has been empirically established that multiobjective evolutionary algorithms do not scale well with the number of conflicting objectives. This paper shows that the convergence rate of all comparison-based multi-objective algorithms, for the Hausdorff distance, is not much better than the convergence rate of the random search under certain conditions. The number of objectives must be very moderate and the framework should hold the following assumptions: the objectives are conflicting and the computational cost is lower bounded by the number of comparisons is a good model. Our conclusions are: (i) the number of conflicting objectives is relevant (ii) the criteria based on comparisons with random-search for multi-objective optimization is also relevant (iii) having more than 3-objectives optimization is very hard. Furthermore, we provide some insight into cross-over operators. Many evolutionary algorithms are comparison-based, in the sense that the only information coming from the objective function and used by the algorithm is the results of binary comparisons between fitness-values. Teytaud and Gelly (2006) has shown that this limitation has non-trivial consequences in continuous mono-objective optimization. We here apply similar techniques in order to show that comparison-based MOO has strong limitations in terms of convergence rates, when it is applied to continuous problems in which (i) the computational cost is well approximated by the number of comparisons (ii) only binary comparisons are used (iii) all objectives are conflicting (iv) the number of objectives is high. This is not simply a general negative result, as it emphasizes some tricks for avoiding these limitations, such as removing non-conflicting objectives as in Brockhoff and Zitzler (2006) or using non-binary comparisons as done in Roudenko and Schoenauer (2004) through "informed" cross-over.. Consider f itness = (f itness 1 , . . . , fitness d ) some real-valued objective functions (to be maximized) on a same domain D. A point y ∈ D dominates (or Paretodominates) a point x ∈ D if for all i ∈ [[1, d] ], f itness i (y) ≥ f itness i (x), and for at least one i 0 , f itness i0 (y) > fitness i0 (x) (i.e., y is at least as good as x for each objective and y is better than x for at least one objective). This is denoted by y x. We denote by y x the fact that ∀i ∈ [[1, d] ], fitness i (y) ≥ f itness i (x). We use the same notation for points in the so-called fitness-space: f itness (x) f itness(y) (resp. ) if and only if x y (resp. x y). Also, we say that a set A dominates a set B if ∀b ∈ B, ∃a ∈ A; a b; This is denoted by A B. A point in D is said Pareto-optimal if it is not Pareto-dominated by any other point in D. Multi-objective optimization (MOO,Bhattacharyya (2000) ; Miettinen (1999); Deb (2001) ) is the search for the set of non-dominated points, i.e., {x ∈ D; y ∈ D, y x}.. ( 1) This set is called the Pareto set. Offline MOO is the search for the whole Pareto set, which is, after the optimization (offline), studied by the user. On the other hand, on-line MOO is the interactive search of an interesting point in the Pareto set; typically, such programs use a weighted average of the various objectives, and the weights are updated by the user depending on his or her preferences, during the run of the MOO. On-line MOO is in some sense easier than offline MOO: the user provides some information during the run of the MOO and this information simplifies the problem by restricting the optimization to a part of the Pareto set chosen by the user. Offline MOO and online MOO are compared in algos 1 and 2.. Algorithm 1 Offline MOO.. Init n = 0. while stopping criterion not met do Modify the population (mutation, selection, crossover, . . . ) (if n > 0) or initialize it (if n = 0). for Each newly visited point x do n ← n + 1 P n ← P n ∪ {x} end for end while Output P n (or possibly only the non-dominated points in P n , or any other subset of P n ). The main result is that the lower-bound on the computation time for all comparisonbased MOO-algorithms and the upper bound on the computation time of the random search are very close to each other when the dimension is large; this shows that no comparison-based algorithm is much better than the baseline random search, at least when the dimension is large. To strengthen the results, the upper bound is proved for a very poor random search (see the poor distribution used in the random search of theorem 1) and the lower bound is proved for a very small family of fitness functions (see theorem 2; the upper bound holds a fortiori for larger families of problems). The lower bound is proved for comparison-based algorithms; we conjecture that the same holds for all offline MOO algorithms, under some slightly stronger assumptions. Let's look at the result in a more concrete manner, emphasizing the assumptions. The criterion is a convergence to the whole Pareto-front. We consider an "offline" algorithm (see introduction), which approximates the full Pareto-front. For iterative multi-objective optimization (on-line algorithms, with interactions with the user), the result does not hold.

O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O