Degree of contribution (DoC) feature selection algorithm for structural brain MRI volumetric features in depression detection Purpose Accurate detection of depression at an individual level using structural magnetic resonance imaging (sMRI) remains a challenge. Brain volumetric changes at a structural level appear to have importance in depression biomarkers studies. An automated algorithm is developed to select brain sMRI volumetric features for the detection of depression. Methods A feature selection (FS) algorithm called degree of contribution (DoC) is developed for selection of sMRI volumetric features. This algorithm uses an ensemble approach to determine the degree of contribution in detection of major depressive disorder. The DoC is the score of feature importance used for feature ranking. The algorithm involves four stages: feature ranking, subset generation, subset evaluation, and DoC analysis. The performance of DoC is evaluated on the Duke University Multi-site Imaging Research in the Analysis of Depression sMRI dataset. The dataset consists of 115 brain sMRI scans of 88 healthy controls and 27 depressed subjects. Forty-four sMRI volumetric features are used in the evaluation. Results The DoC score of forty-four features was determined as the accuracy threshold (Acc_Thresh) was varied. The DoC performance was compared with that of four existing FS algorithms. At all defined Acc_Threshs, DoC out- performed the four examined FS algorithms for the average classification score and the maximum classification score. Conclusion DoC has a good ability to generate reduced-size subsets of important features that could yield high classification accuracy. Based on the DoC score, the most discriminant volumetric features are those from the left-brain region. Depression is the most common mental disorder worldwide and is considered a significant health problem. Stigma and patient denial, extent of clinical experience, time limitations, and reliability of psychometrics are barriers to the clinical diagnosis of depression. Various medical diagnosis systems have been developed for different medical diagnosis problems. However, such systems are generic systems that have not been specifically designed for depression detection [1, 2] . Thus, an automated system that is able to detect depression would assist medical experts in their decision-making process. One of the key components of a sMRI-based depression detection system is feature selection (FS) [1] . Existing FS algorithms are not tailored for the depression detection problem.. FS identifies the most useful features and reduces the feature dimensionality, while the most significant aspects of the data are preserved [3] . Feature subset selection (FSS) finds a reduced subset of features from a dataset described by the feature set [4] . Feature ranking (FR), also called feature weight-ing, assesses individual features and assigns to them weights according to their degree of relevance [5] .. In individual depression detection studies, very few FS methods have been used thus far. Using sMRI data, Costafreda et al. [6] used analysis of variance (ANOVA) to filter the whole-brain voxels to select the areas of maximum group difference between patients and controls. Mwangi et al. [7] implemented a t test to filter voxel-based morphometry (VBM) for identification of the voxels that differed most in major depressive disorder (MDD) patients versus healthy controls. They also investigated a wrapper FS method called RFE.. Using other brain imaging methods, e.g., functional MRI, principal component analysis (PCA) was employed by Fu et al. [8] and Marquand et al. [9] to achieve feature selection. A study by Chyzhyk et al. [10] employed lattice independent component analysis (LICA) and kernel transformation hybrid with dendritic computing classifiers. Other options include the use of multivariate approaches such as the Gaussian mixture model and partial least square which can relieve the small sample size problem, and extract feature vectors from the computation of the activation of each ROI in the brain image. De Martino et al. [11] and Craddock et al. [12] performed FS using the RFE algorithm. Zeng et al. [13] used the Kendall tau rank correlation coefficient to evaluate the power of features.. The application of sMRI in depression diagnosis is in its infancy. The available studies that have reported the detection of depression at an individual level include Costafreda et al. [6] , Nouretdinov et al. [14] , Gong et al. [15] , Mwangi et al. [7, 16] , Bao et al. [17] , and Kipli et al. [18, 19] . These available individual sMRI-based depression detection studies based on sMRI have utilized different features for detection. The features used were voxel-based morphometry, brain shape, and voxels intensity values [1, 19] .. Group-level depression studies reported that alteration of brain volumes was present at structural levels [1] . These sMRI volume analyses of various brain regions have drawn a lot of attention; however, the brain sMRI volumetric features have yet to be explored for individual depression detection. Brain volume changes occur in the hippocampus, amygdala, anterior cingulate cortex, orbitofrontal cortex, dorsolateral prefrontal cortex, subgenual prefrontal cortex, putamen, caudate, and also in the cerebrospinal fluid (CSF) [1] . Volumetric analysis refers to the measurement of the volume of selected brain regions and is carried out by summing all voxels within the traced regions of interest (ROIs). The volume measure is calculated after manual or semiautomated segmentation. For example, the hippocampus volume in the scanning space is measured by multiplying the number of voxels of the hippocampal mask by the size of the voxel of the image.. In order to utilize these sMRI volumetric features for individual depression detection, the most relevant features that contribute to depression detection need to be determined. Selection of reduced-size subsets of features while yielding significant depression detection accuracy is a key objective of the depression detection systems at an individual level. In this paper, the author proposes a new ensemble FS algorithm called the degree of contribution (DoC) algorithm. The idea incorporated in this algorithm was originally introduced in our earlier work [20] . The main focus of this algorithm is to generate FR based on the feature degree of contribution. It has been reported that an ensemble of algorithms perform better compared with a single algorithm [21, 22] . Ensemble FSs are designed to deal with the problem of small sample sizes with high dimensions. Ensemble approaches are also one of the approaches used to improve the robustness of FS algorithms and have been shown to improve the robustness and stability of final FS methods [21, 22] . In this regard, DoC modifies the ensemble approach for integration of multiple FS algorithms and multiple learning algorithms (LAs) in a single algorithm. Moreover, the proposed DoC algorithm also utilizes a hybrid search technique that combines the wrapper and filter approaches. It has often been discovered that hybrid techniques are capable of finding a good solution, even when a single technique is often trapped with an incomplete solution [23] . The proposed DoC algorithm will be further described in the proposed algorithm section.. The key contributions of this paper include: (i) introduction of an ensemble FS algorithm to calculate the degree of contribution of brain sMRI volumetric features to automated detection of depression at an individual level, (ii) identification of the most discriminant brain sMRI volumetric features for detection of depression at an individual level based on the proposed algorithm, and (iii) presentation of empirical evidence to support the improved detection accuracy of depression by the proposed algorithm compared with some existing FS algorithms. This paper presented an ensemble-based DoC algorithm consisting of four stages: feature ranking, subset generation, subset evaluation, and DoC analysis. Firstly, multiple FRs were generated using multiple FS algorithms. Secondly, the top features are used to generate the first subset. The process is repeated until the top (n − 1) features are used to form a subset. Thirdly, each generated subset is evaluated using multiple classification algorithms. Finally, the evaluation accuracy data are analyzed, and the DoC scores are calculated for individual features. The DoC score represents the importance of the feature, and if the scores are arranged in a descending order, they generate FR. The FR can be used to guide the selection of FSS.. The experimental study indicates that a higher T value imposes a better stability of FSS. The T values greater than 0.9 showed better classification performances than the T values below 0.9. The optimum T value for the proposed DoC algorithm for the selection of sMRI volumetric features remains to be explored in our future work. Further analysis on the DoC scores reveals that volumetric features from the leftbrain region are the most discriminant features for depression detection. The total DoC score from the left-brain region is always higher than the total DoC score from the right-brain region or the whole brain regardless of the Acc_Thresh value. From the experiments, the proposed DoC algorithm demonstrated great potential for application in FS of sMRI volumetric features. DoC outperformed IG, OneR, SVM, and ReliefF in terms of the average and maximum classification accuracy. Finally, it will be beneficial to expand the reported findings to larger scale studies.. See Table 5 . From Experiment I, the DoC scores for each feature are calculated based on four different Acc_Threshs: 70, 75, 80, and 85. For Acc_Thresh ≥80 %, forty of the brain volumetric features have nonzero DoC scores. When Acc_Thresh is increased to 85 %, only eighteen of the brain volumetric features have nonzero DoC scores. For Acc_Thresh ≥70 % and Acc_Thresh ≥75 %, all features have nonzero DoC scores. It is obvious that at different Acc_Thresh, the level of contribution is also different. From these varied T values, we learned that a reasonable high T value is necessary for a better separability of the DoC values for features. Furthermore, when T was set greater than 0.9, the subsets of features for Acc_Thresh of 80 % (T = 0.94) and 85 % (T = 0.99) were exactly the same for the Top 5 subset, Top 10 subset, Top 15 subset, and Top 20 subset. However, for T values below 0.9, whereAcc_Thresh is 70 % (T = 0.82) and 75 % (T = 0.88), only the Top 10 subset is exactly the same, while the other subsets (Top 5, Top 15 and Top 20) have a few dissimilar features. Thus, it can be observed that a higher T value could generate higher stability of the feature subsets. From the results in Experiment I, we conducted further analysis to determine the most dominant brain regions for the detection of depression. Table 4 provides the total DoC scores for the left, right, and whole brain calculated from the DoC scores of the their associated features as reported in Table 1 . It can be seen that regardless of the Acc_Thresh value, the volumetric features from the left-brain region are the dominant contributors for depression detection. The total DoC score of the features from the left-brain region improves as Acc_Thresh increases. The volumetric features associated with the whole-brain region are the second important contributors to depression detection, but their contribution reduces gradually with an increment in Acc_Thresh. Finally, the volumetric features associated with the right-brain region have the lowest contribution toward depression detection, and their contribution reduces drastically with an increase in Acc_Thresh. Figure 4 shows the eighteen volumetric features mapped on the left-, right-, and whole-brain regions when Acc_Thresh ≥85 %. The Top 3 features are the features from the left-brain Fig. 4 Distribution of the contributing features (Acc_Thresh ≥85) [48] region with the same DoC score of 32.73. These features are as follows: ltotgm, lhemis, and ltotcsf. The finding in this study is consistent with the findings reported in the literature associating structural alterations of the left-brain region to depression. In our published review paper [1] , we reported studies that suggested depression affect the structure of the left-brain region especially the left GM, the left hemisphere, and the left CSF. Furthermore, this study also supports the published research associating WM and total GM volume to depression [1] . Our evaluation has also identified additional potential discriminant sMRI volumetric features that are useful to differentiate healthy and depress patients. These are non-ventricular CSF volume in the whole brain (nvcsf), nonlesion GM volume in the left cerebral hemisphere (lnonlgm), and lateral ventricle volume in the whole brain (totvent). These findings have not been reported elsewhere and could provide a platform for further investigation of the biomarkers for depression detection.. From Experiment II, it can be observed that the performance of the DoC algorithm is better than that of the four existing FS algorithms. The highest accuracy was achieved by DoC for Acc_Thresh of 80 and 85 % with an accuracy of 88.23 % using the Top 15 feature subset. The maximum classification accuracy achieved by the existing FS algorithms was only 85.29 % using SVM and IG. Thereby, the DoC algorithm achieved a higher classification accuracy by at least 3 % compared with other FS algorithms. It was also observed that when the Acc_Thresh was 75 %, the DoC algorithm was still able to achieve accuracy results that are comparable with other existing FS algorithms; however, the DoC used a lower number of features to achieve this results. For example, the DoC algorithm scored the accuracy of 85.29 % using only 8 features compared with the IG algorithm that needed 15 features to achieve the same accuracy.. Although the focus of this paper has been to present a new FS algorithm, it is also important to compare the achieved classification accuracies with those of the existing works in depression detection [6, 7, [15] [16] [17] . Previously, the highest accuracy of diagnosis was reported by Mwangi et al. [7] as 90.3 %. The second highest accuracy was also reported by Mwangi et al. as 87.1 % [7] . The third highest accuracy was reported by Gong et al. as 84.65 % [15] . The rest of the published studies reported accuracies within 58-76 %. The classification accuracy achieved by the proposed DoC algorithm is comparable with those reported in the existing published studies. The highest classification accuracy result (88.23 %) achieved by the DoC algorithm outperformed four of the existing works except for that in [7] . This paper presents the first proposed FS algorithm for ranking and selection of brain sMRI volumetric features for depression diagnosis at an individual level. From this study, it can be seen that the influence of the volumetric features is not limited to the group-level analysis, but is also meaningful for detection of depression at the individual level. Another interesting finding is that the classification using volumetric features has produced better accuracy than studies using VBM features [6, 15] . The proposed DoC algorithm addresses two important interests of the experts in the disease detection systems. These are the stability of the FS and the classification performance of the FS [76, 77] . The experts are not interested in a FS method that yields very stable feature sets, but return a model with poor performance accuracy [39] .. The proposed DoC algorithm has the advantages of variance reduction in feature selection as well as bias maintenance in prediction through strategies that are incorporated in the algorithm construction. The existence or nonexistence of a feature that shows a noticeable local profile in the training data can substantially affect the feature selection result [77] . In the proposed algorithm, an efficient ranking-based weighting was established to assign a weight to each feature according to its level of importance. The level of importance is determined from the generated feature ranking using the employed FS algorithm. The main idea is to first assign a weight to each feature in the training subsets according to the estimation of the feature's relevance. The weight is assigned during the subsets generation stage whereby high-ranking features are given higher chances in the subset formation, whereas the low-ranking features are given lower chances. For example, for n number of features, the Top 1 feature is included n times meaning that it is included in all of the formed subsets while the Top n feature included one time only. This rule is applied to each of the base FS algorithms that is executed. Then, the weighted training subsets are fed to the LAs at the evaluation stage using the CV strategy. In the tenfold CV, the original sample is randomly partitioned into 10 equal-sized subsamples. Of the 10 subsamples, a single subsample is retained as the validation data for testing the model, and the remaining nine subsamples are used as the training data. Multiple rounds of CV are performed using different partitions, and the validation results are averaged over the rounds. The tenfold CV strategy was employed in order to limit the over-fitting problem that causes bias in prediction. The strategy is also useful when it becomes hard to gather similar type of sample set.. The variance of feature selection quantifies the stability of a feature selection algorithm under variation of the training data, where a lower variance indicates higher stability of the algorithm [77] [78] [79] . In the proposed DoC algorithm, low variance in feature selection is achieved through the use of multiple FS algorithms at the feature ranking stage and multiple LAs at the evaluation stage. Also, the DoC analysis stage where filtering and aggregation steps are incorporated contributes toward variance reduction. The filtering step enables discarding of unimportant/irrelevant subsets of features that do not contribute to the desired Acc_Thresh value. Then, the aggregation step computes the voting score of each feature from the success subsets. The aggregation of the data results in low variance in feature selection while supplied with varying training data. Our initial investigation shows that a high T (above 0.90) would generate a more stable DoC ranking compared with a T below 0.90 when trained using various training sets. However, the stability remains an interesting topic to be investigated in our future work. The stability of a feature selection algorithm is determined by evaluating the stability of its generated feature rankings and feature subsets. To measure the similarity between two rankings, and to measure similarity between two subsets of features, Spearman's rank correlation coefficient [80] and Jaccard Index [39] can be adopted.. As previously stated, the complexity of the proposed DoC algorithm is between that of the filter and the wrapper methods. The idea of combining filter and wrapper-based eval-uations has recently emerged [81] [82] [83] offering the advantage of reducing the computational cost. This is because the number of subsets to be evaluated is smaller than the number of subsets for a wrapper. Given that the features in this study are n = 44, if an exhaustive search was employed, the number of subsets to be evaluated will be very large, 2 n = 1.7592186e + 13 subsets evaluation. While if a wrapper-based method such as SFS [82] was employed, it carried out n 2 = 1936 wrapper evaluations. In the proposed DoC algorithm, only 640 subsets need to be evaluated in the entire search. The execution time associated with the proposed DoC algorithm was measured and demonstrated good average execution time. The total execution time needed to achieve its maximum accuracy of 88.24 % was only 3.420 ms. It is not applicable to compare the execution time of the DoC algorithm with its base algorithm. However, it would be interesting to investigate the time complexity of the proposed DoC algorithm with other state-of-art embedded FS algorithms in our future work. The optimization of the developed codes would reduce the average execution time of the system.. The potential of the brain sMRI volumetric features for depression detection was explored in this study. There is, however, a need for further work using additional population studies beyond the MIRIAD dataset in order to validate the findings in this study. It would also be beneficial to employ the DoC algorithm in selection of other types of features for depression detection. These include other features extracted from brain sMRI or other brain imaging modalities, such as functional MRI, SPECT etc. It will be also interesting to apply the DoC algorithm to other application domains. Two experiments were designed and carried out. These were as follows:. Experiment I: In this experiment, the forty-four features from each of the 115 brain scans were given to the proposed DoC algorithm. For the FR stage, we adopted four FS algorithms: OneR, IG, ReliefF, and SVM. Thus, multiple FRs were generated. In the subset generation stage, the ranked features were used to form 17 individual feature subsets. Then, multiple classifiers were trained and tested on the feature subsets using tenfold cross-validation. This process was repeated for all of the generated subsets. Next, in the DoC analysis stage, T was first defined in order to select the subsets to be included in the next stage. In this work, we set T to four different values. These were T = 0.82, 0.88, 0.94, and 0.99. Then, a number of Acc_Thresh values were calculated. The corresponding Acc_Thresh were 70 % (T = 0.82), 75 % (T = 0.88), 80 % (T = 0.94), and 85 % (T = 0.99). These reasonably high T values were selected to ensure only the best subsets were included in the aggregation step. Next, the calculation of frequency ( f ) was performed for each of the features that met these criteria. The total frequency (T f ) for the defined threshold was also computed. Finally, based on the frequency data, calculation of the DoC score using Eq. (1) was performed, and the associated results were stored.. Experiment II: In this experiment, the performance of the proposed DoC algorithm was compared against that of four single FS algorithms: OneR, IG, ReliefF, and SVM. We are comparing the feature selection algorithms in terms of classification accuracy.. The same sMRI dataset was used in this experiment. Class values were assigned as binary (0, 1) values: 0 represents healthy subjects and 1 stands for depressed subjects. The volumetric features were ranked using the four existing FS methods and the proposed DoC algorithm. Then, the following subsets were generated from each of the FR results: subsets of Top 1 to Top 10, Top 15, and Top 20. Thus, a total of 12 subsets were generated for each of the FS algorithms. Each of these subsets was divided into training and testing datasets using a 70:30 split. 70 % of the healthy controls and 70 % of the depressed subjects were used for training, while the remaining 30 % from each class was used as test sets.. For the classification performance evaluation, five classifiers were selected and used as follows: SVM with radial basis function (SVM RBF), J48, random forest (RF), random tree (RT), simple KMeans classification via clustering (KMeans). The obtained classification accuracy of each classifier was used as the evaluation criterion. This process was repeated for all subsets. The optimum classifiers' parameters were determined by choosing the parameters that maximize the classification accuracy score. We employed the implementation of the existing FS algorithms and classifiers from the open source WEKA library [75] .. The classification performance of the FS algorithms was characterized using the accuracy of the classification. Accuracy is the probability that a diagnostic test is correctly performed or the number of samples correctly classified. The accuracy of the classification process is defined as the portion of true positives and true negatives:. where TP (true positives) are correctly classified positive cases, true negatives (TN) are correctly classified negative cases, false positives (FP) are incorrectly classified negative cases and false negatives (FN) are incorrectly classified positive cases.

B I I O B I O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O B I I O O O O O O O B I O O O O B O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O B O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Small k-pyramids and the complexity of determining k Abstract. Motivated by the computational complexity of determining whether a graph is hamiltonian, we study under algorithmic aspects a class of polyhedra called k-pyramids, introduced in [Zamfirescu and Zamfirescu, Math. Nachr. 284 (2011) 1739-1747, and discuss related applications. We prove that determining whether a given graph is the 1-skeleton of a k-pyramid, and if so whether it is belted or not, can be done in polynomial time for k ≤ 3. The impact on hamiltonicity follows from the traceability of all 2-pyramids and non-belted 3-pyramids, and from the hamiltonicity of all non-belted 2-pyramids. The algorithm can also be used to determine the outcome for larger values of k, but the complexity increases exponentially with k. Lastly, we present applications of the algorithm, and improve the known bounds for the minimal cardinality of systems of bases called foundations in graph families with interesting properties concerning traceability and hamiltonicity. Determining whether a given graph is hamiltonian is a classical NP-complete problem [Karp, 1972] . Based on this, [Garey et al., 1976] showed that determining traceability is an NP-complete problem, too. Even restricted to planar, cubic, 3-connected graphs, determining hamiltonicity remains NP-complete [Garey et al., 1976] . Thus, it is also NP-complete for the class of polyhedral graphs. In this contribution, we focus on k-pyramids, a class of polyhedra which generalizes those having Halin graphs as 1-skeleta and includes pyramids and prisms. Other generalizations on Halin graphs and investigations of their hamiltonian properties have already been made by Skowrońska [1983] , Skowrońska and Sysło [1987] , Skupień [1990] , and Malik et al. [2009] .. All graphs in this paper are finite, undirected, connected and contain neither loops nor edges. Such a graph is called polyhedral if it is planar and 3-connected. (Recall that there is a bijection between polyhedral graphs and the 1-skeleta of polytopes by Steinitz's famous Theorem [Steinitz, 1922] .) In order to study a polyhedral graph G, we will need the concepts of dual graph and independent dominating set. On one hand, constructing the dual graph is, algorithmically speaking, easily dealt with. On the other hand, it was shown by Garey and Johnson [1979] that the problem of finding an independent dominating set of minimal cardinality (MIDS) is NP-complete. It remains NP-complete if restricted to line graphs [Yannakakis and Gavril, 1980] , bipartite graphs [Corneil and Perl, 1984; Irving, 1991] and dually chordal graphs [Brandstädt et al., 1998 ]. Polynomial-time algorithms exist for many families of graphs, e.g. chordal graphs [Farber, 1982] , interval and circulararc graphs [Chang, 1998 ], cocomparability graphs [Kratsch and Stewart, 1993] , asteroidal triple-free graphs [Broersma et al., 1997] , and series-parallel graphs [Pfaff et al., 1984; Grinstead and Slater, 1994] . For more details, see the excellent article by Manlove [1999] . In the same paper it was shown that MIDS, even restricted to cubic planar graphs, still is NP-complete.. There exist several results concerning exact exponential time algorithms for MIDS. For general graphs with n vertices we have O(1.3575 n ), given in [Gaspers and Liedloff, 2007] , and O * (3 n/2 ) [Liu and Song, 2006] . In the same paper they prove that for graphs with degree bounded by 3, we have O * (2 0.465n ).. Motivated by the computational complexity of determining hamiltonicity and traceability, and problem 3 raised in [Zamfirescu and Zamfirescu, 2011] , we present two algorithms with O(n 4 ) time complexity that, given a connected finite graph without loops and multiple edges, (i) determine the minimal value k for which the graph is the 1-skeleton of a k-pyramid, and (ii) whether the graph is belted or not. In the remainder of this paper, we will address the following subsequent problems for a given polyhedral input graph G: (P1) Is there a natural number k such that G is isomorphic to a k-pyramid? (P2) Compute k * , the minimal k for which (P1) holds. (P3) Let (P2) be satisfied, and k * ∈ {2, 3}. Is G non-belted?

O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Distributed vector Processing of a new local MultiScale Fourier transform for medical imaging applications Abstract-The recently developed S-transform (ST) combines features of the Fourier and Wavelet transforms; it reveals frequency variation over both space and time. It is a potentially powerful tool that can be applied to medical image processing including texture analysis and noise filtering. However, calculation of the ST is computationally intensive, making conventional implementations too slow for many medical applications. This problem was addressed by combining parallel and vector computations to provide a 25-fold reduction in computation time. This approach could help accelerate many medical image processing algorithms. The S-transform (ST) [1] , a generalization of the Fourier transform (FT), is a spectral localization transform that utilizes a frequency adapted Gaussian window to achieve optimum resolution at each frequency. Signals may be directly translated between the Fourier and S domains without loss of information, including preservation of phase information. The ST was originally developed for geophysics applications and was recently introduced for medical imaging and signal processing [2] - [5] .. The classic Fourier transform provides information about the frequency content of an entire signal or image while the ST provides a local spectrum for each signal sample. This makes the ST useful for identifying changes in frequency content over time or space. The one-dimensional (1-D) ST, applied to a signal [such as time course functional magnetic resonance imaging (fMRI) data], can be used to localize and remove noise components and artifacts [3] . The two-dimensional (2-D) ST has proven useful for filtering and artefact removal in MR imaging [3] , [4] . The 2-D ST also can provide local textural information that can be used to distinguish tissues and indicate disease activity [2] , [4] .. Unfortunately, calculation of the ST is computationally intensive. Currently, the time required to transform a typical MR image makes The ST has been shown to be useful in a variety of medical image processing tasks. Unfortunately, research is difficult, and clinical relevance of techniques is limited because of the long processing times. We have demonstrated a method for vector computation of the ST, resulting in a speed improvement of 6.8 times for a 256 2 256 pixel image, and a method for distributed computation of the ST that scales acceptably with increasing cluster size. Both techniques were implemented using standard desktop computers and networking hardware. While our single processor, unoptimzed reference implementation took over an hour and a half (5561 s) to calculate the ST of a 256 2 256 pixel image, using a four node vector-computing cluster we were able to perform the same transform in less than four minutes (221 s), over 25 times faster. Future research will focus on investigating how the ST might be accelerated further by leveraging redundancy in the S-domain. Ultimately, calculation of the ST in minutes instead of hours makes techniques for filtering, image enhancement and feature detection based on the ST practical for acute clinical applications. As the FFT is the highest order operation in the ST algorithm, FFT performance is very important to the ST as a whole, especially with larger images. The IDL ST using the Altivec FFT was about 3.5 times faster than the pure IDL implementation for a 256 2 256 image.. When combined with Altivec optimization of the code that applies the Fig. 1 . Computing the ST using a cluster of 1-GHz G4 Powermacs with gigabit Ethernet interconnections. Distributing calculation introduces some overhead, but a 2-node cluster is almost as fast as an entirely local calculation. If additional analysis (such as texture analysis) can be done in a distributed fashion and the full ST results do not need to be transmitted to one machine (nonmarshalling mode), a 2-node cluster is faster than the local ST. The distributed ST performance continues to scale to all cluster sizes tested (up to 6-node).. Gaussian window (the second highest order operation), implementation in C and hand tuning, performance increases of almost 7 times were obtained for the ST of 256 2 256 images. This brings calculation of the ST for this image size to less than 14 minutes, compared to over 90 minutes for the native IDL implementation.. We tested two configurations of the distributed ST. In the first, results from each node are returned to a receiver node (marshaled) and merged into one data structure. This approach is required for analysis that requires the full ST and cannot itself be distributed. In the second case, individual nodes do not return their results to a common node (nonmarshalled). This configuration eliminates a significant amount of network transmission and is useful for analysis algorithms that can be at least partially distributed along with the ST calculation (texture analysis for example [2] ). Fig. 1 shows the performance of the two distributed versions of the ST and the local ST for comparison. The distributed ST runs more slowly on one node than the local version due to overhead associated with scheduling and network communications, but a 2-node cluster that does not marshal its results outperforms the local ST by a factor of 1.4. A 4-node cluster is faster than the local ST for both marshaling and nonmarshaling configurations. The nonmarshalling version using four nodes calculates the ST of a 256 2 256 image in less than five minutes, a speed that is potentially clinically acceptable.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Free-Form Geometric Modeling by Integrating Parametric and Implicit PDEs Abstract-Parametric PDE techniques, which use partial differential equations (PDEs) defined over a 2D or 3D parametric domain to model graphical objects and processes, can unify geometric attributes and functional constraints of the models. PDEs can also model implicit shapes defined by level sets of scalar intensity fields. In this paper, we present an approach that integrates parametric and implicit trivariate PDEs to define geometric solid models containing both geometric information and intensity distribution subject to flexible boundary conditions. The integrated formulation of second-order or fourth-order elliptic PDEs permits designers to manipulate PDE objects of complex geometry and/or arbitrary topology through direct sculpting and free-form modeling. We developed a PDEbased geometric modeling system for shape design and manipulation of PDE objects. The integration of implicit PDEs with parametric geometry offers more general and arbitrary shape blending and free-form modeling for objects with intensity attributes than pure geometric models. We present a novel geometric modeling framework that integrates PDE solids, surfaces, and implicit PDE models for general and arbitrary shape modeling including design, deformation, sculpting, and blending. The integrated model uses elliptic PDEs to govern both geometric and intensity properties of solid objects. The PDE solid geometry can be defined by either boundary surfaces or a set of curves as generalized boundary conditions governed by PDE surface formulation. The geometric PDE solid modeling techniques allow users to manipulate objects satisfying a set of design criteria and functional requirements with lots of degrees of freedom. The incorporation of intensity attributes to solid objects provides modeling advantages of implicit functions to the integrated PDE model. We developed a set of manipulation toolkits that support both global and local deformation of PDE solids subject to various constraints. Our PDE-based geometric modeling system offers PDE-governed boundary surface manipulations to deform the solid geometry, which permit users to model and edit boundary PDE surfaces and the geometric shape of corresponding PDE solid intuitively with ease. The deformation and trimming operations inside the solid space provide a natural way of free-form modeling for objects with arbitrary topology. In addition, users can manipulate intensity distributions of the PDE solid to obtain arbitrary shape blending and deformation for embedded objects. Our unified PDE-based approach greatly expands the geometric coverage and topological flexibilities of conventional PDE solids and implicit PDE models and improves the utility of PDE solids for modeling and design applications. With this integrated framework, it is possible to simulate real world objects with material properties. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib. Our PDE-based geometric modeling system offers a freeform modeling scheme according to the mapping from parametric space to the physical space generated by the trivariate PDE. Comparing with other popular FFD techniques, our method has some unique features. First, the deformed physical space as the solution of the PDE can be viewed as a PDE space which satisfies the given equation. Different from spline-based and subdivision-based FFD techniques that require 3D control lattices/volumes to define the deforming space, only boundary surfaces are necessary to form the PDE space, and the interior information will be recovered and governed by the equation itself. Second, the PDE scheme also provides a continuous deforming space by default and modifications of any region in the space will be smoothly propagated into its neighborhood. Therefore, any object embedded in the PDE space will follow the continuous geometric distribution inside the solid space and avoid self-intersection. Third, we also provide sculpting of arbitrary interior regions for local deformation and direct manipulation similarly to other FFD techniques. Unlike most spline-based FFD techniques that have to modify the control lattices based on the direct manipulations first and then deform the rest of the object accordingly, any sculpting operation on local regions in our PDE model will directly affect neighboring areas according to the equation. However, because of the nature of FDM used in our system, interactivity is a concern when manipulating large regions. For improvement, we consider using GPUs to obtain interactive operations. In addition, the incorporation of implicit PDEs for intensity distribution inherits modeling advantages of implicit models into our system. Thus, we can provide modeling functionalities that are difficult for parametric techniques such as arbitrary shape blending and shape deformation based on intensity manipulation. Compared to algebraic shape blending, our method offers local control and direct interference because our PDE formulation allows additional constraints directly enforced in local regions.

O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O


A speculative approach to parallelization in particle swarm optimization Abstract Particle swarm optimization (PSO) has previously been parallelized primarily by distributing the computation corresponding to particles across multiple processors. In these approaches, the only benefit of additional processors is an increased swarm size. However, in many cases this is not efficient when scaled to very large swarm sizes (on very large clusters). Current methods cannot answer well the question: "How can 1000 processors be fully utilized when 50 or 100 particles is the most efficient swarm size?" In this paper we attempt to answer that question with a speculative approach to the parallelization of PSO that we refer to as SEPSO.In our approach, we refactor PSO such that the computation needed for iteration t + 1 can be done concurrently with the computation needed for iteration t. Thus we can perform two iterations of PSO at once. Even with some amount of wasted computation, we show that this approach to parallelization in PSO often outperforms the standard parallelization of simply adding particles to the swarm. SEPSO produces results that are exactly equivalent to PSO; that is, SEPSO is a new method of parallelization and not a new PSO algorithm or variant.However, given this new parallelization model, we can relax the requirement of exactly reproducing PSO in an attempt to produce better results. We present several such relaxations, including keeping the best speculative position evaluated instead of the one corresponding to the standard behavior of PSO, and speculating several iterations ahead instead of just one. We show that these methods dramatically improve the performance of parallel PSO in many cases, giving speed ups of up to six compared to previous parallelization techniques. Particle swarm optimization (PSO) has been found to be a highly robust and effective algorithm for solving many types of optimization problems (Poli 2008a; Poli et al. 2007 ). For much of the algorithm's history, PSO was run serially on a single machine. However, the world's computing power is increasingly coming from large clusters of processors. In order to efficiently utilize these resources for computationally intensive problems, PSO needs to run in parallel.. Within the last few years, researchers have begun to recognize the need to develop parallel implementations of PSO, publishing many papers on the subject. The methods they have used include various synchronous algorithms (Parsopoulos et al. 2004 ) and asynchronous algorithms (Mostaghim et al. 2006) . Parallelizing the evaluation of the objective function can also be done in some cases using standard techniques (Grama et al. 2003) , though that is not an adaption of the PSO algorithm itself and thus is not the focus of this paper.. These previous parallel techniques distribute the computation needed by the particles in the swarm over the available processors. If more processors are available, these techniques increase the number of particles in the swarm, either by adding individual particles or by adding entire new subswarms. In almost all cases, adding additional particles produces better results in the same amount of time (McNabb et al. 2009 ). In Fig. 1 we see an example of this on the well-known benchmark function Sphere (20 dimensions, reporting the average of twenty runs). In terms of the number of iterations performed (which is equivalent to wallclock time if all particles are evaluated in parallel), every time the swarm size increases, the performance improves.. However, it can be seen from the graph that once the swarm is sufficiently large, there comes a point of diminishing returns with respect to adding particles. The increase in performance seen when moving from 50 to 100 particles is roughly equivalent to the increase seen when moving from 1000 to 4000. In Fig. 2 we show the value obtained after 50,000 function evaluations (not iterations) as a function of swarm size, again for the function Sphere. Increasing the swarm size from 5 to 10 has a significant effect on the value obtained. However, increasing the swarm size from 16 to 30 makes the algorithm less efficient; that is, it reduces the progress the algorithm makes per evaluation. Other functions show similar trends, though often the optimal swarm size is slightly larger. For this reason, previous work has recommended the use of a swarm size of 50 for PSO (Bratton and Kennedy 2007) . Thus, in at least some cases, adding particles indefinitely will not yield an efficient implementation.. Our purpose is to explore the question of what to do with a thousand processors when 50 or 100 particles is the most efficient swarm size, and simply adding particles results in only incremental improvement. We thus consider PSO parallelization strategies for clusters of hundreds or thousands of processors and functions for which a single evaluation will take long enough to merit a parallelization of one particle per processor-at least hundreds of milliseconds, but perhaps several minutes or longer.. In order to solve the problem of diminishing returns, we apply the concept of speculative decomposition (Grama et al. 2003) to particle swarm optimization, using extra processors to perform two iterations of PSO at the same time. Speculative decomposition is analogous to speculative execution (also known as branch prediction), a technique commonly used in processors. Modern processors, when faced with a branch on which they must wait (e.g., a memory cache miss), guess which way the branch will go and start executing, ensuring that any changes can be undone. If the processor guesses right, execution is much farther ahead than if it had idly waited on the memory reference. If it guesses wrong, execution restarts where it would have been anyway. Thus the processor speculates about future paths of execution in an attempt to decrease overall processing time.. In this paper we show that the results of standard PSO can be reproduced exactly, 1 two iterations at a time, using a speculative approach adapted from speculative execution. We show that the standard PSO equations can be factored such that a set of speculative positions can be found which will always include the position computed in the next iteration. By computing the value of the objective function for each of the speculative positions at the same time the algorithm evaluates the objective function for the current position, it is possible to know the objective function values for both the current and the next iteration at the same time. We demonstrate this principle by implementation and show that it produces exactly the same results as standard PSO, but two iterations at a time. The resulting implementation runs efficiently on large clusters where the number of processors is much larger than a typical or reasonable number of particles, producing better results in less wall-clock time.. We refer to this parallelization technique as "speculative evaluation in particle swarm optimization," or SEPSO. It is important to note here that SEPSO is not a variant of PSO. We simply propose a new way to think about the parallelization of PSO that we show takes fewer iterations to reach the same level of fitness than previous parallelizations in many instances.. Furthermore, we show that if we relax the requirements of the algorithm, no longer demanding that it strictly reproduce the exact behavior of standard PSO, we can introduce new speculative techniques that often outperform both standard parallelizations of PSO and SEPSO. These relaxations make better use of the information obtained from the extra exploration made by the speculative function evaluations. We also explore the idea that, like branch prediction in processors, we need not speculatively evaluate all possible future positions, we can accelerate the algorithm even if we are just likely to have guessed right. By pruning the speculation to just paths that are statistically likely to reproduce the paths that are equivalent to PSO we can increase the swarm size without increasing the number of speculative evaluations. We also consider several recovery strategies for cases where the pruned set of speculative evaluations does not contain the evaluation that standard PSO would have done. A further improvement we explore is speculating several iterations ahead instead of just one, which is made possible by pruning the number of speculative evaluations. This paper is organized as follows. Section 2 describes the particle swarm optimization algorithm, and Sect. 3 gives a brief overview of previous parallelization techniques for this algorithm. Section 4 shows mathematically how speculative evaluation can be done in parallel PSO to perform two iterations at once, leaving implementation concerns to the appendices. In Sect. 5, we discuss various methods of improving the performance of speculative evaluation in PSO, all of which break the requirement of strictly reproducing the behavior of the original algorithm. Section 6 describes the experiments we ran, and Sect. 7 presents the results of those experiments. In Sect. 8 and Sect. 9 we conclude and discuss future work. We have described a new technique for using processors in parallel PSO to improve the performance of the algorithm. To our knowledge, this is the first time extra processors have been used to do anything in PSO besides increase the swarm size. In an increasingly parallel world, such advancements will prove to be crucial to the continued effectiveness of PSO.. We have detailed how to perform speculative evaluation in PSO in several different parallel architectures. Using this methodology, the behavior of the original PSO algorithm can either be exactly reproduced, two iterations at a time, or the behavior can be modified in order to improve performance. While exactly reproducing PSO sometimes uses too many extra processors to be useful, when we allow ourselves some freedom with the algorithm, we see great improvements over previous methods. We have shown results that conclusively demonstrate the superiority of our techniques for several functions over the standard practice of adding particles to the swarm when extra processors are available, giving speed ups of up to six compared to previous parallelization techniques.. What we have presented is not a new variant of the PSO equations. We presented a new parallelization technique, so we compared parallelization strategies on the same algorithm, the original PSO. Our most promising techniques do change the behavior of the PSO algorithm slightly, as do some other previously proposed parallelization techniques, such as asynchronous PSO. However, our methods are applicable to almost all PSO variants, and so a comparison using the same variant for each of the parallelization techniques tested is justified. We presented results on a separate PSO variant that give evidence for this claim.. We have given five different possible approaches to speculative evaluation, each of which has different properties. These approaches perform differently on different functions and at different swarm sizes, as would be expected by the No Free Lunch Theorem for Optimization (Wolpert and Macready 1997) . We have given a brief evaluation of the premature convergence properties of these methods on deceptive functions when a smaller number of processors is available. We have also shown evidence that when many processors are available by far the best thing to do in most cases is to speculate as far ahead as the extra processors allow. This point is shown in a single graph in Fig. 10 . In that case, the extra processors in standard parallelizations provided no appreciable benefit. In contrast, speculative methods allow additional processors to provide additional speed ups, as the more processors are available the farther ahead the algorithm can speculate.. Though our methods show great improvements on some functions, they do not work for all functions at the swarm sizes we were able to experiment with. As is commonly known, in PSO there is a trade-off between exploration and exploitation. Some functions need only minimal exploration, and some never seem to have enough. Increasing the swarm size is a natural way to increase exploration in a parallel environment. However, once "enough" exploration has been reached for any particular function, adding additional particles adds only incremental benefits. At this point, a better use of the additional processors, as we have shown, is to perform some amount of speculative evaluation.. Large parallel clusters are often required to successfully optimize practical modern problems, and the world's computing resources are increasingly coming from such clusters. To properly use PSO with these systems, a balance needs to be made between using processors to increase the swarm size and using them to increase the speed of the algorithm. This work is a first step in that direction that opens the door to many future improvements on speculative methods in the parallelization of PSO. In this work we have focused on PSO itself and not all of its variants. It remains as future work to apply speculative approaches to recent and popular PSO variants, such as the Fully Informed Particle Swarm ). While our methods will not always be immediately applicable to every variant, we are confident that some kind of speculative approach will be beneficial to the parallelization of all forms of PSO, especially as the number of processors used gets into the thousands.. We mentioned related work showing that increasing the swarm size throughout the course of the algorithm could provide improved performance over a fixed swarm size in serial PSO (Montes de Oca et al. 2010) . If this method were extended to parallel PSO, most processors would be idle in the first few iterations, while more would be utilized at the end. During iterations where there are many unutilized processors, a natural use of them would be speculative evaluation, performing two or more of those iterations at a time.. The issues of the sampling distribution of speculative relaxations, branch statistics, and why standard PSO fails at 500 dimensions were briefly mentioned in this paper. Each of those issues needs further treatment. The sampling distribution of our speculative methods could be compared to Poli's description of standard PSO's sampling distribution (Poli 2008b) . The branch statistics could be used to analyze topologies and discover why certain topologies work well on some functions but not on others; perhaps PSO performance is more dependent on the branch statistics of a combination of topology and function than on the topology itself. And discovering exactly why constricted PSO fails at 500 dimensions while Many Iterations does not could lead to improvements in the standard PSO algorithm, even when not running in parallel.. We opened the door to speculative parallelization methods in PSO and described the possible speculative evaluations to perform as an infinite tree from which branches are selected. However, we only presented a few of the countless possibilities for selecting those branches. Our methods for determining which speculative evaluations to perform were independent of the particle; all particles performed the same number and type of evaluations. Another way to allocate speculative evaluations is to somehow use the performance of each particle to determine how many and which extra evaluations it can have.. Any other optimization algorithm that only depends on current sampling positions when computing the next position to sample can be parallelized with this technique. In particular, genetic algorithms produce future generations by combining individuals from the current generation. With a large population size there would be an unwieldy amount of possible future individuals, but the potential exists to modify the algorithm to use some kind of speculative evaluation. We frequently present tables summarizing our results. In each table, we bold the "best" method, meaning that it has at least a 90% success rate and its average time to completion is faster than all other methods that have at least a 90% success rate. 4 We used a double-sided t-test to test for statistical significance in these results, and all algorithms shown in boldface have a mean time to completion that is significantly lower than that of other algorithms, with a p-value of less than 10 −5 . We first present results in Sect. 7.1 for the 20-dimensional variants of the benchmark functions we tested. We give a discussion of each function individually at 20 dimensions, as there are interesting characteristics of the algorithms that are worth discussing. In Sect. 7.2 we then give a summary of results for the 50-dimensional variants of the benchmark functions; we do not go into as much detail in our discussion as the results are very similar to those in 20 dimensions. We finish our discussion of benchmark functions with their 500-dimensional variants in Sect. 7.3, and we discuss results on the model fitting problem in Sect. 7.4. The idea of speculative decomposition in the parallelization literature is not new (Grama et al. 2003) . In the field of function optimization, simulated annealing has previously been parallelized using this technique (Witte et al. 1991 ), though we are not aware of other evolutionary or swarm-intelligence-based algorithms having been parallelized with speculative decomposition.. There have been several parallelizations of PSO presented in the literature. The improvements described in these papers come in two major areas: innovations in implementation details and innovations in the use of topology and swarm size to scale PSO to many processors.

O O O O O O B I I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Three-Dimensional Microwave Breast Imaging: Dispersive Dielectric Properties Estimation Using Patient-Specific Basis Functions Abstract-Breast imaging via microwave tomography involves estimating the distribution of dielectric properties within the patient's breast on a discrete mesh. The number of unknowns in the discrete mesh can be very large for 3-D imaging, and this results in computational challenges. We propose a new approach where the discrete mesh is replaced with a relatively small number of smooth basis functions. The dimension of the tomography problem is reduced by estimating the coefficients of the basis functions instead of the dielectric properties at each element in the discrete mesh. The basis functions are constructed using knowledge of the location of the breast surface. The number of functions used in the basis can be varied to balance resolution and computational complexity. The reduced dimension of the inverse problem enables application of a computationally efficient, multiple-frequency inverse scattering algorithm in 3-D. The efficacy of the proposed approach is verified using two 3-D anatomically realistic numerical breast phantoms. It is shown for the case of single-frequency microwave tomography that the imaging accuracy is comparable to that obtained when the original discrete mesh is used, despite the reduction of the dimension of the inverse problem. Results are also shown for a multiple-frequency algorithm where it is computationally challenging to use the original discrete mesh.Index Terms-Breast cancer, computation time, electromagnetic tomography, finite-difference time-domain (FDTD) methods, inverse problems, microwave imaging. We have presented a method for enhancing the computational efficiency of 3-D multiple-frequency microwave breast imaging. The multiple-frequency approach improves the conditioning of the inverse problem and enhances the resolution of the reconstruction. In addition, reconstruction of a frequency dependent model offers an inherent correlation between the real and imaginary parts of the complex permittivity. The importance of such behavior is evidenced by the presence of a correlation, both spatially and in magnitude, in actual tissue distributions as well as the lack of spatial correlation often seen in single-frequency reconstructions. Reducing the dimension of the inverse scattering problem relaxes the computational constraints and allows the consideration of a larger number of frequencies and measurements as compared to the solution of the problem on a fine voxel-based mesh. Specifically, we have proposed the use of patient-specific basis functions to reduce the dimension of the inverse scattering problem. The basis functions conform to the interior volume of the breast and are designed to represent details on the order of the resolution limits of the system. The conformal shape and the smoothness of these continuous basis functions avoid the discretization errors encountered by some other methods of dimensional reduction, while offering a reduction in dimension of nearly two orders of magnitude. We have showed using a single-frequency example that the resolution and profile distribution that can be reconstructed with the patient-specific basis is substantially similar to that of a high resolution voxel-based mesh. We have explored the performance of our 3-D multiple-frequency microwave breast imaging algorithm using two realistic computational testbeds. The testbeds consist of a high resolution numerical breast phantom derived from a 3-D MRI dataset, and represent two BI-RADS classifications of breast tissue distributions. The combined use of the fine detail available from the MRIs and data from the most recent and comprehensive dielectric properties studies results in highly realistic numerical breast phantoms.. Our 3-D microwave imaging results point to the challenge of tumor detection via the reconstruction of dielectric properties distributions in the breast interior. The electrically small features of the true profile are underestimated and the contribution to the reconstruction from even a scattered distribution of fibroglandular heterogeneity interferes with the ability to distinctly image the inclusion. These challenges are magnified for distributions such as the heterogeneously dense breast phantom. Irrespective of the detection challenges, the imaging results demonstrate the utility of this 3-D method to applications such as the characterization and monitoring of tissue distribution and density for all classifications of breast tissue. In order to evaluate the information available from the presence of a malignant inclusion, we have also compared reconstructions with and without the inclusion. Differential images formed by subtracting two 3-D reconstructions (with and without the inclusion) clearly and accurately locate the inclusion; 1-D transections of the two 3-D reconstructions also illustrate the impact of the inclusion on the estimated dielectric properties. It is premature to assess whether the formation of a differential microwave image for the purpose of cancer detection is clinically feasible. Baseline reconstructions of healthy breasts become less valid with time, as factors such as aging and image registration can cause a change in the distribution of normal breast tissues. Differential imaging is potentially applicable to contrast-enhanced microwave breast cancer detection, whereby contrast agents are introduced alter the dielectric properties of malignant tissue [51] , [52] . Such a method would consider two successive scans of the patient's breast before and after the introduction of the agents, such that the differential image will reveal the presence of the contrast agent that has accumulated in the tumor. We first compare the imaging performance of the single-frequency, voxel-based DBIM to that of our patient-specific basis approach. We then present results for the patient-specific basis approach using multiple-frequency DBIM, for which it is not computationally practical to implement a voxel-based approach.. In the examples considered in this section, the patient-specific basis functions are sampled on the original voxel-based mesh, although as noted in Section II-B this does not have to be the case in general. For all the numerical examples of Section V-B, we have added white Gaussian noise to the simulated measurements with a signal to noise ratio (SNR) of 60 dB relative to the total signal. This SNR is considerably higher than the SNR reported in experimental microwave breast imaging systems [1] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I I I B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A simple branching scheme for vertex coloring problems We present a branching scheme for some Vertex Coloring Problems based on a new graph operator called extension. The extension operator is used to generalize the branching scheme proposed by Zykov for the basic problem to a broad class of coloring problems, such as the graph multicoloring, where each vertex requires a multiplicity of colors, the graph bandwidth coloring, where the colors assigned to adjacent vertices must differ by at least a given distance, and graph bandwidth multicoloring, that generalizes both the multicoloring and the bandwidth coloring problems. We report some computational evidence of the new branching scheme effectiveness. Given a graph G = (V, E) and an integer k, a k-coloring of the graph G is a mapping of each vertex to a single color, such that adjacent vertices take different colors. The Minimum Graph Coloring Problem (Min-GCP), which is known to be NP-hard, consists in finding the minimum k such that a k-coloring exists. Such minimum k is the chromatic number of G and is denoted by χ(G), or simply by χ. A second graph coloring problem that generalizes Min-GCP asks to assign to every vertex a given number of colors, while satisfying the constraint that the same color cannot appear in adjacent vertices. This problem is known in the literature as the Minimum Graph Multicoloring Problem (Min-GMP), and though it can be reduced to Min-GCP on an auxiliary (much bigger) graph, it has its own interest and ad hoc algorithms can take advantage of its structure. Let χ m (G) denote the minimum k for Min-GMP. A third version of the vertex coloring problem, defined on an edge-weighted graph, consists of assigning to each vertex a set of colors such that the distance between any color assigned to every pair of adjacent vertices is at least equal to the edge weight. This version of the problem is known in literature as the Bandwidth-Multicoloring Problem, or Bandwidth Coloring if every vertex requires a single color. Let χ mb (G) and χ b (G) denote the minimum number of colors k for Min-GBMP and for the simple Bandwidth Coloring problem, respectively. For a recent survey on Vertex Graph Coloring problems, see [1] .. Branching schemes in implicit enumeration approaches as Branch&Bound, Branch&Cut and Branch&Price, are of utmost importance for the efficiency of the algorithms, especially when strongly structured problems are at hand. The addition of general branching constraints often risks to spoil the structure of the problem. Zykov in 1949 proposed a simple bipartite branching scheme for Min-GCP whose main feature is to generate in both branches two smaller instances of the same problem. For the other coloring problems, as Min-GMP, Min-GBMP and Bandwidth Coloring problem, to the best of our knowledge, no branching schemes with this characteristic have been proposed.. In the Branch&Price algorithm proposed in [2] for Min-GMP, the adopted branching schemes are quite involved and mainly fix variables corresponding to columns of the extended formulation to 0 or to 1. In the case of Frequency Assignment Problems (e.g., [3] ), which basically are Min-GBMP problems on graphs with a special structure, the branching is carried out on binary variables that indicate the assignment of colors (frequencies) to vertices (cells). More involved branching schemes are also devised [4, 5] , but they are always derived from the mathematical programming formulation of the problem.. In the following section, we introduce a new operator generalizing the basic Zykov branching scheme that exploits the problem structure, and maintains the graph size small. Some computational evidence of the branching scheme effectiveness is reported.

O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O


Research and performance evaluation of data replication technology in distributed storage systems Replication is a key technology of distributed storage systems. In this paper, an indirect replication algorithm is proposed following the intrinsic characteristic of distributed storage systems and the peer-to-peer model. In the indirect replication algorithm, the data object is partitioned into several data blocks. These data blocks are encoded in order that there is data redundancy between data blocks. Comparing with the traditional replication algorithm, the indirect replication algorithm has less granularity of replication, less bandwidth and storage costs, and provides higher availability, durability, and security. The performance evaluation shows that the encoding and decoding times are proportional to the data size, and that the irregular cascade bipartite graphs are of great advantage in improving the success ratio of data recovery. Finally, if the number of data blocks used to recover the data object is larger than a certain value, the success ratio of data recovery approaches 100%. With the explosive growth of network bandwidth, computing power, and storage capacity, aggregating the geographically distributed heterogeneous storage resources to form a virtual storage space and to provide secure efficient data storage service becomes a challenging research topic.. The hierarchical model and the peer-to-peer model can be used to build distributed storage systems. In the hierarchical model, layers of control and abstraction to stitch together distributed and disparate storage providers into a single virtual entity axe used. In the peer-to-peer model, many peers work together in a symmetrical way to provide storage services; clients need not install any new software to consume basic storage services. The distributive, autonomy, dynamic, scalabi]ity, and flexibility of the peer-to-peer model can be utilized to improve the efficiency of distributed heterogeneous storage resources. Therefore, many research projects [1] [2] [3] [4] [5] [6] [7] on distributed storage systems are based on the peer-to-peer model, such as [1] [2] [3] [4] .. Many distributed storage systems utilize the traditional replication technology to achieve the high availability and durability of distributed storage systems [4, 5, [8] [9] [10] . In the traditional replication technology, the replica size is equal to the size of data object. In order to guarantee the availability and durability, the number of replicas needs to be increased while the system size increased. However, to some extent, more replicas need larger network bandwidth and storage capacity, especially for large data objects. On the other hand, the increasing number of replicas also influences data security. Therefore, the traditional replication technology is not adequate for data management in distributed storage systems.. This paper addresses the above problems and is organized as below. After a brief introduction in Section 1, Section 2 describes the indirect replication algorithm. Section 3 presents the results of performance evaluation. Section 4 provides a summary of our research work. This paper proposes an indirect replication algorithm and compares it with traditional replication algorithms. It is suggested that the indirect replication algorithm has less bandwidth and storage costs, and can provide higher availability, durability, and security. The results of performance evaluation show that the encoding time and decoding time is proportional to the size of the data object, and that the irregular cascade bipartite graph is of great advantage in improving the success ratio of data recovery. In general, when the number of data blocks used to recover a data object is larger than a certain value, the success ratio of data recovery approaches 100%.

O O B I O O B O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O