Class-switching Neural Network Ensembles This article investigates the properties of class-switching ensembles composed of neural networks and compares them to class-switching ensembles of decision trees and to standard ensemble learning methods, such as bagging and boosting. In a class-switching ensemble, each learner is constructed using a modified version of the training data. This modification consists in switching the class labels of a fraction of training examples that are selected at random from the original training set. Experiments on 20 benchmark classification problems, including real-world and synthetic data, show that class-switching ensembles composed of neural networks can obtain significant improvements in the generalization accuracy over single neural networks and bagging and boosting ensembles. Furthermore, it is possible to build mediumsized ensembles (≈ 200 networks) whose classification performance is comparable to larger class-switching ensembles (≈ 1000 learners) of unpruned decision trees. Ensemble methods in machine learning attempt to induce a collection of diverse predictors which are both accurate and complementary, so that, when the decisions of the different learners are combined, better prediction accuracy on previously unseen data is obtained. The goal is to generate from a given training dataset a collection of diverse predictors whose errors are uncorrelated. Ensembles built in this manner often exhibit significant performance improvements over a single predictor in many regression and classification problems. Ensembles can be built using different base classifiers: decision stumps [27] decision trees [11, 25, 3, 4, 27, 22, 2, 9, 26, 6] , neural networks [14, 31, 23, 28, 22, 26, 8] , support vector machines [17, 29, 30] , etc.. To create diversity, ensemble methods introduce perturbations at some stage in the generation of individual predictors. These modifications, which often involve injecting some amount of randomness, can be either in the algorithm that is used to build the base learners or in the training data used as input for the induction process. The motivation for introducing an aleatory element in the base learning algorithm is that different executions of the randomized training algorithm on the same instance of a learning problem should generate diverse classifiers. For example, in randomization [18] , the base learners are decision trees generated with a modified tree construction algorithm. This algorithm computes the best 20 splits for every internal node and then chooses one of them at random. Another algorithm of this type consists in generating diverse neural networks using different random initializations of the synaptic weights. This simple technique is sufficient to generate fairly accurate ensembles [22] .. Perturbations in the training dataset can be introduced in different ways: using bootstrap samples from the training data, modifying the empirical distribution of the data (either by resampling or reweighting examples), manipulating the input features or altering the output targets, etc. Bagging [3] , one of the most widespread methods for ensemble learning, belongs to this group of techniques. In bagging, each individual classifier is generated using a surrogate training set of the same size as the original one. This surrogate set is obtained by random resampling with replacement from the original training data. In boosting [11] , the empirical distribution is modified by reweighting the training examples depending on the performance of the generated classifiers on those examples. Initially, the weights of the training examples are equal. At each iteration of the boosting process, the weights of the training data are updated according to whether the classification produced by the classifier generated in the previous iteration is correct or not. The weights of correctly classified examples are decreased, and the weights of incorrectly classified ones are increased. In this manner, the subsequent base learner tends to focus on examples that are harder to classify. A weighted voting scheme, where the contribution made by predictors obtained later in the ensemble generally decreases, is finally used to combine the decisions of the individual learners.. Another strategy consists in manipulating the input features. For instance, one can randomly eliminate features of the input data before constructing each individual classifier. In random subspaces [15] , each base learner is generated using a different random subset of the input features. Another data randomization strategy consists in modifying the class labels. In particular, in classification problems with multiple classes, one can build each classifier in the ensemble using a different coding of the class labels [10, 12] .. The ensemble method analyzed in this work belongs to this last group of techniques. It is based on randomly modifying the class label of a fraction of instances of the training set to generate each classifier. This type of ensemble learning algorithms were introduced in [5] (flipping) and further analyzed in [19, 13] (class-switching). In [19] it is shown that class-switching ensembles composed of a sufficiently large number of unpruned decision trees trained on data where a fairly large fraction of the class-labels are switched exhibit good generalization performance in a large number of benchmark classification problems. Typically, these class-switching ensembles obtain accuracies equivalent or better than boosting and much better than bagging [19] . In [13] , the performance of the class-switching algorithm using neural networks as the base learners is analyzed. Because of the different properties of neural networks and decision trees, several modifications of the procedure described in [19] need to be made to generate efficient class-switching ensembles composed of neural networks. In particular, it is found that, in contrast with class-switching ensembles of decision trees, one should not attempt to train the base learners to provide an exact fit of the perturbed training data. Instead, the number of hidden units for the individual networks should be determined using standard architecture selection techniques. Another difference with class-switching ensembles of decision trees is that, when neural networks are used as base learners, the best overall results are obtained with relatively low class-switching rates (p = 1/5 orp = 2/5).. In this work, we extend the analysis of class-switching ensembles of neural networks presented in [13] and compare their performance with class-switching ensembles of decision trees, bagging and boosting in 20 benchmark classification tasks. When the base learners are neural networks, class-switching ensembles significantly outperform bagging and boosting ensembles in most of the classification tasks investigated. Comparing class-switching ensembles of neural networks and of decision trees, a smaller number of networks (around 200 networks) need to be aggregated in the ensembles to achieve the lowest possible (asymptotic) error level. This has the advantage that the memory requirements to store the ensemble are lower than with ensembles of decision trees, which typically need around 1000 trees to converge.. The article is organized as follows: Section 2 introduces the class-switching algorithm and the modifications that are necessary to build accurate ensembles composed of neural networks. Section 3 presents the results of experiments that compare the classification performance of a single neural network, bag-ging, boosting and class-switching ensembles in twenty datasets. Finally, the conclusions of this research are summarized in Section 4. Class-switching ensembles generate a diversity of classifiers using different perturbed versions of the training set [5, 19] . To generate each perturbed set, a fraction of examples is randomly selected and their class labels are switched also at random to a different value. The prescription used for decision trees (generate individual classifiers that achieve 0-error in the perturbed training datasets) is found not to be the appropriate configuration for neural network ensembles constructed with class-switching [13] . Combining neural networks whose architecture is designed by standard architecture selection techniques (and that therefore do not necessarily achieve 0 error in the perturbed training datasets) produces significantly better results than ensembles composed of more complex nets that do achieve 0 error in the perturbed datasets. Since the networks in the ensemble are not constructed to have zero error on the perturbed training sets, they seem to avoid overfitting to the noise injected, at least to a certain extent, and perform well in the original unperturbed problem. As a consequence, the number of base learners needed for the convergence of the ensembles errors to their asymptotic values is smaller than in class-switching ensembles composed of unpruned decision trees.. The classification accuracy of class-switching neural network ensembles is better or equivalent to bagging and better than boosting and single nets in the problems investigated. Ensembles generated with a class-switching rate ofp = 1/5 or 2/5 obtain the best overall results. These configurations (i.e. p = 1/5, 2/5) rarely obtain generalization errors significantly worse than bagging or boosting. This is not the case for other values ofp (that is,p = 0/5, 3/5 and 4/5), where results both better and worse than boosting and bagging have been observed.. The question of whether to use neural networks or decision trees as base learners in a class-switching ensemble has an answer that is problem dependent both in terms of accuracy and of classification speed. Nevertheless, both types of ensembles have demonstrated an excellent overall performance in the classification tasks investigated. This establishes that class-switching can be a useful randomization mechanism for the creation of ensembles of classifiers with very different characteristics, such as decision trees and neural networks.. Autónoma de Madrid and from the European Social Fund under an F.P.I. grant. The performance of class-switching ensembles composed of feedforward neural networks with one hidden layer is investigated in a variety of classification tasks. Experiments are carried out in twenty classification problems from various fields of application and with different characteristics (number of classes, number of attributes, distribution of classes): sixteen classification tasks from the UCI repository [1] and four synthetic datasets (Led24, Ringnorm, Twonorm and Waveform [7, 4] ). Table 1 displays the properties of the selected datasets, the protocol used for testing and the characteristics of the neural networks used as base learners in the ensemble.. Nominal features are binarized by assigning a different input unit for each label of each nominal attribute. In addition, all input features are normalized to have zero mean and unit standard deviation. The weights are randomly initialized between −0.1 and 0.1. Sigmoidal transfer functions for both the hidden and the output layers are used. The number of units in the output layer is equal to the number of classes in the classification task and the networks are trained to approximate the posterior probability of each class. The parameters of the network are determined using an improved RPROP batch algorithm [16] . The optimal architecture and number of training epochs for the neural networks are estimated for every partition of the data using cross-validation within the training dataset. The same architecture and number of epochs is used in bagging, boosting and class-switching ensembles. For the neural networks, the FANN library [21] implementation is used.. The results given are averages over 100 experiments for each dataset. In the real-world datasets these experiments consist in the execution of 10 × 10-fold cross-validation. For the synthetic datasets (Led24, Ringnorm, Twonorm and Waveform) different independent random samples for the training and testing Table 1 Characteristics of the datasets, testing method, number of input units, average number (± standard deviation) of hidden units and average number of training epochs for the neural networks used in the experiments sets are generated in each experiment. The sizes of these sets are given in Table 1 . Each experiment involves the following steps:. (1) Obtain the training and testing sets from the corresponding cross-validation fold in the real-world datasets and by random sampling in the synthetic ones. (2) Build a single neural network using the whole training dataset. The configuration of the network is estimated using 10-fold cross-validation in the training data. Different architectures (3, 5, and 7 hidden units) and different values for the number of epochs (100, 300, 500 and 1000) are explored. The configuration that obtains on average the best accuracy on the separate folds of the training data is employed. For those datasets in which the cross-validation procedure selected most of the times the maximum number of epochs and hidden units, the range of possible hidden units considered is incremented. Boosting is implemented with resampling. The architecture of the networks and the number of learning epochs used for the neural networks in the class-switching, bagging and boosting ensembles is the same as the configuration selected using cross-validation for training a single neural network. (5) As a reference, class-switching ensembles composed of 1000 C4.5 decision trees [24] with a switching rate ofp = 3/5 are built. This configuration has the best overall results for class-switching ensembles composed of decision trees in a wide range of problems [19] . (6) The generalization errors of the different classifiers (a single neural network, bagging, boosting and class-switching) are estimated on the corresponding test sets. Figure 2 displays the evolution of the average generalization error for bagging, boosting and class-switching ensembles as a function of the number of neural networks aggregated in the ensemble for a representative subset of the classification problems investigated. As a reference, the average generalization error of a single net is displayed with a horizontal line in the plots. These graphs show that the convergence of the error of class-switching ensembles is related to the fraction of switched examples (i.e.p): higherp values result in slower convergence rates. For most of the ensemble configurations, combining 200 networks seems to be sufficient for the error curves to level off and attain the asymptotic error rate of the ensemble. However, for some datasets (see for example the German, Waveform and Wine datasets in Fig. 2 ) using a high class-switching probability (class-switching withp = 4/5), 200 does not seem to be sufficient to reach the asymptotic ensemble error rate. In contrast, ran- domly initialized neural network ensembles (p = 0/5) reach their asymptotic error level after combining a fairly small number of networks (≈ 20) . In most of the classification problems investigated the error rate of a single neural network is above the asymptotic error level of the different ensembles. Table 2 presents the test errors averaged over the 100 executions for single networks, bagging, boosting, class-switching ensembles of neural networks for the different class-switching rates,p, and for class-switching ensembles composed of decision trees usingp = 3/5. For each dataset, the generalization errors that are significantly better than bagging are highlighted in boldface. The results that are significantly better than boosting are underlined. The test used to determine whether differences are statistically significant is a paired t-test with an alpha-value 0.01. The cross-validation procedure used in the real-world problems tends to produce estimates of the p-values which are too large [20] . For this reason, a low alpha-value (1% instead of 5%) is used in the test. For the synthetic datasets the p-values are not biased because the experiments are carried out using independent sampling. The standard deviations of the values reported are given after the ± signs.. The best generalization error for neural based algorithms for each dataset is marked with an asterisk. Considering only neural network ensembles, classswitching ensembles exhibit the best results in fourteen of the twenty problems analyzed. Bagging has the best performance in five classification problems, boosting in three and ensembles withp = 0/5 in two. The performance of a single neural network is suboptimal in all the classification tasks analyzed and is generally poorer than most of the ensemble methods investigated. Table 2 also shows that most configurations of class-switchingneural network ensembles reach similar generalization errors in many datasets. In particular, nearly the same results (within 0.2 points) are achieved in Diabetes, E-coli, German, Tic-tac-toe, Twonorm, Votes, Waveform and Wine by class-switching witĥ p = 1/5, 2/5 and 3/5. Thep = 4/5 configuration exhibits significantly worse results in several datasets (namely German, Glass, Ionosphere, Labor, Ringnorm, Vehicle). In some cases this is due to the fact that larger ensembles ought to have been used.. The relative performance of neural network and decision tree ensembles is problem dependent. The differences between the generalization errors of the two types of class switching ensembles are significant in many classification tasks. In Australian, Glass, Ionosphere and New-thyroid, class-switching ensembles of decision trees outperform all the configurations of neural network ensembles. By contrast, in Heart and Vehicle, and to a lesser extent in Labor and Twonorm, the decision tree ensembles are inferior to ensembles of neural networks. Table 3 shows win/draw/loss records. The numbers displayed in each cell correspond to the number of datasets in which the algorithm displayed in the leftmost column wins/draws/losses when compared to the algorithm displayed in the top row. A difference is considered to be significant when a paired t-test Table 2 . Average generalization errors Table 3 . Win/draw/loss records for ensembles composed of pruned trees (the significance of the differences in performance is measured using a paired t-test with an alpha-value of 0.01) has a p-value smaller than 0.01. For each column, the record with the largest value of wins − losses is highlighted in bold face. These results show that, for most of the datasets investigated, the ensembles considered have significantly better classification accuracies than single neural networks. The best performance with respect to a single neural network is the randomly initialized neural network ensemble: eighteen significant wins and no significant losses. Single Neural Networks significantly outperform boosting in one dataset, classswitching withp = 4/5 four times and class-switching ensembles of decision trees in two problems.

B O O O O O O O O O B O O O B I O O O O B O O B I O O O O O O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O B I O B O B O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O B O O O O O O O O O B O O B I O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O B I O O O O O O O O B O O B I B O B O O O O O O O O O O B I B O O O B O B O O O O O O O O O B O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O B O O O O O O O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B I O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I I O O O O O O B I O B I O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B B O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O B B O B O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B B O O B I O O O B O O O B O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O B O O B I O O O


Efficient run-time dispatching in generic programming with minimal code bloat Generic programming using C++ results in code that is efficient but inflexible. The inflexibility arises, because the exact types of inputs to generic functions must be known at compile time. We show how to achieve run-time polymorphism without compromising performance by instantiating the generic algorithm with a comprehensive set of possible parameter types, and choosing the appropriate instantiation at run time. The major drawback of this approach is excessive template bloat, generating a large number of instantiations, many of which are identical at the assembly level. We show practical examples in which this approach quickly reaches the limits of the compiler. Consequently, we combine the method of run-time polymorphism for generic programming with a strategy for reducing the amount of necessary template instantiations. We report on using our approach in GIL, Adobe's open source Generic Image Library. We observed notable reduction, up to 70% at times, in executable sizes of our test programs. Even with compilers that perform aggressive template hoisting at the compiler level, we achieve notable code size reduction, due to significantly smaller dispatching code. The framework draws from both the generic programming and generative programming paradigm, using static metaprogramming to fine tune the compilation of a generic library. Our test bed, GIL, is deployed in a real world industrial setting, where code size is often an important factor. Generic programming, pioneered by Musser and Stepanov [19] , and introduced to C++ with the STL [24] , aims at expressing algorithms at an abstract level, such that the algorithms apply to as broad class of data types as possible. A key idea of generic Copyright is held by the author/owner(s). LCSD '06 October 22nd, Portland, Oregon. ACM [to be supplied].. programming is that this abstraction should incur no performance degradation: once a generic algorithm is specialized for some concrete data types, its performance should not differ from a similar algorithm written directly for those data types. This principle is often referred to as zero abstraction penalty. The paradigm of generic programming has been successfully applied in C++, evidenced, e.g., by the STL, the Boost Graph Library (BGL) [21] , and many other generic libraries [3, 5, 11, 20, 22, 23] . One factor contributing to this success is the compilation model of templates, where specialized code is generated for every different instance of a template. We refer to this compilation model as the instantiation model.. We note that the instantiation model is not the only mechanism for compiling generic definitions. For example, in Java [13] and Eiffel [10] a generic definition is compiled to a single piece of byte or native code, used by all instantiations of the generic definition. C# [9, 18] and the ECMA .NET framework delay the instantiation of generics until run time. Such alternative compilation models address the code bloat issue, but may be less efficient or may require run-time compilation. They are not discussed in this paper.. With the instantiation model, zero abstraction penalty is an attainable goal: later phases of the compilation process make no distinction between code generated from a template instantiation and non-template code written directly by the programmer. Thus, function calls can be resolved statically, which enables inlining and other optimizations for generic code. The instantiation model, however, has other less desirable characteristics, which we focus on in this paper.. In many applications the exact types of objects to be passed to generic algorithms are not known at compile time. In C++ all template instantiations and code generation that they trigger occur at compile time-dynamic dispatching to templated functions is not (directly) supported. For efficiency, however, it may be crucial to use an algorithm instantiated for particular concrete types.. In this paper, we describe how to instantiate a generic algorithm with all possible types it may be called with, and generate code that dispatches at run time to the right instantiation. With this approach, we can combine the flexibility of dynamic dispatching and performance typical for the instantiation model: the dispatching occurs only once per call to a generic algorithm, and has thus a negligible cost, whereas the individual instantiations of the algorithms are compiled and fully optimized knowing their concrete input types. This solution, however, leads easily to excessive number of template instantiations, a problem known as code bloat or template bloat. In the instantiation model, the combined size of the instantiations grows with the number of instantiations: there is typically no code sharing between instantiations of the same templates with different types, regardless of how similar the generated code is. This paper reports on experiences of using the generic programming paradigm in the development of the Generic Image Library (GIL) [5] in the Adobe Source Libraries [1] . GIL supports several image formats, each represented internally with a distinct type. The static type of an image manipulated by an application using GIL is often not known; the type assigned to an image may, e.g., depend on the format it was stored on the disk. Thus, the case described above manifests in GIL: an application using GIL must instantiate the relevant generic functions for all possible image types and arrange that the correct instantiations are selected based on the arguments' dynamic types when calling these functions. Following this strategy blindly may lead to unmanageable code bloat. In particular, the set of instantiations increases exponentially with the number of image type parameters that can be varied independently in an algorithm. Our experience shows that the number of template instantiations is an important design criterion in developing generic libraries.. We describe the techniques and the design we use in GIL to ensure that specialized code for all performance critical program parts is generated, but still keep the number of template instantiations low. Our solution is based on the realization that even though a generic function is instantiated with different type arguments, the generated code is in some cases identical. We describe mechanisms that allow the different instantiations to be replaced with a single common instantiation. The basic idea is to decompose a complex type into a set of orthogonal parameter dimensions (with image types, these include color space, channel depth, and constness) and identify which parameters are important for a given generic algorithm. Dimensions irrelevant for a given operation can be cast to a single "base" parameter value. Note that while this technique is presented as a solution to dealing with code bloat originating from the "dynamic dispatching" we use in GIL, the technique can be used in generic libraries without a dynamic dispatching mechanism as well.. In general, a developer of a software library and the technologies supporting library development are faced with many, possibly competing, challenges, originating from the vastly different context the libraries can be used. Considering GIL, for example, an application such as Adobe Photoshop requires a library flexible enough to handle the variation of image representations at run time, but also places strict constraints on performance. Small memory footprint, however, becomes essential when using GIL as part of a software running on a small device, such as a cellular phone or a PDA. Basic software engineering principles ask for easy extensibility, etc. The design and techniques presented in this paper help in building generic libraries that can combine efficiency, flexibility, extensibility, and compactness.. C++'s template system provides a programmable sub-language for encoding compile-time computations, the uses of which are known as template metaprogramming (see e.g. [25] , [8, §.10] ). This form of generative programming proved to be crucial in our solution: the process of pruning unnecessary instantiations is orchestrated with template metaprograms. In particular, for our metaprogramming needs, we use the Boost Metaprogramming Library (MPL) [2, 14] extensively. In the presentation, we assume some familiarity with the basic principles of template metaprogramming in C++.. The structure of the paper is as follows. Section 2 describes typical approaches to fighting code bloat. Section 3 gives a brief introduction to GIL, and the code bloat problems therein. Section 4 explains the mechanism we use to tackle code bloat, and Section 5 describes how to apply the mechanism with dynamic dispatching functions. In the results section we demonstrate that our method can result in noticeable code size reduction even in the presence of such heuristics. to generic algorithms. We report experimental results in Section 6, and conclude in Section 7. Combining run-time polymorphism and generic programming with the instantiation model of C++ is non-trivial. We show how variant types can be used for this purpose but, without caution, this easily leads to a severe code bloat. As its main contribution, the paper describes library mechanism for significantly reducing code bloat that results from invoking generic algorithms with variant types, and demonstrates their effectiveness in the context of a production quality generic library.. We discussed the problems of the traditional class-centric approach to addressing code bloat: template hoisting within class hierarchies. This approach requires third-party developers to abide by a specific hierarchy in a given module, and can be inflexibleone hierarchy may allow template hoisting for certain algorithms but not for others. Moreover, complex relationships involving two or more objects may not be representable with a single hierarchy.. We presented an alternative, algorithm-centric approach to addressing code bloat, which allows the definition of partitions among types, each specific to one or more generic algorithms. The algorithms need to be instantiated only for one representative of the equivalence class in each partition. Our technique does not enforce a particular hierarchical structure that extensions to the library must follow. The rules for type reduction are algorithm-dependent and implemented as metafunctions. The clients of the library can define their own equivalence classes by specializing a particular type reduction template defined in a generic library, and have the induced type reductions be applied when using the generic algorithms. Also, new algorithms can be introduced by third-party developers and all they need to do is define the reduction rules for their algorithms. Algorithm reduction rules may be inherited; we discussed the copy pixels and resample view algorithms which have identical reduction rules.. The primary disadvantage of our technique is that it relies on a cast operation, the correctness of which is not checked. The reduction specifications declare that a given type can be cast to another given type when used in a given algorithm. That requires intimate knowledge of the type and the algorithm. Nevertheless, we believe the generality and effectiveness of algorithm-centric type reduction justify the safety concerns. We demonstrated that this technique can result in reducing the size of the generated code in half for compilers that don't support template bloat reduction. Even for compilers that employ aggressive pruning of duplicate identical template instantiations, our technique can result in further noticeable decrease in code size.. The framework presented in this paper is essentially an active library, as defined by Czarnecki et al. [7] . It draws from both generic and generative programming, static metaprogramming with C++ templates in particular. We accomplish a high degree of reuse and good performance with the generic programming approach to library design. Static metaprogramming allows us to fine tune the library's internal implementation -for example, to decrease the amount of code to be generated.. Our future plans include experimenting with the framework in domains other than imaging. We have experience on generic libraries for linear algebra, which seems to be a promising domain, sharing similarities with imaging: a large number of variations in many aspects of the data types (matrix shapes, element types, storage orders, etc.). One common strategy to reduce code bloat associated with the instantiation model is template hoisting (see e.g. [6] ). In this approach, a class template is split into a non-generic base class and a generic derived class. Every member function that does not depend on any of the template parameters is moved, hoisted, into the base class; also non-member functions can be defined to operate directly on references or pointers to objects of the base-class type. As a result, the amount of code that must be generated for each different instantiation of the derived class decreases. For example, red-black trees are used in the implementation of associative containers map, multimap, set, and multiset in the C++ Standard Library [15] . Because the tree balancing code does not need to depend on the types of the elements contained in these containers, a high-quality implementation is expected to hoist this functionality to non-generic functions. The GNU Standard C++ Library v3 does exactly this: the tree balancing functions operate on pointers to a non-generic base class of the tree's node type.. In the case of associative containers, the tree node type is split into a generic and non-generic part. It is in principle possible to split a template class into several layers of base classes, such that each layer reduces the number of template parameters. Each layer then potentially has less type variability than its subclasses, and thus two different instantiations of the most derived class may coalesce to a common instantiation of a base class. Such designs seem to be rare.. Template hoisting within a class hierarchy is a useful technique, but it allows only a single way of splitting a data type into sub-parts. Different generic algorithms are generally concerned with different aspects of a data-type. Splitting a data type in a certain way may suit one algorithm, but will be of no help for reducing instantiations of other algorithms. In the framework discussed in this paper, the library developer, possibly also the client of a library, can define a partitioning of data-types, where a particular algorithm needs to be instantiated only with one representative of each equivalence class in the partition.. We define the partition such that differences between types that do not affect the operation of an algorithm are ignored. One common example is pointers -for some algorithms the pointed type is important, whereas for others it is ok to cast to void * . A second example is differences due to constness (consider STL's iterator and const iterator concept). The generated code for invoking a non-modifying algorithm (one which accepts immutable iterators) with mutable iterators will be identical to the code generated for an invocation with immutable iterator. Some algorithms need to operate bitwise on their data, whereas others depend on the type of data. For example, assignment between a pair of pixels is the same regardless of whether they are CMYK or RGBA pixels, whereas the type of pixel matters to an algorithm that sets the color to white, for example.

O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Constructivism in computer science education Constructivism is a theory of learning which claims that students construct knowledge rather than merely receive and store knowledge transmitted by the teacher. Constructivism has been extremely influential 'in science and mathematics education, but not in computer science education (CSE). This paper surveys constructivism inthe context of CSE, and shows how the theory can supply a theoretical basis for debating issues and evaluating proposals.', The dominant theory of learning today is called consyuctivism. This theory claims that knowledge' is actively constructed by the student, not passively absorbed from textbooks and lectures. Since the construction builds recursively on knowledge (facts, ideas and beliefs) that the student already has, each student will construct an idiosyncratic version of knowledge. Teaching techniques derived from the theory of constructivism are supposed to be more successful than traditional techniques, because they explicitly address the inevitable process of knowledge construction. Constructivism has been intensively studied for more than a decade by researchers of science education [fl and mathematics education [l] . This article will first survey constructivism and its application in science education; then I will assess its relevance for CSE and offer some examples of the application of the theory. There is a large literature on the psychology of programming ( [17] , [lOI) , and many of these researchers clearly hold constructivist views. For example, the Permission to make digitaLhard copies of all or part of this material for personal or doom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyl'@UOtb%, the title oftbe publication and its date appear, end notice is given that copyright is by permission of the &%I, Inc. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires specific permission ylflorfee. Passive learning will likely fail, because each student brings a different knowledge framework to the classroom, and will construct new knowledge in a different manner. Learning must be active: the student 'must construct knowledge assisted by guidance from' the teacher and feedback from other students.. The task of the teacher is signifida&ly more difficult than in the classical paradigm, because .the guidance must be based on, the understanfling of each student's currently existing cognitive structures. Constructivists differ among themselves as to the relative importance ascribed' to 'the individual student? and to the group ,iq constructing knowledge; these variants are known as 'radical' and rsocial constructivism, respectively. A discussion of #this topic is beyond the scope of this, article (see [5] ). , ,. Studies have, shown that relatively few students reach an, acceptable level ,of achievement in science and mathematics [4] . Physics teachers seem to have the worst time, as students retain a naive theory of physics despite 'intensive instruction in'NeCtonian mechanics. Apparently, these' ideas are so entrenched that-inere lectures and even experiments have: a' difficult time evicting"them. :I,' It According to constructivism, a teacher cannot ignore the student's existing knowledge;' instead, he or she must question the student in order .to understand what models the student possesses, and only then attempt to guide the student to the 'correct!, theory. A term that is commonly used is alternative j?a,me,~$s, meaning that the students have consistent models-they just happen to be at variance With the (currently accepted) scientific theory., Similarly, von Glasersfeld would never say that a student's knowledge is 'wrong', because he does not believe in the possibility of ,establishing truth. Instead,, he says that concepts are viable 'if they prove adequate in the contexts in which they were created' ( [20] , p. 7).. In most fields of science education (including CSE), there is a large bbdy of reskarch"that catalogs miscdnceptions. A constructivist'would view a misconception *'. 'These terms come from the seminal work of &an Piaget 0; the acquisition of knowledge by children; P&et's work was instrumental in the development of constructivist theories. not as a mistake, but as a logical construction based on consistent, though non-standard concepts, held by the student. Misconceptions form the prior knowledge that is essential to the construction of new knowledge [16] . Merely listing misconceptions is fruitless; a description of the underlying model and a prescription for constructing a modified one must be given.. Criticism of Constructivism Before continuing, we must stress that that there is opposition to constructivism.2 The criticism is not so much of. the constructivist theory of learning, but rather of extreme conclusions drawn from constructivist epistemology. Carried to the extreme, radical constructivism leads t,o solipsism, the philosophical claim that the world is my own creation or dream. In turn, this can lead to a rejection of ethics: if the world is my own creation, why should I care what happens to others? Carried to the extreme, social constructivism leads to a view of science as a merely political enterprise developed by entrenched elitist groups whose main purpose is to ensure their own survival. From the fallibility of scientific knowledge, one slips into relativism of truth, and from the sociology of scientific practice, into demands for empowerment detached from any attempt at objective evaluation of scientific knowledge.

B O O B B B O O B O B O O O O O B O O O O O O B O O O O B O O O O O B O B B O O O O B B O O B O B O B O O O O O O B O O O O O O O O O O O O O B O B O O O O O B O O B O O O O O B O O O O O O O O O O O O O B O O O O O O B O O O B O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O B B O O B B O O O O O O B B O O O O B B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O B O O O O O O O O B O O O O B O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B


Categorical skylines for streaming data The problem of skyline computation has attracted considerable research attention. In the categorical domain the problem becomes more complicated, primarily due to the partially-ordered nature of the attributes of tuples.In this paper, we initiate a study of streaming categorical skylines. We identify the limitations of existing work for offline categorical skyline computation and realize novel techniques for the problem of maintaining the skyline of categorical data in a streaming environment. In particular, we develop a lightweight data structure for indexing the tuples in the streaming buffer, that can gracefully adapt to tuples with many attributes and partially ordered domains of any size and complexity. Additionally, our study of the dominance relation in the dual space allows us to utilize geometric arrangements in order to index the categorical skyline and efficiently evaluate dominance queries. Lastly, a thorough experimental study evaluates the efficiency of the proposed techniques. Abundance of data has been both a boon and a curse, as it has become increasingly difficult to process data in order to isolate useful and relevant information. In order to compensate, the research community has invested considerable effort into developing tools that facilitate the exploration of a data space. One such successful tool is the skyline query. The skyline of a data set is the subset of tuples that are not dominated on all of their attributes by any Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. other tuple. Intuitively, the skyline consists of the tuples that have a uniquely interesting combination of attribute values that no other tuple can match.. Given the practicality of skyline queries, which arises from the elegant and parameterless manner in which they capture a notion of interestingness, previous work has concentrated on the efficient evaluation of skyline queries in both offline and online environments. In an offline environment data resides on disk and skyline queries are answered on demand, while in an online data streaming environment the skyline of the most recent stream data is continuously maintained up to date.. Most of this work reasoned about tuples with numerical attributes and totally ordered domains, and relied on the cleanness of the dominance relation between tuples, induced by the linearity of the numerical domains, in order to derive efficient solutions. However, in real applications data can either include or be exclusively comprised of attributes that are categorical and partially ordered in nature. A hierarchical categorical domain is possibly the most familiar example of a partially ordered domain. Complex relationships and hierarchal structure cannot be captured by a simple mapping of categorical values to numbers and this immensely complicates the skyline computation problem.. Recent work [5] considered the on-demand evaluation of skyline queries on tuples with partially ordered categorical attributes, but in an offline environment. As we will subsequently argue and experimentally demonstrate, the proposed techniques are inappropriate for a highly dynamic data streaming environment where tuples constantly flow into the system. In this setting, we require an efficient solution to continuously maintain the skyline up to date for the most recent tuples that arrived in the stream. This leaves a significant gap in the array of available skyline techniques. The omission becomes more important when one considers the wide applicability of such an online skyline maintenance solution for partially ordered data.. As an example, consider a service that aggregates and displays news articles as they are published by news sources. News posts streaming into the system are associated with categorical attributes like the name of the news source, the subject of the event and the geographical area associated with the event. An expert has defined a partial order over the categorical domains expressing the service's preferences and defining its unique style. The system will then select for display, or more extensive filtering by an expert, the most "interesting" news that comprise the skyline of the most recent articles. For example, a skyline article is of interest because it was published in a high quality news source and is related to a popular geographical area, even though the subject itself might not be that preferable.. In this paper, we identify and study the problem of maintain-ing the skyline of streaming data with partially ordered, categorical attributes and realize two novel techniques that constitute the building blocks of STARS (Streaming Arrangement Skyline), the proposed efficient solution to the problem. In particular, we assume a sliding window model of stream computation and introduce a lightweight, grid-based data structure for indexing the tuples in the system buffer. Our initial, basic indexing solution is progressively refined: we first identify and utilize a property that is unique to the grid-oriented structure of the index and subsequently develop techniques that offer flexibility in controlling the granularity of the grid. The resulting indexing structure can gracefully adapt to tuples with many attributes and partially ordered domains of any size and structure in an optimal manner.. We subsequently study the dominance relation between two tuples with partially ordered attributes in the dual space: tuples are mapped to lines and the interaction of their corresponding lines is used in order to infer their dominance relation. This mapping allows us to utilize powerful tools from computational geometry, known as geometric arrangements, in order to organize and query the skyline efficiently. As we discuss, the arrangement-based organization of the skyline allows us to answer dominance queries by considering only a small fraction of the skyline tuples.. The rest of the paper is organized as follows. In Section 2 we survey related work. In Section 3 we introduce the basic definitions and notations that we will use throughout this paper and offer a high level discussion of the skyline maintenance task. The core techniques comprising our solution are presented in Section 4. Section 5 presents our experimental evaluation of the proposed solution, while Section 6 concludes the paper. To summarize, our experimental evaluation demonstrated the applicability of the proposed solution to a wide range of buffer sizes and data dimensionality, for both synthetic (Figures 11(a)-(c) ) and real ( Figure 12 ) data. We also verified our claim that the skybuffer indexing technique can adapt to posets of any shape and size by offering flexibility in controlling the granularity of the grid-based indexing structure (Figure 15 ). The second claim that we verified was the pruning efficiency of the skyline organization, which was also found to be resilient to increases in data dimensionality (Figure 14) . Lastly, we demonstrated the inapplicability of the existing offline skyline evaluation techniques of [5] in a streaming environment (Figures 11(a)-(c) ) and identified the inherent reasons behind this poor performance (Figure 13 ). In this paper, we identified and motivated the problem of maintaining the skyline of streaming data with partially ordered, categorical attributes and realized two novel techniques that constitute the building blocks of an efficient solution to the problem.. We introduced a lightweight data structure for indexing the tuples in the streaming buffer, that can gracefully adapt to tuples with many attributes and partially ordered domains of any size and complexity. We subsequently studied the dominance relation in the dual space and utilized geometric arrangements in order to index the categorical skyline and efficiently evaluate dominance queries. Lastly, we performed a thorough experimental study to evaluate the efficiency of the proposed techniques. Since the inception of the skyline operator [4] , researchers have considered the efficient evaluation of skyline queries over numerical data in the absence [8, 12] or presence [24, 16, 23, 18] of supporting indexing structures. More recent work has concentrated on variations of the original query [7, 6, 20] . In this section we briefly review the relevant literature. A more comprehensive review of the area is available elsewhere [18] .. The Block-nested-loops algorithm introduced in [4] reads data sequentially from disk, while maintaining in memory the temporary skyline of the tuples that have been read so far. At the end of the computation, when all data have been read, the temporary skyline is the true skyline of the data set. [8] observed that if the data are first sorted in an order that is compatible with the dominance relation, then the maintenance of the in-memory temporary skyline becomes simpler and more efficient. The LESS algorithm of [12] capitalizes on this idea and integrates skyline computation with the external sorting procedure.. Skyline computation can greatly benefit by the presence of a supporting indexing structure. The BBS algorithm of [23] utilizes an R-tree to progressively construct the skyline. The R-tree is used to prioritize access over nodes that are likely to contain skyline tuples and to prune entire nodes whose tuples are definitely dominated by the partially constructed skyline. The BBS algorithm was found to be superior than the index-based solutions presented in [24, 16] . Recent work [18] introduced a B-tree variant that stores and clusters the tuples based on their Z-order, which is compatible with the dominance relation.. The size of the skyline can increase dramatically with data dimensionality. This is incompatible with the skyline's role as an exploratory tool. In order to compensate, [6] introduced the notion of k-dominance: a tuple is declared dominant as long as it is better than another tuple on a subset of its attributes. [20] adopted a different approach to the same problem and introduced algorithms for computing the k most "representative" skyline tuples.. With respect to skyline maintenance, [26, 18] discuss the problem for non-streaming, numerical data. However, most relevant to our work are existing solutions for performing skyline maintenance for a stream of numerical data [25, 19, 21] . These techniques make no provision for partially ordered categorical data. The work of [5] is designed to handle such data, albeit in an offline environment. Because of the limitations of the techniques presented in [5] when applied to streaming data, we initiate a study of the problem of maintaining the skyline of streaming data with partially ordered attributes and design a novel solution for performing this task.

B O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B B O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O


Group recommendations with rank aggregation and collaborative filtering The majority of recommender systems are designed to make recommendations for individual users. However, in some circumstances the items to be selected are not intended for personal usage but for a group; e.g., a DVD could be watched by a group of friends. In order to generate effective recommendations for a group the system must satisfy, as much as possible, the individual preferences of the group's members.This paper analyzes the effectiveness of group recommendations obtained aggregating the individual lists of recommendations produced by a collaborative filtering system. We compare the effectiveness of individual and group recommendation lists using normalized discounted cumulative gain. It is observed that the effectiveness of a group recommendation does not necessarily decrease when the group size grows. Moreover, when individual recommendations are not effective a user could obtain better suggestions looking at the group recommendations. Finally, it is shown that the more alike the users in the group are, the more effective the group recommendations are. Recommender Systems (RSs) are software tools and techniques suggesting to a user a well selected set of items matchPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ing the user's taste and preferences [18, 1, 4] . The suggestions relate to various decision-making processes, such as what items to buy, what music to listen to, or what online news to read. They are widely used in Web-based ecommerce applications to help online users to choose the most suitable products, e.g. movies, CDs, books, or travels.. The large majority of RSs are designed to make recommendations for individual users. Since recommendations are usually personalized, different users receive diverse suggestions. However, in some circumstances the items to be selected are not intended for personal usage but for a group of users; e.g., a DVD could be watched by a group of friends or in a family. These groups can vary from stable groups to ad hoc groups requiring recommendations only occasionally.. For this reason some recent works have addressed the problem of identifying recommendations "good for" a group of users, i.e., trying to satisfy, as much as possible, the individual preferences of all the group's members [8] . Group recommendation approaches are either based on the generation of an integrated group profile or on the integration of recommendations built for each member separately (see section 2). Group RSs have been designed to work in different domains: web/news pages [17] , tourism [15, 13] , music [6, 14] , TV programs and movies [16, 21] .. A major issue in this research area relates to the difficulty of evaluating the effectiveness of group recommendations, i.e., comparing the generated recommendations for a group with the true preferences of the individual members. One general approach for such an evaluation consists of interviewing real users. In this approach there are two options: either to acquire the users' individual evaluations for the group recommendations and then integrate (e.g., averaging) these evaluations into a score that the group "jointly" assigns to the recommendations; or to acquire directly a joint evaluation of the group for the recommendations. In the first case one must decide how the individual evaluations are integrated; this is problematic as different methods will produce different results and there is no single best way to perform such an integration. Another difficulty, which is common to both options, as was observed by [12] , is related to the fact that the satisfaction of an individual is likely to depend on that of other individuals in the group (emotional contagion). Moreover, on-line evaluations can be performed on a very limited set of test cases and cannot be used to extensively test alternative algorithms.. A second approach consists of performing off-line evaluations, where groups are sampled from the users of a tra-ditional (i.e., single-user) RS. Group recommendations are offered to group members and are evaluated independently by them, as in the classical single user case, by comparing the predicted ratings (rankings) with the ratings (rankings) observed in the test set of the user. The group recommendations are generated to suit simultaneously the preferences of all the users in the group and our intuition suggests that they cannot be as good as the individually tailored recommendations. We observe that using this evaluation approach, in order to test the effectiveness of a group recommendation, we do not need the joint group evaluations for the recommended items, and we can reuse the most popular data sets (e,g, Movielens or Netflix) that contain just evaluations (ratings) of individual users.. In this paper we follow the second approach evaluating a set of novel group recommender techniques based on rank aggregation. We first generate synthetic groups (with various criteria), then we build recommendation lists for these groups, and finally we evaluate these group recommendations on the test sets of the users. The group recommendations are built in two steps: first a collaborative filtering algorithm produces individual users' rating predictions for the test items and consequently individual ranking predictions based on these rating predictions are generated. Then a rank aggregation method generates a joint ranking of the items recommended to the group by integrating the individual ranking predictions computed in the previous step. We then measure, for each group member, how good this integrated ranking is, and if this is worse than the initial individual ranking built by the system by taking into account only the ratings contained in the profile of the user. We performed an analysis of the generated group recommendations (ranking) varying the size of the groups, the inner group members similarity, and the rank aggregation mechanism.. As we mentioned above, intuition suggests that aggregating individual rankings, i.e., mediating the potentially contrasting preferences of the group members, should result in a decrease of the recommendation effectiveness. Moreover, one can also conjecture that the larger is the number of people in the group the harder it is to find a consensus among them.. In this work, we show that these intuitions are not always correct. In fact, we show that in certain cases group recommendations with rank aggregation can be more effective than individual recommendations, when effectiveness is measured using normalized discounted cumulative gain (nDCG), a popular IR metric measuring the quality of the ranking produced by a system. We observe that this happens when the individual recommendations are not particularly good, i.e., when the recommender system is not able to make good personalized recommendations. We show that this happens often in real scenarios. Moreover, we confirm the intuition that the effectiveness of the group recommendations raises when the group members are more similar.. It is worth noting that in this paper we focus on the goodness of the predicted ranking of the recommendations rather than in evaluating the accuracy of the predicted ratings' values. We believe that especially for group recommendations it is more important to understand if the RS correctly sorts the proposed items, hence suggests to the group first the items more suitable for all the group members. This approach follows a new trend of evaluating recommender systems [19] . Moreover, we observe that the rank aggregation problem has been largely studied and the notion of optimal rank, i.e., the Kemeny optimal one, has been defined [7] . Some of the rank aggregation techniques that we have used (e.g., Spearman footrule optimal aggregation) are proved to be close to the Kemeny optimal one and therefore our results show important and fundamental properties of the best solutions to group recommendation via rank aggregation. This paper is structured as follows. Section 2 overviews related researches. Section 3 defines our approach based on rank aggregation. Section 5 provides the experimental evaluation of the approach and finally Section 6 draws the conclusions and describes some future work. In this paper we analyzed the effectiveness of ranked list recommendations tailored for a group of users. We propose to use rank aggregation techniques for generating group recommendations from individual recommendations. We show that these recommendations are only slightly less effective that the individual recommendations for groups of moderate size (2, 3, and 4). Only for larger groups, of size 8, the group recommendation is significantly inferior than the individual recommendation.. Moreover, our results revealed some novel facts. First, we observed that the effectiveness of group recommendations does not necessarily decrease when the group size grows. In fact, this is not happening for groups of similar users. Secondly, when the individual recommendations are not correctly ranked, then recommending items ordered for a group recommendation can improve the effectiveness of the recommendations. This indicates that when, for some reason, the individual recommendations are not good, aggregating the ranked list recommendations built for a group of users, which the target user belongs to, can increase recommendation goodness. Third, we have experimentally confirmed a common belief that the more alike are the users in the group, the more satisfied they are with the group recommendations.. In the future, we want to test our findings with other data sets, with other recommendation algorithms, and with real users. In such a way we want to further validate our findings, which are currently based on synthetically generated groups and may also be dependent on the particular collaborative filtering technique that we used, namely SVD.. In particular we would like to assess the impact in our group recommendation techniques of collaborative filtering prediction approaches aimed at producing better rankings [19] rather more accurate rating predictions. We also want to further investigate when it is convenient to make a group recommendation instead of an individual recommendation. We consider this approach as a semi-personalized recommendation strategy. With that respect it would be important to find critical user or group features that can help to decide when it is beneficial to make an individual, or a group, or even a non-personalized recommendation. The major part of the research on group recommendation investigated the core algorithms used for generating the group recommendations. Different strategies are available and two main approaches have been proposed [8] . The first consists of creating a joint user profile for all the users in the group and then performing a recommendation for this artificial user represented by the group profile [14, 21] . This would provide a recommendation for the group that is based on all the user profiles and in some way represents the group interests mediated in the group profile. The second approach aggregates the recommendations for each individual member into a single group recommendation list. In this case, first the recommendations for each individual group member are created independently, and then these ranked lists of items, one for each individual user, are aggregated into a joint group recommendation list. Berkovsky and Freyne [3] compared the two approaches in a recipe recommendation problem and found that the first one performs slightly better. In their study the second approach is implemented by computing an item rating prediction for a group as a domain-dependent linear combination of the predictions for the group members. Actually, this is only one of the many possible approaches for aggregating predictions and an extensive comparison of the two approaches is still missing in the literature.. In our work we are concerned with recommendations generated using this second approach, i.e., aggregating individual recommendations. The core issue in this case is how to aggregate into a single group recommendations' list the ranked lists of recommendations produced for each group member. And then the second problem is related to the estimation of how this integrated ranking would be evaluated by the group members.. Masthoff [11] addressed the related problem of how subjects aggregate recommendation lists, i.e., how a person would select recommendations for a group balancing the preferences of the group members (three, in their experiments), that are expressed by the ratings of these members for 10 options. The subjects were asked which items (clips) the three should view as a group, given that they only had time to see 1, 2, 3, ... or 7. The goal was primarily to understand how humans aggregate group preferences and hence to select the aggregation method that is best evaluated by the users. That study concluded that there is evidence that human subjects use in particular Average, Average Without Misery, and Least Misery (see [11] for the definition of the aggregation strategies). Average and Least Misery are two strategies that we have tested in our experiments. We note however, that [11] did not evaluate how group members evaluate group recommendations but how subjects integrate recommendations for a group.. Another related work in the area of web search [7] deals with rank aggregation methods for the web, and presents a theoretical groundwork for building and evaluating rank aggregation methods. Kemeny optimal aggregation is introduced, and since this is computationally intractable other aggregation methods are defined, showing that their produced rankings are close to the optimal one. We have adopted that approach and used rank aggregation methods for combining the ranking independently produced for each group member by a collaborative filtering RS. The major technical difficulties that we faced are related to the specific characteristics of our application. In fact, in their case the best integrated ranking is defined as the Kemeny optimal one, i.e., that minimizing the (average) Kendall tau distance from the predicted individual recommendation lists.. In the recommendation scenario the ultimate goal is not simply to correctly aggregate the individual recommendation lists. In fact, that aggregation may be optimal, i.e., as close as possible to the individual recommendation lists, but far from the true users' preferences, i.e, those observed in the test data. For instance, suppose that there are three items p 1, p2 and p3, and a group of users all rank these items as [p 1, p2, p3] . Imagine that the RS makes a systematic error and predicts the ratings for the users in the group in such a way that the generated ranking is [p 3, p2, p1], i.e., it is the reverse order of the correct one. Then the predicted ranking for the group is also equal to [p 3, p2, p1]. Clearly, this is the Kemeny optimal aggregated ranking (since all of them are equal) but it is a wrong group recommendation (because the predictions were wrong) when it is compared to the true preferred ranking of the group. To avoid this problem, in our experimental evaluation the aggregated group ranking is compared to the individuals' optimal rankings that we derive from the ratings in the test set. In the experimental evaluation we measure the effectiveness of the proposed group recommendation techniques. For each member of a group, we compare the effectiveness of the ranked recommendations generated for his group with the effectiveness of those computed only for him. Effectiveness of a ranking is computed with nDCG, as it will be illustrated below. We deliberately avoided to compute the joint effectiveness of the group recommendations for the group because, as we discussed in the introduction, it requires to define an arbitrary aggregation formula integrating the effectiveness of the group recommendation for all the group members.. To conduct the experiments we first generated artificial groups of users (see Subsection 4.1 for the details). Then, for evaluating the goodness of the recommendations (both individual and group) we used the standard approach to divide the dataset into two parts: the training set and the testing set. Approximately 60% (randomly chosen) of the ratings from the user profile were assigned to the training set and the rest 40% to the testing set. We used collaborative filtering to generate individual predictions for each user (see Subsection 4.2 for the details). Then, these predictions were aggregated into the group recommendations using the

O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O