A data-driven study of image feature extraction and fusion Feature analysis is the extraction and comparison of signals from multimedia data, which can subsequently be semantically analyzed. Feature analysis is the foundation of many multimedia computing tasks such as object recognition, image annotation, and multimedia information retrieval. In recent decades, considerable work has been devoted to the research of feature analysis. In this work, we use large-scale datasets to conduct a comparative study of four state-of-theart, representative feature extraction algorithms: color-texture codebook (CT), SIFT codebook, HMAX, and convolutional networks (ConvNet). Our comparative evaluation demonstrates that different feature extraction algorithms enjoy their own advantages, and excel in different image categories. We provide key observations to explain where these algorithms excel and why. Based on these observations, we recommend feature extraction principles and identify several pitfalls for researchers and practitioners to avoid. Furthermore, we determine that in a large training dataset with more than 10, 000 instances per image category, the four evaluated algorithms can converge to the same high level of category-prediction accuracy. This result supports the effectiveness of the data-driven approach. Finally, based on learned clues from each algorithm's confusion matrix, we devise a fusion algorithm to harvest synergies between these four algorithms and further improve class-prediction accuracy. Extracting useful features from a scene is an essential subroutine in many multimedia data analysis tasks such as classification and retrieval. Remarkable progress has been made in multimedia computing, computer vision and signal processing in recent decades. Despite this finding, it is still notably difficult for computers to accurately recognize an object or analyze the semantics of a scene. For example, suppose that we want to recognize a piece of white paper in an image. A naive feature we can use is "a white two dimensional rectangle." However, such a feature will not work in most cases because of the following:. 1. The paper may be folded. 2. The viewing angle of the piece of paper may not perpendicular, and hence the paper does not appear to be rectangular. 3 . Environmental factors such as occlusion and lighting can cause changes in its shape and color.. The above challenges are all related to feature invariance issues. A second challenge is called feature aliasing or feature selectivity: how well a feature can differentiate one object from the others. For example, the feature "white twodimensional rectangle" can be used to describe many other objects: a piece of white cloth, a white table, and a white wall, among others. The goal of feature extraction is to find features that are both invariant and selective.. All traditional feature extraction approaches focus on some specific information in the image. For example, the color-texture codebook (CT) focuses on the statistics of colors and textures in small regions of an image. SIFT focuses on local invariant shapes. Recently, neuro-based approaches such as HMAX and convolution networks (ConvNet) have been proposed to model features according to how the human visual system extracts features. HMAX [45] builds computing models that use the pioneering neuroscience work of Hubel [22] . Hubel's work indicates that visual information is transmitted from the primary visual cortex (V1) through extrastriate visual areas (V2 and V4) to the inferotemporal cortex (IT). The IT, in turn, is a major source of input to the prefrontal cortex (PFC), which is involved in linking perception to memory and action [33] . The pathway from the V1 to the IT (called the visual frontend) consists of a number of simple (lower) and complex (higher) layers. The lower layers attain simple features that are invariant to scale, position and orientation at the pixel level.. Higher layers can combine simple features to recognize more complex features at the object-part level. Pattern recognition at the lower layers is unsupervised, whereas recognition at the higher layers involves supervised learning. This particular neuroscience-motivated model appears to enjoy at least a couple of advantages: (1) it balances feature selectivity (at lower layers) and invariance (at higher layers), and (2) it models edges of an object and then combines edges to recognize parts of an object and place these features in a hierarchical context. Similar to HMAX, ConvNet is also a neuro-based approach. It differs from HMAX primarily in the way that ConvNet iterates more over the data to learn a model with a deep architecture [39] . This allows for the capture of both the structure and detail of an object.. Herein, we perform comparative evaluation that demonstrates that different feature extraction algorithms have their own set of advantages and excel in different image categories. We provide key observations about why certain algorithms perform better with different image categories. Based on these observations, we establish feature extraction principles and identify several pitfalls for researchers and practitioners to avoid:. 1. When training data are insufficient, no scheme performs well. However, because simple algorithms such as CT and SIFT do not require much data to learn model parameters, they may be a better choice when training data are scarce. 2. Increases in the amount of training data correlate with a jump in the accuracy of complex models, such as HMAX and ConvNet. Different feature extraction algorithms enjoy their own advantages, and excel in different image categories. 3. When training data are abundant, all the four algorithms, simple or complex, converge to the same level of accuracy.. The major contributions of this paper are summarized as follows:. 1. Through our comparative analysis, we identify pitfalls of past studies: either they did not use enough training data, or their testbed composition already favors a particular feature extraction algorithm. 2. Through our large-scale comparative study, we demonstrate the benefit of employing large training datasets in training, which can make both simple and complex algorithms converge to the same level of accuracy. 3. We devise a fusion algorithm based on learned clues from each algorithm's confusion matrix. Our algorithm harvests synergies between these four algorithms and further improves class-prediction accuracy. 4. We established a large testbed for the research community, namely an annotated dataset of six million PicasaWeb images, which will be released publicly with this paper. The rest of the paper is organized as follows. Section 2 surveys the related work. Section 3 briefly introduces the four feature extraction algorithms evaluated in this paper. Section 4 details an algorithm that fuses multiple feature extraction methods that we demonstrate can perform better than any individual feature extraction scheme alone. Section 5 explains the setup of our experiments and presents their results. Finally, we offer concluding remarks in Section 6. In this paper, we investigated four representative feature extraction algorithms, color-texture codebook (CT), SIFT codebook, HMAX, and convolutional networks (ConvNet). Comprehensive experiments were conducted that revealed differences between these algorithms. We discussed our results using two different views. The first view is the image-category view. We provided an extensive analysis of different categories, and found that different algorithms each have their own advantages that can give them an edge in different categories. This finding is one of the main reasons why different algorithms may achieve significantly different or even contradictory results with different datasets. In addition to the inter-category analysis, this actually reveals patterns of intracategory variance. Different feature extraction algorithms discover different intra-category invariant information, and thus perform differently. The second view is the number of training instances. We observed that different algorithms may perform differently depending on the number of training instances. Simple algorithms tend to perform better when the training data size is small. When the number of training instances reaches up to and beyond a certain amount, complex models can experience a sudden jump in accuracy. To analyze even larger number of training instances, we constructed a new large-scale image dataset from PicasaWeb. With this dataset, we observed that all the four algorithms we studied converge to a good accuracy level with an abundance of training instances. The observation demonstrated the validity of the data-driven approach. Our experimental results reveal that both training data size and dataset category composition can affect the results of algorithm comparisons. Extreme care must be taken by researchers to avoid such pitfalls to ensure the reporting of reliable results.. Finally, we devised and studied a fusion algorithm based on confusion matrices to harvest synergies between these four algorithms. The key idea is that when an algorithm is confused between classes x and y for recognizing an object, one should direct the recognition of the object to an algorithm that can clearly distinguish the two classes. Our fusion method can improve class-prediction accuracy by substantial margins when the training dataset size ranges from small to moderate (below 1, 000). When the dataset size is large (approaching 5, 000), improvement still exists, although it is less significant. Fusion methods that can take advantage of having different views of the raw data can continue to improve class-prediction accuracy even when individual views have reached their limits.. We believe that our proposed confusion matrix-guided fusion scheme can be formulated into an optimization problem to obtain statically optimal class prediction. Our future work will be devoted into formally formulating the problem and comparing different design considerations. The multimedia community has been striving to bridge the semantic gap [20, 46, 62] between low-level features and high-level semantics for decades. (Comprehensive surveys are given in [5, 20] .) With high-quality image features, fancy applications can significantly improve a user's experience [9, 11, 30] . One key problem is how to extract powerful features. Numerous feature extraction algorithms have been developed for image annotation [46] , as well as machine learning algorithms [14, 36, 59, 60, 61] . Roughly speaking, image features can be grouped into four types: color, texture, local features, and shapes. Color features are the most straightforward image feature and therefore were the first to receive sufficient study. Many color-based image retrieval algorithms have been developed [18] . Typical color features include the color histogram [19] , color invariance [17] , and color saliency [51] . Texture features, such as local binary patterns (LBP) [34] , pyramidal-structured wavelet transforms [37] , and tree-structured wavelet transform [8] , are another significant set of signals for recognizing objects. Color and texture features are usually combined to solve image retrieval problems. Color-texture histograms are widely used for object tracking and recognition. After a decade of using color and texture as the main features for an image, a breakthrough came with the scale-invariant feature transform (SIFT) [32] . SIFT was a large step forward for extracting scaleinvariant features. SIFT features are local features, and have been demonstrated to be effective in detecting near-replicas of images. SIFT has often been combined with unsupervised learning algorithms to solve multiple image processing problems. Recently, Y. Bengio [2] proved that it is theoretically impossible to represent some functions by architectures that are too shallow. In response to this finding, researchers learned from neuroscientists how the human visual system works [22] , and it has been proven to be a deep architecture. Since Hinton's work in 2006 [21] , deep models have been developed with good results. The earliest deep architectures include self-organizing neural networks [15] and the predecessor of the convolutional network (ConvNet), which is applied in document recognition [28] . For natural image processing, one of the representative works is HMAX [45] , which strikes a good balance between feature selectivity and feature invariance. Furthermore, ConvNet [39] was introduced to establish a deep unsupervised learning architecture to learn powerful features. These deep learning features are based on edge detection and thus focus on the shapes of objects. Because our evaluation covers color, texture, local feature and shape, the conclusions that we draw hold for any subset of these combinations.. Though many comparative studies on features have been performed in the past, they generally suffer from a couple of limitations. First, most studies employ a relatively small training dataset; this results in better feature extraction algorithms not having the opportunity to demonstrate their superiority with training data increases. Second, some comparative studies' results are highly dependent on the makeup of the testbed. If categories of images favoring a particular algorithm dominate a testbed, that algorithm certainly achieves the best result. Unfortunately, most studies do not pay attention to these two limitations, which results in their conclusions having limited applicability. More importantly, we focus on studying representative state-of-the-art algorithms using large datasets. Our dataset size is much larger than that used in previous attempts (e.g., [13] ) for data-driven image classification [42, 63] .. Because different feature extraction algorithms have different advantages in dif-ferent scenarios, it is a natural consideration to integrate different methods together to overcome any drawbacks from one individual method that may be an advantage to another [1, 56] . This fusion of methods leads to enhanced final performance. There are various ways to combine multiple algorithms. These traditional schemes aim to improve class-prediction accuracy by applying statistical learning to the same set of features. In our work, we pay attention to employing different features, i.e., different views provided by different featureextraction algorithms. Our fusion method uses a confusion matrix to guide the choice of the best feature sets (views) into the raw images and then makes a collective class-prediction decision. Our approach differs from the traditional ensemble schemes in two respects. First, most traditional fusion schemes combine several feature sets into a single one and then use a statistical algorithm to learn the best feature combination [10] . Second, most ensemble schemes work on the same feature space and use statistical methods to mask prediction error. Our approach builds on these schemes and emphasizes that we take advantage of both multiple feature and statistical methods to reduce prediction error.. In general, ensemble methods can be classified into two groups: feature-level and decision-level. The first group pays focuses on combining features. When there are multiple feature sets, one of the most straightforward ways to combine features is to fuse on the feature-level. Algorithms of this type include stitching features together to create a super feature set [49] and constructing a classifier from the super feature set in conjunction with some learned cost [52] . Giridharan et al. proposed a discriminative approach to fusion of multimodal features of a video [23] . Wang et al. proposed a smart approach to fusion of multiple visual features in a graph-based learning scheme [53, 54] . However, these methods may encounter the curse of dimensionality or be confined by a feature-dependent structure [57] , which is hard to generalize to different feature sets. The second group, decision-level fusion [38] , is a good method to avoid the curse of dimensionality. Decision-level fusion can be implemented by either voting or constructing features with meta-classifier results. For instance, pairwise coupling, tree-structure ensemble, and error-correction output coding [41] all attempt to work on the same feature set (the same feature space) with robust statistical methods to minimize class-prediction error. Feature-level fusion and decision-level fusion have also been compared using video datasets, and the performance is related to semantic concepts [48] . In contrast to the feature-level and decision-level approaches, our fusion method conducts statistical inference on multiple views (through different features generated by different algorithms) of the raw images. As each view enjoys its own pros and cons, we employ confusion matrices to guide the class-prediction process through the least inter-class-confusion inference path. This approach not only avoids the curse of dimensionality as in feature-level ensemble application but also displays a superior before to decision-level methods by using advantages from different feature extraction algorithms. Our experiments were designed to address the following questions:. • How do feature extraction algorithms compare with one another?. • Given an image category, which feature extraction algorithm performs the best, and why? • What is the effect of the size of the training dataset?. • How does fusion perform compared with individual classifiers, and how do the results of fusion change with the number of training instances?. To answer the above questions, we conducted experiments on two datasets: an ImageNet dataset and a PicasaWeb dataset. The first dataset was constructed from ImageNet [12] . As our experiments aimed to study the effects that training dataset size has on feature extraction algorithms, we first selected 100 ImageNet categories (each with 1, 100 images), and 10 PicasaWeb categories (each of which we manually annotate for a total of 11, 000 images).. To make a fair comparison, feature vectors extracted from different algorithms were set to be the same length: 1, 000. This was implemented by setting the cluster number for color-texture and SIFT codebook, setting the patch number for HMAX, and setting the number of hidden units in the last layer for ConvNet. After extracting features for each image, the data were divided into training and testing sets for the supervised classifier. An 11-fold cross validation was performed. The data were randomly divided into 11 parts of equal size: 10 parts were used for training and the remaining one for testing. To evaluate the feature performance on different sizes of training instances, we also varied the training data from 1 to 1, 000 instances for each category (10, 000 when available) and evaluated the accuracy using the same 100 testing data set aside for each fold. When we performed small experiments with only a part of the training data, a random subset was sampled. We used SVMs for the supervised learning, with PSVM [7] for large-scale data. Figure 5 shows the overall results for the ImageNet dataset. When comparing algorithms, each algorithm excels in categories with certain characteristics. It is not very meaningful to compare average classification accuracies on all categories because the composition of the testbed can be biased toward some algorithms. For the ImageNet dataset, SIFT always achieved much better results than other algorithms because the dataset contains a large number of categories favoring SIFT (We will provide more analysis later). Therefore, we did not analyze the exact values for overall accuracy, but only focused on the accuracy trends with respect to the growth of the training instance number. We make two observations. First, when there are only several training instances, it is insufficient to learn a practical model. No scheme can perform well. For example, when there is only one training instance for each category, all algorithms just perform slightly better than a random guess. However, as the simple algorithms, such as CT, do not require much data to learn model parameters, their results are relatively good when training data are sparse. For example, when the number of training instances ranges from 10 to 100, CT achieves better results than HMAX and ConvNet. Second, when the amount of training data increases, complex models such as HMAX and ConvNet can improve the results significantly. On large datasets, they achieve a higher rate of accuracy, and outperform CT when the number of training data increases to approximately 200 and above.

O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Learning in games using the imprecise Dirichlet model We propose a new learning model for finite strategic-form two-player games based on fictitious play and Walley's imprecise Dirichlet model (1996, J. Roy. Statistical Society B 58, 3-57). This model allows the initial beliefs of the players about their opponent's strategy choice to be near-vacuous or imprecise instead of being precisely defined. A similar generalization can be made as the one proposed by Fudenberg and Kreps (1993, Games Econ. Behav. 5, 320-367) for fictitious play, where assumptions about immediate behavior are replaced with assumptions about asymptotic behavior. We also obtain similar convergence results for this generalization: if there is convergence, it will be to an equilibrium. This paper describes a new approach to learning for finite strategic-form two-player games in the setting of fictitious play. So, consider successive rounds of a game.. Suppose we are in the t-th round, and that at this point, each player has certain beliefs about which strategy his opponent will play in the next, (t + 1)-th, round.. After each round of the game the players change their beliefs. Indeed, we assume that each player can observe the strategy his opponent plays, which naturally leads to the setting of so-called fictitious play [2, 11] . After each round, this new information affects the player's beliefs about the strategy his opponent will play next. This updating of beliefs is what we call learning about the opponent's strategy choice.. In a Bayesian context, such beliefs are represented by a probability distribution on the opponent's strategies (a precise Dirichlet model or PDM), and learning is implemented by updating with Bayes's rule using a likelihood function relating observations to future strategies. The model we propose for representing and updating these beliefs generalizes this Bayesian approach. It is based on Walley's imprecise Dirichlet model or IDM [13] . Using this generalization is justified by the fact that sometimes the assumptions underlying the Bayesian approach are too strong. The most visible difference is that the beliefs are summarized by a convex set of expected mixed strategies, instead of only one. Now, given such beliefs about his opponent's next strategy choice, and the game's payoff function, which strategy should a player use in the next round to satisfy some optimality criterion? In a Bayesian context, optimal strategies maximize a player's expected payoff. As optimality criteria, we propose two generalizations of the concept of expectation maximization. The main consequence is that -in contrast to the Bayesian context -two different optimal strategies may be incomparable.. If both players use a method of learning and of choosing optimal strategies during the successive rounds of a game, will their behavior converge: Will the optimal strategies they select converge to an equilibrium of the game? 2 Or, when convergence occurs, will it be to an equilibrium? We investigate whether some interesting, existing results [5] for players using a PDM-based learning model can be generalized to the case where the players use our IDM-based learning model.. How is this paper organized? In Sec. 2 we describe its game-theoretic setting and introduce basic game-theoretic concepts and notation. In Sec. 3, we look at how a player can represent and update his beliefs about his opponent's strategy. Both the PDM-based Bayesian model and our IDM-based generalization are improvements of the classical model of fictitious play [2, 11] . In Sec. 4 we discuss the player's options for deciding on an optimal strategy. Most importantly, we investigate options that can be used in conjunction with the PDM-based and IDM-based models. Our contribution up until this point is mainly to show how well imprecise probability theory can be allied with game theory. Then, in Sec. 5, before concluding, this alliance is used to generalize results by Fudenberg and Kreps [5] about the convergence of play to equilibria. All proofs are collected in the Appendix at the end.

B O O O O B I I O O O O B O O O O B I O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Preserving privacy by de-identifying face images Abstract-In the context of sharing video surveillance data, a significant threat to privacy is face recognition software, which can automatically identify known people, such as from a database of drivers' license photos, and thereby track people regardless of suspicion. This paper introduces an algorithm to protect the privacy of individuals in video surveillance data by de-identifying faces such that many facial characteristics remain but the face cannot be reliably recognized. A trivial solution to de-identifying faces involves blacking out each face. This thwarts any possible face recognition, but because all facial details are obscured, the result is of limited use. Many ad hoc attempts, such as covering eyes, fail to thwart face recognition because of the robustness of face recognition methods. This paper presents a new privacy-enabling algorithm, named k-Same, that guarantees face recognition software cannot reliably recognize de-identified faces, even though many facial details are preserved. The algorithm determines similarity between faces based on a distance metric and creates new faces by averaging image components, which may be the original image pixels (k-Same-Pixel) or eigenvectors (k-Same-Eigen). Results are presented on a standard collection of real face images with varying k. T HERE has been a tremendous proliferation of video surveillance cameras in public locations such as stores, ATMs, schools, subway stations, and airports. For example, a recent survey of Times Square found 500 visible surveillance cameras in the area and 2,500 total in New York City [8] . The existence of so many cameras and proposals for even more cameras have caused many citizens to protest their use in Tampa, Florida, and Virginia Beach, Virginia, where video cameras are being used in conjunction with face recognition software [9] , [16] , thereby enabling the eventual possibility of tracking almost all of the people most of the time.. One could imagine in the near future the widespread use of video surveillance cameras, the subsequent sharing of that data, and the use of reliable face recognition software to explicitly identify and track people in real time. This extends beyond American society's current expectations, thereby challenging a person's "reasonable expectation of privacy" as provided in US law. Yet, there are many worthy purposes for sharing video surveillance data. Three examples include law enforcement, bioterrorism surveillance, and medical research.. In the case of law enforcement, obtaining (unaltered) video surveillance footage from its owner may require a search warrant, particularly if the footage was not recorded in a public space. Alternatively, if the subjects in the footage could be "anonymized," then authorities could view the tape beforehand to gain sufficient evidence for a search warrant as warranted [6] . A very specific search warrant could be issued to reveal the identity of an actor exhibiting suspicious behavior or possibly a witness who might have been looking in the direction of a crime taking place. Therefore, the technology described in this work could allow the general sharing of the tapes with law enforcement, while still providing traditional privacy protections to those who are not under suspicion.. Second, government agencies engaged in bioterrorism surveillance want to be able to observe signs of respiratory distress (e.g., coughing) in the general population to discover early evidence of an outbreak or a bioterrorism attack [22] . By observing anonymized video surveillance images, this work allows for the general sharing of video tapes originally captured for safety and security without revealing the identities of the individuals, in accordance with existing public health laws and to allay fears in the commercial sector who may supply video footage.. Third, using video data as a basis for conducting behavioral research is becoming quite common (e.g., Alzheimer's patients living in a specialized care unit [3] ). Because of the restrictions of the Health Information Portability and Accountability Act (HIPAA), this type of medical research is limited in its ability to widely share its recordings unless technology as described in this work can be used to reasonably anonymize the images. The identity of nonconsenting patients as well as employees would need to be concealed.. The goal of this work is to enable the sharing of video data with scientific assurances of privacy protection while keeping the data practically useful. Experiments reported herein demonstrate that many tactics done on television, such as covering the eyes, do not thwart face recognition software. What is needed is an algorithm to de-identify faces in video data such that many facial characteristics remain yet face recognition software cannot reliably identify subjects whose images are captured in the data. This work precisely defines the problem and provides an algorithm (k-Same) that is an effective solution. The algorithm guarantees privacy protection by identifying the "closest" k faces in a video clip and then replacing each of the k faces with the same "averaged" face. By sharing these minimally distorted de-identified images, we can restore the current expectation of privacy so that society does not have to choose sharing video data over privacy, but society can share video data freely with privacy protection.

O B O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O B O


Declarative Ajax and client side evaluation of workflows using iTasks Workflow systems coordinate tasks of humans and computers. The iTask system is a recently developed toolkit with which workflows can be defined declaratively on a very high level of abstraction. It offers functionality which cannot be found in commercial workflow systems: workflows are constructed dynamically depending on the outcome of earlier work, workflows are strongly typed, and they can be of higher order. From the specification, a web-based multi-user workflow system is generated. Up until now we could only generate thin clients. All information produced by a worker triggers a round trip to the server. For real world workflows this is unsatisfactory. Modern Ajax web technology to update part of a web page is required, as well as the ability to execute tasks on clients. The architecture of any system that supports such features is complex: it manages distributed computing on clients and server which generally involves the collaboration of applications written in different programming languages. The contribution of this paper is that we integrate partial updates of web pages and client side task evaluation within the iTask system, while retaining its approach of a single language and declarative nature. The workflow designer uses light-weight annotations to control the run-time behavior of work. The iTask implementation takes care of all the hard work under the hood. Arbitrary tasks (functional programs) can be evaluated at web clients. When such a task cannot be evaluated on the client for some reason, the system switches to server side evaluation. All communication and synchronization issues are handled by the extended iTask system. A workflow system is a computer system that coordinates the work that has to be done by human workers in collaboration with computers. Workflow systems are challenging real-world applications because they need to handle many things. First of all, a workflow system has to provide a way to specify workflows: what are the tasks that have to be done, how do these tasks depend on each other, and who should do them? The specification is used by the system at run-time for the real-time coordination and monitoring of the actual work being performed. Hence, somehow a mapping has to be made between the workflow specification and the real work that has to be done given the concrete human and software resources which are available. Daily work can be structured in quite a complex way which has direct consequences for the way tasks are depending on each other. The result of the work of one worker might determine the work of many others in both a positive or negative way. One needs a good understanding of how tasks depend on each other, and one also needs a sufficiently powerful specification formalism to express such complicated dependencies. In addition, one has to control a process which is quite dynamic: the amount and kind of work, the time it takes to do a job (ranging from split seconds to months), the number of available workers, the allocation of resources (both human, software, and hardware), they may all vary over time and may depend on the concrete work that takes place. Last but not least, one generally has to deal with a technically complicated distributed, heterogeneous environment: people working together all over the world using their own personal computer, pda's, mobile phone, and so on.. How to express this all? How to control this given a specification? It should be clear that a software system that can deal with all the above is bound to be complex. There exist many, mainly commercial, workflow systems. Examples are Business Process Manager, COSA Workflow, FLOWer, i-Flow 6.0, Staffware, Websphere MQ Workflow, and YAWL. Although these systems all have their own way of dealing with the challenges mentioned above, they also have a lot in common. Usually the systems are based on Petri-nets. The advantage is that dependencies between tasks can be depicted which makes them attractive to non-experts, while these drawings can straightforwardly be mapped to a corresponding Petri-net. This Petri-net is used at run-time as scheme to control the real work to do. Furthermore the net can be used as a formal model at compiletime to determine desired properties of the specified workflow: one can calculate reachability of a certain task or determine the absence of deadlock. The kind of task dependencies one can specify in these systems, the so-called workflow patterns, are summarized in [14] , together with a discussion of the systems mentioned above. However, the use of Petri-nets as semantic model also has big disadvantages. The nets are rather static and only first order: tasks cannot deliver new tasks to do. Hence they cannot be used to describe the dynamic way of working that takes place in the real world and therefore they can only be used for the specification of static tasks.. The main research question that we address in this paper has been asked to us by industry being confronted with the limitations of the current systems: can declarative programming, and functional programming in particular, provide new concepts and implementation methods and tools for workflow systems that can deal with the dynamic behavior of daily work? In answering this question, we have developed the iTask toolkit [12] as a first step towards a realistic workflow system. This toolkit is a web-based combinator library written in the lazy, purely functional programming language Clean. The novel and declarative contributions that this toolkit provides, which cannot be found in the existing commercial systems, are:. • workflows are constructed fully dynamically instead of statically: they can depend on the intermediate inputs and outputs that are yielded by workers and computations; • workflows can be higher-order, i.e. yield partially evaluated tasks which can be passed around for further evaluation to other workers at other locations; • workflow cases are specified as pure, strongly-typed functional expressions, using the predefined iTask combinators; • the workflow application can handle multiple workers, multiple tasks, and multiple clients dynamically, yet everything is controlled by one, single Clean application running on the server; • the specification of the workflow is executable; all implementation details like web-page generation, web-page handling, client-server communication and database storage handling is handled fully automatically by making intensive use of generic programming techniques [8, 3] : from the types being used the required code is generated fully automatically.. Tasks have to be offered to the workers in such a way that it is clear what they have to do. The iTask application generates, given the workflow specification and the work that has been done so far, an appropriate web page for each user. The key advantage of using browsers to display the work to do, is of course that they are available on any thinkable platform. No special software needs to be installed to connect iTask workflow users. Workflow systems are distributed software systems, hence it makes sense to not only deploy web technology for rendering purposes, but also for the distribution, communication, and control of tasks. However, although the web seems to be very suited for all this, it is actually technically quite difficult to realize the rendering and communication automatically from a given declarative workflow specification. Workflow systems exhibit state, support multiple users, and guide the flow of work. Neither of these concepts are readily supported by the web and hence additional software is needed for the realization. Commonly, a web application which guides a user through several working steps does not consist of one, single application. The implementation often consists of a collection of software applications and scripts, written in several languages, which somehow together do the job: one can think of HTML-code, php-scripts, Ajax-scripts, SQL-queries. Since they are commonly not generated from one single source code, it is very hard to design, implement and maintain systems which such an architecture. In the iTask system all software is generated from one single source in Clean. To understand what the application is doing, one only needs to look at the iTask specification. It is a specification on a very high level of abstraction which can be read as if we are dealing with an ordinary simple desktop application. We take full advantage of the fact that we are working with a pure functional language. First of all we solve the lack-of-state problem of the web, by using generic programming techniques to store the state of the interactive elements, the iTasks, only. Because we have the most recent states of the iTasks at our disposal, we only need to rerun the function that represents the program and provide it with the most recent input action of any worker to advance to the next state. This reduces the programming burden on the workflow developer. It allows her to focus on the workflow case, rather than its implementation. Another advantage of such an approach is that one obtains a clean separation between the workflow specification and its implementation.. In this paper we explain the use and implementation of two new important features added to the iTask toolkit. In the old system, any event received from an iTask user is handled by the single iTask application on the server. It computes the next state and calculates a whole new web page for a particular user showing her the new tasks to do. New web technology such as Ajax [6] , makes it possible to update only a part of a page. Updating only the relevant part of a page improves the behavior of the web application in a way that resembles desktop behavior. The first feature is that we incorporate partial page updates. This is a challenge since the iTask system dynamically calculates which part of the page has to be updated because it depends on the state of the task being performed and the state of the work of all other users. In most existing systems the part of the page to be updated is fixed rather than computed dynamically. The second feature is that we want that the program executing the tasks can run partially on the client instead of on the server. Client side evaluation is essential to eliminate delays associated with the communication between server and client. The impact of this feature can not be overestimated because it is fundamental to create coarse grained computational tasks on clients with rich interaction and quick response times (think of modern day web applications like Google Docs and gmail). There are three ways to obtain client side evaluation of tasks in a browser: plugins, JavaScript, or Java code. The disadvantages of plugins is the explicit installation that is required. In the current iTasks system we use Java since it seems better suited for the large applications that have to be run on the client than JavaScript.. Instead of a single server, one can also think of using several servers, as well as tasks that are migrating over the internet. Distribution of client tasks is also required when one wants to work with distributed document repositories that can be accessed by client workflows. This is not addressed in this paper, but will be subject of future research. The feature of client side workflows also challenges the underlying architecture of the iTask toolkit.. In this paper we show how these two major web techniques can nevertheless be incorporated smoothly within the iTask toolkit, while fully retaining its declarative nature:. • We rearrange the iTask toolkit in such a way that worker-tasks automatically use the asynchronous, partial page update technology that is offered by Ajax. Besides this default arrangement, we allow the workflow designer to annotate workflow expressions in a light-weight way to give fine-grained control of other parts of the workflow application.. • We rearrange the iTask toolkit in such a way that arbitrary workflow task expressions can be evaluated at the client side.. For the workflow designer this is only a matter of a simple annotation in the existing specification.. The workflow engineer can use the new contributions of the iTask toolkit by annotating ordinary iTask applications. The implementation of these new annotations is however rather challenging. We wish to evaluate complete task expressions on the client instead of the server which requires that we can evaluate full Clean code on the client side within the browser. Worker actions can have non-local effects, hence we need to implement some sort of synchronization. We show that this can be implemented without loss of the semantic model of the system without client side evaluation and partial page updates.. The declarative nature of the iTask toolkit is retained by implementing an evaluation strategy that can automatically switch between client side evaluation and server side task rewriting if necessary. The details are presented in Sections 5 and 6, but roughly speaking this means that the system can perform tasks on the client side (within a browser) as well as on the server side (within the server application). Moreover, if client side evaluation is no longer possible (because of a non-local effect of a remote worker, or because the local computation requires a server resource), the system automatically can continue to perform the computation on the server side. The workflow designer does not have to specify this, unlike other approaches as for instance in Hop [13, 10] (see also Sect. 7). This implies that the approach as described in this paper is not only more declarative, but also more robust: it can handle situations dynamically that would otherwise be considered programming errors.. The iTask toolkit has been created in Clean. A concise overview of the syntactical differences with Haskell is [2] . We assume the reader is familiar with the concept of generic programming.. We start with a short overview of the iTask combinator system in Sect. 2. The new annotations are introduced in Sect. 3. Their ease of usage contrasts strongly with their implementation. To understand why, we present the basic architecture of the standard implementation in Sect. 4. The high level specification of workflows offered by the iTask system is achieved due to the fact that the system is able to reconstruct the state of evaluation of all tasks of all users, the so called Task Tree. To avoid the Task Tree from growing infinitely, a task expression is rewritten by its result in a similar way as function applications are rewritten by their result. This is called Global Task Tree Rewriting. In Sect. 5 we discuss the implementation consequences of asynchronous partial page updates and introduce Local Task Tree Rewriting. In Sect. 6 we do the same for client side evaluation and introduce Client Side Local Task Tree Rewriting. Related work is presented in Sect. 7 and we conclude in Sect. 8. In this paper we have presented a number of contributions to the iTask toolkit, a combinator library written in Clean to create workflow systems that run on the web in a pure functional style. The contributions to the workflow engineer are that she can annotate arbitrary task structures with two annotations: UseAjax and OnClient. Task structures annotated with UseAjax can be handled much more efficiently by the toolkit by using the underlying Ajax technology. Using this Ajax technology only the part of the web page corresponding to the task that is changed is updated instead of the entire web page. Task structures annotated with OnClient can be evaluated completely on the client side. This requires the Ajax technology since the client side evaluation of a single task only updates that task and hence only a fragment of the web page.. The actual gain in efficiency cannot be predicted on forehand because it highly depends on the kind of workflow being specified. The standard evaluation strategy of iTasks is already reasonably efficient thanks to global task rewriting. Local task rewriting is more efficient when there are no global effects. Otherwise local task rewriting will not be more efficient, the use of Ajax even introduces some minor additional run-time overhead. Local task rewriting will be more efficient than global task rewriting when the workflow is large, there are many users, and global effects occur occasionally.. Client side evaluation by Sapl has as advantage that internet traffic is avoided and that server processing load is relieved, but as disadvantage that the Sapl interpreter is slower than the compiled code generated by the Clean compiler. For arithmetical operations Clean can be an order of magnitude faster, but when higher order functions are being calculated Sapl performs quite well. Whether it is better to calculate on the client therefore depends on the size of the task, the kind of computations being performed, the amount of traffic on the internet, the speed of the network, the speed of the server, and the speed of the client. For small tasks the overhead of interpretation on the client usually outweighs the communication delay, even on slow client machines with fast internet connections. It is up to the workflow engineer to make the optimal choices. In any case, the advantage of both the UseAjax and OnClient annotation is the elimination of blank browser windows when waiting for a new page.. The technical contributions are the incorporation of Ajax technology within the iTask toolkit, the ability to convert any Clean expression to a Sapl function call, the introduction of Task Tree rewriting strategies that can automatically switch when required by the tasks that they evaluate, and rearranging the architecture of the iTask toolkit to incorporate these changes. We have maintained the declarative approach of the iTask toolkit. Everything is generated from an annotated, single source specification with a low burden on the workflow designer because the system itself switches automatically between client and server side evaluation when this is necessary without any effort of the workflow engineer. The iTask system integrates all mentioned technologies in a truly transparent and declarative way. With the two new annotations, UseAjax and OnClient, the workflow engineer can control the evaluation of tasks in a light weight way. However, the implementation of these annotations is by no means light weight because it needs to handle many issues. One issue is that for every worker event it needs to figure out which Ajax thread (if any) has to handle the event. However, the event may cause the associated task to terminate. In that case the Ajax thread has to terminate as well, and the parent thread has to be activated to determine the next tasks to deal with. This can result in a cascade of activated-terminated Ajax threads. Another issue is that due to non-locality of worker actions tasks may disappear, and consequently also their associated Ajax threads. In these cases the evaluation strategy has to resort to the standard evaluation technique. Switching of evaluation strategy is also vital for those OnClient tasks for which it turns out that they cannot be evaluated on the client. Using the Ajax infrastructure allows the iTask toolkit to turn a failing OnClient task automatically into a UseAjax task, and hence have the task calculated on the server. This can only be done if the server has either full knowledge of the states of all clients, or if it can completely reconstruct their state on demand. The iTask toolkit uses the latter strategy, and this is explained in Sect. 4 . After that, we show how Ajax technology is incorporated in Sect. 5 and client side evaluation in Sect. 6.. In this section we have shown how to incorporate Ajax threads in the iTask toolkit. As we have explained in Sect. 3, we have chosen to turn every worker task (i.e. a task assigned to a worker with the @: function) into an Ajax thread. As a result, the page that is displayed to a worker consists of a set of tasks, each of which can be handled individually by the worker without the need to wait for the full page to reload. The latter is only necessary in case her action has caused a non-local effect. In this way, the user experiences a smoothly operating workflow application. The workflow engineer can further fine-tune the workflow application by adding UseAjax annotations in the right places.. With the OnClient annotation workflow engineers can create workflow systems that are able to compute arbitrarily complex computations on clients. This can significantly reduce the traditional roundtrip communication between clients and server, it will reduce the workload of server applications, and it will enhance the worker experience as the workflow can respond quicker than in the old setting. Every workflow system has to be aware of non-local effects in any kind of implementation, and the iTask toolkit automatically detects whether this is the case and global Task Tree rewriting is required. The workflow engineer immediately profits from this approach because she does not need to concern herself with the question what parts of the local computations must take place on the client and what parts must be done on the server. The iTask toolkit automatically switches to global Task Tree rewriting in case a client task performs such work. The new iTask toolkit as described in this paper allows high level specification of multi-user workflows. Forms are generated generically from type information, which considerably decreases the amount of HTML programming. Complex dynamic workflows can be created that can be evaluated at both the client and the server without any restriction. Actions of workers can safely affect that of co-workers: the toolkit supports multi-user programming smoothly. The system is robust: computations that cannot be evaluated at the client side can always, and safely, be evaluated at the server side. We are not aware of any other functional system that has these features. However, there are functional approaches for handling web pages.. Links [4] and its recent extension formlets [5] is a functional language based web programming language. Links compiles to JavaScript for rendering HTML pages, and SQL to communicate with a back-end database. A Links program stores its session state at the client side. In a Links program, the keywords client and server force a top-level function to be executed at the client or server respectively. In Links processes can be spawned, and these processes can communicate via message passing. Clientserver communication is implemented using Ajax technology. In iTasks processes are not created explicitly as in Links programs. The novel UseAjax and OnClient are similar to Links functionality, except that we do not limit their use to top-level functions, but instead allow any (nested) task to be annotated. In the iData and iTasks toolkits, forms are generated generically for every data type, whereas in Links and formlets these need to be coded by the programmer. Links and formlets are designed for form based web applications, as is the iData toolkit, whereas the iTask toolkit extends this with multi-user workflow, including recursive, higherorder workflows.. Another functional language based web programming language is Hop [13, 10] . Just as Links, Hop is compiled to JavaScript. It implements a strict separation between programming the user interface and the logic of an application. The main computation runs on the server, and the GUI runs on the client(s). These components can invoke each other (from GUI client to server via function calls, from server to client via signalling events). In particular the latter feature increases the expressive power of web applications, which are usually driven by the browser client side. Hop is a stratified language: each component is programmed in either one stratum in order to prevent it from performing operations that are considered to be illegal (e.g. database access by the GUI client, or rendering operations by the server application). Annotations control what stratum is used: within the main stratum (the server application code)~es-capes to the GUI stratum, and within the GUI stratum one uses $ to escape to the server. Additional server logic can be invoked as a Hop service, which makes the design very modular. This has been implemented with Ajax. In the iData and iTasks toolkits, we do not require a stratified language approach to divide our attention to GUI programming versus application logic, because the GUI is mainly generated generically. The novel iTask annotations UseAjax and OnClient also allow fine-grained tuning of application logic, whereas an "exit strategy" is always available by relying on Global Task Rewriting strategy.. The Flapjax language [1] is an implementation of functional reactive programming in JavaScript. Many of its features are comparable with those of Hop, and indeed both are designed to create intricate web applications. The main difference with our approach is that the iTask system is geared for distributed, multi-user, workflow systems in which the coordination and interaction of work is defined in a highly declarative style.. The enabling technology of client-side evaluation in the iTask toolkit is Sapl. We have seen that the complete Clean application is compiled to Sapl. This enables our approach to use the full expressive power of Clean to perform intricate computations at the client side. A much more restricted approach has been implemented in Curry [7] : only a very restricted subset of Curry is translated to JavaScript to handle client side verification code fragments only.

O B O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O B O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Help features in digital libraries: types, formats, presentation styles, and problems Purpose -The purpose of this paper is to evaluate Help features in digital libraries and identify problems related to their design. Design/methodology/approach -This study selects six digital libraries to represent a variety of digital libraries developed or sponsored by different types of organisations. The Help features of these selected digital libraries are examined by their types (explicit versus implicit), formats (texts, images, screenshots, multimedia materials, and interactive formats), and presentation styles (descriptive, guided, procedural, and exemplary). Findings -This study presents the types of Help features available in the selected digital libraries, and further characterises the formats and presentation styles of these Help features. In the process of analysis, the author also identifies six types of problems: lack of standards; tradeoff between using explicit Help and implicit help; tradeoff between using general Help versus specific Help; lack of interactive Help features; lack of dynamic presentation styles; and lack of Help features for advanced users and users who do not understand English. Research limitations/implications -In order to design Help features that facilitate users to effectively use digital libraries, further research needs to extend studies to what types of help-seeking situations users generally encounter and the corresponding support they need. Originality/value -This study provides insightful information regarding the current status and problems of the Help features in existing digital libraries. Help features are not separate entities of digital libraries. To some extent, the overall interface design is the most important Help feature itself. The more intuitive an interface, the less help a user needs. An intuitive design is self-explanatory, which can reduce problems that users might encounter in their interaction with digital libraries.. That is why in this study, Help features refers to any features that assist users to effectively use digital libraries except general search and browse functions.. This study provides insightful information regarding the current status of the Help features available in existing digital libraries. It is interesting to note that there are a variety of implicit Help features in digital libraries, but the problem is how to label them, standardise them, and organise them, and more important, how to introduce them to novice users. As it is impossible to design a one-size-fits-all Help for different levels of users, and for users with different types of tasks, problems, and knowledge structure, it is essential to design interactive Help features that allow users to engage in communication with digital libraries or librarians via digital libraries.. In order to design Help features that are really helpful to users, further research needs to extend studies to what types of help-seeking situations users generally encounter and the corresponding support they need. It is also important for further research to involve real users with real problems at real settings in their information-searching process when they interact with a variety of IR systems. The results of this study are reported to answer the two proposed research questions. view-and-use-related Help.

O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Agent Technologies for Sensor Networks networks consist of small, battery-powered devices that are physically distributed over a wide area and connected through a wireless communication network. Since these networks often must collect data over extended periods of time and are deployed in inhospitable environments where replacing batteries is inconvenient or impossible, much of the research in this domain addresses the challenge of minimizing each sensor's energy needs. To this end, researchers have developed a wide range of energy-effi cient sensor nodes and wireless communication protocols and demonstrated them in varied applications.In addition to the immediate concerns of energy effi ciency and reliable wireless communication, which are normally the concerns of electronics engineers, the distributed nature of such networks and the autonomous behavior expected of them present several more generic challenges. Specifi cally, the individual sensors in these networks must typically coordinate their sensing actions with nearby sensors to achieve systemwide goals (for example, varying their sense/sleep duty cycles to maximize battery life while reducing the redundant sensing of overlapping areas). Furthermore, the network must autonomously adapt its responses in a dynamically changing environment such that it can achieve the long-term systemwide goals without direct human intervention. Such problems have long been the domain of computer scientists, particularly those researching multiagent systems. As such, the multiagent systems community would seem to have an extensive set of formalisms, algorithms, and methodologies in place to address these challenges.However, the mapping from sensor to agent isn't trivial. Research in the multiagent systems domain typically doesn't address the constrained computational and communication resources of low-powered sensor nodes. Moreover, research often fails to consider that communication might be slow and intermittent, hardware might be unreliable and failure prone, and environments might be highly dynamic. As such, although existing agent technologies are extremely valuable, they cannot be used directly. Rather, to address the specifi c constraints and challenges posed by this application setting, we need a new synthesis that adapts and extends traditional technologies using approaches from other disciplines such as electronic engineering.Against this background, this article describes three examples where this synthesis has succeeded. One showcases the development of communication and computationally effi cient decentralized coordination algorithms to coordinate the behavior of physically distributed sensors. The second addresses the real-world challenges of using sensoragent platforms in the fi eld. Finally, we describe intelligent agents that can autonomously acquire data from these networks and perform informationprocessing tasks such as fusion, inference, and prediction. In each case, researchers have demonstrated the work in the wild, implemented it on real sensor hardware, deployed it in real, hostile environments, and evaluated it on real sensor data.As we mentioned, coordinating the activities of physically distributed devices to achieve good systemwide performance is a fundamental challenge. Such coordination might include routing data through the network, choosing the appropriate sampling rates of sensors that exhibit spatial correlations, or determining W ireless sensor networks are increasingly seen as a solution to the problem of performing continuous wide-area monitoring in many environmental, security, and military scenarios. Such

B I O B I B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O


Effective partitioning method for computing weighted Moore-Penrose inverse We introduce a method and algorithm for computing the weighted MoorePenrose inverse of multiple-variable polynomial matrix and the related algorithm which is appropriated for sparse polynomial matrices. These methods and algorithms are generalizations of algorithms developed in [24] to multiple variable rational and polynomial matrices and improvements of these algorithms on sparse matrices. Also, these methods are generalizations of the partitioning method for computing the Moore-Penrose inverse of rational and polynomial matrices introduced in [22] and [23] to the case of weighted MoorePenrose inverse. Algorithms are implemented in the symbolic computational package MATHEMATICA. Let C m×n be the set of m × n complex matrices, and C m×n r is the set of m × n complex matrices of rank r: C m×n r = {X ∈ C m×n | rank(X) = r}. For any matrix A ∈ C m×n and positive definite Hermitian matrices M and N of the order m and n respectively, consider the following equations in X, where * denotes conjugate and transpose:. (1) AXA = A (2) XAX = X (3) (M AX) * = M AX (4) (N XA) * = N XA.. The matrix X satisfying these equations is called the weighted Moore-Penrose inverse of A, and it is denoted by X = A introduced in [22] . An extension of results from [22] to the set of two-variable rational and polynomial matrices is introduced in the paper [23] . In our recent paper [24] we propose an algorithm for computing the weighted Moore-Penrose of one-variable rational and polynomial matrix. In this work we generalized the results from [24] in the following two ways:. -extends algorithms from [24] to the set of multi-variable rational and polynomial matrices with complex coefficients, -make algorithms from [24] more effective on sparse matrices with a relatively small number of nonzero elements.. The structure of the paper is as follows. In the second section we extend the algorithm for computing the weighted Moore-Penrose from [20] to the set of multiplevariable rational matrices with complex coefficients. Main results are given in the third and the fourth section. In Section 3 we adapt previous algorithm to the set of polynomial matrices. In the fourth section we consider two effective structures which exploit only nonzero addends in polynomial matrices and improve previous results on the set of sparse matrices. In the last section we presented an illustrative example and compared various algorithms. We extend the algorithm for computing the weighted Moore-Penrose from [20] to the set of multiple-variable rational matrices with complex coefficients. We adapt previous algorithm to the set of polynomial matrices. We consider two effective structures which make use of only nonzero addends in polynomial matrices and improve previous results on the set of sparse matrices. In the last section we presented an illustrative example and compared various algorithms.

O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O


Balancing Bounded Treewidth Circuits We use algorithmic tools for graphs of small treewidth to address questions in complexity theory. For our main construction, we prove that multiplicatively disjoint arithmetic circuits of size n O(1) and treewidth k can be simulated by bounded fan-in arithmetic formulas of depth O(k 2 log n). From this we derive an analogous statement for syntactically multilinear arithmetic circuits, which strengthens the central theorem of M. Mahajan and B.V.R. Rao (Proc. 33rd International Symposium on Mathematical Foundations of Computer Science, vol. 5162, pp. 455-466, 2008). We show our main construction has the following three applications:• Bounded width arithmetic circuits of size n O(1) can be balanced to depth O(log n), provided chains of iterated multiplication in the circuit are of length O(1).• Boolean bounded fan-in circuits of size n O(1) and treewidth k can be simulated by bounded fan-in formulas of depth O(k 2 log n). This strengthens in the non-uniform setting the known inclusion that SC 0 ⊆ NC 1 .• We demonstrate treewidth restricted cases of DIRECTED-REACHABILITY and CIRCUIT VALUE PROBLEM that can be solved in LogDCFL.We also give a construction showing, for both arithmetic and Boolean circuits, that any circuit of size n O(1) and treewidth O(log i n) can be simulated by a circuit of width O(log i+1 n) and size n c , where c = O(1), if i = 0, and c = O(log log n) otherwise. It is well-known that many hard graph theoretical problems become tractable when restricted to graphs of small treewidth. 1 For a graph with n nodes and treewidth bounded by some absolute constant, there always exists a balanced tree decomposition of depth O(log n). This yields NC-algorithms for many problems, which are known to be NP-complete in general [4] .. Consider the following question. Suppose one is given a circuit (Boolean or arithmetic) of size s and bounded fan-in, for which the underlying graph has constant treewidth. Does this imply, as intuition might suggest, that there must exist an equivalent bounded fan-in circuit of size poly(s) and depth O(log s) ? We show that in the Boolean case the situation is as expected resulting in the following theorem:

O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Geometry

Constructions

Language

Abstract

Geometry

Constructions

Language

(gcl)

is

a

language

for

explicit

descriptions

of

constructions

in

Euclidean

plane

and

of

their

properties.

Other

mathematical

objects

can

also

be

described

in

the

language.

The

language

gcl

is

intuitive

and

simple,

yet

it

supports

arrays,

flow

control

structures,

user-defined

procedures,

etc.

The

processors

for

the

gcl

language-applications

gclc

and

Wingclc-enable

visualization

of

described

objects

and

producing

of

mathematical

illustrations,

provide

different

semantical

information

and

support

for

automated

proving

of

properties

of

the

constructed

objects.

These

features

make

the

tools

gclc

and

Wingclc

powerful

mechanized

geometry

systems

and

they

have

thousands

of

users

worldwide.

Euclidean

geometry

and

geometric

constructions

have

important

role

in

mathematics

and

in

mathematical

education

for

thousands

of

years.

In

twentieth

century,

there

was

a

shift

from

classical,

synthetic

geometry

in

favor

of

algebraic

geometry

in

university

education.

However,

synthetic

geometry

still

holds

a

very

important

position

in

lower

levels

of

mathematical

education

and

also,

in

recent

years,

it

has

been

making

a

comeback

to

university

education,

thanks

to

important

applications

in

computer-aided

design,

computer

graphics,

computer

vision,

robotics,

etc..

There

is

a

range

of

geometry

software

tools,

covering

different

geometries

and

geometry

problems.

Many

of

them

focus

on

Euclidean

geometry

and

on

construction

problems.

These

problems

are

very

suitable

for

interactive

work

and

animations,

Various

aspects

of

gcl,

gclc,

and

Wingclc

are

described

in

publications

referred

to

in

the

following

text,

in

appropriate

parts..

Overview

of

the

paper

The

rest

of

the

paper

is

organized

as

follows:

Section

briefly

introduces

Euclidean

geometric

constructions,

Section

describes

the

geometry

constructions

language

gcl,

while

Section

describes

its

processors,

Section

reviews

main

areas

of

applications

of

gcl

and

its

processors,

and

Section

discusses

related

languages

and

tools.

In

Section

we

draw

final

conclusions

and

we

briefly

discuss

further

development

of

the

language

gcl

and

its

processors.

We

presented

the

Geometry

Constructions

Language

(gcl),

a

language

for

explicit

describing

constructions

in

Euclidean

plane

and

their

properties.

The

basic

idea

behind

the

gcl

language

is

that

geometric

constructions

are

formal

procedures,

rather

than

drawings.

Therefore,

in

gcl,

one

explicitly

describes

constructions

and

figures,

rather

than

draws

figures..

After

years

of

development,

the

gcl

processor

gclc

is

much

more

than

a

geometry

and

visualization

tool.

There

is

support

for

symbolic

expressions,

for

parametric

curves

and

surfaces,

for

flow

control,

etc.

The

built-in

theorem

provers

can

automatically

prove

a

number

of

complex

geometry

theorems.

Wingclc

is

an

interactive

version

of

gclc,

a

dynamic

mathematical

tool

with

a

range

of

functionalities.

These

features

make

the

tools

gclc

and

Wingclc

powerful

mechanized

geometry

systems,

with

the

main

applications

in

producing

digital

mathematical

illustrations,

storing

mathematical

contents,

mathematical

education,

and

in

studies

of

automated

reasoning

in

geometry.

The

systems

are

publicly

available

and

have

thousands

of

users

worldwide..

Modern

mathematical

tools

have

to

combine

powerful

support

for

visualization,

symbolic

and

numerical

calculations,

and

for

automated

theorem

proving.

The

tools

gclc

and

Wingclc

aim

at

this

direction.

Possible

directions

for

further

development

of

the

language

gcl

and

its

processors

are

in:

improving

support

for

symbolic

and

numeric

calculations

(e.g.,

adding

support

for

symbolic

derivations,

numeric

integration,

etc.),

in

developing

new

theorem

provers,

able

to

generate

human-readable

proofs

(e.g.,

based

on

coherent

logic

in

developing

tools

for

automated

solving

of

construction

problems,

and

in

developing

support

for

automated

discovery

of

geometry

theorems.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O