Compressed Sensing Signal and Data Acquisition in Wireless Sensor Networks and Internet of Things Abstract-The emerging compressed sensing (CS) theory can significantly reduce the number of sampling points that directly corresponds to the volume of data collected, which means that part of the redundant data is never acquired. It makes it possible to create standalone and net-centric applications with fewer resources required in Internet of Things (IoT). CS-based signal and information acquisition/compression paradigm combines the nonlinear reconstruction algorithm and random sampling on a sparse basis that provides a promising approach to compress signal and data in information systems. This paper investigates how CS can provide new insights into data sampling and acquisition in wireless sensor networks and IoT. First, we briefly introduce the CS theory with respect to the sampling and transmission coordination during the network lifetime through providing a compressed sampling process with low computation costs. Then, a CS-based framework is proposed for IoT, in which the end nodes measure, transmit, and store the sampled data in the framework. Then, an efficient cluster-sparse reconstruction algorithm is proposed for in-network compression aiming at more accurate data reconstruction and lower energy efficiency. Performance is evaluated with respect to network size using datasets acquired by a real-life deployment.Index Terms-Compressed sensing (CS), enterprise systems, industrial informatics, information systems, Internet of Things (IoT), wireless sensor networks (WSNs). In this paper, we first proposed a CS framework for WSNs and IoT and introduced how the framework could be utilized to reconstruct the sparse or compressible information data into a variety of information systems involving WSNs and IoT. The CS-based framework provides a promising approach for compressible signal and data in information systems by employing a priori data sparsity information, which makes it an effective new information and data gathering paradigm in networks and information systems.. This work has shown that CS can be a powerful data acquisition tool for saving energy and communication resources in networks and informations systems. It further strengthens the connection between information theory and CS. As part of our ongoing work, we are investigating the best achievable CS reconstruction schemes over information systems and networks.

O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I B I I I B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O


DSP-Based Parallel Implementation of Speeded-Up Robust Features Robust local image features have become crucial components of many state-of-the-art computer vision algorithms. Due to limited hardware resources, computing local features on embedded system is not an easy task. In this paper, we propose an efficient parallel computing framework for speeded-up robust features with an orientation towards multi-DSP based embedded system. We optimize modules in SURF to better utilize the capability of DSP chips. We also design a compact data layout to adapt to the limited memory resource and to increase data access bandwidth. A data-driven barrier and workload balance schemes are presented to synchronize parallel working chips and reduce overall cost. The experiment shows our implementation achieves competitive time efficiency compared with related works. Robust local image features are widely used in many stateof-the-art computer vision algorithms such as calibration, object recognition and motion tracking. As a sound representative, scale invariant feature transform (SIFT) is known for its distinctiveness against image scaling, rotation and view point change [1] . However, its excellent matching result is acquired at the cost of a heavy computation burden. This disadvantage is also common for other local features like maximally stable extremal region (MSER) [2] , and limits their application to PC systems only. To relieve this burden, speeded-up robust features (SURF) is proposed as a light weighted alternate while maintaining a similar performance [3] . Meanwhile, modern DSP chips now acquire more strength with less energy consumption as the hardware techniques develop. There emerges a possibility to extract local features on embedded systems within reasonable time.. Sinha [4] and Cornelis [5] ported SIFT and SURF to graphics processing unit (GPU). GPU plus CPU obtain impressive processing speed, but they still require a PC system and more power supply. Special purpose hardwares on FPGA can provide a good balance between speed and energy, these works are reported in Ehsan's survey [6] . Ko [7] implemented SIFT on a BlackFin DSP inside a wireless camera network. Their solution is not very fast due to the complexity of SIFT and the low speed of DSP chip. Arth [8] implemented MSER on a single fixed-point DSP for an object recognition system, which is most similar to our work. However, we place more emphasis on improving the efficiency by computing in parallel with multiple DSP chips. Zhang [9] decomposed SURF modules into parts and computed with several threads on quad-core CPU. This work is also parallel and achieves real-time performance. But it is not very suitable for DSP systems due to platform discrepancies in memory usage and synchronization.. In this paper, we propose an efficient parallel computing framework for speeded-up robust features oriented towards multi-DSP based embedded system. With a major concern on latency, we optimize a few modules with high call-frequency to improve the efficiency. Due to limited internal memory, we also design a compact data layout to increase data access bandwidth. A data-driven barrier and workload balance schemes are presented to synchronize parallel working chips and reduce overall cost. In this paper, we present a fast parallel implementation of SURF on DSP-based embedded systems. We investigate octave-level parallelism and study the key problems of memory layout, workload balance and synchronization. We achieve competitive time efficiency with 2 TigerSHARC chips compared with related works. In the future, we will further explore the possibility to incorporate more chips and reduce the overhead in multi-core synchronization.

O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Complexity-Regularized Image Denoising We develop a new approach to image denoising based on complexity regularization. This technique presents a flexible alternative to the more conventional l 2 , l 1 , and Besov regularization methods. Different complexity measures are considered, in particular those induced by stateof-the-art image coders. We focus on a Gaussian denoising problem and derive a connection between complexity-regularized denoising and operational rate-distortion optimization. This connection suggests the use of efficient algorithms for computing complexity-regularized estimates. Bounds on denoising performance are derived in terms of an index of resolvability that characterizes the compressibility of the true image. Comparisons with state-of-the-art denoising algorithms are given. Due to the imperfection of image acquisition systems and transmission channels, images are often degraded by noise. The presence of noise is visually annoying and makes it more difficult to perform tasks such as segmentation, recognition, and scene interpretation. It then becomes desirable to construct a good estimate of the original image from the noisy observations.. In this paper, we consider the typical problem of estimating an image degraded by additive white Gaussian noise (AWGN). The problem is formally described as:. where f * is the unknown original image, is white Gaussian noise with zero mean and known variance σ 2 , and y denotes the noisy observations. We restrict our attention to digital images with N pixels. In this case, f * , and y are length-N vectors. The problem is to design a good estimator f for the original image based on the observations y.. A classical framework in which to solve such image estimation problems is penalized maximumlikelihood (ML) estimation [1] . The estimator is given bŷ. where Φ(f) is the penalty or regularization functional, which takes into account a priori knowledge about f * to penalize "unlikely" estimates and stabilize the ML estimator. The regularization parameter µ controls the tradeoff between the log-likelihood term and the regularization penalty.. The choice of Φ(f) is critical to estimation performance. The estimatorf in (2) may also be thought of as the maximum a posteriori (MAP) estimator corresponding to the prior p(f ) ∝ e −µΦ(f ) .. A conventional choice for the penalty function Φ(f) is the quadratic penalty µ||Cf|| 2 (implicitly assuming a Gaussian prior on f ), where C is a differentiating operator [2, 3, 4] . This choice penalizes large local variations in the image. For the Gaussian observational model (1), the estimatef is linear in the data y. However, the estimate often suffers from oversmoothing artifacts [5, 6] , because sharp transitions are too heavily penalized by the quadratic penalty. One way to fix this problem is to use a data-adaptive (spatially varying) regularization parameter µ [4, 7] . Other forms of penalty functions have also been studied, such as l 1 norms [8] , Besov norms [9] , total-variation norms [10, 11] , and Huber functions [5, 6] , which penalize large local variations less heavily in order to preserve edges. This paper has applied the concept of complexity regularization to image denoising under an AWGN model. Complexity regularization is not restricted to this particular framework. Our performance analysis of the complexity-regularized estimator (4) is extended in [23] to more general inverse problems such as image restoration with possibly nonlinear blur, and noise with non-Gaussian (e.g., Poisson or multiplicative noise) and non-white statistics. The natural estimation loss in this case is relative-entropy loss, and it is shown that the estimation risk is still upper bounded by a linear function of a suitably defined index of resolvability. A theoretical discussion of the optimal codebook design problem is also presented in [23] .. A Proof of Theorem 3.1.. First we present a lemma.. Lemma A.1 Let {U i , 0 ≤ i < N } be a set of independent Gaussian random variables with sample. For all τ ≥ 0 and ≥ 0, we have:. Proof: Note thatŪ − EŪ is Gaussian distributed with mean 0 and standard deviation σŪ = V ar(Ū ). For all τ ≥ 0, and ≥ 0, we have. by definition of Marcum's Q function, Q(x) = 2 . Hence we obtain (8). The idea of penalizing estimates with high complexity also appears in Natarajan's Occam filtering technique, which uses lossy data compression to filter noisy data [21] . Using noisy data y as the input to a data compression algorithm, and setting the distortion level equal to the noise variance σ 2 , the algorithm takes the output of the compression algorithm as the estimatef (see Fig. 1 ). The method is named after the principle of Occam's Razor, which states that "the simplest explanation of the observed phenomenon is more likely to be correct." The Occam filter technique has found a theoretical justification in the case of bounded additive noise [21] . However, Natarajan's analysis is not applicable to unbounded noise models such as Gaussian distributions. Unlike (3), his method may also be difficult to extend to problems other than denoising.. The basic theory of complexity regularization was developed by Barron [22] in the context of nonparametric regression. Some extensions of that theory are developed in this paper and in [23] .

O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O


Image Tag Refinement by Regularized Latent Dirichlet Allocation Tagging is nowadays the most prevalent and practical way to make images searchable. However, in reality many manually-assigned tags are irrelevant to image content and hence are not reliable for applications. A lot of recent efforts have been conducted to refine image tags. In this paper, we propose to do tag refinement from the angle of topic modeling and present a novel graphical model, regularized Latent Dirichlet Allocation (rLDA). In the proposed approach, tag similarity and tag relevance are jointly estimated in an iterative manner, so that they can benefit from each other, and the multi-wise relationships among tags are explored. Moreover, both the statistics of tags and visual affinities of images in the corpus are explored to help topic modeling. We also analyze the superiority of our approach from the deep structure perspective. The experiments on tag ranking and image retrieval demonstrate the advantages of the proposed method. The community-contributed multimedia content in the internet, such as Flickr, Picasa, Youtube and so on, has been exploding. To facilitate the organization of the uploaded images or videos, media repositories usually offer a tool to enable consumers to manually assign tags (a.k.a. labels) to describe the media content [1] . These assigned tags are adopted to index the images to help consumers access shared media content.. Reliable tagging results in making shared media more easily accessible to the public. However, the reliability of tagging is not guaranteed in that the tags may be noisy, orderless and incomplete [17] , possibly due to carelessness of the taggers. First, some tags are noises and may be irrelevant to media. According to the statistics in Flickr, there are about only 50% tags indeed relevant to photos [17, 10] . Second, different tags essentially have different relevance degrees to the media, but such information is not indicated in the current tag list, where the order is given according to the input sequence. We did an analysis on the MSRA-TAG dataset [24] , which was crawled from Flickr, about what percentage of images have the most important tags in different positions. A statistics figure is shown in Fig.1 to indicate the result. It can be observed from this statistics that less than 20% images have the most relevant tags at the top position, which shows that the tags are almost in a random order in terms of the relevance. Last, the tags of some photos are incomplete due to the interest limitation of taggers, and even not given.. We address the problem of refining the tags, to facilitate the access of the shared media. To be specific, we investigate the tagging problem in Flickr, one of the most popular photo sharing web sites, and propose to reorder the tags. The available information to refine the tags consists of manual tags and image affinity.. 1. Although they are not completely reliable, the manual tags still reflect the photo content in some degree and their relations can be explored for tag refinement. Existing solutions only make use of the pairwise relation between tags, mined from WordNet [31] , or estimated from Web photo tags [21, 22, 24, 35] . 2. Visually similar images usually tend to have similar semantics and hence have similar tags, which means that the tag refinement of one image may benefit from those of other images. The typical exploration [24, 22] is to utilize the visual popularity of one image among the images having the same tag as a cue to estimate the relevance of the tag with this image.. In this paper, we present a novel probabilistic formulation, to estimate the relevance of a tag by considering all the other images and their tags. To this goal, we propose a novel model called regularized latent Dirichlet allocation (rLDA), which estimates the latent topics for each document, with making use of other documents. The model is applicable in tag refinement due to the observation that the content of an image essentially contains a few topics and the reasonable assumption that the tags assigned to the image accordingly form a few groups. The latent topics are estimated by viewing the tags of each image as a document, and the estimation also benefits from other visually similar images by the regularization term, instead of the estimation by LDA only from the corresponding document. The main contribution of our approach lies in the following aspects. On the one hand, both LDA and rLDA explore the multiple-wise relation among tags through the latent topics, rather than pairwise relations in the random walk based methods. On the other hand, the tag relevance estimation from rLDA can be interpreted using the deep structure [3, 13] . Compared with random walk and LDA, our approach is the deepest, and the illustration is presented in Figure 2 . This paper presents a regularized latent Dirichlet allocation approach for tag refinement. Our approach succeeds from the factors: (1) Our approach explores the multi-wise relationship among the tags that are mined from both textual and visual information; (2) Our approach explores a deep structure that has large capability to refine tags. Experimental results also demonstrate the superiority of our approach over existing state-of-the-art approaches for tag refinement. The automatic image tagging or annotation problem is usually regarded as an image classification task. Typical techniques [2, 4, 7, 9, 11, 12, 14, 16, 20, 28, 29, 33] usually learn a generative/discriminative multi-class classifier from the training data, to construct a mapping function from low level features extracted from the images to tags, and then predict the annotations or tags for the test images. Later, a more precise formulation is presented to regard it as a multi-label classification problem by exploring the relations between multiple labels [15, 30] .. The automatic annotation techniques have shown great successes with small scale tags and the well-labeled training data. But in the social tags, e.g., image tags on Flickr, there exist noisy or low-relevance tags, and the vocabulary of tags is very large, which limits the performance of conventional automatic tagging techniques in social tagging. The study in [17] has shown that classifiers trained with Flickr images and associated tags got unsatisfactory performance and that tags provided by Flickr users actually contain noise. Moreover, the relevance degrees of the tags, i.e., the order of the tags, are not investigated in automatic annotation.. Various approaches have been developed to refine tags using the available tags and visual information. The following reviews some closely-related methods, and more discussions can be found from a survey [34] . The straightforward approach directly exploits the tag relation, e.g., co-occurrence relation mined from WordNet [26] , or the internet, and then refines tags [31, 35, 19] . For example, the tag ambiguities are resolved [35] by finding two tags that appear in different contexts but are both likely to co-occur with the original tag set and then presenting such ambiguous tags to users for further clarification. The random walk approach over the pairwise graph on the provided tags with edges weighted by the tag similarities is presented in [32, 24] . The visual information is proved very useful to help tag refinement. For example, the neighborhood voting approach [21] is to recommend the tags by exploring the tags of the visually similar images. The likelihood that a tag is associated with an image is computed in [32, 24] from probabilistic models learnt the images assigned with such a tag, and then put it into the random walk framework for further refinement. A hybrid probabilistic model [38] is introduced to combine both collaborative and content based algorithms for tagging, which is similar to [24] in using the visual contents. A RankBoost based approach [36] is presented to learn a function to combine ranking features from multi-modalities, including tag and visual information. An optimization framework [23] is proposed to perform tag filter and enrichment by exploring visual similarity and additional knowledge from WordNet [26] . An approach [39] formulates the tag refinement problem as a decomposition of the user-provided tag matrix into a low-rank matrix and a sparse error matrix, targeting the optimality by low-rank, content consistency, tag correlation and error sparsity.. Rather than exploring the pairwise relation among tags, some techniques are proposed to adopt the multiple wise relations among tags, through latent models. Latent topic models, alternatives of latent Dirichlet allocation, is adopted [19, 18, 6] to learn a generative model from the tags, which then can estimate the posterior probability that a tag is associated with an image. Those methods are limited in lack of capabilities of adopting visual information. Therefore, this paper proposes a novel topic model, called regularized latent Dirichlet allocation, to estimate the topic models with exploiting the visual information. The latent topic based models are also justified by the conclusion in [31] that these tags assigned to images span a broad spectrum of the semantic space.. From the perspective of the deep learning theory [13] , the random walk based approaches essentially estimate the tag relevance with a shallow structure, which only consists of two levels, the provided tags as the input level and the tag being considered as the output level. The LDA based approach is with a deep structure, introducing a latent topic level, which has potential to get better performance. The proposed regularized LDA model is deeper, with four levels, the tags associated with other images as the first level, the latent topics of other images and the tags of the image being considered as the second level, the latent topic as the third level, and the tag being considered as the output level.. The relational topic model [8] is closely related to the proposed regularized LDA. But they are clearly different because our approach imposes the regularization over the topic distribution instead of the latent variables and moreover our approach deals multiple modalities and makes use of additional visual similarity to formulate the regularization term. Our approach is also different from topic models for image annotation [4, 2, 28, 29] : The image tagging problem in our paper is more challenging than image annotation as aforementioned. and moreover the proposed regularized LDA aims to impose the consistency of tags within similar images while they are supervised algorithms [28, 29] or aim to find common topics shared by tags and visual contents. This paper is different from the short version [37] because we present a formal derivation of our approach, introduce a new inference approach conduct more experiments, and particularly, we use the deep network structure to analyze the benefit of regularized LDA. To evaluate the performance of our approach, regularized LDA (rLDA), for tag refinement, we also report the experimental results of existing state-of-the-art approaches.. 1. BaseLine. The score is computed based on the original tag ranking provided by users according to the uploading time.. 2. Random walk with restart (RWR) [32] . This method performs a random walk process on a graph that encodes the relationship between tags. It only uses the text information without using the visual information. 3. Tag ranking based on visual and semantic consistency (TRVSC) [24] . This work follows RWR [32] using a random walk based method. The difference is that when constructing the graph over tags TRVSC considers the visual similarity between images. 4. Tag refinement based on low-rank, content-tag prior and error sparsity (LRCT-PES) [39] . This approach formulates the tag refinement problem as a decomposition of the user-provided tag matrix into a low-rank matrix and a sparse error matrix, targeting the optimality by low-rank, content consistency, tag correlation and error sparsity. 5. Collaborative retagging (CRT) [25] . The CRT process is formulated as a multiple graph-based multi-label learning problem. This work also proposes a tag-specific visual sub-vocabulary learning method. In our implementation we did not use the sub-vocabulary part because we focus mainly on the approach of rag refinement, not feature extraction. The parameters of CRT are tuned using the grid search method in [25] . 6. Separate retagging (SRT). SRT performs the same method as CRT [25] , but without considering tag similarity. We also use the grid search method described in [25] to find the best parameters. 7. Latent Dirichlet Allocation (LDA) [5] . We perform LDA by considering each image as a document and each tag as a word.. In our experiments, all the approaches use the same visual features, a 225-dimensional block-wise color moment. The visual similarity is calculated as s ij = exp{−. }, where f i is the visual feature of image I i and γ = 9 E[ f i − f j 2 2 ] with E being the expectation operator. Two images are connected in the rLDA model only when their similarity is higher than 0.2.

B I I O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries Narrative reports in medical records contain a wealth of information that may augment structured data for managing patient information and predicting trends in diseases. Pertinent negatives are evident in text but are not usually indexed in structured databases. The objective Much of the clinical information contained in patient medof the study reported here was to test a simple algorithm for determining ical records is in narrative form and therefore unavailable whether a finding or disease mentioned within narrative medical reports is present or absent. We developed a simple regular expression algoto automated systems that could improve patient care or rithm called NegEx that implements several phrases indicating negafurther medical research. Clinical information described in tion, filters out sentences containing phrases that falsely appear to be narrative reports is also difficult for humans to access for negation phrases, and limits the scope of the negation phrases. We clinical, teaching, or research purposes.. compared NegEx against a baseline algorithm that has a limited set Researchers in information retrieval are creating effective of negation phrases and a simpler notion of scope. In a test of 1235 findings and diseases in 1000 sentences taken from discharge summarmethods for automatically indexing narrative clinical docuies indexed by physicians, NegEx had a specificity of 94.5% (versus ments to facilitate searching on relevant terms [1] [2] [3] . Infor-85.3% for the baseline), a positive predictive value of 84.5% (versus mation retrieval techniques, however, do not generally dis-68.4% for the baseline) while maintaining a reasonable sensitivity of criminate between terms that are mentioned as being present 77 .8% (versus 88.3% for the baseline). We conclude that with little and terms that are negated. In fact, most phrases indicating implementation effort a simple regular expression algorithm for determining whether a finding or disease is absent can identify a large negation are stop words in information retrieval systems and portion of the pertinent negatives from discharge summaries. ᭧ 2001 are not even used for indexing. In clinical reports the presElsevier Science (USA) ence of a term does not necessarily indicate the presence of Key Words: text classification; pertinent negatives; negation; narrathe clinical condition represented by that term. In fact, many tive medical reports; natural language processing; artificial intelligence. of the most frequently described findings and diseases in discharge summaries, radiology reports, history and physical exams, and other transcribed reports are denied in the patient [4] . Physicians often note that a particular disease can be 1 To whom correspondence and reprint requests should be addressed ruled out or that a finding consistent with a suspected disease at Center for Biomedical Informatics, 8084 Forbes Tower, University is absent. We use the term "pertinent negatives" to refer to absent in a patient. Differentiating pertinent negatives from used to determine the scope of the negation phrases are entwined in the MLP system. positive conditions in a clinical report is crucial to accurate indexing of the report.. The most extensive study on negation was recently published by Mutalik et al. [19] . They use a lexical scanner Researchers in the medical language processing community have created methods for automatically extracting inforwith regular expressions and a parser that uses a restricted context-free grammar to identify pertinent negatives in dismation contained in narrative reports for decision support [5] , guideline implementation [6, 7] , detection and managecharge summaries and surgical notes. Like the algorithm described in this paper, their system first identifies proposiment of epidemics [8] , and identification of patients eligible for research studies [9] . Medical language processing (MLP) tions or concepts and then determines whether the concepts are negated. Their system performed with a sensitivity of systems do determine whether the extracted information is negated, but MLP techniques for negating clinical findings 95.7% and a specificity of 91.8% and is fine tuned with rules that apply to particular negation phrases and syntactic in free text are often entwined in the MLP system and are not transferable.. structures. Mutalik's algorithm is quite complex and requires other utilities such as lex and Yacc. The tools required by In this paper we describe and test a computationally simple algorithm that could be implemented quickly and easily to Mutalik's algorithm are easily attainable; however, implementing their system in a preexisting indexing tool would determine whether an indexed term is negated.. be less straightforward than the regular expression algorithm we describe here. Both NegEx and a baseline algorithm were applied to the preprocessed sentences. The baseline algorithm is the We designed an algorithm called NegEx to determine whether findings and diseases indexed from discharge sumalgorithm previously in use by a system called the IPS system [9, 23] that was created at the University of Pittsburgh to maries were negated by the dictating physician. To test our hypothesis that a relatively simple algorithm could produce help researchers identify relevant subsets of patient reports for research studies (we have since replaced the baseline reasonably accurate results, we compared NegEx against a baseline algorithm.. algorithm with NegEx). The baseline algorithm searches for six phrases (see Appendix) that might indicate a negation The input to NegEx is a sentence with indexed findings and diseases. The output is whether an indexed phrase is and negates all UMLS terms following the negation phrase until the end of the sentence. The baseline algorithm is negated in the sentence. For this study, we automatically preprocessed the sentences, indexed the relevant phrases simplistic but had been used by the IPS system for 2 years. Informally tested on approximately 1000 sentences, the basewith UMLS terms, applied the negation algorithms, and compared the negations made by the algorithms against line algorithm was being used in a system where a human user reviews results and can filter out mistakes made by negations made by physicians. For NegEx to work, the indexing algorithm must first identify a UMLS term; when the algorithm. NegEx expands on the baseline algorithm with additional the indexing algorithm does not identify a UMLS term, a pertinent negative cannot be found.. negation phrases and a richer regular expression syntax yet retains much of the simplicity of the baseline method. We preprocessed the reports so that exactly one sentence appeared per line. Processing the reports by individual senThrough a combination of manual scanning and semiautomated learning we identified 35 negation phrases (see Aptences means that information across sentences is not used in determining whether a clinical condition is affirmed or pendix) that could be divided into two groups. The first group (I), which we call "pseudo-negation" phrases, consists denied. Next, we removed all punctuation; information about syntactic structure, such as comma-delimited lists, is not of phrases that appear to indicate negation but instead identify double negatives ("not ruled out"), modified meanings used by NegEx. We did not remove stop words because some commonly used stop words (e.g., "of") are important ("gram-negative"), and ambiguous phrasing ("unremarkable"). The second group (II) consists of phrases we believed parts of the expressions we look for. Finally, we indexed findings and diseases within the sentence by replacing are used to deny findings and diseases when used in one of two regular expressions. In the first regular expression to determine the completeness of NegEx's set of negation phrases by capturing sentences with negation phrases we (II-A) the negation phrase precedes the UMLS term:. had not included. The entire test set contained a total of ͗negation phrase͘ * ͗UMLS term͘. 1000 sentences. In the second (II-B) the negation phrase follows the UMLS term:. 2.3. Gold Standard ͗UMLS term͘ * ͗negation phrase͘.. Three physicians judged the sentences in the test set to In both II-A and II-B the asterisk indicates that up to five establish a "gold standard" against which the computerized tokens (i.e., words or UMLS terms) may fall between the algorithms could be compared. The physicians read overlapnegation phrase and the UMLS term.. ping subsets of the 1000 sentences and marked all UMLS The regular expressions were matched to the longest posterms in the sentences as (a) present-the term was described sible subset of the sentence. For example, the sentence "exby the dictating physician as being present at the current tremities showed no cyanosis, clubbing, or edema" (UMLS visit, (b) absent-the term was explicitly described as being terms in bold, negation phrase in italics) matches the regular absent at the current visit, or (c) ambiguous-whether the expression ͗no͘ * ͗UMLS term͘ twice with both "cyanosis" term was present or absent was not clear from the sentence. and "edema" being labeled as negated.. Each physician judged 400 of the 1000 sentences. To determine interrater reliability, 200 sentences were rated by pairs of physicians. Raters 1 and 2 judged 100 overlapping

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O B I B I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Computation of Maximum Likelihood Estimates for Multiresponse Generalized Linear Mixed Models with Non-nested, Correlated Random Effects Abstract. Estimation of generalized linear mixed models (GLMMs) with non-nested random effects structures requires approximation of high-dimensional integrals. Many existing methods are tailored to the lowdimensional integrals produced by nested designs. We explore the modifications that are required in order to adapt an EM algorithm with first-order and fully exponential Laplace approximations to a non-nested, multiple response model. The equations in the estimation routine are expressed as functions of the first four derivatives of the conditional likelihood of an arbitrary GLMM, providing a template for future applications. We apply the method to a joint Poisson-binary model for ranking sporting teams, and discuss the estimation of a correlated random effects model designed to evaluate the sensitivity of value-added models for teacher evaluation to assumptions about the missing data process. Source code in R is provided in the online supplementary material. Generalized linear mixed models (GLMMs) are popular for fitting data sets with correlated responses from a member of the exponential family, where random effects are used to account for within-subject correlation. They have been used in applications ranging from ecology and evolution (Bolker et al., 2009) to epidemiology (Brumback et al., 2013) to education (Broatch and Lohr, 2012) . Computing maximum likelihood (ML) estimates in GLMMs is more complicated than comparable computations in linear mixed models because the log-likelihood function for the parameters in a GLMM contains integrals over the random effects that must be computed numerically. The dimension of the integration depends on the structure of the random effects. Many of the computational methods in current use for GLMMs concentrate on nested generalized linear mixed models, in which the units at the top level of the hierarchy are independent. The nested structure considerably simplifies the computation of ML estimates because it reduces the dimensions of the integrals that need to be calculated and simplifies the inversion of relevant covariance matrices.. Many longitudinal studies of subjects in social sciences or epidemiology, however, involve a non-nested structure of the random effects. In education, students proceed through different classrooms in their academic career. Thus, the response of a student at time t may depend on the full set of classes the student has taken up to that time. This leads to a multiple membership structure (Browne et al., 2001) in which each student is associated with multiple random classroom effects in sequence. The correlation structure becomes complex in multiple membership models because one student's response is correlated with responses of all the students with whom the student shares classes. Each student may have multiple and mixed responses at each time: for example one response might be a score on a standardized test, while another response might be whether the student is required to attend summer school. Furthermore, allowing binary responses allows for the modeling of a missing data mechanism: one of the responses is whether the student is present for the exam; if the student is absent, then his test score is missing. Jointly modeling the student test scores and missing data mechanism allows for a sensitivity analysis to assumptions about the dropout process (Karl et al., 2013a) .. Multiple membership models have also been used in longitudinal studies of health outcomes. Chandola et al. (2005) use them to examine the effects of neighborhood and family on health outcomes, when individuals can move from one household to another and from one neighborhood to another over time. Gueorguieva and Sanacora (2006) discuss the advantages of jointly modeling continuous and ordinal measures of depression over time. Accounting for patients seeing different therapists over time would lead to a non-nested structure for the random effects. In a different type of application, Page and Page (2010) study the relationship between the home field advantage and effects of different referees in soccer. The home field advantage of a team for a specific referee is modeled by a random effect, and it would be desirable to also include the home and away teams as random effects. Typically, the number of goals scored in soccer games is low, and a Poisson distribution may be better for fitting the response than a normal distribution.. All of these applications share the following structure. At each time period, there is a potentially multivariate outcome for each subject where at least one of the responses has a non-normal distribution. Each subject is associated with multiple higher-level units, or each observation is a function of multiple levels of the same random factor: a student belongs to multiple teachers or classrooms, a person belongs to different neighborhoods over time, a patient sees different therapists, a referee officiates for different teams and at different locations, and game outcomes depend on combinations of different teams' effects. The multiple membership pattern induces a complex correlation structure in which each observation of a subject is potentially correlated with those of every other subject sharing any of the higher-level units. In a GLMM, the dimensionality of the integration is related to the complexity of the random effects design matrix. When this matrix cannot be factored across subjects, the integral in the log-likelihood will have the same dimension as the number of random effects in the model. In addition, the random effects associated with different responses for the same subject are correlated. The parameters of interest are the fixed regression coefficients, the empirical best linear unbiased predictors (EBLUPs), the variance components for low-and high-level units, and the correlations among the variance components. Pinheiro and Chao (2006) note the need for an efficient estimation routine for non-nested models such as these.. In this paper, we propose a general method based on the Expectation-Maximization (EM) algorithm for computing ML estimates in multiresponse GLMMs with correlated, potentially non-nested random effects. We exploit the sparse structure of the random effects design matrices while using first-order and fully exponential Laplace approximations for the multidimensional integrals to furnish a balance of accuracy and speed, providing an improvement over the widely used pseudo-likelihood model approximation. Furthermore, we express the estimation routine in terms of the first four derivatives of the conditional likelihood for each response, facilitating its application to new problems. Estimation routines for GLMMs tend to be more application-dependent than routines available for linear mixed models; this paper presents a template for jointly estimating multiple GLMMs without assuming any special structure for the random effects design matrices corresponding to each response. R code to fit a non-nested Poisson-binary model and a multiple membership binary response is presented in the online supplement, along with sample data. The code provides a foundation for readers wishing to apply the routine in new settings, since the challenging programming aspects have already been addressed.. In Section 2, we describe the multivariate generalized linear mixed model treated in this paper, illustrate the model with a Poisson-binary example, and summarize previous work on computational methods. Section 3 presents the estimation of the joint model via an EM algorithm. Section 4 discusses the techniques that are necessary in order to compute the EM estimates for large data sets. Applications to sports rankings and student test scores are given in Section 5. A simulation study to evaluate the performance of various approximations appears in Section 6. Section 7 discusses the results and outlines possible extensions.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


QCD simulations with staggered fermions on GPUs We report on our implementation of the RHMC algorithm for the simulation of lattice QCD with two staggered flavors on Graphics Processing Units, using the NVIDIA CUDA programming language. The main feature of our code is that the GPU is not used just as an accelerator, but instead the whole Molecular Dynamics trajectory is performed on it. After pointing out the main bottlenecks and how to circumvent them, we discuss the obtained performances. We present some preliminary results regarding OpenCL and multiGPU extensions of our code and discuss future perspectives. Graphics processing units (GPUs) have been developed originally as co-processors meant to fast deal with graphics tasks. In recent years the video game market developments compelled GPUs manufacturers to increase the floating point calculation performance of their products, by far exceeding the performance of standard CPUs in floating point calculations. The architecture evolved toward programmable many-core chips that are designed to process in parallel massive amounts of data. These developments suggested the possibility of using GPUs in the field of high-performance computing (HPC) as low-cost substitutes of more traditional CPU-based architectures: nowadays such possibility is being fully exploited and GPUs represent an ongoing breakthrough for many computationally demanding scientific fields, providing consistent computing resources at relatively low cost, also in terms of power consumption (watts/flops).. Due to their many-core architectures, with fast access to the on-board memory, GPUs are ideally suited for numerical tasks allowing for data parallelism, i.e. for SIMD (Single Instruction Multiple Data) parallelization. The numerical simulation, by Monte Carlo algorithms, of the path integral formulation of Quantum Field Theories, discretized on a Euclidean space-time lattice, is a typical such task: one has to sample the distribution for a system with many degrees of freedom and mostly local interactions.. Quantum Chromodynamics (QCD), the Quantum Gauge Theory describing strong interactions, is a typical example where numerical simulations represent the best tool to investigate systematically specific features of the theory and give an answer to many important unsolved questions, regarding e.g. color confinement, deconfinement and the values of hadron masses. Lattice QCD and its computational needs has represented a challenge in the field of HPC since more than 30 years, being often the stimulus for new machine developments (think e.g. of the series of APE machines [1] ).. The introduction of GPUs in lattice QCD calculations started with the seminal work of Ref. [2] , in which the native graphics APIs were used, but the real explosion of interest in the field followed the introduction of NVIDIA's CUDA (Compute Unified Device Architecture) platform, that effectively disclosed the field of GPGPU (General Purpose GPU [3]), providing a more friendly programming environment.. GPUs have maintained their role of co-processors in most numerical applications, where they are used as accelerators for some specific, time demanding purposes. In the same spirit, most of previous studies on the application of GPUs to lattice QCD calculations were mainly aimed at using them together with the standard architectures in order to speed up some specific steps, typically the expensive Dirac matrix inversion. Our intent is instead to use GPUs in substitution of the usual architectures, actually performing the whole simulation by them: one still needs a CPU to run the main program, but mostly in the role of a mere controller of the GPU instruction flow.. To achieve this result we found simpler to write a complete program from scratch instead of using existing software packages 1 , in order to have a better control of all the steps to be performed and ultimately transferred to the GPU. Our implementation uses NVIDIA's CUDA platform together with a standard C++ serial control program running on CPU. The specific case considered in the present study regards QCD on a hypercubic lattice with quark fields discretized in the standard staggered formulation.. The paper is organized as follows. In Section 2 we give more details about the lattice discretization of QCD considered in our study and the simulation algorithm adopted. In Section 3 we review some of the fundamental features of GPU architectures. In Section 4 we describe our implementation of the algorithm on GPUs and discuss the achieved performances. Finally, in Sections 5.1 and 5.2 we discuss some preliminary comparisons with performances obtained with OpenCL and multiGPU implementations of our code. A preliminary report about our implementation has been presented in Ref. [4] . The extremely high computation capabilities of modern GPUs make them attractive platforms for high-performance computations. Previous studies on lattice QCD applications have been devoted almost exclusively to the Dirac matrix inversion problem. We have shown that it is possible to use GPUs to efficiently perform a complete simulation, without the need to rely on more traditional architectures: in this case the GPU is not just an accelerator, but the real computer.. Our strategy therefore has been that of bringing as much as possible of the computations on the GPU, leaving for the CPU only light or control tasks: in particular the whole molecular dynamics evolution of gauge fields and momenta, which is the most costly part of the Hybrid Monte Carlo algorithm, runs completely on the GPU, thus reducing the costly CPU-GPU communications at the minimum.. Following such strategy, we have developed a single GPU code based on CUDA and tested it on C1060 and C2050 architectures. We have been able to reach boost factors up to ∼ 10 2 as compared to what reached by a twin traditional C++ code running on a single CPU core. Our code is currently in use to study the properties of strong interactions at finite temperature and density and the nature of the deconfinement transition, in particular first production results have been reported in Ref. [32] .. A point worth noting is that in our implementation we have to rely on the reliability of the GPU. If the GPU is used just for the Dirac matrix inversion the result can then be directly checked on the CPU without introducing significant overhead in the computation. Such a simple test can not be performed if the GPU is used to perform a complete MD trajectory. For this reason it was mandatory to use GPUs of the Tesla series.. During the editorial processing of this paper it was signaled us that also the TWQCD Collaboration uses GPUs to completely perform the Hybrid Monte Carlo update of QCD with optimal Domain Wall fermions (their first results were published in [33] ).. Reported performances make surely GPUs the preferred choice for medium size lattice groups who need enough computational power at a convenient cost, in this sense they already represent a breakthrough for the lattice community. Our current lines of development regard the extension of our code to OpenCL and to multiGPU architectures and we have reported preliminary results about that in Section 5: that will open to possibility to use GPU clusters with fast connection links (see for instance Ref. [34] ) in order to make the GPU technology available also for large scale simulations.

O O O O O O O O O O O O O O O O O O O O B I O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On codes over quaternion integers In this study, we followed the procedures used by Neto et al. and suggest a decoding algorithm for an n length cyclic code over quaternion integers to correct errors of quaternion Mannheim weight two up to two coordinates. Furthermore, we establish that; over quaternion integers, for a given n length cyclic code there exist a cyclic code of length 2n − 1. The decoding algorithms for the cyclic code of length 2n − 1 are given, which correct errors of quaternion Mannheim weight one and two. In addition, we show that the cyclic code of length 2n − 1 is maximum-distance separable (MDS) with respect to Hamming distance. In this paper we suggest an effective and consistent decoding procedure for the cyclic code of length n over quaternion integers which can correct errors of quaternion Mannheim weight two. Moreover corresponding to the n length cyclic code, the existence of cyclic code of length 2n − 1 is proven. The 2n − 1 length cyclic code has error correcting capability of quaternion Mannheim weight one and two. It is further established that, for a given prime p, the corresponding [2n − 1, 2n − 2] cyclic code has higher code rate than the cyclic code [n, n − 1]. Astonishingly [2n − 1, 2n − 2] cyclic code has improved the quaternion Mannheim distance than of the cyclic code [n, n − 1].. In customary case, we have developed that the Hamming error correction capability of the cyclic code of length 2n − 1 is also better as compared to the cyclic code of length n.

O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O B I O O O O B I O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O


Quantifying the value of IT-investments We describe a method to quantify the value of investments in software systems. For that, we adopted the classical risk-adjusted discounted cash flow model and geared it towards the field of information technology. This resulted in a scenariobased approach incorporating two IT-specific risks that can substantially influence IT-appraisals. They are requirements creep and time compression. To account for the risk of failed IT-projects and overrun risks we proposed the Weighted Average Cost of Information Technology (WACIT). WACIT adjusts the well-known Weighted Average Cost of Capital, commonly used in discounted cash flow models. We proposed several methods to approximate WACIT ranging from an investment-specific rate to a (company-wide) one inferred from a quantitative IT-portfolio analysis. We illustrated our quantitative IT-investment management approach by way of a published example. For various risk-scenarios, we calculated using this example the impact on the firm's share price, standard economic indicators like NPV, IRR, PBP, ROI, RAROC, and more, in order to come to grips with the appraisal of information technology-the largest production factor of today. Anyone with a background in economics knows that there are established tools and techniques to determine the value of proposed investments. Some investments are harder to appraise than others, but continuous research in financial economics is carried out to increase insight in such hard cases. Normally, a business case is valuated using a discounted cash flow (DCF) analysis. Roughly, the cost are subtracted from the benefits, time value for money is taken into account, and risks are dealt with. The outcome of such an analysis is the net present value (NPV), giving the net value in terms of today's money. If the NPV is positive, and a few other indicators are favorable, for instance the pay back period (PBP) is not too long, the internal rate of return (IRR) is not too close to the used discount rate, and if the appropriate variant of the return on investment (ROI) is not too low, you can decide to invest.. Of course, such economic indicators are a model only, and do not need to reflect reality accurately. For instance, during the dotcom hype a loss-generating free-software reseller could have a higher market value than the entire German automotive industry. For those hard cases additional analysis is required, for instance using real option valuation (ROV), where despite a zero NPV still a positive appraisal of an investment can be found due to the value that options can represent. For instance, the stock price of Netscape at the initial public offering (IPO), could be valued at $14 using a very optimistic DCF appraisal, but was set to $28 by its major underwriters. Via an ROVaugmented DCF it turned out to be possible to justify this higher value [4] . management, and the GAO addresses how to embed this within an organization. Furthermore, we used an example from Gardner and Trotta [10, 9] , but as we will see, this example makes assumptions on IT-development costs, operational costs, its lifespan, and more that are not in accordance with industry benchmarks. Then there is a textbook by Gardner on the value of information technology, which also does not take such issues into account [12] . It discusses networks, hardware, and other issues which are better understood than valuating custom-made IT-investments, which is our focus. Pisello and Strassmann [29] wrote an e-book, which creates awareness for the economic perspective in information technology. It gives an overview of standard economic performance metrics every CIO should know about. To a lesser extent this was also touched upon in the textbook by Boehm from the early 1980s, where net value, marginal analysis, time value for money, and discounted cash flows were briefly mentioned [3] . The focus of this book is on models for software cost estimation, not valuation. This paper dives much deeper into valuation questions, taking IT-specific aspects into account via benchmarked relations. Our work builds on a body of work by the author and his co-workers. We mention an elaborate publication on quantitative IT-portfolio management [35] , recent work on quantitative aspects of outsourcing deals [36] , and a few related contributions [34, 25, 24] . In this paper we illustrated how you can appraise IT-investments. We explained our approach using an existing published IT-investment proposal, and we adapted and extended this example to illustrate how to obtain the most prominent quantitative aspects of estimating net tangible benefits of IT-investment plans. Our approach extends existing methods known from financial theory on the one hand, and on the other hand makes use of quantitative IT-portfolio management to supply important input to conduct the economic calculations. Importantly, based on minimal data points, we are still able to carry out a credible financial analysis of the costs, duration, and financing of ITinvestments, based on benchmarks. Moreover, we use benchmarks to risk-adjust costs, durations, and financing of such investments. Thereby, we are in a position to provide insight into the standard economic indicators: net present value, internal rate of return, return on investment, its risk-adjusted variants, the payback period, break even points, cost-benefit analysis over time, J-curves, and comparisons of different risk scenarios. We also addressed the idea of discounting the projected benefits, and obtain an impression of how much lower the benefits can become before we have a bad investment. We think that this paper brings us one step closer to a common practice of IT-investment management.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O B I I I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I


Radial Basis Function Nets for Time Series Prediction This paper introduces a novel ensemble learning approach based on recurrent radial basis function networks (RRBFN) for time series prediction with the aim of increasing the prediction accuracy. Standing for the base learner in this ensemble, the adaptive recurrent network proposed is based on the nonlinear autoregressive with exogenous input model (NARX) and works according to a multi-step (MS) prediction regime. The ensemble learning technique combines various MS-NARX-based RRBFNs which differ in the set of controlling parameters. The evaluation of the approach includes a discussion on the performance of the individual predictors and their combination.Keywords: NARX Architecture, Radial basis function networks, Ensemble predictors, Multi-step predictionIn learning temporal (and spatial) sequences for predictions purposes, recurrent neural networks have attracted a lot of attention. There exist a number of studies showing different neural architectures and relying on different learning models, i.e., supervised, unsupervised and with reinforcement. While supervised learning recurrent algorithms seem to be the most popular ones 14,8 , unsupervised learning (i.e, clustering) has witnessed increasing attention especially with the advent of self-organized maps and vector quantization networks 2 . Reinforcement on the other hand has been applied for time series in a smaller number of studies 17 .Temporal relationship often are captured using feedback connection in neural networks. This has resulted in a number of architectures and have been classified in 3 main groups 16 . globally recurrent networks 14 , locally recurrent networks 8 and nonlinear autoregressive with exogenous input networks (NARX networks) 18 . In the first class, hidden nodes provide a context (hidden states) and are globally fed back as new input. In locally recurrent networks, the feedback connections are allowed only from neurons to themselves (looped neurons). In the NARX architecture, the output of the network are fed back to the input layer.In this paper we use a new adaptive NARX recurrent radial basis function network as a prototypical base learner for an ensemble learning approach. This neural network is called a MS-NARX-RRBFN standing for multi-step NARXbased recurrent radial basis function network. The present paper deals with a new method of time series predictions based on multiple predictors. Each of these is a multi-step nonlinear autoregressive with exogenous input model (NARX) radial basis function network. Relying n two time series, the experiments have shown that the combination improves the prediction accuracy. However, there are many other issues that will be studied in the future such as the dynamic evolution of the networks' structure, that is the number of RBFs is automatically learned and weighting the contribution of the networks in the ensemble. To assess the proposed approach, we study two aspects: (i) the prediction accuracy of the individual networks, (ii) the accuracy of their combination following the the four types of radial basis functions on both data sets: the sunspots and the chaotic Mackey-Glass time-series. For the sake of the evaluation, the root mean squared error (RMSE) measure is used to quantify the goodness-of-fit. It is given by: Before starting the evaluation of each of the individual NARX-RBFNN, it is important to check the effect the key parameters that characterize the proposed NARX architecture. Basically, these parameters include the time window size (D) used for training the networks and the prediction horizon (p), and the number of radial basis functions. Figures 5 and 6 show the effect of the two parameters on the prediction root mean square error for both data sets. The observed trend from these figures is that for the sunspot time series, small window size is preferred. In fact, when D = 3, a least root mean square error is obtained. In the case of Mackey glass data set, there is no clear trend window size, though larger sizes seem to be preferred. On the other hand, small size of the prediction horizon is preferred for both data sets. These results are consistent with the variability of time series. Indeed, the variability of Mackeyglass, that is 0.0556, is far smaller than that of sunspot (1554.20). Therefore, it is important to set the window time in the case of sunspot also smaller than that applied to the Mackey-glass time series.. This experiment has allowed to estimate the near-optimum D and p (in a certain range of values). In the next experiments, we will consider D sunpot = 3, D M ackey = 9, p sunpot = 2, and. On the other hand, to find the optimal number of radial basis functions used by the neural networks, we have used the Bayesian Information Criterion (BIC) to judge the statistical significance of a number of inspected finite mixture models. We do that by relying on the expectation maximization algorithm. The highest BIC value corresponds to the optimal number of radial basis function.. As expected, the BIC increases as the number of clusters increases. However, a significant increase is obtained after setting the number of clusters to 48 in the case of Mackey glass data and 56 for the sunspot data. This has also been noticed when computing the root mean square error as shown in Figs. 7 and 8 for the case of Gaussian radial basis function as an illustrative example. Therefore, we have considered 60 and 50 clusters for sunspot and Mackey glass respectively for the Gaussian function. Then for the quadratic, inverse multiquadratic and cubic the number has been set to 140, 120 and 240 based on preliminary experiments and guided by the study in Ref. 11 . Applying the set of NARX-RBFNNs under the optimal conditions (i.e., optimal number of radial basis functions, size of the time window, size of the prediction horizon) on the training data sets, we obtain Figs. 9 and 10. The results of both the training and testing show a very good fit as explicitly portrayed in Tab. 4. Despite the high accuracy, it is easy to note that the networks are less accurate on the testing data than on the training set. Now testing the set of NARX-RBFNNs on both data sets allow to appreciate more the capability of the individual classifiers. Figure 11 and 12 show the testing results. While in the case of the sunspot data, the classifiers perform very well, in the case of Mackey glass, the performance was of lesser quality. The combination of the four NARXRBFNNs according to the combination rules portrayed in Tab. 3 yields the results shown in Tab. 4 with respect to both data sets. The accu- racy of the ensemble is much higher compared to each of the individual predictor. The average rule is the best combination rule. Comparing the individual NARX-RBFNNS on these two particular time series and given the differences in the accuracy values, it is not clear enough to state which radial basis function produces more accurate fitting.

O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Providing developmental feedback to individuals from different ethnic minority groups using expert systems The paper is an account of a commercially successful and viable expert system entitled FRED (Feedback Reporter Evaluating Development). Expert systems offer a consistent approach to providing feedback to individuals that people with personal biases and emotions may find difficult to achieve. The text used in the expert system is sensitive and carefully worded by a team of skilled professionals, many of whom were from ethnic and cultural minorities. This is especially important when dealing with internal applicants from ethnic minorities as feedback given is consistent, fair, non-judgemental and devoid of possible ethnic prejudices and emotional statements. FRED offers a sensitive, comprehensive and cost-effective method of producing feedback reports for individuals from an ethnic minority and the paper details FRED's design, use and evaluation in an organizational setting. Future research is recommended in utilizing expert systems to provide feedback in assessment and development settings. The use of commercial expert systems has come under considerable debate recently with several authors demonstrating their limited use (Abdullah, Kimble, Benest & Paige, 2006; Sagheb-Tehrani, 2006; Shaw and Gaines, 2005) . Nevertheless, in Industrial/Organizational Psychology, expert systems have continued from strength to strength predominantly in the field of interpreting psychometric tests (Bushnell, Paltiel & Young, 1998) . Many of the UK test publishers include an expert system interpretation package along with their online or pencil and paper test. These expert systems write a variety of reports from basic interpretations through to complicated in-depth analyses linking the psychometric test taken with other well-known tests.. Expert systems have also had some success in the assessment centre field. Wilson (1998) described how an expert system used in the London Fire Brigade detailed performance ratings as part of a Promotion Centre to applicant managers. The reports specifically produced by the system interpreted the performance ratings of assessors on various exercises and competencies (Wilson, 1998) . Potential managers would thus be able to see which exercises they performed poorly on and more importantly which competencies they needed to develop. This paper expands the use of expert systems further into the initial application phase of the selection process. Expert system technology, as this paper will demonstrate, can be used to report on the performance of applicants early on in the selection process. It can also be used to link consistencies of underperformance together to generate personal development feedback specific to the individual. Feedback to applicants on why their application was not successful may lessen the negative impact of being rejected as some research has suggested (e.g., Gilliland, Groth, Baker, Dew, Polly & Langdon, 2001 ). Expert systems can offer a consistent and sensitive method of delivering feedback to unsuccessful applicants in the early stages of the selection process. The paper outlines a commercially successful venture that performed well in the context of providing individuals from an ethnic minority feedback that was consistent and fair. This feedback was considered important by the Government Department that commissioned the work as concerns about the impact of rejection on applicants were raised by the Multi-cultural team. Arguably, the use of expert systems should be promoted further in the early stages of selection to provide feedback to rejected applicants. This is particularly important for those internal applicants who disheartened by rejection from a selection process may leave the organization. It may also be easier with online application forms that are systematically assessed sometimes without the need for an assessor.. In addition, in line with the results from the implementation of this expert system, further research needs to be undertaken to explore how expert systems can provide feedback in a selection context. Applicant's reactions to the selection process and ways in which feedback may moderate these reactions also need to be raised as a research issue.. 14 Expert systems have enjoyed much commercial success in the area of selection where they are often used to assist the hiring decisions of organizations through the interpretation of psychometric tests. This success as this paper has shown can also be readily transferred into the area of personal development where expert systems can also be effectively utilised.. Expert systems can provide timely, sensitive and meaningful feedback to individuals who may not readily wish to discuss their performance and thus improve the chances that these individuals may do better in their careers. Positive evidence on the Improving and Learning Competency was shown in that you; recognised the need for improvements, explained why the improvement was necessary, suggested improvement ideas and suggested ideas on evaluating the effectiveness of change.. There was an absence of positive evidence on the Improving and Learning Competency in that you failed to demonstrate a clear indication that you; consulted others for suggestions on how to improve the current situation, were open to adopting new ideas, considered a variety of options, adapted existing knowledge to make improvements and calculated the risks associated with the changes. FRED is a commercially successful system that produced developmental feedback reports that were well received by unsuccessful applicants of the Personal Improvement Programme. This approach had major advantages over other traditional forms of feedback offering a completely consistent approach for providing feedback to unsuccessful applicants. Feedback was consistent and fair as given the same answers FRED would generate the same report without personal biases or prejudice. The text was sensitive and carefully thought out by a team of experts. FRED offered a sensitive, comprehensive and cost-effective method of producing feedback reports for individuals from an ethnic minority. The project was thus a successful venture for both the client and the consultancy that provided the system as a service. This project continued on the following year where the system described here was modified slightly to respond to different application questions. After this, the client took over the entire project and source code of the expert system, which still runs today.. Even though FRED is a commercial success, it almost was never implemented. A number of authors have noted that the implementation of expert systems into an organization is a major stumbling block for expert system design (Awad, 1996; Sagheb-Tehrani, 2006) . This was indeed a major hurdle for the development of FRED whereby resistance to using the expert system approach was noted in the consultancy that commissioned the program. In order to overcome this resistance, the advantages of the expert system approach had to be sold as being superior to any other traditional approaches. This was achieved by detailing the system success factors (Awad, 1996) , which were:. • the user of the expert system had prior knowledge of expert systems;. • the user had expert knowledge in the domain of selection techniques and providing feedback to applicants;. • the user was involved in all stages of the knowledge acquisition process and the final evaluation of the system;. • the payoff of the system was large in terms of sensitivity, wording and cost;. • the expert system was designed away from the core business of the consultancy and the data entry into the system was outsourced meaning a minimum of disruption;. • the main advocate of the project used PRINCE2 project management techniques, which aided the project to be delivered on time, on budget and to the required quality.. Through detailing these system success factors the development of FRED was commissioned.. This demonstrated to the consultancy that such expert systems could be easily deployed in a commercial setting. It also demonstrated to the client, that the consultancy was, at the leading edge of selection implementing innovative solutions to solve recruitment issues.. Limitations to the project were noted with the post-system evaluation where only a questionnaire survey had been completed. Dijkstra (1999) described a phenomenon whereby some individuals blindly accepted the advice of the expert system even though this is not necessarily correct. Obviously, in this case great effort was made to ensure that the information provided to applicants was correct. The reactions in the questionnaire were positive and taken as a mark of success for the project in that the feedback reports were not criticised or mentioned in terms of improvement. Nonetheless, consistent with Dijkstra (1999) applicants may blindly accept what was written in the feedback report without question. To combat this limitation the letter enclosing the feedback report strongly suggested to rejected applicants that they talk the report over with their manager. This procedure would thus ensure accuracy of feedback as the managers would be able to support the findings in the expert system report. However, no formal survey of whether this practice was undertaken although some evidence from the client did suggest that applicants were discussing the report with their managers.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O