Top-Down modulation of neural responses in visual perception: a computational exploration Abstract Visual perception is typically performed in the context of a task or goal. Nonetheless, visual processing has traditionally been conceptualized in terms of a fixed, task-independent hierarchy of feature detectors. We explore the computational implications of allowing early visual processing to be task modulated. Using artificial neural networks, we show that significant improvements in task accuracy can be obtained by allowing the weights to be modulated by task. The primary benefits are obtained under resource-limited processing. A relatively modest task-based modulation of weights and activities can lead to a large performance boost, suggesting an efficient means of increasing effective cortical capacity. Our simulations show a clear benefit of top-down task-based modulation of neural responses. Although the architectures we studied had little of the structure of the human visual system, and the ANN is a highly idealized neural network, we are confident that the results apply to neurobiological systems. If anything, our results are probably conservative because the visual system is a multilayered hierarchy, and each layer introduces nonlinearities. As a result, small modulations to the response of neurons in an early layer can amplify as they propagate forward.. Top-down modulation increases the effective resources of a neural architecture, and is therefore most useful to overcome resource limitations. However, our simulations showed that top-down modulation may also be useful in overcoming some amount of input noise. We observed that a moderate top-down modulation (e.g., a = 0.5) yielded better performance than a pure bottom-up (a = 0) or a strong top-down model (a = 1). In a sense this result should not be terribly surprising. The bottom-up model is ideal if the tasks share strong similarities, and therefore internal representations of the input should be identical. The strong top-down model is ideal if the tasks are unrelated, allowing for flexible representations from task to task. Clearly, the tasks we studied have an intermediate degree of similarity, and therefore benefited from an intermediate strength of top-down modulation. A surprising and interesting finding of our simulations is that even small task-based modulations yield significant performance improvements, as evidenced by the steep slopes of our AROC curves for the a, V w , and V a graphs. Because similar tasks should utilize similar representations, it makes sense that not much modulation is required to achieve a performance boost. Our simulations provide a computational justification for observed task-dependent modulation of neural activity, and suggest that experimental studies should be even more sensitive to such modulations.. Our simulations are limited by the number of data sets we explored. We also did not have the opportunity to explore another possible benefit of top-down modulation: the ability to generalize to novel tasks (Caruana 1997) . For example, suppose one was asked to search for a target defined by a combination of color and shape. Previous experience with searching for the color and/or the shape may facilitate search for the novel combination via task-based modulation. Regardless of how such an exploration might turn out, we have demonstrated how task-based modulation can increase the effective processing and representational capacity of a hardware limited system like the brain. We studied four data sets, summarized in Table 1 . Sets A, C, and D are from the UCI Machine Learning Repository (pendigits , letter À recognition , and isolet , respectively), and Set B is derived from the MNIST database and available at yann.lecun.com/exdb/mnist/index.html. Sets A-C are visual images, utilizing three different feature representations: stroke based, pixel based, and statistical moments and edge counts. Set D uses the 26 spoken letters of the alphabet. Because our modeling is at an abstract level, it did not seem unreasonable to include a non-visual data set. Our primary goal in picking data sets was to find a diverse collection using different representations. For any a, the training procedure involved a search in the model's underlying parameter space, fŵ t g . These parameters are translated into weights used in the activation dynamics, fw t g , via Eq. 1. The logic of this procedure is to obtain weights whose strength of taskspecific modulation is related to a. The training procedure and the fŵ t g are not meant to have any neurobiological reality; they simply provide a means of obtaining sets of weights with given functional properties.. Training was performed via on-line steepest descent in the underlying parameter space, fŵ t g , using a mean-squared-error objective function. For each training example on each epoch, gradients were computed with respect to each task's underlying parameters, and weights were updated. An adaptive learning rate was used, which increased by a small constant if the error dropped from one epoch to the next, and decreased by a constant of proportionality if the error rose. To ensure that the network was trained to a local minimum, we used a conservative criterion that terminated training only when the epochwise mean squared-error dropped by less than 1% over 100 epochs. For each value of a, each hidden-layer size, and each train/test split, we performed 3-8 replications of training with different random weight initializations. (The number of replications was inversely related to the number of data splits, ensuring a roughly fixed number of total runs.). For each training run, an ROC curve was computed for the test/validation set, and the area under the ROC curve (AROC) was determined. AROC is a measure of a model's intrinsic ability to perform a two-way discrimination-present versus absent in the case of our tasks. The AROC measure ranges from 0.5, indicating chance discrimination, to 1.0, indicating perfect discrimination. An AROC score is obtained for each task, and we compute the mean AROC score across tasks. size for each task in the top-down net. Thus, comparing the bottom-up and top-down AROC values for a given value on the x-axis allows a factor of n more free parameters for the top-down net. This assumption is sensible if top-down modulation can completely rewire the available hidden units for each task. At this extreme, it's not surprising that the top-down net outperforms the bottom-up net. At the other extreme, when no rewiring can take place, then the natural comparison is betwen nets matched on total hardware: a top-down net with h units per task versus a bottom-up net with nh units. As examination of the first column of Figure 3 reveals, little or no advantage is obtained for the top-down net in this case. Thus, the benefit of task-specific modulation of unit responses arises in conditions where hardware resources are limited and can be effectively reused by top-down modulation. For this reason, all subsequent explorations were performed by selecting a particular hidden layer size for each data set, such that resource constraints arose. The size of the hidden layer for each data set is shown in Table 1 .

O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Semi-supervised sequence classification using abstraction augmented Markov models Supervised methods for learning sequence classifiers rely on the availability of large amounts of labeled data. However, in many applications because of the high cost and effort involved in labeling the data, the amount of labeled data is quite small compared to the amount of unlabeled data. Hence, there is a growing interest in semi-supervised methods that can exploit large amounts of unlabeled data together with small amounts of labeled data. In this paper, we introduce a novel Abstraction Augmented Markov Model (AAMM) based approach to semi-supervised learning. We investigate the effectiveness of AAMMs in exploiting unlabeled data. We compare semi-supervised AAMMs with: (i) the Markov models (MMs) (which do not take advantage of unlabeled data); and (ii) an expectation maximization (EM) based approach to semi-supervised training of MMs (that make use of unlabeled data). The results of our experiments on three protein subcellular localization prediction tasks show that semi-supervised AAMMs: (i) can effectively exploit unlabeled data; and (ii) are more accurate than both the MMs and the EM based semi-supervised MMs. Many problems in computational biology, e.g., protein function prediction, subcellular localization prediction, can be formulated as sequence classification tasks [2] . Sequence * This research was funded in part by an NSF grant IIS 0711356 to Vasant Honavar and Doina Caragea. data contain intrinsic dependencies between their constituent elements. Hence, effective sequence models need to incorporate these dependencies. Markov models (MMs) are examples of such sequence models [12] . They capture dependencies between neighboring elements in a sequence and, thus, provide more accurate models of the data. The accuracy of MMs obtained in a supervised setting depends on the availability of large amounts of labeled data. Labeling sequences often involves costly and time consuming manual curation.. Recent advances in sequencing technologies have resulted in an exponential increase in the rate at which DNA and protein sequence data are being acquired [1] . Because of the growing gap between the rate of acquisition of sequence data and the rate of manual curation, there is significant interest in semi-supervised algorithms that can exploit large amounts of unlabeled data together with limited amounts of labeled data in training sequence classifiers [32] .. Formally, the semi-supervised learning problem can be defined as follows: Given training data D = (DL, DU ) of labeled and unlabeled examples, where DL = (x l , y l ) l=1,··· ,|D L | , x l ∈ R d , y l ∈ Y, and DU = (xu) u=1,··· ,|D U | , xu ∈ R d , |DL| |DU |, respectively; a hypothesis space H; and a performance criterion P , a learning algorithm L outputs a classifier h ∈ H that optimizes P . If |DL| = 0, the problem reduces to supervised learning; if |DU | = 0, it reduces to unsupervised learning. The input x can represent sequences over a finite alphabet X , x ∈ X * . During classification, the task of the classifier h is to accurately assign a new example xtest to a class label y ∈ Y.. We introduce a novel approach to semi-supervised learning of sequence classifiers. Specifically, we use abstraction augmented Markov models (AAMMs), which are variants of Markov models, to incorporate information available in the unlabeled data. AAMMs model the dependency of each element in a sequence on abstractions of k preceding elements [7] . The abstractions are organized into an abstraction hierarchy that groups together k-grams that induce similar conditional probabilities of the next letter in the sequence. An AAMM corresponds to a generative model for sequence data expressed in terms of random variables whose values correspond to abstractions over k-grams, in addition to the MMs random variables [7] . AAMMs provide a simple way to incorporate unlabeled data into the model: first, the abstraction hierarchy is constructed using the entire training set including the unlabeled data. Next, the labeled data is used to estimate the parameters of a set of AAMMs (one for each class) based on the resulting abstraction hierarchy.. Thus, in effect, AAMMs: (i) exploit the relatively large amount of unlabeled data to discover abstractions that transform the sequence data x and, hence, effectively reduce the number of parameters used to specify the probability p(x); and (ii) use the resulting representation to estimate the posterior probability p(y|x). Hence, AAMMs are likely to yield more robust estimates of p(y|x) than MMs when the amount of labeled data is much smaller compared to the amount of unlabeled data.. To test our hypothesis, we compare AAMMs that use both labeled and unlabeled data with AAMMs that use only labeled data, with the standard MMs, which can not make use of unlabeled data, and also with MMs that can incorporate unlabeled data through an expectation maximization approach (EM-MM). The results of our experiments show that AAMMs can make effective use of unlabeled data and significantly outperform EM-MMs when the amount of labeled data are very small, and relatively large amounts of unlabeled data are readily available. Here, because of the small amounts of labeled data available for estimating parameters, the ability of AAMMs to minimize overfitting (through parameter smoothing) turns out to be especially useful.. The rest of the paper is organized as follows: We discuss related work in Section 2 and provide some background on Markov models for sequence classification in Section 3. We describe our novel AAMM-based approach to semi-supervised learning in Section 4. We present results of experiments comparing AAMMs, MMs and EM-MMs in Section 5. We conclude with a summary and discussion in Section 6. We trained AAMMs, MMs, and EM-MMs using 3-grams extracted from the data 3 (for psortNeg, plant, and nonplant data sets, the number of 3-grams is 7970, 7965, and 7999, respectively). In the case of AAMMs, we trained classifiers for m = 1500, where m is the cardinality of the set of abstractions A used as "features" in the classification model (m is set to 1500 because this partitioning of the set of k-grams produces classifiers that use substantially smaller number of "features" compared to MMs, i.e., ≈ 8000 k-grams, and at the same time, the model compression is not very stringent so as to lose important information in the data through abstraction). We denote by AAMM(l+u) an AAMM trained using an AH constructed from both labeled and unlabeled data, and by AAMM(l) an AAMM trained using an AH constructed only from labeled data, when it is necessary to distinguish between AAMMs training procedures. Note that EM-MMs are trained on the same fractions of labeled and unlabeled data as their AAMM(l+u) counterparts, and AAMM(l) and MMs are trained on the same fraction of labeled data as their AAMM(l+u) counterparts. Figure 3 : Comparison of AAMMs trained using an abstraction hierarchy learned from both labeled and unlabeled data, AAMM(l+u), with (i) AAMMs trained using an abstraction hierarchy learned only from labeled data, AAMM(l); (ii) Expectation-Maximization with Markov models, EM-MM; and (iii) Markov models, MM, on non-plant (left), plant (center), and psortNeg (right) data sets. x axis indicates the number of labeled examples in each data set corresponding to fractions of 1%, 5%, 10%, 15%, 20%, 25%, 35%, 50% of training data being treated as labeled data. The fraction of unlabeled data in each data set is fixed to 50%. Figure 3 shows results of the first set of experiments that compare AAMM(l+u) with AAMM(l), MM, and EM-MM on non-plant, plant, and psortNeg data sets. The x axis indicates the number of labeled examples in each data set. The number of unlabeled examples is kept fixed and is equal to the rightmost number of labeled examples on the x axis of each plot.. Comparison of AAMM(l+u) with AAMM(l) trained on a fixed amount of unlabeled data and varying amounts of labeled data. As can be seen from Figure 3 , AAMM(l+u) significantly outperforms AAMM(l) on all three data sets when small fractions of labeled data are available. For example, with 110 labeled sequences on non-plant (i.e., 5% of labeled data), AAMM(l+u) achieves 63% accuracy while AAMM(l) achieves 52%, which gives 23% reduction in classification error (|t| = 7.2). Strikingly, on the same data set, with only 22 labeled sequences (i.e., 1% of labeled data), AAMM(l+u) achieves 59% accuracy as compared to 43% obtained by AAMM(l), which gives 28% reduction in classification error (|t| = 9.73). Hence, AAMM(l+u) are able to incorporate information available in the unlabeled data (i.e., joint probability distributions of contiguous amino acids in a sequence) to learn more robust abstraction hierarchies than AAMM(l) when the labeled training set is limited in size (thereby, reducing the risk of overfitting).. Furthermore, AAMM(l+u) decreases the need for large numbers of labeled data. Specifically, on non-plant, AAMM(l+u) achieves 63% accuracy with 110 labeled examples, which is matched by that of AAMM(l) with 438 labeled examples (≈ 4 times more labeled data). However, when the fraction of labeled data is large, and hence, good estimates of model parameters can be obtained from such data, there is not much need for unlabeled data. For example, AAMM(l+u) becomes similar in performance with AAMM(l) on nonplant using 35% and 50% of labeled data (the null hypothesis is not rejected, |t| = 1.38 and |t| = 0.26, respectively).. As expected, the performance of AAMM(l+u) increases with the increase in the amount of labeled data. For example, on psortNeg with 12 labeled sequences (i.e., 1% of labeled data), AAMM(l+u) achieves 32% accuracy while AAMM(l+u) with 289 labeled sequences (i.e., 25% of labeled data) achieves 58% accuracy, which corresponds to 38% reduction in classification error.. Comparison of AAMM(l+u) with MM and EM-MM trained on a fixed amount of unlabeled data and varying amounts of labeled data. Figure 3 also shows the comparison of AAMM(l+u) with MM. AAMM(l+u) is superior in performance to MM, especially when small amounts of labeled data are available. For example, on plant, with 75 labeled sequences (i.e., 10% of labeled data), MM achieves 39% accuracy as compared to 44% obtained using AAMM(l+u) (|t| = 3.07). On non-plant, with 219 labeled sequences (i.e., 10% of labeled data), MM achieves 51% accuracy whereas AAMM(l+u) achieves 64% (|t| = 14). AAMM(l+u) not only incorporates information available in the unlabeled data (see previous comparison), but also performs parameter smoothing. Thus, AAMM(l+u) provides more robust estimates of model parameters than MMs, and hence, help reduce overfitting when the labeled training set is limited in size.. Both AAMM(l+u) and EM-MM make use of information available in the unlabeled data (i.e., both improve the performance of their counterpart classifiers trained only from labeled data) on all three data sets, although the improvement is not very large on psortNeg (Figure 3) . However, AAMM(l+u) uses the joint distribution over amino acids (independent of the class variable) to learn a more robust abstraction hierarchy (i.e., a finer partitioning of the set of kgrams), especially when the amount of labeled data is small, so that better estimates of parameters can be obtained. On the other hand, EM-MM uses the joint distribution over amino acids after an initial classifier has made predictions on the unlabeled data. When small amounts of labeled data are available, the predictions made by the initial classifier may not be reliable. As can be seen in Figure 3 , AAMM(l+u) significantly outperforms EM-MMs on non-plant, plant, and psortNeg data sets, when the fraction of labeled data is small. For example, with only 22 labeled sequences on Figure 4 : Comparison of AAMMs with EM-MMs for three different fractions of labeled data (i.e., 1%, 10%, and 25%) while varying the amount of unlabeled data on non-plant (left), plant (center), and psortNeg (right) data sets. x axis indicates the number of unlabeled examples in each data set corresponding to fractions of 1%, 10%, 25%, 50%, 75%, 90%, 99% of training data being treated as unlabeled data (by ignoring the class).. non-plant (i.e., 1% of labeled data), AAMM(l+u) achieves 59% accuracy while EM-MM achieves 42%, which gives 29% reduction in classification error (|t| = 8.83). Similarly, with only 8 labeled sequences on plant (i.e., 1% of labeled data), AAMM(l+u) achieves 34% accuracy as compared to 28% obtained by EM-MM, which gives 8% reduction in classification error (|t| = 4.66). As the amount of labeled data increases, EM-MM significantly outperforms AAMM(l+u). For example, with 767 labeled sequences on non-plant (i.e., 35% of labeled data), EM-MM achieves 69% accuracy while AAMM(l+u) achieves 67% (|t| = 4.87).. We conclude that AAMM(l+u) is able to incorporate information available in the unlabeled data, and hence, produce more robust classifiers than AAMM(l), MM, and EM-MM, when the fraction of labeled data is small. When larger amounts of labeled data become available, EM-MM makes better use of unlabeled data than AAMM(l+u).. Comparison of AAMM with EM-MM trained on a fixed amount of labeled data and varying amounts of unlabeled data. Figure 4 shows results of the second set of experiments that compare AAMMs with EM-MMs on non-plant, plant, and psortNeg data sets, respectively, while varying the amount of unlabeled data for three different fractions of labeled data (i.e., 1%, 10%, and 25% of labeled data) that are kept fixed. The x axis indicates the number of unlabeled examples in each data set.. As can be seen from Figure 4 , the improvement in performance of AAMMs over EM-MMs is rather dramatic when the amount of labeled data is quite small. For example, when only 1% of labeled data is used regardless of the amount of unlabeled data, AAMMs consistently significantly outperform EM-MMs on non-plant and plant data sets (the largest and smallest t values on non-plant are 10.96 and 5.66, respectively). However, the difference in performance between AAMMs and EM-MMs diminishes as more and more labeled data become available (and eventually levels off). When the amount of labeled data is increased (e.g., 25% of labeled data), EM-MMs often significantly outperform AAMMs (Figures 4(a) and 4(c) ). For example, on non-plant with 25% of unlabeled data, EM-MM achieves 68% accuracy, whereas AAMM achieves 66% (|t| = 7).. Again as can be seen from Figure 4 , the classification accuracy of AAMMs typically increases with the amount of unlabeled data (when the subset of labeled data is fixed). For example, on non-plant, AAMM with 22 labeled sequences (i.e., 1% of labeled data) and 219 unlabeled sequences (i.e., 10% of unlabeled data) achieves an accuracy of 56% as compared to 49% obtained by AAMM with 22 labeled sequences (i.e., 1% of labeled data) and 22 unlabeled sequences (i.e., 1% of unlabeled data), 14% reduction in classification error.. We conclude that AAMMs significantly outperforms EMMMs when there is less labeled data. Also the more unlabeled data is available, the higher the performance of AAMMs. A variety of approaches to semi-supervised learning have been studied in the literature (see [9] , [32] for reviews). Most of the existing semi-supervised learning algorithms including those based on co-training [5] , Expectation Maximization (EM) [25] , Transductive Support Vector Machines (TSVM) [19] , cluster kernel [30] , manifold based approaches [4, 18] , essentially involve different means of transferring labels from labeled to unlabeled samples in the process of learning a classifier that can generalize well on new unseen data.. EM-based methods provide a way to estimate the parameters of a generative model from incomplete data [10] , i.e., samples that contain missing values for some of the variables. Semi-supervised learning is a special case of such inference where it is the class labels that are missing for a subset of the data [25] . Specifically, the parameters of the model are estimated initially from the labeled fraction of the training data, DL, and the resulting model is used to predict p(y|x) for each of the unlabeled samples in DU . The parameters are re-estimated using the entire training data D and this process is repeated until the estimates converge. Co-training [5] is a variant of this approach where unlabeled data are labeled with two different classifiers trained on different subsets of the features in x.. Several semi-supervised learning algorithms based on discriminative approaches to classification have been investigated. TSVM [19] can be seen as a discriminative counterpart of EM. TSVM starts by training an SVM on the labeled data and uses the trained SVM to label the unlabeled data. The algorithm iteratively attempts to maximize the margin of separation between the sets of samples labeled by the SVM (by considering at each iteration, alternative labels for pairs of originally unlabeled samples that have been assigned different labels by the SVM). A similar outcome can be achieved by adding an additional regularization term for unlabeled data to the objective function optimized by SVM [32] . Similar approaches for exploiting unlabeled data in training discriminative classifiers include [23] , [29] , [16] , [17] .. An alternative approach to exploiting unlabeled data relies on the manifold assumption: high-dimensional data lies on a lower dimensional manifold, making it possible to propagate labels from labeled samples to unlabeled samples based on some measure of closeness of the data points on the manifold. The manifold can be approximated by a weighted graph in which the nodes correspond to data samples and the weights on the links between nodes correspond to the pairwise similarity of the corresponding data points [3] . A number of techniques for label propagation have been proposed [4] , [18] . Note that graph laplacian based techniques can be interpreted as a more general type of regularization where not only the L2 norm of the hypothesis is penalized but also the L2 norm of the hypothesis gradient.. Semi-supervised learning methods have been successfully applied in many areas including text classification [25] , [5] , [19] , natural language processing [26] , [15] , [27] , image annotation [6] , and more recently bioinformatics and computational biology, [20] , [22] , [21] .. In contrast to the approaches reviewed above, we present a novel abstraction-based approach to semi-supervised learning of sequence classifiers.

O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O


Relating Weight Constraint and Aggregate Programs: Semantics and Representation Weight constraint and aggregate programs are among the most widely used logic programs with constraints. In this paper, we relate the semantics of these two classes of programs, namely the stable model semantics for weight constraint programs and the answer set semantics based on conditional satisfaction for aggregate programs. Both classes of programs are instances of logic programs with constraints, and in particular, the answer set semantics for aggregate programs can be applied to weight constraint programs. We show that the two semantics are closely related. First, we show that for a broad class of weight constraint programs, called strongly satisfiable programs, the two semantics coincide. When they disagree, a stable model admitted by the stable model semantics may be circularly justified. We show that the gap between the two semantics can be closed by transforming a weight constraint program to a strongly satisfiable one, so that no circular models may be generated under the current implementation of the stable model semantics. We further demonstrate the close relationship between the two semantics by formulating a transformation from weight constraint programs to logic programs with nested expressions which preserves the answer set semantics. Our study on the semantics leads to an investigation of a methodological issue, namely the possibility of compact representation of aggregate programs by weight constraint programs. We show that almost all standard aggregates can be encoded by weight constraints compactly. This makes it possible to compute the answer sets of aggregate programs using the ASP solvers for weight constraint programs. This approach is compared experimentally with the ones where aggregates are handled more explicitly, which show that the weight constraint encoding of aggregates enables a competitive approach to answer set computation for aggregate programs. Answer set programming (ASP), namely logic programming under the answer set semantics (Gelfond and Lifschitz 1988; Niemelä 1999) , is a constraint programming paradigm, which has been successfully deployed in many applications (Balduccini et al. 2001; Wu et al. 2007; Caldiran et al. 2009; Oetsch et al. 2009; Ielpa et al. 2009; Delgrande et al. 2009; Erdem et al. 2009 ). Recently, ASP was extended to include constraints to facilitate reasoning with sets of atoms. These constraints include weight constraints (Simons et al. 2002) , aggregates (Faber et al. 2004; Ferraris 2005; Pelov et al. 2007; and abstract constraints (Marek and Remmel 2004; Marek et al. 2007; Liu and Truszczyński 2006; You et al. 2007; Shen et al. 2009; Liu et al. 2010) . Among them, weight constraints and aggregates are the most widely used constraints in practice. In this paper, logic programs with weight constraints and aggregates will be referred to as weight constraint and aggregate programs, respectively.. The semantics of weight constraint programs, called the stable model semantics, is well established and implemented in a number of ASP solvers (Simons et al. 2002; Giunchiglia et al. 2006; ). Especially, the results of the ASP solver competitions Denecker et al. 2009) show that CLASP is an efficient solver that implements this semantics.. For aggregate programs, various semantics have been proposed (Faber et al. 2004; Ferraris 2005; Pelov et al. 2007; . The one proposed in (Pelov et al. 2007 ) (previously in (Denecker et al. 2001; Pelov et al. 2004) ), called the ultimate stable semantics, is based on an iterative construction on partial interpretations. The same semantics is reformulated by ) and extended to logic programs with arbitrary abstract constraint atoms, which embodies a key concept called conditional satisfaction. Since this reformulation is conceptually simpler, as it does not resort to 3-valued logic, in this paper we call this semantics conditional satisfaction-based. Among the semantics for aggregate programs, this semantics is known to be the most conservative, in the sense that any answer set under this semantics is an answer set under others, but the reverse may not hold. The relationships of these semantics have been studied in Shen et al. 2009; Liu et al. 2010) . In this paper, we refer to the semantics based on conditional satisfaction as the answer set semantics. Despite the fact that weight constraint and aggregate programs are among the most popular classes of programs in practice, the relationship among them has not been fully studied, both in semantics and in representation.. In this paper, we study the relationship between the stable model semantics and the answer set semantics. We show that for a broad class of weight constraint programs, called strongly satisfiable programs, the stable model semantics agrees with the answer set semantics. For example, weight constraint programs where weight constraints are upper bound free are all strongly satisfiable. This result is useful in that we are now sure that the known properties of the answer sets also hold for these programs. One important property is that any answer set is a well-supported model , ensuring that any conclusion must be supported by a non-circular justification in the sense of (Fages 1994) .. Our study further reveals that for weight constraint programs where the stable model and answer set semantics disagree, stable models may be circularly justified. We then show that the gap between the two can be closed by a transformation, which translates an arbitrary weight constraint program to a strongly satisfiable program so that the answer sets of the original program are exactly the stable models of the translated program.. We further demonstrate the precise difference between the two semantics using a more general logic programming framework, logic programs with nested expressions. We propose yet another transformation from weight constraint programs to logic programs with nested expressions which preserves the answer set semantics. We compare this transformation to the one given in (Ferraris and Lifschitz 2005) , which is faithful to the stable model semantics. Interestingly, the difference is small but subtle: given a weight constraint l [S ]u, where l and u are lower and upper bounds, respectively, and [S ] expresses a collection of literals with weights, while in our transformation the satisfaction of the upper bound is interpreted directly as "less than or equal to u", in (Ferraris and Lifschitz 2005) the interpretation is by negation-as-failure "not greater than u".. The observation that the gap between the answer set and the stable model semantics can be closed by a transformation leads to an approach for computing answer sets of aggregate programs using the ASP solvers that implement the stable models semantics of weight constraint programs. We propose such an approach where aggregate programs are encoded compactly as weight constraint programs and their answer sets are computed using a stable model solver. We conducted a series of experiments to evaluate this approach. The results suggest that representing aggregates by weight constraints is a promising alternative to the explicit handling of aggregates in logic programs.. Besides efficiency, another advantage is at the system level: an aggregate language can be built on top of a stable models solver with a simple front end that essentially transforms standard aggregates to weight constraints in linear time. This is in contrast with the state-ofthe-art in handling aggregates in ASP, which typically requires an explicit implementation for each aggregate.. The paper is organized as follows. The next section gives preliminary definitions. In Section 3 we relate the stable model semantics with the answer set semantics. We first establish a sufficient condition for the two to coincide, and then discuss their differences. In Section 4, we present a transformation to close the gap between the two semantics, followed by Section 5 where we show how to represent aggregate programs by weight constraint programs. Further in Section 6, to pinpoint the precise difference between the stable model semantics and the answer set semantics for weight constraint programs, by proposing a transformation from weight constraint programs to logic programs with nested expressions which preserves the answer set semantics, and comparing this with that of (Ferraris and Lifschitz 2005) . We implemented a prototype system called ALPARSE and in Section 7 we report some experimental results. Section 8 concludes the paper.. A preliminary version of this paper has appeared as (Liu and You 2008) . The main extensions here include: (i) Section 6, where we propose a transformation from weight constraint programs to logic programs with nested expressions which preserves the answer set semantics -this transformation shows exactly what makes the answer set semantics differ from the stable model semantics; (ii) Section 7, where experiments are expanded including the benchmarks for aggregate programs used in the 2007 ASP Solver Competition ; and (iii) the proofs of all the theorems and lemmas. We have shown that for a large class of programs the stable model semantics coincides with the answer set semantics based on conditional satisfaction. In general, answer sets admitted by the latter are all stable models. When a stable model is not an answer set, it may be circularly justified. We have proposed a transformation, by which a weight constraint program can be translated to strong satisfiable program, such that all stable models are answer sets and thus well-supported models. We have also given another transformation from weight constraint programs to logic programs with nested expressions which preserves the answer set semantics. In conjunction with the one given in (Ferraris and Lifschitz 2005) , their difference reveals precisely the relation between stable models and answer sets.. As an issue of methodology, we have shown that most standard aggregates can be encoded by weight constraints. Therefore the ASP systems that support weight constraints can be applied to efficiently compute the answer sets of logic programs with aggregates. The experimental results demonstrate the effectiveness of this approach.. Currently, ALPARSE does not handle programs with aggregates like SUM (·) = k or AVG(·) = k , due to the fact that the complexity of such programs is higher than NP . What is the best way to include this practically requires further investigation. The theoretical studies show that an aggregate program can be translated to a weight constraint program whose stable models are precisely the answer sets of the original program. The experiments are run on Scientific Linux release 5.1 with 3GHz CPU and 1GB RAM. The reported time of ALPARSE consists of the transformation time (from aggregate programs to strongly satisfiable programs), the grounding time (calling to LPARSE version 1.1.2 for SMODELS and GRINGO version 2.0.3 for CLASP), and the search time (by SMODELS or CLASP). The time of SMODELS A consists of grounding time, search time and unfolding time (computing the solutions to aggregates). The time of DLV includes the grounding time and search time (the grounding phase is not separated from the search in DLV). All times are in seconds.

O B I O O O O O O B I O O O O O O O O O B I I I O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O


Who leads research productivity growth? Guidelines for R&D policy-makers Abstract This paper evaluates to what extent policy-makers have been able to promote the creation and consolidation of comprehensive research groups that contribute to the implementation of a successful innovation system. Malmquist productivity indices are applied in the case of the Spanish Food Technology Program, finding that a large size and a comprehensive multi-dimensional research output are the key features of the leading groups exhibiting high efficiency and productivity levels. While identifying these groups as benchmarks, we conclude that the financial grants allocated by the program, typically aimed at small-sized and partially oriented research groups, have not succeeded in reorienting them in time so as to overcome their limitations. We suggest that this methodology offers relevant conclusions to policy evaluation methods, helping policy-makers to readapt and reorient policies and their associated means, most notably resource allocation (financial schemes), to better respond to the actual needs of research groups in their search for excellence (micro-level perspective), and to adapt future policy design to the achievement of medium-long term policy objectives (meso and macro-level). Public support to Research and Development (R&D) and technology transfer activities are totally incorporated into Spanish Science and Technology (S&T) policies. However, the evaluation of these activities is not fully internalized into the policy cycle yet. Furthermore, the evaluation processes carried out so far deal with the elaboration of static indicators which barely provide an accurate picture regarding the way the results of these activities are evolving over time. In addition, many scholars claim that the structuralist-evolutionary context under which these sorts of policies are being built (Lipsey et al. 2005 ) need an alternative evaluation context, different to the efficiency of outcomes in a return on investment sense (Potts 2007) .. Another important claim that policy-makers and scholars raise with respect to these activities' outcomes and impacts is the need for a long term perspective to be able to actually appreciate them into the territory. However, this is an issue that has not been extensively treated in the policy evaluation literature. Moreover, this totally fair claim implies that conventional short term-cross section-evaluation processes of these activities might render narrow results that do not shed light on issues that could be useful to establish guidelines for long run policy reorientations. Therefore, an evaluation methodology that provides a dynamic overview on the evolution of R&D and technology transfer activities should be able to capture, on the one hand, the behavioral evolution (Buisseret et al. 1995) of the agents participating in the policy (i.e. the micro-level perspective) and the complexity of the economic order that S&T policies pose on any given innovation system on the other (i.e. the macro-level perspective).. This paper proposes a long term evaluation framework for a Spanish public policy supporting R&D and technology transfer activities within the food technology field based on efficiency and productivity measures. To offer this dynamic view on the impacts and outcomes that such policy has shown we follow a threefold perspective: micro, meso and macro. The micro-level perspective constitutes the focus of our study (research groups participating within the food technology field in Spain), the meso-level represents the plane where the recommendations to be concluded from the study are to be applied (Spanish S&T policy), while the macro-level corresponds to the context of analysis (Spanish food innovation system). Our goal is to determine the policy impact on the research groups' outputs (micro-level perspective), to gauge to what extent the policy contributes to consolidate the research groups 0 position on the food technology field (meso-level perspective), and how this relative position is helping the policy to construct a complex and articulated innovation system on the referred field (macro-level perspective). That is, we aim to contribute to the literature with a dynamic framework that could offer a set of guidelines for decision-makers involved in the management of multi-level S&T policies (Laranja et al. 2008) .. Studying the evolution of the system in time implies determining the characteristics of the most successful agents from a dynamic perspective-which in turn implies sorting them out according to their heterogeneity, so as to categorize those best practices that allow R&D managers to change policy guidelines in a way that encourages less successful agents to adopt benchmark practices. Our study performs such dynamic analysis and the results provide R&D managers with consistent evidence of those best practices over time, which will allow them to design and implement new strategies (financial schemes and their associated requirements) that would render the system more efficient and productive.. In order to accomplish this target, we perform a Malmquist productivity index (MPI) analysis that helps us understand how the policy is affecting the research groups participating into this policy. The paper focuses on research groups within the food technology sector that belong to the Spanish National Research Council (CSIC) and that have participated in the Spanish Food Technology Program (SFTP) between 1988 and 1999. We explore how our methodological approach allows to: (i) analyze the behavior and interactions of agents within the innovation system, and their effect on the productivity; (ii) contribute to policy evaluation and the sort of recommendations that will emerge under this approach.. From this micro-level perspective, our analysis is able to capture the degree of heterogeneity among research groups, both in terms of their research behavior and productive scale-i.e., relative size. This fact combined with the temporal perspective helps us characterize the contribution of research groups to the articulation of the innovation system as the final policy goal from the macro-level perspective. However, policy-makers do act in the meso-level (Dopfer et al. 2004) . Hence, our conclusions are addressed to provide them with guidelines in terms of what characteristics allow research groups to increase their internal capabilities and how that evolution fosters the innovation system towards an articulated one. Therefore, policy-makers can reorient and adjust the policy in specific directions that provide agents with the incentives to change in the desired direction. This is in fact the case of R&D managers in the food technology field, whose policy guidelines regarding the funding of particular projects and research groups have changed over the years in a way that is consistent with our results by promoting research activities of groups performing multidimensional and comprehensive research that contribute to the articulation of the innovation system, and exhibit higher efficiency and productivity levels.. The paper is structured as follows. The next section discusses the approaches that have been proposed in the literature to assess the dynamics of an innovation system and the policies related to it. This is followed by a discussion of the institutional framework that characterizes the SFTP and the research units participating in it. Next, we present the technology and its representation by way of the generalized distance function. In Sect. 5 we present the definition of the MPI and its decomposition in different terms, which allows us to determine the contribution that technological change, technical efficiency change and scale efficiency change make to productivity change. Generally, the decompositions of the MPI found in the literature are based on a changing base approach, whose main consequence is that the indices do not comply with the circularity property that allow consistent aggregation of period by period and sub-period productivity changes. To avoid this weakness that would not allow us to carry out a dynamic analysis, we introduce the necessary chained index definitions of all the alternative decompositions, which will be later used in the empirical application. In Sect. 6 we concisely present the data envelopment analysis (DEA) techniques that allow calculation of the generalized distance functions on which the MPI is based. We undertake our empirical analysis of productivity change in Sect. 7, where the productivity trends of the different research units involved in the SFTP are presented and the different sources of productivity change discussed. Section 8 concludes illustrating to what extent the SFTP has fostered productivity growth among those research units that have obtained financial support within the Spanish R&D plan. neoclassical approach, public intervention rests on the existence of market failures and production of new knowledge is associated with a positive externality (Arrow 1962) . The structuralist-evolutionary approach on the other hand, sees knowledge as an imperfect good that does not satisfy the usual characteristic of non-excludability, so the agents generating it will only be able to appropriate a small fraction of the social benefit produced (David et al. 1994 ). This approach is also linked to the concept of innovation system (IS), which justifies the existence of different agents and the relationships among them to carry out innovation activities (Freeman 1987; Lundvall 1992) . Therefore, under a structuralistevolutionary approach, R&D public policies, to an extent, respond to the need to strengthen the role and involvement of IS agents (Lipsey and Carlaw 1998; Metcalfe 2002) .. This interactive behavior relates to Rip and Nederhof's (1986) terminology of IS's articulation. Their concept of articulation correlates with the description in Gibbons et al. (1994) of the change in scientific knowledge production from mode I-summarized as the pursuit of scientific truth by scientists-to mode II-the production of knowledge from application-and the succeeding relationships among agents. Hence an articulated IS enables the different types of agents (i.e. policy-makers, scientists, technologists, business men, etc.) to maintain continuity in their relationships, over time. By relying on this articulation concept, we aim to assess the Spanish Food Innovation System's (SFIS) capacity to establish a network of fluent and continuous knowledge flows among its constituting agents.. In this paper we focus on the impact that the SFTP has had on the research groups within the food technology field in Spain, as one of the most relevant instruments used by Spanish S&Td policies to encourage and support the articulation of the IS. From this perspective we want to link the idea of public policies promoting a growing multidimensional output of research units, as an instrumental policy goal toward the articulation of a successful IS. To assess whether this instrumental goal has been successful we evaluate such policy using productivity analysis. In particular, we will analyze the productivity gains observed in the research groups that belong to the CSIC. Our research question is thus: to what extent has the SFTP become a suitable tool to promote the productivity increases of research units (micro level) contributing therefore to a multidimensional research output mix and, by extension, to the SFIS's articulation (meso level)?. One of the main limitations of the existing studies on the evaluation of innovation policies is the static view they offer. The literature agrees that innovation is a dynamic phenomenon (Autio 1997) and there is still a strong need to study the trends in technological change (Grimpe and Sofka 2007) . Lee and von Tunzelman (2005) consider that the study of systems through time allows for the analysis of complex behaviors that shed light on how policies, decisions, structure, and delays are interrelated and influence growth and stability. In recent years, there have been attempts to provide the IS approach with a more dynamic view. Markard and Truffer (2008) follow an actor-oriented perspective relating the micro (individual strategies and resources) and meso (system characteristics) levels in the case of stationary fuel cells in Germany. Similarly, Miettinen (1999) illustrates the possibilities of studying the dynamics of research-driven innovations using activity and actor-network theories. In addition, the literature discerns a series of functions accomplished within the frame of IS as one of the main attempts to characterize these system dynamics (Balzat and Hanusch 2004) . In contrast to the traditional agent-based view of innovation, which mainly focuses on the structure or a certain system, the functions view of innovation is based on mapping the activities that result in technological change and finally in the performance of an IS (Bergek et al. 2008; Edquist and Hommen 2008; Hekkert et al. 2007 ).. However, not only the dynamic assessment of an IS becomes a key issue, but also that of the innovation policies supporting its future development. In fact, the evolution followed by the IS approach and science, technology and innovation policies show an interactive and co-evolving process (Molas-Gallart and Davies 2006; Mytelka and Smith 2002) . Accordingly, the innovation policy evaluation related literature is also challenged by the need to provide policy evaluations with a dynamic view (Arbel 1981; Kuhlmann 2003; van Raan 2000) . This change in the role of evaluation in policy-making has also implications in the rationales for intervention, the behavior of institutions and framework conditions, and last but not least the role of the policy-maker (Arnold 2004) .. From a policy management perspective few are the efforts done in the evaluations of the innovation policy instruments implemented so as to assess their influence and results in the long run, both on the actors the policy is oriented to (micro level) and on the conclusions that may be drawn on the policy (meso) level. A recent contribution is done by Grammatikopoulos et al. (2004) in education policies in Greece, following a temporal evaluation approach. Schmidt et al. (2003) , in turn, conclude about the organization and leadership of research environments; the framework and the conditions for research; and the resource allocation policy as the key drivers of research policies in Denmark. From an industrial perspective, one of the few contributions is that of Laitinen (2002) who presents the results of a dynamic integrated performance measurement system applied to small Finnish technology companies.. 1 This is precisely our major target, to provide policy-makers with a tool to dynamically assess the performance of the research units the policy is aimed at. The SFTP, as other R&D Programs within the Spanish R&D plan, which are comparable to similar plans in developed countries, was designed to cover all the stages in the innovation process, offering possibilities for participation to a wide variety of agents, and fostering cooperation among them. Our goal is to propose an evaluation framework that allows R&D managers to assess the efficiency and productivity performance of research units participating as well as to measure their contribution to the Spanish Food Innovation System. In this respect, ex-post evaluation methodologies are mainly focused on outputs/input ratios, results achieved, impacts and the like indicators with little concern on how those indicators were achieved over time. We offer an evaluation framework where research groups' dynamics are a key aspect of policy implementation and evaluation, becoming an input for policy-makers in the next policy generation. This enhanced evaluation framework seeds light on micro, meso and macro-level aspects of any policy, helping policy-makers to ground future changes in policy design based on the observed results and particular trajectories.. To show its practicality in strategic planning we draw several specific and practical conclusions that may constitute guidelines for research managers, and make the following policy recommendations:. (i) Since the Spanish SFTP has exhibited an outstanding inter-periodical productivity growth with an average 19.3 % increase every 3 years-around 6 % yearly, we conclude that the implementation of the SFTP, and the particular allocation of (diminishing) inputs has been successful in general. However, this trend is not observed to the same extend across research units since our analysis unveils a high heterogeneity that can be discussed according to the typology of comprehensive, partial and specialized research units. (ii) Groups undertaking a comprehensive research should be promoted by the program as they prove themselves not only efficient in managing the scarce resources made available to them, but also capable of fostering research productivity growth while increasing their multidimensional output. Over the 12 year period this group increased its productivity by 41.7 % on average, outgrowing the productivity rates of other groups of efficient units that, despite being more numerous, do not contribute to the same extent to the achievement of the goals of the SFTP because of their partial or specialized research orientation-these units in particular exhibit a productivity decline to the tune of -5.5 %). Moreover, since comprehensive units rank high in terms of their efficiency levels (some of them being always efficient throughout the whole period) we confirm that the main source of this remarkable productivity growth is the expansion of the research frontier-technical change. (iii) R&D policy managers should be worried about the fact that the higher share of the units decide for a partial research orientation, focusing their work solely on science and technology outputs (mainly articles published in international journals), rather than undertaking personnel training or signing bilateral R&D contracts with the private sector. The reason behind this narrow research orientation is that the promotion of their members is based by far on this criterion. This is particularly grave since most of the inefficient units follow this partially oriented research strategy. This suggests that the incentives of academics do not agree with those of the R&D managers, and that research activities that contribute to a larger extent to the articulation of the Spanish Food Innovation System are prone to principal-agent problems that result in inefficient research practices. (iv) Looking at the average evolution in productivity growth for those units participating in the SFTP that start out from an inefficient situation, ours results confirm that they are not able to converge toward the production frontier, casting a shadow on their performance. On average they are able to attain productivity growth levels that barely match those of their efficient counterparts. This is rather unsatisfactory from a policy perspective because it implies that they are not able to profit from a catchingup process thereby reducing average inefficiency within the SFTP. One of the reasons why inefficiency levels remain constant over the period is that inefficient units are not able to converge toward the optimal production scale represented by the comprehensive units, whose size in terms of the amount of output and inputs is well above the average. (v) The analysis shows that large units undertaking a comprehensive research must constitute the benchmark peers against which all remaining units are confronted, and therefore their best practices should become the guidelines underlying the financial scheme of the program. Based on this conclusion we believe that a new financial line introduced in the announcement of the 2006 Spanish R&D plan (including the SFTP), reorienting some of the funding so as to promote the creation and consolidation of this kind of units, must be welcomed. The new line, known as ''consolider'', extends the duration of the average project form 3 to 5 years and grants an average budget of 1 million Euros. Nevertheless, to apply to this line, a minimum size must be met, i.e., it is required that a minimum of five units-with at least four researchers each-agree on a single proposal also contributing to the networking objective. Besides the general objectives of all programs, the declared goal of this line is to increase the competitiveness of Spanish research groups at international levels, e.g. the European framework program, by increasing the ''critical mass'' of research groups-i.e. creating large comprehensive units-that should translate into higher research productivity. 13 In a sense this change in the R&D plans acknowledges the pitfalls of the financial scheme existing until then. Since grants could not be awarded to large groups because there was not a particular financial line specifically aimed at promoting the consolidation of larger groups, most of the funding would end up in units carrying out a partial research orientation, whose results are less satisfactory as already discussed.. There are some limitations to this study which encourage further research efforts. One of these is the focus on a single national program which affects the degree to which our conclusions can be generalized to other research environments. That would allow policymakers to benefit from the application of the conclusions obtained not only in countries where similar practices are implemented, but also on those where similar research group distributions are observed. In this sense, Schmoch and Schubert (2009) , discuss the need to include a wide set of indicators when attributing excellence to the behavior of certain research units. We find that their conclusions are very much illustrated by our case study, where we also use a wide array of indicators.. Further work on our evaluation methodology is concerned with the actual combination of neoclassical methods into an evolutionary policy framework, also reflecting Lundvall et al.'s (2002) concern on how to link input/output analyses with evolutionary economics that will benefit the policy evaluation discipline. Besides, to lay additional bridges between the neoclassical and evolutionary realms we also suggest the addition of qualitative information into the quantitative framework that is already being developed so as critical aspects can be incorporated into the evaluation process that so far can hardly be included into a quantitative evaluation methodology (e.g. the concepts of strategic capital, social 13 In 2011, research proposals were evaluated using different criteria depending on their characteristics: (i) the ''consolider'' line already described, (ii) the general and conventional line that did not require a minimum size (i.e. using the criteria exiting until then) and (iii) a line reserved to young researchers under 40 years old and whose proposal clearly departs from those of their supervisors. This segmentation of the financial scheme system guarantees that funds are allocated among researchers competing in the same category, as using one single set of criteria did not give managers the flexibility to finance large projects on a long term basis or ensure that enough funding would reach young researchers.. Guidelines for R&D policy-makers 299 network capital, etc.) A first step would be to consider them as categorical variables within our quantitative DEA approach. Finally, we consider that our study of the key features of research units exhibiting a best practice behavior associated to high efficiency and productivity levels shows the potential of the proposed Malmquist productivity change analysis as a valid methodology to undertake research performance evaluations and achieve the proposed policy-guidelines goal. The conclusions we draw are useful to policy and decision-makers when reorienting the policy based on a dynamic ex-post evaluation of research groups' performance, thereby allowing them to undertake a command and control strategic planning of the role that research groups should play within an articulated innovation system, so as to better contribute to its final goal of social wealth creation. We constructed a data base including inputs and outputs provided to and generated by the research units participating in R&D projects financed by the SFTP between 1988 and 1999. We conduct our analysis at the micro level. Following Olazarán et al. (2004) , we do not consider the host public research centers as the decision making unit, but the research groups operating within them. This implies that different research units operating in the Guidelines for R&D policy-makers 285 same center can participate in the program, and therefore are individually evaluated. Our target decision making units (DMUs) include thus research units receiving financial and human capital inputs from the Spanish Central Administration to promote applied research within the SFTP. From an institutional perspective they belong to the CSIC. Due to the large proportion of R&D projects obtained by CSIC research groups in the SFTP and the homogeneity of CSIC centers in terms of internal structure, institutional framework, research behavior and other contextual variables-most notably the absence of teaching duties-we have restricted our analysis to these types of research groups. By focusing on a smaller, but nevertheless homogenous and quite representative set of research groups, we considered that the dynamic evaluation of the SFTP would provide more conclusive results. Data were gathered from the central administration body responsible for the project management-Direccio´n General de Ensen˜anza Superior e Investigacio´n Cientı´fica. For the purposes of our study we focus on the role of R&D projects in terms of financial and human capital inputs and three categories of outputs jointly representing a multidimensional output mix, namely training (PhD dissertations and trained scientific personnel), science-technology outputs (international articles and patents), and socio-economic outputs (bilateral R&D contracts with firms).. The time period under study, 1988-1999, comprises the first three Spanish R&D Plans-each covering a period of 4 years. However, as R&D projects within the SFTP may last up to 3 years (CICYT 1987; Jiménez-Sáez 2005) we followed the same rationale in our analysis. Hence, our analysis is split into four periods, covering the natural periodicity length of R&D projects: 1st period: 1988-1990; 2nd: 1991-1993; 3rd: 1994-1996; and 4th: 1997-1999. 6 Table 1 summarizes the variables used in the analysis, classified by input and output categories, their inter-periodical change rate, as well as the overall change rate over the entire time span.. Both the number of personnel and overall budget devoted to the SFTP decline markedly from the first to the last period (1997-1999/1988-1990) , as well as in consecutive periods. From an output perspective, there is a marked growth in the number of R&D contracts signed between research units and private firms to promote joint partnerships leading to practical innovations (166.5 % when accounting for the change in the whole period). This remarkable increase suggests that the Spanish public research bodies are contributing extensively to the articulation of the SFIS (García-Martínez and Briz 2000). 7 In terms of the output variables related to training, while the number of trained people shows a noticeable increase (33.2 %) from 1988-1990 to 1997-1999 , the number of doctoral theses decreases by 36.5%. The remaining variables representing S&T outputs, both number of international papers and registered patents show positive values (40.3 and 28.9 % respectively). Summing up, with decreasing input variables and increasing output variables-except for PhD theses, it is expected that research productivity growth is to be observed throughout the period. 6 We are aware that a three-year periodicity may question the contemporaneous nature of the inputs employed and outputs attained since publications, patents and so on may have a longer time lag. To test the robustness of our results we have replicated all the calculations taking as output variables the-movingaverage of the outputs observed in periods t and t ? 1, and find that efficiency and productivity trends, as well as raking positions do not differ significantly. 7 We consider R&D contracts as an output because it is normally the outcome of a demonstrated research activity-successful enough to convince private firms about the utility and profitability of the applied research they are co-financing. Therefore, when comparing the efficiency and productivity of research groups it must be considered as an outcome of the group's trajectories. that involve relationships with other agents, such as embedding personnel of firms within the units to train it, as well as bilateral R&D contracts with firms, representative both of actions contributing to the articulation of an IS. Based on the profile of the different research groups, these authors propose a taxonomy of the efficient research units depending on their research strategies, which can be divided into: (i) comprehensive, (ii) partial, (iii) specialized and (iv) ''shooting stars''. Comprehensive groups perform an efficient multidimensional research strategy by producing all outputs and have an in depth knowledge of the SFIS. Partial research units represent the largest group comprising those observations whose activities are directed towards the two output dimensions that characterize scientific knowledge production in mode I, i.e. training and S&T variables. Specialized groups are those research units that are consistently efficient by focusing on either S&T variables or socio-economic goals related to profitable bilateral contracts with interest in particular research actions. Finally, ''shooting stars'' describes those efficient research units that sporadically participate in the SFTP with the objective of achieving a particular goal (i.e. accomplishing a specific project, signing a bilateral contract with a firm, etc.), but are not able to raise funds within the SFTP consistently in more than one period. 8 We consider this typology when assessing the overall productivity growth of the efficient research units, examine the sources contributing to its increase, as shown in Sects. 5 and 6, and highlight the most relevant trends followed by each of these groups.. The mean growth rates for all periods of the cumulated MPI are presented in Table 2 , sorted by their (in)efficiency status and group typology: comprehensive, partial and specialized (values for the individual units are reported in ''Appendix'' section).. 9 Calculating average inter-periodical productivity growth rates is necessary so as to render comparable the productivity change of units participating in different number of periods, i.e. not all the CSIC research units included in the analysis participate in the four sub-periods comprising the whole time length under study. Considering the whole period of time, if we relate our results to those presented by Jiménez-Sáez (2004) we can conclude that research groups leading productivity gains have only captured a small proportion of the SFTP financial support (less than 30 %), while they have been engaged in the generation of a larger proportion of research results in any category (more than 40 % of results).. Malmquist productivity change within the SFTP increased inter-periodically by 19.3 % = (1.193 -1) 9 100. The main driver of productivity growth is the change in the technology led by the efficient units operating at the MPSSs, as technical change TC 1;t G -i.e. the upward shift in the production frontier presents a 15.5 % annual increase. Decomposing PTC 1;t G into its two sources, it is the shift in the production frontier allowing for variable returns to scale TC 1;t G what brings higher gains, 23.5 %-this measure can be interpreted as 8 Therefore it is not possible to study productivity change for these efficient units, as well as any other inefficient unit participating in the SFTP in a single period. 9 Note that in Table 2 we present mean values of the Malmquist indices and the different terms in which it decomposes; therefore, multiplying the values reported in Table 2 will not normally result in their aggregates. However, this multiplicative nature of the Malmquist index and its components is preserved in Annex 1, where all terms can be obtained by direct multiplication. the frontier shift for the average output-input scales corresponding to each group. The remaining term, STC 1;t G shows that productivity change at those average output-input scales exceeds that observed at the MPSSs by 4.7 %, and therefore technical change presents a bias in favor of the average productive scale when compared to that observed at the optimal ones. Productivity growth is barely boosted by efficiency change, EC a relevant and generalized catch-up process within the SFTP according to which inefficient research groups would converge toward the efficient frontier by adopting the best practice research strategies and behavior of the leading units, resulting in a slow rate of convergence. Finally, the last source of productivity growth corresponding to the contribution that returns to scale RTS 1;t G shows that changes in the output-input size carry decreasing returns to scale resulting in productivity decline. We remark that the relative contribution of these terms to productivity change is similar across all groups of research units, either efficient or inefficient, as well as when sorting the former according to the previously discussed categories, i.e. as previously discussed the major source of productivity growth corresponds in every group to TC 1;t G while scale efficiency play a very limited role. Virtually all productivity growth is attributable to technical change TC, since it is the main driver of PTC growth, while efficiency change barely contributes to the increment in the Malmquist index. For this reason we depict in Fig. 1 the connection between these two measures for each individual research group classified according to efficiency statusinefficient and efficient (sorted by group category), and whose size is proportional to the average of funding throughout the four periods. The correlation coefficient for the most comprehensive units 0.47 is clearly influenced by IFI-05 that exhibits a Malmquist index of 3.130, and would be statistically significant and rather high, 0.760 if this particular unit were excluded. 10 We focus now on the distribution of the inter-periodical cumulated productivity growth. Table 3 10 However, anticipating our discussion on individual leading units, we stress that the remarkable productivity change value of IFI-05 cannot be recognized as an outlier resulting from data measurement errors, but as the outcome of an outstanding performance. This unit is able to increase outputs while reducing inputs resulting in a productivity growthM 1;t G = 3.130 that can be mostly explained in terms of a remarkable scale efficiency change SEC 1;t G = 1.566, jointly with a substantial technological progress equal to TC 1;t G = 1.491.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O