Computational models of reinforcement learning: the role of dopamine as a reward signal Abstract Reinforcement learning is ubiquitous. Unlike other forms of learning, it involves the processing of fast yet content-poor feedback information to correct assumptions about the nature of a task or of a set of stimuli. This feedback information is often delivered as generic rewards or punishments, and has little to do with the stimulus features to be learned. How can such low-content feedback lead to such an efficient learning paradigm? Through a review of existing neuro-computational models of reinforcement learning, we suggest that the efficiency of this type of learning resides in the dynamic and synergistic cooperation of brain systems that use different levels of computations. The implementation of reward signals at the synaptic, cellular, network and system levels give the organism the necessary robustness, adaptability and processing speed required for evolutionary and behavioral success. In computer sciences, reinforcement learning (RL) combines theories from machine learning, artificial intelligence and dynamic programming. It refers to the trial-and-error learning of the set of actions an agent must take to maximize future rewards or minimize punishments (Sutton and Barto 1998) . RL is fundamental to the psychological and neuroscientific studies of reinforcement and conditioning, and to the neuroscience of decision-making in general including neuroeconomics (Glimcher and Rustichini 2004; Camerer 2008) . Since the early 1980s, there has been a growing interest in mapping computational theories of RL to their underlying neural mechanisms. On the theoretical side, it has become clear that RL results from the complex interactions between different computational subsystems describing processes internal to the organisms and accounting for its interactions with the environment ( Fig. 1 ; Sutton and Barto 1998; Freeman 2007) . On the biological side, it has also become clear that there is no single brain area responsible for the implementation of RL and that learning and reward processing are highly distributed functions involving dozens of dynamically interacting brain structures (Dayan and Balleine 2002) . As this review will show, RL is computationally implemented at multiple levels, from chemical to systems.. In RL, an agent interacts with its environment in order to learn the best actions it must perform to maximize the sum of future rewards. RL models are typically composed of 4 main components (Fig. 1; Sutton and Barto 1998) . (1) A reward function, which attributes a desirability value to a state. The reward is often a one-dimensional scalar r(t), computed on the basis of information streams from the environment. It is an instantaneous scalar that can carry a positive or a negative value. (2) A value function (also known as critic) which determines the long term desirability of a state, based on how much future rewards can be expected for being in that state. The value function may or may not be contingent upon actions taken by the agent. In most models, the output of this function is computed as the Temporal Difference (TD) error between estimated and actual rewards. (3) A policy function (also known as actor) which maps the agent states to possible actions, using the output of the value function (the reinforcement signal) to determine the best action to choose. (4) A model of the environment which includes a representation of the environment dynamics required for maximizing the sum of future rewards. These general RL concepts are further explained and elaborated by various authors (Dayan and Daw 2008; Kaelbling et al. 1996; Montague et al. 2004a; Sutton and Barto 1998; Worgotter and Porr 2005; Dayan and Abbott 2001) .. The general goal of RL models is to maximize the sum of future rewards. However, if rewards are delivered after a delay, or with low probabilities, it can be challenging to determine which state or action to associate with the current reward. This issue is known as the 'temporal creditassignment problem' (Barto et al. 1983; Sutton 1984) . One way of addressing this issue with computational models is to incorporate an eligibility trace. An eligibility trace can be defined as a slowly decaying temporal trace of an event, used to determine which state or action is to be reinforced when a reward arrives later in time under a given policy (Sutton and Barto 1998) . Eligibility traces have been implemented computationally at various levels and a few examples will be presented in this review.. The computer science notion of RL was originally inspired from the psychological theories of reinforcement, and the underlying process of associative learning (Cardinal et al. 2002; Mackintosh 1983; Wise 2004; Wise and Rompre 1989) . At the neural level, reinforcement allows for the strengthening of synaptic associations between pathways carrying conditioned and unconditioned stimulus information. At cognitive and system levels, reinforcement processes may be the basis of the neural substrates of the emotional state (Rolls 2000) . The intrinsic nature of rewards in psychology and computer science, however, is different. In computational RL theories, rewards are simple, 'information poor' scalars influencing learning, while in psychology, rewards are signals from the environment that can arouse the organism (Berridge 2007) .. Despite the differences between computational RL theories and the psychological and physiological aspects of conditioning, findings from these research fields are complementary and converging toward a better understanding of the brain mechanisms underlying reinforcement. RL models have contributed to-and continue to unveil the process underlying reinforcement from the neurochemical to systems levels. We present an overview of the neurobiological underpinnings of reinforcement learning and review the various ways in which the role of rewards has been implemented computationally. We point to the fact that reward signals act as reinforcers for multiple neural mechanisms, being at neurochemical levels or at the system levels. We review models of dopamine (DA) neuron activity elicited by reward-predicting stimuli, models of DA transmission, models of the effect of DA on synaptic transmission and plasticity, and models of the effect of DA on complex neural circuits mediating behavior.. Neurochemical level: models of dopamine neuron activity A large body of experimental work has shown that rewardmotivated behavior depends on the activity of DA neurons from the midbrain ventral tegmental area (VTA) and substantia nigra pars compacta (SNc; Ljungberg et al. 1992; O'Doherty et al. 2003; Seymour et al. 2004; Schultz 1998; Wise and Rompre 1989) . Dopamine has been shown to play a role in motivation (Fibiger and Phillips 1986; Robbins and Everitt 1996; Wise and Hoffman 1992; Wise 2004 Wise , 2005 , in the acquisition of appetitive conditioning tasks (Berridge and Robinson 1998; Everitt et al. 1999; Ikemoto and Panksepp 1999) , in many aspects of drug addiction (Berke and Hyman 2000; Di Chiara 2002; Everitt and Robbins 2005; Kelley and Berridge 2002; Koob 1992; Robinson and Berridge 2008; Wise 1996a, b) and is involved in disorders such as Parkinson disease (Canavan et al. 1989; Voon et al. 2010; Frank 2005; Knowlton et al. 1996; Moustafa et al. 2008) .. Although computational theories of reinforcement learning were in full force in the early 1980s, their popularity significantly increased later, in the early 1990s, with the finding that DA cells were responsive to rewards and reward predictive stimuli Schultz 1994, 1996; Schultz 1992 Schultz , 1998 Waelti et al. 2001) . Compatible with RL theories, these neurons transiently increased their activity in response to the presentation of unexpected rewards (i.e. time limited, in the order of 100 ms) and in response to cues predicting upcoming reward delivery (Hikosaka et al. 2008; Romo and Schultz 1990; Schultz et al. 1997) . As illustrated in Fig. 2 , during classical conditioning, unexpected reward initially triggered an increase in phasic activity of DA neurons, which then shifted with learning to the conditioned stimulus (Ljungberg et al. 1992; Mirenowicz and Schultz 1994; Pan et al. 2005) . When an expected reward was omitted, the activity of DA neurons paused at the precise time when the reward should have been delivered (Ljungberg et al. 1992; Roesch et al. 2007; Satoh et al. 2003) . This activity pattern suggests that DA neurons signal a 'reward prediction error'. After learning therefore, DA neurons do not respond to the reward itself, but the difference between expected and received reward. This notion of 'prediction error' is a central component of current RL theories. Reinforcement learning is at the basis of many forms of adaptation of the organism to its environment. More than simply a form of stimulus-outcome association, RL is intrinsically designed to optimize. Optimization can be that of an amount of rewards, a speed of learning, or that of robustness to perturbing events. This feature of the RL paradigm is puzzling because RL relies on relatively information-poor feedback signals. We reviewed some of the main models of RL and outlined how they can be understood and implemented by the nervous system at multiple levels, from synaptic to system. Rare are the models that attempt multi-level integration. It is however in the integration between these levels that lay the biological success of RL. Finally, we note that this review did not cover the many artificial intelligence approaches that have attempted to address reinforcement at more cognitive and abstract levels. Future work should be done to bridge and contrast the artificial intelligence and computational neuroscience approaches.

O O O B I O O O B O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Aperiodic tilings with one prototile and low complexity atlas matching rules We give a constructive method that can decrease the number of prototiles needed to tile a space. We achieve this by exchanging edge to edge matching rules for a small atlas of permitted patches. This method is illustrated with Wang tiles, and we apply our method to present via these rules a single prototile that can only tile R 3 aperiodically, and a pair of square tiles that can only tile R 2 aperiodically. The field of aperiodic tilings was created by Berger's discovery [7] of a set of 20426 square tiles which were strongly aperiodic in the sense that they could only tile the plane in a non-repeating global structure. Unsurprisingly there has been some interest on how far this number could be decreased. The number of tiles was reduced over time, to 6 square tiles by Robinson [10] in 1971, and, relaxing to non-square tiles, to 2 tiles by Penrose [11] two years later. This led naturally to serious consideration about the possible existence of a single tile. While a simple example has not been forthcoming, if we relax the requirement that the monotile be completely defined by its shape alone, there has been progress.. In [9] Socolar studied a more general problem, 'k-isohedral' monotiles, which had the monotile as a limiting case. Relaxing conditions on edge-coloring, nonconnected tiles or space-filling provided positive results, but not in the limit. In 1996 Gummelt [1] considered tiles that where allowed to overlap, and produced a decorated tile which could force strong aperiodicity. This paper extends work by Goodman-Strauss on 'atlas matching rules'. In [2] Goodman-Strauss describes how by requiring a tiling be covered by a suitable finite atlas of permitted bounded configurations, a domino can serve as a monotile. Sadly the atlas requires patches of extremely large radius. This paper describes a method of altering matching rules from coloured tiles to atlas matching rules with very small patches. Furthermore if two of the tiles have the same shape, the number of prototiles needed is decreased.. We use this method to construct a pair of square tiles which tile R 2 aperiodically, and a single cubic tile that tiles R 3 aperiodically.

O O O O B O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O B O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


PARTITIONING PROBLEMS IN DENSE HYPERGRAPHS Abstract. We study the general partitioning problem and the discrepancy problem in dense hypergraphs. Using the regularity lemma [16] and its algorithmic version proved in [5], we give polynomial time approximation schemes for the general partitioning problem and for the discrepancy problem. For some NP-complete problems good approximation algorithms were developed for the dense instances of the problems. A typical example is the Max-Cut problem in which one tries to partition a vertex set of a graph into two subsets in such a way that the number of edges that have the endpoints in different sets is maximized. The MaxCut problem is known to be NP-complete [13] and, moreover, hardness results for the corresponding approximation problem have been recently proved [3] . In contrast to the difficulty of the general case, for the dense instances of the problem (graph (V, E) has at least c|V | 2 edges for some positive constant c) polynomial time approximation schemes were developed. Arora et al. [2] used the sampling method to design an approximation algorithm that finds in O(|V | O(1/ 2 ) ) time a cut of value which is within 1 − factor of the optimal. Fernandez de la Vega [8] presented a O(2 1/ 2 +o(1) ) algorithm which finds a cut of value within 1 − factor of the optimal. Using a version of the regularity lemma Frieze and Kannan [11] gave an algorithm which finds in O(n 2.376 ) time a cut with value which is within 1 − of the optimal. Other partitioning problems were also considered in [11] . In addition, both sampling method and a version of regularity combined with sampling lead to fast randomized algorithms ( [11] , [12] ). It should be noted then in most of the applications of the regularity lemma a constant hidden in big O depends on and is enormous. However, version proposed by Frieze and Kannan has constant which is relatively small. Many of the problems for which the algorithmic version of the regularity lemma was successfully applied have interesting generalizations to hypergraphs. The situation in hypergraphs is however much more complicated and the obvious generalization of the concept of graph regularity (weak regularity) doesn't always lead to the extensions of applications for graphs. Therefore different approaches to measure the regularity of hypergraphs were developed (strong regularity) [9] , [10] .. We should also mention that the original proof of Szemerédi [16] was not constructive, in a sense that it did not provide a polynomial time algorithm that would find a desired partition. The algorithmic version of graph regularity lemma was given by Alon, Duke, Lefmann, Rödl, and Yuster in [1] and was applied to various problems ( [1] , [6] , [4] , [11] , [14] ). The algorithm from [1] is based on the characterization of regularity which roughly states that a pair of sets is -regular with density d if and only if "most" of the vertices have degree around dn and "most" of the pairs of vertices have co-degree around d 2 n. This characterization does not have a natural generalization to hypergraphs, the algorithmic version of the weak regularity lemma for hypergraphs was however proved recently in [5] (see also [12] ).. In this paper, we will be interested in applications of the algorithmic version of hypergraph regularity lemma. We will discuss two applications, first we will concentrate on the general partitioning problem and then we will discuss the discrepancy problem. Let us start we definitions and notation.. An l-uniform hypergraph is a pair H = (V, E) where V is a nonempty finite set of vertices and E is a set of l-element subsets of V called edges. We will use [k] = {1, . . . , k}.. . . , V k be pairwise disjoint nonempty subsets of the vertex set V . An edge e ∈ E is called crossing in. For an l-uniform hypergraph H = (V, E) we will be interested in finding a partition of V into k (where k ≥ l) nonempty sets that maximizes the number of crossing hyperedges.. For a partition. and letf. where the maximum is taken over all partitions of V into k nonempty sets (k is fixed and independent of the size of H). Finding the value off(H) is NP-complete as even in the case l = k = 2, we get the well studied Max-Cut problem. Using the regularity lemma, we will device an approximation algorithm which finds a partition V 1 ∪ V 2 ∪ · · · ∪ V k , the value of which is within ηn l off(H).. Theorem 2. For l and k ≥ l there is an algorithm which for every l-uniform hypergraph H = (V, E) with |V | = n and every 0 < η < 1 finds in O(n 2l−1 log 2 n) time a partition. The algorithm of Theorem 2 can be used to approximatef(H) in case when the number of edges in H is at least cn l for some positive constant c.. Our second application concerns the discrepancy of a coloring of l-element subsets of [n] . Let χ :. [n] l → {−1, +1} be a {−1, +1}-coloring of l-element subsets of [n] . Define the discrepancy of a set S ⊂ [n] as. where the sum is taken over all e ∈. [n] l which are contained in S and the discrepancy of the coloring χ disc(χ) = max. Our problem will be to find a set S * ⊂ [n] that maximizes the value of d. Using the algorithmic version of the regularity lemma for hypergraphs, we will show the following hypergraph analog of the result in [4] .. Theorem 3. For every l there is an algorithm which for a coloring of the l-element subsets of [n], χ :. Clearly, the algorithm of Theorem 3 will give meaningful results only if disc(χ) ≥ cn l for some positive constant c. If in addition c is known in advance, the algorithm can be easily modified to a polynomial time approximation scheme.

O O O O O O O O O O B I O O O O O O O O B I I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Enabling technologies on hybrid camera networks for behavioral analysis of unattended indoor environments and their surroundings This paper presents a layered network architecture and the enabling technologies for accomplishing vision-based behavioral analysis of unattended environments. Specifically the vision network covers both the attended environment and its surroundings by means of hybrid cameras. The layer overlooking at the surroundings is laid outdoor and tracks people, monitoring entrance/exit points. It recovers the geometry of the site under surveillance and communicates people positions to a higher level layer. The layer monitoring the unattended environment undertakes similar goals, with the addition of maintaining a global mosaic of the observed scene for further understanding. Moreover, it merges information coming from sensors beyond the vision to deepen the understanding or increase the reliability of the system. The behavioral analysis is demanded to a third layer that merges the information received from the two other layers and infers knowledge about what happened, happens and will be likely happening in the environment. The paper also describes a case study that was implemented in the Engineering Campus of the University of Modena and Reggio Emilia, where our surveillance system has been deployed in a computer laboratory which was often unaccessible due to lack of attendance. Giving a quick glance to the panorama of information and communication technologies within research, development and production, it is quite clear the strong push on topics such as wireless communications, mobile computing, distributed networks, sensor networks and on top of them, high-level inferences on the huge amount of data produced with such technologies. Within such ferment, computer vision plays an ever increasing role, and as a specific branch of it, in recent times behavioral analysis has gained more and more attention thanks to two different enabling "technologies": on one side the rich variety of the sensed data (in the meaning of quality, quantity, multi-modality, distribution, etc.), on the other side the advances in machine learning and pattern recognition which can effectively process them. The research results are definitely promising and new and deeper level of behavior understanding are continuously uncovered; actually, it seems really hard to define a reasonable limit, if there is any, to the degree of behavior understanding which machines might attain.. If research is pushing forward, through this paper we claim that the technology is also mature enough to successfully step from research to development in the analysis of simple (or evident) behaviors which are characterized by well defined constraints and goals.. This paper specifically deals with the enabling technologies, mostly from computer vision and distributed sensor networks, that are used as building blocks of an automatic system for the attendance of indoor environments, which can successfully replace the tedious (and costly) humanbased activity often limited to very low-level people monitoring (people positioning, counting, evident abuses of devices, thefts, etc). The field of applicability is wide: it is enough to quote the attendance of shops, libraries, labs, data-centers, etc. Many times such environments are left unattended, since the cost of a human attendance would be definitely bigger than the cost of the problems deriving from its lack.. The proposed architecture is based on a double-layer of camera networks: the first monitors the target environment, the second one its surroundings. Such outer layer does not simply extend the domain of observation, but increases the range of achievable understanding and inference of the system. The employed cameras are of different typologies (fixed and PTZ -Pan-Tilt-Zoom), each assigned to complementary tasks. Moreover sensors beyond the vision could be used to deepen the understanding or increase the reliability of the system. The paper also describes a case study that very recently was implemented in the Engineering Campus of the University of Modena and Reggio Emilia, where our surveillance system has been introduced in a computer laboratory which was often unaccessible due to lack of attendance.. Several works on indoor [16, 42] or outdoor [20, 26, 10, 12, 19] surveillance are available in the literature. In our work, in order to obtain system scalability and efficient resource allocation, Wireless Multimedia Sensor Networks (WMSNs) are used. WMSNs [1] are enabling several new applications such as: multimedia surveillance, traffic avoidance and control systems, environmental monitoring, person locator services and many others. This type of network allows enhancing [14, 18] traditional monitoring and surveillance systems by using multi-sensor data to obtain multi-resolution and multi-source descriptions of the same scene. Moreover onboard processing algorithms allow to reduce bandwidth consumption. Hence, it is possible to obtain faster and more reliable systems but it is necessary to develop architectures to control distributed and collaborative information processing. Beside usual wired distributed systems [33, 43] , there have been several studies on WMSNs. In [28] a model to decompose logical surveillance functionalities into a set of modules (e.g. tracking and classification of objects) is proposed. Each module is then optimally allocated among a set of physical processing units structured in a hierarchical surveillance network. Chu et al. [11] propose a system which uses onboard camera processing to filter out uninteresting events. A factor-graph-based resource allocation algorithm is then used to move cameras mantaining local and peripheral knowledge of new events.. This works does not claim to step into new surveillance fields or scenarios, but to propose and clarify a rendez-vous architecture and application for all these enabling technologies, which are mature to produce real solutions to real problems, fact that is confirmed as more and more world-wide technology companies and corporations are opening up to the video surveillance as part of their business areas.. The paper has the following structure: after a general architecture overview (section 2.1), a detailed description of each single layer will be given: perimeter vision layer (section 2.2), environment vision layer (section 2.3) and reasoning vision layer (section 2.4). We want to underline that what is presented in these sections is (and wants to be) a general and open description of an architecture, which can be specifically implemented in several different ways. Indeed, section 3 will give implementation details of the case study we deployed in our campus, providing the results obtained within such environment. Conclusions will follow. The paper presents a panoramic view over the enabling technologies that, through an ad-hoc architecture, are already mature for being deployed in successful applications for behavior analysis of unattended indoor environments. The presented architecture is based on a scalable and modular approach and purposely separates vision from reasoning and environment sensing from surroundings sensing. We believe that in the next future many application similar to what is described here, with modifications and additions, will seriously step into the market of surveillance solutions and services. The deployment of a real prototype at the University of Modena and Reggio Emilia allowed to design and progressively refine the system architecture using a realword scenario. In our previous works we have evaluated the performance of each single component independently from each other, while the evaluation of the overall three-layer system is still in progress. Our next future commitment is to collect data for experimental results, both on robustness of the behavioral analysis and on the effectiveness of the architecture deployment.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Interactive visual dialog In this paper we propose a paradigm called the Interactive Visual Dialog (IVD) as a means of facilitating a system's ability to recognize objects presented to it by a human. The presentation centers around a supermarket checkout scenario in which an operator presents an item to be tallied to a stationary television camera. An active vision approach is used to provide feedback to the operator in the form of an image (or images) depicting what the system thinks the operator is most likely holding, shown in a viewpoint that suggests how the object should next be presented to improve the certainty of interpretation. Interaction proceeds iteratively until the system converges on the correct interpretation. We show how the IVD can be implemented using an entropy-based gaze planning strategy and a sequential Bayes recognition system using optical flow as input. Experimental results show that the system does, in practice, improve recognition accuracy, leading to convergence to a correct solution in a minimal number of iterations. In this paper, we explore how a visual dialog between a person and a machine can be used to facilitate interpretive tasks such as recognizing objects. The presentation will center around a supermarket checkout scenario in which an operator sweeps an object to be identified in front of a stationary television camera. Instead of a bar code, the system must recognize the object from the sequence of images generated as a result of its motion in front of the camera. This task is difficult because imaging conditions cannot be precisely controlled (e.g. object pose, distance to camera, illumination, etc.) so that it is likely that the system will fail to correctly identify the object in a significant number of instances. In previous work, [1] we showed how an active vision approach could be used to solve a similar recognition problem in the context of a mobile robot in a stationary environment. There, ambiguity of recognition in the form of entropy measures was used to calculate gaze trajectories that minimized the uncertainty of interpretation. The present problem is more difficult for two reasons i) motion is induced by a human instead of a robot and ii) the requested motions must somehow be communicated to the human.. Since the variability of human motions cannot be controlled, they must be treated as noise and accounted for by the recognition process. Rather than attempting to base recognition on a single measurement, a hypothesis filtering strategy is applied to the entire sequence so that evidence for different object hypotheses can be accumulated over time until a clear assertion can be made. The strategy is implemented using a sequential Bayes estimator [1] and is also required to regularize the recognition process. This is because recognition is based on the appearance of the optical flow patterns induced by the object's motion in front of the camera. Although somewhat analogous to approaches employed for gesture recognition [10, 4, 7, 3] , confounding of motion and structure makes it difficult (if not impossible) to make a confident assertion from a single viewpoint. However the likelihood of different objects giving rise to similar appearances from different viewpoints diminishes in the number of viewpoints (temporal regularization). Hence it becomes possible to make a confident assertion once sufficient data have been gathered.. Generating the appearance manifold for each object [12, 11] presents another problem as it must account for all of the expected motions. We argue elsewhere that this process can be made tractable by invoking constraints on the local appearance of rigid-body motion and camera to object distance [1] . However it is not always feasible to expect human generated motions to lie within the permissible range. This is where the active vision approach comes in by means of a paradigm we call the Interactive Visual Dialog (IVD). In addition to the appearance manifold, we construct a second representation called an entropy map which relates the ambiguity of recognition associated with different viewpoints (or poses of the object). The idea behind the IVD is for the system to present to the operator an ordered set of images corresponding to the most plausible hypotheses for what is being presented to the system, but shown in a different pose -the one that suggests how the object should next be presented to improve the certainty of interpretation. The operator then moves the object along a trajectory that mimics the optimal motion sequence. The process repeats until the confidence of interpretation exceeds a prescribed threshold, generally within 1 or 2 iterations.. The remainder of the paper is organized as follows. Section 2 describes the appearance manifold used in our scheme, along with the details of how it is constructed and the recognition strategy used to identify it using subspace methods. The sequential recognition approach used to regularize the interpretations follows next along with an overview of the entropy maps which serve as the basis for active vision. Section 3 lays out the structure of the IVD and the details of our particular implementation, with experimental results presented next in Section 4. Finally we conclude in Section 5 with a brief discussion about the results and pointers to further work. In this paper we have demonstrated how view-based entropy measures can be effectively used to steer an appearance-based recognition system towards viewpoints or poses that minimize ambiguity. The specific case studied was object recognition through the optical flow patterns induced by an operator sweeping an object by hand in front of a stationary camera. A paradigm called the Interactive Visual Dialog was used to provide feedback to the operator in the form of images depicting how the object should be presented to improve the certainty of interpretation. Experimental results demonstrated the effectiveness of this feedback, both for keeping the on-line presentation within the training set and for quickly taking the operator through a sequence of poses by which the correct hypotheses could be identified with certainty. Although motion-based appearances were used in this study, the approach is applicable to any view-based method.. Future work will involve more extensive testing of the algorithm, particularly over a much broader range of permissible motions. This will be possible by using chromatic keying to remove the presence of moving hands from the images. Although it is highly unlikely that this approach will soon be found in your local supermarket, it does point out the possibilities for other applications of machine vision.

B I I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Visual analytics in cheminformatics: user-supervised descriptor selection for QSAR methods AbstractThe design of QSAR/QSPR models is a challenging problem, where the selection of the most relevant descriptors constitutes a key step of the process. Several feature selection methods that address this step are concentrated on statistical associations among descriptors and target properties, whereas the chemical knowledge is left out of the analysis. For this reason, the interpretability and generality of the QSAR/QSPR models obtained by these feature selection methods are drastically affected. Therefore, an approach for integrating domain expert's knowledge in the selection process is needed for increase the confidence in the final set of descriptors.In this paper a software tool, which we named Visual and Interactive DEscriptor ANalysis (VIDEAN), that combines statistical methods with interactive visualizations for choosing a set of descriptors for predicting a target property is proposed. Domain expertise can be added to the feature selection process by means of an interactive visual exploration of data, and aided by statistical tools and metrics based on information theory. Coordinated visual representations are presented for capturing different relationships and interactions among descriptors, target properties and candidate subsets of descriptors. The competencies of the proposed software were assessed through different scenarios. These scenarios reveal how an expert can use this tool to choose one subset of descriptors from a group of candidate subsets or how to modify existing descriptor subsets and even incorporate new descriptors according to his or her own knowledge of the target property.The reported experiences showed the suitability of our software for selecting sets of descriptors with low cardinality, high interpretability, low redundancy and high statistical performance in a visual exploratory way. Therefore, it is possible to conclude that the resulting tool allows the integration of a chemist's expertise in the descriptor selection process with a low cognitive effort in contrast with the alternative of using an ad-hoc manual analysis of the selected descriptors. The selection of a subset of molecular descriptors during the design of a QSAR/QSPR model is a hard task, where several simultaneous goals must be satisfied [1, 35] . Many feature selection methods used for dealing with this problem are focused on statistical relationships among the descriptors and target properties, leaving aspects associated with the chemical knowledge out of the picture. Therefore, the interpretability and generality of the models obtained by these methods are drastically affected. For this reason, a strategy that incorporated expert knowledge in the selection process is required in order to improve the user confidence in the final set of descriptors.. In this paper we address this challenge by proposing a software tool that conjoins statistical methods with interactive visualizations. The main idea is to exploit the expertise of chemists by an interactive visual exploration of different pieces of data generated from statistical and information theory based metrics. Several coordinated visual representations were designed with the aim of capturing different aspects and relationships among descriptors and target properties. Some key insights that can be gained aided by the interactive visualizations are: redundant descriptors, descriptors that provide discriminative information, relevant descriptors by consensus among alternative models, and descriptors whose knowledge helps reduce the uncertainty about the value of the target property.. The capabilities of the proposed tool were assessed through two different scenarios. The first case study corresponds to the prediction of log P liver . In this experiment the tool was used to choose one subset of descriptors from a group of alternative subsets generated automatically by one or more descriptor selection methods. The second study corresponds to the prediction of a relevant mechanical property useful in the design of polymers. In this case, we illustrate the case where the analyst wants to make an intervention in the automatic selection of descriptors in order to incorporate an experimental descriptor to the model. Thus, we showed how the expert can use our tool to analyze information coming from existing descriptor subsets and even incorporate new descriptors according to her own knowledge of the target property.. The results in both cases showed the suitability and convenience of this methodology for selecting sets of descriptors with desirable characteristics (low cardinality, high interpretability, low redundancy and high statistical performance) in an exploratory and versatile way, and with a lower "cognitive effort" in comparison with the alternative of using an ad-hoc manual analysis of the selected descriptors.. Besides the aforementioned advantages of incorporating the analyst in the loop, it is also important to establish the limitations and scope of this software. Here we focus on the descriptor selection stage. In order to address the more general problem of activity/property prediction, we would need to include aspects such as studying the compounds individually or in families, as well as the incorporation of visualizations related to the applicability domain of the QSAR models or the analysis of its outliers.. Finally, this analysis can be repeated by using different prediction methods, which could lead to different results. Yet the focus of this use case is not in the prediction or validation of the results themselves but on a rationale set of steps to find good candidate descriptors. These descriptors should then be evaluated in an unbiased manner and preferably using an external dataset [10] . The second column shows the predictive accuracy of the "best" model after applying 4-fold cross validation on three different methods (linear regression, decision trees, and neural networks). In this case, the best predictive accuracy for the three models was obtained by using a decision tree (M5P) and evaluating using 4-fold cross validation. The parameter setup and predictive accuracy for all methods is available in the Additional file 1: Table S3 . Quantitative structure-activity/structure-property relationship (QSAR/QSPR) models are regression or classification models widely used in cheminformatics, where a biological activity or a chemical property of chemical compounds are modeled in terms of their molecular descriptors. The design of QSAR/QSPR models requires dealing with several problems. One of them is the selection of the most relevant set of molecular descriptors for the property or activity that is intended to be modeled. Chemical structures are usually encoded by a variety of descriptor families such as functional groups, topological, constitutional, thermodynamic, quantum mechanical, etc. Several of them may contribute similar information or may be irrelevant for the biological activity under study, and thus, affecting the discovery of Open Access *Correspondence: ip@cs.uns.edu.arthe descriptor-activity relationship. For this reason, the selection of the most relevant descriptors is regarded as one of the most difficult and crucial tasks for QSAR/ QSPR modeling [1, 2] .. There are several families of methods for addressing the descriptor selection problem, but most of them can be classified into two main categories: classic methods, such as those using multiple linear regression [3] or VSMP [4] , and those using artificial intelligence-based techniques, such as the ones inspired on genetic algorithms, artificial neural networks or fuzzy logic methods [5] [6] [7] . In particular, the artificial intelligence-based methods are better suited to discover strong nonlinearities between the set of descriptors and a given biological activity (or property) and can overcome some limitations of classic descriptor selection methods [8] .. Despite the numerous statistical approaches for feature or descriptor selection, real users that want to extract the most informative descriptors to understand or predict a property/activity still face an important challenge. The main reason is because none of these methods can be claimed as the best approach for any possible combination of dataset and prediction method. Therefore, in a real setting, especially when unseen data is likely to be presented in a future, users do not have a certain way of knowing what descriptor selection method would work best.. Another common criticism that descriptor selection methods receive is that most of these approaches are deemed as "black-boxes" by chemists. This is due to the fact that in order to fine-tune the results or introduce domain knowledge in the selection criteria it is necessary to know the inner workings of the method [9] or, in the best case, to encode this knowledge in the form of some prior probabilities. While some works have overcome this limitation by ad-hoc analyses [10, 11] , we claim that there is a need for better support in the descriptor selection task, so that these methods can be embraced by the cheminformatics community.. There are two research questions that we aim at answering in this paper. The first one is whether we can leverage outcomes from different descriptor selection approaches to arrive at a more thorough decision on the subset of descriptors to be selected. While there exist several methods based on ensembles and voting schemes [12, 13] these approaches leave the chemical knowledge out of the selection process. The second research question is how we can involve the domain expert (e.g. a chemist), so that he or she can incorporate his or her knowledge and expertise during the feature selection process in a semi-automated manner.. In this paper we propose to deal with these two challenges by means of the proposal of a tool that combines statistical methods with interactive visualizations. This type of approaches falls under the emerging area of visual analytics [14] . The main idea behind visual analytics approaches is to merge the computational capacity of statistical and machine learning methods with the human natural ability of identifying patterns in visualizations. By allowing some form of interaction in the visualizations, domain experts can explore the data and provide feedback to the tool, and/or use the tool to arrive at more informative decisions. Visual analytics have been applied to different bioinformatics problems [15] [16] [17] , while its application on cheminformatics has not been considered until recently. In the area of cheminformatics some approaches have been presented, such as CheS-Mapper [18] , ChemMine [19] or MQN-Mapplet [20] , which aim at interactively exploring chemical datasets by means of clustering and dimensionality reduction methods. QSARINS [21] is a comprehensive tool for multiple linear regression QSAR models that aggregates different statistical methods with a special focus on the model validation. INFUSE [22] is a recent visual analytics tool designed to help the analyst understand the predictive power of features in predictive modeling. One limitation of this approach is that the visual analysis is done on a per-feature basis. In contrast to the previous methods, in this tool we propose an approach that focuses on the challenging task of analyzing and selecting subsets of descriptors for QSAR/QSPR. This is done by means of explorative and visual analytics techniques, where the tool facilitates the comparison of multiple descriptors and descriptor subsets coming from the output of different descriptor selection methods. As it was mentioned previously, when a physical chemistry expert develops a QSAR/QSPR predictive model, the choice of the most appropriate descriptors for this model constitutes the first complex challenge. Once the molecular descriptors have been computed for the dataset, different combinations of them should be analyzed in order to obtain a good quality model. In this context, the QSAR/QSPR model must satisfy two quality standards: high prediction accuracy, preferably evaluated with an external dataset, and good interpretability, evaluated from a physicochemical point of view. Considering the high number of molecular descriptors that are usually computed for a dataset, which could be in the range of thousands, a common practice consists of exploring different feature selection methods in order to obtain different smaller subsets of relevant descriptors [1, 5] .. In a next step, these subsets should be systematically compared to find the best combination of descriptors. This process encompasses several aspects, such as analyzing descriptor co-occurrence in the different candidate models, avoiding redundant descriptor sets, and analyzing descriptor-target relationships. This analysis has been traditionally carried out by the expert combining her expertise with the design of plots and tables in an ad-hoc manner [10, 11] .. This task can be tedious and time inefficient, because the relationships among the different pieces of information does not emerge in a straightforward way. At this point, it becomes valuable the contribution that visual analytics techniques can incorporate to this process. The main idea behind these strategies is to summarize and integrate data in a graphical and interactive way, facilitating the finding of key information for supporting domain expert decision making [14] .. At this stage an interactive visual analytics tool, which we will refer to as Visual and Interactive DEscriptor ANalysis (VIDEAN), is proposed for comparing alternative descriptor subsets. These descriptor subsets are obtained from different descriptor selection methods as candidates for the QSAR/QSPR model. The main idea is to use this tool as a decision support system. Undoubtedly, the analyst's task is complex since multiple aspects are involved in the modeling decisions. Therefore, different data exploration strategies are required in order to integrate these multiple pieces of information.. In this tool, we propose to visualize different aspects related to the information required for modeling, uncovering hidden connections among the descriptors and their relationships with the target property. In this way, the specific objectives that guided the design of these visualizations can be defined as follows:. 1. The avoidance of redundant descriptors in QSAR/ QSPR models. This means that if two or more descriptors are conveying similar information, the tool should help choose the most meaningful or suitable descriptors for the model, and thus keeping the descriptor subset small. 2. The complementarities among the selected descriptors and the target property. This means that each descriptor of the model should be relevant for prediction. In other words, if one descriptor were removed, the prediction accuracy of the model should worsen.

B I O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Meetings scheduling solver enhancement with local consistency reinforcement Abstract Meeting scheduling (MS) represents an important real-world group decision application that denotes one of the actual combinatorial problems. Solving this problem consists of scheduling all the meetings while satisfying all the constraints related to both the users and the meetings. However, given human nature, the solution is usually delineated by the encountering of conflicting preferences. Most of existing research efforts allow the relaxation of the users' preferences in order to reach an agreement between all the participants, which is not always possible. In addition, they do not deal with the achievement of any level of local consistency to enhance the efficiency of the solving process, and finally, they do not address the real difficulty of distributed systems, which is the complexity of message passing operations.Here we propose a new approach to facilitate and streamline the scheduling meetings process in any organization. This approach is based on the distributed reinforcement of arc consistency model, which takes into account the difficulties mentioned above. The present work focuses mainly on satisfying meetings hosts' preferences as much as possible, while taking into consideration all users' availability. The underlying selfish protocol is able to efficiently reach the best solution for the host of the meeting (according to the predefined criteria) whenever possible. This process is achieved with the minimal number of exchanged messages and while In daily life, meeting scheduling (MS) is a pre-eminent and typical group decision support problem. It can be described by the process of scheduling events (meetings) involving individual constraints, i.e., private preferences over alternative mutual decisions. These constraints are crucially related to the availabilities and preferences of the users who should participate in the meetings. The MS problem is naturally distributed and cannot be solved by a centralized approach. Solving it involves determining the date, the time, place and the duration of meetings 1 that must be held among several users, depending on their available times, calendar, and preferences. Moreover, each meeting has a priority that indicates its degree of importance. Therefore, solving an MS problem means to find an optimal solution (satisfying some predefined optimality criteria) whenever possible. Various optimality criteria have been suggested in game theory, economics and voting theory. Thus, the solver process should seek for a compromise among different human users' requirements regarding both the potential meetings' times and meetings' priorities.. Many significant research efforts dealing with the MS problem have been proposed in the literature; among them, [1, 5, 7, 11, 12, 13, 20, 22] will be outlined in the next section.. The basic common points stressed in the majority of works in the literature are:. -The relaxation of any of the users' constraints, even nonavailability constraints, to achieve agreement between all of the agents and consequently to solve the problem. However, in the real world problems, it is not always possible to relax the constraints of users. For example, if a user has to travel for business, then such a constraint would oblige him/her to cancel the trip in order to attend the meeting, which may not be always possible. -None of these works has considered the high complexity of message passing operations in real distributed systems. -Only the authors in [10, 11] have dealt with the use of some inferred knowledge to maintain the coherence between meetings in order to steer the selection of the next proposal; none of the other efforts tried to maintain any level of consistency [14] during the negotiation process despite the pre-eminent role of filtering techniques in solving NP-Complete problems. These techniques allow the simplification of any constraint problems by eliminating values or a combination of values that cannot be involved in any solution. Integrating the enforcement of local consistency within the search process is worthwhile for pruning inconsistent values, and consequently saving much fruitless exploration of the search tree, especially on hard and large problems leading to an enhancement of the solving process.. We have learned from all the previous work and focused our research essentially on a new multi-agent MS approach that closely reflects real applications while improving the process of scheduling meetings. The proposed protocol is based on a distributed reinforcement for arc consistency (DRAC) approach [2] . This approach is an asynchronous agent-based approach to enforce arc consistency on any binary constraint network. Note that several levels of local consistency [14] exist in the literature (node, arc, path, kconsistency). Nevertheless, arc consistency (AC) is the most used one due to its low cost. 2 The basic idea of this work is to benefit from the main goal of DRAC in order to reduce the complexity of the meetingscheduling problem solving process. Thus, the meetingscheduling problem is contemplated as a set of distributed reactive self-interested agents in communication; each of them acts on behalf of one user, and maintains its personal information in private. All the agents cooperate and negotiate by exchanging only relevant information. Their decisions are based only on current available local knowledge and without necessarily having any global view. Thus the final result, i.e., the scheduling of the meetings maintained by agents, is obtained as a consequence of the agents' interactions. In such a manner, all of the agents act in parallel and asynchronously via the sending of asynchronous point-to-point messages. This paper significantly extends our previous work [3] .. In this work, we propose to formalize the MS problem as a valued constraint satisfaction problem (VCSP) [17] in which each user maintains two kinds of constraints: hard and soft constraints related to themselves, in addition to the other strong constraints defining the problem. The hard constraints (which can never be violated) represent the nonavailability of the user, while the soft constraints (which can be violated) represent the preference calendar of a user. Furthermore, each new scheduled event is considered as a hard constraint.. This paper is organized as follows. In Section 2, we outline some of the related works. In Section 3, we give the proposed formalization for the MS problem. In Section 4, we present the DRAC model adapted to the MS problem. In Section 5, we describe the global MS dynamic. In Section 6, we discuss the termination and complexity properties. In Section 7, we give the experimental results. Finally, Section 8 concludes the paper. The objective of this paper is to propose a new approach for meeting scheduling (MS) problems that reflects real-world applications. To fulfill such condition, we have considered, in our model, two kinds of constraints to model the users' requirements: hard constraints to model the non-availability of a user and soft constraints to define his/her preferences.. The underlying multi-agent architecture associates an agent with each user and makes the agents interact by sending asynchronous point-to-point messages containing only relevant information to keep, as much as possible, their privacy. The basic idea of this approach consists of two steps: the first reduces the initial problem by reinforcing some level of local consistency (node and arc consistency); and the second step solves the resulting meeting scheduling problem with a minimum number of exchanged messages.. This approach was implemented with Actalk under the Smalltalk-80 environment and compared with an existing approach in literature [22] on randomly generated instances, in mean of CPU time, percentage of scheduled meetings and the number of exchanged messages. The obtained results show that our approach is scalable and worthwhile for processing strong constraints. In addition, they show the importance of the first, i.e., the reduction step. Other experiments were made to measure the percentage of non-viable values discarded from the meetings' calendars. The obtained results showed that this process is appropriate for reducing the MS problem, and consequently the search space, without loss of solutions.. In future work, we will try to improve this approach by allowing a more relaxed system. Then, we will try to implement the improved approach on a multi-processor platform using real data.. Tu Bao Ho received a B. Eng. degree from Hanoi University of Technology in 1978, M.S. and PhD. Degrees from University Paris 6, in 1984 and 1987, a Habilitation diploma in 1998 from University Paris Dauphine. He is currently a professor at School of Knowledge Science, Japan Advanced Institute of Science and Technology (JAIST). His research interests include artificial intelligence, machine learning, knowledge based systems, knowledge discovery and data mining. Among the research studies dealing with solving MS problems are those based on CSP (constraint satisfaction problem) formalism [15] . The underlying problem is formalized as centralized CSP in which all the users' information is centralized in the same process [1, 5] . These works are essentially focused on over-constraint CSPs.. However, recent researchers have argued that the best way to solve MS problems is to use an agent-based approach. The main reason is that agents can accomplish their tasks through cooperation while allowing the users to keep their privacies. A mechanism design approach based on multi-agent system (MAS) to solve MS problems was reported first by Ephrati et al. [7] . The authors defined two paradigms of MS scenarios, open scheduling systems and closed scheduling systems. In our work we focus on the second type, 3 in which the participants belong to a same body, e.g., an organization or a company. The authors proposed three scheduling mechanisms for the closed system, which differ in the information type that each user has to reveal about his individual preferences. They tried to approximate the optimal utilitarian choice while avoiding manipulability by using the Clark Tax method [8] . Garrido and Sycara [12] reported another MAS work that focused on using distributed autonomous and independent agents to solve the problem. Each agent has its individual goal, to schedule the meeting while maximizing its individual preferences. This work is based on the communication protocol presented in [18] where in agents are capable of negotiating and relaxing their constraints in order to reach an agreement on a schedule with high joint utility.. Sen et al. [20] have proposed another technique based on how an application domain for intelligent surrogate agents can be analyzed, understood and represented in order to make these agents able to carry out tasks on behalf of human users, taking into account their environment. Their prior work has focused on agents adapting to environmental changes [21] , but in [20] their efforts were directed towards the integration of user preferences. Often users' preferences are mutually conflicting, so the authors used techniques from voting theory to formally represent and reason with conflicting preferences.. Three other multi-agent approaches to MS problems, using the Partial CSP formalism introduced by [9] , appear in the literature. The first work proposed by [13] offered a new approach to MS problems using fuzzy constraints. The underlying protocol is called the selfish protocol, in which each user tries to maximize his/her preferences during the negotiation process. The second, in [22] , used the distributed valued constraint satisfaction problem (DVCSP) formalism to model the MS problem. This approach is used in our experimental evaluation.. The third work, based on multi-agent systems and using fuzzy constraints to express users' preferences, was presented by Franzin et al. [11] . This meeting scheduling system was based on an existing system that includes hard constraints [10] . The authors proposed, in their work, to integrate preferences to their system and focused on observing the behavior of this new system under several conditions [11] . Their main objective was to evaluate the relations among solution quality, efficiency and privacy. The correlated protocol is based on inferring some new knowledge during the solving process, which may clash with the desires of agents to keep their information private. Two basic global optimization criteria, considered in this work were fuzzy optimality and Pareto optimality. However, in this protocol the number of exchanged messages increases with the size of the problem (number of possible proposals and users). In addition, in the worst case each agent has to reveal all its proposals in order to reach optimality.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


What makes a great journal great in the sciences? Which came first, the chicken or the egg? Abstract The paper is concerned with analysing what makes a great journal great in the sciences, based on quantifiable Research Assessment Measures (RAM). Alternative RAM are discussed, with an emphasis on the Thomson Reuters ISI Web of Science database (hereafter ISI). Various ISI RAM that are calculated annually or updated daily are defined and analysed, including the classic 2-year impact factor (2YIF), 5-year impact factor (5YIF), Immediacy (or 0-year impact factor (0YIF)), Eigenfactor, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, Zinfluence, PI-BETA (Papers Ignored-By Even The Authors), Impact Factor Inflation (IFI), and three new RAM, namely Historical Self-citation Threshold Approval Rating (H-STAR), 2 Year Self-citation Threshold Approval Rating (2Y-STAR), and Cited Article Influence (CAI). The RAM data are analysed for the 6 most highly cited journals in 20 highly-varied and well-known ISI categories in the sciences, where the journals are chosen on the basis of 2YIF. The application to these 20 ISI categories could be used as a template for other ISI categories in the sciences and social sciences, and as a benchmark for newer journals in a range of ISI disciplines. In addition to evaluating the 6 most highly cited journals in each of 20 ISI categories, the paper also highlights the similarities and differences in alternative RAM, finds that several RAM capture similar performance characteristics for the most highly cited scientific journals, determines that PI-BETA is not highly correlated with the other RAM, and hence conveys additional information regarding research performance. In order to provide a meta analysis summary of the RAM, which are predominantly ratios, harmonic mean rankings are presented of the 13 RAM for the 6 most highly cited journals in each of the 20 ISI categories. It is shown that emphasizing THE impact factor, specifically the 2-year impact factor, of a journal to the exclusion of other informative RAM can lead to a distorted evaluation of journal performance and influence on different disciplines, especially in view of inflated journal self citations. Although there is a multitude of contradictory biblical, bibliographical, conceptual, Darwinian, definitional, evolutionary, genetic, grammatical, literary, logical, logistical, mathematical, paleontological, paradoxical, philosophical, processional , and theoretical possibilities associated with the perennial question as to which came first, the chicken or the egg, the same question may also be asked in the context of which came first, the great paper or the great journal in which the paper was published. In short, does the quality, somehow defined, of a journal define the quality of a paper, the reverse, or both?. High quality published research is fundamental to all individual researchers in the sciences and social sciences. In this regard, research assessment rankings are essential to evaluate the research performance of individuals and the quality of academic journals. The perceived research performance of individual researchers can be crucial for hiring, firing, tenure and promotion decisions. In the absence of suitable information regarding the perceived quality of research output, the quality of a journal has frequently been used as a proxy for the research quality of an academic paper. Publishing in a prestigious journal can provide great stimulus to moving in the right direction along the career path. In this context, how might quality and prestige be defined quantitatively?. The perceived quality of a journal would be seen by many as an inappropriate and misleading proxy for the inherently latent quality of a paper, especially in the early years of publication. The quality and prestige of a leading journal is based on the quality of published papers. However, a leading journal cannot be an accurate reflection of the quality of a recently published paper, especially when the paper has received few if any citations to date, and all the more so if a paper is yet to appear in the pages of a journal (for more on this, see Seglen (1997) , who finds that citation rates of papers determine the impact factor of journals, and not vice versa).. It should be borne in mind that the acceptance of a paper for publication in a journal is typically based on one or more of the Editor, Co-editor, Associate Editor, and 1-3 referees. This small group of experts decides the rejection rate prior to publication. It is well known that even experts can and do make mistakes. The rejection rate of a journal after publication depends on the profession. For this reason, it is essential to know the proportion of published papers that is ignored by the profession, and by even the authors. The large market of researchers worldwide is less prone to making errors regarding the quality of academic research papers than a small group of experts at any journal, great or otherwise. This is the reason Chang et al. (2010) proposed PI-BETA (Papers Ignored-By Even The Authors) as a RAM to capture such mistakes after publication.. Type I and Type II errors arise in any decision made under uncertainty, namely bad papers that are accepted for publication in a journal, and good papers that are rejected. As noted in Chang et al. (2010) : ''Great papers appear in great journals [and] All great journals publish great papers [but] Not all papers in great journals are great.'' Papers that have zero citations after a number of years are clearly errors, whether or not they appear in great journals. Editors, Co-editors, Associate Editors and referees may not care about bad papers that are accepted for publication in a journal, but PI-BETA allows the reader to decide whether decisions to publish papers have been prescient.. For purposes of evaluating the research performance of individual researchers and the quality of academic journals, some Research Assessment Measures (RAM) are subscription based, while others can be downloaded free of charge from the Internet. A leading high quality database for generating RAM is the Thomson Reuters ISI Web of Science database (hereafter ISI). This paper examines the importance of ranking RAM, emphasizes the importance of RAM as viable rankings criteria, highlights the usefulness of existing RAM from Thomson Reuters ISI (hereafter ISI RAM), and evaluates the usefulness of three new RAM.. The empirical analysis of RAM presented in this paper concentrates on the 6 most highly cited journals in 20 ISI categories in the sciences. Garfield (1972) developed citation analysis as a tool in evaluating the impact of journals. Although the question posed in the title of the paper refers to greatness, virtually all RAM measures are based on recent citations, that is, within the past 2 or 5 years. Citations capture scientific impact, visibility, performance, influence, utility and prestige, which should not be confused with scientific excellence, quality, content, importance, innovation, scientific progress, intellectual property, industrial transfers, and social benefits, among others. There is no substitute for peer review in evaluating the scientific quality of an individual paper, and hence all of the papers published in journals. Greatness of a journal may be hard to define quantitatively, but the question in the title of the paper still needs to be asked.. As citations form the foundations of most bibliometric measures, the application of a meta analysis of several RAM to the journals in these 20 ISI categories could be used as a template for other ISI categories in both the sciences and social sciences, and as a benchmark for newer journals in a range of ISI disciplines. In addition to evaluating the most highly cited journals in these 20 ISI categories, the paper examines the impact of journal self citations on journal performance, compares alternative RAM, and highlights the similarities and differences of alternative RAM. It is shown that emphasizing THE impact factor, specifically the 2-year impact factor, of a journal to the exclusion of other useful and illuminating RAM, can lead to a distorted evaluation of journal performance and influence on the professions, especially in view of inflated journal self citations.. The plan of the remainder of the paper is as follows. ''Some key bibliometrics research'' section presents some key bibliometric research. ''Research assessment measures (RAM)'' section discusses alternative RAM, with an emphasis on the Thomson Reuters ISI Web of Science database. Alternative RAM that are calculated annually or updated daily are defined and analysed, including the 2-year impact factor (2YIF), both with and without self citations, 5-year impact factor (5YIF), Immediacy (or 0-year impact factor (0YIF)), Eigenfactor score, Article Influence, C3PO (Citation Performance Per Paper Online), h-index, Zinfluence, PI-BETA (Papers Ignored-By Even The Authors), Self-citation Approval Rating (STAR), both historical and 2-year, Impact Factor Inflation (IFI), and Cited Article Influence (CAI). ''Analysis of ISI RAM data'' section discusses and analyses ISI RAM data for the 6 most highly cited journals in each of 20 ISI categories in the sciences, and presents harmonic mean rankings of the 13 RAM. ''Conclusion'' section summarizes the outcomes and discusses some future uses of RAM. The paper discussed alternative Research Assessment Measures (RAM), with an emphasis on the Thomson Reuters ISI Web of Science (hereafter ISI) database. Alternative RAM that are calculated annually or updated daily were defined and analysed, including the classic 2-year impact factor (2YIF), with and without journal self citations, 5-year impact factor (5YIF), Immediacy (or 0-year impact factor (0YIF)), Eigenfactor, Article Influence, h-index, C3PO, Zinfluence, PI-BETA, Impact Factor Inflation (IFI), and three new RAM, namely Historical Self-citation Threshold Approval Rating (H-STAR), 2 Year Self-citation Threshold Approval Rating (2Y-STAR), and Cited Article Influence (CAI).. The RAM data were analysed for the 6 most highly cited journals in each of 20 highlyvaried and well-known ISI categories in the sciences. As citations form the foundations of most bibliometric measures, the application of a meta analysis of RAM to the journals in these 20 ISI categories could be used as a template for other ISI categories in both the sciences and social sciences, and as a benchmark for newer journals in a range of ISI disciplines.. In addition to evaluating high quality research in the 6 most highly cited journals in each of 20 ISI categories, with 112 Distinct journals, the paper also compared alternative RAM, highlighted the similarities and differences in alternative RAM, found that several RAM captured similar performance characteristics for the most highly cited journals, determined that PI-BETA was not highly correlated with the other RAM, and hence conveyed additional information. Harmonic mean rankings of the 13 RAM for the 6 most highly cited journals in the 20 ISI categories were also presented. It was shown that emphasizing THE impact factor, specifically the 2-year impact factor, of a journal to the exclusion of other useful and illuminating RAM, could lead to a distorted evaluation of journal performance and influence on the profession, especially in view of inflated journal self citations.. What is the road map for existing and future RAM, most of which are based on citations? Likely future uses of RAM include using RAM for research assessment exercises, and as input into academic appointments and promotions. Conundrums such as whether or not it is better to publish in a journal with: (i) high rather than low 2-year impact factor (with and without self citations); (ii) high rather than low 5-year impact factor; (iii) high rather than low Immediacy; (iv) high rather than low h-index; (v) high rather than low C3PO; (vi) low rather than high PI-BETA; (vii) high rather than low Eigenfactor score; (viii) high rather than low Article Influence; (ix) high rather than low STAR (both historical and 2-year); (x) low rather than high IFI; and (xi) high rather than low CAI; can also be analysed critically, as such choices will likely increase the probability of being highly cited.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B B I B O O O O O B B B O O O O O O O O B O O O O O O O O O O B O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O B I B O O O O O B B B O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B B I B B B B O O O B O O O O O O O O O O B O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O B O O O O O B O O O O O B O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O