URSA: A System for Uniform Reduction to SAT Abstract. There are a huge number of problems, from various areas, being solved by reducing them to sat. However, for many applications, translation into sat is performed by specialized, problem-specific tools. In this paper we describe a new system for uniform solving of a wide class of problems by reducing them to sat. The system uses a new specification language ursa that combines imperative and declarative programming paradigms. The reduction to sat is defined precisely by the semantics of the specification language. The domain of the approach is wide (e.g., many np-complete problems can be simply specified and then solved by the system) and there are problems easily solvable by the proposed system, while they can be hardly solved by using other programming languages or constraint programming systems. So, the system can be seen not only as a tool for solving problems by reducing them to sat, but also as a general-purpose constraint solving system (for finite domains). In this paper, we also describe an open-source implementation of the described approach. The performed experiments suggest that the system is competitive to state-of-the-art related modelling systems. Following spectacular advances made over the last years, the sat solving technology has many successful applications nowadays -there is a wide range of problems being solved by reducing them to sat. Very often solving based on reduction to sat is more efficient than using a problem-specific solution. Therefore, sat solvers are already considered to be a Swiss army knife for solving many hard csp and np-complete problems and in many areas including software and hardware verification, model checking, termination analysis, planning, scheduling, cryptanalysis, electronic design automation, etc. [6, 3, 12, 25, 74, 48, 50] . Typically, translations into sat are performed by specialized, problem-specific tools. However, using a general-purpose system capable of reducing a wide range of problems to sat can simplify this task, make it less error prone, and make this approach more easily accessible and more widely accepted.. There are already a number of approaches for solving combinatorial and related problems by general-purpose systems that reduce problems to underlying theories and domains (instead of developing special purpose algorithms and implementations). A common motivation is that it is much easier to develop a problem specification for a general system than a 2 P. JANIČIĆ new, special-purpose solver. The general problem solving systems include libraries for general purpose programming languages, but also modelling and programming languages built on top of specific solvers [36, 33, 42, 14] . Most modelling languages are highly descriptive and have specific language constructs for certain sorts of constraints. Specific constraints are translated to underlying theories by specific reduction techniques. Some modelling systems use sat as the target problem [31, 35] and some of them focus on solving np-complete problems by reduction to sat [10] .. In this paper we present a novel approach for solving problems by reducing them to sat. The approach can be seen also as a general-purpose constraint programming system (for finite domains). The approach consists of a new specification/modelling language ursa (from Uniform Reduction to SAt) and an associated interpreter. In contrast to other modelling languages, the proposed language combines features of declarative and imperative programming paradigms. What makes the language declarative is not how the constraints are expressed, but only the fact that a procedure for finding solutions does not need to be explicitly given. On the other hand, the system has features of imperative languages and may be seen as an extension of the imperative programming paradigm, 1 similarly as some constraint programming systems are extensions of logic programming paradigm. In contrast to other modelling languages, in the proposed specification language loops are represented in the imperative way (not by, generally more powerful, recursion), destructive updates are allowed, there is support for constraints involving bitwise operators and operators for arithmetic modulo 2 n . There are problems for which, thanks to these features, the modelling process is simpler and specifications are more readable and easier to maintain than for other languages and constraint systems. However, of course, the presented system does not aim to replace other constraint systems and languages, but rather to provide a new alternative with some distinctive features.. The used uniform approach enables a simple syntax and semantics of the ursa language, a simple, uniform reduction to sat and, consequently, a simple architecture of the whole system. This enables a straightforward implementation of the proposed system and a rather straightforward verification of its correctness. This is very important because, although it is often easier for a declarative program than for a corresponding imperative program to verify that it meets a given specification, this still does not lead to a high confidence if the constraint solving system itself cannot be trusted.. The presented approach is accompanied with an open-source implementation, publicly available on the Internet. A limited experimental comparison suggest that the system (combined with state-of-the-art sat solvers) yields good performance, competitive to other modern approaches.. Overview of the paper. In Section 2 we give relevant definitions; in Section 3 we provide motivation and basic ideas of the proposed approach. In Section 4 we describe the specification language ursa, in Section 5 its semantics, in Section 6 the corresponding interpreter, and in Section 7 pragmatics of the language. In Section 8 we discuss related techniques, languages and tools. In Section 9 we discuss directions for future work and in Section 10 we draw final conclusions. There are constraint programming libraries for imperative languages, but these still do not follow the spirit of imperative programming and are substantially declarative. The current system (with the presented semantics and the corresponding implementation) uses one way (binary representation) for representing (unsigned) integers but (as shown in the given examples), it still enables using different encoding styles in specifications. For further convenience, we are planning to natively support other representations for integers, so the user could choose among several encodings. Also, signed integers and floating point numbers could be supported.. The language ursa (and the interpreter) can be extended by new language constructs (e.g., by division). A new form of the assert can be added, such that it propagates intermediate solutions to subsequent commands of the same sort. Support for global constraints can also be developed, but primarily only as a ,,syntactic sugar" -the user could express global constraints more easily, but internally they would be expanded as if they were expressed using loops (i.e., as in the current version of the system). Alternative forms of support for global constraints would require substantial changes in the sat-reduction mechanism.. On the lower algorithmic and implementation level, we are planning to further improve the current version of transformation to cnf.. In the current version, ground integers are represented by built-in fixed-precision integers, which is typically sufficient. However, in order to match symbolic integers, ground integers should be represented by arbitrary (but fixed) length integers and we are planning to implement that.. Concerning the underlying sat solvers, currently only two complete sat solvers are used. We are planning to integrate additional solvers, since some solvers are better suited to some sorts of input instances, as the sat competitions show. Within this direction of work, we will also analyze performance of stochastic solvers within ursa. In addition, we are exploring potentials of using non-cnf sat solvers [67, 53, 38] within ursa, which avoids the need for transformation to cnf [69] . Choosing among available solvers can be automated by using machine learning techniques for analysis of the generated sat instances (or even input specifications) [73, 55] . For solving optimization problems, instead of the existing naive implementation, we are planning to implement more advanced approaches and to explore the use of MaxSAT and pseudo-Boolean solvers [3] .. On the theoretical side, the full operational semantics outlined in this paper can be formally defined and it could be proved that solutions produced by the ursa system indeed meet the specifications and if there are no produced solutions, then the specifications is inconsistent. Along with the formal verification (i.e., verification within a proof assistant) of the sat solver ArgoSAT used [45, 44] , that would provide a formal correctness proof of the ursa system (which would make it, probably, the first trusted constraint solver).. In the presented version of ursa, reducing to sat is tightly integrated (and defined by the semantics of the system) in the program execution phase. An alternative would be as follows: during the program execution phase, a first-order formula is generated and only before the solving phase it is translated to a propositional formula. Moreover, the generated formula would not need to be translated to a propositional formula, but could be tested for satisfiability by using smt (satisfiability modulo theory) solvers (e.g., for linear arithmetic, equality theory, alldifferent theory etc.) [2] . In particular, symbolic computations employed by the ursa system are closely related to the theory of bit-vector arithmetic and to decision procedures for this theory based on "bit-blasting" [8, 9] . Since solvers for this theory typically cover all the operators used in ursa, the theory of bit-vector arithmetic can be used as an underlying theory (instead of propositional logic) and any solver for bit-vectors arithmetic can be used as a solving engine. Generally, the reduction could be adaptable to smt solvers available -if some solver is available, then its power can be used, otherwise all generated constraints are exported to propositional logic. This would make the approach more powerful and such development is the subject of our current work -the system ursa major will be able to reduce constraints not only to sat but to different smt theories. That system will be a general constraint solver and also a high-level front-end to the low level smt-lib interchange format, and, further, to all smt solvers that supports it. Reduction to the theory of bit-vector arithmetic is firstly explored in this context [46] and it shows that reducing to bit-vector arithmetic does not necessarily lead to more efficient solving process than reducing to sat (and the same holds for other smt theories).. With the increased power of the presented system (by using both sat and smt solvers), we are planning to further consider a wide range of combinatorial, np-complete problems, and potential one-way functions and also to apply the ursa system to real-world problems (e.g., the ones that are already being solved by translating them to sat). Some applications in synthesis of programs are already the subject of our current work.. For the sake of easier practical usability of the ursa system, we are planning to develop a support for integration of ursa with popular imperative languages (C, C++, Java). In this paper we described a novel approach for uniform representation and solving of a wide class of problems by reducing them to sat. The approach relies on:. • a novel specification language that combines features of imperative and declarative languages -a problem is specified by a test, expressed in an imperative form, that some values indeed make a solution to the problem; • symbolic computations over unknowns represented by (finite) vectors of propositional formulae. The approach is general and elegant, with precisely defined syntax and induced (by the concept of symbolic execution) semantics of the specification language. This enables straightforward implementation of the system and it works as a "clear box". The proposed language is a novel mixture of imperative and declarative paradigms, leading to a new programming paradigm. Thanks to the language's declarative aspects -the problem is described by what makes a solution and not by describing how to find it -using the system does not require human expertise on the specific problem being solved. On the other hand, specifications are written in imperative form and this gives the following advantages compared to other modelling languages (all of them are declarative):. • problem specifications can involve destructive assignments, which is not possible in declarative languages and this can be essential for many sorts of problems (e.g., from software verification); • modelling problems that naturally involve loops (and nested loops) is simple and the translation is straightforward; • for users familiar with imperative programming paradigm, it should be trivial to acquire the specification language ursa (since there are no specific commands or flow-controls aimed at constraint solving);. • the user has a fuller control of internal representation of the problem, so can influence the efficiency of the solving phase.. • a specification can be taken, almost as-is, from and to languages such as C (within C, such code would check if some given concrete values are indeed a solution of the problem).. • the system can smoothly extend imperative languages like C/C++ or Java, (as constraint programming extends logic programming).. • the system can be verified to be correct in a straightforward manner.. In addition, the system ursa, in contrast to most of (or all) other modelling systems, supports bit-wise logical operators and constraints involving modular arithmetic (for the base of the form 2 n ), which is essential for many applications, and can also enable efficient problem representation and problem solving.. The proposed approach can be used for solving all problems with Boolean and numerical unknowns, over Boolean parameters and numerical parameters with finite domains that can be stated in the specification language (e.g., the domain of the system is precisely determined by expressiveness of its specification language).. The search for required solutions of the given problem is performed by modern sat solvers that implement very efficient techniques not directly applicable to other domains. While sat is already used for solving a wide range of various problems, the proposed system makes these reductions much easier and can replace a range of problem-specific tools for translating problems to sat. The tool ursa can be used not only as a powerful general problem solver (for finite domains), but also as a tool for generating sat benchmarks (both satisfiable and unsatisfiable). In this section we give a brief account of the sat and csp problems and related notions.. Propositional logic. We assume standard notions of propositional logic: literal, clause, propositional formula, conjunctive normal form (cnf), valuation (or assignment), interpretation, model, satisfiable formula, etc. We denote by ⊤ and ⊥ the Boolean constants true and false and the logical connectives by ¬ (negation), ∧ (conjunction), ∨ (disjunction), ⊕ (exclusive disjunction), ⇒ (implication), ⇔ (equivalence). Two formulae A and B are said to be equivalent if A and B have the same truth value in any valuation. Two formulae A and B are said to be weakly equivalent (or equisatisfiable) if whenever A is satisfiable then B is satisfiable and vice versa.. Constraint Satisfaction Problem. A constraint satisfaction problem (csp) is defined as a triple (X, D, C), where X is a finite set of variables x 1 , x 2 , . . ., x n , D is a set of domains d 1 , d 2 , . . ., d n for these variables, and C is a set of constraints c 1 , c 2 , . . ., c k . In a finitedomain csp, all sets from D are finite. Constraints from C may define combinations of values assigned to variables that are allowed or that are prohibited. A problem instance is satisfiable if there is an assignment to variables such that all constraints are satisfied. Such assignment is called a solution. A constraint optimization problem is a csp in which the goal is to find a solution maximizing (or minimizing) a given objective function over all allowed values of the given variables.. sat Problem and sat Solvers. sat is the problem of deciding if a given propositional formula in cnf is satisfiable, i.e., if there is any assignment to variables such that all clauses are true. Obviously, sat is a special case of csp, with all variables ranging over the domain {0, 1} and with constraints given as clauses. sat was the first problem shown to be np-complete [15] , and it still holds a central position in the field of computational complexity. Stochastic sat solvers cannot prove the input instance to be unsatisfiable, but may find a solution (i.e., a satisfying variable assignment) for huge satisfiable instances quickly. On the other hand, for a given sat instance, a complete sat solver always finds a satisfying variable assignment or shows that there is no such assignment. Most of the state-of-the-art complete sat solvers are cdcl (conflict-driven, clause-learning) based extensions of the Davis-PutnamLogemann-Loveland algorithm (dpll) [18, 17, 3] . In recent years, a tremendous advance has been made in sat solving technology [52, 22, 75, 3] . These improvements involve both high-level and low-level algorithmic techniques. The advances in sat solving make possible deciding satisfiability of some industrial sat problems with tens of thousands of variables and millions of clauses.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Restoring subsampled color images Abstract. In some capturing devices, such as digital cameras, there is only one color sensor at each pixel. Usually, 50% of the pixels have only a green sensor, 25% only a red sensor, and 25% only a blue sensor. The problem is then to restore the two missing colors at each pixel -this is called "demosaicing", because the original samples are usually arranged in a mosaic pattern. In this short paper, a few demosaicing algorithms are developed and compared. They all incorporate a notion of "smoothness in chroma space", by imposing conditions not only on the behavior of each color channel separately, but also on the correlation between the three channels. With the advent of color technology, more and more of the images which are captured, copied, transmitted, and processed, are color images. Naturally, this results in a growing interest in extending various algorithms for gray-level images, to color images. Various researchers have discovered that applying standard deblurring or enhancement algorithms separately to every color channel is not optimal; better results can be obtained in view of the fact that, in natural images, there are strong correlations between the channels [6, 7, 9-12, 14-17, 20] . In this paper, the so-called "demosaicing" problem is addressed; this problem is relatively new, and is motivated by the fact that, in digital cameras and other capturing devices, there is often only one color sensor at each pixel. Thus, the missing colors have to be "filled in". The array of sensors is often called a "mosaic", hence the name "demosaicing" for the task of filling in the missing colors. The following algorithms are introduced and studied.. -Regularization with a "color correlation term". This algorithm resembles ordinary regularization, however, an. Some of the algorithms in this paper are included in a patent application to the US Patent Office.. Correspondence to: D. Keren additional term is introduced to account for the betweenchannel correlations, using the estimated local averages and covariance matrices. -The local MAP algorithm, which also uses the estimated local averages and covariance matrices of the color channels. Here, the missing colors are filled in simply by choosing, at each pixel, values which maximize the probability with regard to the Gaussian distribution induced by the local average and covariance matrices. The algorithm can be improved by segmenting each neighborhood before the local averages and covariance matrices are computed; this improves the results, particularly in areas of strong discontinuities between uniform regions. -Processing in the angle domain. This is the simplest among the suggested algorithms, and consists of representing the color image in spherical coordinates, and then filtering it in the new representation; we tried both standard regularization and median filtering. However, only the angle components of the spherical representation are regularized. This allows preservation of luminance discontinuities in the image, while smoothing it in chroma space. -Regularization with a vector-product-based betweenchannel term. Here, correlation between the channels is forced by minimizing not only the standard regularization smoothness term, but also a smoothness term which consists of the sum of the squared norms of vector products between neighboring pixels, where the color values are viewed as vectors in R 3 . The advantage of this algorithm is that it does not require any estimation of the between-channel correlation; also, it gives rather good results, and preserves image discontinuities. The vector-product-based term also proved useful for denoising color images [12] . Most previous research on demosaicing took place in industry, due to the great importance of demosaicing in colorcapturing devices. In [5] , the missing color values are filled in by linear interpolation; then, the color differences are median filtered and the image restored according to the new values, which reduces "color fringes" near edges. In [2] , the missing green values are first filled in by a simple interpolation, and then the other missing values are filled, in a manner which forces the ratios between the different color channels to change slowly between neighboring pixels. In [3] , it is suggested to first segment the image, based on the green (luminance) values, and fill in the missing colors in each region independently. In [19] , the optimal approximation (in the mean square error sense) of the missing colors as linear combinations of the measured ones is derived, and used to interpolate the missing colors. In [1] , the Bayesian restoration paradigm is extended to color images, by assuming a prior distribution over color images which favors positive correlation between the different channels. In [4] , a nonlinear algorithm is suggested, which locates structures such as edges in the image, and tries to preserve discontinuities during the demosaicing process.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O


A geometric aggregation operator for decision making Abstract We proposed an aggregation operator which is used to aggregate decision makers' opinions in group decision making process. First, a Choquet integral-based distance between generalized interval-valued trapezoidal fuzzy numbers is defined. Then combining the generalized intervalvalued trapezoidal fuzzy number aggregation operator with Choquet integral-based distance, an extension of technique for order preference by similarity to ideal solution method is developed to deal with a multi-criteria generalized intervalvalued trapezoidal fuzzy number group decision making problems, where inter-dependent or interactive characteristics among criteria preference of decision makers are also considered. Finally, an illustrative example is provided to elaborate the proposed method.Keywords Multi-criteria group decision making · Generalized interval-valued trapezoidal fuzzy numbers · Fuzzy measures · Technique for order preference by similarity to ideal solution Technique for order preference by similarity to ideal solution (TOPSIS) was initiated by Hwang and Yoon [21] . TOPSIS is a useful and practical technique for selection and ranking of alternatives [8] . The basic principle is that the chosen alterna- In the TOPSIS theory, crisp values are used for weights and performance ratings of the criteria. Hwang and Yoon [21] developed a classical approach to multiattribute/multi-criteria decision making (MADM/MCDM) problems by using TOPSIS. Often, human judgements and preferences are ambiguous, vague and cannot be estimated with exact numeric value under many conditions, so the crisp values are not suitable to model real-world situations. Zadeh [45] proposed the concept of fuzzy set theory and successfully used to handle imprecision (or uncertainty) in decision making problems, to solve the ambiguity and vagueness in information from human judgements and preferences. TOP-SIS has been extended to fuzzy environment in [23] . Fuzzy numbers are applied and used to establish a prototype fuzzy TOPSIS [12, 32] . Recently a lot of work on fuzzy TOPSIS has been developed by several authors [6, 16, 30] . Extension of TOPSIS method for triangular fuzzy numbers is developed in [10] . TOPSIS was further extended by Xu and Chen [42] , an interactive method for fuzzy multiple attribute group decision making.. Interval-valued fuzzy TOPSIS method and experimental analysis has been proposed by Chen and Tsao [14] and they also gave the ranking technique of alternatives based on distance measure. Ashtiani et al. [2] gave an extension of fuzzy TOPSIS for interval-valued fuzzy sets. Chen and Lee [13] further developed interval type-2 TOPSIS method for fuzzy multiple attributes group decision making. On the basis of interval arithmetic, Chu and Lin [16] introduced the fuzzy TOPSIS model. The concept of interval-valued intuitionistic fuzzy sets (IVIFS) was given by Atanassov [4] , as a generalization of IFS. The basic characteristic of the IVIFS is that the values of its membership function and nonmembership function are intervals rather than exact numbers.. Some operational laws of the IVIFS are defined in [3] . In [38] a novel method for multiple attribute decision making based on IVIFS and TOPSIS method in uncertain environments is presented. Wei [41] applied IIFWGA aggregation functions to deal with dynamic multiple attribute decision making where all the attribute values are expressed in intuitionistic fuzzy numbers or interval-valued intuitionistic fuzzy numbers. TOPSIS-based nonlinear programming methodology for multi-attribute decision making with interval-valued intuitionistic fuzzy sets was proposed by Li [27] . Beg and Rashid [7] proposed the concept of trapezoidal valued intuitionistic fuzzy sets and used it for multi-criteria fuzzy decision making with Choquet integral-based TOPSIS. Recently, Li and Chen proposed a multi-criteria group decision making method based on trapezoidal intuitionistic fuzzy information in [28] .. Several aggregation process are based on the assumption that the criteria (attribute) or preferences of decision makers are independent, and the aggregation operators are linear operators based on additive measures, which is characterized by an independence axiom [24, 40] . For real decision making problems, there is a phenomenon that there exists some degree of inter-dependent or interactive characteristics between criteria [19] . Usually, decision makers invited from same or similar fields for a decision problem. They have similar social status, knowledge and preference. Their subjective preference show non-linearity. Independence phenomena among these criteria and mutual preferential independence of decision makers are thus violated. Sugeno [37] introduced the concept of non-additive fuzzy measure, which makes monotonicity instead of additivity property of fuzzy measure. It is a more effective tool to model interaction phenomena [22, 26] and to deal with decision making problems [19, 33] . Liginlal and Ow [29] is an excellent review on analyzing decision maker behavior using fuzzy measure. In the real decision making problems, the attributes of the problem are often correlated or inter-dependent. Choquet integral [15] is a useful tool to model the correlation or inter-dependence. It has been studied and applied in the decision making methods [9, 17, 31] . Choquet integral is also used for interaction between criteria in hierarchy process [1] . Aggregation of decision makers' opinions is very important in group decision making problems to perform evaluation process. Group decision making involves weighted aggregation of all individual decisions to obtain a single collective decision. In [36] , aggregation operator of intuitionistic fuzzy group decision making is proposed with the weights of decision makers. The weights of decision makers play an important role in the process of aggregation. In [38, 43] , aggregation of the interval-valued intuitionistic fuzzy group decision making environment with the Choquet integral is studied.. The concept of interval-valued fuzzy number is extended to the concept of generalized fuzzy number in [44] . Furthermore, the concept of generalized fuzzy number is extended to the generalized interval-valued trapezoidal fuzzy numbers (GITFNs) by Chen [11] . GITFNs are suitable for imprecise and uncertain preference or information in multi-criteria decision method. Chen [11] extended TOPSIS method for multiple criteria decision analysis (MCDA) based on GITFNs. Recently, TOPSIS is modified for generalized interval-valued trapezoidal fuzzy numbers to select the most suitable robot in [34] . Dubois [18] gave some literature about some old and new techniques for fuzzy decision analysis. Moreover, Dubois discussed membership function, aggregation operators, linguistic variables, fuzzy intervals and the valued preference relations. By the motivation of new techniques of decision analysis in [18] and the use of Choquet integral in MCDM in [38, 39] , we extended TOPSIS for GITFNs with aggregation operators and preference of decision makers. Combining this proposed aggregation operator with TOPSIS on Choquet integral-based distance, a multicriteria generalized interval-valued trapezoidal fuzzy group decision making is proposed, where interaction phenomena among the decision making problem and weights of decision makers are taken into account.. The rest of the article is organized as follows: in Sect. 2, we review fuzzy set, fuzzy measure, triangular norm, interval-valued fuzzy sets and generalized interval valued trapezoidal fuzzy numbers (GITFNs). In Sect. 3, we introduce an exponent law on GITFN, order relation and some of its properties are also studied in this section. In Sect. 4, based on some operational laws, a generalized intervalvalued trapezoidal fuzzy number geometric aggregation (GITFNGA) operator is proposed, and some of its properties are examined. In Sect. 5, according to definition of Choquet integral, we define the Choquet integral-based distance between any two generalized interval-valued trapezoidal fuzzy sets. Combining the generalized interval-valued trapezoidal fuzzy geometric aggregation operator with Choquet integral-based distance, an extension of TOPSIS is developed to deal with a multi-criteria generalized interval-valued trapezoidal fuzzy group decision making problem where inter-dependent or interactive characteristics among criteria and preferences of decision makers are considered. In Sect. 6, an illustrative example is constructed to understand the application of the method and to demonstrate its practicality and feasibility.. Let X be a crisp universe of generic elements, a fuzzy set A in the universe X is a mapping from X to [0, 1]. For any x ∈ X, the value A(x) is called the degree of membership of x in A.. Let X = {x 1 , x 2 , . . . , x n } be the set of the attributes, P(X ) be the power set of X. Definition 2.1 [37] A λ-fuzzy measure μ on the set X is a set function μ : P(X ) → [0, 1] satisfying the following properties:. In the above definition, if λ = 0, then the third condition reduces to the axiom of the additive measure: μ(B ∪ C) = μ(B) + μ(C) for all B, C ⊆ X and B ∩ C = φ, which indicates that there is no interaction between B and C. Also λ = 0 indicates that the λ-fuzzy measure μ is non-additive and there is interaction between B and C.. For λ > 0, μ(B ∪C) > μ(B) +μ(C), which implies that μ is a super-additive measure. When λ < 0, then μ(B ∪C) < μ(B)+μ(C), which shows that μ is a sub-additive measure. The interaction between sets can be represented by λ.. If X is a finite set, then. To determine λ-fuzzy measure μ on X, Sugeno [37] gave the following Eq. (2.1). It can be noted that μ(x i ) for a subset with a single element x i is called a fuzzy density.. Specially for any subset A ⊂ X, we have. Based on Eq. (2.1), the value of λ can be uniquely determined from μ(X ) = 1, which is equivalent to solving. If the elements of B in X are independent, we have. So the interaction between sets or elements of set depends upon the value of λ and μ. Definition 2.2 [25] A binary operation T : [0, 1] 2 → [0, 1] is a triangular norm (t-norm) if it satisfies the following: . . Set of all IVFSs on X is denoted by IVFS(X ). We have studied the situation that the attributes in the decision making problem are interactive or inter-dependent and the evaluation values are GITFNs. We have defined an aggregation operator with Choquet integral for generalized interval-valued trapezoidal fuzzy group decision making process based TOPSIS, where the inter-dependent of attributes is considered. The GITFS is the best way to deal with uncertainty. G I T F N G A operator is used to aggregate the values of decision makers. Generalized interval-valued trapezoidal fuzzy positive and negative ideal solution calculated by using distance based on Choquet integral. The relative closeness coefficient is used to rank alternatives. The properties of GITFNGA operator are studied, such as idem-potency, commutativity, boundedness and monotonicity. We have applied this operator to the multi-criteria generalized interval-valued trapezoidal fuzzy group decision making with Choquet integral-based TOPSIS. Finally, an example has been provided to show the feasibility of our proposed decision making method. The proposed method is different from all the previous techniques for group decision making due to the fact that the proposed method uses generalized interval trapezoidal fuzzy set theory with interdependent criteria taken into account and GITFNGA operator. So it is efficient and feasible for real-world decision making applications. In future, we plan to continue working in the extension and application of the developed multi-criteria generalized interval trapezoidal fuzzy group decision making with Choquet integral-based TOPSIS to some extended forms of intuitionistic spaces and other domains.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O B I I I I I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O B I I I I O B I O B I I I I I I I I B I I I I I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


OneLOop: for the evaluation of one-loop scalar functions ONELOOP is a program to evaluate the one-loop scalar 1-point, 2-point, 3-point and 4-point functions, for all kinematical configurations relevant for collider-physics, and for any non-positive imaginary parts of the internal squared masses. It deals with all UV and IR divergences within dimensional regularization. Furthermore, it provides routines to evaluate these functions using straightforward numerical integration. The experiments at LHC for the study of elementary particles demand precise predictions of signals and potential backgrounds for new physics. This implies that for many processes the hard part has to be evaluated at least at the next-to-leading (NLO) level in perturbation theory, and this includes processes with four or more particles in the final state. There has been a considerable progress in this respect in recent years, resulting in full differential calculations for such processes [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] . One of the bottlenecks that had to be resolved in order to achieve this was the calculation of the one-loop amplitude necessary to determine the virtual contribution, which in the mentioned publications involves at least 6-point functions. Developments have reached the stage that by now public programs exist that, given the machinery to calculate treelevel amplitudes, can calculate one-loop amplitudes involving 6-point functions and beyond at a high level of automation [13, 14] .. Besides dealing with an increase in combinatorial complexity, moving from tree-level amplitudes to one-loop amplitudes demands the treatment of one-loop integrals. The most successful approach has been to express the one-loop amplitudes as a linear combination of universal oneloop functions, dependent on the particular scattering process under consideration only through The program ONELOOP for the evaluation of one-loop scalar functions up to and including 4-point functions was presented. It deals with all kinematical configurations relevant for colliderphysics, and any non-positive imaginary parts for the internal squared masses, and it deals with all UV and IR divergences within dimensional regularization. Furthermore, it provides routines to evaluate all these functions using straightforward numerical integration, and the methods applied to achieve this were presented. The program can be obtained from http://helac-phegas.web.cern.ch/ 1 Within the convention of Eq. (14), I 3 (j) actually is a triangle integral divided by −(1 + ε). c = Whereas the integral for triangle (6) was divergent due to the large-x behavior, this integral is divergent due to the small-x behavior. Notice that a factor x 1+ε can be taken out of the denominator. Using this fact, we apply subtraction to write . where we used the identity. −1+2ε. Both integrals in Eq.(52) are now finite. To obtain the ε −1 -term and the ε 0 -term, the integrand of the second integral may be expanded to order ε, and ε may be put to zero in the first integral. Notice that the first, 2-dimensional, integral vanishes for m 2 = 0, i.e. for triangle (2) in [15] .

O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Reinforcement learning with via-point representation In this paper, we propose a new learning framework for motor control. This framework consists of two components: reinforcement learning and via-point representation. In the field of motor control, conventional reinforcement learning has been used to acquire control sequences such as cart-pole or stand-up robot control. Recently, researchers have become interested in hierarchical architecture, such as multiple levels, and multiple temporal and spatial scales. Our new framework contains two levels of hierarchical architecture. The higher level is implemented using via-point representation, which corresponds to macro-actions or multiple time scales. The lower level is implemented using a trajectory generator that produces primitive actions. Our framework can modify the ongoing movement by means of temporally localized via-points and trajectory generation. Successful results are obtained in computer simulation of the cart-pole swing up task. q In our daily lives, it seems that we usually make good choices unconsciously when determining an action to take from among several choices under various environmental conditions. We must make such choices on both large and small scales. For example, we must select a route to a restaurant from a variety of routes. After being seated, we have more details to decide. Consider a person reaching for a menu on the table. The goal of the movement is to move the arm toward the menu to grab it. We must select one desirable trajectory out of an infinite number of possible trajectories. Animals and humans are able to empirically learn how to select good actions from past experiences. We can also learn from demonstrations performed by other people (Atkeson & Schaal, 1997a,b; Miyamoto & Kawato, 1998; Miyamoto et al., 1996; Schaal, 1999) , which is called 'learning by watching' or 'teaching by showing.' However, it is still difficult to start from scratch in a conventional learning machine. A reinforcement learning framework is fascinating and can obtain such autonomous behavior (Sutton & Barto, 1998 for a detailed and comprehensive description).. A conventional reinforcement learning framework is very powerful for small-scale problems such as a simple maze on a computer screen, but it cannot be easily applied to large-scale problems (e.g. driving a car in a real environment). For application to realistic and large-scale problems, the recent trend of research in reinforcement learning is to use modularization and hierarchical architecture. Several methods for constructing such higher levels have been studied, such as macro-actions, sub-goals, and multiple time scales (McGovern, Precup, Ravindran, Singh, & Sutton, 1998; .. Since the feedback gain of biological systems is low and delay is large, high-speed movement must be controlled with feed-forward control. Recent results of a psychological experiment indicated that the sensory-motor system modifies the ongoing movement with a continually updated internal estimate of the environment (Todorov, 1998) . In this paper, we propose a reinforcement learning framework for motor control. With via-point representation, this framework acquires feed-forward control, on-line planning, and environmental adaptation abilities like those of biological systems. This framework contains two levels of hierarchical architecture. The higher level, which is implemented using via-point representation, corresponds to macro-actions or multiple time scales. The lower level, which is implemented using a trajectory generator, corresponds to primitive actions. In Section 2, we present our new method in detail. In Section 3, we describe computer simulation that uses our method. In this paper, we demonstrated that a hierarchical reinforcement learning framework containing via-point representation is effective for learning motor control. Since the feedback gain of biological systems is low and delay is large, high-speed movement must be controlled with feedforward control. In the framework proposed in this paper, feed-forward-like control was acquired by introducing the via-point representation. In the conventional TD-learner, we must choose the best control interval to achieve good performance. When a parameter of the environment (e.g. the mass or length of the cart) is changed, we must readjust the control interval. In the via-point framework, the best control intervals are automatically obtained accordingly, when a parameter of the environment is changed, the system is able to adapt to a new environment automatically.. According to the recent results of a psychological experiment, when we move the hand through a small set of sequential targets, the sensory-motor system modifies the ongoing movement with a continually updated internal estimate of the environment (Todorov, 1998) . The on-line planning of via-points and trajectory in our model is in accordance with the results of this psychological experiment.. When humans make goal-directed arm movements, we show highly stereotypical trajectories, even though there are an infinite number of possible trajectories. Based on optimization principles for trajectory planning, several models have been proposed, such as the minimum-jerk (Flash & Hogan, 1985) , the minimum-torque-change (Uno et al., 1989a) , the minimum-muscle-tension-change (Uno et al., 1989b) , and the minimum-motor command-change models (Kawato, 1992) . Recently, Harris & Wolpert (1998) have proposed a minimum variance model to generate an optimal trajectory. They present a unifying theory of eye and arm movements based on the single physiological assumption that the neural control signals are corrupted by noise whose variance increases with size of the motor command amplitude. In the presence of such signaldependent noise, the shape of a trajectory is selected to minimize the variance of the final eye or arm position. For the nonlinear two-jointed planar arm, they derived optimal trajectories by parameterizing them by cubic splines and by adjusting the knot locations using the simplex algorithm. However, when we consider a biologically plausible way of obtaining the minimum-variance trajectory, the reinforcement learning framework seems to be suitable. Although we used the minimum-jerk trajectory as an optimal trajectory in this paper, we will integrate such a trajectory planning mechanism into our model in the near future.. Computational Neuroscience Laboratories for their useful discussions.. Appendix A. Minimum jerk trajectory Fig. A1 shows an optimal trajectory passing through the start, via-, and end points. Let ðx s ; _ x s ; € x s be position, velocity, and acceleration at the start point where x [ R n : Similarly, we denote ðx v ; _ x v ; € x v and ðx f ; _ x f ; € x f as via-and end points. t v and t f ; respectively, are the time intervals between the start and via-points and between the via-and end points. Here, we use the minimum jerk trajectory (fifthorder polynomial) as the optimal trajectory. The minimumjerk trajectory can be obtained by using a recurrent neural network when the velocity and acceleration at the end point are equal to 0 (Hoff & Arbib, 1993) . We can extend their model to generate the minimum-jerk trajectory under general conditions so that the velocity and acceleration at the end points are not equal to 0 (Wada & Kawato, 1993) . If the position, velocity, and acceleration at these points are all fixed, then we can calculate the minimum jerk trajectory. Even if the velocity and acceleration at a via-point are not fixed, we can calculate _ x v and € x v Consequently, we can calculate the minimum jerk trajectory passing through the start, via-and end point, if the following variables are given: ðx s ; _ x s ; € x s Þ; x v ; ðx f ; _ x f ; € x f Þ; t v ; t f : In this paper, we set the velocity and acceleration at the end point to 0 for simplicity. As described in Section 2.2, the first half of the trajectory is used for control. Fig . 5 shows an example of the learned behavior of the system (the 6991st to the 7000th trials are superimposed). Fig. 5a and b show the time course of x and u; respectively. In Fig. 5a , the circles ðWÞ and xs ð£Þ show the via-points and tentative via-points, respectively (the positions of W and £ are almost the same). In the figure, the perturbation was not applied to the 7000th trial (shown as a thick line). The 6991st to 6999th trials are also shown but are indistinguishable because the lines overlap each other. After a preparatory swing to build up the inertial energy, the system succeeded in swinging up the pendulum and maintaining its balance. At the first period of the movement, the via-points are sparsely located in time. In this period, the system is controlled in a feed-forward manner. After the pole swings up, the via-points are densely located in time, since the system needs fine feedback to keep the balance.. The system generates such via-point locations in the following way. In the initial part of the movement, in climbing up the slope of the value function, the system can get a larger reward with a small number of steps by taking a long interval of via-points. When the system tries to keep the balance, the via-points have to be densely located in time. If the via-points are sparsely located, the state will quickly fall down from the peak of the value function. Fig. 6 shows the relationship between the holding time at the inverted position and the trial number. As a measure of the swing up performance, we defined the time in which the pole stayed up ðcosðuÞ . cosðp=4ÞÞ as t up : Fig. 6a shows the performance averaged over 10 runs. Fig. 6b shows the individual performances of 10 runs. In Fig. 6b , each line was made by averaging every 100 trials over a single run. We can see that the system shows a good learning performance.

O O O B O O O O O O O O O O O B I O O O O O O O O O B O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O B O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Taxonomy-based

partitioning

of

the

Gene

Ontology

The

Gene

Ontology

(GO)

project

is

a

collaborative

effort

to

construct

ontologies

which

facilitate

biologically

meaningful

annotation

of

gene

products.

In

some

situations,

only

a

generic

or

a

species-specific

subset

of

all

GO

terms

is

required

to

annotate

and

analyze

the

results

of

a

particular

biomedical

experiment.

We

show

that

by

defining

explicit

links

between

terms

in

the

GO

and

terms

in

the

Taxonomy

of

Species

(TS)

it

is

possible

to

automatically

create

partitions

of

the

GO

according

to

various

taxonomic

criteria.Our

framework

is

based

on

three

logically

defined

relations-validity,

specificity,

and

relevance-used

to

link

terms

in

the

Gene

Ontology

with

terms

in

the

Taxonomy.

The

major

advantages

of

this

approach,

as

compared

to

the

traditional

GO

slims

methodology,

are:

unambiguous

semantics

of

GO-TS

annotations,

significant

reduction

of

the

effort

needed

to

manually

select

GO

terms

appropriate

for

a

particular

taxonomic

context,

ability

to

generate

views

of

the

GO

even

for

taxa

for

which

no

explicit

links

with

GO

terms

exist,

logical

consistency

of

such

views,

and

automated

updates

of

TS-dependent

GO

subsets.

Incorporation

of

the

proposed

framework

into

the

GO

may

improve

the

usability

of

the

ontology

for

those

scientists

who

focus

their

research

on

a

particular

species

or

a

specific

class

of

organisms.

During

the

past

decade

there

has

been

a

rapid

growth

of

interest

in

ontological

engineering,

i.e.,

in

designing,

implementing,

and

deploying

structured

representations

of

various

real

world

domains

One

of

the

most

visible

testimonies

of

this

trend

is

the

ontological

activity

in

biomedicine

and

bioinformatics,

perhaps

best

represented

by

the

Open

Biomedical

Ontologies

(OBO)

project

and

its

successor

OBO

Foundry

which

is

supported

by

the

National

Center

for

Biomedical

Ontology

(NCBO;

and

is

a

central

part

of

its

BioPortal.

The

Gene

Ontology

(GO;

which

served

as

the

initial

kernel

of

OBO

and

successfully

continues

to

be

its

driving

force,

is

the

result

of

an

effort

aimed

at

providing

a

structured,

precise,

shared

vocabulary

for

describing

roles

of

genes

and

gene

products

in

any

organism

Ideally,

ontologies

should

be

built

to

enable

automated

agents

to

communicate

and

process

information

as

if

they

had

the

understanding

of

the

domain

that

human

experts

possess.

On

the

other

hand,

ontologies

are

valuable

sources

of

explicit,

systematized

knowledge

for

human

users,

especially

those

who

are

not

experts

in

the

domain

represented

by

a

particular

ontology

Automated

agents

use

an

ontology

to

retrieve

relevant

pieces

of

information

annotated

with

its

terms,

but

it

is

human

users

that

must

understand

the

ontology

to

provide

such

annotations.

(Even

if

annotations

are

suggested

by

automated

services,

such

as

text

mining

of

scientific

literature,

it

is

still

human

experts

that

must

curate-accept,

modify,

or

reject-those

suggestions.)

This

dual

role

of

ontologies

is

reflected

in,

e.g.,

the

principle

of

intelligible

definitions:

definitions

within

an

ontology

should

be

both

humanly

intelligible

and

formally

specifiable

Biomedical

ontologies

grow

considerably

in

number,

size,

and

coverage,

a

trend

clearly

illustrated

by

the

list

of

bioontologies

available

on

the

OBO

website.

It

has

long

been

recognized

that

large

knowledge

bases

need

to

be

easily

partitionable

into

subsets

tailored

for

the

convenience

of

both

a

human

and

an

automated

agent

(see,

e.g.,

On

the

one

hand,

focusing

on

a

narrow

partition

improves

the

efficiency

of

inference;

on

the

other

hand,

human

users

may

be

presented

with

a

constrained

and

tersely

expressed

portion

of

knowledge,

relevant

to

the

particular

problem

at

hand.

In

the

case

of

OBO,

partitioning

of

the

represented

biomedical

domain

is

realized

in

two

ways.

Firstly,

the

whole

of

our

biomedical

knowledge

is

divided

into

a

number

of

separate

ontologies

covering

different

subdomains,

such

as

gross

anatomy

of

Drosophila,

human

diseases,

cellular

components,

protein-protein

interactions,

etc.

This

partitioning

reflects

the

perspective

on

reality

taken

by

the

authors

of

the

ontologies.

Secondly,

terms

within

a

single

ontology

may

be

selected

or

hidden

to

provide

a

partial

view

of

the

partition

of

reality

represented

by

the

ontology.

This

subpartitioning

reflects

a

user-defined

perspective

on

reality

(its

part

captured

within

the

scope

of

the

chosen

ontology)..

The

Gene

Ontology

was

originally

designed

to

be

a

vocabulary

that

could

be

applied

to

all

eukaryotes

but

now

includes

a

large

number

of

terms

that

cover

gene

products

found

in

prokaryotes

and

viruses

as

well.

One

consequence

of

this

development

is

that

not

all

GO

terms

are

necessarily

of

interest

to

a

researcher

focused

on

organisms

of

a

particular

kind,

such

as

viruses,

flowering

plants,

or

fruit

flies;

not

all

terms

need

to

be

visible

to

every

user,

and

many

terms

will

be

irrelevant

to

the

inferences

a

particular

experiment

may

require

to

be

carried

out.

We

show

that

by

relating

terms

in

the

Gene

Ontology

to

terms

in

the

Taxonomy

of

Species

(TS)

one

is

able

to

partition

the

GO

according

to

various

criteria,

and

answer

taxon-specific

queries

such

as

''Which

metabolic

processes

are

specific

to

vertebrates,

but

are

not

found

in

birds?'',

for

example.

The

proposed

framework

is

in

line

with

the

efforts

of

the

OBO

community

aimed

not

only

at

developing

and

maintaining

biomedical

ontologies,

but

also

at

their

full

semantic

integration.

The

NCBO's

BioPortal

provides

access

to

a

large

collection

of

ontologies

spanning

many

species,

and

an

extension

of

the

framework

to

cover

ontologies

other

that

the

GO

is

also

plausible.

In

this

article,

we

argue

that

GO

slims,

the

current

approach

to

building

constrained

views

of

the

GO,

may

be

inconvenient

as

a

means

of

subsetting

the

GO

based

on

its

relation

to

the

Taxonomy

of

Species.

As

an

alternative,

we

propose

a

solution

that

allows

one

to

link

GO

terms

with

taxa

in

an

unambiguous

way,

reduce

the

effort

needed

to

manually

select

all

GO

terms

appropriate

for

a

particular

taxonomic

context,

automatically

create

views

of

the

GO

based

on

criteria

involving

taxa

that

are

not

explicitly

linked

with

GO

terms,

and

thus

also

to

reduce

the

effort

needed

to

maintain

a

consistent

set

of

GO-TS

term-term

links..

In

the

Gene

Ontology,

there

are

a

number

of

terms

which

represent

features

that

have

not

been

found

in

organisms

of

some

taxa

(including

features

that

by

definition

cannot

be

found

in

some

organisms).

These

terms

are

an

essential

part

of

the

GO:

since

the

intention

of

its

inventors

was,

among

others,

to

provide

a

vocabulary

for

cross-annotation

of

various

molecular

biology

databases,

taxon-specific

terms

are

unavoidable.

The

yearly

database

issue

of

(Nucleic

Acids

Research

and

other

articles

in

that

volume)

illustrates

well

the

difference

in

taxonomic

All

symbols

as

in

that

figure.

GO

terms

for

which

no

relations

are

asserted

or

inferred

are

omitted

here.

Table

Relations

between

three

GO

terms

(rows)

and

three

TS

terms

(columns);

GO

identifiers

are

omitted

coverage

among

various

databases.

However,

we

argue

that

if

species-specific

terms

are

to

be

included

in

the

GO

all

of

the

issues

mentioned

above

should

be

properly

addressed.

We

believe

that

the

framework

proposed

here

is

a

step

towards

an

efficient

solution.

In

a

related

study,

Schlicker

et

al.

show

that

queries

such

as

''Which

biological

processes

are

present

in

Saccharomyces

cerevisiae

but

not

in

human?''

can

be

answered

according

to

measures

of

semantic

similarity

between

GO

terms.

However,

this

data-driven

approach

is

based

on

existing

annotations,

and

may

thus

miss

some

of

the

knowledge

which

domain

experts

already

have,

but

which

is

not

explicitly

accounted

for

in

the

annotation

database.

The

framework

proposed

here

has

been

informally

discussed

with

some

members

of

the

GO

community

(including

scientific

curators),

to

the

conclusion

that

the

problem

needs

to

be

addressed

and

that

our

solution

may

indeed

be

a

plausible

candidate.

The

solution

is

not

intended

as

a

replacement,

but

rather

as

a

partially

overlapping

alternative

to

GO

slims.

The

two

technologies

are

compatible:

of

them

excludes

the

other,

and

custom

GO

slims

may

of

course

be

defined

on

a

taxonomically

enhanced

version

of

the

Gene

Ontology.

In

this

section,

we

elaborate

on

a

framework

that,

we

believe,

allows

for

a

substantial

improvement

in

performance,

consistency,

and

flexibility

of

creating

taxon-based

partitions

of

the

GO,

as

compared

to

what

can

be

obtained

using

the

traditional

GO

slims.

The

framework

is

not

intended

to

be

a

replacement

of

GO

slims,

however,

as

we

explore

only

the

very

specific

problem

of

selecting

terms

based

on

their

relations

with

taxa,

while

GO

slims

may

be

created

according

to

criteria

of

any

other

sort.

In

this

article,

we

focus

on

how

species-dependence

can

be

defined

and

used

to

automatically

partition

terms

within

a

biomedical

ontology.

We

first

analyze

(in

Section

how

suitable

for

this

task

are

the

so-called

GO

slimssubsets

of

GO

slims

intended

to

provide

user-defined

selections

of

terms

from

the

three

branches

of

the

Gene

Ontology,

the

molecular

function,

biological

process,

and

cellular

component

ontologies.

In

Section

we

observe

that

by

explicitly

linking

GO

terms

with

taxonomic

terms

from

the

Taxonomy

of

Species,

one

is

able

to

take

advantage

of

the

hierarchical

structure

of

both

the

Gene

Ontology

and

the

Taxonomy.

Following

this

observation,

we

present,

in

the

form

of

precise

definitions,

a

solution

that

allows

a

user

to

form

partitions

of

the

GO

in

a

dynamic

fashion,

by

specifying

and

combining

various

criteria

on

term-taxon

relations.

It

should

be

noted

that

the

framework

is

not

intended

to

address

all

of

the

problems

discussed

in

Section

In

particular,

it

allows

one

to

define

the

specificity

of

GO

terms

on

the

basis

of

their

correspondence

to

more

or

less

inclusive

taxa,

but

it

does

not

address

the

issue

of

specificity

of

GO

terms

in

general..

Section

philosophical,

terminological,

and

practical

issues

related

to

the

presented

framework..

Our

analysis

of

the

structure

and

content

of

the

Gene

Ontology

and

GO

slims

is

based

on

data

obtained

from

the

Gene

Ontology

downloads

site

Where

taxonomical

information

is

involved,

our

reference

is

data

obtained

from

the

NCBI

Taxonomy

downloads

site

gov/pub/taxonomy).

Both

databases

were

most

recently

accessed

in

January

both

databases

are

also

available

for

online

browsing.

O B O O B I O B I O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O


Universal psychometrics: Measuring cognitive abilities in the machine kingdom We present and develop the notion of 'universal psychometrics' as a subject of study, and eventually a discipline, that focusses on the measurement of cognitive abilities for the machine kingdom, which comprises any (cognitive) system, individual or collective, either artificial, biological or hybrid. Universal psychometrics can be built, of course, upon the experience, techniques and methodologies from (human) psychometrics, comparative cognition and related areas. Conversely, the perspective and techniques which are being developed in the area of machine intelligence measurement using (algorithmic) information theory can be of much broader applicability and implication outside artificial intelligence. This general approach to universal psychometrics spurs the re-understanding of most (if not all) of the big issues about the measurement of cognitive abilities, and creates a new foundation for (re)defining and mathematically formalising the concept of cognitive task, evaluable subject, interface, task choice, difficulty, agent response curves, etc. We introduce the notion of a universal cognitive test and discuss whether (and when) it may be necessary for exploring the machine kingdom. On the issue of intelligence and very general abilities, we also get some results and connections with the related notions of no-free-lunch theorems and universal priors. The measurement of intelligence and other cognitive abilities is the subject of several disciplines. On one hand, psychometrics has developed valid and reliable tests for human intelligence and related abilities. On the other hand, comparative psychology (and cognition) has explored animal behaviour and the range of cognitive abilities for many different species.. While psychometrics and comparative psychology usually work independently, there are areas and approaches where their connection becomes more explictly, such as evolutionary psychology, the analysis of the personality of animals (Gosling, 2001 ) and the studies of comparative cognition on apes (Herrmann et al., 2007 (Herrmann et al., , 2010 . As a result, there is nowadays a less anthropocentric view of human and animal abilities. Nonetheless, when we widen the scope from humans to a range of species, some fundamental questions about the nature of intelligence become more difficult: what intelligence really is, exactly what cognitive tests measure, which factors there are and whether the relations among cognitive abilities are only valid for a few species or general for the entire animal kingdom. Despite the enormous progress in the past decades, comparative cognition, psychometrics and related disciplines are still very far from drawing a comprehensive and accurate chart of the set of cognitive abilities in the animal kingdom (including humans) and the degree to which each is present in each species (and in each stage of development).. If this is already a vast sea for exploration, there is an uncharted ocean of cosmic dimension: the set of all possible machines, with any conceivable (computable) behaviour. We call this set the machine kingdom. Any computable behaviour is possible -at least in theory-, including the emulation of any animal (an animat, Wilson, 1991; Webb, 2009) . While the diversity and richness of behaviour is still much higher in natural systems than state-of-theart artificial systems, machines can be designed (or conceived of) without any constraint related to evolution niches and species survival. Actually, computer systems are becoming more and more complex and unpredictable (Bullock & Cliff, 2004) . This is not different for robots and other cognitive systems in artificial intelligence (AI), as some properties may emerge as the result of a learning process in an environment or as the outcome of some kind of artificial evolution. This suggests that there is an increasing need of tools for evaluating the behaviour (e.g., the abilities) of all these artefacts.. Given this need, do we have a well-founded and functional set of cognitive tests for bots, robots, artificial agents, avatars, 'animats', etc.? There are certainly many anthropocentric approaches, such as the the Turing Test and variants (Turing, 1950; Oppy & Dowe, 2011) , the more recent CAPTCHAs (von Ahn et al., 2004) and the mere use of human IQ tests for machines (Sanghi & Dowe, 2003; Detterman, 2011; Dowe & Hernández-Orallo, 2012 ). We will review some of these works in subsequent sections and discuss their limitations. There is also an incipient line of research where algorithmic information theory plays a crucial role, as we will see. Despite this more mathematical approach, the evaluation of artificial cognitive systems is still much more immature than the evaluation of natural cognitive systems in psychometrics, comparative psychology and ethology.. In this paper we claim that the statement of a more general problem, the measurement of cognitive abilities for any kind of individual or collective -either artificial, biological or hybrid-may be advantageous over the particular problems dealt with separately by human psychometrics, animal evaluation and AI metrics. Thus, we propose here the notion of 'universal psychometrics' as a subject of study, and eventually a discipline, that would focus on the measurement of any interactive (cognitive) system. In this spirit, and with the general view of the machine kingdom, this paper re-explores and, in most cases, mathematically formalises the concept of cognitive ability, evaluable subject, task choice, interface, difficulty, factorisation, cognitive test, administration, scales, etc. One of the key concepts is the definition of cognitive task class, as a weighting (more formally a distribution or probability measure) of tasks. This is useful to see what a cognitive ability can and, most especially, cannot be, by using some results about task distributions, the no-free-lunch theorems and the universal distributions.. The paper is organised as follows. Section 2 describes how the area of analysis has been enlarged from human evaluation to animal evaluation, with a natural further generalisation in the realm of machines, known as the machine kingdom. We define what an evaluable subject in the machine kingdom is and we give a definition of universal psychometrics. Section 3 overviews the different approaches taken in human psychometrics, animal evaluation in comparative cognition and machine evaluation. Section 4 focusses on what is being measured, properly defining the notion of cognitive task and interface between a subject and a task. Here we also develop the notion of task class and distribution, provide some examples and deal with the notion of task difficulty. Section 5 discusses how expected performance can be approximated for a task class, discusses some other possible indicators and shows decompositions and curves based on difficulty. Section 6 focusses on the exploration of the machine kingdom, introducing a general adaptive test, the use of populations or reference machines for scaling, and the notion of universal test. Section 7 investigates how the space of abilities can be explored around the concepts of validity and the analysis of relations and hierarchies among abilities. It touches upon the notion of general ability and intelligence, and its connections with the so-called no-free-lunch theorems and related results. Section 8 closes the paper with a discussion about the possible objections to universal psychometrics, its possibilities and applicability, and the main contributions of this work. 'Universal psychometrics', as presented so far, opens many new questions and may be subject to many objections. The most important objection is about its feasibility. We do not expect a conclusive answer to this objection, as this will be, in any case, a progressive endeavour, where better and more general theories and tools will be eventually introduced. In this paper, we have just outlined some fundamental elements and properties, which may of course be questioned and revised in the future. Nonetheless, the construction of tests for the machine kingdom cannot be eluded. The following decades will increasingly demand cognitive tests for new cognitive systems, including machines and hybrids. Once (and if) the technological singularity (Solomonoff, 1962; Good, 1965; Solomonoff, 1985) is achieved, this will be a necessity (Hernández-Orallo & Dowe, 2010 .. Of course we can have ethical objections. Some could say that it is too pretentious to put humans, non-human animals, machines and hybrids (and communities) thereof on the same scale, and measuring them with the same ruler. We cannot disagree more. Measuring things with the same rulers is one of the principles in science. That does not mean that it has to be one ruler but rather a set of (possibly related) rulers for many abilities. The fact that the 'scala naturae' scheme (see section 3.2) has been abandoned in evolution and in the study of cognition does not mean that we cannot have different scales for many abilities. What we argue here is that the ruler for each ability must be the same for all. In fact, focussing on intelligence alone may give a poor characterisation of the agents we want to evaluate, since it is the range of abilities and the results an agent obtains in each of them which characterises the agent.. We have clearly defined the machine kingdom as the set of all resource-bounded computable machines. There is an exceptional opportunity whenever a realm or field of study is broadened. The good thing about universal psychometrics -and the ruling principle for the discipline-is that its tests are falsifiable (in Popper's sense) by experimentation. If a test is assumed to measure an ability and we find (or construct) a subject for which the test repeatedly evaluates the ability in a wrong way, then we have falsified the test. This principle has been used regularly in psychometrics, but the scope of the machine kingdom (including virtually any possible system) makes falsifiability more powerful. For instance, for humans, it has been shown that several abilities correlate, but we do not know whether these abilities correlate theoretically or whether it is just because they happen frequently together. Evaluating these abilities on machines (which is far easier than on other animals) may help us to clarify some of these questions.. Nonetheless, it is important to state that we are not proposing here that classical and functional IQ tests should be replaced by more general "universal" tests when the goal is evaluating humans or better understanding human intelligence. Most probably, universal tests will be less effective, since they assume fewer things about the examinee. Evaluating humans and animals has many specific issues which are difficult to address with such a general view originating from universal psychometrics, such as the notion of "species", the evolutionary background, the genetic load, the appraisal of intelligence development, etc. The view of "universal psychometrics" as a superdiscipline should be better substituted by a view of "universal psychometrics" as an overlapping or transversal discipline which would allow the exchange of theories, techniques and methodologies among some of the previously isolated disciplines. However, universal psychometrics should be mainly constructed from formal, computational, principles. In this sense, this work has presented the foundations of a general approach for universal psychometrics, with the space of evaluable systems (the machine kingdom) and the space of tasks as the field of study. We have revisited and formally redefined the notions of (expected) performance, difficulty, interface, test, etc., and we have discussed some key issues around them, as well as some results showing possible directions -and some dead-ends, too.. The general perspective of universal psychometrics enlarges both the theoretical and the experimental possibilities. In one direction, we can bring the experience, terminology, experimentation methodology, findings and expertise from psychometrics to the realm of machine intelligence measurement and to the field of AI in general. In particular, AI evaluation may receive an important impact from the view of a cognitive system being characterised by the diversity and degree of its cognitive abilities, the analysis of the relation between the spectrum of abilities, the distinction between traits and measuring instruments, or the borrowing of item response theory, as well as other theories and concepts developed in psychometrics. In the other direction, as already mentioned, we can derive tasks and problems from non-anthropocentric mathematical principles, and determine their difficulty theoretically. Similarly, as the areas of animal behaviour and cognition have become more and more related to computational disciplines such as artificial life, artificial evolution, swarm computation and biorobotics, there are many possibilities to be explored when we focus on cognitive system evaluation and ability measurement. The notion of animat can be crucial in this connection, as different animats need to be compared to real animals (and other animats) and the evaluation tools can benefit from comparative cognition and artificial intelligence alike. Finally, universal psychometrics can strengthen the already regular connections between psychometrics and comparative cognition, as there is a continuum in the machine kingdom between animats and androids; avatars, assistants, robots, chatterbots and robotic pets are designed and chosen because of their range of abilities and personality.. It is very difficult to tell how much time and effort it will take to get a universal intelligence test which could be applied to any machine, and, as a result of the physical Church-Turing thesis, as already discussed, be applicable to any animal, including humans, or any hybrid or collective. It is perhaps better to ask the question for some other simpler cognitive abilities, and see whether an aggregated cognitive ability called intelligence makes sense in the context of universal psychometrics. Also, in this paper we have mostly talked about cognitive abilities, but there are some other features and traits which would be useful to measure on machines, in the same way psychology and psychiatry do: empathy, aggressiveness, perseverance, etc. But this is still further away.. We can conclude referring to the main motivation already expressed by some earlier works in the 1990s about measuring cognitive systems: artificial intelligence requires an accurate, non-anthropocentric, meaningful and formal way of evaluating its progress, by evaluating its artefacts. Once we have clarified the realm of subjects we want to measure, we need to analyse the theories, techniques and methodologies which have been used in each parcel, with a special emphasis on those which might be still applicable and more useful for universal psychometrics.

O B O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B