nils-herrmann/clean_scirepeval huggingface dataset:
A Catalog of Elemental Neural Circuits 2

It is well-known that abstract neurons of the McCulloch-Pitts type compute input-output functions that are formally equivalent to suitably defined Boolean functions. Boolean functions, in turn, can be classified (up to isomorphism of underlying logical structure) into a limited number of basic types, each of which is non-isomorphic to each other type. These basic types are conveniently expressed by means of a canonic logical formula involving a minimal number of constituent operations. Each such canonic formula can be translated into an equivalent representation as a neural circuit, resulting in a catalog of basic, distinct neural circuits. Each circuit in this catalog computes a distinct Boolean function, and every Boolean function is computed by one such network, making this a kind of “periodic table” of elemental neural circuits. This brief article presents and discusses this catalog of circuits. An intriguing possibility raised by this analysis is that neural circuits in biological networks might, like the circuits in this catalog, tend to take forms that are minimally complex given the functions they compute—a kind of “principle of neural parsimony.” A Catalog of Elemental Neural Circuits 3 A Catalog of Elemental Neural Circuits The modern mathematical investigation of the properties of neural networks began with McCulloch and Pitts’s model of an “abstract neuron,” which sums binary inputs, each either excitatory or inhibitory, and fires if and only if the total excitation meets or exceeds a predefined threshold . McCulloch and Pitts showed that networks of these simple devices were of extraordinary computational power, in fact equivalent to that of a Turing Machine or modern digital computer (McCulloch & Pitts, 1943). Underlying their proof is the idea that any abstract neuron, or circuit of abstract neurons, computes a Boolean function: that is, a mapping from binary variables (the inputs on the abstract “dendrites”) to one variable (the output on the abstract “axon”). The mathematics of neural networks are thus inextricably tied up with the mathematics of Boolean functions. A similar relation exists between electrical circuit diagrams, of the type that illustrate textbooks on electrical engineering, and logical formulae, as one would find in a textbook of mathematical logic. The two representations are in a very literal sense equivalent, and, as long recognized by electrical engineers, it is convenient to investigate the properties of real electrical circuits by translating them into equivalent symbolic expressions. Similarly, in this paper we catalog neural circuits by reference to the mathematical properties of the functions they compute.
The Simplicity Principle in Human Concept Learning

How do we learn concepts and categories from examples? Part of the answer might be that we induce the simplest category consistent with a given set of example objects. This seemingly obvious idea, akin to simplicity principles in many fields, plays surprisingly little role in contemporary theories of concept learning, which are mostly based on the storage of exemplars, and avoid summarization or overt abstraction of any kind. This article reviews some evidence that complexity minimization does indeed play a central role in human concept learning. The chief finding is that subjects' ability to learn concepts depends heavily on the concepts' intrinsic complexity; more complex concepts are more difficult to learn. This pervasive effect suggests, contrary to exemplar theories, that concept learning critically involves the extraction of a simplified or abstracted generalization from examples.
Categorization Under Complexity: A Unified MDL Account of Human Learning of Regular and Irregular Categories

We present an account of human concept learning—that is, learning of categories from examples—based on the principle of minimum description length (MDL). In support of this theory, we tested a wide range of two-dimensional concept types, including both regular (simple) and highly irregular (complex) structures, and found the MDL theory to give a good account of subjects' performance. This suggests that the intrinsic complexity of a concept (that is, its description length) systematically influences its learnability.
Theoretical note A catalog of Boolean concepts

Boolean concepts are concepts whose membership is determined by a Boolean function, such as that expressed by a formula of propositional logic. Certain Boolean concepts have been much studied in the psychological literature, in particular with regard to their ease of learning. But research attention has been somewhat uneven, with a great deal of attention paid to certain concepts and little to others, in part because of the unavailability of a comprehensive catalog. This paper gives a complete classification of Boolean concepts up to congruence (isomorphism of logical form). Tables give complete details of all concepts determined by up to four Boolean variables. For each concept type, the tables give a canonic logical expression, an approximately minimal logical expression, the Boolean complexity (length of the minimal expression), the number of distinct Boolean concepts of that type, and a pictorial depiction of the concept as a set of vertices in Boolean D-space. Some psychological properties of Boolean concepts are also discussed. r 2003 Elsevier Science (USA). All rights reserved.
Bayesian contour integration

The process by which the human visual system parses an image into contours, surfaces, and objects—perceptual grouping—has proven difficult to capture in a rigorous and general theory. A natural candidate for such a theory is Bayesian probability theory, which provides optimal interpretations of data under conditions of uncertainty. But the fit of Bayesian theory to human grouping judgments has never been tested, in part because methods for expressing grouping hypotheses probabilistically have not been available. This paper presents such methods for the case ofcontour integration—that is, the aggregation of a sequence of visual items into a “virtual curve.” Two experiments are reported in which human subjects were asked to group ambiguous configurations of dots (in Experiment 1, a sequence of five dots could be judged to contain a “corner” or not; in Experiment 2, an arrangement of six dots could be judged to fall into two disjoint contours or one smooth contour). The Bayesian theory accounts extremely well for subjects’ judgments, explaining more than 75% of the variance in both tasks. The theory thus provides a far more quantitatively precise account of human contour integration than has been previously possible, allowing a very precise calculation of the subjective goodness of a virtual chain of dots. Because Bayesian theory is inferentially optimal, this finding suggests a “rational justification,” and hence possibly an evolutionary rationale, for some of the rules of perceptual grouping.
Minimization of Boolean complexity in human concept learning

One of the unsolved problems in the field of human concept learning concerns the factors that determine the subjective difficulty of concepts: why are some concepts psychologically simple and easy to learn, while others seem difficult, complex or incoherent? This question was much studied in the 1960s but was never answered, and more recent characterizations of concepts as prototypes rather than logical rules leave it unsolved. Here I investigate this question in the domain of Boolean concepts (categories defined by logical rules). A series of experiments measured the subjective difficulty of a wide range of logical varieties of concepts (41 mathematically distinct types in six families—a far wider range than has been tested previously). The data reveal a surprisingly simple empirical ‘law’: the subjective difficulty of a concept is directly proportional to its Boolean complexity (the length of the shortest logically equivalent propositional formula)—that is, to its logical incompressibility.
Perception of Animacy from the Motion of a Single Object

We demonstrate that a single moving object can create the subjective impression that it is alive, based solely on its pattern of movement. Our displays differ from conventional biological motion displays (which normally involve multiple moving points, usually integrated to suggest a human form) in that they contain only a single rigid object moving across a uniform field. We focus on motion paths in which the speed and direction of the target object change simultaneously. Naive subjects' ratings of animacy were significantly influenced by (i) the magnitude of the speed change, (ii) the angular magnitude of the direction change, (iii) the shape of the object, and (iv) the alignment between the principal axis of the object and its direction of motion. These findings are consistent with the hypothesis that observers classify as animate only those objects whose motion trajectories are otherwise unlikely to occur in the observed setting.
Bias toward regular form in mental shape spaces.

The distribution of figural "goodness" in 2 mental shape spaces, the space of triangles and the space of quadrilaterals, was examined. In Experiment 1, participants were asked to rate the typicality of visually presented triangles and quadrilaterals (perceptual task). In Experiment 2, participants were asked to draw triangles and quadrilaterals by hand (production task). The rated typicality of a particular shape and the probability that that shape was generated by participants were each plotted as a function of shape parameters, yielding estimates of the subjective distribution of shape goodness in shape space. Compared with neutral distributions of random shapes in the same shape spaces, these distributions showed a marked bias toward regular forms (equilateral triangles and squares). Such psychologically modal shapes apparently represent ideal forms that maximize the perceptual preference for regularity and symmetry.
Mapping the Mental Space of Rectangles

The cognitive structure of a shape space—the space of rectangles—is explored by a nonmetric scaling technique. Our experiment was designed to extract the major transformational paths or ‘modes’ that characterize the mental shape space. Earlier studies of rectangle similarities using multidimensional scaling have provided conflicting evidence about whether the coordinate system of the mental rectangle space is based on height and width or on area and shape (ie aspect ratio). Our study reveals shape to be the single dominant factor. We suspected that earlier evidence for a height – width parameterization might have been due to the presentation of rectangles upright in a pseudo-gravitational coordinate system (whereas our rectangles are randomly rotated). In a control experiment with upright (vertical or horizontal) rectangles, the heavy bias towards shape preservation was still the dominant mode. In addition, however, a secondary bias towards change of height or width emerged, exactly following the pattern expected from the biasing change in context. This finding established a concrete path by which context and frame can influence the way shape is represented. The relevance of these findings to the cognitive organization of more complex shape spaces is discussed.
Shape-based indexing in a medical image database

Reports on a prototype of a clinical radiograph image database to be indexed by image content. The underlying content-based search engine is based on the modal shape description method for characterizing the shape of a 2-D image region (S.E. Sclaroff, 1995). Using a similarity metric defined in a modal vector space, the authors' system successfully classified radiographic images according to the dental pathologies they depicted. This successful classification demonstrates that the proposed similarity metric effectively captures clinical similarity between images in the database. The prototype is implemented in a Web-based environment, allowing remote users in the field to search a central repository of images. Examples of classification performance and typical queries are provided.
' s personal copy What ’ s magic about magic numbers ? Chunking and data compression in short-term memory

Short term memory is famously limited in capacity to Miller’s (1956) magic number 7 ± 2— or, in many more recent studies, about 4 ± 1 ‘‘chunks’’ of information. But the definition of ‘‘chunk’’ in this context has never been clear, referring only to a set of items that are treated collectively as a single unit. We propose a new more quantitatively precise conception of chunk derived from the notion of Kolmogorov complexity and compressibility: a chunk is a unit in a maximally compressed code. We present a series of experiments in which we manipulated the compressibility of stimulus sequences by introducing sequential patterns of variable length. Our subjects’ measured digit span (raw short term memory capacity) consistently depended on the length of the pattern after compression, that is, the number of distinct sequences it contained. The true limit appears to be about 3 or 4 distinct chunks, consistent with many modern studies, but also equivalent to about 7 uncompressed items of typical compressibility, consistent with Miller’s famous magical number.
Efficient regularity-based grouping

This paper summarizes a novel logic-based approach to grouping and perceptual organization, and presents novel efficient methods for computing interpretations in this framework. Grouping interpretations are first defined as logical structures, built out of atomic premises ("regularities") that are derived from considerations of non-accidentalness. These interpretations can then be partially ordered by their degree of regularity or constraint (measured numerically by their codimension). The genericity constraint-the principle that interpretations should minimize coincidences in the observed configuration-dictates that the preferred interpretation will be the minimum in this partial order, i.e. the interpretation with maximum codimension. The preferred interpretation, called the qualitative parse, corresponds neatly to the interpretation intuitively preferred by human observers. As a side-effect, the "most salient" or most structured part of the scene can be identified, as the highest-codimension subtree of the qualitative parse. An efficient (O(n/sup 2/)) method for computing the maximum codimension interpretation is presented, along with examples.
Regularity‐based Perceptual Grouping

This paper investigates perceptual grouping from a logical point of view, defining a grouping interpretation as a particular kind of logical expression, and then developing an explicit inference theory in terms of such expressions. First, a regularity‐based interpretation language is presented, in which an observed configuration is characterized in terms of the regularities(special configurational classes, e.g. nonaccidental properties) it obeys. The most preferred interpretation in such a system is shown to be the most‐regular (maximum “codimension”) model the observed configuration obeys, which is also the unique model in which it is generic(typical). Inference then reduces to a straiightforward exercise in LogicProgramming. Because generic model assignment involves negation, this reduction requires that a version of the Closed World Assumption (CWA) be adopted.
Regularity Vs Genericity in the Perception of Collinearity

The perception of collinearity is investigated, with the focus on the minimal case of three dots. As suggested previously, from the standpoint of probabilistic inference, the observer must classify each dot triplet as having arisen either from a one-dimensional curvilinear process or from a two-dimensional patch. The normative distributions of triplets arising from these two classes are unavailable to the observer, and are in fact somewhat counterintuitive. Hence in order to classify triplets, the observer invents distributions for each of the two opposed types, ‘regular’ (collinear) triplets and ‘generic’ (ie not regular) triplets. The collinear prototype is centered at 0° (ie perfectly straight), whereas the generic prototype, contrary to the normative statistics, is centered at 120° away from straight—apparently because this is the point most distant in triplet space from straight and thus creates the maximum possible contrast between the two prototypes. By default, these two processes are assumed to be equiprobable in the environment. An experiment designed to investigate how subjects' judgments are affected by conspicuous environmental deviations from this assumption is reported. The results suggest that observers react by elevating or depressing the expected probability of the generic prototype relative to the regular one, leaving the prototype structure otherwise intact.
Priors, preferences and categorical percepts

Visual perception is the process of inferring world structure from image structure. If the world structure we recover from our images \makes sense" as a plausible world event, then we have a \percept" and can often ooer a concise linguistic description of what we see. For example, in the upper panel of Figure 3.1, if asked, \What do you see?", a typical response might be a pillbox with a handle either erect (left) or at (right). This concise and conndent response suggests that we have identiied a model type that ts the image observation with no residual ambiguities at the level of the description. In contrast, when asked to describe the two lower drawings in Figure 3.1, there is some hesitancy and uncertainty. Is the handle erect or not? Does it have a skewed or rectangular shape? The depiction leaves us somewhat uncertain, as if several options were possible, but none where all aspects of the interpretation collectively support each other. What then, leads us to the certainty in the upper set and to the ambiguity in the lower pair? To be a bit more precise about our goal, let us assume that some Waltz-like algorithm has already identiied the base of the pillbox and the wire-frame handle as separate 3D parts. Even with this decomposition, there remains an innnity of possible interpretations for any of these drawings. Yet we conndently commit to one interpretation in the case of the upper panel, but otherwise for the lower pair. Our aim, then, is to understand why the image structures in the upper panel support the assertion that they must arise only from very particular world structures, whereas the lower two structures seem more ambiguous. Our analysis will consist of three parts: rst we will lay out the domain associated with pillboxes having handles. Then the role of preferences for 80
A Biased View of Perceivers Commentary on Observer theory Bayes theory and psychophysics by B Bennett

The target chapter introduces the notion of an observer as an entity which applies a given Bayesian theory to the task of inferring world properties from sense data In fact most of this volume is concerned with observers in one form or another The basic idea is that the perceptual system uses a model of the current world context that is a domain theory to reduce the ambiguity inherent in the sense data and to select particular interpretations for the state of the world Example domain theories are the qualitative probabilistic models used for the motion of a ball in a box in Chapter the preference based scheme for the interpretation of handle shape and orientation discussed in Chapter the Bayesian models for shape from texture and shading used in Chapter and the cost function formulation proposed for the interpretation of shape illumination and pigmentation in Chapter In order to include such alternative forms of world models we use the term observer to refer loosely to any entity which applies a xed domain theory to the task of inferring world properties from sensory data Our bias is to view the perceptual system as a theorist in its own right not just an observer We claim that in order to understand perception we need to understand the way this theorist works that is how it selects a theory for a new domain how it applies a theory to choose particular interpretations for sense data how it compares di erent theories and how it revises and learns theories under the in uence of new information In contrast an observer is concerned with only one of these aspects namely how a xed theory is applied The distinction between theorist and observer is perhaps blurred by notions such as Yuille s competitive priors see Chapter or Each prior may lead to a single distinct Bayesian observer and the selection of one prior in favor of others can be thought of as a theory selection process Thus the overall system can be thought of as a theorist On
Efficient Computation of Gapped Substring Kernels on Large Alphabets

We present a sparse dynamic programming algorithm that, given two strings s and t , a gap penalty λ, and an integer p, computes the value of the gap-weighted length-p subsequences kernel. The algorithm works in time O(p |M| log |t|), where M = {(i,j) | si = tj} is the set of matches of characters in the two sequences. The algorithm is easily adapted to handle bounded length subsequences and different gap-penalty schemes, including penalizing by the total length of gaps and the number of gaps as well as incorporating character-specific match/gap penalties.The new algorithm is empirically evaluated against a full dynamic programming approach and a trie-based algorithm both on synthetic and newswire article data. Based on the experiments, the full dynamic programming approach is the fastest on short strings, and on long strings if the alphabet is small. On large alphabets, the new sparse dynamic programming algorithm is the most efficient. On medium-sized alphabets the trie-based approach is best if the maximum number of allowed gaps is strongly restricted.
Learning hierarchical multi-category text classification models

We present a kernel-based algorithm for hierarchical text classification where the documents are allowed to belong to more than one category at a time. The classification model is a variant of the Maximum Margin Markov Network framework, where the classification hierarchy is represented as a Markov tree equipped with an exponential family defined on the edges. We present an efficient optimization algorithm based on incremental conditional gradient ascent in single-example subspaces spanned by the marginal dual variables. Experiments show that the algorithm can feasibly optimize training sets of thousands of examples and classification hierarchies consisting of hundreds of nodes. The algorithm's predictive accuracy is competitive with other recently introduced hierarchical multi-category or multilabel classification learning algorithms.
Variational Inference in Stochastic Dynamic Environmental Models

Dan Cornford, Manfred Opper, John Shawe-Taylor, Ian Roulstone, Peter Clark 1 NCRG, School of Engineering and Applied Science, Aston University, Birmingham B4 7ET 2 ISIS, Computer Science, Southampton University, Southampton SO17 1BJ 3 Dept. of Maths and Statistics, School of Electronics and Physical Sciences, University of Surrey, Guildford GU2 7XH. 4 Joint Centre for Mesoscale Meteorology, Met Office, Reading RG6 6BB.
On the eigenspectrum of the gram matrix and the generalization error of kernel-PCA

In this paper, the relationships between the eigenvalues of the m/spl times/m Gram matrix K for a kernel /spl kappa/(/spl middot/,/spl middot/) corresponding to a sample x/sub 1/,...,x/sub m/ drawn from a density p(x) and the eigenvalues of the corresponding continuous eigenproblem is analyzed. The differences between the two spectra are bounded and a performance bound on kernel principal component analysis (PCA) is provided showing that good performance can be expected even in very-high-dimensional feature spaces provided the sample eigenvalues fall sufficiently quickly.