nils-herrmann/clean_scirepeval huggingface dataset:
A Simple Neural Network Models Categorical Perception of Facial Expressions

The performance of a neural network that categorizes facial expressions is compared with human subjects over a set of experiments using interpolated imagery. The experiments for both the human subjects and neural networks make use of interpolations of facial expressions from the Pictures of Facial Affect Database [Ekman and Friesen, 1976]. The only difference in materials between those used in the human subjects experiments [Young et al., 1997] and our materials are the manner in which the interpolated images are constructed – image-quality morphs versus pixel averages. Nevertheless, the neural network accurately captures the categorical nature of the human responses, showing sharp transitions in labeling of images along the interpolated sequence. Crucially for a demonstration of categorical perception [Harnad, 1987], the model shows the highest discrimination between transition images at the crossover point. The model also captures the shape of the reaction time curves of the human subjects along the sequences. Finally, the network matches human subjects’ judgements of which expressions are being mixed in the images. The main failing of the model is that there are intrusions of “neutral” responses in some transitions, which are not seen in the human subjects. We attribute this difference to the difference between the pixel average stimuli and the image quality morph stimuli. These results show that a simple neural network classifier, with no access to the biological constraints that are presumably imposed on the human emotion processor, and whose only access to the surrounding culture is the category labels placed by American subjects on the facial expressions, can nevertheless simulate fairly well the human responses to emotional expres-
How Can I Know What You Think ? : Assessing Representational Similarity in Neural Systems

How do my mental states compare to yours? We suggest that, while we may not be able to compare experiences, we can compare neural representations, and that the correct wa y to compare neural representations is through analysis of th e distances between them. In this paper, we present a technique for measuring the similarities between representati ons at various layers of neural networks. We then use the measure to demonstrate empirically that different artificial neura l networks trained by backpropagation on the same categorizatio n task, even with different representational encodings of th e input patterns and different numbers of hidden units, reach st ate in which representations at the hidden units are similar.
A Connectionist Simulation of the Empirical Acquisition of Grammatical Relations

This paper proposes an account of the acquisition of grammatical relations using the basic concepts of connectionism and a construction-based theory of grammar. Many previous accounts of first-language acquisition assume that grammatical relations (e.g., the grammatical subject and object of a sentence) and linking rules are universal and innate; this is necessary to provide a first set of assumptions in the target language to allow deductive processes to test hypotheses and/or set parameters.
Assessing the contribution of representation to results

In this paper, we make a methodological point concerning the contribution of the representation of the output of a neural network model when using the model to compare to human error performance. We replicate part of Dell, Juliano & Govindjee’s work on modeling speech errors using recurrent networks (Dell et al., 1993). We find that 1) the error patterns reported by Dell et al. do not appear to remain when more networks are used; and 2) some components of the error patterns that are found can be accounted for by simply adding Gaussian noise to the output representation they used. We suggest tha t when modeling error behavior, the technique of adding noise to the output representation of a network should be used as a control to assess to what degree errors may be attributed to t he underlying network.
Extreme Attraction: On the Discrete Representation Preference of Attractor Networks

Attractor networks play an important role in many connectionist cognitive models. The dynamic behavior of these recurrent networks, which typically involves the convergence of activation levels to stable fixed-points over time, provides a means for modeling the time course of cognitive processes. Networks of this kind are often used as associative memories and as resolvers of soft constraint satisfaction problems. Early attractor networks used step activation functions, allowing only two activation values for each processing element. Contemporary networks, however, often use sigmoidal activation functions. Continuous outputs allow for continuous target patterns, and these have at least two apparent advantages: (1) they allow for richer representational schemes, such as mapping activations to probabilities, and (2) they allow for a larger number of attractors. Attractor network models, like those for lexical semantics (Clouse and Cottrell, 1995) and consciousness (Mathis and Mozer, 1995), would seem to benefit from continuous patterns, and other models using continuous vectors, like the CHARM memory model (Metcalfe Eich, 1982), would benefit from attractor dynamics. In this work we argue that these apparent advantages of continuous targets are illusory, at least when attractors are being learned using standard methods. Our simulation results indicate that attractor networks, even those with continuous activation functions, are best suited for use with target vectors consisting of polarized discrete elements.
Extreme Attraction: the Beneets of Corner Attractors

Connectionist attractor networks have played a central role in many cognitive models involving associative memory and soft constraint satisfaction. While early attractor networks used step activation functions , permitting the construction of attractors for only binary (or bipolar) patterns, much recent work has focused on networks with continuous sigmoidal activation functions. The incorporation of sig-moidal processing elements allows for the use of expressive real vector representations in attractor networks. The empirical studies reported here, however, reveal that the learning performance of sigmoidal at-tractor networks is best when such general real vectors are avoided | when training patterns are explicitly placed in the extreme corners of the network's activation space. Using binary (or bipolar) patterns produces beneets in the number of attractors learnable by a network, in the accuracy of the learned attractors, and in the amount of training required. These beneets persist under conditions of sparse patterns. Furthermore, these experiments show that the advantages of extreme-valued patterns are not solely eeects of the large separation between training patterns aaorded by corner attractors.
User Lenses { Achieving 100 % Precisionon Frequently

The concept of a \user lens" is introduced. The lens is a sequence of linear transformations used to reweight document or query vectors , and is trained automatically via relevance data provided by the user. Experiments verify that in the adhoc task the lens can improve performance on training data while not degrading test data performance, and that larger lenses result in nearly perfect performance on the training set. The lens provides a mechanism for automatically capturing long-term, user-speciic information about an improved weighting scheme.
Task and Spatial Frequency Effects on Face Specialization

There is strong evidence that face processing is localized in the brain. The double dissociation between prosopagnosia, a face recognition deficit occurring after brain damage, and visual object agnosia, difficulty recognizing other kinds of complex objects, indicates that face and nonface object recognition may be served by partially independent mechanisms in the brain. Is neural specialization innate or learned? We suggest that this specialization could be tbe result of a competitive learning mechanism that, during development, devotes neural resources to the tasks they are best at performing. Furtber, we suggest that the specialization arises as an interaction between task requirements and developmental constraints. In this paper, we present a feed-forward computational model of visual processing, in which two modules compete to classify input stimuli. When one module receives low spatial frequency information and the other receives high spatial frequency information, and the task is to identify the faces while simply classifying the objects, the low frequency network shows a strong specialization for faces. No other combination of tasks and inputs shows this strong specialization. We take these results as support for the idea that an innately-specified face processing module is unnecessary.
Serial Order in Reading Aloud: Connectionist Models and Neighborhood Structure

Dual-Route and Connectionist Single-Route models of reading have been at odds over claims as to the correct explanation of the reading process. Recent Dual-Route models predict that subjects should show an increased naming latency for irregular words when the irregularity is earlier in the word (e.g. chef is slower than glow) - a prediction that has been confirmed in human experiments. Since this would appear to be an effect of the left-to-right reading process, Coltheart & Rastle (1994) claim that Single-Route parallel connectionist models cannot account for it. A refutation of this claim is presented here, consisting of network models which do show the interaction, along with orthographic neighborhood statistics that explain the effect.
Time-delay neural networks: representation and induction of finite-state machines

In this work, we characterize and contrast the capabilities of the general class of time-delay neural networks (TDNNs) with input delay neural networks (IDNNs), the subclass of TDNNs with delays limited to the inputs. Each class of networks is capable of representing the same set of languages, those embodied by the definite memory machines (DMMs), a subclass of finite-state machines. We demonstrate the close affinity between TDNNs and DMM languages by learning a very large DMM (2048 states) using only a few training examples. Even though both architectures are capable of representing the same class of languages, they have distinguishable learning biases. Intuition suggests that general TDNNs which include delays in hidden layers should perform well, compared to IDNNs, on problems in which the output can be expressed as a function on narrow input windows which repeat in time. On the other hand, these general TDNNs should perform poorly when the input windows are wide, or there is little repetition. We confirm these hypotheses via a set of simulations and statistical analysis.
HUMOUR: Degenerative Grammar: The Story of Outa

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.
Modeling Interference Effects In Instructed Category Learning

Category learning is often seen as a process of inductive generalization from a set of class-labeled exemplars. Human learners, however, often receive direct instruction concerning the structure of a category before being presented with examples. Such explicit knowledge may often be smoothly integrated with knowledge garnered by exposure to instances, but some interference effects have been observed. Specifically, errors in instructed rule following may sometimes arise after the repeated presentation of correctly labeled exemplars. Despite perfect consistency between instance labels and the provided rule, such inductive training can drive categorization behavior away from rule following and towards a more prototype-based or instance-based pattern. In this paper we present a general connectionistmodel of instructed category learning which captures this kind of interference effect. We model instruction as a sequence of inputs to a network which transforms such advice into a modulating force on classification behavior. Exemplarbased learning is modeled in the usual way: as weight modification via backpropagation. The proposed architecture allows these two sources of information to interact in a psychologically plausible manner. Simulation results are provided on a simple instructed category learning task, and these results are compared with human performance on the same task.
Categorical Perception in Facial Emotion Classification

We present an automated emotion recognition system that is capable of identifying six basic emotions (happy, surprise, sad, angry, fear, disgust) in novel face images. An ensemble of simple feed-forward neural networks are used to rate each of the images. The outputs of these networks are then combined to generate a score for each emotion. The networks were trained on a database of face images that human subjects consistently rated as portraying a single emotion. Such a system achieves 86% generalization on novel face images (individuals the networks were not trained on) drawn from the same database. The neural network model exhibits categorical perception between some emotion pairs. A linear sequenceof morph images is created between two expressions of an individual’s face and this sequence is analyzed by the model. Sharp transitions in the output response vector occur in a single step in the sequence for some emotion pairs and not for others. We plan to us the model’s response to limit and direct testing in determining if human subjects exhibit categorical perception in morph image sequences.
Representation and Induction of Finite State Machines using Time-Delay Neural Networks

This work investigates the representational and inductive capabilities of time-delay neural networks (TDNNs) in general, and of two subclasses of TDNN, those with delays only on the inputs (IDNN), and those which include delays on hidden units (HDNN). Both architectures are capable of representing the same class of languages, the definite memory machine (DMM) languages, but the delays on the hidden units in the HDNN helps it outperform the IDNN on problems composed of repeated features over short time windows.
Selective attention in the acquisition of the past tense

It is well known that children generally exhibit a “Ushaped” pattern of development in the process of acquiring the past tense. Plunkett & Marchman (1991) showed that a connectionist network, trained on the past tense, would exhibit U-shaped learning effects. This network did not completely master the past tense mapping, however. Plunkett & Marchman (1993) showed that a network trained with an incrementally expanded training set was able to achieve acceptable levels of mastery, as well as show the desired U-shaped pattern. In this paper, we point out some problems with using an incrementally expanded training set. We propose a model of selective attention that enables our network to completely master the past tense mapping and exhibit U-shaped learning effects without requiring external manipulation of its training set.
Representing Face Images for Emotion Classification

We compare the generalization performance of three distinct representation schemes for facial emotions using a single classification strategy (neural network). The face images presented to the classifiers are represented as: full face projections of the dataset onto their eigenvectors (eigenfaces); a similar projection constrained to eye and mouth areas (eigenfeatures); and finally a projection of the eye and mouth areas onto the eigenvectors obtained from 32×32 random image patches from the dataset. The latter system achieves 86% generalization on novel face images (individuals the networks were not trained on) drawn from a database in which human subjects consistently identify a single emotion for the face.
Learning to Retrieve Information

Information retrieval diiers signiicantly from function approximation in that the goal is for the system to achieve the same ranking function of documents relative to queries as the user: the outputs of the system relative to one another must be in the proper order. We hypothesize that a particular rank-order statistic, Guttman's point alienation, is the proper objective function for such a system, and demonstrate its eecacy by using it to nd the optimal combination of retrieval experts. In application to a commercial retrieval system, the combination performs 47% better than any single expert.
Towards Instructable Connectionist Systems

At least three disparate channels have been used to install new knowledge into artificial intelligence systems. The first of these is the programmer channel, through which the knowledge in the system is simply edited to include the desired new knowledge. While this method is often effective, it may not be as efficient as learning directly from environmental interaction. The second channel may be called the linguistic channel, through which knowledge is added by explicitly telling the system facts or commands encoded as strings of quasi-linguistic instructions in some appropriate form. Finally, there is, for want of a better phrase, the learning channel, through which the system learns new knowledge in an inductive way via environmental observations and simple feedback information. These latter two channels are the ones upon which we wish to focus, as they are the hallmarks of instructable systems. Most instructable systems depend upon, or at least heavily favor, one of these two channels for the bulk of their knowledge acquisition. Specifically, symbolic artificial intelligence systems have generally depended upon the explicit use of sentential logical expressions, rules, or productions for the transmission of new knowledge to the system. In contrast, many connectionist network models have relied solely on inductive generalization mechanisms for knowledge creation. There is no apparent reason to believe that this rough dichotomy of technique is necessary, however. Systems which are capable of receiving detailed instruction and also generalizing from experience the both possible and potentially very useful.
Optimization of Entropy with Neural Networks

OF THE DISSERTATION Optimization of Entropy with Neural Networks by Nicol Norbert Schraudolph Doctor of Philosophy in Cognitive Science and Computer Science University of California, San Diego, 1995 Professor Terrence J. Sejnowski, Chair The goal of unsupervised learning algorithms is to discover concise yet informative representations of large data sets; the minimum description length principle and exploratory projection pursuit are two representative attempts to formalize this notion. When implementedwith neural networks, both suggest the minimization of entropy at the network’s output as an objective for unsupervised learning. The empirical computation of entropy or its derivative with respect to parameters of a neural network unfortunately requires explicit knowledge of the local data density; this information is typically not available when learning from data samples. This dissertation discusses and applies three methods for making density information accessible in a neural network: parametric modelling, probabilistic networks, and nonparametric estimation. By imposing their own structure on the data, parametric density models implement impoverished but tractable forms of entropy such as the log-variance. We have used this method to improve the adaptive dynamics of an anti-Hebbian learning rule which has proven successful in extracting disparity from random stereograms.
A Connectionist Model Of Instruction Following

In this paper we describe a general connectionist model of “learning by being told”. Unlike common network models of inductive learning which rely on the slow modification of connection weights, our model of instructed learning focuses on rapid changes in the activation state of a recurrent network. We view stable distributed patterns of activation in such a network as internal representations of provided advice – representations which can modulate the behavior of other networks. We suggest that the stability of these configurations of activation can arise over the course of learning an instructional language and that these stable patterns should appear as articulated attractors in the activation space of the recurrent network. In addition to proposing this general model, we also report on the results of two computational experiments. In the first, networks are taught to respond appropriately to direct instruction concerning a simple mapping task. In the second,networks receive instructions describing procedures for binary arithmetic, and they are trained to immediately implement the specified algorithms on pairs of binary numbers. While the networks in these preliminary experiments were not designed to embody the attractor dynamics inherent in our general model, they provide support for this approach by demonstrating the ability of recurrent backpropagation networks to learn an instructional language in the service of some task and thereafter exhibit prompt instruction
Representing documents using an explicit model of their similarities

A method is proposed for creating vector space representations of documents based on modeling target interdocument similarity values. The target similarity values are assumed to capture semantic relationships, or associations, between the documents. The vector representations are chosen so that the inner product similarities between document vector pairs closely match their target interdocument similarities. The method is closely related to the Latent Semantic Indexing approach; in fact, they are equivalent when the target similarities are derived directly from document similarities based on term co‐occurrence. However, our method allows for external sources of interdocument semantic constraints to be used in the indexing, though at greater computational expense. The method is applied to three standard text databases from the information retrieval literature. On the CISI database of information science abstracts, performance (measured by precision averaged over a range of recall levels) improves by 28% compared to a weighted term‐vector approach, and improves 10% compared to Latent Semantic Indexing. Similar improvement is obtained on the Cranfield database, but no improvement is obtained for the artificial MED database of medical abstracts. The generally favorable performance suggests interesting potential for methods which explicitly modify the retrieval system to meet interdocument semantic constraints.