nils-herrmann/clean_scirepeval huggingface dataset:
An Empirical Evaluation of Chernoff Faces, Star Glyphs, and Spatial Visualisations for Binary Data

Data visualization has the potential to assist humans in analyzing and comprehending large volumes of data, and to detect patterns, clusters and outliers that are not obvious using non-graphical forms of presentation. For this reason, data visualizations have an important role to play in a diverse range of applied problems, including data exploration and mining, information retrieval, and intelligence analysis. Unfortunately, while a variety of different approaches are available for data visualization, there have been few rigorous evaluations of their effectiveness. This paper presents the results of a controlled experiment comparing the ability four different visualization approaches to help people answer meaningful questions for binary data sets. Two of these visualizations, Chernoff faces and star glyphs, represent objects using simple icon-like displays. The other two visualizations use a spatial arrangement of the objects, based on a model of human mental representation, where more similar objects are placed nearer each other. One of these spatial displays uses a common features model of similarity, while the other uses a distinctive features model. It is found that both glyph visualizations lead to slow, inaccurate answers being given with low confidence, while the faster and more confident answers for spatial visualizations are only accurate when the common features similarity model is used.
The roles of the convex hull and the number of potential intersections in performance on visually presented traveling salesperson problems

The planar Euclidean version of the traveling salesperson problem requires finding the shortest tour through a two-dimensional array of points. MacGregor and Ormerod (1996) have suggested that people solve such problems by using a global-to-local perceptual organizing process based on the convex hull of the array. We review evidence for and against this idea, before considering an alternative, local-to-global perceptual process, based on the rapid automatic identification of nearest neighbors. We compare these approaches in an experiment in which the effects of number of convex hull points and number of potential intersections on solution performance are measured. Performance worsened with more points on the convex hull and with fewer potential intersections. A measure of response uncertainty was unaffected by the number of convex hull points but increased with fewer potential intersections. We discuss a possible interpretation of these results in terms of a hierarchical solution process based on linking nearest neighbor clusters.
OPTIMIZATION AND INTELLIGENCE : INDIVIDUAL DIFFERENCES IN PERFORMANCE ON THREE TYPES OF VISUALLY PRESENTED OPTIMIZATION PROBLEMS

Although problem solving is an essential expression of intelligence, both experimental and differential psychology have neglected an important class of problems, for which it is difficult or impossible for systematic procedures to provide a definitive solution. A study is described, in which participants’ solutions to three such computationally difficult problems (a Travelling Salesperson, a Minimal Spanning Tree, and a Generalized Steiner Tree problem) all showed consistent individual differences that intercorrelated reliably and correlated moderately with scores on Raven’s Advanced Progressive Matrices. The results are interpreted in terms of a theory of visual perception based on detecting minimal structures and selecting transformations that maximize self-similarity. This interpretation is discussed in the context of a perspective in which intelligence is viewed as the capacity to solve problems that (either in principle or in practice) defy definitive solution by a mechanical algorithm.
Are these two groups of scores significantly different ? A Bayesian approach

A common statistical inference problem in experimental psychology involves deciding whether or not there are differences in the means and variances of two groups of scores. Standard current practice for this problem is to use Null Hypothesis Significance Testing (NHST) methods, such as the t-test and Levene’s test. There are, however, serious theoretical and practical flaws in NHST, particularly in terms of its violation of the likelihood principle. Accordingly, this paper develops an alternative method using Bayesian statistical inference. The basic idea is to consider four different models, each of which assumes the groups scores comes from a Gaussian distribution, but which make different assumptions about the equality of means and variances across the groups. By finding maximum likelihood parameter values for these models, it is possible to calculate Bayesian Information Criterion values for each model, and hence estimate the Bayes Factors between each pair of models. The Bayes Factors quantify the evidence provided by the data in a naturally meaningful and interpretable way, allowing the assessment of whether differences are “significant” to be made in terms of the standards of scientific evidence required for the research problem at hand. An illustrative application of this Bayesian approach to real data is presented. The extension of the approach to cases where data have different distributions is discussed, as is the possibility of using Bayesian statistical inference for other standard problems of data analysis in psychology. ∗I thank Nick Burns, Daniel Navarro and Douglas Vickers for helpful comments, and Nick Burns for supplying the window tinting data. Correspondence should be addressed to: Michael D. Lee, Department of Psychology, University of Adelaide, SA 5005,  michael.lee@psychology.adelaide.edu.au, URL: http://www.psychology.adelaide.edu.au/members/staff/michaellee/homepage
The roles of the convex hull and number of intersections upon performance on visually presented traveling salesperson problems

The planar Euclidean version of the Traveling Salesperson Problem (TSP) requires finding a tour of minimal length through a two-dimensional set of points. Despite the computational intractability of these problems, most empirical studies have found that humans are able to find good solutions. For this reason, understanding human performance on TSPs has the potential to offer insights into basic perceptual and cognitive decision making processes, as well as potentially informing theorizing in individual differences and intelligence. Through the convex hull hypothesis, previous researchers (MacGregor & Ormerod 1996; MacGregor, Ormerod, & Chronicle 1999; MacGregor, Ormerod, & Chronicle 2000; Ormerod & Chronicle 1999) have suggested people use a global-to-local solution process. We review the empirical evidence for and against this idea, before suggesting an alternative local-to-global solution process, based on the avoidance of intersections in constructing a tour. To compare these two competing approaches, we present the results of an experiment that measures the different effects the number of points on the convex hull and the number of potential intersections have on human performance. It is found that both have independent effects on the degree to which people deviate from the optimal solution, with performance worsening as more points are added to the convex hull and as fewer potential intersections are present. A measure of response uncertainty, capturing the degree to which different people produce the same types of solution, is found to be unaffected by the number of points on the convex hull but to increase as fewer potential intersections are present. A possible interpretation of these results in terms of a generative transformational theory of human visual perception is discussed. ∗Correspondence should be addressed to: Douglas Vickers, Department of Psychology, University of Adelaide, SA 5005,
Extending the ALCOVE model of category learning to featural stimulus domains

The ALCOVE model of category learning, despite its considerable success in accounting for human performance across a wide range of empirical tasks, is limited by its reliance on spatial stimulus representations. Some stimulus domains are better suited to featural representation, characterizing stimuli in terms of the presence or absence of discrete features, rather than as points in a multidimensional space. We report on empirical data measuring human categorization performance across a featural stimulus domain and show that ALCOVE is unable to capture fundamental qualitative aspects of this performance. In response, a featural version of the ALCOVE model is developed, replacing the spatial stimulus representations that are usually generated by multidimensional scaling with featural representations generated by additive clustering. We demonstrate that this featural version of ALCOVE is able to capture human performance where the spatial model failed, explaining the difference in terms of the contrasting representational assumptions made by the two approaches. Finally, we discuss ways in which the ALCOVE categorization model might be extended further to use “hybrid” representational structures combining spatial and featural components.
Generating Additive Clustering Models with Minimal Stochastic Complexity

Additive clustering models provide a conceptually simple and representation-ally powerful approach to extracting features from similarity data. Objects are represented according to the presence or absence of a set of weighted features, and their observed similarities are modeled using the recovered features that objects have in common. Unlike partitioning or hierarchical clustering approaches, most approaches to additive clustering place no constraints on the way features may be assigned to objects. This representational freedom demands, however, that the issue of additive clustering model complexity is addressed. It is important that additive clustering models are generated so as to balance the competing demands of goodness-of-fit and complexity for substantive interpretation. This paper uses previous analytic results to derive a stochastic complexity criterion measure for additive clustering models. This measure simultaneously takes into account the goodness-of-fit, the number of clusters, and the complexity associated with the patterns of cluster inclusion and overlap within the model. A new algorithm for fitting additive clustering models to similarity data is then developed, using the stochastic complexity measure to control the balance between goodness-of-fit and complexity. The ability of the algorithm to recover known features and weights is assessed using Monte Carlo techniques, and its application to empirical data is demonstrated using a previously examined data set that measures the similarities among kinship terms.
Spatial Visualisation of Document Similarity

Sorting through a large set of documents when only some are relevant to the topic at hand is a central problem facing analysts. Graphical depiction of the document set has the potential to be a powerful tool, by providing insight into the documents at a glance. The strength of such document visualisations is the possibility of quickly imparting large amounts of information to the human operator. Correspondingly, however, reliance on visual inspection requires that visualisations be guided by an understanding of human perception and cognition (Lee and Vickers 1998). This paper presents preliminary work on employing spatial visualisations of document similarity employing both the well-established multidimensional scaling (MDS) technique and a recently developed approach known as Isomap.
On the Complexity of Additive Clustering Models.

Additive clustering provides a conceptually simple and potentially powerful approach to modeling the similarity relationships between stimuli. The ability of additive clustering models to accommodate similarity data, however, typically arises through the incorporation of large numbers of parameterized clusters. Accordingly, for the purposes of both model generation and model comparison, it is necessary to develop quantitative evaluative measures of additive clustering models that take into account both data-fit and complexity. Using a previously developed probabilistic formulation of additive clustering, the Bayesian Information Criterion is proposed for this role, and its application demonstrated. Limitations inherent in this approach, including the assumption that model complexity is equivalent to cluster cardinality, are discussed. These limitations are addressed by applying the Laplacian approximation of a marginal probability density, from which a measure of cluster structure complexity is derived. Using this measure, a preliminary investigation is made of the various properties of cluster structures that affect additive clustering model complexity. Among other things, these investigations show that, for a fixed number of clusters, a model with a strictly nested cluster structure is the least complicated, while a model with a partitioning cluster structure is the most complicated.
Clustering Using the Contrast Model

An algorithm is developed for generating featural representations from similarity data using Tversky’s (1977) Contrast Model. Unlike previous additive clustering approaches, the algorithm fits a representational model that allows for stimulus similarity to be measured in terms of both common and distinctive features. The important issue of striking an appropriate balance between data fit and representational complexity is addressed through the use of the Geometric Complexity Criterion to guide model selection. The ability of the algorithm to recover known featural representations from noisy data is tested, and it is also applied to real data measuring the similarity of
Determining the Dimensionality of Multidimensional Scaling Representations for Cognitive Modeling.

Multidimensional scaling models of stimulus domains are widely used as a representational basis for cognitive modeling. These representations associate stimuli with points in a coordinate space that has some predetermined number of dimensions. Although the choice of dimensionality can significantly influence cognitive modeling, it is often made on the basis of unsatisfactory heuristics. To address this problem, a Bayesian approach to dimensionality determination, based on the Bayesian Information Criterion (BIC), is developed using a probabilistic formulation of multidimensional scaling. The BIC approach formalizes the trade-off between data-fit and model complexity implicit in the problem of dimensionality determination and allows for the explicit introduction of information regarding data precision. Monte Carlo simulations are presented that indicate, by using this approach, the determined dimensionality is likely to be accurate if either a significant number of stimuli are considered or a reasonable estimate of precision is available. The approach is demonstrated using an established data set involving the judged pairwise similarities between a set of geometric stimuli.
Fast Text Classification Using Sequential Sampling Processes

A central problem in information retrieval is the automated classification of text documents. While many existing methods achieve good levels of performance, they generally require levels of computation that prevent them from making sufficiently fast decisions in some applied setting. Using insights gained from examining the way humans make fast decisions when classifying text documents, two new text classification algorithms are developed based on sequential sampling processes. These algorithms make extremely fast decisions, because they need to examine only a small number of words in each text document. Evaluation against the Reuters-21578 collection shows both techniques have levels of performance that approach benchmark methods, and the ability of one of the classifiers to produce realistic measures of confidence in its decisions is shown to be useful for prioritizing relevant documents.
Dynamic Models of Simple Judgments: II. Properties of a Self-Organizing PAGAN (Parallel, Adaptive, Generalized Accumulator Network) Model for Multi-Choice Tasks

This is the second of two papers comparing connectionist and traditional stochastic latency mechanisms with respect to their ability to account for simple judgments. In the first, we reviewed evidence for a self-regulating accumulator module for two- and three-category discrimination. In this paper, we examine established neural network models that have been applied to predicting response time measures, and discuss their representational and adaptational limitations. We go on to describe and evaluate the network implementation of a Parallel Adaptive Generalized Accumulator Network (PAGAN), based on the interconnection of a number of self-regulating, generalized accumulator modules. The enhancement of PAGAN through the incorporation of distributed connectionist representation is briefly discussed.
Towards a transformational approach to perceptual organization

A transformational approach to visual perception is presented, in which image structure is encoded in the parameters of those transformations that produce an output maximally symmetric with the current input. Results are presented for a computer program, dubbed SMART (Symmetry Maximizing Array using Random Transformations). SMART consists of a parallel array of independent symmetry detectors. Each detector attempts to find a transformation that maximizes the symmetry between the original and the transformed configuration. The weighted output of the detectors is collated in a connection matrix, which summarizes the image structure and provides a continuously-varying measure of relative symmetry. The program is applied to constrained and random arrays, Glass (1969) figures and the detection of hidden symmetric targets. More general implications are briefly discussed.
An Extraction and Regularization Approach to Additive Clustering

Additive clustering provides a conceptually simple similarity model which is, nevertheless, capable of accommodating arbitrary similarity structures. The discrete nature of the clusters, coupled with the general flexibility of the model, however, means that the derivation of additive clustering models from given similarity data is difficult. After reviewing a number of previously developed algorithms, a new two stage algorithm for generating additive cluster models is developed. In the first stage, an extraction process generates a manageable number of candidate clusters which, in the second stage, are subject to a regularization process. The number of clusters included in the derived model is controlled by a parameter specifying the target level of variance to be accounted for by the final model. Several applications of the proposed algorithm are presented, including three involving previously examined data sets that facilitate an evaluation of performance relative to several other algorithms. It is argued that the proposed algorithm exhibits comparable performance in relation to these previous algorithms, and has the advantage of being developed within a framework that potentially allows the optimization of the tradeoff between goodness-of-fit and model parsimony.
Algorithms for Representing Similarity Data

This report develops and demonstrates algorithms for representing and displaying similarity data using three established cognitive models. The first representational model, multidimensional scaling, represents objects as points in a coordinate space so that similar objects lie near each other. The second representational model, the additive tree, represents objects as terminal nodes in a tree so that the similarity of two objects is modelled by length of the path between them. The third representational model, additive clustering, specifies a number of clusters with associated weights, so that the similarity of two objects is modelled by the sum of the weights of their common clusters. As well as listing and demonstrating MATLAB algorithms for finding these representations, a survey is presented of ways in which similarity and proximity data may be generated, and a principled Bayesian method of controlling the complexity of each representational model is presented. Finally, a number of suggestions are made regarding the use of the three representational models, and the relative strengths and weaknesses of the algorithms in relation to previously developed alternative algorithms are discussed.
Psychological approaches to data visualisation

The aim of data visualization is to display a body of information in a way which allows accurate and effortless human comprehension and analysis. Accordingly, the development of data visualization techniques should be constrained by an understanding of both human perception and cognition. This report develops and examines a psychological framework for the development of data visualization techniques based on the notion of similarity structure modelling. Through a series of case studies, a range of established approaches to data visualization is reviewed and evaluated within this framework, and a number of suggestions for the development of new techniques is made.
Towards intelligent search interfaces: visualising query results by relating meta-data to user-relevance

Our concern is to exploit user behaviour in the interaction between humans and computers. By doing this we expect to move towards adaptive human-computer interaction. The work reported addresses the coordinated use of user behaviours and meta-data generated during interactive computer-mediated information searches. The context used to develop this idea is that of searching the Web for information. This work is in the spirit of the Okapi Projects (Robertson, 1997; Beaulieu, 1997), and the TREC Interactive Track Matrix (Lagergren and Over, 1998), but differs in that access to user behaviours is completely unobtrusive.
Neural Feature Abstraction from Judgments of Similarity

The common neural network modeling practice of representing the elements of a task domain in terms of a set of features lacks justification if the features are derived through some form of ad hoc preabstraction. By examining a featural similarity model related to established multidimensional scaling techniques, a neural network is developed that generates features from similarity data and attaches weights to these features. The network performs a constrained search of a continuous solution space to determine the features and uses a previously developed regularization technique to minimize the number of features it derives. The network is demonstrated on artificial data, from which it recovers known features and weights, and on two real data sets involving the similarity of a set of geometric shapes and the abstract conceptual similarities of the 10 Arabic numerals. On the basis of these results, the relationship between the multidimensional scaling approach adopted by the network and an alternative additive clustering approach to feature extraction is discussed.
Dynamic Models of Simple Judgments: I. Properties of a Self-Regulating Accumulator Module

This is the first of two papers comparing connectionist and traditional stochastic latency mechanisms with respect to their ability to account for simple judgments. In this paper, we show how the need to account for additional features of judgment has led to the formulation of progressively more sophisticated models. One of these, a self-regulating, generalized accumulator process, is treated in detail, and its simulated performance across a sample of tasks is described. Since an adaptive decision module of this kind possesses all the ingredients of intelligent behavior, it is eminently suited as a basic computing element in more complex networks.