nils-herrmann/clean_scirepeval huggingface dataset:
A Bayesian analysis of bivariate survival data from a multicentre cancer clinical trial.

I present an analysis of data from a clinical trial of new chemotherapies for colorectal cancer. I model participating clinical sites via random effects, to allow for heterogeneous patient responses across centres. Patient response is measured by both progression and survival times. The data suggest that patient responses are homogeneous before, but heterogeneous after, disease progression. For one new therapy, an increase in efficacy relative to standard therapy is detected before, but not after, progression.
Prediction and Change Detection

We measure the ability of human observers to predict the next datum in a sequence that is generated by a simple statistical process undergoing change at random points in time. Accurate performance in this task requires the identification of changepoints. We assess individual differences between observers both empirically, and using two kinds of models: a Bayesian approach for change detection and a family of cognitively plausible fast and frugal models. Some individuals detect too many changes and hence perform sub-optimally due to excess variability. Other individuals do not detect enough changes, and perform sub-optimally because they fail to notice short-term temporal trends.
The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth

We present statistical analyses of the large-scale structure of 3 types of semantic networks: word associations, WordNet, and Roget's Thesaurus. We show that they have a small-world structure, characterized by sparse connectivity, short average path lengths between words, and strong local clustering. In addition, the distributions of the number of connections follow power laws that indicate a scale-free pattern of connectivity, with most nodes having relatively few connections joined together through a small number of hubs with many connections. These regularities have also been found in certain other complex natural networks, such as the World Wide Web, but they are not consistent with many conventional models of semantic organization, based on inheritance hierarchies, arbitrarily structured networks, or high-dimensional vector spaces. We propose that these structures reflect the mechanisms by which semantic networks grow. We describe a simple model for semantic growth, in which each new word or concept is connected to an existing network by differentiating the connectivity pattern of an existing node. This model generates appropriate small-world statistics and power-law connectivity distributions, and it also suggests one possible mechanistic basis for the effects of learning history variables (age of acquisition, usage frequency) on behavioral performance in semantic processing tasks.
The dynamics of experimentally induced criterion shifts.

Investigations of decision making have typically assumed stationarity, even though commonly observed "context effects" are dynamic by definition. Mirror effects are an important class of context effects that can be explained by changes in participants' decision criteria. When easy and difficult conditions are blocked alternately and a mirror effect is observed, participants must repeatedly change their decision criteria. The authors investigated the time course of these criterion changes and observed the buildup of mirror effects on a trial-by-trial basis. The data are consistent with slow, systematic changes in decision criteria that lag behind stimulus changes. The length of this lag is considerable: analysis of a simple dynamic signal-detection model suggests participants take an average of around 14 trials to adjust to new decision environments. This trial-level measurement of experimentally induced changes has implications for traditional blockwise analyses of data and for models of decision making.
Integrating Topics and Syntax

Statistical approaches to language learning typically focus on either short-range syntactic dependencies or long-range semantic dependencies between words. We present a generative model that uses both kinds of dependencies, and can be used to simultaneously find syntactic classes and semantic topics despite having no representation of syntax or semantics beyond statistical dependency. This model is competitive on tasks like part-of-speech tagging and document classification with models that exclusively use short- and long-range dependencies respectively.
The Author-Topic Model for Authors and Documents

We introduce the author-topic model, a generative model for documents that extends Latent Dirichlet Allocation (LDA; Blei, Ng, & Jordan, 2003) to include authorship information. Each author is associated with a multinomial distribution over topics and each topic is associated with a multinomial distribution over words. A document with multiple authors is modeled as a distribution over topics that is a mixture of the distributions associated with the authors. We apply the model to a collection of 1,700 NIPS conference papers and 160,000 CiteSeer abstracts. Exact inference is intractable for these datasets and we use Gibbs sampling to estimate the topic and author distributions. We compare the performance with two other generative models for documents, which are special cases of the author-topic model: LDA (a topic model) and a simple author model in which each author is associated with a distribution over words rather than a distribution over topics. We show topics recovered by the author-topic model, and demonstrate applications to computing similarity between authors and entropy of author output.
Nonword Repetition in Lexical Decision: Support for two Opposing Processes

We tested and confirmed the hypothesis that the prior presentation of nonwords in lexical decision is the net result of two opposing processes: (1) a relatively fast inhibitory process based on global familiarity; and (2) a relatively slow facilitatory process based on the retrieval of specific episodic information. In three studies, we manipulated speed-stress to influence the balance between the two processes. Experiment 1 showed item-specific improvement for repeated nonwords in a standard “respond-when-ready” lexical decision task. Experiment 2 used a 400-ms deadline procedure and showed performance for nonwords to be unaffected by up to four prior presentations. In Experiment 3 we used a signal-to-respond procedure with variable time intervals and found negative repetition priming for repeated nonwords. These results can be accounted for by dual-process models of lexical decision (e.g., Balota & Chumbley, 1984; Balota & Spieler, 1999).
Probabilistic author-topic models for information discovery

We propose a new unsupervised learning technique for extracting information from large text collections. We model documents as if they were generated by a two-stage stochastic process. Each author is represented by a probability distribution over topics, and each topic is represented as a probability distribution over words for that topic. The words in a multi-author paper are assumed to be the result of a mixture of each authors' topic mixture. The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm. We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and learn a model with 300 topics. We discuss in detail the interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, significant trends in the computer science literature between 1990 and 2002, parsing of abstracts by topics and authors and detection of unusual papers by specific authors. An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer.
Finding scientific topics

A first step in identifying the content of a document is determining which topics that document addresses. We describe a generative model for documents, introduced by Blei, Ng, and Jordan [Blei, D. M., Ng, A. Y. & Jordan, M. I. (2003) J. Machine Learn. ], in which each document is generated by choosing a distribution over topics and then choosing each word in the document from a topic selected according to this distribution. We then present a Markov chain Monte Carlo algorithm for inference in this model. We use this algorithm to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics. We show that the extracted topics capture meaningful structure in the data, consistent with the class designations provided by the authors of the articles, and outline further applications of this analysis, including identifying “hot topics” by examining temporal dynamics and tagging abstracts to illustrate semantic content.
The effect of normative context variability on recognition memory.

According to some theories of recognition memory (e.g., S. Dennis & M. S. Humphreys, 2001), the number of different contexts in which words appear determines how memorable individual occurrences of words will be: A word that occurs in a small number of different contexts should be better recognized than a word that appears in a larger number of different contexts. To empirically test this prediction, a normative measure is developed, referred to here as context variability, that estimates the number of different contexts in which words appear in everyday life. These findings confirm the prediction that words low in context variability are better recognized (on average) than words that are high in context variability.
Conceptual interrelatedness and caricatures

Concepts are interrelated to the extent that the characterization of each concept is influenced by the other concepts, and are isolated to the extent that the characterization of one concept is independent of other concepts. The relative categorization accuracy of the prototype and caricature of a concept can be used as a measure of concept interrelatedness. The prototype is the central tendency of a concept, whereas a caricature deviates from the concept’s central tendency in the direction opposite the central tendency of other acquired concepts. The prototype is predicted to be relatively well categorized when a concept is relatively independent of other concepts, but the caricature is predicted to be relatively well categorized when a concept is highly related to other concepts. Support for these predictions comes from manipulations of the labels given to simultaneously acquired concepts (Experiment 1) and of the order of categories during learning (Experiment 2).
Feature frequency effects in recognition memory

Rare words are usually better recognized than common words, a finding in recognition memory known as theword-frequency effect. Some theories predict the word-frequency effect because they assume that rare words consist of more distinctive features than do common words (e.g., Shiffrin & Steyvers's, 1997, REM theory). In this study, recognition memory was tested for words that vary in the commonness of their orthographic features, and we found that recognition was best for words made up of primarily rare letters. In addition, a mirror effect was observed: Words with rare letters had a higher hit rate and a lower false-alarm rate than did words with common letters. We also found that normative word frequency affects recognition independently of letter frequency. Therefore, the distinctiveness of a word's orthographic features is one, but not the only, factor necessary to explain the word-frequency effect.
FREQUENCY EFFECT 1 Running head : FEATURE FREQUENCY EFFECT Feature Frequency Effects in Recognition Memory

Rare words are usually better recognized than common words, a finding in recognition memory known as the word-frequency effect. Some theories predict the word-frequency effect because they assume that rare words consist of more distinctive features than common words (e.g., REM, Shiffrin & Steyvers, 1997). In this study, recognition memory was tested for words that vary in the commonness of their orthographic features, and we found that recognition was best for words made up of primarily rare letters. In addition, a mirror effect was observed: words with rare letters had a higher hit rate and a lower false-alarm rate than words with common letters. We also found that normative word frequency affects recognition independently of letter frequency. Therefore, the distinctiveness of a word's orthographic features is one but not the only factor necessary to explain the word-frequency effect. FEATURE FREQUENCY EFFECT 3 Rare words are better recognized than common words (Schulman, 1967; Shepard, 1967; but see Wixted, 1992), a finding in recognition memory known as the wordfrequency effect (WFE). For single-word old-new recognition, hit rates (HRs, correctly responding “old” to an old word) are higher and false-alarm rates (FARs, incorrectly responding “old” to an new word) are lower for low-frequency words than for highfrequency words (Glanzer & Adams 1985). Examples of some of the accounts for the advantage for low-frequency (LF) words include: differences in the distribution of attentional resources (e.g., Glanzer & Adams, 1990; Maddox & Estes, 1997), multiple retrieval processes (e.g., Joordens & Hockley, 2000), the number of different contexts in which words appear (e.g., Dennis & Humphreys, 2001), and differences in encoding variability (e.g., McClelland & Chappell, 1998). No consensus yet exists regarding how many -if any -of these accounts is correct, but it is quite clear that normative word frequency is correlated with a large number of variables that could theoretically produce the WFE (cf. Gillund & Shiffrin, 1984; Shiffrin & Steyvers, 1997). Here, we directly test an hypothesis other than those listed above, that differences in the distinctiveness of the features that comprise words of differing frequency produce the WFE (e.g., Shiffrin &
A Bayesian model for the time course of lexical processing.

A Bayesian-based model for lexical decision, REM-LD, is fit to data from a novel version of a signal-to-respond paradigm. REM-LD calculates the odds that a test item is a word, by accumulating likelihood ratios for each lexical entry in a small neighborhood of similar words. The new model predicts the time course of observed effects of nonword lexicality, word frequency and repetition priming. It can also make qualitative predictions for the response time distributions in tasks with subject paced responding.
Multidimensional Scaling Multidimensional Scaling , Similarity Judgments , Proximity data , Geometric Models , Knowledge Representation , data visualization , dimension reduction , multivariate analysis

Multidimensional Scaling (MDS) describes a family of techniques for the analysis of proximity data on a set of stimuli to reveal the hidden structure underlying the data. The proximity data can come from similarity judgments, identification confusion matrices, grouping data, same-different errors or any other measure of pairwise similarity. The main assumption in MDS is that stimuli can be described by values along a set of dimensions that places these stimuli as points in a multidimensional space and that the similarity between stimuli is inversely related to the distances of the corresponding points in the multidimensional space. The Minkowski distance metric provides a general way to specify distance in a multidimensional space: r n k r jk ik ij x x d 1 1       − = ∑ = , where n is the number of dimensions, and x ik is the value of dimension k for stimulus i. With r =2, the metric equals the Euclidian distance metric while r=1 leads to the city-block metric. A Euclidian metric is appropriate when the stimuli are composed of integral or perceptually fused dimensions such as the dimensions of brightness and saturation for colours. The city-block metric is appropriate when the stimuli are composed of separable dimensions such as size and brightness (Attneave, 1950). In practice, the Euclidian distance metric is often used because of mathematical convenience in MDS procedures. MDS can be applied with different purposes. One is exploratory data analysis; by placing objects as points in a low dimensional space, the observed complexity in the original data matrix can often be reduced while preserving the essential information in the data. By a representation of the pattern of proximities in two or three dimensions, researchers can visually study the structure in the data.
MODELING SEMANTIC AND ORTHOGRAPHIC SIMILARITY EFFECTS ON MEMORY FOR INDIVIDUAL WORDS

Many memory models assume that the semantic and physical features of words can be represented by collections of features abstractly represented by vectors. Most of these memory models are process oriented; they explicate the processes that operate on memory representations without explicating the origin of the representations themselves; the different attributes of words are typically represented by random vectors that have no formal relationship to the words in our language. In Part I of this research, we develop Word Association Spaces (WAS) that capture aspects of the meaning of words. This vector representation is based on a statistical analysis of a large database of free association norms. In Part II, this representation along with a representation for the physical aspects of words such as orthography is combined with REM, a process model for memory. Three experiments are presented in which distractor similarity, the length of studied categories and the directionality of association between study and test words were varied. With only a few parameters, the REM model can account qualitatively for the results. Developing a representation incorporating features of actual words makes it possible to derive predictions for individual test words. We show that the moderate correlations between observed and predicted hit and false alarm rates for individual words are larger than can be explained by models that represent words by arbitrary features. In Part III, an experiment is presented that tests a prediction of REM: words with uncommon features should be better recognized than words with common features, even if the words are equated for word frequency.
Interactions Between Perceptual and Conceptual Learning

Confusions arise when 'stable' is equated with 'foundational.' Spurred on by the image of a house`s foundation, it is tempting to think that something provides effective support to the extent that it is rigid and stable. We will argue that when considering the role of perception in grounding our concepts, exactly the opposite is true. Our perceptual system supports our ability to acquire new concepts by being flexibly tuned to these concepts. Whereas the concepts that we learn are certainly influenced by our perceptual representations, we will argue that these perceptual representations are also influenced by the learned concepts. In keeping with one of the central themes of this book, behavioral adaptability is completely consistent with representationalism. In fact, the most straightforward account of our experimental results is that concept learning can produce changes in perceptual representations, the 'vocabulary' of perceptual features, that are used by subsequent tasks. 

This chapter reviews theoretical and empirical evidence that perceptual vocabularies used to describe visual objects are flexibly adapted to the demands of their user. We will extend arguments made elsewhere for adaptive perceptual representations (Goldstone, Schyns, & Medin, 1998; Schyns, Goldstone, & Thibaut, 1998), and discuss research from our laboratory illustrating specific interactions between perceptual and conceptual learning. We will describe computer simulations that provide accounts of these interactions using neural network models. These models have detectors that become increasingly tuned to the set of perceptual features that support concept learning. The bulk of the chapter will be organized around mechanisms of human perceptual learning, and computer simulations of these mechanisms.
A model for recognition memory: REM—retrieving effectively from memory

A new model of recognition memory is reported. This model is placed within, and introduces, a more elaborate theory that is being developed to predict the phenomena of explicit and implicit, and episodic and generic, memory. The recognition model is applied to basic findings, including phenomena that pose problems for extant models: the list-strength effect (e.g., Ratcliff, Clark, & Shiffrin, 1990), the mirror effect (e.g., Glanzer & Adams, 1990), and the normal-ROC slope effect (e.g., Ratcliff, McKoon, & Tindall, 1994). The model assumes storage of separate episodic images for different words, each image consisting of a vector of feature values. Each image is an incomplete and error prone copy of the studied vector. For the simplest case, it is possible to calculate the probability that a test item is “old,” and it is assumed that a default “old” response is given if this probability is greater than .5. It is demonstrated that this model and its more complete and realistic versions produce excellent qualitative predictions.
Fast Kernel Classifiers with Online and Active Learning

Very high dimensional learning systems become theoretically possible when training examples are abundant. The computing cost then becomes the limiting factor. Any efficient learning algorithm should at least take a brief look at each example. But should all examples be given equal attention?This contribution proposes an empirical answer. We first present an online SVM algorithm based on this premise. LASVM yields competitive misclassification rates after a single pass over the training examples, outspeeding state-of-the-art SVM solvers. Then we show how active example selection can yield faster training, higher accuracies, and simpler models, using only a fraction of the training example labels.
Structural bioinformatics Semi-supervised protein classification using cluster kernels

Building an accurate protein classification system depends critically upon choosing a good representation of the input sequences of amino acids. Recent work using string kernels for protein data has achieved state-of-the-art classification performance. However, such representations are based only on labeled data—examples with known 3D structures, organized into structural classes—whereas in practice, unlabeled data are far more plentiful.  In this work, we develop simple and scalable cluster kernel techniques for incorporating unlabeled data into the representation of protein sequences. We show that our methods greatly improve the classification performance of string kernels and outperform standard approaches for using unlabeled data, such as adding close homologs of the positive examples to the training data. We achieve equal or superior performance to previously presented cluster kernel methods and at the same time achieving far greater computational efficiency.  The Spider matlab package is available at www.kyb.tuebingen.mpg.de/bs/people/spider
Joint Kernel Maps

We develop a methodology for solving high dimensional dependency estimation problems between pairs of data types, which is viable in the case where the output of interest has very high dimension, e.g., thousands of dimensions. This is achieved by mapping the objects into continuous or discrete spaces, using joint kernels. Known correlations between input and output can be defined by such kernels, some of which can maintain linearity in the outputs to provide simple (closed form) pre-images. We provide examples of such kernels and empirical results.