nils-herrmann/clean_scirepeval huggingface dataset:
Automatic Extraction of Phase and Frequency Information from Raw Voice Data

We use a simple network which uses negative feedback of activation and simple Hebbian learning to self-organise in such a way as to produce a feature map which has the property of identifying the relative proportions of the components of the input data. Thus it evaluates the angular properties of the input data space and ignores the magnitude of the input data. When used on unprocessed voice data, the network is shown to extract both the phase and the frequency information from the raw data. We show how to use this network for classification of vowels.
An Angular Quantising Self Organising Map for Scale Invariant Classification

We use a simple network that uses negative feedback of activation and simple Hebbian learning to self organise in such a way as to produce a feature map which has the property of identifying the relative proportions of the components of the input data. Thus, it evaluates the angular properties of the input data space and ignores the magnitude of the input data. We have previously shownn1] that we can easily nd a one dimensional mapping of input data; we now present an improved algorithm which nds a two dimensional mapping of the data. When we train this network on uniformly distributed artiicial data, we can observe convergence to the shape of a sphere which exhibits the scale invariant classifying properties previously seen in the network.
Study and comparison of multilayer perceptron NN and radial basis function NN in oceanographic forecasting

The research presented in this paper describes a comparison of the use of Multi-Layer Perceptron Neural Network and the Radial Basis Functions Neural Networks in predicting the physical structure of the ocean up to 20 km ahead of a sea going vessel. The data used in this experiment was acquired on board a vessel transecting the Atlantic Ocean from 50 degree(s) South (the Falkland Island) to 50 degree(s) North (the UK). The research is part of the Strategic Tactical Environmental Bubble project which aims to develop a methodology for predicting the physical structures in three dimensions, around a sea going vessel from data acquired in situ and historical records. A Sobel filter has been applied to pre- process the raw data, enhancing features and smoothing stable areas. Some characteristics of both types of Neural Networks are outlined, as well as an exhaustive comparison of both methods. The advantages and disadvantages of both to this particular problem are discussed. A final system is presented together with a critical evaluation of its performance. For this purpose a real-time simulator has been built and its results are presented here.
Project Monitoring Intelligent Agent system

A multi-agent system architecture that facilitates the design, implementation and communication of autonomous dedicated agents is discussed. The architecture allows the learning abilities of agents using both connectionist and symbolic artificial intelligence (AI) techniques to be investigated. Identifying a particular supervisor for a student is a very time-consuming task. Processing all the information related to students and staff members, distributing this information, obtaining feedback from the participants, making sure that all students have supervisors and that all members of staff have similar numbers of students to supervise can be a very tedious task. The Project Monitoring Intelligent Agent (PMIA) system has been developed with the aim of automating as much of this process as possible. PMIA is a multi-agent based tool designed using the philosophy of Internet software agents and uses a hybrid architecture that combines the advantages of both symbolic and connectionist approaches to artificial intelligence problem solving. The system design is influenced by the agent architecture proposed by the Knowledge Sharing Effort using the Knowledge, Query and Manipulation Language for agent communication; this is an Internet agent based system, designed using HTML, Perl and Visual C++, that requires a WWW server and the help of Web browsers.
A scale-invariant feature map.

We use a simple network which uses negative feedback of activation and simple Hebbian learning to self-organize in such a way as to produce a hierarchical classification network. By adding neighbourhood relations to its learning rule, we create a feature map which has the property of retaining the angular properties of the input data, i.e. vectors of similar directions are classified similarly regardless of their magnitude. We use neither re-normalization of weights nor data preprocessing in the network despite using competition based on maximizing the neuron's activation.
Edge Enhancement and Exploratory Projection Pursuit

We present a neural network algorithm based on simple Hebbian learning which allows the finding of higher order structure in data. The neural network uses negative feedback of activation to self-organise; such networks have previously been shown to be capable of performing Principal Component Analysis (PCA). In this paper, this is extended to Exploratory Projection Pursuit (EPP) which is a statistical method for investigating structure in high-dimensional data sets.
Introducing Asymmetry into Interneuron Learning

A review is given of a new artificial neural network architecture in which the weights converge to the principal component subspace. The weights learn by only simple Hebbian learning yet require no clipping, normalization or weight decay. The net self-organizes using negative feedback of activation from a set of "interneurons" to the input neurons. By allowing this negative feedback from the interneurons to act on other interneurons we can introduce the necessary asymmetry to cause convergence to the actual principal components. Simulations and analysis confirm such convergence.
Finding compact and sparse-distributed representations of visual images

Some recent work has investigated the dichotomy between compact coding using dimensionality reduction and sparse-distributed coding in the context of understanding biological information processing. We introduce an artificial neural network which self-organizes on the basis of simple Hebbian learning and negative feedback of activation and show that it is capable both of forming compact codings of data distributions and of identifying filters most sensitive to sparse-distributed codes. The network is extremely simple and its biological relevance is investigated via its response to a set of images which are typical of everyday life. However, an analysis of the network's identification of the filter for sparse coding reveals that this coding may not be globally optimal and that there exists an innate limiting factor which cannot be transcended.
Predictive Discretization during Model Selection

We present an approach to discretizing multivariate continuous data while learning the structure of a graphical model. We derive a joint scoring function from the principle of predictive accuracy, which inherently ensures the optimal trade-off between goodness of fit and model complexity including the number of discretization levels. Using the so-called finest grid implied by the data, our scoring function depends only on the number of data points in the various discretization levels (independent of the metric used in the continuous space). Our experiments with artificial data as well as with gene expression data show that discretization plays a crucial role regarding the resulting network structure.
To appear in "Semi-supervised learning", O. Chappelle, B. Scholkopf, and A. Zien, Eds., 2005

Information regularization is a principle for assigning labels to unlabeled data points in a semi-supervised setting. The broader principle is based on finding labels that minimize the information induced between examples and labels relative to a topology over the examples; any label variation within a small local region of examples ties together the identities of examples and their labels. Such variation should be minimized unless supported directly or indirectly by the available labeled examples. The principle can be cast in terms of Tikhonov style regularization for maximizing likelihood of labeled examples with an information theoretic regularization penalty. We consider two ways of representing the topology over examples, either based on complete knowledge of the marginal density, or by grouping together examples whose labels should be related. We discuss the learning algorithms and sample complexity issues that result from each representation.
Focused Inference

We develop a method similar to variable elimination for computing approximate marginals in graphical models. An underlying notion in this method is that it is not always necessary to compute marginals over all the variables in the graph, but focus on a few variables of interest. The Focused Inference (FI) algorithm introduced reduces the original distribution to a simpler one over the variables of interest. This is done in an iterative manner where in each step the operations are guided by (local) optimality properties. We exemplify various properties of the focused inference algorithm and compare it with other methods. Numerical simulation indicates that FI outperform competing methods.
Using term informativeness for named entity detection

Informal communication (e-mail, bulletin boards) poses a difficult learning environment because traditional grammatical and lexical information are noisy. Other information is necessary for tasks such as named entity detection. How topic-centric, or informative, a word is can be valuable information. It is well known that informative words are best modeled by "heavy-tailed" distributions, such as mixture models. However, informativeness scores do not take full advantage of this fact. We introduce a new informativeness score that directly utilizes mixture model likelihood to identify informative words. We use the task of extracting restaurant names from bulletin board posts as a way to determine effectiveness. We find that our "mixture score" is weakly effective alone and highly effective when combined with Inverse Document Frequency. We compare against other informativeness criteria and find that only Residual IDF is competitive against our combined IDF/Mixture score.
Validation and refinement of gene-regulatory pathways on a network of physical interactions

As genome-scale measurements lead to increasingly complex models of gene regulation, systematic approaches are needed to validate and refine these models. Towards this goal, we describe an automated procedure for prioritizing genetic perturbations in order to discriminate optimally between alternative models of a gene-regulatory network. Using this procedure, we evaluate 38 candidate regulatory networks in yeast and perform four high-priority gene knockout experiments. The refined networks support previously unknown regulatory mechanisms downstream of SOK2 and SWI4.
Time Series Analysis of Gene Expression and Location Data

We develop a method for integrating time series expression profiles and factor-gene binding data to quantify dynamic aspects of gene regulation. We estimate latencies for transcription activation by explaining time correlations between gene expression profiles through available factor-gene binding information. The resulting aligned expression profiles are subsequently clustered and again combined with binding information to determine groups or subgroups of co-regulated genes. The predictions derived from this approach are consistent with existing results.1,2 Our analysis also provides several hypotheses not implicated in previous studies.
MAP estimation via agreement on trees: message-passing and linear programming

We develop and analyze methods for computing provably optimal maximum a posteriori probability (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of tree-structured distributions, we obtain an upper bound on the optimal value of the original problem (i.e., the log probability of the MAP assignment) in terms of the combined optimal values of the tree problems. We prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common. An important implication is that any such shared configuration must also be a MAP configuration for the original distribution. Next we develop two approaches to attempting to obtain tight upper bounds: a) a tree-relaxed linear program (LP), which is derived from the Lagrangian dual of the upper bounds; and b) a tree-reweighted max-product message-passing algorithm that is related to but distinct from the max-product algorithm. In this way, we establish a connection between a certain LP relaxation of the mode-finding problem and a reweighted form of the max-product (min-sum) message-passing algorithm.
Approximate Expectation Propagation for Bayesian Inference on Large-scale Problems

where k indexes experimental replicates, i indexes the probe positions, j indexes the binding positions, andN ( jPj aji jjsjbj; i) represents the probability density function of a Gaussian distribution with mean Pj aji jjsjbj and variance i. We assign prior distributions on the binding event bj and the binding strength sj: p(bjj j) = bj j (1 j)1 bj (3) p0(sj) = Gamma(sjjc0; d0) (4) where Gamma( jc0; d0) stands for the probability density functions of Gamma distributions with hyperparameters c0 and d0. We assign a hyperprior distribution on the binding probability j as: p0( j) = Beta( jj 0; 0) (5)
Modeling the Combinatorial Functions of Multiple Transcription Factors

A considerable fraction of gene promoters are bound by multiple transcription factors. It is therefore important to understand how such factors interact in regulating the genes. In this paper, we propose a computational method to identify groups of co-regulated genes and the corresponding regulatory programs of multiple transcription factors from protein- DNA binding and gene expression data. The key concept is to characterize a regulatory program in terms of two properties of individual transcription factors: the function of a regulator as an activator or a repressor, and its direction of effectiveness as necessary or sufficient. We apply a greedy algorithm to find the regulatory models which best explain the available data. Empirical analysis indicates that the inferred regulatory models agree with known combinatorial interactions between regulators and are robust against various parameter choices.
A new class of upper bounds on the log partition function

We introduce a new class of upper bounds on the log partition function of a Markov random field (MRF). This quantity plays an important role in various contexts, including approximating marginal distributions, parameter estimation, combinatorial enumeration, statistical decision theory, and large-deviations bounds. Our derivation is based on concepts from convex duality and information geometry: in particular, it exploits mixtures of distributions in the exponential domain, and the Legendre mapping between exponential and mean parameters. In the special case of convex combinations of tree-structured distributions, we obtain a family of variational problems, similar to the Bethe variational problem, but distinguished by the following desirable properties: i) they are convex, and have a unique global optimum; and ii) the optimum gives an upper bound on the log partition function. This optimum is defined by stationary conditions very similar to those defining fixed points of the sum-product algorithm, or more generally, any local optimum of the Bethe variational problem. As with sum-product fixed points, the elements of the optimizing argument can be used as approximations to the marginals of the original model. The analysis extends naturally to convex combinations of hypertree-structured distributions, thereby establishing links to Kikuchi approximations and variants.
MAP estimation via agreement on (hyper)trees: Message-passing and linear programming

We develop and analyze methods for computing provably optimal {\em maximum a posteriori} (MAP) configurations for a subclass of Markov random fields defined on graphs with cycles. By decomposing the original distribution into a convex combination of tree-structured distributions, we obtain an upper bound on the optimal value of the original problem (i.e., the log probability of the MAP assignment) in terms of the combined optimal values of the tree problems. We prove that this upper bound is tight if and only if all the tree distributions share an optimal configuration in common. An important implication is that any such shared configuration must also be a MAP configuration for the original distribution. Next we develop two approaches to attempting to obtain tight upper bounds: (a) a {\em tree-relaxed linear program} (LP), which is derived from the Lagrangian dual of the upper bounds; and (b) a {\em tree-reweighted max-product message-passing algorithm} that is related to but distinct from the max-product algorithm. In this way, we establish a connection between a certain LP relaxation of the mode-finding problem, and a reweighted form of the max-product (min-sum) message-passing algorithm.
Tree consistency and bounds on the performance of the max-product algorithm and its generalizations

Finding the maximum a posteriori (MAP) assignment of a discrete-state distribution specified by a graphical model requires solving an integer program. The max-product algorithm, also known as the max-plus or min-sum algorithm, is an iterative method for (approximately) solving such a problem on graphs with cycles. We provide a novel perspective on the algorithm, which is based on the idea of reparameterizing the distribution in terms of so-called pseudo-max-marginals on nodes and edges of the graph. This viewpoint provides conceptual insight into the max-product algorithm in application to graphs with cycles. First, we prove the existence of max-product fixed points for positive distributions on arbitrary graphs. Next, we show that the approximate max-marginals computed by max-product are guaranteed to be consistent, in a suitable sense to be defined, over every tree of the graph. We then turn to characterizing the nature of the approximation to the MAP assignment computed by max-product. We generalize previous work by showing that for any graph, the max-product assignment satisfies a particular optimality condition with respect to any subgraph containing at most one cycle per connected component. We use this optimality condition to derive upper bounds on the difference between the log probability of the true MAP assignment, and the log probability of a max-product assignment. Finally, we consider extensions of the max-product algorithm that operate over higher-order cliques, and show how our reparameterization analysis extends in a natural manner.
Distributed Information Regularization on Graphs

We provide a principle for semi-supervised learning based on optimizing the rate of communicating labels for unlabeled points with side information. The side information is expressed in terms of identities of sets of points or regions with the purpose of biasing the labels in each region to be the same. The resulting regularization objective is convex, has a unique solution, and the solution can be found with a pair of local propagation operations on graphs induced by the regions. We analyze the properties of the algorithm and demonstrate its performance on document classification tasks.
Training and test performance : sampling

The results from the small scale application are encouraging. Our model successfully reproduces known behavior of the λ−switch on the basis of molecular level competition and resource constraints, without the need to assume protein-protein interactions between cI2 dimers and cI2 and RNA-polymerase. Even in the context of this well-known sub-system, however, few quantitative experimental results are available about binding. Proper validation and use of our model therefore relies on estimating the game parameters from available protein-DNA binding data (in progress). Once the game parameters are known, the model provides valid predictions for a number of possible perturbations to the system, including changing nuclear concentrations and knock-outs.
Inferring regulatory networks from multiple sources of genomic data

This thesis addresses the problems of modeling the gene regulatory system from multiple sources of large-scale datasets. In the first part, we develop a computational framework of building and validating simple, mechanistic models of gene regulation from multiple sources of data. These models, which we call physical network models, annotate the network of molecular interactions with several types of attributes (variables). We associate model attributes with physical interaction and knock-out gene expression data according to the confidence measures of data and the hypothesis that gene regulation is achieved via molecular interaction cascades. By applying standard model inference algorithms, we are able to obtain the configurations of model attributes which optimally fit the data. Because existing datasets do not provide sufficient constraints to the models, there are many optimal configurations which fit the data equally well. In the second part, we develop an information theoretic score to measure the expected capacity of new knock-out experiments in terms of reducing the model uncertainty. We collaborate with biologists to perform suggested knockout experiments and analyze the data. The results indicate that we can reduce model uncertainty by incorporating new data. The first two parts focus on the regulatory effects along single pathways. In the third part, we consider the combinatorial effects of multiple transcription factors on transcription control. We simplify the problem by characterizing a combinatorial function of multiple regulators in terms of the properties of single regulators: the function of a regulator and its direction of effectiveness. With this characterization, we develop an incremental algorithm to identify the regulatory models from protein-DNA binding and gene expression data. These models to a large extent agree with the knowledge of gene regulation pertaining to the corresponding regulators. The three works in this thesis provide a framework of modeling gene regulatory networks.