nils-herrmann/clean_scirepeval huggingface dataset:
Multivariate Density Estimation: an SVM Approach

We formulate density estimation as an inverse operator problem. We then use convergence results of empirical distribution functions to true distribution functions to develop an algorithm for multivariate density estimation. The algorithm is based upon a Support Vector Machine (SVM) approach to solving inverse operator problems. The algorithm is implemented and tested on simulated data from different distributions and different dimensionalities, gaussians and laplacians in $\R^2$ and $\R^{12}$. A comparison in performance is made with Gaussian Mixture Models (GMMs). Our algorithm does as well or better than the GMMs for the simulations tested and has the added advantage of being automated with respect to parameters.
Multivariate Density Estimation: a Support Vector Machine Approach

A Support Vector Machine (SVM) algorithm for multivariate density estimation is developed based on regularization principles and bounds on the convergence of empirical distribution functions. The algorithm is compared to Gaussian Mixture Models (GMMs). Our algorithm outperforms GMMs for data drawn from mixtures of gaussians in IR and IR. Our algorithm is also automated with respect to parameters.  This report describes research done within the Center for Biological and Computational Learning in the Department of Brain and Cognitive Sciences and at the Arti cial Intelligence Laboratory at the Massachusetts Institute of Technology. This research is sponsored by a grant from the O ce of Naval Research under contract No.  Additional support is provided by Eastman Kodak Company, Daimler-Benz AG, Siemens Corporate Research Inc., AT&T, Digital Equipment and Central Research Institute of Electric Power Industry.
PLAGA: a highly parallelizable genetic algorithm for programmable logic arrays test pattern generation

An evolutionary algorithm (EA) approach is used to generate test vectors for the detection of shrinkage faults in programmable logic arrays (PLA). Three basic steps are performed during the generation of the test vectors: crossover, mutation and selection. A new mutation operator is introduced that helps increase the Hamming distance among the candidate solutions. Once crossover and mutation have occurred, the new candidate test vectors with higher fitness function scores replace the old ones. With this scheme, population members steadily improve their fitness level with each new generation. The resulting process yields improved solutions to the problem of the PLA test vector generation for shrinkage faults. PLA testing and fault simulation is computationally prohibitive in uniprocessor machines. However, PLAGA is well suited for powerful parallel processing machines with vectorization capability,.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES

We have simulated the behavior of several arti cial ies, interacting visually with each other. Each y is described by a simple tracking system (Poggio and Reichardt, 1973; Land and Collett, 1974) which summarizes behavioral experiments in which individual ies xate a target. Our main nding is that the interaction of these simple modules gives rise to a variety of relatively complex behaviors. In particular, we observe a swarm-like behavior of a group of many arti cial ies for certain reasonable ranges of our tracking system parameters.  This research is sponsored by grants from ONR under contract N00014-93-1-0385 and from ARPA-ONR under contract N00014-92-J-1879; and by a grant from the National Science Foundation under contract ASC-9217041 (this award includes funds from ARPA provided under the HPCC program). Additional support is provided by the North Atlantic Treaty Organization, ATR Audio and Visual Perception Research Laboratories, Mitsubishi Electric Corporation, Sumitomo Metal Industries, and Siemens AG. Support for the A.I. Laboratory's arti cial intelligence research is provided by ARPA-ONR contract N00014-91-J-4038. Tomaso Poggio is supported by the Uncas and Helen Whitaker Chair at MIT's Whitaker College. Martino Poggio is with the Roxbury Latin School, St. Theresa Ave., West Roxbury, MA
Multi-class Pattern Classification Based on a Probabilistic Model of Combining Binary Classifiers

We propose a novel probabilistic model for constructing a multi-class pattern classifier by weighted aggregation of general binary classifiers including one-versus-the-rest, one-versus-one, and others. Our model has a latent variable that represents class membership probabilities, and it is estimated by fitting it to probability estimate outputs of binary classfiers. We apply our method to classification problems of synthetic datasets and a real world dataset of gene expression profiles. We show that our method achieves comparable performance to conventional voting heuristics.
Nonlinear and Noisy Extension of Independent Component Analysis: Theory and Its Application to a Pitch Sensation Model

In this letter, we propose a noisy nonlinear version of independent component analysis (ICA). Assuming that the probability density function (p.d.f.) of sources is known, a learning rule is derived based on maximum likelihood estimation (MLE). Our model involves some algorithms of noisy linear ICA (e.g., Bermond & Cardoso, 1999) or noise-free nonlinear ICA (e.g., Lee, Koehler, & Orglmeister, 1997) as special cases. Especially when the nonlinear function is linear, the learning rule derived as a generalized expectation-maximization algorithm has a similar form to the noisy ICA algorithm previously presented by Douglas, Cichocki, and Amari (1998). Moreover, our learning rule becomes identical to the standard noise-free linear ICA algorithm in the noiseless limit, while existing MLE-based noisy ICA algorithms do not rigorously include the noise-free ICA. We trained our noisy nonlinear ICA by using acoustic signals such as speech and music. The model after learning successfully simulates virtual pitch phenomena, and the existence region of virtual pitch is qualitatively similar to that observed in a psychoacoustic experiment. Although a linear transformation hypothesized in the central auditory system can account for the pitch sensation, our model suggests that the linear transformation can be acquired through learning from actual acoustic signals. Since our model includes a cepstrum analysis in a special case, it is expected to provide a useful feature extraction method that has often been given by the cepstrum analysis.
Multi-scale clustering for gene expression profiling data

In cluster analyses, setting the scale parameter which is implicitly related to the complexity of the data distribution is an important issue; different scale values lead to different results and hence cause different interpretation. In this study, we discuss a framework of multi-scale clustering, where clustering is done with multiple scale values and then the obtained results are compiled into a visually appropriate form to observe overall structures of the clusters. For such purpose, a brick view method is proposed in this study. The construction of a brick view diagram consists of a reindexing procedure of clusters obtained with various scale values and a sorting procedure of samples so as to minimize the distortion defined based on the multiple clustering results. Although some popular clustering methods, such as K-means, spherical K-means, and hierarchical clustering, have been used within the multi-scale framework, we introduce mean-shift clustering based on the kernel density estimation for directional data. We evaluate our approach and existing hierarchical clustering by using an artificial data set and a real data set of gene expression profiles. The results show global structures of distributions can be observed well and in a stable manner, in the brick view diagram.
Prediction of recurrence in advanced gastric cancer patients after curative resection by gene expression profiling

The prognosis of patients with advanced gastric cancer remains unfavorable. Even after curative resection, 40% of patients with advanced gastric cancer die of recurrence. Conventional clinicopathlogic findings are sometimes inadequate for predicting recurrence in individuals. Hence, we tried to construct a new diagnostic system, which predicts recurrence in patients with advanced gastric cancer after curative resection based on molecular analysis. Gastric cancer progression is a function of multiple genetic events that may affect the expression of large number of genes. We performed gene expression profiling with 2,304 genes in 60 advanced gastric cancer patients who underwent curative resection using a PCR array technique, a high‐throughput quantitative RT‐PCR technique. The diagnostic system, which was constructed from the learning set comprised of 40 patients with the most informative 29 genes, classified each case into a good‐signature or poor‐signature group. Then, we confirmed the predictive performance in an additional test set comprised of 20 patients, and the prediction accuracy for recurrence was 75%. Kaplan‐Meier analysis revealed significant difference between the good‐signature and the poor‐signature group (p = 0.0125). Especially in patients with smaller tumor (≤ 5 cm), less developed LN metastasis (N0,1), or earlier stage (stages I and II), the prediction accuracy was high (88.9%, 84.6%, or 81.8%, respectively). Our diagnostic system based on systematic analysis of gene expression profiling can predict the recurrence at clinically meaningful level. By combining our system with conventional clinicopathologic factors, we can improve the prediction of recurrence in patients with advanced gastric cancer who underwent curative surgery.
A Selection Criterion for Robust Classifiers : Cancer Prognosis with Microarray Gene Expression

Microarray technology is now enabling us to measure the expression levels of large number of genes simultaneously. In clinical medicine, this technology is going to be applied to predict clinical features, such as malignancy of cancer . For such purposes, we need classifiers which show a high prediction accuracy. However, estimation of the true prediction performance of a classifier is highly variable because of the small size of samples and non-negligible observation noise. Consequently, even when we have a classifier with high expected accuracy, it is not safe if its variance is high. We present here a new method, the Parametric Noise Bootstrap and Percentile (PNBP) method, to estimate the variance by generating artificial datasets which simulate an original dataset with observation noise. The advantages of PNBP are, firstly, we overcome the smallness of sample size to estimate the variance. Secondly, we set a clear criterion to select a robust and reliable classifier which does not depend on only one dataset but considers variance of classifier performance estimation with enough simulated datasets.
Molecular Prediction of Response to 5-Fluorouracil and Interferon-α Combination Chemotherapy in Advanced Hepatocellular Carcinoma

The prognosis of hepatocellular carcinoma (HCC) is very poor, particularly in patients with tumors that have invaded the major branches of the portal vein. Combination chemotherapy with intra-arterial 5-fluorouracil and subcutaneous interferon-α has shown promising results for such advanced HCC, but it is important to develop the ability to accurately predict chemotherapeutic responses.  We analyzed the expression of 3,080 genes using a polymerase chain reaction-based array in 20 HCC patients who were treated with combination chemotherapy after reduction surgery. After unsupervised analyses, a supervised classification method for predicting chemotherapeutic responses was constructed. To minimize the number of predictive genes, we used a random permutation test to select only significant (P < 0.01) genes. A leave-one-out cross-validation confirmed the gene selection. We also prepared an additional 11 cases for validation of predictive performance.  Hierarchical clustering analysis and principal component analysis with all 3,080 genes revealed distinct gene expression patterns in responders (those with complete response or partial response) and nonresponders (those with stable disease or progressive disease) to the combination chemotherapy. Using a weighted-voting classification method with either all genes or only significant genes as assessed by permutation testing, the objective responses to treatment were correctly predicted in 17 of 20 cases (accuracy, 85%; positive predictive value, 100%; negative predictive value, 80%). Moreover, patients in the validation dataset could be classified into two distinct prognostic groups using 63 predictive genes.  Molecular analysis of 63 genes can predict the response of patients with advanced HCC and major portal vein tumor thrombi to combination chemotherapy with 5-fluorouracil and interferon-α.
A NOISY NON LI NEAR INDEPENDENT COMPONENT ANALYSIS

In this study, we propose a noisy nonlinear extension of independent component analysis (ICA). There have been proposed several extensions of the original noise-free linear ICA, e.g.: noisy ICA or nonlinear ICA. There are few studies dealing with both noisy and nonlinear situations, however, because of the difficulty in integral calculation of the likelihood. In this study, we approximate the integral by a Taylor expansion and a Laplace approximation. The derived algorithm formnlated as an expectation-maximization (EM) algorithm generalizes several of existing ICA algorithms. We also obtain an optimal step size for our EM algorithm and discuss the reason why various noisy linear ICA algorithms based on max- imum likelihood estimation are unsuccessful in being the noise-free linear ICA in the noiseless limit.
Character string extraction from color documents using supervised learning

In this study, we deal with a pattern classification problem for extracting character string regions from digital color documents that contain both character images and other types of images. We applied supervised classification methods to data set represented in the feature space. By utilizing the characteristics that especially the negative data distribute as to have a cluster structure, we devised a method using a mixture of classifiers. Additionally, we propose a parametric feature that allows to improve the discriminative performance by optimizing its own parameters. As a consequence, we have achieved a high classification accuracy, which is as high as 96.5%, when applied to various color documents even with complicated layouts.
混合 von Mises-Fisher 分布に対する変分法的ベイズ推定 Variational Bayes Inference of Mixtured von Mises-Fisher Distribution

High dimensional directional data is getting important in several applications such as analysis of text and gene expression data. In this report, we proposed variational Bayes algorithm of mixtured von Mises-Fisher distribution, which is the probability density function defined on unit hypersphere. We applied this algorithm to artificial data and computed free energy, and showed that free energy can be used as a criterion to determine the number of units.
Operon Prediction by DNA Microarray: An Approach with a Bayesian Network Model

An operon is a set of genes in prokaryotes, which are transcribed to a single mRNA transcript.Although the operon organization has not yet been completely revealed even in model organisms,such as E. Coli and B. subtilis, the understanding of the operon organization is important for varioustranscriptome analyses and for the prediction of genes’ functions.Many methods that predict transcription units use correlation coecients, denoted by r, betweengenes, which are calculated from DNA microarray data. Because the r’s distribution of operon pairs(OPs) is di erent from that of non-operon pairs (NOPs), OPs and NOPs can be discriminated basedon r[2]. For example, Bockhorst et al.  proposed a method with a Bayesian network to link manykinds of observations such as spacer sizes, expression pro les, and codon usage. Tjaden et al.  useda hidden Markov model (HMM) to predict transcription boundaries.Although these methods utilize correlations between only adjacent genes on a single DNA strand,a pair of genes that are not immediately next to each other can be an operon pair when the membersof the operon are more than two. In this report, we propose a new method for the operon prediction,which utilizes correlations between not only adjacent but also distant genes on a DNA strand.
Prior Hyperparameters in Bayesian PCA

Bayesian PCA (BPCA) provides a Bayes inference for probabilistic PCA, in which several prior distributions have been devised; for example, automatic relevance determination (ARD) is used for determining the dimensionality. However, there is arbitrariness in prior setting; different prior settings result in different estimations. This article aims at presenting a standard setting of prior distribution for BPCA. We first define a general hierarchical prior for BPCA and show an exact predictive distribution. We show that several of the already proposed priors can be regarded as special cases of the general prior. By comparing various priors, we show that BPCA with nearly non-informative hierarchical priors exhibits the best performance.
A Bayesian missing value estimation method for gene expression profile data

Gene expression profile analyses have been used in numerous studies covering a broad range of areas in biology. When unreliable measurements are excluded, missing values are introduced in gene expression profiles. Although existing multivariate analysis methods have difficulty with the treatment of missing values, this problem has received little attention. There are many options for dealing with missing values, each of which reaches drastically different results. Ignoring missing values is the simplest method and is frequently applied. This approach, however, has its flaws. In this article, we propose an estimation method for missing values, which is based on Bayesian principal component analysis (BPCA). Although the methodology that a probabilistic model and latent variables are estimated simultaneously within the framework of Bayes inference is not new in principle, actual BPCA implementation that makes it possible to estimate arbitrary missing variables is new in terms of statistical methodology.



When applied to DNA microarray data from various experimental conditions, the BPCA method exhibited markedly better estimation ability than other recently proposed methods, such as singular value decomposition and K-nearest neighbors. While the estimation performance of existing methods depends on model parameters whose determination is difficult, our BPCA method is free from this difficulty. Accordingly, the BPCA method provides accurate and convenient estimation for missing values.


AVAILABILITY
The software is available at http://hawaii.aist-nara.ac.jp/~shige-o/tools/.
Variational Bayes method for Mixture of Principal Component Analyzers

When dealing with high-dimensional data, it is proper to construct an appropriate feature space consisting of relevant features extracted from given data. Mixture of Principal Component Analyzers (MPCA) is a feature extracting tool in which clustering in the data space and principal component analysis in each cluster are simultaneously done. A probabilistic model of MPCA and its maximum likelihood (ML) inference were presented by Tipping and Bishop (1999). Although the probabilistic formulation can be satisfactorily advantageous within Bayesian inference instead of ML inference, Bayesian inference often requires difficult integrations. Such difficulty comes to be overcome by the variational Bayes (VB) approximation that has recently been developed. In this paper, we present a VB inference algorithm for MPCA, and its application to a recognition problem of handwritten digit images.
Significance Test of Clusters in Gene Expression Profile Data

Clustering is a popular technique used in gene expression analyses. However, conventionally well-used clustering methods often produce variety in results depending on, e.g., initial algorithmic condition. Due to this instability, analysis results may be questionable. For example , although B-cell lymphomas had been classified into two distinct classes using hierarchical clustering, another analysis by a different clustering method failed to find these classes [1, 2]. In spite of such instability, the preprocessing or visualization of data by a clustering technique is still important for gene expression analyses. For utilizing technique with certain reliability, it is required to develop not only a stable method but also a method to evaluate quantitatively the reliability of the clustering results. In this report, we propose a quantitative evaluation method of clusters, which determines whether a data group, seemingly a cluster, constructs a statistically significant cluster. In particular, we propose a statistical test examining the existence possibility of a cluster within a parametric model of background noise. A seeming cluster is expressed by a window function represented by characteristic parameters. This statistical test can be extended to examining of feature spaces; we also propose a significance test of feature subspace extracted from PCA analysis.
Identification of expressed genes linked to malignancy of human colorectal carcinoma by parametric clustering of quantitative expression data

BackgroundIndividual human carcinomas have distinct biological and clinical properties: gene-expression profiling is expected to unveil the underlying molecular features. Particular interest has been focused on potential diagnostic and therapeutic applications. Solid tumors, such as colorectal carcinoma, present additional obstacles for experimental and data analysis.ResultsWe analyzed the expression levels of 1,536 genes in 100 colorectal cancer and 11 normal tissues using adaptor-tagged competitive PCR, a high-throughput reverse transcription-PCR technique. A parametric clustering method using the Gaussian mixture model and the Bayes inference revealed three groups of expressed genes. Two contained large numbers of genes. One of these groups correlated well with both the differences between tumor and normal tissues and the presence or absence of distant metastasis, whereas the other correlated only with the tumor/normal difference. The third group comprised a small number of genes. Approximately half showed an identical expression pattern, and cancer tissues were classified into two groups by their expression levels. The high-expression group had strong correlation with distant metastasis, and a poorer survival rate than the low-expression group, indicating possible clinical applications of these genes. In addition to c-yes, a homolog of a viral oncogene, prognostic indicators included genes specific to glial cells, which gives a new link between malignancy and ectopic gene expression.ConclusionsThe malignancy of human colorectal carcinoma is correlated with a unique expression pattern of a specific group of genes, allowing the classification of tumor tissues into two clinically distinct groups.
Clustering of Gene Expression Data by Mixture of PCA Models

Clustering techniques, such as hierarchical clustering, k- means algorithm and self-organizing maps, are widely used to analyze gene expression data. Results of these algorithms depend on several parameters, e.g., the number of clusters. However, there is no theoretical criterion to determine such parameters. In order to overcome this problem, we propose a method using mixture of PCA models trained by a variational Bayes (VB) estimation. In our method, good clustering results are selected based on the free energy obtained within the VB estimation. Furthermore, by taking an ensemble of estimation results, a robust clustering is achieved without any biological knowledge. Our method is applied to a clustering problem for gene expression data during a sporulation of Bacillus subtilis and it is able to capture characteristics of the sigma cascade.