nils-herrmann/clean_scirepeval huggingface dataset:
Variational methods for the Dirichlet process

Variational inference methods, including mean field methods and loopy belief propagation, have been widely used for approximate probabilistic inference in graphical models. While often less accurate than MCMC, variational methods provide a fast deterministic approximation to marginal and conditional probabilities. Such approximations can be particularly useful in high dimensional problems where sampling methods are too slow to be effective. A limitation of current methods, however, is that they are restricted to parametric probabilistic models. MCMC does not have such a limitation; indeed, MCMC samplers have been developed for the Dirichlet process (DP), a nonparametric distribution on distributions (Ferguson, 1973) that is the cornerstone of Bayesian nonparametric statistics (Escobar & West, 1995; Neal, 2000). In this paper, we develop a mean-field variational approach to approximate inference for the Dirichlet process, where the approximate posterior is based on the truncated stick-breaking construction (Ishwaran & James, 2001). We compare our approach to DP samplers for Gaussian DP mixture models.
Probabilistic models of text and images

Managing large and growing collections of information is a central goal of modern computer science. Data repositories of texts, images, sounds, and genetic information have become widely accessible, thus necessitating good methods of retrieval, organization, and exploration. In this thesis, we describe a suite of probabilistic models of information collections for which the above problems can be cast as statistical queries. 
We use directed graphical models as a flexible, modular framework for describing appropriate modeling assumptions about the data. Fast approximate posterior inference algorithms based on variational methods free us from having to specify tractable models, and further allow us to take the Bayesian perspective, even in the face of large datasets. 
With this framework in hand, we describe latent Dirichlet allocation (LDA), a graphical model particularly suited to analyzing text collections. LDA posits a finite index of hidden topics which describe the underlying documents. New documents are situated into the collection via approximate posterior inference of their associated index terms. Extensions to LDA can index a set of images, or multimedia collections of interrelated text and images. 
Finally, we describe nonparametric Bayesian methods for relaxing the assumption of a fixed number of topics, and develop models based on the natural assumption that the size of the index can grow with the collection. This idea is extended to trees, and to models which represent the hidden structure and content of a topic hierarchy that underlies a collection.
Modeling annotated data

We consider the problem of modeling annotated data---data with multiple types where the instance of one type (such as a caption) serves as a description of the other type (such as an image). We describe three hierarchical probabilistic mixture models which aim to describe such data, culminating in correspondence latent Dirichlet allocation, a latent variable model that is effective at modeling the joint distribution of both types and the conditional distribution of the annotation given the primary type. We conduct experiments on the Corel database of images and captions, assessing performance in terms of held-out likelihood, automatic annotation, and text-based image retrieval.
Variational inference in a truncated Dirichlet process

The N -component truncated Dirichlet process (DPN ) is defined in Ishwaran and James [2001] and converges almost surely to a true Dirichlet process (DP∞). Like a full Dirichlet process, this distribution can be used as a nonparametric Bayesian prior in a mixture model. Ishwaran and James show that this approximation allows a blocking strategy in the corresponding Gibbs sampler which can be faster than the classical Gibbs samplers developed for the DP∞ prior [Escobar and West, 1995]. In this paper, we develop a variational inference algorithm to approximate the posterior in a Bayesian mixture model with a DPN prior. An exponential family mixture model with DPN prior on the natural parameter of the mixture component is illustrated in Figure 2. The random variables are distributed as follows:
Topic segmentation with an aspect hidden Markov model

We present a novel probabilistic method for topic segmentation on unstructured text. One previous approach to this problem utilizes the hidden Markov model (HMM) method for probabilistically modeling sequence data . The HMM treats a document as mutually independent sets of words generated by a latent topic variable in a time series. We extend this idea by embedding Hofmann's aspect model for text  into the segmenting HMM to form an aspect HMM (AHMM). In doing so, we provide an intuitive topical dependency between words and a cohesive segmentation model. We apply this method to segment unbroken streams of New York Times articles as well as noisy transcripts of radio programs on SpeechBot, an online audio archive indexed by an automatic speech recognition engine. We provide experimental comparisons which show that the AHMM outperforms the HMM for this task.
Deep learning?

Unsupervised multilayered ("deep") models are considered for imagery. The model is represented using a hierarchical convolutional factor-analysis construction, with sparse factor loadings and scores. The computation of layer-dependent model parameters is implemented within a Bayesian setting, employing a Gibbs sampler and variational Bayesian (VB) analysis that explicitly exploit the convolutional nature of the expansion. To address large-scale and streaming data, an online version of VB is also developed. The number of dictionary elements at each layer is inferred from the data, based on a beta-Bernoulli implementation of the Indian buffet process. Example results are presented for several image-processing applications, with comparisons to related models in the literature. Index Terms—Bayesian, deep learning, convolutional, dictionary learning, factor analysis
Finding roots of arbitrary high order polynomials based on neural network recursive partitioning method

This paper proposes a novel recursive partitioning method based on constrained learning neural networks to find an arbitrary number (less than the order of the polynomial) of (real or complex) roots of arbitrary polynomials. Moreover, this paper also gives a BP network constrained learning algorithm (CLA) used in root-finders based on the constrained relations between the roots and the coefficients of polynomials. At the same time, an adaptive selection method for the parameter δP with the CLA is also given. The experimental results demonstrate that this method can more rapidly and effectively obtain the roots of arbitrary high order polynomials with higher precision than traditional root-finding approaches.
Document Image Binarization With Feedback For Improving Character Segmentation

Binarization of gray scale document images is one of the most important steps in automatic document image processing. In this paper, we present a two-stage document image binarization approach, which includes a top-down region-based binarization at the first stage and a neural network based binarization technique for the problematic blocks at the second stage after a feedback checking. Our two-stage approach is particularly effective for binarizing text images of highlighted or marked text. The region-based binarization method is fast and suitable for processing large document images. However, the block effect and regional edge noise are two unavoidable problems resulting in poor character segmentation and recognition. The neural network based classifier can achieve good performance in two-class classification problem such as the binarization of gray level document images. However, it is computationally costly. In our two-stage binarization approach, the feedback criteria are employed to keep the well binarized blocks from the first stage binarization and to re-binarize the problematic blocks at the second stage using the neural network binarizer to improve the character segmentation quality. Experimental results on a number of document images show that our two-stage binarization approach performs better than the single-stage binarization techniques tested in terms of character segmentation quality and computational cost.
Genetic evolution processing of data structures for image classification

This paper describes a method of structural pattern recognition based on a genetic evolution processing of data structures with neural networks representation. Conventionally, one of the most popular learning formulations of data structure processing is backpropagation through structures (BPTS) [C. Goller et al., (1996)]. The BPTS algorithm has been successfully applied to a number of learning tasks that involved structural patterns such as image, shape, and texture classifications. However, this BPTS typed algorithm suffers from the long-term dependency problem in learning very deep tree structures. In this paper, we propose the genetic evolution for this data structures processing. The idea of this algorithm is to tune the learning parameters by the genetic evolution with specified chromosome structures. Also, the fitness evaluation as well as the adaptive crossover and mutation for this structural genetic processing are investigated in this paper. An application to flowers image classification by a structural representation is provided for the validation of our method. The obtained results significantly support the capabilities of our proposed approach to classify and recognize flowers in terms of generalization and noise robustness.
Zeroing polynomials using modified constrained neural network approach

This paper proposes new modified constrained learning neural root finders (NRFs) of polynomial constructed by backpropagation network (BPN). The technique is based on the relationships between the roots and the coefficients of polynomial as well as between the root moments and the coefficients of the polynomial. We investigated different resulting constrained learning algorithms (CLAs) based on the variants of the error cost functions (ECFs) in the constrained BPN and derived a new modified CLA (MCLA), and found that the computational complexities of the CLA and the MCLA based on the root-moment method (RMM) are the order of polynomial, and that the MCLA is simpler than the CLA. Further, we also discussed the effects of the different parameters with the CLA and the MCLA on the NRFs. In particular, considering the coefficients of the polynomials involved in practice to possibly be perturbed by noisy sources, thus, we also evaluated and discussed the effects of noises on the two NRFs. Finally, to demonstrate the advantage of our neural approaches over the nonneural ones, a series of simulating experiments are conducted.
Locating Human Eyes Using Edge and Intensity Information

In this paper, a new eye detection method is presented. The method consists of three steps: (1) extraction of binary edge image (BEI) based on the multi-resolution analysis of wavelet transform; (2) extraction of eye region and segments from BEI, and (3) eye localization using light dot or intensity information. An improved face region extraction algorithm and a light dot detection method are proposed to improve eye detection performance. Experimental results show that our approach can achieve a correct eye detection rate of 98.7% on 150 Bern images with variations in view and gaze direction and a rate of 96.6% on 564 AR images with different facial expressions and lighting conditions.
Scene Classification Using Adaptive Processing of Tree Representation of Rectangular-Shape Partition of Images

Image classification is very helpful for organizing large image databases and content based image retrieval (CBIR). However, it is very complex and challenging because of lacking effective methods. In this paper, we present a tree representation of images based on rectangular-shape partition. Then an adaptive processing algorithm is adopted to perform the classification task. Experimental results on seven categories of scenery images show that the structural representations are better than the traditional methods and our previous work based on quadtree representation of fixed partition.
MR Brain Image Segmentation Based on Self-Organizing Map Network

Magnetic resonance imaging (MRI) is an advanced medical imaging technique providing rich information about the human soft tissue anatomy. The goal of magnetic resonance (MR) image segmentation is to accurately identify the principal tissue structures in these image volumes. A new unsupervised MR image segmentation method based on self-organizing feature map (SOFM) network is presented. The algorithm includes spatial constraints by using a Markov Random Field (MRF) model. The MRF term introduces the prior distribution with clique potentials and thus improves the segmentation results without having extra data samples in the training set or a complicated network structure. The simulation results demonstrate that the proposed algorithm is promising.
Bark texture feature extraction based on statistical texture analysis

This paper quantitatively describes and discusses the usefulness of texture analysis methods for the recognition of bark. Comparative studies of bark texture feature extraction are performed for the four texture analysis methods such as the gray level run-length method (RLM), co-occurrence matrices method (COMM) and histogram method (HM) as well as auto-correlation method (ACM). Specifically, we use three classifiers of nearest neighbor (l-NN), k-nearest neighbor (k-NN) and moving median centers (MMC) hypersphere classifiers to verify the validity of the extracted bark texture features. To gain good results we added to color information that proved very efficient. Moreover, the experimental results also demonstrate that from the viewpoint of the recognition accuracy and computational complexity, the COMM method is superior to the other three methods.
B-spline over-complete wavelet based fractal signature analysis for texture image retrieval

In the paper, we propose novel overcomplete B-spline wavelet based statistical features and fractal signatures for texture image analysis and retrieval. The discrete wavelet frame takes the first order derivative of the smoothing function into account, which is equivalent to Canny edge detection, with the specific case using a Gaussian function as smoothing function. Meanwhile, the feature set based on the fractal surface area function in a Besov space is very accurate and robust for gray scale texture classification. Experimental results have shown that the proposed method is reasonable for describing the characteristics of the texture in temporal-frequency and fractal domain and it can reach the highest retrieval rate compared with the Gabor filter based feature descriptor and B-spline overcomplete wavelet transformation based feature representation only.
Automatic generation of pen-and-ink drawings from photos

In this paper, a new algorithm for automatic generating pen-and-ink drawings from photos is presented. The basic processing step of our system is the edge extraction based on the multiresolution property of wavelet transform. In order to enhance the variations in texture and tone, the proposed algorithm includes two binarization steps and a noise removing step. Experimental results show that the generated pen-and-ink drawings are of high quality, which accurately illustrate the scenic contents of the original photos.
erived Fuzzy ize zzification for Handwritt Numeral Recognition

In this paper, we present a technique to produce fuzzy rules based on the ID3 approach and to optimize defuzzification parameters by using a two-layer perceptron. The technique overcomes the difficulties in a conventional syntactic approach to handwritten Character recognition, including problems of choosing a starting or reference point, scaling, and learning by machines. Our technique provides us with: a way to produce meaningful and simple fuzzy rules; a method to fuzzify ID3-derived rules to deal with uncertain, noisy, or fuzzy data; and a framework to incorporate fuzzy rules learned from the training data and those extracted from human recognition experience. Our experimental results on NIST Special Database 3 show that the technique out-performs the straight forward ID3 approach. Moreover, ID3-derived fuzzy rules can be combined with an optimized nearest neighbor classifier, which uses intensity features only, to achieve a better classification performance than either of the classifiers. The combined classifier achieves a correct classification rate of 98.6% on the test set.
Texture image segmentation based on entropy theory

A two-stage segmentation algorithm for textured object image is proposed based on variational approaches and entropy theory. Structure tensor of the image is adopted as the texture feature for the discrimination of texture patterns. In order to make the four information channels cooperatively push the evolving contour towards the textured foreground object, a pre-processing based on mean shift algorithm is applied, which can implement edge-preserving smoothing and has a definite stopping criterion. The Dream/sup 2/s framework is extended to the structure tensor data in the first segmentation stage in which the differential entropy is adopted as regional descriptors, and Gaussian distribution for both the foreground part and the background part is assumed. In the second stage, the nonparametric density distributions are exploited, and the mutual information between the intensity random variables of different information channels and the binary label random variable is adopted as the criterion to minimize. The nonparametric representation can describe the truly nongaussianity of data, and thus can refine the initial segmentation result obtained from the first stage.
Bark classification by combining grayscale and binary texture features

In this paper, a texture feature based bark classification method is presented. Our method uses two types of texture features: the co-occurrence matrix feature and the long connection length emphasis (LCLE) feature, which is extracted from the binary bark image. For the extraction of binary texture maps, an improved wavelet-based edge detection algorithm is proposed. It includes two binarization steps and a post-processing step. The paper also presents an approach to combine two feature sets. Experiments on 18 different tree species, and in total 90 bark images, show that a combination of these two feature sets can achieve a much higher bark classification rate than that when each feature set is utilized individually.
Computational Visual Sensitivity Models - a Review

The Human Visual Systems (HVSs) is imperfect and contains highly selective visual acquisition sensors. Not all useful information presented to human eyes can be perceived. Generally, a technique to determine whether a signal can be perceived and how well it can be perceived is called human visual sensitivity analysis. More than one hundred years of psychophysical research on HVSs has revealed that human visual sensitivity is not only determined by local characteristics of visual contents (luminance, contrast orientation, spatial and temporal frequency), but also global modulatory factors (visual attention and motion suppression). In this paper, we provide a review on various factors that affect human visual sensitivity and on various computational models for human visual sensitivity. A comparative study on the performance of various visual sensitivity models by simulations is also reported in the paper. Subjective evaluation on noise-embedded video sequences confirms that the introduction of global modulatory factors does improve the performance of the Just Noticeable Difference (JND) profile used in noise shaping.
Efficient Learning in Adaptive Processing of Data Structures

Many researchers have explored the use of neural network models for the adaptive processing of data structures. The learning formulation for one of the models is known as the Backpropagation Through Structure (BPTS) algorithm. The main limitations of the BPTS algorithm are attributed to the problems of slow convergence speed and long-term dependency. In this Letter, a novel heuristic algorithm is proposed. The idea of this algorithm is to optimize the free parameters of the node representation in data structure by using a hybrid type of learning algorithm. Encouraging results achieved demonstrate that this proposed algorithm outperforms the BPTS algorithm.
Split decision function using fuzzy integral [image coding applications]

The quadtree partition is often used in fractal image compression to achieve variable bit rates. In this paper, we propose a split decision function based on fuzzy integrals - fuzzy mean error (FME), that can improve image coding quality while speeding up the encoding process. Our proposed split function is compared with other commonly used split decisions. In the paper, we also demonstrate that it is difficult for a split decision function to obtain an a priori estimate of the threshold for encoding a given image at a given quality level.
Locatization of human eyes based on a series of binary images

A new method for eyes localization is presented. By incorporating some eye geometrical constraints, our method includes such procedures as eyes image segmentation, eye candidates selection and eyeball detection. In order to locate the eyes more precisely, the valley map transformed from the grayscale face image using an arithmetical morphology method is successively binarized with a series of threshold values determined adaptively, and many possible eyeball candidates are extracted from them. The final position of the two eyes is obtained using a statistical method. Experimental results on AR and Yale face databases show that a locating rate of over 94% and an average locating disparity of below 2 pixels can be achieved
Extraction of face image edges with application to expression analysis

This paper presents a new method for the extraction of face image edges and a scheme for facial expression analysis based on the binary edge images (BEIs). Making use of the multi-resolution property of wavelet transform, the edge detection method includes two binarization steps and a noise removing step. Our method provides a robust solution to face image edge extraction, in particular, if changes of lighting conditions are encountered. Experimental results show that different components in a BEI are well separated and pixels of the same face component are connected well. Preliminary experimental results on facial expression analysis on the subsets of AR and Yale face databases show that a satisfactory recognition rate can be achieved when our facial analysis scheme is used for recognizing four action units.