nils-herrmann/clean_scirepeval huggingface dataset:
Constructing shared-state hidden Markov models based on a Bayesian approach

In this paper, we propose a method for constructing sharedstate triphone HMMs (SST-HMMs) within a practical Bayesian framework. In our method, Bayesian model selection criterion is derived for SST-HMM based on the Variational Bayesian approach. The appropriate phonetic decision tree structure of SST-HMM is found by using the criterion according to a given data set. This criterion, unlike the conventional MDL criterion, is applicable even in the case of insufficient amounts of data. We conduct two experiments on speaker independent word recognition in order to prove the effectiveness of the proposed method. The first experiment demonstrates that the Bayesian approach is valid for determining the tree structure. The second experiment demonstrates that the Bayesian criterion can design SST-HMMs with higher recognition performance than the MDL criterion when dealing with small amounts of data.
Modeling of growing networks with communities

We propose a growing network model and its learning algorithm. Unlike the conventional scale-free models, we incorporate community structure, which is an important characteristic of many real-world networks including the Web. In our experiments, we confirmed that the proposed model exhibits a degree distribution with a power-law tail, and our method can precisely estimate the probability of a new link creation from data without community information. Moreover, by introducing a measure of dynamic hub-degrees, we could predict the change of hub-degrees between communities.
Single-shot detection of multiple categories of text using parametric mixture models

In this paper, we address the problem of detecting multiple topics or categories of text where each text is not assumed to belong to one of a number of mutually exclusive categories. Conventionally, the binary classification approach has been employed, in which whether or not text belongs to a category is judged by the binary classifier for every category. In this paper, we propose a more sophisticated approach to simultaneously detect multiple categories of text using parametric mixture models (PMMs), newly presented in this paper. PMMs are probabilistic generative models for text that has multiple categories. Our PMMs are essentially different from the conventional mixture of multinomial distributions in the sense that in the former several basis multinomial parameters are mixed in the parameter space, while in the latter several multinomial components are mixed. We derive efficient learning algorithms for PMMs within the framework of the maximum a posteriori estimate. We also empirically show that our method can outperform the conventional binary approach when applied to multitopic detection of World Wide Web pages, focusing on those from the "yahoo.com" domain.
Parametric Mixture Models for Multi-Labeled Text

We propose probabilistic generative models, called parametric mixture models (PMMs), for multiclass, multi-labeled text categorization problem. Conventionally, the binary classification approach has been employed, in which whether or not text belongs to a category is judged by the binary classifier for every category. In contrast, our approach can simultaneously detect multiple categories of text using PMMs. We derive efficient learning and prediction algorithms for PMMs. We also empirically show that our method could significantly outperform the conventional binary methods when applied to multi-labeled text categorization using real World Wide Web pages.
Application of Variational Bayesian Approach to Speech Recognition

In this paper, we propose a Bayesian framework, which constructs shared-state triphone HMMs based on a variational Bayesian approach, and recognizes speech based on the Bayesian prediction classification; variational Bayesian estimation and clustering for speech recognition (VBEC). An appropriate model structure with high recognition performance can be found within a VBEC framework. Unlike conventional methods, including BIC or MDL criterion based on the maximum likelihood approach, the proposed model selection is valid in principle, even when there are insufficient amounts of data, because it does not use an asymptotic assumption. In isolated word recognition experiments, we show the advantage of VBEC over conventional methods, especially when dealing with small amounts of data.
HMMs for both labeled and unlabeled time series data

An insufficiency of training data often results in a poorly learned classifier. To mitigate this problem, several learning methods using both labeled and unlabeled data have been proposed. In these methods, however, only static data are considered; time series unlabeled data cannot be utilized. In this paper, we first present an extension of HMMs, named Extended Tied-Mixture HMMs (ETM-HMMs) in which both labeled and unlabeled time series data can be used simultaneously to obtain a better classification accuracy than the case only labeled data are used. The learning algorithm for the ETM-HMMs is also presented. Experiments on synthetic and gesture data demonstrated that unlabeled time series data can help improve the classification performance.
SMEM algorithm for mixture regression problems

We investigate applications of the SMEM (Split and Merge EM) algorithm to regression problems. We use the NGnet (Normalized Gaussian network) as a mixture regression model in which a set of input and output variables is dealt with as a joint probability. We describe the split and merge operations and criteria necessary for applying the SMEM algorithm to the NGnet. We show the usefulness of the SMEM algorithm for regression problems through experiments using artificial and real data.
Optimal Linear Combination of Neural Networks for Improving Classification Performance

This paper presents a new method for linearly combining multiple neural network classifiers based on the statistical pattern recognition theory. In our approach, several neural networks are first selected based on which works best for each class in terms of minimizing classification errors. Then, they are linearly combined to form an ideal classifier that exploits the strengths of the individual classifiers. In this approach, the minimum classification error criterion is utilized to estimate the optimal linear weights. In this formulation, because the classification decision rule is incorporated into the cost function, a more suitable better combination of weights for the classification objective could be obtained. Experimental results using artificial and real data sets show that the proposed method can construct a better combined classifier that outperforms the best single classifier in terms of overall classification errors for test data.
EM algorithm with split and merge operations for mixture models

The maximum-likelihood estimate of a mixture model is usually found by using the EM algorithm. However, the EM algorithm suffers from the local-optimum problem and therefore we cannot obtain the potential performance of mixture models in practice. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split and merge operations using a new criterion for efficiently selecting the split and merge candidates. We apply the proposed algorithm to the training of Gaussian mixtures and the dimensionality reduction based on a mixture of factor analyzers using synthetic and real data and show the effectiveness of using the split and merge operations to improve the likelihood both of the training data and of reserved test data.
Law discovery from financial data using neural networks

We describe an experimental study for discovering underlying laws of market capitalization using BS (Balance Sheet) items. For this purpose, we apply law discovery methods based on neural networks: RF5 (Rule Finder) discovers a single numeric law from data containing only numeric values, RF6 discovers a set of nominally conditioned polynomials from data containing both nominal and numeric values, and MCV regularizer is used to improve both the generalization performance and the readability. Our preliminary experimental results show that these methods are promising for discovering underlying laws from financial data.
Adaptive clustering method using modular learning architecture

あらまし クラス内のデータが連続的に分布しクラス間で重なりがない場合において,クラス数未知の状態から逐次処理によりデー タをクラス分けする手法を提案する.提案する手法はモジュール競合ネットワークに統計力学のアナロジーを用いる手法 である.即ち,入力データを粒子と見なし,粒子の分布に適したエネルギー状態を各モジュールが推定することにより学 習を行なう手法である.この学習の結果,連続的に分布する粒子は同一のモジュールに振り分けられ,同時に,粒子の存 在しないモジュールも現れる.従って,予めモジュール数を多めに用意し,学習後,粒子の集まったモジュールを1つの クラスと見なすことで,入力データのクラス分けとクラス数推定が可能となる.本論文では提案手法の詳細と,2次元人 工データ及び顔画像を用いた実験結果を示す. キーワード クラスタリング,教師無し学習,競合学習,逐次処理,自由エネルギー,統計力学
A Model for Self-Organization of Feature Columns using Modular Structure

We propose a computational model for the self-organization of feature columns based on a modular framework. The proposed model consists of two-stage networks; each network is composed of several modules each of which consists of a collection of Gaussian units. Computer simulation shows that the ability of each network to find out continuity among input patterns forces feature columns to self-organize into modules and classifies objects by combining the created feature columns. 1 はじめに 近年、物体認識に深く関係すると考えられている大脳 皮質下側頭部が特定の複雑なパターンに反応するコラム 構造を持つことが生理実験により明らかにされた 。 この結果は脳が複雑な特徴を抽出し、それらを組み合 わせることで物体の認識を行っている可能性を示してい る。計算論的アプローチではKohonen [3]、田中 [6]等 が初期視覚における自己組織化モデルを提案している。 しかしながら、認識などに直接関わる高次視覚機能に おける自己組織化モデルはまだほとんど検討されてい ない。本稿ではKohonen [3]の自己組織化モデルをモ ジュール化および階層化することにより部分画像からの 特徴のコラム化とそれらのコラムを利用して物体の類別 を行うモデルを提案する。計算機実験では初期視覚野か らの入力を想定し、画像に方向成分のフィルタをかけた ものを用いてモデルの有効性を示した。 2 モデル 提案するモデルは図1に示す様に、上位の1つのネッ トワーク(NW)と下位の4つのNWから成る。各NW は同一の構造であり、モジュール構造を持っている。こ こで、各モジュールはKohonen [3]の自己組織化モデル に等しい。図1に示した例ではNWは2つのモジュール から成り、各モジュールはリング状に並んだ10個のガ ウシアン・ユニットから成る。 2.1 各ネットワークの構成と学習 NWに入力 xが与えられるとNW内のすべてのユニッ ト(U 個)が同じ入力を受取り、第 iモジュールの第 j ユニットは yj|i = cj|i hj|i , (1) を出力する。ここで、 cj|i は第 iモジュール第 jユニッ トのガウシアンの平均ベクトルである。 hj|i は同ユニッ トの2乗誤差 ∥∥x− cj|i ∥∥2 のソフトマックス関数であ り、 hj|i = exp [ −‖x−cj|i‖2/2σ2] U ∑ j=1 exp [ −‖x−cj|i‖2/2σ2] , (2) 1キーワード:特徴コラム,自己組織化,物体認識,モジュール構造 The lower networks The higher network
Self-Organization of Feature Columns and its Application to Object Classification

We propose a computational model for the self-organization of feature columns based on a modular framework. The proposed model consists of several modules; each module is composed of a collection of Gaussian units. The combination of hierarchical competition within and between modules and a smoothness constraint nds out continuity among input patterns, and topographically maps these series of patterns into dierent modules. Computer simulations show an example of self-organization and object classication by combining the created feature columns.
Combining discriminant-based classifiers using the minimum classification error discriminant

Focusing on classification problems, this paper presents a new method for linearly combining discriminant-based classifiers to improve classification performance, in the sense of the minimum classification errors. In our approach, the problem of estimating linear weights in combination is reformulated as the problem of designing a linear discriminant function using the minimum classification error discriminant. In this formulation, because the classification decision rule is incorporated into the cost function, better combination weights suitable for the classification objective can be obtained. Experimental results using neural network classifiers support the effectiveness of the proposed method.
Generalization error of ensemble estimators

It has been empirically shown that a better estimate with less generalization error can be obtained by averaging outputs of multiple estimators. This paper presents an analytical result for the generalization error of ensemble estimators. First, we derive a general expression of the ensemble generalization error by using factors of interest (bias, variance, covariance, and noise variance) and show how the generalization error is affected by each of them. Some special cases are then investigated. The result of a simulation is shown to verify our analytical result. A practically important problem of the ensemble approach, ensemble dilemma, is also discussed.
Tracking Moving Contours Using Energy-Minimizing Elastic Contour Models

This paper proposes a robust method for tracking an object contour in a sequence of images. In this method, both object extraction and tracking problems can be solved simultaneously. Furthermore, it is applicable to the tracking of arbitrary shapes since it does not need a priori knowledge about the object shapes. In the contour tracking, energy-minimizing elastic contour models are utilized, which is newly presented in this paper. The contour tracking is formulated as an optimization problem to find the position that minimizes both the elastic energy of its model and the potential energy derived from the edge potential image that includes a target object contour. We also present an algorithm which efficiently solves energy minimization problems within a dynamic programming framework. The algorithm enables us to obtain optimal solution even when the variables to be optimized are not ordered. We show the validity and usefulness of the proposed method with some experimental results.
Parrot-like speaking using optimal vector quantization

Parrot-like speaking can be considered as one of the most fundamental abilities of humans or robots. It is not a transformation of a target speech signal, but a perception-and-action process: recognizing the target speech and producing a mimic one using a voice obtained from a voice owner. This paper presents a connectionist parrot-like speaking system. Our approach employs the record-and-edit approach with an acoustic wave segment as the processing unit, and uses a vector quantizer for two purposes: to build a segment database as a natural voice of a robot, and to cluster the segment database to speed up the mimicking. The experimental parrot system works mostly well, mimicking any target speech and sounding like a voice owner.
Competitive and selective learning method for vector quantizer design - Equidistortion principle and its algorithm

A new competitive learning approach for optimal vector quantizer is presented. First, it is shown that the original Competitive Learning (CL) algorithm can be derived from the two necessary conditions forming the basis of the traditional nonconnectionist vector quantizer (VQ) design algorithm called the LBG algorithm. It is then shown that the conventional conscience principle or equiprobable principle is not optimal from the standpoint of the minimization of the expected dis-tortion. Next, a basic principle is introduced. Called the equidistortion principle (a necessary condition for optimal vector quantizers) it is derived by using Gersho's asymptotic theory. It is shown to be applicable to distributions consisting of disjoint clusters. Then a new competitive learning algorithm with a selection mechanism, called the competitive and selective learning (CSL) algorithm and based on the equidistortion principle, is proposed here. Since the selection mechanism enables the system to escape from local minima, the proposed algorithm can obtain better performance without a particular initialization procedure even when the input data clusters in a number of regions in the input vector space. In a synthetic one-dimensional quantizer problem in which the optimum quantizer can be numerically obtained, less than 2% of the minimum distortion has been found which had not been obtained previously by con-ventional algorithms.
Estimating expected error rates of neural network classifiers in small sample size situations: a comparison of cross-validation and bootstrap

We compare the cross-validation and bootstrap methods for estimating the expected error rates of feedforward neural network classifiers in small sample size situations. The cross-validation method, a commonly applied method, provides nearly unbiased classification error rates, using only the original samples. The cross-validated estimates, however, may suffer from a large variance. In this paper, we apply a statistical resampling technique, called the bootstrap method, to this estimation problem and compare the performances of these methods. Our results show that the variance of the bootstrap estimates can be smaller than those of the cross-validated estimates.
Multi-armed Bandit Algorithms and Empirical Evaluation

The multi-armed bandit problem for a gambler is to decide which arm of a K-slot machine to pull to maximize his total reward in a series of trials. Many real-world learning and optimization problems can be modeled in this way. Several strategies or algorithms have been proposed as a solution to this problem in the last two decades, but, to our knowledge, there has been no common evaluation of these algorithms. 
 
This paper provides a preliminary empirical evaluation of several multi-armed bandit algorithms. It also describes and analyzes a new algorithm, Poker (Price Of Knowledge and Estimated Reward) whose performance compares favorably to that of other existing algorithms in several experiments. One remarkable outcome of our experiments is that the most naive approach, the e-greedy strategy, proves to be often hard to beat.
Finite-state rate-distortion for individual sequences

We introduce a class of lossy finite-state machines for lossy compression of an individual sequence drawn from a finite alphabet at a fixed distortion, and define a fundamental quantity finite-state rate-distortion that is an asymptotically attainable lower bound on the compression rate of any lossy finite-state machine. For Hamming distortion, we obtain a universal lower bound on the finite-state rate-distortion of any individual sequence.
A Cost-Shaping LP for Bellman Error Minimization with Performance Guarantees

We introduce a new algorithm based on linear programming that approximates the differential value function of an average-cost Markov decision process via a linear combination of pre-selected basis functions. The algorithm carries out a form of cost shaping and minimizes a version of Bellman error. We establish an error bound that scales gracefully with the number of states without imposing the (strong) Lyapunov condition required by its counterpart in . We propose a path-following method that automates selection of important algorithm parameters which represent counterparts to the "state-relevance weights" studied in .
Exploration-Exploitation Tradeoffs for Experts Algorithms in Reactive Environments

A reactive environment is one that responds to the actions of an agent rather than evolving obliviously. In reactive environments, experts algorithms must balance exploration and exploitation of experts more carefully than in oblivious ones. In addition, a more subtle definition of a learnable value of an expert is required. A general exploration-exploitation experts method is presented along with a proper definition of value. The method is shown to asymptotically perform as well as the best available expert. Several variants are analyzed from the viewpoint of the exploration-exploitation tradeoff, including explore-then-exploit, polynomially vanishing exploration, constant-frequency exploration, and constant-size exploration phases. Complexity and performance bounds are proven.
Learning and teaching in repeated games: A machine learning approach to long-term best-response play

Repeated games and long-term best-response play. We consider the problem of learning during a play of a a repeated game. Existing approaches often focus on algorithms that asymptotically achieve an equilibrium of the one-shot game. We shift away from that paradigm to consider learning strategies that attempt to play long-term best-response to the opponent’s observed strategy. This goal may be more appropriate for several reasons:
How to Combine Expert (and Novice) Advice when Actions Impact the Environment?

The so-called "experts algorithms" constitute a methodology for choosing actions repeatedly, when the rewards depend both on the choice of action and on the unknown current state of the environment. An experts algorithm has access to a set of strategies ("experts"), each of which may recommend which action to choose. The algorithm learns how to combine the recommendations of individual experts so that, in the long run, for any fixed sequence of states of the environment, it does as well as the best expert would have done relative to the same sequence. This methodology may not be suitable for situations where the evolution of states of the environment depends on past chosen actions, as is usually the case, for example, in a repeated non-zero-sum game. 
 
A new experts algorithm is presented and analyzed in the context of repeated games. It is shown that asymptotically, under certain conditions, it performs as well as the best available expert. This algorithm is quite different from previously proposed experts algorithms. It represents a shift from the paradigms of regret minimization and myopic optimization to consideration of the long-term effect of a player's actions on the opponent's actions or the environment. The importance of this shift is demonstrated by the fact that this algorithm is capable of inducing cooperation in the repeated Prisoner's Dilemma game, whereas previous experts algorithms converge to the suboptimal non-cooperative play.
The Linear Programming Approach to Approximate Dynamic Programming

The curse of dimensionality gives rise to prohibitive computational requirements that render infeasible the exact solution of large-scale stochastic control problems. We study an efficient method based on linear programming for approximating solutions to such problems. The approach "fits" a linear combination of pre-selected basis functions to the dynamic programming cost-to-go function. We develop error bounds that offer performance guarantees and also guide the selection of both basis functions and "state-relevance weights" that influence quality of the approximation. Experimental results in the domain of queueing network control provide empirical support for the methodology.
On constraint sampling in the linear programming approach to approximate linear programming

In the linear programming approach to approximate dynamic programming, one tries to solve a certain linear program - the ALP -, which has a relatively small number K of variables but an intractable number M of constraints. In this paper, we study a scheme that samples and imposes a subset of m /spl Lt/ M constraints. A natural question that arises in this context is: How must m scale with respect to K and M in order to ensure that the resulting approximation is almost as good as one given by exact solution of the ALP? We show that, under certain idealized conditions, m can be chosen independently of M and need grow only as a polynomial in K.