estimates of error probability for complex gaussian channels with generalized likelihood ratio detection we derive approximate expressions for the probability of error in a two class hypothesis testing problem in which the two hypotheses are characterized by zero mean complex gaussian distributions these error expressions are given in terms of the moments of the test statistic employed and we derive these moments for both the likelihood ratio test appropriate when class densities are known and the generalized likelihood ratio test appropriate when class densities must be estimated from training data these moments are functions of class distribution parameters which are generally unknown so we develop unbiased moment estimators in terms of the training data with these accurate estimates of probability of error can be calculated quickly for both the optimal and plug in rules from available training data we present a detailed example of the behavior of these estimators and demonstrate their application to common pattern recognition problems which include quantifying the incremental value of larger training data collections evaluating relative geometry in data fusion from multiple sensors and selecting a good subset of available features

modularity of simple termination of term rewriting systems with shared constructors abstract kurihara m and a ohuchi modularity of simple termination of term rewriting systems with shared constructors theoretical computer science 103 1992 273 282 a term rewriting system is simply terminating if there exists a simplification ordering showing its termination let r and r be term rewriting systems which share no defined symbol but may share constructors constructors are function symbols which do not occur at the leftmost position in left hand sides of rewrite rules the rest of the function symbols are defined symbols in this paper we prove that r u r is simply terminating if and only if both r and r are so 1 introduction a property p of term rewriting systems trss 4 6 is modular if r u r has the property p if and only if both r0 and r have that property starting with toyama 141 several authors studied modular aspects of trss under the disjointness condition which requires that

modularity in term rewriting revisited we revisit modularity in term rewriting which for the last 25 years has been a very active and fruitful research field starting with the pioneering works of yoshihito toyama on the modularity of confluence and the non modularity of termination he thus initiated an extremely productive line of research with many non trivial and deep results striking counterexamples and a substantial amount of systematic theoretical foundations methodological principles and novel proof techniques in this focused summary we will revisit the modularity analysis for ordinary term rewriting systems considering various confluence and termination properties and restricting ourselves mainly to the case of disjoint unions we will summarize known results on the non modularity of various confluence and termination properties and exhibit crucial ideas constructions and phenomena later on we will also briefly consider various extensions applications related questions and open problems as well as recent developments

relations between information criteria for model structure selection part 1 the role of bayesian model order estimation model structure estimation is first accommodated in the general framework of prediction error identification the concept of the best model structure is defined a theorem is given on bayesian estimation from which the strong consistency of well known information criteria with penalty terms is inferred

divergence based feature selection for multimodal class densities a new feature selection procedure based on the kullback j divergence between two class conditional density functions approximated by a finite mixture of parameterized densities of a special type is presented this procedure is suitable especially for multimodal data apart from finding a feature subset of any cardinality without involving any search procedure it also simultaneously yields a pseudo bayes decision rule its performance is tested on real data

exact sample conditioned mse performance of the bayesian mmse estimator for classification error part i representation in recent years biomedicine has been faced with difficult high throughput small sample classification problems in such settings classifier error estimation becomes a critical issue because training and testing must be done on the same data a recently proposed error estimator places the problem in a signal estimation framework in the presence of uncertainty permitting a rigorous solution optimal in a minimum mean square error sense the uncertainty in this model is relative to the parameters of the feature label distributions resulting in a bayesian approach to error estimation closed form solutions are available for two important problems discrete classification with dirichlet priors and linear classification of gaussian distributions with normal inverse wishart priors in this work part i of a two part study we introduce the theoretical mean square error mse conditioned on the observed sample of any estimate of the classifier error including the bayesian error estimator for both bayesian models thus bayesian error estimation has a unique advantage in that its mathematical framework naturally gives rise to a practical expected measure of performance given an observed sample in part ii of the study we examine consistency of the error estimator demonstrate various mse properties and apply the conditional mse to censored sampling

a comparison of asymptotic error rate expansions for the sample linear discriminant function abstract several asymptotic expansions for approximating the expected or unconditional probability of misclassification for the sample linear discriminant function are compared for accuracy in terms of yielding the smallest mean absolute deviation from the exact value for 104 population configurations the actual expected probabilities of misclassification are found via monte carlo simulation a simple and relatively obscure asymptotic expansion derived by raudys tech cybern 4 168 174 1972 is found to yield better approximation than the well known asymptotic expansions

searchlight based multi voxel pattern analysis of fmri by cross validated manova abstract multi voxel pattern analysis mvpa is a fruitful and increasingly popular complement to traditional univariate methods of analyzing neuroimaging data we propose to replace the standard decoding approach to searchlight based mvpa measuring the performance of a classifier by its accuracy with a method based on the multivariate form of the general linear model following the well established methodology of multivariate analysis of variance manova we define a measure that directly characterizes the structure of multi voxel data the pattern distinctness d our measure is related to standard multivariate statistics but we apply cross validation to obtain an unbiased estimate of its population value independent of the amount of data or its partitioning into training and test sets the estimate d can therefore serve not only as a test statistic but also as an interpretable measure of multivariate effect size the pattern distinctness generalizes the mahalanobis distance to an arbitrary number of classes but also the case where there are no classes of trials because the design is described by parametric regressors it is defined for arbitrary estimable contrasts including main effects pattern differences and interactions pattern changes in this way our approach makes the full analytical power of complex factorial designs known from univariate fmri analyses available to mvpa studies moreover we show how the results of a factorial analysis can be used to obtain a measure of pattern stability the equivalent of cross decoding

resolvability of the parameters of multiexponentials and other sum models it is shown that model fitting solutions for closely located nonlinear parameters of sums of functions of the same family such a multiexponentials are only distinct in a sub space of the euclidean space of the errors in the observations in the complementary space the solutions for two or more of the nonlinear parameters exactly coincide subspaces of the complementary space each correspond with the coincidence of the solutions for the parameters of specific terms of the sum model earlier studies concentrated on the case that the linear parameters such as the amplitudes of the exponentials are known and only two solutions for the nonlinear parameters coincide the analysis presented generalizes these results and extends them to include the solving for the linear parameters and multiple coincidence also a procedure is proposed for the computation of the errors that have minimum energy among all errors causing coincidence this minimum energy may be used as a simple scalar criterion to find out to what extent res olution is to be expected for assumed parameter values and er ror energy for least squares model fitting the proposed pro cedure is shown to be particularly simple

discriminative wavelet packet filter bank selection for pattern recognition this paper addresses the problem of discriminative wavelet packet wp filter bank selection for pattern recognition the problem is formulated as a complexity regularized optimization criterion where the tree indexed structure of the wp bases is explored to find conditions for reducing this criterion to a type of minimum cost tree pruning a method well understood in regression and classification trees cart for estimating the conditional mutual information adopted to compute the fidelity criterion of the minimum cost tree pruning problem a nonparametric approach based on product adaptive partitions is proposed extending the darbellay vajda data dependent partition algorithm finally experimental evaluation within an automatic speech recognition asr task shows that proposed solutions for the wp decomposition problem are consistent with well understood empirically determined acoustic features and the derived feature representations yield competitive performances with respect to standard feature extraction techniques

thresholding method for dimensionality reduction in recognition systems often recognition systems must be designed with a relatively small amount of training data plug in test statistics suffer from large estimation errors often causing the performance to degrade as the measurement vector dimension increases choosing a better test statistic or applying a method of dimensionality reduction are two possible solutions to this problem in this paper we consider a recognition problem where the data for each population are assumed to have the same parametric distribution but differ in their unknown parameters the collected vectors of data as well as their components are assumed to be independent the system is designed to implement a plug in log likelihood ratio test with maximum likelihood ml estimates of the unknown parameters instead of the true parameters because a small amount of data is available to estimate the parameters the performance of such a system is strongly degraded relative to the performance with known parameters to improve the performance of the system we define a thresholding function that when incorporated into the plug in log likelihood ratio significantly decreases the probability of error for binary and multiple hypothesis testing problems for the exponential class of populations we analyze the modified test statistic and present the results of monte carlo simulation special attention is paid to the complex gaussian model with zero mean and unknown variances

termination proofs by multiset path orderings imply primitive recursive derivation lengths it is shown that a termination proof for a term rewriting system using multiset path orderings i e recursive path orderings with multiset status only yields a primitive recursive bound on the length of derivations measured in the size of the starting term confirming a conjecture of plaisted pla78 this result holds for a great variety of path orderings including path of subterms ordering recursive decomposition ordering and the path ordering of kapur narendran sivakumar if lexicographic status is not incorporated the result is essentially optimal as such derivation lengths can be found in each level of the grzegorczyk hierarchy even for string rewriting systems

simple termination of rewrite systems abstract in this paper we investigate the concept of simple termination a term rewriting system is called simply terminating if its termination can be proved by means of a simplification order the basic ingredient of a simplification order is the subterm property but in the literature two different definitions are given one based on strict partial orders and another one based on preorders or quasi orders we argue that there is no reason to choose the second one as the first one has certain advantages simplification orders are known to be well founded orders on terms over a finite signature this important result no longer holds if we consider infinite signatures nevertheless well known simplification orders like the recursive path order are also well founded on terms over infinite signatures provided the underlying precedence is well founded we propose a new definition of simplification order which coincides with the old one based on partial orders in case of finite signatures but which is also well founded over infinite signatures and covers orders like the recursive path order we investigate the properties of the ensuing class of simply terminating systems

model selection through a statistical analysis of the global minimum of a weighted nonlinear least squares cost function this paper presents a model selection algorithm for the identification of parametric models that are linear in the measurements it is based on the mean and variance expressions of the global minimum of a weighted nonlinear least squares cost function the method requires the knowledge of the noise covariance matrix but does not assume that the true model belongs to the model set unlike the traditional order estimation methods available in literature the presented technique allows to detect undermodeling the theory is illustrated by simulations on signal modeling and system identification problems and by one real measurement example

on the modularity of termination of term rewriting systems it is well known that termination is not a modular property of term rewriting systems i e it is not preserved under disjoint union the objective of this paper is to provide a uniform framework for sufficient conditions which ensure the modularity of termination we will prove the following result whenever the disjoint union of two terminating term rewriting systems is nonterminating then one of the systems is not ce terminating i e it loses its termination property when extended with the rules cons x y x and cons x y y and the other is collapsing this result has already been achieved by gramlich 7 for finitely branching term rewriting systems a more sophisticated approach is necessary however to prove it in full generality most of the known sufficient criteria for the preservation of termination 24 15 13 7 follow as corollaries from our result and new criteria are derived this paper particularly settles the open question whether simple termination is modular in general we will moreover shed some light on modular properties of combined systems with shared constructors for instance it will be shown that the above result does not carry over to constructor sharing systems

model validity tests for non linear signal processing applications abstract time series model validity tests based on general correlations are presented in this paper it is shown that the tests xi xi xi xi 2 and xi xi xi 1 t2 can only detect a subset of any possible unmodelled terms in the residuals whereas xi 2 xi 2 detects all possible terms these basic results are then extended to include functions of process or residual terms as entries in the correlation simulation studies are included to demonstrate the effectiveness of the tests when applied to estimated models of both simulated and real data sequences

general framework for asymptotic properties of generalized weighted nonlinear least squares estimators with deterministic and stochastic weighting this paper studies the asymptotic properties strong consistency convergence rate asymptotic normality of a generalized weighted nonlinear least squares estimator under weak noise assumptions both deterministic and stochastic weighting are handled and the presence of model errors is considered for particular models estimators and noise assumptions the general framework boils down to known time and frequency domain estimators

model selection and validation methods for non linear systems the theory of hypothesis testing is used to select a model with the correct structure and the relation of such a method to the aic and fpe criteria is investigated parametric validation and correlation validation methods are developed for non non linear difference equation models several shortcomings of traditional methods especially when applied to non linear systems are described

asymptotic expansions for the means and variances of error rates summary the problem of classifying an observation x into one of k multivariate normal distributions is considered when samples are used to estimate the population parameters the probabilities of correct classification and the associated error rates are random variables asymptotic expansions for the expected values and variances of these random variables in terms of the inverses of the sample sizes are found simulations have been performed to evaluate the expansions

valid population inference for information based imaging from the second level t test to prevalence inference in multivariate pattern analysis of neuroimaging data second level inference is often performed by entering classification accuracies into a t test vs chance level across subjects we argue that while the random effects analysis implemented by the t test does provide population inference if applied to activation differences it fails to do so in the case of classification accuracy or other information like measures because the true value of such measures can never be below chance level this constraint changes the meaning of the population level null hypothesis being tested which becomes equivalent to the global null hypothesis that there is no effect in any subject in the population consequently rejecting it only allows to infer that there are some subjects in which there is an information effect but not that it generalizes rendering it effectively equivalent to fixed effects analysis this statement is supported by theoretical arguments as well as simulations we review possible alternative approaches to population inference for information based imaging converging on the idea that it should not target the mean but the prevalence of the effect in the population one method to do so permutation based information prevalence inference using the minimum statistic is described in detail and applied to empirical data

moments and root mean square error of the bayesian mmse estimator of classification error in the gaussian model the most important aspect of any classifier is its error rate because this quantifies its predictive capacity thus the accuracy of error estimation is critical error estimation is problematic in small sample classifier design because the error must be estimated using the same data from which the classifier has been designed use of prior knowledge in the form of a prior distribution on an uncertainty class of feature label distributions to which the true but unknown feature distribution belongs can facilitate accurate error estimation in the mean square sense in circumstances where accurate completely model free error estimation is impossible this paper provides analytic asymptotically exact finite sample approximations for various performance metrics of the resulting bayesian minimum mean square error mmse error estimator in the case of linear discriminant analysis lda in the multivariate gaussian model these performance metrics include the first second and cross moments of the bayesian mmse error estimator with the true error of lda and therefore the root mean square rms error of the estimator we lay down the theoretical groundwork for kolmogorov double asymptotics in a bayesian setting which enables us to derive asymptotic expressions of the desired performance metrics from these we produce analytic finite sample approximations and demonstrate their accuracy via numerical examples various examples illustrate the behavior of these approximations and their use in determining the necessary sample size to achieve a desired rms the supplementary material contains derivations for some equations and added figures highlightsthe bayesian minimum mean square error bmmse error estimator of lda is studied the first second and cross moments of the bmmse estimator with true error are considered both conditional and unconditional performance metrics are analyzed for the first time the kolmogorov double asymptotic is used in a bayesian setting asymptotically exact finite sample approximations of performance metrics are derived

orderings for term rewriting systems abstract methods of proving that a term rewriting system terminates are presented they are based on the intuitive notion of simplification orderings orderings in which any term that is syntactically simpler than another is smaller than the other as a consequence of kruskal s tree theorem any nonterminating system must be self embedding in the sense that it allows for the derivation of some term from a simpler one thus termination is guaranteed if every rule in the system is a reduction in some simplification ordering most of the orderings that have been used for proving termination are indeed simplication orderings using this notion often allows for much easier proofs a particularly useful class of simplification orderings the recursive path orderings is defined examples of the use of simplication orderings in termination proofs are given

numerically robust transfer function modeling from noisy frequency domain data using vector orthogonal polynomials as basis functions for the representation of the rational form of a linear time invariant system in frequency domain identification problems it is shown that the notorious numerical ill conditioning of these maximum likelihood problems can be overcome completely for the identification of high order 100 100 systems operating over a wide frequency band or even in the situation of over or undermodeling condition numbers less than ten are reported for real measurements

time series analysis in the frequency domain this correspondence presents a parametric frequency domain identification algorithm for autoregressive moving average arma processes that does not suffer from spectral leakage errors it is based on an extended transfer function model that takes into account the begin and end effect of the finite data record the relationship with the one step ahead prediction error method is established the advantages of the proposed method are easy prefiltering and leakage free spectral representation of the raw data

arma model order estimation based on the eigenvalues of the covariance matrix an approach to model order determination based on the minimum description length mdl criterion is proposed and shown to depend on the minimum eigenvalues of a covariance matrix derived from the observed data a selection procedure for estimating the model order by means of the mdl method is proposed examples are given to illustrate the significantly improved accuracy of the technique

relative undecidability in term rewriting i the termination hierarchy for a hierarchy of properties of term rewriting systems related to termination we prove relative undecidability for implications x y in the hierarchy the property x is undecidable for term rewriting systems satisfying y for most implications we obtain this result for term rewriting systems consisting of a single rewrite rule

effects of sample size in classifier design the effect of finite sample size on parameter estimates and their subsequent use in a family of functions are discussed general and parameter specific expressions for the expected bias and variance of the functions are derived these expressions are then applied to the bhattacharyya distance and the analysis of the linear and quadratic classifiers providing insight into the relationship between the number of features and the number of training samples because of the functional form of the expressions an empirical approach is presented to enable asymptotic performance to be accurately estimated using a very small number of samples results were experimentally verified using artificial data in controlled cases and using real high dimensional data

parametric identification of transfer functions in the frequency domain a survey this paper gives a survey of frequency domain identification methods for rational transfer functions in the laplace s or z domain the interrelations between the different approaches are highlighted through a study of the equivalent cost functions the properties of the various estimators are discussed and illustrated by several examples

termination of term rewriting using dependency pairs we present techniques to prove termination and innermost termination of term rewriting systems automatically in contrast to previous approaches we do not compare left and right hand sides of rewrite rules but introduce the notion of dependency pairs to compare left hand sides with special subterms of the right hand sides this results in a technique which allows to apply existing methods for automated termination proofs to term rewriting systems where they failed up to now in particular there are numerous term rewriting systems where a direct termination proof with simplification orderings is not possible but in combination with our technique well known simplification orderings such as the recursive path ordering polynomial orderings or the knuth bendix ordering can now be used to prove termination automatically unlike previous methods our technique for proving innermost termination automatically can also be applied to prove innermost termination of term rewriting systems that are not terminating moreover as innermost termination implies termination for certain classes of term rewriting systems this technique can also be used for termination proofs of such systems

on signal representations within the bayes decision framework this work presents new results in the context of minimum probability of error signal representation mpe sr within the bayes decision framework these results justify addressing the mpe sr criterion as a complexity regularized optimization problem demonstrating the empirically well understood trade off between signal representation quality and learning complexity contributions are presented in three folds first the stipulation of conditions that guarantee a formal tradeoff between approximation and estimation errors under sequence of embedded transformations are provided second the use of this tradeoff to formulate the mpe sr as a complexity regularized optimization problem and an approach to address this oracle criterion in practice is given finally formal connections are provided between the mpe sr criterion and two emblematic feature transformation techniques used in pattern recognition the optimal quantization problem of classification trees cart tree pruning algorithms and some versions of fisher linear discriminant analysis lda

linear discriminant analysis with misallocation in training samples abstract linear discriminant analysis for a two class case is studied in the presence of misallocation in training samples a general approach to modeling of misallocation is formulated and the mean vectors and covariance matrices of the mixture distributions are derived the asymptotic distribution of the discriminant boundary is obtained and the asymptotic first two moments of the error rates are given certain numerical results for the error rates are presented by considering the random and two nonrandom misallocation models

generalization error of linear discriminant analysis in spatially correlated sensor networks generalization error the probability of error of a detection rule learned from training samples on new unseen samples is a fundamental quantity to be characterized however characterizations of generalization error in the statistical learning theory literature are often loose and practically unusable for optimizing detection systems in this work focusing on learning linear discriminant analysis detection rules from spatially correlated sensor measurements a tight generalization error approximation is developed that can be used to optimize the parameters of a sensor network detection system as such the approximation is used to optimize network settings the approximation is also used to derive a detection error exponent and select an optimal subset of deployed sensor nodes a gauss markov random field is used to model correlation and weak laws of large numbers in geometric probability are employed in the analysis

on dimensionality sample size and classification error of nonparametric linear classification algorithms this paper compares two nonparametric linear classification algorithms the zero empirical error classifier and the maximum margin classifier with parametric linear classifiers designed to classify multivariate gaussian populations formulae and a table for the mean expected probability of misclassification mep sub n are presented they show that the classification error is mainly determined by n p a learning set size dimensionality ratio however the influences of learning set size on the generalization error of parametric and nonparametric linear classifiers are quite different under certain conditions the nonparametric approach allows us to obtain reliable rules even in cases where the number of features is larger than the number of training vectors