advances on byy harmony learning information theoretic perspective generalized projection geometry and independent factor autodetermination the nature of bayesian ying yang harmony learning is reexamined from an information theoretic perspective not only its ability for model selection and regularization is explained with new insights but also discussions are made on its relations and differences from the studies of minimum description length mdl bayesian approach the bit back based mdl akaike information criterion aic maximum likelihood information geometry helmholtz machines and variational approximation moreover a generalized projection geometry is introduced for further understanding such a new mechanism furthermore new algorithms are also developed for implementing gaussian factor analysis fa and non gaussian factor analysis nfa such that selecting appropriate factors is automatically made during parameter learning

a latent class approach to fitting the weighted euclidean model clascal a weighted euclidean distance model for analyzing three way proximity data is proposed that incorporates a latent class approach in this latent class weighted euclidean model the contribution to the distance function between two stimuli is per dimension weighted identically by all subjects in the same latent class this model removes the rotational invariance of the classical multidimensional scaling model retaining psychologically meaningful dimensions and drastically reduces the number of parameters in the traditional indscal model the probability density function for the data of a subject is posited to be a finite mixture of spherical multivariate normal densities the maximum likelihood function is optimized by means of an em algorithm a modified fisher scoring method is used to update the parameters in the m step a model selection strategy is proposed and illustrated on both real and artificial data

model selection and akaike s information criterion aic the general theory and its analytical extensions during the last fifteen years akaike s entropy based information criterion aic has had a fundamental impact in statistical model evaluation problems this paper studies the general theory of the aic procedure and provides its analytical extensions in two ways without violating akaike s main principles these extensions make aic asymptotically consistent and penalize overparameterization more stringently to pick only the simplest of the true models these selection criteria are called caic and caicf asymptotic properties of aic and its extensions are investigated and empirical performances of these criteria are studied in choosing the correct degree of a polynomial model in two different monte carlo experiments under different conditions

on model selection criteria in multimodel analysis 1 hydrologic systems are open and complex rendering them prone to multiple conceptualizations and mathematical descriptions there has been a growing tendency to postulate several alternative hydrologic models for a site and use model selection criteria to 1 rank these models 2 eliminate some of them and or 3 weigh and average predictions and statistics generated by multiple models this has led to some debate among hydrogeologists about the merits and demerits of common model selection also known as model discrimination or information criteria such as aic aicc bic and kic and some lack of clarity about the proper interpretation and mathematical representation of each criterion we examine the model selection literature to find that 1 all published rigorous derivations of aic and aicc require that the true model having generated the observational data be in the set of candidate models 2 though bic and kic were originally derived by assuming that such a model is in the set bic has been rederived by cavanaugh and neath 1999 without the need for such an assumption and 3 kic reduces to bic as the number of observations becomes large relative to the number of adjustable model parameters implying that it likewise does not require the existence of a true model in the set of alternatives we explain why kic is the only criterion accounting validly for the likelihood of prior parameter estimates elucidate the unique role that the fisher information matrix plays in kic and demonstrate through an example that it imbues kic with desirable model selection properties not shared by aic aicc or bic our example appears to provide the first comprehensive test of how aic aicc bic and kic weigh and rank alternative models in light of the models predictive performance under cross validation with real hydrologic data

estimating latent distributions in recurrent choice data this paper introduces a flexible class of stochastic mixture models for the analysis and interpretation of individual differences in recurrent choice and other types of count data these choice models are derived by specifying elements of the choice process at the individual level probability distributions are introduced to describe variations in the choice process among individuals and to obtain a representation of the aggregate choice behavior due to the explicit consideration of random effect sources the choice models are parsimonious and readily interpretable an easy to implement em algorithm is presented for parameter estimation two applications illustrate the proposed approach

bayesian inference for finite mixtures of generalized linear models with random effects we present an hierarchical bayes approach to modeling parameter heterogeneity in generalized linear models the model assumes that there are relevant subpopulations and that within each subpopulation the individual level regression coefficients have a multivariate normal distribution however class membership is not known a priori so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions this approach combines the flexibility of semiparametric latent class models that assume common parameters for each sub population and the parsimony of random effects models that assume normal distributions for the regression parameters the number of subpopulations is selected to maximize the posterior probability of the model being true simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub populations an application is presented concerning preferences for various aspects of personal computers

a stochastic multidimensional scaling vector threshold model for the spatial representation of pick any n data this paper presents a new stochastic multidimensional scaling vector threshold model designed to analyze pick any n choice data e g consumers rendering buy no buy decisions concerning a number of actual products a maximum likelihood procedure is formulated to estimate a joint space of both individuals represented as vectors and stimuli represented as points the relevant psychometric literature concerning the spatial treatment of such binary choice data is reviewed the nonlinear probit type model is described as well as the conjugate gradient procedure used to estimate parameters results of monte carlo analyses investigating the performance of this methodology with synthetic choice data sets are presented an application concerning consumer choices for eleven competitive brands of soft drinks is discussed finally directions for future research are presented in terms of further applications and generalizing the model to accommodate three way choice data

hierarchical classes models for three way three mode binary data interrelations and model selection several hierarchical classes models can be considered for the modeling of three way three mode binary data including the indclas model leenen van mechelen de boeck and rosenberg 1999 the tucker3 hiclas model ceulemans van mechelen and leenen 2003 the tucker2 hiclas model ceulemans and van mechelen 2004 and the tucker1 hiclas model that is introduced in this paper two questions then may be raised 1 how are these models interrelated and 2 given a specific data set which of these models should be selected and in which rank in the present paper we deal with these questions by 1 showing that the distinct hierarchical classes models for three way three mode binary data can be organized into a partially ordered hierarchy and 2 by presenting model selection strategies based on extensions of the well known scree test and on the akaike information criterion the latter strategies are evaluated by means of an extensive simulation study and are illustrated with an application to interpersonal emotion data finally the presented hierarchy and model selection strategies are related to corresponding work by kiers 1991 for principal component models for three way three mode real valued data

a thurstonian pairwise choice model with univariate and multivariate spline transformations a probabilistic choice model is developed for paired comparisons data about psychophysical stimuli the model is based on thurstone s law of comparative judgment case v and assumes that each stimulus is measured on a small number of physical variables the utility of a stimulus is related to its values on the physical variables either by means of an additive univariate spline model or by means of multivariate spline model in the additive univariate spline model a separate univariate spline transformation is estimated for each physical dimension and the utility of a stimulus is assumed to be an additive combination of these transformed values in the multivariate spline model the utility of a stimulus is assumed to be a general multivariate spline function in the physical variables the use of b splines for estimating the transformation functions is discussed and it is shown how b splines can be generalized to the multivariate case by using as basis functions tensor products of the univariate basis functions a maximum likelihood estimation procedure for the thurstone case v model with spline transformation is described and applied for illustrative purposes to various artificial and real data sets finally the model is extended using a latent class approach to the case where there are unreplicated paired comparisons data from a relatively large number of subjects drawn from a heterogeneous population an em algorithm for estimating the parameters in this extended model is outlined and illustrated on some real data

model selection principles in misspecified models summary model selection is of fundamental importance to high dimensional modelling featured in many contemporary applications classical principles of model selection include the bayesian principle and the kullback leibler divergence principle which lead to the bayesian information criterion and akaike information criterion respectively when models are correctly specified yet model misspecification is unavoidable in practice we derive novel asymptotic expansions of the two well known principles in misspecified generalized linear models which give the generalized bayesian information criterion and generalized akaike information criterion a specific form of prior probabilities motivated by the kullback leibler divergence principle leads to the generalized bayesian information criterion with prior probability gbicp which can be naturally decomposed as the sum of the negative maximum quasi log likelihood a penalty on model dimensionality and a penalty on model misspecification directly numerical studies demonstrate the advantage of the new methods for model selection in both correctly specified and misspecified models

a note on pareto tails and lognormal body of us cities size distribution abstract luckstead and devadoss 2017 fitted a distribution made up of lower tail pareto lognormal body and upper tail pareto to city sizes in the us here we propose a class of distributions providing better fits the fits are assessed in terms of akaike information criterion bayesian information criterion and probability plots

model order selection based on information theoretic criteria design of the penalty information theoretic criteria itc have been widely adopted in engineering and statistics for selecting among an ordered set of candidate models the one that better fits the observed sample data the selected model minimizes a penalized likelihood metric where the penalty is determined by the criterion adopted while rules for choosing a penalty that guarantees a consistent estimate of the model order are known theoretical tools for its design with finite samples have never been provided in a general setting in this paper we study model order selection for finite samples under a design perspective focusing on the generalized information criterion gic which embraces the most common itc the theory is general and as case studies we consider a the problem of estimating the number of signals embedded in additive white gaussian noise awgn by using multiple sensors b model selection for the general linear model glm which includes e g the problem of estimating the number of sinusoids in awgn the analysis reveals a trade off between the probabilities of overestimating and underestimating the order of the model we then propose to design the gic penalty to minimize underestimation while keeping the overestimation probability below a specified level for the considered problems this method leads to analytical derivation of the optimal penalty for a given sample size a performance comparison between the penalty optimized gic and common aic and bic is provided demonstrating the effectiveness of the proposed design strategy

detecting the number of clusters in n way probabilistic clustering recently there has been a growing interest in multiway probabilistic clustering some efficient algorithms have been developed for this problem however not much attention has been paid on how to detect the number of clusters for the general n way clustering n 2 to fill this gap this problem is investigated based on n way algebraic theory in this paper a simple yet efficient detection method is proposed by eigenvalue decomposition evd which is easy to implement we justify this method in addition its effectiveness is demonstrated by the experiments on both simulated and real world data sets

supervising isodata with an information theoretic stopping rule abstract new biomedical imaging modalities such as magnetic resonance imaging mri provide fertile multidimensional environments for the automatic identification of biological soft tissues but lack the a priori information required to appropriately train supervised classifiers hierarchical cluster analysis techniques can generate this information but they are inefficient on large data sets we have developed an unsupervised clustering method that is a variant of the well known isodata algorithm we replaced the heuristic rules that control isodata with rules that search for the minimum value of an information theoretic criterion the criteria investigated in this study are akaike s information criterion aic and the consistent aic caic both measure the global fit of a cluster model to the input data and the smallest criterion value suggests the best fit we tested the new method on multivariate gaussian and real world data including mr images of normal and diseased tissue in vivo

a model selection criterion for high dimensional linear regression statistical model selection is a great challenge when the number of accessible measurements is much smaller than the dimension of the parameter space we study the problem of model selection in the context of subset selection for high dimensional linear regressions accordingly we propose a new model selection criterion with the fisher information that leads to the selection of a parsimonious model from all the combinatorial models up to some maximum level of sparsity we analyze the performance of our criterion as the number of measurements grows to infinity as well as when the noise variance tends to zero in each case we prove that our proposed criterion gives the true model with a probability approaching one additionally we devise a computationally affordable algorithm to conduct model selection with the proposed criterion in practice interestingly as a side product our algorithm can provide the ideal regularization parameter for the lasso estimator such that lasso selects the true variables finally numerical simulations are included to support our theoretical findings

the robust sequential estimator a general approach and its application to surface organization in range data presents an autonomous statistically robust sequential function approximation approach to simultaneous parameterization and organization of possibly partially occluded surfaces in noisy outlier ridden not gaussian functional range data at the core of this approach is the robust sequential estimator a robust extension to the method of sequential least squares unlike most existing surface characterization techniques the authors method generates complete surface hypotheses in parameter space given a noisy depth map of an unknown 3 d scene the algorithm first selects appropriate seed points representing possible surfaces for each nonredundant seed it chooses the best approximating model from a given set of competing models using a modified akaike information criterion with this best model each surface is expanded from its seed over the entire image and this step is repeated for all seeds those points which appear to be outliers with respect to the model in growth are not included in the possibly disconnected surface point regions are deleted from each newly grown surface in the prune stage noise outliers or coincidental surface alignment may cause some points to appear to belong to more than one surface these ambiguities are resolved by a weighted voting scheme within a 5 spl times 5 decision window centered around the ambiguous point the isolated point regions left after the resolve stage are removed and any missing points in the data are filled by the surface having a majority consensus in an 8 neighborhood

cluster number selection for a small set of samples using the bayesian ying yang model one major problem in cluster analysis is the determination of the number of clusters in this paper we describe both theoretical and experimental results in determining the cluster number for a small set of samples using the bayesian kullback ying yang byy model selection criterion under the second order approximation we derive a new equation for estimating the smoothing parameter in the cost function finally we propose a gradient descent smoothing parameter estimation approach that avoids complicated integration procedure and gives the same optimal result

on the linear quadratic regulator problem in one dimensional linear fractional stochastic systems in this paper for one dimensional stochastic linear fractional systems in terms of the riemann liouville fractional derivative the optimal control is derived it is assumed that the state is completely observable and all the information regarding this is available the formulation leads to a set of stochastic fractional forward and backward equation in the riemann liouville sense the proposed method has been checked via some numerical simulations which show the effectiveness of the fractional stochastic optimal algorithm

discrepancy risk model selection test theory for comparing possibly misspecified or nonnested models a new model selection statistical test is proposed for testing the null hypothesis that two probability models equally effectively fit the underlying data generating process dgp the new model selection test called the discrepancy risk model selection test drmst extends previous work see vuong 1989 on this problem in four distinct ways first generalized goodness of fit measures which include log likelihood functions can be used second unlike the classical likelihood ratio test the models are not required to be fully nested where the nesting concept is defined for generalized goodness of fit measures the drmst also differs from the likelihood ratio test by not requiring that either competing model provides a completely accurate representation of the dgp and fourth the drmst may be used to compare competing time series models using correlated observations as well as data consisting of independent and identically distributed observations

nested logit models for multiple choice item response data nested logit item response models for multiple choice data are presented relative to previous models the new models are suggested to provide a better approximation to multiple choice items where the application of a solution strategy precedes consideration of response options in practice the models also accommodate collapsibility across all distractor categories making it easier to allow decisions about including distractor information to occur on an item by item or application by application basis without altering the statistical form of the correct response curves marginal maximum likelihood estimation algorithms for the models are presented along with simulation and real data analyses

a note on impact factor distribution revisited abstract motivated by huang 2017 we analyze impact factor data in all subject categories for each year from 2010 to 2015 the results are provided by subject categories and by years the two exponent distribution due to mansilla et al 2007 and the normal distribution are shown to give the best fits for the data sets the best fits are assessed in terms of probability plots quantile plots and five other criteria

range image segmentation using surface selection criterion in this paper we address the problem of recovering the true underlying model of a surface while performing the segmentation first and in order to solve the model selection problem we introduce a novel criterion which is based on minimising strain energy of fitted surfaces we then evaluate its performance and compare it with many other existing model selection techniques using this criterion we then present a robust range data segmentation algorithm capable of segmenting complex objects with planar and curved surfaces the presented algorithm simultaneously identifies the type order and geometric shape of each surface and separates all the points that are part of that surface this paper includes the segmentation results of a large collection of range images obtained from objects with planar and curved surfaces the resulting segmentation algorithm successfully segments various possible types of curved objects more importantly the new technique is capable of detecting the association between separated parts of a surface which has the same cartesian equation while segmenting a scene this aspect is very useful in some industrial applications of range data analysis

a stochastic multidimensional scaling procedure for the spatial representation of three mode three way pick any j data this paper presents a new stochastic multidimensional scaling procedure for the analysis of three mode three way pick any j data the method provides either a vector or ideal point model to represent the structure in such data as well as floating model specifications e g different vectors or ideal points for different choice settings and various reparameterization options that allow the coordinates of ideal points vectors or stimuli to be functions of specified background variables a maximum likelihood procedure is utilized to estimate a joint space of row and column objects as well as a set of weights depicting the third mode of the data an algorithm using a conjugate gradient method with automatic restarts is developed to estimate the parameters of the models a series of monte carlo analyses are carried out to investigate the performance of the algorithm under diverse data and model specification conditions examine the statistical properties of the associated test statistic and test the robustness of the procedure to departures from the independence assumptions finally a consumer psychology application assessing the impact of situational influences on consumers choice behavior is discussed

a maximum likelihood method for latent class regression involving a censored dependent variable the standard tobit or censored regression model is typically utilized for regression analysis when the dependent variable is censored this model is generalized by developing a conditional mixture maximum likelihood method for latent class censored regression the proposed method simultaneously estimates separate regression functions and subject membership in k latent classes or groups given a censored dependent variable for a cross section of subjects maximum likelihood estimates are obtained using an em algorithm the proposed method is illustrated via a consumer psychology application

comparing dynamic causal models this article describes the use of bayes factors for comparing dynamic causal models dcms dcms are used to make inferences about effective connectivity from functional magnetic resonance imaging fmri data these inferences however are contingent upon assumptions about model structure that is the connectivity pattern between the regions included in the model given the current lack of detailed knowledge on anatomical connectivity in the human brain there are often considerable degrees of freedom when defining the connectional structure of dcms in addition many plausible scientific hypotheses may exist about which connections are changed by experimental manipulation and a formal procedure for directly comparing these competing hypotheses is highly desirable in this article we show how bayes factors can be used to guide choices about model structure both concerning the intrinsic connectivity pattern and the contextual modulation of individual connections the combined use of bayes factors and dcm thus allows one to evaluate competing scientific theories about the architecture of large scale neural networks and the neuronal interactions that mediate perception and cognition

temporal byy encoding markovian state spaces and space dimension determination as a complementary to those temporal coding approaches of the current major stream this paper aims at the markovian state space temporal models from the perspective of the temporal bayesian ying yang byy learning with both new insights and new results on not only the discrete state featured hidden markov model and extensions but also the continuous state featured linear state spaces and extensions especially with a new learning mechanism that makes selection of the state number or the dimension of state space either automatically during adaptive learning or subsequently after learning via model selection criteria obtained from this mechanism experiments are demonstrated to show how the proposed approach works

deriving ultrametric tree structures from proximity data confounded by differential stimulus familiarity this paper presents a new procedure called treefam for estimating ultrametric tree structures from proximity data confounded by differential stimulus familiarity the objective of the proposed treefam procedure is to quantitatively filter out the effects of stimulus unfamiliarity in the estimation of an ultrametric tree a conditional alternating maximum likelihood procedure is formulated to simultaneously estimate an ultrametric tree under the unobserved condition of complete stimulus familiarity and subject specific parameters capturing the adjustments due to differential unfamiliarity we demonstrate the performance of the treefam procedure under a variety of alternative conditions via a modest monte carlo experimental study an empirical application provides evidence that the treefam outperforms traditional models that ignore the effects of unfamiliarity in terms of superior tree recovery and overall goodness of fit

application of model selection criteria to some problems in multivariate analysis a review of model selection criteria is presented with a view toward showing their similarities it is suggested that some problems treated by sequences of hypothesis tests may be more expeditiously treated by the application of model selection criteria consideration is given to application of model selection criteria to some problems of multivariate analysis especially the clustering of variables factor analysis and more generally describing a complex of variables

on the robustness of maximum likelihood scaling for violations of the error model a monte carlo study was conducted to investigate the robustness of the assumed error distribution in maximum likelihood estimation models for multidimensional scaling data sets generated according to the lognormal the normal and the rectangular distribution were analysed with the log normal error model in ramsay s multiscale program package the results show that violations of the assumed error distribution have virtually no effect on the estimated distance parameters in a comparison among several dimensionality tests the corrected version of thex2 test as proposed by ramsay yielded the best results and turned out to be quite robust against violations of the error model

feature specific penalized latent class analysis for genomic data genomic data are often characterized by a moderate to large number of categorical variables observed for relatively few subjects some of the variables may be missing or noninformative an example of such data is loss of heterozygosity loh a dichotomous variable observed on a moderate number of genetic markers we first consider a latent class model where conditional on unobserved membership in one of k classes the variables are independent with probabilities determined by a regression model of low dimension q using a family of penalties including the ridge and lasso we extend this model to address higher dimensional problems finally we present an orthogonal map that transforms marker space to a space of features for which the constrained model has better predictive power we demonstrate these methods on loh data collected at 19 markers from 93 brain tumor patients for this data set the existing unpenalized latent class methodology does not produce estimates additionally we show that posterior classes obtained from this method are associated with survival for these patients

a graphical technique for determining the number of components in a mixture of normals abstract when a population is assumed to be composed of a finite number of subpopulations a natural model to choose is the finite mixture model it will often be the case however that the number of component distributions is unknown and must be estimated this problem can be difficult for instance the density of two mixed normals is not bimodal unless the means are separated by at least 2 standard deviations hence modality of the data per se can be an insensitive approach to component estimation we demonstrate that a mixture of two normals divided by a normal density having the same mean and variance as the mixed density is always bimodal this analytic result and other related results form the basis for a diagnostic and a test for the number of components in a mixture of normals the density is estimated using a kernel density estimator under the null hypothesis the proposed diagnostic can be approximated by a stationary gaussian process under the alternative hypothesis components in the mixtu

a model based approach for visualizing the dimensional structure of ordered successive categories preference data a cyclical conditional maximum likelihood estimation procedure is developed for the multidimensional unfolding of two or three way dominance data e g preference choice consideration measured on ordered successive category rating scales the technical description of the proposed model and estimation procedure are discussed as well as the rather unique joint spaces derived we then conduct a modest monte carlo simulation to demonstrate the parameter recovery of the proposed methodology as well as investigate the performance of various information heuristics for dimension selection a consumer psychology application is provided where the spatial results of the proposed model are compared to solutions derived from various traditional multidimensional unfolding procedures this application deals with consumers intending to buy new luxury sport utility vehicles suvs finally directions for future research are discussed

parameter estimation of poisson mixture with automated model selection through byy harmony learning finite mixture is widely used in the fields of information processing and data analysis however its model selection i e the selection of components in the mixture for a given sample data set has been still a rather difficult task recently the bayesian ying yang byy harmony learning has provided a new approach to the gaussian mixture modeling with a favorite feature that model selection can be made automatically during parameter learning in this paper based on the same byy harmony learning framework for finite mixture we propose an adaptive gradient byy learning algorithm for poisson mixture with automated model selection it is demonstrated well by the simulation experiments that this adaptive gradient byy learning algorithm can automatically determine the number of actual poisson components for a sample data set with a good estimation of the parameters in the original or true mixture where the components are separated in a certain degree moreover the adaptive gradient byy learning algorithm is successfully applied to texture classification

the byy annealing learning algorithm for gaussian mixture with automated model selection bayesian ying yang byy learning has provided a new mechanism that makes parameter learning with automated model selection via maximizing a harmony function on a backward architecture of the byy system for the gaussian mixture however since there are a large number of local maxima for the harmony function any local searching algorithm such as the hard cut em algorithm does not work well in order to overcome this difficulty we propose a simulated annealing learning algorithm to search the global maximum of the harmony function being expressed as a kind of deterministic annealing em procedure it is demonstrated by the simulation experiments that this byy annealing learning algorithm can efficiently and automatically determine the number of clusters or gaussians during the learning process moreover the byy annealing learning algorithm is successfully applied to two real life data sets including iris data classification and unsupervised color image segmentation

location estimation of a random signal source based on correlated sensor observations the problem of location estimation of a source of random signals using a network of sensors is considered a novel maximum likelihood estimation mle based approach using copula functions is proposed the measurements received at the sensors are often spatially correlated and characterized by a multivariate distribution using the theory of copulas the joint parametric density of sensor observations joint likelihood is approximated assuming only the knowledge of the marginal likelihood functions of the sensor observations the problem of selecting the best copula function to model the joint likelihood is approached as one of model selection and a model fusion strategy is used to reduce the effect of selection bias an example involving source localization of a poisson source is presented to illustrate the proposed approach and demonstrate its performance

information and posterior probability criteria for model selection in local likelihood estimation local likelihood estimation has proven to be an effective method for obtaining estimates of parameters that vary with a covariate to obtain useful estimates of such parameters approximating models are used in such cases it is useful to consider window based estimates we may need to choose between competing approximating models in this article we propose a modification to the methods used to motivate many information and posterior probability criteria for the weighted likelihood case we derive weighted versions for two of the most widely known criteria namely the aic and bic via a simple modification the criteria are also made useful for window span selection the usefulness of the weighted version of these criteria is demonstrated through a simulation study and an application to three datasets

a byy scale incremental em algorithm for gaussian mixture learning abstract gaussian mixture model has been used extensively in the fields of information processing and data analysis however its model selection i e the selection of number of components or gaussians in the mixture is still a difficult problem fortunately the new established bayesian ying yang byy harmony function provides an efficient criterion for the model selection of gaussian mixture with a set of sample data in this paper we propose a byy scale incremental em algorithm for gaussian mixture learning via a component split rule to increase the byy harmony function incrementally particularly starting from two components and adding one component sequentially via the split rule after each em procedure until a maximum number of components the algorithm increases the scale of the mixture incrementally and leads to the maximization of the byy harmony function together with the correct model selection and a good parameter estimation of the gaussian mixture it is demonstrated well by the simulation experiments that this byy scale incremental em algorithm can make both model selection and parameter estimation efficiently for gaussian mixture modeling moreover the byy scale incremental em algorithm is successfully applied to two real life data sets including iris data classification and unsupervised color image segmentation

a mixture model for distributions of correlation coefficients an old problem in personnel psychology is to characterize distributions of test validity correlation coefficients the proposed model views histograms of correlation coefficients as observations from a mixture distribution which for a fixed sample sizen is a conditional mixture distributionh r n j j h r j n wherer is the correlation coefficient j are population correlation coefficients and j are the mixing weights the associated marginal distribution ofr is regarded as the parent distribution underlying histograms of empirical correlation coefficients maximum likelihood estimates of the parameters j and j can be obtained with an em algorithm solution and tests for the number of componentst are achieved after the one component density ofr is replaced with a tractable modeling densityh r j n two illustrative examples are provided

model selection an integral part of inference we argue that model selection uncertainty should be fully incorporated into statistical inference whenever estimation is sensitive to model choice and that choice is made with reference to the data we consider different philosophies for achieving this goal and suggest strategies for data analysis we illustrate our methods through three examples the first is a poisson regression of bird counts in which a choice is to be made between inclusion of one or both of two covariates the second is a line transect data set for which different models yield substantially different estimates of abundance the third is a simulated example in which truth is known

catching up faster by switching sooner a predictive approach to adaptive estimation with an application to the aic bic dilemma summary prediction and estimation based on bayesian model selection and model averaging and derived methods such as the bayesian information criterion bic do not always converge at the fastest possible rate we identify the catch up phenomenon as a novel explanation for the slow convergence of bayesian methods which inspires a modification of the bayesian predictive distribution called the switch distribution when used as an adaptive estimator the switch distribution does achieve optimal cumulative risk convergence rates in non parametric density estimation and gaussian regression problems we show that the minimax cumulative risk is obtained under very weak conditions and without knowledge of the underlying degree of smoothness unlike other adaptive model selection procedures such as the akaike information criterion aic and leave one out cross validation bic and bayes factor model selection are typically statistically consistent we show that this property is retained by the switch distribution which thus solves the aic bic dilemma for cumulative risk the switch distribution has an efficient implementation we compare its performance with aic bic and bayesian model selection and averaging on a regression problem with simulated data

data model conclusion doing it again this paper explores the robustness of conclusions from a statistical model against variations in model choice rather than variations in random sampling and random assignment to treatments which are the usual variations covered by inferential statistics after the problem formulation in section 1 section 2 presents an example from box and tiao in which variation in parent distribution is modeled for a one sample location problem an adaptive bayesian procedure permits to use a weighted mixture of parent distributions rather than choosing just one such as a normal or a uniform distribution in section 3 similar considerations are applied to an event history model for the influence of education and gender on age at first marriage but here the conclusions are hardly influenced by the choice of the duration distribution in section 4 a brief discussion of model choice in factor analysis and structural equation models is followed by a more elaborate treatment of the choice of integer valued slopes for item response functions in the oplm model which is an extension of the rasch model a modest simulation study suggests that adaptive bayesian modeling with a mixture of sets of slopes works better than fixing one set of postulated slopes the paper concludes with some remarks on the roles and sources of prior distributions followed by a short epilogue which argues that simultaneous consideration of a class of models for the same data is sometimes superior to exclusively analyzing the data under one specific model chosen from such a class

facaic model selection algorithm for the orthogonal factor model using aic and caic this paper describes the authors fortran algorithm facaic for choosing the number of factors for an orthogonal factor model using akaike s information criterion facaic utilizes the imsl subroutine ofcomm

using convex switching techniques for partially observable decision processes we present and examine a novel method for obtaining solutions to specific discrete time optimal control problems our approach is based on linear state dynamics and convexity assumptions commonly satisfied in practical applications we show that the important class of optimal switching problems under partial observation is covered by our methodology and we exploit specific model features to achieve simple algorithmic form of a numerical solution

on the choice of a model to fit data from an exponential family let x 1 cdots x n be iid observations coming from an exponential family the problem of interest is this given a finite number of models m j smoothly curved manifolds in mathbb r k choose the best model to fit the observations with some penalty for choosing models with dimensions which are too large a result of schwarz is made more specific and is extended to the case where the models are curved manifolds if s y n j is up to a constant c n independent of the model the log of the posterior probability of the j th model where the sample mean y n 1 n sum n i 1 x i has been replaced by y schwarz suggested an asymptotic expansion of s y n j whose leading terms are gamma y n j n sup psi in m j cap theta y psi b psi frac 1 2 k j log n in the case where the models are affine subspaces of mathbb r k we establish a similar asymptotic expansion including the next term with uniform bounds for y in a compact neighborhood of nabla b theta where theta is the true value of the parameter we suggest a criterion for the choice of the best model that consists of maximizing the three leading terms in the expansion s y n j we show that the criterion gives the correct model with probabilities p n theta rightarrow 1 as n rightarrow infty

bootstrap choice of estimators in parametric and semiparametric families an extension of eic ishiguro sakamoto et kitagawa 1997 ont propose le eic comme une extension du critere d akaike aic le eic est construit en corrigeant le biais de la log vraisemblance consideree comme un estimateur de l information de kullback leibler par la technique du bootstrap nous developpons ce critere dans un cadre d estimation semi parametrique multivarie et nous montrons qu il peut etre utilise pour choisir un estimateur parmi une famille d estimateurs parametriques et semi parametriques une etude de simulation basee sur un modele de regression montre que l eic est meilleur que les criteres usuels bien que la validation croisee appliquee a la vraisemblance donne des resulats tres proches sauf dans le cas d echantillons de faible taille nous avons applique le eic afin d estimer l evolution moyenne de la charge virale d enfants infectes par le vih

model order selection rules for covariance structure classification in radar the adaptive classification of the interference covariance matrix structure for radar signal processing applications is addressed in this paper this represents a key issue because many detection architectures are synthesized assuming a specific covariance structure which may not necessarily coincide with the actual one due to the joint action of the system and environment uncertainties the considered classification problem is cast in terms of a multiple hypotheses test with some nested alternatives and the theory of model order selection mos is exploited to devise suitable decision rules several mos techniques such as the akaike takeuchi and bayesian information criteria are adopted and the corresponding merits and drawbacks are discussed at the analysis stage illustrating examples for the probability of correct model selection are presented showing the effectiveness of the proposed rules

parsimonious structural equation models for repeated measures data with application to the study of consumer preferences recent research reflects a growing awareness of the value of using structural equation models to analyze repeated measures data however such data particularly in the presence of covariates often lead to models that either fit the data poorly are exceedingly general and hard to interpret or are specified in a manner that is highly data dependent this article introduces methods for developing parsimonious models for such data the underlying technology uses reduced rank representations of the variances covariances and means of observed and latent variables the value of this approach which may be implemented using standard structural equation modeling software is illustrated in an application study aimed at understanding heterogeneous consumer preferences in this application the parsimonious representations characterize systematic relationships among consumer demographics attitudes and preferences that would otherwise be undetected the result is a model that is parsimonious illuminating and fits the data well while keeping data dependence to a minimum

bayesian model estimation and selection for epipolar geometry and generic manifold fitting computer vision often involves estimating models from visual input sometimes it is possible to fit several different models or hypotheses to a set of data and a decision must be made as to which is most appropriate this paper explores ways of automating the model selection process with specific emphasis on the least squares problem of fitting manifolds in particular algebraic varieties e g lines algebraic curves planes etc to data points illustrated with respect to epipolar geometry the approach is bayesian and the contribution three fold first a new bayesian description of the problem is laid out that supersedes the author s previous maximum likelihood formulations this formulation will reveal some hidden elements of the problem second an algorithm mapsac is provided to obtain the robust map estimate of an arbitrary manifold third a bayesian model selection paradigm is proposed the bayesian formulation of the manifold fitting problem uncovers an elegant solution to this problem for which a new method gric for approximating the posterior probability of each putative model is derived this approximations bears some similarity to the penalized likelihoods used by aic bic and mdl however it is far more accurate in situations involving large numbers of latent variables whose number increases with the data this will be empirically and theoretically demonstrated

admissible selection of an accurate and parsimonious normal linear regression model let mo be a normal linear regression model and let m1 mk be distinct proper linear submodels of mo let k e 0 k be a model selection rule based on observed data from the true model given k let the unknown parameters of the selected model ms be fitted by the maximum likelihood method a loss function is introduced which depends additively on two parts i a measure of the difference between the fitted model ms and the true model and ii a measure cs of the complexity of the selected model a natural model selection rule k which minimizes an empirical version of this loss is shown to be admissible and very nearly bayes

model based clustering of large data sets tracing the development of spelling ability there are two main theories with respect to the development of spelling ability the stage model and the model of overlapping waves in this paper exploratory model based clustering will be used to analyze the responses of more than 3500 pupils to subsets of 245 items to evaluate the two theories the resulting clusters will be ordered along a developmental dimension using an external criterion solutions for three statistical problems will be given 1 an algorithm that can handle large data sets and only renders non degenerate clusters 2 a goodness of fit test that is not affected by the fact that the number of possible response vectors by far out weights the number of observed response vectors and 3 a new technique data expunction that can be used to evaluate goodness of fit tests if the missing data mechanism is known

a parametric procedure for ultrametric tree estimation from conditional rank order proximity data the psychometric and classification literatures have illustrated the fact that a wide class of discrete or network models e g hierarchical or ultrametric trees for the analysis of ordinal proximity data are plagued by potential degenerate solutions if estimated using traditional nonmetric procedures i e procedures which optimize a stress based criteria of fit and whose solutions are invariant under a monotone transformation of the input data this paper proposes a new parametric maximum likelihood based procedure for estimating ultrametric trees for the analysis of conditional rank order proximity data we present the technical aspects of the model and the estimation algorithm some preliminary monte carlo results are discussed a consumer psychology application is provided examining the similarity of fifteen types of snack breakfast items finally some directions for future research are provided

filter order selection in adaptive maximum likelihood estimation the choice of the number of parameters is an important problem in adaptive filtering in the general ground of maximum likelihood ml parameter estimation akaike criterion has proved to be a very useful tool significant results and the importance of the kullback information measure as the implicit measure of distance associated with ml estimation are discussed application of these results to adaptive filtering is then considered for time invariant and time varying parametric models the corresponding practical forms of akaike criterion are given in the time varying case this criterion may be used as a way to fit the model into the nonstationarity of an exact filter the possible generalization of the akaike criterion to non ml estimation is discussed and application is made for weighted least squares estimation a more general approach to the nonstationary case is given by assuming a linear stochastic model for the vector of the filter coefficients in which case the ml estimate is computed recursively using the kalman filtering equation then the random sequence of kalman gains is close to a deterministic average sequence this sequence may be precomputed and gives an optimal choice for the gain matrices in an adaptive gradient type filter

identification of piecewise affine systems based on statistical clustering technique this paper is concerned with the identification of a class of piecewise affine systems called a piecewise affine autoregressive exogenous pwarx model the pwarx model is composed of arx sub models each of which corresponds to a polyhedral region of the regression space under the temporary assumption that the number of sub models is known a priori the input output data are collected into several clusters by using a statistical clustering algorithm we utilize support vector classifiers to estimate the boundary hyperplane between two adjacent regions in the regression space in each cluster the parameter vector of the sub model is obtained by the least squares method it turns out that the present statistical clustering approach enables us to estimate the number of sub models based on the information criteria such as caic and mdl the estimate of the number of sub models is performed by applying the identification procedure several times to the same data set after having fixed the number of sub models to different values finally we verify the applicability of the present identification method through a numerical example of a hammerstein model

on model selection and the arc sine laws generalizations of the arc sine laws are shown to provide insight into the operating characteristics of certain techniques for selecting models to fit a given data set when the available models are nested as a corollary one sees that a popular technique may be expected to include about one superfluous parameter even if the sample size is large

multiclus a new method for simultaneously performing multidimensional scaling and cluster analysis this paper develops a maximum likelihood based method for simultaneously performing multidimensional scaling and cluster analysis on two way dominance or profile data this multiclus procedure utilizes mixtures of multivariate conditional normal distributions to estimate a joint space of stimulus coordinates andk vectors one for each cluster or group in at dimensional space the conditional mixture maximum likelihood method is introduced together with an e m algorithm for parameter estimation a monte carlo analysis is presented to investigate the performance of the algorithm as a number of data parameter and error factors are experimentally manipulated finally a consumer psychology application is discussed involving consumer expertise experience with microcomputers

asymptotics of aic bic and rmsea for model selection in structural equation modeling model selection is a popular strategy in structural equation modeling sem to select an optimal model many selection criteria have been proposed in this study we derive the asymptotics of several popular selection procedures in sem including aic bic the rmsea and a two stage rule for the rmsea rmsea 2s all of the results are derived under weak distributional assumptions and can be applied to a wide class of discrepancy functions the results show that both aic and bic asymptotically select a model with the smallest population minimum discrepancy function mdf value regardless of nested or non nested selection but only bic could consistently choose the most parsimonious one under nested model selection when there are many non nested models attaining the smallest mdf value the consistency of bic for the most parsimonious one fails on the other hand the rmsea asymptotically selects a model that attains the smallest population rmsea value and the resea 2s chooses the most parsimonious model from all models with the population rmsea smaller than the pre specified cutoff the empirical behavior of the considered criteria is also illustrated via four numerical examples

biclustering models for two mode ordinal data the work in this paper introduces finite mixture models that can be used to simultaneously cluster the rows and columns of two mode ordinal categorical response data such as those resulting from likert scale responses we use the popular proportional odds parameterisation and propose models which provide insights into major patterns in the data model fitting is performed using the em algorithm and a fuzzy allocation of rows and columns to corresponding clusters is obtained the clustering ability of the models is evaluated in a simulation study and demonstrated using two real data sets

the chic model a global model for coupled binary data often problems result in the collection of coupled data which consist of different n way n mode data blocks that have one or more modes in common to reveal the structure underlying such data an integrated modeling strategy with a single set of parameters for the common mode s that is estimated based on the information in all data blocks may be most appropriate such a strategy implies a global model consisting of different n way n mode submodels and a global loss function that is a weighted sum of the partial loss functions associated with the different submodels in this paper such a global model for an integrated analysis of a three way three mode binary data array and a two way two mode binary data matrix that have one mode in common is presented a simulated annealing algorithm to estimate the model parameters is described and evaluated in a simulation study an application of the model to real psychological data is discussed

high snr consistent linear model order selection and subset selection high snr consistency of model order selection criteria in linear regression models has attracted a lot of attention in the signal processing community recently it is now known that exponentially embedded family versions of minimum description length etc are high snr consistent however a general framework for high snr consistency in linear regression is still missing this paper fills this gap by deriving necessary and sufficient conditions nscs for model order selection criteria in both known and unknown noise variance situations to be high snr consistent many popular model order selection criteria are proved to be high snr consistent using these nscs we also provide a convergence rate analysis to discriminate between various high snr consistent model order selection criteria a direct application of model order selection techniques to the very important subset selection problem is computationally infeasible this paper establish the high snr consistency of an existing t statistics based index ordering scheme that allows the conversion of a subset selection problem into a model order selection problem combining this index ordering with high snr consistent model order selection criteria leads to a novel subset selection procedure ssp that is numerically shown to outperform existing high snr consistent ssps

semiparametric estimation of count regression models this paper develops a semiparametric estimation approach for mixed count regression models based on series expansion for the unknown density of the unobserved heterogeneity we use the generalized laguerre series expansion around a gamma baseline density to model unobserved heterogeneity in a poisson mixture model we establish the consistency of the estimator and present a computational strategy to implement the proposed estimation techniques in the standard count model as well as in truncated censored and zero inflated count regression models monte carlo evidence shows that the finite sample behavior of the estimator is quite good the paper applies the method to a model of individual shopping behavior

factor analysis and aic the information criterion aic was introduced to extend the method of maximum likelihood to the multimodel situation it was obtained by relating the successful experience of the order determination of an autoregressive model to the determination of the number of factors in the maximum likelihood factor analysis the use of the aic criterion in the factor analysis is particularly interesting when it is viewed as the choice of a bayesian model this observation shows that the area of application of aic can be much wider than the conventional i i d type models on which the original derivation of the criterion was based the observation of the bayesian structure of the factor analysis model leads us to the handling of the problem of improper solution by introducing a natural prior distribution of factor loadings

optimal product positioning based on paired comparison data a new approach for analyzing paired comparison data is proposed which combines a probabilistic ideal point model with product positioning issues unlike traditional approaches based on paired comparison data the same formulation is used for estimating a joint space representation of consumer segments and products as well as for determining optimal new product positioning options in a relevant product market a monte carlo experiment is presented and real world coffee market data are used to show advantages of the new approach

mixtures of constrained ultrametric trees this paper presents a new methodology concerned with the estimation of ultrametric trees calibrated on subjects pairwise proximity judgments of stimuli capturing subject heterogeneity using a finite mixture formulation we assume that a number of unobserved classes of subjects exist each having a different ultrametric tree structure underlying the pairwise proximity judgments a new likelihood based estimation methodology is presented for those finite mixtures of ultrametric trees that accommodates ultrametric as well as other external constraints various assumptions on the correlation of the error of the dissimilarities are accommodated the performance of the method to recover known ultrametric tree structures is investigated on synthetic data an empirical application to published data from schiffman reynolds and young 1981 is provided the ability to deal with external constraints on the tree topology is demonstrated and a comparison with an alternative clustering based method is made

density estimation with confidence sets exemplified by superclusters and voids in the galaxies abstract a method is presented for forming both a point estimate and a confidence set of semiparametric densities the final product is a three dimensional figure that displays a selection of density estimates for a plausible range of smoothing parameters the boundaries of the smoothing parameter are determined by a nonparametric goodness of fit test that is based on the sample spacings for each value of the smoothing parameter our estimator is selected by choosing the normal mixture that maximizes a function of the sample spacings a point estimate is selected from this confidence set by using the method of cross validation an algorithm to find the mixing distribution that maximizes the spacings functional is presented these methods are illustrated with a data set from the astronomy literature the measurements are velocities at which galaxies in the corona borealis region are moving away from our galaxy if the galaxies are clustered the velocity density will be multimodal with clusters correspond

selecting a separable parametric spatiotemporal covariance structure for longitudinal imaging data longitudinal imaging studies allow great insight into how the structure and function of a subject s internal anatomy changes over time unfortunately the analysis of longitudinal imaging data is complicated by inherent spatial and temporal correlation the temporal from the repeated measures and the spatial from the outcomes of interest being observed at multiple points in a patients body we propose the use of a linear model with a separable parametric spatiotemporal error structure for the analysis of repeated imaging data the model makes use of spatial exponential spherical and matern and temporal compound symmetric autoregressive 1 toeplitz and unstructured parametric correlation functions a simulation study inspired by a longitudinal cardiac imaging study on mitral regurgitation patients compared different information criteria for selecting a particular separable parametric spatiotemporal correlation structure as well as the effects on type i and ii error rates for inference on fixed effects when the specified model is incorrect information criteria were found to be highly accurate at choosing between separable parametric spatiotemporal correlation structures misspecification of the covariance structure was found to have the ability to inflate the type i error or have an overly conservative test size which corresponded to decreased power an example with clinical data is given illustrating how the covariance structure procedure can be done in practice as well as how covariance structure choice can change inferences about fixed effects

least informative distributions in maximum q log likelihood estimation abstract we use the maximum q log likelihood estimation for least informative distributions lids in order to estimate the parameters in probability density functions pdfs efficiently and robustly when data include outlier s lids are derived by using convex combinations of two pdfs a convex combination of two pdfs is composed of an underlying distribution and a contamination the optimal criterion is obtained by minimizing the change of maximum q log likelihood function when the data contain small amount of contamination in this paper we make a comparison between ordinary maximum likelihood estimation maximum q log likelihood estimation mqle and lids based on mqle for parameter estimation from data with outliers we derive a new fisher information matrix based on the score function for lid from m function and use it for choice of optimal estimator in the class of mqle the model selection is done by the robust information criteria we test the methods on the real data with outliers and estimate shape and scale parameters of probability distributions as a result we show that the lids based on mqle provide the most robust and efficient estimation of the model parameters

byy harmony learning independent state space and generalized apt financial analyses first the relationship between factor analysis fa and the well known arbitrage pricing theory apt for financial market is discussed comparatively with a number of to be improved problems listed an overview is made from a unified perspective on the related studies in the literatures of statistics control theory signal processing and neural networks next we introduce the fundamentals of the bayesian ying yang byy system and the harmony learning principle we further show that a specific case of the framework called byy independent state space iss system provides a general guide for systematically tackling various fa related learning tasks and the above to be improved problems for the apt analyses third on various specific cases of the byy iss system in three typical architectures adaptive algorithms regularization methods and model selection criteria are provided for either or both of parameter learning with automated model selection and parameter learning followed by model selection finally we introduce some other financial applications that are based on the underlying independent factors via the apt analyses

global convergence of the em algorithm for unconstrained latent variable models with categorical indicators convergence of the expectation maximization em algorithm to a global optimum of the marginal log likelihood function for unconstrained latent variable models with categorical indicators is presented the sufficient conditions under which global convergence of the em algorithm is attainable are provided in an information theoretic context by interpreting the em algorithm as alternating minimization of the kullback leibler divergence between two convex sets it is shown that these conditions are satisfied by an unconstrained latent class model yielding an optimal bound against which more highly constrained models may be compared

bayesian information criterion for longitudinal and clustered data when a number of models are fit to the same data set one method of choosing the best model is to select the model for which akaike s information criterion aic is lowest aic applies when maximum likelihood is used to estimate the unknown parameters in the model the value of 2 log likelihood for each model fit is penalized by adding twice the number of estimated parameters the number of estimated parameters includes both the linear parameters and parameters in the covariance structure another criterion for model selection is the bayesian information criterion bic bic penalizes 2 log likelihood by adding the number of estimated parameters multiplied by the log of the sample size for large sample sizes bic penalizes 2 log likelihood much more than aic making it harder to enter new parameters into the model an assumption in bic is that the observations are independent in mixed models the observations are not independent this paper develops a method for calculating the effective sample size for mixed models based on fisher s information the effective sample size replaces the sample size in bic and can vary from the number of subjects to the number of observations a number of error models are considered based on a general mixed model including unstructured compound symmetry random intercept first order autoregression with observational error and random intercept and slope copyright 2011 john wiley sons ltd

factor analysis with mixed observed and latent variables in the exponential family we develop a general approach to factor analysis that involves observed and latent variables that are assumed to be distributed in the exponential family this gives rise to a number of factor models not considered previously and enables the study of latent variables in an integrated methodological framework rather than as a collection of seemingly unrelated special cases the framework accommodates a great variety of different measurement scales and accommodates cases where different latent variables have different distributions the models are estimated with the method of simulated likelihood which allows for higher dimensional factor solutions to be estimated than heretofore the models are illustrated on synthetic data we investigate their performance when the distribution of the latent variables is mis specified and when part of the observations are missing we study the properties of the simulation estimators relative to maximum likelihood estimation with numerical integration we provide an empirical application to the analysis of attitudes

almost nonparametric inference for repeated measures in mixture models summary we consider ways to estimate the mixing proportions in a finite mixture distribution or to estimate the number of components of the mixture distribution without making parametric assumptions about the component distributions we require a vector of observations on each subject this vector is mapped into a vector of os and 1 s and summed the resulting distribution of sums can be modelled as a mixture of binomials we then work with the binomial mixture the efficiency and robustness of this method are compared with the strategy of assuming multivariate normal mixtures when typically the true underlying mixture distribution is different it is shown that in many cases the approach based on simple binomial mixtures is superior let pi x and p2 x be probability distributions with ai and a2 positive real numbers that sum to 1 then p x al pi x a2 p2 x defines a two component mixture distribution with com ponents pl x and p2 x and mixture weights al and a2 mixture distributions arise when observations come from different populations each with a different distribution and the population sampled is unobserved interesting examples include those in the social sciences in which important variables are latent and unobserved as an example consider the problem of trying to infer the solution strategies which young children employ in solving a cognitive task when different solution strategies can be employed a group of same age children can be considered a sample from a mixture distribution in which the components correspond to the various strategies see thomas and horton 1997 recently there has been an expanding interest in mixtures and there is now a large literature examples and theory may be found in many sources including lindsay 1995 mclachlan and basford 1988 titterington et al 1985 and everitt and hand 1981 computational issues in estimating the model parameters rely mainly on the em algorithm see mclachlan and krishnan 1997 for a detailed account two main issues remain unresolved in applications of mixtures one is the specification of the number of components although there may sometimes be substantive theory to guide this specification in exploratory settings

a unified perspective and new results on rht computing mixture based learning and multi learner based problem solving on one hand multiple object detection approaches of hough transform ht type and randomized ht type have been extended into an evidence accumulation featured general framework for problem solving with five key mechanisms elaborated and several extensions of ht and rht presented on the other hand another framework is proposed to integrate typical multi learner based approaches for problem solving particularly on gaussian mixture based data clustering and local subspace learning multi sets mixture based object detection and motion estimation and multi agent coordinated problem solving typical learning algorithms especially those that base on rival penalized competitive learning rpcl and bayesian ying yang byy learning are summarized from a unified perspective with new extensions furthermore the two different frameworks are not only examined with one viewed crossly from a perspective of the other with new insights and extensions but also further unified into a general problem solving paradigm that consists of five basic mechanisms in terms of acquisition allocation amalgamation admission and affirmation or shortly a5 paradigm