low rank doubly stochastic matrix decomposition for cluster analysis cluster analysis by nonnegative low rank approximations has experienced a remarkable progress in the past decade however the majority of such approximation approaches are still restricted to nonnegative matrix factorization nmf and suffer from the following two drawbacks 1 they are unable to produce balanced partitions for large scale manifold data which are common in real world clustering tasks 2 most existing nmf type clustering methods cannot automatically determine the number of clusters we propose a new low rank learning method to address these two problems which is beyond matrix factorization our method approximately decomposes a sparse input similarity in a normalized way and its objective can be used to learn both cluster assignments and the number of clusters for efficient optimization we use a relaxed formulation based on data cluster data random walk which is also shown to be equivalent to low rank factorization of the doubly stochastically normalized cluster incidence matrix the probabilistic cluster assignments can thus be learned with a multiplicative majorization minimization algorithm experimental results show that the new method is more accurate both in terms of clustering large scale manifold data sets and of selecting the number of clusters

bootstrapping a regression equation some empirical results abstract the bootstrap like the jackknife is a technique for estimating standard errors the idea is to use monte carlo simulation based on a nonparametric estimate of the underlying error distribution the main object of this article is to present the bootstrap in the context of an econometric equation describing the demand for energy by industry as it turns out the conventional asymptotic formulas for estimating standard errors are too optimistic by factors of nearly three when applied to a particular finite sample problem in a simpler context this finding can be given a mathematical proof

bootstrap inference for instrumental variable models with many weak instruments this study s main contribution is to theoretically analyze the application of bootstrap methods to instrumental variable models when the available instruments may be weak and the number of instruments goes to infinity with the sample size we demonstrate that a standard residual based bootstrap procedure cannot consistently estimate the distribution of the limited information maximum likelihood estimator or fuller 1977 estimator under many many weak instrument sequence the primary reason is that the standard procedure fails to capture the instrument strength in the sample adequately in addition we consider the restricted efficient re bootstrap of davidson and mackinnon 2008 2010 2014 that generates bootstrap data under the null restricted and uses an efficient estimator of the coefficient of the reduced form equation efficient we find that the re bootstrap is also invalid however it effectively mimics more key features in the limiting distributions of interest and thus is less distorted in finite samples than the standard bootstrap procedure finally we propose modified bootstrap procedures that provide a valid distributional approximation for the two estimators with many many weak instruments a monte carlo experiment shows that hypothesis testing based on the asymptotic normal approximation can have severe size distortions in finite samples instead our modified bootstrap procedures greatly reduce these distortions

bootstrap prediction intervals for autoregression abstract the nonparametric bootstrap is applied to the problem of prediction in autoregression let y t t 0 1 2 be a stationary autoregressive process of known order p ar p given a realization of the series up to time t y 1 y 2 y t a 100 prediction interval for y t k is desired standard forecasting techniques which assume that the error sequence of the process y t is gaussian rely on the fact that the conditional distribution of y t k given the data is gaussian as well as a nonparametric alternative the bootstrap provides an estimate of the conditional distribution of y t k the method is similar to other applications of the bootstrap for linear models because the residuals are resampled the proposed methodology represents a different approach since an alternative representation for ar p series is used allowing for bootstrap replicates generated backward in time it follows that the resulting replicates all have the same conditionally fixed values at the end

k means type algorithms a generalized convergence theorem and characterization of local optimality the k means algorithm is a commonly used technique in cluster analysis in this paper several questions about the algorithm are addressed the clustering problem is first cast as a nonconvex mathematical program then a rigorous proof of the finite convergence of the k means type algorithm is given for any metric it is shown that under certain conditions the algorithm may fail to converge to a local minimum and that it converges under differentiability conditions to a kuhn tucker point finally a method for obtaining a local minimum solution is given

bootstrapping regression models the regression and correlation models are considered it is shown that the bootstrap approximation to the distribution of the least squares estimates is valid and some error bounds are given

higher order multivariate markov chains and their applications abstract markov chains are commonly used in modeling many practical systems such as queuing systems manufacturing systems and inventory systems they are also effective in modeling categorical data sequences in a conventional n th order multivariate markov chain model of s chains and each chain has the same set of m states the total number of parameters required to set up the model is o m ns such huge number of states discourages researchers or practitioners from using them directly in this paper we propose an n th order multivariate markov chain model for modeling multiple categorical data sequences such that the total number of parameters are of o ns 2 m 2 the proposed model requires significantly less parameters than the conventional one we develop efficient estimation methods for the model parameters an application to demand predictions in inventory control is also discussed

some asymptotic theory for the bootstrap efron s bootstrap method of distribution approximation is shown to be asymptotically valid in a large number of situations including t statistics the empirical and quantile processes and von mises functionals some counter examples are also given to show that the approximation does not always succeed

convex and semi nonnegative matrix factorizations we present several new variations on the theme of nonnegative matrix factorization nmf considering factorizations of the form x fgt we focus on algorithms in which g is restricted to containing nonnegative entries but allowing the data matrix x to have mixed signs thus extending the applicable range of nmf methods we also consider algorithms in which the basis vectors of f are constrained to be convex combinations of the data points this is used for a kernel extension of nmf we provide algorithms for computing these new factorizations and we provide supporting theoretical analysis we also analyze the relationships between our algorithms and clustering algorithms and consider the implications for sparseness of solutions finally we present experimental results that explore the properties of these new methods

variable length markov chains we study estimation in the class of stationary variable length markov chains vlmc on a finite space the processes in this class are still markovian of high order but with memory of variable length yielding a much bigger and structurally richer class of models than ordinary high order markov chains from an algorithmic view the vlmc model class has attracted interest in information theory and machine learning but statistical properties have not yet been explored provided that good estimation is available the additional structural richness of the model class enhances predictive power by finding a better tradeoff between model bias and variance and allowing better structural description which can be of specific interest the latter is exemplified with some dna data a version of the tree structured context algorithm proposed by rissanen in an information theoretical setup is shown to have new good asymptotic properties for estimation in the class of vlmcs this remains true even when the underlying model increases in dimensionality fur thermore consistent estimation of minimal state spaces and mixing properties of fitted models are given we also propose a new bootstrap scheme based on fitted vlmcs we show its validity for quite general stationary categorical time series and for a broad range of statistical procedures

recursive estimation of high order markov chains high order markov chain is approximated by mixture of low order markov chains corresponding bayesian approximate recursive estimator is proposed forgetting counteracting error accumulation is tailored to this case a simple example related to ultimatum game indicates the power of the approach a high order markov chain is a universal model of stochastic relations between discrete valued variables the exact estimation of its transition probabilities suffers from the curse of dimensionality it requires an excessive amount of informative observations as well as an extreme memory for storing the corresponding sufficient statistic the paper bypasses this problem by considering a rich subset of markov chain models namely mixtures of low dimensional markov chains possibly with external variables it uses bayesian approximate estimation suitable for a subsequent decision making under uncertainty the proposed recursive sequential one pass estimator updates a product of dirichlet probability densities pds used as an approximate posterior pd projects the result back to this class of pds and applies an improved data dependent stabilised forgetting which counteracts the dangerous accumulation of approximation errors

subsampling for heteroskedastic time series abstract in this article a general theory for the construction of confidence intervals or regions in the context of heteroskedastic dependent data is presented the basic idea is to approximate the sampling distribution of a statistic based on the values of the statistic computed over smaller subsets of the data this method was first proposed by politis and romano 1994b for stationary observations we extend their results to heteroskedastic observations and prove a general asymptotic validity result under minimal conditions in contrast the usual bootstrap and moving blocks bootstrap are typically valid only for asymptotically linear statistics and their justification requires a case by case analysis our general asymptotic results are applied to a regression setting with dependent heteroskedastic errors

double preference relations for generalised belief change many belief change formalisms employ plausibility orderings over the set of possible worlds to determine how the beliefs of an agent ought to be modified after the receipt of a new epistemic input while most such possible world semantics rely on a single ordering we investigate the use of an additional preference ordering representing for instance the epistemic context the agent finds itself in to guide the process of belief change we show that the resultant formalism provides a unifying semantics for a wide variety of belief change operators by varying the conditions placed on the second ordering different families of known belief change operators can be captured including agm belief contraction and revision rott and pagnucco s severe withdrawal the systematic withdrawal of meyer et al as well as the linear liberation and s liberation operators of booth et al our approach also identifies novel classes of belief change operators worthy of further investigation

heuristic algorithms for the cardinality constrained efficient frontier abstract this paper examines the application of genetic algorithm tabu search and simulated annealing metaheuristic approaches to finding the cardinality constrained efficient frontier that arises in financial portfolio optimisation we consider the mean variance model of markowitz as extended to include the discrete restrictions of buy in thresholds and cardinality constraints computational results are reported for publicly available data sets drawn from seven major market indices involving up to 1318 assets our results are compared with previous results given in the literature illustrating the effectiveness of the proposed metaheuristics in terms of solution quality and computation time

a model for high order markov chains on introduit un modele pour des chaines de markov du leme ordre qui comporte seulement un parametre additionnel pour chaque retard

structured partitioning problems in many important combinatorial optimization problems such as bin packing allocating customer classes to queueing facilities vehicle routing multi item inventory replenishment and combined routing inventory control an optimal partition into groups needs to be determined for a finite collection of objects each is characterized by a single attribute the cost is often separable in the groups and the group cost often depends on the cardinality and some aggregate measure of the attributes such as the sum or the maximum element an upper bound capacity may be specified for the cardinality of each group and the number of groups in the partition may either be fixed or variable the objects are indexed in nondecreasing order of their attribute values and within a given partition the groups are indexed in nondecreasing order of their cardinalities we identify easily verifiable analytical properties of the group cost function under which it is shown that an optimal partition exists of one of three increas

a high order markov switching model for risk measurement in this paper we introduce a high order markov switching hms model for measuring the risk of a portfolio we suppose that the rate of return from a risky portfolio follows an hms model with the drift and the volatility modulated by a discrete time weak markov chain the states of the weak markov chain are interpreted as observable states of an economy we adopt the value at risk var as a metric for market risk quantification and examine the high order effect of the underlying markov chain on the risk measures via backtesting

non negative matrix factorization with sparseness constraints non negative matrix factorization nmf is a recently developed technique for finding parts based linear representations of non negative data although it has successfully been applied in several applications it does not always result in parts based representations in this paper we show how explicitly incorporating the notion of sparseness improves the found decompositions additionally we provide complete matlab code both for standard nmf and for our extension our hope is that this will further the application of these methods to solving novel data analysis problems

sieve bootstrap with variable length markov chains for stationary categorical time series a bootstrap for stationary categorical time series based on the method of sieves is studied here the data generating process is approximated by the so called variable length markov chain vlmc a flexible class of markov models that allows for parsimonious structure then the resampling is given by simulating from the fitted model it is shown that for a whole class of stationary categorical time series that is more general than vlmc the vlmc sieve has faster rate of convergence for variance estimation than the more general block bootstrap results are illustrated from a theoretical and empirical perspective for the latter a real data application about in homogeneity classification of a dna strand is also presented finally the vlmc sieve scheme enjoys an implementational advantage of using the plug in rule for bootstrapping a statistical procedure which generally is not the case for the block method

calibrating prediction regions abstract suppose that the variable x to be predicted and the learning sample yn that was observed have a joint distribution which depends on an unknown parameter the parameter can be finite or infinite dimensional a prediction region dn for x is a random set depending on yn that contains x with prescribed probability this article studies methods for controlling simultaneously the conditional coverage proability of dn given yn and the overall unconditional coverage probability of dn the basic construction yields a prediction region dn which has the following properties in regular models both the conditional and overall coverage probabilities of dn converge to as the size n of the learning sample increases the convergence of the former is in probability moreover the asymptotic distribution of the conditional coverage probability about is typically normal and the overall coverage probability tends to at rate n 1 can one reduce the dispersion of the conditional coverage pr

bootstrap confidence regions computed from autoregressions of arbitrary order summary given a linear time series e g an autoregression of infinite order we may construct a finite order approximation and use that as the basis for bootstrap confidence regions the sieve or autoregressive bootstrap as this method is often called is generally seen as a competitor with the better understood block bootstrap approach however in the present paper we argue that for linear time series the sieve bootstrap has significantly better performance than blocking methods and offers a wider range of opportunities in particular since it does not corrupt second order properties then it may be used in a double bootstrap form with the second bootstrap application being employed to calibrate a basic percentile method confidence interval this approach confers secondorder accuracy without the need to estimate variance that offers substantial benefits since variances of statistics based on time series can be difficult to estimate reliably and partly because of the relatively small amount of information contained in a dependent process are notorious for causing problems when used to studentize other advantages of the sieve bootstrap include considerably greater robustness against variations in the choice of the tuning parameter here equal to the autoregressive order and the fact that in contradistinction to the case of the block bootstrap the percentile t version of the sieve bootstrap may be based on the raw estimator of standard error in the process of establishing these properties we show that the sieve bootstrap is second order correct

similarity based clustering by left stochastic matrix factorization for similarity based clustering we propose modeling the entries of a given similarity matrix as the inner products of the unknown cluster probabilities to estimate the cluster probabilities from the given similarity matrix we introduce a left stochastic non negative matrix factorization problem a rotation based algorithm is proposed for the matrix factorization conditions for unique matrix factorizations and clusterings are given and an error bound is provided the algorithm is particularly efficient for the case of two clusters which motivates a hierarchical variant for cases where the number of desired clusters is large experiments show that the proposed left stochastic decomposition clustering model produces relatively high within cluster similarity on most data sets and can match given class labels and that the efficient hierarchical variant performs surprisingly well

an optimal graph theoretic approach to data clustering theory and its application to image segmentation a novel graph theoretic approach for data clustering is presented and its application to the image segmentation problem is demonstrated the data to be clustered are represented by an undirected adjacency graph g with arc capacities assigned to reflect the similarity between the linked vertices clustering is achieved by removing arcs of g to form mutually exclusive subgraphs such that the largest inter subgraph maximum flow is minimized for graphs of moderate size approximately 2000 vertices the optimal solution is obtained through partitioning a flow and cut equivalent tree of g which can be efficiently constructed using the gomory hu algorithm 1961 however for larger graphs this approach is impractical new theorems for subgraph condensation are derived and are then used to develop a fast algorithm which hierarchically constructs and partitions a partially equivalent tree of much reduced size this algorithm results in an optimal solution equivalent to that obtained by partitioning the complete equivalent tree and is able to handle very large graphs with several hundred thousand vertices the new clustering algorithm is applied to the image segmentation problem the segmentation is achieved by effectively searching for closed contours of edge elements equivalent to minimum cuts in g which consist mostly of strong edges while rejecting contours containing isolated strong edges this method is able to accurately locate region boundaries and at the same time guarantees the formation of closed edge contours

propositional knowledge base revision and minimal change abstract the semantics of revising knowledge bases represented by sets of propositional sentences is analyzed from a model theoretic point of view a characterization of all revision schemes that satisfy the gardenfors rationality postulates is given in terms of minimal change with respect to an ordering among interpretations revision methods proposed by various authors are surveyed and analyzed in this framework the correspondences between gardenfors like rationality postulates and minimal changes with respect to other orderings are also investigated

nonmonotonic reasoning preferential models and cumulative logics abstract many systems that exhibit nonmonotonic behavior have been described and studied already in the literature the general notion of nonmonotonic reasoning though has almost always been described only negatively by the property it does not enjoy i e monotonicity we study here general patterns of nonmonotonic reasoning and try to isolate properties that could help us map the field of nonmonotonic reasoning by reference to positive properties we concentrate on a number of families of nonmonotonic consequence relations defined in the style of gentzen 13 both proof theoretic and semantic points of view are developed in parallel the former point of view was pioneered by gabbay 10 while the latter has been advocated by shoham 38 five such families are defined and characterized by representation theorems relating the two points of view one of the families of interest that of preferential relations turns out to have been studied by adams 2 the preferential models proposed here are a much stronger tool than adams probabilistic semantics the basic language used in this paper is that of propositional logic the extension of our results to first order predicate calculi and the study of the computational complexity of the decision problems described in this paper will be treated in another paper

on the logic of iterated belief revision abstract we show in this paper that the agm postulates are too weak to ensure the rational preservation of conditional beliefs during belief revision thus permitting improper responses to sequences of observations we remedy this weakness by proposing four additional postulates which are sound relative to a qualitative version of probabilistic conditioning contrary to the agm framework the proposed postulates characterize belief revision as a process which may depend on elements of an epistemic state that are not necessarily captured by a belief set we also show that a simple modification to the agm framework can allow belief revision to be a function of epistemic states we establish a model based representation theorem which characterizes the proposed postulates and constrains in turn the way in which entrenchment orderings may be transformed under iterated belief revision

robust perron cluster analysis in conformation dynamics the key to molecular conformation dynamics is the direct identification of metastable conformations which are almost invariant sets of molecular dynamical systems once some reversible markov operator has been discretized a generalized symmetric stochastic matrix arises this matrix can be treated by perron cluster analysis a rather recent method involving a perron cluster eigenproblem the paper presents an improved perron cluster analysis algorithm which is more robust than earlier suggestions numerical examples are included

a tabu search based heuristic for clustering abstract this paper considers a clustering problem where a given data set is partitioned into a certain number of natural and homogeneous subsets such that each subset is composed of elements similar to one another but different from those of any other subset for the clustering problem a heuristic algorithm is exploited by combining the tabu search heuristic with two complementary functional procedures called packing and releasing procedures the algorithm is numerically tested for its effectiveness in comparison with reference works including the tabu search algorithm the k means algorithm and the simulated annealing algorithm

on blocking rules for the bootstrap with dependent data summary we address the issue of optimal block choice in applications of the block bootstrap to dependent data it is shown that optimal block size depends significantly on context being equal to n 3 n 4 and nll5 in the cases of variance or bias estimation estimation of a onesided distribution function and estimation of a two sided distribution function respectively a clear intuitive explanation of this phenomenon is given together with outlines of theoretical arguments in specific cases it is shown that these orders of magnitude of block sizes can be used to produce a simple practical rule for selecting block size empirically that technique is explored numerically

a c eigenvalue problem for tensors with applications to higher order multivariate markov chains abstract in this paper we study a new tensor eigenvalue problem which involves e and s eigenvalues as its special cases some theoretical results such as existence of an eigenvalue and the number of eigenvalues are given for an application of the proposed eigenvalue problem we establish a tensor model for a higher order multivariate markov chain the core issue of this problem is to study a stationary probability distribution of a higher order multivariate markov chain a sufficient condition of the unique stationary positive distribution is given an algorithm for computing stationary probability distribution is also developed numerical examples of applications in stock market modeling sales demand prediction and biological sequence analysis are given to illustrate the proposed tensor model and the computed stationary probability distribution can provide a better prediction in these markov chain applications

a tabu search heuristic procedure in markov chain bootstrapping abstract markov chain theory is proving to be a powerful approach to bootstrap finite states processes especially where time dependence is non linear in this work we extend such approach to bootstrap discrete time continuous valued processes to this purpose we solve a minimization problem to partition the state space of a continuous valued process into a finite number of intervals or unions of intervals i e its states and identify the time lags which provide memory to the process a distance is used as objective function to stimulate the clustering of the states having similar transition probabilities the problem of the exploding number of alternative partitions in the solution space which grows with the number of states and the order of the markov chain is addressed through a tabu search algorithm the method is applied to bootstrap the series of the german and spanish electricity prices the analysis of the results confirms the good consistency properties of the method we propose