layered dynamic textures a novel video representation the layered dynamic texture ldt is proposed the ldt is a generative model which represents a video as a collection of stochastic layers of different appearance and dynamics each layer is modeled as a temporal texture sampled from a different linear dynamical system the ldt model includes these systems a collection of hidden layer assignment variables which control the assignment of pixels to layers and a markov random field prior on these variables which encourages smooth segmentations an em algorithm is derived for maximum likelihood estimation of the model parameters from a training video it is shown that exact inference is intractable a problem which is addressed by the introduction of two approximate inference procedures a gibbs sampler and a computationally efficient variational approximation the trade off between the quality of the two approximations and their complexity is studied experimentally the ability of the ldt to segment videos into layers of coherent appearance and dynamics is also evaluated on both synthetic and natural videos these experiments show that the model possesses an ability to group regions of globally homogeneous but locally heterogeneous stochastic dynamics currently unparalleled in the literature

tensor product of n isometries abstract a hilbert space operator a is an m isometry if k 0 m 1 k m k a m k a m k 0 we prove that the tensor product of an m isometry a with an n isometry b is an m n 1 isometry this proves a conjecture of botelho and jamison 5 remark 5 1

multi modal clique graph matching for view based 3d model retrieval multi view matching is an important but a challenging task in view based 3d model retrieval to address this challenge we propose an original multi modal clique graph mcg matching method in this paper we systematically present a method for mcg generation that is composed of cliques which consist of neighbor nodes in multi modal feature space and hyper edges that link pairwise cliques moreover we propose an image set based clique edgewise similarity measure to address the issue of the set to set distance measure which is the core problem in mcg matching the proposed mcg provides the following benefits 1 preserves the local and global attributes of a graph with the designed structure 2 eliminates redundant and noisy information by strengthening inliers while suppressing outliers and 3 avoids the difficulty of defining high order attributes and solving hyper graph matching we validate the mcg based 3d model retrieval using three popular single modal data sets and one novel multi modal data set extensive experiments show the superiority of the proposed method through comparisons moreover we contribute a novel real world 3d object data set the multi view rgb d object data set to the best of our knowledge it is the largest real world 3d object data set containing multi modal and multi view information

statistical clustering of temporal networks through a dynamic stochastic block model statistical node clustering in discrete time dynamic networks is an emerging field that raises many challenges here we explore statistical properties and frequentist inference in a model that combines a stochastic block model sbm for its static part with independent markov chains for the evolution of the nodes groups through time we model binary data as well as weighted dynamic random graphs with discrete or continuous edges values our approach motivated by the importance of controlling for label switching issues across the different time steps focuses on detecting groups characterized by a stable within group connectivity behavior we study identifiability of the model parameters propose an inference procedure based on a variational expectation maximization algorithm as well as a model selection criterion to select for the number of groups we carefully discuss our initialization strategy which plays an important role in the method and compare our procedure with existing ones on synthetic datasets we also illustrate our approach on dynamic contact networks one of encounters among high school students and two others on animal interactions an implementation of the method is available as a r package called dynsbm

m isometric operators and their local properties abstract in this paper we give necessary and sufficient conditions for a bounded linear hilbert space operator to be an m isometry for an unspecified m written in terms of conditions that are applied to one vector at a time we provide criteria for orthogonality of generalized eigenvectors of an a priori unbounded linear operator t on an inner product space that correspond to distinct eigenvalues of modulus 1 we also discuss a similar question of when jordan blocks of t corresponding to distinct eigenvalues of modulus 1 are orthogonal

k vectors an alternating minimization algorithm for learning regression functions the k vectors algorithm for learning regression functions proposed here is akin to the well known k means algorithm both algorithms partition the feature space but unlike the k means algorithm the k vectors algorithm aims to reconstruct the response rather than the feature the partitioning rule of the algorithm is based on maximizing the correlation inner product of the feature vector with a set of k vectors and generates polyhedral cells similar to the ones generated by the nearest neighbor rule of the k means algorithm similarly to k means the learning algorithm alternates between two types of steps in the first type of steps k labels are determined via a centroid type rule in the response space which uses a surrogate hinge type loss function to the mean squared error loss function in the second type of steps the k vectors which determine the partition are updated according to a multiclass classification rule in the spirit of support vector machines it is proved that both steps of the algorithm only require solving convex optimization problems and that the algorithm is empirically consistent as the length of the training sequence increases to infinity fixed points of the empirical version of the algorithm tend to fixed points of the population version of the algorithm learnability of the predictor class posit by the algorithm is also established

adaptive iterative detectors for phase uncertain channels via variational bounding the problem of iterative detection decoding of data symbols transmitted over an additive white gaussian noise awgn channel in the presence of phase uncertainty is addressed in this paper by modelling the phase uncertainty either as an unknown deterministic variable process or random variable process with a known a priori probability density function a number of non bayesian and bayesian detection algorithms with various amount of suboptimality have been proposed in the literature to solve the problem in this paper a new set of suboptimal iterative detection algorithms is obtained by utilizing the variational bounding technique especially applying the generic variational bayesian vb framework efficient iterative joint estimation and detection decoding schemes are derived for the constant phase model as well as for the dynamic phase model in addition the relation of the vb based approach to the optimal noncoherent receiver as well as to the classical approach via the expectation maximization em algorithm is provided performance of the proposed detectors in the presence of a strong dynamic phase noise is compared to the performance of the existing detectors furthermore an incremental scheduling of the vb or em algorithm is shown to reduce the overall complexity of the receiver

on higher order selfadjoint operators abstract for a positive integer m and a hilbert space h an operator t in b h the space of all bounded linear operators on h is called m selfadjoint if k 0 m 1 k m k t k t m k 0 in this paper we show that if t b h and the spectrum of t consists of a finite number of points then it is m selfadjoint if and only if it is an n jordan operator for some integer n moreover we prove that if t is m selfadjoint then t is nilpotent when it is quasinilpotent then we characterize m selfadjoint weighted shift operators also we show that if t is m selfadjoint then so is p t when p z is a polynomial with real coefficients after that we investigate an elementary operator and a generalized derivation operator on the hilbert schmidt class c 2 h which are m selfadjoint finally we prove that no m selfadjoint operator on an infinite dimensional hilbert space can be n supercyclic for any n 1

decomposing algebraic m isometric tuples abstract we show that any m isometric tuple of commuting algebraic operators on a hilbert space can be decomposed as a sum of a spherical isometry and a commuting nilpotent tuple our approach applies as well to tuples of algebraic operators that are hereditary roots of polynomials in several variables

a convergence theorem for the fuzzy subspace clustering fsc algorithm we establish the convergence of the fuzzy subspace clustering fsc algorithm by applying zangwill s convergence theorem we show that the iteration sequence produced by the fsc algorithm terminates at a point in the solution set s or there is a subsequence converging to a point in s in addition we present experimental results that illustrate the convergence properties of the fsc algorithm in various scenarios

a proof of convergence of the concave convex procedure using zangwill s theory the concave convex procedure cccp is an iterative algorithm that solves d c difference of convex functions programs as a sequence of convex programs in machine learning cccp is extensively used in many learning algorithms including sparse support vector machines svms transductive svms and sparse principal component analysis though cccp is widely used in many applications its convergence behavior has not gotten a lot of specific attention yuille and rangarajan analyzed its convergence in their original paper however we believe the analysis is not complete the convergence of cccp can be derived from the convergence of the d c algorithm dca proposed in the global optimization literature to solve general d c programs whose proof relies on d c duality in this note we follow a different reasoning and show how zangwill s global convergence theory of iterative algorithms provides a natural framework to prove the convergence of cccp this underlines zangwill s theory as a powerful and general framework to deal with the convergence issues of iterative algorithms after also being used to prove the convergence of algorithms like expectation maximization and generalized alternating minimization in this note we provide a rigorous analysis of the convergence of cccp by addressing two questions when does cccp find a local minimum or a stationary point of the d c program under consideration and when does the sequence generated by cccp converge we also present an open problem on the issue of local convergence of cccp

a m isometries on hilbert spaces abstract a bounded linear operator t on a hilbert space h is called an a m isometry for some positive operator a on h and integer m if k 0 m 1 m k m k t k a t k 0 we give some properties of a m isometries in particular we focus on spectral properties and the relation between a m isometries and m isometries also we obtain some dynamic properties of a m isometries as a negative answer to 22 question 1 with an example of an a isometric which is n supercyclic and sufficient conditions for an a m isometry to be not n supercyclic moreover we prove that the perturbation of a m isometry by a bigger class than nilpotent operators is not n supercyclic

elementary properties of isometries on a hilbert space inspired by recent works on m isometries for a positive integer m in this paper we introduce the classes of isometries and unitaries on a hilbert space we show that an isometry on a finite dimensional complex hilbert space h with dimension n is in fact an 2n 1 isometry we describe the spectra of such operators study the quasinilpotent perturbations of isometries and characterize when tensor products of isometries are also isometries as a surprising by product we obtain a generalization of nagy foias langer decomposition of a contraction into an unitary and a completely nonunitary contraction

new consistent and asymptotically normal parameter estimates for random graph mixture models summary random graph mixture models are very popular for modelling real data networks parameter estimation procedures usually rely on variational approximations either combined with the expectation maximization em algorithm or with bayesian approaches despite good results on synthetic data the validity of the variational approximation is however not established moreover these variational approaches aim at approximating the maximum likelihood or the maximum a posteriori estimators whose behaviour in an asymptotic framework as the sample size increases to remains unknown for these models in this work we show that in many different affiliation contexts for binary or weighted graphs parameter estimators based either on moment equations or on the maximization of some composite likelihood are strongly consistent and p n convergent when the number n of nodes increases to as a consequence our result establishes that the overall structure of an affiliation model can be asymptotically caught by the description of the network in terms of its number of triads order 3 structures and edges order 2 structures moreover these parameter estimates are either explicit as for the moment estimators or may be approximated by using a simple em algorithm whose convergence properties are known we illustrate the efficiency of our method on simulated data and compare its performances with other existing procedures a data set of cross citations among economics journals is also analysed

adomian s decomposition method for solving an intermediate fractional advection dispersion equation this paper is concerned with a model that describes the intermediate process between advection and dispersion via fractional derivative in the caputo sense adomian s decomposition method is used for solving this model the solution is obtained as an infinite series which always converges to the exact solution

an isometry plus a nilpotent operator is an m isometry applications abstract we prove that if an isometry a and a nilpotent operator q of order n commute then a q is a strict 2 n 1 isometry as an application of the main result we prove that a q cannot be n supercyclic for any n also we find an m isometry with prescribed spectrum k where k is the closed unit disk or a compact subset of the unit circle

a convergence theorem for graph shift type algorithms the robust graph mode seeking by graph shift liu and yan 2010 rggs algorithm represents a recent promising approach for discovering dense subgraphs in noisy data however there are no theoretical foundations for proving the convergence of the rggs algorithm leaving the question as to whether an algorithm works for solid reasons in this paper we propose a generic theoretical framework consisting of three key graph shift gs components the simplex of a generated sequence set the monotonic and continuous objective function and closed mapping we prove that the gs type algorithms built on such components can be transformed to fit zangwill s theory and the sequence set generated by the gs procedures always terminates at a local maximum or at worst contains a subsequence which converges to a local maximum of the similarity measure function the framework is verified by theoretical analysis and experimental results of several typical gs type algorithms highlightswe theoretically analyze the behaviors of the rggs algorithm we prove that the rggs algorithm is convergent with zangwill s theory a convergence proof framework is built to apply to other gs type algorithms

classification of hereditary matrices abstract a classical approach used to obtain basic facts in the theory of square matrices involves an analysis of the relationship between polynomials p in one variable and square matrices t such that p t 0 we consider matrices and operators which satisfy a different type of polynomial constraint let h be a complex hilbert space t be a bounded linear transformation of h t be the adjoint of t and c x y be the algebra of polynomials in x and y with complex coefficients for a polynomial p c x y in two variables with complex coefficients define p t m n 0 p m n t n t m where p m n is the coefficient of ynxm in the expansion of p in a power series about the point 0 0 t is called a root of p if and only if p t 0 note that if p c x y is a polynomial in the single variable x then the definition of p t given here agrees with the classical definition in this paper we study the relationships which p t 0 forces between p and t when t is an algebraic operator i e there exists n 1 and complex numbers a0 an 1 such that 0 a0 a1t an 1tn 1 tn the classification starts with the following observation suppose p c x y and an algebraic operator t l h satisfy p t 0 then certain subspaces of h which are invariant for t must be orthogonal or certain coefficients of p must vanish this leads to the notions of a graph attached to each p c x y and a graph attached to each square matrix t for diagonalizable t a necessary and sufficient graph theoretic condition for solving p t 0 is given for nondiagonalizable t this condition is necessary but not sufficient the use of these graphs does however reduce the problem to the problem of solving the equation p t 0 for t with exactly one or two eigenvalues for t with one eigenvalue we give a necessary and sufficient condition for solving p t 0 this leaves the case of solving p t 0 when t has exactly two eigenvalues this problem mixes algebra involving polynomials with matrix theory we show that it is equivalent to the purely algebraic problem of determining if equations of the form i j eci jxi r j s 0 have solutions of finite support with certain nonvanishing properties we call these equations bi hankel equations subordinate to a given subset e of the lattice of integer pairs i j 0 i m 1 0 j n 1 it turns out that there is an algorithm which uses grobner bases for determining if the type of solution we seek exists and for computing it

algebraic properties of operator roots of polynomials abstract properties of m selfadjoint and m isometric operators have been investigated by several researchers particularly interesting to us are algebraic properties of nilpotent perturbations of such operators mccullough and rodman showed in the nineties that if q n 0 and a is a selfadjoint operator commuting with q then the sum a q is a 2 n 1 selfadjoint operator very recently bermudez martinon and noda proved a similar result for nilpotent perturbations of isometries via a new approach we obtain simple proofs of these results and other generalizations to operator roots of polynomials

pattern alternating maximization algorithm for missing data in high dimensional problems we propose a novel and efficient algorithm for maximizing the observed log likelihood of a multivariate normal data matrix with missing values we show that our procedure based on iteratively regressing the missing on the observed variables generalizes the standard em algorithm by alternating between different complete data spaces and performing the e step incrementally in this non standard setup we prove numerical convergence to a stationary point of the observed log likelihood for high dimensional data where the number of variables may greatly exceed sample size we perform regularization using a lasso type penalty this introduces sparsity in the regression coefficients used for imputation permits fast computation and warrants competitive performance in terms of estimating the missing entries we show on simulated and real data that the new method often improves upon other modern imputation techniques such as k nearest neighbors imputation nuclear norm minimization or a penalized likelihood approach with an l1 penalty on the concentration matrix

some results on higher order isometries and symmetries products and sums with a nilpotent operator abstract we study the sum of an m isometry or an m symmetric operator with a nilpotent operator we also study the product of two m isometries or two n symmetries we obtain several theorems generalizing previous work our method is straightforward and thus offers direct and simple insights into why these theorems are true

lattice fractional diffusion equation in terms of a riesz caputo difference abstract a fractional difference is defined by the use of the right and the left caputo fractional differences the definition is a two sided operator of riesz type and introduces back and forward memory effects in space difference then a fractional difference equation method is suggested for anomalous diffusion in discrete finite domains a lattice fractional diffusion equation is proposed and the numerical simulation of the diffusion process is discussed for various difference orders the result shows that the riesz difference model is particularly suitable for modeling complicated dynamical behaviors on discrete media

elementary operators which are m isometries abstract the study of m isometry elementary operator of length one is initiated by botelho and jamison several authors have contributed to obtain this result if the two operator coefficients involved are m isometries then the elementary operator is m isometry on the hilbert schmidt operator ideal in this paper we show that the converse of this result is also true we also give a necessary and sufficient condition for the generalized derivation to be an m isometry the sufficiency follows from results on the sum of m isometries and nilpotent operators in two recent papers

products of m isometries abstract an operator t on a banach space x is called an m p isometry if it satisfies the equality k 0 m m k 1 m k t k x p 0 for all x x in this paper we prove that if t is an n p isometry s is an m p isometry and they commute then ts is an m n 1 p isometry this result applied to elementary operators of length 1 defined on the hilbert schmidt class proves a conjecture in 11

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

exp function method for traveling wave solutions of nonlinear evolution equations in this paper we apply the exp function method to construct generalized solitary and periodic solutions of nonlinear evolution equations the proposed technique is tested on the modified zakharov kuznetsov zk and zakharov kuznetsov modified equal width zk mew equations these equations play a very important role in mathematical physics and engineering sciences the suggested algorithm is quite efficient and is practically well suited for use in these problems numerical results clearly indicate the reliability and efficiency of the proposed exp function method

complexiton solutions of the toda lattice equation a set of coupled conditions consisting of differential difference equations is presented for casorati determinants to solve the toda lattice equation one class of the resulting conditions leads to an approach for constructing complexiton solutions to the toda lattice equation through the casoratian formulation an analysis is made for solving the resulting system of differential difference equations thereby providing the general solution yielding eigenfunctions required for forming complexitons moreover a feasible way is presented to compute the required eigenfunctions along with examples of real complexitons of lower order

variational iteration method some recent results and new interpretations this paper is an elementary introduction to the concepts of variational iteration method first the main concepts in variational iteration method such as general lagrange multiplier restricted variation correction functional are explained heuristically subsequently the solution procedure is systematically addressed in particular for nonlinear oscillators particular attention is paid throughout the paper to give an intuitive grasp for the method the main motivation is to put things together in a convenient form for later reference and systematic use

a fast improvement to the em algorithm on its own terms the em algorithm is a numerical technique for the evaluation of maximum likelihood estimates for parameters describing incomplete data models it is easy to apply in many problems and is stable but slow the algorithm fails to provide a consistent estimator of the standard errors of the maximum likelihood estimates unless the additional analysis required by the louis method is performed newton type or other gradient methods are faster and provide error estimates but tend to be unstable and require the analytical evaluation of likelihoods to derive expressions for the score function and at least approximations to the fisher information matrix the purpose of this paper is to expand on a result by fisher that permits a unification of em methodology and newton methods the evaluation of the individual observation by observation score functions of the incomplete data is a byproduct of the application of the e step of the em algorithm once these become available the fisher information matrix may be consistently estimated and the m step may be replaced by a fast newton type step

regularized image reconstruction algorithms for dual isotope myocardial perfusion spect mps imaging using a cross tracer prior in simultaneous dual isotope myocardial perfusion spect mps imaging data are simultaneously acquired to determine the distributions of two radioactive isotopes the goal of this work was to develop penalized maximum likelihood pml algorithms for a novel cross tracer prior that exploits the fact that the two images reconstructed from simultaneous dual isotope mps projection data are perfectly registered in space we first formulated the simultaneous dual isotope mps reconstruction problem as a joint estimation problem a cross tracer prior that couples voxel values on both images was then proposed we developed an iterative algorithm to reconstruct the mps images that converges to the maximum a posteriori solution for this prior based on separable surrogate functions to accelerate the convergence we developed a fast algorithm for the cross tracer prior based on the complete data os em cosem framework the proposed algorithm was compared qualitatively and quantitatively to a single tracer version of the prior that did not include the cross tracer term quantitative evaluations included comparisons of mean and standard deviation images as well as assessment of image fidelity using the mean square error we also evaluated the cross tracer prior using a three class observer study with respect to the three class mps diagnostic task i e classifying patients as having either no defect reversible defect or fixed defects for this study a comparison with conventional ordered subsets expectation maximization os em reconstruction with postfiltering was performed the comparisons to the single tracer prior demonstrated similar resolution for areas of the image with large intensity changes and reduced noise in uniform regions the cross tracer prior was also superior to the single tracer version in terms of restoring image fidelity results of the three class observer study showed that the proposed cross tracer prior and the convergent algorithms improved the image quality of dual isotope mps images compared to os em

combining monte carlo and mean field like methods for inference in hidden markov random fields issues involving missing data are typical settings where exact inference is not tractable as soon as nontrivial interactions occur between the missing variables approximations are required and most of them are based either on simulation methods or on deterministic variational methods while variational methods provide fast and reasonable approximate estimates in many scenarios simulation methods offer more consideration of important theoretical issues such as accuracy of the approximation and convergence of the algorithms but at a much higher computational cost in this work we propose a new class of algorithms that combine the main features and advantages of both simulation and deterministic methods and consider applications to inference in hidden markov random fields hmrfs these algorithms can be viewed as stochastic perturbations of variational expectation maximization vem algorithms which are not tractable for hmrf we focus more specifically on one of these perturbations and we prove their almost sure convergence to the same limit set as the limit set of vem in addition experiments on synthetic and real world images show that the algorithm performance is very close and sometimes better than that of other existing simulation based and variational em like algorithms

variable separation method for a nonlinear time fractional partial differential equation with forcing term abstract in this paper a variable coefficient nonlinear time fractional partial differential equation pde with initial and boundary conditions is solved by using the variable separation method as a result some new explicit and exact solutions of the time fractional pde are obtained including airy function solution hyperbolic function solution trigonometric function solution and rational solution it is shown that the variable separation method can provide a useful mathematical tool for solving some other nonlinear time fractional pdes in science and engineering

latent log linear models for handwritten digit classification we present latent log linear models an extension of log linear models incorporating latent variables and we propose two applications thereof log linear mixture models and image deformation aware log linear models the resulting models are fully discriminative can be trained efficiently and the model complexity can be controlled log linear mixture models offer additional flexibility within the log linear modeling framework unlike previous approaches the image deformation aware model directly considers image deformations and allows for a discriminative training of the deformation parameters both are trained using alternating optimization for certain variants convergence to a stationary point is guaranteed and in practice even variants without this guarantee converge and find models that perform well we tune the methods on the usps data set and evaluate on the mnist data set demonstrating the generalization capabilities of our proposed models our models although using significantly fewer parameters are able to obtain competitive results with models proposed in the literature

convergence theorems for generalized alternating minimization procedures the em algorithm is widely used to develop iterative parameter estimation procedures for statistical models in cases where these procedures strictly follow the em formulation the convergence properties of the estimation procedures are well understood in some instances there are practical reasons to develop procedures that do not strictly fall within the em framework we study em variants in which the e step is not performed exactly either to obtain improved rates of convergence or due to approximations needed to compute statistics under a model family over which e steps cannot be realized since these variants are not em procedures the standard g em convergence results do not apply to them we present an information geometric framework for describing such algorithms and analyzing their convergence properties we apply this framework to analyze the convergence properties of incremental em and variational em for incremental em we discuss conditions under these algorithms converge in likelihood for variational em we show how the e step approximation prevents convergence to local maxima in likelihood