on some classes of almost automorphic functions and applications to fractional differential equations abstract we study in this paper the properties of c n almost automorphic and asymptotically c n almost automorphic functions a new concept with values in a banach space we then give a new result related to the existence and uniqueness of an asymptotically almost automorphic solution of a semilinear fractional differential equation of the form d x t a x t f t x t b x t with t r 0 1 where a generates a family of resolvent family s t and f satisfies some lipschitz conditions

discriminative least squares regression for multiclass classification and feature selection this paper presents a framework of discriminative least squares regression lsr for multiclass classification and feature selection the core idea is to enlarge the distance between different classes under the conceptual framework of lsr first a technique called e dragging is introduced to force the regression targets of different classes moving along opposite directions such that the distances between classes can be enlarged then the e draggings are integrated into the lsr model for multiclass classification our learning framework referred to as discriminative lsr has a compact model form where there is no need to train two class machines that are independent of each other with its compact form this model can be naturally extended for feature selection this goal is achieved in terms of l2 1 norm of matrix generating a sparse learning model for feature selection the model for multiclass classification and its extension for feature selection are finally solved elegantly and efficiently experimental evaluation over a range of benchmark datasets indicates the validity of our method

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

bounded solutions to a class of semilinear integro differential equations in banach spaces abstract let a be the generator of an immediately norm continuous c 0 semigroup defined on a banach space x we study the existence and uniqueness of bounded solutions for the semilinear integro differential equation with infinite delay u t a u t t e t s a u s d s f t u t t r r for each f r x x satisfying diverse lipschitz type conditions sufficient conditions are established for the existence and uniqueness of an almost periodic almost automorphic and asymptotically almost periodic solution among other types of distinguished solutions these results have significance in viscoelasticity theory finally an example is presented to illustrate the feasibility and effectiveness of the results

testing for factor loading structural change under common breaks abstract this paper proposes a new test for factor loading structural change in dynamic factor models we first show that the leading test proposed by breitung and eickmeier 2011 exhibits nonmonotonic power essentially because the breaks are considered as spurious factors with stable factor loadings the proposed test eliminates the effects of the spurious factors by maximizing the test statistic over possible numbers of the original factors monte carlo simulations and an empirical example using u s treasury yield curve data clearly illustrate the usefulness of the proposed test

controllability of fractional integro differential evolution equations with nonlocal conditions this paper concerns the controllability for a class of fractional integro differential evolution equations with nonlocal initial conditions by using the fractional calculus measure of noncompactness and the monch fixed point theorem we obtain a controllability result for the nonlocal cauchy problem of the fractional integro differential evolution equations involving noncompact semigroups and the nonlocal functions without lipschitz continuity an example is given to illustrate the effectiveness of the abstract results

forecasting using principal components from a large number of predictors this article considers forecasting a single time series when there are many predictors n and time series observations t when the data follow an approximate factor model the predictors can be summarized by a small number of indexes which we estimate using principal components feasible forecasts are shown to be asymptotically efficient in the sense that the difference between the feasible forecasts and the infeasible forecasts constructed using the actual values of the factors converges in probability to 0 as both n and t grow large the estimated factors are shown to be consistent even in the presence of time variation in the factor model

nonparametric trending regression with cross sectional dependence panel data whose series length t is large but whose cross section size n need not be are assumed to have common time trend of unknown form the model includes additive unknown individual specific components and allows for spatial or other cross sectional dependence and or heteroscedasticity a simple smoothed nonparametric trend estimate is shown to be dominated by an estimate which exploits availability of cross sectional data asymptotically optimal bandwidth choices are justified for both estimates feasible optimal bandwidths and feasible optimal trend estimates are asymptotically justified finite sample performance of the latter being examined in a monte carlo study potential extensions are discussed

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

study of the convergence behavior of the complex kernel least mean square algorithm the complex kernel least mean square cklms algorithm is recently derived and allows for online kernel adaptive learning for complex data kernel adaptive methods can be used in finding solutions for neural network and machine learning applications the derivation of cklms involved the development of a modified wirtinger calculus for hilbert spaces to obtain the cost function gradient we analyze the convergence of the cklms with different kernel forms for complex data the expressions obtained enable us to generate theory predicted mean square error curves considering the circularity of the complex input signals and their effect on nonlinear learning simulations are used for verifying the analysis results

an information theoretic approach of designing sparse kernel adaptive filters this paper discusses an information theoretic approach of designing sparse kernel adaptive filters to determine useful data to be learned and remove redundant ones a subjective information measure called surprise is introduced surprise captures the amount of information a datum contains which is transferable to a learning system based on this concept we propose a systematic sparsification scheme which can drastically reduce the time and space complexity without harming the performance of kernel adaptive filters nonlinear regression short term chaotic time series prediction and long term time series forecasting examples are presented

asymptotically almost automorphic solutions for some integrodifferential equations with nonlocal initial conditions abstract of concern is a class of abstract semilinear integrodifferential equations with nonlocal initial conditions under some suitable hypotheses we establish some new theorems about the existence of asymptotically almost automorphic solutions to the integrodifferential equations moreover an example is given to illustrate our results

testing for structural stability of factor augmented forecasting models abstract mild factor loading instability particularly if sufficiently independent across the different constituent variables does not affect the estimation of the number of factors nor subsequent estimation of the factors themselves see e g stock and watson 2009 this result does not hold in the presence of large common breaks in the factor loadings however in this case information criteria overestimate the number of breaks additionally estimated factors are no longer consistent estimators of true factors hence various recent research papers in the diffusion index literature focus on testing the constancy of factor loadings however forecast failure of factor augmented models can be due to either factor loading instability regression coefficient instability or both to address this issue we develop a test for the joint hypothesis of structural stability of both factor loadings and factor augmented forecasting model regression coefficients our proposed test statistic has a chi squared limiting distribution and we are able to establish the first order validity of block bootstrap critical values empirical evidence is also presented for 11 us macroeconomic indicators

discriminative elastic net regularized linear regression in this paper we aim at learning compact and discriminative linear regression models linear regression has been widely used in different problems however most of the existing linear regression methods exploit the conventional zero one matrix as the regression targets which greatly narrows the flexibility of the regression model another major limitation of these methods is that the learned projection matrix fails to precisely project the image features to the target space due to their weak discriminative capability to this end we present an elastic net regularized linear regression enlr framework and develop two robust linear regression models which possess the following special characteristics first our methods exploit two particular strategies to enlarge the margins of different classes by relaxing the strict binary targets into a more feasible variable matrix second a robust elastic net regularization of singular values is introduced to enhance the compactness and effectiveness of the learned projection matrix third the resulting optimization problem of enlr has a closed form solution in each iteration which can be solved efficiently finally rather than directly exploiting the projection matrix for recognition our methods employ the transformed features as the new discriminate representations to make final image classification compared with the traditional linear regression model and some of its variants our method is much more accurate in image classification extensive experiments conducted on publicly available data sets well demonstrate that the proposed framework can outperform the state of the art methods the matlab codes of our methods can be available at http www yongxu org lunwen html

specification test for panel data models with interactive fixed effects abstract in this paper we propose a consistent nonparametric test for linearity in a large dimensional panel data model with interactive fixed effects both lagged dependent variables and conditional heteroskedasticity of unknown form are allowed in the model we estimate the model under the null hypothesis of linearity to obtain the restricted residuals which are then used to construct the test statistic we show that after being appropriately centered and standardized the test statistic is asymptotically normally distributed under both the null hypothesis and a sequence of pitman local alternatives by using the concept of conditional strong mixing that was recently introduced by prakasa rao 2009 to improve the finite sample performance we propose a bootstrap procedure to obtain the bootstrap p value a small set of monte carlo simulations illustrates that our test performs well in finite samples an application to an economic growth panel dataset indicates significant nonlinear relationships between economic growth initial income level and capital accumulation

bounded mild solutions of perturbed volterra equations with infinite delay abstract we study existence and regularity of bounded mild solutions on the real line to perturbed integral equations with infinite delay in the space of almost periodic functions in the bohr sense the space of compact almost automorphic functions the space of almost automorphic functions and the space of asymptotically almost automorphic functions

linear regression for panel with unknown number of factors as interactive fixed effects in this paper we study the least squares ls estimator in a linear panel regression model with unknown number of factors appearing as interactive fixed effects assuming that the number of factors used in estimation is larger than the true number of factors in the data we establish the limiting distribution of the ls estimator for the regression coefficients as the number of time periods and the number of cross sectional units jointly go to infinity the main result of the paper is that under certain assumptions the limiting distribution of the ls estimator is independent of the number of factors used in the estimation as long as this number is not underestimated the important practical implication of this result is that for inference on the regression coefficients one does not necessarily need to estimate the number of interactive fixed effects consistently

sketch kernel ridge regression using circulant matrix algorithm and theory kernel ridge regression krr is a powerful method for nonparametric regression the time and space complexity of computing the krr estimate directly are mathcal o n 3 and mathcal o n 2 respectively which are prohibitive for large scale data sets where n is the number of data in this article we propose a novel random sketch technique based on the circulant matrix that achieves savings in storage space and accelerates the solution of the krr approximation the circulant matrix has the following advantages it can save time complexity by using the fast fourier transform fft to compute the product of matrix and vector its space complexity is linear and the circulant matrix whose entries in the first column are independent of each other and obey the gaussian distribution is almost as effective as the i i d gaussian random matrix for approximating krr combining the characteristics of the circulant matrix and our careful design theoretical analysis and experimental results demonstrate that our proposed sketch method making the estimate kernel methods scalable and practical for large scale data problems outperforms the state of the art krr estimates in time complexity while retaining similar accuracies meanwhile our sketch method provides the theoretical bound that keeps the optimal convergence rate for approximating krr

shrinkage estimation of dynamic panel data models with interactive fixed effects abstract we consider the problem of determining the number of factors and selecting the proper regressors in linear dynamic panel data models with interactive fixed effects based on the preliminary estimates of the slope parameters and factors a la bai 2009 and moon and weidner 2015 we propose a method for simultaneous selection of regressors and factors and estimation through the method of adaptive group lasso least absolute shrinkage and selection operator we show that with probability approaching one our method can correctly select all relevant regressors and factors and shrink the coefficients of irrelevant regressors and redundant factors to zero further we demonstrate that our shrinkage estimators of the nonzero slope parameters exhibit some oracle property we conduct monte carlo simulations to demonstrate the superb finite sample performance of the proposed method we apply our method to study the determinants of economic growth and find that in addition to three common unobserved factors selected by our method government consumption share has negative effects whereas investment share and lagged economic growth have positive effects on economic growth

tests for parameter instability and structural change with unknown change point this paper considers tests for parameter instability and structural change with unknown change point the results apply to a wide class of parametric models that are suitable for estimation by generalized method of moments procedures the asymptotic distributions of the test statistics considered here are nonstandard because the change point parameter only appears under the alternative hypothesis and not under the null the tests considered here are shown to have nontrivial asymptotic local power against all alternatives for which the parameters are nonconstant the tests are found to perform quite well in a monte carlo experiment reported elsewhere copyright 1993 by the econometric society

eigenvalue ratio test for the number of factors this paper proposes two new estimators for determining the number of factors r in static approximate factor models we exploit the well known fact that the r largest eigenvalues of the variance matrix of n response variables grow unboundedly as n increases while the other eigenvalues remain bounded the new estimators are obtained simply by maximizing the ratio of two adjacent eigenvalues our simulation results provide promising evidence for the two estimators

quantized kernel least mean square algorithm in this paper we propose a quantization approach as an alternative of sparsification to curb the growth of the radial basis function structure in kernel adaptive filtering the basic idea behind this method is to quantize and hence compress the input or feature space different from sparsification the new approach uses the redundant data to update the coefficient of the closest center in particular a quantized kernel least mean square qklms algorithm is developed which is based on a simple online vector quantization method the analytical study of the mean square convergence has been carried out the energy conservation relation for qklms is established and on this basis we arrive at a sufficient condition for mean square convergence and a lower and upper bound on the theoretical value of the steady state excess mean square error static function estimation and short term chaotic time series prediction examples are presented to demonstrate the excellent performance

determining the number of factors in approximate factor models in this paper we develop some econometric theory for factor models of large dimensions the focus is the determination of the number of factors r which is an unresolved issue in the rapidly growing literature on multifactor models we first establish the convergence rate for the factor estimates that will allow for consistent estimation of r we then propose some panel criteria and show that the number of factors can be consistently estimated using the criteria the theory is developed under the framework of large cross sections n and large time dimensions t no restriction is imposed on the relation between n and t simulations show that the proposed criteria have good finite sample properties in many configurations of the panel data encountered in practice

on time varying factor models estimation and testing abstract conventional factor models assume that factor loadings are fixed over a long horizon of time which appears overly restrictive and unrealistic in applications in this paper we introduce a time varying factor model where factor loadings are allowed to change smoothly over time we propose a local version of the principal component method to estimate the latent factors and time varying factor loadings simultaneously we establish the limiting distributions and uniform convergence of the estimated factors and factor loadings in the standard large n and large t framework we also propose a bic type information criterion to determine the number of factors which can be used in models with either time varying or time invariant factor models based on the comparison between the estimates of the common components under the null hypothesis of no structural changes and those under the alternative we propose a consistent test for structural changes in factor loadings we establish the null distribution the asymptotic local power property and the consistency of our test simulations are conducted to evaluate both our nonparametric estimates and test statistic we also apply our test to investigate stock and watson s 2009 u s macroeconomic data set and find strong evidence of structural changes in the factor loadings

time series modelling with semiparametric factor dynamics high dimensional regression problems which reveal dynamic behavior are typically analyzed by time propagation of a few number of factors the inference on the whole system is then based on the low dimensional time series analysis such high dimensional problems occur frequently in many different fields of science in this article we address the problem of inference when the factors and factor loadings are estimated by semiparametric methods this more flexible modeling approach poses an important question is it justified from an inferential point of view to base statistical inference on the estimated times series factors we show that the difference of the inference based on the estimated time series and true unobserved time series is asymptotically negligible our results justify fitting vector autoregressive processes to the estimated factors which allows one to study the dynamics of the whole high dimensional system with a low dimensional representation we illustrate the theory with a simulati

decorrelation of neutral vector variables theory and applications in this paper we propose novel strategies for neutral vector variable decorrelation two fundamental invertible transformations namely serial nonlinear transformation and parallel nonlinear transformation are proposed to carry out the decorrelation for a neutral vector variable which is not multivariate gaussian distributed the conventional principal component analysis cannot yield mutually independent scalar variables with the two proposed transformations a highly negatively correlated neutral vector can be transformed to a set of mutually independent scalar variables with the same degrees of freedom we also evaluate the decorrelation performances for the vectors generated from a single dirichlet distribution and a mixture of dirichlet distributions the mutual independence is verified with the distance correlation measurement the advantages of the proposed decorrelation strategies are intensively studied and demonstrated with synthesized data and practical application evaluations

consistent factor estimation in dynamic factor models with structural instability abstract this paper considers the estimation of approximate dynamic factor models when there is temporal instability in the factor loadings we characterize the type and magnitude of instabilities under which the principal components estimator of the factors is consistent and find that these instabilities can be larger than earlier theoretical calculations suggest we also discuss implications of our results for the robustness of regressions based on the estimated factors and of estimates of the number of factors in the presence of parameter instability simulations calibrated to an empirical application indicate that instability in the factor loadings has a limited impact on estimation of the factor space and diffusion index forecasting whereas estimation of the number of factors is more substantially affected

semiparametric trending panel data models with cross sectional dependence abstract a semiparametric fixed effects model is introduced to describe the nonlinear trending phenomenon in panel data analysis and it allows for the cross sectional dependence in both the regressors and the residuals a pooled semiparametric profile likelihood dummy variable approach based on the first stage local linear fitting is developed to estimate both the parameter vector and the nonlinear time trend function as both the time series length t and the cross sectional size n tend to infinity the resulting estimator of the parameter vector is asymptotically normal with a root n t convergence rate meanwhile the asymptotic distribution for the nonparametric estimator of the trend function is also established with a root n t h convergence rate two simulated examples are provided to illustrate the finite sample performance of the proposed method in addition the proposed model and estimation method are applied to a cpi data set as well as an input output data set

estimating number of factors by adjusted eigenvalues thresholding determining the number of common factors is an important and practical topic in high dimensional factor models the existing literature is mainly based on the eigenvalues of the covariance matrix

testing against constant factor loading matrix with large panel high frequency data abstract in this paper we introduce a nonparametric test against the constancy of the factor loading matrix of a high dimensional continuous time factor model using high frequency data the central limit theorems on the test statistics with and without perturbation are established under the null hypothesis that the factor loading matrix is constant as time evolves the tests perform well in size and power interestingly the test statistic without perturbation converges at a rate that depends not only on the sample size but also on the dimension through the cross sectional dependence of the residual process which is a distinctive feature that contrasts with the low dimensional setting extensive numerical studies including monte carlo simulations and real data analysis validate the performance of our test

mixed precision kernel recursive least squares kernel recursive least squares krls is a widely used online machine learning algorithm for time series predictions in this article we present the mixed precision krls producing equivalent prediction accuracy to double precision krls with a higher training throughput and a lower memory footprint the mixed precision krls applies single precision arithmetic to the computation components being not only numerically resilient but also computationally intensive our mixed precision krls demonstrates the 1 32 1 15 1 29 1 09 and 1 08x training throughput improvements using 24 95 24 74 24 89 24 48 and 24 20 less memory footprint without losing any prediction accuracy compared to double precision krls for a 3 d nonlinear regression a lorenz chaotic time series a mackey glass chaotic time series a sunspot number time series and a sea surface temperature time series respectively

kernel based least squares temporal difference with gradient correction a least squares temporal difference with gradient correction ls tdc algorithm and its kernel based version kernel based ls tdc kls tdc are proposed as policy evaluation algorithms for reinforcement learning rl ls tdc is derived from the tdc algorithm attributed to tdc derived by minimizing the mean square projected bellman error ls tdc has better convergence performance the least squares technique is used to omit the size step tuning of the original tdc and enhance robustness for kls tdc since the kernel method is used feature vectors can be selected automatically the approximate linear dependence analysis is performed to realize kernel sparsification in addition a policy iteration strategy motivated by kls tdc is constructed to solve control learning problems the convergence and parameter sensitivities of both ls tdc and kls tdc are tested through on policy learning off policy learning and control learning problems experimental results as compared with a series of corresponding rl algorithms demonstrate that both ls tdc and kls tdc have better approximation and convergence performance higher efficiency for sample usage smaller burden of parameter tuning and less sensitivity to parameters

on factor models with random missing em estimation inference and cross validation abstract we consider the estimation and inference in approximate factor models with random missing values we show that with the low rank structure of the common component we can estimate the factors and factor loadings consistently with the missing values replaced by zeros we establish the asymptotic distributions of the resulting estimators and those based on the em algorithm we also propose a cross validation based method to determine the number of factors in factor models with or without missing values and justify its consistency simulations demonstrate that our cross validation method is robust to fat tails in the error distribution and significantly outperforms some existing popular methods in terms of correct percentage in determining the number of factors an application to the factor augmented regression models shows that a proper treatment of the missing values can improve the out of sample forecast of some macroeconomic variables

controllability of fractional evolution nonlocal impulsive quasilinear delay integro differential systems in this work the controllability result of a class of fractional evolution nonlocal impulsive quasilinear delay integro differential systems in a banach space has been established by using the theory of fractional calculus fixed point technique and also we introduced a new concept called a u resolvent family as an application that illustrates the abstract results an example is given

almost automorphic mild solutions to fractional differential equations abstract we introduce the concept of resolvent families to prove the existence of almost automorphic mild solutions to the differential equation d t u t a u t t n f t 1 2 n z considered in a banach space x where f r x is almost automorphic we also prove the existence and uniqueness of an almost automorphic mild solution of the semilinear equation d t u t a u t f t u t 1 2 assuming f t x is almost automorphic in t for each x x satisfies a global lipschitz condition and takes values on x finally we prove also the existence and uniqueness of an almost automorphic mild solution of the semilinear equation d t u t a u t f t u t u t 1 2 under analogous conditions as in the previous case

the kernel adaptive autoregressive moving average algorithm in this paper we present a novel kernel adaptive recurrent filtering algorithm based on the autoregressive moving average arma model which is trained with recurrent stochastic gradient descent in the reproducing kernel hilbert spaces this kernelized recurrent system the kernel adaptive arma kaarma algorithm brings together the theories of adaptive signal processing and recurrent neural networks rnns extending the current theory of kernel adaptive filtering kaf using the representer theorem to include feedback compared with classical feedforward kaf methods the kaarma algorithm provides general nonlinear solutions for complex dynamical systems in a state space representation with a deferred teacher signal by propagating forward the hidden states we demonstrate its capabilities to provide exact solutions with compact structures by solving a set of benchmark nondeterministic polynomial complete problems involving grammatical inference simulation results show that the kaarma algorithm outperforms equivalent input space recurrent architectures using first and second order rnns demonstrating its potential as an effective learning solution for the identification and synthesis of deterministic finite automata

quantized kernel recursive least squares algorithm in a recent paper we developed a novel quantized kernel least mean square algorithm in which the input space is quantized partitioned into smaller regions and the network size is upper bounded by the quantization codebook size number of the regions in this paper we propose the quantized kernel least squares regression and derive the optimal solution by incorporating a simple online vector quantization method we derive a recursive algorithm to update the solution namely the quantized kernel recursive least squares algorithm the good performance of the new algorithm is demonstrated by monte carlo simulations

the kernel least mean square algorithm the combination of the famed kernel trick and the least mean square lms algorithm provides an interesting sample by sample update for an adaptive filter in reproducing kernel hilbert spaces rkhs which is named in this paper the klms unlike the accepted view in kernel methods this paper shows that in the finite training data case the klms algorithm is well posed in rkhs without the addition of an extra regularization term to penalize solution norms as was suggested by kivinen kivinen smola and williamson ldquoonline learning with kernels rdquo ieee transactions on signal processing vol 52 no 8 pp 2165 2176 aug 2004 and smale smale and yao ldquoonline learning algorithms rdquo foundations in computational mathematics vol 6 no 2 pp 145 176 2006 this result is the main contribution of the paper and enhances the present understanding of the lms algorithm with a machine learning perspective the effect of the klms step size is also studied from the viewpoint of regularization two experiments are presented to support our conclusion that with finite data the klms algorithm can be readily used in high dimensional spaces and particularly in rkhs to derive nonlinear stable algorithms with comparable performance to batch regularized solutions

a least squares framework for component analysis over the last century component analysis ca methods such as principal component analysis pca linear discriminant analysis lda canonical correlation analysis cca locality preserving projections lpp and spectral clustering sc have been extensively used as a feature extraction step for modeling classification visualization and clustering ca techniques are appealing because many can be formulated as eigen problems offering great potential for learning linear and nonlinear representations of data in closed form however the eigen formulation often conceals important analytic and computational drawbacks of ca techniques such as solving generalized eigen problems with rank deficient matrices e g small sample size problem lacking intuitive interpretation of normalization factors and understanding commonalities and differences between ca methods this paper proposes a unified least squares framework to formulate many ca methods we show how pca lda cca lpp sc and its kernel and regularized extensions correspond to a particular instance of least squares weighted kernel reduced rank regression ls wkrrr the ls wkrrr formulation of ca methods has several benefits 1 provides a clean connection between many ca techniques and an intuitive framework to understand normalization factors 2 yields efficient numerical schemes to solve ca techniques 3 overcomes the small sample size problem 4 provides a framework to easily extend ca methods we derive weighted generalizations of pca lda sc and cca and several new ca techniques

on the algorithmic implementation of multiclass kernel based vector machines in this paper we describe the algorithmic implementation of multiclass kernel based vector machines our starting point is a generalized notion of the margin to multiclass problems using this notion we cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function unlike most of previous approaches which typically decompose a multiclass problem into multiple independent binary classification tasks our notion of margin yields a direct method for training multiclass predictors by using the dual of the optimization problem we are able to incorporate kernels with a compact set of constraints and decompose the dual problem into multiple optimization problems of reduced size we describe an efficient fixed point algorithm for solving the reduced optimization problems and prove its convergence we then discuss technical details that yield significant running time improvements for large datasets finally we describe various experiments with our approach comparing it to previously studied kernel based methods our experiments indicate that for multiclass problems we attain state of the art accuracy

supervised discrete hashing with relaxation data dependent hashing has recently attracted attention due to being able to support efficient retrieval and storage of high dimensional data such as documents images and videos in this paper we propose a novel learning based hashing method called supervised discrete hashing with relaxation sdhr based on supervised discrete hashing sdh sdh uses ordinary least squares regression and traditional zero one matrix encoding of class label information as the regression target code words thus fixing the regression target in sdhr the regression target is instead optimized the optimized regression target matrix satisfies a large margin constraint for correct classification of each example compared with sdh which uses the traditional zero one matrix sdhr utilizes the learned regression target matrix and therefore more accurately measures the classification error of the regression model and is more flexible as expected sdhr generally outperforms sdh experimental results on two large scale image data sets cifar 10 and mnist and a large scale and challenging face data set frgc demonstrate the effectiveness and efficiency of sdhr

testing for smooth structural changes in time series models via nonparametric regression checking parameter stability of econometric models is a long standing problem almost all existing structural change tests in econometrics are designed to detect abrupt breaks little attention has been paid to smooth structural changes which may be more realistic in economics we propose a consistent test for smooth structural changes as well as abrupt structural breaks with known or unknown change points the idea is to estimate smooth time varying parameters by local smoothing and compare the fitted values of the restricted constant parameter model and the unrestricted time varying parameter model the test is asymptotically pivotal and does not require prior information about the alternative a simulation study highlights the merits of the proposed test relative to a variety of popular tests for structural changes in an application we strongly reject the stability of univariate and multivariate stock return prediction models in the postwar and post oil shocks periods

testing slope homogeneity in large panels abstract this paper proposes a standardized version of swamy s test of slope homogeneity for panel data models where the cross section dimension n could be large relative to the time series dimension t the proposed test denoted by exploits the cross section dispersion of individual slopes weighted by their relative precision in the case of models with strictly exogenous regressors but with non normally distributed errors the test is shown to have a standard normal distribution as n t j such that n t 2 0 when the errors are normally distributed a mean variance bias adjusted version of the test is shown to be normally distributed irrespective of the relative expansion rates of n and t the test is also applied to stationary dynamic models and shown to be valid asymptotically so long as n t as n t j where 0 using monte carlo experiments it is shown that the test has the correct size and satisfactory power in panels with strictly exogenous regressors for various combinations of n and t similar results are also obtained for dynamic panels but only if the autoregressive coefficient is not too close to unity and so long as t n

regularity of mild solutions for a class of fractional order differential equations in this article we show sufficient conditions ensuring the existence and uniqueness of a mild solution to the equation d au t au t d a 1f t u t 1 a 2 email protected r in the same space where f belongs here a is a sectorial operator defined in a banach space x d a is the fractional derivative in the riemann liouville sense and f x email protected x for each email protected x where w x is a vector valued subspace of the space of continuous and bounded functions the subspaces w x that we will consider in this article are the space of periodic almost periodic almost automorphic and compact almost automorphic vector valued functions among others in particular we extend and unify recent results established for the equation in the papers agarwal et al 2010 cuevas et al 2010 and cuevas and lizama 2008

retargeted least squares regression algorithm this brief presents a framework of retargeted least squares regression relsr for multicategory classification the core idea is to directly learn the regression targets from data other than using the traditional zero one matrix as regression targets the learned target matrix can guarantee a large margin constraint for the requirement of correct classification for each data point compared with the traditional least squares regression lsr and a recently proposed discriminative lsr models relsr is much more accurate in measuring the classification error of the regression model furthermore relsr is a single and compact model hence there is no need to train two class binary machines that are independent of each other the convex optimization problem of relsr is solved elegantly and efficiently with an alternating procedure including regression and retargeting as substeps the experimental evaluation over a range of databases identifies the validity of our method

testing for structural breaks in dynamic factor models in this paper we investigate the consequences of structural breaks in the factor loadings for the specification and estimation of factor models based on principal components and suggest procedures for testing for structural breaks it is shown that structural breaks severely inflate the number of factors identified by the usual information criteria the hypothesis of a structural break is tested by using lr lm and wald statistics the lm test which performs best in our monte carlo simulations is generalized to test for structural breaks in factor models where the break date is unknown and the common factors and idiosyncratic components are serially correlated the proposed test procedures are applied to datasets from the us and the euro area

groupwise retargeted least squares regression in this brief we propose a new groupwise retargeted least squares regression grelsr model for multicategory classification the main motivation behind grelsr is to utilize an additional regularization to restrict the translation values of relsr so that they should be similar within same class by analyzing the regression targets of relsr we propose a new formulation of relsr where the translation values are expressed explicitly on the basis of the new formulation discriminative least squares regression can be regarded as a special case of relsr with zero translation values moreover a groupwise constraint is added to relsr to form the new grelsr model extensive experiments on various machine leaning data sets illustrate that our method outperforms the current state of the art approaches

qml estimation of dynamic panel data models with spatial errors abstract we propose quasi maximum likelihood qml estimation of dynamic panel models with spatial errors when the cross sectional dimension n is large and the time dimension t is fixed we consider both the random effects and fixed effects models and prove consistency and derive the limiting distributions of the qml estimators under different assumptions on the initial observations we propose a residual based bootstrap method for estimating the standard errors of the qml estimators monte carlo simulation shows that both the qml estimators and the bootstrap standard errors perform well in finite samples under a correct assumption on initial observations but may perform poorly when this assumption is not met

estimation of large dimensional factor models with an unknown number of breaks abstract in this paper we study the estimation of a large dimensional factor model when the factor loadings exhibit an unknown number of changes over time we propose a novel three step procedure to detect the breaks if any and then identify their locations in the first step we divide the whole time span into subintervals and fit a conventional factor model on each interval in the second step we apply the adaptive fused group lasso to identify intervals containing a break in the third step we devise a grid search method to estimate the location of the break on each identified interval we show that with probability approaching one our method can identify the correct number of changes and estimate the break locations simulation studies indicate superb finite sample performance of our method we apply our method to investigate stock and watson s 2009 u s monthly macroeconomic dataset and identify five breaks in the factor loadings spanning 1959 2006

fast sparse approximation for least squares support vector machine in this paper we present two fast sparse approximation schemes for least squares support vector machine ls svm named fsals svm and pfsals svm to overcome the limitation of ls svm that it is not applicable to large data sets and to improve test speed fsals svm iteratively builds the decision function by adding one basis function from a kernel based dictionary at one time the process is terminated by using a flexible and stable epsilon insensitive stopping criterion a probabilistic speedup scheme is employed to further improve the speed of fsals svm and the resulting classifier is named pfsals svm our algorithms are of two compelling features low complexity and sparse solution experiments on benchmark data sets show that our algorithms obtain sparse classifiers at a rather low cost without sacrificing the generalization performance

msdlsr margin scalable discriminative least squares regression for multicategory classification in this brief we propose a new margin scalable discriminative least squares regression msdlsr model for multicategory classification the main motivation behind the msdlsr is to explicitly control the margin of dlsr model we first prove that the dlsr is a relaxation of the traditional l 2 support vector machine based on this fact we further provide a theorem on the margin of dlsr with this theorem we add an explicit constraint on dlsr to restrict the number of zeros of dragging values so as to control the margin of dlsr the new model is called msdlsr theoretically we analyze the determination of the margin and support vectors of msdlsr extensive experiments illustrate that our method outperforms the current state of the art approaches on various machine leaning and real world data sets

online learning with kernels kernel based algorithms such as support vector machines have achieved considerable success in various problems in batch setting where all of the training data is available in advance support vector machines combine the so called kernel trick with the large margin idea there has been little use of these methods in an online setting suitable for real time applications in this paper we consider online learning in a reproducing kernel hilbert space by considering classical stochastic gradient descent within a feature space and the use of some straightforward tricks we develop simple and computationally efficient algorithms for a wide range of problems such as classification regression and novelty detection in addition to allowing the exploitation of the kernel trick in an online setting we examine the value of large margins for classification in the online setting with a drifting target we derive worst case loss bounds and moreover we show the convergence of the hypothesis to the minimizer of the regularized risk functional we present some experimental results that support the theory as well as illustrating the power of the new algorithms for online novelty detection

boosting high dimensional predictive regressions with time varying parameters abstract high dimensional predictive regressions are useful in wide range of applications however the theory is mainly developed assuming that the model is stationary with time invariant parameters this is at odds with the prevalent evidence for parameter instability in economic time series but theories for parameter instability are mainly developed for models with a small number of covariates in this paper we present two l 2 boosting algorithms for estimating high dimensional models in which the coefficients are modeled as functions evolving smoothly over time and the predictors are locally stationary the first method uses componentwise local constant estimators as base learner while the second relies on componentwise local linear estimators we establish consistency of both methods and address the practical issues of choosing the bandwidth for the base learners and the number of boosting iterations in an extensive application to macroeconomic forecasting with many potential predictors we find that the benefits to modeling time variation are substantial and they increase with the forecast horizon furthermore the timing of the benefits suggests that the great moderation is associated with substantial instability in the conditional mean of various economic series

a constrained least squares regression model abstract least squares regression lsr is a widely used regression technique for multicategory classification conventional lsr model assumes that during the learning phase the labeled samples can be exactly transformed into a discrete label matrix which is too strict to learn a regression matrix for fitting the labels to overcome this drawback lots of lsr s variants utilize the soft target label which contains the continuous values to replace this discrete label to improve the learning performance since the regression matrix can be learnt from these soft target labels it is reasonable to assume that the samples in the same class have similar soft target labels nevertheless most of existing lsr based models don t adequately consider this similarity assumption in this paper we propose a constrained least squares regression clsr model for multicategory classification the main motivation of clsr is to force the samples in the same class to obtain the similar soft target labels to effectively optimize clsr we propose a novel alternating algorithm which can converge to the globally optimal solution extensive experiments results on face and digit data sets confirm the effectiveness of our proposed model

remarks on fractional derivatives in this paper we further discuss the properties of three kinds of fractional derivatives the grunwald letnikov derivative the riemann liouville derivative and the caputo derivative especially we compare the riemann liouville derivative with the caputo derivative and sequential property of the caputo derivative is also derived which is helpful in translating the higher fractional order differential systems into lower ones besides we also compare the riemann liouville derivative and the caputo derivative with the classical derivative

quantized minimum error entropy criterion comparing with traditional learning criteria such as mean square error the minimum error entropy mee criterion is superior in nonlinear and non gaussian signal processing and machine learning the argument of the logarithm in renyi s entropy estimator called information potential ip is a popular mee cost in information theoretic learning the computational complexity of ip is however quadratic in terms of sample number due to double summation this creates the computational bottlenecks especially for large scale data sets to address this problem in this paper we propose an efficient quantization approach to reduce the computational burden of ip which decreases the complexity from o n 2 to o mn with m ll n the new learning criterion is called the quantized mee qmee some basic properties of qmee are presented illustrative examples with linear in parameter models are provided to verify the excellent performance of qmee

stochastic conjugate gradient algorithm with variance reduction conjugate gradient cg methods are a class of important methods for solving linear equations and nonlinear optimization problems in this paper we propose a new stochastic cg algorithm with variance reduction 1 and we prove its linear convergence with the fletcher and reeves method for strongly convex and smooth functions we experimentally demonstrate that the cg with variance reduction algorithm converges faster than its counterparts for four learning models which may be convex nonconvex or nonsmooth in addition its area under the curve performance on six large scale data sets is comparable to that of the liblinear solver for the l2 regularized l2 loss but with a significant improvement in computational efficiency 1 cgvr algorithm is available on github https github com xbjin cgvr

sparse on line gaussian processes we develop an approach for sparse representations of gaussian process gp models which are bayesian types of kernel machines in order to overcome their limitations for large data sets the method is based on a combination of a bayesian on line algorithm together with a sequential construction of a relevant subsample of the data that fully specifies the prediction of the gp model by using an appealing parameterization and projection techniques in a reproducing kernel hilbert space recursions for the effective parameters and a sparse gaussian approximation of the posterior process are obtained this allows for both a propagation of predictions and bayesian error measures the significance and robustness of our approach are demonstrated on a variety of experiments

extended kernel recursive least squares algorithm this paper presents a kernelized version of the extended recursive least squares ex krls algorithm which implements for the first time a general linear state model in reproducing kernel hilbert spaces rkhs or equivalently a general nonlinear state model in the input space the center piece of this development is a reformulation of the well known extended recursive least squares ex rls algorithm in rkhs which only requires inner product operations between input vectors thus enabling the application of the kernel property commonly known as the kernel trick the first part of the paper presents a set of theorems that shows the generality of the approach the ex krls is preferable to 1 a standard kernel recursive least squares krls in applications that require tracking the state vector of general linear state space models in the kernel space or 2 an ex rls when the application requires a nonlinear observation and state models the second part of the paper compares the ex krls in nonlinear rayleigh multipath channel tracking and in lorenz system modeling problem we show that the proposed algorithm is able to outperform the standard krls and ex rls in both simulations

density dependent quantized least squares support vector machine for large data sets based on the knowledge that input data distribution is important for learning a data density dependent quantization scheme dqs is proposed for sparse input data representation the usefulness of the representation scheme is demonstrated by using it as a data preprocessing unit attached to the well known least squares support vector machine ls svm for application on big data sets essentially the proposed dqs adopts a single shrinkage threshold to obtain a simple quantization scheme which adapts its outputs to input data density with this quantization scheme a large data set is quantized to a small subset where considerable sample size reduction is generally obtained in particular the sample size reduction can save significant computational cost when using the quantized subset for feature approximation via the nystrom method based on the quantized subset the approximated features are incorporated into ls svm to develop a data density dependent quantized ls svm dqls svm where an analytic solution is obtained in the primal solution space the developed dqls svm is evaluated on synthetic and benchmark data with particular emphasis on large data sets extensive experimental results show that the learning machine incorporating dqs attains not only high computational efficiency but also good generalization performance

inferential theory for factor models of large dimensions this paper develops an inferential theory for factor models of large dimensions the principal components estimator is considered because it is easy to compute and is asymptotically equivalent to the maximum likelihood estimator if normality is assumed we derive the rate of convergence and the limiting distributions of the estimated factors factor loadings and common components the theory is developed within the framework of large cross sections n and a large time dimension t to which classical factor analysis does not apply we show that the estimated common components are asymptotically normal with a convergence rate equal to the minimum of the square roots of n and t the estimated factors and their loadings are generally normal although not always so the convergence rate of the estimated factors and factor loadings can be faster than that of the estimated common components these results are obtained under general conditions that allow for correlations and heteroskedasticities in both dimensions stronger results are obtained when the idiosyncratic errors are serially uncorrelated and homoskedastic a necessary and sufficient condition for consistency is derived for large n but fixed t