julia a fresh approach to numerical computing bridging cultures that have often been distant julia combines expertise from the diverse fields of computer science and computational science to create a new approach to numerical computing julia is designed to be easy and fast and questions notions generally held to be laws of nature by practitioners of numerical computing beginlist item high level dynamic programs have to be slow item one must prototype in one language and then rewrite in another language for speed or deployment item there are parts of a system appropriate for the programmer and other parts that are best left untouched as they have been built by the experts endlist we introduce the julia programming language and its design a dance between specialization and abstraction specialization allows for custom treatment multiple dispatch a technique from computer science picks the right algorithm for the right circumstance abstraction which is what good computation is really about recognizes what remains the same after dif

information transfer between dynamical system components we present a rigorous formalism of information transfer for systems with dynamics fully known this follows from an accurate classification of the mechanisms for the entropy change of one component into a self evolution plus a transfer from the other component the formalism applies to both continuous flows and discrete maps the resulting transfer measure possesses a property of asymmetry and is qualitatively consistent with the classical measures it is further validated with the baker transformation and the henon map

information based detection of nonlinear granger causality in multivariate processes via a nonuniform embedding technique we present an approach framed in information theory to assess nonlinear causality between the subsystems of a whole stochastic or deterministic dynamical system the approach follows a sequential procedure for nonuniform embedding of multivariate time series whereby embedding vectors are built progressively on the basis of a minimization criterion applied to the entropy of the present state of the system conditioned to its past states a corrected conditional entropy estimator compensating for the biasing effect of single points in the quantized hyperspace is used to guarantee the existence of a minimum entropy rate at which to terminate the procedure the causal coupling is detected according to the granger notion of predictability improvement and is quantified in terms of information transfer we apply the approach to simulations of deterministic and stochastic systems showing its superiority over standard uniform embedding effects of quantization data length and noise contamination are investigated as practical applications we consider the assessment of cardiovascular regulatory mechanisms from the analysis of heart period arterial pressure and respiration time series and the investigation of the information flow across brain areas from multichannel scalp electroencephalographic recordings

nonlinear dynamics delay times and embedding windows in order to construct an embedding of a nonlinear time series one must choose an appropriate delay time d often d is estimated using the autocorrelation function however this does not treat the nonlinearity appropriately and it may yield an incorrect value for d on the other hand the correct value of d can be found from the mutual information but this process is rather cumbersome computationally here we suggest a simpler method for estimating d using the correlation integral we call this the c c method and we test it on several nonlinear time series obtaining estimates of d in agreement with those obtained using the mutual information furthermore some researchers have suggested that one should not choose a fixed delay time d independent of the embedding dimension m but rather one should choose an appropriate value for the delay time window w d m 1 which is the total time spanned by the components of each embedded point unfortunately w cannot be estimated using the autocorrelation function or the mutual information and no standard procedure for estimating w has emerged however we show that the c c method can also be used to estimate w basically w is the optimal time for independence of the data while d is the first locally optimal time as tests we apply the c c method to the lorenz system a three dimensional irrational torus the rossler system and the rabinovich fabrikant system we also demonstrate the robustness of this method to the presence of noise c 1999 elsevier science b v all rights reserved

finite time entropy a probabilistic approach for measuring nonlinear stretching abstract transport and mixing processes in dynamical systems are often difficult to study analytically and therefore a variety of numerical methods have been developed finite time lyapunov exponents ftles or related stretching indicators are frequently used as a means to estimate transport barriers alternatively eigenvectors singular vectors or oseledets vectors of numerical transfer operators find almost invariant sets finite time coherent sets or time asymptotic coherent sets respectively which are minimally dispersed under the dynamics while these families of approaches geometric ftles and the probabilistic transfer operator often give compatible results a formal link is still missing here we present a small step towards providing a mathematical link we propose a new entropy based methodology for estimating finite time expansive behaviour along trajectories in autonomous and nonautonomous dynamical systems we introduce the finite time entropy fte field as a simple and flexible way to capture nonlinear stretching directly from the entropy growth experienced by a small localised density evolved by the transfer operator the fte construction elucidates in a straightforward way the connection between the evolution of probability densities and the local stretching experienced we develop an extremely simple and numerically efficient method of constructing an estimate of the fte field the fte field is instantaneously calculable from a numerical transfer operator a transition matrix of conditional probabilities that describes a discretised version of the dynamical system once one has such a transition matrix the fte field may be computed for free we also show i how to avoid long time integrations in autonomous and time periodic systems ii how to perform backward time computations by a fast matrix manipulation rather than backward time integration and iii how to easily employ adaptive methods to focus on high value fte regions

minimal locally linearly dependent spaces of operators abstract let s be an n dimensional minimal locally linearly dependent space of operators acting between vector spaces u and v we obtain the sharp lower bound and the sharp upper bound for dim s u and give a complete description of those minimal locally linearly dependent spaces at which the upper bound is attained

discrete observability of the wave equation on bounded domains in euclidean space the study of observability is the study of deducing information about the state of a dynamical system from incomplete measurements discrete observability asks if we can recover the initial data with only a discrete set of measurements discrete observability of the heat equation has been studied for bounded domains in euclidean space and compact homogeneous spaces by gilliam li and martin 4 and wallace and wolf 6 this paper examines discrete observability of the wave equation on bounded domains in euclidean space the solutions to the wave equation involve an additional initial condition and the terms of the series solution no longer decay with time so a new approach is needed

computer simulations at the fixed point using an inverse renormalization group transformation abstract following brandt and ron s suggestion of inverting the renormalization group transformation used in monte carlo renormalization it is shown that efficient computer simulations of the fixed point of the transformation can be carried out on very large systems without critical slowing down we illustrate the new method with calculations of critical exponents for the two and three dimensional ising models based on several different transformations

a robust method for detecting interdependences application to intracranially recorded eeg we present a measure for characterizing statistical relationships between two time sequences in contrast to commonly used measures like cross correlations coherence and mutual information the proposed measure is non symmetric and provides information about the direction of interdependence it is closely related to recent attempts to detect generalized synchronization however we do not assume a strict functional relationship between the two time sequences and try to define the measure so as to be robust against noise and to detect also weak interdependences we apply our measure to intracranially recorded electroencephalograms of patients suffering from severe epilepsies

a study of causality structure and dynamics in industrial electricity consumption based on granger network abstract based on industrial electricity consumption we model industrial networks by granger causality method and mst minimum spanning tree and then further stick onto an industrial coupling mechanism from energy consumption perspective first we construct granger causality networks of five provinces in south china of gd gx gz hn and yn based on their industrial electricity consumption data and we demonstrate from a network topology perspective the distribution of weight of links of all industrial electricity consumption granger causality networks approximately follows power law distribution revealing a phenomenon that few industries may bring a tremendous influence on the rest moreover correlation analysis between weight and degree of a node shows that in most granger causality networks both span and strength of influence of a given industry will significantly increase further we analyze the relationship between the thresholds of granger causality significance and density of corresponding networks results show gd and hn could be classified into a group with relatively greater global differentiation in industries and unbalanced industrial development however gx gz and yn are grouped as second cluster with relatively balanced industrial development furthermore using chu liu edmonds mst algorithm we extract graphs of msts or maximal cliques from industrial electricity consumption granger causality networks and research on energy transmission structure feedback loop and bootstrap reliability by analyzing msts we find that only gd gx and yn can be extracted with mst graphs and capture the probable transmission routes of key nodes besides we illustrate all three mst graphs are involved with feedback loops structures with various characteristics gx has complete feed forward section feed back section and feedback loop section yn has only feed forward section and feedback loop section gd has multiple feedback loops section finally we conduct bootstrap reliability tests on msts results show branches of reliability of gd gx and yn are decentralized while those feedback loops sections are only concentrated on highly reliable branches as neither hn nor gz can be extracted with mst graphs we characterize them by maximal cliques it is demonstrated that both industrial systems of hn and gz are significantly grouped into two independent branches and those internal industries inside independent branches are highly correlated however from perspectives of whole south china the causal relationship is only converged on a few of industries for regions without mst graphs like hn and gz the reliability of links is also greater than that of gd gx and yn

a missing family of classical orthogonal polynomials we study a family of classical orthogonal polynomials which satisfy apart from a three term recurrence relation an eigenvalue problem with a differential operator of dunkl type these polynomials can be obtained from the little q jacobi polynomials in the limit q 1 we also show that these polynomials provide a nontrivial realization of the askey wilson algebra for q 1

discrete observability and dirichlet series abstract the problem of recovering the initial data of the heat equation when the output is measured at points discrete in time and space is considered it is shown that the solution of this problem is linked to the theory of dirichlet series and a solution is found in the case of the one dimensional problem

financial networks based on granger causality a case study connectivity analysis is performed on a long financial record of 21 international stock indices employing a linear and a nonlinear causality measure the conditional granger causality index cgci and the partial mutual information on mixed embedding pmime respectively both measures aim to specify the direction of the interrelationships among the international stock indexes and portray the links of the resulting networks by the presence of direct couplings between variables exploiting all available information however their differences are assessed due to the presence of nonlinearity the weighted networks formed with respect to the causality measures are transformed to binary ones using a significance test the financial networks are formed on sliding windows in order to examine the network characteristics and trace changes in the connectivity structure subsequently two statistical network quantities are calculated the average degree and the average shortest path length the empirical findings reveal interesting time varying properties of the constructed network which are clearly dependent on the nature of the financial cycle

measuring synchronization in coupled model systems a comparison of different approaches the investigation of synchronization phenomena on measured experimental data such as biological time series has recently become an increasing focus of interest different approaches for measuring synchronization have been proposed that rely on certain characteristic features of the dynamical system under investigation for experimental data the underlying dynamics are usually not completely known therefore it is difficult to decide a priori which synchronization measure is most suitable for an analysis in this study we use three different coupled model systems to create a controlled setting for a comparison of six different measures of synchronization all measures are compared to each other with respect to their ability to distinguish between different levels of coupling and their robustness against noise results show that the measure to be applied to a certain task can not be chosen according to a fixed criterion but rather pragmatically as the measure which most reliably yields plausible information in test applications although certain dynamical features of a system under investigation e g power spectra dimension may render certain measures more suitable than others c 2006 elsevier b v all rights reserved

a significance test for the lasso in the sparse linear regression setting we consider testing the significance of the predictor variable that enters the current lasso model in the sequence of models visited along the lasso solution path we propose a simple test statistic based on lasso fitted values called the covariance test statistic and show that when the true model is linear this statistic has an exp 1 asymptotic distribution under the null hypothesis the null being that all truly active variables are contained in the current lasso model our proof of this result for the special case of the first predictor to enter the model i e testing for a single significant predictor variable against the global null requires only weak assumptions on the predictor matrix x on the other hand our proof for a general step in the lasso path places further technical assumptions on x and the generative model but still allows for the important high dimensional case p n and does not necessarily require that the current lasso model achieves perfect recovery of the truly active variables of course for testing the significance of an additional variable between two nested linear models one typically uses the chi squared test comparing the drop in residual sum of squares rss to a formula see text distribution but when this additional variable is not fixed and has been chosen adaptively or greedily this test is no longer appropriate adaptivity makes the drop in rss stochastically much larger than formula see text under the null hypothesis our analysis explicitly accounts for adaptivity as it must since the lasso builds an adaptive sequence of linear models as the tuning parameter decreases in this analysis shrinkage plays a key role though additional variables are chosen adaptively the coefficients of lasso active variables are shrunken due to the formula see text penalty therefore the test statistic which is based on lasso fitted values is in a sense balanced by these two opposing properties adaptivity and shrinkage and its null distribution is tractable and asymptotically exp 1

correlation structure and dynamics of international real estate securities markets a network perspective abstract in this paper we investigate the correlation structure and dynamics of international real estate securities markets by using daily returns of 20 national markets during the period 2006 2012 from a network perspective we construct the minimum spanning tree mst the hierarchical tree ht and the planar maximally filtered graph pmfg obtained from the correlation matrix computed by the daily returns during the investigated period and analyze the corresponding clustering structure hierarchical structure and community structure we also build the time varying mst and pmfg networks by a rolling window to examine the dynamics of correlation structure the empirical results show that 1 the distribution of correlation coefficients is asymmetric fat tailed and non gaussian 2 the distributions of the influence strength of the mst and pmfg networks obey a power law 3 two clusters i e the european and asia pacific clusters are found in the mst network three hierarchical clusters i e two like in the mst and the north american cluster in the ht and three communities in the pmfg network which shows that national markets are linked together according to their geographical distributions 4 the descriptive statistics of correlation coefficients and distances of the msts and pmfgs are time varying especially during periods of crisis they have a large fluctuation 5 a huge number of linkages between national markets survive from one time to the next and the long term stability of the correlation structure in international real estate securities markets descends as time goes on our obtained results are new insights in international real estate securities markets and have wide applications for investment portfolio and risk management

measures of conditional linear dependence and feedback between time series abstract measures of linear dependence and feedback for two multiple time series conditional on a third are defined the measure of conditional linear dependence is the sum of linear feedback from the first to the second conditional on the third linear feedback from the second to the first conditional on the third and instantaneous linear feedback between the first and second series conditional on the third the measures are non negative and may be expressed in terms of measures of unconditional feedback between various combinations of the three series the measures of conditional linear feedback can be additively decomposed by frequency estimates of these measures are straightforward to compute and their distribution can be routinely approximated by bootstrap methods an empirical example involving real output money and interest rates is presented

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

learning driver response relationships from synchronization patterns we test recent claims that causal driver response relationships can be deduced from interdependencies between simultaneously measured time series we apply two recently proposed interdependence measures that should give results similar to cross predictabilities used by previous authors the systems that we study are asymmetrically coupled simple models lorenz roessler and henon models the couplings being such that they lead to generalized synchronization if the data were perfect noise free infinitely long we should be able to detect at least in some cases which of the coupled systems is the driver and which the response this might no longer be true if the time series has finite length instead estimated interdependencies depend strongly on which of the systems has a higher effective dimension at the typical neighborhood sizes used to estimate them and causal relationships are more difficult to detect we also show that slightly different variants of the interdependence measure can have quite different sensitivities

failure in distinguishing colored noise from chaos using the noise titration technique identifying chaos in experimental data noisy data remains a challenging problem for which conclusive arguments are still very difficult to provide in order to avoid problems usually encountered with techniques based on geometrical invariants dimensions lyapunov exponent etc poon and barahona introduced a numerical titration procedure which compares one step ahead predictions of linear and nonlinear models proc natl acad sci u s a 98 7107 2001 we investigate the aformentioned technique in the context of colored noise or other types of nonchaotic behaviors the main conclusion is that in several examples noise titration fails to distinguish such nonchaotic signals from low dimensional deterministic chaos

nonuniform state space reconstruction and coupling detection we investigate the state space reconstruction from multiple time series derived from continuous and discrete systems and propose a method for building embedding vectors progressively using information measure criteria regarding past current and future states the embedding scheme can be adapted for different purposes such as mixed modeling cross prediction and granger causality in particular we apply this method in order to detect and evaluate information transfer in coupled systems as a practical application we investigate in records of scalp epileptic eeg the information flow across brain areas

regularized estimation in sparse high dimensional time series models many scientific and economic problems involve the analysis of high dimensional time series datasets however theoretical studies in high dimensional statistics to date rely primarily on the assumption of independent and identically distributed i i d samples in this work we focus on stable gaussian processes and investigate the theoretical properties of ell 1 regularized estimates in two important statistical problems in the context of high dimensional time series a stochastic regression with serially correlated errors and b transition matrix estimation in vector autoregressive var models we derive nonasymptotic upper bounds on the estimation errors of the regularized estimates and establish that consistent estimation under high dimensional scaling is possible via ell 1 regularization for a large class of stable processes under sparsity constraints a key technical contribution of the work is to introduce a measure of stability for stationary processes using their spectral properties that provides insight into the effect of dependence on the accuracy of the regularized estimates with this proposed stability measure we establish some useful deviation bounds for dependent data which can be used to study several important regularized estimates in a time series setting

ldb division algebras abstract an ldb division algebra is a triple a in which and are regular bilinear laws on the finite dimensional non zero vector space a such that x x y is a scalar multiple of y for all vectors x and y of a this algebraic structure has been recently discovered in the study of the critical case in meshulam and semrl s estimate of the minimal rank in non reflexive operator spaces in this article we obtain a constructive description of all ldb division algebras over an arbitrary field together with a reduction of the isotopy problem to the similarity problem for specific types of quadratic forms over the given field in particular it is shown that the dimension of an ldb division algebra is always a power of 2 and that it belongs to 1 2 4 8 if the characteristic of the underlying field is not 2

geometry of nonlinear least squares with applications to sloppy models and optimization parameter estimation by nonlinear least squares minimization is a common problem that has an elegant geometric interpretation the possible parameter values of a model induce a manifold within the space of data predictions the minimization problem is then to find the point on the manifold closest to the experimental data we show that the model manifolds of a large class of models known as sloppy models have many universal features they are characterized by a geometric series of widths extrinsic curvatures and parameter effect curvatures which we describe as a hyper ribbon a number of common difficulties in optimizing least squares problems are due to this common geometric structure first algorithms tend to run into the boundaries of the model manifold causing parameters to diverge or become unphysical before they have been optimized we introduce the model graph as an extension of the model manifold to remedy this problem we argue that appropriate priors can remove the boundaries and further improve the convergence rates we show that typical fits will have many evaporated parameters unless the data are very accurately known second bare model parameters are usually ill suited to describing model behavior cost contours in parameter space tend to form hierarchies of plateaus and long narrow canyons geometrically we understand this inconvenient parametrization as an extremely skewed coordinate basis and show that it induces a large parameter effect curvature on the manifold by constructing alternative coordinates based on geodesic motion we show that these long narrow canyons are transformed in many cases into a single quadratic isotropic basin we interpret the modified gauss newton and levenberg marquardt fitting algorithms as an euler approximation to geodesic motion in these natural coordinates on the model manifold and the model graph respectively by adding a geodesic acceleration adjustment to these algorithms we alleviate the difficulties from parameter effect curvature improving both efficiency and success rates at finding good fits

reflexivity defect of spaces of linear operators abstract for a finite dimensional linear subspace s l v w and a positive integer k the k reflexivity defect of s is defined by rd k s dim ref k s s where ref k s is the k reflexive closure of s we study this quantity for two dimensional spaces of operators and for single generated algebras and their commutants

functional connectivity in resting state fmri is linear correlation sufficient functional connectivity fc analysis is a prominent approach to analyzing fmri data especially acquired under the resting state condition the commonly used linear correlation fc measure bears an implicit assumption of gaussianity of the dependence structure if only the marginals but not all the bivariate distributions are gaussian linear correlation consistently underestimates the strength of the dependence to assess the suitability of linear correlation and the general potential of nonlinear fc measures we present a framework for testing and estimating the deviation from gaussianity by means of comparing mutual information in the data and its gaussianized counterpart we apply this method to 24 sessions of human resting state fmri for each session matrix of connectivities between 90 anatomical parcel time series is computed using mutual information and compared to results from its multivariate gaussian surrogate that conserves the correlations but cancels any nonlinearity while the group level tests confirmed non gaussianity in the fc the quantitative assessment revealed that the portion of mutual information neglected by linear correlation is relatively minor on average only about 5 of the mutual information already captured by the linear correlation the marginality of the non gaussianity was confirmed in comparisons using clustering of the parcels the disagreement between clustering obtained from mutual information and linear correlation was attributable to random error we conclude that for this type of data practical relevance of nonlinear methods trying to improve over linear correlation might be limited by the fact that the data are indeed almost gaussian

statistical mechanics of the us supreme court we build simple models for the distribution of voting patterns in a group using the supreme court of the united states as an example the maximum entropy model consistent with the observed pairwise correlations among justices votes an ising spin glass agrees quantitatively with the data while all correlations perhaps surprisingly are positive the effective pairwise interactions in the spin glass model have both signs recovering the intuition that ideologically opposite justices negatively influence each another despite the competing interactions a strong tendency toward unanimity emerges from the model organizing the voting patterns in a relatively simple energy landscape besides unanimity other energy minima in this landscape or maxima in probability correspond to prototypical voting states such as the ideological split or a tightly correlated conservative core the model correctly predicts the correlation of justices with the majority and gives us a measure of their influence on the majority decision these results suggest that simple models grounded in statistical physics can capture essential features of collective decision making quantitatively even in a complex political context

scale free brain functional networks functional magnetic resonance imaging is used to extract functional networks connecting correlated human brain sites analysis of the resulting networks in different tasks shows that a the distribution of functional connections and the probability of finding a link versus distance are both scale free b the characteristic path length is small and comparable with those of equivalent random networks and c the clustering coefficient is orders of magnitude larger than those of equivalent random networks all these properties typical of scale free small world networks reflect important functional information about brain states

pairwise network information and nonlinear correlations reconstructing the structural connectivity between interacting units from observed activity is a challenge across many different disciplines the fundamental first step is to establish whether or to what extent the interactions between the units can be considered pairwise and thus can be modeled as an interaction network with simple links corresponding to pairwise interactions in principle this can be determined by comparing the maximum entropy given the bivariate probability distributions to the true joint entropy in many practical cases this is not an option since the bivariate distributions needed may not be reliably estimated or the optimization is too computationally expensive here we present an approach that allows one to use mutual informations as a proxy for the bivariate probability distributions this has the advantage of being less computationally expensive and easier to estimate we achieve this by introducing a novel entropy maximization scheme that is based on conditioning on entropies and mutual informations this renders our approach typically superior to other methods based on linear approximations the advantages of the proposed method are documented using oscillator networks and a resting state human brain network as generic relevant examples

variance of operators and derivations abstract the variance of a bounded linear operator a on a hilbert space h at a unit vector is defined by d a a 2 a 2 we show that two operators a and b have the same variance at all vectors h if and only if there exist scalars c with 1 such that b a 1 or a is normal and b a 1 further if a is normal then the inequality d b d a holds for some constant and all unit vectors if and only if b f a for a lipschitz function f on the spectrum of a variants of these results for c algebras are also proved where vectors are replaced by pure states we also study the related inequalities b x x b a x x a supposed to hold for all x b h or for all x b h n and all n n we consider the connection between such inequalities and the range inclusion d b b h d a b h where d a and d b are the derivations on b h induced by a and b if a is subnormal we study these conditions in particular in the case when b is of the form b f a for a function f

estimating mutual information we present two classes of improved estimators for mutual information m x y from samples of random points distributed according to some joint probability density mu x y in contrast to conventional estimators based on binnings they are based on entropy estimates from k nearest neighbor distances this means that they are data efficient with k 1 we resolve structures down to the smallest possible scales adaptive the resolution is higher where data are more numerous and have minimal bias indeed the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale giving typically systematic errors which scale as functions of k n for n points numerically we find that both families become exact for independent distributions i e the estimator m x y vanishes up to statistical fluctuations if mu x y mu x mu y this holds for all tested marginal distributions and for all dimensions of x and y in addition we give estimators for redundancies between more than two random variables we compare our algorithms in detail with existing algorithms finally we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis ica for improving ica and for estimating the reliability of blind source separation

separating vectors and reflexivity abstract there is a close connection between separating vectors and reflexivity but the existence of a separating vector does not guarantee reflexivity this paper clarifies the relation we first find that every subspace of linear transformations whose dimension is less than the cardinality of the scalar field and having a disjoint pair of separating vectors is algebraically reflexive then we refine the result and conclude that a finite dimensional subspace of operators having a two dimensional subspace of separating vectors is algebraically reflexive if its scalar field is algebraically closed examples are provided to show that these results are sharp an application of the first result shows that finite dimensional spaces of operators of large rank are reflexive

correlation network analysis for multi dimensional data in stocks market this paper shows how the concept of vector correlation can appropriately measure the similarity among multivariate time series in stocks network the motivation of this paper is i to apply the rv coefficient to define the network among stocks where each of them is represented by a multivariate time series ii to analyze that network in terms of topological structure of the stocks of all minimum spanning trees and iii to compare the network topology between univariate correlation based on r and multivariate correlation network based on rv coefficient

reliable detection of directional couplings using rank statistics to detect directional couplings from time series various measures based on distances in reconstructed state spaces were introduced these measures can however be biased by asymmetries in the dynamics structure noise color or noise level which are ubiquitous in experimental signals using theoretical reasoning and results from model systems we identify the various sources of bias and show that most of them can be eliminated by an appropriate normalization we furthermore diminish the remaining biases by introducing a measure based on ranks of distances this rank based measure outperforms existing distance based measures concerning both sensitivity and specificity for directional couplings therefore our findings are relevant for a reliable detection of directional couplings from experimental signals

sparse model from optimal nonuniform embedding of time series an approach to obtaining a parsimonious polynomial model from time series is proposed an optimal minimal nonuniform time series embedding schema is used to obtain a time delay kernel this scheme recursively optimizes an objective functional that eliminates a maximum number of false nearest neighbors between successive state space reconstruction cycles a polynomial basis is then constructed from this time delay kernel a sparse model from this polynomial basis is obtained by solving a regularized least squares problem the constraint satisfaction problem is made computationally tractable by keeping the ratio between the number of constraints to the number of variables small by using fewer samples spanning all regions of the reconstructed state space this helps the structure selection process from an exponentially large combinatorial search space a forward stagewise algorithm is then used for fast discovery of the optimization path results are presented for the mackey glass system

continuous observability for parabolic system under observations of discrete type for the linear partial differential equation of parabolic type the problem of continuous observability under observations of discrete type received from the moving or stationary sensors is considered the techniques exploit the knowledge about observability of an associated system with observations that are continuous in time on the basis of sensitivity points of the latter a method of constructing discrete observations that preserve the property of continuous observability at final time is developed

granger causality in multivariate time series using a time ordered restricted vector autoregressive model granger causality has been used for the investigation of the inter dependence structure of the underlying systems of multivariate time series in particular the direct causal effects are commonly estimated by the conditional granger causality index cgci in the presence of many observed variables and relatively short time series cgci may fail because it is based on vector autoregressive models var involving a large number of coefficients to be estimated in this paper the var is restricted by a scheme that modifies the recently developed method of backward in time selection bts of the lagged variables and the cgci is combined with bts further the proposed approach is compared favorably to other restricted var representations such as the top down strategy the bottom up strategy and the least absolute shrinkage and selection operator lasso in terms of sensitivity and specificity of cgci this is shown by using simulations of linear and nonlinear low and high dimensional systems and different time series lengths for nonlinear systems cgci from the restricted var representations are compared with analogous nonlinear causality indices further cgci in conjunction with bts and other restricted var representations is applied to multichannel scalp electroencephalogram eeg recordings of epileptic patients containing epileptiform discharges cgci on the restricted var and bts in particular could track the changes in brain connectivity before during and after epileptiform discharges which was not possible using the full var representation

the trapezoidal rule for analytic functions of rapid decrease abstract the evaluation of the integral of an analytic function f over the entire real line may be accurately approximated by the trapezoidal method of integration or by mapped trapezoidal rules typically the latter is employed to accelerate the convergence rate of the rule at the expense of a bit more complicated formula in the case that the integrand f decreases rapidly and has its singularities sufficiently removed from the real line the trapezoidal rule provides not only a simpler method of evaluation but also a more accurate method of approximation

locally linearly dependent operators and reflexivity of operator spaces we obtain two results on the existence of large subspaces of operators of small rank in locally linearly dependent spaces of operators as a consequence we obtain an upper bound for the rank of operators belonging to a minimal locally linearly dependent space of operators it has been known that the only obstruction to the reflexivity of a finite dimensional operator space comes from the operators with small ranks our results improve known bounds on the minimal rank that guarantees the reflexivity

state space reconstruction in the presence of noise takens theorem demonstrates that in the absence of noise a multidimensional state space can be reconstructed from a scalar time series this theorem gives little guidance however about practical considerations for reconstructing a good state space we extend takens treatment applying statistical methods to incorporate the effects of observational noise and estimation error we define the distortion matrix which is proportional to the conditional covariance of a state given a series of noisy measurements and the noise amplification which is proportional to root mean square time series prediction errors with an ideal model we derive explicit formulae for these quantities and we prove that in the low noise limit minimizing the distortion is equivalent to minimizing the noise amplification we identify several different scaling regimes for distortion and noise amplification and derive asymptotic scaling laws when the dimension and lyapunov exponents are sufficiently large these scaling laws show that no matter how the state space is reconstructed there is an explosion in the noise amplification from a practical point of view determinism is lost and the time series is effectively a random process in the low noise large data limit we show that the technique of local singular value decomposition is an optimal coordinate transformation in the sense that it achieves the minimum distortion in a state space of the lowest possible dimension however in numerical experiments we find that estimation error complicates this issue for local approximation methods we analyze the effect of reconstruction on estimation error derive a scaling law and suggest an algorithm for reducing estimation errors

observability and determination of surface temperature part 1 the problem of determining the surface temperature of a thin rod and thin rectangular plate from temperature readings at interior points is investigated it is shown that this problem can be solved by an infinite differentiation operator acting on the known temperature readings examples are also obtained which show that the surface temperature cannot be determined by a discrete set of temperature readings this is in contrast to the recent discrete observability results obtained for the determination of initial temperature

evaluation of dynamical models dissipative synchronization and other techniques some recent developments for the validation of nonlinear models built from data are reviewed besides giving an overall view of the field a procedure is proposed and investigated based on the concept of dissipative synchronization between the data and the model which is very useful in validating models that should reproduce dominant dynamical features like bifurcations of the original system in order to assess the discriminating power of the procedure four well known benchmarks have been used namely duffing ueda duffingholmes and van der pol oscillators plus the henon map the procedure developed for discrete time systems is focused on the dynamical properties of the model rather than on statistical issues for all the systems investigated it is shown that the discriminating power of the procedure is similar to that of bifurcation diagrams which in turn is much greater than say that of correlation dimension but at a much lower computational cost

data driven non markovian closure models abstract this paper has two interrelated foci i obtaining stable and efficient data driven closure models by using a multivariate time series of partial observations from a large dimensional system and ii comparing these closure models with the optimal closures predicted by the mori zwanzig mz formalism of statistical physics multilayer stochastic models msms are introduced as both a generalization and a time continuous limit of existing multilevel regression based approaches to closure in a data driven setting these approaches include empirical model reduction emr as well as more recent multi layer modeling it is shown that the multilayer structure of msms can provide a natural markov approximation to the generalized langevin equation gle of the mz formalism a simple correlation based stopping criterion for an emr msm model is derived to assess how well it approximates the gle solution sufficient conditions are derived on the structure of the nonlinear cross interactions between the constitutive layers of a given msm to guarantee the existence of a global random attractor this existence ensures that no blow up can occur for a broad class of msm applications a class that includes non polynomial predictors and nonlinearities that do not necessarily preserve quadratic energy invariants the emr msm methodology is first applied to a conceptual nonlinear stochastic climate model of coupled slow and fast variables in which only slow variables are observed it is shown that the resulting closure model with energy conserving nonlinearities efficiently captures the main statistical features of the slow variables even when there is no formal scale separation and the fast variables are quite energetic second an msm is shown to successfully reproduce the statistics of a partially observed generalized lotka volterra model of population dynamics in its chaotic regime the challenges here include the rarity of strange attractors in the model s parameter space and the existence of multiple attractor basins with fractal boundaries the positivity constraint on the solutions components replaces here the quadratic energy preserving constraint of fluid flow problems and it successfully prevents blow up

stationary sparse causality network learning recently researchers have proposed penalized maximum likelihood to identify network topology underlying a dynamical system modeled by multivariate time series the time series of interest are assumed to be stationary but this restriction is never taken into consideration by existing estimation methods moreover practical problems of interest may have ultra high dimensionality and obvious node collinearity in addition none of the available algorithms provides a probabilistic measure of the uncertainty for the obtained network topology which is informative in reliable network identification the main purpose of this paper is to tackle these challenging issues we propose the s2 learning framework which stands for stationary sparse network learning we propose a novel algorithm referred to as the berhu iterative sparsity pursuit with stationarity bisps where the berhu regularization can improve the lasso in detection and estimation the algorithm is extremely easy to implement efficient in computation and has a theoretical guarantee to converge to a global optimum we also incorporate a screening technique into bisps to tackle ultra high dimensional problems and enhance computational efficiency furthermore a stationary bootstrap technique is applied to provide connection occurring frequency for reliable topology learning experiments show that our method can achieve stationary and sparse causality network learning and is scalable for high dimensional problems

measurement of linear dependence and feedback between multiple time series abstract measures of linear dependence and feedback for multiple time series are defined the measure of linear dependence is the sum of the measure of linear feedback from the first series to the second linear feedback from the second to the first and instantaneous linear feedback the measures are nonnegative and zero only when feedback causality of the relevant type is absent the measures of linear feedback from one series to another can be additively decomposed by frequency a readily usable theory of inference for all of these measures and their decompositions is described the computations involved are modest

small random perturbations of dynamical systems and the definition of attractors the strange attractors plotted by computers and seen in physical experiments do not necessarily have an open basin of attraction in view of this we study a new definition of attractors based on ideas of conley we argue that the attractors observed in the presence of small random perturbations correspond to this new definition

network information and connected correlations entropy and information provide natural measures of correlation among elements in a network we construct here the information theoretic analog of connected correlation functions irreducible n point correlation is measured by a decrease in entropy for the joint distribution of n variables relative to the maximum entropy allowed by all the observed n 1 variable distributions we calculate the connected information terms for several examples and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source

arma identification of graphical models consider a gaussian stationary stochastic vector process with the property that designated pairs of components are conditionally independent given the rest of the components such processes can be represented on a graph where the components are nodes and the lack of a connecting link between two nodes signifies conditional independence this leads to a sparsity pattern in the inverse of the matrix valued spectral density such graphical models find applications in speech bioinformatics image processing econometrics and many other fields where the problem to fit an autoregressive ar model to such a process has been considered in this paper we take this problem one step further namely to fit an autoregressive moving average arma model to the same data we develop a theoretical framework and an optimization procedure which also spreads further light on previous approaches and results this procedure is then applied to the identification problem of estimating the arma parameters as well as the topology of the graph from statistical data

stock market networks the dynamic conditional correlation approach we demonstrate the economic relevance of minimum spanning trees msts constructed from dynamic conditional correlations dcc for a sample of s p 100 constituents an empirical comparison of mst properties shows that using the standard approach of rolling or sliding window correlations yields trees that are more robust have higher densities and exhibit higher industry clustering than msts based on dcc our results suggest that these properties are achieved at the expense of the smoothing of market dynamics which is better preserved by dcc the dcc approach offers a new perspective for the analysis of complex systems such as stock markets

statistical mechanics of letters in words we consider words as a network of interacting letters and approximate the probability distribution of states taken on by this network despite the intuition that the rules of english spelling are highly combinatorial and arbitrary we find that maximum entropy models consistent with pairwise correlations among letters provide a surprisingly good approximation to the full statistics of words capturing ensuremath sim 92 mathrm of the multi information in four letter words and even discovering words that were not represented in the data these maximum entropy models incorporate letter interactions through a set of pairwise potentials and thus define an energy landscape on the space of possible words guided by the large letter redundancy we seek a lower dimensional encoding of the letter distribution and show that distinctions between local minima in the landscape account for ensuremath sim 68 mathrm of the four letter entropy we suggest that these states provide an effective vocabulary which is matched to the frequency of word use and much smaller than the full lexicon

beyond hirsch conjecture walks on random polytopes and smoothed complexity of the simplex method the smoothed analysis of algorithms is concerned with the expected running time of an algorithm under slight random perturbations of arbitrary inputs spielman and teng proved that the shadow vertex simplex method has polynomial smoothed complexity on a slight random perturbation of an arbitrary linear program the simplex method finds the solution after a walk on polytope s with expected length polynomial in the number of constraints n the number of variables d and the inverse standard deviation of the perturbation 1 sigma we show that the length of walk in the simplex method is actually polylogarithmic in the number of constraints n spielman teng s bound on the walk was o n 86 d 55 sigma 30 up to logarithmic factors we improve this to o log 7n d 9 d 3 sigma 4 this shows that the tight hirsch conjecture n d on the length of walk on polytopes is not a limitation for the smoothed linear programming random perturbations create short paths between vertices we propose a randomized phase i for solving arbitrary linear programs which is of independent interest instead of finding a vertex of a feasible set we add a vertex at random to the feasible set this does not affect the solution of the linear program with constant probability so in expectation it takes a constant number of independent trials until a correct solution is found this overcomes one of the major difficulties of smoothed analysis of the simplex method one can now statistically decouple the walk from the smoothed linear program this yields a much better reduction of the smoothed complexity to a geometric quantity the size of planar sections of random polytopes we also improve upon the known estimates for that size showing that it is polylogarithmic in the number of vertices

a discrete sampling inversion scheme for the heat equation we present a simple and extremely accurate procedure for approximating initial temperature for the heat equation on the line using a discrete time and spatial sampling the procedure is based on the sinc expansion which for functions in a particular class yields a uniform exponential error bound with exponent depending on the number of spatial sample locations chosen further the temperature need only be sampled at one and the same temporal value for each of the spatial sampling points forn spatial sample points the approximation is reduced to solving a linear system with a 2n 1 2n 1 coefficient matrix this matrix is a symmetric centrosymmetric toeplitz matrix and hence can be determined by computing only 2n 1 values using quadratures

estimation of connectivity measures in gappy time series a new method is proposed to compute connectivity measures on multivariate time series with gaps rather than removing or filling the gaps the r ows of the joint data matrix containing empty entries are removed and the calculations are done on the remainder matrix the method called measure adapted gap removal magr can be applied to any connectivity measure that uses a joint data matrix such as cross correlation cross mutual information and transfer entropy magr is favorably compared using these three measures to a number of known gap filling te chniques as well as the gap closure the superiority of magr is illustrated on time series from synthetic systems and financial time series

comparison of six methods for the detection of causality in a bivariate time series in this comparative study six causality detection methods were compared namely the granger vector autoregressive test the extended granger test the kernel version of the granger test the conditional mutual information transfer entropy the evaluation of cross mappings between state spaces and an assessment of predictability improvement due to the use of mixed predictions seven test data sets were analyzed linear coupling of autoregressive models a unidirectional connection of two h enon systems a unidirectional connection of chaotic systems of r ossler and lorenz type and of two different r ossler systems an example of bidirectionally connected two species systems a fishery model as an example of two correlated observables without a causal relationship and an example of mediated causality we tested not only 20 phantom rule 0 16em 0ex 000 points long clean time series but also noisy and short variants of the data the standard and the extended granger tests worked only for the autoregressive models the remaining methods were more successful with the more complex test examples although they differed considerably in their capability to reveal the presence and the direction of coupling and to distinguish causality from mere correlation

topology selection in graphical models of autoregressive processes an algorithm is presented for topology selection in graphical models of autoregressive gaussian time series the graph topology of the model represents the sparsity pattern of the inverse spectrum of the time series and characterizes conditional independence relations between the variables the method proposed in the paper is based on an l1 type nonsmooth regularization of the conditional maximum likelihood estimation problem we show that this reduces to a convex optimization problem and describe a large scale algorithm that solves the dual problem via the gradient projection method results of experiments with randomly generated and real data sets are also included

inverse parabolic problems and discrete orthogonality this paper considers a discrete sampling scheme for the approximate recovery of initial data for one dimensional parabolic initial boundary value problems on a bounded interval to obtain a given approximate data is sampled at a single time and at a finite number of spatial points the significance of this inversion scheme is the ability to accurately predict the error in approximation subject to choice of sample time and spatial sensor locations the method is based on a discrete analogy of the continuous orthogonality for sturm liouville systems this property which is of independent mathematical interest is the notion of discrete orthogonal systems which loosely speaking provides an exact or approximate gauss type quadrature for the continuous biorthogonality conditions

direct coupling information measure from nonuniform embedding a measure to estimate the direct and directional coupling in multivariate time series is proposed the measure is an extension of a recently published measure of conditional mutual information from mixed embedding mime for bivariate time series in the proposed measure of partial mime pmime the embedding is on all observed variables and it is optimized in explaining the response variable it is shown that pmime detects correctly direct coupling and outperforms the linear conditional granger causality and the partial transfer entropy we demonstrate that pmime does not rely on significance test and embedding parameters and the number of observed variables has no effect on its statistical accuracy it may only slow the computations the importance of these points is shown in simulations and in an application to epileptic multichannel scalp electroencephalograms

transfer entropy computation using the perron frobenius operator we propose a method for computing the transfer entropy between time series using ulam s approximation of the perron frobenius transfer operator associated with the map generating the dynamics our method differs from standard transfer entropy estimators in that the invariant measure is estimated not directly from the data points but from the invariant distribution of the transfer operator approximated from the data points for sparse time series and low embedding dimension the transfer operator is approximated using a triangulation of the attractor whereas for data rich time series or higher embedding dimension we use a faster grid approach we compare the performance of our methods with existing estimators such as the k nearest neighbors method and kernel density estimation method using coupled instances of well known chaotic systems coupled logistic maps and a coupled rossler lorenz system we find that our estimators are robust against moderate levels of noise for sparse time series with less than 100 observations and low embedding dimension our triangulation estimator shows improved ability to detect coupling directionality relative to standard transfer entropy estimators

numerical methods based on whittaker cardinal or sinc functions this paper summarizes the results known to date for using sine functions composed with other functions as bases for approximations in numerical analysis described in this paper are methods of interpolation and approximation of functions and their derivatives quadrature the approximate evaluation of transforms hilbert fourier laplace hankel and mellin and the approximate solution of differential and integral equations the methods have many advantages over classical methods which use polynomials as bases in addition all of the methods converge at an optimal rate if singularities on the boundary of approximation are ignored

optimum choice of moving sensor trajectories for distributed parameter system identification the problem is considered of the optimal choice of moving sensor trajectories from the viewpoint of distributed parameter system identification accuracy the determinant of the information matrix of parameter estimates is taken as a measure of identification accuracy necessary and sufficient conditions for the trajectories to be optimal are derived it is shown that the optimum trajectories can be found by solving a sequence of optimal sensor allocation problems a number of examples illustrates the proposed approach

phase transitions in the coloring of random graphs we consider the problem of coloring the vertices of a large sparse random graph with a given number of colors so that no adjacent vertices have the same color using the cavity method we present a detailed and systematic analytical study of the space of proper colorings solutions we show that for a fixed number of colors and as the average vertex degree number of constraints increases the set of solutions undergoes several phase transitions similar to those observed in the mean field theory of glasses first at the clustering transition the entropically dominant part of the phase space decomposes into an exponential number of pure states so that beyond this transition a uniform sampling of solutions becomes hard afterward the space of solutions condenses over a finite number of the largest states and consequently the total entropy of solutions becomes smaller than the annealed one another transition takes place when in all the entropically dominant states a finite fraction of nodes freezes so that each of these nodes is allowed a single color in all the solutions inside the state eventually above the coloring threshold no more solutions are available

discrete observability of the heat equation on bounded domains the problem of discrete observation for the heat equation is considered it is shown that under a wide variety of conditions it is possible to recover the initial data for the heat equation from measurements that are discrete in time and space the conditions under which the initial data is recoverable depend on the geometry of the underlying domain and consequently on the spectrum of the associated eigenvalue problem

controlling the false discovery rate a practical and powerful approach to multiple testing summary the common approach to the multiplicity problem calls for controlling the familywise error rate fwer this approach though has faults and we point out a few a different approach to problems of multiple significance testing is presented it calls for controlling the expected proportion of falsely rejected hypotheses the false discovery rate this error rate is equivalent to the fwer when all hypotheses are true but is smaller otherwise therefore in problems where the control of the false discovery rate rather than that of the fwer is desired there is potential for a gain in power a simple sequential bonferronitype procedure is proved to control the false discovery rate for independent test statistics and a simulation study shows that the gain in power is substantial the use of the new procedure and the appropriateness of the criterion are illustrated with examples

on the minimal rank in non reflexive operator spaces over finite fields let u and v be vector spaces over a field k and s be an n dimensional linear subspace of l u v the space s is called algebraically reflexive whenever it contains every linear map g u v such that for all x u there exists f s with g x f x a theorem of meshulam and semrl states that if s is not algebraically reflexive then it contains a non zero operator of rank at most 2n 2 provided that k has more than n 2 elements in the present article we prove that the provision on the cardinality of the underlying field is unnecessary to do so we demonstrate that the above result holds for all finite fields

range compatible homomorphisms on matrix spaces abstract let k be a commutative field and u and v be finite dimensional vector spaces over k let s be a linear subspace of the space l u v of all linear operators from u to v a map f s v is called range compatible when f s im s for all s s obvious examples of such maps are the evaluation maps s s x with x u in this article we classify all the range compatible group homomorphisms on s provided that codim l u v s 2 dim v 3 unless k has cardinality 2 and codim l u v s 2 dim v 3 under those assumptions it is shown that the linear range compatible maps are the evaluation maps and the above upper bound on the codimension of s is optimal for this result to hold as an application we obtain new sufficient conditions for the algebraic reflexivity of an operator space and with the above conditions on the codimension of s we give an explicit description of the range restricting and range preserving homomorphisms on s

reflexivity algebraic reflexivity and linear interpolation introduction this paper was motivated by a discovery that most finite dimensional subalgebras of b h for h an infinite dimensional hilbert space are reflexive with the only obstructions to reflexivity being finite rank considerations proofs section 2 do not depend on multiplicative structure nor on topology so extend to linear subspaces of transformations in an abstract setting abstract reflexivity has been studied in 3 4 7 primarily for singly generated algebras reflexivity properties can be interpreted as linear interpolation properties and we shall adopt this point of view the results of section 2 extend in section 3 to algebraic reflexivity counterparts for countably generated algebraically linear subspaces of bounded linear transformations acting on a banach space as consequences in section 4 we obtain generalizations of two single operator results one is a multivariate version of the result due to kaplansky 10 theorem 15 that a bounded locally algebraic operator acting on a banach space is algebraic the other is a multivariate version of the result due to douglas and foias 5 for hilbert space and to hadwin 7 for a general banach space extending work of fillmore 6 that a bounded non algebraic operator acting on a banach space is topologically algebraically reflexive in section 5 we give an application kindly suggested by d hadwin concerning joint strong similarity orbits of n tupules of operators we wish to thank e azoff k davidson j erdos d hadwin j kraus c lance and a sourour for conversations and some correspon

chaotic hypothesis onsager reciprocity and fluctuation dissipation theorem it is shown that the chaoticity hypothesis recently introduced in statistical mechanics which is analogous to ruelle s principle for turbulence implies the onsager reciprocity and the fluctuation dissipation theorem in various reversible models for coexisting transport phenomena

reflexivity and hyperreflexivity of operator spaces abstract in this paper we show that if s is an n dimensional subspace of l v such that every nonzero transformation of s has rank greater than or equal to 2 n 1 then s is algebraically reflexive if s is an n dimensional subspace of b h such that every nonzero transformation of s has rank greater than or equal to 2 n 1 then s is hyperreflexive we also consider how to construct some new hyperreflexive subspaces

modeling and synchronizing chaotic systems from time series data the problem of obtaining ordinary differential equations ode s from chaotic time series data is addressed the vector fields for the ode s are polynomials constructed from a basis set that is orthonormal on the data the method for constructing the model is similar to the integration of ode s using adams predictor corrector integration the method is compared to the usual euler model and is shown to be accurate for much larger sampling intervals in addition the method used to construct the model is capable of determining the optimal polynomial vector field for the given data finally we demonstrate that it is possible to synchronize in the sense of fujisaka and yamada prog theor phys 69 32 1983 as well as pecora and carroll phys rev lett 64 821 1990 phys rev a 44 2374 1991 the model to a time series synchronization is used as a nontrivial test to determine how close the model vector field is to the true vector field implications and possible applications of synchronization are discussed

statistical mechanics for natural flocks of birds flocking is a typical example of emergent collective behavior where interactions between individuals produce collective patterns on the large scale here we show how a quantitative microscopic theory for directional ordering in a flock can be derived directly from field data we construct the minimally structured maximum entropy model consistent with experimental correlations in large flocks of starlings the maximum entropy model shows that local pairwise interactions between birds are sufficient to correctly predict the propagation of order throughout entire flocks of starlings with no free parameters we also find that the number of interacting neighbors is independent of flock density confirming that interactions are ruled by topological rather than metric distance finally by comparing flocks of different sizes the model correctly accounts for the observed scale invariance of long range correlations among the fluctuations in flight direction