common randomness in information theory and cryptography i secret sharing as the first part of a study of problems involving common randomness at distance locations information theoretic models of secret sharing generating a common random key at two terminals without letting an eavesdropper obtain information about this key are considered the concept of key capacity is defined single letter formulas of key capacity are obtained for several models and bounds to key capacity are derived for other models

a copula approach to assessing granger causality in neuroscience as in many other fields of science and engineering it is crucial to assess the causal interactions among multivariate time series granger causality has been increasingly used to identify causal influence between time series based on multivariate autoregressive models such an approach is based on linear regression framework with implicit gaussian assumption of model noise residuals having constant variance as a consequence this measure cannot detect the cause effect relationship in high order moments and nonlinear causality here we propose an effective model free copula based granger causality measure that can be used to reveal nonlinear and high order moment causality we first formulate granger causality as the log likelihood ratio in terms of conditional distribution and then derive an efficient estimation procedure using conditional copula we use resampling techniques to build a baseline null hypothesis distribution from which statistical significance can be derived we perform a series of simulations to investigate the performance of our copula based granger causality and compare its performance against other state of the art techniques our method is finally applied to neural field potential time series recorded from visual cortex of a monkey while performing a visual illusion task

coding theorems for a continuous time gaussian channel with feedback the main aim of the paper is to prove a coding theorem for a continuous time gaussian channel with feedback under an average power constraint in the case of discrete time the coding theorem for the feedback gaussian channel has been shown by cover and pombra 1989

coordination capacity we develop elements of a theory of cooperation and coordination in networks rather than considering a communication network as a means of distributing information or of reconstructing random processes at remote nodes we ask what dependence can be established among the nodes given the communication constraints specifically in a network with communication rates ri j between the nodes we ask what is the set of all achievable joint distributions p x1 xm of actions at the nodes of the network several networks are solved including arbitrarily large cascade networks distributed cooperation can be the solution to many problems such as distributed games distributed control and establishing mutual information bounds on the influence of one part of a physical system on another

measuring sample path causal influences with relative entropy we present a sample path dependent measure of causal influence between time series the proposed causal measure is a random sequence a realization of which enables identification of specific patterns that give rise to high levels of causal influence we show that these patterns cannot be identified by existing measures such as directed information di we demonstrate how sequential prediction theory may be leveraged to estimate the proposed causal measure and introduce a notion of regret for assessing the performance of such estimators we prove a finite sample bound on this regret that is determined by the worst case regret of the sequential predictors used in the estimator justification for the proposed measure is provided through a series of examples simulations and application to stock market data within the context of estimating di we show that because joint markovicity of a pair of processes does not imply the marginal markovicity of individual processes commonly used plug in estimators of di will be biased for a large subset of jointly markov processes we introduce a notion of di with stale history which can be combined with a plug in estimator to upper and lower bound the di when marginal markovicity does not hold

multivariate granger causality and generalized variance granger causality analysis is a popular method for inference on directed interactions in complex systems of many variables a shortcoming of the standard framework for granger causality is that it only allows for examination of interactions between single univariate variables within a system perhaps conditioned on other variables however interactions do not necessarily take place between single variables but may occur among groups or ensembles of variables in this study we establish a principled framework for granger causality in the context of causal interactions among two or more multivariate sets of variables building on geweke s seminal 1982 work we offer additional justifications for one particular form of multivariate granger causality based on the generalized variances of residual errors taken together our results support a comprehensive and theoretically consistent extension of granger causality to the multivariate case treated individually they highlight several specific advantages of the generalized variance measure which we illustrate using applications in neuroscience as an example we further show how the measure can be used to define partial granger causality in the multivariate context and we also motivate reformulations of causal density and granger autonomy our results are directly applicable to experimental data and promise to reveal new types of functional relations in complex systems neural and otherwise

measures of conditional linear dependence and feedback between time series abstract measures of linear dependence and feedback for two multiple time series conditional on a third are defined the measure of conditional linear dependence is the sum of linear feedback from the first to the second conditional on the third linear feedback from the second to the first conditional on the third and instantaneous linear feedback between the first and second series conditional on the third the measures are non negative and may be expressed in terms of measures of unconditional feedback between various combinations of the three series the measures of conditional linear feedback can be additively decomposed by frequency estimates of these measures are straightforward to compute and their distribution can be routinely approximated by bootstrap methods an empirical example involving real output money and interest rates is presented

secret key agreement by public discussion from common information the problem of generating a shared secret key s by two parties knowing dependent random variables x and y respectively but not sharing a secret key initially is considered an enemy who knows the random variable z jointly distributed with x and y according to some probability distribution p sub xyz can also receive all messages exchanged by the two parties over a public channel the goal of a protocol is that the enemy obtains at most a negligible amount of information about s upper bounds on h s as a function of p sub xyz are presented lower bounds on the rate h s n as n to infinity are derived for the case in which x x sub 1 x sub n y y sub 1 y sub n and z z sub 1 z sub n result from n independent executions of a random experiment generating x sub i y sub i and z sub i for i 1 n it is shown that such a secret key agreement is possible for a scenario in which all three parties receive the output of a binary symmetric source over independent binary symmetric channels even when the enemy s channel is superior to the other two channels

exploring transient transfer entropy based on a group wise ica decomposition of eeg data this paper presents a data driven pipeline for studying asymmetries in mutual interdependencies between distinct components of eeg signal due to volume conductance estimating coherence between scalp electrodes may lead to spurious results a group based independent component analysis ica which is conducted across all subjects and conditions simultaneously is an alternative representation of the eeg measurements within this approach the extracted components are independent in a global sense while short lived or transient interdependencies may still be present between the components in this paper functional roles of the ica components are specified through a partial least squares pls analysis of task effects within the time course of the derived components functional integration is estimated within the information theoretic approach using transfer entropy analysis based on asymmetries in mutual interdependencies of reconstructed phase dynamics a secondary pls analysis is performed to assess robust task specific changes in transfer entropy estimates between functionally specific components

distributed channel synthesis two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel in which a stochastic channel output is generated based on a compressed description of the channel input wyner s common information is the minimum description rate needed however when common randomness independent of the input is available the necessary description rate reduces to shannon s mutual information this paper characterizes the optimal tradeoff between the amount of common randomness used and the required rate of description we also include a number of related derivations including the effect of limited local randomness rate requirements for secrecy applications to game theory and new insights into common information duality our proof makes use of a soft covering lemma known in the literature for its role in quantifying the resolvability of a channel the direct proof achievability constructs a feasible joint distribution over all parts of the system using a soft covering from which the behavior of the encoder and decoder is inferred with no explicit reference to joint typicality or binning of auxiliary interest this paper also generalizes and strengthens this soft covering tool

rate distortion theory for secrecy systems secrecy in communication systems is measured herein by the distortion that an adversary incurs the transmitter and receiver share secret key which they use to encrypt communication and ensure distortion at an adversary a model is considered in which an adversary not only intercepts the communication from the transmitter to the receiver but also potentially has side information in particular the adversary may have causal or noncausal access to a signal that is correlated with the source sequence or the receiver s reconstruction sequence the main contribution is the characterization of the optimal tradeoff among communication rate secret key rate distortion at the adversary and distortion at the legitimate receiver it is demonstrated that causal side information at the adversary plays a pivotal role in this tradeoff it is also shown that measures of secrecy based on normalized equivocation are a special case of the framework

the likelihood encoder for lossy compression a likelihood encoder is studied in the context of lossy source compression the analysis of the likelihood encoder is based on the soft covering lemma it is demonstrated that the use of a likelihood encoder together with the soft covering lemma yields simple achievability proofs for classical source coding problems the cases of the point to point rate distortion function the rate distortion function with side information at the decoder i e the wyner ziv problem and the multi terminal source coding inner bound i e the berger tung problem are examined in this paper furthermore a non asymptotic analysis is used for the point to point case to examine the upper bound on the excess distortion provided by this method the likelihood encoder is also related to a recent alternative technique using the properties of random binning

spectral methods based on prolate spheroidal wave functions for hyperbolic pdes we examine the merits of using prolate spheroidal wave functions pswfs as basis functions when solving hyperbolic pdes using pseudospectral methods the relevant approximation theory is reviewed and some new approximation results in sobolev spaces are established an optimal choice of the band limit parameter for pswfs is derived for single mode functions our conclusion is that one might gain from using the pswfs over the traditional chebyshev or legendre methods in terms of accuracy and efficiency for marginally resolved broadband solutions

transfer entropy as a log likelihood ratio transfer entropy an information theoretic measure of time directed information transfer between joint processes has steadily gained popularity in the analysis of complex stochastic dynamics in diverse fields including the neurosciences ecology climatology and econometrics we show that for a broad class of predictive models the log likelihood ratio test statistic for the null hypothesis of zero transfer entropy is a consistent estimator for the transfer entropy itself for finite markov chains furthermore no explicit model is required in the general case an asymptotic 2 distribution is established for the transfer entropy estimator the result generalizes the equivalence in the gaussian case of transfer entropy and granger causality a statistical notion of causal influence based on prediction via vector autoregression and establishes a fundamental connection between directed information transfer and causality in the wiener granger sense

directed information causal estimation and communication in continuous time a notion of directed information between two continuous time processes is proposed a key component in the definition is taking an infimum over all possible partitions of the time interval which plays a role no less significant than the supremum over space partitions inherent in the definition of mutual information properties and operational interpretations in estimation and communication are then established for the proposed notion of directed information for the continuous time additive white gaussian noise channel it is shown that duncan s classical relationship between causal estimation error and mutual information continues to hold in the presence of feedback upon replacing mutual information by directed information a parallel result is established for the poisson channel the utility of this relationship is demonstrated in computing the directed information rate between the input and output processes of a continuous time poisson channel with feedback where the channel input process is constrained to be constant between events at the channel output finally the capacity of a wide class of continuous time channels with feedback is established via directed information characterizing the fundamental limit on reliable communication

estimating mutual information we present two classes of improved estimators for mutual information m x y from samples of random points distributed according to some joint probability density mu x y in contrast to conventional estimators based on binnings they are based on entropy estimates from k nearest neighbor distances this means that they are data efficient with k 1 we resolve structures down to the smallest possible scales adaptive the resolution is higher where data are more numerous and have minimal bias indeed the bias of the underlying entropy estimates is mainly due to nonuniformity of the density at the smallest resolved scale giving typically systematic errors which scale as functions of k n for n points numerically we find that both families become exact for independent distributions i e the estimator m x y vanishes up to statistical fluctuations if mu x y mu x mu y this holds for all tested marginal distributions and for all dimensions of x and y in addition we give estimators for redundancies between more than two random variables we compare our algorithms in detail with existing algorithms finally we demonstrate the usefulness of our estimators for assessing the actual independence of components obtained from independent component analysis ica for improving ica and for estimating the reliability of blind source separation

analyzing functional connectivity using a network likelihood model of ensemble neural spiking activity analyzing the dependencies between spike trains is an important step in understanding how neurons work in concert to represent biological signals usually this is done for pairs of neurons at a time using correlation based techniques chornoboy schramm and karr 1988 proposed maximum likelihood methods for the simultaneous analysis of multiple pair wise interactions among an ensemble of neurons one of these methods is an iterative continuous time estimation algorithm for a network likelihood model formulated in terms of multiplicative conditional intensity functions we devised a discrete time version of this algorithm that includes a new efficient computational strategy a principled method to compute starting values and a principled stopping criterion in an analysis of simulated neural spike trains from ensembles of interacting neurons the algorithm recovered the correct connectivity matrices and interaction parameters in the analysis of spike trains from an ensemble of rat hippocampal place cells the algorithm identified a connectivity matrix and interaction parameters consistent with the pattern of conjoined firing predicted by the overlap of the neurons spatial receptive fields these results suggest that the network likelihood model can be an efficient tool for the analysis of ensemble spiking activity

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

estimating the directed information and testing for causality the problem of estimating the directed information rate between two discrete processes x n and y n via the plug in or maximum likelihood estimator is considered when the joint process x n y n is a markov chain of a given memory length the plug in estimator is shown to be asymptotically gaussian and to converge at the optimal rate o 1 sqrt n under appropriate conditions this is the first estimator that has been shown to achieve this rate an important connection is drawn between the problem of estimating the directed information rate and that of performing a hypothesis test for the presence of causal influence between the two processes under fairly general conditions the null hypothesis which corresponds to the absence of causal influence is equivalent to the requirement that the directed information rate be equal to zero in that case a finer result is established showing that the plug in converges at the faster rate o 1 n and that it is asymptotically chi 2 distributed this is proved by showing that this estimator is equal to a scalar multiple of the classical likelihood ratio statistic for the above hypothesis test finally it is noted that these results facilitate the design of an actual likelihood ratio test for the presence or absence of causal influence

capacity and error exponent for the direct detection photon channel ii for pt i see ibid vol 34 no 6 p 1449 61 1980 the discussion of the capacity and error exponent of the direct detection optical channel is continued the channel input in a t second interval is a waveform satisfying certain peak and average power constraints for the optical signals the channel output is a poisson process with an intensity parameter that accounts for the dark current component an upper bound is obtained on the error exponent which coincides with the lower bound thus this channel is one for which the error exponent can be known exactly

granger causality and transfer entropy are equivalent for gaussian variables granger causality is a statistical notion of causal influence based on prediction via vector autoregression developed originally in the field of econometrics it has since found application in a broader arena particularly in neuroscience more recently transfer entropy an information theoretic measure of time directed information transfer between jointly dependent processes has gained traction in a similarly wide field while it has been recognized that the two concepts must be related the exact relationship has until now not been formally described here we show that for gaussian variables granger causality and transfer entropy are entirely equivalent thus bridging autoregressive and information theoretic approaches to data driven causal inference

a large deviations approach to secure lossy compression a shannon cipher system for memoryless sources in which distortion is allowed at the legitimate decoder is considered the source is compressed using a secured rate distortion code which satisfies a constraint on the compression rate as well as a constraint on the exponential rate of the excess distortion probability at the legitimate decoder secrecy is measured by the exponential rate of the exiguous distortion probability at the eavesdropper rather than by the traditional measure of equivocation the perfect secrecy exponent is defined as the maximal exiguous distortion exponent achievable when the key rate is unlimited the reproduction based estimate exponent is defined as the maximal exiguous distortion exponent achievable for a genie aided eavesdropper which knows the secret key under limited key rate it is proved that the maximal achievable exiguous distortion exponent is equal to the minimum between the key rate plus the reproduction based estimate exponent and the perfect secrecy exponent the result is generalized to a fairly general class of variable key rate and coding rate codes

measurement of linear dependence and feedback between multiple time series abstract measures of linear dependence and feedback for multiple time series are defined the measure of linear dependence is the sum of the measure of linear feedback from the first series to the second linear feedback from the second to the first and instantaneous linear feedback the measures are nonnegative and zero only when feedback causality of the relevant type is absent the measures of linear feedback from one series to another can be additively decomposed by frequency a readily usable theory of inference for all of these measures and their decompositions is described the computations involved are modest

nonlinear transformation on the transfer entropy of financial time series abstract transfer entropy te now is widely used in the data mining and economic field however te itself demands that time series intend to be stationary and meet markov condition naturally we are interested in investigating the effect of the nonlinear transformation of the two series on the te therefore the paper is designed to study the te of five nonlinear volatile transformations based on the data which are generated by the linear modeling and the logistic maps modeling as well as the dataset that come from financial markets with only one of the te of nonlinear transformations fluctuating around the te of original series the te of others all have increased with different degrees

semantic security capacity for wiretap channels of type ii the secrecy capacity of the type ii wiretap channel wtc ii with a noisy main channel is currently an open problem herein its secrecy capacity is derived and shown to be equal to its semantic security ss capacity in this setting the legitimate users communicate via a discrete memoryless dm channel in the presence of an eavesdropper that has perfect access to a subset of its choosing of the transmitted symbols constrained to a fixed fraction of the blocklength the secrecy criterion is achieved simultaneously for all possible eavesdropper subset choices the ss criterion demands negligible mutual information between the message and the eavesdropper s observations even when maximized over all message distributions a key tool for the achievability proof is a novel and stronger version of wyner s soft covering lemma specifically a random codebook is shown to achieve the soft covering phenomenon with high probability the probability of failure is doubly exponentially small in the blocklength since the combined number of messages and subsets grows only exponentially with the blocklength ss for the wtc ii is established by using the union bound and invoking the stronger soft covering lemma the direct proof shows that rates up to the weak secrecy capacity of the classic wtc with a dm erasure channel ec to the eavesdropper are achievable the converse follows by establishing the capacity of this dm wiretap ec as an upper bound for the wtc ii from a broader perspective the stronger soft covering lemma constitutes a tool for showing the existence of codebooks that satisfy exponentially many constraints a beneficial ability for many other applications in information theoretic security

information networks with in block memory a class of channels is introduced for which there is memory inside blocks of a specified length and no memory across the blocks the multiuser model is called an information network with in block memory nibm it is shown that block fading channels channels with state known causally at the encoder and relay networks with delays are nibms a cut set bound is developed for nibms that unifies strengthens and generalizes existing cut bounds for discrete memoryless networks the bound gives new finite letter capacity expressions for several classes of networks including point to point channels and certain multiaccess broadcast and relay channels cardinality bounds on the random coding alphabets are developed that improve on existing bounds for channels with action dependent state available causally at the encoder and for relays without delay finally quantize forward network coding is shown to achieve rates within an additive gap of the new cut set bound for linear additive gaussian noise channels symmetric power constraints and a multicast session

source coding with feed forward rate distortion theorems and error exponents for a general source in this work we consider a source coding model with feed forward we analyze a system with a noiseless feed forward link where the decoder has knowledge of all previous source samples while reconstructing the present sample the rate distortion function for an arbitrary source with feed forward is derived in terms of directed information a variant of mutual information we further investigate the nature of the rate distortion function with feed forward for two common types of sources discrete memory less sources and gaussian sources we then characterize the error exponent for a general source with feed forward the results are then extended to feed forward with an arbitrary delay larger than the block length

generalized holomorphic szego kernel in 3d spheroids monogenic orthogonal polynomials over 3d prolate spheroids were previously introduced and shown to have some remarkable properties in particular the underlying functions take values in the quaternions identified with r 4 and are generally assumed to be nullsolutions of the well known moisil theodoresco system in this paper we show that these polynomial functions play an important role in defining the szego kernel function over the surface of 3d prolate spheroids as a concrete application we prove an explicit expression of the monogenic szego kernel function over 3d prolate spheroids and present two numerical examples

finite state channels with time invariant deterministic feedback we consider capacity of discrete time channels with feedback for the general case where the feedback is a time invariant deterministic function of the output samples under the assumption that the channel states take values in a finite alphabet we find a sequence of achievable rates and a sequence of upper bounds on the capacity the achievable rates and the upper bounds are computable for any n and the limits of the sequences exist we show that when the probability of the initial state is positive for all the channel states then the capacity is the limit of the achievable rate sequence we further show that when the channel is stationary indecomposable and has no intersymbol interference isi its capacity is given by the limit of the maximum of the normalized directed information between the input xn and the output yn i e c limnrarrinfin 1 n max i xnrarryn where the maximization is taken over the causal conditioning probability q xnparzn 1 defined in this paper the main idea for obtaining the results is to add causality into gallager s results on finite state channels the capacity results are used to show that the source channel separation theorem holds for time invariant determinist feedback and if the state of the channel is known both at the encoder and the decoder then feedback does not increase capacity

local distortion of m conformal mappings a conformal mapping in a plane domain locally maps circles to circles more generally quasiconformal mappings locally map circles to ellipses of bounded distortion in this work we study the corresponding situation for solutions to stein weiss systems in the n 1 d euclidean space this class of solutions is a transformation of a subset of monogenic locally quasiconformal mappings with nonvanishing jacobian in the theoretical part of this work we prove that an m conformal mapping locally maps the unit hypersphere onto explicitly characterized hyperellipsoids and vice versa then we discuss quasiconformal radial mappings and their relations with the cauchy kernel and p monogenic mappings this is followed by the consideration of quadratic m conformal mappings in the applications part of this work we provide the reader with some plot examples that demonstrate the effectiveness of our approach

an operational approach to information leakage given two random variables x and y an operational approach is undertaken to quantify the leakage of information from x to y the resulting measure mathcal l left x to y right is called maximal leakage and is defined as the multiplicative increase upon observing y of the probability of correctly guessing a randomized function of x maximized over all such randomized functions a closed form expression for mathcal l left x to y right is given for discrete x and y and it is subsequently generalized to handle a large class of random variables the resulting properties are shown to be consistent with an axiomatic view of a leakage measure and the definition is shown to be robust to variations in the setup moreover a variant of the shannon cipher system is studied in which performance of an encryption scheme is measured using maximal leakage a single letter characterization of the optimal limit of normalized maximal leakage is derived and asymptotically optimal encryption schemes are demonstrated furthermore the sample complexity of estimating maximal leakage from data is characterized up to subpolynomial factors finally the guessing framework used to define maximal leakage is used to give operational interpretations of commonly used leakage measures such as shannon capacity maximal correlation and local differential privacy

on the reliability function of the ideal poisson channel with noiseless feedback a coding scheme for the channel under peak power and average power constraints on the input is presented and its asymptotic error exponent is shown to coincide at all rates below capacity with the sphere packing error exponent which for the case at hand is known to be unachievable without feedback for rates below the critical rate an upper bound on the error exponent achievable with feedback is also derived and shown under a capacity reducing average power constraint to coincide with the error exponent achieved by the proposed coding scheme in such a case the coding scheme is asymptotically optimal thus the ideal poisson channel limited by a capacity reducing average power constraint provides a nontrivial example of a channel for which the reliability function is known exactly both with and without feedback it is shown that a slight modification of the coding scheme to one of random transmission time can achieve zero error probability for any rate lower than the ordinary average error channel capacity

feedback capacity of the compound channel in this work we find the capacity of a compound finite state channel fsc with time invariant deterministic feedback we consider the use of fixed length block codes over the compound channel our achievability result includes a proof of the existence of a universal decoder for the family of fscs with feedback as a consequence of our capacity result we show that feedback does not increase the capacity of the compound gilbert elliot channel additionally we show that for a stationary and uniformly ergodic markovian channel if the compound channel capacity is zero without feedback then it is zero with feedback finally we use our result on the fsc to show that the feedback capacity of the memoryless compound channel is given by infthetas maxqx i x y thetas

a correlation measure based on vector valued l_p norms in this paper we introduce a new measure of correlation for bipartite quantum states this measure depends on a parameter alpha and is defined in terms of vector valued textit l textit p norms the measure is within a constant of the exponential of alpha renyi mutual information and reduces to the trace norm total variation distance for alpha 1 we will prove some decoupling type theorems in terms of this measure of correlation and present some applications in privacy amplification as well as in bounding the random coding exponents in particular we establish a bound on the secrecy exponent of the wiretap channel under the total variation metric in terms of the alpha renyi mutual information according to csiszar s proposal

the capacity of channels with feedback in this paper we introduce a general framework for treating channels with memory and feedback first we prove a general feedback channel coding theorem based on massey s concept of directed information second we present coding results for markov channels this requires determining appropriate sufficient statistics at the encoder and decoder we give a recursive characterization of these sufficient statistics third a dynamic programming framework for computing the capacity of markov channels is presented fourth it is shown that the average cost optimality equation acoe can be viewed as an implicit single letter characterization of the capacity fifth scenarios with simple sufficient statistics are described sixth error exponents for channels with feedback are presented

interpretations of directed information in portfolio theory data compression and hypothesis testing we investigate the role of directed information in portfolio theory data compression and statistics with causality constraints in particular we show that directed information is an upper bound on the increment in growth rates of optimal portfolios in a stock market due to causal side information this upper bound is tight for gambling in a horse race which is an extreme case of stock markets directed information also characterizes the value of causal side information in instantaneous compression and quantifies the benefit of causal inference in joint compression of two stochastic processes in hypothesis testing directed information evaluates the best error exponent for testing whether a random process y causally influences another process x or not these results lead to a natural interpretation of directed information i yn xn as the amount of information that a random sequence yn y1 y2 yn causally provides about another random sequence xn x1 x2 xn a new measure directed lautum information is also introduced and interpreted in portfolio theory data compression and hypothesis testing

prolate spheroidal wavefunctions as an alternative to chebyshev and legendre polynomials for spectral element and pseudospectral algorithms prolate spheroidal functions of order zero are generalizations of legendre polynomials which when the bandwidth parameter c 0 oscillate more uniformly on x 1 1 than either chebyshev or legendre polynomials this suggests that compared to these polynomials prolate functions give more uniform spatial resolution further when used as the spatial discretization for time dependent partial differential equations in combination with explicit time marching prolate functions allow a longer stable timestep than legendre polynomials we show that these advantages are real and further that it is almost trivial to modify existing pseudospectral and spectral element codes to use the prolate basis however improvements in spatial resolution are at most a factor of 2 approached slowly as n the timestep can be lengthened by several times but not by a factor that grows rapidly with n the prolate basis is not likely to radically expand the range of problems that can be done on a workstation however for production runs on the bleeding edge edge of arithmurgy such as numerical weather prediction the rewards for switching to a prolate basis may be considerable

information transfer in continuous processes we discuss a recently proposed quantity called transfer entropy which uses time series data to measure the amount of information transferred from one process to another in order to understand its foundation merits and limitations we review some aspects of information theoretic functionals while for symbol sequences these measures have an intuitive interpretation their application to continuous state processes and in particular their estimation from finite data sets is problematic for mutual information finite length scale estimates converge from below and can thus be used to reject the assumption that the observed processes are independent however mutual information does not provide any directional information conversely transfer entropy does resolve the directionality of information exchange but no similar monotonic convergence seems to hold thus only in the case of zero transfer entropy in one direction we can reliably infer an asymmetry of the information exchange

radial basis function approach to nonlinear granger causality of time series we consider an extension of granger causality to nonlinear bivariate time series in this frame if the prediction error of the first time series is reduced by including measurements from the second time series then the second time series is said to have a causal influence on the first one not all the nonlinear prediction schemes are suitable to evaluate causality indeed not all of them allow one to quantify how much knowledge of the other time series counts to improve prediction error we present an approach with bivariate time series modeled by a generalization of radial basis functions and show its application to a pair of unidirectionally coupled chaotic maps and to physiological examples

a coding theorem for a class of stationary channels with feedback a coding theorem is proved for a class of stationary channels with feedback in which the output yn f xn m n zn m n is the function of the current and past m symbols from the channel input xn and the stationary ergodic channel noise zn in particular it is shown that the feedback capacity is equal to limnrarr infin supp x n y n 1 1 n i xn rarr yn where i xn rarr yn sigmai 1 n i xi yi yi 1 denotes the massey directed information from the channel input to the output and the supremum is taken over all causally conditioned distributions p xn yn 1 pii 1 n p xi xi 1 yi 1 the main ideas of the proof are a classical application of the shannon strategy for coding with side information and a new elementary coding technique for the given channel model without feedback which is in a sense dual to gallager s lossy coding of stationary ergodic sources a similar approach gives a simple alternative proof of coding theorems for finite state channels by yang kavcic tatikonda chen berger and permuter weissman goldsmith

granger causality for state space models granger causality has long been a prominent method for inferring causal interactions between stochastic variables for a broad range of complex physical systems however it has been recognized that a moving average ma component in the data presents a serious confound to granger causal analysis as routinely performed via autoregressive ar modeling we solve this problem by demonstrating that granger causality may be calculated simply and efficiently from the parameters of a state space ss model since ss models are equivalent to autoregressive moving average models granger causality estimated in this fashion is not degraded by the presence of a ma component this is of particular significance when the data has been filtered downsampled observed with noise or is a subprocess of a higher dimensional process since all of these operations commonplace in application domains as diverse as climate science econometrics and the neurosciences induce a ma component we show how granger causality conditional and unconditional in both time and frequency domains may be calculated directly from ss model parameters via solution of a discrete algebraic riccati equation numerical simulations demonstrate that granger causality estimators thus derived have greater statistical power and smaller bias than ar estimators we also discuss how the ss approach facilitates relaxation of the assumptions of linearity stationarity and homoscedasticity underlying current ar methods thus opening up potentially significant new areas of research in granger causal analysis