D

D!V!C !^!' 1 ' , 3

pi« < < NATIONAL BUREAU OF STANDARDS REPORT

1293

INTRODUCTION TO THE THEORY OF STOCHASTIC PROCESSES

DEPENDING ON A CONTINUOUS PARAMETER

By

Henry B e Mann

Ohio State University

U. S, DEPARTMENT OF COMMERCE

NATIONAL BUREAU OF STANDARDS
U., S. DEPARTMENT OF COMMERCE

Charles Sawyer, Secretary

NATIONAL BUREAU OF STANDARDS

A. V. As tin , Acting Director

THE NATIONAL BUREAU OF STANDARDS

The scope of activities of the National Bureau of Standards is suggested in the following listing of the divisions and sections engaged in technical work. In general, each section is en- gaged in specialised research, development, and engineering in the field indicated by its

title. A brief description of the activities, and of the resultant reports and publications,

appears on the inside of the back cover of this report.

1. ELECTRICITY. Resistance Measurements. Inductance and Capacitance. Electrical Instruments. Magnetic Measurements. Electrochemistry.

2. OPTICS AND METROLOGY. Photometry and Colorimetry. Optical Instruments. Photographic

Technology. Length. Gage.

3. HEAT AND POWER. Temperature Measurements. Thermodynamics. Cryogenics. Engines and

Lubrication. Engine Fuels.

1. ATOMIC AND RADIATION PHYSICS. Spectroscopy. Radiometry. Mass Spectrometry. Physical Electronics. Electron Physics. Atomic Physics. Neutron Measurements. Nuclear Physics. Radioactivity. X-Rays. Betatron. Nucleonic Instrumentation. Radiological Equipment. Atomic Energy Commission Instruments Branch.

5. CHEMISTRY. Organic Coatings. Surface Chemistry. Organic Chemistry. Analytical Chemistry. Inorganic Chemistry. Electrodeposition. Gas Chemistry. Physical Chemistry. Thermochemistry. Spectrochemistry. Pure Substances.

6. MECHANICS. Sound. Mechanical Instruments. Aerodynamics. Engineering Mechanics. Hydraulics. Mass. 'Capacity, Density, and Fluid Meters.

7. ORGANIC AND FIBROUS MATERIALS. Rubber. Textiles. Paper. Leather. Testing and

Specifi cations. Organic Plastics. Dental Research.

8. METALLURGY. Thermal Metallurgy. Chemical Metallurgy. Mechanical Metallurgy . Corrosion.

9. MINERAL PRODUCTS. Porcelain and Pottery. Glass. Refractories. Enameled Metals. Building Stone. Concreting Materials. Constitution and Microstructure. Chemistry of Mineral

Products.

10. BUILDING TECHNOLOGY. Structural Engineering. Fire Protection . Heating and Air Conditioning. Exterior and Interior Coverings. Codes and Specifications.

11. APPLIED MATHEMATICS. Numerical Analysis. Computation. Statistical Engineering. Ma- chine Development.

12- FLECTRCa ICS. Engineering Electronics. Electron Thbes. Electronic Coaiputers. Elee13. ORDNANCE DEVELOPMENT. Mechanical Research and Development. Electromechanical

Fuzes. Technical Services. Missile Fuzing Research. Missile Fuzing Bevelop^jent. Projectile Fuzes. Ordnance Coniponen ts . Ordnance Teats. Ordnance Research.

IN. RADIO PROPAGATION. Upper Atmosphere Research. Ionospheric Research. Regular Propagation Services. Frequency Utilization Research. Tropospheric Propagation Research. High

Frequency Standards, t -’icu..rave Stan .rds. 15. MISSILE DEVELOPMENT. Missile Engineering. Missile Dynamics. Missile Intelligence.

Missile Instr •station. Technical Services. Combustion.
NBS PROJECT NBS REPORT

May 1951

1103 - 11-1107 1293

INTRODUCTION TO THE THEORY OP STOCHASTIC PROCESSES

DEPENDING ON A CONTINUOUS PARAMETER

This monograph to be published in the

NBS Applied Mathematics Series is the

outgrowth of a series of lectures given

at the National Bureau of Standards in

June, 19h9 , under the sponsorship of the

Statistical Engineering Laboratory

By

Henry B. Mann

Ohio State University

This report is issued for

in sny form, either in whol

froa the Office of the Dire

Approved for public release by the

Director of the National Institute of

Standards and Technology (NIST) on October 9, 2015

reprinting, or reproduction

sion in writing is obtained

ngton 25, D. C.
** - - 4ii h

;

,

;

• :

'

,
Contempiator enim*cum solis lumina cumqu©

Insert! fundunt radii per op&ca domorum^

Multi minuta media multi s per inane videbis

Corpora mi scer5 0 radiorum lumine in ipso©

Et velut eterno certsunine proeliep pugnas

Edere turmatim eertantia nec dare pausam*,

Conciliis et discidiis exercita crebris©

Coni cere ut possis ex hocpprimordia rerum

Qual© sit in magno iactari semper inani©

Bumtaxat rerum magnarum parva potest res

Exemplar© dare et vestigia notitiai©

Hoc etlam magi s haec animum te advertere par est

Corpora^ qua© in soils radiis turbare videntur

Quod tales turba© ruotus quoque materiai

Significant clandestinos caecosque subesse©

Multa videbis enim plagis ibi percita caecis

Commutax*© viam retroque pulsa revert!

Nunc hue* nunc illuc in cunetas undique partes 0 Titus Lucre tiu3 Carus

De Rerum NaturasVol ©IloVers 113^130*

Let us observe as brightly the rays of the sun

Penetrate in streams the darkness of our houses

Thousands of tiny bodies dancing in space

Approaching each other and parting in the bright light of the sun©

As if fighting a battle without pause through the ages e Like an army of soldiers restlessly warring

They advance and retreat in motion never to cease©

May you conjecture from this the very nature of matter^

How it is ceaselessly tossed through the vastness of space©

Thus a phenomenons small as it seems and of little importance

Often does indicate things highly important and great©

Hence it is well worthwhile to observe these bodies

Whirling and dancing without rest in the sunlight^

Sine© such irregular motion of visible bodies

Is a sure indication of the invisible motion of matter©

For you can see these bodies constantly changing direction^

Often reversing their motion all of a sudden

And propelled by invisible impacts moving this way and that way©
t

-

'

H
TABLE OP CONTENTS

£ OX*@WOI*d a«oeo 6 «e* 60 Goeo*Qcaaoo X

Introduction eaooQ©o©oo©Q©aec-ooQc XXX

Chapter 1» Fundamental Concepts 0 • * 0 « e « © » 0 1

X. Random variables 1

2o Convergence * . 2

3 ® Stochastic processes 10

Convergence in the mean 11

£*>-• a Differentiation iii. 6 0 Stochastic processes of second order 15

r*

1 e Integration 17

Chapter 2, Special Processes oeooooooooo 32

lo The fundamental random process 32

2a Further properties of the FwR'oP* 37

3 , Frictional effects* The

Ornate in~Uhlenbeck process 47

Chapter 3» Estimation of Parameters a*®*®.*® 55

lo Estimation of th® parameter of the F®R 0 £*. 55

2 9 FoRoPo with mean value function 57

3. Estimation of parameters for th© O s UoP. 64

Chapter k. 9 The General Differential Process * e » o 72

Chapter 5o Differential Processes Modified by

Mechanical Devices ao *.aeo 9 »®® 86

lo Filter effect

2 * Counter data

86

91

Chapter 8® The Fourier Analysis of Stochastic

^rocessss <> ® © <> © o ® • ® « « ® ® • a 1® General theory

2* Trigonometric expansion of the F e RoPo

3* Stationary processes

ko The mean ergodic theorem

100

100

103

108

112

FOREWORD

Th© theory of stochastic processes is steadily gaining

in importance and the applications ar© ever widening© Never^

theless* it is at present not easy to study this subject# since

the literature g although extensive# is widely scattered©

This situation motivated the National Bureau of Standards

to invite Professor Henry B© Mann to give a series of lectures

on stochastic processes and to write a monograph on the subject©

The lectures "were given in the period from March 1949 to June

1949 0 during which Dr© Mann was a member of the staff of th©

Bureau® s Statistical Engineering Laboratory©

It is well known that the theory of stochastic processes

depending on a continuous parameter can be developed in a

satisfactory way by studying random functions or by consider^

ing probability measures in function space© The author of

th© present monograph has however adopted a different approach

which is similar to the definition of a stochastic process

given by E© Slutskyp A random variable is considered to be

a symbol with which a distribution function is associated#

and a stochastic process is then defined as a set of random

variables® This approach loads to a tbeery which for many

practical purposes is equivalent to th© direct measure**

theoretical approach© It has th© advantage that the tech®

nicali ties of measure theory seem loss obstrusiv© at th©

outset# although for logical completness they must enter

sooner or later if the theory is to be developed in a w©ll=>

rounded way0
n

-

.

J

*
ii

It is hoped that this modest little volume, written

by i distinguished contemporary mathematician, will b© useful and Interesting in various vrays© The argument is addressed

uncompromisingly to educated mathematicians, and they will

not fail to be impressed by the skillful way in which the

author develops the theory from his chosen starting point©

The user of time ^continuous processes in the applied fields

who is not interested in the methods of proof may still

appreciate having a number of important definitions and results

conveniently gathered here between two covers.

Finally, it is hoped that the publication of the monograph will stimulate further expository efforts in the import®

ant field of time -continuous stochastic processes ^ and that

in particular the day will come a little sooner than it other®

wise might have, when a comprehensive but readable textbook

en this subject v using the measure®theoretical approach,

appears in the English language©

Jo Ho Curtiss

National Applied Mathematics Laboratories

National Bureau of Standards

V'ashington 25 * D* G©

October 1951
I

.

I
INTRODUCTION

The study of stochastic processes Is becoming increas: gly

' Important in nany branches of science and accord5.ngTy the r th®~

matlcal theo::rT of stochastic processes has progressed rapi j

during the last two decades* This rapid progress has resul ad in

a large diversification of notation and terminology which ; keg

It difficult even for a mathematician to inform himself on he

subject* It seemed, therefore, advisable to bring together under

a unified terminology and notation some of the basic definl ions

and results of this theory* The viewpoint taken was that : the

mathematical atisticiar^ and the stochastic process was a : ord®>

ingly defined as a family of distribution functions satis Ing

certain consistency relations. It was on® of the goals of be

present monograph to develop the theory of stochastic prcc scs

from this viewpoint with as little appeal to abstract meas e theory

as possible* l.n most practical problems information about sndom

variables can be obtained only in terms of their joint dls ibution

function*, and f t is the opinion of the author that a treat: e on

stochastic processes will be most useful to the statistic! if

* the definitions, theorems^ and proofs are given In these tc &«» It

is in many cases almost Impossible to trace a result to or par®

tlcular autho % and it was therefore decided to omit refer e©i

altogether* This does not mean that the author claims ere: t for

any particular result. To the author's knowledge only the: em 7

of chapter 1 and most of chapter 3 are new* (After, complc ; on of

chapter 3 the author was Informed by H* Rubin that some of he

results of this chapter had previously been obtained by him ana

L* Savage, but their results were never published,,) In hi

sentation of ie theory of stochastic processes* as well as

I

chapter 4, ti author has followed the presentations of M«

given in Paul ^evy’s book on stochastic processes and in M

paper ”0n set of probability laws and their limit element

(University c California Press, 1950), respectively* In

treatment of hunter data in chapter 5 the author hac usee

Feller 1 s appi ich and his masterful presentation In the Co

# < / Anniversary v; lume* The treatment of the Omstein Uhlenbe

process in cl, :>ter 2 follows & presentation given by J* L©

(Anno of Math , Vol© 43, No* 2)*

My than! are duo to Dr* Eugene Lukacs for his v&luab

help in prep* ng th ? final form of the manuscript and to

P. Moranda wl read ;he proofs and prepared the Index* I

wish to then} Professor M. Loeve for many helpful discuss!

the subject*

H* B* MANN

Ohio State Ur varsity

May 1951

> 3V®

> eve * s

9

infc

;Ob

Ip

t 3 on

Chapter 1

FUNDAMENTAL CONCEPTS

l ar * xnsjw. Random r-~~'-.xc. '•jtarjazrcT*variables, ttrzJ-aa? .Mfen.i jcjouts*. xxvwistt* We consider a finite or infinite set of

symbols (x*y5odcs ) such that to every finite set of symbols

*>*••***. there is defined a light continuous distribution function

JL " <]£& U0 l} F

1 2 .*3^

C VVo ® c ® n ) s P(z1 < a. $ q 9 o 9 EL S

' e w n

called the probability of the ©vent x^ < a^, ,* df x^ < a^ 0

The distribution functions- of th© family given by (! e l)

satisfy the following equations?

. Cai 99{0 j^| I — . v ^11 8«ef\l xi I

s ooof> n

J 1 o ? } '‘i pooss

J 1 ~a

where 1* is any permutation of the numbers lj>2f 00 o*&j

i 1 0 C? }

‘v v v* ' 9 0 55 * 9 ^W*s=i^ 03* ) a* i? ( { e 9 »J ^

,‘*D”| £ O C Q 9

4S*‘g* X4 ^ **»2 ® o G 6 ©

“* '*

The symbol x- is called a random variable,, For every Borel set A

in the n-dimensional Euclidean space we define the symbol

P[(x-f>in0 .,,x j cAL called the probability that the "point”

{x^ o0st r 5 lies in A by the equation

) C A] ss j 4F

a

9 © © o 9 (a

3l 8 9 0 9$

2

If gif, 9 0 * J is a Bore! measurable function, than we can define

in M

a -new random variable glx^© w * 0 *x^) by the equations

' £7 vr ^»^i»oeo^a) = Pig < &9 yX <

^t eo» 3L, < ^ ) j w " 4 • 1 1 -a m

~ W *j » O 60g 1 O sjy

a m

© o o © £ 5

We now consider sequences {x^} of random variables,, The

« notion of confergene® of such a sequence can be defined in various

ways, Ir our representation of the theory, of stochastic processes

we shall hcaaver use mainly the following definition,,

S 0 C onvergenc e

for ©very £ >

' \ q

A ©equenoe {x } will b© called convergent if

V

> O there exists an j such that

U0 4J

for n > !(&, and all h* If there exists a random variable x such

that 11m Pllx -x| > ©) =• 0 for all s then w© shall write

n-*co 3 a j -

plim x*- x and say that {j; } converges to x ©r that x is the

n~$>oo a &

probability limit of the sequence {x }« The convergence defined

above L 8 U£i-,al2y termed convergence in probability* This definition

can be extended In an obvious manner to random vectors,,

We proceed to formulate an important property of convergent

sequences

T/it' n . 1 f; t

rx } “bo a sequence of random variables. There y| • w

•inia-ra a random variable x such that plia x & x if and only if

n~>oo n

ta© aequeno© {% } converges, Moreover, if F

x

n

yi

, ** y«

are the

dlefci/lbubioo in etlaaa of x (n«*l,2, 0 . # ) and y a# *y then

X m

Um ? „ ,.:, E

a:v .

... for all points (t,b1( ) for

n*>co -"n^l 9 5 : x ®

the ft-natlor 11. . , v. (t5 b. , ec e ,b ) is continuous in t 0 3

« * * # _ •* “

which

'Phccr-jjj iol gives a condition for convergence in probability

similar tc t--ucty*s criterion* This condition was first established

by £« Slut; ehr.

:

- - « rgi

Wc> proceed to prove theorem Id. As a first

step w© sssune that the sequence |x^| is convergent and show the

existence of a random variable x » plim xH * In th© following we

114 co n

write for abbreviation

gc'«*{ q ) F ' 2’L j, &

*.7, *** et5

a

)

We ilrat pie*'© the following lemma*

LI] Itetror > ,5-89 (19&5) j 0* E e Aoad 0 Sei* Peris, 187, 370-372 (13 £8).

I 2] The p 2T- k‘? of theorem l r l may be skipped in a first reading

without affecting the understanding of the rest of the

monograph

,

4

i25£!L_i*JL ij a^y positive numbers, then £02* sufficiently

. rge n and all h > 0

' 9 ^) En

(*+S) * n > Sn4, h

(o) > g.Jo-6)- sj

r all c

,

For abbraviation we write for any ©vent &

p

bk

^S) p y, < ? « • » * y < b )

"

* 1 ~ll m — &

> that In particular

-Vx

n i') = V0)

ten

b Un£°* s > *VX»£ 8 * 6’ *

a*h

< «) 2: Whs °- >n*h~z

3

i S s>

laoe tho set of points <x , * ) for which ix - x I > s & 1 n*h a 1 "

^eludes the points for which \x — x ] > S and x < © ;, we have

n+n. n n^ 3, *'»

b

(za*h ?- °» ^ x

n+h“ V - 51 ^ V^n-h- °> ~ p< l x

n+h~

*n< > 5 > aae for sufficiently large a and all h > 0

Sn

(o+ 5 ) £ W0J n o

ra&larly Ws * 2«a!c ' s !

d ( 1 o 5 ) fellows

n

For a sequence of functions If (t) i we shall write Lim f (t) 1 & i n^oo n

:?( t ) if li^f

n

(t) s f(t) for every continuity point of f(t) w

Lemma 1 n 3 There exists a non-decreasing function g{a) such that

e

© function* g (e) are non decreasing and

n

eorem^ tiers exists a ^subsequence g (c

ni

hounded hence by Helly

J suoh that

6) f4S> sn

t

l °J *

« l °)

ere g(e) ie non-decreasing*

Let t b© a continuity point of g(o) 0 Fix q>0 and choose 6

positive and arbitrarily small and so that t + 5 and t-5 are conInuity points of g(o) and

g(t*3) - g(tj < jj , g(t) - «(t- 5) < 55 „

? sufficiently large and n we then have by Cl 5 )

«», <*&) ij> «a

(t) > g (t-5) - ij ”'4 ^

1 therefore

1-

JJ > g^tt) > g( t-'ft) - IJ e

y choice of 5 w© have

g(t) >2Xi > gn

lt) > g(t)

• ij san b$ made arbitrarily small for sufficiently large n.

Helly a theorem { seep for instance^ VoWidderp The Laplace Transform

2?) states 0 If the real non«4eoreasing functions qlv ( x ) and th8

Ltivs constant A are suoh that jcu(x)j< A in ^ 0, lf z9 . 0 «^a<x<b $

. a there exists a subsequence {o^ (x)} of {c^Cx)} and a non«=

r easing bounded function a(x) such that

Lim gl, lx) — a(x) (a ^ x <. bi

i-#QO a

j[ — **
.

,

.

.

/
6

It than follows that

Lia g ("t 5 = g(t)

Ol G

and l©mma l e 2 is pr oved c Wo now define a symbol x by the equations

?r, ooof>K)« - *, a ; - **y .y ,xy4 0 0 0 y

' ”\9 * • 09 w i«ls ** *1* • * eS

g(t 4-0)

To prove that sr is a random variable we have to show that F

^3^1* e osTja

is a iigtributioa function and that

Uob) •A «F"•? C « oFlm

'* ^1® O O O 9 ^ — ^ 2V V

(tpb«0 O oo*^_

jl m-x

and

(1 0 9 } i-~~. .lia -F . (t'gKp , 9a pt ) — {b,^b • 0

0

eS b ) e t ^ ^ w -jo9o«^ ^ 2 ®

That the interval function corresponding to FTM v is non-negativ©

o 0 <* a

is obvious since it is a limit of factions F~ - - with this

n

w x

0 * 0 *

in

property,, Wa therefor© have merely to show that (1 0 8) and (1 0 9) hold

and that F_„ _ tends to tiers if any one of its arguments tends

z!x c o ®ym

to ~co Q

ffe lave

a

( tg tf J

§ 0 o <? 0 ^ S r w

! ?v 4 S m

0 - ?x

n

t

i

. . .yj

*' b

i» • • • • V £ V,( V

M hence

0 < F.

o that

Um._ F -73

Furthermore from l 1*5) for arbitrarily small ^ and all t

y

i

4

(t,b,,...,b ^ 3

1 } ~ 0 ^ 5* C I* ^ x

^ v ~m

b.-*-ao xy. . J4 y_ - m

w xMy/. . . y* * * °* V • * * * *» *

'' r* TM "*y,...y t ', nim *V •••***) ^

4 m

’or sufficiently larg© n uniformly In t a ffe therefore have

lim F

5*°> «* Furthermore

t*^ °oo © © oy^

( ^ls 0009 *\> } ~ o

0 - l

y. . , .y,

*• V0 "’Vi1 - \j . . .7

ii 1 ** 1 ®

< 1 - Fw (fcj | ym

a

if we let first &-t»oo and then to ~$»oo we obtain (1 0 6 | m

To proto (1 0 9) we choose a continuity point © <t

F»y oooV (t„bA

...ba) »o IMS* that for fixed 5 asfi q

1 "a

%b& also

; y V

^ ® **’’'** * ° °

«>t' j ss ,-

: c AnJ, }

oo#,i

4 » ’W V ^ ^ 1 9 © o o g> 0=3

F

'n^i° 3 c ^m

t 5j> tp o o oS> F«- « ' .9 o o o 5

! p? 2S ®

° c ^ ^

^ - is

where 0 < e <, 1 and 0 < ©* < 1
.
8

t ; n ;; Ski a into i . 0) .va

9 O % y

nohri j

i

*1? b, p *J& A

e £

iO > 0 0 9 yJ-V €90

»

/ ie tf* vg h-s»QO and oonaiderlng that f may be ohosez arbitrarily

:aXX we obtain (1 0 9J 0

file proceed to pro that plim s

n«*> oo a

x, tfe represent

domain ]y®x| > e by th© sum of a denumerable number of

Intervals whose comer points are continuity points of F for

xnx

Ll a„ To do this we remember that there can be at moat a denumerable

number of points with -« positive probability in the' plane9 so

we can construct an interval netting 14 ^ W

’aioh avoids these

points^ Since the- set ( y~ x j > e is open0 every point of this set

1* Of n

; ined in one inte rval of this netting which H entirely con-

- ; - - u L'or •'< we have mere ly to tal s- Ll the inter vale,

of cur netting which lie entirely in the set Jy~ * 0 Thereby

' 9? *d :9. sue*; h • shat- i,f! 4.. 0 I are two sue* intervals and

_ Ixn-ivs lei ; - j: ftjiu v , r.i hi /ides t .a

a«- nto £3t-s:; ••. « is called a ue&h, The area of

& / these reo^a,- -lea 4s- called the gcdulue • >* f the mesh0 A

e {11^} of -3 .-has mo

h

that id. is a re inemeat of :ji. 2 ’.;'?h that the oduXi sa| converge to aero i- call ed a netting

.

then only Ij_ Is chosen for our interval covering*

Let Ij , Ig, B a 0 he these intervals and denote by P . (I^j

the probability that the point (xn

*y) will fall into the interior

either or

of the interval l

v or on A

its right etsd: upper boundary0 Then for

sufficiently large n and arbitrary ij

p( iwxJ>£ > s Fxn

xnth

lI *>

for all h c Furthermore

P(| Vxi > ) s |i>XnX

tlk

) Both sums converge* From now on consider n as fixed 0 Choose I

so that for some vj > 0

JjfVl

PV (Ik> < •

Next choose h so that for the first N intervals

! Uk) " ?xn

x

(I * ,! - f~*'* o Then

Pl l*n

“ x l > e

) s pV (I k> i f

P

xn

xn4h

(Ik> 2n< 3n .

t Since 2

|

was arbitrary ^n^P(|xa~x| > e).s 0 or plim^x^ax 0

[Th© relation glim^x^x also follows from the fact that the

characteristic function of x-xF^n+h 00nV®rg@8 to characteristlo function of xn

=»x ] o
II
On the other- hand if pllm than tor sufficiently

large n and arbitrary q

P( |x! •xOT | < £) > P( |x ~x| < £ and lx . < £ J n*h n ! w 9 — f n 1 - *x

1 n*h * — jt 1 #

> p< |xn

-x| < «j-p( |xn+h=.xj>

|

) > i-n o

Hence the seg,uenoe{xn

l converges and theorem 1 0 1 is proved 0

3 0 Stochastic processes ,

it set of random variables x^ where t

is chosen out of some set of real numbers is called a stochastic

process #

Lf the '•eet of indices t is an interval then the stochastic

process is said to depend on a continuous parameter 0 Such a process

is called continuous in [a*b] if for every sequence [hfo^tlth

11m h. « 0 • plio x. . s x, for a « t < b y 1«^00 A l -»0

0

** ~ —

The expression

^oo

f t dF«(t) s Ely}

®oo *

is called the mathematical expectation of y0 The expression

E{[2 •Elx)][y»E!yj]}g gxy

is called the covariance between x and y0 The covariance between

and will be denoted by cr^ ^ and called the oovariano©

1 3 1 3

function of the process 0

11

4* Convergence In the mean,, A sequence of random variables {x^}

Is said to converge In the mean to a random variable x in symbols

X o i o m0 xa s x if

n-^oo

'1,18) lim E(x--x)2 5 0-

shall now prove several very useful lemmas on oonvergena®

in the mean and convergence in probability,,

Lanuaa 1„3. If thsa gUm x

n

sx „

This follows Immediately from Tohabicheff "s inequality,,

Lemma 1,4. If l„i 0 m„ x.sx then lim„ E(x„) s E(xj 0

n«^oo ^ n~>oo u

Sine® E(x *»xj 2 < e for sufficiently large a we also have

e > E(x -x) 2 sa| . x

*(E(xn

-x)] 2 > [E(xn

)-E(x) ]2 0

ju n

Lemma 1 B 5 Q If{y^ is & sequence of non~neg&tiv©^ D -' random variables

and if glim yh g y and E(yh

) <M then E(y) < M 0

Under the conditions of the lemma end in view of theorem 1 0 1 w©

have F^it ) s Fit

)

where and F are the cumulative distribution,

functtonsof and y respectively* Suppose E(y} > M then there

exists a continuity point A of F(t) such that ft dF(t) > M * A . A A

' 0

However f/dF^( t) < * and 31m

]

%

tdFj1 (t) s ftdF(t) B a contra^

0 a 0*

dlotion rt

[5] 1„9„» if £(yh< OJ S 0 o

12

Lemma_i^ 6 0 The sequence {x^} conr&t gas in the me&n to a random variable x if and only if to every £ > 0 there exists an M such that

U0 lS) Etx^-x^}2 < e for all mp n > IS 0 Suppose first that there exist a a random variable z such that

X^i.mo xn %

x

c Then for sufficiently large n and m and arbitrary £

n*<*w

B(xa

-xi 2 < e j, E(xm

-x ) 2 < e 0

But

E(zm“X }

2 Elxn

»x )2 - 2E[(x

ffl-x)(xn

-x)]

and by Schwarts's inequality

f E{xm~x)Un-xj J <^/B(xm*x)S e(xh

«x )2 < e *

Rena a ^^nTxn^ .« the other hand from E(xm^xn

j- < £ it follows

by Tchabioheff's inequality that

±*l \x *xw | > tcft ) < 1 1 a 23

1

- ' ~ T2

Thus plim. x„ sz exists by theorem l 0 l e It follows also that

2£«f> GO ** plim (x; m

*x ) s • (x «x ) 2 and thus bj? lemma 1 0 5

h-**^cd a u S^-*) S < e “4 i'lS* *»* X «

lemma l .? 0 If l 0 l 0 m 0 x_sx »• l^i.ffic y„sy and If

n«^a> n^co

E(x2

) j, E(y2| exist then lias S^.yJ s ^y)„ n ** n***© **

' h rtapoot to show first that £(x'

r

} , E(y^) z.ie bounded

n n

By virtue of lemma l e 6 there exists an m and an M such that

2 ^ 3

‘- !a *x

n

) 2

. 5 14 for aii & From the almost trivial inequaliy i 2[la-b) 2*b £

] we see that for all m

E( *q) £ StEiXa-X^) 2 E(x 2

)] < 2[M * E(x2

)]

. rom this inequality it follows that E(x2

} is bounded for all m

nos

' z(xayn) | < <^{x|}s^y2) it follows moreover that E(xy| exists

nd fur hermore

ECx^y^^xy ) | ® E ^n * ) * yi*£»x)]j

<. 7E(x|)E(y^y;2 * ./E(.y2)E(xn

«x}2

: iy"; exists by lemma 1,5 and Efr2

) i S bounded and since E(yn

»y)2 e

£ *^2

1

converge to z s ’o8 the right-hand side of (I„l4| converges

vi ioh prc?v6'$ ler;jia 5l 0 ^ 0 ^emma 1 0 7 may also b<g written in

he form

; X 0 15J E{U 0 i 0 m0 xn )U 0 i 0 m0 yn f}r= He EU„yJ

xn )U,i,«0 ya J} S Urn a.

n—* co n-^^> n—»co '«*n

Corollary to lemma 1„7„ She sequence {x£ } converges In the mean

ano o :ly if li* B(x^) exists irrespective of the manner in

ioh n 4nd :a tens to infinity 0

- ] Thig ie seen if we determine N -- aooordin^ to leama l c 6 —, so

£ ^hat &{ < e for m,n> H and then take, for a fixed m>BP a * aax{E(xm

-x1

)2, E(xk

.x

2

) 2

, ...„ E(x

ffi-xK

) 2 . e}
I

I as &•**•**»* — a - J * -At ** «<««“ »* * * ( **-«* '
14

From lemma X 0 7 it follows immediately that th© condition is

necessary* I'o show that th© condition is also sufficient we aseuma

that lim E(xnx } exists and is independent of th© maim©? in

<=£*&• ^ *** m-f>oo

whioh n and m go to infinity* Then lim E(v3L) 2 :0 fetnce it is

a—

®

ffi-^03

possible to find for ©very & > 0 an fi§K(ej such that

E(xa

«»x )

2 < s for ng, m > I „ «Ve see therefore from lemma X e s that

X 0 i*m 0 exists* This corollary is due to m 0 Loev@ 0 n—*© - u

5 C Differentiation ,, In order to be able to define derivatives of

stochastic processes w© have to extend the concepts of limits in

probability and limits in th@ mean* Suppose that for @v®ry h in an

interval (a#b') a random variable is defined,. If for every . sequence

{hfl with lim h, s a 9 plim xhjt §i exists then we writ® 1 %l

i.-*oo i 1—>» hi

glim s x 0 In a similar manner we define x..

v g| 0 i^m 0 xj^

The process is called differentiable at the point t if

glim xt»fr oS:

t z exist®* Th© stochastic process is

called the derivative ©f x^ 0

In the following we assume E(x^)sO a Th@ modifications ©f

©ur statements for th® case s(x^) ^ 0 will be obvious 0

15

6 0 Stochastic processes of second or d er n A stochastic process x^

is called of second order if for any values t

7 ^t^ the covariance

ctlt exists. The process x., ia called differentiable l.i.m. if

l.i.m. xt*h”xt __ x

* exists.

h~*» 0 S“ *

t

Theorem 1.2 . fleoessary and. sufficient that the proeesg x ia

t

differentiable l.i.m. is that the limit

U0 16J lim t-T - Z*

k-£o 1

exist 0 The covariance function at t

is then twioe different! *- 3S\t. sdjl® ana

3St . x 3 1 B Moreover x! is a stochastic process

Stl

3t

2

3"

2

3tl ?

e ‘'°t 1

t 2 of second ord©r and its covariance function is • ^ 2

3^t*t ° Th® cov&rianc© between x^ and x^ is given by —“pr

Proof: Consider a sequence of difference quotients c

W© have

gt*h.t*k~ g

t+h.t ~ g

t . t»k** gt.t

hK

and by the corollary to lemma 1.7 the relation (1.16) is necessary

and sufficient for l.i.m. ^t+h s

t _ to exist. £^3 exh-*- 0 h

pression Z^ t

in (1.16) is called the generalized second, derivative.

16

we moreover hav® by lemma 1 0 4 E(x^j~0 since Etx^J ^ Elx^j

s 0 0 Furthermore by lemma 1 # 7 cr~t**texists and

1„17 1 a*. =• 11m E lx* * ^t^h^tl _ lim at*lu t*“ att* _

3<ytt 1 x

t*x

t h^O [* —Yj* h"»0 g

' - ~WT'

Thus gott* exists A It also follows from lemma l n 7 that *

• exists and xtxt*

a,-?-* sr lim E[xt*e*h~ xt « xt*$*k aXt*

l ** fct8 —e—

= ^

qt«h.t**k~ qt»t*«k~ qt»ha t**gt.t* _. Z^*

It easily follows that c^.* is twio© differentiable and that

S^tt*

IFsT"

&

2

gtt*

3F"W"

©

It is well known that the generalized second derivative of any

function f(x,y) exists if i'f exists and is continuous,,

ax ay

Thus we have

Corollary to theorem l n 2 0 If x^ is a stochastic process of second

order with covariance function a*** and if 3 G

u %% exists and is

at at

*

continuous^ act than x-£ exists l e i.a, and its covariance

function is ^gtt* _ 2, . * e

The oor Hilary to theoi I c 2 giv<*e a coiv anient raebl >1 to

:i ki t 'O variance : u* Lo> oi x! il x* exists 1„: mc

\

Int gration Q Let x^ be a stochastic process defined for

< t < b , tie subdivide the interval from a to b into n parts

y means of the points a «: t )9 t^, 09(,,tn ~ b aad put

ax(t ) - 8 0 The number 6 is called the modulus of the

subdivision* within every interval t. , < t < t we choose a

'aluo t

‘ and fora the sum

x

i

7

^i-1

)

is a random variable,, flow consider a sequence {SM} of subt vie ions S ra with moduli such that lim --t 0 c l*et

m m m *>& m

*& be the random variable oorraspon&ir:$ by {1 0 18} to the subivlsio i and some she ice of ;he t a , £ then

lim X.I rnj ~ X exists and is equal for all sequence {S’* with

ol-*q&

modulus converging to Zi&o and all choices of t* th a X is called

he Integral of x.v and we write

18

[7 ]

Strong contlnnl ty 0 In th e

the event that the relatione

following we denote by £(89 e,S)

l xtr x < £ are simultaneously

satisfied for all pairs with | < 5

to a finite set S of points contained in [e=>b1

is then the probability that the inequalities ]x^

and belonging

o £[<£ ]

are simultaneously fulfilled for all pairs (t^t, | of a finite set

* TM 4SU S of points for which Jtj-tjJ < $ 0

The process is called strongly continuous in an interval

[a*b] if to every £ and- i) tfcere exists a S-Slt^ri? aash that

for every finite set S of points contained In [a^b]

Uc20J P [g(5a e0 sq > ln c

For any stochastic process consider a set S =*(t^f 0 0 0 *t )

where a < t^ < b ( i = I^^n) 0 We denote by M^g the largest cf the

values 000 „x^ 0 Let {S^} be a sequence of subdivisions of

the interval [a9 b] whose moduli converge to £©ro 0 If

?y&M&bs, - ^ab @xlsts and is th® same for all sequences {S^}

X =4- itO J ^ whose moduli converge to zero then we shall call K;1> the maximum

of in [a^b] 0 The minimum m is similarly defined.

concept

[7] The delimittea is due to Pc levy 0

19

To simplify the notation we also introduce ?

ab - M

afo~

ffi ab

aEd Y&bS =? 0 tie next derive a criterion for the strong

continuity of a process

s

Theorem 1 0 3 0 a process xt Is strongly continuous in [a9 h] if and

only if

(l) it possesses a maximum and a minimum in

every subint erval [t*t‘"] of [a 9 hj f

(ii) for every £ > 0 9 rj > 0 there exists & d such that

for every subdivision S ~ (a « t ^ 0 e G ® t^s b $ with

modulus less than o

^

f h ts i~rue t’h^t

do El) £

jtjp* ^ 38

« • ••f 11 ) 5^ ^*”*1 o

tfe emphasize that (l^El) means that the probability of the simultaneous fulfillment of all the inequalities t < s !is.L, 30e? nj

* 4 „1 ^ -S*”* . , [&3 " " ’ must exceed 1 -*j 0

[83 If H

;i (lul9 29 0 6 0 »nj are n events then P(R^t l9eooj>a)

means the probability that all n evants occur simultaneously c Thus PCRj^tsl* e 0&9 n) > k means that the probability of the

simultaneous occurrence of all n events exceeds kg this should

ba carefully distinguished from the statement P(Rj

)

> k9

( t~X 3 0 0 0? n)g, which means that the probability of the occurrence

% of each single event H.s exceeds k which does not imply any

thing about their joint occurrence^ We further emphasis©

that in conditional probabilities the condition is separated

not by a semi col on but by a vertical bur „

20

The proof of lemma i a 8 is left to the reader 0

Proof of theorem l„3 a Suppose first that conditions { i j and (iij

are fulfilled^ Let S he a finite set of points 0 vVe can consider

sequences of subdivisions {Sa } such that each S„ contains S 0

It follows then from lemma l e 8 that

{Sa}of subdivisions^,,., 0<>o8 ta ) with t,, s a +MMI „ for auffioifatly largs a ana arbitrary e,jj we ha’/a by (lij

Ko® let t

(1„22J

( 1*235

The relations

t < s , isl»2» .. „»n Imply the relations

'9 o o o 8 n-1 0 Hence from (l 0 £3J and lemma

I«6 it fellows that

t — i-lf E

i-lid 9 © o © S n-l) > 1-ij o

?x

Any two points tj9 t' with 1 j < i He together in one

Qy^Ji (i'2Z)

of the intervale (t, „ Henae the above inequal ityA

tmplliUL

1

1

0 24 ) 2 [g (|9 £ 9 S)J > l- n „

Ehus (i) and (11) imply strong continuity*

>v© next show that the condition is necessary,, We assume

that x, is strongly continuous and let a < £ < H < b 0 Wo consider

t

two subdivisions S„ and Sm of both of modulus less than 5

tei4, S consist °f fhc f>e.»i-s o.nd Sm - and the maxima and C A

*' " relation

|M_, - U'„ | > e Implies that for «Qr two points t 9 t

f of 3

abSa

a5Sm

'

we must have J > £ * jt—

t

\ 5 h^nce ty (

1

0 ^ : l

* o

sufficiently small 5 and arbitrary

(1„25) P M%5s “ M

5S3 l >£ !^

n m

On account of theorem 1 0 1 the relation (1 0 25) implies that

plim Jd- rQ ss Mgg exists* In a similar manner it is shown that

h=$»cj &

7i\ m, exists so that condition (i) of theorem X c 3 is satisfied,,

Hoy? choose 5 so that for every finite set S of pointis

t

i$ 0 oo»tn

and arbitrary e9 q

1 « 26 ^ } 1 ^

How let Ss {a^tof ooopt^s b} be any subdivision of modulus leas

than 5, {S^} a sequenoe of subdivisions with moduli conversing to zero , and containing the points of S„ The relation (1 0 B6) implies

( 1oB7 ) 1*1 tS *5 » ^ eoo»n) > 1*^^

*1-1 in 2

How ohoose 9

* so that < S* < * and so that e* is a continuity point of

the distributions of V . is l8 a# 000 ,n0 It then follows from ll 0 27)

ft

-h i § » isl^ s® 0 0 0& n j

- > 1 — ij o *1-1 u i

-

This completes the proof of theorem l 0 3 n ShooreF. 1 q 4 0 t>et x be a strongly oontinuous proaess,, than

IS X. -

21 x* =

> x^dx exists for every t

dXj

•t ar*

Proof” By theorem 1.3 exist for all pairs Cj'

and we have for every choice of points axt „t ,t . t st

0 1 2 • «

and h-l ^ **?.

(1 0 28J 2m,

To understand this inequality correctly w© must remember that

^ ^ (i»l#2f 00 ©«ni random variables and

" rv? •' joint distribution is such that th© inequality (1 6 28)

holds with probability one c
.

ustufo

N

31

I

-

I *

I V

*>

. / 1

* 9

’

•

.

•

.

•

’

.
83

Since the process is strongly continuous we have for any

subdivision with sufficiently small modulus 6

P K, ‘VW 5 Mb" a)

]

* 1 = s

> • L 1-1 i I®1 1 ^

If S is a subdivision then we call YCSjs^M^

^ the

1

ct 1 i

upper sum and y(S)s £m It -t ) the lowsr sum corresponding

** ij t v - A A A

i®JL 1

to the subdivision S 0 rte consider now a g?®%uene© of subdivisions

{S.} with moduli {5., } such that

« 3

11m 5.s0 and

$~4*x> 3

If Y.^YCS^ji and ^sy(Sj)

SC8 if m < n „ m n

are the corresponding upper But lower

sums then

and hence for sufficiently largo n

P

L

0 - yn*k'yn - s(b"a 0 £ lo>J

l °

Hence the sequences of random variables {/.. } and {Yn j converge

and plim y s plim Y D From hare on the proof of the existence

h-^s© "o n~»*co n

t

of X^sfx^dir is precisely the same as that of the existence of

a

! " the ordinary Riemann integral of a continuous function,.

24

It follows also from 11*38} th

tf

(‘ tU ) > J z Yi

Consider now th© quotient

Xti-

<S£» X1}* h 2

b l "’"irr^Hrr

** 3 1

tg

f xt

fi t1 2

nV © ha¥@

xt - xt, m . , <

t; E 1 {* «$= S1 < M

“18

z

m H

t-j t > 2

aM

m.. . <•£, < M. . * 1*2 “ H

" * 1*2

and sine® th@ procst;, i

f3 strongly continuous-'

• <T: &* s?,

v Mp “% t . Henoo pH® m. , s plim M, , 4: aiaafe^ »i »/w ifcesa^jj «*8 4; 2 ,1 ““S’ Tt , w p j -p

e& Cj <t-. <*i .Ci 1

and thus

plim

’• r

*h-

«* ^t

' A4t C« * j •

Tills cempletea the proof of theorem i-b 4 « $© shall now consider siochastie proaosses of secr.aci ordCT.

.. i

X. ^ f x.dt exists

if the Hi ©mann sums 2x. w (t 4 «*t

C V

5 i

4^*1

• on^r ©3 in th© m®an0

,

25

Theorem 1 0 5 0

The process

Let x

t

"be a process with ©©variance funotion 0^ f

X 2

is integr&ble X 0 i 0 m 0 in [a^b 3 if mid only if for any

o

t t

t in [a^b], f J t dtj&tg exists. The covariance funotion

a a

w

,l * 2

t

2; t t

of X

t = J[

x^dt is given by

.1.89) t l

t

2 ”

sf

^1^2

0. 4'sidT

2

The process X^ is differentiable X 0 i«m 0 and X^ssx^

continuous 0

To prove theorem 1 0 5 we apply the corollary to

if 0a i is

*1*8

| lemma 1 0 ? to

a sequence of Kiemann euas {2 } with moduli going to

w© have

e‘v«* s EK?C1vwzx

tt

( yvi) ]

J *f5« yw{1Wi

If 21 and m go to infinity in any manner we have

t t

lim E(J J )= f T of * dt dt

n~& QO n m a a

• *1^0 1 1

Thus X 0 i 0 m 0 2 « 2 exists 0

n-^oo 11 Moreover if {2^} is any other sequence of Kiemam sum© we

put 2^s 2a

0 2n0 Sinoe 2” ©xists we must have

2 s X 0 i 0 ® 0 2 n

TM l,i c ®o 2 = J

x at ^ X

n~»® n n->oo n 5

26

It follows also easily from lemma I,? that J5(x^X

+ /

}

exists and

that

(lo2S&) E «xtV >=

* a & id

w© further have

Y . '•Xa.

**. <0 »* Xt « ?Vx

t 4, , t

t

v ~Fn

From Jl 0 29a) it foil ova that

.t' t'

*t»y x*Trrajf&V"°t t'

0

t T *°tt

^'c

'

If a , is continuous then by the mean value theorem of in

tegral calculus ov v becomes arbitrarily small if t* approaches

t 0 Thus x^ss X

t

" L0 l ouo

*

Let x^ p y^ b© two ft = than to every a

division S s Ca^t^t^g t^ 00 0 *ta

«b) we can form Riamann-Stislt Job

sums

n

ilS)» rx- (y* -

y

t 1*1 *t i i-l

(lo30J

27

If nov* for every sequence {S } of subdivisions v;ith modulfe

converging to zero piim xis )' exists and is Independent of the

n-^oo

particular sequence {S Q } and of the choice of points t

(t^^ < t* < tjJ then we shall write

X=s plim I(S j - S xt d^- n-* oo n

a

w

*Ve shall call X the integral of x^. with respect to y+ c If the

random variables XjSn J converge in the mean to a we shall say

that k^ x^dy. exists l 0 i 0 m.

* t

Theorem 1^ 6^ l«et x.. 9 y^ be two independent stochastic processes

of second order (that is to say is independent /

g and any t) with covariance functions * Pttf *®speotively„

The integral of x^ with respect to y^ exists 1 ;i aa foi

terval ra*t] contained in [a;b] if and only if

( 1 . 31 )

t t

JJ

exist

s

0 The covariance function of

x

t. - S x

t

4

^t

a

is moreover given by

h t t J j o <¥*

*1*2 a a t

x t

2

o

The proof of theorem 1,6 is analogous to that (

1,5 and is left to the I'e&dsr,

By P(S|E) we denote the conditional probability that E '.'fill

r o i 4? happen provided E has happened, 1 ’ 7

.ia ha?j called a

tiauoua if dim x

’S-&Q - X 4* o A process will be celled uniformly

continuous in [ag b] if to every e > 0 g q >0 there a .tie

5? £<,ii | of t 3\ii3 h. tnsi ti

$e a

pl! ztn xtl & s )^ 1 ’ 1

l every |t] < o( e v s| J

Lemma l 0 9 e if a process is continuous in a doted in.ervd

[&£>&] then it is uniformly continuous in ; &s b'j

Troo^t Consider a monotone decreasing sequence

dim t4 »0 c For every i consi&sr the set of points for ?hieh

p( l xu< xtl > *1 >*«

c&

Assume the lemma to he false* then we can construct i aegu^uoo

ti<,tg9o ,, such that /e/- ;<,.«« f'><>, 'rj>o

P(|xt^ T

"x

tJ

>;?£ >

for some % <. t* * Let t he an accumulation point of & e ecue-nee

5 >o

{t.} c Then for every «

w

-a—ft we can find a

to t such that

| > Zt) > £*} for some j-cj <> J|

1 * i

[ 3 ] ^ 02? vile concept of conditional probability the read a 7 Is

referred to Kolmogoroffg Grundbegriffe dsr Wahu 5 chain ~ lio hkai t sreefenung, Chapter 5, par, ; end 3.

29

Choo> e now |t.~tj < and t

^ 30 clos to t that

P( >*}<!),

S Then

?( I^ + T-Xtl > Z PM*t . ?***.! > ge * !*

tl- xtl i e 2

> p( 5-3 >*}>*» . &

Honoc for arbitrary &>Q and some e>0a q> 0 there exist values

£ < h such that

p( l x

t*T* x

tt

> e) > n

in contradiction with our assumption of continuity of the process x^ c

rie derive next a sufficient condition for the existence of

M h

f nd hare to aon«lde** in this connection the event

{* <a X

i f0r k < * 9 x

fc < Xi

f0r k > Rnd W® denote this

by A and state the following

Theorem l n 7 Q Let Ss{a < t i < t £S>oooy < b} b© a sat of points

in [a^b] and let x

t

be a stochastic process which

(ij is continuous in [ay b]$

{iij is such that for sufficiently small t —which is independent of 3 — 0

*

(1 Q 3£) > *l A

i>

*

< xi^nr X

t| : > "

>!

where K is a constant independent of the choice of S-Q Then and

m

afe

exist 0
I

’

V

'
30

Proofs Let SiS (t1# . 0#f t

n ) and S =

(

0 . c

, l£) be two subdivisions

of module less than 6 and put for eh. or • ^ = o q

To every we can find an Xj s y^

a

such that t

J- t. < 6

For P(A^J 4^ Q we thus have cm account of lemma 1*9 for arbitrary

ij and sufficiently small 5

(1,33) P(Ak

,z

lc-yk > e)»P(Ak

)P(xk-yk > eUfc ) < ,

If P(A.j'sO the inequality (1 0 33) i3 also valid*.

Denote by 3^ the Joint occurrence of the evenia A^ and

xk~^k < 2 an& 1*^ B be the event that at least one of

the ©vents Bv occurs* By definition .A Implies

& K.

x. *® M , _ so that B implies the existence of soma y, K aoo^ “ jf

such that y, > M . 0 - £ 0 From this it follows in turn that

i£ “» aob|

M. > M, ~ s abSj £ ~ ° Therefor© we see that

•^afcSi •- MahS|““ £ J £ p ^ B i S °

From ( 1 0 33 ) obtain easily

P(Bk ) = P{Ak

)P(xk-yk < e|Ak ) > (I-K^) P{Ak

) o

The ©vents exclude each other an£ exhaust all the- poesi

bilitiee so that by adding these Inequalities we obtain

Therefor©,

(1.34)

2P(BV ) > X-Kir,

k

*'

f ^ MabSj MabS^~ ® | ^ 1 o

Similarly w© obtain

Cl 0 34a'j ^(“abSj £

M

abS,~ E

^ £ 1-1 ^ •

hence

U.35) P M\bS1 " MabS J i

e > > 1

‘' 2K’}°

The existence of M

afe

follows easily from (l c 55) using i heoram 1 , and th© existence of is proved similarly 3

CHAPTER 2

SPECIAL PROCESSES

/I A email partial© suspended' in a gas is subjected to a continual bombardment by the molecules of this gas* The individual impacts

imparted by these molecules are small compared to the mass of th©

and the n.imber of impacts per second is very large* The impacts are received from all directions and are randomly distri—

t buted* Moreover , if w® neglect the velocity of th© particle : itself* which is small compared to th© velocity of the molecules*

th© distribution of these impacts at time t will be independent

of th© momentum of th© particle at time t*< t 0 If we denote by

the momentum of th© particle* it will therefore be reasonable

to assume that x^ is independent of x^# for t* < t 0 The

motion of the particle is called the Brownian motion*

The momentum of th© particle is a special example of a more

general type of stochastic processes, called Markoff processes,

which satisfy for t^ < t

g .

a < t

n < t and i > 0 the equation

P$Xt+T ^ Al Z

t1

»»«*» X

l! » Xt)- ptxt* T i A l Xt> •

In words, the conditional distribution of (where ? > OJ,

given the values of , # 0 0 ,x

t

,x^ (where t^ < t

g

<oe# < t

n

< t)

is the same as the conditional distribution of 6i ven 2t

©
-
More conversationally speaking* if the present value of x*

is known the distribution of any future valuer ie independent of

the way in wbioh the present value was reached 0 In our special case of the motion of a particle suspended

in a gas wa shall make the following assumptions about its momentum o Assumption X c

r-uMXK^a*

where e* ^ 1® a random variable with moan sera and is indepen*

t jT

dent of li, and also of s * * if the interval a (t^t+xK

i* w jV

(s„l)

Assumption 2 o The distribution of e*. depends only on

o

and hence

From (2 0 2J and aseumption 3 it follows by a well known

£X01 , theorem taat

( 2 . 3 ) C

2 ~ 0%

T

where o 1 a some positive constant 0 Thus x^ converges to x^

in t*h& mean with decreasing t and the process is continuous

l 0 i.0m0 ^ Suppose further that x

0~ 0 c It follows then from

(2 0 3) and assumption 1 that

t <£,- •*

(2.4) i

l

°x

t

x

t

^= °K!xtn*Vxtl) s ^

s ot -

The assumptions 1 to 3 define fin important class of Markoff prorii » di

ceases* sometimes called differential processes* 1 J "in order

to define completely our mathematical model for tha Brownian

notion we must also take account of the fact that we regard the

impacts from the molecul.es as coming in a continuous stream so that

large changes of the momentum in a short time interval became

much less likely than small ones c

[10] If a measurable function f(x) satisfies the functional

aquation f(x+y)s f(x) + f(yj then f(xj=f 0x c Proof of this

theorem may be found in H 0 Hah

n

f Theorie hex* reellen Funktionen* Frster Ban&9 pp c s&l-Sg J 0 Spring&r* Berlin (192lJ c

[11] we distinguish differential process from "general differential processes 71 (Chapter 4) 0

o 5

W® therefore impose th£ following additional condition*

called tbs Lindebarg -condition* on the distribution functions

*

Tu> ot

Aaavaaptlon 4 (Lindabacs oonditloaj g

Post sufficiently small X and arbitrary P > 0 „ ij > 0

12.5) f a

2 dP (.a) <. no

2

v T

s 4 T

las >p

Condition *{2 0 5) zaay perhaps bast be understood if we discuss an

important case where it is fulfilled G

oo e. -Theorem 2„1„ If o

2 = f

^d? (a) exists and if P{_^5 < a)

£ -00 *

-s Flaj is independent of % then fulfills the Lin&eoerg

condition (£ 0 5J 0

ffo^'Tvom the conditions of the theorem vie have

F

T

laa

T)=*F{a) » Fjaj ~ Fia/ej 0

Hence for arbitrarily small p > 0 and ^ > 0

? a"' arT(a) * $

** dF(a/ffT

) = a

2

f &

g

dF(JL)

!&!>/> W>f> ltl>js* x

s a

2

j* y

2 dF(y) < D,a

2

>1I >/>/«*•

v

z for sufficiently email » since CsOi «

36

'M next prov©

Theorem 2 0 B o If e*. • fulfills the Lind©berg condition then

.mi i.ri^. af i i |JU <W|| (i «0 %

uaJ^L’

is normally distributed with variance q % 0 as long as we consider only the distribution of there

will not b@ any danger of confusion If we writ® e for e t 0

nil a shall uae the following

Lemma 2 p jLs Let XgPooo£> b© independent random variables witii^

2 distribution functions F^ oco? respectively and a^ri4, )^i

then to every 8 there exists a p and a*s^ such that

a

2^o* . < a % «- ifl c^< a)

a>

@~X ^

dxf < 5

whenev©r for all k

jxg dPk

(x) < Jjof , c

. >P &

A proof of this lemma can be found for instance in Khintchine® ®

KAsymptoti@eh@ Gesets© her ^ahrecheinli©hkeitsreohnung f! £ p 6 S

(Ergebniss.e der Juathematik* J Q Springer^ Berlln 1933) 0.

To prove theorem Z Q Z we put s r /,/cv j than c^, has variance

« 1* «*• divide the interval 0 < t < % into n equal parts and put

& f0* l “lj29 ® ® » 9 n

1

6

B?

The £*. are independently distributed and ei s= e,*60+ *s

“ in0 ® ta5 l

i fulfill the iindeberg condition we sea from x®m&a z t \

to8:-t the distribution of differs arbitrarily little from the normal distribution with unit varianoi®. Heno® e^« & e' i 3 normally

distributed wife variance wo^ai ot *

• * "*

. ’

, r

• -kQ processes defined by assu/nptioiia I p 2 # 3, &&& 4 will be

sailed fundamental random prooesses (abbreviated F E p \ e

b 0 ' & g -o

' in th® following we shall repeatedly use the fact that the

ul-isrioufciQii of the limit of a sequence of random variables equals

s.

** *u »*' •“«»*»*

wary y.a.P. is Strongly continuous.

With0ilt 1088 of Senerality we .ifcaU a«a©» o * ij that is,

'*»< **

\

d9rlVi n8it aowral lemas needed for the

prhcf of theorem 2.3.

2|Jr a«, For a > Q w© have

. m P

fy. K

3 a

IS) jV*;/2 dx < .-*“/*/.

a'

TO ’

-=*roQf :

* ;• 6 ,|»"A /"'

a

a

d:s < jx©"2* dx S ©

- /

“k

an of length d and for S

o*. points in this interval

a

p(?

tt'S > M> < (8e*M“/8 V4^F

o8

aM the points t and t' to S, This can only increase the value

of ^t/ 3

s %S',ffl U'3 c »^taout loss of generality we may further

assume x^—Q 9 since we could otherwise consider the process —

x

s

-x

t , Let t 0 »t, t

1

, tg, t

n

* t

# be the subdivision points

of S and put x.j =x

t

c Let k

% {Uls 2foco9 n} be the event

{x^ < Mj* COC f>x ic3,j < MgX^. > M) * The events are exclusive and exhaust all oases for which > M f

«Ve have with iQ s 0 0 M > 0

i

> iA

i#)

X

2a > Mj^P(A

i

) PU

n

> M|A

t

)srP(A

1

) > M-^|A

i

5

> P( A^ ) P(x£-x^ > 0 |x, «. M ? 0 0 0() xH «, Me x

t > M| g our

on account a£^ assumptions we know that x^-x^ is independently distributed of i1 Coool) X| and has a normal distribution with E#ro mean

and variance fc »fc. so that

n x

CQ

i>ui» x

n i M> t *l v j

®SE- (x-x,

II

0

a it -T.

n i

]

d{x

n i

33 1 r ( Ai ji ^ 8

The event© {A^Xg > M} comprise all case© for which xa ^

M and ar®

mutually exclusive Adding C

S

0 B

)

ov@r all i we therefor© obtain

{2 0 S) SPjxn > M| > P(Mtt , s > Mj o

She left aide is by (2 t 6J smaller than

(2 0 1Q) Z28

T'ha same estira&t© is obtained, also for < ~Mj 0

Furthermore

i’U'tVg £ MJ - P{elthaf MU'S * 1

°r “tt'S

g

< swu* 3 > |j *P(«

tt / s

< -|j < iajT/^Tw)f& /8S

9

whioh establishes (2 0 7’)

c Lemma 2 Q 4 0 Let t and consider subdivisions { tst^t^* 0 0 <i9

t' J such that t^-t^^^= £/n » 6n c Then there exists for tv^rj

positive e*^ an H such that for an arbitraryV rr 8 of points

3 0 11

)

P(V

%

1 -1

* 1 *

< £$ i~l*•$ o o o £ nj > 1 -ij for a > 2?

To prove (2 0 11) we add the points of to 3» This will at

meet decrease the probability in (

2

0 11 > ^ Since the distribution

of V* ^ a ia independent of the distribution of that of

*ig. for j we have by lemma 2 0 3

J

** J ^

U0 12J

40

where k and k* are^ constants Independent of n e Sine© it is

~nk f n

easily seen that lim (1-ke ) s 1» lesnna 2 0 4 follows*

n^oo

Lenma 2 C 5. and m^9 exist for every internal [t9 t*j « Consider a sequence of subdivisions^^} of the internal

[t 9 t

5 ] of aodul« { 8 } with lim 5 = 0« If S. and s a have both

sufficiently small molul^ then in every interval of length

8

Q

of lemma 2®4 there will be at least on© point of and one of

3 . • Hence applying lemma 2 C 4 to the union of S and S we obtain

« 1 ,1

Thus

pMmu' s^^t'

s

I > 6) 4 J, , V

exists for every sequence {Sn } whose

moduli converge to zero and is the same for ©very such s©qu®no@ 0

In a similar manner it is possible to prove that plim ie. 1-^cpo

”

s

i

exists 0

To prove theorem 2 0 3 let S ba any subdivision of modulus

< 6/E and consider

(2 0 l3) t S

eg D 0 ®nj c *i~l i

tie form a new subdivision with 6/E < < 6 by deleting points

of S c The probability (2 c l3) for this new subdivision is smaller

than that for S 0
- . * - / . .
41

By lemma w© have*, alnoe the distribution of V, , . converges

c i-l c i &

to the distribution of 0

C

3

0 14 j P|

^ ^ ^ i«X* > [1”8 exp( -v&"/8 5* ]

Ei/5

HI cT/c?ir

The limit on the right of (2 0 X4) is 1 for 6-^0 , which proves theorem

p

'* *'»0

Theorem 2 „4„ For the F C R 0 P-C

(2,15) P(M

at)

-x

a > M)= 2P(x0

~x

a > M)

we consider the proof of lemma 2 0 3 and write t s b p t s

a

we see from (2„9) that 2P(xb > M) > P(M

8feS __ > M) c Here S* may

i

bs any' set of points in the Interval [&9 b] c, Lot S| be an ©lament

of a sequence of subdivisions {S*} whose moduli go to A®<~

cording to lemma 2 0 s plia M - g

M - exist b p henoe

(2„I6J 2P(rb > H) > P(Ma% > U)

W© consider again the events A^ of lemma 2 0 3 for a subdivision

of [af b} into equal parts 0 We then have far ; ^ ao, with a > li cs and with x& - 0

42

(2 0 17) JPUj,»X0 > M) = > 0) vP(A4 ,M < x

g

< Xj)

= t ) HAj*M < X0 < X|

}

< Aj I 4Pi Aj,X£_^ < Xg < X| J c

Let e be an arbitrary positive number# then

U0 A3j p ^i»xi-i < x

0

< x

i) sP{ H* xrx

i-i i

e » x

i-i < x

0 < x

i)

PUi**rxi-i > e » x

i-i < x

o

< *i>

< J’UpQ < x

t

-xc. < ej Pjxj-x^ > ej

(JS

< P(A

jL ^£b

-x

ct < e) P(xi

-x

i-]L

> S)

since X|-x

Q

has larger variance than Xj«x

Q

and both ar® normally

distributed with mean zero* Adding the inequalities (2 0 17) and using

i

&

0 18 j w® obtain

C2J19) P{x

a > Ma Mat > M) < |PlMab > M) <-P«0 < X

b-XQ < t)

ZPlx^-x^ > e) 0

From C2 0 ?} w® see easily that ZPCx^-x^^ > eonvsrges to zero

« for ©very positive s and every sequence {8,rJ whose module converges

to z®ro 0 Since e was arbitrary we have

C 2 o 80 | P(x

c > Ms tlab > U} <. |p(Mab > MJ o

43

Since c maybe chosen arbitrarily close to it follows from (2 o 20

that

(2,21) 2P(xb > MJ < P{Mab > M) .

The inequalities {2„21) and (2,16) together imply theorem 2„4 0

Corollary to theorem 2„4. „ Lot S, p 3 £0 0 „ 0 b® a sequence of subdivisions of the interval (t g t+x) with modulw converging to zero and

consider for each S

fi ; {t®t, s the probability

Pn* P^

xt“ x

t - °»*’«»x

t“

x

t -

then X&m P sO 0

n-froo a

Yroc^% a 9 nave

n

lim P„ - P(Mt>t+T-x

t = 0)sH!Kt(ttfx

t > 0)= > 0,- 0

'Theorem 2.5, If /> > 1/2 » plim (x. -ij// doss net exist.

-- ' * ~~ —> o

** % *

It is aaro if p < 1/2 t Th® proof of this theorem is left to the reader©

Pet [ Is

j

be any interval and arbitrary positive

numbers and p < 1/2 C) Than there exists a 6 such that for ©very

subdivision S * {a^stQ# t„ 9 0 0 0 , t^b} of modulus not exceeding 5

nv

t iJlVVi/ < > 1“J}

44

Far tbs proof we need the following

Lemma _2_ ^6. For > 0, 6^ > Q* k > Qa k* > 0* * > 0* and 5j+5g

sufficiently email

C3 o aaj [1-k expl-lsf /c%)][l-k expt-k'/dg) 3 > 1-k exp[-k//{ 8j*52 ) v

]

Proof of lemma 2a 6s The left side of (2 0 22j Is not smaller than

l-k{exp(-k'/8*) «xp(-k#

/*»p) .

Hero® lZ e ZZ) is proved if we prove for sufficiently small and

FfSp 5

g }s «xp[-kV(5i^5g)v

] -expf-k'/fij]- axpl-k'/op > 0

tfo have

lim F( 5* 9 $2 ) s 0 and

Oyfr Q

X

a

exp[-k'/op

*b |

i J>V+1 8

1

r8xp[-kV^ 5i

,«'5

2 )

v

. expf-k^/Sg]

*V )

1 - „ * v*l

• C

j>j

+ So ) °2

}

}

45

The function x

* ^ exp {®k'/xv

) is monotopically increasing for

sufficiently small x 0 We haye therefore aF/a^ > 0 0 aF/a5g > 0 for sufficiently small 5* + dg , Hence

is positive for sufficiently small 5, * 6r> t

Ws proceed to prove theorem 2 0 6 0 we have by (2 0 7) with

l®£p s v o S

- €

] -

1= «*p

^

a

/as.Ji

> 1 - k 9xp(- ]£ /6

}

where k and k

; are independent of the subdivision..

Thus

How let S have modulus J

then by lemma 2 C 6 we may combine the

intervals to the right of (2 0 23) in such a way that all intervals

are at least of length J| and at most of length 6 c Hence

Uc 24) B > jY- k azp («k

/

/5

T

)J

2 ^"“^8)

and the right hand side of (2 0 24j is arbitrarily close to one if

5 is sufficiently small 0 ‘This completes the proof of theorexn 2 0 6 0

[

1 - k expC

h

U.23 1 P - P j ,p

< e 8 i-i t o e o £> i i~l

i i*n

»

J

>TT

i*i

46

A process is called Gaussian if the Joint distribution of

Xj, f x. fooo0 X. is normal for ©very choice of t^ t£0ooo0 t

ft

c 6 1 *£ b n

vV 0 now consider the integral of a F 0 R*? 0 and pr&ve

Theorem 2 0 ? r, Let x^ be a F 0 R 0 P C

. with variance ot 0 The process

cic /6or i Con C

j[^=f |x di? Is a Gaussian process with mean zero and oo»«9totlon

o

fuuotioa - £> max( t 0 1* j[min(t c t* }]* - Slminttyt" )]®*.

The integral X^ exists l o i 0 m<) by theorem 1 0 5 0 By lemma 1 0 4

we have S(X^)s 0 and by theorem 1 0 5 for t^> t

t t t t t

y o J Jraint t0 t'

|

d^dT^=o Jld's-jV j\ j\sV

Q 0 ® Q 0 %*

t't2

Each of the approximating Hiemann sums is normally distributed and

that itself is Gaussian follows from the following lemma®

Lemma 2 e 7 c Let x

n ^ (x^0 x|0oodr xj) be a sequence of normally distrX

touted vectors with mean 0 and assume that alfn ox

i

x

J TM @13 ex^sts 0

If plim x. s x than x is normally distributed with mean zero and

n

covariance matrix g

11 o

47

Proofs inequality (1 0 5J may be derived also for vectors if we

interpret x <* a to mean that the vector a-x has non«negative components 0 Lemma E c ^ then follows easily from the fact that for

arbitrarily small® and sufficiently large a

• F

n

Sa+5| 8 £ > F

n

(a-8} - 8 „

where f (a) P Flaj. are the distribution functions of x

ft

and x respeo

3. Friction a! Tk e Orns'te,'* -U/fUnbcc l ,^>gef£ , We have so far in the Brownian motion neglected the effect of

the motion of the particle itself on 0 If the particle has the

momentum x^ then the random impulses will have a mean value proper?*

tional to x^ it Self 0 This leads to the equation

! Xt+ T= Vt + *t,T

(2*25) 1

ft o* a

t * \ <lfw t > 0 f, E(x

o

Js: 0

wh»ro again s. • la normally distribute with mean Tain© zero and

>0 * variance* or^ and is independent of and of if the intervals

{t%t'+x*) do not overlapo we shall further assume that

xA Is normally distributed and that a is a measurable function of

* x

T C
I

I

• ' ''II

/ « , " - * 9 "

* \

..

'

.

.

*

• '

- .

- "

i

'”‘w

p . I

.

A

-

' •

* X

• -
48

It follows from (20 25l that

(2*26) J

" XUx^a

T

sS

Xt * %‘W

E(x^ ^

|x. ) s a x^sa^ & x* Ut1+t2

t t1^t

2 * T

i T

S

15

Honoo ^ s a^ from which it follows that a^s $

ax

and since a < 1 p a = e~^T

, B >

% T p 0 c wo further have from (-3 0 25}

and ('2

C 26)

a_ e* _ + s. T2*** T1

"" ****!/

2

® fe t.rT^T

g

Thus

a! a? a? s a? n 2 _2 ^ 2 a o $ q t

2 t

1 t

2 Vt8 t

1 T

S

or

«°V°v (1-a^j^l-al js (l-e’

<’^Tl)^il-*“

to ^Ta

)

, e

Therefore

(7, 37) a®*®^X-®" 2PT

) r p > 0 ,

Moreover from {

2

0 25J &n& our assumption about ^ wo have

jJ2 0 28} cr^ s a^a| * crs

( l-o“ s PT

} ^ If x approaches infinity than of approaches 0 That is

xUx

to say* if the particle has been subjected to these random impacts

for a long time then the distribution of its momentum approaches a

8 toady state 0
*

;

.

*

.

.

.

'

*

.
49

We shall therefore assume that the process is stationary*) that

Is to say0 that the joint distribution of x, 0ooo0 xt is the same

as that of xt jv. hf) 0 00& 3 Under this assumption of s

1 V x

%

and it follows from lz 0 27) that

(2 C 29 J o' y » a o

2s a

2 exp(-pv) for t > 0 f; t t*ir T

/ The x^ process c satisfying the assumptions listed above* was first

considered by L 0 S c Ornstoin and G 0 Ec Uhlenbeok* As will call it

the Ornstein-Uhlenbeok process (abbreviated 0 o yt{ P0 ) p

We next consider the process given by £2 e 53)c

£S o 30» I

1€S» V*8

^og t)/S|3

! *«. - 0 for t < 0

? CSSS*

sue have for > t

E(v

tvy }^oh

Jv

g

3= 0

(2c 33.) '

Moreover the ft ,, 0(, e v

t 1 n

the process is a F0 R0 P0

for t > 0

for 0 <a<t<&' c

are jointly normally distributed., Thus

From the properties of the y*. process

we obtain
\

.

*

'

'

-

1

.

•

.

' r ,

'

i

'
50

Theorem s ft 8 a Let be an 0 o Uo P o Then

(X) la continuous^

C XX j x.

?

. is not different tables

(XXX> M

ftb

and exist for the x^ process and it is strongly

continuous^

(XYJ the following equation —> derived from theorem $

.holds m > o

c*o

(

B

0 32) P(e?\

T < M} ss 1- 2 JU//2j)exp(-xB

/H) dx

«oo<^<(l/2p)log t „JjL

' <r/i"

In equation (2 0 32) we have written

P (y. < for P(M . < M)

&<t<b *» w 1 ** - s

where 1® the maximum of the process in a < t < b c. This

notation will also bo used In what follows,,

i Equation does not seem of great use as it stand

s

0 but

we can obtain from It a bound for P (x < Mj as follow®

P (aFx « M) <. P («PV. < M) < P Cx_ < ***P*lj •

t, -- aa '"S l==5' " ®se „ esmt * «&> <=> ^ - @i> <?<? *lSTS*s

and thm

5l

(2 c 33j!

/*; * /o

P (x T < M) > 1- 2 f 4= e”

x /d

Tl <•.%<*%o Af00 c=> #5 dr

•ft-*

t

Theorem a 0 9 0 Let x^ be an 0 o

lT o P o The integral I^s jx (it of this

©

process exists then X 0 ! 0 m0 end fear t

g > t ?v its ©©variance function

ia given by

(8,343 R* ... - SC

2 r ~02

-,k' -o -i

B ,. = mrnm I 9 *« <>2pt,”l-*‘ “ *1”S o2 1 A

9

(

8

0 35 3

is a F 0 R 0 Ptt Proa £: For t„ > t.

' 2 TMr* j

t© „ „

, a a r

* ® r ®

®t** P*W * XfX0

have by theorem 1 0 5

s 0

t

2f*f %«, <!» © ©

dTl

fi,r2

<$» l$*

15 -fa ** <n> 0 f \ ,n ca> ‘#“5

jj r @ 2 1

•5*0 , 0- T

dT

2

4,fl

1

2 r 'X <rse 2

J

53

anfl. for

EUt

-X0 )

2 =

|sp«'P*+pt--l) ,

A straightforward, calculation gives for t

g

> tj > > 8. g

C 2C 36 } 1[{X -X ~ x > 3

"2 *1 °2 *1

ps pa -ptj -pt

o -e ue ”-e

A

\

-% 2[(x

t

-z

t jUB ~x )]

P

x "2 l 3

8

i

Uein? FU/.J1 “ s(x i -.U«o

X

8*fe ~S ) f'a * h-?»o "•““-g

we obtain from lemma 1 0 7 and (2„34 j

20

(2„a?i s«xtx

g j

From (g 0 3fj we

f?. 0 s8)

P

[a if t >

E[(X

t

-X

t )Cx -*

2 “'l 2 1

if * <

0 * ~ ~ ( 0

~ - S£ll

g

^-X

g ^| (Xj-X^

53

It Is easily sear, from th®^Jgc.3SJ and (2 0 28) that

(a 0 35 ) B

t P<W ~ X

Q

has th« property that fos? tg > tj_ > s, > the difference

Ba. ®° B^

is independent of B

g - B

Q

- aM sine© is normally die

tributed it is a F0 R 0 P 0

From iz 0 35 } w@ have

Ihu@ for ©very function fit) for which the operations indicated

b@l®e have mis&aing^w© hav©

t t t S_, .. 'fCtj tet --fsjfU)^ at Jfltj fiB

t

a a

W© may writ© ^B 0 S^| as a stochastic differential equation

Cg 0 4Q| s - px^ &t e

. In the form Cs o'40j a stochastic differential equation too moaning

OT©a if th© procesa©s are not different iable 0 This interpretation

i

1

. j of a stochastic differential equation is du© to J 0L0 Boob 0 Th© ©quation Cs o 40| may to interpreted as th@ Ion of the motion of a

partial© (x^ boing its velocity at time t$ subject to fandom impacts

when the frictional for©® ie proportional to its velo@ity0 On® ©ould

also interpret x^ as an electric potential subject to random changes

when the d©or@as© in potential is proportional to the potent ini Itself
' !

'

v

, 9 II

’ - ’

.

' _

- ...

ii . ’

. Ji ... Vi v? ;

.-

a H *

. . .. V

. .

. ..

'

. | |

' I.:

j

-

.
54

{Thus we may consider a oqjidenser which is charged by a randomly

fluctuating current and at the same time grounded through a resist

anoo

1

0 In short 0 ©action (a o 40J describes any situation in which a

quantity is subject to random changes and to a systematic decrease

proportional to x^ iteelfc

i

V
-

l"

, . i, , ' * 1

- ' ,'tm

'

'

'

•

:; m

.
CHAPTER S

ESTIMATION OF PARAMETERS

In th© preceding chapter we discussed Markoff processes ^ we

shall now apply our results to obtain estimates of the parameters

determining these processes from observations. In our estimating

procedures we shall assume that we have at least one curve at our

disposal registering the values x^ for all values 0 < t < T

Actually it would be sufficient to know for any dense get in

this interval^ This procedure may not seem realistic sine© we never

observe the process for every time point o Every method of registering the curve described by x^ will itself affect and In particular smooth the path curve of Thus what we observe is really

a modified process a However the methods of observation may be so refined as to give

us the value of x^ in a large number of points and at any rate the

variances of our estimates if obtained from discrete points may alf><

be computed

The F o R 0 P o is completely known if the

tejR.

the constant

Eix

is known 0 We first discuss the estimation of the parameter o of

56

Theorem 3,1 If x*. la a F.R d P 0 and if it Is known in a dense set

in an arbitrary small interval 9 then it is possible to estimate

the parameter o with arbitrarily high preoieion.

ftoofi Assume that H observations are taken in the interval 0 € let t g T/21 and x in 6 Q,X 8 E 9 , ,

*

the sample value at the

time nx. Since x ia a F e R e P 0 the variates

V

x

(n.l)<e ^n-ly^T

a /r /x 8 2s 8 ( ® «Hj

are normally and independently distributed with mean %m*o and varl

o 0 The maximum likelihood estimate of the variance of Jn

Is there

fore given by

H

• N

i3 0 l) ° 8 f *

J^l

' Xbv~ x (n~l)V j^$/

x

nv~

x (n*lJv^

v/e have Ease, Moreover 5$ /o has the ohi»aqu&re distribution wi 1 K degrees of freedom,, Its variance is therefore gK e Henoe ^ has

the variance Bo * fB and this ©an bemad© arbitrarily email by UMi.

I large enough0 Thug if it were possible to observe the procesa completely in

any interval^ however small, we oould determine o accurately.

Actually however every registering instrument will introduce a ti

lag and will thus smooth the process,. We may Infer however from

our result that the points for which we read the value of x shoul

t

be spaced as closely together as possible*, That is to say* as ole

as ie consistent with the assumption that the values of x ~x

t*“f t

obtained still represent the actual values supplied by the £ , R 0 !*

5^ More generally we prove

Theorem 3 e 2 If y , is a stochastic process such that y

t

*5 Z. * f

{

t

where x. is a F,R 0 P 0 and f{t) a function of bounded variation

t

satisfying in (Q S T) a Lipsohitz condition < M for

some M^then it is possible to estimate Vne parameter o of the

F.R x^ with arbitrarily high precision 0 P

Fro*£: Let again t s T/H and consider the sample points x^{m

• H ,

. . \Z

2 8 ..„ e H). Denote by o « ^ J_ yRT °yUr4) .T. ... ® n«l x

9 I $ Then

%> “ pj

tx,jT

= f(nt) = f(n!It )]8 .2

CT

Her© and in the immediately following formulae the summation is to

be extended from n-1 to n* B 0 We write also n^l for (n®l) to

simplify such expressions as x, „ x s , y,

‘ s or

(n-l)T n4t (n-lR n~X<¥

f[(n«iW] s f (nTix} 0 Then

-ft

Z[«lax) - KEltJ

-i- 2[f(n^)= f{n=l *?} l

2

}

2

CT

We expand the right member of this expression; a considerable

simplification follows from the assumption that x is a F.R.P,, we

w use in particular the fact that 5P£~2ELs is normally distribut

/or

58

X

with zero mean and unit variance and is independent of , mT i ; ...,

JmL£

0 %

for n 0 Thus we obtain

El |, 2 „i + 2 + 2r [f (n-t) - )]

3 4 r [f(cT) - f(n^Ic ) 3

2

o'

" i • 1 * 1 t- c X

i» “ O X

jg { t [f(nT)

0

” f - 5-1

y

and

Etl) - i»i.T [f(nT)

o c<?

Hence

E(c - o)

2 - -S2

2

4. is y

[f(nx) - fin^T) I 2

* _L (y [f(nx) - fU-1 -elf

)1

H H2 T

jjS- It, . J

Thus o converges in the mean to c and o»c is stochastically of

the order 1 fj§ 0 In fact E(c-c) 2 < JtSU + iS MV + i(M¥) 2

y ~ E Jr N*

g

Here f{t»T) - fit) < M and ¥ is the variation of f(t) so that

also ¥ < MT where T is the length of the interval.

Thus in estimating the function f{t) w© may assume o to be

known*, if we know y^ in any interval completely^

tf© shall discuss two examples of the function f(t} 9 In the

first we assume that

fit) ^ at

(Since we can always consider the process this assumption

is identical with the assumption f(t) g at * b e ) We then kno7

that the y\ are normally distributed, and independent in nonover lapping intervals with mean at and variance o% 0 Hence the

maximum likelihood estimate of given the values at time

09 %9 coo* becomes

a

1

1

z( ya** - y

n®1 ! v

) v-s - yQ

Its variance i®

?or the second example we assume that fjt) is given by

m

fit) sJZ Cameos Jt p 4 sin ^t)

J«1

J * and that the values of y are toomx in ths Interval (0, Elf) ani

that y„ * Q„ If we 4ust choose the values yQ

» y^» y^. , yJt

where nv e g 7J"» the maximum likelihood estimates of the and £?. will be given by those values, which minimize the expression

& m

ZL {yiT - G.[003 i^v - eosti-l) Jv]

i-1 4 1 AT j»l * -r C fain ijv - sin{. i-1)

i-l i

80

Henoa the maximum likelihood equations are

n I v/,.a . (3«2 0 l) 21 (ylT

- )[00S ik? « ©os(i-l)k?]

n m A

s 2 I ou[©03 i^T“ coe(i-l) Jt][oos ik.T~ oosti-Dkr]

1*1 5*1 « a m A ^ F ]T s«[sin I^t

sinti*-!) Jt][oos ik^ - oos( l-Uk^]

l»l 3-1 * k «• 1p o c © 9 ^

and

n

(3„2o2) E ty{r- yp-j.j ) tain ik-r - 8 in! i-‘l)k<S

n m

= L L

ui

JT d [ooa y-s ~ oos(i-l) JxHsin ik-r - aia{

3

a m A

t r. z p-csm y*

i«l 4*1 J - aln(i-l) Jt][ela ik-r - 8inU-l)kr]

k s£ 1, 3# e a o # m

If w© dirid© the first of these equatione by t and let % go to

setro* we obtain because of the orthogonality of .the sine and cceinfe

functions

zr

k

| sla kt dy. ss cl* l sin *kt dt ^ cl irk

vrt & k fcV •£

61

The rules of calculation for the integral in the left member

of this equation are completely analogous to those for the ordinary

Riemann-Stieltjes integral. Integration by parte on the left

gives therefore

j y oo s kt dt » a^Tf

and thus

(3.3,1)

A

.STT

s

ir J H

cos kt dt s . 1

71 !

zr

sin kt dy^

0

Similarly* we obtain from (3.B.2)

Err

2

- k

J

j’ oo s kt dy 2 f k* p

0

*

Integration by parts gives again

(3.3.2)

a

+ V

8ia

^ ^ * A C'C06 kt d7

t

The integrals in (3,3.1) and &30 3,2) are to be understood as

stochastic limits of Riemann sums* and it is easy to see from the

corollary to lemma 1.7 that these Riemann sums converge in the mean.

From lemma 1.4 we see therefore

E(\) * C* , E(gk

) s ^

and similarly from lemma 1 0 7

63

o*« sfct n® 1

4

8in kti6111 kt

j

0 {(y* - y* Hy+ - yt )} a

fc T ^ 5^0 1J 1 3 \

t i.i “4 S.i

5f*0

nJaNr o. - ra&xj t^ * t„ | and 5, s maxj t . - 1 4 | . l i sj 2 j'-l

Since the increments of y in non- overlapping intervals are

w independent of each other we havs

so that

( 3*85 %

and similarly

(^»9) °1 s ip

Further

(3.10)

ft s ap 1

Tc

at Te .£, Ca * s 0

“k*3

*

) singlet.

c

£ir 9

f sin^kt dt

J 0

= 0 for k Z

Frc& lemma £ 0 6 we see moreover that are normally distributed*

Suppose now that we take observations in the Interval {0*

so that we have

*zo. ii) f( t) 7 [a^ cos 21m ^

sin r 2im-|]

r> wo put .an yj - yt .

<*"T> « ^ « xhofi

I & : - *

o

2

0k »pv»® maximum likelihood estimate a and B?. then oeooaie

iw

A

IS

1 ^ fE

t A o,_

88 - •sag, J

sin(27f&

0

rp A

Pk

~ 4- | eo-a (2rk

o

2

0-^

&

ID v *?S _ *2 n <0 Pfc s-12 **

v ;

-

i ors«sTe*iWrifi>^-5f^»"r«» a «s£ 0* ,CTa a s& €f>& £ » t -'5- V

a, a.

k t

?y P*

u- a confidence region for the avp 83p M #

where the sum runs over those terms (a^~ a^) 9 ( p

) which are

not zero by assumption, and $,

2 has the %

2-distr ibution with the

number of degrees of freedom ec^ual to the number of terms in the

sums on the right. The estimates a^f p^ are consistent in the

following sense. Suppose f(t) is given by (3.11) and we observe

y^ in the interval YT where V is an integer. We then have

^ 008 2jr "Tr^ * Bln

3

so that

2

Cf^

®k

2

a*

Pfc

Y To s To

2 rMk* 3 JT^Vk^

and thus piin cL = gl . Similar ly p plim h* 3 0 Y-f>co s * Wes ' * k

Q.4.P. We now turn to the discussion of the 0„Uo £„ given by (2

and prove

Theorem 3.3 If x^ is an 0 o U9 P o determined by $ha two parameters 'ft and

2

or and if the values of x. are known in a dense set in any interval

%

2

0 < t < T e then it is possible ,to determine c p

with arbitrarily

high precision.

Proo ^ : We form with s T

(3.12) J)

i r vXnT " ^rt”! *r

'

we have E(D) * JBg5(l-aJ - 2o2 .

For t-I^O this converges to 2a p,

65

We now compute the variance of De For this purpose we shall

need the value of } with t < t

1 < t

,f < t HS

e We have for t < V < t

18 < t

908 on replacing x^*, hy

o f zf

* et^tm -tn &&& analogous transformations

E(x

t

x

t

,x|,,) = x

t

, e

t

»jt n.t

<) J

2 , \ / -» 2 » 4

s a

t

r, «t 8 ^ * a V ~t

^ ~ a

t

w “t 9

J °

and

E(z

t

x?.i * B

C

x

t

la t*-* x

t

* e

t,t»=t

,s

3

=

»t*-t E(zt

) + »*•-* E(x?) <r

‘i(1 ° a

iS»~t :

s c^£3a^i_^ 3a^i«,$ •= s Sa^» = ^« Thus

and

E^ xtx

t sXt

n ) ~ a !>t"~t 9 a

t

9 “t 4 a t®®t^" a

t

w ®t 3 U

4 2

c ^ a

t

3 -t

41 ^a

t

w ~t 0 a

t

f, -t^

4 g s ° ^ a

tm-t" at

3 =t

41 ^a

tm-t ,f a

t

n -t 9 a

t

8 ~t'

A

s a a

t*»t * Satm -t r

' a

t

n

=>t' or

13) E(x

tx

t?

x

t

^

tm ) s a exp r=p( t

re «t n *t 8 ~ 1 )1 + Z exp£®p{t w

66

For n < m we find easily

^3xax" x

n=I t

' 1 i

et° z

m=I t ]

E <

(

* x

ztt“

x

n=l t

} ^

£ (a

<t”

1 )x5=l t

+ ES^'ri t

3

*)

2

° 1} CE(x

nx

J4=Ix 1 “ j4=1t) * ^*£=It*S=I x

]

2 a

4 (! a^)]*

From (3 0 l3) w© 838 then

$H«- xn^X x

] 8(xbt “ xi^I t

] *J

2 4 -3B(a~n-l)T =8t «BT[2 (m-n) -1]

T “l) 0^|U 2 e

1 - 2e ’ -4©

«2B(m®n)T

1 2e + 2 t 2 a ! •

Since a & e PT we finally have for m > n

(3 0 14) #* -<=> ZJL —-=» ; • X —-b* n«l t mT m~X n

(l^a.

2 "• 28 (m<:»2i'*l 5 x .

) 9 ' 1
i* • % V
Farther

E(X;are» X£SI T

4 h; lift, «=. IT + €J “*==3 n^i-r - nra i *?: 4

2 2 _ 2 ^ 2 §

l)

4

6(a

T

- X) Bixjq T )(1» a“)(T 3a U~&T

),

3 a

4

[l

4 , Z 2 2,2

1) + 2(a « X) U ° <•

3a

4 2

that Is

L. ) ^ 2 ( X - a® ) S X + &

r

) ]

*J

4 * 12 c

4(l~ a

T ),

SVom (3,125, (3,14), and (3„l5) wa have

4

j a _|_g (1- a^) 12 H 4H(H"1 ) 4 ( 1 - a,,)

Ml rri<=>n«a

bo that

£ _ O’ « D

4cr

4

d”> ajtl_ fgH + ( 1 » a

T

) 2 2p(m®n®X)T 1

J

f

now compute

I 7

<*'>“

$38 MzffcH

Y_ (»v”n

as

By using the formula for the sun of a geometric series

it is easily seen that

•I ^4

L

m~n

b 3 1 - b

H

*° “ ® C

1

88 b )

hence

A e

3 1 • &>

23

i-a2 (1 = a

2

) 2

We substitute this in the expression for cr and obtain

B

4 cf (l~aT

)^ f E(l«a,r)

3*t L

1-a*

1*sl (l*aJ 2

j

or

f3 0 l6)

£

4c

4 (l-ax ) 2

r H-l

u

I

2 2H

<? “i (Lap

2 Equation (5 0 lS) shows that a" can he made arbitrarily small by

making H large enough, In fact

lim qu s 8 B^ and lim E{J>) « 2 8 / =&rn WTtr T) r ' V 3-^qo i3 D 2T-£ao

£(D-2Pa

2

) 2 = a

2 4a 4 (I~£!2 \6

>)

also

and

Kjysu-apoSg S 2 //ei [iisl!! = p ]

and therefore since tsT/K

lim Ef/^D ~ 2p03 Q” 0

Tf “*C®

£

proceed to prove that the limit distribution of </$( D - 2($cT^

is normal. This may be seen as follows:;

nZi x V5 fe

H~' [(eT-^)x » g 1

m~ '

' nT^TJ ht ?, t< SBSS^SKt A

or

, t „ n _ i

^

2 2 (*T-1) y _ 1 y gnv. - ;.

(3.17) */® % L,

7

'a%

*

r L 'ni'n'r.T ' ,/W I—

ti-l J°

“nt~aT,t ' 0

The last sum is a sum of independently distributed variables

all with the same distribution and converges to the normal distribution by lemma 2 0 X 0 Thus the normality of the limit dis®

tribution will be proved if we can prove that the first two sms

in (

3

0 17) converge to zero &

70

tfe therefore put

(3 0 18) ~ 2

x^ and % Q & -JL f x £

v

_ 1 ^ * V nT Z %

o

nT nT* ^

»Ys ha~e by (3 c l3)

4

2(zf) 5 t* * U^1 '~

[5

2

* 25 * 4??r*Pln-n)T

] #

3 JW

The double sum in the bracket oan be easily determined and we have

E(zi>

^(a^-l) 4

r 2(l*«w)

5 <* ¥ t 1 -a'

i

H U=a2)2i.

Clearly lim E{£, ) ® 0

5<“f>oo

Fur ther

, a,

E(z

g

)

*« a //=* / i jsu2 *i^L „*u-4)

f- nx ^ *e

o

and therefore lim E(2 0 ) = 0 , 5~#oo 2

Therefore l 0 i*m. 2^ ~ l 0 i 0 aio Z ^ 0 so that

w 4 5 oo ^ CD

h'-t 5 oo d - •%

/H £Si x

n

and since the second term on the left is normally distributed

2 with mean 2 it follows that 2j3a^) Is in the limit

4.2 normally distributed with mean zero and variance 8cr p

2

To estimate a separately one might use the estimate

*2 \

2 . Q = -m J ax‘ A o

b

Its variance is given by

hT T

-h I X

E ( xtV )dt d4 ’ * ° T2 o o

4

%) f f

e

*' **

««•<* . /;* .-***'-»>«

* O' t

2<r

4

+

g

4

(e"

S3 ~- 1)

pT

A

If p.T is large enough compared with a" this comparatively simple

estimate may be quite satisfactory.

CHAPTER 4

THE GENERAL DIFFERENTIAL PROCESS

W@ stoll, consider processes xwith the following properties

g

Cl} x,^ ie a continuous process (not necessarily

strongly continuous )|

2 > L8t

„ X. C 6 , X 0 6**0

x° i

*2* *a

Th® random Y&riablea

are completely

independent of each other if the intervals

(t1D t^i1

} c (t 2l.li 2

+-v£

) F , 0 , (,(tnr t

n

4-T

n

J do not

OTorX&pg

{3} Th® distribution of ^

is independent of t c Prooeagas satisfying thss® thro© oonditions will b© called

differential processes 0 In this chapter we shall find a general

expression giving the distributions of x* _-ac* for all possible

rr % « ^<4/1

^ differential processes,, Processes satisfying equation (2 0 X| and

assumptions 1 0 z 0 30 of chapter 2 are differential processes of

second order and we discussed in chapter z the special, case where

ac * 4.‘f~z

fc normally distributed*,

We stoll first discuss another special c&ee in mioh the

increments *»x^ have a discontinuous distribution* A practical example for such » process ia? for instance© the total of

insurfence claims raised against an insurance company as a resultof randomly distributed accidents* ah important special oase 0 fundamental also for the understanding of the more general problem^-

73

is that In which increases a randomly distributed number of

times within @v#ry time interval but each time by the game amount n

ft’© shall call such an increase a 8hot c make the following assumet longs

(1} The probability jP^th&t k shots will occur

during the Interval (t p t^x| is independent

of t end of the number of shots that have

occurred up to and including the time t a iz\ The probability q^

2 ^ that more than one shot

will occur in a time Interval of length x

is of smaller or&sr than x0 la symbols

a

qf,

2 ^ s o(t) or limlq^/T) = 0 o % x

o

f

pi^' la a measurable function of xa m ^

vVe clearly have 0 if Xj*x9 — *

1

4

c l) 4°»

1 2

From {

4

0 1 j and the measurability of It follows that ax

and since p^

v * <. 1 we have a < 0 0 Moreover c if are any shots

to be expected we must haw a < 0 9 ss»]i where pi > 0 o Thus

{0) «. e-l« U*2) o 11 > 0 o

74

We bow divide the interval {t*t*'r) into N parts. Then

for sufficiently large N the probability that two shots will

occur in any of the intervals oan be made arbitrarily small so

that If denotes the probability that k shots will ooour

during the time interval % -

(4,3) p

v

SJ

=(k) [l ~ «cpl-££ {H-kj3 o(l)i

For 14 ->-09 we then have

(4 0 *) p

(3t *- dlfei?TM

*t k*

The distribution (4 C 4) is the Poisson distribution* its mean and

variance are both equal to p.v 0 We next consider a situation in which the assumptions (1)

(k)

*2 if 3) regarding hold but where the increment of x+ at each

shot varies and has itself a probability distribution fyix) =s and we shall also assume that the increases in different

shots are independent of eaoh other 0 If $-(x) is th© distribution

of the ©um of k independent random variables * eaoh with distribu

tion &(x) 0 then th© distribution of the total increase provided that

k shots have occurred ia $k

(x) c Thus th© distribution of xt

is given by

75

(4,5} P(xUT _Xt ^ -

00

Z

k*Q

with

for A < 0

for A > 0

Is the following we shall need the characteristic function

[abbreviated o c f0 J

U.&) f^U) = E{8Spfi i8(x

t^ T

-2

t )3}

cr. th © distribution (4 0 5jf : An er.»y calculation gives, if gCs|

denotes the c 0 f 0 of ${Xi

*-(•!« £ - ^

lVR }

*

Cgtsj]* 8* exp{jiT[gl»J ““!]}•

v ko X t The distribution (4 0 5J is called the generalized Poisson distribution.

,tfe return now to the general differential process. ^e hare

at,T® s

t c i* *t*jf i *# i

n an n a

Hence r lfdL(s) denotes the o,f Q of 6. we have

A 16 - t ot

f 4,8) # f

Cs? = t*f x

Cs> 3

1

"

.

n

Froa the continuity of x* it follows that lim 0_ A<r (s) as *3 is) c * &£*£»<& 1

T h' iB (4^8) ispli 6

8

U.9? 0^5sj = 10^(8

_
'
76

On the other hand every family of distribution functions H^(x)

whose characteristic functions satisfy aquation (4„9) is the distribution function of the increment of some differential process,

Hence the general form of a differential process will he found if

we find the general form of characteristic functions 0(sj that

satisfy the condition that for every % > is a o0 fo A

distribution law whose 0 o f o satisfies this condition is called an

infinitely divisible law (abbreviated* ! 0 d 0 l 0 j 0

Oar main result will be the following^

Theorem Uet ^f(sj » log 0(e) „ The function 0{s$ is 0 o f c of an infinitely divisible law if and only if

<$» oo

(4,10) ty{«) a iea ? (e

,i0X - 1- 4&(x;

~Q© 1*XS X^

where a la real and G{xj non-decreasing and bounded and the inte2 grand is defined by continuity to be-,® for x«G 0

fundamental for the proof ©f this theorem is the powerful

continuity theorem of P0 Levy0

Continuity Theorem 8 Let {F^lxj } be a sequence of dlatrlbution

functions* the oorresDonding sequence of o 0 f0

5 8 0 The sequence {Fn

ix}) converges to a distribution function F(x) if and

only if f Ce) converges to a function fls) continuous for bs 0o

U8J Theorem 4 C 1 is due to PD Lovy fsee* for instance* his "Theorie

do l' addition des variables aleatoires vv 9 Cauthiers Villars

P&ris^XSS1^

p 0 180) 0 The following elegant proof is due to

M0 Loeve* University of California rubl 0 in 3tat o0 vol 0 1*

W0^0 53-88Cl950j o
I

\ <

• ' <

1

(

' *

* !.

"

' ' * - • '<

.

I

'

/

, <

• *

' •

• ‘ *

J

*
Far & proof the reader is referred to H0 Crambo Mathematical

Method© of Statiatloso 10 jQ9

Via shall need this theorem in the following slightly mor®

gonsrale fora 0

Corollary to tfog Continuity Theorem s Let {F^lx|} be a sequene©

of bounded monotone functions*, F

n

(~oo ) =s 0 and let {fa

{ s ) } be the

sequence of their Fourier transforms

U«U» t U)* / e

lM «F U) • -00

The sequence {F (x)} converges to a bounded monotonia function

Fix* and lim [F 5 -

F

(-o© j] « Floo 5 - F(-o» ) if and only if

« »

the sequence } oonvergee to a function f(s} continuous at

a ss 0 0

The corollary follows easily from the continuity theorem in

observing that

V°J* X «.U, - v

n .

Hence the variations V„ of F (x) ©onverge to f(0j o If fCQj^Q

a a

then «£-«—» converges by the continuity theorem to a distribution T*a

function H(x) aM F

a

(x) 0 onverges to Fix) HixjfCO) 0 If f^OjssO

then lira F_{x) - 0 and the theorem also holds 0

n-M» a
- . . • > I ^ I .
TO

shall also need the

Let {F (xj) be a sequence of distribution

AA functions and iim F (x} » F(xj( r Let further g{x) be everywhere

n~t*oo “

continuous and assume that to every e > 0 there exists an A such

that f |fi(i) j AFjjfi) v e 0 Ih«n

m &a

AGO ‘ ^*5

lisa f gU] 4F (xj « f six} O.Flx) .

r*

\

%

* For a proof of this theorem the reader is referred to Cramer

®l>o ©iSoo p 0

fM0

It is easy to verify that the conditions of the HeXly-Bray

theorem are satisfied if

lim [F_(oo j ~ F (“COJ 3 s? Fioo j — F(-o? )

and if gixj is bounded 0

$© ©hall first show that ©v&?y function T$r(ej given by (4 o 10j

is the logarithm of the 0 o f o of an i 0 d 0 l 0 For this it will be

sufficient to show that ®vsry fyis) given by (4 o 10) is the logarithm

of a 0 o fo sine© it is obvious that with ^( 0 ) also idJLi satisfies

2 q see this# consider first

Uoi2)

with 0 •; £ <, 1 0

I I®} may ho written as the limit of Riem&nn-StieXtjes sums

w

(4 CIS) Sk"^[>.k

{# '"^-l) isjxjj.] v

with

xk

Pk 5* > ~ 0' x

k^ 3 •

It is easy to v®r5.fy that ® 13u is the Ooto of the distribution of & random variable which equals u with probability

oMo Hence on account of (4o7) we see that

l9Xy

Xt-le -U * is£>

X.

is logarithm of a © ft £0 ana so consequently is 0 Thus X^Js) is

a limit of logarithms of o c f 0

s i c Also I^ls) is obviously continu

oua at s*&G 0 By u£t$% $ continuity theorem I

g

( aj is thus the

logarithm of the a 0 f c of a distribution funotion a By the $am®

theorem also
%
80

IAb} ~ Xim Io ( 8) Q J >e iax_ , AJSLj lax 1*^1 dG(x)

la the logarithm of a 0 o f,

But dr(*)

1 <»

ssiaa+jf (eisx ~l~ -iSS^) d&Cx) *t I

Q

(s) + isa -jp[G(0+) - G(O-) ]

«=* qo x

Is obtained' by adding to 2 (a) * isa the logarithm of the c o f0 of a normal &istrlbution0 Thus SJtCg) is th© logarithm of a o 0 f 0

Thus the equation C4 oX0) is sufficient for fyls) to be the logarithm

of a © 0 f0 of an i p d oX 0

To prove also the neoeesity of (4 0 XQ) w© need sot&t&I Xemmas 0

hamma 4„l n a|xj and a in (4 o 10) are uniquely determined by fCa3 0

»«fe consider

1

Ua M) &Ib) « X f>< 8 > - ^

2^b|i8^Ij dh *

o

-#>0$ 2

00' j e

i8XU-

^

ac-{x5 * j* ?

ie* &*<x)

-do* x *®oo

where

X £

( 4 0l5) $(s) * /ClTM &i§I) At®. aafy)

-a>

• y J

81

4 It is easy to verify that (1- SiSt) is bounded above

and below by positive constants,, Thus |(x) is monotone and of

bounded variation,, Xh© Fourier inversion formula determines uniquely

${x) given e(s) and thus also

x

2 -1

U0 1&) G(x) ^ f &f){y) ur y

Finally a is determined from (d o X0) o Lemma 40 2 0 If ^a

(e) determined by G

p

(x) and by a

n converges uniformly in every finite interval to a function b(e) continuous at

the origin than Aim £ (x) =2. G(x) and lim a =s a exist and

n-^oo • n-*>o»

bdsjss ty(s) Is determined by a and G(x) by equation (4 C XG) C i^roofg Frcm P0 Levy2 a continuity theorem It follows that

b f s ) e

lt is a 0 o fc hens© everywh<sre oontinuous 0 Thus also b(s) 0 it®

logarithm,, Therefore* ea

l©) JP defined by ^a

{s) by m'sane of (4 0 14)

converges to a continuous funotion0 . By the corollary to the Continuity Theorem it then follows that Lim $n {x) = ${x) existe c More®

n^oo a

over |lx) is non-deor easing and bounded,, It follows from the

Helly-Bray theorem that

Lim G (xj x. Lim fd,. ?A£X.)

*

&$n

iy)

-go

x

i ?

= fii-alsafA

a$< y) - s{x)

-a? * i+r

8B

since the intograni Is bounded*, It further follows 0 also from the

HeXly-Sray th sor-3m D that

*c©

11m I ( s ) — Urn f (eisx lex

?)

u

s- »y*>

*#•00

A / isx

=s j

(

e

dG(x) a* 1 (b) ^

'Finally It fallows from the oonvergenoe of ^(s) and I

n

(e) that

also the sequence {aQ } must converse and thus b(s) — ^(s) 0

The converse of lemma 4 0 2 follows Immediately from the HollyBray theorem 0 Lemma £

p 3 0 Th« 0 o f0 0(@) of an i 0 d 0 I 0 is everywhere different

i from zero 0 Conti dor {0(3}*^/^ We. have lire ~ w(sl

n-frco*”'

where w(s) «s 1 for 0 (b)^=O and w(g) =5 0 for 0(s) ~ Q 0 By

f . • Levy's theorem w{g) Is a 0 o f o Since w(s) - 1 for s and w(s)

is continuous being a o 0 f 0 we must have w{s) =. l everywhere 0

\ Thus, 0 ( s J ^ 0 everywhera 9 Lemma 4C 4& log x » lim n[x^n ^ - 13, x > O' n Lemma 4-4 follows ~~ n~^oa

immediately from the rule of 4® 1' Hospital,.

Lemma 4 ? 5 0 If 0 (b) is the o 0 f0 of an i 0 d 0 l 0 then there exists

a aeguenoo of functions ^|x) given by (4 o 10) gush that

log 0(g)- lim

a -*oo a

83

Proofg W© have by lemma 4 C 4

log 0(6} « 11m n{[0(8)] ll /n) -1} * lim ^ (s) n^co n-^css s

uniformly In ©very finite interval of s since 0£s)=£o with

*00 x g

&

n

* a J £~*jpr ^ G

a

!xj « nj dF

a

{y) 0

Hare F (x) is the & c f G whose 0 o f o ia [0(a) 3 a

•Proo£ of tha necessity of {4 IQ) a By lemma 4 0 3 0(s)^EsO 4? hence

log 0{s) is defiiied everywhere and continuous 0 Moreover

log 0 {a}^ lim &.(s) uniformly in s with Ir ( 3 ) given by (4 o 10) 0

&-f» CO M »

But by lemma 4 0 2 lim ^n (a)s 4r(s) where ^(s) 1 b itself determined by (4. o 10) o Thus theorem 4 0 1 Is completely proved 0

From our proof of the sufficiency ©f equation (4 o 10) follows

the following corollary to theorem 4 01 0

Corollary t© theorem 4 0 X§ If x is distributed according to an I 0 d o £ 0 then x s* y * a wh@re y is normally distributed and 2 is distributed &e Is the limit of a sequence of finite sums of independent

random variables each of which is distributed according to (4 c s) with

V

0 for x < 0

1 for x > a

84

Theorem 4 a Z a Let x.u be a differential process of second order r than ECxfc*T~xt^

251 1' m VarCx^^-x^) » t where m and g^ are

constants independent of t and t 0 Froofg het ty^Cs) be the logarithm of the o c f c of x^^, -

x^ , From

(4 C 9) we see then that ^(s) t^(s) where ^{e) is determined

by (4 ,.,10 } 0 Therefore ty'(O) = v^[(0) and ^(0) =* ^tx(O) 0 From

this and from 0 ) - iE(xu<r -xt

) and t^O) = -VarU^-x^ 5 we see that tho iamaa holds,

. The estimation procedures in the case of a process given by

(4 0 4) are vary $impla 0 The parameter to be estimated is u f its

maximum likelihood estimate is X-ffar0 the number of shots observed

per unit of time* and the variance of this estimate is ^/T 0 If the process is given by 14 0 5) then the increments observed

are a sample from a population with the distribution ${r) 0 If

» $lx) is given in parametric form then the proper estimation procedures are those appropriate for estimating the parameters of

|ix) from the observed sample of increments*,

t

r
.

.

'

1

x.

''

*

. IS- S

.

;

. 1

• •

•

'

1

«

'1 f

I

.
CHAPTER 5

DIFFERENTIAL PROCESSES MODIFIED BY MECHANICAL DEVICES

X 0 Filter effec t.

In registering a stochastic process the registering device

often spread® the affect over a certain period of time in such a

way that an increase occurring at time t s* 0 will produce an effect

at time t and the observed value0 or the output process,, is obtained

from the superposition of all the effects produced from increases

that occurred in preceding time periods 0 If we let y^ be the output process and x^ the input process

and assume that the effect is proportional to the increase we then

have

C 5 yt s J f(t -t) d*

f

c -CIO

C». -»GQ>

where J"' |f(t)|dt and J

1 [f{t)J2 dt and the integral (o Q l) are

- C£> - CO

assumed to exists

taking the upper limit of the integral ©qnal to infinity instead

of zm?o leaves the possibility open that future changes will influence the pre8.ezit 0 If the present is not affected by the

future j> then f(t) will be 0 for negative values of t 0 In previous work we have often talrsn the point of view that

the process start® at some fixed timi T 0 However wo can also

speak of the conditional distribution "£ x given x^ for v < t

B

and thus f f(t-T) &x may be formed for every A and B and we

A* T

proceed to prove
I

.
86

Theorem 5 0 1 0 Xf x^ is a differential process of second order with

-v-CO a weight function f{t) then the integral

j fit- %) exists

- oo loio^o

In the following v/e use a simplified notation by writing

H and \ So* H-H-x o

Proof of theorem 5 e XS tfe have

a p

E[ j

fit-t) 5x

x

3 s lim

- ^ i

f

- lim Eft^U-'cfjU.-x, ,)

2

i 414 4

• + 51 f(t " T

i> f(t ' T

i>

{xi' xi-l> (x4- 3C4-l)>

whare t,„x < t| <• l

Since x^ is a differential prooess of second order we have,

writing 0

| j

for the covariance of (x^- x

i~l^ (x^-x^)

0 if

°S(Ti- H~x' if is j

E(xl- x

l-l?

* aixi- Vl! •

Thie follows from theorem 4 0 2 0

•

8?

Thus

B B B

(5*2) E[ J tU - T)dx,.3

2 = o

2

ft

8

(t ~ T)fix m

2

[ A A A

t-A ' t-B

« cr2

j £

2

{t)<1x * o

2

[ j*. f (\)dx]6

c t-B t-A

Both integrals on the right of ( 5 o 2 5 converge to zero if A and B

converge both to *<&> or to - m 0 In fact* if m^Q*, only the conGO

vergenoe of J[f(t)]*dfc need be assuznedgthus by lemma l 0 a

B

l 0 i 0 m 0 If (t - Tjdx exists.

AH> -00 J

v A

We denote the characteristic function of the increment

~ a process by $^{s) „ From (4 0 9J w@ see

that

(5,3) fjU) =r exptt log ^(sjj

We compute next the characteristic function of y^ *> that

is %C®3 - E(e'4syfc ) 0 We have

|5,4j- * E{exp[is JfCt -Tjto

x

])

-CO

» E{axp[is plim Zf(t-rf S(^ 1 — x ui)l

Aj~#*0 j

4 4 * A
'

.

.

;

'

'
Stia random TarlabIs- in the exponent at the rl$it of Cg i;/4). converges

In probability to

i. tin o t i on o onverges

*3 j' f(t-T)dx rt Hence its characteristic

to u* Ja'j c »<e thus have

(5.5) i)tU) «U» B{exp £is Ztlt- xplxj- *j-i)

Since the summands in the exponent are independent random variables

we have

i* (s$ - lira TTE{exp[is-f(t

%P® j

Th® characteristic function of x

t

* therefore

the characteristic function of £lt->

^Llsf(t ~?||J » exp {A

j

log ^[sftt «tJ

(5*6) , log s| t (sj- lim^Aj log fjtsftt- t*

4

so that

* '00 * 00

*

J* lag (^[afU - tJJAt *

J* leg ^[afUjJdt 0

«. w> « oo

The oharaet'eristio function of the joint distribution of

y,, 9 which completely determines the output process i@

*1

‘ ' *n

formed In an analogous manner 0 Me have

89

fit t a i»o»os3n J = E{exp ! Js,

y

t + a y )}

''i f o « o? A n A *1 n j n

oo — r% E{exp 1 j tSjfftj.- t)+„..+8 -tj 3}dx J

-G2 z

The argument employed in the oase nsl shews that

<5o?J *lt- o 0 t o o ©S> 9 o o o 9 A

OP

S j* logi^LejfUj- 8

n

f{tn" t ^)d'c

It Is saen from (5 0 ?) that ri+ f (biPoooSI 8 ) and thus also

»1 # OOOJ A 11

the Joint distribution of y^ 9000? ¥% is invariant under trans«

1 n

lations in timo c lienee th£ process is stationary

If the distribution of x^ is Gaussian (a differential process

which is Gaussian is a F0R 0 P0 ) then also the distribution of y*

oo is Gaussian,, The variance of yv is given by ^

j*£f(t }j

2dt

oS

and tae covariance function is given by

oc OB

(5„8> R(t - t

#

) s E{[ f f(t-vj«SxT][ J*

f( V - t

)

dx^ ]

}

a a

-00

£ r J Z

j* f(t~TT)f It^T^dT £S. 0* j. f(T)f{t# - t+T^dT

00

114 ] A process is called stationary if the variables x. , o , 0 *x.

t

l *n

have the same distribution as the variables x^ ^, 0(8 ,x f ^

1 n*

0
. . , ... "

’

.

'W

*

.

: Umivn e At (

' 9ass8 6
90

This follows immediately by writing the integrals as limits of

Kiemann-Stielt^es sums and by then applying theorem 5 C 1 and lemma

X 0 4 0 Thus the resulting process is a stationary®® Gaussian process with covariance function (5 0 G) 0 If we put# for instance#

for t > 0 / 0 tor % <. Q

fit J*-

th©n we obtain the Q 0 U0 P0 of Chapter Z with Styles \|£

*P

It may be seen from C5 0 7) and (5 0 8J that a larg© variety of

output processes may be obtained from .differential processes,. If

the differential process can be specified in parametric form and

the function f{tj la also known at least in parametric form then

(5*7) or in the most important special case C

0 B ) will give the

output process in parametric form# so that the procedures of testing hypotheses about a finite number of parameters and of estimation

in the parametric ®ase become applicable although the difficulties

of calculation may still be formidable,,

la case nothing is known about either fit

j

or #f(s) the only

way known at present by which some inferences can be obtained is

by the spectral analysis deoribed in Chapter 5 0

A process 1 called/stationi If |he variables x

y© the /lam® dl^tributionlas the/variable®

91

The modifying devloe may also operate in such a way that

the modification of the input process is itself dependent on provlous values of the input or output process 0 A frequently occur®

ring example of this type of modification is provided by certain

counter devices* which count random events 0 Due to the inertia of • the counts device not all events will be counted. In particular

we shall consider two types of such devices 0

Type 1 0 After an event has been registered the counter remains

locked during a certain time t

.

Type After an event has happened the counter remains

locked during a certain time t c A general and comprehensive treatment of probability problems

in counter devices has been given by *» 0 Feller $Couraat Anniversary

volume* 1948$ pp c 105-115) and we shall here follow essentially

Feller5 a representation We shall assume that the input process

is a Poisson process described by (4 C 4) 0

*

* Aet T^ $ i > 1 be the time interval between the i-th and

the (UX)st registration^ the time from the beginning* when

the counter is looked* to the first registration,, Tbs * i > 1

are independently distributed all with the same distribution,,

*Ve denote the time up to the (k+l}st registration by

Let I? be the number of registrations during time t 0 #e clear

ly have
•

•

*

„ m

‘ ,

' \

•

' • •

.
92

p.^t) = Plfl-k) == JflS^ < %)- <. t)

Let T k s i>l have the distribution function F , so that P(Tv < t)

sF(tj 0 vie write moreover F C t ) for the distribution function

of H

a 0 Let Fk

(t J-= P(Sk < t) then

(5 C 10 5 pk

(t) « F

k_x

(t)- F

k

(t) #

Since r and since and 5 are Independent

we have

t

**n+1(t} = JFn(t-z| dFjsj , ©

The characteristic function of the random variable N

is thus given by

qo * . » (Sell) P0 (t5,^^ e

lBic [Fk_1(t)-Fk

(t)]

= P0

Ct) * e

4sF

0

(t)

Thus since p @

Ctj P0

tt) « 1 p we obtain

*(*) a 1 (e4e -l& 2sis

($ 6 12j \lt! #

•

93

Henoe

(5 0 13J

{ S ^ X 0® £ — T_ tt (s) =

*

. =

(®la - 1; k*o

= F

e

(tl + f

9is\^lt ~x)dF(x

“ P

0

t*) * 9iS jVs

CBJdPU}.

0

In type 1 as well as in type Z counters the value of N is hounded

oo Vv

so that JL converges*, where V, » V- (tj is-the*k-th mom@nt 0

k&o * *

Thus w© may write

* 9? Yv v

^C©) °s X ^4i@j

kso *•*

and for © < X also

V8 >=[| §

is ...]

-L

isV« ~ C Y-^ *o* *$* *3 + . % ^00*1#

Henoe the constant t©m in the ^cpansion of ijr^(s) beoomes and

the aoeffiolent of (is)' 'becomes tYg-Y^)/2 0 From this ami (5 0 2*3j

we obtain equations for Y^ and Vg

64

& .14 K

%

VjCtj « P

0

(t) + JVj(t -x)dF(x) »

0

t

Vg(t) = 2V

x

(t) -F0(t) jV2 (t- x}4F(x) =; EjH8

) o

»Va begin with the discussion of counters of type i 0 From

at

(4 c .2j we see that F (t) ^ 1 - e where a > 0 ie the mean number

of events per unit of time 0 Further

Fit) « X- for t > ^ 5

F(t) ^ 0 for t < ?

sine© the counter remains looked during the time ^ after every

registration The first of the equations Cs-X^i then becomes

1~ •at for t x

{ 5 ^X 5 }

^ - 1 ^

, * ajV^(t - xje~a ^ s' 1 * .

tfe compare (s 0 l5) with the more general equation

for t > %

for t <. %

Alt

t

t\ 4- xio-ajx-?) fQr t > t a * at If HCt)<l- 6“6t then A(t) < V,(t) 0 If Hf.t) > 1 -

100

|

' “=B

<Lr than A(t) > Y^lt) 0 This Is certainly true tor 0 < t < v and can,

easily be ^iosm to hold for t y-w <, (a Xh'/ if it holds for t r^j < n* 0

95

at

rte put Alt) 7S * o „0 Than

i * o for fc < x

j !+&*£ ~

Bit \\ £i*S i8® 1- Lae” a ^

^ for t > it

An elementary calculation shows that

H( % J < 1 - e~

a* if OssO

Bit) > 1 - •'•* at if e« sfj

2

irzTSsf

Hence

E 2

A V« £L C5.,isi xdrf < VtJ i its? * sn+afT

It la also possible to obtain from (5 0 14) an exact expression

for Y|(tj0 H owever* this does not seem to be of great interest

since C 5c i$J shows that approximates V^C

1

1 very closely with

a bounded error whieli is small compared to Y^lt) anless a? is very

large c The exact expression for V-^(t) ie moreover very Involved

and hard to evaluate 0 H

For tha Marianos Bit) of H glvoa by B(t J » Yg(t) -£V^{t jj ^ Feller found the asymptotic expression

(5,17) B(t ) =

at

(l+a**)'

o(t)

' 96

We now put

00

( 5 0 18 ) •<

tla) s jVst dF(t) g f

k

(e)

o

« jVst dFk

(t

0

m

fi(3)ta jV1 (t)e"8t at

we have by Is,

v^t) = Zfcpjtj * nkCPv.1 it)-pk

(t)j„ s Fjt) A ksl ~ k=l “ A K k«o a J

and fey induction^ using the well-known multiplicative property of

the Laplace transform*)

ijjtaj a f

tt (8)[f( 8)3

k

00 - wher © f I s ) s JV*

0 * dF { t ) 0

o

Thus

Q© , e© co (5.1») *(#)* II jVSt

F

fc {t)dt s £ |

fe“

St4Fk

Jt) «

® 0 <0 “

q> a

We now proceed to dismiss counters of type 2 0 teution function' F(t) of Tj^must first b© obtained,,

1 f

0

<8 > ¥ I~rl s

j

*

The distriTo this purpose

we shall first obtain the distribution of the time T during which

the crnmtcr is lock@d c The probability that once the counter is

locked exactly v events will' prolong the locked time T is given

by 4?P» where p «s p"®*

8 qssl- e~

aT

$ since p fey i4 0 E) is the preferability that no event will occur during time t and q

ir the probability

97

th-M the time intervals between v successive events will all be smallC 1 ^ er than t 0 T*st now T be the time elapsed between the (1-lJ^sb

and the i-th ev©nt D The total looked time T0 provided exactly v

events prolong the looked time 0 is thiin given by

(5,20) 1 « f'

1 * I

C2J I** 1 T 0

The conditional probability 'tf(t) that an event will occur

during time t provided that t 5 t is then given by

(50 *V

Thus

*5.22 Ufsjs jV

o

&<*)« «| (l-e"at

)

st AO(tl - i. JL,[ 1 - e"?a+8 ) T

3

q

a<fs

Ltt now v event e prolong the locked time and write (t

000

( v * T < tjjvj for the probability that the locked time

will be at most t % « provided v events prolong the looked time.

From f§ 0 22) and C5 0 X8| w© see that

vt

? 9”Ss 4W ft) * [ttisjf 0

©

{1} ( v) Consider now v itself as a random variable 0 Then wCtjasP.lT oao T. •* ^ O

is the probability that the looked time T is at most t v c We

thus have

98

>"St drtlt) = pltaule)]** z-jsxcfi = Pi1 " af¥[1 ‘ «~ (a+3|T3}”1

„ ja»s ^e"

a<g

# ~ 8+aata+8 ^

Let now Gr(t) s P(T < t) then G(t) — W(t-*c) for % > %

and 0 for t < xn Hence

5o2s5 jV8t

0

«at

cw

•8*5 £^-8t je~ow dW(t-T) s Je“

eb dw(t ^ o

Ca an-*8* 8 )*

— I TT—H u, Ha4>a|^

a &e

The time between two successive registrations is composed

of the resolving time T and the time from the moment when the

counter is free to the next event 0 The distribution of the latter

eaft

j is by (4 0 2) l~e " and has the Laplace transform a/ (a+s) thus

f(ai - iV8t

* AfCt i - - ^ J s a ekpi«^s

whil® fQ (e

s

0 Substituting this into (s 0 19j yields

C 5 0

^

)

fiU) s

a Cs

|

a„ 6^^i

8 (a«e)
«

'

.

'
99

This is the Laplace transform i$ 0 ±8) of the function

1 ~ ©

-at for 0 < % < n

lo 0 25}

X- ®’ a£’ for % > %

Since is completely determined by its Laplace transform*

formula i$ 0 Z§)) gi^es the expected number of registrations during

time t in a counter of type £ 0 ^

A calculation similar to the on© leading^! 5 D 25) shows that

the Y&riEne© B(tj of the number of counted ©Vents ik by

C5o«6) B(») » V

g (t)- [V1

(t)3

aeaT(t-T)tl- e“

B' s U»a*sfe”2**

GEA.PT EH 6

The Fourier Analysis of Stochastic Processes ,, A tJu^i 4 A function in two variables is called raonotonoid if

f(t#V} ss gO^t' I - ) where g and h are two functions monotonia in t and t

7 in the same sense 0 rte now proveg

Theorem 6 q 1 0 let he a stochastics process with & monotonoid and

continuous covariance function g^* and E{x$) —* 0 then

For 0 < t ^ T we have the expansion

ns^m

(S 0 1) xt = l 0 l 0m 0 2* on exp[2irint/T]

* Qs-rn

wher e

T

°a ~ I

jxt «xpt-2rtn*/T] dt

f ii J This limit Is uniform in 0 < e < t <. T- s T T

Uli) %o,= 4 j f

au. exp[.2m^SSl]dt dt'

f 0 0

Ctv) If the process is Gaussian# then any finite set

of 0

jx*®n a^d °n

ra ^n affe Jointly normally distributed*.

To simplify the proof we put t s ^21 # y.r s n , Th@n f

goes from G to 27T as t goes from G to T and have to prove the

formula

{b - M ^

^ °in%

» °n s

iff J^e'

in%Ax *

We thus have to prove

uniformly

We have

$6„3J

How

7is~m

Putt ins

(6 0 4)

lim Sty^J ~y ]

£ =0

\ in every interval £ < T < E7T- e where

r

. M _ in«-

v x T Za n ° " ni

gff

_(s) i P „

;srj0

V

*y5

LzA -m

’ >j

Ax'

s e

•ima i_ ei(2m+l}a e~ima m e

i(m+l)a

1 «= @

la 1 08 e

- e

ima _ eiCm*l)a_ ^ Aima

iwi OTi nnr 9 *e i wnw v i Wp £U ~ 008 0 )

oos ma - Qoe

gm+1

s ' s t + h we thus have

y

(»)

E?r

*.T-T

I yT*h

sin

&

sin §

£h

and

1 02

.sq-t ztr.T

(6 0 55 e[ 4

k) ~ yT

3

2

L i

r -r

3in^ih 8 iaS¥ik

Vh,^k —pk1 —T

-V" dh dk

* sin sin #

35T I 0' 25T J "''tpt+h „

a. h \n g'

, Sm^l: —%

ah . c

h vx

By well-known theorems on the Dirichlet integral l ^ aj we have

uniformly in £ < *c < 27T-& is > 0) p

.

(.6.6) lim

m~$>© 4Tf" 1 L

a^h»

ain 8ln Sgik

C33» V

-IT

v*k . TT”’ . k sin «g sin «g

dh dk

ss 1 im I o’—, _ . v. sa~»oo^ i.

ain - T dh

sin 4

- a

From C

6

C 5| and id 0 6) it follows that

(6.7) lim £ [y* m 5 - y )

2 , 0 or l 0 i.m. y^ a y

uniformly in s < ? < 27T- £ for every e > 0 0

[l5] For the double Dirichlet integral see Hobsons "The theory

of Emotions of a real variable and the theory of Fourier

series"* voX G II* pp 0 7Qo-9 c
' :

i

-

,

' r

? W'i , J, . >

' '•

,

'

,

)
105

• v

• is completes the proof of the first two statements of theorem

, ii - is easily obtained by an elementary computation while

7 } follows from the representation of the Fourier coefficients

«s limits of Riamann sums*

% ft ffrigonomet ri o exTmng i on of t he ?„ R» ?„

as an example we shall represent the F0 RC P0 & trigono-

,;. IjIc series with fandom coefficient s 0 The o ova?ianoe fuiiv t i o n • of the F 0R 0 P0 Is o minSt^t^J [see formula C3 0 4)3& this is a monomi© and non-dsereasing function of t and t* so that theorem 6 0 i

i applicable 0

.In this case (that 1 b the real and imaginary part of o

an )

; normally distributed with mean aero and we have

id0B) ElF^,S - ^ o min{ts»t'“ } =* ') s o'mlnfo*?

I hue

2Bp?

Mon

^m 'i ss j" ) expt- InT-loT^ 3 d-t d^! ****** <f ©

4 l 1» ar gif

^ p* « -2w| f f fV exp(~ixiT-imT* idT* fftnp* - f I !t axpC^iOT-iaT* 14*?

4:^^ L

*f

" J 0* " T

' J

j

ar nssmstO we < Fi

For m^O we have

jfflh

1

‘ 3

fhl3 gives

37T

Z

27f oT oH

{,

5

0 9 „3 } SCo0

Om) » £

J~" 5

1 3 OT" for sa

( 6 0 9 0 4 ) E^ sm°.jnS

*

o2

2/'m2

for m 0 *

For 0 wo obtain from C6*9 a g)

U0 9 0 5) EC o„q | xg q m*

f{1 _~m3T

4 ir^ran

If we write

we have - o„ * c „$> *> ss i(o - o j for n > 0 * a & n « n* n xx -a? o

and from this and th© formulae (6 f,1 0 l3 - (6.0 9 0 5) we find

E(&o) a OT/3

E(a0an j a -cT/2TT*n

Eia^a^j s 0 for is^n, n^O, n^O

E(a

2

) — eT/2n^n2

iSa 10j <

] W“ 0 for n £ 0

E(a b^) s ~cT/27Tm

= 3oT/2r3

n

8

= oT/rr ran for ra^fc n*

n in/ * <

'

rfe shall now estimate eTx ~ £j

m H ** for a fixed a wher®

, ax a

xim * -s a ,S a 003 arfit 21 b sin Efi^i % ® n«l n (F Ra1

11 T

oo

n*«H4i

tfe havOg using (8C X0)

oo ^

E[x

t

- x^a?

]

2

s ZL E(a2

)ooa

2

2lfS£ « 21 Ei'e^ela** 2ff£S

" " ' * #a*W A

00 00

2 i„. S{Vbk

)Bln 2jr~i sin 2u

05 05 . antni2

f

= sh £ V jf - ia

fl* 2

^

n

4

! L-*

L n«mW ~ si

Expanding the function 7T- a Into a Courier series we get

nr-o » zE SiS_SS,

n*l a y

heno e

055 Z

sin^ na

n

si , IlTT- s'j - S SiSLM m f( a),

a n*i n

Differentiating this equation we have

m * i ain(mv|ja

f « ri S=> <—BBaegs -^xg^*xrr’“.fc> ^ (a) - - i - Haas n * ~ „ _ 4 2

^

2 8lnf ^

and therefor®

osT sin na

L.

~

Hence’

* A ,2.

2 . 1 f 8tn(cH-2) t dt _ tt _ j 8ln(2a-»l)Y dv

2 2 1 sin | “2 -

E£zt - 4“>]2 s S!fi f -i- , ar^r-sin^IjY 1

" ’ 1*

1* fan* ‘- s J • S

“ "

' j

Thus far values of t not t.vo close to 0 or T ff xi J is a good

approximation to x^ 0

Another and perhaps are useful formula may he obtained by

deriving the following expansion

t T T

[oosSi^i-l] * <3 a8ln2^. J

o
V

J

, / .. < V. •

. *

'

\ ' !

•

.

'

v ••• v : 3 SJS. S . ~A

• •

.

V

'

.

,

\

-

.

.

.

.
10?

In this expansion the and b^ are independently and normally

distributed variables with mean zero and variances * the

aAs

anand bnar© also independent of e Th© right side converge^

moreover^ uniformly in the mean to the left side 0 The proof can

be obtained by first applying theorem s 0 l to the stochastic process ac^-^Xij and determining the Fourier coefficients end their

variances 0 It is then seen that the Fourier expansion thus obtained

oo converges also l 0 i 0

ra 0 for tsl and thus - Z && o proof w' n«X

Is rather laborious but elementary and is therefore omitted,

Thus writing x^j 33 &

a

w@ have

m

<6oU) *t “ a

o t * & {»a

toos^-13 b

n

where

0

2 *

a

o

0S • of * «§

cT

n 2lPn‘

. Vj'Vj " 0 1#J ’ Vi 3 0

Except for the constant term* - £a . this Is essentially the j n»i n-> expansion discovered by PaXey and Wiener,;[16]

[163 Fourier transforms in the complex domain^ p 0 14? 0

108

S 0 Stationary processes . f© now return to the general theory and consider stationary

processes 0

exist a o We then have cr^.* =5 where R(xJ is an ©¥@n function Of T o We she.ll also consider a slightly more general class of

processes* called guasistationary processes 0 A process x^ is

aid to he quasiatationary if E[x^] is independent of t and if its

covariance function exists and is given hy cr^t* zz R{t~ £'} where

R|x) is an even function of t 4

we assume now that R(t) is continuous at the point x^O

ad show that R(tJ is then continuous everywhere 0 If R|tJ is continuous at tsO then we have lim E(xt+T -x

t

) 2 s 0 „ From the

definition of Rlxj w© see that

h~>©

lim iR{%+h) -R(x)J = HmE[’(x_ . -x }x 0

.-®*o h^o c,u y

since

0
• '

...

•

. . .

. : . .. , .

‘ c C

.

. >

’

'

•

-

'

. m-i m

•

: I . 1® I
109

wle next introduce the following definition 3 A function f(t)

is said to be positive definite if

(a) f(t) is continuous end bounded on the real axis*

(b) f(t) is Bicrmitianj, that is £(~t) s f( t fg

(c) for any positive integer m and any real numbers

\9 z

z

»ooob zm

and any complex numbers ugS?

0009 ^ we have

From the preceding it is clear that H(t) satisfies conditions

{a} and (b) since H(t) is real and ev©n0 ue have only to prove

that

m m

h-X ,?t k*l A f ^ 2h" z

k^ u

huk ~ 0 0

m m

3 - Z Z B(th

- vv\ - 0 0

We have

s =

1 f Vk E5 vys^fe

m m

S ^ s{{| v**th)(f

function^

uo

According to a theorem of S 0 Boolmor^7 ^ every positive

definite function fit} may he represented in the form

* oo ^ fit) a J

e

ita dV(a)

~0D

where Via) ie a hounded non^decreasing funotion a Thus w© have

I

6

0 l5) R(t)

*co = S e

1 *®

~oo

dgl <o)

where gl©} is a bounded and non-decreasing function 0 we may take

g(-oo) - 0 D Then gloo) s& R##and glct}/R(d) could therefor© be

defined as a distribution funotlon 0 It will however simplify our

formulae If we determine gla) so that

«(a)= &Latl ± .sUdl. . 2

Binoe Hit} ss R{-tj.we have

Rlt

'00. J

.it©

&«(©) S J

6

too

3 -it©

4-00 „ s - /

a

itS) dg(-o)

-0©

•*•00 =s j

4 d£g(CO )-g(~©) ] ; — 00

tl?3 S c Bochner9 Yorlesungen ubar Fouriersch® Int©grales p 0 7Q & Sats 23 0

Ill

end ainoe the function gUo) is unions if g( = ®! s 0

g(o5 s £L&l~±JSk$zl9 w© must have g{»5 s g{oo } « g(~$) H

end. fa? osO, g(co) = 2g{0) and g{®) * g(O) = g{0) » g(=a).

It Is further well known that

, -2

e it ffl- , g( K )- g(0 ) = i

nm

ii5.JR(t)_=i dt 0

-

//e may also writ©

R( t i ss ®

e

it®A«*• „- n®

fig!®}

OO 00

3 joos tjD dgiCDjs J

00 3 to d[g(o)-g{-fl)}]

—00 0

^00 - Joos t© dF(©5

0

7

where

[

( 6 oi6 ) <

fC®j - gl©)-gU©)ss

Fjce ) S g(co } S R(0)

FiOj s 0 ,

Further

?{») g(oo) - sI-cd) = 11m

T

so that

Fla) 5 |

JrU) at c G

It may also be remarked that to every positive definite

function R(t) we may construct a Gaussian process with R(^) as

covariance function* i'his can he done by defining the distribution

of xt ? 9 0 08 n to be a multivariate Gaussian distribution with

1 n

covariance matrix JjRtt^-tj) |{ 0 Since R{t) is positive definite auoh a distribution always exist

s

0 It is then easy to

verify that the family of distribution functions so defined satisfies the consistency conditions of chapter X c Combining this with

the result of Boohner we obtain

Theorem 6 Q S 0 The function R(t) is the covariance function of a

quasistationary process If and only if it is the Fourier transform

of a bounded non-decreasing function*

4 0 The mean ergodio theorem,

V/e shall conclude this chapter with a proof of the mean

orgodic theorem 0

Theorem 5 0 5 0 (Mean ergodio theorem \ g Bet be a quasi stationary

process with continuous covariance function R(t) and mean value zero

[16] This theorem is due to J 0 v c Neumann g Proo 0 Hat, Aoad 0 Soi e9

vol. 1811932), pp. 70-8S.
•' c .. . »< I O , a r tl

'

;

.

K

1
113

Then

{ 6e l7

)

1 o 1 O ^ ft T ~*$*G0

fit

o

dt ~ a

x

where is a random variable with variance g(X+)-g(X-J and mean

z@ro and where E(a^a ) «• 0 for X4* -p. ft The function g(Xj 1 b

defined hy ld 0 l5} 0 w© first prove the following lemma

Lemma 6 0 l g For any 0 < * < % # T > 1 aad for ©V9ry £

dx- [g a^j-g(\-|)]

I 0

«. [g(X*£} - g(X* )*g(-X*cj-g(-X4 1

3

g'L g(^«+ £ J 4

£)3* «Rr[g(ao')- £(~gd }]

Proofs *ve put

T

-» f &{t-T

1 ! e

ai’

^ 3! ^GO

d-v ?» i P fexp[iX(t-T}*lo)| t«Tj J&gCaj) d*s

* j J

Q-CO

:s J 1

* J

2

* J

3

+ h

* S

z

* *T3

where

i

114

x” r iMt-^3 | 1 if* eft

Is $ , 18

o

*. i f

dgu-'dir | Jjss

|>

j “t)

j

a""

4,05 ' '"^dg!®)^

X f*iXSt”'E} f _i©(t“?)

fj 8

j

" dg(»)&Tj J

s

a f |

®~*®it *T

^dglo)d,f

<&> £

r

: ~ i ! o

ie)U’';)

T ^

Q

| Jr> ^ i

I 3 2

T

9

imxsS T

ioC t^v)

$

These Integrals converge absolutely,, Henoe we may Interchange the

order of integration whenever necessary,, In this manner w# obtain

\h ! *

?X

*T

c

t

a^| ^ ji , . T

i(<D*X)t

so that

( 6 * 18 )

Similarly

U0 l8&)

< JL [gCoo ) -g(-cx) 3 ] - £T

©T

«=» gtAT? o

tf© have

S® - 1 i 1*1

I


o

using this inequality we obtain from

%

f- jexp[iOv*s)Mt-T)]dT igiio)

o J

|6 C 19) |I 2 ] < l[g(x+e - g(X+ +g(-X+e - g(-X+ ]

\

and similarly

(6 c X9a) jJg J < ^~[glX+e) - gDw4-} -s-gi-X^s) - g(~X*) ]

Since

|[g(X«£)-g(x-£)] = A J

JVgU) fiT

tj/4 o

we have0 using

|I3 »|[«JX<-|)-g(X-£53| a

|y j* J

[e i(' A’ 0 ^ ~ ijdg(ffl) 4t

T

* f ~ 8T8 ^

J J

|X*o)j (t-T) dg(eo) d*s

< ||>CgfX*|>)~

r

Sine®

lift

X > I and sine© g(x) is non- decreasing it is seen easily that

(

6

20} j Xg - - s(X-£)J | - gfX-©} ]

Similarly we obtain

(6 o 80a) e<T«»

”2®'

t) X*c )

Lemma 6 0 X then follows easily from C6 0 ls), (6 0 X8a}* i6 0 I9)£ (6 o X9a) 0

(6 o 20j» (6 o20&) o

Corollary 1 to lemma 6 0 1 0 T

(6 0 2l) Ilia U f e^lW’H(t-T}d-E * gCX*)-g(X”) * X ~*»00 1 %

Corollary 2 to lemma 6«l ft * x*t* »mmrm I r > w **»* J rea^>a»«g^«»r* Tifci^ia^KJW UJiTTMB wMiT^er^r lfcMiaW

( 6C 22 ) T» rr gfo* ) - g(X- ) 0

'Proofs may always write the double integral so that T

f > X

so that lemma 6 0 1 is applicable and corollary 2 followB easily

sine a e is arbitrary0

II?

In tha proof of the mean ©rgodic theorem w® shall operate

/ with complex random va3:*i&bXes 0 If zs^ij is a complex random

variable with mean zero we shall define

(

6

0 SS) s Et zz) S o| 0*

where zgx- ij is the complex eonjy^gate to z c A sequence

{sa } s {xn

4-i'yn } of complex random variables converges if both

{x^} and {yn} converge,. From lemma l c 8 it follows that {za}

converges l 0 i c m 0 if and only if B[tzn

-

z

m ) zm) ] is arbitrarily

email for sufficiently large n* a 0

T

To show that X^a A at converges in the mean we

o

consider

L

n>

* EfUj- X^, ) (Xj- Ij,)]

T T

A

J j

' T

2

<3 %

o 9 V T

R(t-V )4t dt' «• -jiff *R(t-t')dt &t

#

£

ff'

ns

All thro© integrals converge to the same limit by (6 0 22) 0 Thus

l 0 i 0 m0 X « a* exists 0 Moreover * by lemma X 0 7 and (6 0 22)

]£ <*=£• GO

lim

T~3»gq

of ~ g(X-) « i

For we further have

T T

E(«\a ) a lii i f fexpUxUigit* )R(t-t' )dt 4t" A £ T-#>oe m2 ^ ^

A 0 ©

^ lim i jy.cx-ni.3ii x

fe-ip.Ct-v ) S( t. t*

j dfc 4t

' o 'a"

The second integral converges by corollary 1 of lemma.

0 1 to

g(p.*) - g(p,-) uniformly in t 0 Thus'

r J

ECa^a ) -3 lim i ^

) -gC^-) ]dt * tj(T)

T«^co ^ 0

where lim ji(T) ss. 0 0 It easily follows that E(au a } ss 0 0

’ T A A

Theorem 8~4 C Let be any quasistationary. process with

function R{^) and let gC©) be defined by {© 0 l5) 0 Further let

Xg^ooo ba the discontinuities of g(s>) and

x- l 0 l 0 m0 1 jzt

®at dt

T-»oo T

q

j *

then

34 - JL* 4K %

' 3*3.

it

j

" M2?* ^ la a quasi.stationary process such that

*g

l oi 0 m o i •f'V^.e"^

3

' fit 3 0

£<-*»<& fj* *

:?a* ail rtal nmfe«8 u 0

ls?rtoafs She n

m

v *x « *3

•K>5 4< oonvargas in the mean

* ‘ * Ism >

«»i> v*.o n

X t «(>..) - g<X,- >3

Jsffi

a: .’ &* tt-O 3*1

} - &i\r) 1 < s for all

‘;

‘ ^

i f a®, f©r "jl SS Aa C 1 “ 1^> 189 oO«J

rP.JCft»#S& X ** M ",

? 0 ©tharwlse 9

5 hioh 'i ’O^- &

INDEX 120

BOCHNER , S^llOo

Brownian motion^ 32* 34s 4?o

Characteristic function; definition, 75? of increment of differential process, 8? I of number of registrations, 92;

logarithm of, 76©

Conditional probability, 28 0

Confidence region for FOURIER coefficients, 63o

Convergence in probability, 2 0

Convergence in the mean, 11 0

Convergence of distribution functions; at points of continuity, 4

S

• HELLY-BRAY theorem, 78; KELLY 8 S theorem, 5 1 LEVY9 S continuity

theorem, 7&o

/ Counters, 91 ffoo

Covariance, 10

©

Covariance function, 10

o

CRiiMER, H.oa 11, 77* 76o

Derivative of stochastic process, l4o

Differentiable l o iomO0 15o

Differential processes, 34? of second order, 84l modified, 85 o Distribution functions,! ff c ? of number of registrations, 92;

of registration times, 9^, 960

D00B p Jo Lo, CIV), 53o

Estimation of parameters, 55 ffoo

Expectation, 10

o

FELLER , Wo, (IV)

,

91* 95o

Filter effects, 85 ffoo

121

FOURIER analysis of stochastic processes* 100 ff oc, FOURIER transform,, ?7 o Fr 1 c 1 1 onal effe ct s* 4? o Fundamental Random Process (F 0 R©Po)s 32 ff e ; covariance function* 89s

definition 37? estimation of parameters* 55 ff 0 ; integral* 46*

properties* 37 ff*J trigonometric expansion* 103 ffo* with

mean value function* 57

o

Gaus s i an proc e s s* 46 * 89 » 100 * 112 <> Generalised differential process^ 72 ff»«

Generalised POISSON distribution;, 75©

Generalised second derivative* 15 0

HAHN 9 H c* 34 o HELLT-BRAY theorem* 760

HELLY 9 S theorera* 5 o Hermit! an function, 109

o

HOBSON* So

^

102

o

Independent stochastic processes* 27

o

Infinitely divisible law (i 0 d e l 0 )

|) 76 ff*o

Input process* 85

o

Integrate!© l©lom®*24©

Integration of stochastic processes* 17; of on® process with

respect to another* 2? 0

KHIOTCHINE, Aos

, 360

K0LM0G0R0FF, A„, 28 „ LAFItiGE transfortn

8 960

122

LEVY 9 ? 0»{IV) 9 13, ?6, Si*

LEVY 1 S c ont inulty the orem, ?6 0 Limit in the mean {

l

0 ioM0 )„ llo

LIKLEBERG condition* 35-37*

Locked time* 97

©

LQEVE* M 0& (IV)* 14* 76o

LUKA GS * So, (IV) o MARKOFF processes* 32* 34® 55

o

Mathematical expectation^ 10o

Maximum of process on interval* 18

©

Mean ergodic theorem* 112 ff 00

Mesh*

8

0

Minimum of process on intervalfi 18 0

Modified differential processes, 85 ff*o

Modulus % of mesh* 8; of subdivision* 17

o

Monotonid. 100 ©

1 e MpRANDA, P 0

„

| IV)

Netting, 8 . ORNSTEIN, L„ 49 o OKNSTEIH<=UHLSNBECR process (0 0 Uo P„), 47 ff«; covariance function,

51

1

estimation of parameters, 64» integral of, 51<>

Output process, 8f>«

PALES’, R<- B u A, Co, 107.

plim, 2 o POISSON distribution, 74, 7$,

123

POISSON process, 91o 4 • Positive definite function, 109 < Probability limit of a sequence of random variables, 2©

Quas 1st at ionary processes, 108 ff® 0 Random variables,! ff oe ; complex valued, 11? I convergence of

sequence, 2; non-negatlv^ 11; probability limit of sequence,

2; sequences of, 2 0 Registration of stochastic processes, 85 ffoo

RUBIN, Ho, (III)

SAVaGE, Lo, (IV)

Shot, 73

o

SLUTSHY, E., 3o

Spectral analysis, 100 ff 0 © , Stationary processes, 49, 89, 90, 108 o Stochastic differential equation, 53o

Stochastic processes; continuous, 10; differentiable, 14;

derivative of, 14; generalized second derivative, 15;

integration of, 1?; maximum of, 18; minimum of, 18; of a

continuous parameter, 10; of second order, 15; registration

of, 85 1 stationary, 49; strongly continuous, 22; uniformly

continuous, 28 0

Strong continuity, l8o

UHLENBECK, G & E 0, 49©

« VON NEUMANN, J„, 112 0

WIENER, N 06 107 o

- WIDDER, D, V„, 5.

THE NATIONAL BUREAU OF STANDARDS

Functions and Activities

The National Bureau of Standards is the principal agency of

the Federal Government for fundamental and applied research in physics, mathematics, chemistry, and engineering. Its activities range

from the determination of physical constants and properties of materials, the development and maintenance of the national standards

of measurement in the physical sciences, and the development of

methods and instruments of measurement, to the development of special

devices for the military and civilian agencies of the Government.

The work includes basic and applied research, development, engineering, instrumentation, testing, evaluation, calibration services,

and various scientific and technical advisory services. A major

portion of the NBS work is performed for other government agencies,

particularly the Department of Defense and the Atomic Energy Commission. The functions of the National Bureau of Standards are set

forth in the Act of Congress, March 3, 1901, as amended by Congress

in Public Law 619, 1950. The scope of activities is suggested in

the listing of divisions and sections on the inside of the front

cover.

Reports and Publications

The results of the Bureau's work take the form of either actual

equipment and devices or published papers and reports. Reports are

issued to the sponsoring agency of a particular project or program.

Published papers appear either in the Bureau's own series of publications or in the journals of professional and scientific societies.

The Bureau itself publishes three monthly periodicals, available

from the Government Printing Off ice : the Journal of Research, which

presents complete papers reporting technical investigations; the

Technical News Bulletin, which presents summary and preliminary re- ports on work in progress; and Basic Radio Propagation Predictions,

which provides data for determining the best frequencies to use for

radio communications throughout the world. There are also five

series of i nonperiodical publications: the Applied Mathematics Series, Circulars, Handbooks, Building Materials and Structures Reports, and Miscellaneous Publications.

Information on the Bureau's publications can be found in

NBS Circular 460, Publications of the National Bureau of Standards

($1.00). Information on calibration services and fees can be found

in NBS Circular 483, Testing by the National Bureau of Standards

(25 cents). Both are available from the Government Printing Of fice

.

Inquiries regarding the Bureau’s reports and publications should be

addressed to the Office of Scientific Publications „ National Bureau

of Standards, Washington 25, D. C.