aarjaneiro/planetmath huggingface dataset:
odds ratio

odds ratio

Suppose the probability of an event

A

is not 1. The

odds

of the event

A

is the ratio

odds

⁡

(

A

)

:=

P

⁢

(

A

)

P

⁢

(

A

c

)

=

P

⁢

(

A

)

1

-

P

⁢

(

A

)

.

For example, if the odds of landing a head in a coin flip is 2, then the probability of landing a head is twice
as likely as that of landing a tail.

In a

2

×

2

contingency table

, with a

dichotomous

explanatory variable

X

having levels 1 and 2, and a binary response variable

Y

with success and failure as two possible levels of outcomes:

Title

odds ratio

Canonical

name

OddsRatio

Date of creation

2013-03-22 14:41:07

Last modified on

2013-03-22 14:41:07

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

10

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62H20

Classification

msc 62H17

Defines

odds

Defines

log-odds ratio

\@unrecurse

Generated on Sat Feb 10 12:05:17 2018 by

LaTeXML
covariance matrix

covariance matrix

Let

𝐗

=

(

X

1

,

…

,

X

n

)

T

be a

random vector

. Then the

covariance matrix

of

𝐗

, denoted by

𝐂𝐨𝐯

⁢

(

𝐗

)

, is

{

C

⁢

o

⁢

v

⁢

(

X

i

,

X

j

)

}

. The

diagonals

of

𝐂𝐨𝐯

⁢

(

𝐗

)

are

C

⁢

o

⁢

v

⁢

(

X

i

,

X

i

)

=

V

⁢

a

⁢

r

⁢

[

X

i

]

. In matrix notation,

𝐂𝐨𝐯

⁢

(

𝐗

)

=

(

V

⁢

a

⁢

r

⁢

[

X

1

]

⋯

C

⁢

o

⁢

v

⁢

(

X

1

,

X

n

)

⋮

⋮

C

⁢

o

⁢

v

⁢

(

X

n

,

X

1

)

⋯

V

⁢

a

⁢

r

⁢

[

X

n

]

)

.

It is easily seen that

𝐂𝐨𝐯

⁢

(

𝐗

)

=

𝐕𝐚𝐫

⁢

[

𝐗

]

via

(

E

⁢

[

X

1

2

]

-

E

⁢

[

X

1

]

2

⋯

E

⁢

[

X

1

⁢

X

n

]

-

E

⁢

[

X

1

]

⁢

E

⁢

[

X

n

]

⋮

⋮

E

⁢

[

X

n

⁢

X

1

]

-

E

⁢

[

X

n

]

⁢

E

⁢

[

X

1

]

⋯

E

⁢

[

X

n

2

]

-

E

⁢

[

X

n

]

2

)

=

𝐄

⁢

[

(

𝐗

-

𝐄

⁢

[

𝐗

]

)

⁢

(

𝐗

-

𝐄

⁢

[

𝐗

]

)

𝐓

]

.

The covariance matrix is

symmetric

and if the

X

i

’s are

independent

,

identically distributed

(iid) with

variance

𝝈

2

, then

𝐂𝐨𝐯

⁢

(

𝐗

)

=

𝝈

2

⁢

𝐈

.

Title

covariance matrix

Canonical

name

CovarianceMatrix

Date of creation

2013-03-22 14:27:23

Last modified on

2013-03-22 14:27:23

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

8

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62H99

Synonym

variance covariance matrix

Generated on Sat Feb 10 12:05:19 2018 by

LaTeXML
Fisher information matrix

Fisher information matrix

Given a

statistical model

{

f

𝐗

(

𝒙

∣

𝜽

)

}

of a

random vector

X

, the

Fisher information matrix

,

I

, is the

variance

of the

score function

U

. So,

I

=

Var

⁡

[

U

]

.

If there is only one parameter involved, then

I

is simply called the

Fisher information

or

information

of

f

𝐗

(

𝒙

∣

θ

)

.

Remarks

•

If

f

𝐗

(

𝒙

∣

𝜽

)

belongs to the

exponential family

,

I

=

E

⁡

[

U

T

⁢

U

]

. Furthermore, with some regularity conditions imposed, we have

I

=

-

E

⁡

[

∂

⁡

U

∂

⁡

𝜽

]

.

•

As an example, the

normal distribution

,

N

⁢

(

μ

,

σ

2

)

, belongs to the exponential family and its

log-likelihood function

ℓ

(

𝜽

∣

x

)

is

-

1

2

⁢

ln

⁡

(

2

⁢

π

⁢

σ

2

)

-

(

x

-

μ

)

2

2

⁢

σ

2

,

where

𝜽

=

(

μ

,

σ

2

)

. Then the score function

U

⁢

(

𝜽

)

is given by

(

∂

⁡

ℓ

∂

⁡

μ

,

∂

⁡

ℓ

∂

⁡

σ

2

)

=

(

x

-

μ

σ

2

,

(

x

-

μ

)

2

2

⁢

σ

4

-

1

2

⁢

σ

2

)

.

Taking the derivative with respect to

𝜽

, we have

∂

⁡

U

∂

⁡

𝜽

=

(

∂

⁡

U

1

∂

⁡

μ

∂

⁡

U

2

∂

⁡

μ

∂

⁡

U

1

∂

⁡

σ

2

∂

⁡

U

2

∂

⁡

σ

2

)

=

(

-

1

σ

2

-

x

-

μ

σ

4

-

x

-

μ

σ

4

1

2

⁢

σ

4

-

(

x

-

μ

)

2

σ

6

)

.

Therefore, the Fisher information matrix

I

is

-

E

⁡

[

∂

⁡

U

∂

⁡

𝜽

]

=

1

2

⁢

σ

4

⁢

(

2

⁢

σ

2

0

0

-

1

)

.

•

Now, in

linear regression model

with constant variance

σ

2

, it can be shown that the Fisher information matrix

I

is

1

σ

2

⁢

𝐗

T

⁢

𝐗

,

where

X

is the design matrix of the regression model.

•

In general, the Fisher information meansures how much “information” is known about a parameter

θ

. If

T

is an

unbiased estimator

of

θ

, it can be shown that

Var

⁡

[

T

⁢

(

X

)

]

≥

1

I

⁢

(

θ

)

This is known as the

Cramer-Rao inequality

, and the number

1

/

I

⁢

(

θ

)

is known as the

Cramer-Rao lower bound

. The smaller the variance of the

estimate

of

θ

, the more information we have on

θ

. If there is more than one parameter, the above can be generalized by saying that

Var

⁡

[

T

⁢

(

X

)

]

-

I

⁢

(

𝜽

)

-

1

is

positive semidefinite

, where

I

is the Fisher information matrix.

Title

Fisher information matrix

Canonical name

FisherInformationMatrix

Date of creation

2013-03-22 14:30:15

Last modified on

2013-03-22 14:30:15

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

14

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62H99

Classification

msc 62B10

Classification

msc 62A01

Synonym

information matrix

Defines

Fisher information

Defines

information

Defines

Cramer-Rao inequality

Defines

Cramer-Rao lower bound

Generated on Sat Feb 10 12:05:21 2018 by

LaTeXML
random vector

random vector

A

random vector

is a

finite-dimensional

formal vector of

random variables

. The random vector can be written either as a
column or row of random variables, depending on its

context

and use.
So if

X

1

,

X

2

,

…

,

X

n

are random variables, then

𝐗

=

(

X

1

X

2

⋮

X

n

)

=

(

X

1

,

X

2

,

…

,

X

n

)

T

is a
random (column) vector. Similarly, one defines a

random
matrix

to be a formal matrix whose entries are all random
variables. The size of a random vector and the size of a
random matrix are assumed to be finite fixed

constants

.

The

distribution

of a random vector

𝐗

=

(

X

1

,

X

2

,

…

,

X

n

)

is defined to be the

joint
distribution

of its

coordinates

X

1

,

…

,

X

n

:

F

𝐗

⁢

(

𝐱

)

:=

F

X

1

,

…

,

X

n

⁢

(

x

1

,

…

,

x

n

)

.

Similarly, the

distribution of a random matrix

is the joint
distribution of its matrix components.

Let

𝐗

=

(

X

1

,

X

2

,

…

,

X

n

)

be a random vector. If

E

⁡

[

X

i

]

exists (

<

∞

) for each

i

, then the

expectation

of

X

, called the

mean vector

and denoted by

𝐄

⁢

[

𝐗

]

, is defined to be:

𝐄

⁢

[

𝐗

]

:=

(

E

⁡

[

X

1

]

,

E

⁡

[

X

2

]

,

…

,

E

⁡

[

X

n

]

)

.

Clearly

𝐄

⁢

[

𝐗

]

T

=

𝐄

⁢

[

𝐗

T

]

. The
expectation of a random matrix is similarly defined. Note that the
definitions of expectations can also be defined via measure theory. Then,
using Fubini’s

Theorem

, one can show that the two sets of definitions coincide.

Again, let

𝐗

=

(

X

1

,

X

2

,

…

,

X

n

)

T

be a random vector.
If

𝝁

=

𝐄

⁢

[

𝐗

]

is defined and

E

⁡

[

X

i

⁢

X

j

]

are defined for all

1

≤

i

,

j

≤

n

, then the

variance

of

X

, denoted by

𝐕𝐚𝐫

⁢

[

𝐗

]

, is
defined to be:

𝐕𝐚𝐫

⁢

[

𝐗

]

:=

𝐄

⁢

[

(

𝐗

-

𝝁

)

⁢

(

𝐗

-

𝝁

)

T

]

.

It is not hard to see that

𝐕𝐚𝐫

⁢

[

𝐗

]

is an

n

×

n

symmetric matrix

and it is equal to the

covariance matrix

of the

X

i

’s.

:

1.

If

X

is an

n

-dimensional random vector with

A

a

m

×

n

constant matrix and

𝜶

an

m

-dimensional constant vector, then

𝐄

⁢

[

𝐀𝐗

+

𝜶

]

=

𝐀𝐄

⁢

[

𝐗

]

+

𝜶

.

2.

Same set up as above. Then

𝐕𝐚𝐫

⁢

[

𝐀𝐗

+

𝜶

]

=

𝐀𝐕𝐚𝐫

⁢

[

𝐗

]

⁢

𝐀

T

.

If the

X

i

’s are

iid

(

independent identically distributed

), with variance

𝝈

2

, then

𝐕𝐚𝐫

⁢

[

𝐀𝐗

+

𝜶

]

=

𝝈

2

⁢

𝐀𝐀

T

.

3.

Let

𝐗

be an

n

-dimensional random vector with

𝝁

=

𝐄

⁢

[

𝐗

]

,

𝚺

=

𝐕𝐚𝐫

⁢

[

𝐗

]

.

𝐀

is an

n

×

n

constant matrix. Then

𝐄

⁢

[

𝐗

T

⁢

𝐀𝐗

]

=

tr

⁡

(

𝐀

⁢

𝚺

)

+

𝝁

T

⁢

𝐀

⁢

𝝁

.

Title

random vector

Canonical name

RandomVector

Date of creation

2013-03-22 14:27:20

Last modified on

2013-03-22 14:27:20

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

15

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62H99

Classification

msc 15A52

Defines

random matrix

Defines

distribution of a random vector

Defines

distribution of a random matrix

Defines

mean vector

Generated on Sat Feb 10 12:05:24 2018 by

LaTeXML
Cholesky decomposition

Cholesky decomposition

1

Cholesky Decomposition

A

symmetric

and

positive definite matrix

can be efficiently decomposed into a lower and

upper triangular matrix

. For a matrix of any type, this is achieved by the

LU decomposition

which factorizes

A

=

L

⁢

U

. If

A

satisfies the above criteria, one can decompose more efficiently into

A

=

L

⁢

L

T

where

L

is a

lower triangular matrix

with

positive

diagonal

elements.

L

is called the

Cholesky triangle

.

To solve

A

⁢

x

=

b

, one solves first

L

⁢

y

=

b

for

y

, and then

L

T

⁢

x

=

y

for

x

.

A variant of the

Cholesky decomposition

is the form

A

=

R

T

⁢

R

, where

R

is upper triangular.

Cholesky decomposition is often used to solve the

normal equations

in

linear least squares

problems; they give

A

T

⁢

A

⁢

x

=

A

T

⁢

b

, in which

A

T

⁢

A

is symmetric and

positive definite

.

To derive

A

=

L

⁢

L

T

, we simply equate coefficients on both sides of the equation:

[

a

11

a

12

⋯

a

1

⁢

n

a

21

a

22

⋯

a

2

⁢

n

a

31

a

32

⋯

a

3

⁢

n

⋮

⋮

⋱

⋮

a

n

⁢

1

a

n

⁢

2

⋯

a

n

⁢

n

]

=

[

l

11

0

⋯

0

l

21

l

22

⋯

0

⋮

⋮

⋱

⋮

l

n

⁢

1

l

n

⁢

2

⋯

l

n

⁢

n

]

⁢

[

l

11

l

21

⋯

l

n

⁢

1

0

l

22

⋯

l

n

⁢

2

⋮

⋮

⋱

⋮

0

0

⋯

l

n

⁢

n

]

Solving for the unknowns (the nonzero

l

j

⁢

i

s), for

i

=

1

,

⋯

,

n

and

j

=

i

-

1

,

…

,

n

, we get:

l

i

⁢

i

=

(

a

i

⁢

i

-

∑

k

=

1

i

-

1

l

i

⁢

k

2

)

l

j

⁢

i

=

(

a

j

⁢

i

-

∑

k

=

1

i

-

1

l

j

⁢

k

⁢

l

i

⁢

k

)

/

l

i

⁢

i

Because

A

is symmetric and positive definite, the

expression

under the

square root

is always positive, and all

l

i

⁢

j

are real.

References

1

Originally from The Data Analysis Briefbook
(

http://rkb.home.cern.ch/rkb/titleA.html

http://rkb.home.cern.ch/rkb/titleA.html)

Title

Cholesky decomposition

Canonical

name

CholeskyDecomposition

Date of creation

2013-03-22 12:07:38

Last modified on

2013-03-22 12:07:38

Owner

gufotta (12050)

Last modified by

gufotta (12050)

Numerical id

16

Author

gufotta (12050)

Entry type

Definition

Classification

msc 62J05

Classification

msc 65-00

Classification

msc 15-00

Synonym

Cholesky factorization

Synonym

matrix square root

Related topic

SquareRootOfPositiveDefiniteMatrix

Defines

Cholesky triangle

Generated on Sat Feb 10 12:05:28 2018 by

LaTeXML
Gauss-Markov theorem

Gauss-Markov theorem

A

Gauss-Markov linear model

is a linear

statistical model

that satisfies all the conditions of a

general linear model

except
the normality of the error terms. Formally, if

𝒀

is
an

m

-dimensional

response variable

vector, and

𝒁

𝒊

=

z

i

⁢

(

𝑿

)

,

i

=

1

,

…

,

k

are the

m

-dimensional functions of the explanatory variable vector

𝑿

, a Gauss-Markov linear model has the form:

𝒀

=

β

0

⁢

𝒁

𝟎

+

⋯

+

β

k

⁢

𝒁

𝒌

+

ϵ

,

with

ϵ

the error vector such that

1.

E

⁡

[

ϵ

]

=

𝟎

, and

2.

Var

⁡

[

ϵ

]

=

σ

2

⁢

𝑰

.

In other words, the observed responses

Y

i

,

i

=

1

,

…

,

m

are not
assumed to be normally distributed, are not correlated with one
another, and have a common

variance

Var

⁡

[

Y

i

]

=

σ

2

.

Gauss-Markov Theorem

. Suppose the response variable

𝒀

=

(

Y

1

,

…

,

Y

m

)

and the explanatory variables

𝑿

satisfy a Gauss-Markov linear model as described
above. Consider any

linear combination

of the responses

Y

=

∑

i

=

1

m

c

i

⁢

Y

i

,

(1)

where

c

i

∈

ℝ

. If each

μ

i

is an

estimator

for response

Y

i

,

parameter

θ

of the form

θ

=

∑

i

=

1

m

c

i

⁢

μ

i

,

(2)

can be used as an estimator for

Y

. Then, among all

unbiased estimators

for

Y

having form (2), the ordinary least square estimator (OLS)

θ

^

=

∑

i

=

1

m

c

i

⁢

μ

i

^

,

(3)

yields the smallest variance. In other words, the OLS estimator is the

uniformly minimum variance unbiased

estimator.

Remark

.

θ

^

in equation (3) above is more
popularly known as the

BLUE

, or the

best linear unbiased estimator

for a linear combination of the responses in a Gauss-Markov linear
model.

Title

Gauss-Markov theorem

Canonical

name

GaussMarkovTheorem

Date of creation

2013-03-22 15:02:53

Last modified on

2013-03-22 15:02:53

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

8

Author

CWoo (3771)

Entry type

Theorem

Classification

msc 62J05

Synonym

BLUE

Related topic

LinearLeastSquaresFit

Defines

Gauss-Markov linear model

Defines

best linear unbiased estimator

Generated on Sat Feb 10 12:05:30 2018 by

LaTeXML
Linear Plexing

Linear Plexing

We have fixed

α

as the maximum acceptable probability of commiting a
type

I

error. But associated with each normalized test of

significance

is

γ

, the probability of commiting a type

I

error.

Notice that as we make

γ

smaller, the smaller the critical region
becomes and thus the larger the value of the

test statistic

must be for the

null hypothesis

to be rejected.

Find

p

, defined as the smallest

γ

for which we can still reject

H

0

.

Reject

H

0

if

α

<

p

.

Title

Linear Plexing

Canonical

name

LinearPlexing

Date of creation

2013-03-22 15:27:22

Last modified on

2013-03-22 15:27:22

Owner

cogent (10347)

Last modified by

cogent (10347)

Numerical id

4

Author

cogent (10347)

Entry type

Definition

Classification

msc 62J05

Generated on Sat Feb 10 12:05:32 2018 by

LaTeXML
Cochran’s theorem

Cochran’s theorem

Let

X

be multivariate normally distributed as

𝑵

𝒑

⁢

(

𝟎

,

𝑰

)

such that

𝐗

T

⁢

𝐗

=

∑

i

=

1

k

Q

i

,

where each

1.

Q

i

is a

quadratic form

2.

Q

i

=

𝐗

T

⁢

𝐁

i

⁢

𝐗

, where

𝐁

i

is a

p

by

p

square matrix

3.

𝐁

i

is

positive semidefinite

4.

rank

⁡

(

𝐁

i

)

=

r

i

Then any two of the following imply the third:

1.

∑

i

=

1

k

r

i

=

p

2.

each

Q

i

has a chi square distribution (

http://planetmath.org/ChiSquaredRandomVariable

) with

r

i

of freedom,

χ

2

⁢

(

r

i

)

3.

Q

i

’s are mutually independent

As an example, suppose

X

1

2

∼

χ

2

⁢

(

m

1

)

and

X

2

2

∼

χ

2

⁢

(

m

2

)

. Furthermore, assume

X

1

2

≥

X

2

2

and

m

1

>

m

2

, then

X

1

2

-

X

2

2

∼

χ

2

⁢

(

m

1

-

m

2

)

.

This

corollary

is known as

Fisher’s theorem

.

Title

Cochran’s theorem

Canonical

name

CochransTheorem

Date of creation

2013-03-22 14:33:01

Last modified on

2013-03-22 14:33:01

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

8

Author

CWoo (3771)

Entry type

Theorem

Classification

msc 62J10

Classification

msc 62H10

Classification

msc 62E10

Defines

Fisher’s theorem

Generated on Sat Feb 10 12:05:34 2018 by

LaTeXML
general linear model

general linear model

In statistical modeling of

N

data

observations

(

N

<

∞

), two types of variables are usually defined. One is the response variable or

variate

, usually denoted by

Y

, and the other is the explanatory variable or covariate

X

. While there is only one response variable, there may be one or more than one explanatory variables. The response variable is considered random, where as the explanatory variable(s) may or may not be random.

Based on the above setup, a

general linear model

, or

normal linear model

, is a

statistical model

with the following

assumptions

:

1.

the response variable

Y

is a

continuous random variable

2.

the response variable

Y

can be expressed as a

linear combination

of

functions

z

i

⁢

(

𝐗

)

, of the explanatory variables, plus a random error term

ε

:

Y

=

β

0

⁢

z

0

⁢

(

𝐗

)

+

⋯

+

β

k

⁢

z

k

⁢

(

𝐗

)

+

ε

.

The portion of

Y

without the error term is known as the

systematic

component

of

Y

.

3.

the error component and the systematic component are

independent

4.

random error variables

ε

i

for the

N

observations are iid normal with mean 0 and

variance

σ

2

Remarks

•

Conditioning on the explanatory variables, the

random variables

Y

i

corresponding to the

individual

responses are independent, normally distributed, with mean

μ

=

E

⁡

[

Y

∣

𝐗

=

𝒙

]

=

β

0

⁢

z

0

⁢

(

𝐗

)

+

⋯

+

β

k

⁢

z

k

⁢

(

𝐗

)

and variance

σ

2

.

•

A

linear regression model

is a special case of the general linear model where all explanatory variables are assumed to be

continuous

.

•

Analysis of variance

model, or

ANOVA

, is another special case of the general linear model, where all of the explantory variables are

categorical

in nature (for example, gender, marital status, etc..).

•

Analysis of covariance

, or

ANCOVA

, sits between a linear regression model and the ANOVA, where some of the explanatory variables are continuous and some are categorical.

•

The general linear model is a special case of the

generalized linear model

, where the assumption that the response variable

Y

has a

normal distribution

is dropped.

Title

general linear model

Canonical name

GeneralLinearModel

Date of creation

2013-03-22 14:31:23

Last modified on

2013-03-22 14:31:23

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

6

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62J10

Classification

msc 62J05

Synonym

normal linear model

Defines

analysis of variance

Defines

ANOVA

Defines

analysis of covariance

Defines

ANCOVA

Generated on Sat Feb 10 12:05:35 2018 by

LaTeXML
mean square error

mean square error

The

mean square error

of an

estimator

θ

^

of a

parameter

θ

in a

statistical model

is defined as:

MSE

⁡

(

θ

^

)

:=

E

⁡

[

(

θ

^

-

θ

)

2

]

.

From the

definition

of the

variance

Var

[

X

]

=

E

[

X

2

]

-

E

[

X

]

2

,
we can express the mean square error in terms of the bias by
expanding the right hand side above:

MSE

(

θ

^

)

=

Var

[

θ

^

]

+

Bias

(

θ

^

)

2

.

If

θ

^

is an

unbiased estimator

, then its mean square
error is identical to its variance:

MSE

⁡

(

θ

^

)

=

Var

⁡

[

θ

^

]

.
An unbiased estimator such that

MSE

⁡

(

θ

^

)

is a minimum value among all unbiased estimators for

θ

is
called a

minimum variance unbiased estimator

, abbreviated

MVUE

, or

uniformly minimum variance unbiased estimator

, abbreviated

UMVU

estimator.

Example

. Suppose

X

1

,

X

2

,

…

,

X

n

are iid

random
variables

(

n

independent

measurements of the radius of a coin,
etc…) from a

normal distribution

N

⁢

(

μ

,

σ

2

)

(for example,

μ

would be the true radius of the coin, and

σ

2

would be
the error

component

of the measurements). Suppose

X

¯

(

=

X

¯

n

) is the

sample mean

. Then

X

¯

is an
unbiased estimator, so that

MSE

⁡

(

X

¯

)

=

Var

⁡

[

X

¯

]

=

Var

⁡

[

1

n

⁢

∑

i

=

1

n

X

i

]

=

1

n

2

⁢

(

∑

i

=

1

n

σ

2

)

=

σ

2

n

.

Remark

. The square root of MSE is called the “root mean square error”, or

rms error

for short.

Title

mean square error

Canonical

name

MeanSquareError

Date of creation

2013-03-22 12:07:42

Last modified on

2013-03-22 12:07:42

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

11

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62J10

Classification

msc 94A12

Synonym

MSE

Synonym

MVUE

Synonym

UMVU

Synonym

UMVUE

Synonym

uniformly minimum variance unbiased

Related topic

MeanSquareDeviation

Defines

minimum variance unbiased estimator

Defines

rms error

Defines

root-mean-square

Defines

root mean square

Defines

rms

Generated on Sat Feb 10 12:05:37 2018 by

LaTeXML
generalized linear model

generalized linear model

Given a

random vector

, or the

response variable

,

Y

, a

generalized linear model

, or GLM for short, is a

statistical model

{

f

𝐘

(

𝒚

∣

𝜽

)

}

such that

1.

the

components

of

Y

are mutually independent of each other,

2.

f

Y

i

(

y

i

∣

θ

i

)

belongs to the

exponential family

of

distributions

and has the following canonical form:

f

Y

i

(

y

i

∣

θ

i

)

=

exp

[

y

θ

i

-

b

(

θ

i

)

+

c

(

y

)

]

,

where the

parameter

θ

i

is called the

canonical parameter

and

b

⁢

(

θ

i

)

is called the

cumulant function

.

3.

for each component or variate

Y

i

, with a corresponding set of

p

covariates

X

i

⁢

j

, there exists a

monotone

differentiable function

g

, called the

link function

, such that

g

⁢

(

E

⁡

[

Y

i

]

)

=

𝐗

i

T

⁢

𝜷

,

where

𝐗

i

T

=

(

X

i

⁢

1

,

…

,

X

i

⁢

p

)

, and

𝜷

=

(

β

1

,

…

,

β

p

)

T

is a parameter vector.

In practice, an extra parameter called the

dispersion parameter

,

ϕ

, is introducted to the model to lower a phenonmenon known as overdispersion. The GLM now looks like:

f

Y

i

(

y

i

∣

θ

i

)

=

exp

[

y

⁢

θ

i

-

b

⁢

(

θ

i

)

a

⁢

(

ϕ

)

+

c

(

y

,

ϕ

)

]

Remarks

•

Below is a table of canonical parameters and cumulant functions for some well-known distributions from the exponential family:

Title

generalized linear model

Canonical name

GeneralizedLinearModel

Date of creation

2013-03-22 14:30:11

Last modified on

2013-03-22 14:30:11

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

17

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62J12

Synonym

GLM

Defines

link function

Defines

canonical parameter

Defines

cumulant function

Defines

variance function

\@unrecurse

Generated on Sat Feb 10 12:05:46 2018 by

LaTeXML
overdispersion

overdispersion

When applying the

generalized linear model

or GLM to the real world, a phenomenon called

overdispersion

occurs when the observed

variance

of the data is larger than the predicted variance. This is particularly apparent in the case of a Poisson regression model, where

predicted variance = predicted mean,

or the binary

logistic regression

model, where

predicted variance = predicted mean(1- predicted mean).

A parameter, called the

dispersion parameter

,

ϕ

, is introducted to the model to lower this overdispersion effect.

The GLM, with the inclusion of this dispersion parameter, has the following

density function

:

f

Y

i

(

y

i

∣

θ

i

)

=

exp

[

y

⁢

θ

i

-

b

⁢

(

θ

i

)

a

⁢

(

ϕ

)

+

c

(

y

,

ϕ

)

]

Dispersion parameters for some of the well known

distributions

from the

exponential family

are listed in the following table:

Title

overdispersion

Canonical name

Overdispersion

Date of creation

2013-03-22 14:30:34

Last modified on

2013-03-22 14:30:34

Owner

CWoo (3771)

Last modified by

CWoo (3771)

Numerical id

10

Author

CWoo (3771)

Entry type

Definition

Classification

msc 62J12

Defines

dispersion parameter

\@unrecurse

Generated on Sat Feb 10 12:05:51 2018 by

LaTeXML