Minimum

Message

Length

Encoding

Evolutionary

Trees

and

Multiple-Alignment

L.

Allison

C.

S.

Wallace

and

C.

N.

Yee

Department

of

Computer

Science

Monash

University

Australia

xxx@cs.monash.edu.au

where

xxx

lloyd

csw

cyee

Tel

-LRB-03-RRB- 565 5205

Fax

Technical

Report

September

Aversion

of

this

paper

nowappears

in

Hawaii

International

Conference

on

System

Sciences

Jan

Abstract

Amethod

of

Bayesian

inference

known

as

minimum

message

length

-LRB-

MML

-RRB-

encoding

is

applied

to

problems

overk

strings

inference

of

evolutionary-trees

and

multiple-alignment

The

method

allows

the

posterior

odds-ratio

of

twocompeting

hypotheses

for

example

twotrees

to

be

calculated

A

tree

that

is

a

good

hypothesis

for

the

strings

forms

the

basis

of

a

short

message

describing

them

The

work

is

a

natural

extension

of

applying

MML

encoding

to

the

alignment

of

twostrings

as

previously

reported

It

is

seen

that

tree

inference

and

multiple-alignment

are

intimately

connected

The

mutation

of

one

string

into

another

is

modelled

by

a

nite-state

machine

The

machine

can

copy

change

insert

and

delete

characters

Each

machine

has

one

or

more

parameters

Va

rying

the

machine

architecture

varies

the

mutation

cost-functions

The

tree

inference

problem

is

broken

down

into

inter-linked

sub-problems

what

is

the

best

topology

what

are

the

optimum

parameter

values

for

each

edge

of

the

tree

what

is

the

implied

cost

function

on

character-tuples

in

a

multiple-alignment

what

is

the

probability

of

each

multiple-alignment

and

nally

what

is

the

probability

of

an

optimal

alignment

or

of

all

alignments

taken

together

Each

edge

of

the

tree

relates

two

possibly

hypothetical

strings

and

is

described

by

one

machine

Methods

of

estimating

the

parameter

values

for

each

edge

-LRB-

machine

-RRB-

are

described

The

length

of

a

message

describing

the

strings

and

based

on

a

tree

includes

of

the

sum

of

the

probabilities

of

all

multiple-alignments

as

implied

by

the

topology

and

parameters

of

the

tree

Computing

this

overall

alignments

is

not

feasible

heuristics

and

approximate

methods

to

circumvent

this

problem

are

described

Q

What

is

the

difference

between

a

hypothesis

and

a

theory

A

Think

of

a

hypothesis

as

a

card

Atheory

is

a

house

made

of

hypotheses

-LRB-

From

rec.humor.funny

attributed

to

Marilyn

vos

Sava

nt

-RRB-

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

Introduction

In

previous

papers

-LSB-

-RSB-

minimum

message

length

-LRB-

MML

-RRB-

encoding

was

used

to

infer

the

relation

if

any

between

twostrings

and

to

compare

models

of

mutation

or

relation

Here

we

apply

MML

encoding

to

problems

ove

r

k

of

average

length

n

multiple-alignment

and

inference

of

ev

olutionary-trees

First

we

introduce

the

MML

method

MML

encoding

is

a

method

of

Bayesian

inference

In

it

we

consider

the

message

length

of

an

explanation

of

givendata

D

An

explanation

consists

of

a

message

which

rst

states

an

hypothesis

H

then

states

the

data

using

a

code

which

would

be

optimal

were

H

true

It

is

a

basic

fact

of

coding

theory

that

the

message

length

-LRB-

ML

-RRB-

under

optimal

encoding

of

an

event

of

probability

p

is

-LRB-

p

-RRB-

bits

From

Bayes

theorem

in

the

discrete

case

givenahypothesis

H

and

data

D

P

-LRB-

H&D

-RRB-

P

-LRB-

H

-RRB-

P

-LRB-

D

H

-RRB-

P

-LRB-

D

-RRB-

P

-LRB-

H

D

-RRB-

or

equivalently

ML

-LRB-

H&D

-RRB-

ML

-LRB-

H

-RRB-

ML

-LRB-

D

H

-RRB-

ML

-LRB-

D

-RRB-

ML

-LRB-

H

D

-RRB-

P

-LRB-

D

H

-RRB-

is

called

the

likelihood

of

the

hypothesis

although

it

is

a

function

of

the

data

giventhe

hypothesis

it

is

not

the

probability

of

the

hypothesis

P

-LRB-

H

-RRB-

is

the

prior

probability

of

the

hypothesis

P

-LRB-

H

D

-RRB-

is

the

posterior

probability

of

the

hypothesis

If

ML

-LRB-

H

-RRB-

and

ML

-LRB-

D

H

-RRB-

can

be

calculated

for

givenHand

D

then

the

posterior

odds-ratio

of

twohypotheses

can

be

calculated

Analogous

results

hold

in

the

continuous

case

and

imply

an

optimum

accuracyfor

the

estimation

of

real-valued

parameters

although

the

calculations

can

be

dif?cult

MML

encoding

can

be

used

to

compare

models

test

hypotheses

and

estimate

parameters

Wallace

and

Freeman

-LSB-

-RSB-

give

the

statistical

foundations

Maximum-likelihood

methods

and

MML

encoding

are

related

Asigni?cant

difference

is

that

the

former

do

not

cost

the

model

or

hypothesis

complexity

ML

-LRB-

H

-RRB-

and

so

tend

to

favour

overly

complexor

detailed

models

In

sequence

analysis

the

issue

covers

the

estimation

of

parameters

-LRB-

such

as

time

-RRB-

to

optimal

accuracy

the

complexity

of

models

of

mutation

and

of

evolution

and

the

fact

that

a

single

alignment

is

too

precise

a

theory.Inthose

cases

where

the

model

complexity

is

constant

it

can

be

ignored

and

then

maximum-likelihood

and

MML

encoding

give

the

same

results

Tw

o

being

a

special

case

of

k

we

next

link

inference

overtwo

strings

to

the

more

general

problems

Givenstrings

A

and

B

the

following

questions

can

be

raised

Are

theyrelated

Are

theyrelated

by

a

particular

model

of

relation

and

parameter

values

-LRB-

time

since

divergence

-RRB-

Are

theyrelated

by

a

particular

alignment

Are

theyrelated

by

descent

from

a

particular

common

ancestor

An

answer

to

anyone

of

these

questions

can

only

be

a

hypothesis

An

explanation

of

the

strings

can

be

based

on

such

an

answer.Explanations

based

on

successive

questions

carry

increasing

amounts

of

information

and

must

have

increasing

message

lengths

Aand

B

have

in?nitely

manypossible

common

ancestors

It

is

possible

to

dismiss

most

as

wildly

improbable

but

it

is

generally

impossible

to

identify

a

common

ancestor

with

anycertainty

unless

extra

information

is

available

A

single

alignment

implies

a

subset

of

possible

ancestors

in

a

natural

way.Conventional

alignment

programs

return

a

single

alignment

which

is

optimal

under

some

explicit

or

implicit

assumptions

Agiv

enmodel

can

relate

A

and

B

by

many

different

alignments

and

we

called

the

answer

to

the

r-theory

-LRB-

for

relation

-RRB-

Its

statement

embodies

the

inferred

parameters

for

the

evolution

of

A

and

B

followed

by

the

strings

A

and

B

in

a

code

based

on

all

possible

alignments

One

of

the

by-products

is

a

probability

density

plot

of

all

alignments

under

those

parameter

values

The

answer

to

would

include

the

sum

overall

models

of

the

integral

overall

parameter

values

This

calculation

is

impractical

and

the

result

would

be

of

little

use

is

the

question

usually

asked

There

is

a

natural

null-theory

in

sequence

analysis

If

the

strings

are

unrelated

it

is

not

possible

to

do

better

than

transmit

them

one

after

the

other

atabout

twobits

per

character

for

DNA

Note

that

the

lengths

of

the

strings

must

also

be

transmitted

Anyexplanation

that

has

a

message

length

greater

than

that

of

the

null-theory

is

not

acceptable

The

approach

to

problems

overtwo

strings

as

summarised

above

generalises

to

problems

over

several

strings

Questions

analogous

to

arise

when

givenk

which

might

be

related

by

an

ev

olutionary-tree

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

-LRB-

a

-RRB-

Are

the

strings

related

or

slightly

more

detailed

-LRB-

b

-RRB-

are

theyrelated

by

a

particular

tree-topology

Consideration

is

usually

restricted

to

binary

trees

and

we

followthis

convention

here

There

is

no

dif?culty

in

allowing

arbitrary

branching

factors

Forests

should

be

allowed

so

as

to

permit

independent

families

but

the

message

length

of

a

forest

includes

the

sum

of

the

message

lengths

of

its

trees

and

the

generalisation

is

obvious

Are

theyrelated

by

a

particular

tree

-LRB-

topology

parameter

values

for

each

edge

-RRB-

Each

edge

of

the

tree

represents

a

simple

two-string

relation

the

inferred

parameters

for

each

edge

-LRB-

time

-RRB-

are

required

This

is

the

generalised

r-theory

Are

theyrelated

by

a

particular

multiple

alignment

Are

theyrelated

by

descent

through

particular

ancestors

The

chain

matches

the

chain

and

in

particular

matches

although

this

requires

some

discussion

Aparticular

tree

-LRB-

topology

edge

parameters

-RRB-

implies

a

probability

distribution

on

character-tuples

and

thus

on

multiple-alignments

The

probability

of

a

tree

and

the

strings

-LRB-

-RRB-

is

the

product

of

the

prior

probability

of

the

tree

and

the

sum

of

the

probabilities

of

all

such

multiple-alignments

giventhe

tree

Forkstrings

where

k

the

number

of

alignments

is

very

large

Therefore

heuristic

and

approximate

methods

are

required

to

makethe

computations

feasible

can

be

taken

in

twoways

Wetakethe

word

related

to

indicate

related

by

an

evolutionary-tree

A

tree

has

edges

and

thus

has

O

-LRB-

k

-RRB-

parameters

but

there

are

possible

tuples

in

a

multiple

alignment

There

is

a

smaller

but

still

exponential

number

of

possible

patterns

Manyofthe

tuples

probabilities

are

locked

together

under

an

evolutionary-tree

hypothesis

If

related

were

taken

to

mean

anyarbitrary

relation

there

would

be

more

degrees

of

freedom

in

the

form

of

hypotheses

which

would

enable

the

data

to

be

tted

better

butatthe

higher

overhead

in

general

of

stating

the

increased

number

of

parameters

Reichert

et

al

-LSB-

Reic

-RSB-

were

the

rst

to

apply

compact

encoding

to

the

alignment

of

twostrings

They

used

one

particular

model

of

relation

and

sought

a

single

optimal

alignment

This

was

dif?cult

to

do

ef?ciently

for

the

model

that

theychose

Bishop

and

Thompson

-LSB-

Bish

-RSB-

de?ned

maximum-likelihood

alignment

for

what

is

a

model

although

theyrecognised

the

possibility

of

other

models

Allison

et

al

-LSB-

-RSB-

objectively

compared

and

models

of

mutation

using

MML

encoding

Felsenstein

-LSB-

-RSB-

developed

a

maximum-likelihood

method

for

infering

evolutionary-trees

To

date

it

is

the

method

with

the

soundest

statistical

credentials

His

program

does

not

do

alignment

itself

and

requires

a

multiple-alignment

as

input

Friday

-LSB-

Frid

-RSB-

givesageneral

discussion

of

the

problem

of

assigning

gures

of

merit

to

inferred

evolutionary-trees

Wallace

and

Boulton

-LSB-

-RSB-

developed

an

MML

method

for

classi?cation

of

arbitrary

things

-LRB-

It

is

not

directly

applicable

to

the

alignment

problem

-RRB-

The

things

can

have

continuous

and/or

discrete-valued

attributes

A

further

development

-LSB-

-RSB-

inferred

hierarchical

classi?cations

Cheeseman

and

Kanefsky

-LSB-

Chee

-RSB-

applied

MML

encoding

to

evolutionary-trees

Milosavljevic

-LSB-

Milo

-RSB-

has

examined

the

classi?cation

of

macro-molecule

sequences

-LRB-

In

classi?cation

a

class

usually

contains

manythings

-RRB-

The

methods

described

in

this

paper

relate

to

other

methods

as

follows

Theygeneralise

Felsenstein

sapproach

by

allowing

inserts

and

deletes

to

be

inferred

and

by

including

the

cost

of

the

model

In

contrast

to

Cheeseman

and

Kanefsky

a

ncestral

strings

are

not

inferred

for

to

do

so

involves

arbitrary

choices

Howeverprobabilities

can

be

assigned

to

possible

ancestors

and

to

possible

alignments

giventhe

parameters

that

are

inferred

Mutation-Models

We

consider

twobroad

models

to

relate

twostrings

A

and

B

-LRB-

-RRB-

In

the

mutation-model

a

simple

machine

reads

string

A

and

a

sequence

of

mutation

instructions

and

outputs

string

B

In

the

generation-model

asimple

machine

reads

a

sequence

of

generation

instructions

and

outputs

strings

A

and

B

on

a

single

tape

twocharacters

wide

There

an

obvious

correspondence

between

a

mutation

sequence

a

generation

sequence

and

an

alignment

Results

based

on

these

notions

are

equivalent

and

the

notions

can

be

used

interchangeably

as

convenient

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

The

machines

that

we

consider

are

probabilistic

nite-state

machines

as

described

in

-LSB-

-RSB-

theyare

equivalent

to

hidden

Markov

models

Normally

we

consider

symmetric

machines

where

p

-LRB-

ins

-RRB-

p

-LRB-

del

-RRB-

p

-LRB-

insA

-RRB-

=p

-LRB-

insB

-RRB-

The

message

length

of

an

instruction

is

of

its

probability

plus

either

twobits

for

one

character

or

-LRB-

-RRB-

bits

for

twodiffering

characters

for

simplicity

only

itisassumed

that

all

characters

are

equally

probable

In

highly

conserved

sequences

the

probability

of

a

mutation

is

lowand

its

message

length

is

high

The

probability

of

an

instruction

also

depends

on

the

current

state

S

of

the

machine

ML

-LRB-

copy

S

-RRB-

-LRB-

P

-LRB-

copy

S

-RRB-

-RRB-

ML

-LRB-

change

S

-RRB-

-LRB-

P

-LRB-

change

S

-RRB-

-RRB-

-LRB-

-RRB-

ML

-LRB-

ins

S

-RRB-

ML

-LRB-

del

S

-RRB-

-LRB-

P

-LRB-

ins

S

-RRB-

-RRB-

machine

corresponds

to

Sellers

edit

distance

-LSB-

Sell

-RSB-

with

adjustable

weights

machine

corresponds

to

linear

indel

costs

or

weights

Machines

with

more

than

three

states

can

correspond

to

piecewise

linear

indel

costs

Note

that

multiple

states

allowaninstruction

to

have

d

ifferent

probabilities

in

different

contexts

In

particular

for

a

machine

a

run

of

indels

can

have

a

higher

probability

of

continuing

than

of

beginning

giving

a

linear

message

length

-LRB-

cost

-RRB-

as

a

function

of

length

Amessage

based

on

one

generation

sequence

-LRB-

alignment

-RRB-

states

-LRB-

i

-RRB-

the

parameter

values

for

the

machine

and

-LRB-

ii

-RRB-

the

sequence

of

instructions

The

parameters

are

estimated

by

an

iterative

process

Initial

values

are

chosen

and

used

to

nd

an

optimal

sequence

It

is

an

elementary

result

of

coding

theory

that

in

an

optimal

code

for

the

sequence

the

probabilities

of

instructions

should

come

from

the

observed

frequencies

of

instructions

-LRB-

Tosee

this

for

an

alphabet

of

twoletters

consider

a

sequence

of

length

n

and

its

probability

pf

-LRB-

-RRB-

n-f

differentiate

with

respect

to

p

and

set

to

zero

-RRB-

The

change

in

the

parameter

values

can

not

increase

the

message

length

of

the

givenalignment

but

may

cause

another

alignment

to

become

optimal

Iteration

is

therefore

necessary

and

is

guaranteed

to

converge

A

few

iterations

typically

four

to

eight

are

suf?cient

for

convergence

The

process

could

nd

only

a

local

minimum

in

theory

but

this

is

not

a

problem

in

practice

The

parameters

must

be

stated

to

an

optimum

accuracy

the

instructions

are

chosen

from

a

multistate

distribution

and

Boulton

and

Wallace

-LSB-

-RSB-

give

the

necessary

calculations

The

r-theory

-LRB-

-RRB-

is

based

on

all

instruction

sequences

-LRB-

alignments

-RRB-

The

algorithms

-LSB-

-RSB-

that

compute

the

r-theory

for

the

various

machines

are

dynamic

programming

algorithms

-LSB-

DPA

-RSB-

that

perform

logplus

on

message

lengths

in

the

inner

step

or

on

probabilities

instead

of

the

usual

min

on

edit

distances

or

max

on

lengths

of

common

subsequences

-LRB-

logplus

-LRB-

log

-LRB-

A

-RRB-

log

-LRB-

B

-RRB-

-RRB-

log

-LRB-

A+B

-RRB-

-RRB-

The

iterative

process

is

used

to

estimate

the

parameter

values

based

on

weighted

frequencies

Results

reported

in

-LSB-

-RSB-

support

the

widely

held

viewthat

the

and

machines

are

better

models

for

DNAthan

the

machine

The

and

the

machines

have

increasing

numbers

of

parameters

to

be

stated

which

tends

to

increase

the

message

length

but

this

is

more

than

made

up

for

by

a

saving

in

stating

the

strings

There

are

practical

dif?culties

in

going

beyond

about

We

donot

yet

have

enough

data

to

reliably

judge

between

the

and

machines

We

donot

yet

knowhow

best

to

incorporate

a

mutation

model

with

more

than

into

multiple

alignment

this

is

discussed

later

Traditionally

string

alignment

programs

have

been

based

on

Sellers

metric

-LSB-

Sell

-RSB-

linear

and

piece-wise

linear

cost

functions

for

indels

-LSB-

-RSB-

or

arbitrary

concave

cost

functions

-LSB-

Mill

-RSB-

Some

of

these

programs

have

sev

eral

parameters

requiring

ad-hoc

adjustments

MML

encoding

provides

an

objective

way

to

compare

models

of

differing

complexities

and

to

infer

their

parameter

values

We

now

examine

the

questions

informally

on

the

example

strings

A

TAATA

CTCGGC

and

B

TAT

AACTGCCG

There

are

twoimportant

cautionary

notes

The

strings

are

necessarily

short

and

results

on

short

strings

must

be

taken

with

a

pinch

of

salt

We

u

se

the

model

only

because

it

is

the

simplest

to

discuss

not

because

it

is

strongly

supported

Are

strings

A

and

B

related

Intuitively

it

seems

that

theyprobably

are

related

but

it

is

hard

to

see

exactly

how

Theyhav

e

at

the

least

diverged

considerably

How

far

and

in

what

ways

have

A

and

B

diverged

The

front

sections

TAT

A

A

TAATA

can

be

related

in

several

ways

involving

either

twomismatches

or

twoindels

The

central

CTs

probably

match

The

nal

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

sections

GCCG

CGGC

can

be

related

with

twoindels

and

a

mismatch

or

four

indels

or

four

mismatches

Overall

and

on

average

we

expect

about

sevenmatches

one

or

twomismatches

and

a

fewindels

What

is

the

best

alignment

It

is

not

possible

to

say

more

than

it

probably

involves

T

-LSB-

A

AA

-RSB-

T

-LSB-

AA

A

-RSB-

CT

-LSB-

GCCG

CGGC

-RSB-

-LRB-

-RRB-

As

noted

above

the

model

is

used

in

this

example

for

simplicity.The

and

models

would

penalise

isolated

indels

more

heavily

What

was

the

most

recent

common

ancestor

of

A

and

B

It

is

not

possible

to

say

more

than

it

was

probably

one

of

the

strings

described

by

the

pattern

TA

-LSB-

A

-RSB-

TA

-LSB-

A

-RSB-

CTa

where

a

invo

lved

twotofour

Cs

and/or

Gs

When

the

r-theory

algorithm

is

run

on

strings

A

and

B

the

results

-LRB-

-RRB-

quantify

and

support

the

informal

arguments

made

above.The

optimal

alignments

stand

out

in

the

density

plot

but

there

are

several

of

them

and

manyother

near-optimal

alignments

Anyone

alignment

has

a

message

length

greater

than

the

null-theory

and

is

not

acceptable

but

taking

them

all

together

the

r-theory

is

acceptable

Trees

overTwo

Strings

Giventwo

related

present-day

strings

A

and

B

we

think

of

them

as

having

evolved

from

a

most

recent

common

ancestor

P.This

involves

twounknown

steps

or

mutation

machines

MA

and

MB

-LRB-

-RRB-

The

usual

models

are

reversible

and

mutating

P

into

A

by

MA

is

equivalent

to

mutating

A

back

into

P

by

MA

Howeverthe

composition

of

twos-state

machines

MA'and

MB

isnot

in

general

equivalent

to

an

s-state

machine

for

s

Fortunately

for

typical

overall

indel

rates

a

single

s-state

machine

is

a

good

approximation

to

the

composition

of

twoormore

s-state

machines

It

is

thus

acceptable

to

relate

A

and

B

directly

by

a

single

s-state

machine

givesthe

probability

distribution

for

indel

lengths

for

compositions

of

and

machines

Figure

-LRB-

a

-RRB-

is

for

machines

that

give

linear

indel

costs

Figure

-LRB-

b

-RRB-

is

for

machines

that

can

cause

short

-LRB-

s

-RRB-

or

long

-LRB-

L

-RRB-

indels

and

that

give

asymptotically

piece-wise

linear

costs

The

calculations

are

based

on

machines

that

can

copyorinsert

only.Itisthe

shape

of

the

curves

not

their

absolute

values

that

is

important

This

also

suggests

that

when

considering

anytwo

strings

out

of

k

and

the

path

of

machines

-LRB-

edges

-RRB-

between

them

in

a

tree

in

order

to

get

partial

information

on

a

multiple-alignment

it

is

acceptable

to

combine

twoormore

adjacent

s-state

machines

-LRB-

edges

-RRB-

into

one

such

machine

Note

that

there

are

twoforests

overAand

B

The

r-theory

tree

-LRB-

Figure

-RRB-

forms

one

forest

The

forest

of

twosingleton

trees

corresponds

to

the

null-theory

If

the

ancestor

P

and

the

mutation

instruction

sequences

that

MA

and

MB

executed

were

known

it

would

be

possible

to

form

an

equivalent

generation

sequence

-LRB-

-RRB-

for

a

single

generation

machine

G

relating

A

and

B.

Howeverneither

P

nor

the

mutation

sequences

are

known

The

idea

to

circumvent

this

problem

is

to

synchronise

MA

and

MB

as

theyread

P

and

to

remove

dependencyonthe

unknown

P

by

summing

overall

possibilities

MA

and

MB

are

forced

to

execute

copy

change

and

delete

instructions

in

parallel

note

that

each

of

these

instructions

reads

one

character

from

P.Equivalently

Gexecutes

a

match

mismatch

insA

or

insB

instruction

as

appropriate

except

that

if

both

MA

and

MB

execute

a

delete

G

does

nothing

Note

that

an

insert

reads

no

character

from

P.IfMA

is

about

to

execute

an

insert

instruction

and

MB

is

about

to

execute

some

other

kind

of

instruction

then

MB

is

delayed

and

G

executes

an

insA

Similarly

ifMB

-LRB-

only

-RRB-

is

about

to

execute

an

insert

G

executes

an

insB.Ifboth

MA

and

MB

are

about

to

execute

one

or

more

inserts

theyand

G

can

be

made

to

do

one

of

several

things

double-counting

can

be

avoided

if

this

is

done

carefully

-LRB-

Figure

-RRB-

The

inserts

can

be

interleavedinsev

e

ral

different

ways

For

example

G

can

execute

an

insA

and

then

an

insB

or

v.v.Each

of

these

options

corresponds

to

a

part

of

an

alignment

Some

such

as

an

insA

immediately

followed

by

an

insB

are

of

low

but

non-zero

probability

and

are

not

usually

seen

in

alignments

Forexample

a

single

optimal

alignment

always

explains

a

pair

of

characters

Ai

Bj

as

change

-LRB-

Ai

Bj

-RRB-

and

neveras

insA

-LRB-

Ai

-RRB-

insB

-LRB-

Bj

-RRB-

and

thus

givesbiased

estimates

We

expect

P

Aand

B

to

have

the

same

number

of

characters

on

average

provided

that

pA

-LRB-

i

-RRB-

pA

-LRB-

d

-RRB-

and

pB

-LRB-

i

-RRB-

pB

-LRB-

d

-RRB-

Howeverweexpect

the

alignment

to

be

longer

than

P

for

a

tuple

is

gained

whenever

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

MA

or

MB

does

an

insert

but

a

character

is

only

lost

when

both

MA

and

MB

delete

it

Forthis

reason

rates

of

instructions

per

character

of

P

ie

pA'and

pB

have

a

denominator

of

-LRB-

-LRB-

i

-RRB-

-RRB-

or

-LRB-

-LRB-

i

-RRB-

-RRB-

The

expected

number

of

pairs

in

the

alignment

per

character

of

P

is

pA

-LRB-

i

-RRB-

pB

-LRB-

i

-RRB-

The

above

considerations

are

generalised

to

k

in

the

next

section

Note

that

is

a

noncharacter

not

a

proper

character.Similarly

an

edge

with

a

at

each

end

represents

a

non-operation

for

a

machine

not

a

proper

operation

It

is

introduced

only

to

synchronise

MA

and

MB

The

parameters

of

G

could

be

calculated

from

the

parameters

of

MA

and

MB

if

the

latter

were

known

Howeverneither

P

MA

nor

MB

are

known

In

this

case

parameter

values

for

G

can

still

be

inferred

from

A

and

B

under

the

r-theory

The

values

embody

the

weighted

average

overall

possible

P

and

instruction

sequences

for

MA

and

MB.Ifaconstant

rate

of

mutation

is

assumed

and

MA

MB

then

it

is

possible

to

infer

the

parameters

of

MA.For

three

or

more

strings

it

is

possible

to

get

a

three-way

x

on

all

hypothetical

strings

and

estimate

all

machines

without

assuming

a

constant

rate

of

mutation

Evolutionary

Trees

and

Multiple-Alignment

Of

the

questions

overk

we

consider

to

be

the

most

important

one

to

answer

with

a

close

second

to

recap

Are

theyrelated

by

a

particular

tree

Are

theyrelated

-LSB-

under

an

ev

olutionary-tree

-RSB-

by

a

particular

multiple-alignment

The

topology

of

the

tree

is

important

but

of

little

use

without

the

amount

and

form

of

mutation

represented

by

each

edge

Strictly

detailed

to

judge

a

tree

on

although

an

optimal

alignment

is

often

useful

As

for

the

two-string

case

optimal

multiple-alignments

are

not

unique

in

general

and

even

sub-optimal

alignments

may

be

plausible

An

MML

encoding

of

the

strings

based

on

an

answer

to

or

consists

of

-LRB-

i

-RRB-

the

value

of

k

-LRB-

ii

-RRB-

the

tree

topology

-LRB-

iii

-RRB-

the

parameter

values

for

each

edge

and

either

-LRB-

iv

-RRB-

the

strings

based

on

an

all

alignments

-LRB-

-RRB-

or

-LRB-

iv

-RRB-

the

strings

based

on

one

optimal

alignment

-LRB-

-RRB-

The

tree

that

best

explains

the

strings

givesthe

shortest

message

length

The

difference

in

message

lengths

of

twotrees

of

their

posterior

odds-ratio

Note

the

most

colourless

assumption

is

that

all

topologies

are

equally

likely

so

part

-LRB-

ii

-RRB-

is

of

constant

length

givenk

and

is

only

needed

for

comparison

with

the

null-theory

Arooted

binary

tree

can

only

be

inferred

if

an

assumption

of

a

-LRB-

near

-RRB-

constant

rate

of

mutation

is

made

If

this

assumption

is

not

made

an

unrooted

binary

tree

is

inferred

and

can

be

broken

down

into

sub-problems

What

is

the

best

tree-topology

What

are

the

parameters

for

each

edge

What

is

the

cost

function

for

tuples

in

a

multiple

alignment

What

is

the

message

length

of

an

alignment

What

is

the

message

length

of

the

strings

giventhe

tree

considering

all

alignments

taken

together

-LRB-

-RRB-

What

is

the

message

length

of

the

strings

giventhe

tree

and

based

on

one

optimal

alignment

-LRB-

-RRB-

Note

that

answers

to

these

sub-problems

are

not

independent

The

topology

and

parameter

values

de?ne

the

tuple

cost

which

affects

the

alignment

probabilities

which

decide

the

plausibility

of

each

tree

Tw

o

competing

sets

of

answers

to

the

sub-problems

can

be

judged

easily

and

objectively

but

nding

the

best

answers

is

hard

On

the

other

hand

the

algorithms

for

the

sub-problems

are

largely

independent

and

can

be

studied

separately.Aprototype

program

has

been

written

for

based

on

particular

approaches

to

these

problems

In

addition

some

other

possibilities

have

been

examined

These

are

discussed

in

the

following

section

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

Tr

e

e-Topology

An

unrooted

tree

overk

or

leaveshas

internal

hypothetical

strings

and

edges

There

are

Uk

unrooted

tree

topologies

overkstrings

where

Uk

-LRB-

-RRB-

k

Each

is

considered

equally

probable

apriori

and

-LRB-

Uk

-RRB-

bits

are

required

to

state

its

identity

ML

-LRB-

topology

-RRB-

-LRB-

Uk

-RRB-

The

prototype

program

requires

the

user

to

specify

one

or

more

trees

to

be

evaluated

Standard

distance-matrix

methods

could

be

used

to

propose

plausible

topologies

The

ideal

distance

is

proportional

to

time

It

is

thought

that

the

parameters

derivedfrom

the

two-string

r-theory

give

more

accurate

pair-wise

distances

than

a

simple

edit-distance

Felsenstein

-LSB-

-RSB-

describes

howlocal

rearrangements

can

be

used

to

nd

at

least

a

locally

optimal

tree

There

is

no

great

dif?culty

in

allowing

n-ary

trees

except

that

there

are

more

of

them

Note

that

any

multiple-alignment

method

that

does

not

explicitly

consider

a

tree

nevertheless

embodies

an

implicit

model

of

relation

of

some

kind

possibly

a

xed

tree

Edge

Parameters

Each

edge

of

the

tree

relates

two

possibly

hypothetical

strings

and

is

represented

by

a

machine

Anyrealistic

machine

requires

at

least

one

parameter

to

specify

instruction

probabilities

being

related

to

the

amount

of

time

or

mutation

the

edge

represents

The

parameter

-LRB-

s

-RRB-

of

each

of

the

machines

must

be

included

in

the

message

The

instructions

are

chosen

from

a

multi-state

distribution

and

Boulton

and

Wallace

-LSB-

-RSB-

give

the

calculations

to

determine

the

precision

with

which

to

state

the

instruction

probabilities

The

prototype

program

is

based

on

machines

In

that

case

each

machine

has

three

parameters

P

-LRB-

copy

-RRB-

P

-LRB-

change

-RRB-

P

-LRB-

indel

-RRB-

and

twodegrees

of

freedom

MML

encoding

is

invariant

under

smooth

monotonic

transformations

of

parameters

so

the

probabilities

are

stated

rather

than

being

translated

into

a

measure

of

time

No

correlation

between

P

-LRB-

change

-RRB-

and

P

-LRB-

indel

-RRB-

is

assumed

although

some

saving

in

stating

the

parameters

would

result

if

one

were

to

be

assumed

If

a

constant

rate

of

mutation

were

assumed

it

would

only

be

necessary

to

describe

machines

This

would

permit

an

objective

test

of

this

much

discussed

hypothesis

There

is

a

saving

of

approximately

on

the

cost

of

stating

parameters

but

presumably

a

rise

in

the

cost

of

describing

the

strings

An

intermediate

theory

hypothesising

a

near

constant

rate

of

mutation

is

also

possible

with

different

tradeoffs

An

iterative

scheme

similar

to

that

used

for

twostrings

is

used

to

estimate

the

parameter

values

of

the

machines

-LRB-

edges

-RRB-

This

is

described

later

Tuple

Costs

Here

we

consider

the

tree

topology

and

parameters

to

be

xed

The

probability

of

an

alignment

depends

on

the

probabilities

of

the

k-tuples

of

characters

forming

the

alignment

Each

tuple

can

be

thought

of

as

an

instruction

of

a

k-string

generation

machine

that

generates

all

of

the

k

strings

in

parallel

This

machine

is

implied

by

the

machines

on

the

edges

of

the

tree

The

probability

of

a

tuple

can

be

found

by

assigning

its

characters

-LRB-

possibly

-RRB-

to

the

leavesofthe

tree

and

considering

all

explanations

of

the

tuple

An

explanation

makes

assignments

of

characters

to

the

internal

hypothetical

characters

of

the

tree

This

de?nes

the

operation

on

each

edge

and

a

probability

can

be

calculated

for

the

explanation

The

probability

of

the

tuple

is

the

sum

of

the

probabilities

of

all

of

its

explanations

The

message

length

of

the

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

tuple

is

of

its

probability

We

rst

consider

the

case

where

the

mutation-model

on

twostrings

is

a

model

Felsenstein

-LSB-

-RSB-

gav

e

the

necessary

algorithm

to

calculate

the

probability

of

a

tuple

of

characters

given

the

probability

of

a

change

on

each

edge

of

the

tree

It

ef?ciently

sums

the

probabilities

of

all

explanations

for

a

tuple

overall

possible

assignments

to

internal

hypothetical

characters

Foreach

edge

all

possible

assignments

to

the

characters

at

both

ends

are

considered

The

time

complexity

is

O

-LRB-

k

alphabet

-RRB-

for

each

tuple

The

algorithm

can

be

used

directly

by

allowing

the

possibility

of

indels

Care

must

be

taken

that

only

one

insert

is

allowed

per

explanation

of

a

tuple

as

an

insert

begins

a

newtuple

The

tuple

's

probability

is

computed

in

a

post-order

traversal

of

the

tree

The

probabilities

of

hypothetical

characters

and

of

machine

operations

giventhe

tuple

-LRB-

-RRB-

are

computed

in

a

subsequent

pre-order

traversal

Atable

of

tuple

probabilities

can

be

precomputed

for

small

k

If

k

is

larger

probabilities

can

be

computed

as

tuples

are

encountered

in

alignments

and

stored

in

a

hash-table

for

reuse

There

is

a

considerable

saving

when

n

alphabet

This

is

the

technique

used

in

the

prototype

program

Sankoffand

Cedergren

-LSB-

Sank

-RSB-

give

a

similar

algorithm

that

nds

the

single

most

probable

explanation

for

each

tuple

this

is

too

detailed

a

hypothesis

in

general

and

is

another

case

where

a

max

over

probabilities

should

be

replaced

with

a

Howeverthe

complexity

of

their

algorithm

is

only

O

-LRB-

k

-RRB-

per

tuple

It

givesagood

approximation

when

all

mutation

probabilities

are

small

and

all

edges

are

similar

The

situation

is

more

complexwhen

the

model

relating

twostrings

has

more

than

one

state

The

model

giving

linear

indel

costs

is

of

particular

interest

There

are

patterns

of

indels

possible

in

atuple

-LRB-

-RRB-

The

k-string

generation

machine

therefore

has

states

-LRB-

is

also

the

number

of

neighbours

that

each

cell

depends

on

in

a

k-dimensional

DPAmatrix

-RRB-

Each

tuple

can

therefore

occur

in

contexts

with

a

different

probability

in

each

one

This

is

a

serious

problem

unless

a

simpli?cation

can

be

found

There

is

some

hope

because

the

probabilities

of

instructions

in

these

states

are

not

independent

as

there

are

only

O

-LRB-

k

-RRB-

free

parameters

in

the

tree

Alignment

Message

Length

The

message

for

the

multiple-alignment

itself

states

-LRB-

i

-RRB-

the

length

of

the

alignment

and

-LRB-

ii

-RRB-

the

sequence

of

tuples

in

the

alignment

The

former

is

encoded

under

the

log

prior.The

message

length

of

a

tuple

was

giveninthe

previous

section

It

is

therefore

easy

to

compare

twomultiple-alignments

and

conclude

which

is

best

on

the

basis

of

message

length

The

dif?culty

is

to

nd

a

good

alignment

Optimal

Multiple-Alignment

The

dynamic-programming

algorithm

-LSB-

DPA

-RSB-

can

be

naive

ly

extended

to

k

strings

to

nd

an

optimal

multiple-alignment

The

k-dimensional

DPAmatrix

has

O

-LRB-

nk

-RRB-

entries

Each

entry

depends

on

neighbours

If

the

mutation-model

has

one

state

the

extension

is

straightforward

and

the

time

complexity

is

O

-LRB-

nk

k

-RRB-

Only

O

-LRB-

-RRB-

space

is

needed

to

calculate

the

value

of

the

DPA

The

nk

factor

becomes

painful

in

both

time

and

space

for

n

and

k

as

small

as

three

It

is

necessary

to

reduce

the

volume

of

the

DPAmatrix

that

is

searched

to

makethe

algorithms

practical

and

there

are

a

number

of

possible

approaches

Altschul

and

Lipman

-LSB-

Alts

-RSB-

devised

a

safe

method

to

restrict

the

volume

investigated

for

an

editdistance

based

on

xed

tree-costs

-LRB-

theyalso

give

a

method

for

star-costs

-RRB-

-LRB-

The

tree-cost

of

a

tuple

is

the

minimum

number

of

mutations

required

to

give

the

tuple

summing

overall

tuples

givesthe

tree-cost

of

an

alignment

-RRB-

Pair-wise

alignments

are

done

rst

and

used

to

delimit

a

volume

that

is

guaranteed

to

contain

an

optimal

multiple-alignment

However

noresults

are

available

to

indicate

howpractical

the

method

is

for

tree-costs

The

method

relies

on

the

triangle

inequality

and

it

might

be

applicable

to

the

MML

calculations

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

Arelated

heuristic

approach

is

to

use

alignment-regions

as

fuzzy

strings.Pair-wise

alignments

are

performed

on

sibling

leaf

strings

Athreshold

-LRB-

in

edit-distance

or

in

message

length

-RRB-

is

applied

to

limit

the

area

of

each

alignment

Fuzzy

strings

are

aligned

in

the

obvious

way

until

a

complete

multiple-alignment

is

obtained

The

method

is

fast

if

the

threshold

is

tight

but

is

not

guaranteed

to

nd

an

optimal

alignment

We

have

done

some

experiments

with

such

an

algorithm

using

xed

tree-costs

Wi

th

a

threshold

of

eight

strings

of

characters

can

be

aligned

on

a

-LRB-

With

this

kind

of

complexity

the

model

of

computer

makes

only

a

small

difference

to

the

practical

limits

-RRB-

Hein

-LSB-

Hein

-RSB-

gav

e

amultiple

alignment

algorithm

which

uses

directed

acyclic

graphs

-LRB-

DAGs

-RRB-

to

represent

fuzzy

ancestral

strings

The

algorithm

is

fast

but

an

optimal

alignment

is

not

guaranteed

Fixed

tree-costs

are

used

but

this

is

by

no

means

necessary.The

value

of

DAGsisthat

in

sections

where

there

is

consensus

between

the

strings

the

effective

number

of

neighbours

of

a

cell

in

the

DPA

matrix

is

much

reduced

Long

exact

matches

between

strings

are

fast

to

nd

using

hash-tables

or

suf?x

trees

Heuristics

for

multiple-alignment

based

on

splitting

the

strings

on

a

long

exact

match

ideally

near

their

centres

are

clearly

possible

The

prototype

program

operates

in

twostages

During

an

initial

post-order

traversal

strings

and

then

alignments

are

aligned

recursively

until

a

complete

multiple-alignment

is

obtained

procedure

RA

-LRB-

var

Result

Alignment

T

Tree

-RRB-

var

Left

Right

Alignment

begin

if

leaf

-LRB-

T

-RRB-

then

StringToAlignment

-LRB-

T

s

Result

-RRB-

else

RA

-LRB-

Left

T

left

-RRB-

RA

-LRB-

Right

T

right

-RRB-

Align

-LRB-

Left

Right

Result

-RRB-

end

if

end

The

machine

parameters

are

held

constant

at

some

reasonable

but

otherwise

arbitrary

values

during

the

initial

stage

An

improvement

stage

follows

During

asecond

post-order

traversal

the

complete

alignment

is

projected

onto

subalignments

which

are

realigned

procedure

Improve

-LRB-

var

Algn

Alignment

-RRB-

var

A

B

Alignment

ss

StringSet

procedure

Im

-LRB-

T

Tree

var

ss

StringSet

-RRB-

var

StringSet

begin

if

not

leaf

-LRB-

T

-RRB-

then

Im

-LRB-

T

left

-RRB-

Im

-LRB-

T

right

-RRB-

Project

-LRB-

Algn

A

B

-RRB-

Align

-LRB-

A

B

Algn

-RRB-

if

AllStrings

then

Project

-LRB-

Algn

A

B

-RRB-

Align

-LRB-

A

B

Algn

-RRB-

end

if

else

-LCB-

leaf

-LRB-

T

-RRB-

-RCB-

ss

-LSB-

T

s

-RSB-

end

if

end

begin

Im

-LRB-

T

ss

-RRB-

end

The

machine

parameters

are

continually

reestimated

during

this

second

stage

The

overall

process

takes

O

-LRB-

-RRB-

time

and

global

optimality

is

not

guaranteed

The

improvement

stage

can

be

iterated

More

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

research

needs

to

be

done

on

strategies

for

optimal

multiple-alignment

Note

that

if

the

mutation-model

on

twostrings

has

three

states

to

model

linear

indel

costs

the

k

string

generation

machine

has

states

and

each

entry

in

the

DPAmatrix

has

entries

This

also

adds

amultiplicative

factor

of

to

the

time

complexity

Sum

overall

Alignments

The

part

of

the

message

that

describes

the

strings

proper

is

ideally

based

on

all

alignments

The

number

of

bits

required

to

state

the

strings

is

of

the

sum

of

the

probabilities

of

all

alignments

This

can

be

computed

by

a

logplus

dynamic

programming

algorithm

It

suffers

from

the

same

combinatorial

explosion

as

nding

an

optimal

alignment

Fortunately

most

of

the

probability

is

concentrated

in

a

narrowregion

which

raises

hopes

for

a

fast

approximate

algorithm

-LRB-

This

has

been

used

to

implement

a

fast

approximate

r-theory

algorithm

for

twostrings

-LSB-

-RSB-

-RRB-

Feng

-LSB-

-RSB-

has

performed

experiments

on

three

strings

First

probability

density

plots

are

calculated

for

pair-wise

comparisons

These

are

thresholded

to

give

a

small

alignment

region

The

regions

are

projected

into

the

DPAcube

The

r-theory

is

calculated

overthis

volume

Afurther

approach

that

we

intend

to

implement

is

to

to

sum

overall

alignments

in

a

region

within

a

distance

w

of

a

multiple-alignment

found

as

described

in

the

previous

section

Note

that

if

the

generation

machine

has

s

states

pairs

of

states

must

be

considered

at

each

transition

So

if

linear

indel

costs

are

modelled

a

multiplicative

f

actor

of

-LRB-

-RRB-

orO

-LRB-

-RRB-

is

included

This

is

a

severe

problem

unless

a

simpli?cation

or

an

approximation

can

be

found

Estimating

Edge

Parameters

The

probability

of

a

tuple

can

be

calculated

as

described

previously

if

all

the

edge

parameters

are

known

An

iterative

scheme

is

used

to

estimate

unknown

parameters

it

is

a

generalisation

of

the

scheme

used

for

twostrings

Initial

values

are

assumed

and

used

to

nd

and

to

compute

the

message

length

of

a

multiple

alignment

-LRB-

say

-RRB-

and

hence

of

the

tree

hypothesis

For

each

tuple

the

frequencies

of

operations

for

each

machine

are

calculated

These

frequencies

are

accumulated

and

used

to

calculate

the

instruction

probabilities

for

the

next

iteration

The

newvalues

can

not

increase

the

message

length

of

the

multiplealignment

but

theymay

cause

another

alignment

to

be

become

optimal

but

this

must

reduce

the

message

length

further

so

the

process

converges

Atest

program

has

been

written

to

investigate

the

estimation

process

alone

for

the

model

It

assumes

that

a

long

and

perfect

alignment

is

known

its

tuples

occurring

with

their

predicted

frequencies

It

then

attempts

to

estimate

the

true

machine

parameters

from

arbitrary

starting

values

Convergence

is

rapid

The

message

length

usually

converges

to

within

of

the

optimum

value

within

three

iterations

All

parameters

are

reestimated

simultaneously.Note

that

the

perfect

alignment

would

be

sub-optimal

Its

statistical

properties

could

only

derive

from

a

weighted

average

overall

alignments

This

test

program

is

also

used

to

calculate

limits

on

acceptable

hypotheses

for

asymptotically

long

strings

The

overhead

of

stating

parameter

costs

and

string

lengths

becomes

a

vanishing

fraction

of

the

total

message

length

for

very

long

strings

and

can

be

ignored

Brie?y

the

message

length

of

the

null-theory

is

calculated

and

the

critical

point

is

where

this

equals

the

message

length

of

the

perfect

alignment

For

example

when

the

machines

in

a

tree

are

all

identical

and

P

-LRB-

change

-RRB-

=P

-LRB-

indel

-RRB-

then

the

following

results

are

obtained

for

different

values

of

k

k

critical

P

-LRB-

copy

-RRB-

The

critical

values

of

P

-LRB-

copy

-RRB-

would

be

higher

for

short

strings

Theywould

be

lower

if

the

probabilities

of

all

alignments

were

added

-LSB-

-RSB-

givesempirical

results

for

twostrings

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

The

estimation

scheme

described

above

has

been

built

into

the

prototype

alignment

program

It

will

also

be

used

in

anyr-theory

algorithm

in

which

case

weighted

frequencies

will

be

used

for

estimation

The

programs

do

not

correct

for

the

invisible

tuple

which

has

a

probability

of

approximately

-LRB-

-RRB-

p

-LRB-

del

-RRB-

This

results

in

an

underestimate

of

indels

by

a

similar

amount

but

this

is

considerably

less

than

the

accuracywith

which

parameters

can

be

estimated

Null-Theory

The

null-theory

for

the

k

strings

is

that

theyare

not

related

Its

statement

consists

of

-LRB-

i

-RRB-

the

value

of

k

-LRB-

ii

-RRB-

the

lengths

of

the

strings

and

-LRB-

iii

-RRB-

the

characters

in

the

strings

Part

-LRB-

iii

-RRB-

requires

twobits

per

character

for

DNAifall

characters

are

equally

probable

Part

-LRB-

ii

-RRB-

is

potentially

a

large

penalty

overatree

hypothesis

because

the

latter

states

only

a

single

length

of

an

instruction

sequence

Areasonable

approach

which

mitigates

this

penalty

istotransmit

the

total

length

of

the

strings

encoded

under

the

log

prior

followed

by

the

set

of

lengths

according

to

a

k-nomial

probability

distribution

The

second

part

is

short

if

the

lengths

are

similar.This

generalises

the

binomial

distribution

used

for

twostrings

in

-LSB-

-RSB-

The

total

message

length

of

the

null-theory

is

therefore

MLnull

-LRB-

P

-LRB-

k

-RRB-

-RRB-

-LRB-

P

-LRB-

total

-RRB-

-RRB-

-LRB-

P

-LRB-

lengthk

total

-RRB-

-RRB-

total

where

total

lengthk

Anyhypothesis

that

has

a

message

length

longer

than

that

of

the

null-theory

is

not

acceptable

The

previous

section

included

predictions

of

critical

values

of

P

-LRB-

copy

-RRB-

for

asymptotically

long

strings

and

some

special

trees

Extensive

tests

have

not

been

done

but

when

the

alignment

program

is

run

on

arti?cially

generated

data

the

results

of

the

signi?cance

test

are

consistent

with

the

predictions

In

particular

random

strings

are

alwa

ys

detected

as

being

unrelated

Results

shows

the

results

of

the

prototype

program

on

a

small

example

All

three

trees

are

acceptable

hypotheses

but

tree

most

plausible

by

a

signi?cant

margin

All

trees

are

un-rooted

Note

that

little

bene?t

is

gained

from

the

hash-table

for

such

short

strings

givesresults

on

transthyretin

DNAand

RNAsequences

-LSB-

Duan

-RSB-

The

grouping

-LRB-

-LRB-

human

sheep

-RRB-

-LRB-

mouse

rat

-RRB-

-RRB-

is

favoured

on

those

four

sequences

Joining

chicken

to

the

edge

between

the

twopairs

is

narrowly

the

best

option

There

is

some

debate

overthe

branching

pattern

of

human

chimpanzee

and

gorilla

in

evolutionary

trees

for

the

primates

shows

results

on

globin

pseudo-gene

sequences

from

primates

They

favour

the

grouping

-LRB-

gorilla

-LRB-

human

chimp

-RRB-

-RRB-

and

are

consistent

with

the

conclusions

of

Holmes

-LSB-

Holm

Ch

-RSB-

on

these

sequences

First

the

three

trees

overgorilla

human

chimpanzee

and

owlmonkey

sequences

were

evaluated

Owl-monkey

was

a

consistent

outsider.The

orangutan

sequence

was

then

added

and

three

trees

were

evaluated

Conclusions

We

h

av

e

shown

howtouse

MML

encoding

to

infer

evolutionary-trees

and

multiple-alignments

We

have

argued

that

these

problems

are

intimately

connected

The

posterior

probability

of

a

tree

depends

on

the

sum

of

the

implied

probabilities

of

all

multiple-alignments

In

this

we

agree

with

the

sentiment

but

not

the

detailed

methods

of

Hogewegand

Hesper

-LSB-

Hoge

-RSB-

that

the

alignment

of

sets

of

sequences

and

the

construction

of

phyletic

trees

can

not

be

treated

separately

The

use

of

a

single

multiple-alignment

leads

to

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

an

underestimate

of

the

posterior

probability

of

the

tree

hypothesis

and

a

bias

in

the

parameter

estimates

Nevertheless

it

is

a

good

approximation

when

the

strings

are

similar.Aprototype

program

of

this

kind

has

been

written

for

the

model

The

MML

method

can

compute

the

posterior

odds-ratios

of

competing

hypotheses

such

as

different

trees

The

ev

olutionary-tree

hypothesis

is

explicitly

built

into

the

message

The

existence

of

a

natural

nulltheory

leads

to

an

in-built

signi?cance

test

These

remarks

are

quali?ed

with

subject

to

the

validity

of

the

underlying

model

The

MML

method

requires

the

model

to

be

de?ned

explicitly.Inprinciple

two

competing

models

can

be

compared

objective

ly

this

was

done

for

twostrings

in

-LSB-

-RSB-

The

most

signi?cant

doubts

about

the

model

used

are

as

follows

Wehav

e

not

as

yet

modelled

different

transition/transversion

rates

but

this

would

be

straightforward

What

seems

to

be

more

dif?cult

is

to

bring

a

multi-state

mutation

model

into

multiple-alignment

Complexmutation

events

such

as

transpositions

duplications

and

reversals

could

be

modelled

in

principle

although

nding

an

optimal

alignment

would

become

much

harder.Itmust

also

be

pointed

out

that

what

is

completely

lacking

from

our

method

and

from

most

-LRB-

all

-RRB-

sequence

analysis

programs

as

yet

is

anymodel

of

the

pressure

of

selection

The

program

discussed

above

iswritten

in

Pascal

and

is

ava

ilable

free

for

non-pro

t

non-classi

ed

teaching

and

research

purposes

by

email

-LRB-

only

-RRB-

from

lloyd@bruce.cs.monash.edu.au

The

alignment

programs

for

twostrings

based

on

and

models

of

relation

and

discussed

in

-LSB-

-RSB-

are

written

in

Cand

are

also

available

At

least

a

good

work-station

with

oating

point

hardware

is

required

for

their

use

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

References

-LSB-

-RSB-

Allison

L.

C.

N.

Yee

Minimum

message

length

encoding

and

the

comparison

of

macro-molecules

Bull

Math

Biol

-LRB-

-RRB-

-LSB-

-RSB-

Allison

L.

C.

S.

Wallace

C.

N.

Yee

Inductive

inference

overmacro-molecules

AAAI

Spring

Symposium

on

A.I.

Mol

Biol

Stanford

March

-LRB-

extended

abstract

-RRB-

and

TR

Dept.

Comp

Sci

Monash

University

-LRB-

in

full

-RRB-

-LSB-

-RSB-

Allison

L.

Du

Xiaofeng

Relating

three

strings

by

minimum

message

length

encoding

-LRB-

abstract

-RRB-

Genes

Proteins

and

Computers

Chester

April

-LSB-

Alts

-RSB-

Altschul

S.

F.

D.J.Lipman

Trees

stars

and

multiple

biological

sequence

alignment

SIAM

J.

Appl

Math

-LRB-

-RRB-

197-209 1989

-LSB-

Bish

-RSB-

Bishop

M.

J.

E.

A.

Thompson

Maximum

likelihood

alignment

of

DNAsequences

J.

Mol

Biol

90 159-165 1986

-LSB-

-RSB-

Boulton

D.

M.

C.

S.

Wallace

The

information

content

of

a

multistate

distribution

J.

Theor.Biol

23 269-278 1969

-LSB-

-RSB-

Boulton

D.

M.

C.

S.

Wallace

A

program

for

numerical

classi?cation

Comp

J.

-LSB-

-RSB-

Boulton

D.

M.

C.

S.

Wallace

An

information

measure

for

hierarchic

classi?cation

Comp

J.

16 254-261 1973

-LSB-

Chee

-RSB-

Cheeseman

P.

B.Kanefsky

Evo

lutionary

tree

reconstruction

-LRB-

extended

abstract

-RRB-

AAAI

Spring

Symposium

on

the

Theory

and

Application

of

Minimal-Length

Encoding

Stanford

March

-LSB-

Duan

-RSB-

Duan

W.

M.

G.

Achen

S.

J.

Richardson

M.

C.

Lawrence

R.

E.

H.

Wettenhall

A.

Jaworowski

G.

Schreiber

Isolation

characterization

cDNAcloning

and

gene

expression

of

an

avian

transthyretin

implications

for

the

evolution

of

the

structure

and

function

of

transthyretin

in

vertebrates

Euro

J.

Biochem

to

appear

-LSB-

-RSB-

Felsenstein

J.

Evolutionary

trees

from

DNAsequences

a

maximum

likelihood

approach

J.

Mol

Evol

17 368-376 1981

-LSB-

-RSB-

Felsenstein

J.

Inferring

evo

lutionary

trees

from

DNAsequences

Statistical

Analysis

of

DNASequence

Data.B.S.Weir

-LRB-

ed

-RRB-

Marcel

Dekker

133-150 1983

-LSB-

Frid

-RSB-

Friday

A.

Quantitative

a

spects

of

the

estimation

of

evolutionary

trees

Folia

Primatol

53 221-234 1989

-LSB-

-RSB-

Gotoh

O

An

improvedalgorithm

for

matching

biological

sequences

J.

Mol

Biol

162 705-708 1982

-LSB-

-RSB-

Gotoh

O.

Optimal

sequence

alignment

allowing

for

long

gaps

Bull

Math

Biol

-LRB-

-RRB-

359-373 1990

-LSB-

Hein

-RSB-

Hein

J.

Anew

m

ethod

that

simultaneously

reconstructs

ancestral

sequences

for

anynumber

of

homologous

sequences

when

the

phylogenyisgiv

en

Mol

Biol

Evol

649-668 1989

-LSB-

Hoge

-RSB-

HogewegP

B.Hesper.The

alignment

of

sets

of

sequences

and

the

construction

of

phyletic

trees

an

integrated

method

J.

Mol

Evol

20 175-186 1984

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

-LSB-

Holm

-RSB-

Holmes

E.

C.

Pa

ttern

and

Process

in

the

Molecular

Evolution

of

the

Order

Primates

PhD

Thesis

Cambridge

University

-LSB-

Mill

-RSB-

Miller

W.

E.W.Myers

Sequence

comparison

with

concave

weighting

functions

Bull

Math

Biol

-LRB-

-RRB-

97-120 1988

-LSB-

Milo

-RSB-

Milosavljevic

A.

D.

Categorization

of

macro-molecular

sequences

by

minimum

length

encoding

University

of

California

at

Santa

Cruz

UCSC

CRL

-LSB-

Reic

-RSB-

Reichert

T.A.

D.

N.

Cohen

K.

C.

Wong

An

application

of

information

theory

to

genetic

mutations

and

the

matching

of

polypeptide

sequences

J.

Theor.Biol

42 245-261 1973

-LSB-

Sank

-RSB-

SankoffD

R.J.Cedergren

Simultaneous

comparison

of

three

or

more

sequences

related

by

a

tree

Time

Warps

String

Edits

and

Macromolecules

The

Theory

and

Practice

of

Sequence

Comparison.D.Sankoff

J.

B.

Kruskal

-LRB-

eds

-RRB-

Addison

-LSB-

Sell

-RSB-

Sellers

P.H.Onthe

theory

and

computation

of

evolutionary

distances

SIAM

J.

Appl

Math

-LRB-

-RRB-

787-793 1974

-LSB-

-RSB-

Wallace

C.

S.

D.

M.

Boulton

An

information

measure

for

classi?cation

Comp

J.

11 185-195 1968

-LSB-

-RSB-

Wallace

C.

S.

P.R.Freeman

Estimation

and

inference

by

compact

coding

J.

Royal

Stat

Soc

B

-LRB-

-RRB-

240-265 1987

L.A.

Computer

Science

Monash

University

MML

Encoding

Evolutionary

Trees

and

Multiple-Alignment

string

A

TAATACTCGGC

string

B

TATAACTGCCG

mutation

instructions

copy

change

-LRB-

ch

-RRB-

NB

ch

differs

from

the

corr

char

in

string

A.

del

ins

-LRB-

ch

-RRB-

amutation

sequence

copy

copy

delete

copy

copy

ins

-LRB-

A

-RRB-

copy

copy

del

copy

change

-LRB-

C

-RRB-

copy

ins

-LRB-

G

-RRB-

generation

instructions

match

-LRB-

ch

-RRB-

mismatch

-LRB-

chA

chB

-RRB-

NB

chA?chB

insA

-LRB-

chA

-RRB-

insB

-LRB-

chB

-RRB-

ageneration

sequence

match

-LRB-

T

-RRB-

match

-LRB-

A

-RRB-

insA

-LRB-

A

-RRB-

match

-LRB-

T

-RRB-

match

-LRB-

A

-RRB-

insB

-LRB-

A

-RRB-

match

-LRB-

C

-RRB-

match

-LRB-

T

-RRB-

insA

-LRB-

C

-RRB-

match

-LRB-

G

-RRB-

mismatch

-LRB-

G

C

-RRB-

match

-LRB-

C

-RRB-

insBG

equivalent

alignment

TAATA-CTCGGC

TA-TAACT-GCCG

Basic

Models

TATAA

TA-TAA

T-ATAA

TAATA

TAATA

TAAT-A

etc.

CGGC

CGGC

CGGC

GCCG

GCCG

GCCG

CGGC

CGGC

CG-GC

GCCG

GCCG

GC-CG

CGGC

GC

CG

etc

Possible

Sub-alignments

L.A.

Computer

Science

Monash

University

TATAACTGCCG

T

A

A

T

A

C

T

C

G

G

C

machine

key

MML

plus

bits

algn

bits

null-theory

bits

r-theory

bits

indel

probability

related

Probability

Density

Plot

P

MA

MB

AB

Common

Ancestor

LA

Generation

Explanations

and

Rates

Instruction

match

-LRB-

x

-RRB-

xx

x

x

x

x

y

x

pA

-LRB-

c

-RRB-

pB

-LRB-

c

-RRB-

pA

-LRB-

ch

-RRB-

pB

-LRB-

ch

-RRB-

mismatch

-LRB-

x

y

-RRB-

xy

x

x

y

x

y

y

pA

-LRB-

c

-RRB-

pB

-LRB-

ch

-RRB-

pA

-LRB-

ch

-RRB-

pB

-LRB-

c

-RRB-

x

z

y

pA

-LRB-

ch

-RRB-

pB

-LRB-

ch

-RRB-

insA

-LRB-

x

-RRB-

x

x

x

x

y

pA

-LRB-

c

-RRB-

pB

-LRB-

d

-RRB-

pA

-LRB-

ch

-RRB-

pB

-LRB-

d

-RRB-

x

pA

-LRB-

i

-RRB-

insB

-LRB-

x

-RRB-

x

x

x

y

x

pA

-LRB-

d

-RRB-

pB

-LRB-

c

-RRB-

pA

-LRB-

d

-RRB-

pB

-LRB-

ch

-RRB-

x

pB

-LRB-

i

-RRB-

x

invisible

pA

-LRB-

d

-RRB-

pB

-LRB-

d

-RRB-

these

do

not

appear

c-copy

ch-change

i-insert

d-delete

define

pA

-LRB-

instn

-RRB-

pA

-LRB-

instn

-RRB-

-LRB-

-LRB-

i

-RRB-

-RRB-

pB

-LRB-

instn

-RRB-

pB

-LRB-

instn

-RRB-

-LRB-

-LRB-

i

-RRB-

-RRB-

rates

per

character

of

P

Note

pA

-LRB-

c

-RRB-

pA

-LRB-

ch

-RRB-

pA

-LRB-

d

-RRB-

pB

-LRB-

c

-RRB-

pB

-LRB-

ch

-RRB-

pB

-LRB-

d

-RRB-

sum

rates

-LRB-

pA

-LRB-

i

-RRB-

pB

-LRB-

i

-RRB-

-RRB-

-LRB-

-LRB-

i

-RRB-

pB

-LRB-

i

-RRB-

-RRB-

-LRB-

-LRB-

-LRB-

i

-RRB-

-RRB-

-LRB-

-LRB-

i

-RRB-

-RRB-

-RRB-

Explanations

for

Generation

Instructions

LA

Actual

machine

parameters

copy

change

indel

AA

A

A

C

C

CC

Probable

hypothetical

characters

Prior

tuple

probability

Estimated

operations

carried

out

copy

change

indel

-LRB-

Probabilities

less

than

omitted

-RRB-

Example

explanations

for

tuple

ACAC

string

A

A-A-A

string

B

B

BB

string

C

C

CC

Indel

Patterns

k

LA

Input

acgtacgtacagt

string

actgtacgtacgt

acgtactagct

accgtactgagct

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

Tree

M.L.

bits

estimate

of

parameters

-LSB-

-RSB-

ac

gtacgt-acagt

-LSB-

-RSB-

ac-tgtacgt-ac-gt

-LSB-

-RSB-

ac

gtac-t-ag-ct

-LSB-

-RSB-

acc-gtac-tgag-ct

Tree

M.L.

bits

Tree

M.L.

bits

Null-Theory

bits

NB

all

trees

are

unrooted

CPU

time

secs/tree

-LRB-

Pyramid

-RRB-

Example

Run

LA

HUMPALA

Human

prealbumin

-LRB-

transthyretin

-RRB-

mRNA

complete

cds

RATPALTA

Rat

prealbumin

-LRB-

transthyretin

-RRB-

mRNA

complete

MUSPALB

Mouse

mRNA

for

prealbumin

sheep

Transythyretin

cDNA

chicken

TTR

cDNA

transthyretin

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Null-Theory

bits

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-LRB-

-RRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-LRB-

-RRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

incl

length

params

alignment

Null-Theory

bits

estimate

of

parameters

copy

change

indel

CPU

time

approx

min/tree

-LRB-

sparc

station

-RRB-

Transthyretin

DNA/RNAsequences

LA

GGBGLOP

Gorilla

beta-type

globin

pseudogene

HSBGLOP

Human

beta-type

globin

pseudogene

standard

DNA

PRI

BP

PTBGLOP

Chimpanzee

beta-type

globin

pseudogene

standard

DNA

PRI

BP

Owl-monkey

psi

beta

globin

pseudogene

bp

ds-DNA

ORAHBBPSE

Orangutan

bases

to

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Null-Theory

bits

Elapsed

time

min

's

-LRB-

Sparc

station

-RRB-

Tree

-LRB-

-LRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

Tree

-LRB-

-LRB-

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

-RRB-

M.L.

bits

incl

length

params

alignment

Null-Theory

bits

NB

all

trees

are

un-rooted

Elapsed

time

min

's

-LRB-

Sparc

station

-RRB-

estimate

of

parameters

copy

change

indel

GGB

HSB

ATR

PTB

ORA

Primate

Globin

Pseudo-genes

LA

evo lutionarytree
minimum message/ description length
multiple-alignment
phylogenetic tree


A Remark on Algorithm AS57 -LCB- Printing Multidimensional Tables By Tim Hopkins and David Morse Computing Laboratory , University of Kent , Canterbury , Kent CT2 7NF , UK . LANGUAGE Fortran 77 . DESCRIPTION AND PURPOSE This routine is a Fortran 77 version of that appearing in Griffiths and Hill -LRB- 1985 -RRB- -LRB- originally Haberman -LRB- 1973 -RRB- -RRB- . It uses character variables rather than storing character information in integers and reals which was the only way of implementing such an algorithm in standard Fortran 66 . Where possible data statements have been replaced by parameter statements . The code has been restructured and variable names have been altered to avoid clashes with Fortran 77 intrinsic function names and the Fortran specifier name UNIT . The algorithm prints one or more NV AR-dimensional parallel tables stored in an array TABLE of length NTAB . For example , TABLE might contain a table of observations , a table of fitted values and a table of residuals . To be specific , suppose that a three-dimensional table fnijkg has been studied , where 1 < = i < = 4 , 1 < = j < = 3 , 1 < = k < = 2 . From this investigation , a fitted table fmijkg and a residual table frijkg have been derived . For purposes of display it is desired that the printed table have the format shown in Figure 1 . In this case , TABLE is divided into three subtables as follows . The subtable corresponding to the observations fnijkg begins at TABLE -LRB- 1 -RRB- , the subtable for the fit fmijkg begins at TABLE -LRB- 4 ? 3 ? 2 + 1 = 25 -RRB- and the subtable of residuals frijkg begins at TABLE -LRB- 49 -RRB- . Each subtable is arranged in standard Fortran fashion ; that is , TABLE -LRB- 1 -RRB- = n111 , TABLE -LRB- 2 -RRB- = n211 , TABLE -LRB- 3 -RRB- = n311 , etc. -LRB- Note : that standard Fortran 77 would allow TABLE in the calling routine to be a four-dimensional array dimensioned -LRB- 4,3,2,3 -RRB- where the last dimension denotes the number of tables . -RRB- To permit proper printing of the table , the algorithm requires descriptive information concerning the structure of the table , and information which indicates how the table should be displayed . The basic structural information is provided by an integer array DIM of length NVAR and an integer array LOC of length COL. The array DIM gives the number of categories in each of the Var 1 . Cat . 11 Cat . 12 Cat . 13 Cat . 14 Var 3 . Var 2 . Cat . 31 Cat . 21 Res . n111 n112 n113 n114 Fit . m111 m112 m113 m114 Obs . r111 r112 r113 r114 Cat . 31 Cat . 22 Res . n121 n122 n123 n124 Fit . m121 m122 m123 m124 Obs . r121 r122 r123 r124 Cat . 31 Cat . 23 Res . n131 n132 n133 n134 Fit . m131 m132 m133 m134 Obs . r131 r132 r133 r134 Cat . 32 Cat . 21 Res . n211 n212 n213 n214 Fit . m211 m212 m213 m214 Obs . r211 r212 r213 r214 Cat . 32 Cat . 22 Res . n221 n222 n223 n224 Fit . m221 m222 m223 m224 Obs . r221 r222 r223 r224 Cat . 32 Cat . 23 Res . n231 n232 n233 n234 Fit . m231 m232 m233 m234 Obs . r231 r232 r233 r234 Figure 1 : Sample test . table variables . Thus DIM -LRB- 1 -RRB- = 4 , DIM -LRB- 2 -RRB- = 3 and DIM -LRB- 3 -RRB- = 2 in the example . The array LOC gives the location in TABLE of each of the COL parallel tables . In the example , COL = 3 , LOC -LRB- 1 -RRB- = 1 , LOC -LRB- 2 -RRB- = 25 and LOC -LRB- 3 -RRB- = 49 . Label information is required for the table variables , the categories of each variable and the names of the parallel tables . In addition , the title for the complete display is needed . The character arrays T ITLE , V ARNAM , CATNAM and COLLAB provide these data . All label arrays are declared to be assumed size character arrays ; the variables T ITLEN and LABLEN control how much of each label is actually printed as well as the field width used for output of the labels . Thus the first T ITLEN characters of each of the NT2 lines of the title are output and the maximum length of any label is LABLEN characters . The integer constant GAP -LRB- see Adjustable constants below -RRB- controls the amount of space inserted between a label and the following data values . Of the NV AR labels of V ARNAM , the last V ERT of them are printed on the left-hand side of the page , while NVAR ? V ERT labels are printed along the top . In the example , V ERT = 2 . The body of the table is printed by use of F - conversion . The format for an entry from parallel table J is F a : b , where a is LABLEN + GAP and b is DEC -LRB- J -RRB- . The integer array DEC has length COL. When many tables are to be printed , it is sometimes useful to print several small tables on the same page . To permit this practice , the user may use the value of the line counter LINE as returned by a prior call to TABWRT along with the argument RESTOR . If RESTOR is set : F ALSE : , the table will be printed , space permitting , within the remainder of the page . If RESTOR is set : T RUE : the new table will be printed at the top of a new page . STRUCTURE SUBROUTINE TABWRT -LRB- TITLE , TITLEN , NT2 , TABLE , NTAB , DIM , NVAR , + LOC , COL , DEC , VARNAM , LABLEN , CATNAM , MAXCAT , COLLAB , + VERT , RESTOR , LINE , SKIP , PAGE , WIDTH , UNIT , IFAULT -RRB- Formal parameters T ITLE Character * -LRB- * -RRB- array -LRB- NT 2 -RRB- input : the title of the table . T ITLEN Integer input : the number of characters per line of the title . NT2 Integer input : the number of lines in the title . TABLE Real array -LRB- NTAB -RRB- input : the tables to be printed . NTAB Integer input : the number of elements in TABLE . DIM Integer array -LRB- NV AR -RRB- input : the number of categories in each variable of the table . NVAR Integer input : the number of variables -LRB- dimensions -RRB- in the table . LOC Integer array -LRB- COL -RRB- input : the locations in TABLE of the subtables to be printed . COL Integer input : the number of subtables to be printed . DEC Integer array -LRB- COL -RRB- input : the number of places to the right of the decimal point for each subtable . V ARNAM Character * -LRB- * -RRB- array -LRB- NV AR -RRB- input : the variable names . LABLEN Integer input : the maximum number of characters of each label to be printed . CATNAM Character * -LRB- * -RRB- array -LRB- MAXCAT -RRB- input : the names of the variable categories . MAXCAT Integer input : the maximum number of categories per variable . COLLAB Character * -LRB- * -RRB- array -LRB- COL -RRB- input : the table names . V ERT Integer input : the number of labels to be printed on the left side of the page . RESTOR Logical input : If : T RUE : the table will be printed on a new page , if : F ALSE : then an attempt will be made to print the table on the same page -LRB- see Description and Purpose for details -RRB- . LINE Integer input : and output : the line counter . SKIP Integer input : the number of lines between tables on the same page . PAGE Integer input : the number of lines per page . W IDTH Integer input : the number of characters per line -LRB- excluding carriage control character -RRB- . UNIT Integer input : the number designating the output device . IFAULT Integer output : 1 if NVAR , COL , or DIM is incorrectly specified ; 2 if table will not fit on the defined page ; 3 if T ITLEN > W IDTH or LABLEN > W IDTH ? GAP ; 4 if RESTORE is : F ALSE : and LINE < 0 ; otherwise . Adjustable constants The following values are defined in PARAMETER statements : MAXV AR Integer constant : maximum number of variables permitted in a table . GAP Integer constant : number of characters between label and start of data values . RESTRICTIONS None . PRECISION It does not seem worth making this algorithm adjustable for double precision . If double precision data are to be used , values should first be copied into a single precision array . REFERENCES Griffiths , P. and Hill , I. D. , editors -LRB- 1985 -RRB- Applied statistics algorithms . Chichester : Ellis Horwood . Haberman , S. J. -LRB- 1973 -RRB- Printing Multidimensional Tables . Appl . Statist , 22 , 118 -LCB- 126 .

O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O B O O O O B O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O