ALGORITHMS

FOR

THE

SANDBAG

AN

APPROACH

TO

IMPRECISE

SET

REPRESENTATION

Robert

L.

Read

Donald

S.

Fussell

and

Avi

Silberschatz

Department

of

Computer

Sciences

University

of

Texas

at

Austin

Austin

Texas

May

DEPARTMENT

OF

COMPUTER

SCIENCES

THE

UNIVERSITY

OF

TEXAS

AT

AUSTIN

austin

texas

Algorithms

for

the

Sandbag

An

Approach

to

Imprecise

Set

Representation

Robert

L.

Read

Donald

S.

Fussell

and

Avi

Silberschatz

Department

of

Computer

Sciences

University

of

Texas

Austin

TX

Email

fread

fussell

avig@cs.utexas.edu

May

Abstract

The

sandbag

expressively

models

uncertainty

imprecision

and

varying

quality

of

information

about

a

set

This

paper

describes

an

approach

to

constructing

sandbags

incrementally

from

a

set

of

constraints

called

a

fact

set

The

complexities

of

fundamental

sandbag

operations

using

the

fact

set

approach

depend

on

the

application

but

are

generally

intractable

However

by

sacrificing

optimality

reasonably

efficient

algorithms

can

be

obtained

This

paper

describes

a

data

structure

and

algorithms

that

produce

suboptimal

results

with

polynomial

worst

case

running

times

in

the

number

of

facts

Further

it

proves

that

optimal

polynomial

algorithms

exist

for

important

special

cases

It

also

suggests

uses

for

the

sandbag

such

as

as

a

general

approach

to

representing

sets

generated

by

statistical

sampling

or

as

a

component

of

a

multimedia

DBMS

The

goals

of

this

document

are

to

motivate

and

explain

the

sandbag

convince

the

reader

that

it

can

be

implemented

and

to

provide

sufficiently

detailed

data

structures

and

algorithms

that

the

reader

can

evaluate

them

and

undertake

an

implementation

of

the

sandbag

Introduction

We

define

an

imprecise

set

representation

to

be

a

representation

of

a

set

S

that

informs

us

about

the

members

of

S

but

from

which

the

membership

of

S

can

This

material

is

based

in

part

upon

work

supported

by

the

Texas

Advanced

Technology

Program

under

Grant

No

ATP

the

National

Science

Foundation

under

Grant

No

IRI

and

grants

from

the

IBM

and

Hewlett-Packard

corporations

not

necessarily

be

exactly

determined

Several

imprecise

set

representations

have

been

suggested

by

the

database

research

community

to

model

uncertainty

in

data

-LSB-

-RSB-

to

reconcile

discrepancies

in

heterogeneous

databases

and

as

a

basis

for

imprecise

computation

-LSB-

-RSB-

The

sandbag

is

an

imprecise

set

representation

that

has

been

used

as

a

basis

for

a

relational

data

model

that

addresses

the

issues

of

imprecise

computation

-LSB-

-RSB-

This

paper

takes

three

steps

towards

an

implementation

that

allows

sandbags

to

be

queried

and

incrementally

constructed

ffl

We

define

the

fact

set

approach

and

discuss

its

expressiveness

The

complexity

of

optimally

computing

sandbag

operations

is

analyzed

ffl

We

describe

a

data

structure

the

SDAG

and

its

important

invariant

ffl

We

detail

a

particular

set

of

algorithms

that

allow

suboptimal

querying

and

incremental

construction

of

the

sandbag

in

polynomialworst-case

time

The

complexities

of

fundamental

sandbag

operations

using

the

fact

set

approach

are

highly

application-dependent

and

generally

intractable

The

algorithms

presented

in

this

paper

are

suboptimal

algorithms

that

represent

a

practical

tradeoff

between

quality

and

efficiency

The

algorithms

described

in

this

paper

are

implemented

in

about

lines

of

C++

This

is

part

of

an

on-going

implementation

project

We

chose

to

report

on

these

algorithms

at

this

time

because

they

have

been

tested

experimentally

by

our

implementation

that

is

described

very

briefly

in

Section

This

paper

does

not

present

any

formal

performance

results

The

overarching

goals

of

this

document

are

to

explain

the

sandbag

convince

the

reader

that

it

can

be

implemented

and

to

provide

sufficiently

detailed

data

structures

and

algorithms

that

the

reader

can

evaluate

them

and

undertake

an

implementation

of

the

sandbag

Although

designed

as

a

basis

for

a

DBMS

supporting

high-performance

for

multimedia

and

other

very

large

database

applications

the

sandbag

applies

widely

both

inside

and

outside

the

realm

of

databases

Section

summarizes

the

conceptual

and

formal

definition

of

the

sandbag

and

describes

how

it

can

be

employed

The

fact

set

approach

to

incrementally

constructing

sandbags

is

described

in

Section

with

extensive

examples

Section

analyzes

the

complexity

of

the

fact

set

approach

It

relates

sandbag

operations

to

integer

linear

programming

and

points

out

some

consequences

of

that

relationship

Section

describes

a

data

structure

called

the

SDAG

that

implements

the

fact

set

approach

Section

discusses

the

use

of

the

SDAG

and

its

expected

performance

in

certain

situations

In

Section

we

briefly

described

the

current

state

of

our

implementation

Section

describes

related

research

We

conclude

in

Section

with

future

work

The

Sandbag

A

general

approach

to

imprecise

set

representation

is

to

represent

information

about

a

set

S

with

constraints

on

S

The

nature

of

these

constraints

determines

the

kinds

of

information

or

uncertainty

about

S

that

can

be

expressed

and

strongly

influences

the

ease

of

implementation

and

efficiency

The

sandbag

differs

from

other

imprecise

set

representations

in

that

it

constrains

the

cardinality

of

various

subsets

of

S

thereby

constraining

S

The

particular

subsets

that

are

thus

constrained

are

constructed

from

a

designed

partial

order

of

incomplete

partial

approximate

or

lowresolution

data

elements

-LSB-

-RSB-

Because

informationtheoretic

partial

orders

are

essential

to

the

definition

and

understanding

of

the

sandbag

we

summarize

the

intuition

behind

them

before

formally

defining

the

sandbag

although

some

readers

will

be

familiar

with

this

material

Information-theoretic

Partial

Orders

Data

describes

the

real

world

Some

data

is

more

descriptive

than

other

data

For

instance

the

daily

list

of

stock

volumes

opening

prices

and

closing

prices

is

more

descriptive

of

market

history

than

averages

and

indices

computed

over

many

stocks

such

as

the

Dow

Jones

Industrial

Average

Similarly

a

high

quality

audio

recording

better

describes

music

played

than

a

poor

scratchy

recording

The

more

descriptive

data

is

the

more

information

it

contains

Only

by

considering

the

meaning

of

computermanipulated

data

or

the

descriptions

of

the

real

world

we

obtain

from

it

can

we

define

the

informationtheoretic

notion

of

approximation

A

data

object

X

approximates

a

data

object

Y

if

every

world

described

by

Y

is

described

by

X

The

approximates

relation

is

a

natural

partial

order

of

data

that

could

be

called

the

goodness

or

precision

of

the

data

X

approximates

Y

if

and

only

if

Y

describes

the

world

at

least

as

well

as

than

X

and

is

consistent

with

X.

Intuitively

if

Y

describes

the

world

then

X

is

a

version

of

Y

that

tells

us

less

about

the

world

than

does

Y

but

from

which

we

will

not

draw

any

false

conclusions

The

meaning

of

data

and

hence

the

notion

of

approximation

is

always

application

dependent

We

use

the

term

resolution

synonymously

with

information

content

We

call

a

data

object

that

is

not

very

descriptive

low

resolution

and

a

more

descriptive

data

object

higher-resolution

The

highest

possible

resolution

data

object

is

called

total

or

complete

We

call

the

use

of

data

at

different

levels

of

resolution

multiresolution

abbreviated

MR.

Multiresolution

Types

We

find

it

convenient

to

organize

the

partially

ordered

data

elements

that

describe

particular

types

of

total

data

into

abstract

data

types

Definition

A

multiresolution

type

is

a

set

D

equipped

with

a

partial

order

v

having

a

unique

bottom

element

and

a

predicate

tot

D

ftrue

falseg

-LRB-

We

denote

function

application

by

an

infix

dot

Thus

tot

applied

to

x

is

written

tot

x

When

a

function

takes

more

than

one

argument

we

parenthesize

those

arguments

For

instance

if

f

takes

three

arguments

we

denote

an

application

of

it

to

x

y

and

z

by

f

-LRB-

x

y

z

-RRB-

-RRB-

Although

there

can

be

no

formal

requirement

primitive

MR

data

types

should

not

pervert

the

intention

that

the

formally

defined

relation

v

corresponds

to

the

meaning

of

approximates

presented

informally

above

The

practicality

of

a

multiresolution

type

is

application

dependent

For

use

by

a

general

multiresolution

database

management

system

the

practicality

depends

directly

on

three

factors

ffl

the

compactness

of

partial

data

elements

ffl

the

efficiency

of

calculating

the

relation

v

and

ffl

the

efficiency

of

determining

if

two

elements

have

no

upper

bound

in

common

The

importance

of

compact

partial

data

elements

is

demonstrated

in

examples

below

The

efficiency

of

calculating

the

relation

v

and

determining

if

two

elements

have

an

upper

bound

in

common

will

become

apparent

in

Section

We

illustrate

these

concepts

with

a

series

of

examples

that

demonstrate

the

diverse

potential

application

of

MR

data

types

The

first

is

the

canonical

example

of

a

partially

ordered

data

type

of

real

intervals

-LSB-

-RSB-

Example

A

real

number

can

be

approximated

by

a

real

interval

that

contains

it

We

say

a

narrower

interval

is

higher-resolution

than

a

wider

interval

The

type

of

closed

real

intervals

consists

of

all

intervals

and

is

partially

ordered

by

containment

-LSB-

a

b

-RSB-

v

-LSB-

c

d

-RSB-

a

c

b

d

where

a

b

c

and

d

are

in

R

The

total

elements

of

this

type

are

the

intervals

-LSB-

a

a

-RSB-

that

only

contain

one

real

number

-LRB-

i.e.

tot

-LSB-

-RSB-

true

-RRB-

The

bottom

element

-LRB-

-RRB-

of

this

domain

is

-LSB-

-RSB-

Similarly

an

MR

primitive

type

of

intervals

of

the

integers

Z

can

be

constructed

Example

Consider

a

multiresolution

raster

image

primitive

type

with

five

distinct

levels

of

resolution

The

JPEG

standard

-LSB-

-RSB-

supports

inherently

scalable

compression

rates

for

raster

images

An

application

programmer

might

construct

the

resolution

levels

such

that

ffl

level

images

are

encoded

with

compression

ffl

level

images

are

encoded

at

compression

ffl

level

images

are

encoded

at

compression

ffl

level

images

are

black-and-white

icon-sized

pictures

and

ffl

level

is

a

value

for

this

type

The

type

of

raster

images

consists

of

all

images

at

these

five

resolution

levels

In

contrast

to

Example

that

has

an

infinite

number

of

resolution

levels

this

example

has

only

five

resolution

levels

The

ordering

v

would

be

programmed

by

the

user

so

that

X

vY

X

Y

or

X

and

Y

are

compressed

from

the

same

picture

and

X

is

more

compressed

than

Y

Since

each

low-resolution

raster

is

in

fact

a

function

of

some

higher-resolution

raster

the

determination

of

X

vY

is

efficient

if

each

low-resolution

raster

records

a

unique

identifier

of

the

the

higher-resolution

raster

from

which

it

was

computed

The

total

elements

of

this

type

are

the

level

rasters

The

level

element

is

the

bottom

element

-LRB-

-RRB-

of

this

type

Example

Protocols

for

the

storage

of

moving

pictures

support

different

levels

of

audio

and

video

quality

For

instance

the

proposed

MPEG

-LSB-

-RSB-

standard

is

inherently

scalable

in

terms

of

megabits

per

second

of

motion

video

and

audio

within

certain

ranges

The

DVI

-LSB-

-RSB-

standard

defines

two

motion

video

quality

and

several

different

audio

qualities

p

-LSB-

-RSB-

is

an

inherently

scalable

motion

video

protocol

used

for

teleconferencing

with

limited

bandwidth

A

multiresolution

motion-video/audio

type

based

on

a

scalable

protocol

can

be

constructed

analogously

to

Example

Each

element

of

this

type

is

a

video

clip

of

a

certain

length

encoded

at

a

certain

quality

using

a

specific

protocol

The

type

might

be

designed

such

that

ffl

level

clips

are

HDTV

quality

ffl

level

clips

are

MPEG

quality

ffl

level

clips

are

p

clips

at

kilobits

per

second

ffl

level

clips

are

frame

per

second

JPEG

still

images

and

ffl

level

is

a

value

for

this

type

Any

type

of

digitally

sampled

continuous

function

such

as

sounds

or

those

that

arise

in

scientific

computing

and

seismic

prospecting

can

be

the

basis

of

an

MR

type

whose

lower-resolutions

elements

are

lossy

compressions

like

Examples

and

Multiresolution

types

of

a

more

abstract

flavor

are

described

below

Example

Archived

technical

documents

often

have

titles

list

of

keywords

abstracts

and

bodies

The

title

of

document

A

approximates

the

keyword

list

of

document

A

the

keyword

list

of

A

approximates

the

abstract

of

A

and

the

abstract

of

A

approximates

the

full

text

of

A

These

relationships

can

be

formalized

into

a

useful

multiresolution

type

Example

A

graphic

geometric

scene

such

as

the

buildings

in

a

city

is

sometimes

represented

with

solid

polyhedra

Such

a

scene

can

be

approximated

by

a

smaller

set

of

polyhedra

that

fit

the

same

volume

with

minimal

error

For

instance

large

buildings

may

be

represented

by

single

parallelpipeds

A

higher-resolution

representation

might

allow

the

representation

of

peaked

roofs

As

resolution

is

added

the

decorative

masonry

is

represented

In

future

computing

environments

each

brick

and

its

texture

might

be

modelled

A

general

approach

to

constructing

types

is

to

define

the

cross

product

of

types

as

a

new

type

This

idea

underlies

the

relation

and

tuple

concept

almost

ubiquitous

to

databases

and

is

sometimes

called

the

separated

sum

in

more

formal

treatments

-LSB-

-RSB-

This

fundamental

concept

can

be

extended

to

multiresolution

data

types

Definition

For

a

tuple-type

T

constructed

from

a

sequence

of

N

primitive

types

TN

ffl

A

tuple

x

in

T

approximates

a

tuple

y

in

T

if

and

only

if

every

component

of

x

approximates

the

corresponding

component

of

y.

xv

y

-LRB-

c

c

c

N

xc

v

yc

-RRB-

ffl

The

bottom

element

-LRB-

-RRB-

of

T

is

the

sequence

of

the

bottom

elements

from

each

component

type

of

the

tuple-type

ffl

A

tuple

is

total

if

and

only

if

every

component

of

it

is

total

tot

x

-LRB-

c

c

c

N

tot

-LRB-

x

-LSB-

c

-RSB-

-RRB-

-RRB-

We

follow

the

notation

of

Gries

-LSB-

-RSB-

for

guarded

quantification

The

expression

-LRB-

v

G

v

P

v

-RRB-

consisting

of

a

universal

quantification

through

variable

v

with

guard

G

and

predicate

P

is

more

often

written

as

-LRB-

G

-LRB-

v

-RRB-

-RRB-

P

-LRB-

v

-RRB-

-RRB-

The

meaning

of

the

guard

in

existential

quantification

is

slightly

different

-LRB-

v

G

v

P

v

-RRB-

is

the

same

as

-LRB-

G

-LRB-

v

-RRB-

P

-LRB-

v

-RRB-

-RRB-

Example

Partial

orders

and

multiresolution

tuples

can

be

constructed

by

the

user

to

serve

convenient

data

modelling

and

type

hierarchy

purposes

A

type

of

car

make

is

useful

for

representing

information

about

cars

It

has

partial

elements

sport

and

sedan

The

partial

element

sport

approximates

the

total

element

Ferrari

-LRB-

i.e.

sport

v

Ferrari

-RRB-

This

type

may

be

paired

with

a

color

type

that

contains

only

colors

that

are

total

elements

and

a

bottom

element

The

resulting

type

of

and

the

partial

order

defined

on

it

by

Definition

can

be

used

to

represent

information

about

cars

The

Interpretation

of

Partially

Ordered

Domain

as

Encodings

of

Subsets

of

the

Total

Universe

The

examples

of

the

previous

section

suggest

that

partial

data

elements

have

a

useful

meaning

in

themselves

and

that

they

can

be

useful

in

the

same

spirit

if

not

to

the

same

degree

that

total

data

elements

are

However

it

is

also

convenient

to

use

a

partial

data

element

as

a

name

for

a

particular

subset

of

the

space

of

total

data

elements

that

form

the

universe

of

discourse

This

section

connects

the

point

of

view

of

formal

semantics

to

the

point

of

view

of

data

retrieval

Except

for

the

following

definition

this

section

may

be

skipped

Definition

The

function

seta

-LRB-

A

U

-RRB-

returns

the

set

of

total

elements

in

U

above

any

element

of

A.

seta

-LRB-

A

U

-RRB-

fx

j

x

U

tot

x

-LRB-

d

d

A

d

vx

-RRB-

g

A

partial

data

element

d

can

be

thought

of

as

a

name

for

the

set

of

total

elements

seta

-LRB-

fdg

U

-RRB-

where

U

is

the

universe

of

data

elements

This

set

of

total

elements

is

often

considered

a

set

of

possibilities

that

are

indistinguishable

on

the

basis

of

limited

information

represented

by

the

partial

data

element

d

As

d

becomes

better

defined

-LRB-

i.e.

moves

up

the

partial

order

towards

totality

-RRB-

fewer

possibilities

are

consistent

with

it

When

d

is

total

there

is

only

one

consistent

possibility

This

interpretation

is

traditional

-LSB-

-RSB-

and

is

captured

by

a

theorem

that

follows

from

the

transitivity

of

v

and

Theorem

xv

y

seta

-LRB-

fxg

U

-RRB-

seta

-LRB-

fyg

U

-RRB-

We

generally

think

of

partial

elements

as

encoding

sets

of

total

elements

when

stating

formal

properties

and

discussing

semantics

and

think

of

the

partial

elements

as

intrinsically

useful

when

designing

partial

data

domains

and

algorithms

These

two

points

of

view

while

quite

different

are

in

fact

consistent

and

complementary

Contrasting

a

practically

useful

example

from

the

previous

section

with

a

partially

ordered

data

type

that

is

useful

mainly

for

theoretical

purposes

reveals

the

connection

between

these

two

approaches

Example

Consider

the

universe

of

integers

Z

A

partially

ordered

data

domain

P

is

defined

by

where

x

approximates

y

exactly

when

y

is

a

subset

of

x

-LRB-

i.e.

xvy

x

y

-RRB-

The

empty

set

is

the

only

total

element

of

this

domain

and

Z

is

the

bottom

element

The

data

domain

of

Example

is

useful

theoretically

because

most

partially

ordered

data

domains

over

countable

universes

are

isomorphic

to

some

subset

of

this

semi-lattice

However

it

is

not

very

practical

No

applications

leap

to

mind

There

are

an

infinite

number

of

partial

data

elements

each

of

which

may

require

a

great

deal

of

space

to

represent

and

thus

are

expensive

to

manipulate

In

contrast

the

domain

D

of

Example

allows

efficient

representation

of

low-resolution

or

partial

data

elements

because

it

gives

up

a

great

deal

of

representational

power

Consider

a

level

raster

image

d

The

set

of

total

images

that

are

approximated

by

this

image

-LRB-

i.e.

are

possibly

consistent

with

it

or

in

seta

fdg

:D

-RRB-

are

presumably

exactly

those

whose

color

pixels

when

averaged

and

discretized

into

black/white

pixels

produce

the

level

raster

image

This

is

a

conceptually

interesting

set

of

total

rasters

since

they

share

the

prop

erty

of

looking

similar

and

can

be

represented

-LRB-

approximately

-RRB-

by

a

compact

representation

of

the

level

raster

image

This

domain

is

finite

though

huge

and

segregated

into

only

five

levels

in

contrast

to

Example

Rasters

of

level

require

the

space

of

level

rasters

Unlike

Example

incomparable

elements

of

Example

have

no

upper

bounds

in

common

In

general

the

partially

ordered

data

domains

over

which

sandbags

are

constructed

will

have

the

property

that

partial

data

elements

are

informative

-LRB-

allow

some

computation

or

inference

to

be

carried

out

-RRB-

and

compact

-LRB-

can

be

represented

space-efficiently

-RRB-

However

the

theoretical

generality

of

our

approach

is

an

advantage

even

though

in

practice

it

will

not

be

possible

to

exploit

it

The

Sandbag

Scheme

The

sandbag

approximates

a

set

B

of

total

elements

from

a

primitive

data

type

D

by

using

partial

data

elements

of

D.

Each

partial

data

element

d

where

d

D

is

mapped

into

lower

and

upper

bounds

on

the

number

of

elements

in

B

that

are

approximated

by

d

We

call

the

number

of

total

elements

approximated

by

d

in

some

set

B

the

number

above

d

The

term

above

corresponds

to

the

convention

that

more

informative

elements

are

thought

of

as

being

above

less

informative

elements

as

in

a

Hasse

diagram

To

formalize

this

concept

we

define

the

following

Definition

Let

D

be

a

partially

ordered

data

domain

B

be

a

set

of

total

elements

from

D

and

d

be

any

data

element

from

D

The

number

of

total

elements

above

the

element

d

in

the

set

B

is

denoted

by

numa

-LRB-

B

d

-RRB-

and

defined

by

numa

-LRB-

B

d

-RRB-

jfx

j

d

vx

x

Bgj

or

equivalently

numa

-LRB-

B

d

-RRB-

jseta

-LRB-

d

D

-RRB-

Bgj

For

instance

consider

the

set

of

total

elements

B

f

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

g

from

the

partially

ordered

domain

of

real

intervals

-LRB-

Example

-RRB-

The

number

of

elements

above

the

partial

data

element

d

-LSB-

-RSB-

is

-LRB-

i.e.

numa

-LRB-

B

d

-RRB-

-RRB-

A

sandbag

describing

B

might

provide

a

lower

bound

for

d

of

and

an

upper

bound

of

These

are

consistent

with

B

but

are

not

the

tightest

consistent

bounds

which

are

a

lower

bound

of

and

an

upper

bound

of

This

capacity

for

imprecision

is

essential

to

the

sandbag

Definition

Let

be

the

set

of

naturals

-LRB-

N

-RRB-

plus

infinity

N

-LSB-

and

let

D

be

a

partially

ordered

data

domain

A

sandbag

S

is

a

pair

of

functions

denoted

mina

S

and

maxa

S

with

types

mina

S

D

and

maxa

S

D

We

write

the

application

of

the

sandbag

function

mina

S

to

a

partial

data

element

d

is

written

mina

S

:d

that

is

the

same

as

-LRB-

mina

S

-RRB-

:d

Informally

mina

S

:d

is

a

lower

bound

on

the

number

of

total

elements

in

any

set

consistent

with

S

that

are

above

d

and

maxa

S

:d

is

an

upper

bound

on

the

number

of

total

elements

in

any

set

consistent

with

S

that

are

above

d

We

now

present

a

definition

that

formalizes

the

concept

that

a

sandbag

is

consistent

with

a

set

Definition

Let

D

be

a

partially

ordered

data

domain

and

B

D

a

set

of

total

elements

We

say

that

a

sandbag

S

is

consistent

with

a

set

B

denoted

sbcons

-LRB-

S

B

-RRB-

when

-LRB-

d

d

D

mina

S

:d

numa

-LRB-

B

d

-RRB-

maxa

S

:d

numa

-LRB-

B

d

-RRB-

Extensive

examples

of

the

use

of

the

sandbag

are

given

in

Section

The

functions

mina

and

maxa

are

a

means

of

querying

a

sandbag

and

extracting

useful

information

from

it

In

practice

however

other

operations

are

required

for

the

sandbag

to

be

a

useful

class

or

abstract

data

type

In

particular

a

means

of

extracting

interesting

partial

elements

from

a

sandbag

is

required

Analogously

the

dictionary

abstract

data

type

is

usually

defined

to

support

the

operations

of

insertion

deletion

and

membership

testing

However

a

means

of

enumerating

the

elements

contained

in

a

dictionary

is

often

required

The

SDAG

data

structure

briefly

described

in

Section

solves

this

problem

for

sandbags

A

Partial

Order

of

Sandbags

Sandbags

like

primitives

types

are

partially

ordered

by

the

amount

of

information

they

provide

about

the

sets

that

they

approximate

The

tighter

the

bounds

provided

by

a

sandbag

the

more

informative

it

is

A

sandbag

P

approximates

a

sandbag

Q

if

the

bounds

provided

by

mina

and

maxa

on

Q

are

at

least

as

tight

as

and

consistent

with

those

of

P

Definition

Let

D

be

a

partially

ordered

data

domain

The

approximates

relation

v

SB

on

sandbags

is

defined

by

P

v

SBQ

-LRB-

x

x

D

mina

:P

x

mina

Q

x

maxa

:P

x

maxa

Q

x

-RRB-

The

relation

v

SB

defined

as

above

is

in

fact

a

partial

order

of

sandbags

because

it

is

reflexive

transitive

and

antisymmetric

Since

an

approximates

relation

is

defined

for

sandbags

they

meet

the

requirement

of

a

multiresolution

type

defined

in

Section

However

the

computational

complexity

of

computing

v

SB

on

sandbags

is

highly

dependent

on

the

types

of

elements

in

the

sandbags

so

sandbags

do

not

universally

form

practical

multiresolution

types

Sandbags

have

a

bottom

element

and

a

notion

of

totality

just

as

did

primitives

and

tuples

If

a

sandbag

does

not

constrain

the

cardinality

of

any

subset

then

it

provides

no

information

whatsoever

This

is

the

value

of

sandbags

in

our

model

Since

a

sandbag

S

is

a

pair

of

functions

mina

S

and

maxa

S

we

formally

define

a

sandbag

by

defining

the

value

of

mina

S

and

maxa

S

on

every

element

of

D.

Definition

The

bottom

element

of

the

sandbag

type

is

denoted

SB

and

defined

by

ffl

mina

SB

:d

ffl

maxa

SB

:d

for

every

element

d

in

a

partially

ordered

data

domain

D.

The

bottom-most

element

of

sandbags

under

the

partial

order

v

SB

is

in

fact

SB

Similarly

if

for

all

data

elements

the

upper

and

lower

bound

on

the

number

of

approximated

elements

in

a

consistent

set

are

equal

then

the

sandbag

is

total

Definition

Let

D

be

a

partially

ordered

data

domain

A

sandbag

S

is

total

denoted

tot

S

if

and

only

if

-LRB-

d

d

D

mina

S

:d

maxa

S

:d

-RRB-

Total

sandbags

are

in

fact

the

top-most

elements

of

the

type

of

sandbags

If

a

sandbag

is

total

there

is

at

most

one

set

consistent

with

it

and

the

members

of

that

set

can

be

exactly

determined

from

the

sandbag

Between

these

two

extremes

of

zero

and

total

information

a

sandbag

provides

varying

degrees

of

information

about

a

set

it

approximates

Alternatively

we

think

of

a

sandbag

as

limiting

the

number

of

sets

consistent

with

it

The

more

informative

a

sandbag

is

the

fewer

sets

are

consistent

with

it

All

sets

are

consistent

with

SB

Unfortunately

a

sandbag

need

not

be

consistent

with

any

set

because

the

values

of

mina

and

maxa

can

easily

contradict

each

other

We

say

a

sandbag

S

is

externally

consistent

if

there

exists

any

set

consistent

with

it

Definition

Let

S

be

a

sandbag

and

B

be

a

set

of

total

elements

We

say

that

S

is

externally

consistent

denoted

by

extcons

S

if

and

only

if

-LRB-

B

sbcons

-LRB-

S

B

-RRB-

-RRB-

As

might

be

expected

we

are

exclusively

concerned

with

externally

consistent

sandbags

because

we

are

only

interested

in

using

sandbags

to

inform

us

about

actual

sets

Henceforth

we

assume

all

sandbags

are

externally

consistent

Using

the

Sandbag

This

section

provides

seven

examples

of

the

use

of

the

abstract

sandbag

Each

of

these

examples

are

elaborated

in

Section

after

the

fact

set

approach

to

constructing

sandbags

is

defined

The

sandbag

can

model

sets

of

total

elements

by

simply

mapping

elements

of

the

set

to

and

nonmembers

to

Example

Given

a

universe

D

of

total

data

elements

construct

a

trivial

domain

D

D

-LSB-

f?g

and

v

define

so

that

d

v

e

d

A

total

sandbag

S

trivially

models

a

set

of

B

D

if

-LRB-

d

D

-LRB-

d

B

mina

S

:d

-RRB-

-LRB-

:d

B

maxa

S

:d

-RRB-

-RRB-

mina

S

jBj

The

sandbag

models

the

sandwich

-LSB-

-RSB-

Example

A

sandwich

is

an

imprecise

set

representation

consisting

of

a

pair

of

sets

hC

P

i

If

the

sandwich

is

approximating

a

set

B

each

member

of

C

approximates

some

member

of

B

and

each

member

of

B

approximates

some

member

of

P

The

set

B

is

thus

sandwiched

between

the

sets

C

of

elements

known

to

be

in

B

or

have

an

upper

bound

in

B

and

P

that

contains

upper

bounds

for

every

element

in

B

A

sandbag

S

models

a

sandwich

hC

P

i

if

S

is

the

least

informative

externally

consistent

sandbag

such

that

-LRB-

d

d

D

d

C

-RRB-

mina

S

:d

-RRB-

-LRB-

d

D

-LRB-

x

P

xv

d

-RRB-

-RRB-

maxa

S

:d

-RRB-

A

sandbag

can

express

incomplete

information

based

on

a

type

hierarchy

Example

IS-A

class

hierarchies

have

been

used

to

efficiently

encode

incomplete

information

in

an

imprecise

set

representation

called

approximate

relations

based

on

the

sandwich

-LSB-

-RSB-

By

constructing

a

partially

ordered

data

type

corresponding

to

a

class

hierarchy

the

sandbag

can

model

this

construct

Let

D

be

the

set

of

all

class

names

and

set

of

data

values

and

define

d

to

approximate

e

if

and

only

if

d

is

a

subclass

of

e

or

a

member

of

e

Then

maxa

S

:d

where

d

is

a

class

name

means

that

no

member

of

that

class

exists

in

any

set

consistent

with

S.

Thus

if

only

one

of

several

subclasses

has

a

non-zero

maxa

value

the

fact

that

any

value

in

the

set

is

a

member

of

that

subclass

can

be

efficiently

represented

-LSB-

-RSB-

-LRB-

sport

-RRB-

-LRB-

Corvette

-RRB-

-LRB-

Corvette

red

-RRB-

-LRB-

Ferrari

blue

-RRB-

Figure

A

sandbag

representing

facts

about

a

set

of

cars

The

following

examples

demonstrates

a

more

interesting

use

of

the

ability

of

a

sandbag

to

constrain

cardinality

Example

Suppose

it

is

known

that

a

car

rental

company

has

between

and

cars

ready

for

rental

to

of

them

are

sports

cars

of

unspecified

color

at

most

are

Corvettes

at

least

is

a

blue

Ferrari

and

exactly

is

a

red

Corvette

This

information

can

be

expressed

by

a

sandbag

In

this

example

we

use

the

partial

ordering

of

Example

and

assume

partial

orders

for

both

the

color

type

and

the

make

type

color

and

make

have

a

bottom

element

and

make

is

assumed

to

have

at

least

one

intermediate

resolution

of

car

style

of

which

sport

is

an

example

Thus

sport

v

Corvette

Sandbags

can

be

represented

diagrammatically

as

in

Figure

In

this

style

of

diagrammatic

representation

of

a

sandbag

a

node

represents

an

element

of

the

multiresolution

type

over

which

the

sandbag

is

defined

An

arrow

indicates

the

approximation

relationship

-LRB-

an

arrow

from

p

to

q

indicates

p

v

q.

Redundant

arrows

are

not

drawn

technically

only

the

transitive

reduction

of

the

approximates

relation

is

indicated

with

arrows

The

small

numbers

to

the

left

and

right

of

a

node

indicate

the

mina

and

maxa

value

of

that

node

respectively

If

a

node

p

is

drawn

above

node

q

in

the

diagram

then

node

p

is

higher

resolution

than

node

q.

Example

Consider

a

relation

of

bank

accounts

with

schema

bank

-LRB-

AcctNum

Balance

-RRB-

where

no

account

has

more

than

If

integer

intervals

are

an

MR

type

the

sandbag

could

represent

a

conglomeration

of

facts

such

as

-LRB-

a

-RRB-

At

most

five

and

at

least

four

accounts

between

account

and

have

a

balance

in

the

range

to

-LRB-

b

-RRB-

Account

has

exactly

Account

Number

d

e

a

b

c

50 100 107108

Balance

Figure

Some

facts

about

a

bank

account

relation

and

its

sandbag

-LRB-

c

-RRB-

Account

has

between

and

-LRB-

d

-RRB-

Accounts

through

have

exactly

accounts

with

less

than

-LRB-

e

-RRB-

Accounts

through

have

at

most

and

at

least

accounts

with

more

than

The

partial

order

integer

interval

is

generated

by

Definition

as

the

cross

product

of

two

integer

interval

multiresolution

types

These

facts

are

depicted

in

Figure

wherein

a

partial

data

element

is

represented

as

a

rectangle

labeled

near

its

lower

left

corner

The

mina

and

maxa

values

of

these

partial

elements

are

to

the

left

and

right

of

the

label

respectively

The

mina

and

maxa

functions

enable

queries

on

the

sandbag

to

obtain

interesting

information

The

precision

in

the

results

will

depend

on

the

quality

of

the

facts

from

which

it

is

constructed

For

instance

given

the

facts

above

ffl

A

selection

of

accounts

having

between

and

will

definitely

include

account

-LRB-

i.e.

mina

applied

to

will

produce

-RRB-

but

can

not

exclude

That

is

maxa

-LRB-

gen

F

-RRB-

-LSB-

-RSB-

and

mina

-LRB-

gen

F

-RRB-

-LSB-

-RSB-

ffl

The

number

of

accounts

in

the

range

to

with

less

than

is

at

most

-LRB-

i.e.

maxa

-LRB-

gen

F

-RRB-

-LSB-

-RSB-

-RRB-

and

we

can

not

be

sure

that

there

are

any

such

accounts

-LRB-

i.e.

mina

-LRB-

gen

F

-RRB-

-LSB-

-RSB-

-RRB-

Example

Consider

a

sandbag

S

over

the

MR

type

of

real

intervals

from

Example

consistent

with

a

set

F

of

one

million

positive

reals

each

of

which

is

between

and

A

lower

bound

of

the

sum

of

F

can

be

calculated

by

making

k

calls

to

mina

S.

Let

call

number

j

use

the

partial

data

element

-LSB-

j

k

-LRB-

j

-RRB-

k

-RSB-

Then

-LRB-

j

k

X

j

-LRB-

j

k

-RRB-

mina

S

-LSB-

j

k

-LRB-

j

-RRB-

k

-RSB-

-RRB-

-LRB-

X

x

-RRB-

The

error

of

this

underestimate

will

depend

on

the

precision

of

the

sandbag

S.

Its

usefulness

as

an

underestimate

of

P

x

will

be

application

dependent

However

since

it

can

be

computed

in

only

k

calls

to

mina

S

presumably

there

will

be

some

applications

that

benefit

from

this

underestimate

if

mina

S

can

be

computed

efficiently

An

overestimate

can

be

computed

similarly

Besides

representing

information

about

large

sets

of

points

continuous

spaces

such

as

those

of

Example

the

sandbag

can

also

represent

small

sets

of

very

large

objects

such

as

sets

of

multimedia

objects

Example

The

sandbag

can

represent

a

set

of

digitally

encoded

motion

video

sequences

at

different

resolutions

Suppose

that

a

televised

debate

is

broken

into

a

set

of

digitally

encoded

answers

each

of

which

is

-LRB-

supposedly

-RRB-

only

minutes

long

A

sandbag

can

serve

as

a

container

for

these

clips

that

remains

consistent

as

new

clips

are

transmitted

over

a

network

to

a

studio

At

some

point

in

time

two

clips

-LRB-

A

and

B

-RRB-

may

be

available

at

resolution

level

clip

D

is

known

to

have

been

destroyed

and

hence

unavailable

there

will

be

between

and

answers

in

the

entire

debate

and

no

other

information

is

currently

known

Let

D

be

the

partially

ordered

domain

of

clips

described

by

Example

Let

xi

represent

a

clip

of

level

i

depicting

scene

x

Then

mina

S

and

maxa

S

produce

the

following

results

ffl

mina

S

maxa

S

ffl

mina

S

maxa

S

ffl

mina

S

maxa

S

ffl

mina

S

maxa

S

ffl

mina

S

and

maxa

S

This

sandbag

of

Example

is

depicted

in

Figure

Example

illustrates

the

previously

mentioned

need

to

interrogate

and

enumerate

the

interesting

partial

elements

of

a

sandbag

Although

mina

S

and

maxa

S

are

powerful

functions

the

sandbag

itself

does

not

provide

any

hint

which

partial

data

elements

they

should

be

queried

with

to

obtain

interesting

results

Figure

A

sandbag

representing

Presidential

debate

footage

The

Fact

Set

Approach

The

previous

section

defines

sandbags

abstractly

but

does

not

address

their

construction

There

exists

a

spectrum

of

possible

approaches

to

constructing

the

functions

mina

and

maxa

At

one

end

the

constraints

may

be

very

general

such

as

arbitrary

predicates

in

some

logic

Such

a

system

is

difficult

or

impossible

to

efficiently

implement

At

the

other

end

of

the

spectrum

are

very

simple

constraint

mechanisms

that

can

not

conveniently

express

some

kinds

of

information

and

are

more

likely

to

be

efficiently

computable

Simple

maybe

tuples

-LSB-

-RSB-

are

representative

of

this

point

This

section

outlines

the

fact

set

approach

to

sandbag

construction

that

is

an

educated

guess

at

an

appropriate

compromise

between

implementation

efficiency

and

expressiveness

Perhaps

the

most

obvious

type

of

constraints

that

could

be

incorporated

into

a

sandbag

are

facts

similar

to

those

produced

by

the

sandbag

Definition

We

denote

the

fact

that

a

set

of

total

data

elements

B

from

a

partially

ordered

set

D

has

at

least

a

and

at

most

z

total

data

elements

above

d

by

a

triple

hd

a

zi

that

we

call

a

revelation

A

revelation

hd

a

zi

constrains

the

membership

of

a

set

in

proportion

to

the

tightness

of

the

bound

a

and

z

A

set

of

such

constraints

restricts

the

sets

consistent

with

it

in

proportion

to

their

number

tightness

and

independence

By

making

a

set

of

revelations

F

more

restrictive

the

set

of

sets

consistent

with

F

is

progressively

refined

This

capacity

for

incremental

improvement

is

essential

to

the

usefulness

of

the

sandbag

The

revelation

restricts

the

possible

membership

of

the

approximated

set

above

some

element

A

complementary

notion

is

to

restrict

the

membership

of

total

elements

not

above

some

element

It

is

common

and

convenient

to

be

able

to

express

that

if

a

total

element

is

above

some

element

d

it

is

above

one

element

of

a

set

P

each

element

of

which

is

above

d.

Definition

We

can

express

the

fact

that

a

set

B

of

total

data

elements

chosen

from

a

partially

ordered

set

D

contains

no

elements

above

d

that

are

not

above

some

element

of

a

set

P

of

partial

elements

with

a

pinning

a

tuple

of

the

form

hd

P

i

where

d

D

P

D

P

is

a

cochain

under

v

and

every

element

of

P

is

approximated

by

d

The

set

P

is

called

a

pin

set

and

the

element

d

is

called

a

pin

We

call

a

conglomeration

of

pinnings

of

the

form

hd

P

i

and

revelations

of

the

form

he

a

zi

a

fact

set

Each

fact

provides

a

grain

of

information

about

the

approximated

set

A

sandbag

is

filled

up

with

grains

of

information

like

a

sack

is

filled

up

with

grains

of

sand

Definition

A

set

B

is

consistent

with

a

fact

set

F

denoted

by

fscons

-LRB-

F

B

-RRB-

if

and

only

if

every

fact

of

F

is

true

of

B.

fscons

-LRB-

F

B

-RRB-

-LRB-

a

zi

hd

a

zi

F

a

numa

-LRB-

B

d

-RRB-

z

numa

-LRB-

B

d

-RRB-

-RRB-

-LRB-

P

i

hd

P

i

F

-LRB-

e

B

d

v

e

-LRB-

q

P

q

v

e

-RRB-

-RRB-

-RRB-

Definition

We

use

the

symbols

min

max

and

sum

both

as

functions

and

as

quantifiers

When

used

as

functions

they

are

applied

to

sets

of

extended

naturals

and

produce

a

single

extended

natural

that

is

either

the

minimum

maximum

or

sum

of

that

set

and

thus

have

the

type

When

used

as

quantifiers

the

have

the

following

meanings

ffl

-LRB-

min

x

P

x

f

x

-RRB-

min

ff

x

j

P

xg

ffl

-LRB-

max

x

P

x

f

x

-RRB-

max

ff

x

j

P

xg

ffl

-LRB-

max

x

P

x

f

x

-RRB-

-LRB-

PP

x

x

f

x

-RRB-

The

sandbag

constructed

from

a

fact

set

should

be

the

most

informative

that

guarantees

that

a

set

consistent

with

a

fact

set

is

consistent

with

the

sandbag

Definition

The

function

gen

F

maps

fact

sets

into

sandbags

and

is

defined

by

ffl

mina

-LRB-

gen

F

-RRB-

x

-LRB-

min

B

fscons

-LRB-

F

B

-RRB-

numa

-LRB-

B

x

-RRB-

-RRB-

ffl

maxa

-LRB-

gen

F

-RRB-

x

-LRB-

max

B

fscons

-LRB-

F

B

-RRB-

numa

-LRB-

B

x

-RRB-

-RRB-

Note

that

this

definition

is

not

constructive

it

is

simply

a

theoretically

useful

target

shot

at

by

an

implementation

This

definition

ties

sandbag

consistency

to

fact

set

consistency

Theorem

fscons

-LRB-

F

B

-RRB-

-RRB-

sbcons

:-LRB-

-LRB-

gen

F

-RRB-

B

-RRB-

Just

as

a

sandbag

may

be

inconsistent

a

fact

set

may

by

self-contradictory

and

therefore

not

define

a

sandbag

However

we

restrict

our

attention

only

to

fact

sets

for

which

there

exists

a

B

such

that

fscons

-LRB-

F

B

-RRB-

We

call

such

a

fact

set

externally

consistent

following

our

terminology

for

sandbags

Examples

of

Fact

Sets

The

following

extensions

to

Examples

provide

fact

sets

that

realize

the

sandbags

described

in

those

examples

Example

Given

a

universe

D

of

total

data

elements

construct

a

trivial

domain

D

D

-LSB-

f?g

and

define

v

so

that

xv

y

x

The

total

sandbag

gen

F

models

a

set

of

B

D

where

F

is

defined

by

hd

a

zi

F

d

B

a

z

-LRB-

d

a

z

a

jBj

-RRB-

Example

Given

a

sandwich

hC

P

i

construct

a

fact

set

F

by

including

the

revelation

hd

for

every

d

in

C

and

the

pinning

h

P

i

Then

a

set

B

is

fact

set

consistent

with

F

if

and

only

if

B

is

consistent

with

the

with

the

sandwich

hC

P

i.

Example

Vrbsky

and

Liu

-LSB-

-RSB-

describe

a

technique

whereby

a

block

of

tuples

are

read

from

disk

and

a

class

designator

is

replaced

by

this

set

of

tuples

thus

making

their

approximate

relational

representation

more

precise

This

can

be

modeled

by

a

fact

set

by

adding

a

revelation

hd

for

each

tuple

d

read

and

the

revealing

hc

n

ni

where

c

is

class

designator

being

upgraded

and

n

is

the

number

of

tuples

that

are

read

Example

Suppose

it

is

known

that

a

car

rental

company

has

between

and

cars

ready

for

rental

to

of

them

are

sports

cars

of

unspecified

color

at

most

are

Corvettes

at

least

is

a

blue

Ferrari

and

exactly

is

a

red

Corvette

In

this

example

we

subscript

the

bottom

elements

so

that

they

can

be

distinguished

M

is

the

bottom

element

of

the

make

type

C

is

the

bottom

element

of

the

color

type

and

R

h?M

Ci

is

the

bottom

element

of

the

tuples

used

by

the

rental

car

company

Using

the

type

of

pairs

defined

in

Example

the

sandbag

depicted

in

Figure

is

gen

F

where

F

is

f

h?R

h?R

hhsport

Ci

hhCorvette

Ci

hhFerrari

bluei

hhCorvette

redi

g.

Example

Consider

the

relation

of

bank

accounts

with

schema

bank

-LRB-

AcctNum

Balance

-RRB-

from

Example

and

the

set

of

facts

-LRB-

a

-RRB-

At

most

five

and

at

least

four

accounts

between

account

through

have

a

balance

in

the

range

to

-LRB-

b

-RRB-

Account

has

exactly

-LRB-

c

-RRB-

Account

has

between

and

-LRB-

d

-RRB-

Accounts

through

have

exactly

accounts

with

less

than

-LRB-

e

-RRB-

Accounts

through

have

at

most

and

at

least

accounts

with

more

than

The

sandbag

of

Example

is

gen

F

where

F

is

f

h

hh

-LSB-

-RSB-

-LSB-

-RSB-

i

hh

-LSB-

-RSB-

-LSB-

-RSB-

i

hh

-LSB-

-RSB-

-LSB-

-RSB-

i

hh

-LSB-

-RSB-

-LSB-

-RSB-

i

hh

-LSB-

-RSB-

-LSB-

-RSB-

i

Example

Example

defines

no

particular

sandbag

However

one

reasonable

approach

to

compactly

representing

one

million

reals

between

and

is

to

partition

the

interval

-LSB-

-RSB-

into

equallysized

bins

and

to

similar

partition

each

bin

into

subbins

Then

a

fact

set

of

revelations

that

bound

the

number

of

reals

falling

into

bins

and

sub-bins

can

be

constructed

The

sandbag

resulting

from

such

a

fact

set

provides

useful

information

about

the

number

of

points

falling

in

large

intervals

but

is

not

very

informative

at

a

scale

smaller

than

of

the

unit

interval

However

a

fact

set

consisting

of

revelations

is

nine

thousand

times

more

compact

than

a

million

floats

Example

Let

D

be

the

partially

ordered

domain

of

motion

video

clips

described

by

Example

Let

xi

represent

a

clip

of

level

i

of

answer

x

As

revelations

consistent

with

F

of

the

form

hxi

are

added

to

a

revelation

set

F

more

about

S

can

be

determined

from

F

Let

f

h

i

g

The

fact

set

of

Example

is

gen

Revelation

sets

can

model

improvements

in

the

quality

of

clips

in

a

set

of

clips

The

addition

of

two

revelations

might

give

us

f

h

i

g

provides

an

improvement

of

B

to

resolution

level

and

has

provided

an

approximation

of

answer

C

at

level

Complexity

of

the

Fact

Set

Approach

The

function

gen

generates

the

mina

and

maxa

functions

from

fact

sets

This

section

addresses

the

question

What

is

the

computational

complexity

of

calculating

mina

-LRB-

gen

F

-RRB-

:d

and

maxa

-LRB-

gen

F

-RRB-

:d

for

a

fact

set

F

and

a

partial

data

element

d

The

answer

to

this

question

is

highly

dependent

on

the

structure

of

the

underlying

multiresolution

type

that

the

fact

set

and

sandbag

employ

Theorem

For

arbitrary

data

domains

mina

and

maxa

can

not

be

exactly

computed

without

taking

into

account

the

structure

of

the

data

domain

Proof

Sketch

If

the

underlying

domain

is

the

domain

of

real

intervals

maxa

-LRB-

gen

fg

-RRB-

-LSB-

-RSB-

but

if

it

is

integers

maxa

-LRB-

gen

fg

-RRB-

-LSB-

-RSB-

The

fact

set

approach

allows

mina

and

maxa

to

interact

in

such

a

way

that

mina

also

can

not

be

computed

As

shown

in

the

next

section

if

a

function

that

computes

jseta

-LRB-

fdg

D

-RRB-

j

is

available

then

mina

and

maxa

can

be

calculated

For

all

the

multiresolution

types

D

suggested

so

far

jseta

-LRB-

fdg

D

-RRB-

j

can

be

efficiently

computed

We

call

a

constraint

of

the

form

maxa

S

:d

jseta

fdg

Dj

a

domain

constraint

A

Relationship

to

Integer

Linear

Programming

The

problem

of

calculating

mina

-LRB-

gen

F

-RRB-

:d

and

maxa

-LRB-

gen

F

-RRB-

:d

reduces

to

finding

an

optimal

integer

solution

to

a

linear

system

of

constraints

if

domain

constraints

are

computable

We

discuss

the

complexity

of

this

reduction

and

its

implications

after

explaining

it

A

very

important

special

case

in

which

the

sandbag

can

be

exactly

computed

in

polynomial

time

is

then

discussed

Consider

a

fact

set

F

that

mentions

a

set

of

data

elements

M

from

D

and

a

query

element

q.

Partition

the

total

elements

of

D

into

disjoint

sets

I

such

that

-LRB-

m

M

-LSB-

fqg

-LRB-

Y

Y

I

seta

-LRB-

fmg

D

-RRB-

seta

-LRB-

Y

D

-RRB-

-RRB-

-RRB-

-LRB-

-RRB-

Figure

is

an

illustration

of

this

idea

on

the

domain

of

real

intervals

The

intervals

in

the

lower

part

of

the

figure

correspond

to

the

mentioned

and

query

elements

of

the

fact

set

The

intervals

along

the

top

of

the

figure

represent

the

independent

sets

that

are

the

partition

I

of

the

universe

of

total

elements

In

this

example

each

such

set

could

be

represented

by

a

particular

partial

Encoding

of

the

constraints

from

Eqn

Each

revelation

generates

two

lines

one

negated

to

capture

the

minimum

Encoding

of

the

domain

constraints

from

Eqn

Each

line

limits

a

variable

xi

x

a

z

jseta

-LRB-

D

-RRB-

j

jseta

-LRB-

D

-RRB-

j

jseta

-LRB-

fkijIjg

D

-RRB-

j

Figure

The

form

of

the

integer

linear

program

corresponding

to

a

fact

set

Figure

Example

partition

of

mentioned

elements

data

element

In

general

however

the

components

of

I

need

not

correspond

to

any

seta

-LRB-

d

D

-RRB-

for

a

particular

element

d

This

partition

into

independent

sets

yields

a

reduction

to

integer

linear

programming

by

allowing

us

to

associate

a

variable

with

each

set

in

the

partition

I.

Constraints

on

these

variables

can

be

constructed

that

correspond

to

the

constraints

on

sets

imposed

by

a

fact

set

F

so

that

a

feasible

solution

to

this

set

of

constraints

will

be

related

to

a

set

B

that

is

consistent

with

F

Construct

of

vector

x

of

jIj

variables

The

variable

xi

is

associated

with

the

set

i

I

We

relate

a

vector

x

to

a

set

B

via

a

relation

x

B

-LRB-

i

i

I

xi

jseta

-LRB-

i

B

-RRB-

j

-RRB-

Our

goal

is

to

construct

a

set

of

constraints

C

from

F

such

that

if

x

is

related

to

the

set

B

by

then

B

is

fact

set

consistent

with

F

if

and

only

if

x

is

feasible

on

C

x

B

-RRB-

fscons

-LRB-

F

B

-RRB-

-LRB-

x

is

feasible

on

C

-RRB-

The

pinnings

in

F

will

be

considered

first

since

they

effectively

allow

variables

to

be

excluded

from

further

consideration

Each

pinning

hd

P

i

F

constrains

the

members

of

x

corresponding

to

sets

above

a

pin

but

not

above

some

element

of

its

pin

set

to

be

-LRB-

sum

k

-LRB-

k

I

seta

-LRB-

fdg

D

-RRB-

k

-LRB-

a

P

seta

-LRB-

fag

D

-RRB-

k

-RRB-

-RRB-

xk

-RRB-

-LRB-

-RRB-

This

constraint

effectively

allows

us

to

consider

in

the

reduction

only

those

variables

xi

that

are

not

excluded

by

the

pinnings

of

F

these

variables

can

be

chosen

in

time

polynomial

in

jIj

Construct

a

set

of

constraints

corresponding

to

the

revelations

hd

a

zi

of

the

set

F

such

that

-LRB-

sum

k

k

I

seta

-LRB-

fdg

D

-RRB-

k

xk

-RRB-

z

-LRB-

sum

k

k

I

seta

-LRB-

fdg

D

-RRB-

k

xk

-RRB-

a

-LRB-

-RRB-

Construct

a

set

that

includes

the

pertinent

domain

constraints

-LRB-

k

k

I

xk

jseta

-LRB-

fkg

D

-RRB-

j

-RRB-

-LRB-

-RRB-

Additionally

the

condition

that

each

xk

is

nonnegative

is

required

since

each

xk

represents

the

cardinality

of

some

actual

subset

of

the

approximated

set

which

of

course

can

not

be

negative

However

that

constraint

will

be

implicit

in

the

fact

that

we

seek

a

non-negative

solution

to

the

ILP

that

we

construct

Similarly

since

these

represent

actual

cardinalities

they

must

be

integral

Let

C

-LSB-

These

constraints

on

the

variables

x

are

depicted

in

Figure

in

the

form

of

an

integer

linear

program

A

I

x

-LRB-

-RRB-

where

x

This

linear

property

has

the

desired

property

that

any

feasible

solution

x

corresponds

directly

via

the

relation

to

some

set

that

is

fact

set

consistent

with

F

Since

mina

and

maxa

are

defined

to

be

quantification

over

consistent

sets

they

can

be

computed

by

choosing

suitable

objective

functions

to

optimize

since

-LRB-

sum

k

k

I

q

seta

-LRB-

fkg

D

-RRB-

xk

-RRB-

numa

-LRB-

S

q

-RRB-

because

the

desired

conditions

is

now

a

theorem

by

the

construction

of

C.

Theorem

By

the

construction

of

C

from

F

x

S

-RRB-

fscons

-LRB-

F

S

-RRB-

-LRB-

x

is

feasible

on

C

-RRB-

Theorem

Consider

the

objective

function

x

-LRB-

sum

k

k

I

seta

-LRB-

fqg

D

-RRB-

k

xk

-RRB-

and

let

xmax

be

the

optimal

feasible

solution

to

C

It

then

follows

that

xmax

maxa

-LRB-

gen

F

-RRB-

q

The

value

of

mina

-LRB-

gen

F

-RRB-

q

is

similarly

obtained

by

optimizing

OE

where

OE

x

x.

If

the

set

of

constraints

C

and

the

vector

x

can

be

constructed

in

polynomial

time

from

F

and

the

domain

constraints

can

be

calculated

in

polynomial

time

for

each

member

of

I

then

the

calculation

of

the

optimal

values

of

mina

and

maxa

polynomially

reduces

to

an

integer

linear

programming

-LRB-

ILP

-RRB-

problem

and

is

thus

in

NP

Such

a

construction

is

possible

for

all

example

data

domains

mentioned

in

this

paper

This

reduction

suggests

a

close

connection

between

the

structure

of

a

data

domain

and

the

computational

complexity

of

computing

against

that

domain

The

cardinality

of

the

partition

I

is

the

dimension

of

the

space

that

must

be

optimized

therefore

informally

the

smaller

the

partition

I

the

easier

the

problem

The

cardinality

of

I

is

dependent

on

both

the

elements

mentioned

in

the

fact

set

and

the

structure

of

the

underlying

data

domain

This

reduction

also

suggests

the

possibility

that

a

general

Linear

Programming

-LRB-

LP

-RRB-

engine

could

be

employed

to

implement

approximately

the

fact

set

approach

Since

every

solution

to

an

ILP

is

also

a

solution

to

the

ILP

relaxed

to

non-integral

solutions

an

LP

always

produces

more

optimal

points

with

respect

to

the

linear

objective

function

These

optimal-butnot-integral

solutions

could

be

used

as

upper

bounds

on

maxa

and

lower

bounds

on

mina

Linear

Programming

is

polynomial

The

reduction

above

yields

conditions

under

which

the

the

problem

is

in

NP

It

is

generally

NP-hard

for

arbitrary

domains

Theorem

For

arbitrary

domains

that

can

be

polynomially

reduced

to

independent

sets

as

per

Equation

the

calculation

of

mina

and

maxa

is

NP

complete

Proof

Sketch

Data

domains

that

encode

the

structure

of

graphs

can

be

constructed

and

fact

sets

over

them

formed

that

have

particular

minimums

or

maximum

if

and

only

if

the

graphs

have

properties

that

are

NP-hard

to

calculate

More

detailed

proofs

are

provided

in

Appendix

A.

An

Important

Special

Case

In

the

previous

section

we

suggest

that

the

difficulty

of

calculating

mina

and

maxa

from

a

fact

set

decrease

as

the

structure

of

the

underlying

data

domains

become

more

tree-like

For

many

very

restrictive

but

practical

data

domains

mina

and

maxa

can

be

calculated

in

polynomial

time

Definition

A

data

domain

D

is

called

perfectly

tree-like

exactly

when

elements

incomparable

with

v

have

no

common

upper

bound

-LRB-

y

x

D

y

D

-LRB-

x

vy

y

vx

-RRB-

-RRB-

-LRB-

z

xv

z

y

v

z

-RRB-

-RRB-

Example

The

usefulness

of

a

multiresolution

primitive

type

depends

in

part

on

its

compactness

Consider

the

problem

of

constructing

a

useful

multiresolution

type

of

integer

intervals

The

naive

idea

of

storing

two

integers

is

clearly

not

space

efficient

By

giving

up

our

ability

some

ability

to

represent

intervals

we

can

construct

a

much

more

efficient

representation

We

construct

a

data

type

whose

every

element

can

be

represented

in

bits

If

we

restrict

the

domain

to

those

intervals

that

are

either

the

upper

or

lower

half

of

some

larger

interval

the

number

of

partial

data

elements

or

intervals

that

must

be

represented

where

k

is

the

number

of

divisions

we

allow

Since

a

practical

data

type

should

contain

total

data

elements

that

are

completely

specific

the

finest

divisions

should

be

integer

intervals

that

only

contain

one

integer

and

hence

are

naturally

total

The

number

of

nodes

in

a

complete

binary

tree

is

where

h

is

the

height

of

the

tree

Thus

any

node

of

a

complete

binary

tree

of

height

can

be

represented

in

a

bit

quantity

Let

us

construct

a

multiresolution

primitive

data

type

of

restricted

integer

intervals

by

associating

integer

intervals

with

nodes

in

a

complete

binary

tree

of

height

Since

this

tree

has

leaves

the

total

elements

might

be

the

integers

between

and

The

remaining

interior

elements

are

partial

data

elements

generated

recursively

by

defining

the

interval

-LSB-

-RSB-

to

be

the

partial

element

and

defining

the

upper

or

lower

half

of

any

data

element

that

can

be

divided

to

be

a

data

element

The

approximates

relation

on

this

domain

is

just

interval

containment

as

in

Example

data

element

Figure

Example

constraint

matrix

transformed

to

interval

matrix

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

-LSB-

-RSB-

Figure

Example

data

elements

from

perfectly

treelike

integer

domain

This

domain

may

be

more

useful

than

the

domain

of

arbitrary

real

intervals

since

it

can

be

compactly

represented

Furthermore

it

is

perfectly

tree-like

since

for

any

two

data

elements

d

and

e

the

following

condition

holds

d

v

e

e

v

d

-LRB-

:d

v

e

e

v

d

-RRB-

The

transitive

reduction

of

a

set

of

data

elements

from

this

domain

is

displayed

in

Figure

Note

that

although

the

domain

is

a

complete

binary

tree

subsets

of

it

need

not

be

although

they

are

always

trees

When

a

multiresolution

domain

is

perfectly

treelike

the

cardinality

of

the

partition

I

of

the

reduction

described

in

the

previous

subsection

is

at

most

the

number

of

elements

mentioned

in

F

-LRB-

i.e.

jIj

jMj

-RRB-

Under

most

data

domains

that

are

tree-like

the

production

of

the

partition

jIj

is

polynomial

For

instance

for

the

data

elements

Example

the

independent

sets

that

disjointly

exhaust

the

space

of

total

elements

are

and

The

cross

product

of

two

data

domains

is

not

perfectly

tree-like

under

the

standard

construction

-LSB-

-RSB-

However

important

domains

such

as

Examples

and

are

perfectly

tree-like

and

the

other

examples

are

treelike

under

certain

restrictions

so

the

following

theorem

is

quite

often

applicable

Theorem

If

a

data

domain

D

is

perfectly

treelike

and

can

be

polynomially

reduced

to

an

ILP

the

calculation

of

optimal

mina

and

maxa

is

polynomial

This

theorem

follows

from

the

fact

that

the

system

of

constraints

generated

by

a

fact

set

over

a

perfectly

tree-like

domain

is

integral

-LSB-

-RSB-

Any

solution

that

optimizes

the

linear

function

is

integral

hence

the

system

can

be

treated

as

a

linear

program

on

reals

rather

than

as

an

integer

linear

program

Proof

Sketch

Recall

that

the

system

of

constraints

produced

by

the

reduction

is

of

the

form

A

I

x

If

A

is

unimodular

-LRB-

A

I

-RRB-

is

unimodular

The

constraint

matrix

A

is

unimodular

because

it

can

be

transformed

into

an

interval

matrix

using

operations

that

preserve

unimodularity

-LSB-

-RSB-

and

interval

matrices

are

unimodular

In

particular

the

operations

of

transposing

multiplying

a

row

-LRB-

column

-RRB-

by

and

the

the

rearrangement

of

rows

-LRB-

columns

-RRB-

preserve

unimodularity

An

interval

matrix

is

a

matrix

containing

only

O

's

and

's

such

that

all

's

in

a

row

occur

consecutively

The

matrix

A

can

be

transformed

into

an

interval

matrix

by

multiplying

every

column

that

contains

's

by

The

fact

that

is

an

interval

matrix

follows

from

the

fact

that

every

element

xi

is

uniquely

associated

with

some

member

of

the

set

of

mentioned

elements

M

Thus

each

xi

can

be

thought

of

as

labeling

a

node

of

the

tree

of

the

elments

of

M

ordered

by

v

If

we

write

the

columns

of

A

-LRB-

and

hence

-RRB-

in

the

order

produced

by

for

instance

a

pre-order

traversal

of

the

tree

M

then

each

row

of

A

will

have

its

non-zero

elements

completely

consecutive

since

each

row

has

non-zero

elements

in

exactly

those

columns

corresponding

to

those

partial

elements

approximated

by

the

element

corresponding

to

that

constraint

A

suggestion

of

the

construction

of

such

an

interval

matrix

for

Example

is

depicted

in

Figure

The

matrix

is

a

constraint

matrix

which

has

had

the

negative

rows

duplicated

rows

and

the

identity

matrix

for

domain

constraints

removed

by

unimodularitypreserving

operations

Each

column

represents

one

of

the

disjoint

sets

in

the

independent

partion

of

the

total

elements

of

Example

Each

row

is

labeled

by

the

mentioned

fact

set

element

that

generated

the

constraint

corresponding

to

that

row

A

Data

Structure

and

its

Algorithms

In

this

section

we

motivate

and

describe

a

data

structure

for

the

sandbag

Algorithms

for

calculating

mina

and

maxa

from

this

structure

are

then

detailed

in

separate

subsections

We

then

describe

routines

that

maintain

this

structure

's

important

invariants

which

form

the

basis

of

routines

that

allow

revelations

and

pinnings

to

be

incorporated

into

a

sandbag

data

structure

The

high

computational

complexity

associated

with

the

fact

set

approach

is

disappointing

However

by

giving

up

the

optimality

of

mina

and

maxa

and

settling

for

under-estimates

and

over-estimates

respectively

efficient

algorithms

can

be

obtained

that

allow

the

sandbag

to

be

used

soundly

The

imperfect

bounds

thus

computed

will

be

useful

for

two

reasons

Firstly

the

sandbag

is

generally

used

as

a

description

of

imperfect

knowledge

so

that

presumably

limited

computation

against

it

is

acceptable

Secondly

in

those

cases

that

the

sandbag

is

used

as

a

container

rather

than

as

a

means

of

deducing

information

from

powerful

constraints

the

error

in

derived

bounds

will

be

small

or

nonexistent

The

previous

section

suggests

that

the

fact

set

approach

could

be

implemented

using

linear

programming

and

integer

linear

programming

technology

Although

we

hope

to

explore

that

idea

further

fact

setspecific

data

structures

and

algorithms

are

appropriate

for

several

reasons

ffl

an

efficient

means

of

incrementally

maintaining

the

fact

set

is

needed

in

any

case

ffl

the

reduction

to

independent

linear

constraints

is

expensive

ffl

algorithms

and

data

structures

specific

to

the

fact

set

approach

may

be

more

efficient

and

ffl

any

data

structure

implementing

the

fact

set

approach

must

support

the

enumeration

of

the

fact

set

and

the

rapid

lookup

and

extraction

of

data

elements

The

remainder

of

this

section

therefore

develops

a

data

structure

and

a

set

of

polynomial

algorithms

The

SDAG

data

structure

This

subsection

describes

the

data

structure

After

considerable

machinery

is

introduced

that

will

be

needed

in

the

next

subsections

we

are

able

state

its

invariants

formally

To

implement

the

fact

set

approach

to

sandbags

we

use

a

directed

acyclic

graph

of

records

called

an

SDAG

denoted

by

G

The

SDAG

is

the

transitive

reduction

of

the

set

of

data

elements

mentioned

in

a

fact

set

under

the

partial

ordering

v

We

use

G

to

denote

the

type

of

SDAGs

R

to

denote

the

type

of

an

SDAG

node

and

to

denote

the

type

of

a

set

of

SDAG

nodes

The

fields

of

an

SDAG

node

r

are

ffl

r

data

D

a

partial

-LRB-

or

total

-RRB-

data

element

corresponding

to

a

revealed

pin

set

or

pin

data

element

ffl

r

kmin

representing

the

known

mina

value

for

r

ffl

r

kmax

representing

the

known

maxa

value

for

r

ffl

r

pars

the

immediate

ancestors

of

r

in

the

SDAG

ffl

r

kids

the

immediate

descendants

of

r

in

the

SDAG

ffl

r

pnset

corresponding

to

the

pin

set

of

r

and

ffl

r

baset

corresponding

to

the

set

of

records

whose

pnset

includes

r

We

will

intentionally

use

the

names

of

these

data

fields

as

functions

that

extract

the

field

of

that

name

from

a

record

This

allows

us

to

map

these

functions

over

sets

An

application

of

the

map

function

to

a

function

f

to

a

set

Z

denoted

map

-LRB-

f

Z

-RRB-

is

equivalent

to

ff

x

j

x

Zg

So

for

instance

map

-LRB-

data

Z

-RRB-

is

equal

to

the

set

fx

data

j

x

Zg

Conceptually

the

SDAG

is

constructed

from

and

represents

a

fact

set

in

a

straightforward

manner

Each

data

field

of

every

record

in

the

SDAG

is

a

mentioned

element

whether

in

a

revelation

a

pin

element

or

in

a

pinning

set

The

edges

of

the

SDAG

are

the

transitive

reduction

of

the

data

fields

under

v

and

each

record

is

linked

to

its

immediate

ancestors

with

the

pars

field

and

its

immediate

descendants

with

the

kids

field

We

Algorithm

Type

-LRB-

G

d

-RRB-

G

D

R

-LSB-

fnullg

-LRB-

G

d

-RRB-

G

D

-LRB-

G

d

-RRB-

G

D

-LRB-

G

d

-RRB-

G

D

-LRB-

G

Z

-RRB-

G

-LRB-

G

d

-RRB-

G

D

-LRB-

G

R

P

d

-RRB-

G

-LRB-

G

d

-RRB-

G

D

suremin

-LRB-

G

r

a

-RRB-

G

R

G

suremax

-LRB-

G

r

z

-RRB-

G

R

G

dealloc

-LRB-

G

Z

-RRB-

G

G

addnode

-LRB-

G

r

-RRB-

G

R

G

lkpormk

-LRB-

G

d

-RRB-

G

D

G

R

rev

-LRB-

G

d

a

z

-RRB-

G

D

G

updatepin

-LRB-

G

r

A

B

-RRB-

G

R

G

cocadd

-LRB-

b

P

G

Z

-RRB-

D

G

G

pin

-LRB-

G

d

P

-RRB-

G

D

G

Table

Summary

of

Types

of

Implemented

Functions

formally

define

the

the

values

of

the

pars

and

kids

fields

of

an

SDAG

below

The

purpose

of

this

data

structure

is

to

support

efficient

calculation

of

suboptimal

values

of

mina

S

:d

and

maxa

S

:d

for

any

partial

data

value

d

The

approach

is

straightforward

the

kmin

and

kmax

values

store

suboptimal

values

of

elements

which

have

been

mentioned

in

the

construction

of

the

SDAG

From

the

kmin

and

kmax

values

of

data

elements

near

to

a

queried

element

d

we

can

deduce

bounds

on

mina

S

:d

and

maxa

S

:d

The

pnset

field

of

a

node

will

represent

a

valid

pin

set

for

that

node

However

a

fact

set

may

have

many

pin

sets

for

a

particular

pin

element

each

of

which

provides

a

slightly

different

constraint

on

the

sets

consistent

with

the

fact

set

As

it

is

expensive

to

store

all

of

these

sets

our

strategy

is

to

produce

a

good

-LRB-

highly

constraining

-RRB-

pnset

field

consistent

with

the

pin

sets

of

a

given

fact

set

This

strategy

is

a

tradeoff

of

some

of

the

restrictiveness

of

the

many

pin

sets

in

a

fact

set

for

the

compactness

of

the

pnset

field

In

the

next

six

pages

we

introduce

a

number

of

functions

and

algorithms

We

collect

the

types

of

the

concrete

implemented

functions

in

Table

and

present

them

here

before

their

definitions

so

that

the

reader

may

refer

back

to

them

If

an

algorithm

may

modify

one

of

its

arguments

than

the

type

of

that

argument

is

repeated

in

the

range

of

the

type

For

instance

dealloc

-LRB-

G

Z

-RRB-

is

a

procedure

that

has

no

return

value

but

since

it

modifies

the

SDAG

G

by

deallocating

the

nodes

Z

from

it

its

type

is

denoted

G

G

In

order

to

formalize

the

structure

of

the

SDAG

as

an

invariant

we

formally

define

the

notions

of

mentioned

elements

immediate

ancestors

and

immediate

descendants

Definition

For

a

fact

set

F

an

element

d

is

mentioned

denoted

by

ment

-LRB-

F

d

-RRB-

if

and

only

if

it

appears

in

any

fact

of

the

set

ment

-LRB-

F

d

-RRB-

-LRB-

hd

a

bi

hd

a

bi

F

-RRB-

-LRB-

hd

Li

hd

Li

F

-RRB-

-LRB-

hx

Li

hx

Li

F

d

L

-RRB-

Definition

For

an

SDAG

G

a

record

r

such

that

r

data

d

is

returned

by

lookup

-LRB-

G

d

-RRB-

if

it

exists

in

G

If

not

lookup

-LRB-

G

d

-RRB-

returns

-LRB-

-LRB-

r

r

G

r

data

d

-RRB-

-RRB-

lookup

-LRB-

G

d

-RRB-

-RRB-

-LRB-

-LRB-

r

r

G

r

data

d

-RRB-

-RRB-

lookup

-LRB-

G

d

-RRB-

r

-RRB-

Definition

x

y

-LRB-

x

y

xv

y

-RRB-

Definition

For

a

subset

of

a

domain

D

D

the

immediate

ancestors

and

immediate

descendants

of

any

element

d

are

denoted

by

imanc

-LRB-

D

d

-RRB-

and

imdec

-LRB-

D

d

-RRB-

defined

by

ffl

imanc

-LRB-

D

d

-RRB-

fa

j

a

d

a

D

-LRB-

c

c

D

a

c

c

d

-RRB-

g

ffl

imdec

-LRB-

D

d

-RRB-

fa

j

d

a

a

D

-LRB-

c

c

D

c

a

d

c

-RRB-

g

Note

that

imanc

and

imdec

are

defined

for

every

data

value

We

adopt

the

convention

of

referring

to

the

actual

algorithms

that

implement

abstract

functions

such

as

imanc

and

imdec

by

affixing

digits

to

their

names

to

distinguish

them

We

also

use

function

and

procedure

names

that

contain

a

digit

for

algorithms

that

produce

approximate

or

suboptimal

results

that

bound

abstract

functions

Hence

our

algorithm

for

the

abstract

function

mina

is

Definition

For

a

fact

set

F

and

an

SDAG

G

the

following

structural

invariant

is

maintained

This

invariant

states

that

the

edges

in

the

SDAG

properly

reflect

the

structure

of

the

fact

set

from

which

it

was

created

that

all

pnset

fields

are

cochains

and

that

the

baset

fields

properly

mirrors

the

pnset

fields

sinv

-LRB-

F

G

-RRB-

-LRB-

-LRB-

lookup

-LRB-

G

d

-RRB-

-RRB-

ment

-LRB-

F

d

-RRB-

-RRB-

-LRB-

r

G

r

pars

imanc

-LRB-

map

-LRB-

data

G

-RRB-

r

data

-RRB-

r

kids

imdec

-LRB-

map

-LRB-

data

G

-RRB-

r

data

-RRB-

-LRB-

r

G

-LRB-

y

x

r

pnset

y

r

pnset

xv

y

y

vx

r

data

vx

-RRB-

-RRB-

-LRB-

s

r

G

s

G

s

r

baset

r

s

pnset

-RRB-

-RRB-

Under

this

invariant

the

functions

lookup

imanc

and

imdec

for

an

SDAG

can

be

implemented

using

variants

of

depth-first

search

Algorithms

for

and

are

provided

in

Appendix

B.

Theorem

The

algorithms

-LRB-

G

d

-RRB-

-LRB-

G

d

-RRB-

and

-LRB-

G

d

-RRB-

correctly

calculate

the

functions

lookup

-LRB-

G

d

-RRB-

imanc

-LRB-

F

d

-RRB-

and

imdec

-LRB-

F

d

-RRB-

ffl

sinv

-LRB-

F

G

-RRB-

lookup

-LRB-

G

d

-RRB-

-LRB-

G

d

-RRB-

ffl

sinv

-LRB-

F

G

-RRB-

lookup

-LRB-

G

d

-RRB-

-RRB-

imanc

-LRB-

F

d

-RRB-

-LRB-

G

d

-RRB-

ffl

sinv

-LRB-

F

G

-RRB-

lookup

-LRB-

G

d

-RRB-

-RRB-

imdec

-LRB-

F

d

-RRB-

-LRB-

G

d

-RRB-

The

functions

and

are

used

by

our

main

algorithms

described

below

In

addition

to

these

we

define

other

auxiliary

functions

that

operate

on

sets

of

records

of

the

SDAG

itself

The

essence

of

the

SDAG

is

that

the

r

kmin

r

kmax

and

r

pnset

fields

for

each

record

are

sound

with

respect

to

the

fact

set

from

which

an

SDAG

is

being

built

This

is

captured

by

the

invariant

of

the

SDAG

INV

-LRB-

F

G

-RRB-

Definition

For

an

SDAG

G

and

a

fact

set

F

ffl

-LRB-

F

G

-RRB-

-LRB-

r

r

G

r

kmin

mina

-LRB-

gen

F

-RRB-

-LRB-

r

data

-RRB-

-RRB-

ffl

-LRB-

F

G

-RRB-

-LRB-

r

r

G

r

kmax

maxa

-LRB-

gen

F

-RRB-

-LRB-

r

data

-RRB-

-RRB-

ffl

-LRB-

F

G

-RRB-

-LRB-

r

G

-LRB-

r

data

v

a

tot

a

-LRB-

q

r

pnset

q

v

a

-RRB-

-RRB-

maxa

-LRB-

gen

F

-RRB-

a

-RRB-

-RRB-

ffl

INV

-LRB-

F

G

-RRB-

sinv

-LRB-

F

G

-RRB-

-LRB-

F

G

-RRB-

-LRB-

F

G

-RRB-

-LRB-

F

G

-RRB-

The

usefulness

of

this

invariant

is

that

suboptimal

values

of

mina

-LRB-

gen

F

-RRB-

:d

and

maxa

-LRB-

gen

F

-RRB-

:d

can

be

calculated

reasonably

efficiently

for

any

data

element

d

In

the

next

five

subsections

we

describe

the

six

main

algorithms

that

use

this

data

structure

in

addition

to

a

number

of

auxiliary

algorithms

ffl

algorithms

that

calculate

approximate

bounds

and

ffl

algorithms

that

propagate

an

improved

constraint

throughout

the

SDAG

suremin

and

suremax

and

ffl

algorithms

that

add

a

revelation

or

pinning

to

the

SDAG

rev

and

pin

respectively

The

Calculation

of

mina

This

subsection

defines

some

necessary

preliminaries

before

giving

the

algorithms

for

mina

Consider

the

problem

of

calculating

mina

S

:d

from

an

SDAG

If

a

record

r

corresponding

to

d

-LRB-

r

data

d

-RRB-

already

exists

in

the

SDAG

then

r

kmin

is

a

lower

bound

for

mina

S

:d

by

INV

-LRB-

F

G

-RRB-

If

no

such

record

exists

then

a

lower

bound

can

be

calculated

from

the

records

of

the

SDAG

that

correspond

to

the

immediate

descendants

of

d

If

these

descendants

whose

data

fields

are

approximated

by

d

can

be

partitioned

into

two

sets

A

and

B

such

that

there

is

no

upper

bound

in

D

that

is

approximated

by

an

element

from

both

A

and

B

then

the

sum

of

the

minimum

above

A

and

the

minimum

above

B

is

at

most

the

minimum

above

d

This

fact

is

the

essence

of

the

algorithm

described

below

that

recursively

partitions

sets

of

records

into

sets

without

shared

upper

bounds

and

calculates

lower

bounds

for

the

number

of

total

data

elements

above

sets

of

records

Throughout

this

paper

we

use

the

language

and

proof

system

of

Gries

-LSB-

-RSB-

with

two

exceptions

Pure

functions

those

that

have

no

side-effects

will

be

treated

as

functions

rather

than

procedures

that

have

a

result

parameter

Secondly

we

use

an

iteration

construct

that

loops

over

the

members

of

a

set

Definition

We

extend

the

language

presented

in

-LSB-

-RSB-

with

a

construct

for

This

is

a

simple

loop

construct

over

the

elements

of

an

immutable

set

As

such

it

is

guaranteed

to

terminate

Given

that

statement

S

does

not

mention

Y

and

Z

is

an

otherwise

unused

identifier

it

is

defined

by

Z

Y

for

-LRB-

x

x

Y

-RRB-

do

Z

x

choose

Z

S

S

Z

Z

fxg

rof

od

where

choose

Z

returns

any

member

of

Z.

The

determination

of

whether

or

not

two

elements

have

an

upper

bound

in

common

is

crucial

to

the

partition

of

the

set

of

records

required

by

the

algorithm

The

predicate

no-ub

-LRB-

a

b

-RRB-

is

programmed

by

the

user

for

each

primitive

multiresolution

type

Definition

Given

a

partially

ordered

domain

D

an

implementation

of

no-ub

is

correct

if

and

only

if

it

satisfies

the

specification

-LRB-

no-ub

-LRB-

a

b

-RRB-

-RRB-

-LRB-

x

x

D

-LRB-

a

vx

b

vx

-RRB-

-RRB-

-RRB-

-LRB-

-LRB-

tot

a

tot

b

a

b

-RRB-

-RRB-

no-ub

-LRB-

a

b

-RRB-

-RRB-

Note

in

particular

that

this

is

not

an

equivalence

Although

the

efficiency

of

our

algorithms

depends

in

part

on

the

use

of

no-ub

-LRB-

a

b

-RRB-

it

is

legal

for

no-ub

-LRB-

a

b

-RRB-

to

return

false

when

in

fact

no

upper

bound

may

exist

for

a

and

b

unless

a

and

b

are

total

We

extend

the

notion

of

no-ub

to

tuples

in

the

obvious

way

Definition

Given

any

two

values

a

and

b

from

a

partially

ordered

tuple

type

T

constructed

from

N

types

TN

no-ub

-LRB-

a

b

-RRB-

is

true

if

and

only

if

there

exists

a

pair

of

components

from

a

and

b

that

have

no

upper

bound

in

common

no-ub

-LRB-

a

b

-RRB-

-LRB-

c

c

c

N

no-ub

-LRB-

ac

bc

-RRB-

-RRB-

Definition

The

function

sharenoub

-LRB-

A

B

-RRB-

returns

a

subset

of

A

that

has

no

upper

bound

with

B

The

function

mayhaveub

-LRB-

A

d

-RRB-

returns

the

subset

of

A

that

might

have

an

upper

bound

with

d

The

predicate

cochain

A

returns

true

if

A

is

a

cochain

ffl

sharenoub

-LRB-

A

B

-RRB-

fa

j

a

A

-LRB-

b

b

B

no-ub

-LRB-

a

b

-RRB-

-RRB-

g

ffl

mayhaveub

-LRB-

A

d

-RRB-

fa

j

a

A

no-ub

-LRB-

a

d

-RRB-

g

ffl

cochain

A

-LRB-

a

b

a

A

b

A

-LRB-

a

v

b

-RRB-

-RRB-

We

define

a

function

noubsep

that

takes

a

set

of

records

Z

and

returns

a

set

of

disjoint

subsets

of

Z

that

share

no

upper

bounds

with

each

other

This

function

is

useful

because

we

can

make

inferences

about

the

number

of

elements

above

partial

elements

that

share

no

upper

bounds

We

formally

define

the

important

property

of

noubsep

Definition

An

implementation

of

noubsep

is

correct

if

and

only

if

-LRB-

R

R

S

noubsep

Q

R

S

sharenoub

-LRB-

R

S

-RRB-

R

Q

-RRB-

Note

that

some

records

in

Z

may

not

be

represented

in

the

set

of

returned

sets

A

good

but

not

necessarily

optimal

in

our

context

way

to

compute

noubsep

Z

is

to

construct

an

undirected

graph

with

nodes

Z

with

an

edge

from

a

to

b

if

and

only

no-ub

-LRB-

a

b

-RRB-

Then

find

a

cutnode

set

-LSB-

-RSB-

of

this

graph

remove

those

nodes

and

return

the

separated

components

as

the

elements

of

noubsep

Z

This

can

be

done

in

time

proportional

to

the

number

of

edges

in

a

graph

In

our

current

implementation

we

use

a

much

simpler

approach

that

simply

picks

a

set

of

nodes

that

share

no

upper

bounds

in

common

at

all

This

approach

has

had

no

observable

deleterious

effects

on

performance

however

we

plan

to

investigate

it

further

Algorithm

The

algorithm

takes

a

set

Z

of

records

and

computes

an

underestimate

of

the

minimum

nuber

of

total

data

elements

that

can

occur

above

any

element

of

Z.

func

-LRB-

G

G

Z

-RRB-

var

x

m

if

Z

m

-LSB-

-RSB-

Z

frg

m

r

kmin

-LSB-

-RSB-

jZj

x

sum

-LRB-

map

-LRB-

G

noubsep

Z

-RRB-

-RRB-

m

max

-LRB-

fxg

-LSB-

map

-LRB-

kmin

Z

-RRB-

-RRB-

return

m

-LRB-

Note

that

map

is

used

here

to

apply

with

G

as

a

Curried

argument

to

every

element

of

the

partition

generated

by

noubsep

In

C++

map

is

simply

implemented

as

a

loop

-RRB-

Algorithm

The

function

-LRB-

G

d

-RRB-

takes

a

partial

or

total

data

element

d

and

yields

a

lower

bound

on

the

number

of

elements

that

can

consistently

be

above

d

in

the

set

approximated

by

G.

func

-LRB-

G

G

d

D

-RRB-

var

r

R

m

r

-LRB-

G

d

-RRB-

if

r

m

r

kmin

-LSB-

-RSB-

r

m

-LRB-

G

-LRB-

G

d

-RRB-

-RRB-

return

m

Theorem

For

any

d

-LRB-

G

d

-RRB-

correctly

computes

an

underestimate

of

mina

-LRB-

gen

F

-RRB-

:d

from

an

SDAG

G

satisfying

INV

INV

-LRB-

F

G

-RRB-

-RRB-

-LRB-

d

-LRB-

G

d

-RRB-

mina

-LRB-

gen

F

-RRB-

:d

-RRB-

Proof

Sketch

By

INV

-LRB-

F

G

-RRB-

if

d

is

the

data

field

of

some

record

it

is

at

most

mina

-LRB-

gen

F

-RRB-

:d

If

not

-LRB-

G

-LRB-

G

d

-RRB-

-RRB-

is

returned

This

either

returns

kmin

or

the

sum

of

the

applied

to

sets

that

have

no

upper

bounds

in

common

By

the

invariant

and

induction

on

the

size

of

Z

these

are

all

at

most

mina

-LRB-

gen

F

-RRB-

:d

The

calculation

of

maxa

This

subsection

describes

algorithms

for

calculating

maxa

that

is

similar

to

the

mina

algorithms

Calculating

an

upper

bound

for

maxa

S

:d

begins

by

looking

up

d

If

a

record

corresponding

to

d

is

not

found

then

a

calculation

on

the

would-be

ancestors

of

d

is

required

This

is

in

contrast

to

mina

S

:d

that

calculates

a

lower

bound

from

would-be

descendants

of

d.

Additionally

since

pinnings

can

be

used

to

derive

an

upper

bound

on

the

number

of

elements

above

d

the

calculation

of

an

upper

bound

for

maxa

S

:d

that

occurs

in

employs

the

ancestors

of

d

that

are

pins

as

well

as

the

immediate

ancestors

of

d.

Algorithm

The

algorithm

looks

to

see

if

a

node

corresponding

to

its

argument

exists

if

it

does

it

returns

the

kmax

value

associated

with

that

node

If

it

does

not

it

calls

func

-LRB-

G

G

d

D

-RRB-

var

r

R

m

r

:d

if

r

m

r

kmax

-LSB-

-RSB-

r

m

-LRB-

G

-LRB-

G

d

-RRB-

-LRB-

G

d

-RRB-

d

-RRB-

return

m

Algorithm

The

algorithm

takes

a

set

of

existing

nodes

that

are

the

immediate

ancestors

of

d

and

a

partial

data

value

and

calculates

an

upper

bound

for

maxa

:d

func

-LRB-

G

G

R

P

d

D

-RRB-

var

m

m

for

-LRB-

t

t

R

-RRB-

m

min

-LRB-

fmg

-LSB-

ft

kmax

-LRB-

G

sharenoub

-LRB-

t

kids

fdg

-RRB-

-RRB-

g

-RRB-

rof

for

-LRB-

t

t

P

-RRB-

m

min

-LRB-

fmg

-LSB-

fsum

-LRB-

map

-LRB-

kmax

mayhaveub

:-LRB-

-LRB-

t

pnset

-RRB-

d

-RRB-

-RRB-

g

-RRB-

rof

return

m

A

precondition

of

is

that

R

P

is

any

subset

of

the

pin

ancestors

of

d

including

Just

as

with

if

the

element

is

already

mentioned

we

simply

return

the

kmax

value

and

avoid

running

Theorem

For

any

d

-LRB-

G

d

-RRB-

correctly

computes

an

overestimate

of

maxa

-LRB-

gen

F

-RRB-

:d

from

an

SDAG

G

satisfying

INV

INV

-LRB-

F

G

-RRB-

-RRB-

-LRB-

d

-LRB-

G

d

-RRB-

maxa

-LRB-

gen

F

-RRB-

:d

-RRB-

Proof

Sketch

Note

that

xvy

-RRB-

maxa

y

maxa

x

so

the

kmax

value

of

any

ancestor

of

d

limits

maxa

:d

Those

elements

above

an

ancestor

of

d

that

share

no

upper

bound

with

d

are

not

above

d

so

we

can

subtract

an

underestimate

of

that

number

from

kmax

Any

element

above

d

must

be

above

an

element

of

the

pnset

of

any

pin

ancestor

of

d

The

sum

of

the

kmaxs

of

the

pnsets

is

at

least

the

maxa

of

d.

Maintaining

the

Invariant

In

this

subsection

we

provide

two

algorithms

that

recursively

effect

the

changes

required

by

the

addition

of

a

new

constraint

to

a

fact

set

throughout

an

entire

SDAG

When

the

addition

of

a

revelation

or

pinning

to

a

fact

set

allows

us

to

infer

an

improvement

in

the

precision

of

the

SDAG

the

value

of

kmin

and

kmax

should

be

updated

appropriately

Changing

the

value

of

kmin

or

kmax

in

one

place

may

allow

for

improvements

in

other

places

that

should

be

realized

immediately

so

that

the

SDAG

will

provide

bounds

via

and

that

are

as

precise

as

possible

The

following

two

routines

are

designed

to

recursively

update

the

values

of

kmin

and

kmax

appropriately

throughout

the

SDAG

By

adopting

the

policy

that

the

kmin

and

kmax

data

fields

are

accomplished

entirely

through

these

two

procedures

the

concern

of

maintaining

sound

kmin

and

kmax

values

is

isolated

to

these

two

procedures

Algorithm

The

procedure

suremin

-LRB-

G

r

a

-RRB-

insures

that

record

r

has

a

kmin

of

at

least

a

and

if

this

change

allows

an

increase

in

the

mina

value

of

r

descendants

to

be

inferred

it

effects

those

changes

recursively

proc

suremin

-LRB-

val-res

G

G

val

r

R

a

-RRB-

if

a

r

kmin

r

kmin

a

for

-LRB-

s

s

r

pars

-RRB-

suremin

-LRB-

G

s

-LRB-

-LRB-

G

s

kids

-RRB-

-RRB-

-RRB-

rof

Theorem

When

called

with

sound

values

suremin

maintains

the

essential

invariant

INV

-LRB-

F

G

-RRB-

a

mina

-LRB-

gen

F

-RRB-

:d

-RRB-

-RRB-

wp

-LRB-

suremin

-LRB-

G

d

a

-RRB-

INV

-LRB-

F

G

-RRB-

-RRB-

Proof

Sketch

By

INV

-LRB-

F

G

-RRB-

we

know

the

values

calculated

by

-LRB-

G

s

kids

-RRB-

to

be

sound

and

hence

propagating

it

with

suremin

maintains

the

invariant

Since

suremin

recurses

only

on

ancestors

of

its

argument

it

terminates

Algorithm

The

procedure

suremax

-LRB-

G

r

z

-RRB-

insures

that

record

r

has

a

kmax

of

at

most

z

and

if

this

change

allows

a

decrease

in

the

maxa

value

of

r

descendants

or

those

records

which

include

r

in

their

pin

sets

to

be

inferred

it

effects

those

changes

recursively

proc

suremax

-LRB-

val-res

G

G

val

r

R

z

-RRB-

if

b

r

kmax

r

kmax

b

if

b

dealloc

-LRB-

G

r

kids

-RRB-

for

-LRB-

s

s

r

baset

-RRB-

suremax

-LRB-

G

s

sum

-LRB-

map

-LRB-

kmax

s

pnset

-RRB-

-RRB-

-RRB-

rof

for

-LRB-

s

s

r

kids

-RRB-

suremax

-LRB-

G

s

b

-RRB-

rof

if

r

kmax

r

baset

dealloc

-LRB-

G

r

-RRB-

The

calls

to

dealloc

are

extremely

important

because

they

decrease

the

overall

space

usage

of

a

sandbag

If

a

sandbag

eventually

is

refined

to

totality

and

approximates

the

empty

set

it

will

require

only

one

record

of

space

Definition

The

procedure

dealloc

-LRB-

G

Z

-RRB-

removes

the

records

Z

and

all

of

there

ancestors

updating

kids

baset

and

pnset

fields

appropriately

The

pre-condition

of

dealloc

-LRB-

G

Z

-RRB-

is

that

for

all

elements

x

of

Z

maxa

-LRB-

gen

F

-RRB-

x

The

first

call

to

dealloc

is

legal

since

the

children

of

r

can

not

have

any

elements

above

them

and

hence

can

be

removed

from

further

consideration

The

second

call

is

legal

because

if

any

of

the

pnsets

of

a

node

s

below

r

includes

r

and

r

kmax

then

there

can

be

no

elements

above

r

itself

and

hence

it

can

be

removed

from

the

pnset

of

r

and

deallocated

Technically

dealloc

keeps

a

list

of

the

nodes

that

need

to

be

deallocated

When

the

recursive

execution

of

suremax

has

completed

these

nodes

are

all

deallocated

carefully

so

that

INV

-LRB-

F

G

-RRB-

is

maintained

Theorem

When

called

with

sound

values

suremax

maintains

the

essential

invariant

INV

-LRB-

F

G

-RRB-

z

maxa

-LRB-

gen

F

-RRB-

:d

-RRB-

-RRB-

wp

-LRB-

suremax

-LRB-

G

d

z

-RRB-

INV

-LRB-

F

G

-RRB-

-RRB-

Proof

Sketch

The

routine

suremax

terminates

because

it

only

recurses

with

a

value

equal

or

greater

than

its

original

argument

and

if

called

a

second

time

on

the

same

node

with

the

same

or

greater

argument

it

will

visit

the

node

at

most

once

Thus

it

will

visit

any

node

in

the

SDAG

at

most

once

For

any

pin

element

s

the

sum

of

the

kmax

values

of

its

pnset

is

greater

than

than

maxa

s

since

any

total

element

will

be

counted

by

some

pin

set

element

and

must

be

above

at

least

one

member

of

the

pin

set

In

the

other

case

the

kmax

value

of

a

node

is

passed

to

its

immediate

descendants

by

INV

-LRB-

F

G

-RRB-

and

this

is

greater

than

or

equal

to

maxa

-LRB-

gen

F

-RRB-

:d

If

the

kmax

value

of

a

record

r

is

set

to

the

maxa

value

of

every

descendant

is

Since

the

descendants

of

such

a

node

can

contribute

nothing

even

in

terms

of

pin

sets

and

we

can

remove

every

descendant

of

r

if

we

maintain

the

invariant

by

adjusting

baset

pnset

kids

and

pars

appropriately

Theorems

and

assure

us

of

the

soundness

of

using

suremin

and

suremax

However

these

algorithms

are

clearly

a

compromise

between

efficiency

and

completeness

and

many

similar

and

sound

algorithms

can

be

generated

by

making

slightly

different

design

decisions

Incremental

Fact

Set

Realization

An

advantage

of

the

sandbag

and

the

fact

set

approach

in

particular

is

that

they

are

incremental

in

nature

This

subsection

provides

the

algorithms

that

allow

individual

revelations

to

be

added

to

a

fact

set

and

reflected

in

an

SDAG

implementing

that

fact

set

The

addition

of

a

new

fact

to

a

fact

set

tightens

the

bounds

of

the

sandbag

it

generates

incrementally

improving

it

with

a

reasonable

amount

of

computation

This

incremental

improvement

is

accomplished

in

the

SDAG

with

procedures

that

propagate

the

effect

of

a

revelation

-LRB-

rev

-RRB-

or

a

pinning

-LRB-

pin

-RRB-

throughout

the

SDAG

suremin

and

suremax

are

employed

by

rev

and

pin

as

a

means

of

improving

the

SDAG

and

maintaining

the

essential

invariant

that

allows

and

to

be

calculated

The

rev

algorithm

may

require

the

addition

of

a

node

with

a

new

data

field

to

the

SDAG

This

is

accomplished

by

the

routine

lkpormk

-LRB-

lookup

or

make

-RRB-

This

routine

is

also

used

by

pin

described

in

the

next

subsection

Algorithm

The

procedure

lkpormk

returns

the

node

with

data

d

if

it

exists

or

creates

it

if

it

does

not

and

returns

it

proc

lkpormk

-LRB-

val-res

G

G

val

d

D

res

r

-RRB-

var

r

R

a

z

r

-LRB-

G

d

-RRB-

if

r

r

allocnode

:d

r

pars

-LRB-

G

d

-RRB-

r

kids

-LRB-

G

d

-RRB-

a

-LRB-

G

d

-RRB-

z

-LRB-

G

d

-RRB-

addnode

-LRB-

G

r

-RRB-

r

kmin

a

r

kmax

z

return

r

The

routine

lkpormk

uses

allocnode

to

obtain

space

and

addnode

to

maintain

the

structural

invariant

Definition

The

procedure

allocnode

:d

allocates

a

new

record

with

data

d.

Algorithm

The

procedure

addnode

-LRB-

r

G

-RRB-

adds

a

record

r

to

an

SDAG

in

that

it

makes

the

immediate

ancestors

of

r

point

to

r

with

the

kids

fields

and

the

immediate

descendants

point

to

r

with

the

pars

fields

proc

addnode

-LRB-

val-res

G

SDAG

val

r

R

-RRB-

G

G

-LSB-

frg

for

-LRB-

c

c

r

pars

-RRB-

c

kids

-LRB-

c

kids

r

kids

-RRB-

-LSB-

frg

rof

for

-LRB-

c

c

r

kids

-RRB-

c

pars

-LRB-

c

pars

r

pars

-RRB-

-LSB-

frg

rof

Theorem

If

r

has

correct

kids

and

pars

fields

addnode

r

maintains

the

structural

invariant

-LRB-

sinv

-LRB-

F

G

-RRB-

r

kids

imdec

-LRB-

r

data

-RRB-

r

pars

imanc

-LRB-

r

data

-RRB-

-RRB-

-RRB-

wp

-LRB-

addnode

-LRB-

G

r

-RRB-

sinv

-LRB-

F

G

-RRB-

-RRB-

The

procedure

lkpormk

is

the

only

procedure

in

our

set

of

algorithms

that

adds

a

node

to

the

SDAG

since

it

does

so

with

addnode

-LRB-

G

d

-RRB-

and

then

sets

the

kmin

value

of

the

created

record

to

-LRB-

G

d

-RRB-

and

the

kmax

to

-LRB-

G

d

-RRB-

it

maintains

and

The

revelation

operation

is

simply

the

execution

of

lkpormk

with

a

partial

data

value

followed

by

suremin

and

suremax

to

tighten

bounds

where

possible

based

on

the

revelation

hd

a

zi

Algorithm

The

revelation

operation

proc

rev

-LRB-

val

G

G

d

D

a

z

-RRB-

var

r

R

lkpormk

-LRB-

G

d

r

-RRB-

suremin

-LRB-

G

r

a

-RRB-

suremax

-LRB-

G

r

z

-RRB-

Theorem

The

revelation

algorithm

correctly

updates

the

SDAG

INV

-LRB-

F

G

-RRB-

-RRB-

wp

:-LRB-

rev

-LRB-

G

d

a

z

-RRB-

INV

-LRB-

F

-LSB-

fhd

a

zig

G

-RRB-

Proof

Sketch

By

the

definition

of

consistency

with

a

revelation

hd

a

zi

it

preserves

INV

-LRB-

F

G

-RRB-

to

call

suremin

-LRB-

G

r

a

-RRB-

and

suremax

-LRB-

G

r

z

-RRB-

The

client

should

insure

that

a

and

z

are

tighter

than

could

be

derived

by

-LRB-

G

d

-RRB-

and

-LRB-

G

d

-RRB-

perhaps

by

calling

those

functions

before

calling

rev

so

that

uninformative

elements

are

not

stored

in

the

sandbag

The

realization

of

pinnings

Unfortunately

the

algorithm

to

reflect

a

pinning

-LRB-

Figure

-RRB-

throughout

the

SDAG

described

in

this

subsection

is

rather

complicated

pin

relies

on

the

meaning

of

a

pinning

to

set

the

kmax

value

of

ancestors

of

a

node

which

are

not

in

the

pin

set

to

zero

Algorithm

The

pinning

operation

See

Figure

Invariants

of

this

algorithm

are

boxed

Note

that

in

this

algorithm

the

variable

Z

holds

records

whose

values

may

be

tightened

by

the

addition

of

the

new

pin

set

Recalculating

these

values

in

the

last

three

statements

of

the

routine

is

essential

to

informative

and

calculation

but

does

not

affect

correctness

The

routine

updatepin

-LRB-

G

r

A

B

-RRB-

updates

the

baset

pnset

fields

of

its

arguments

so

that

the

sinv

is

maintained

The

argument

A

is

a

set

of

records

that

are

removed

from

the

pnset

of

r

and

B

is

a

set

of

records

that

is

added

to

the

pnset

of

r

The

basets

of

the

appropriate

records

are

updated

accordingly

so

that

sinv

is

maintained

A

pre-condition

of

updatepin

-LRB-

G

r

A

B

-RRB-

is

that

the

addition

of

A

and

deletion

of

B

to

the

pnset

of

r

leaves

it

a

cochain

and

that

the

new

pnset

of

r

contains

an

element

that

approximates

any

element

s

above

r

data

if

maxa

-LRB-

s

data

-RRB-

A

further

condition

that

avoids

ambiguity

is

that

A

B

proc

pin

-LRB-

val-res

G

G

val

d

D

P

-RRB-

Pre-condition

INV

-LRB-

F

G

-RRB-

var

y

R

R

A

Z

W

Q

lkpormk

-LRB-

G

d

y

-RRB-

if

y

pnset

U

cocadd

-LRB-

r

P

G

Z

-RRB-

updatepin

-LRB-

G

r

-LRB-

U

Z

-RRB-

-RRB-

A

sharenoub

-LRB-

map

-LRB-

data

y

kids

-RRB-

map

-LRB-

data

y

pnset

-RRB-

-RRB-

-LSB-

-RSB-

y

pnset

U

R

A

Z

y

pnset

Loop

Invariant

INV

F

-LSB-

fhd

P

ig

G

-LRB-

x

x

A

maxa

-LRB-

gen

-LRB-

F

-LSB-

fhd

P

ig

-RRB-

-RRB-

x

-RRB-

for

-LRB-

x

x

R

-RRB-

Q

mayhaveub

-LRB-

P

x

data

-RRB-

if

Q

updatepin

-LRB-

G

y

fxg

;-RRB-

A

A

-LSB-

fxg

-LSB-

-RSB-

-LRB-

-LRB-

p

p

Q

x

data

p

data

-RRB-

cochain

-LRB-

map

-LRB-

data

Q

-LSB-

-LRB-

y

pnset

fxg

-RRB-

-RRB-

cocadd

-LRB-

y

Q

G

W

-RRB-

updatepin

-LRB-

G

y

W

fxg

-RRB-

Z

Z

-LSB-

W

-LSB-

-RSB-

otherwise

skip

rof

INV

F

-LSB-

fhd

P

ig

G

-LRB-

x

x

A

maxa

-LRB-

gen

-LRB-

F

-LSB-

fhd

P

ig

-RRB-

-RRB-

x

-RRB-

for

-LRB-

s

s

A

-RRB-

suremax

-LRB-

s

G

-RRB-

rof

INV

-LRB-

F

-LSB-

fhd

P

ig

G

-RRB-

for

-LRB-

s

s

Z

-RRB-

suremax

-LRB-

G

s

-LRB-

G

s

pars

fs

datag

s

data

-RRB-

-RRB-

rof

suremax

-LRB-

G

y

-LRB-

G

y

pars

fy

datag

y

data

-RRB-

-RRB-

suremin

-LRB-

G

y

-LRB-

G

y

kids

-RRB-

-RRB-

for

-LRB-

s

s

y

pnset

-RRB-

if

s

kmax

dealloc

-LRB-

G

s

-RRB-

rof

Post-condition

INV

-LRB-

F

-LSB-

fhd

P

ig

G

-RRB-

Figure

Algorithm

Theorem

The

function

updatepin

preserves

the

invariant

when

the

preconditions

are

met

-LRB-

INV

-LRB-

F

G

-RRB-

cochain

-LRB-

map

-LRB-

data

-LRB-

-LRB-

r

pnset

-LSB-

A

-RRB-

B

-RRB-

-RRB-

-RRB-

A

B

-LRB-

r

data

v

a

tot

a

-LRB-

q

map

-LRB-

data

r

pnset

-LSB-

A

B

-RRB-

q

v

a

-RRB-

-RRB-

maxa

-LRB-

gen

F

-RRB-

a

-RRB-

-RRB-

-RRB-

wp

-LRB-

updatepin

-LRB-

G

r

A

B

-RRB-

INV

-LRB-

F

G

-RRB-

-RRB-

The

function

cocadd

-LRB-

b

P

G

Z

-RRB-

takes

a

cochain

of

data

elements

P

and

insures

that

every

element

of

P

is

represented

by

a

record

in

the

SDAG

by

adding

records

if

necessary

It

returns

a

set

of

records

corresponding

to

P

as

the

result

Z.

G

is

considered

a

value

result

parameter

because

nodes

may

be

added

Theorem

The

pinning

algorithm

pin

preserves

the

essential

invariant

Let

the

precondition

of

pin

on

P

and

d

be

abbreviated

by

PINCON

cochain

:P

-LRB-

a

a

P

d

v

a

-RRB-

INV

-LRB-

F

G

-RRB-

-RRB-

wp

-LRB-

pin

-LRB-

G

hd

P

i

-RRB-

INV

-LRB-

F

-LSB-

fhd

P

ig

G

-RRB-

-RRB-

Proof

Sketch

The

first

three

statements

merely

make

sure

everything

exists

and

the

structural

invariant

is

preserved

The

large

if

statement

sets

the

pin

set

of

this

node

either

to

the

argument

or

to

some

more

restrictive

pin

set

that

is

guaranteed

to

be

correct

by

virtue

of

the

existing

pin

set

and

the

new

pin

set

After

this

statement

the

set

A

contains

nodes

that

have

a

maxa

value

of

hence

applying

suremax

to

them

with

an

argument

of

is

legal

The

remaining

statements

are

correct

by

the

correctness

of

and

INV

-LRB-

F

G

-RRB-

Using

the

SDAG

This

section

discusses

the

informativeness

of

the

suboptimal

SDAG

algorithms

describes

their

worst-case

polynomial

complexity

and

discusses

informally

their

expected

performance

The

theorems

of

the

Section

are

summarized

in

the

following

theorem

Theorem

Let

G

be

an

SDAG

data

structure

built

up

by

calls

to

rev

-LRB-

G

d

a

z

-RRB-

and

pin

-LRB-

d

P

G

-RRB-

where

for

every

call

hd

a

zi

or

hd

P

i

is

in

an

externally

consistent

fact

set

F

Queries

of

this

function

using

and

never

yield

a

result

inconsistent

with

the

fact

set

This

can

be

stated

-LRB-

-LRB-

G

d

-RRB-

mina

-LRB-

gen

F

-RRB-

:d

-LRB-

G

d

-RRB-

maxa

-LRB-

gen

F

-RRB-

:d

-RRB-

If

we

momentarily

treat

the

implementation

of

the

sandbag

as

a

sandbag

this

can

be

stated

more

concisely

Gv

SBgen

F

This

critical

theorem

essentially

states

the

correctness

of

the

SDAG

structure

and

its

algorithms

However

this

theorem

does

not

address

the

utility

or

efficiency

of

the

sandbag

since

the

following

implementation

of

mina

and

maxa

would

also

satisfy

that

theorem

-LRB-

d

mina

:d

maxa

:d

-RRB-

This

answer

is

consistent

but

not

informative

Since

optimal

calculation

of

mina

and

maxa

are

in

general

intractable

and

the

SDAG

is

designed

to

be

practical

will

not

be

equal

to

mina

in

every

case

although

by

Theorem

is

sound

Similarly

is

guaranteed

to

provide

a

consistent

overestimate

of

maxa

but

will

not

in

general

be

the

best

bound

computable

However

the

fact

set

approach

is

not

meant

to

be

a

reasoning

mechanism

but

an

efficient

and

incremental

representation

of

imprecise

knowledge

about

sets

We

believe

the

SDAG

structures

is

informative

enough

and

efficient

enough

to

be

useful

in

practice

Unfortunately

we

have

not

been

able

to

discover

a

precise

measure

of

how

informative

these

algorithms

are

in

part

due

to

the

problem

outlined

above

Our

implementation

appears

to

experimentally

validate

our

supposition

that

the

SDAG

is

in

practice

reasonably

informative

However

a

more

direct

means

of

comparison

to

some

other

existing

technique

that

would

allow

quantification

of

this

performance

is

future

work

The

best

guarantee

of

informativeness

at

present

is

Theorem

that

follows

from

Theorem

Theorem

The

following

invariant

is

maintained

by

rev

and

pin

r

data

v

s

data

-RRB-

r

kmin

s

kmin

r

kmax

s

kmax

Proof

Sketch

When

any

node

is

added

by

addnode

this

is

true

because

lkpormk

calls

and

that

maintain

this

property

The

function

suremin

maintains

this

invariant

because

it

always

calls

itself

on

each

parent

with

an

argument

that

is

at

least

the

kmin

value

of

any

child

since

returns

at

least

the

kmin

value

of

any

child

The

function

suremax

maintains

this

invariant

because

it

is

called

on

every

child

with

its

arguments

so

no

child

of

r

will

have

a

kmax

value

greater

than

r

kmax

The

deallocation

of

nodes

does

not

change

this

invariant

Therefore

rev

maintains

this

invariant

Furthermore

pin

maintains

this

invariant

since

it

only

adds

nodes

and

uses

suremin

and

suremax

Theorem

For

any

SDAG

G

built

up

from

F

using

rev

and

pin

the

functions

and

produce

bounds

at

least

as

good

as

those

that

are

directly

revealed

to

it

he

a

zi

F

e

v

d

-RRB-

-LRB-

G

d

-RRB-

z

he

a

zi

F

d

v

e

-RRB-

-LRB-

G

d

-RRB-

a

Proof

Sketch

This

follows

directly

from

Theorem

since

any

call

to

examines

the

parents

of

d

one

of

which

by

the

premise

has

a

kmax

value

of

at

most

z.

Similarly

examines

the

ancestors

of

d

one

of

which

by

premise

is

at

most

a

and

any

any

node

between

that

and

d

is

therefore

at

most

a.

If

a

fact

is

directly

revealed

it

will

be

reflected

in

the

SDAG

and

not

forgotten

or

missed

The

SDAG

can

thus

be

used

confidently

as

a

container

for

multiresolution

data

items

However

this

theorem

tells

us

nothing

about

the

SDAGs

ability

to

draw

more

complicated

inferences

Complexity

of

the

SDAG

Algorithms

Although

it

is

not

yet

possible

to

be

precise

about

the

informativeness

of

the

SDAG

algorithms

interesting

upper

bounds

on

the

worst

case

running

times

of

these

algorithms

can

be

stated

Definition

Given

an

SDAG

G

we

refer

to

various

parameters

of

G

with

lower

case

letters

in

expressions

describing

complexities

ffl

n

stands

for

the

number

of

nodes

ffl

e

stands

for

the

number

of

edges

ffl

o

stands

for

the

maximum

out-degree

-LRB-

the

maximum

cardinality

of

the

kids

field

-RRB-

ffl

i

stands

for

the

maximum

in-degree

-LRB-

the

maximum

cardinality

of

the

pars

field

-RRB-

ffl

p

stands

for

the

maximum

cardinality

of

pnset

for

any

node

and

ffl

b

stands

for

the

maximum

cardinality

of

baset

for

any

node

All

sets

are

implemented

with

a

data

structure

that

allows

linear

time

traversal

Complexities

a

v

b

O

-LRB-

-RRB-

no-ub

-LRB-

a

b

-RRB-

O

-LRB-

-RRB-

tot

:d

O

-LRB-

-RRB-

mayhaveub

-LRB-

A

d

-RRB-

-LRB-

jAj

-RRB-

sharenoub

-LRB-

A

B

-RRB-

-LRB-

jAj

jBj

-RRB-

noubsep

Z

O

-LRB-

-RRB-

Table

Summary

of

Simple

Functions

Function

call

Worst

case

for

Brief

Explanation

Arbitrary

SDAG

-LRB-

G

d

-RRB-

O

-LRB-

e

-RRB-

at

worst

looks

at

each

edge

-LRB-

G

d

-RRB-

O

-LRB-

e

-RRB-

d.f.s.

of

graph

-LRB-

G

d

-RRB-

O

-LRB-

b

e

-RRB-

d.f.s.

of

graph

and

main

of

ancs

-LRB-

G

d

-RRB-

O

-LRB-

n

o

-RRB-

d.f.s.

with

cochain

maintenance

-LRB-

G

Z

-RRB-

O

-LRB-

-RRB-

PjZj

i

O

-LRB-

-RRB-

-LRB-

noubsep

quadratic

-RRB-

-LRB-

G

d

-RRB-

O

-LRB-

e

n

o

-RRB-

O

-LRB-

:o-RRB-

-LRB-

G

R

P

d

-RRB-

O

-LRB-

jRj

jPj

p

-RRB-

cost

of

in

R

loop

and

p

in

P

loop

-LRB-

G

d

-RRB-

O

-LRB-

e

b

n

p

i

-RRB-

calc

of

args

and

n

p

subsumed

by

suremin

-LRB-

G

r

a

-RRB-

O

-LRB-

e

n

-RRB-

every

edge

plus

at

most

once

per

node

dealloc

-LRB-

G

Z

-RRB-

O

-LRB-

jZj

-LRB-

e

n

-RRB-

-RRB-

may

go

hit

every

edge

above

Z

modify

basets

suremax

-LRB-

G

r

z

-RRB-

O

-LRB-

e

n

-LRB-

b

p

-RRB-

-RRB-

baset

loop

at

most

once

per

node

plus

e

addnode

-LRB-

G

r

-RRB-

i

o

-LRB-

i

o

-RRB-

worst

case

i

o

lkpormk

-LRB-

G

d

-RRB-

O

-LRB-

b

i

i

o

-LRB-

i

o

-RRB-

-RRB-

addnode

plus

plus

rev

-LRB-

G

d

a

z

-RRB-

O

-LRB-

io

-LRB-

i

o

-RRB-

lkpormk

plus

suremax

plus

suremin

en

-LRB-

b

p

-RRB-

-RRB-

updatepin

-LRB-

G

r

A

B

-RRB-

O

-LRB-

b

jAj

-RRB-

subtract

A

from

worst

baset

cocadd

-LRB-

b

P

G

Z

-RRB-

O

-LRB-

jPj

-LRB-

e

n

o

i

o

-LRB-

i

o

-RRB-

-RRB-

-RRB-

pin

-LRB-

G

d

P

-RRB-

O

-LRB-

This

bound

assumes

cocadd

implemented

via

p

-LRB-

e

no

io

-LRB-

i

o

-RRB-

pnb

-RRB-

multiple

calls

to

lkpormk

-LRB-

Note

this

is

in

O

-LRB-

-RRB-

-RRB-

Table

Summary

of

Complexities

of

Major

Functions

Upper

bounds

on

the

worst-case

running

times

for

the

algorithms

defined

or

mentioned

in

this

paper

are

expressed

in

terms

of

these

parameters

in

Table

and

Table

These

bounds

could

be

lowered

with

more

careful

analysis

or

by

amortization

Although

all

of

these

complexities

could

have

been

described

in

terms

of

n

or

e

greater

detail

has

been

used

because

in

practice

the

out-degree

in-degree

and

pin

set

cardinalities

will

be

much

smaller

than

the

worst

cases

Furthermore

the

worst

case

running

times

will

rarely

be

observed

To

a

large

extent

the

construction

of

fact

sets

influences

these

running

times

In

particular

the

algorithms

are

both

faster

and

closer

to

the

optimal

to

the

extent

that

the

underlying

data

domain

has

the

property

that

incomparable

data

elements

have

no

upper

bound

in

common

This

produces

SDAGs

with

low

average

in-degrees

that

are

not

very

graphy

but

more

tree-like

This

is

true

of

many

partial

data

types

used

in

practice

it

is

conspicuously

not

the

case

for

tuples

of

partially

ordered

elements

under

the

ordering

for

tuples

However

in

practice

it

may

be

possible

to

choose

revelations

in

such

a

way

that

the

average

in-degree

is

low

or

These

suspicions

are

supported

by

our

current

implementation

The

SDAG

is

not

monotonic

Unfortunately

the

SDAG

algorithms

presented

here

are

not

monotonic

That

is

a

revelation

or

pinning

may

actually

make

the

value

returned

by

or

worse

for

some

particular

value

although

of

course

in

general

revelations

and

pinnings

make

a

sandbag

more

informative

-LRB-

-LRB-

G

d

-RRB-

-LRB-

H

d

-RRB-

-LRB-

G

d

-RRB-

-LRB-

H

d

-RRB-

-RRB-

where

H

is

an

SDAG

generated

by

applying

consistent

revelations

and

pinnings

to

G.

By

limiting

the

kind

of

inferences

used

by

the

mina

and

maxa

algorithm

it

might

be

possible

to

construct

monotonic

algorithms

However

it

is

probably

not

possible

to

construct

monotonic

algorithms

without

loosening

the

bounds

provided

for

mina

and

maxa

overmuch

If

monotonicity

is

required

by

some

application

the

dirty

trick

of

simply

storing

the

return

value

of

every

mina

and

maxa

query

and

never

returning

looser

values

may

be

used

in

practice

Current

Implementation

This

document

uses

formal

notation

that

may

put

off

some

readers

or

make

it

difficult

for

them

to

imagine

an

implementation

of

the

SDAG

In

this

section

we

briefly

describe

our

current

implementation

of

the

SDAG

so

that

readers

can

have

some

inkling

how

the

SDAG

is

actually

implemented

Specific

performance

results

and

implementation

details

are

not

discussed

in

this

paper

Our

implementation

is

about

lines

of

C++

It

leverages

the

class

structure

of

C++

as

much

as

possible

The

sandbag

class

contains

public

members

for

rev

and

pin

Additionally

it

contains

a

large

number

of

private

members

such

as

suremin

and

updatepin

This

class

uses

a

virtual

class

tuple

as

the

type

of

the

data

elements

of

the

sandbag

The

tuple

class

supports

the

composition

of

subclasses

of

a

virtual

class

primitive

that

implements

multiresolution

primitive

types

Because

this

class

is

virtual

it

can

work

for

any

multiresolution

data

type

that

is

expressed

as

a

subclass

of

primitive

The

salient

features

of

the

primitive

class

that

a

programmer

of

a

new

data

type

must

implement

are

the

tot

noub

seta

and

approx

members

that

corresponds

to

the

tot

no-ub

seta

and

v

functions

in

our

formal

framework

These

functions

need

not

be

implemented

for

the

tuple

class

as

their

meaning

and

implementation

follow

directly

from

Definition

The

only

subclass

of

primitive

that

have

been

implemented

so

far

are

integer

intervals

Unfortunately

sandbags

are

not

directly

comparable

to

any

existing

data

structures

or

algorithms

A

fairly

complicated

testbed

framework

has

been

implemented

that

allows

revelations

and

pinnings

to

be

made

in

a

variety

of

ways

These

include

interactive

mouse-based

creation

of

revelations

and

pinnings

random

revelations

and

pinnings

and

random

revelations

and

pinnings

that

are

guaranteed

not

to

overlap

and

hence

be

as

informative

as

possible

In

an

attempt

to

describe

the

performance

of

our

system

information

theoretic

measures

have

been

developed

that

bound

the

number

of

bits

-LCB-

based

on

a

particular

probability

model

of

the

content

of

a

set

-LCB-

that

a

sandbag

contains

This

has

been

used

in

testing

to

interactively

demonstrate

that

some

revelations

are

more

informative

than

others

and

to

compare

design

decisions

that

trade

informativeness

for

performance

The

theoretical

and

practical

aspects

of

these

measures

are

outside

the

scope

of

this

paper

The

performance

of

our

testbed

system

at

present

is

very

difficult

to

describe

but

the

current

implementation

appears

efficient

enough

to

be

practical

for

applications

where

I/O

costs

dominate

computation

time

such

as

multimedia

applications

The

construction

of

sandbags

is

currently

not

competitive

with

normal

set

operations

as

might

be

expected

On

a

fast

SPARC

a

sandbag

of

several

hundred

facts

can

be

constructed

in

about

ten

seconds

The

execution

of

a

single

mina

or

maxa

query

on

a

sandbag

is

much

faster

than

the

realization

of

a

revelation

or

a

pinning

The

main

lesson

our

implementation

has

taught

us

so

far

is

that

the

performance

of

an

SDAG

is

highly

dependent

on

the

nature

of

the

revelations

and

pinnings

from

which

it

is

constructed

By

designing

revelations

performance

can

be

made

reasonable

but

not

competitive

with

a

normal

set

implementation

Related

Research

There

has

been

a

great

deal

of

work

on

incomplete

information

in

databases

-LSB-

-RSB-

much

of

which

has

focused

on

disjunctive

tuples

This

paper

has

some

bearing

on

that

body

of

research

however

our

emphasis

has

been

somewhat

different

Our

goal

is

to

improve

performance

by

avoiding

the

manipulation

or

access

of

large

quantities

of

data

In

contrast

much

of

the

work

on

incomplete

information

in

databases

has

focused

on

clean

semantics

-LSB-

-RSB-

characterizing

the

degree

of

incompleteness

-LSB-

-RSB-

and

the

complexity

of

obtaining

complete

answers

relative

to

the

available

information

-LSB-

-RSB-

The

sandbag

has

been

most

influenced

by

work

based

on

the

sandwich

idea

-LSB-

-RSB-

that

inspired

its

name

The

sandwich

can

not

represent

information

about

cardinalities

which

is

the

foundation

of

the

sandbag

The

sandbag

is

thus

technically

more

expressive

than

the

sandwich

The

sandbag

may

be

efficient

because

it

allows

underlying

partially

ordered

multiresolution

types

to

be

used

as

compact

codes

for

subsets

of

the

domain

The

problem

of

compactly

and

efficiently

representing

the

very

large

sets

of

tuples

that

are

generated

by

a

naive

implementation

of

the

sandwich

has

been

addressed

by

Vrbsky

Smith

and

Liu

in

a

working

object-oriented

implementation

-LSB-

-RSB-

In

this

system

tuples

are

part

of

a

class

hierarchy

and

the

names

of

the

classes

can

stand

for

the

whole

classes

in

the

set

uncertain

tuples

thus

representing

very

large

uncertain

sets

efficiently

This

is

very

closely

related

to

the

partially

ordered

data

types

used

in

this

paper

and

could

be

represented

by

them

The

partially

ordered

types

used

in

this

paper

differ

from

the

class

hierarchy

approach

-LSB-

-RSB-

mainly

in

being

more

general

and

in

emphasizing

multimedia

objects

rather

than

more

traditional

tuple-style

data

values

Those

researchers

have

developed

and

implemented

algorithms

for

approximate

relational

operators

which

is

future

work

for

us

The

object-oriented

class

hierarchy

approach

is

simpler

than

the

sandbag

and

offers

commensurate

advantages

such

as

ease

of

implementation

and

completeness

but

is

less

expressiveness

and

potentially

less

efficient

The

notion

of

constructing

a

query

system

that

can

trade

off

precision

for

performance

-LSB-

-RSB-

is

one

of

our

motivations

This

idea

is

particularly

beneficial

when

applied

to

multimedia

and

scientific

databases

because

larger

tradeoffs

are

possible

for

those

applications

-LSB-

-RSB-

This

paper

is

a

necessary

step

towards

testing

these

claims

The

sandbag

approach

gives

up

relative

completeness

immediately

Similarly

this

paper

is

not

concerned

with

the

difficult

issues

of

semantics

of

values

that

arise

in

databases

but

do

not

arise

when

approximating

individual

sets

in

isolation

that

is

the

focus

of

this

paper

Our

emphasis

has

been

on

expressiveness

compactness

and

implementation

performance

Indirectly

Related

Research

A

number

of

research

areas

are

indirectly

or

philosophically

related

to

this

paper

The

reader

is

invited

to

explore

these

relationships

The

sandbag

is

clearly

related

to

a

fuzzy

set

-LSB-

-RSB-

in

principle

although

in

practice

there

is

little

in

common

The

imprecision

of

fuzzy

sets

is

expressed

in

terms

of

probabilities

whereas

the

sandbag

is

based

on

interval

and

partial

data

elements

that

though

imprecise

provide

information

free

of

any

probabilities

The

notion

of

a

rough

set

-LSB-

-RSB-

is

similar

to

sandwiches

and

sandbags

but

the

research

that

we

know

of

on

rough

sets

is

concerned

more

with

data

modelling

than

with

saving

time

through

the

use

of

approximations

Our

DAG

based

implementation

is

closely

related

to

NP-complete

problems

such

as

integer

linear

programming

and

polynomial

problems

such

as

linear

programming

-LSB-

-RSB-

and

sensitivity

analysis

This

relationship

needs

to

be

explored

further

in

future

work

Similarly

there

are

many

special

cases

in

which

the

overall

complexity

and

completeness

could

be

greatly

improved

For

instance

an

important

special

case

occurs

when

the

partial

data

elements

in

a

sandbag

are

rectangles

-LRB-

in

whatever

dimension

-RRB-

In

this

case

computational

geometry

-LSB-

-RSB-

and

the

existence

of

metrics

on

the

elements

allow

solutions

better

than

our

general

approach

The

incremental

construction

of

the

sandbag

is

a

special

instance

of

a

general

constraint

solver

-LSB-

-RSB-

There

may

be

a

connection

between

the

sandbag

and

constraint

logic

programming

However

the

sandbag

is

specifically

an

imprecise

set

representation

rather

than

an

approach

to

more

general

constraints

or

to

programming

via

constraints

This

specificity

the

limited

power

of

the

revelation

and

pinning

and

the

limitation

of

the

sandbag

to

partially

ordered

data

types

may

preclude

any

direct

connection

to

that

work

There

is

a

philosophical

relationship

between

the

ap

proach

of

abstract

interpretation

and

the

sandbag

in

that

both

suggest

deriving

useful

information

from

a

minimal

computation

Additional

Applications

The

sandbag

is

a

very

general

construct

If

it

can

be

implemented

efficiently

it

will

support

a

number

of

possible

applications

outside

the

immediate

realm

of

multiresolution

data

retrieval

-LSB-

-RSB-

ffl

The

sandbag

is

a

convenient

framework

for

statistical

sampling

because

it

can

fluidly

and

incrementally

combine

the

notions

of

a

histogram

and

a

sampling

ffl

The

sandbag

is

a

framework

for

lossy

compression

of

sets

of

essentially

arbitrary

data

types

Although

lossy

compression

is

relatively

common

for

descriptions

of

continuous

functions

such

as

images

and

sounds

it

is

much

less

common

for

other

kinds

of

data

-LCB-

in

part

because

lossy

versions

of

other

kinds

of

data

are

less

useful

ffl

One

of

many

forms

of

incompatibility

of

databases

that

complicate

attempts

to

build

heterogeneous

databases

or

multi-databases

is

precision

discrepancy

-LSB-

-RSB-

the

representation

of

data

at

different

precisions

in

different

databases

The

expressiveness

of

the

sandbag

may

allow

a

consistent

treatment

of

data

at

different

precisions

ffl

Query

result

size

estimation

-LSB-

-RSB-

facilitates

query

optimization

The

sandbag

may

be

a

useful

contribution

to

this

research

because

it

is

a

framework

for

describing

the

cardinality

of

relations

in

various

levels

of

detail

Conclusion

and

Future

Work

This

paper

explains

the

fact

set

approach

to

incrementally

constructing

sandbags

explores

its

complexity

and

describes

a

promising

data

structure

and

set

of

algorithms

that

implement

it

Although

experimental

validation

of

the

SDAG

data

structure

is

future

work

this

paper

is

a

necessary

and

significant

basis

for

that

and

other

future

work

In

describing

the

complexity

of

the

fact

set

approach

to

sandbags

a

point

in

the

spectrum

of

imprecise

set

representations

has

been

explored

that

may

be

of

interest

to

researchers

working

on

other

imprecise

set

representations

However

the

main

contribution

of

this

paper

is

progress

towards

a

practical

implementation

of

the

sandbag

which

is

a

new

approach

to

approximating

sets

that

has

great

potential

A

number

of

interesting

tasks

remain

that

would

increase

or

elucidate

the

efficiency

of

the

fact

set

implementation

ffl

The

expected

case

performance

is

highly

dependent

on

the

structure

of

SDAG

that

is

a

function

of

the

fact

set

and

the

underlying

partially

ordered

data

domain

This

has

in

fact

been

observed

in

the

implementation

An

analysis

of

expected

case

performance

in

terms

of

this

structure

would

allow

us

to

predict

the

applicability

of

the

sandbag

to

various

problems

and

to

design

the

construction

of

fact

sets

in

the

most

economical

manner

ffl

In

the

very

important

special

case

of

perfectly

tree-like

data

domains

the

sandbag

can

be

perfectly

computed

in

polynomial

time

Efficient

algorithms

for

this

special

case

need

to

be

developed

ffl

The

algorithms

and

data

structure

presented

in

this

paper

do

not

consider

the

important

special

case

that

the

information

about

a

certain

part

of

the

sandbag

is

in

fact

complete

When

a

portion

of

the

sandbag

is

completely

specified

it

may

be

more

efficient

to

use

a

traditional

set

data

structure

to

represent

that

portion

of

the

sandbag

This

issue

is

related

to

the

issue

of

totality

preservation

-LSB-

-RSB-

ffl

To

determine

which

children

of

a

node

approximate

some

data

value

e

each

child

is

examined

by

the

current

algorithms

If

there

exists

a

transitive

antisymmetric

irreflexive

ordering

OE

that

has

the

the

property

x

OE

y

-RRB-

xvy

that

ordering

can

be

used

to

avoid

examining

certain

child

nodes

This

is

simply

an

extension

of

the

idea

of

binary

search

tree

For

instance

real

intervals

can

be

ordered

by

-LSB-

a

b

-RSB-

OE

-LSB-

c

d

-RSB-

b

c

If

we

keep

the

children

of

a

node

in

non-decreasing

order

of

this

relation

we

need

not

examine

those

after

the

first

child

f

that

has

the

property

f

OE

e

Such

orders

may

exist

for

many

important

data

domains

and

will

generally

decrease

the

cost

of

lookup

imanc

and

imdec

that

are

used

throughout

the

algorithms

presented

in

this

paper

This

becomes

more

important

as

an

SDAG

approaches

totality

and

serves

the

traditional

purposes

of

set

representation

ffl

In

practice

several

revelations

and

pinnings

may

be

grouped

together

Considerable

savings

may

be

possible

by

performing

these

operations

together

rather

than

sequentially

The

general

utility

of

the

sandbag

for

many

applications

depends

on

several

issues

that

are

currently

unresolved

and

independent

of

the

efficiency

of

the

implementation

ffl

The

development

of

algorithms

for

the

analogs

of

set-theoretic

and

relational

algebra

operators

-LRB-

-LSB-

oe

ss

and

-RRB-

that

operate

on

and

produce

sandbags

-LSB-

-RSB-

will

greatly

expand

the

usefulness

of

the

sandbag

and

allow

a

database

management

system

to

be

constructed

from

them

ffl

If

the

SDAG

structure

itself

can

be

efficiently

encoded

then

the

SDAG

may

be

a

flexible

and

reasonably

efficient

lossy

compression

mechanism

for

kinds

of

data

for

which

no

lossy

compression

mechanism

is

currently

used

ffl

An

extension

of

an

SDAG

structure

to

a

diskbased

storage

structure

might

allow

efficient

retrieval

of

low

information

content

-LRB-

low-resolution

-RRB-

sandbags

A

final

idea

is

to

use

some

of

the

ideas

of

our

implementation

here

to

build

a

non-monotonic

or

reactive

implementation

of

a

sandbag

This

would

allow

not

only

the

addition

of

information

about

a

set

but

the

relaxation

of

constraints

as

well

to

model

a

view

of

some

world

that

is

changing

but

not

always

improving

We

do

not

intend

to

pursue

this

idea

References

-LSB-

-RSB-

Robert

L.

Read

Donald

S.

Fussell

and

Avi

Silberschatz

A

multi-resolution

relational

data

model

In

International

Conference

on

Very

Large

Data

Bases

pages

-LCB-

Vancouver

Canada

August

-LSB-

-RSB-

F.

Bancilhon

and

S.

Khoshafian

Calculus

for

complex

objects

In

PODS

pages

-LCB-

-LSB-

-RSB-

P.

Buneman

S.

Davidson

and

A.

Watters

A

semantics

for

complex

objects

and

approximate

queries

In

Seventh

Symposium

on

the

Principles

of

Database

Systems

pages

-LCB-

March

-LSB-

-RSB-

J.

M.

Morrissey

Imprecise

information

and

uncertainty

in

information

systems

ACM

Transactions

on

Information

Systems

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

D.

Maier

The

Theory

of

Relational

Databases

Computer

Science

Press

Taft

Court

Rockville

Maryland

Chapter

-LSB-

-RSB-

S.

V.

Vrbsky

and

J.

W.

S.

Liu

An

object-oriented

query

processor

that

returns

monotonically

improving

answers

In

Proceedings

of

the

IEEE

Conf

on

Data

Engineering

Kobe

Japan

April

-LSB-

-RSB-

K.

Liu

and

R.

Sunderraman

On

representing

indefinite

and

maybe

information

in

relational

databases

In

Proceedings

The

Fourth

International

Conference

on

Data

Engineering

pages

-LCB-

February

-LSB-

-RSB-

T.

Imielinski

and

W.

Lipski

On

representing

incomplete

information

in

a

relational

database

In

International

Conference

on

Very

Large

Data

Bases

pages

-LCB-

Cannes

France

September

-LSB-

-RSB-

T.

Imielinski

and

W.

Lipski

Incomplete

information

in

relational

databases

Journal

of

the

ACM

-LRB-

-RRB-

-LCB-

October

-LSB-

-RSB-

R.

Reiter

A

sound

and

sometimes

complete

query

evaluation

algorithm

for

relational

databases

with

values

Journal

of

the

ACM

-LRB-

-RRB-

-LCB-

April

-LSB-

-RSB-

J.

Grant

Partial

values

in

a

tabular

database

model

Information

Processing

Letters

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

A.

Ohori

Denotational

semantics

of

relational

databases

Technical

Report

Department

of

Computer

and

Information

Science

University

of

Pennsylvania

School

of

Engineering

and

Applied

Science

University

of

Pennsylvania

Philadelphia

PA

-LSB-

-RSB-

L.

G.

DeMichiel

Resolving

database

incompatibility

An

approach

to

performing

relational

operations

over

mismatched

domains

Transactions

on

Knowledge

and

Data

Engineering

-LRB-

-RRB-

-LCB-

December

-LSB-

-RSB-

L.

G.

DeMichiel

Performing

database

operations

over

mismatched

domains

PhD

thesis

Stanford

University

Stanford

CA

-LSB-

-RSB-

P.

Buneman

A.

Jung

and

A.

Ohori

Using

powerdomains

to

generalize

relational

databases

Theoretical

Computer

Science

-LCB-

December

-LSB-

-RSB-

J.

E.

Stoy

Denotational

Semantics

The

ScottStrachey

Approach

to

Programming

Language

Theory

MIT

Press

Cambridge

Massachusetts

-LSB-

-RSB-

Gregory

K.

Wallace

The

JPEG

still

picture

compression

standard

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

April

-LSB-

-RSB-

Didier

Le

Gall

MPEG

A

video

compression

standard

for

multimedia

applications

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

April

-LSB-

-RSB-

G.

David

Ripley

DVI

-LCB-

a

digital

multimedia

technology

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

July

-LSB-

-RSB-

Ming

Liou

Overview

of

the

p

kbit/s

video

coding

standard

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

April

-LSB-

-RSB-

David

Gries

The

Science

of

Programming

SpringerVerlag

-LSB-

-RSB-

L.

DeMichiel

Performing

operations

over

mismatched

domains

In

Knowledge

Engineering

Conference

Proceedings

pages

-LCB-

-LSB-

-RSB-

Alexander

Schrijver

Theory

of

Integer

and

Linear

Programming

John

Wiley

Sons

Ltd.

WileyInterscince

Series

in

Discrete

Mathematics

-LSB-

-RSB-

R.

Tarjan

Depth-first

search

and

linear

graph

algorithms

SIAM

Journal

of

Computing

-LRB-

-RRB-

-LCB-

June

-LSB-

-RSB-

J.

E.

Hopcroft

and

R.

Tarjan

Planarity

testing

in

V

log

V

steps

Technical

Report

Stanford

CA

March

-LSB-

-RSB-

K.

Paton

An

algorithm

for

the

blocks

and

cutnodes

of

a

graph

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

July

-LSB-

-RSB-

T.

Imielinski

and

K.

Vadaparty

Complexity

of

query

processing

with

OR-objects

In

Principles

of

Database

Systems

pages

-LCB-

-LSB-

-RSB-

S.

Davidson

and

A.

Watters

Partial

computation

in

real

time

databases

In

Fifth

Workshop

on

Real

Time

Operating

Systems

May

-LSB-

-RSB-

Liu

and

Sunderraman

A

generalized

relational

model

for

indefinite

and

maybe

information

IEEE

Transactions

on

Knowledge

and

Data

Engineering

-LRB-

-RRB-

March

-LSB-

-RSB-

Vilem

Novak

Fuzzy

Sets

and

their

Applications

Bristol

Hilger

-LSB-

-RSB-

Zdislaw

Pawlak

Rough

Sets

Theoretical

Aspects

of

Reasoning

and

Data

Kluwer

Academic

Publishers

-LSB-

-RSB-

Franco

P.

Preparata

and

Michael

Ian

Shamos

Computational

Geometry

Springer-Verlag

New

York

Inc.

Fifth

Avenue

New

York

New

York

USA

-LSB-

-RSB-

Bjorn

N.

Freeman-Benson

John

Maloney

and

Alan

Borning

An

incremental

constraint

solver

Communications

of

the

ACM

-LRB-

-RRB-

-LCB-

January

-LSB-

-RSB-

P.

Buneman

S.

B.

Davidson

and

A.

Watters

Querying

independent

databases

Information

Sciences

-LRB-

-RRB-

-LCB-

August

-LSB-

-RSB-

Richard

J.

Lipton

and

Jeffrey

E.

Naughton

Practical

selectivity

estimation

through

adaptive

sampling

In

SIGMOD

Conference

Proceedings

pages

-LCB-

-LSB-

-RSB-

Wen-Chi

Hou

Gultekin

Ozsoyoglu

and

Baldeo

K.

Taneja

Processing

aggregate

relational

queries

with

hard

time

constraints

In

SIGMOD

Conference

Proceedings

pages

-LCB-

-LSB-

-RSB-

M.

Muralikrishna

and

David

J.

DeWitt

Equi-depth

historgrams

for

estimating

selectivity

factor

for

multidimensional

queries

In

ACM

SIGMOD

Conference

Proceedings

pages

-LCB-

A

Proofs

that

mina

and

maxa

are

NP-hard

Unfortunately

the

calculation

of

the

optimal

values

of

the

functions

mina

-LRB-

gen

F

-RRB-

and

maxa

-LRB-

gen

F

-RRB-

is

NP-hard

for

an

arbitrary

partially

ordered

domain

and

fact

set

F

over

that

domain

The

reductions

for

both

mina

and

maxa

use

slightly

different

constructions

of

a

partially

ordered

domains

from

a

graph

in

order

to

show

that

the

calculation

of

mina

and

maxa

solve

NP-hard

graph

problems

hence

are

NP-hard

Definition

We

construct

a

partially

ordered

domain

hE

v

E

i

called

the

edge

domain

from

an

arbitrary

undirected

graph

G

hV

Ei

Let

E

be

the

set

V

-LSB-

E

-LSB-

f?g

Define

v

E

by

x

v

E

y

if

only

if

x

is

x

y

or

x

corresponds

to

a

node

and

y

corresponds

to

an

edge

touching

x.

Note

that

v

E

is

indeed

a

partial

order

and

that

it

has

two

levels

above

bottom

-LRB-

-RRB-

The

elements

of

the

first

level

correspond

to

the

nodes

of

G

and

the

second

higher

and

maximal

level

correspond

to

the

edges

of

G

The

total

elements

of

E

correspond

to

edges

of

G.

Theorem

maxa

is

NP-hard

Proof

maxa

reduces

to

determining

if

an

undirected

graph

has

a

Hamiltonian

Circuit

Consider

an

undirected

graph

G

with

N

nodes

Construct

the

edge

domain

E

from

G

as

given

by

the

Definition

Construct

a

fact

set

F

on

E

that

is

the

set

of

revelations

such

that

he

F

e

corresponds

to

an

edge

in

G

hd

F

d

corresponds

to

an

node

in

G

Such

a

set

can

clearly

be

constructed

in

polynomial

time

from

a

representation

of

G.

Note

also

that

this

fact

set

is

externally

consistent

since

is

consistent

with

it

We

show

that

maxa

-LRB-

gen

F

-RRB-

N

if

and

only

if

the

graph

G

has

a

Hamiltonian

Circuit

Lemma

If

G

has

a

Hamiltonian

circuit

then

maxa

-LRB-

gen

F

-RRB-

N

Proof

If

G

has

a

Hamiltonian

circuit

then

there

is

a

set

S

of

N

edges

of

G

that

touch

each

node

and

each

node

in

G

is

touched

by

exactly

two

of

those

edges

Such

a

set

S

is

consistent

with

gen

F

because

each

constraint

in

F

is

satisfied

Each

fact

in

F

corresponding

to

an

edge

is

consistent

since

no

edge

is

repeated

in

S

-LRB-

since

S

is

a

set

-RRB-

and

each

fact

corresponding

to

a

node

approximates

exactly

two

elements

of

S

-LRB-

those

corresponding

to

the

two

edges

incident

on

that

node

-RRB-

Therefore

maxa

-LRB-

gen

F

-RRB-

N

since

maxa

-LRB-

gen

F

-RRB-

:d

is

defined

to

be

the

max

number

above

d

in

any

consistent

set

and

everything

is

above

Further

maxa

B

N

because

any

set

corresponding

to

more

than

N

edges

touches

some

node

from

a

set

of

N

more

than

once

by

the

pigeon-hole

principle

The

fact

set

constraints

preclude

a

node

n

touching

more

than

edges

because

hn

is

a

member

of

F

Therefore

maxa

F

N

Thus

if

G

has

a

Hamiltonian

circuit

then

maxa

-LRB-

gen

F

-RRB-

N

Lemma

If

maxa

-LRB-

gen

F

-RRB-

N

then

G

has

a

Hamiltonian

circuit

Proof

If

S

is

a

set

consistent

with

gen

F

with

N

elements

each

node

of

G

is

touched

by

at

most

two

edges

of

S

since

S

is

consistent

with

gen

F

and

F

contains

the

revelation

hn

for

each

node

n

Since

there

are

N

edges

and

no

node

is

touched

by

more

than

two

edges

every

node

must

be

touched

since

S

must

touch

N

nodes

and

G

only

contains

N

nodes

S

thus

corresponds

to

a

Hamiltonian

circuit

of

G

since

it

touches

every

node

and

no

node

is

touched

by

more

than

two

edges

Thus

if

maxa

-LRB-

gen

F

-RRB-

is

N

then

G

contains

contains

a

Hamiltonian

circuit

Thus

maxa

reduces

to

the

Hamiltonian

circuit

problem

and

is

NP-hard

-LRB-

End

of

Proof

-RRB-

Theorem

mina

is

NP-hard

Proof

Definition

We

construct

a

partially

ordered

domain

hN

vN

i

called

the

node

domain

from

an

arbitrary

undirected

graph

It

differs

from

the

edge

domain

in

that

the

maximal

elements

are

nodes

rather

than

edges

of

G

hV

Ei

Let

N

be

the

set

V

-LSB-

E

-LSB-

f?g

Define

vN

by

x

vN

y

if

only

if

x

is

x

y

or

x

corresponds

to

an

edge

and

y

corresponds

to

a

node

touched

by

x.

Note

that

vN

is

indeed

a

partial

order

and

that

it

has

two

levels

above

bottom

-LRB-

-RRB-

The

elements

of

the

first

level

correspond

to

the

edges

of

G

and

the

second

higher

and

maximal

level

correspond

to

the

nodes

of

G

The

total

elements

of

N

correspond

to

edges

of

G.

The

problem

of

calculating

mina

reduces

to

determining

if

an

undirected

graph

has

a

vertex

cover

of

size

k.

Consider

an

undirected

graph

G

hV

Ei

with

N

nodes

Construct

the

node

domain

N

from

G

as

given

by

the

Definition

Construct

a

fact

set

F

by

hd

d

is

a

node

in

G

he

e

is

an

edge

in

G

Note

that

this

fact

set

is

externally

consistent

since

V

is

consistent

with

it

and

it

can

be

constructed

in

polynomial

time

from

a

graph

We

will

show

that

the

graph

G

has

a

vertex

k-cover

if

and

only

if

mina

-LRB-

gen

F

-RRB-

is

less

than

or

equal

to

k.

Lemma

If

mina

-LRB-

gen

F

-RRB-

k

then

G

has

a

vertex

cover

of

size

k.

Proof

If

mina

-LRB-

gen

F

-RRB-

k

then

there

exists

a

set

S

corresponding

to

at

most

k

vertices

of

G

that

is

consistent

with

gen

F

by

definition

S

is

a

vertex

cover

of

G

since

to

be

consistent

with

F

at

least

one

element

of

S

is

above

-LRB-

in

the

partial

ordering

of

N

-RRB-

each

element

of

N

that

corresponds

to

an

edge

of

G.

Nodes

are

above

edges

incident

upon

them

in

vN

therefore

any

S

consistent

with

F

is

a

cover

of

every

edge

of

G

If

i

j

and

a

graph

has

an

i-cover

then

it

has

a

j-cover

Thus

if

mina

B

k

then

G

has

a

k-cover

Lemma

If

G

as

a

vertex

cover

of

size

k

then

mina

-LRB-

gen

F

-RRB-

k.

Proof

If

G

has

a

k-cover

S

then

S

is

a

set

of

k

nodes

touched

by

every

edge

of

G.

By

construction

of

F

such

a

set

is

consistent

with

every

fact

of

F

Since

mina

-LRB-

SBF

F

-RRB-

is

defined

as

a

minimum

of

numa

over

sets

consistent

with

F

the

existence

of

a

set

S

with

k

members

consistent

with

F

implies

that

mina

-LRB-

gen

F

-RRB-

is

at

most

k

since

numa

Q

jQj

for

any

set

Q.

Thus

if

G

has

a

k-cover

then

mina

-LRB-

gen

F

-RRB-

is

at

most

k.

Therefore

G

has

a

vertex

k-cover

if

and

only

if

mina

F

k

Since

the

vertex

cover

problem

is

NP

complete

mina

is

NP-hard

-LRB-

End

of

Proof

-RRB-

B

Some

Auxiliary

Algorithms

This

Appendix

defines

some

of

the

basic

graph

manipulation

algorithms

used

without

definition

in

the

body

of

this

tech

report

These

codes

and

the

proofs

sketched

here

essentially

prove

Theorem

Most

of

these

algorithms

utilize

depth-first

search

These

searches

usually

begin

at

the

node

in

the

SDAG

whose

data

field

is

which

is

the

universal

and

primitive

ancestor

of

all

other

nodes

We

denote

this

node

r

Algorithm

The

algorithm

for

determining

if

a

node

with

a

given

data

field

is

in

the

graph

func

-LRB-

G

G

d

D

v

R

-RRB-

R

var

s

R

K

if

v

data

d

r

v

-LSB-

-RSB-

v

data

d

K

v

kids

r

do

K

s

choose

K

K

K

fsg

if

s

data

v

d

K

r

-LRB-

G

d

s

-RRB-

od

return

r

Algorithm

The

function

-LRB-

G

d

-RRB-

just

calls

-LRB-

G

d

r

-RRB-

func

-LRB-

G

G

d

D

-RRB-

R

return

-LRB-

G

d

r

-RRB-

Algorithm

The

-LRB-

G

d

v

-RRB-

algorithm

is

a

simple

modification

of

depth

first

search

on

the

SDAG

func

-LRB-

G

G

d

D

v

R

I

-RRB-

var

f

if

v

flag

v

flag

f

for

-LRB-

k

k

v

kids

-RRB-

if

k

data

vd

f

if

k

flag

I

-LRB-

G

d

v

I

-RRB-

rof

if

f

I

I

-LSB-

fvg

return

I

Algorithm

-LRB-

G

d

-RRB-

just

calls

-LRB-

G

d

r

;-RRB-

func

-LRB-

G

G

d

D

-RRB-

return

-LRB-

G

d

r

;-RRB-

Lemma

The

function

computes

the

set

of

nodes

in

the

SDAG

that

have

data

fields

less

than

d

in

v

and

for

which

no

greater

nodes

in

the

v

partial

ordering

are

less

than

d.

For

an

SDAG

G

constructed

from

a

fact

set

F

-LRB-

G

d

r

-RRB-

fa

j

a

d

lookup

-LRB-

G

a

-RRB-

-LRB-

c

lookup

-LRB-

F

c

-RRB-

a

c

c

d

-RRB-

g

Proof

Sketch

The

function

-LRB-

G

d

r

;-RRB-

performs

a

depth-first

search

of

the

SDAG

starting

at

bottom

that

includes

every

node

v

v

d

If

v

has

no

kids

which

are

less

than

d

then

v

is

an

immediate

ancestor

of

the

partial

data

element

d

and

placed

in

the

set

I

Since

we

start

at

bottom

and

search

all

nodes

using

this

condition

I

contains

all

the

immediate

ancestors

of

d

when

we

are

done

Algorithm

The

algorithm

is

a

simple

modification

of

depth

first

search

on

the

SDAG

It

collects

all

ancestors

of

d

that

are

pin

elements

func

-LRB-

G

G

d

D

v

R

I

-RRB-

if

v

flag

v

flag

ifv

pnset

U

I

I

-LSB-

fvg

-LSB-

-RSB-

v

pnset

U

for

-LRB-

k

k

v

kids

-RRB-

if

k

data

v

d

I

-LRB-

G

d

k

I

-RRB-

rof

return

I

Algorithm

-LRB-

G

d

-RRB-

just

calls

-LRB-

G

d

r

;-RRB-

func

-LRB-

d

D

-RRB-

return

-LRB-

G

d

r

;-RRB-

Definition

The

function

mincocadd

-LRB-

A

r

-RRB-

returns

a

set

of

records

that

form

a

v

minimal

cochain

that

approximates

every

record

of

A

-LSB-

r.

mincocadd

-LRB-

A

r

-RRB-

fx

j

x

A

-LSB-

frg

-LRB-

y

y

A

-LSB-

frg

y

data

v

x

data

-RRB-

g

Repeated

calls

to

this

function

in

the

algorithm

below

are

used

to

maintain

a

cochain

of

the

lowest-resolution

elements

that

are

guaranteed

to

approximate

particular

data

elements

Algorithm

The

function

-LRB-

G

d

v

I

-RRB-

is

a

modification

of

depth

first

search

as

well

func

-LRB-

G

G

d

D

v

R

I

-RRB-

if

v

flag

v

flag

if

no-ub

-LRB-

v

;d

-RRB-

if

dv

v

I

mincocadd

-LRB-

I

v

-RRB-

-LSB-

-RSB-

else

for

-LRB-

k

k

v

imdec

-RRB-

if

k

flag

I

-LRB-

G

d

k

I

-RRB-

rof

return

I

Algorithm

The

function

-LRB-

G

d

-RRB-

just

calls

-LRB-

G

d

v

I

-RRB-

with

any

immediate

ancestor

of

d

in

G

func

-LRB-

G

G

d

D

-RRB-

var

x

x

choose

-LRB-

G

;d

-RRB-

return

-LRB-

G

d

x

;-RRB-

Lemma

If

v

imanc

-LRB-

G

;d

-RRB-

then

the

function

-LRB-

G

d

-RRB-

computes

the

set

of

nodes

in

the

SDAG

that

have

data

fields

greater

than

d

in

v

and

for

which

no

lesser

nodes

in

the

v

partial

ordering

are

greater

than

d.

For

an

SDAG

G

constructed

from

a

fact

set

F

-LRB-

G

d

-RRB-

fa

j

d

a

lookup

-LRB-

G

a

-RRB-

-LRB-

c

lookup

-LRB-

G

c

-RRB-

c

a

d

c

-RRB-

g

Proof

Sketch

The

function

-LRB-

G

d

-RRB-

performs

a

depth-first

search

of

the

subgraph

of

G

that

includes

the

nodes

above

the

ancestor

of

d

supplied

as

the

argument

to

the

first

call

If

a

node

has

the

property

no-ub

-LRB-

v

d

-RRB-

it

can

not

have

any

descendant

that

is

a

descendant

of

d.

All

descendants

of

d

are

placed

in

I

or

some

ancestor

of

them

is

placed

in

I

Since

I

is

maintained

as

a

minimal

cochain

and

an

ancestor

of

every

descendant

of

d

is

placed

in

I

upon

completion

I

contains

the

immediate

descendants

of

d.

The

following

algorithm

is

very

straightforward

A

much

more

complicated

and

slightly

more

efficient

algorithm

for

adding

a

cochain

to

a

SDAG

exists

Algorithm

This

is

a

naive

version

of

cocadd

-LRB-

b

P

G

Z

-RRB-

that

is

extensionally

equivalent

to

a

more

complicated

algorithm

that

utilizes

the

fact

that

P

is

a

cochain

proc

cocadd

-LRB-

val

b

R

;P

val-res

G

res

Z

-RRB-

var

r

R

Z

for

-LRB-

c

c

P

-RRB-

lkpormk

-LRB-

G

c

r

-RRB-

Z

Z

-LSB-

frg

rof

The

following

theorem

about

cocadd

is

useful

Theorem

INV

-LRB-

F

G

-RRB-

cochain

:P

-LRB-

a

a

P

b

va

-RRB-

-RRB-

wp

-LRB-

cocadd

-LRB-

b

P

G

Z

-RRB-

Z

map

-LRB-

data

P

-RRB-

INV

-LRB-

F

G

-RRB-

-RRB-

databases
imprecise computation
incomplete information
multiresolution

incomplete data model
partial data model
partial information
progressive refinement