In

Journal

of

Artificial

Neural

Networks

vol

no.

pp.

Priority

ASOCS

Tony

R.

Martinez

Douglas

M.

Campbell

Brent

W.

Hughes

Computer

Science

Department

Brigham

Young

University

Provo

Utah

Abstract

This

paper

presents

an

ASOCS

-LRB-

Adaptive

Self-Organizing

Concurrent

System

-RRB-

model

for

massively

parallel

processing

of

incrementally

defined

rule

systems

in

such

areas

as

adaptive

logic

robotics

logical

inference

and

dynamic

control

An

ASOCS

is

an

adaptive

network

composed

of

many

simple

computing

elements

operating

asynchronously

and

in

parallel

An

ASOCS

can

operate

in

either

a

data

processing

mode

or

a

learning

mode

During

data

processing

mode

an

ASOCS

acts

as

a

parallel

hardware

circuit

During

learning

mode

an

ASOCS

incorporates

a

rule

expressed

as

a

Boolean

conjunction

in

a

distributed

fashion

in

time

logarithmic

in

the

number

of

rules

This

paper

proposes

a

learning

algorithm

and

architecture

for

Priority

ASOCS

This

new

ASOCS

model

uses

rules

with

priorities

The

new

model

has

significant

learning

time

and

space

complexity

improvements

over

previous

models

Introduction

Non-von

Neumann

architectures

such

as

neural

networks

attack

the

word-at-a-time

bottleneck

of

traditional

computing

systems

-LSB-

-RSB-

Neural

networks

learn

input-output

mappings

using

highly

distributed

processing

and

memory

-LSB-

-RSB-

Their

numerous

simple

processing

elements

with

modifiable

weighted

links

permit

a

high

degree

of

parallelism

A

typical

neural

network

has

fixed

topology

It

learns

by

modifying

weighted

links

between

nodes

A

new

class

of

connectionist

architectures

has

been

proposed

called

ASOCS

-LRB-

Adaptive

SelfOrganizing

Concurrent

Systems

-RRB-

-LSB-

-RSB-

ASOCS

models

support

efficient

computation

through

self-organized

learning

and

parallel

execution

Learning

is

done

through

the

incremental

presentation

of

rules

and/or

examples

ASOCS

models

learn

by

modifying

their

topology

Data

types

include

Boolean

and

multi-state

variables

recent

models

support

analog

variables

The

model

incorporates

rules

into

an

adaptive

logic

network

in

a

parallel

and

self

organizing

fashion

In

processing

mode

ASOCS

supports

fully

parallel

execution

on

actual

inputs

according

to

the

learned

rules

The

adaptive

logic

network

acts

as

a

parallel

hardware

circuit

during

execution

mapping

n

input

boolean

vectors

into

m

output

boolean

vectors

in

a

combinatoric

fashion

The

overall

philosophy

of

ASOCS

follows

the

high

level

goals

of

current

neural

network

models

However

the

mechanisms

of

learning

and

execution

vary

significantly

The

ASOCS

logic

network

is

topologically

dynamic

with

the

network

growing

to

efficiently

fit

the

specific

application

Current

ASOCS

models

are

based

on

digital

nodes

ASOCS

also

supports

use

of

symbolic

and

heuristic

learning

mechanisms

thus

combining

the

parallelism

and

distributed

nature

of

connectionist

computing

with

the

potential

power

of

AI

symbolic

learning

A

proof

of

concept

ASOCS

chip

has

been

developed

-LSB-

-RSB-

ASOCS

models

have

two

significant

advantages

over

other

learning

models

a

-RRB-

they

are

guaranteed

to

learn

any

arbitrary

set

of

legal

rules

b

-RRB-

learning

time

is

both

fast

and

bounded

-LRB-

linear

with

the

depth

of

the

network

and

logarithmic

with

the

number

of

nodes

-RRB-

Application

targets

for

ASOCS

include

adaptive

logic

devices

logical

inference

robotics

embedded

systems

and

pattern

recognition

A

number

of

formal

ASOCS

models

have

been

proposed

to

date

-LSB-

-RSB-

This

paper

presents

a

new

model

of

ASOCS

called

Priority

ASOCS

-LSB-

-RSB-

The

Prioritized

ASOCS

architecture

provides

a

number

of

advantages

over

other

ASOCS

architectures

The

number

of

nodes

required

never

exceeds

the

number

of

rules

presented

to

the

system

The

allocation

of

nodes

is

simpler

and

more

parsimonious

Though

not

using

a

specific

generalization

method

in

the

basic

model

PASOCS

allows

for

the

efficient

introduction

of

flexible

and

arbitrary

generalization

techniques

Section

presents

a

simplified

high

level

view

of

PASOCS

to

give

the

reader

a

basic

understanding

of

the

model

Section

gives

essential

definitions

Section

describes

the

learning

algorithm

in

detail

Section

proposes

a

specific

PASOCS

architecture

to

support

the

learning

algorithm

Section

presents

a

detailed

example

of

PASOCS

learning

Section

demonstrates

the

simple

extension

to

allow

multiple

outputs

Section

concludes

the

paper

While

this

paper

presents

a

self-contained

detailed

discussion

of

PASOCS

execution

and

learning

a

more

rigorous

approach

with

proofs

of

all

learning

rules

may

be

found

in

-LSB-

-RSB-

High

Level

Overview

of

Prioritized

ASOCS

To

give

the

reader

the

overall

flavor

of

PASOCS

processing

we

informally

describe

a

Prioritized

ASOCS

architecture

In

sections

through

we

give

the

technical

detail

of

how

the

learning

and

processing

are

accomplished

in

a

parallel

and

self-organizing

fashion

A

Priority

ASOCS

is

an

ASOCS

model

which

has

the

following

attributes

PASOCS

processes

inputs

in

the

form

of

Boolean

variables

and

outputs

Boolean

results

accepts

rules

made

up

of

a

conjunction

of

boolean

inputs

which

imply

a

boolean

output

learns

new

rules

over

time

automatically

resolves

rule

conflicts

combines

specific

rules

into

more

general

rules

where

appropriate

maintains

an

associated

priority

with

each

rule

We

begin

with

the

black

box

of

figure

having

four

input

variables

-LCB-

a

b

c

d

-RCB-

and

the

single

output

variable

z

In

general

an

ASOCS

maps

n

input

variables

into

m

output

variables

where

the

size

of

m

and

n

do

not

need

to

be

known

a

priori

The

system

allocates

input

and

output

lines

as

new

variables

are

introduced

in

rules

a

b

c

d

z

PASOCS

Network

Figure

PASOCS

black

box

Nodes

and

Rules

The

black

box

of

figure

contains

nodes

connected

by

a

binary

tree

structure

Each

node

represents

a

rule

A

rule

is

represented

as

a

The

first

element

is

a

conjunction

of

boolean

variables

-LRB-

the

left

hand

side

of

the

rule

-RRB-

the

second

element

is

the

rule

output

-LRB-

or

-RRB-

and

the

third

element

is

the

priority

of

the

rule

The

priority

is

a

positive

integer

value

with

being

the

lowest

priority

The

rule

of

node

in

figure

has

priority

It

says

to

output

-LRB-

z

-RRB-

whenever

-LRB-

a

-RRB-

and

-LRB-

b

-RRB-

and

-LRB-

c

-RRB-

Node

is

currently

empty

-LRB-

abc

-RRB-

-LRB-

abd

-RRB-

-LRB-

a

bc

'd

-RRB-

a

b

c

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

Figure

Internal

node

representations

Execution

When

the

black

box

of

figure

receives

the

input

-LRB-

a

b

c

d

-RRB-

it

broadcasts

these

values

down

the

binary

tree

structure

to

each

node

Each

node

independently

and

in

parallel

compares

itself

to

the

input

If

the

input

matches

the

rule

then

the

node

passes

up

the

binary

tree

a

message

consisting

of

the

rule

output

and

the

rule

priority

If

the

node

does

not

match

no

message

is

sent

When

two

messages

arrive

at

a

junction

in

the

tree

the

one

with

higher

priority

-LRB-

larger

integer

-RRB-

is

passed

on

while

the

other

is

discarded

In

the

case

of

figure

and

the

given

input

nodes

and

both

fire

Since

node

has

priority

and

node

has

priority

the

system

outputs

the

result

of

node

which

specifies

an

output

of

-LRB-

z

-RRB-

-LRB-

abc

-RRB-

-LRB-

abd

-RRB-

-LRB-

a

bc

'd

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

Figure

PASOCS

during

input

processing

Learning

When

a

new

rule

is

presented

to

the

system

it

is

broadcast

down

the

tree

to

the

nodes

Each

node

compares

itself

with

the

rule

and

depending

on

how

it

matches

the

node

initiates

an

appropriate

operation

The

matching

and

operations

occur

independently

and

in

parallel

These

operations

make

up

the

learning

algorithm

We

present

some

examples

to

give

a

flavor

for

the

basic

learning

algorithm

In

the

following

assume

we

wish

to

add

the

rule

bd

z

to

the

black

box

of

figure

The

following

actions

would

take

place

Though

we

present

them

sequentially

they

would

take

place

in

parallel

in

an

actual

system

Node

Addition

and

Consistency

Resolution

To

resolve

inconsistencies

the

system

need

only

give

the

new

rule

a

priority

one

greater

than

any

rule

it

contradicts

The

rule

bd

z

is

broadcast

and

all

nodes

which

contradict

the

new

rule

-LRB-

i.e.

nodes

which

could

be

simultaneously

matched

for

some

input

but

would

cause

the

opposite

output

-RRB-

send

their

priority

up

the

tree

The

highest

priority

arrives

at

the

top

and

the

new

rule

is

added

to

the

network

with

priority

one

higher

than

the

highest

inconsistent

priority

In

the

case

of

figure

only

node

-LRB-

abc

z

-RRB-

is

inconsistent

with

bd

z

and

it

has

priority

The

system

therefore

adds

bd

z

with

priority

as

shown

in

figure

-LRB-

abc

-RRB-

-LRB-

abd

-RRB-

-LRB-

a

bc

'd

-RRB-

a

b

c

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

-LRB-

bd

-RRB-

Figure

Adding

a

new

rule

Generalization

The

new

rule

bd

z

has

made

node

redundant

since

node

is

matched

in

all

cases

where

node

is

matched

As

part

of

the

learning

algorithm

the

redundant

node

deletes

itself

as

shown

in

figure

-LRB-

abc

-RRB-

-LRB-

a

bc

'd

-RRB-

a

b

c

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

-LRB-

bd

-RRB-

Figure

After

self

deletion

The

new

rule

bd

z

makes

it

possible

for

node

of

figure

to

generalize

itself

by

removing

the

variable

d

As

part

of

the

learning

algorithm

node

minimizes

itself

as

shown

in

figure

-LRB-

abc

-RRB-

-LRB-

a

bc

-RRB-

a

b

c

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

-LRB-

bd

-RRB-

Figure

After

further

generalization

This

example

gives

a

quick

view

of

the

style

of

learning

mechanisms

available

in

a

PASOCS

system

The

body

of

the

paper

presents

the

full

learning

model

and

a

potential

architecture

We

now

conclude

the

list

of

PASOCS

attributes

begun

earlier

In

Prioritized

ASOCS

-LRB-

PASOCS

-RRB-

whenever

a

new

rule

is

learned

at

most

one

node

need

be

added

to

the

system

PASOCS

is

guaranteed

to

learn

any

set

of

valid

rules

PASOCS

need

only

see

a

rule

once

when

learning

a

rule

set

PASOCS

does

not

unlearn

old

rules

when

learning

new

rules

except

in

the

case

of

conflict

resolution

PASOCS

can

support

multiple

output

variables

-LRB-

See

section

-RRB-

Due

to

the

localist

node-per-rule

nature

of

PASOCS

each

node

is

independent

of

every

other

node

Thus

any

network

topology

can

be

chosen

to

best

fulfill

such

needs

as

cost

speed

and

fault

tolerance

The

particular

topological

instantiation

proposed

above

is

that

of

a

binary

tree

Due

to

this

decision

we

can

state

that

the

PASOCS

processes

inputs

in

O

-LRB-

log

-LRB-

n

-RRB-

-RRB-

time

where

n

is

the

number

of

nodes

-LRB-

rules

-RRB-

in

the

network

learns

individual

rules

in

O

-LRB-

log

-LRB-

n

-RRB-

-RRB-

time

where

n

is

the

number

of

nodes

-LRB-

rules

-RRB-

in

the

network

The

PASOCS

model

presented

above

generalizes

in

the

sense

of

simplifying

rules

However

no

mechanism

is

defined

for

what

should

be

output

when

the

input

environment

matches

no

learned

rule

Other

ASOCS

models

-LSB-

-RSB-

use

a

distributed

mechanism

to

accomplish

automatic

generalization

for

novel

inputs

Generalization

in

a

learning

system

is

at

best

a

always

a

guess

The

proper

style

of

generalization

is

application

dependent

Thus

one

style

of

generalization

is

not

sufficient

to

allow

a

learning

system

to

support

a

broad

spectrum

of

applications

Current

connectionist

models

including

earlier

ASOCS

models

use

one

specific

style

of

generalization

which

is

implicit

in

the

learning

algorithm

We

suggest

that

the

type

of

generalization

used

be

a

self-organizing

parameter

of

the

learning

system

which

can

be

discovered

as

learning

takes

place

This

requires

a

-RRB-

a

model

which

allows

flexible

generalization

styles

and

b

-RRB-

mechanisms

to

guide

the

system

into

the

best

style

of

generalization

for

the

problem

being

learned

In

PASOCS

the

style

of

generalization

can

be

both

arbitrary

and

potentially

dynamic

Generalization

techniques

being

studied

with

the

PASOCS

model

include

hamming

distance

percentage

mismatch

weighted

hamming

subsets

of

critical

variables

confidence

levels

-LRB-

allowing

both

noisy

and

training

set

style

learning

-RRB-

and

structural

prototype

generalization

Schemes

which

guide

the

automatic

discovery

of

the

best

combination

of

generalization

mechanisms

are

also

part

of

current

ASOCS

research

Discussion

of

these

particular

concepts

is

beyond

the

scope

of

this

paper

Definitions

Propositional

Rules

The

atomic

knowledge

element

in

ASOCS

is

the

instance

-LRB-

rule

-RRB-

Each

instance

is

a

-LRB-

partial

-RRB-

function

from

a

set

of

Boolean

variables

to

a

Boolean

variable

An

instance

consists

of

a

left

hand

side

and

a

right

hand

side

Following

are

four

examples

of

instances

I.

a

b

z

II

a

b

c

a

III

a

b

z

IV

c

d

e

z

Instance

Variables

The

variables

that

appear

in

the

left

hand

side

of

an

instance

are

called

instance

input

variables

The

variable

that

appears

in

the

right

hand

side

of

an

instance

is

called

the

instance

output

variable

Semantics

of

an

Instance

Instance

I

forces

z

to

become

true

whenever

a

and

b

are

false

It

says

nothing

about

what

z

should

be

when

a

or

b

are

true

Instance

II

forces

a

to

become

false

whenever

a

and

c

are

true

and

b

is

false

Instance

III

forces

z

to

become

false

whenever

a

and

b

are

false

Matching

and

Firing

A

set

of

Boolean

values

J

matches

-LRB-

fires

-RRB-

an

instance

I

if

and

only

if

for

every

literal

v

in

the

left

hand

side

of

I

if

v

is

uncomplemented

then

the

variable

v

has

the

value

true

-LRB-

-RRB-

in

J

else

if

v

is

complemented

then

the

variable

v

has

the

value

false

-LRB-

-RRB-

in

J.

For

example

the

input

a

b

c

d

and

e

matches

instances

I

III

and

IV

above

but

does

not

match

instance

II

Polarity

An

instance

with

a

non-complemented

output

variable

is

said

to

have

positive

polarity

an

instance

with

a

complemented

output

variable

is

said

to

have

negative

polarity

Instances

I

and

IV

have

positive

polarity

instances

II

and

III

have

negative

polarity

Two

instances

with

the

same

polarity

are

concordant

with

respect

to

each

other

Two

instances

with

opposite

polarity

are

discordant

Inconsistent

An

instance

and

an

instance

are

inconsistent

if

and

only

if

-RRB-

and

are

discordant

and

-RRB-

there

exists

a

set

of

Boolean

values

J

that

fires

both

and

Instances

I

and

III

are

inconsistent

-RRB-

they

are

discordant

and

-RRB-

when

a

and

b

are

false

instance

I

sets

c

to

true

while

instance

III

sets

c

to

false

Discriminant

variable

Let

and

be

two

instances

let

v

be

a

Boolean

variable

If

v

is

an

input

variable

of

and

v

is

an

input

variable

of

-LRB-

or

vice

versa

-RRB-

then

v

is

said

to

be

a

discriminant

variable

for

and

Instances

I

and

II

have

a

as

a

discriminant

variable

Neither

b

nor

c

is

a

discriminant

variable

for

instances

I

and

II

The

following

theorem

gives

a

simple

computational

test

for

inconsistency

in

terms

of

polarity

and

discriminant

variables

Theorem

Two

instances

are

inconsistent

if

and

only

if

they

are

of

opposite

polarity

-LRB-

discordant

-RRB-

and

have

no

discriminant

variable

Prioritized

Instances

A

prioritized

instance

is

an

instance

with

an

associated

positive

integer

called

its

priority

Instance

Set

A

set

S

of

prioritized

instances

is

called

an

instance

set

Any

subset

of

an

instance

set

is

also

an

instance

set

The

sets

S

-LRB-

J

-RRB-

S

-LRB-

J

-RRB-

and

S

-LRB-

J

-RRB-

Let

S

be

an

instance

set

Let

J

be

a

set

of

Boolean

values

Let

S

-LRB-

J

-RRB-

be

the

set

of

instances

in

S

that

J

fires

Let

S

-LRB-

J

-RRB-

be

the

set

of

instances

in

S

-LRB-

J

-RRB-

with

positive

polarity

Let

S

-LRB-

J

-RRB-

be

the

set

of

instances

in

S

-LRB-

J

-RRB-

with

negative

polarity

The

MaxPriority

function

Let

S

be

an

instance

set

Define

MaxPriority

-LRB-

S

-RRB-

as

follows

if

S

is

non-empty

then

return

the

highest

priority

of

any

instance

in

S

else

if

S

is

empty

then

return

If

S

-LRB-

J

-RRB-

is

empty

-LRB-

no

instance

matches

J

-RRB-

then

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

Consistency

of

an

Instance

Set

An

instance

set

S

is

consistent

if

and

only

if

for

all

sets

of

Boolean

values

J

and

for

all

pairs

of

discordant

instances

and

of

S

if

J

fires

and

then

priority

-LRB-

-RRB-

priority

-LRB-

-RRB-

The

following

two

lemmas

relate

consistency

and

the

MaxPriority

function

Lemma

An

instance

set

S

is

consistent

if

and

only

if

for

every

set

of

Boolean

values

J

whenever

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

is

positive

then

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

does

not

equal

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

Lemma

Let

S

be

a

consistent

instance

set

and

J

be

a

set

of

boolean

values

If

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

is

positive

then

all

instances

in

S

that

match

J

and

have

priority

equal

to

MaxPriority

-LRB-

J

-RRB-

have

the

same

polarity

Execution

We

now

introduce

the

-LRB-

partial

-RRB-

function

MAP

-LRB-

S

J

-RRB-

which

returns

the

polarity

of

the

highest

priority

instance

that

is

matched

in

the

instance

set

Let

S

be

a

consistent

instance

set

and

J

be

a

set

of

Boolean

variables

Define

MAP

-LRB-

S

J

-RRB-

as

follows

MAP

-LRB-

S

J

-RRB-

Set

m

to

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

if

m

then

MAP

-LRB-

S

J

-RRB-

is

undefined

else

MAP

-LRB-

S

J

-RRB-

is

the

polarity

of

any

instance

in

S

-LRB-

J

-RRB-

with

priority

m.

Theorem

MAP

-LRB-

S

J

-RRB-

is

a

well

defined

partial

function

Proof

Let

J

be

any

set

of

Boolean

values

and

let

m

be

MaxPriority

-LRB-

S

-LRB-

J

-RRB-

-RRB-

If

m

then

MAP

-LRB-

S

J

-RRB-

is

undefined

If

m

the

set

of

all

instances

that

have

priority

m

is

non-empty

By

lemma

each

such

instance

has

the

same

polarity

Thus

when

m

MAP

-LRB-

S

J

-RRB-

is

well

defined

This

concludes

the

proof

of

the

theorem

Note

that

when

m

-LRB-

MAP

undefined

-RRB-

a

generalization

scheme

such

as

mentioned

in

section

can

be

used

to

give

an

output

Fulfillment

A

network

is

said

to

fulfill

an

instance

set

S

if

and

only

if

for

every

set

of

boolean

inputs

J

the

network

computes

MAP

-LRB-

S

J

-RRB-

When

MAP

-LRB-

S

J

-RRB-

is

undefined

the

network

may

generalize

and

set

the

output

variable

to

true

false

or

do

n't

know

Subsumption

Let

and

be

instances

We

say

that

subsumes

if

the

variables

of

are

a

subset

of

the

variables

of

If

subsumes

then

whenever

fires

must

fire

One-Difference

Subset

Let

and

be

two

instances

that

share

exactly

one

discriminant

variable

v

We

say

is

one-difference

subset

with

if

the

variables

of

-LCB-

v

v

-RCB-

are

a

proper

subset

of

the

variables

of

-LCB-

v

v

-RCB-

For

example

ab

z

is

one-difference

subset

with

ab

c

z.

One-Difference

Equal

Let

and

be

two

instances

that

share

exactly

one

discriminant

variable

v

We

say

is

one-difference

equal

with

if

the

variables

of

-LCB-

v

v

-RCB-

are

the

same

as

the

variables

of

-LCB-

v

v

-RCB-

For

example

ab

z

is

one-difference

equal

with

ab

z.

One-Difference

Superset

Let

and

be

two

instances

that

share

exactly

one

discriminant

variable

v

We

say

is

one-difference

superset

with

if

the

variables

of

-LCB-

v

v

-RCB-

are

a

proper

superset

of

the

variables

of

-LCB-

v

v

-RCB-

For

example

a

bc

z

is

one-difference

superset

with

ab

z.

Clearly

is

one-difference

subset

with

if

and

only

if

is

onedifference

superset

with

Circumstances

determine

which

we

emphasize

Learning

Algorithm

In

this

section

we

introduce

a

specific

learning

algorithm

for

priority

ASOCS

A

more

rigorous

presentation

with

proofs

of

all

learning

rules

is

be

found

in

-LSB-

-RSB-

We

assume

that

a

network

N

has

been

created

that

fulfills

an

instance

set

IS

Suppose

that

a

new

instance

-LRB-

NI

-RRB-

is

presented

to

the

network

The

network

must

reconfigure

itself

to

fulfill

the

new

instance

set

which

is

the

combination

of

the

old

IS

and

the

NI

The

learning

algorithm

defines

how

this

is

done

in

a

self-organizing

fashion

When

a

NI

is

presented

to

the

system

each

OI

independently

and

in

parallel

-LRB-

see

section

-RRB-

compares

itself

with

the

NI

Depending

on

how

it

matches

the

NI

the

OI

may

a

-RRB-

require

no

modification

b

-RRB-

be

deleted

c

-RRB-

be

modified

d

-RRB-

cause

the

NI

to

be

deleted

e

-RRB-

cause

the

NI

to

be

modified

The

basic

actions

which

take

place

in

the

learning

algorithm

are

-RRB-

Resolve

any

inconsistencies

caused

by

the

NI

-LRB-

Section

-RRB-

-RRB-

Generalize

and

simplify

by

a

-RRB-

removing

basic

redundancies

-LRB-

Section

-RRB-

b

-RRB-

removing

one-difference

redundancies

-LRB-

Section

-RRB-

Section

gives

the

actual

learning

algorithm

using

the

mechanisms

presented

in

sections

Resolution

of

New

Instance

Inconsistency

A

NI

may

be

inconsistent

with

one

or

more

OI

's

in

the

IS

One

solution

would

be

to

give

the

NI

a

priority

greater

than

any

priority

in

the

IS

However

to

restore

consistency

it

is

sufficient

to

give

the

NI

a

priority

one

greater

than

the

inconsistent

OI

having

the

highest

priority

For

example

assume

the

new

instance

a

c

z

is

added

to

the

IS

I.

a

b

z

-LRB-

-RRB-

II

b

c

z

-LRB-

-RRB-

-LRB-

The

priority

of

each

instance

is

shown

in

parenthesis

to

the

right

of

the

instance

In

general

NI.P

stands

for

the

priority

of

the

new

instance

and

OI.P

the

priority

of

a

specific

old

instance

-RRB-

The

NI

and

instance

II

are

consistent

since

c

is

a

discriminant

variable

But

instance

I

-LRB-

with

priority

-RRB-

is

inconsistent

with

the

NI

since

there

is

no

discriminant

variable

Therefore

if

the

NI

is

added

with

priority

the

consistency

of

the

instance

set

is

restored

Removing

Basic

Redundancies

This

section

presents

two

mechanisms

for

removing

instances

from

the

instance

set

If

the

OI

is

subsumed

by

the

NI

then

the

OI

can

be

deleted

from

the

IS

Note

that

the

OI

deletion

does

not

depend

on

the

priority

or

polarity

of

the

NI

or

OI

Let

-LRB-

-RRB-

represent

an

instance

priority

that

can

take

on

any

value

We

illustrate

this

mechanism

by

example

Assume

that

new

instance

a

z

is

added

to

the

following

instance

set

I.

a

b

z

-LRB-

-RRB-

II

a

z

-LRB-

-RRB-

III

a

b

c

z

-LRB-

-RRB-

IV

b

z

-LRB-

-RRB-

Since

instances

I

II

and

III

are

subsumed

by

the

NI

mechanism

allows

them

to

be

deleted

from

the

instance

set

If

-RRB-

OI

and

NI

are

concordant

-RRB-

subsumes

the

NI

and

-RRB-

OI.P

NI.P

then

the

NI

can

be

deleted

As

noted

in

section

it

suffices

to

set

the

priority

of

the

NI

to

one

greater

than

the

inconsistent

OI

with

the

highest

priority

Thus

if

a

concordant

old

instance

subsumes

the

NI

and

if

has

priority

greater

than

the

NI

then

already

covers

all

cases

where

the

NI

could

be

matched

The

NI

therefore

adds

no

new

information

to

the

IS

and

can

be

discarded

We

illustrate

mechanism

by

example

Assume

the

new

instance

a

b

c

z

is

added

to

the

instance

set

I.

a

b

z

-LRB-

-RRB-

II

a

c

z

-LRB-

-RRB-

The

NI

would

be

assigned

the

priority

of

since

it

is

inconsistent

with

instance

I

Since

the

NI

is

subsumed

by

the

concordant

instance

II

and

since

instance

II

has

a

greater

priority

mechanism

lets

the

NI

be

deleted

The

new

instance

a?b

c

z

adds

no

new

information

since

instance

II

already

covers

that

case

On

the

other

hand

if

is

a

concordant

OI

which

subsumes

the

NI

and

if

has

priority

lower

than

the

NI

then

there

exists

at

least

one

discordant

inconsistent

instance

-LRB-

having

priority

less

than

the

NI

but

greater

than

-RRB-

to

which

the

NI

is

an

exception

We

illustrate

this

case

by

example

Assume

the

same

new

instance

a

b

c

z

is

added

to

the

instance

set

I.

a

b

z

-LRB-

-RRB-

II

a

c

z

-LRB-

-RRB-

The

NI

would

be

assigned

the

priority

since

it

is

inconsistent

with

instance

I

The

NI

is

subsumed

by

the

concordant

instance

II

But

the

assigned

NI

priority

is

greater

than

that

of

instance

II

Therefore

does

not

apply

and

the

NI

is

added

to

the

instance

set

The

new

instance

abc

z

is

an

exception

to

instance

I

when

a

b

c

Instance

II

still

contains

information

since

it

sets

z

to

true

when

a

b

and

c

a

case

not

covered

by

either

instance

I

or

the

NI

Removing

One-Difference

Redundancies

In

this

section

we

state

and

illustrate

two

types

of

mechanisms

We

first

present

a

b

and

mechanisms

for

simplifying

one

difference

concordant

instances

Then

we

present

and

mechanisms

for

simplifying

one

difference

discordant

instances

Mechanisms

permit

the

removal

of

discriminant

variables

when

the

NI

is

onedifference

concordant

with

an

OI

-RRB-

If

the

NI

and

OI

are

concordant

and

the

OI

is

one-difference

superset

with

respect

to

the

NI

then

the

discriminant

variable

can

be

deleted

from

the

OI

We

illustrate

mechanism

Assume

the

new

instance

a

b

z

is

added

to

the

instance

set

I.

a

b

c

z

-LRB-

-RRB-

Since

I

and

NI

are

concordant

and

the

instance

I

is

one-difference

superset

with

respect

to

the

NI

mechanism

allows

us

to

delete

the

discriminant

variable

b

from

I

The

old

instance

a

b

c

z

becomes

a

c

z.

a

-RRB-

If

the

OI

and

NI

are

concordant

the

OI

is

one-difference

equal

with

respect

to

the

NI

and

OI.P

NI.P

then

the

discriminant

variable

can

be

deleted

from

the

OI

We

illustrate

mechanism

a.

Assume

the

new

instance

a

b

c

z

is

added

to

the

instance

set

I.

a

b

c

z

-LRB-

-RRB-

II

c

d

z

-LRB-

-RRB-

The

NI

is

inconsistent

with

instance

II

and

is

assigned

priority

Since

instance

I

and

the

NI

are

concordant

one-difference

equal

and

the

priority

of

the

NI

is

greater

than

the

priority

of

instance

I

then

by

a

the

discriminant

variable

b

can

be

removed

from

instance

I

The

old

instance

a

b

c

z

becomes

a

c

z.

b

-RRB-

If

the

OI

and

NI

are

concordant

the

OI

is

one-difference

equal

with

respect

to

the

NI

and

OI.P

NI.P

then

-RRB-

the

discriminant

variable

can

be

deleted

from

the

OI

and

-RRB-

the

NI

itself

can

be

deleted

We

illustrate

mechanism

b.

Assume

the

new

instance

a

b

c

z

is

added

to

the

instance

set

I.

a

b

c

z

-LRB-

-RRB-

II

c

d

z

-LRB-

-RRB-

The

NI

is

inconsistent

with

instance

II

and

is

assigned

priority

Since

instance

I

and

the

NI

are

concordant

one-difference

equal

and

the

priority

of

the

instance

I

is

greater

than

that

of

the

NI

then

by

b

-RRB-

the

discriminant

variable

b

can

be

removed

from

instance

I

and

-RRB-

the

NI

can

be

deleted

-LSB-

Since

instance

I

has

higher

priority

than

the

NI

and

since

deleting

the

discriminant

variable

from

instance

I

causes

it

to

subsume

the

NI

the

NI

adds

no

new

information

-RSB-

-RRB-

If

the

OI

and

NI

are

concordant

the

OI

is

one-difference

subset

with

respect

to

the

NI

and

OI.P

NI.P

then

the

discriminant

variable

can

be

deleted

from

the

NI

We

illustrate

mechanism

Assume

the

new

instance

a

b

c

z

is

added

to

the

instance

set

I.

a

b

z

-LRB-

-RRB-

II

c

d

z

-LRB-

-RRB-

III

a

c

z

-LRB-

-RRB-

The

NI

is

inconsistent

with

instance

II

and

is

assigned

priority

Since

instance

I

and

the

NI

are

concordant

the

NI

is

one-difference

superset

with

respect

to

instance

I

and

the

priority

of

the

instance

I

is

greater

than

that

of

the

NI

then

by

the

discriminant

variable

b

can

be

removed

from

the

NI

Note

that

instance

III

also

fulfills

the

requirements

to

simplify

the

NI

-LRB-

namely

it

is

concordant

one-difference

subset

with

higher

priority

-RRB-

However

simplification

by

can

be

done

with

respect

to

only

one

OI

Either

I

or

III

could

have

been

used

for

the

simplification

If

III

had

been

chosen

rather

than

I

the

discriminant

variable

c

would

have

been

removed

from

the

NI

We

show

how

the

restriction

of

at

most

one

application

of

is

supported

in

an

actual

architecture

in

section

We

now

present

a

case

where

does

not

apply

Assume

the

new

instance

a

b

c

z

is

added

to

the

instance

set

I.

a

b

z

-LRB-

-RRB-

II

c

d

z

-LRB-

-RRB-

The

NI

is

inconsistent

with

instance

II

and

is

assigned

priority

Instance

I

and

the

NI

are

concordant

the

NI

is

one-difference

superset

with

respect

to

instance

I

but

the

priority

of

the

NI

is

greater

than

that

of

instance

I

In

this

case

no

removing

action

can

take

place

since

the

NI

is

an

exception

to

instance

II

This

makes

sense

since

the

NI

's

higher

priority

adds

information

We

now

state

and

illustrate

two

simplification

mechanisms

for

one-difference

discordant

instances

-RRB-

If

the

OI

and

NI

are

discordant

the

OI

is

one-difference

equal

with

respect

to

the

NI

and

OI.P

NI.P

then

the

discriminant

variable

can

be

deleted

from

the

OI

We

illustrate

mechanism

Assume

the

new

instance

a

b

z

is

added

to

the

instance

set

I.

a

b

z

-LRB-

-RRB-

II

a

b

z

-LRB-

-RRB-

III

c

d

z

-LRB-

-RRB-

The

NI

is

inconsistent

with

instance

III

and

is

assigned

priority

Instance

I

and

II

are

onedifference

equal

with

respect

to

the

NI

By

the

discriminant

variable

b

can

be

removed

from

instance

I.

-LSB-

No

change

is

made

to

instance

II

since

instance

II

has

higher

priority

than

the

NI

-RSB-

-RRB-

If

the

OI

and

NI

are

discordant

the

OI

is

one-difference

superset

with

respect

to

the

NI

and

OI.P

NI.P

then

the

discriminant

variable

can

be

deleted

from

the

OI

We

illustrate

mechanism

Assume

the

new

instance

a

b

z

is

added

to

the

instance

set

I.

a

b

c

z

-LRB-

-RRB-

II

a

b

d

z

-LRB-

-RRB-

The

NI

is

inconsistent

with

instance

III

and

is

assigned

priority

Instance

I

and

II

are

onedifference

superset

with

respect

to

the

NI

By

the

discriminant

variable

a

can

be

removed

from

instance

I.

-LSB-

No

change

is

made

to

instance

II

since

instance

II

has

higher

priority

than

the

NI

-RSB-

Instance

Set

Learning

Algorithm

Before

giving

the

learning

algorithm

we

define

a

mechanism

for

how

the

OI

matches

the

NI

This

matching

is

specified

by

the

-LRB-

Polarity

Discriminants

Set

Relation

-RRB-

with

each

value

taken

from

the

following

disjoint

sets

Polarity

-LCB-

Concordant

Discordant

-RCB-

Discriminants

-LCB-

NoDiscriminant

OneDiscriminant

Many

-RCB-

Set

Relation

-LCB-

Subset

Equal

Superset

-RCB-

Let

OI

be

an

old

instance

and

NI

be

a

new

instance

Their

matching

tuple

is

the

three-tuple

-LRB-

polarity

discriminants

set

relation

-RRB-

It

is

assigned

as

follows

The

polarity

is

concordant

if

the

two

instances

are

concordant

discordant

otherwise

The

discriminants

is

NoDiscriminant

if

the

two

instance

have

no

discriminant

variable

OneDiscriminant

if

the

two

instances

share

exactly

one

discriminant

variable

and

Many?otherwise

If

discriminants

is

NoDiscriminant

then

set

relation

is

Subset

if

the

variables

of

the

OI

are

a

proper

subset

of

the

variables

of

the

NI

Equal

if

the

variables

of

the

OI

are

equal

to

the

variables

of

the

NI

Superset

if

the

variables

of

the

OI

are

a

proper

superset

of

the

variables

of

the

NI

otherwise

If

discriminants

is

OneDiscriminant

then

set

relation

is

Subset

if

except

for

the

discriminant

variable

the

variables

of

the

OI

are

a

proper

subset

of

the

variables

of

the

NI

Equal

if

except

for

the

discriminant

variable

the

variables

of

the

OI

are

equal

to

the

variables

of

the

NI

Superset

if

except

for

the

discriminant

variable

the

variables

of

the

OI

are

a

proper

superset

of

the

variables

of

the

NI

otherwise

If

discriminants

is

Many

then

set

relation

is

For

example

the

new

instance

a

b

z

would

cause

the

following

old

instances

to

have

the

following

associated

matching

tuples

a

b

c

z

-LCB-

Concordant

NoDiscriminant

Superset

-RCB-

a

b

z

-LCB-

Discordant

NoDiscriminant

Equal

-RCB-

a

b

c

z

-LCB-

Discordant

OneDiscriminant

Superset

-RCB-

d

e

z

-LCB-

Concordant

NoDiscriminant

-RCB-

We

restate

in

terms

of

these

triples

if

OI

is

-LCB-

NoDiscriminant

Superset

or

Equal

-RCB-

with

the

NI

then

the

OI

can

be

deleted

If

OI

is

-LCB-

Concordant

NoDiscriminant

Subset

or

Equal

-RCB-

with

the

NI

and

OI.P

NI.P

then

the

NI

can

be

deleted

If

OI

is

-LCB-

Concordant

OneDiscriminant

Superset

-RCB-

with

the

NI

then

the

OI

's

discriminant

variable

can

be

deleted

a

If

OI

is

-LCB-

Concordant

OneDiscriminant

Equal

-RCB-

with

the

NI

and

OI.P

NI.P

then

the

OI

's

discriminant

variable

can

be

deleted

b

If

OI

is

-LCB-

Concordant

OneDiscriminant

Equal

-RCB-

with

the

NI

and

OI.P

NI.P

then

-RRB-

the

OI

's

discriminant

variable

can

be

deleted

and

-RRB-

the

NI

can

be

deleted

If

OI

is

-LCB-

Concordant

OneDiscriminant

Subset

-RCB-

with

the

NI

and

OI.P

NI.P

then

the

NI

's

discriminant

variable

can

be

deleted

If

OI

is

-LCB-

Discordant

OneDiscriminant

Superset

-RCB-

with

the

NI

and

OI.P

NI.P

then

the

OI

's

discriminant

variable

can

be

deleted

If

OI

is

-LCB-

Discordant

OneDiscriminant

Equal

-RCB-

with

the

NI

and

OI.P

NI.P

then

the

OI

's

discriminant

variable

can

be

deleted

Having

discussed

the

matching

notation

and

the

mechanisms

of

sections

we

present

the

basic

learning

algorithm

The

italicized

text

gives

the

high

level

description

of

each

step

The

plain

text

gives

an

algorithmic

instantiation

of

the

steps

Present

the

new

instance

-LRB-

NI

-RRB-

to

the

system

Each

OI

assigns

its

matching

tuple

with

respect

to

the

NI

Determine

the

priority

of

the

NI

If

an

OI

is

-LCB-

Discordant

NoDiscriminant

-RCB-

then

it

presents

its

priority

to

the

system

else

the

OI

presents

priority

to

the

system

Let

the

highest

such

priority

be

called

TopPriority

The

NI

's

priority

-LRB-

NI.P

-RRB-

is

set

to

TopPriority

Determine

whether

the

NI

is

redundant

If

so

then

the

NI

is

not

added

to

the

instance

set

and

no

further

processing

occurs

otherwise

continue

at

step

If

an

OI

is

-LCB-

Concordant

NoDiscriminant

Subset

or

Equal

-RCB-

and

OI.P

NI.P

then

return

without

adding

the

NI

to

the

instance

set

-LRB-

-RRB-

Delete

subsumed

OIs

If

an

OI

is

-LCB-

NoDiscriminant

Superset

or

Equal

-RCB-

then

the

OI

self-deletes

-LRB-

-RRB-

Generalize

concordant

one-difference

superset

OIs

If

an

OI

is

-LCB-

Concordant

OneDiscriminant

Superset

-RCB-

then

the

OI

removes

the

discriminant

variable

from

its

variable

list

-LRB-

-RRB-

Generalize

concordant

one-difference

equal

OIs

If

an

OI

is

-LCB-

Concordant

OneDiscriminant

Equal

-RCB-

and

OI.P

NI.P

then

the

OI

removes

the

discriminant

variable

from

its

variable

list

-LRB-

a

-RRB-

If

an

OI

is

-LCB-

Concordant

OneDiscriminant

Equal

-RCB-

and

OI.P

NI.P

then

-RRB-

the

OI

removes

the

discriminant

variable

from

its

variable

list

and

-RRB-

the

NI

is

deleted

-LRB-

b

-RRB-

Generalize

the

NI

by

concordant

one-difference

subset

OIs

Let

T

-LCB-

OI

OI

is

-LCB-

Concordant

OneDiscriminant

Subset

-RCB-

and

OI.P

NI.P

-RCB-

If

T

is

nonempty

then

choose

any

instance

I

of

T

and

remove

the

discriminant

variable

of

I

from

the

NI

-LRB-

-RRB-

Generalize

discordant

one-difference

OIs

If

an

OI

is

-LCB-

Discordant

OneDiscriminant

Superset

or

Equal

-RCB-

and

OI.P

NI.P

then

the

OI

removes

the

discriminant

variable

from

its

variable

list

-LRB-

and

-RRB-

Add

the

NI

to

the

instance

set

Add

the

NI

with

its

assigned

priority

NI.P

For

simplicity

we

have

presented

the

above

steps

as

a

sequential

algorithm

Many

of

the

operations

can

take

place

in

parallel

and

we

present

a

parallel

version

of

the

algorithm

after

overviewing

the

PASOCS

architecture

in

section

Prioritized

ASOCS

Architectural

Model

In

this

section

we

give

a

simple

prioritized

ASOCS

architecture

its

associated

execution

algorithm

and

its

associated

learning

algorithm

Emphasis

has

been

placed

on

ease

of

explanation

rather

than

efficiency

Interconnect

and

Node

Requirements

The

Interconnect

We

choose

a

binary

tree

for

our

basic

topology

Consider

the

n-layered

tree

with

N

leaf

nodes

and

junction

-LRB-

internal

-RRB-

nodes

shown

in

figure

A

separate

message

controller

exists

outside

the

tree

called

the

adaption

unit

-LRB-

AU

-RRB-

Communication

flows

up

and

down

the

tree

in

O

-LRB-

log

n

-RRB-

where

n

is

the

number

of

nodes

in

the

network

AU

Inputs

-LRB-

New

Instances

-RRB-

Outputs

Figure

PASOCS

Network

Leaf

Nodes

PASOCS

nodes

-LRB-

the

nodes

that

store

the

rules

-RRB-

are

the

leaf

nodes

of

the

tree

Junction

Nodes

The

internal

nodes

of

the

binary?tree

are

junction

nodes

They

can

transmit

a

message

store

a

messages

and

compare

a

message

against

a

stored

value

Junction

nodes

have

no

rule

bearing

capacity

in

this

particular

architectural

instantiation

Downward

Messages

To

send

a

message

to

all

PASOCS

nodes

the

controller

creates

a

downward

message

and

sends

it

to

the

top

junction

node

When

a

junction

node

receives

a

downward

message

it

duplicates

the

message

and

broadcasts

it

to

each

of

its

children

When

a

PASOCS

-LRB-

leaf

-RRB-

node

receives

a

downward

message

it

reads

and

executes

the

message

Upward

messages

To

send

a

message

to

the

controller

a

PASOCS

node

creates

an

upward

message

attaches

its

internal

priority

and

sends

the

message

to

its

junction

-LRB-

parent

-RRB-

node

Each

junction

node

checks

messages

from

its

children

determines

which

message

has

higher

priority

sends

the

higher

priority

message

up

the

tree

and

discards

the

other

message

In

case

of

a

tie

one

message

is

arbitrarily

chosen

and

sent

Adaption

Unit

-LRB-

AU

-RRB-

This

unit

communicates

directly

with

the

root

node

of

the

network

During

execution

the

AU

receives

the

input

variables

and

sets

the

output

variables

During

learning

the

AU

receives

new

instances

sends

messages

to

the

network

and

receives

messages

from

the

network

PASOCS

Nodes

The

PASOCS

nodes

are

the

leaf

nodes

of

the

tree

Each

PASOCS

leaf

node

contains

-RRB-

a

variable

list

-LRB-

the

list

of

input

variables

over

which

the

node

does

a

conjunction

-RRB-

-RRB-

a

priority

field

-RRB-

a

polarity

flag

-RRB-

a

matching

triple

-LRB-

Polarity

Discriminants

Set

Relation

-RRB-

Initialization

All

leaf

node

priorities

are

set

to

initially

indicating

that

the

leaf

node

is

free

An

active

leaf

node

has

a

positive

priority

Each

active

leaf

node

in

the

tree

represents

an

instance

Execution

Algorithm

During

execution

a

set

of

Boolean

values

J

is

input

to

the

AU

and

broadcast

down

the

tree

Each

leaf

node

executes

the

same

simple

algorithm

If

the

input

J

matches

a

leaf

node

's

variable

list

then

the

leaf

node

sends

the

upward

message

consisting

of

the

node

's

polarity

and

priority

The

highest

priority

message

bubbles

to

the

top

of

the

tree

in

log

time

If

the

winning

message

has

non-zero

priority

and

positive

polarity

then

the

system

outputs

true

If

the

winning

message

has

non-zero

priority

and

negative

polarity

then

the

system

outputs

false

If

the

winning

message

has

zero

priority

then

the

output

is

system

dependent

it

could

output

a

or

a

generalized

value

Learning

Algorithm

Functions

This

section

defines

the

basic

operations

which

can

be

initiated

by?the

adaption

unit

-LRB-

AU

-RRB-

during

learning

The

-LRB-

AU

-RRB-

initiates

six

types

of

messages

BroadcastNI

GetPriority

SendPriority

Update

FindAvailableNodes

and

CreateNode

BroadcastNI

-LRB-

NI

-RRB-

BroadcastNI

transmits

the

NI

's

variable

list

and

the

NI

's

polarity

allowing

each

leaf

node

to

assign

its

matching

tuple

The

logic

for

a

leaf

node

executing

the

BroadcastNI

message

is

if

active

then

assign

one

flag

from

each

the

following

three

disjoint

sets

-LCB-

Concordant

Discordant

-RCB-

-LCB-

NoDiscriminant

OneDiscriminant

Many

-RCB-

-LCB-

Subset

Equal

Superset

-RCB-

otherwise

do

nothing

The

flag

's

values

indicate

the

OI

's

relationship

to

the

NI

in

the

same

manner

as

presented

in

section

GetPriority

-LRB-

-RRB-

GetPriority

allows

the

AU

to

determine

the

highest

priority

of

any

node

which

contradicts

the

NI

The

logic

for

a

leaf

node

executing

the

GetPriority

message

is

If

the

node

is

active

and

-LCB-

Discordant

NoDiscriminant

-RCB-

then

it

sends

an

upward

message

of

the

node

's

priority

otherwise

it

sends

an

upward

message

of

Each

junction

node

passes

on

the

highest

priority

and

the

root

node

passes

the

highest

priority

to

the

AU

SendPriority

-LRB-

NI.P

-RRB-

SendPriority

lets

the

AU

inform

the

network

of

the

assigned

priority

of

the

new

instance

NI.P

-LRB-

The

AU

sets

NI.P

to

one

greater

than

the

value

returned

by

GetPriority

-RRB-

The

logic

for

a

leaf

node

executing

the

SendPriority

-LRB-

NI.P

-RRB-

message

is

if

active

then

store

a

copy

of

NI.P

otherwise

do

nothing

Update

-LRB-

-RRB-

Update

allows

the

nodes

to

do

independent

modification

using

mechanisms

as

introduced

in

section

These

actions

allow

the

system

to

become

consistent

with?the

updated

instance

set

The

logic

for

a

leaf

node

executing

the

Update

message

is

contained

in

section

Each

node

generates

an

upward

message

during

Update

as

explained

in

section

The

AU

accepts

a

single

highest

priority

message

as

part

of

the

Update

FindAvailableNodes

-LRB-

-RRB-

FindAvailableNodes

marks

unused

nodes

The

logic

for

a

leaf

node

executing

the

FindAvailableNodes

message

is

If

the

node

is

active

or

inactive

send

node.P

-LRB-

the

node

's

priority

-RRB-

in

an

upward

FindAvailableNode

message

The

logic

for

a

junction

node

executing

an

upward

FindAvailableNode

message

is

choose

the

lowest

priority

of

either

child

and

mark

which

child

sent

the

lowest

priority

-LRB-

ties

are

broken

arbitrarily

-RRB-

The

AU

receives

the

lowest

priority

in

the

network

If

the

priority

is

positive

then

there

are

no

inactive

nodes

left

in

the

network

and

the

AU

signals

an

error

CreateNode

-LRB-

NI

-RRB-

CreateNode

causes

a

new

node

to

be

allocated

for

the

NI

If

the

result

of

FindAvailableNodes

is

a

priority

of

then

the

AU

initiates

the

CreateNode

message

sending

the

NI

as

a

parameter

The

logic

for

a

junction

node

executing

the

CreateNode

message

is

forward

the

message

to

the

marked

child

The

message

is

guaranteed

to

arrive

at

a

single

inactive

leaf

node

The

logic

for

the

leaf

node

executing

the

CreateNode

message

is

record

the

NI

's

variable

list

polarity

and

priority

At

this

point

the

node

is

active

and

represents

the

NI

A

Parallel

PASOCS

Learning

Algorithm

The

basic

PASOCS

learning

algorithm

was

presented

in

section

Section

has

proposed

a

specific

architecture

on

which

to

implement

the

algorithm

This

section

illustrates

how

the

architecture

supports

the

learning

algorithm

of

section

in

a

parallel

and

distributed

fashion

The

architectural

learning

algorithm

uses

the

operations

defined

in

section

The

basic

algorithm

is

shown

in

italics

with

each

step

followed

by

comments

BroadcastNI

-LRB-

NI

-RRB-

The

BroadcastNI

message

is

sent

down

the

tree

and

each

active

node

assigns

its

matching

three-tuple

GetPriority

-LRB-

-RRB-

The

GetPriority

message

is

sent

down

the

tree

Each

contradictory

node

sends

its

priority

in

an

upward

message

The

AU

receives

P

the

highest

contradictory

priority

The

AU

assigns

NI.P

the

value

P

SendPriority

-LRB-

NI.P

-RRB-

The

SendPriority

message

is

sent

down

the

tree

with

the

value

NI.P

Each

active

leaf

node

stores

NI.P

Update

Each

node

does

independent

adaptation

depending

on

NI.P

and

its

matching

tuple

The

independent

updates

are

discussed

in

section

Some

actions

in

Update

require

modification

of

the

NI

To

support

this

all

active

leaf

nodes

send

one

of

the

following

three

upward

messages

during

update

DeleteNI

Causes

the

NI

to

be

deleted

-LRB-

not

added

-RRB-

GeneralizeNI

-LRB-

DV

-RRB-

The

discriminant

variable

passed

in

the

message

is

deleted

from

the

NI

NoChange

No

NI

modification

takes

place

A

junction

node

passes

on

only

one

message

At

a

junction

DeleteNI

has

priority

over

GeneralizeNI

which

has

priority

over

NoChange

Ties

are

broken

arbitrarily

If

Update

message

DeleteNI

then

exit

algorithm

else

If

Update

message

GeneralizeNI

-LRB-

DV

-RRB-

then

remove

the

discriminant

variable

DV

from

the

NI

FindAvailableNodes

-LRB-

-RRB-

Inactive

nodes

are

marked

preparatory

to

CreateNode

CreateNode

-LRB-

NI

-RRB-

An

inactive

node

is

allocated

for

the

NI

After

the

learning

algorithm

is

finished

the

system

can

either

add

another

NI

or

can

go

to

execution

mode

Update

During

the

Update

phase

-LRB-

step

of

section

-RRB-

each

node

executes

an

independent

and

parallel

update

depending

on

the

state

of

NI.P

and

the

matching

triple

The

parallel

actions

correspond

to

the

sequential

actions

of

section

All

node

actions

occur

simultaneously

If

the

node

is

-LCB-

Concordant

NoDiscriminant

Subset

or

Equal

-RCB-

and

node.P

NI.P

then

send

an

upward

DeleteNI

message

-LSB-

In

this

case

the

NI

is

already

fulfilled

and

will

be

deleted

-LRB-

-RRB-

-RSB-

else

if

the

node

is

-LCB-

NoDiscriminant

Superset

or

Equal

-RCB-

then

self-delete

-LRB-

set

priority

to

-RRB-

and

send

the

upward

NoChange

message

-LRB-

-RRB-

else

if

the

node

is

-LCB-

Concordant

OneDiscriminant

Superset

-RCB-

then

self-generalize

by

removing

the

discriminant

variable

from

the

node

variable

list

and

send

the

upward

NoChange

message

-LRB-

-RRB-

else

if

the

node

is

-LCB-

Concordant

OneDiscriminant

Equal

-RCB-

then

self-generalize

by

removing

the

discriminant

variable

-LRB-

If

node.P

NI.P

then

send

the

upward

DeleteNI

message

else

send

the

upward

NoChange

message

-RRB-

-LSB-

The

node

always

removes

the

discriminant

variable

in

this

case

regardless

of

its

priority

Also

if

node.P

NI.P

the

NI

will

be

deleted

-LRB-

a

and

b

-RRB-

-RSB-

else

if

the

node

is

-LCB-

Concordant

OneDiscriminant

Subset

-RCB-

and

node.P

NI.P

then

send

the

upward

GeneralizeNI

-LRB-

DV

-RRB-

message

-LSB-

If

the

GeneralizeNI

message

reaches

the

AU

the

NI

will

be

generalized

by

removing

the

discriminant

variable

-LRB-

-RRB-

-RSB-

else

if

the

node

is

-LCB-

Discordant

OneDiscriminant

Superset

-RCB-

and

node.P

NI.P

then

selfgeneralize

by

removing

the

discriminant

variable

from

the

node

variable

list

and

send

the

upward

NoChange

message

-LRB-

and

-RRB-

else

send

the

upward

NoChange

message

PASOCS

Example

This

section

illustrates

how

PASOCS

learns

An

inactive

leaf

node

-LRB-

priority

-RRB-

is

shown

as

empty

We

assume

the

network

begins

without

any

active

nodes

and

that

the

following

seven

instances

are

entered

in

the

given

order

A.

abd

z

B.

abc

z

C.

abc

'd

z

D.

a

bc

'd

z

E.

bc

z

F.

a

b

'd

z

G.

ab

'd

z

A

Since

there

are

no

instances

in

the

system

initially

the

system

assigns

abd

z

the

priority

and

adds

it

to

the

network

-LRB-

See

figure

-RRB-

-LRB-

abd

-RRB-

a

b

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

Figure

After

First

Instance

B

Since

abc

z

is

inconsistent

with

abd

z

of

node

with

priority

the

system

assigns

abc

z

the

priority

No

other

action

takes

place

and

the

NI

is

added

to

the

network

-LRB-

See

figure

-RRB-

-LRB-

abd

-RRB-

a

b

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

c

-LRB-

abc

-RRB-

Figure

Modified

Network

C

Since

abc

'd

z

is

consistent

with

every

node

in

the

network

the

system

assigns

abc

'd

z

the

priority

Since

node

-LRB-

abc

z

-RRB-

is

-LCB-

Concordant

NoDiscriminant

Subset

-RCB-

with

respect

to

the

NI

abc

'd

z

and

since

the

priority

of

node

is

greater

than

the

priority

of

the

NI

the

NI

is

deleted

by

a

DeleteNI

message

during

Update

The

network

remains

as

in

figure

D

Since

a

bc

'd

z

is

consistent

with

every

node

in

the

network

the

system

assigns

a

bcd

z

priority

No

other

action

takes

place

and

the

NI

is

added

to

the

network

-LRB-

See

figure

-RRB-

-LRB-

abd

-RRB-

a

b

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

c

-LRB-

abc

-RRB-

-LRB-

a

bc

'd

-RRB-

Figure

Modified

Network

E

Since

bd

z

is

inconsistent

with

abc

z

-LRB-

node

-RRB-

with

priority

the

system

assigns

bd

z

the

priority

Node

-LRB-

abd

z

-RRB-

is

-LCB-

Concordant

NoDiscriminant

Superset

-RCB-

with

respect

to

the

NI

Thus

it

is

subsumed

by

the

NI

and

self-deletes

by

setting

its

node.P

to

Node

-LRB-

a

bc

'd

z

-RRB-

is

-LCB-

Concordant

OneDiscriminant

Superset

-RCB-

Thus

node

generalizes

itself

by?removing

the

discriminant

variable

d.

-LRB-

See

figure

-RRB-

a

b

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

c

-LRB-

abc

-RRB-

-LRB-

a

bc

-RRB-

-LRB-

bd

-RRB-

Figure

Modified

Network

F

Since

a

b

z

is

consistent

with

every

node

in

the

network

the

system

assigns

it

the

priority

Since

node

-LRB-

a

bc

z

-RRB-

is

-LCB-

Concordant

OneDiscriminant

Superset

-RCB-

with

respect

to

the

NI

node

generalizes

itself

by

removing

the

discriminant

variable

b.

-LRB-

See

figure

-RRB-

a

b

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

c

-LRB-

abc

-RRB-

-LRB-

a

c

-RRB-

-LRB-

bd

-RRB-

-LRB-

a

b

-RRB-

Figure

Modified

Network

G

Since

ab

c

z

is

consistent

with

every?node

in

the

network

the

system

assigns

it

the

priority

Because

node

-LRB-

abc

z

-RRB-

is

-LCB-

Concordant

OneDiscriminant

Equal

-RCB-

with

respect

to

the

NI

it

generalizes

itself

by

removing

the

discriminant

variable

b.

Also

since

the

priority

of

node

is

greater

than

the

priority

of

the

NI

it

sends

the

DeleteNI

message

during

update

The

DeleteNI

message

lets

the

AU

discard

the

NI

The

final

network

is

shown

in

figure

a

b

d

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

z

c

-LRB-

ac

-RRB-

-LRB-

a

c

-RRB-

-LRB-

bd

-RRB-

-LRB-

a

b

-RRB-

Figure

Final

Network

Multiple

Outputs

Up

to

this

point

the

PASOCS

discussion

has

been

limited

to

a

single

output

variable

We

want

to

extend

the

system

to

permit

multiple

outputs

for

example

I.

a

b

x

II

a

c

y

III

b

c

z

IV

a

b

d

z

V.

a

b

d

x

Of

course

one

implementation

could

use

a

separate

PASOCS

for

each

output

variable

However

more

efficient

use

of

hardware

can

be

attained

if

nodes

share

responsibility

for

multiple

outputs

One

extension

augments

the

node

's

variable

list

and

learning

flag

set

with

a

separate

polarity

and

priority

for

each

output

variable

it

represents

Thus

a

node

may

have

positive

polarity

for

x

negative

polarity

for

y

and

be

undefined

for

other

variables

Figure

shows

a

node

representing

the

output

variables

z

and

x

for

instances

IV

and

V

above

Assume

they

have

the

priorities

and

respectively

a

b

d

z

polarity

priority

x

polarity

priority

Figure

Multiple

Output

Node

During

execution

if

the

node

of

figure

is

matched

then

its

upward

message

contains

information

for

both

output

variables

In

this

case

the

message

could

be

-LCB-

-LRB-

z

-RRB-

-LRB-

x

-RRB-

-RCB-

During

learning

the

BroadcastNI

message

is

augmented

with

the

new

instance

's

output

variable

For

example

if

the

new

instance

ab

x

is

entered

then

when

the

node

of

figure

receives

the

message

BroadcastNI

and

x

the

node

assigns

its

matching

triple

to

-LCB-

Discordant

NoDiscriminant

Superset

-RCB-

Therefore

by

it

can

delete

its

x

output

variable

reference

However

the

node

still

retains

its

variable

z

values

When

all

output

variable

fields

for

a

node

are

deleted

then

the

node

itself

can

self-delete

As

a

second

example

assume

we

have

the

original

node

of

figure

and

the

new

instance

ab

'd

y

is

added

to

the

network

The

node

of

figure

can

send

an

upward

message

indicating

that

it

matches

the

variable

list

of

the

NI

Then

if

a

later

CreateNode

function

is

issued

by

the

AU

the

node

can

simply

add

the

new

polarity

and

priority

for

variable

y

to

its

variable

fields

with

no

new

node

allocation

required

Though

informal

this

section

gives

the

basic

flavor

of

an

extension

to

support

multiple

outputs

Conclusion

We

have

introduced

the

Priority

ASOCS

model

and

have

described

one

possible

PASOCS

architecture

The

significant

features

of

PASOCS

include

a

-RRB-

guaranteed

learning

of

any

set

of

rules

and

b

-RRB-

fast

execution

and

learning

times

-LRB-

O

-LRB-

log

-LRB-

n

-RRB-

-RRB-

where

n

is

the

number

of

rules

-RRB-

The

advantages

of

PASOCS

over

previous

ASOCS

models

are

-RRB-

at

most

one

node

is

added

for

each

instance

learned

and

-RRB-

the

execution

and

learning

time

bounds

are

based

on

the

number

of

rules

rather

than

the

number

of

nodes

as

in

other

models

Current

work

with

PASOCS

and

other

ASOCS

models

is

concentrating

on

a

-RRB-

improved

generalization

for

inputs

not

explicitly

represented

in

training

b

-RRB-

extension

to

analog

and

higher-order

data

c

-RRB-

integration

of

rules

and

examples

d

-RRB-

applications

and

e

-RRB-

physical

implementation

studies

Bibliography

Backus

J.

Can

programming

be

liberated

from

the

von

Neumann

style

A

functional

style

and

its

algebra

of

programs

Communications

of

the

ACM

-LRB-

-RRB-

Chang

J.

and

J.

J.

Vidal

Inferencing

in

Hardware

Proceedings

of

the

MCC-University

Research

Symposium

Austin

TX

-LRB-

July

-RRB-

Hughes

B.

Prioritized

Rule

Systems

MS

Thesis

Brigham

Young

University

Martinez

T.

R.

Adaptive

Self-Organizing

Logic

Networks

PhD

Dissertation

University

of

California

Los

Angeles

Los

Angeles

California

Martinez

T.

R.

Adaptive

Self-Organizing

Concurrent

Systems

Progress

in

Neural

Networks

vol

Ch

Ablex

Publishing

Martinez

T.

R.

and

Vidal

J.J.

Adaptive

Parallel

Logic

Networks

Journal

of

Parallel

and

Distributed

Computing

vol

Martinez

T.

R.

and

Campbell

D.

M.

A

Self-Adjusting

Dynamic

Logic

Module

Journal

of

Parallel

and

Distributed

Computing

vol

no.

pp.

Martinez

T.

R.

and

Campbell

D.

M.

A

Self-Organizing

Binary

Decision

Tree

for

Incrementally

Defined

Rule

Based

Systems

IEEE

Transactions

on

System

Man

and

Cybernetics

vol

Martinez

T.

R.

Consistency

and

Generalization

of

Incrementally

Trained

Connectionist

Models

Proceedings

of

the

International

Symposium

on

Circuits

and

Systems

pp.

Minsky

M.

and

Papert

S.

Perceptrons

an

Introduction

to

Computational

Geometry

MIT

Press

Cambridge

MA

Rosenblatt

F.

Principles

of

Neurodynamics

Spartan

Books

Washington

D.

C.

Rumelhart

D.

and

McClelland

J.

Parallel

Distributed

Processing

Explorations

in

the

Microstructure

of

Cognition

MIT

Press

Cambridge

MA


Weighted search in the plane Richa Agarwala?and David Fern?andez - Bacay Abstract We present a simple two-dimensional weighted version of Megiddo 's multidimensional search technique . This speeds up algorithms for certain convex optimization problems in the plane . 1 Introduction We present a two-dimensional weighted version of the multidimensional search technique of Megiddo -LSB- 4 , 7 , 9 -RSB- . The result has applications to a class of two-dimensional convex optimization problems -LSB- 2 , 3 , 5 , 6 , 11 -RSB- . We need to present some background before stating our result precisely . Define sign : R ! f ? 1 ; ; +1 g as sign -LRB- x -RRB- = 8 > < > : if x = +1 if x > ? 1 if x < Let > = ? be a point in Rd. . An affine function h : Rd ! R is resolved if sign -LRB- h -LRB- > = ? -RRB- -RRB- has been computed . Suppose we have a finite set H of n d-dimensional affine functions and an oracle B that can resolve any h. Let S be a finite set on which a weight function w : S ! R + has been defined . For S0 ? S we write w -LRB- S0 -RRB- to denote Ps2S0 w -LRB- s -RRB- . The following result is known -LSB- 2 -RSB- . Theorem 1 For each d > = , there exist constants -LRB- d -RRB- and ffi -LRB- d -RRB- and an algorithm Weighted-Search with the following property . Given a finite set H of n d-dimensional affine functions and a weight function w : H ! R + , Weighted-Search either finds a subset H0 ? H with w -LRB- H0 -RRB- > = ffi -LRB- d -RRB- ? w -LRB- H -RRB- and resolves every h 2 H0 by making at most -LRB- d -RRB- calls to B or it returns an affine function h such that sign -LRB- h -LRB- > = ? -RRB- -RRB- = . Furthermore , the work done by Weighted-Search in addition to the oracle calls is O -LRB- n -RRB- . ? DIMACS -LRB- Center for Discrete Mathematics and Theoretical Computer Science -RRB- , Rutgers University , Piscataway , NJ 08855 -LRB- agarwala@dimacs.rutgers.edu -RRB- . Part of this work was done while this author was at Iowa State University and was supported by the National Science Foundation under grant CCR-9211262 . yDepartment of Computer Science , Iowa State University , Ames , IA 50011 -LRB- fernande@cs.iastate.edu -RRB- . Supported in part by the National Science Foundation under grant CCR-9211262 . Agarwala and Fern?andez - Baca -LSB- 2 -RSB- gave two algorithms for this problem , one of which achieved -LRB- d -RRB- = 2d ? 1 and ffi -LRB- d -RRB- = 12 = 242d ? 1 , the other achieved -LRB- d -RRB- = 2 -LRB- 60d ? 1 -RRB- and ffi -LRB- d -RRB- = 1 = 12 . Subsequently , Agarwal , Sharir , and Toledo -LSB- 1 -RSB- gave an algorithm that achieved -LRB- d -RRB- = O -LRB- d3 log d -RRB- and ffi -LRB- d -RRB- = 1 = 2 . All these algorithms are quite complex and require many oracle calls , even for small d . Since oracle calls tend to be expensive , it is of interest to develop simple algorithms which require fewer oracle calls . Here we give a comparatively simple weighted search algorithm for the plane which always finds a subset H0 ? H with w -LRB- H0 -RRB- > = w -LRB- H -RRB- = 4 and resolves every h 2 H0 by making at most 3 calls to B , thereby achieving -LRB- 2 -RRB- = 3 and ffi -LRB- 2 -RRB- = 1 = 4 with O -LRB- n -RRB- additional work . This algorithm uses ideas that were first introduced by Megiddo -LSB- 10 -RSB- for a partitioning problem on the plane . 2 The algorithm Suppose H = fh1 ; : : : ; hng , where hi -LRB- x ; y -RRB- = aix + biy + ci . If ai ; bi = 0 , sign -LRB- hi -RRB- is simply the sign of ci , and no oracle calls are needed . Thus , the presence of such hi 's can only help . We shall henceforth assume that ai < > or bi < > for i = 1 ; ? ? ? ; n . In this case , each affine function hi corresponds to a line Hi 2 R2 where Hi = f -LRB- x ; y -RRB- : hi -LRB- x ; y -RRB- = g. Computing sign -LRB- hi -RRB- is thus equivalent to determining whether Hi contains > = ? and , if not , finding which side of Hi contains > = ? . We shall find it convenient to deal interchangeably with lines and affine functions and to extend the weight function w to these lines by making w -LRB- Hi -RRB- = w -LRB- hi -RRB- . Suppose bi = for some i 2 f1 ; ? ? ? ; ng . Pick a such that < > -LRB- ? bi = ai -RRB- for i = 1 ; ? ? ? ; n. By using a change of variables x = x0 + ? y and bi = b0i ? ? ai , we get a set of lines in which b0i < > for i = 1 ; ? ? ? ; n . This change of variables simplifies the exposition and needs to be reversed before making an oracle call . For convenience , we now drop the primes on x0 and b0i . Henceforth , bi < > for i = 1 ; ? ? ? ; n . Since for every scalar l < > 0 , sign -LRB- h -LRB- x ; y -RRB- -RRB- = sign -LRB- l -RRB- ? sign -LRB- h -LRB- x ; y -RRB- = l -RRB- , we can rewrite the equations of these lines so that ai > = 0 . Let the slope ffi of Hi be the same as that of aix + biy + ci = with respect to y = 0 ; i.e. , let ffi = -LRB- ? ai = bi -RRB- . Since bi < > for i = 1 ; ? ? ? ; n , the slopes are well defined . Let ff ? be the weighted median of the set fffig where the weight of ffi is w -LRB- Hi -RRB- . Now we make the slopes of roughly weighted half of the lines nonnegative and weighted half nonpositive by using the change of variables y = y0 + ff ?? x and ai = a0i?ff ?? bi . As before , this change of variables is only done to simplify the exposition and needs to be reversed before making an oracle call . We drop the primes on y0 and a0i . Recalculate the slopes of the lines after making this change in variables . All lines that originally had a slope of ff ? will have slope . Let H0 = fHi : ffi = g , H ? = fHi : ffi < g , and H + = fHi : ffi > g. Let W = w -LRB- H -RRB- . Since is our new weighted median slope , w -LRB- H ? -RRB- < = W = 2 and w -LRB- H ? -RRB- + w -LRB- H0 -RRB- > = W = 2 . Therefore , w -LRB- H ? -RRB- > = W = 2 ? w -LRB- H0 -RRB- . Similarly , W = 2 > = w -LRB- H + -RRB- > = W = 2 ? w -LRB- H0 -RRB- . Assume without loss of generality that H + = fH1 ; ? ? ? ; Hkg , H ? = fHk +1 ; ? ? ? ; Hmg , and H0 = fHm +1 ; ? ? ? ; Hng . Let WP -LRB- c -RRB- -LSB- WN -LRB- c -RRB- -RSB- be the weighted median of the set fp1 ; ? ? ? ; pkg -LSB- fpk +1 ; ? ? ? ; pmg -RSB- , where pi is the point of intersection of lines Hi and y = c -LRB- i.e. , pi = -LRB- ? -LRB- c ? bi + ci -RRB- = ai ; c -RRB- -RRB- and w -LRB- pi -RRB- = w -LRB- Hi -RRB- . Let WP -LSB- WN -RSB- be the curve that we get if we plot WP -LRB- c -RRB- -LSB- WN -LRB- c -RRB- -RSB- from c = ? 1 to c = 1 -LRB- See Figure 1 -RRB- . y = b * x = a * W W N P p * Figure 1 : WP and WN . Observation 1 WP -LSB- WN -RSB- is a monotonically increasing -LSB- decreasing -RSB- function . Let p ? = -LRB- a ? ; b ? -RRB- be the point where WP intersects WN . Let H?P 2 H + and H?N 2 H ? be the lines whose intersection point is p ? . Suppose that , using two oracle calls , we resolve the lines y = b ? and x = a ? . We have the following cases : Case 1 : > = ? lies on the line x = a ? and on the line y = b ? . Then > = ? = p ? and we have resolved all the lines in H. Case 2 : > = ? lies on y = b ? . In this case we have resolved a set of lines H1 ? H + and H2 ? H ? such that w -LRB- H1 -RRB- > = w -LRB- H + -RRB- = 2 and w -LRB- H2 -RRB- > = w -LRB- H ? -RRB- = 2 . -LRB- For example , if > = ? lies on the line described by y = b ? and x > a ? , then we resolve fhi 2 H + -LSB- H ? : ? -LRB- b ? ? bi + ci -RRB- = ai < = a?g . -RRB- Furthermore , every H 2 H0 is also resolved . Case 3 : > = ? lies on x = a ? . As in the previous case , we resolve H1 ? H + and H2 ? H ? such that w -LRB- H1 -RRB- > = w -LRB- H + -RRB- = 2 and w -LRB- H2 -RRB- > = w -LRB- H ? -RRB- = 2 . Case 4 : > = ? lies in one of the following four quadrants : y < b ? and x > a ? , y < b ? and x < a ? , y > b ? and x > a ? , y > b ? and x < a ? In this case either we resolve H0 ? H + such that w -LRB- H0 -RRB- > = w -LRB- H + -RRB- = 2 or we resolve H0 ? H ? such that w -LRB- H0 -RRB- > = w -LRB- H ? -RRB- = 2 . -LRB- For example , if > = ? lies in the quadrant described by y < b ? and x > a ? , then we resolve fhi 2 H + : ? -LRB- b ? ? bi + ci -RRB- = ai < = a?g ; see -LSB- 9 -RSB- for details . -RRB- In cases 1 and 2 , we have resolved a subset of H with weight at least W = 2 . Thus , we have achieved a fraction ffi -LRB- 2 -RRB- = 1 = 2 with -LRB- 2 -RRB- = 2 oracle calls . In cases 3 and 4 , we have resolved a subset of H + -LSB- H ? of weight at least -LRB- 1 = 2 -RRB- -LRB- W = 2 ? w -LRB- H0 -RRB- -RRB- . Let c ? is the weighted median of the set fpi : pi = ? ci = bi ; hi 2 H0g , where w -LRB- pi -RRB- = w -LRB- hi -RRB- . By resolving the line y = c ? , we will be resolving a subset of H0 having total weight at least w -LRB- H0 -RRB- = 2 . The total weight of the lines resolved is thus at least -LRB- 1 = 2 -RRB- -LRB- W = 2 ? w -LRB- H0 -RRB- -RRB- + w -LRB- H0 -RRB- = 2 = W = 4 , giving us a fraction ffi -LRB- 2 -RRB- = 1 = 4 with -LRB- 2 -RRB- = 3 oracle calls . In the next section we describe how to find point p ? in O -LRB- m -RRB- time by adapting a technique due to Megiddo -LSB- 10 -RSB- . Hence , we have a search algorithm for Euclidean plane with -LRB- 2 -RRB- = 3 and ffi -LRB- 2 -RRB- = 1 = 4 , which does O -LRB- n -RRB- work , in addition to the time spent on oracle calls . 3 Finding p ? We rely on an idea introduced by Megiddo -LSB- 10 -RSB- who gave an algorithm for the case where WN and WP correspond to unweighted medians . The technique relies on a line query algorithm that finds the position of p ? with respect to any straight line in O -LRB- m -RRB- time . We solve our problem by invoking the line query algorithm repeatedly ; each step either finds p ? or it discards at least 1 = 8 of the lines as candidates for H?P and H?N . Point p ? is just the intersection point of H?P and H?N . An adaptation of Megiddo 's line query algorithm to handle the weighted case is presented in Section 3.1 . A summary of the steps required to find H?P and H?N is presented in Section 3.2 . 3.1 The line query Suppose L is a vertical line given by x = c . The analysis for the case where L is horizontal is analogous . L intersects WP and WN exactly once . Let P = -LRB- c ; bP -RRB- be the point where WP intersects L and N = -LRB- c ; bN -RRB- be the point where WN intersects L. N -LSB- P -RSB- is the weighted median of the y values of the intersection points of lines in H ? -LSB- H + -RSB- with L . It is easy to see that if bN < bP -LSB- bN > bP -RSB- then a ? < c -LSB- a ? > c -RSB- . Furthermore , if bN = bP we have p ? = -LRB- c ; bN -RRB- . Now consider the case where L has positive slope ; the analysis for the case where L has negative slope is analogous . Line L will intersect with WN exactly once , at a point N = -LRB- a ; b -RRB- that is the weighted median of the x values of the intersection points of lines in H ? with L. Thus N can be found in O -LRB- m ? k -RRB- time . Let P be the weighted median of the y values of the intersection points of lines in H + and the line x = a . This takes O -LRB- k -RRB- time . As before , the relative position of N and P along with the fact that N lies on WN and P lies on WP is sufficient to determine the relative position of L and p ? . 3.2 Finding H?P and H?N Let R be the set of candidate lines for H?P and H?N at the beginning of the current iteration . At the start of the algorithm R = H + -LSB- H ? . Each iteration either finds p ? or discards 1 = 8 of the candidate lines as follows : Step 1 . Find the median slope s of the lines in R. Step 2 . Partition R into sets G and S such that the lines in G -LSB- S -RSB- have slopes greater -LSB- smaller -RSB- than s. Step 3 . Form disjoint pairs of lines so that each pair has one line from G and another from S. Step 4 . Compute the x values of the intersection points of the paired lines . Let c be the median of these values . Step 5 . Do a line query on line x = c . This either finds p ? , in which case we are done , or it determines that a ? < c , or a ? > c. Suppose a ? < c . The case for a ? > c is analogous . Step 5.1 Let L be the line with slope s and y intercept equal to the median of the set fy ? sxg , where -LRB- x ; y -RRB- runs over the intersection points with x values greater than or equal to c. Note that there are at least jRj = 4 such points and that L divides them into two equal-size sets . Step 5.2 Do a line query on L. Lines L and x = c identify a quadrant Q containing p ? . The quadrant opposite to Q has jRj = 8 intersection points ; with respect to each such point , there is a line that does not cross Q and thus can not be H?P or H?N . These lines are discarded from R and we are left with a similar problem with 7jRj = 8 lines . Acknowledgements We thank Naoki Katoh for drawing our attention to -LSB- 10 -RSB- . The referee noticed an error in an earlier version of this note and gave suggestions for improving the presentation . References -LSB- 1 -RSB- P.K. Agarwal , M. Sharir , and S. Toledo . An efficient multidimensional searching technique and its applications , Technical Report CS-1993-20 , Duke University , July 1993 . -LSB- 2 -RSB- R. Agarwala and D. Fern?andez - Baca . Weighted multidimensional search and its application to convex optimization , DIMACS Technical Report 92-51 , November , 1992 . To appear in SIAM J. Comput . -LSB- 3 -RSB- Y. P. Aneja and S. N. Kabadi . Polynomial algorithms for lagrangean relaxations in combinatorial problems , Manuscript . -LSB- 4 -RSB- K.L. Clarkson . Linear programming in O -LRB- n ? 3d2 -RRB- time , Information Processing Letters 22 -LRB- 1986 -RRB- , 21 -LCB- 24 . -LSB- 5 -RSB- E. Cohen . Combinatorial algorithms for optimization problems , Report No . STAN - CS-91-1366 , Department of Computer Science , Stanford University , Stanford , CA 94305 . -LSB- 6 -RSB- E. Cohen and N. Megiddo . Maximizing concave functions in fixed dimension , in : P. M. Pardalos , editor , Complexity in Numerical Optimization -LRB- 1993 -RRB- , 74 -LCB- 87 . -LSB- 7 -RSB- M. E. Dyer . On a multidimensional search technique and its application to the Euclidean one -LCB- centre problem , SIAM J. Comput . 15 -LRB- 3 -RRB- -LRB- 1986 -RRB- , 725 -LCB- 738 . -LSB- 8 -RSB- M. L. Fisher . The Lagrangian relaxation method for solving integer programming problems , Management Science 27 -LRB- 1 -RRB- -LRB- 1981 -RRB- , 1 -LCB- 18 . -LSB- 9 -RSB- N. Megiddo . Linear programming in linear time when the dimension is fixed , J. Assoc. Comput . Mach. , 31 -LRB- 1984 -RRB- , 114 -LCB- 127 . -LSB- 10 -RSB- N. Megiddo . Partitioning with two lines in the plane , J. Algorithms , 6 -LRB- 1985 -RRB- , 430 - 433 . -LSB- 11 -RSB- C.H. Norton , S.A. Plotkin , and ? E. Tardos . Using separation algorithms in fixed dimension , J. Algorithms 13 -LRB- 1992 -RRB- , 79 -LCB- 98 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O

algorithms
combinatorial problems
convex optimization