A Relevance Feedback Image Retrieval Scheme Using Multi-Instance and Pseudo Image Concepts SUMMARY Content-based image search has long been considered a difficult task. Making correct conjectures on the user intention (perception) based on the query images is a critical step in the content-based search. One key concept in this paper is how we find the user preferred low-level image characteristics from the multiple positive samples provided by the user. The second key concept is how we generate a set of consistent "pseudo images" when the user does not provide a sufficient number of samples. The notion of image feature stability is thus introduced. The third key concept is how we use negative images as pruning criterion. In realizing the preceding concepts, an image search scheme is developed using the weighted lowlevel image features. At the end, quantitative simulation results are used to show the effectiveness of these concepts. Due to the increasing popularity of digital capturing devices such as digital camera, the dramatically large size of digital contents demands highly efficient multimedia content management. For a particular application, in order to achieve the desired matching accuracy, a content-based image retrieval (CBIR) system often has a distinct set of configurations [1] including selected image features and a processing architecture. A known approach for constructing a satisfactory CBIR system is to incorporate semantic related features for matching. Researches on CBIR topics show that semantic features are critical in boosting the query accuracy. These high-level features can be either extracted at the stage of content analysis, or acquired at run-time. The former is used to provide semantic content descriptions and often requires sophisticated feature extraction processes, such as object segmentation or textual descriptions. The latter is often used to capture the user preferences, and often tries to derive the information from run-time input sources, for example, the user-provided query images.. Features can be roughly categorized into two groups according to their lifetime. The first group is used to describe the static information of an image, such as the color and the textual description; the second group is used to describe the run-time properties of a content-based processing system, such as the user preferences and the content distribution in a database. Although these two kinds of features represent different aspects of a CBIR problem, they are often coupled. A CBIR application is often designed for a specific problem domain, and only processes a limit number of static features. Based on these selected features, it incorporates means to conjecture about user preferences for each query. There are no general rules in acquiring user preferences; thus, many CBIR systems have been proposed to interact with users by using relevance feedback. A typical Query-by-Example (QBE) CBIR system with relevance feedback generally analyzes the user query images and/or relevant feedback images to derive the necessary search parameters. The search parameters are often defined in terms of the image features pre-chosen in the system. Then, the system searches the database and returns a list of the top-N similar images for further feedback actions. This process can be repeated and hopefully it will eventually produce satisfactory results to that particular user and query. If we treat the relevance feedback function as a means to obtain multiple user inputs, the system can be generalized to be an architecture which:. • acquires multiple query instances (positive ones and negative ones).. • analyzes the collected instances based on the prechosen features.. • makes a "guess" on the user intention, represented by using the pre-chosen features.. • performs the search according to the derived user intention.. In addition to the user preference issue, we often expect that a CBIR application can be efficiently implemented. Many of the conventional CBIR methods adopt classification techniques, and assume that some global distribution parameters are available without penalty. For example, the MARS [2] scheme needs the mean and the variance values in normalizing distances. However, when an application is applied to a very large database or a collection of distributed databases, it not only faces the computational complexity problem, but also it needs to solve the frequent database update problem. Thus, updating the database distribution parameters can be a costly task. Although the terms, positive and negative queries, appear in [3] , they are referred to (image) regions that compose an image. Image similarity is decided by a number of set operations on the pre-classified (region) feature clusters. The approach in [3] is very different from the approach in this paper.. In this paper, we are interested in estimating low- Thus, we will focus on the content-based image retrieval (CBIR) methods using only low-level image features. In designing the user-intention estimation algorithm, we try to keep the algorithm independent of the distribution parameters to reduce the computational cost. As shown in Sect. 4, it may degrade the accuracy in some cases. However, it is a trade-off between accuracy and scalability. Our paper is organized as follows. In Sect. 2, we briefly discuss the concept of multiple query instances (relevance feedback) and the problems in using this technique. In Sect. 3, based on a few assumptions, we propose a straightforward yet effective method that incorporates multiple samples and image multi-scale property for estimating user intention. Also in this section, we propose a method to deal with the negative samples. In Sect. 4, we conduct simulations to evaluate our conjectures. Base on a fairly recognized objective performance index, we compare a few different methods. At the end, we conclude this presentation with Sect. 5. Based on the above simulation results, we briefly summarize our observations below.. • Distance-based weight estimation outperforms when multiple input instances are available.. • Pseudo images improve query accuracy in many cases, especially when our method is used with SNR scalability.. • Experiments show that one pseudo image per input image gives significant performance boost in most cases.. • Negative instances used as a pruning criterion produce better results than those used as negative samples in weight calculation.. • When input instances are few, negative feedback may even degrade the performance of the MARS-like and the Gaussian-normalized schemes.. • When sufficient input instances are available, the Gaussian normalized feature distance does not provide as precise estimation as our method.. • The SNR multi-scaled pseudo images provide better ANMRR values; they also lead to more consistent improvements in accuracy.. An overall comment about the performance of our scheme is as follows:. • When only one input image is available, our scheme looses about 0.8 in log(ANMRR). However, with the assistance of pseudo images, the gap shrinks to about 0.25. • In the case of two input images, our scheme improves.. Without pseudo images, ours looses about 0.2; with pseudo images, ours may win or loose in the average of 0.05.. • When we have three input images, our scheme wins.. The figure in log(ANMRR) is about 0.3 to 0.5.. • From the above results, we can summarize that Scheme C is good for sufficient query images. For small-sample cases, though not as good as other schemes, it produces comparable accuracy by including pseudo images. In this paper, the multi-instance image retrieval problem was investigated. The main contributions of this paper are listed below.. 1. Two distance-based methods to estimate the user perception based on the given positive instances were proposed. One is a geometric approach and the other is a statistical approach. 2. Two schemes for generating consistent pseudo images were investigated. We showed that the pseudo image concept improves the query accuracy in many cases, particularly when the query set is too small. 3. A method of pruning irrelevant outcomes based on the given negative images was proposed.. The first concept was realized by analyzing the scattering magnitude of the query instances in the feature space. Our conjecture is that a scattered feature implies less importance in deciding the perceptual similarity. The second concept was realized through the notion of feature stability. Our conjecture is that a stable image feature (for a particular image) would have similar numerical values (small scatter numbers) at different spatial or SNR scales of the same image. Therefore, pseudo images were created by scaling the original image at various spatial and SNR resolutions. The third one was realized by creating pruning regions in the combined feature space. Our conjecture is that negative instances carry only the information of the undesired image feature values. Namely, undesired images should not look like the opposites of positive images. Because negative images may be close or far away, they are only suitable for pruning not for distance estimation. All the preceding concepts can be integrated into one algorithm using the same basic structure. We examined the performance of our scheme using the ANMRR criterion. Simulations showed that multiple instances are helpful in achieving better query accuracy. In the case that the user input set is small, the synthesized pseudo images improve the results in most cases. As we conjectured, negative feedbacks used for pruning performs better than those used for weight estimation.. Additional conclusions are:. 1. the SNR multi-scaled images generally perform better than the spatial multi-scaled ones for pseudo images purpose; 2. although distance normalization is conceptually useful for weight estimation, simulations show that normalization is not always needed for producing good results; 3. our method does not require a priori information about the data distribution in the database, which not only reduces the computational complexity but also makes it more suitable for searching in a distributed networking environment. In our previous work [18] , we have conducted a preliminary experiment to evaluate the proposed method against a 1050-image database. The results show that the multi-instance user perception weighting method is promising, and the pruning concept always improves the query accuracy in our method; also, the pseudo-image concept improves the accuracy in many cases.. In this paper, we extend the evaluation process to a much larger scale. We collect 38 sets of outdoor scenic images as the ground truth. They are similar in terms of low-level descriptions. We prepare the ground-truth images as follows: each set of ground-truth images is taken on the same spot with slightly different camera pan and tilt angles by hand. The size of a ground-truth set varies from 4 to 10. Images in each set are perceptually similar. However, by examining the low-level features, we observe that the feature values can be quite different. There are several possible causes. The first is that these pictures are taken by hands. They are inevitably somewhat shifted and blurred. The second is that different shots have slightly different focus and shutter speed. The third is that photos with shooting angle variation may have different background lighting, which may change the shade of each picture.. Our experiments simulate a typical image query scenario. A user first chooses one or a few "similar" input images to start a query. The matching process returns an ordered list of results; we call it the positive-only query result. If the result is not perfect; that is, not all ground-truth images occupy the highest ranks, or simply NMRR 0, then the highest ranked non-ground-truth image is assigned as the negative feedback item. Then, we repeat the query process with both positive and negative images and produce the positive-and-negative query result. If the positive-only result is perfect, both NMRR positive-only and NMRR positive-and-negative are set to zero. Since the smallest ground truth set has only four images, we simulate the conditions of one to three positive images per query. All possible combinations of images in all ground truth sets are tested to derive the ANMRR values.. Two multi-scale schemes are tested: spatial and SNR. The spatial scaling factor (for both width and height) for each down-sampled image is defined as follows: the n-th scale factor (for the n-th pseudo image) is α − 0.1(n − 1), where n = 1, 2. We perform experiments at α = 0.9, 0.8, 0.7, 0.6, 0.5 to look for the best parameter values that lead to the best ANMRR. The SNR-scaled images are generated by applying JPEG compression with a quality factor of β − 0.1(n − 1) for the n-th scaled version. The test values are β = 0.7, 0.6, 0.5, 0.4, 0.3.. To examine the effectiveness of our method, we simulate another two weighting schemes under the same assumptions. The first scheme is a variation derived from the MARS system. In this scheme, the distance metric d j ( f 1 , f 2 ) for each feature F j is normalized as follows:. where µ j and σ . The final step clamps all calculated distance values between zero and one.. The original MARS system adopts a 5-level relevance feedback. To make it comparable with our simulation environment, we reduce the relevance feedback levels to three: positively relevant (Score l = +1), no opinion (Score l = 0), and negatively relevant (Score l = −1). The weighting process is similar to that in the original MARS. Assume the overall query result list is RT , and the result list of feature F j is RT j . To calculate the weight w j , we first initialize W j = 0, and then update W j as follows:. for each item l which appears in both RT and RT j .. After all W j have been updated, the negative weights are adjusted to 0 (means "irrelevant" in MARS). Then, we compute the weighting factor for each feature F j as. A final remark about this MARS-like scheme is the RT list. According to the original proposal [2] , it is an iterative procudure that leads to the "optimal" RT . The original proposal selects P f d = 3 as the maximum number of iterations and shows good convergence in general. In our simulation, we set P f d = 5. This is called Scheme A in the rest of this section.. The second scheme we simulate has the same basic structure as our proposed scheme in Sect. 3. However, it adopts the same distance normalization in MARS. This is Scheme B. We simulate this scheme for two reasons. One is to compare with a MARS-like scheme to see the effects of different weighting estimation procedures. The other is to compare it with our scheme to see the effects of different distance normalization methods. Our scheme is labeled as Scheme C.

O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Designing

and

embedding

reliable

virtual

infrastructures

In

a

virtualized

infrastructure

where

physical

resources

are

shared,

a

single

physical

server

failure

will

terminate

several

virtual

servers

and

crippling

the

virtual

infrastructures

which

contained

those

virtual

servers.

In

the

worst

case,

more

failures

may

cascade

from

overloading

the

remaining

servers.

To

guarantee

some

level

of

reliability,

each

virtual

infrastructure,

at

instantiation,

should

be

augmented

with

backup

virtual

nodes

and

links

that

have

sufficient

capacities.

This

ensures

that,

when

physical

failures

occur,

sufficient

computing

resources

are

available

and

the

virtual

network

topology

is

preserved.

However,

in

doing

so,

the

utilization

of

the

physical

infrastructure

may

be

greatly

reduced.

This

can

be

circumvented

if

backup

resources

are

pooled

and

shared

across

multiple

virtual

infrastructures,

and

intelligently

embedded

in

the

physical

infrastructure.

These

techniques

can

reduce

the

physical

footprint

of

virtual

backups

while

guaranteeing

reliability.

With

infrastructure

rapidly

becoming

virtualized,

shared

and

dynamically

changing,

it

is

essential

to

have

strong

reliability

support

from

the

physical

infrastructure,

since

a

single

physical

server

or

link

failure

affects

several

shared

virtualized

entities.

Providing

reliability

is

often

linked

with

Permission

to

make

digital

or

hard

copies

of

all

or

part

of

this

work

for

personal

or

classroom

use

is

granted

without

fee

provided

that

copies

are

not

made

or

distributed

for

profit

or

commercial

advantage

and

that

copies

bear

this

notice

and

the

full

citation

on

the

first

page.

To

copy

otherwise,

to

republish,

to

post

on

servers

or

to

redistribute

to

lists,

requires

prior

specific

permission

and/or

a

fee.

Copyright

(c)

ACM

This

is

a

minor

revision

of

the

work

published

in

ACM

SIGCOMM

Workshop

over-provisioning

both

computational

and

network

capacities,

and

employing

load

balancing

for

additional

robustness.

Such

highly

reliable

systems

are

good

for

applications

where

large

discontinuity

may

be

tolerable,

e.g.

restart

of

network

flows

while

re-routing

over

link

or

node

failures,

or

partial

job

restarts

at

node

failures.

A

higher

level

of

fault

tolerance

is

required

at

applications

where

some

failures

have

a

substantial

impact

on

the

current

state

of

the

system.

For

instance,

virtual

networks

with

servers

which

perform

admission

control,

scheduling,

load

balancing,

bandwidth

broking,

AAA

or

other

NOC

operations

that

maintain

snapshots

of

the

network

state,

cannot

tolerate

total

failures.

In

masterslave/

worker

architectures,

e.g.

MapReduce,

failures

at

the

master

nodes

waste

resources

at

the

slaves/workers.

Through

synchronization

and

migration

techniques

on

virtual

machines

and

routers,

we

postulate

that

fault

tolerance

can

be

introduced

at

the

virtualization

layer.

This

has

several

benefits.

Different

levels

of

reliability

can

be

customized

and

provisioned

over

the

same

physical

infrastructure.

There

is

no

need

for

specialized,

fault

tolerant

servers.

Instead,

redundant

(backup)

virtual

servers

can

be

created

dynamically,

and

resources

are

pooled

together,

increasing

the

primary

capacity.

Both

will

lead

to

a

better

overall

utilization

of

the

physical

infrastructure..

In

this

paper,

we

propose

an

Opportunistic

Redundancy

Pooling

(ORP)

mechanism

to

leverage

the

properties

of

the

virtualized

infrastructure

and

achieve

a

n

k

redundancy

architecture,

where

k

redundant

resources

can

be

backups

for

any

of

the

n

primary

resources,

and

share

the

backups

across

multiple

virtual

infrastructures

(VInfs)..

For

a

quick

motivating

example,

consider

two

VInfs

with

and

computing

nodes.

They

would

require

and

redundancy

to

be

guaranteed

reliability

of

and

respectively.

Sharing

the

backups

with

ORP

will

achieve

a

redundancy

of

k

with

the

same

levels

of

reliability,

reducing

the

resources

that

are

provisioned

for

fault

tolerance

by

at

most

In

addition,

there

is

joint

node

and

link

redundancy

such

that

a

redundant

node

can

take

over

a

failed

node

with

guaranteed

connectivity

and

bandwidth.

ORP

ensures

VInfs

do

not

connect

to

more

redundant

nodes

than

necessary

in

order

to

keep

the

number

of

redundant

links

low..

The

other

contribution

of

this

paper

is

a

method

to

statically

allocate

physical

resources

(compute

capacity

and

bandwidth)

to

the

primary

and

redundant

VInfs

simultaneously,

taking

into

account

the

output

of

the

ORP

mechanism.

It

attempts

to

reduce

resources

allocated

for

redundancy

by

utilizing

existing

redundant

nodes,

and

overlapping

band-widths

of

the

redundant

virtual

links

as

much

as

possible..

Our

paper

focuses

on

the

problem

of

resource

allocation

for

virtual

infrastructure

embedding

with

reliability

guarantee.

Practical

issues

such

as

system

health

monitoring,

protocol

design,

recovery

procedures,

and

timing

issues

are

out

of

the

scope

of

this

paper..

The

organization

of

this

paper

is

as

follows.

In

the

next

section,

we

briefly

describe

the

background,

notations

and

define

reliability

in

Section

Then,

we

describe

a

virtual

architecture

that

can

provide

fault

tolerance

and

estimate

the

benefits

of

sharing

redundancies

in

Section

We

see

how

the

link

topology

is

preserved

under

failures

in

Section

and

how

resources

can

be

efficiently

allocated

in

the

physical

infrastructure

in

Section

Finally,

we

evaluate

and

validate

the

ideas

through

simulation

in

Section

present

related

work

in

Section

Section

concludes

this

paper.

We

considered

the

problem

of

efficiently

allocating

resources

in

a

virtualized

physical

infrastructure

for

Virtual

Infrastructure

(VInfs)

with

reliability

guarantees,

which

is

guaranteed

through

redundant

nodes

and

links.

The

resource

allocation

approach

disregards

whether

the

redundant

nodes

are

passive

or

hot

standbys

so

long

as

ample

resources

are

available

when

recovering

from

failures..

Since

a

physical

infrastructure

hosts

multiple

VInfs,

it

is

more

resource

efficient

to

share

redundant

nodes

between

VInfs.

We

introduced

a

pooling

mechanism

ORP

to

share

these

redundancies

for

both

independent

and

cascading

types

of

failures.

The

physical

footprint

of

redundant

links

can

be

reduced

as

well,

by

considering

the

maximum

over

all

failure

scenarios

while

allocating

resources

with

a

linear

program

adapted

from

the

Multi-Commodity

Flow

problem.

Both

mechanisms

have

significant

impact

in

conserving

resources

and

improving

VInf

acceptance

rates.

Network

virtualization

is

a

promising

technology

to

reduce

the

operating

costs

and

management

complexity

of

networks,

and

it

is

receiving

an

increasing

amount

of

research

interest

Reliability

is

bound

to

become

a

more

prominent

issue

as

infrastructure

providers

move

toward

virtualizing

their

networks

over

cheaper

commodity

hardware

Analysis

on

the

reliability

of

overlay

networks

in

terms

of

connectivity

in

the

overlays

has

been

developed

Unfortunately,

it

is

not

applicable

to

our

problem

as

we

are

concerned

with

critical

virtual

nodes

and

embedding

them

as

an

entire

infrastructure

with

reliability

guarantees..

Fault

tolerance

is

provided

in

data

centers

through

excessive

redundant

nodes

and

links

organized

in

a

special

way.

These

works

provide

reliability

but

do

not

customize

reliability

guarantees

to

embedded

VInfs..

While

we

are

not

aware

of

works

studying

the

allocation

of

reliable

virtual

networks,

considered

the

use

of

"shadow

VNet",

namely

a

parallel

virtualized

slice,

to

study

the

reliability

of

a

network.

However,

such

slice

is

not

used

as

a

back-up,

but

as

a

monitoring

tool,

and

as

a

way

to

debug

the

network

in

the

case

of

failure.

considered

the

use

of

virtualized

router

as

a

management

primitive

that

can

be

used

to

migrate

routers

for

maximal

reliability..

Meanwhile

some

works

address

node

fault

tolerance

at

the

virtualization

level.

Bressoud

was

the

first

few

to

introduce

fault

tolerance

at

the

hypervisor.

Two

virtual

slices

residing

on

the

same

physical

node

can

be

made

to

operate

in

sync

through

the

hypervisor

on

the

same

physical

node.

Others

have

made

progress

for

the

virtual

slices

to

be

duplicated

and

migrated

over

a

network.

Various

duplication

techniques

and

migration

protocols

were

proposed

for

different

types

of

applications

(web

servers,

game

servers,

and

benchmarking

applications)

Remus

and

Kemari

are

two

other

systems

that

allows

for

state

synchronization

between

two

virtual

nodes

for

full,

dedicated

redundancy.

However,

these

works

focus

on

the

practical

issues,

and

do

not

address

the

resource

allocation

issue..

VNsnap

is

another

method

to

take

static

snapshots

of

an

entire

virtual

infrastructure

to

some

reliable

storage,

in

order

to

recover

from

failures.

This

can

be

stored

as

reliably

in

a

distributed

manner

as

replicas

or

as

parities

VNsnap

does

not

address

synchronization,

nor

guarantee

sufficient

resources

for

recovery

from

snapshots..

Fundamentally,

there

are

methods

to

construct

topologies

for

redundant

nodes

that

address

both

nodes

and

links

reliability

Based

on

some

input

graph,

additional

links

are

introduced

such

that

the

least

number

is

needed.

However,

a

node

failure,

in

this

case,

may

involve

migrations

among

the

remaining

nodes

to

preserve

the

original

topology.

This

may

not

be

suitable

in

a

scenario

where

migrations

may

disrupt

other

running

parts

of

the

network.

Our

problem

involves

virtual

network

embedding

with

added

node

and

link

redundancy

for

reliability.

Our

model

employs

the

use

of

path-splitting

which

allows

a

link

to

be

split

over

multiple

routes

such

that

the

aggregate

flow

across

those

routes

equal

to

the

demand

between

the

two

nodes.

This

gives

more

resilience

to

link

failures

and

allows

for

graceful

degradation.

A

related

work

that

does

not

use

path-splitting

for

embedding

reliability

is

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O


A network hierarchical feedback system for Taiwanese universities based on the integration of total quality management and innovation Fuzzy analytical hierarchical process (FAHP) Fuzzy analytical network process (FANP) Decision-making trial and evaluation laboratory (DEMATEL) Gray relational analysis (GRA) Network hierarchical feedback system (NHFS) a b s t r a c t An increasing number of Taiwanese universities are improving operational performance through innovation and total quality management (TQM). In addition, the National Quality Award (NQA), which is based on TQM, is now used to evaluate quality performance in various industries in Taiwan. Thus, several models for performance measurement have been proposed in recent years. However, these models do not take into account several features germane to performance within the Taiwanese university system, such as characteristics unique to the integration of TQM and innovation, comprehensive focuses in operational performance improvement across different types of universities, and interrelations among the different variables used to measure performance. Thus, precisely measuring and improving operation performance has proven to be a difficult task. The aim of this paper is to construct a network hierarchical feedback system (NHFS) based on the integration of TQM and innovation to overcome these problems. To that end, we adopted a decision-making trial and evaluation laboratory (DEMATEL) method to address the complex, interdependent relationships among the variables and thereby construct a relation structure among the measurement criteria for evaluation purposes. A fuzzy analytic network process (FANP) is employed to overcome the problem of dependence and feedback among each of the TQM measurement criteria. A fuzzy analytic hierarchical process (FAHP) is used to evaluate the measurement criteria for innovation performance. Lastly, a gray relational analysis (GRA) is utilized to find optimal alternatives. The value of this study comes from providing all types of universities in Taiwan the most complete evaluation system of operational performance, as well as opportunities to realize improved competitive advantages and enhanced prospects for survival. A country's higher education system fosters high-tech talent, which is the key contributor to rising national quality and one of the main ways to upgrade national competitiveness [21, 39] . Based on records from the Taiwanese Ministry of Education in Taiwan, the number of universities in Taiwan has increased to 157 and continues to rise [40] . These universities' levels of quality and operational performance, however, have not increased equally across the board [47] . In addition, with birth rates continuously dropping, the number of universities increasing, and Taiwan joining the World Trade Organization (WTO), the competitive advantages of a strong university system have decreased drastically [10] . Thus, these problems are serious issues for both governments and universities [17] .. The main method of improving and measuring the operational performance of universities involves measuring innovation performance [8] and total quality management (TQM) performance. These two factors have gained significant attention in recent years in Taiwan. A growing number of studies have started to develop separate models for measuring innovation and TQM. However, we argue that a model that addresses both TQM and innovation is necessary for Taiwanese universities to gain competitive advantages and thereby ensure their future survival. In this paper, we propose a measurement system for Taiwanese universities based on the integration of TQM and innovation.. Although the criteria used for the measurement of TQM [34,43] and innovation [8] are numerous, these criteria seem to ignore the characteristics unique to the integration of TQM and innovation. In addition, Taiwanese universities can be categorized into three main types: the research-intensive university (RU), the teachingintensive university (TU), and the professional-intensive university (PU). Another university type, the education-in-practice-intensive 1568-4946/$ -see front matter © 2012 Elsevier B.V. All rights reserved. doi:10.1016/j.asoc.2012.03.003 university (EIPU), has recently emerged. It focuses on operations performance improvement while conducting TQM and innovation independently from one another [9] . In addition, the measurement criteria proposed in the literature are assumed to be independent of each other, even though this separation does not reflect real world circumstances [38] . To maximize the evaluation and improvement of operational performance in Taiwanese universities, measurement criteria must consider characteristics related to the integration of TQM and innovation, the comprehensive focuses across the four types of universities in terms of operational performance, and the interrelationships among the criteria.. To overcome these problems and construct a useful model, a decision-making trial and evaluation laboratory (DEMATEL) method is used to address the complex interdependent relationships of TQM and to construct a relation structure that includes the measurement criteria for evaluation purposes. A fuzzy analytic network process (FANP) is employed to overcome the problems of dependence between and feedback among TQM measurement criteria. A fuzzy analytic hierarchical process (FAHP) is used to evaluate the measurement criteria for innovation performance based on the effects of TQM criteria. The FANP and FAHP methods are widely used for multiple-criteria decision making, and the practical applications reported in the literature have demonstrated related advantages with regard to handling unquantifiable/qualitative criteria and obtaining reliable results [6] [7] [8] . Finally, gray relational analysis (GRA) is utilized to find optimal alternatives based on weights of innovation criteria. Unlike the overly subjective nature of the simple average weighted method (SAW), the GRA method has been adopted largely due to its proficiency in uncertainty information management and its simple calculations [53] . Hence, we combine the DEMATEL, the fuzzy ANP, the fuzzy AHP and the GRA approaches to construct a network hierarchical feedback system (NHFS) based on the integration of TQM and innovation.. The remainder of this paper is organized as follows. An overview of total quality management and innovation is discussed in Section 2. Research methods are proposed in Section 3. Our empirical study is detailed in Section 4. Conclusions are discussed in the last section.

O O O O O O O O O O O O O O O O O O B I I I I B I I I I B I I I I I B I I I B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O B I I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Real-time document collaboration using iPads This paper presents a real-time collaborative interface for reading groups. We exploit the light and portable properties of iPad hardware to facilitate an intimate interaction experience that gives users the ability to simultaneously mark up their own copy of a document, while instantly viewing notes made by other members of the group. We also introduce a tool that allows group members to quickly 'point out' information within a document to other members without physically gesturing with their hands. We finish with a short explanation of a study proposed to test the implemented system. Reading is not usually considered a social activity -being engrossed within your own copy of a novel does not promote collaboration. However, although it is unusual for several people to sit and read a book together, readers will often meet to talk about their experience and understanding of books that they have read. This is seen, for example, in study groups and book clubs.. Paper is the traditional and established medium for reading. When reading in a group environment, users will often make use of individual copies of the same document which, in itself, has clear advantages, e.g. everyone within the group gets to see and make notes on their own copy of the document. These advantages are seen particularly with individual articles (perhaps downloaded and printed), but even with large books, the medium is readily manipulated and does little to impede the social interaction between people. The readers can sit in whatever space is available, without being constrained by the technology, and with minimal physical intrusion from the reading medium. While these benefits enhance the collaborative experience, however, they do give rise to some practical problems. For example, referencing a specific section can be tricky -the reader who is guiding their colleagues to a particular place must identify its physical location, and this can give rise to co-ordination problems. A second issue is that the readers' individual notes are separate and sharing ideas can lead to considerable duplication of effort.. Using a single shared resource has limitations, whether using a computer or paper. Collaborative web search is a task closely related to collaborative reading, and research has shown that problems arise with both the human-computer interaction and the use of physical space [2] when using a single PC. Desktop computers restrict the physical space, and this limits the area available for co-ordination between users, and their collective control of the primary (search) task. Using a single resource is noted as encouraging a hierarchical group dynamic, with one user dominating.. Recently, computer technologies have multiplied, and there are large interactive displays that overcome some of the physical constraints of the relatively small display of a traditional desktop or laptop PC. This physical layout has also been examined in the context of collaborative search, covering interactive whiteboards, table-top PCs and similar devices [5, 9, 13] . These technologies can promote simultaneous collaboration, and permit everyone to see a shared display that co-ordinates the group's view of the media being worked with -i.e. the referencing problem is by-passed through the use of a single view. However, this does not give each person an individual view. This paper introduces a collaborative annotation system that uses a set of individual iPads. The size and shape of the hardware resembles clipboards and allows users the space to sit around the same desk while at the same time giving them their own document to work on. The wireless connectivity of the devices allows users to contribute easily (and not necessarily vocally) via real-time changes to all iPads in the group. This method not only facilitates simultaneous annotation, but also keeps users up to date on all notes made within the session. It supports both individual and group reading, as naturally arises when readers have their own individual printed text. The following sections describe the background and design of such a system along with a short conclusion of the expected benefits. We have created a system that encourages a close working environment by combining portable digital annotation devices with a real-time digital collaboration tool. The iPad hardware and software architecture have been carefully selected to give users their own space to work, while providing them access to the activities of others and also the ability to remotely point out useful pieces of information [12] . This paper has explored a new interaction method for group collaboration. By incorporating the sleek design of iPad hardware with real-time working group annotation, we are able to incorporate the versatility and portability of paper with the instant sharing possibilities of the Internet.. The main features of our new system are:. 1. Hardware interaction: iPads are sleek, portable and can be held in the same way as a clipboard and paper; One area of the system that we have not yet considered, is the possibility of interconnecting it with other types of device. Our individual displays may be combined with the shared displays mentioned in Section 1. Morris et al [10] suggest that co-located collaboration tends to include a separate input device for each group member but sometimes incorporates a shared display. This idea could easily extend to our implementation if we could incorporate other devices into the system. For example, as Morris suggests, we could add to the system of iPads, a shared display in the form of projection or table top to add more diversity into the working group and aid in collaboration. Alternatively, not everyone in the group need work on iPads: e.g. one member of the group, perhaps the team leader, may require a larger screen than the other members and will therefore want the system to be executable on a desktop machine instead. Another area of future study is to test our implementation on several sets of collaborative groups and observe their behavior with this system as well as their usual methods.. The design for this study will start by the selection of an appropriate group of end users: ideally, a group of people who collaborate often either on paper or computers. First then, to form a basis for comparison, we will sit in on one of their regular collaborative sessions and observe their current practices. For example, what equipment do they use? Do they have a single collaborative document or one for every member? How do they make notes? How do they point things out to one another? How many members of the group participate? Are there any dominant members? How many notes were made in total?. Next, we will introduce them to our iPad implementation and sit in on another session where we will observe the behaviors and techniques using the new technology. What we will be looking for here is the differences between the two media. For example, was there a difference in the number of marks made between the systems? Did some users contribute more or less with the new system rather than the old one? Were any problems encountered?. Finally to round off the study, we will conduct a series of individual 3 interviews with the participants probing their preferences for each system.. The information gathered from this study will hopefully shed some light on the success of the current system. Firstly, we would hope that users like the idea of the implementation: i.e. does having personal space to work aid in the collaborative process? Is having everyones marks on the same artifact useful? And was it easy to point out information to others? Furthermore, we would also hope that the system as a whole actually encourages collaboration: i.e. are more notes being made and/or are less dominant members of the group contributing more than they originally did. This study will hopefully address these issues and perhaps even open up new avenues of future work in the area of realtime collaboration. Before we discuss the design for our system, it would be useful to take a look at similar collaborative systems to gain a better understanding of the techniques used for information sharing within working groups.. The work in [6] describes the importance of real-time collaboration tools for digital ink applications. It tests commercially available peer-to-peer ink-enabled software with penbased interaction tools in a bid to identify the most efficient features. The hardware used in the study consisted of pen tablet monitors with separate CPU and peripherals and "hybrid" tablets with detachable keyboards: two devices that require a large amount of desk space and are significantly immobile. The main conclusion drawn from this study was the challenge and importance of facilitating deixis: a ubiquitous technique whereby an idea is demonstrated by speaking about it and pointing to it simultaneously.. This now leads us onto the problem of shared references [7] : i.e. if one person refers to something specific in a document how does everyone in the discussion know they are looking at the same thing? This can be achieved (somewhat clumsily) in paginated documents like books by pointing out the section with your finger, e.g. "go to page 22, paragraph 4", but can prove complicated if people in the group are reading different editions or versions of the text. This method of referencing can also cause problems for electronic documents that lose their sense of location, such as eReaders where the page numbers alter depending upon the zoom level on the device, or documents in HTML format. Clearly then, using page numbers in this context is not a consistent method of referencing. It is for this reason, that the collaborative iPad system will make use of document locations as opposed to pages much like the format used by the Amazon Kindle software. (See Section 3.3 for details).. [11] investigates face-to-face or co-present group collaborations and formulated the model known as Single Display Groupware (SDG). This model is defined as a single user interface that allows simultaneous collaboration by multiple users via numerous input devices. There have been several attempts to improve the collaborative process by incorporating multiple input devices on the same display. [2] , proposes a system to improve the often solitary task of websearch via a co-located collaboration group. The system known as CoSearch uses readily available devices such as mice and mobile phones to aid in traditional shared computer tasks. The main benefit of the system was that although the group members were gathered around the same computer, they each had facility to control it while at the same time maintaining group communication and awareness. Thus, all members of the group could a) see the same collaborative document and b) make changes to it. This is similar in part to what we are trying to achieve with our iPad system. The difference here is the interaction side: we want all users to feel they are working on their own copy of a document (hence the individual iPads) while at the same time allowing them real-time access to others notes.. Whilst much of the research on group collaboration focuses on table top displays and SDG, some uses more portable 'paper-like' devices such as tablet PCs. Hinckley et al [4] proposed a novel collaboration technique that connects together two tablet PCs fitted with physical sensors. The sensors are used to identify switches between shared and individual tasks, and to identify physical gestures that trigger document sharing.. There are also several commercially available systems that focus on collaborations on Apple portable hardware. CrowdMap from Barking Minds 1 , allows users to build mind maps collaboratively on their iPads or iPhones. The tool, which operates in real-time, has been designed for the organization and sharing of mind maps within working groups. Another system that makes use of 'live' collaboration using iPads and iPhones is letsannotate 2 that allows users to view and annotate PDF files. The portable iPad version is currently only in its beta stage, and does not run as an application, but through a web browser.

O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Incremental volume rendering using hierarchical compression AbstractFactorisation algorithm by a factor of more than two, while maintaining good rendering performance. These factors make our octree algorithm more suitable for implementation on average desktop workstations for the purposes of interactive exploration of volume models over a network. Results from tests using typical volume datasets will be presented which demonstrate the ability of the algorithm to achieve high rendering rates for both incremental rendering and standard rendering while reducing the runtime memory requirements.Keywords: incremental, volume rendering, octree, Shear-WarpWith the rising popularity of the Internet and the ever increasing range of data available through systems such as the World-Wide-Web, new methods of transferring and displaying various data types become necessary. Specifically, volumetric datasets, such as those produced by MRI or CT scanning methods, will be available for interactive exploration.Due to the extreme size of these datasets their reception over a low bandwidth network connection is time consuming. This has lead previous researchers only to transmit static images [1], and to perform all the rendering on the server. However in a World-Wide-Web environment the amount of potential simultaneous accesses to this data could be very high, and it would then be more efficient to perform the rendering on the client's workstations. This of course requires the transmission of the entire volume dataset to the client and given that this is a time consuming operation, it would be necessary for the client to render the data incrementally as it arrives in order to achieve reasonable user feedback. Our algorithm builds a compressed hierarchical representation of the volume data using an octree which may then be transmitted to a client (Section 3). On reception of this data the client may incrementally classify (Section 4) and render the data (Section 5) using a modified Shear-Warp Factorisation algorithm. Due to the fact that the incremental classification and rendering times are in the order of seconds, while the network transmission times are in the order of 10 minutes or more, multiple incremental renderings may be performed of the data as it arrives, giving the user the ability to rapidly examine overall characteristics of the data. The result of our method is a rapid incremental volume renderer which reduces the memory requirements of previous direct volume rendering algorithms.2.Our algorithm falls into the lossless compression category (i.e. all information content of original is maintained) and is also notable for its ability to render the data in its compressed state, therefore not incurring the decompression overhead.An early lossless and highly used approach is that of run-length encoding of the data. Montani and Scopigno [2] defined the STICKS representation scheme, which performed run-length encoding along a set axis (typically the z axis). While providing better overall compression ratios than a pure octree method it is unfortunately not symmetrical in 3-space and therefore the memory access patterns change dramatically depending on the traversal order in the volume. Wilhelms and Van Gelder [3] have performed a lot of work in the area of hierarchical data representation using variants of the basic octree method. Their research has indicated that hierarchical representation provides flexibility, and while not providing optimal compression, allows the compressed version to be used almost as quickly as the original as well as providing the ability to naturally represent the data in reduced accuracy. Ning and Hesselink [4] propose the use of vector quantization on blocks of voxels to generate a compressed volume which may be rendered in its compressed state. Their scheme does not however maintain the original content exactly and their rendering rates are fairly low. Other approaches include the truncated Huffman coding used by Fowler and Yagel [5], and the DCT based scheme of Yeo and Liu [6], but in both cases there is no ability to render the data in its compressed state.The rendering of volumetric datasets historically consisted of isosurface methods such as Lorensen and Cline's [7] Marching Cubes Algorithm, and our Mesh Propagation Algorithm (Howie and Blake [8]). Recent work has concentrated on direct volume rendering, with approaches such as ray-casting and splatting methods. Laur and Hanrahan [9] outline an algorithm for progressive refinement rendering of volume data using hierarchical splatting. Their algorithm, while also using octrees, makes use of dedicated polygon rendering and compositing hardware to achieve efficiency.Incremental rendering, or rendering at lower resolutions, has also been achieved in the past through frequency space manipulations of the data. Muraki [ 10] and Westermann [ 1 13 propose a wavelet decomposition of the data, which allows for a multiresolution representation of the data and Totsuka and Levoy [12] propose a frequency domain volume renderer. These methods, while allowing for facilities such as edge enhancement and reducing run-time data size, are still too complex to be considered for implementation on average desktop workstations.Most recently the Shear-Warp rendering algorithm was presented by Lacroute and Levoy [ 13], which allows for near real-time rendering on an average workstation. Their algorithm makes use of a run-length encoding technique similar to the STICKS scheme mentioned above, except that the compression must be in a direction orthogonal to the viewing direction. Due to the dependence of the compression direction on viewing angle, three versions of the compressed volume have to be maintained in memory to achieve acceptable rendering rates.In order to achieve an incremental rendering scheme the data has to be represented in some hierarchical fashion where basic structure is first transmitted followed by refinements to the basic structure. Our algorithm implements the Shear-Warp rendering approach using an octree representation of the data which © Eurographics Association, 1996M. B. Haley et al. / Incremental Volume Renderingremoves the need to store three versions of the volume (due to the symmetry of the octree) while not having any significant impact on rendering time. The algorithm then makes use of the implicit structural information contained in the octree to provide the ability to render approximated areas of the volume when the physical data is not available. Unlike Laur and Hanarahan's [9] method, error terms are not used to determine which nodes in the octree to render as the objective here is to achieve the best possible image. This implies the use of the lowest possible nodes in the octree.In this section we deal with the initial compression of the data and the construction of the hierarchical data structure. The next two sections will cover the incremental phases of classification and rendering of the volume.The choice of data structure for the nodes in the octree is critical in achieving efficient traversal and in providing enough information to facilitate approximate rendering of entire sub-trees. Each node structure contains the following:An empty flag (denoting a region with all transparent voxels), a full flag (denoting a region which is not further subdivided as it is mostly opaque), or a child node reference (which references another eight nodes).A raw data reference (if leaf-node), as well as minimum, maximum, and average voxel values. Also the values of the eight comer voxels.Minimum, maximum, and average spatial gradients, as well as the average normal vector.One of the many advantages of the octree representation over the run-length encoding (RLE) representation is that the octree data structure contains references into the raw volume data and is completely separable from it. This implies that different octree structures may be used at the various processing stages. (e.g. It is not necessary to transmit the minimum and maximum gradient fields as they are calculated during classification.) By using a different octree structure during and after classification the transmitted data can be reduced.Our compression algorithm has four main stages: (1) correction of range of data values and reduction to 8-bit data; (2) initial construction of octree; (3) reordering of octree; and (4) compression of leaf nodes.In the first stage the three dimensional input array of scalars is converted into an array of eight bit values by selecting the range of "usefil" values in the original dataset and then mapping these values into an eight bit range. This process is akin to the contrast stretch algorithm used in image processing. The initial range is selected by heuristics, working with the histogram of the original data, and is then refined through user interaction.In the second stage, minimum and maximum threshold values are selected which allow the construction algorithm to discard data which is not relevant (e.g. air spaces, or filtering out of low-density tissue). The construction process begins by recursively subdividing the volume into eight sub-volumes. At each level, if all child nodes either all contain relevant data or are all empty, these nodes are gathered together into one parent node. The leaf nodes then either reference the raw data for their sub-volumes or have flags marking them as empty. All higher nodes in the octree store the averaging information for the nodes below.Thirdly, once the octree is constructed, the nodes which are stored in a linear array in memory are then reordered such that nodes higher up in the octree occur before nodes further down in the octree. This is to allow for the incremental transmission of nodes. Finally, to produce more efficient compression during the impending transmission of this data, the data at the leaf nodes is compressed using run-length encoding and is stored in the order of node occurrence after the array of node structures. It should be noted however, that this node compression is only used during the transmission of the volume, and it should be decompressed on reception if the advantage of using the symmetrical octree data structure is to be maintained.Tests were performed at a variety of minimum threshold levels (the maximum threshold level was always set to the maximum value), for both our octree method and the pure run-length encoding method. The maximum octree depth was set to 6 in all cases as this was found to give optimal memory and processing performance. The datasets used are typical medical datasets as well as an engine block dataset (their sizes are given in the table below). Table 1 presents both the resultant data size (in bytes) after compression using the octree method (in bold) and the size after pure run length compression for each test case. From these results it is clear that the octree algorithm achieves compression ratios very close to those of pure run-length encoding even though an entire octree data structure containing averaging information is included in the octree data sizes. (Note these values do not represent the run-time memory sizes of the algorithms during rendering.) The duration of the entire octree construction process is in the order of 15 seconds, but this is irrelevant in the entire system as this operation only needs to be performed once by the volume server. With the rising popularity of the Internet and the ever increasing range of data available through systems such as the World-Wide-Web, new methods of transferring and displaying various data types become necessary. Specifically, volumetric datasets, such as those produced by MRI or CT scanning methods, will be available for interactive exploration.. Due to the extreme size of these datasets their reception over a low bandwidth network connection is time consuming. This has lead previous researchers only to transmit static images [1] , and to perform all the rendering on the server. However in a World-Wide-Web environment the amount of potential simultaneous accesses to this data could be very high, and it would then be more efficient to perform the rendering on the client's workstations. This of course requires the transmission of the entire volume dataset to the client and given that this is a time consuming operation, it would be necessary for the client to render the data incrementally as it arrives in order to achieve reasonable user feedback. © Eurographics Association, 1996 Our algorithm builds a compressed hierarchical representation of the volume data using an octree which may then be transmitted to a client (Section 3). On reception of this data the client may incrementally classify (Section 4) and render the data (Section 5) using a modified Shear-Warp Factorisation algorithm. Due to the fact that the incremental classification and rendering times are in the order of seconds, while the network transmission times are in the order of 10 minutes or more, multiple incremental renderings may be performed of the data as it arrives, giving the user the ability to rapidly examine overall characteristics of the data. The result of our method is a rapid incremental volume renderer which reduces the memory requirements of previous direct volume rendering algorithms. Users of a networked workstation can make use of this algorithm to perform interactive exploration of volumetric data over a typical Internet connection. Where data transmission rates are low, the algorithm provides for rapid incremental rendering of the data as it is received by the workstation. This allows the initial volume classification and rendering to be performed during the transmission period such that efficient user feedback is maintained. The octree rendering algorithm achieves rendering rates which are comparable Tests were performed at a variety of minimum threshold levels (the maximum threshold level was always set to the maximum value), for both our octree method and the pure run-length encoding method. The maximum octree depth was set to 6 in all cases as this was found to give optimal memory and processing performance. The datasets used are typical medical datasets as well as an engine block dataset (their sizes are given in the table below). Table 1 presents both the resultant data size (in bytes) after compression using the octree method (in bold) and the size after pure run length compression for each test case. From these results it is clear that the octree algorithm achieves compression ratios very close to those of pure run-length encoding even though an entire octree data structure containing averaging information is included in the octree data sizes. (Note these values do not represent the run-time memory sizes of the algorithms during rendering.) The duration of the entire octree construction process is in the order of 15 seconds, but this is irrelevant in the entire system as this operation only needs to be performed once by the volume server.. The RLE data still has to be transposed into three different volumes which was found to take roughly 8 seconds.. The transmission time (over a slow network link) could be in the order of 10 minutes or more so the initial classification may be amortised over the transmission time of the volume. This is not possible with the RLE data. The octree overhead is however significant on further reclassifications.. Plates 1, 2, and 3" depict the results of rendering the MR Brain volume at various levels of approximation (given by the amount of data available). It is clear that the image quality rapidly improves as lower levels of the octree become available. Figure 3 shows the results of parallel projection rendering of the MR Brain volume at various threshold levels. The times at each level are calculated by averaging the rendering times of the volume at a variety of rotations. The octree algorithm improves on the RLE algorithm for higher levels as the octree is Figure 3 : Times at various levels for full and partial octree completely omitting larger areas of the volume. Performance of the octree algorithm rendering compared to normal RLE rendering. is seen to be comparable for both full rendering and partial rendering of the volume. (The partial rendering above is such that every Table 3 , presents the run-time memory usage (in bytes) of each of the algorithms for the four test volumes at threshold levels of 20 and 160. The octree algorithm is a lot more memory efficient. Classification of volume data primarily requires the estimation of a spatial gradient at every voxel and the calculation of a normal vector. This process is generally performed using finite differences between sets of neighbouring voxels. In our octree scheme this poses a problem (we use first order differences) as the referencing of voxel neighbours can be time consuming (due to a tree traversal) when a voxel's neighbour lies in a different octree node to itself. Our algorithm alleviates this problem through the use of a octree node caching mechanism and an efficient cache search method. (See below.). The classification stage also involves the elimination of transparent areas of the volume given an opacity transfer function. (This function takes a voxel's parameters and returns an opacity.) In the original ShearWarp Factorisation method a Min-Max octree was constructed and used in conjunction with a summed-area table to make the classification process more efficient In our algorithm the octree data structure already closely resembles the Min-Max octree as it already contains the maximum and minimum values for the various voxels at any given node. Thus the overhead of constructing a Min-Max octree is eliminated using our octree method. Another advantage of the octree data structure is that the classification process may flag an entire sub-tree as transparent thus not referencing it at all during rendering. In the first stage the three dimensional input array of scalars is converted into an array of eight bit values by selecting the range of "usefil" values in the original dataset and then mapping these values into an eight bit range. This process is akin to the contrast stretch algorithm used in image processing. The initial range is selected by heuristics, working with the histogram of the original data, and is then refined through user interaction.. In the second stage, minimum and maximum threshold values are selected which allow the construction algorithm to discard data which is not relevant (e.g. air spaces, or filtering out of low-density tissue). The construction process begins by recursively subdividing the volume into eight sub-volumes. At each level, if all child nodes either all contain relevant data or are all empty, these nodes are gathered together into one parent node. The leaf nodes then either reference the raw data for their sub-volumes or have flags marking them as empty. All higher nodes in the octree store the averaging information for the nodes below.. Thirdly, once the octree is constructed, the nodes which are stored in a linear array in memory are then reordered such that nodes higher up in the octree occur before nodes further down in the octree. This is to allow for the incremental transmission of nodes.. Finally, to produce more efficient compression during the impending transmission of this data, the data at the leaf nodes is compressed using run-length encoding and is stored in the order of node occurrence after the array of node structures. It should be noted however, that this node compression is only used during the transmission of the volume, and it should be decompressed on reception if the advantage of using the symmetrical octree data structure is to be maintained.

B B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O