Binary partitions with applications to hidden surface removal and solid modelling We consider schemes for recursively dividing a set of geometric objects by hyperplanes until all objects are separated. Such a binary partition is naturally considered as a binary tree where each internal node corresponds to a division and the leaves correspond to the resulting fragments of objects. The goal is to choose the hyperplanes properly so that the size of the binary partition, i.e., the number of resulting fragments of the objects, is minimized. We construct binary partitions of size O(nlogn) for n edges in the plane, and of size O(n) if the edges are orthogonal. In three dimensions, we obtain binary partitions of size O(n2) for n planar facets, and prove a lower bound of n(n%). Two applications of efficient binary partitions are given. The first is an O(n2)-sized data structure for implementing a hidden-surface removal scheme of Fuchs, Kedem and Naylor [5] . The second application is in solid modelling: given a polyhedron described by its n faces, we show how to generate an O(n2)-sized CSG (consirudive-solid-ge~~e~r~) formula whose literals correspond to half-spaces supporting the faces of the polyhedron (see Peterson [9] and Dobkin et al. [3] ). The best previous results for both of these problems were O(n3). Recursive partitioning is a basic problem-solving technique which has proven to be extremely useful in algorithm design. For geometric problems where the input *The research was done while this author was visiting Xerox Palo Alto Research Center.. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. is a set of objects in the plane or in space, it is natural for this 'divide and conquer strategy' to be employed in such a way that the divide step is accomplished by making a linear cut of the input, that is, by splitting the objects along a line (in the 2-D case) or along a plane (in the 3-D case). This c'reates two subproblems which can then be divided recursively, again by linear cuts, until finally subproblems of some trivial size are obtained. Since each divide step may split some of the objects into several parts, the process described above can lead to a proliferation of objects and result in an inefficient algorithm. Thus, one is motivated to choose the dividing cuts carefully so that fragmentation of the input objects is kept to a minimum. We refer to the recursive partition mentioned above as a binary partition, and we are interested in the question of constructing binary partition trees whose size is not too large as a function of the original input size.. For example, in the three-dimensional formulation of our problem, we take the input to consist of a set of n non-intersecting triangular faces in R3, since triangular tiling is common for representing surfaces in space, Let p(n) be the maximum value over all inputs of size n of the size of a minimal binary partition, as measured by the number of internal nodes in the corresponding binary tree. (Precise definitions of binary partitions and p(n) are given in Section 2.) A straightforward upper bound for p(n) is O(n3). One of the main results of this paper is that p(n) = O(n2). A corresponding lower bound of p(n) = R(n1,5) follows from an example due to Thurston [lo] .. As applications of efficient binary partitions, we describe two well-known problems in computer graphics and show that solutions better than those previously known can be obtained readily from our results. The first problem arises in removing hidden surfaces in realtime. In some graphics applications such as flight simulation and computer animation, one needs to generate rapidly images of a 3-D scene as viewed from changing positions. A good strategy is to preprocess the scene suitably so as to simplify the hidden-surface computation at run-time. Fuchs, Kedem and Naylor [5] observed that, if the objects comprising the scene are represented as a binary partition tree, then traversal of the tree in a symmetric order relative to the viewing position will produce a correct priority order of (the fragments of) the objects for achieving the desired obscuring effect. In this scheme, storage space as well as tree traversal time is proportional to the size of the tree. The only previous bound known for the tree size is O(n3) [5] . As a direct consequence of our main theorem, this bound can be improved to O(n2).. The second application of binary partitions is found in converting boundary representations of three-dimensional objects into constructive-solid-geometry (CSG) representations. The boundary representation of an object specifies the surface elements forming its boundary; whereas the CSG representation expresses the interior of the object by a boolean formula where the operations are intersection and union and the literals are primitive solids such as boxes, cylinders, etc. It is important in solid modelling to be able to convert efficiently between the two styles of representation.. In a special type of CSG representation considered by Peterson [9] , the literals correspond to the half-spaces supporting faces of the given object. A natural question is: given a polyhedron described by its n faces, can one generate a short Peterson-style formula? A straightforward upper bound on the size of the formula is O(n3). We will show that our result on binary partitions implies an O(n') bound on formula size.. In Section 2, we give the definition and basic properties of binary partitions.. In Sections 3 and 4 partitions of size O(nlog n) for the planar case are presented.. Section 5 contains our main result, an algorithm to find partitions of size O(n2) in three dimensions, which is complemented by the Q(ns) lower bound of Section 6. In Section 7 we show that for orthogonal line segments in the plane a linear solution is achievable. The applications are discussed in Section 8, and we conclude with some comments and open problems in Section 9.

O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O B O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O B O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O B O O O O O O B O O B O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O


CONTROL OF PORT HAMILTONIAN SYSTEMS BY DISSIPATIVE DEVICES AND ITS APPLICATION TO IMPROVE THE SEMI-ACTIVE SUSPENSION BEHAVIOUR Abstract: The port Hamiltonian framework is a powerful tool for modeling a wide class of nonlinear systems such as robots and, more generally, mechatronic systems. A wide variety of mechatronic systems are controlled by operating dissipative components and the standard approaches for the control of port Hamiltonian systems are not applicable. Facing the limitation that the controlled devices can only dissipate power, the issue is to find a proper control law to satisfy the control requirements. This paper proposes to choose the control inputs to lead the input power of a subsystem in oder to satisfy the requirements by controlling the energy stored or the power dissipated in that subsystem. A slight extension of the definition of port Hamiltonian system is proposed to allow the description of a larger set of mechatronic systems. Although some important issues remain open, the example of the semi-active suspension shows that some positive results can be achieved by applying the proposed approach. Copyright From a mathematical perspective, the port controlled Hamiltonian systems (PCH) (van der Schaft, 2000) , are natural candidates to model many real systems, as shown in the application examples cited in (Ortega et al., 2002) . Basically, PCH are systems defined with respect to a geometric structure capturing the basic interconnection and dissipation laws, and a Hamiltonian function given by the total stored energy of the system. The control of PCH is an interesting research topic and outstanding results have already been obtained. The main results presented so far consider the possibility to operate on the power-ports of a system in order to obtain a controlled closed-loop system that is still a PCH with desired Hamiltonian function, interconnection laws and damping, see (Ortega et al., 2002) and the references therein. Another approach is the control by interconnection of PCH described in (Garcia-Canseco et al., 2005) . However many mechatronic systems are not controlled by means of the power ports but controlled by operating dissipative components such as variable resistors, variable dampers, clutches, some electro-valves and more. The issue is to find a proper control law that allows to satisfy the control requirements facing the limitation that the energy can only be dissipated by the controlled devices. To the best of our knowledge the problem of controlling a PCH by means of dissipative components has not already been addressed. The key idea proposed in this paper is to divide the PCH system into two or more PCH subsystems that are connected by a power preserving connection. The control inputs are then chosen to control the power flowing towards a certain subsystems or to control the stored energy in that subsystem. To this aim, a slight extension of the definition of PCH is proposed to allow the description of a larger set of mechatronic systems and to obtain an explicit representation of the power flowing to a subsystem. Thanks to the dissipative nature of the controlled devices the passivity properties of the given PCH system are preserved.. The semi-active vehicle suspensions (Savaresi et al., 2003) are an example of a mechatronic system with a controllable dissipative device. By following the proposed approach, some of the control laws already presented in literature for the semi-active suspensions are derived again and an energetic interpretation is given. Moreover a new control with improved performances is proposed. The paper is organized as follows: section 2 gives a brief introduction on Hamiltonian systems and extends the definition of PCH. The proposed control law for the dissipative components are presented in section 3. The application example to semiactive suspensions is described in section 4. The paper has addressed the problem of controlling port Hamiltonian systems by operating its dissipative terms. The key idea is to divide the Hamiltonian system into two or more subsystems that are connected by a power preserving connection. To this aim a slight extension of the definition of port Hamiltonian system has been proposed. The control inputs are then chosen to control the stored energy or the power dissipated of a certain subsystems. The paper presents only some preliminary results, many problems and questions remain open, however the semi-active suspension example has shown that some positive results can be achieved by applying the proposed approach. body acceleration (top) and jerk (bottom) for PDD(solid) and ADD(dashed) controls.

O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O


Stability Analysis of Nonlinear Multivariable Takagi-Sugeno Fuzzy Control Systems Abstract-This paper points out how the nonlinearities involved in multivariable Takagi-Sugeno (T-S) fuzzy control systems could originate complex behavior phenomena, such as multiple equilibrium points or limit cycles, that cannot be detected using conventional stability analysis techniques. In the paper, the application of MIMO frequency-domain methods to predict the existence of multiple equilibria and of limit cycles are presented. The proposed method is based on the formulation of a Lur'e problem from the original structure of a T-S fuzzy system with a fuzzy controller. Furthermore, this technique makes straightforward the application of input-output stability techniques such as the multivariable circle criterion, also called the conicity criterion, and the harmonic balance method. Moreover, in the paper, the application of the harmonic balance method has been generalized to the case of MIMO fuzzy system with asymmetric nonlinearities and improved by the decreasing conservatism. A new and more general stability index which could be used to perform a bifurcation analysis of fuzzy control systems is presented. The paper includes a collection of examples where the advantages of the proposed approach are made explicit comparing it to the input-output conicity criterion and the Lyapunov direct method.Index Terms-First harmonic method, input-output stability, Lyapunov methods, MIMO systems, TS fuzzy systems. Global stability analysis of fuzzy logic controllers is a complex problem. Some difficulties appear because the controller is designed to work well at the operating point, but for nonlinear systems this stability is only guaranteed around this point. Then, the system can be locally stable around the operating point but not globally stable, so some initial conditions or large enough perturbations can lead it out of control. However, knowing how the global stability can be lost, the problem can be at least minimized if not avoided altogether. This stresses the interest in the global analysis of fuzzy control systems.. Methods based on the qualitative theory of nonlinear dynamical systems deserve special mention. These methods are normally of a significant mathematical sophistication. However, as has been shown in this paper, there is a frequency response method, the harmonic balance method, well known by the control engineer, that allows to cope with global problems. It has been shown how to tackle with multiple equilibria and with limit cycles with the help of that technique, even in the case of MIMO systems with asymmetric nonlinearities. To do that, a new technique to decompose a T-S system as a linear and a feedback nonlinear part has been also presented. The method has been applied to additively decomposable T-S systems. Some examples are included where the advantage of the method has been pointed out. It has been shown that in cases where the conventional methods of Lyapunov and conicity fail to give any answer to the stability analysis, the proposed methodology is able to perform the analysis. Furthermore, a robustness measure has been introduced. This measure could be used as a stability index for bifurcation analysis.. Thus a true global method has been implemented. The conservatism in the application of the harmonic balance method has been reduced in two ways: 1) solving the balance equations in a less conservative way; and 2) by using all the structural information available in the fuzzy system. The method has some degree of approximation but it gives good results from a qualitative point of view. Nevertheless, it seems to suit well to the qualitative basis of fuzzy controllers.. The analysis included in this paper is restricted to continuous-time systems. Future research should include the study of discrete-time systems.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I B I B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


RSD:

A

Metric

for

Achieving

Range-Free

Localization

beyond

Connectivity

Abstract-Wireless

sensor

networks

have

been

considered

as

a

promising

tool

for

many

location-dependent

applications.

In

such

deployments,

the

requirement

of

low

system

cost

prohibits

many

range-based

methods

for

sensor

node

localization;

on

the

other

hand,

range-free

approaches

depending

only

on

radio

connectivity

may

underutilize

the

proximity

information

embedded

in

neighborhood

sensing.

In

response

to

these

limitations,

this

paper

introduces

a

proximity

metric

called

RSD

to

capture

the

distance

relationships

among

neighboring

nodes

in

a

range-free

manner.

With

little

overhead,

RSD

can

be

conveniently

applied

as

a

transparent

supporting

layer

for

state-of-the-art

connectivity-based

localization

solutions

to

achieve

better

accuracy.

We

implemented

RSD

with

three

well-known

algorithms

and

evaluated

using

two

outdoor

test

beds:

an

linear

network

with

MICAz

motes,

and

a

regular

network

covering

an

area

of

square

feet

with

motes.

Results

show

that

our

design

helps

eliminate

estimation

ambiguity

with

a

subhop

resolution,

and

reduces

localization

errors

by

as

much

as

percent.

In

addition,

simulations

confirm

its

effectiveness

for

large-scale

networks

and

reveal

an

interesting

feature

of

robustness

under

unevenly

distributed

radio

path

loss.

W

IRELESS

sensor

networks

(WSN)

have

been

proposed

for

many

location-sensitive

applications

such

as

battlefield

monitoring

environment

surveillance

object

tracking

etc.

In

addition,

many

routing

and

management

protocols

proposed

for

such

networks

are

built

on

the

assumption

that

geographic

parameters

of

sensor

nodes

are

available.

Although

node

localization

plays

an

important

role

in

all

those

systems,

itself

is

a

challenging

problem

due

to

the

severe

resource

constraints

at

each

low-cost

and

tiny

sensor

node..

Many

ideas

have

been

proposed

for

node

localization

in

WSN.

Based

on

whether

accurate

ranging

is

required,

there

are

basically

two

types

of

methods:

range-based,

and

range-free.

Range-based

methods

can

be

accurate,

but

costly

for

requiring

per-node

ranging

hardware

extensive

system

calibration

or

environment

profiling

which

may

not

be

possible

for

largescale

outdoor

sensor

networks.

Range-free

approaches

localize

nodes

based

on

simple

sensing,

such

as

wireless

connectivity

anchor

proximity

and

in-field

event

detections

Among

them,

solutions

applying

radio

connectivity

feature

a

low

system

cost,

however,

by

sacrificing

localization

accuracy.

Li

et

al.

made

early

contributions

to

improve

connectivity-based

methods

by

proposing

the

idea

of

sorted

radio

signal

strength

quantization

that

features

a

simple

system

design,

but

under

the

assumption

of

a

preknown

and

uniform

radio

range

throughout

the

network..

The

work

in

this

paper

is

motivated

from

the

finding

that

localization

by

means

of

mere

connectivity

may

underutilize

the

proximity

information

available

from

neighborhood

sensing.

Although

radio

signal

strength

(RSS)

is

considered

irregular

in

many

cases

for

unknown

radio

propagation

loss,

multipath

effects,

hardware

discrepancy,

and

so

forth,

our

empirical

study

shows

that

in

outdoor

open-air

scenarios,

RSS

weakens

approximately

monotonically

with

increasing

physical

distance,

especially

from

the

viewpoint

of

a

single

node,

which

might

provide

the

useful

information

about

which

neighboring

node

is

closer

and

which

is

further..

Following

this

finding,

we

propose

the

regulated

signature

distance,

or

RSD

for

short,

as

a

metric

of

proximity

among

neighboring

nodes.

RSD

brings

in

a

novel

range-free

approach

to

extracting

the

relative

distance

information

from

neighborhood

orderings

easily

obtained

from

RSS

sensing.

With

little

overhead,

RSD

can

be

conveniently

embedded

in

connectivity-based

localization

algorithms

to

improve

accuracy.

We

have

augmented

three

such

algorithms

(MDS-MAP

DV-Hop

and

RPA

with

RSD,

and

evaluated

in

two

types

of

outdoor

test

beds:

an

linear

network

with

MICAz

motes,

and

a

network

covering

an

area

of

square

feet

with

motes.

Results

show

noticeable

performance

gains

including

eliminating

ambiguity

and

reducing

localization

errors

by

as

much

as

percent.

Besides,

simulations

confirm

the

effectiveness

of

RSD

for

large-scale

networks,

and

reveal

an

interesting

feature

of

robustness

to

unevenly

distributed

radio

path

loss..

In

the

following,

Section

starts

with

empirical

data

analysis.

Section

gives

the

design

and

Section

reports

testbed

experiments.

Due

to

space

constraints,

we

provide

related

work

and

simulation

results

in

Appendices

and

which

can

be

found

on

the

Computer

Society

Digital

Library

at

respectively.

And

Section

concludes

the

paper.

B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A framework for recognizing the simultaneous aspects of American Sign Language The major challenge that faces American Sign Language (ASL) recognition now is developing methods that will scale well with increasing vocabulary size. Unlike in spoken languages, phonemes can occur simultaneously in ASL. The number of possible combinations of phonemes is approximately 1.5 × 10 9 , which cannot be tackled by conventional hidden Markov model-based methods. Gesture recognition, which is less constrained than ASL recognition, suffers from the same problem. In this paper we present a novel framework to ASL recognition that aspires to being a solution to the scalability problems. It is based on breaking down the signs into their phonemes and modeling them with parallel hidden Markov models. These model the simultaneous aspects of ASL independently. Thus, they can be trained independently, and do not require consideration of the different combinations at training time. We show in experiments with a 22-sign-vocabulary how to apply this framework in practice. We also show that parallel hidden Markov models outperform conventional hidden Markov models. Computers still have a long way to go before they can interact with users in a truly natural fashion. From a user's perspective, the most natural way to interact with a computer would be through a speech and gesture interface. Although speech recognition has made significant advances in the past 10 years, gesture recognition has been lagging behind. Yet, gestures are an integral part of human-to-human communication and convey information that speech alone cannot [20] . A working speech-and-gesture interface is likely to entail a major paradigm shift away from point-and-click user interfaces toward a natural language dialogue-and-spoken command-based interface.. Sign language recognition enters the picture in three ways. First, such a paradigm shift would leave those deaf people who depend on sign language as their primary mode of communication behind. There is a sense of urgency, because of the improvements in speech recognition. Unless we get sign language recognition to the same level of performance as speech recognition, accessibility of computers will become a major issue for the deaf.. Second, gesture recognition in itself is a difficult problem, because gestures are unconstrained, but gestures take place in the same visual medium as sign languages, and the latter possess a high degree of structure. This structure makes it easier to solve problems in sign language recognition first, before applying them to gesture recognition.. Third, a working sign language recognition system would make deaf-hearing interaction easier. Particularly public functions, such as the courtroom, conventions, and meetings, would become much more accessible to the deaf.. The main challenge in sign language recognition is to find a modeling paradigm that is powerful enough to capture the language, yet scales to large vocabularies. Signed languages are highly inflected, which means that each sign can appear in many different forms, depending on subject, object, and numeric agreement. Thus, it is futile to model each form separately-there are simply too many of them. Instead, sign language recognizers must capture the commonalities among all signs. In speech recognition this problem is solved by modeling the language in terms of its constituent phonemes. In principle, the same solution applies to sign language recognition.. However, modeling the phonology of sign languages is much more difficult than modeling the phonology of spoken languages. In speech, the phonemes appear sequentially. In signed languages the phonemes can appear both in sequences and simultaneously. For example, a sign can consist of two hand movements in sequence, but the handshape and hand orientation can change at the same time. As a consequence, there is a large number of possible combinations of phonemes that can occur in parallel. Attempting to capture all the possible different combinations of phonemes statically-for example, by training a hidden Markov model (HMM) for each combination-would be futile for anything but the smallest vocabularies.. In this paper we present a novel framework for modeling and recognizing American Sign Language (ASL). It consists of breaking the simultaneous aspects of ASL down into its constituent phonemes and modeling them with parallel hidden Markov models (PaHMMs). This is a new extension to hidden Markov models (HMMs).. In previous work, researchers have proposed other extensions to HMMs to model the interaction of several interacting processes in parallel, such as factorial hidden Markov models (FHMMs) [7] or coupled hidden Markov models (CHMMs) [3] . These extensions require modeling the interactions of the processes during the training phase, and thus require training examples of every conceivable combination of actions that can occur in parallel. Thus, it is doubtful that FHMMs and CHMMs will scale well in ASL recognition.. PaHMMs avoid these scalability problems by assuming that the processes are independent of one another ("independent channels"). As a consequence, the channels can be trained completely independently, before they are combined at recognition time. Thus, it is not necessary to provide training examples of all possible combinations of phonemes. There is linguistic evidence that ASL can be modeled at least partially as independent channels [18] . Hence, PaHMMs stand a much better chance than FHMMs and CHMMs of being scalable. Because gesture recognition is even less constrained than ASL recognition, PaHMMs are highly significant to gesture recognition research, as well.. We use 3D data as the input to our recognition framework. These data can be collected either with 3D computer vision methods, such as physics-based modeling [14] [15] [16] , or with a magnetic tracking system, such as the Ascension Technologies MotionStar system. We use these 3D data to recognize continuous sentences over a 22-sign vocabulary, where the individual signs are modeled in terms of their constituent phonemes.. The remainder of this paper is organized as follows: First, we give an overview on related work. Then we describe the fundamentals of modeling ASL, as they apply to our recognition framework. We show how and why breaking down signs into their constituent phonemes is beneficial. We then describe the necessary extensions to existing phonological models of ASL to adapt them to ASL recognition. We also develop the phonological basis for modeling ASL in terms of independent channels.. Then we give a brief introduction to HMMs and describe the token-passing algorithm as the main recognition method. We briefly discuss FHMMs and CHMMs and why they cause problems for large-scale ASL recognition. We then develop the mathematics and algorithms behind PaHMMs, in order to overcome the scalability problems of FHMMs and CHMMs. We briefly discuss implementation issues that arise during the adaptation of HMMs for ASL recognition and provide experimental results that compare PaHMMs with conventional HMMs. We demonstrated that PaHMMs can improve the robustness of ASL recognition even on a small scale. Together with breaking down the signs into phonemes, they provide a powerful and potentially scalable framework for modeling ASL. Because PaHMMs are potentially more scalable than other extensions to HMMs, they are an interesting research topic for gesture and sign language recognition.. Future research should establish how PaHMMs behave with larger vocabularies, and particularly with highly inflected signs that can exhibit a large number of phoneme combinations within one single sign. Future research should also add hand configuration and orientation as new channels to the PaHMM framework.. Once the viability of PaHMMs has been established for more channels and for larger vocabularies, the major outstanding challenges in modeling ASL recognition will be the integration of facial expressions, the use of space. Facial expressions are important, because they constitute 80% of the grammar of ASL. Space is important, because almost all subject-object relations are expressed in terms of locations in front of the body. Both facial expressions and the use of space will be essential in building a complete grammatical representation of the recognized ASL.. Semantic representation of ASL will also be important, particularly for deaf-hearing interaction. Because the structure of ASL is so different from spoken languages, it is necessary to do more research into parsing the recognized ASL constructs and converting them into a semantic representation. In the discussion of related work, we focus on previous work in sign language recognition. For coverage of gesture recognition, the survey in [24] is an excellent starting point. Other, more recent work is reviewed in [35] .. Much previous work has focused on isolated sign language recognition with clear pauses after each sign, although the research focus is slowly shifting to continuous recognition. These pauses make it a much easier problem than continuous recognition without pauses between the individual signs, because explicit segmentation of a continuous input stream into the individual signs is very difficult. For this reason, and because of coarticulation effects, work on isolated recognition often does not generalize easily to continuous recognition.. Erensthteyn and colleagues used neural networks to recognize fingerspelling [6] . Waldron and Kim also used neural networks, but they attempted to recognize a small set of isolated signs [34] instead of fingerspelling. They used Stokoe's transcription system [29] to separate the handshape, orientation, and movement aspects of the signs.. Kadous used Power Gloves to recognize a set of 95 isolated Auslan signs with 80% accuracy, with an emphasis on computationally inexpensive methods [13] . Grobel and Assam used HMMs to recognize isolated signs with 91.3% accuracy out of a 262-sign vocabulary. They extracted 2D features from video recordings of signers wearing colored gloves [9] .. Braffort described ARGo, an architecture for recognizing French Sign Language. It attempted to integrate the normally disparate fields of sign language recognition and understanding [2] . Toward this goal, Gibet and colleagues also described a corpus of 3D gestural and sign language movement primitives [8] . This work focused on the syntactic and semantic aspects of sign languages, rather than phonology.. Most work on continuous sign language recognition is based on HMMs, which offer the advantage of being able to segment a data stream into its constituent signs implicitly. It thus bypasses the difficult problem of segmentation entirely.. Starner and Pentland used a view-based approach with a single camera to extract twodimensional features as input to HMMs with a 40-word vocabulary and a strongly constrained sentence structure [27] . They assumed that the smallest unit in sign language is the whole sign. This assumption leads to scalability problems, as vocabularies become larger. In [28] they applied their methods to wearable computing by mounting a camera on a hat.. Hienz and colleagues used HMMs to recognize a corpus of German Sign Language [12] . Their work was an extension of the work by Grobel and Assam in [9] ; that is, it used colored gloves, and it was 2D-based. They also experimented with stochastic bigram language models to improve recognition performance. The results of using stochastic grammars largely agreed with our results in [31] .. Nam and Wohn [23, 22] used three-dimensional data as input to HMMs for continuous recognition of gestures. They introduced the concept of movement primes, which make up sequences of more complex movements. The movement prime approach bears some superficial similarities to the phoneme-based approach in [33] and in this paper.. Liang and Ouhyoung used HMMs for continuous recognition of Taiwanese Sign Language with a vocabulary between 71 and 250 signs [17] . They worked with Stokoe's model [29] to detect the handshape, position, orientation, and movement aspects of the running signs. Unlike other work in this area, they did not use the HMMs to segment the input stream implicitly. Instead, they segmented the data stream explicitly based on discontinuities in the movements. They integrated the handshape, position, orientation, and movement aspects at a level higher than that of the HMMs.. We used HMMs and 3D computer vision methods to model phonological aspects of ASL [31, 33] with an unconstrained sentence structure. We used the Movement-Hold phonological model by Liddell and Johnson [18] extensively, so as to develop a scalable framework. In [32] we extended the conventional HMM framework to capture the parallel aspects of ASL, which ordinarily would make the recognition task too complex. We ran several continuous recognition experiments with 3D data to test the feasibility of modeling the movements of the left and the right hands with PaHMMs. Our database consisted of 400 training sentences and 99 test sentences over a vocabulary of 22 signs. The full transcriptions of these signs are listed in Appendix 1. The sentence structure was constrained only by what is grammatical in ASL. We performed all training and FIG. 18 . Example of the 3D position signal for the sentence "woman try teach." The solid line is from the x coordinate, the dashed line is from the y coordinate, and the dotted line is from the z coordinate. The unlabeled parts of the signal are epenthesis movements.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O B I O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Bi-criteria algorithm for scheduling jobs on cluster platforms We describe in this paper a new method for building an efficient algorithm for scheduling jobs in a cluster. Jobs are considered as parallel tasks (PT) which can be scheduled on any number of processors. The main feature is to consider two criteria that are optimized together. These criteria are the makespan and the weighted minimal average completion time (minsum). They are chosen for their complementarity, to be able to represent both user-oriented objectives and system administrator objectives.We propose an algorithm based on a batch policy with increasing batch sizes, with a smart selection of jobs in each batch. This algorithm is assessed by intensive simulation results, compared to a new lower bound (obtained by a relaxation of ILP) of the optimal schedules for both criteria separately. It is currently implemented in an actual real-size cluster platform. In this paper we presented a new algorithm for scheduling a set of independent jobs on a cluster. The main feature is to optimize two criteria simultaneously. The experiments show that in average the performance ratio is very good, and the algorithm is fast enough for practical use. The algorithm has been assessed by comparing the minsum performance to a new lower bound based on the relaxation of an ILP, and comparing the makespan performance to the best known approximation. Actual results are not available at the moment, but we are currently implementing this algorithm on a full-scale platform (Icluster2).. Several technical problems still have to be solved for an even more efficient practical solution, namely the reservation of nodes which reduces the size of the cluster and the mix of different types of jobs (moldable jobs, rigid jobs, and divisible load jobs). More formally, we detail below the algorithm starting with the input describing the instances:. • n tasks available at time 0. • pi(k) the processing time of task i on k processors. • wi is its weight • m the number of processors Compute the approximate C * max with the dual approximation algorithm.. Merge the small sequential tasks sorted by decreasing weight. Select the set Sj ⊆ S of tasks to schedule in the current batch (using a knapsack). Schedule the batch between tj and tj+1.. Remove Sj from T . end for Compact the schedule with a list algorithm using the batch ordering.. First, our algorithm calls a dual approximation makespan algorithm (defined in [7] ) to determine an approximation of the optimal makespan of the instance. With this value C * max and the smallest possible duration of a task tmin, we compute the smallest useful batch size t0 (such that at least one task can be done) and K + 1 the number of batches. The values tj are the length of our batches. For every j, tj+1 is twice the value of tj.. The main loop of the algorithm corresponds to the selection of the jobs to be scheduled in the current batch. We first select the tasks which are not too long to run in the batch. If there are several tasks that can be run in less than half the batch size on one processor, we can merge some of these tasks by stacking them together. In order to have as much weight as possible, this merge is done by decreasing weight order.. The next step is to run a knapsack selection, written with integer dynamic programming. We want to maximize the sum of the weight of the selected tasks while using at most m processors. The allocation of the task i is alloti, the smallest allocation that fits (in length) into the batch. Values of W (i, j) are initialized to −∞ for j < 0 and 0 otherwise. For i going from 1 to n and for j going from 1 to m, we compute:. The largest W (n, ·) is the maximum weight that can be done in the batch. The complexity of this knapsack is O(mn).. The first schedule is simple: we start all the selected tasks of one batch at the same time. A straightforward improvement is to start a task at an earlier time if all the processors it uses are idle. A further improvement is to use a list algorithm with the batch ordering and a local ordering within the batches, as it allows to change the set of processors alloted to the tasks.. Finally, an additional optimization step is used. The batch order is shuffled several times and the best resulting compact schedule is kept. This only leads to small improvements.. The overall complexity of this algorithm is O(mnK).

B O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


The hierarchical model for load balancing on two machines Following previous work, we consider the hierarchical load balancing model on two machines of possibly different speeds. We first focus on maximizing the minimum machine load and show that no competitive algorithm exists for this problem. We overcome this barrier in two ways, both related to previously known models. The first one is fractional assignment, where each job can be arbitrarily split between the machines. The second one is a semi-online model where the sum of jobs is known in advance. We design algorithms of best possible competitive ratios for both these cases. Furthermore, we show that the combination of the two models leads to the existence of an optimal algorithm (i.e., an algorithm of competitive ratio 1). This algorithm is clearly optimal for the makespan minimization problem as well. For the latter problem, we consider the fractional assignment model and design an algorithm of best possible competitive ratio for it. We study load balancing on two machines for cases where the two processors or machines do not have the same capabilities.. We consider online algorithms. For an algorithm A, we denote its cost by A as well. The cost of an optimal offline algorithm that knows the complete sequence of jobs is denoted by opt. In this paper we measure the performance of algorithms using the (absolute) competitive ratio. For minimization problems, the competitive ratio of A is the infimum R ≥ 1 such that for any input, A ≤ R · opt. For maximization problems, the competitive ratio of A is the infimum R ≥ 1 such that for any input, opt ≤ R · A.. If the competitive ratio of an online algorithm is at most C we say that it is C-competitive. If no R satisfying the inequality exists, we say that the competitive ratio is unbounded or ∞.. The most general non-preemptive online scheduling model assumes m machines 1, . . . , m and n jobs, arriving one by one. The information associated with a job j is a vector of p j of length m, where p i j is the processing time or size of job j if it is completely assigned to machine i. Each job is to be assigned to a machine before the arrival of the next job. The load of a machine i is the sum of the processing times on machine i of jobs assigned to this machine. The typical goal is to minimize the maximum load of any machine but other goals such as maximizing the minimum load of any machine [5, 3] and minimizing the p norm of the load vector [2] were studied as well.. This model is known as unrelated machines [1] . Many simplified models were defined, both in order to allow the design of algorithms with good performance (which is often difficult, or even impossible, for unrelated machines), and to make the studied model more similar to reality. In the sequel we describe a few models which are relevant to our study.. Uniformly related machines [1, 7] are machines with speeds associated with them, thus machine i has speed s i and the information that a job j needs to provide upon its arrival is just its size, or processing time on a unit speed machine, which is denoted by p j . Then we have p i j = p j /s i . If all speeds are equal, we get identical machines [11] .. Restricted assignment [4] is a model where each job may be run only on a subset of the machines. A job j is associated with a processing time p j which is the time to run it on any of its permitted machines M j . Thus if i ∈ M j we have p i j = p j and otherwise p i j = ∞. The hierarchical model represents a situation where there is a clear order between the strength of machines, in terms of the jobs they are capable of performing. Thus, the set M j is a prefix of the machines for any j.. In this paper we consider the restricted related hierarchical model, where machine i has speed s i , job j has a processing time of p j on a unit speed machine, and may run on machines 1, . . . , g j . Therefore,. We study the case of two machines. In this case, it is reasonable to assume that the machine that is capable of running any job is faster, since this is a stronger machine. However, the opposite case can occur in real life as well, when the machine that cannot run any job, is more specialized, and works faster when it is running the jobs that it is capable of running.. Since we consider the case of two machines, we have g j = 1, if the job may run only on the first machine, and g j = 2 if it can run on both. We denote by D the sum of jobs j such that g j = 1. Throughout the paper, we denote the speed of the first machine by q and assume that the second machine has unit speed. Since the machines are different, we need to consider both cases q ≥ 1 and q < 1, together or separately. The jobs are denoted by j 1 , j 2 , . . ., in the order of arrival and have respective processing times p 1 , p 2 , . . .. Their hierarchies are denoted by g 1 , g 2 , . . .. We denote the final loads of the two machines by L 1 and L 2 , and the total processing time assigned to the machines by P 1 and P 2 . We have P 2 = L 2 and P 1 = q · L 1 . We denote the sum of all processing times by Σ, and if the value of Σ is known in advance, we assume Σ = 1 (which can be achieved by scaling). Previous results. The hierarchical model for general m was studied in [6] (see also [8] ). They designed a non-preemptive e + 1 ≈ 3.718-competitive algorithm.. Park, Chang and Lee [13] and independently Jiang, He and Tang [12] studied the problem on two identical speed hierarchical machines. They both designed 5 3 -competitive algorithms and showed that this ratio is best possible. In [13] , the semi-online variant where the sum of job processing times is known in advance was studied as well. A The paper [12] considered a restricted type of a preemptive model in which idle time is not allowed. They designed a 3 2 -competitive algorithm and showed this is best possible. Additional work on preemptive models can be found in [9] .. Another model where each job can be arbitrarily split between the machines, and parts of the same job can run on different machines, possibly in parallel, was studied in the restricted assignment model [4] . It was shown there that this option does not change the order of growth of the best possible competitive ratio, which is Θ(log n). We refer to this model as fractional assignment and to the model where each job is assigned completely to one machines as integral assignment. Our results.. We provide a complete solution for several problems, for which we consider a setting of two hierarchical machines, and all possible speed combinations.. The max-min model is the one where the goal is to maximize the load of the least loaded machine. The min-max model is the one where the goal is to minimize the load of the most loaded machine.. In this paper we first focus on the max-min model. We show that no competitive algorithm exists for this problem. To overcome this barrier, we study two more specific previously known models. The first one is fractional assignment, defined above. The second one is a semi-online model where the sum of jobs is known in advance. We design algorithms of best possible competitive ratios for both these cases. Furthermore, we show that the combination of the two models leads to the existence of an optimal algorithm (i.e., an algorithm of competitive ratio 1). Note that the overall competitive ratio for fractional assignment (i.e., the supremum competitive ratio for any speed) turns out to be 2, whereas the overall competitive ratio for our semi-online model is unbounded. The values of these ratios for q = 1, i.e., for identical speed machines are 3 2 and 2, respectively. The optimal algorithm for the combined model is clearly optimal for the makespan minimization problem as well. For this latter problem, we consider the fractional assignment model and design an algorithm of best possible competitive ratio for it. The overall competitive ratio of this algorithm is 4 3 , and this value is achieved for q = 1. Note that this is the only case we find in this paper, where the competitive ratio is the same for two machines of speed ratio q, no matter which one of them is the stronger machine in terms of hierarchy.. We summarize our results in table 1. All bounds are tight, thus there is a single entry for each problem.. Note that for two uniformly related machines of speed ratio q, the best competitive ratio for the max-min model is q + 1, which is achieved by a greedy algorithm [10] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Index Information Algorithm with Local Tuning for Solving Multidimensional Global Optimization Problems with Multiextremal Constraints Multidimensional optimization problems where the objective function and the constraints are multiextremal non-differentiable Lipschitz functions (with unknown Lipschitz constants) and the feasible region is a finite collection of robust nonconvex subregions are considered. Both the objective function and the constraints may be partially defined. To solve such problems an algorithm is proposed, that uses Peano space-filling curves and the index scheme to reduce the original problem to a Hölder one-dimensional one. Local tuning on the behaviour of the objective function and constraints is used during the work of the global optimization procedure in order to accelerate the search. The method neither uses penalty coefficients nor additional variables. Convergence conditions are established. Numerical experiments confirm the good performance of the technique. In the last decades there has been a growing interest in approaching Global Optimization problems by different numerical techniques (see, for example, [4] - [11] , [15] - [20] , [25, 27] , [42] - [46] and references given therein). Such an interest is motivated by a large number of real-life applications where such problems arise (see, for example, [1] - [3] , [6, 7, 8, 10, 15, 21, 22, 23, 25, 28, 30, 34, 36] ). These problems often lead to deal with multiextremal non-differentiable objective function and constraints. In such a context the Lipschitz condition becomes the unique information about the problem.. It has been proved by Stephens and Baritompa [37] that if the only information about the objective function ϕ(y) is that it belongs to the class of Lipschitz functions and the Lipschitz constant is unknown, there does not exist any deterministic or stochastic algorithm that, after a finite number of evaluations of ϕ(y), is able to provide an underestimate of the global minimum ϕ(y * ). Of course, this result is very discouraging because usually in practice it is difficult to know the constant. Nevertheless, the necessity to solve practical problems remains. That is why in such problems instead of the statement (S1) 'Find an algorithm able to stop in a given time and provide an ε-approximation of the global optimum ϕ(y * )' the statement (S2) 'Find an algorithm able to stop in a given time and return the lowest value of ϕ(y) obtained by the algorithm' is used. Under this statement, either the computed solution (possibly improved by a local analysis) is accepted by final users (engineers, physicists, chemists, etc.) or the global search is repeated with changed parameters of the algorithm.. Theoretical analysis of algorithms (depending on parameters) for solving problems (S2) is similar to analysis of penalty methods. It is proved for global optimization algorithms that for a fixed problem there exists a parameter P * such that parameters P ≥ P * allow to solve the problem (obviously, the parameter P * is problem-dependent). The problem 'How to determine P * ?' is not discussed in such an analysis, and in every concrete case it is solved using additional information about the problem. For example, in methods using a given Lipschitz constant (see survey [12] ) P * is the Lipschitz constant and it is not discussed how to obtain it. Other examples are diagonal methods (see [25] ) and information algorithms (see [31, 38] ) using similar parameters. For all those methods it is possible to prove that if a value P ≥ P * is used as the parameter, then they converge only to global minimizers. An alternative approach is represented by methods converging to every point in the search domain (see [16, 44] ).. In this paper a constrained Lipschitz global minimization problem is considered. In an informal way it can be stated as follows:. (i) The objective function is multiextremal, non-differentiable, 'black box', and requires a high time to be evaluated;. (ii) Constraints are non-convex (or even multiextremal) and non-differentiable, leading to a complex feasible region consisting of disjoint, non-convex subregions;. (iii) Both the objective function and constraints are Lipschitz functions with unknown Lipschitz constants;. (iv) Both the objective function and constraints may be partially defined, i.e., if a constraint is not satisfied at a point, the rest of constraints and the objective function may be not defined at that point.. It can be seen from this statement that the problem belongs to the class of problems considered in [37] , thus statement (S2) will be used hereinafter. One promising way to face problem (i)-(iv) is the information approach introduced by Strongin in [38, 39, 40] . It uses Peano type space-filling curves (see [2, 5, 26, 38, 42] for examples of usage of space-filling curves in mathematical programming) to reduce the original Lipschitz multi-dimensional problem to a Hölder univariate one (a comprehensive presentation of this approach can be found in [42] ). Global optimization of Hölder functions (see [9, 19, 45] ) has given new tools for solving the reduced one-dimensional problem. Peano curves avoid constructions of support (or auxiliary) functions usually used in the multi-dimensional Lipschitz optimization (see, for example, [15, 17, 18, 25, 27, 42] and references given therein). Of course, if the user knows that the objective function is differentiable and/or the problem is convex, there is no sense to work with Peano curves and specific methods explicitly using information about differentiability or convexity can be applied. In contrast, when you deal with non-differentiable 'black box' multiextremal problems, it is not possible to work with sophisticated techniques using derivatives (or other strong a priori suppositions) and such a reduction to one dimension can help significantly.. In this paper, a novel algorithm belonging to the family of information methods is proposed. It uses two powerful ideas for solving problem (i)-(iv). The first one is the index scheme (see [35, 38, 39, 41, 42] ), allowing to solve Lipschitz problems where both the objective function ϕ(y) and constraints G i (y), 1 ≤ i ≤ m, may be multiextremal and partially defined. Its importance increases in this case because it is not clear how to solve such problems by using, for example, the penalty approach. In fact, the latter requires that ϕ(y) and G i (y), 1 ≤ i ≤ m, are defined over the whole search domain. It seems that missing values can be simply filled in with either a big number or the function value at the nearest feasible point. Unfortunately, in the context of Lipschitz algorithms, incorporating such ideas can lead to infinitely high Lipschitz constants, causing degeneration of the methods and non-applicability of the penalty approach. Thus, for problems where the Lipschitz condition is almost a unique additional information, the ability of the index scheme to work with partially defined problems becomes crucial. Moreover, the index scheme does not introduce any additional parameter and/or variable, whereas the penalty approach requiring determination of the penalty coefficient.. The second idea being at the basis of the new method is the local tuning on the behaviour of the objective function and constraints (see [31, 32, 34, 35] ). Original information methods work with global adaptive estimates of the Lipschitz constants, i.e., the same estimates are used over the whole search region for ϕ(y) and functions G i (y), 1 ≤ i ≤ m. However, global estimates (adaptive or given a priori) of the Lipschitz constants may provide a poor information about the behavior of the objective function over every small sub-region of D. It has been shown in [25, 31, 32, 34, 35] , for different classes of global optimization problems, that local estimates for different sub-regions of D can accelerate the search significantly. Of course, it is necessary to balance global and local information about ϕ(y) obtained by the method during the search. Such a balancing is very important because using only the local information can lead to missing the global solution (see [33, 37] ).. The next section contains a formal statement of the problem (i)-(iv) and presents the new algorithm. Convergence conditions of the new method are established in Section 3. Numerical experiments collected in Section 4 show a satisfactory performance of the new algorithm in comparison with two global optimization techniques. Finally, Section 5 gives concluding remarks. In this paper, a novel global optimization algorithm for solving multi-dimensional Lipschitz global optimization problems with multiextremal partially defined constraints and objective function has been presented. The new algorithm uses Peano type space-filling curves and the index scheme to reduce the original constrained problem to a Hölder one-dimensional problem. Local tuning on the behaviour of the objective function and constraints is executed during the work of the global optimization procedure in order to accelerate the search. The new method works without introducing any penalty coefficients and/or auxiliary variables. Convergence conditions of a new type have been established for the algorithm.. The new algorithm enjoys the following properties:. -in order to guarantee convergence to a global minimizer y * it is not necessary to know the precise Lipschitz constants for the objective function and constraints for the whole search region; on the contrary, only fulfillment of condition (31) for a segment of the Peano curve containing one of the images of y * is required;. -the usual problem of determining the moment to stop the global procedure in order to start a local search does not arise because local information is taken into consideration throughout the duration of the global search;. -local information is taken into account not only in the neighborhood of a global minimizer but also in the whole search region, thus permitting a significant acceleration of the search;. -thanks to usage of the index scheme, the new algorithm does not introduce any additional parameters and/or variables. Constraints are evaluated at every point one at a time until the first violation of one of them, after that the rest of constraints and the objective function are not evaluated at this point. In its turn, the objective function is evaluated only for that points where all the constraints have been satisfied.

O O O O B I O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Conservative Occlusion Culling for Urban Visualization using a Slice-wise Data Structure . An occlusion culling algorithm using our slice-wise data structure sends approximately 54 % fewer triangles to the graphics pipeline and increases frame rate by 46 %, as compared to an occlusion culling algorithm using building-level granularity of visibility in this 1.5M-triangle environment. The red colored sections show occluded regions, which are discarded from the graphics pipeline. In addition, the data structure decreases Potentially Visible Set (PVS) storage requirement drastically.In this paper, we propose a new shrinking process for conservative from-point occlusion culling algorithms and a data structure for the visualization of urban environments. The visible geometry in a typical urban walkthrough mainly consists of partially visible buildings. Occlusion-culling algorithms, in which the granularity is based on buildings, render these partially visible buildings completely. We observe that the visibility in urban walkthroughs shows certain characteristics. The proposed slice-wise data structure represents the buildings, exploiting these characteristics, in terms of slices parallel to the coordinate axes. This forms the base for occlusion culling where the occlusion granularity is at slice level. The proposed slice-wise data structure has minimal storage requirements. The visible parts can be accessed at constant time during navigation with the help of a preprocessing stage. We also propose to shrink the occluders in a scene. This is necessary for a conservative from-point occlusion culling algorithm, which can also be applied to nonconvex general 3D occluders. Empirical results show that a 54 % decrease in the number of processed polygons and 46 % speed-up in frame-rate can be achieved by using the proposed conservative occlusion-culling algorithm with rather than an occlusion-culling method where the granularity is based on individual buildings. The efficiency of the visibility algorithm is vitally important for making an urban visualization system usable on ordinary hardware. View-frustum culling and back-face culling are ways to March 14, 2006 DRAFT speed-up the visualization, and for them, the current algorithms seem to be efficient enough.. However, occlusion-culling algorithms are still very costly.. In occlusion-culling algorithms where the granularity is individual buildings, an object could be sent to the graphics pipeline even if a small portion of it becomes visible. In most cases, this would result in unnecessary overloading of the hardware, especially if the objects are very complex, containing hundreds of thousands of polygons. An efficient approach is needed to create a tight visibility set without causing further overheads. Although it is feasible to traverse the nodes in the hierarchy of an object to see which parts are visible, it is usually impractical to store the visibility lists.. The first contribution of this paper is the slice-wise structure. This is a simple data structure particularly suitable for urban scenes. It automatically exploits real-world occlusion characteristics in urban scenes by subdividing the objects into slices parallel to the coordinate axes. Other object hierarchies such as octrees and regular grids can well be used to partition the objects; even individual triangles can be checked. However, the storage of Potentially Visible Sets (PVSs). limits the scalability of their usage. The PVS storage requirement of the proposed slice-wise structure is very low (as low as three bytes for each viewpoint and partially visible building). An index is stored for a partially visible building, indicating the visible slices along each coordinate axis.. The second contribution of the paper is a shrinking algorithm developed on the principles of conservative from-point occlusion culling. In order to achieve conservative from-point visibility, the occluders need to be shrunk and the test must be performed on the shrunk versions of the occluders. To our knowledge, this is the first demonstrated attempt that can also be applied to general nonconvex occluders.. The organization of the paper is as follows. We give related work in Section 2. In Section 3,. we describe the proposed slice-wise data structure. In Section 4, we describe using the proposed slice-wise structure for occlusion culling based on conservative from-point visibility, thereby describing our shrinking algorithm. In Section 5, we give experimental results and comparisons.. Finally, we give conclusions. March 14, 2006 DRAFT II. RELATED WORK Scene representation has a crucial impact on the performance of the visibility algorithm in terms of memory requirement and processing time. Many data structures have been adopted for scene and object representation such as octrees [1] , or scene graph hierarchy [2] . Scene graph usage that provides fast traversal algorithms is particularly popular [3] . However, these are useful mainly for the definition of object hierarchies. Their use in determining visibility may require them to be augmented with additional information, thereby increasing their storage requirements. In addition, the natural object structure is modified in some applications. In [4] , the triangles that belong to many nodes of the octree are subdivided across the nodes for easy traversal. In [5] , the objects could be divided into subobjects to create a balanced scene hierarchy, if necessary.. Occlusion-culling algorithms detect the parts of the scene that are occluded by other objects and do not contribute to the overall image; these parts should not be sent to the graphics pipeline.. They can be classified as either conservative or approximate [6] . Conservative algorithms may classify some invisible objects as visible but never call a visible building invisible. Instead of traversing an object's internal hierarchy for fine tuned visibility, most conservative algorithms either accept the entire object as visible or reject it. Approximate occlusion-culling algorithms, such as [7] , [8] , [9] , the visible primitives are rendered up to a specified threshold, i.e., some of them may not be sent to the graphics pipeline although they are visible. There are also approaches to occlusion culling that use parallel processing approaches, such as [10] , [5] , [11] .. Exact visibility algorithms provide accurate images at the expense of degrading rendering performance and increasing storage requirements. An example of this class is [12] , where the authors make use of a 5D subdivision of the polygons. A detailed survey on occlusion-culling algorithms can be found in [6] .. Applications where the scene objects are culled and stored using octree structures are mainly suitable for scenes where there are large occluders and a large portion of the model is behind these occluders. Visibility determination by traversing a scene hierarchy requires the quick selection of occluders; this is a difficult task [10] , [13] , [14] , [15] , [16] , [17] , [18] .. The proposed slice-wise structure can be used to create a tight visibility set of slices of objects March 14, 2006 DRAFT for any kind of occlusion-culling algorithm. The visibility set thus produced is smaller than those that measure occlusion at the building level, but larger than the exact ones that operate at the polygon level: it groups polygons by exploiting visibility characteristics in a typical urban walkthrough.. The purpose of clustering visibility is to improve scalability. There are many different approaches to compressing data, such as [19] , [20] , [21] , [22] , [23] . Our slice-wise data structure naturally decreases the amount of information that needs to be stored. Additional compression schemes may further increase its advantages. In this paper, we propose a data structure that exploits the visibility characteristics of buildings in the visualization of urban scenes. The proposed approach avoids sending a building entirely to the graphics pipeline if only a small portion of it is visible, thereby solving the partial occlusion problem. The objects are divided into slices at each axis and then the slices rather than the whole objects are checked for occlusion.. We also show how to shrink objects in a scene, including nonconvex ones, in order to use them as occluders for from-point conservative occlusion culling. Our shrinking algorithm can be used for any kind of object, not just buildings in an urban scene.. Our experiments show the proposed slice-wise occlusion culling provides a 46 % faster visualization than a building-level occlusion culling. Similarly, a visualization using our approach sends 54 % less polygons to the rendering pipeline as compared to a visualization using building-level occlusion culling.. The slice-wise structuring of objects can also be used to visualize scenes other than urban scenery, although we did not test this. Another application for our method would be for the scenes where buildings are touching (as in some European cities). In this case, a subdivision at the object level could be done to create smaller objects.. The proposed approach works for flythrough type navigations where the user is above buildings.. It would be helpful for real time rendering to integrate our method with other approaches such. as view dependent refinement.

O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A generative probabilistic approach to visualizing sets of symbolic sequences There is a notable interest in extending probabilistic generative modeling principles to accommodate for more complex structured data types. In this paper we develop a generative probabilistic model for visualizing sets of discrete symbolic sequences. The model, a constrained mixture of discrete hidden Markov models, is a generalization of density-based visualization methods previously developed for static data sets. We illustrate our approach on sequences representing web-log data and chorals by J.S. Bach. Topographic visualisation techniques have been an important tool in multi-variate data analysis and data mining. Generative probabilistic approaches [2, 3, 7] have been developed and demonstrated to offer numerous advantages over non-probabilistic alternatives in terms of a flexible and technically sound framework that makes various extensions possible in a principled manner.. However, in their current form, most of these methods make an assumption that observation data can be represented in the form of a set of i.i.d. unstructured numerical vectors. While this may be a reasonable assumption in some cases, many of the most recent practical problems face us with the notion of structured observation types. This creates new challenges for data analysis research. Algorithms that are able to discover structure in sample sets of such structured entities need to be developed. Practical examples include various user profiling tasks, where, in the simplest case, one observation consists of a log trace left by a user as Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. There has been a notable interest in extending probabilistic generative modeling principles to accommodate for more complex structured data types [4, 6, 13, 14] . The work in [4] essentially provides a method of visualisation of user navigation sequences based on clustering of Markov chains. Probabilistic clustering of hidden Markov models have also been developed [13] and applied to user navigation modelling in [14] . In addition to clustering models, the work in [6] proposes a computationally efficient convex distributed dynamic model for profiling and prediction of dynamic user activity.. However, none of these methods provide features for a topographic organization of these more complex data types. Topographic mappings require nonlinear distributed models in order to preserve the core of the information contained in the possibly complex and heterogeneous set of structured observations in a 2D visualisation plane.. The necessity of visualising non-vectorial structured types of data has been recognised in the literature and there are non-probabilistic, SOM-based approaches to e.g. visualising time series [5, 9] . In [5] , a self organising map of first order Markov chains is developed and it is pointed out that the Euclidean distance measures initially employed in the nonprobabilistic SOM method are not appropriate for clustering or visualisation based on probabilistic models. A KullbackLeibler divergence is then employed within the SOM, however, this heuristic does not follow from a consistent model formulation.. In this paper we develop a consistent generative probabilistic model for visualizing sets of discrete symbolic sequences, where an appropriate divergence measure is defined by the noise model 1 . The model is a constrained mixture of discrete hidden Markov models (HMM) [11] ), where the constraint, introduced in the spirit of [2, 7] , provides the model with topographic organisation capabilities.. The remainder of the paper is organised as follows: Section 2 introduces the model and the training algorithm. Section 3 provides experimental illustration of the method on two real world data sets: melodic lines of chorales by J.S. Bach and web navigation sequences. We conclude the paper with a brief summary of the key ideas and findings in section 4. We have presented a generative probabilistic model for visualizing sets of discrete symbolic sequences. The model is essentially a constrained mixture of hidden Markov models that allows us to represent non-Markovian dynamical structures in a two-dimensional visualisation plane. Formally the model is a generalization of density-based visualization methods previously developed for static data sets [7] . We illustrated the model on sequences representing web-log data and chorals by J.S. Bach. We experimented with hidden Markov noise models with more than two hidden states, however, the structure extracted by two-state models was already rich and interesting enough.. Our model constitutes a principled approach to visualization of sets of symbolic sequences. Probabilistic formulation enables the model to deal with e.g missing data, hierarchy building, or model selection in a consistent manner. Visualization plots can be naturally interpreted by plotting the state-transition and emission structures of the hidden Markov noise models corresponding to local regions of the latent space. In this sense, hidden Markov models are a much more viable option than standard fixed-order Markov noise models -going beyond the first-order Markov chain structure can prohibitively increase the number of states.. When the task we are facing is, for example, building a good probabilistic model for a given data set of sequences, without any concern for data visualization, then a suitable approach may be to use e.g. mixtures of HMMs, with appropriately chosen number of mixture components using a model selection technique. On the other hand, for model based visualization of sequential data, we may use many HMM components, but constrain them with a tight two-dimensional grid neighborhood structure. Such a constrained mixture of HMM may not be able to compete with appropriately constructed (probably smaller) unconstrained mixture of HMM on the grounds of density modeling, but it is suitable for data visualization and importantly, the tight grid topology prevents constrained models with many components (suitable for high-quality visualization) from excessively overfitting the data. The issue of data explanation vs. data prediction is covered e.g. in [12] . To evaluate tendency of our model to overfit the training data, we split the 100 Bach chorals into training and test sets containing 80 and 20 sequences, respectively. The model is trained solely on training sequences. Figure 4 shows the evolution of negative log-likelihood (NLL) per symbol measured on the training and test sets. After the 12th training epoch, the test set NLL stops decreasing, but crucially, due to the constrained nature of our model, it does not tend to increase at the expense of better modeling of the training data.. Scaling is, however, an issue. The E-step complexity is O(NCT K 2 ). Obviously, a generative topographic formulation with fixed-order Markov chains would be much cheaper, since the noise models would not involve hidden variables. On the other hand, as argued above, the flexibility and interpretability of such formulations would be compromised. Also, the model can be trained on a subsample of the available data and then used to visualize the whole data set. We are currently working on speeding up computations in our model through a variety of cheaper approximation techniques. -For each sequence n = 1 : N and for each grid. Both quantities are determined by the forwardbackward algorithm [11] , using Θc = { c, Ì c, c}, parameters of the c-th HMM. 2 uniform prior distribution of latent classes encourages full use of the available visualization space -Compute 'responsibilities' of HMMs corresponding to grid points Üc, c = 1 : C, for sequences ×. where p(× n |Üc) is the likelihood of the n-th sequence under the c-th HMM.. • M step: Solve the following non-linear equations In the experiments reported below the latent space centres. Üc were positioned on a regular 10×10 square grid (C = 100) and there were M = 16 basis functions φi. The basis functions were spherical Gaussian functions of the same width σ = 1.0. The basis functions were centred on a regular 4×4 square grid, reflecting uniform distribution of the latent classes. We account for a bias term by using an additional constant basis function φ17(Ü) = 1.. Free parameters of the model were randomly initialized in the interval [−1, 1]. Training consisted of repeating EM integrations. Typically, the likelihood levelled up after 30-50 EM cycles.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O