stgenreg: A Stata Package for General Parametric Survival Analysis In this paper we present the Stata package stgenreg for the parametric analysis of survival data. Any user-defined hazard function can be specified, with the model estimated using maximum likelihood utilising numerical quadrature. Models that can be fitted range from the Weibull proportional hazards model to the generalized gamma model, mixture models, cure rate models, accelerated failure time models and relative survival models. We illustrate the features of stgenreg through application to a cohort of women diagnosed with breast cancer with outcome all-cause death. Parametric models remain a standard tool for the analysis of survival data. Through a fully parametric approach, we can not only obtain relative effects, such as hazard ratios in a proportional hazards model, but also clinically relevant absolute measures of risk, such as differences in survival proportions (Lambert, Dickman, Nelson, and Royston 2010) . Parametric models are also useful where extrapolation is required, such as in the economic decision modelling framework (Weinstein et al. 2003) .. The most popular tool for analysing survival data remains the Cox proportional hazards model (Cox 1972) , which avoids making any assumptions for the shape of the baseline hazard function. One of the reasons the Cox model remains the prefered choice over parametric models is that standard parametric models available in standard software are often not flexible enough to capture the underlying shape of the hazard function seen in real data.. The traditional approach to estimation of parametric models is through maximum likelihood. This is relatively simply when using a known probability distribution function, such as the Weibull or Gompertz. Many commonly used parametric survival models are implemented in a variety of software packages, such as the streg package in Stata (StataCorp. 2011), survreg (Therneau 2012) in R (R Core Team 2013) and LIFEREG in SAS (SAS Institute Inc. 2008). However, every parametric model has underlying assumptions, for example, the widely used Weibull proportional hazards model assumes a monotonically increasing or decreasing baseline hazard rate. Such assumptions can be considered restrictive, leading to the development of other more flexible parametric approaches (Royston and Parmar 2002; Royston and Lambert 2011) .. In this paper we present the Stata command stgenreg which enables the user to fit general parametric models through specifying any baseline hazard function which can be written in a standard analytical form. This is implemented through numerical integration of the userdefined hazard function. This allows complex extensions to standard parametric models, for example, modelling the log baseline hazard function using splines or fractional polynomials, as well as complex time-dependent effects; methods that are unavailable in standard software. Time-varying covariates can also be incorporated through using multiple records per subject. We do not consider frailty (unobserved heterogeneity) in this article.. One of the key advantages of such a general framework for survival analysis is in the development of new models, for example in one line of code a parametric survival model can be fitted rather than having to directly program the likelihood evaluator. We have presented the stgenreg command in Stata, for the general parametric analysis of survival data. Through specification of a user-defined hazard function, we have illustrated how to implement standard proportional hazards models, novel restricted cubic spline survival models and a generalized gamma model with proportional hazards. In essence, stgenreg may be used to implement a parametric survival model defined by anything from a very simple one parameter proportional hazards model, to models which contain highly flexible functions of time, for both the baseline and time-dependent effects. Any parameter defined in the hazard function can be dependent on complex functions of time, including fractional polynomials or restricted cubic splines.. The choice of the number of quadrature nodes is left to the user. An increasing number of quadrature nodes should be used to establish consistent parameter estimates.. As it is a general framework, it may not be the most computationally efficient; however, it is a useful tool for the development of novel models. For example, it may be useful to develop ideas and test new models, but then spend time developing more computationally efficient methods for specific cases.. In future developments we aim to allow for interval censoring, the extension to incorporate frailty and a post-estimation command to calculate the cumulative incidence function for competing risks. The package is available from the Statistical Software Components archive (Crowther and Lambert 2013) and can be installed from Stata by typing ssc install stgenreg.

O O B O O O O B I O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O


A Tabu Search Algorithm for the Multi-period Inspector Scheduling Problem This paper introduces a multi-period inspector scheduling problem (MPISP), which is a new variant of the multi-trip vehicle routing problem with time windows (VRPTW). In the MPISP, each inspector is scheduled to perform a route in a given multi-period planning horizon. At the end of each period, each inspector is not required to return to the depot but has to stay at one of the vertices for recuperation. If the remaining time of the current period is insufficient for an inspector to travel from his/her current vertex A to a certain vertex B, he/she can choose either waiting at vertex A until the start of the next period or traveling to a vertex C that is closer to vertex B. Therefore, the shortest transit time between any vertex pair is affected by the length of the period and the departure time.We first describe an approach of computing the shortest transit time between any pair of vertices with an arbitrary departure time. To solve the MPISP, we then propose several local search operators adapted from classical operators for the VRPTW and integrate them into a tabu search framework. In addition, we present a constrained knapsack model that is able to produce an upper bound for the problem. Finally, we evaluate the effectiveness of our algorithm with extensive experiments based on a set of test instances. Our computational results indicate that our approach generates high-quality solutions. This paper studies a new manpower routing and scheduling problem faced by a company that procures products from over one thousand suppliers across Asia. The company places orders with a large number of suppliers and must inspect the goods at the factories of the suppliers before shipment. Therefore, the suppliers are required to make inspection requests with the company when the ordered goods are ready for delivery. An inspection request is characterized by the workload, the inspection site and the time window within which the inspection can be started. In turn, the company dispatches a team of professional quality inspectors to perform all on-site inspections. In order to facilitate coordination between inspectors and suppliers, the inspections could only be carried out during working hours (e.g., 8 :00 am to 6:00 pm). Usually, a weekly schedule is created to assign inspectors to requests for the upcoming week. The company has a stable of in-house inspectors, each having a specified weekly workload limit, and the unfulfilled inspection requests are outsourced to external agencies with additional costs. After receiving their weekly inspection schedules, the inspectors depart from the regional office and will not report back until they have performed all their assigned inspections for the week. More specifically, they leave the regional office on Monday, visit a set of inspection sites and return to the regional office on Friday or some earlier workday. In each workday, an inspector generally travels to diffident locations, completes several inspections and finds overnight accommodation (i.e., hotel) in the vicinity of his/her last/next inspection site at or before the end of the office hours. The objective of the problem is to assign as many inspection workloads as possible to the stable of in-house inspectors while satisfying all the above-mentioned practical constraints.. We call this problem the multi-period inspector scheduling problem (MPISP), which can be viewed as a variant of the multi-trip vehicle routing problem with time windows (VPRTW) (Azi et al., 2010; Macedo et al., 2011) . There are four main features that distinguish the MPISP from the multi-trip VRPTW. First, the scheduling subjects, e.g., vehicles or inspectors, are not required to return to the regional office every workday. Second, at the end of each workday, each scheduling subject must stay at one of the vertices for recuperation. Third, each vertex can be visited more than once. If the remaining time of the current period is insufficient for an inspector to travel from his/her current vertex A to a certain vertex B, he/she can choose either waiting at vertex A until the start of the next period or traveling to a vertex C that is closer to vertex B. The vertex C is called a waypoint, which only acts as the intermediate point in a route. Fourth, the objective is to maximize the total inspected workload rather than to minimize the number of inspectors used and/or the total distance traveled.. In this study, we propose a tabu search algorithm to solve the MPISP. This algorithm employs a tailored fitness function consisting of three lexicographically ordered components, a local improvement procedure with tabu moves, an ejection pool improvement process and a perturbation phase. The contributions of this study are fourfold. First, we introduce a new and practical multi-period manpower routing and scheduling problem that considers multiple working periods. Second, we provide an effective tabu search algorithm that uses a set of problem-specific neighborhood search operators. Third, we construct a constrained knapsack model that can produce an upper bound for the MPISP. Fourth, the comprehensive experimental results on a large number of test instances show the effectiveness of our approach.. The remainder of this paper is organized as follows. We first provide an overview of related research in Section 2. In Section 3, we then give a formal definition of the MPISP.. In Section 4, we describe an approach of computing the shortest transit time for any pair of vertices with any departure time. Our proposed tabu search algorithm is detailed in Section 5 and the constrained knapsack model is presented in Section 6. Section 7 reports the experiments results and Section 8 concludes this study with some closing remarks. 3 This paper introduces an inspector scheduling problem in which a set of inspectors are dispatched to complete a set of inspection requests at different locations in a multi-period planning horizon. At the end of each period, each inspector is not required to return to the depot but has to stay at one of the inspection locations for recuperation. We first studied the way of computing the shortest transit time between any pair of locations when the working time periods are considered. Next, we introduced several local search operators that were adapted from classical VRPTW operators and integrated these adapted operators in a tabu search framework. Moreover, we presented a constrained knapsack model that is able to produce an upper bound for the MPISP. Finally, we evaluated the algorithm based on 370 TOPTW instances and 672 MPISP instances. The experimental results reported in this study show the effectiveness of our algorithm and can serve as benchmarks for future researchers. Since the working time windows of the scheduling subjects and the use of waypoint are very common considerations in practice, a possible research direction can focus on studying the variants of other existing vehicle routing models that involves these two factors. The MPISP is one type of manpower scheduling problems. Scheduling staff members is a traditional research area; example problems include the nurse rostering problem (Cheang et al., 2003) , the technician planning problem (Ernst et al., 2004) and the airline crew rostering problem (Kohl and Karisch, 2004) . As for the manpower scheduling problems that involve creating routes for staff members, we refer the reader to Li et al. (2005) ; Tang et al. (2007) ; Zäpfel and Bögl (2008) ; Cai et al. (2013) ; Zhang et al. (2013) .. Since each inspector has to perform inspections at different locations, the MPISP is essentially a variant of the vehicle routing problem (Toth and Vigo, 2002) . One of the defining characteristics is its objective of maximizing the total inspected workload. Two previously studied problems with similar objective are the team orienteering problem with time windows (TOPTW) (Vansteenwegen et al., 2009 (Vansteenwegen et al., , 2011 Labadie et al., 2012; Hu and Lim, 2014) and the vehicle routing problem with time windows and a limited number of vehicles (m-VRPTW) (Lau et al., 2003; Lim and Zhang, 2007) . The m-VRPTW is an extension of the TOPTW with the consideration of vehicle capacity and customer demands.. These two problems both aim to determine a set of routes that maximizes the total reward of the vertices visited during a single period with a distance or duration limit. The multiperiod planning horizon of the MPISP is related to the periodic vehicle routing problem (PVRP) (Gaudioso and Paletta, 1992; Hemmelmayr et al., 2009 ) and the multiple trip vehicle routing problem (MTVRP) (Battarra et al., 2009 ). However, the MPISP is quite different from the PVRP and MTVRP. In the PVRP, each customer requires a certain number of visits within the planning horizon, and two types of decisions are involved in the planning, namely determining the visit days for each customer and the routing plan for each time period. The PVRP and MTVRP both require that each vehicle must return to the depot at the end of each period.. Another defining characteristic of the MPISP is the consideration of multiple working periods. Working hour regulations have recently received increasing attention from some researchers studying vehicle routing problems. Savelsbergh and Sol (1998) proposed a dynamic and general pickup and delivery problem in which lunch and night breaks must be taken into account. Xu et al. (2003) applied column generation based solution approaches to solve a pickup and delivery vehicle routing problem that involves a set of practical complications, such as heterogeneous vehicles, last-in-first-out loading and unloading operations, pickup and delivery time windows, and working hour restrictions by the United States Department of Transportation. Similarly, Goel (2009 Goel ( , 2010 and Kok et al. (2010) investigated combined vehicle routing and driver scheduling problems under the European Union regulations for drivers.. The MPISP problem can be viewed as a natural extension of the orienteering problem with hotel selection (OPHS) (Divsalar et al., 2013 (Divsalar et al., , 2014 . In the OPHS, a scheduling subject can visit a set of vertices each with a score and find accommodation at a given set of hotels.. The tour is divided into multiple trips, each with a limited duration and starting from and ending at one of the hotels. The objective of the OPHS is to determine a tour that maximizes the total collected score. The OPHS is a variant of the traveling salesperson problem with hotel selection (TSPHS) (Vansteenwegen et al., 2012; Castro et al., 2013) , which aims to serve all vertices with the minimum number of connected trips and the minimum total travel distance. The common characteristic of the above three problems is the involvement of hotel selection. Tang et al. (2007) and Zäpfel and Bögl (2008) introduced two manpower routing and scheduling problems that involve maximization of profits, multiple periods and working hour restrictions. However, their problems require that the trip in each period must start from and end at the depot. Most recently, Zhang et al. (2013) proposed an inspector scheduling problem which is very similar to our problem. Their problem differs from our problem in the following four assumptions: (1) each vertex can only be visited at most once; (2) if a vertex is visited by an inspector, its inspection request must be fulfilled by this inspector;. (3) an inspector reaches a vertex and completes the corresponding inspection task in the same period; and (4) each vehicle stays at the last served vertex at the end of each period and begins the trip of the next period from that vertex. By ignoring these four assumptions,. the MPISP is more difficult but practical. 5

O B I O O O O B I I O O O O O B I I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


An

improved

alignment-free

model

for

dna

sequence

similarity

metric

Background:

DNA

Clustering

is

an

important

technology

to

automatically

find

the

inherent

relationships

on

a

large

scale

of

DNA

sequences.

But

the

DNA

clustering

quality

can

still

be

improved

greatly.

The

DNA

sequences

similarity

metric

is

one

of

the

key

points

of

clustering.

The

alignment-free

methodology

is

a

very

popular

way

to

calculate

DNA

sequence

similarity.

It

normally

converts

a

sequence

into

a

feature

space

based

on

words'

probability

distribution

rather

than

directly

matches

strings.

Existing

alignment-free

models,

e.g.

k-tuple,

merely

employ

word

frequency

information

and

ignore

many

types

of

useful

information

contained

in

the

DNA

sequence,

such

as

classifications

of

nucleotide

bases,

position

and

the

like.

It

is

believed

that

the

better

data

mining

results

can

be

achieved

with

compounded

information.

Therefore,

we

present

a

new

alignment-free

model

that

employs

compounded

information

to

improve

the

DNA

clustering

quality.

Results:

This

paper

proposes

a

Category-Position-Frequency

(CPF)

model,

which

utilizes

the

word

frequency,

position

and

classification

information

of

nucleotide

bases

from

DNA

sequences.

The

CPF

model

converts

a

DNA

sequence

into

three

sequences

according

to

the

categories

of

nucleotide

bases,

and

then

yields

a

feature

vector.

The

feature

values

are

computed

by

an

entropy

based

model

that

takes

both

local

word

frequency

and

position

information

into

account.

We

conduct

DNA

clustering

experiments

on

several

datasets

and

compare

with

some

mainstream

alignment-free

models

for

evaluation,

including

k-tuple,

DMk,

TSM,

AMI

and

CV.

The

experiments

show

that

CPF

model

is

superior

to

other

models

in

terms

of

the

clustering

results

and

optimal

settings.

Conclusions:

The

following

conclusions

can

be

drawn

from

the

The

hybrid

information

model

is

better

than

the

model

based

on

word

frequency

For

DNA

sequences

no

more

than

characters,

the

preferred

size

of

sliding

windows

for

CPF

is

two

which

provides

a

great

advantage

to

promote

system

The

CPF

model

is

able

to

obtain

an

efficient

stable

performance

and

broad

generalization.

It

is

believed

that

an

alignment-free

model

containing

more

useful

information

can

achieve

better

data

mining

results.

This

paper

presents

the

CPF

model

that

employs

the

word

frequency,

position

and

nucleotide

bases

classification

information

from

DNA

sequences.

The

experimental

results

show

that

CPF

is

superior

to

other

models,

including

k-tuple,

DMk,

TSM,

AMI

and

CV.

The

following

conclusions

can

be

drawn

from

the

experiments..

The

hybrid

information

model

is

better

than

the

model

only

based

on

word

frequency..

For

DNA

sequences

no

more

than

characters,

the

preferred

size

of

sliding

windows

for

CPF

is

two,

which

provides

a

great

advantage

to

promote

system

performance.

The

CPF

model

is

able

to

make

an

efficient

stable

performance

and

broad

generalization..

In

the

future,

we

will

perform

the

CPF

model

on

a

large

scale

DNA

data

to

deeply

observe

DNA

sequence

similarity

and

mining

relationships

among

them.

And

a

more

efficient

clustering

method

will

be

presented

to

promote

the

clustering

results

on

unknown

DNA

sequences.

There

are

a

variety

of

features

extracted

from

a

raw

DNA

sequence,

such

as

word

frequency,

classifications

of

nucleotide

bases,

position

and

so

on.

Different

alignmentfree

models

employ

different

features

to

build

feature

As

far

as

I

know,

the

theoretical

boundary

is

not

presented

by

now.

But

it

is

a

fact

that

the

longer

feature

vector

may

not

guarantee

the

better

clustering

result.

On

the

small

size

datasets,

our

experiments

verified

that

the

CPF

model

outperforms

the

k-tuple

model.

Moreover,

the

experiments

illustrate

that

the

CPF

model

gets

the

best

result

when

the

sliding

window

is

two,

i.e.

the

dimension

of

a

feature

vector

is

Namely,

the

longer

CPF

vector

(i.e.

the

longer

sliding

window)

is

not

always

better

than

the

CPF

vector.

We

have

tested

the

CPF

model

on

several

datasets,

the

results

show

that

CPF's

performance

is

steady.

As

a

result,

for

not

too

long

DNA

sequences,

the

best

configuration

of

CPF

model

is

fixed..

However,

the

length

of

a

DNA

sequence

is

not

very

huge

in

our

experiments,

which

is

no

more

than

characters.

It

believed

that

the

vector

is

not

adequate

for

huge

DNA

sequences.

Obviously,

if

characters

size

of

sequences

are

compressed

into

vectors,

too

much

information

are

lost

so

that

they

can

not

be

effectively

distinguished..

When

the

dataset

grows

large,

all

six

models

becomes

worse.

The

best

F-measure

value

declines

from

near

to

near

when

the

family

number

of

dataset

grows

from

six

to

All

models'

F-measure

value

will

less

than

on

the

more

larger

dataset.

Since

the

larger

the

dataset

is,

the

more

families

may

be

overlapped

eath

other.

That

is

a

hard

obstacle

for

alignment-free

clustering

models.

It

is

a

big

challenge

to

solve

the

issue

on

the

wild

large

scale

datasets..

Consequently,

it

is

still

an

problem

for

future

research

to

estimate

the

optimal

size

of

feature

vectors

for

different

size

of

DNA

sequences.

Our

work

illustrates

that

the

CPF

model

can

get

an

excellent

clustering

result

for

DNA

sequences

no

more

than

characters.

Our

direct

motivation

is

to

improve

the

DNA

clustering

quality

rather

than

detect

homologous

sequences.

It

is

well

known

that

the

basic

k-tuple

method

only

containing

word

frequency

information

is

not

sufficient

to

fully

describe

a

DNA

sequence.

Unsufficient

information

in

a

feature

vector

is

the

most

important

reason

that

causes

poor

clustering

results.

For

instance,

Dong

and

Pei

argued

that

the

position

inside

sequence

is

important

information

for

the

sequence

data

clustering

or

classification.

Thus,

many

modified

algorithms

adopt

the

position

information

of

nucleotide

bases.

Among

them,

the

DMk

method

considers

the

occurrence,

location

and

order

relation

of

k-tuple

in

a

DNA

sequence.

It

produces

a

feature

vector

based

on

the

Shannon

entropy

that

reflects

the

degree

of

importance

of

positions

in

a

sequence,

instead

of

simply

adding

new

statistical

information.

The

vector

size

of

DMk

is

equal

to

that

of

k-tuple

under

the

same

sliding

window

size.

However,

the

DMk

method

ignores

the

classifications

of

nucleotide

bases,

which

are

very

useful

information

to

discriminate

DNA

pieces..

Many

researches

show

that

the

classification

over

nucleotide

bases

improves

the

efficiency

when

comparing

DNA

sequences.

TSM

converts

a

DNA

sequence

into

three

symbolic

sequences

utilizing

the

classifications

of

nucleotide

bases

and

their

chemical

properties.

TSM

benefits

from

such

a

treatment.

But

the

improvement

of

TSM

is

limited

because

it

only

involves

the

classification

and

word

frequency

information,

while

the

position

information

is

not

included..

In

this

paper,

we

aim

at

using

the

word

frequency,

position

and

nucleotide

bases

classification

in

calculating

the

DNA

sequence

similarity.

We

integrate

them

together

to

enrich

the

feature

vector

but

without

big

dimensions.

This

idea

inspired

us

to

design

a

new

model,

named

Category-Position-Frequency

model

(CPF).

With

the

rapid

development

of

bioinformatics,

the

collected

biologic

data

has

become

a

giant

monster

and

is

still

explosively

growing.

It

is

necessary

to

use

data

mining

methods

to

analyze

this

tremendous

data

and

find

useful

or

interesting

information

from

the

data

sets.

Due

to

the

extremely

huge

amount

and

complex

structure

of

the

data,

sequence

analysis

of

DNA

and

protein

is

a

challenging

issue

in

the

bioinformatics

field.

There

are

many

approaches

proposed

for

the

sequence

analysis

on

DNA

and

protein.

Among

them,

the

Clustering

approach

is

one

of

the

most

popular

approaches

because

it

requires

less

transcendental

knowledge

and

need

not

mark

the

targets'

category

before

learning.

After

clustering,

DNA

sequence

segments

can

be

automatically

divided

into

clusters

to

show

their

similarity

in

structure,

which

implies

their

functional

similarity

Such

a

treatment

has

many

benefits.

For

example,

it

is

a

powerful

intelligent

way

to

predict

a

genome's

function

and

learn

the

new

world

of

bioinformatics.

When

an

unknown

genome

is

assigned

to

a

known

cluster,

it

can

be

convinced

that

the

new

genome

may

have

the

very

similar

function

with

others

in

the

same

cluster.

But

the

DNA

clustering

quality

can

still

be

improved

greatly.

Since

the

DNA

sequence

similarity

metric

has

a

vital

impact

on

the

clustering

result.

We

present

a

new

DNA

sequence

similarity

model

to

improve

the

DNA

clustering

quality..

The

similarity

of

DNA

sequence

is

a

fundamental

metric

in

bioinformatics,

which

is

a

basis

for

many

applications

including

predicting

unknown

sequences'

functions

or

effects,

constructing

creatures(or

species)

phylogenetic

tree,

and

analyzing

homologous.

Generally,

there

are

two

categories

of

DNA

sequence

similarity

measuring

approaches.

One

is

alignment-based

and

the

other

is

alignment-free.

The

alignment-based

method

directly

compares

two

DNA

sequences

using

string

matching

algorithms,

such

as

BLAST

FASTA

UCLUST

and

CD-HIT

Obviously,

it

is

a

time-consuming

process

to

match

strings

in

a

large

scale

database

and

the

violent

changes

of

sequence

lengths

decline

the

performance

of

clustering..

The

alignment-free

method

converts

each

piece

of

DNA

sequence

into

a

feature

vector

in

a

new

space,

in

which

the

similarity

can

be

quickly

computed.

Some

alignmentfree

algorithms

exploit

probabilistic

models

to

generate

feature

vectors,

of

which

the

Markov

model

is

extremely

important

and

widely

used

in

bioinformatic

applications.

However,

there

are

some

arguments

on

the

Markov

model.

Deshpande

and

Karypis

reported

that

the

SVM-based

approaches

are

more

effective

than

many

traditional

sequence

classification

algorithms,

especially

Markov

model

based

techniques,

in

the

DNA

sequence

comparison.

Lu

et

al.

argued

that

the

assumption

of

Markov

model

on

the

DNA

sequences

impairs

its

capability..

Qi

et

al.

proposed

a

comparison

method

based

on

the

probability

of

appearance

of

K-Strings.

In

order

to

suppress

single-sequence

noises,

Reinert

et

al.

proposed

D

S

and

D

for

sequence

comparison

based

on

k-tuple

content.

They

are

two

variants

of

the

D

word

count

statistic.

The

D

based

alignment-free

models

measure

the

difference

between

two

word

probability

distributions

for

sequence

comparison.

It

is

a

widely

used

statistics

method

for

sequence

comparison

based

on

the

joint

ktuple

content

in

the

two

sequences.

Bauer

et

al.

confirmed

the

existence

of

a

species

specific

Average

Mutual

Information

(AMI)

profile

and

took

these

profiles

to

measure

the

evolutionary

relationships

between

genomic

sequences..

The

k-tuple

algorithm

is

a

very

popular

alignment-free

method.

It

segments

a

DNA

sequence

via

a

sliding

window

of

length

k.

A

segment

of

DNA

sequence

in

the

window

is

a

tuple,

usually

called

a

word

of

length

k.

The

k-tuple

algorithm

counts

the

frequency

of

each

tuple,

i.e.

word,

to

build

a

feature

vector

based

on

the

frequency

value.

Since

a

DNA

sequence

is

converted

into

a

fixed

length

vector,

it

can

be

quickly

measured

by

some

distance

metric..

It

has

been

proved

in

many

researches

that

the

simple

k-tuple

method

cannot

completely

describe

all

information

contained

in

a

DNA

sequence,

since

it

only

contains

the

word

frequency

information.

Therefore,

many

modified

k-tuple

algorithms

are

proposed

to

contain

more

information

in

models.

Liu

et

al.

appended

the

mean

and

variance

value

of

each

word's

position

distribution

into

the

feature

vector.

As

a

result,

the

size

of

the

final

feature

vector

becomes

three

times

of

k

This

method

increases

the

information

contained

in

a

feature

vector

at

the

expense

of

computing

overhead,

in

terms

of

larger

processing

latency

and

memory.

Wei

et

al.

presented

a

Distance

Measure

based

k-tuples

(DMk)

method

for

DNA

sequence

clustering.

According

to

the

position

distribution

of

each

word,

the

DMk

method

calculates

its

entropy

value

to

construct

a

feature

vector.

Dai

et

al.

utilized

both

the

word

frequency

and

overlapping

structure

of

words

to

improve

the

efficiency

of

sequence

comparison.

Li

and

Wang

counted

the

information

of

codon

positions,

and

calculated

the

relative

entropy

over

feature

vectors

to

discriminate

protein

coding

and

non-coding

sequences

in

the

yeast

genome.

Wang

and

Zheng

presented

the

Weighted

Sequence

Entropy

(WSE)

based

comparison

on

word

frequencies

to

modify

the

classical

relative

entropy.

Zhao

et

al.

transformed

the

DNA

sequence

into

the

distribution

vectors.

Lu

et

al.

summarized

the

word

frequency

information

over

a

serial

of

sliding

windows

with

their

size

varying

from

to

k.

Consequently,

they

have

to

observe

all

k-mer

strings'

probability,

and

the

amount

of

sub-strings

is

up

to

k

The

key

issue

of

the

alignment-free

method

is

that

various

DNA

characteristics

and

features

should

be

integrally

considered

and

carefully

composed

so

as

to

contain

sufficient

original

DNA

information

in

the

converted

feature

space.

Shi

and

Huang

proposed

a

Three

Sequence

Method

(TSM)

to

build

a

twelve-component

feature

vector.

Yu

converted

a

DNA

sequence

into

three

cumulative

ratio

curves

instead

of

symbolic

sequences.

Li

and

Wang

presented

a

binary

vector

based

on

the

group

of

nucleotide

bases..

A

segment

of

DNA

data

can

be

considered

as

a

text

written

by

using

a

four-letters

alphabet.

So

some

researchers

apply

text

clustering

methods

to

DNA

data,

such

as

It

is

confirmed

that

key

words

are

flocking

and

not

randomly

distributed

in

DNA

sequences

In

this

paper,

we

present

an

improved

alignment-free

model,

named

as

CPF

model,

which

combines

advantages

of

other

algorithms,

such

as

k-tuple

DMk

and

TSM

The

CPF

model

converts

a

DNA

sequence

into

three

new

sequences

according

to

the

classification

of

nucleotide

bases,

takes

both

frequency

and

position

distribution

information

into

account,

and

measures

the

similarity

in

a

space.

Thus,

the

CPF

model

contains

more

information

than

traditional

alignmentfree

algorithms

and

achieves

better

clustering

quality.

The

experimental

results

demonstrate

the

effectiveness

of

the

CPF

model.

O O B I O B I I O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I I O O O O O O O O B O O O O O B I I O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O


A field model for human detection and tracking Abstract-The large shape variability and partial occlusions challenge most object detection and tracking methods for nonrigid targets such as pedestrians. This paper presents a new approach based on a two-layer statistical field model that characterizes the prior of the complex shape variations as a Boltzmann distribution and embeds this prior and the complex image likelihood into a Markov field. A probabilistic variational analysis of this model reveals a set of fixed-point equations characterizing the equilibrium of the field. It leads to computationally efficient methods for calculating the image likelihood and for training the model. Based on that, effective algorithms for detecting nonrigid objects are developed. This new approach has several advantages. First, it is intrinsically suitable for capturing local nonrigidity. In addition, due to the distributed likelihood, this approach is robust to partial occlusions. Moreover, the two-layer structure provides large flexibility of modeling the image observations, which makes the new method robust to clutters. Extensive experiments demonstrate its effectiveness. T HE research into human detection and tracking has received more and more attention in recent years, due to the drive from many emerging applications, such as perceptual interfaces, ubiquitous computing, and smart video surveillance [4] , [10] , [26] , [27] . Different applications are concerned with different image resolutions of the subjects, thus requiring different techniques. For example, in perceptual interfaces, the motions of the human body parts need to be determined for action recognition; thus, these applications require fairly high resolution for analyzing the articulated motion of the body parts. In contrast, in many video surveillance applications, since the human typically is associated with small image regions, the human needs to be treated as a nonrigid entity for detection and tracking, while the detailed motion of the body parts is no longer the major focus here. This paper addresses the detection and tracking problem in the latter context.. A critical issue in object detection and tracking is to calculate the likelihood pðZjyÞ of the image measurements Z given the rigid motion parameters y (such as the location, the orientation, and the scale) of the target. If the target presents apparent visual invariants (or features), calculating the image likelihood is straightforward. For example, despite the uncertainty in the visual appearances, frontal faces have a similar image pattern that allows the use of the Harr features for face detection [35] . On the other hand, if apparent invariants are not available, the image likelihood pðZjyÞ needs to be broken into a set of conditionals pðZjy; XÞ and integrate them, where X, for example, can be the nonrigid motion of the target. If X is complicated, calculating such an integration can be very difficult, leading to the nontrivial nature of detection and tracking in this scenario. Unfortunately, this is the case when treating the human as a nonrigid entity.. Although the research of object detection has greatly moved forward with the success of face detection, these face detection methods may not apply to human detection. The visual appearances of the human present tremendous variability while lacking apparent visual invariants, as the diversified clothing and the body articulation may significantly change the image of a person. In addition, handling partial occlusion is more concerned in detecting humans because in practice the detector should be robust to the missing body parts, but this challenges most existing methods. Therefore, it is desirable to investigate new human detection methods that cope with the large uncertainty of the visual appearances and are robust to partial occlusions. This paper addresses these two difficulties.. Because the human-like shapes are more or less unique in the real world, they may provide a powerful clue for human detection and tracking [8] , [11] , [34] , [38] . The difficulty of analyzing human shapes lies in the fact that the local shape nonrigidity has a large number of degrees of freedom thus having very complicated uncertainties, which makes rulebased methods unsuitable. Thus, learning-based methods are generally adopted for learning the shape distributions from training data. If the uncertainty is simple, parametric methods such as Gaussian models or Gaussian mixture models can do a good job, e.g., in face shape [5] . Unfortunately, because of the local nonrigidity, the uncertainty in human shapes is too complex to be sufficiently modeled by reasonable Gaussian mixture models. On the other hand, nonparametric methods, e.g., by using exemplars [11] , can be quite flexible. However, a huge set of exemplars is needed to represent the concept of the human-like shapes in order to accommodate the possible variations. As a trade-off, because the Gibbs distribution is flexible to capture a large variety of densities, it can naturally be employed for this task. This idea has been exploited for modeling the face deformations [20] , where the face is represented as a random vector and an inhomogeneous Gibbs distribution can be learned from training data. Although this model is complicated and needs quite complex training, its excellent performance suggests that the Gibbs distribution is useful for characterizing the large and complex variability of the human-like shapes.. Unfortunately, it is difficult for the above learning-based methods to handle partial occlusions because it is not practical, if not impossible, to have the training data that cover all possible situations of partial occlusions. Actually, this difficulty lies in the fact that these methods represent a pattern as a centralized random feature vector. Thus, the missing elements due to occlusion will greatly change the feature vector, thus affecting the classification dramatically. Different from such vectorized methods, the componentbased methods [22] , [29] divide the entity into parts and take advantage of the structures or correlations of the parts. Their excellent performances on detecting partial occluded targets suggest the needs beyond the vectorized models.. The contribution of this paper is a new nonvectorized method based on a two-layer field model for detecting and tracking complex targets such as the human. This new method stands out because of its robustness to partial occlusions, which is difficult for the vectorized methods.. Different from most existing methods, this new approach embeds the complicated nonrigid shape prior into a statistical field and distributes the complex image likelihood to the local sites of the field. This new model has two layers. The hidden layer is a hidden Markov field that captures the shape prior. Every node of this Markov field is associated with an observation node describing the conditional likelihood of image observations of this hidden node, thus constituting the observation layer of this field model. The proposed method models the prior of the nonrigid human shapes as a Boltzmann distribution. Although it is a special Gibbs distribution, our method is different from [20] because the proposed method does not characterize the Boltzmann distribution directly in a vector space, but distribute it into the Markov field. This treatment results in quite simple inference and learning algorithms in both theory and practice. Although the structure of this field model is similar to that in [9] , the difference is apparent, as our method employs probabilistic variational analysis that leads to rigorous and elegant analytical approximations [15] , [17] . Another theoretical benefit is that the image likelihood estimates are lower bounded. This new approach enables effective and efficient detection and tracking algorithms for many nonrigid objects such as pedestrians.. This new approach has a number of advantages over many existing methods. First, since this model employs a field rather than a vector to describe a shape, it can sufficiently capture the local variability of the shape by the local network structure, thus enabling accurate modeling of the complex shape prior. Second, since the model captures shape variability and performs image measurements in a distributed fashion, it is more robust against occlusion than the vectorbased global approaches (such as PCA) in which image measurements have to be performed in a centralized fashion, i.e., conditioned on all shape parameters. In addition, having an observation layer leads to more flexibility and robustness for handling cluttered backgrounds. Third, the variational approximation provides a computationally efficient way to compute the likelihood of image observations, to infer the hidden states of the model, and to facilitate fast learning. Last but not least, it integrates the top-down and bottom-up methodologies for tracking nonrigid objects. The top-down approaches involve evaluating a large number of hypotheses, and the bottom-up approaches require large efforts in grouping and detection. Given the huge variability in the nonrigid human shapes, neither approach would be satisfactory because the number of hypotheses would be tremendously large and grouping a nonrigid object is very difficult. The proposed tracking method is able to balance these two methodologies and to combine the advantages of both: The global variability is handled in a top-down fashion by particle filtering [2] , [14] , while the local nonrigidity is coped with by a bottom-up approach by directly evaluating the likelihood of image measurements.. The paper is organized as follows: After a brief description of the related work in Section 2, this paper presents the twolayer field model in Section 3. The probabilistic variational analysis of this model is given in Section 4, and the learning algorithm is presented in Section 5. Section 6 describes our methods for pedestrian detection and tracking, and our extensive results are reported in Section 7. The paper concludes in Section 8. Characterizing priors of nonrigid shapes is critical for analyzing nonrigid objects. Global or vectorized approaches such as PCA prove to be effective in capturing global deformation by reducing global correlations. However, these vectorized models are neither suitable for handling local nonrigidity nor robust to partial occlusion, which are both important for many real-world applications such as pedestrian detection and tracking. This paper proposed a new statistical method to capture the local nonrigidity based on a two-layer field model, where a Boltzmann distribution was employed to characterize the complicated prior for local variability and a variational mean field approximation was presented for computationally efficient inference, likelihood calculation and model training. Due to the distributed likelihood model, this new field method is robust to occlusion. Based on the framework of this field model, the detection and tracking problems were also investigated and results presented. The success of applying the proposed method to pedestrian detection and tracking showed its effectiveness and general applicability. Aligning training data in the proposed approach is easier than labeling landmark data in the active shape model [5] , but it does leave a problem: How sensitive is the trained model to the alignment errors? We leave this for further studies. In addition, in our future work, we plan to investigate the capacity of the proposed Boltzmann field model, i.e., to what extent the model can capture local nonrigidity. Moreover, better image observation models will be studied to reduce the false alarm rate. In the past few decades, many methods have been proposed for object detection, mainly for the human face and cars. They are based on different classification schemes. Neural network have been employed for detecting faces [31] by classifying the candidate image patches into face or nonface classes. A learned histogram model for wavelet coefficients can be used for face/car detection [32] because the histograms approximate the distributions of object features for discrimination. In addition, combined with the Harr features, the AdaBoost classifier has been very successful for frontal face detection [35] and has been extended for pedestrian detection [36] with the help of motion information. Support vector machines are also widely used in the detection tasks [7] , [23] , [25] .. But, most existing approaches seem not to be suitable for detecting targets with large shape variations, such as the pedestrian. For example, methods based on raw pixel features, such as [24] , [31] , cannot handle large variability in the appearance of pedestrians. It turns out that the shape features of these deformable targets need to be used. It is suggested that both local and global cues should be combined for such a challenging task [19] .. The research on nonrigid shape analysis has a long history, and various approaches have been proposed and investigated. For all these methods, three important common issues should be addressed, i.e., the shape representation X, the shape prior pðXÞ, and the conditional likelihood of image observation pðZjXÞ. (Here, the rigid motion y is dropped for clarity.) Different shape representations can be categorized into either parametric or nonparametric models. Examples of parametric representations includes Fourier descriptors, B splines [2] , [18] , the deformable template [39] , etc., where shape deformation is controlled by the shape parameters and smoothness constraints. A typical nonparametric representation is the point distribution model [5] where a shape is described by an ordered and labeled set of landmark points, and the shape deforms when the points change. Although it provides great flexibility, registration of landmark points is not a trivial task. An even more radical approach is to use a 2D mask [11] , [16] , [34] , where the shape deforms when multiplied by a sparse permutation matrix [16] , or by selecting different exemplars [11] , [34] . In all these representations, a deformable shape is mapped to a point in a vector space (i.e., the shape space), although the dimensionality varies for different approaches. These vectorized models are global since the image observations are conditioned on all the shape parameters. Thus, these global methods are generally not likely to be able to cope with partial occlusions, unless the training data have incorporated all possible occlusion situations. In this paper, rather than using a global representation, a field representation is proposed, with which the complex variability of the nonrigid shape and the occlusion difficulties can be handled easily.. Obviously, in reality, a shape cannot be allowed to deform arbitrarily; thus, the allowable shape space needs to be characterized by having a shape deformation prior model pðXÞ. A possible approach is to reduce the correlations among different shape parameters and model the variance of deformation by a multivariate Gaussian distribution in a lower-dimensional subspace. This is the spirit of the principal component analysis (PCA) and has been widely adopted for learning deformation priors [2] , [5] . Since PCA identifies a linear subspace and catches linear correlations, it is powerful to capture and decorrelate certain global deformations, but insufficient for the local nonrigidity. Thus, it motivated the methods that use mixture distributions [16] or exemplar databases [11] , [34] . Although mixture distributions can represent arbitrarily complicated densities in theory, it becomes unrealistic when the number of mixtures increases tremendously. To alleviate this difficulty, an inhomogeneous Gibbs model has been proposed and applied successfully to face deformation [20] , although the model needs expensive training. The approach proposed in this paper also models the shape deformation prior, but instead of modeling the prior in a global fashion, our approach is based on the field representation and the prior, i.e., a Boltzmann distribution (a specific Gibbs distribution), is distributed into a Markov field, and a variational analysis is employed for analytical results (details in Sections 3 and 4). The new approach stands out from the existing methods by this new representation.. Different approaches have been investigated to fit a shape model to image observations. This can be done through minimizing an energy function [18] , or based on the Bayesian framework where it is important to characterize the conditional likelihood of image observation pðZjXÞ.. Analytical forms can be obtained by assuming the independence among a set of discrete points on shape contours [2] , [21] . To bypass the independence assumption which may be invalid in reality, the conditional likelihood can be modeled as a metric exponential density obtained from the Chamfer distance based on exemplars [34] . When separating global motion from local nonrigidity, the likelihood conditioned on only global motion can be obtained by the mixture (integral) of all exemplar components in the metric mixture model [34] . The proposed approach in this paper also provides tractable ways to calculate the likelihood only conditioned on global motion, but the differences from [34] are: 1) In our model, pðZjXÞ factorizes by independent components and 2) pðZÞ is an integral over almost infinite number of X instead of a finite set of exemplars, and our method obtains a lower bound of pðZÞ.. There have been many excellent works on nonrigid shape matching [1] , [3] , [6] , [30] , [33] . These methods are more concerned with the matching of extracted shapes for shape registration, where the nonrigid motion needs to be explicitly estimated, while this paper is more concerned with integrating out the variability of the nonrigid motion. In addition, since this paper is based on field model, it is also related to Markov random fields (MRF) that have been widely used for image restoration [13] , texture analysis [40] , surface reconstruction [12] , etc. This paper extends MRF to a two-layer model that consists of a random field and an image observation layer. It is more like the Markov network [9] , [37] and the detailed differences will be presented in later sections. More importantly, this paper deals with the nonrigid target detection and tracking problem that has not been addressed by the above methods. In order to validate the proposed approach and demonstrate the applicability of this field model, we performed experiments on pedestrian detection and tracking and compared the proposed detection method with the AdaBoost detector.

O O O O O O O O O O B O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Multifacility ordered median problems on networks: A further analysis In this paper, we address the ordered p-median problem, which includes as special cases most of the classical multifacility location problems discussed in the literature. Finite dominating sets (FDS) are known for particular instances of this problem: p-median, p-center, and p-centdian. We find an FDS for the ordered p-median problem. This set allows us to gain a better insight into the general FDS structure of network location problems. This FDS is later used to present the first polynomial time algorithm for p-facility ordered median problems on tree networks. This result is combined with some approximation algorithms to give an O(log M log log M) approximate solution of these problems on general networks, where M is the number of vertices. One of the most important and well-developed branches in location theory are location problems on networks. Numerous surveys and textbooks give evidence of this fact. (See Mirchandani and Francis [13] , Labbé et al. [12] , Drezner [4] , Puerto [17] , Drezner and Hamacher [5] and references therein.) The starting point of this development may be considered the node-dominance result of Hakimi [7] and the extensions by Hooker et al. [8] , which we will show to be essential. In a series of previous papers, a new type of objective function in location theory was introduced and analyzed (see Nickel and Puerto [15] , Puerto and Fernández [18] , Rodríguez-Chía et al. [20] , Francis et al. [6] , Nickel [14] , and Kalcsics et al. [10] ). In this paper, we will analyze the p-facility version of this new type of objective function called ordered median function which is a generalization of the most popular objective functions: median, center, centdian, k-centrum, among many others.. For networks, this new objective function was first defined in Nickel and Puerto [15] to prove many well-known results in a much easier way and to gain better insight into the geometrical structure of the network with respect to different criteria. In this paper, we discuss the conditions under which finite dominating sets (FDS) for the multifacility formulation of these problems can be derived. Recall that an FDS is a set which always contains an optimal solution of the problem. See Hooker et al. [8] for further details. Identifying a general FDS for this family of problems has important implications on the development of the same kind of efficient algorithms for all these problems simultaneously.. Let ᏺ ϭ (Ᏻ, l ) denote a network with underlying undirected graph Ᏻ ϭ (ᐂ, Ᏹ), with node set ᐂ ϭ {v 1 , . . . , v M } and edge set Ᏹ ϭ {e 1 , . . . , e N }. For an undirected graph Ᏻ, an edge e ʦ Ᏹ is defined as e ϭ. Associated to each edge e ʦ Ᏹ, there is a positive length l(e) defined by the function l : Ᏹ 3 ޒ ϩ . By d(v i , v j ), we denote the length of a shortest path between v i and v j measured by l. Through w : ᐂ 3 ,ޒ every vertex is assigned a nonnegative weight. For short, we write w i :ϭ w(v i ) for a node v i ʦ ᐂ.. A point on an undirected edge e ϭ [v i , v j ] is defined as a pair x ϭ (e, t), t ʦ [0, 1], with. for any v k ʦ ᐂ. The set of all points of a network ᏺ is denoted by ᏼ(Ᏻ). It should be noted that this set also contains the nodes ᐂ. The sets (e, (t 1 , t 2 )) :ϭ {(e, t) ʦ ᏼ(Ᏻ) : t ʦ (t 1 A point x ϭ (e, t) on an edge e ϭ [v i , v j ] ʦ Ᏹ is called a bottleneck point if there exists some node v k with w k 0, such that. Let BN i denote the set of all bottleneck points of a node v i ʦ ᐂ and let Ꮾᏺ :ϭ ഫ iϭ1 M BN i be the set of all bottleneck points of the graph.. For all v i , v j ʦ ᐂ, v i v j , w i w j 0 define EQЈ ij :ϭ ͕x ʦ ᏼ͑Ᏻ͒ : w i d͑v i , x͒ ϭ w j d͑v j , x͖͒.. Let EQ ij be the relative boundary of EQЈ ij , that is, the set of endpoints of the closed subedges forming the elements in EQЈ ij , and let Ᏹᏽ :ϭ ഫ i, j,i j EQ ij . The points in Ᏹᏽ are called Equilibrium points of ᏺ. To simplify the presentation, we will denote by EQ ij kl ʕ EQ ij the equilibrium points of nodes. In the case that ͉EQ ij kl ͉ ϭ 1, we denote for the sake of simplicity by EQ ij kl also the only element of the set EQ ij kl . For X p ʕ ᏼ(Ᏻ), we define. with ٪ being a permutation of the set {1, . . . , M} satisfying. To simplify the notation, we will denote the entries. The ordered p-median problem on ᏺ is defined as. This problem is the natural multifacility extension of the ordered median problem considered in Nickel and Puerto [15] . The function OM ⌳ (X p ) is called the ordered p-median function. Note that this function is defined pointwise. In the context of continuous location theory, a similar objective function was introduced in Puerto and Fernández [18] and later studied in Francis et al. [6] , Rodríguez-Chía et al. [20] , and Puerto and Fernández [19] .. The rest of the paper is organized as follows: In the next section, we study the ordered p-median problem with a special structure in the ⌳-modeling weights. Under this assumption, we can identify an FDS for the problem, namely, the set of pseudo-equilibrium points (r-extremes). Section 3 uses the characterization of the FDS to present the first polynomial time algorithm for solving these problems on tree networks. This polynomial time algorithm for trees is combined with the general approximation algorithms of Bartal [1] and Charikar et al. [2] to obtain an O(log M log log M) approximate solution for the p-facility ordered median problem in general networks. The paper ends with some conclusions.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


On Classification with Bags, Groups and Sets Many classification problems may be difficult to formulate in the traditional supervised setting, where both training and test samples are individual feature vectors. It may be the case that samples are better described by sets of feature vectors, that labels are only available for sets rather than individual samples, or, if individual labels are available, that these are not independent. To better deal with such problems, several extensions of supervised learning have been proposed, where either training and/or test objects are sets of feature vectors. However, such extensions are often proposed independently, disregarding important similarities and differences with other existing classification problems. In this work, we provide an overview of such learning scenarios, propose a taxonomy to illustrate the relationships between them, and discuss directions for further research in these areas. Some pattern recognition problems are difficult to formulate as regular supervised classification problems where (feature vector, label) pairs are available to train a classifier that, in turn, can predict labels for previously unseen feature vectors. In such cases, the objects are, in fact, not individual feature vectors, but sets or bags of feature vectors or instances. Although terminology differs in different fields, we use the terms bags and instances in the rest of this paper.. The first reason for using a bag of instances is that a single feature vector is often too restrictive to describe an object. For example, in drug activity prediction, an application originally addressed in multiple instance learning [10] , we are interested in classifying molecules as having the desired effect (active) or not. However, a molecule is not just a list of its elements: most molecules can fold into different shapes or conformations, which may have an influence on the activity of that molecule. Furthermore, the number of stable shapes is different per molecule. A more logical choice is therefore to represent a molecule as a set of its conformations.. The second reason is that labels on the level of feature vectors are difficult, costly and/or time-consuming to obtain, but labels on a coarser level may be obtained more easily. For computer aided diagnosis applications, it can be very expensive for a radiologist to label individual pixels or voxels in an image as healthy or diseased, while it is more feasible to label a full image, or large image regions. Such coarsely labeled scans or regions can then be used for predicting labels on bag level by labeling new patient scans or, on a finer grained instance level, by labeling individual pixels or voxels.. Another reason to consider the labeling of bags of instances, instead of single feature vectors, is that there may be structure in the labels of the instances. For example, in face verification, where a video of a person is available, considering all the video frames jointly can provide more confident predictions than labeling each of the frames individually and combining the decisions. Similarly, neighboring objects in images, videos, sounds, time series and so forth are typically very correlated, and thus should not be classified independently.. For all these reasons it may be advantageous to represent the objects by bags of instances. Different applications may ask for different representations in the training and the test phase. All possibilities shown in Fig. 1 occur: both training and test objects may be bags or instances. Traditional supervised learning is in the SI-SI scenario in the top left, where both training and test objects are instances. Predicting molecule activity is in the MI-MI scenario, where both training and test objects are bags. Image classification problems can be found in the MI-MI scenario (training on images, testing on images) as well as the MI-SI scenario (training on images, testing on pixels or patches). The face verification problem is best represented by the SI-MI scenario (training on a single face, testing on a set of faces).. The greater representational power, reduced need for labels and improved classification performance are attractive properties of learning domains where objects can be expressed as sets of feature vectors. This idea has been therefore applied in image recognition [32, 9] , face verification [3, 37] , tracking [4] , computer aided diagnosis [40] , molecule activity prediction [10] and document categorization [2] , amongst others.. The success of a classifier in one (source) application may motivate other researchers to use the same method in a different (target) application. However, it is not necessarily the case that the assumptions of the source application still hold in the target application, which can lead to poor performances. On the other hand, it may also happen that the same type of problem occurs in two different applications, and that researchers in the respective fields approach the problem in different ways, without benefiting from each other's findings. We therefore believe that understanding the relationships between such learning scenarios is of importance to researchers in different fields.. With this work, our goal is to provide an overview of learning scenarios in which bags of instances play a role at any of the stages in the learning or classification process. We have gathered the papers in this overview by searching for papers that proposed novel learning scenarios, as well as by combining synonyms of the word "set" with words such as "classification" or "learning". This work is intended as a survey of learning problems, not of classifiers for a particular scenario, although we refer to existing surveys of this type whenever possible. Furthermore, we mainly focus on a single-label, binary classification scenario, as Figure 1 : Supervised learning (SI-SI) and extensions. In the MI-MI scenario (Section 4), both training and test objects are bags. In the MI-SI scenario (Section 5), the training objects are bags and test objects are instances, while in the MI-SI scenario (Section 6), the training objects are instances and the test objects are bags.. Instance space Figure 2 : Overview of important notation many problem formulations can be easily extended to a multi-label [45] , multiclass setting. This paper begins with an overview of notation and learning scenarios in Section 2. We explain the categories of learning scenarios in more detail in Sections 3 to 6. The paper concludes with a discussion in Section 7. Many classification problems deal with objects that are represented as sets of feature vectors, or so-called bags of instances. This popularity is not surprising, as there are several motivating reasons for choosing such a representation at one or more stages of the classification process. Firstly, a set of feature vectors provides greater representational power than a single feature vector, and it might not be logical to express multiple entities (such as several face images of one person) as a single entity. Secondly, often labels may be available only on bag level, and too costly to obtain on instance level, therefore using the bag of instances representation as a form of weak supervision. Lastly, it can be advantageous to consider bags as a whole rather than as independent instances, because of relationships of the instances in a single bag.. This popularity is not without dangers: several different learning scenarios may be defined for the same problem, or several different problems may be incorrectly grouped under the same learning scenario. We proposed a taxonomy that illustrates the relationships of scenarios that deal with bags, groups or sets, and could help researchers relate novel problems to existing applications and research directions.. While the proposed taxonomy allows for heterogeneity in training and test Figure 4 : Variants of the SI-MI scenario. The training objects are instances and the test objects are bags, although the bag may be labeled by a set of instance labels (situation on the right). Note that in this case, the instance labels are decided jointly (as a bag super-label) by a bag classifier F , not by an instance classifier f . objects (i.e., where training objects are bags and test objects are instances and vice versa), it is limited because the training or test objects themselves are homogeneous. It would be interesting to investigate what happens in the case where in the training phase both labeled bags and labeled instances are available. As we already discussed in Section 5, the optimal bag classifier does not necessarily correspond with the optimal instance classifier. Therefore, deciding how to best use the available labels should depend on whether bags or instances are to be classified in the test phase. However, what if bags and instances can be expected in both the classification and test phases? A straightforward solution would be to train separate bag and instance classifiers, but when the bag and instance labels are related, an integrated classifier would perhaps be more suitable.. Another interesting observation is that the "hybrid" categories in the taxonomy (Section 6: SI-MI and Section 5: MI-SI) have attracted a lot of attention, and that the learning scenarios proposed here all need to rely on strong assumptions about the relationships of the instance and bag labels. One of the questions this raises is, what are the minimal assumptions needed to learn in such situations? Furthermore, the learning scenarios we reviewed do not exhaustively cover the types of constraints that could be present between the instance and bag labels. Learning scenarios that will be proposed in the future to fill some of these gaps, can now be easily placed in the context of the works described in this overview.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O