On

the

Approximability

of

Removing

the

Smallest

Number

of

Relations

from

Linear

Systems

to

Achieve

Feasibility

Edoardo

Amaldi

Department

of

Mathematics

Swiss

Federal

Institute

of

Technology

Lausanne

amaldi@dma.epfl.ch

Viggo

Kann

Department

of

Numerical

Analysis

and

Computing

Science

Royal

Institute

of

Technology

Stockholm

viggo@nada.kth.se

August

Abstract

We

investigate

the

computational

complexity

of

the

problem

which

consists

given

a

system

of

linear

relations

of

finding

a

solution

violating

as

few

relations

as

possible

while

satisfying

all

the

others

This

general

combinatorial

problem

referred

to

as

Min

ULR

is

considered

for

the

four

basic

types

of

relational

operators

and

We

proved

in

-LSB-

-RSB-

that

Min

ULR

with

or

relations

is

NP-hard

even

when

restricted

to

homogeneous

systems

with

bipolar

coefficients

whereas

it

is

trivial

for

relations

In

this

paper

we

determine

strong

bounds

on

the

approximability

of

various

intractable

variants

including

constrained

ones

where

the

variables

are

restricted

to

take

bounded

discrete

values

The

various

NP-hard

versions

of

Min

ULR

belong

to

different

approximability

classes

depending

on

the

type

of

relations

and

the

additional

constraints

but

of

them

can

be

approximated

within

any

constant

factor

unless

P

NP

In

the

process

of

studying

Min

ULR

we

also

derive

strong

bounds

on

the

approximability

of

a

class

of

closely

related

problems

which

consist

given

a

feasible

linear

system

of

finding

a

solution

with

as

few

nonzero

variables

as

possible

Finally

we

comment

on

the

approximability

of

two

important

special

cases

that

arise

in

discriminant

analysis

and

machine

learning

Introduction

We

consider

the

general

problem

of

finding

the

minimum

set

of

relations

that

must

be

removed

from

a

given

linear

system

to

make

it

feasible

for

the

four

types

of

relational

operators

and

The

basic

versions

referred

to

as

Min

ULRR

with

R

f

g

for

minimum

Unsatisfied

Linear

Relations

are

defined

as

follows

Given

a

linear

system

AxRb

with

a

matrix

A

of

size

p

n

find

a

solution

x

Rn

which

violates

as

few

relations

as

possible

while

satisfying

all

the

others

Many

variants

of

these

combinatorial

problems

arise

in

various

fields

such

as

operations

research

-LSB-

-RSB-

pattern

recognition

-LSB-

-RSB-

and

artificial

neural

networks

-LSB-

-RSB-

It

is

well-known

that

feasible

linear

systems

with

equations

or

inequalities

can

be

solved

in

polynomial

time

using

an

adequate

linear

programming

method

-LSB-

-RSB-

For

infeasible

systems

least

mean

squared

algorithms

which

are

suited

to

linear

regression

are

not

appropriate

when

the

objective

is

to

minimize

the

number

of

unsatisfied

relations

A

number

of

algorithms

have

been

proposed

for

tackling

various

versions

of

Min

ULR

The

weighted

variants

were

also

considered

in

which

each

relation

has

an

associated

weight

and

the

goal

is

to

minimize

the

total

weight

of

the

unsatisfied

relations

Johnson

and

Preparata

proved

that

special

cases

of

Min

ULR

and

Min

ULR

with

homogeneous

systems

are

NP-hard

and

devised

a

complete

enumeration

method

which

is

also

applicable

to

the

weighted

and

mixed

variants

-LSB-

-RSB-

Greer

developed

a

tree

algorithm

for

optimizing

functions

of

systems

of

linear

relations

that

is

more

efficient

than

complete

enumeration

but

still

exponential

in

the

worst

case

-LSB-

-RSB-

This

general

procedure

can

be

used

to

solve

Min

ULR

with

any

of

the

four

types

of

relations

During

the

last

decade

many

mathematical

programming

formulations

have

been

studied

to

design

linear

discriminant

classifiers

-LRB-

see

-LSB-

-RSB-

as

well

as

the

included

references

-RRB-

When

the

goal

is

to

determine

optimal

linear

classifiers

which

misclassify

the

least

number

of

points

in

the

training

set

the

problem

amounts

to

a

special

case

of

Min

ULR

and

Min

ULR

Increasingly

sophisticated

models

have

been

proposed

in

order

to

try

to

avoid

unacceptable

or

trivial

solutions

-LSB-

-RSB-

The

same

type

of

problem

has

also

attracted

a

considerable

interest

in

learning

theory

because

it

arises

when

training

perceptrons

in

particular

when

minimizing

their

number

of

errors

While

some

heuristic

algorithms

were

devised

in

-LSB-

-RSB-

Amaldi

showed

that

solving

these

problems

to

optimality

is

NP-hard

even

when

restricted

to

perceptrons

with

bipolar

inputs

or

-LSB-

-RSB-

In

-LSB-

-RSB-

minimizing

the

number

of

misclassifications

was

proved

at

least

as

hard

to

approximate

as

minimum

set

cover

and

the

special

case

with

binary

variables

not

easier

than

all

minimization

problems

with

polynomially

bounded

objective

functions

In

recent

years

a

growing

attention

has

been

paid

to

infeasible

linear

programs

-LSB-

-RSB-

When

formulating

or

modifying

very

large

and

complex

models

it

is

hard

to

prevent

errors

and

to

guarantee

feasibility

Infeasible

programs

with

thousands

of

constraints

frequently

occur

and

can

not

be

repaired

by

simple

inspection

With

the

new

software

and

hardware

advances

dealing

with

infeasible

programs

is

becoming

a

major

bottleneck

in

linear

programming

Several

methods

have

been

proposed

in

order

to

try

to

locate

the

source

of

infeasibility

-LSB-

-RSB-

and

to

remove

as

few

constraints

as

possible

to

achieve

feasibility

-LSB-

-RSB-

There

have

recently

been

new

substantial

progresses

in

the

study

of

the

approximability

of

NP-hard

optimization

problems

Various

classes

have

been

defined

and

different

reductions

preserving

approximability

have

been

used

to

compare

the

approximability

of

optimization

problems

-LRB-

see

-LSB-

-RSB-

-RRB-

Moreover

the

striking

results

which

have

recently

been

obtained

in

the

area

of

interactive

proofs

triggered

new

advances

in

computational

complexity

theory

Strong

bounds

were

derived

on

the

approximability

of

several

famous

problems

like

maximum

independent

set

minimum

graph

colouring

and

minimum

set

cover

-LSB-

-RSB-

These

results

have

also

important

consequences

on

the

approximability

of

other

optimization

problems

In

-LSB-

-RSB-

we

performed

a

thorough

study

of

the

approximability

of

the

complementary

problems

named

Max

FLS

where

one

looks

for

maximum

feasible

subsystems

of

linear

systems

In

particular

we

proved

that

the

basic

versions

with

or

relations

are

NP-hard

even

for

homogeneous

systems

with

bipolar

coefficients

While

Max

FLS

with

equations

can

not

be

approximated

within

p

for

some

where

p

is

the

number

of

relations

the

variants

with

strict

or

nonstrict

inequalities

can

be

approximated

within

but

not

within

every

constant

factor

Although

complementary

pairs

of

problems

such

as

Min

ULR

and

Max

FLS

are

equivalent

to

solve

optimally

their

degree

of

approximability

can

differ

enormously

This

is

for

instance

the

case

for

the

minimum

node

cover

and

the

maximum

independent

set

problems

-LSB-

-RSB-

By

delving

into

the

interactive

proof

connection

Arora

et

al.

lately

established

that

Min

ULR

can

not

be

approximated

within

any

constant

unless

P

NP

and

within

a

factor

of

n

for

any

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

-LSB-

-RSB-

Moreover

they

extended

this

nonapproximability

result

to

the

problem

of

minimizing

the

number

of

misclassifications

of

a

perceptron

This

paper

is

organized

as

follows

Section

gives

a

brief

account

on

approximability

classes

and

their

hierarchy

for

minimization

problems

In

section

we

recall

the

known

results

about

the

complexity

of

solving

the

basic

Min

ULRR

with

R

f

g

to

optimally

and

we

determine

alternative

bounds

on

their

approximability

In

section

we

study

the

approximability

of

Min

RVLS

a

closely

related

class

of

problems

which

consist

given

a

feasible

linear

system

of

finding

a

solution

with

as

few

nonzero

variables

as

possible

We

also

discuss

its

consequences

on

Min

ULR

While

section

is

devoted

to

the

weighted

variants

where

a

different

importance

may

be

attached

to

each

relation

section

deals

with

those

where

the

variables

are

restricted

to

take

a

finite

number

of

discrete

values

The

relation

between

Min

ULR

with

real

and

binary

variables

is

analysed

in

section

In

section

we

comment

on

the

approximability

of

two

interesting

special

cases

of

Min

ULR

and

Min

RVLS

with

inequalities

which

arise

in

discriminant

analysis

and

machine

learning

Finally

in

section

the

various

nonapproximability

results

are

summarized

and

some

open

questions

are

mentioned

Approximability

hierarchy

of

minimization

problems

Definition

-LSB-

-RSB-

An

NP

optimization

-LRB-

NPO

-RRB-

problem

over

an

alphabet

is

a

four-tuple

-LRB-

I

S

m

opt

-RRB-

where

I

is

the

space

of

input

instances

S

-LRB-

I

-RRB-

is

the

space

of

feasible

solutions

on

input

I

I

m

I

N

the

objective

function

is

a

polynomial

time

computable

function

opt

fmax

ming

tells

if

is

a

maximization

or

a

minimization

problem

The

set

I

must

be

recognizable

by

a

Turing

machine

in

polynomial

time

The

only

requirement

on

S

is

that

there

exist

a

polynomial

q

and

a

polynomial

time

computable

predicate

ss

such

that

for

all

I

in

I

S

can

be

expressed

as

S

-LRB-

I

-RRB-

fy

jyj

q

-LRB-

jI

j

-RRB-

ss

-LRB-

I

y

-RRB-

g

where

q

and

ss

only

depend

on

Solving

an

optimization

problem

given

the

input

I

I

means

finding

a

y

S

-LRB-

I

-RRB-

such

that

m

-LRB-

I

y

-RRB-

is

optimum

that

is

as

large

as

possible

if

opt

max

and

as

small

as

possible

if

opt

min

Let

opt

-LRB-

I

-RRB-

denote

this

optimal

value

of

m

Approximating

an

optimization

problem

given

the

input

I

I

means

finding

any

S

-LRB-

I

-RRB-

How

good

the

approximation

is

depends

on

the

relation

between

m

-LRB-

I

-RRB-

and

opt

-LRB-

I

-RRB-

The

performance

ratio

of

a

feasible

solution

with

respect

to

the

optimum

of

a

minimization

problem

is

defined

as

R

-LRB-

I

y

-RRB-

m

-LRB-

I

y

-RRB-

opt

-LRB-

I

-RRB-

where

I

I

and

y

S

-LRB-

I

-RRB-

Definition

An

optimization

problem

can

be

approximated

within

c

for

a

constant

c

if

there

exists

a

polynomial

time

algorithm

A

such

that

for

all

instances

I

I

A

-LRB-

I

-RRB-

S

-LRB-

I

-RRB-

and

R

-LRB-

I

A

-LRB-

I

-RRB-

-RRB-

c

More

generally

an

optimization

problem

can

be

approximated

within

p

-LRB-

n

-RRB-

for

a

function

p

Z

R

if

there

exists

a

polynomial

time

algorithm

A

such

that

for

every

n

Z

and

for

all

instances

I

I

with

jI

j

n

we

have

that

A

-LRB-

I

-RRB-

S

-LRB-

I

-RRB-

and

R

-LRB-

I

A

-LRB-

I

-RRB-

-RRB-

p

-LRB-

n

-RRB-

Although

various

reductions

preserving

approximability

within

constants

have

been

proposed

-LRB-

see

-LSB-

-RSB-

-RRB-

we

will

use

the

S-reduction

which

is

suited

to

relate

problems

that

can

not

be

approximated

within

any

constant

Definition

-LSB-

-RSB-

Given

two

NPO

problems

and

and

a

polynomial

time

transformation

f

I

I

f

-LRB-

a

-LRB-

n

-RRB-

c

-RRB-

is

an

S-reduction

with

size

amplification

a

-LRB-

n

-RRB-

from

to

if

i

-RRB-

are

polynomial

time

computable

functions

a

-LRB-

n

-RRB-

is

a

monotonously

increasing

positive

function

and

c

is

a

positive

constant

ii

-RRB-

I

I

and

I

and

S

-LRB-

-LRB-

I

-RRB-

-RRB-

-LRB-

I

y

-RRB-

S

-LRB-

I

-RRB-

iii

-RRB-

I

and

S

-LRB-

-LRB-

I

-RRB-

-RRB-

R

-LRB-

I

-LRB-

I

y

-RRB-

-RRB-

c

R

-LRB-

-LRB-

I

-RRB-

y

-RRB-

iv

-RRB-

I

-LRB-

I

-RRB-

j

a

-LRB-

jI

j

-RRB-

The

composition

of

S-reductions

is

an

S-reduction

If

S-reduces

to

with

size

amplification

a

-LRB-

n

-RRB-

and

can

be

approximated

within

some

monotonously

increasing

function

u

-LRB-

n

-RRB-

in

the

size

of

the

input

instance

then

can

be

approximated

within

c

u

-LRB-

a

-LRB-

n

-RRB-

-RRB-

For

constant

and

polylogarithmic

approximable

problems

the

S-reduction

preserves

approximability

within

a

constant

for

any

polynomial

size

amplification

For

nc

approximable

problems

the

S-reduction

preserves

approximability

within

a

constant

just

for

linear

size

amplification

An

S-reduction

with

c

is

a

cost

preserving

transformation

Definition

-LSB-

-RSB-

An

NPO

problem

is

polynomially

bounded

if

there

is

a

polynomial

p

such

that

I

S

-LRB-

I

-RRB-

m

-LRB-

I

y

-RRB-

p

-LRB-

jI

j

-RRB-

The

class

of

all

polynomially

bounded

NPO

problems

is

called

NPO

PB

All

unweighted

versions

of

Min

ULR

are

included

in

NPO

PB

since

their

objective

function

is

the

number

of

unsatisfied

relations

The

weighted

versions

of

Min

ULR

are

included

in

NPO

PB

if

the

weights

are

polynomially

bounded

since

the

objective

function

is

a

sum

of

a

polynomially

bounded

number

of

polynomially

bounded

weights

Definition

Given

an

NPO

problem

and

a

class

C

is

C-hard

if

every

C

can

be

S-reduced

with

some

size

amplification

to

is

C-complete

if

C

and

is

C-hard

The

range

of

approximability

of

NPO

minimization

problems

stretches

from

problems

which

can

be

approximated

within

every

constant

in

polynomial

time

i.e.

that

have

a

polynomial

time

approximation

scheme

like

the

knapsack

problem

to

problems

that

can

not

be

approximated

within

for

every

where

n

is

the

size

of

the

input

instance

unless

P

NP

Recently

Lund

and

Yannakakis

proved

using

results

from

interactive

proofs

that

Min

Graph

Colouring

can

not

be

approximated

within

n

for

some

unless

P

NP

Moreover

they

derived

a

strong

bound

on

the

approximability

of

Set

Covering

and

of

several

closely

related

minimization

problems

such

as

Min

Dominating

Set

-LSB-

-RSB-

In

-LSB-

-RSB-

Bellare

et

al.

improved

this

result

by

showing

that

approximating

Set

Covering

-LRB-

or

some

of

the

related

problems

-RRB-

within

c

log

n

for

any

c

would

imply

NP

DTIME

-LRB-

nlog

log

n

-RRB-

Since

DTIME

-LRB-

T

-LRB-

n

-RRB-

-RRB-

denotes

the

class

of

problems

which

can

be

solved

in

time

T

-LRB-

n

-RRB-

this

inclusion

is

widely

believed

to

be

unlikely

Furthermore

it

was

established

that

Min

Dominating

Set

and

the

related

problems

can

not

be

approximated

within

any

constant

factor

unless

P

NP

If

there

is

an

approximation

preserving

reduction

from

Min

Dominating

Set

to

an

NPO

problem

we

say

that

is

Min

Dominating

Set-hard

which

means

that

it

is

at

least

as

hard

to

approximate

as

the

minimum

dominating

set

problem

If

we

require

the

dominating

set

in

Min

Dominating

Set

to

be

independent

we

get

the

minimum

independent

dominating

set

problem

or

Min

Ind

Dom

Set

Halld?orsson

established

in

-LSB-

-RSB-

that

Min

Ind

Dom

Set

is

very

hard

to

approximate

Assuming

NP

the

problem

can

not

be

approximated

within

a

factor

of

for

any

where

n

is

the

number

of

nodes

in

the

graph

Inspection

of

Halld?orsson

's

proof

shows

that

the

result

is

still

valid

if

n

is

the

input

size

i.e.

the

sum

of

the

number

of

nodes

and

edges

in

the

graph

Furthermore

Kann

proved

that

Min

Ind

Dom

Set

is

complete

for

NPO

PB

in

the

sense

that

every

polynomially

bounded

NP

optimization

problem

can

be

reduced

to

it

using

an

approximation

preserving

reduction

-LSB-

-RSB-

The

purpose

of

this

paper

is

to

investigate

the

approximability

status

of

the

different

Min

ULR

variants

and

to

locate

them

in

the

hierarchy

of

approximability

classes

We

will

see

that

the

hardness

of

approximating

apparently

similar

and

closely

related

versions

-LRB-

for

instance

a

pair

of

complementary

problems

-RRB-

can

differ

enormously

Approximability

of

Min

ULRR

with

R

f

g

In

this

section

we

focus

on

the

basic

versions

of

Min

ULRR

with

R

f

g.

Unlike

the

other

problems

Min

is

trivially

solvable

because

any

such

system

is

feasible

Indeed

for

any

finite

set

of

hyperplanes

associated

with

a

set

of

linear

relations

there

exists

a

vector

x

Rn

that

does

not

belong

to

any

of

them

In

order

to

determine

the

complexity

of

solving

Min

ULRR

with

R

f

g

to

optimality

we

consider

the

corresponding

decision

versions

that

are

no

harder

than

the

original

optimization

problems

Given

a

linear

system

AxRb

where

A

is

of

size

p

n

and

an

integer

K

with

K

p

does

there

exist

a

solution

x

Rn

violating

at

most

K

relations

of

the

system

and

satisfying

all

the

others

For

Min

ULR

and

Min

ULR

with

homogeneous

systems

we

are

not

interested

in

the

trivial

solutions

where

all

variables

occurring

in

the

satisfied

relations

are

zero

Even

if

we

forbid

the

solution

x

there

might

be

other

trivial

solutions

where

almost

all

variables

are

zero

except

a

few

that

only

occur

in

a

few

equations

In

order

to

rule

out

all

these

trivial

solutions

for

the

equality

and

nonstrict

inequality

case

we

only

consider

solutions

where

the

variable

-LRB-

variables

-RRB-

occurring

in

the

largest

number

of

satisfied

equations

is

-LRB-

are

-RRB-

nonzero

Theorem

-LRB-

-LSB-

-RSB-

-RRB-

Min

ULRR

with

R

f

g

is

NP-hard

even

when

restricted

to

homogeneous

systems

with

bipolar

coefficients

in

f

In

other

words

Min

ULRR

with

R

f

g

is

intractable

even

when

the

points

corresponding

to

the

rows

of

A

belong

to

the

n-dimensional

hypercube

Since

these

problems

are

NP-hard

for

bipolar

coefficients

they

turn

out

to

be

strongly

NP-hard

i.e.

intractable

even

with

respect

to

unary

coding

of

the

data

Thus

these

polynomially

bounded

problems

do

not

have

a

fully

polynomial

time

approximation

scheme

-LRB-

an

approximation

scheme

where

the

running

time

is

bounded

by

a

polynomial

in

both

the

size

of

the

instance

and

-RRB-

unless

P

NP

-LSB-

-RSB-

It

is

worth

noting

that

if

the

number

of

variables

n

is

constant

these

three

basic

versions

of

Min

ULR

can

be

solved

in

polynomial

time

using

Greer

's

algorithm

which

has

an

O

-LRB-

n

pn

-RRB-

time-complexity

where

p

denotes

the

number

of

relations

and

n

the

number

of

variables

-LSB-

-RSB-

These

problems

are

trivial

when

the

number

of

relations

p

is

constant

because

all

subsystems

can

be

checked

in

O

-LRB-

n

-RRB-

time

Furthermore

they

are

easy

when

all

maximal

feasible

subsystems

contain

a

maximum

number

of

relations

because

a

greedy

procedure

is

guaranteed

to

give

a

solution

that

minimizes

the

number

of

unsatisfied

relations

Since

it

is

extremely

unlikely

that

the

three

basic

versions

of

Min

ULR

can

be

solved

optimally

in

polynomial

time

we

are

interested

in

the

existence

of

efficient

algorithms

providing

solutions

that

are

guaranteed

to

be

a

fixed

percentage

away

from

the

actual

optimum

Such

approximate

algorithms

are

of

great

interest

in

practice

Before

establishing

strong

bounds

on

the

approximability

of

Min

ULR

and

Min

ULR

we

point

out

the

relationship

between

the

approximability

of

Min

ULR

and

that

of

Min

ULR

Proposition

Min

ULR

is

at

least

as

hard

to

approximate

as

Min

ULR

but

not

harder

than

Min

ULR

with

nonnegative

variables

Proof

We

first

reduce

Min

ULR

to

Min

ULR

and

then

reduce

Min

ULR

to

Min

ULR

with

nonnegative

variables

Let

-LRB-

A

b

-RRB-

be

an

arbitrary

instance

of

Min

ULR

For

each

equation

aix

bi

where

ai

denotes

the

ith

row

of

A

i

p

we

consider

the

two

inequalities

aix

bi

and

aix

bi

Since

any

solution

x

satisfies

at

least

one

inequality

of

each

such

pair

finding

a

solution

that

minimizes

the

number

of

unsatisfied

equations

in

Ax

b

is

equivalent

to

finding

a

solution

that

minimizes

the

number

of

unsatisfied

inequalities

of

the

corresponding

system

of

size

n.

Let

-LRB-

A

b

-RRB-

be

an

arbitrary

instance

of

Min

ULR

By

replacing

each

variable

xi

unrestricted

in

sign

by

the

difference

of

two

nonnegative

variables

and

by

adding

a

slack

variable

for

each

inequality

we

obtain

an

equivalent

system

with

p

equations

and

p

nonnegative

variables

Proposition

The

problems

Min

ULR

and

Min

ULR

with

integer

coefficients

are

equivalent

Proof

We

simply

use

the

fact

that

any

system

Ax

b

has

a

solution

if

and

only

if

the

system

Ax

b

has

a

solution

where

and

L

is

the

size

-LRB-

in

bits

-RRB-

of

the

binary

encoded

input

instance

-LSB-

-RSB-

It

has

recently

been

shown

in

-LSB-

-RSB-

that

Min

ULR

can

not

be

approximated

within

any

constant

unless

P

NP

and

within

a

factor

of

n

for

any

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

where

n

is

for

instance

the

number

of

variables

According

to

propositions

and

this

is

also

true

for

Min

ULR

with

strict

and

nonstrict

inequalities

We

now

prove

a

nonapproximability

result

for

Min

ULR

and

Min

ULR

which

is

more

likely

to

be

true

but

whose

bound

is

not

as

strong

Theorem

Min

ULR

and

Min

ULR

are

Min

Dominating

Set-hard

They

can

not

be

approximated

within

any

constant

unless

P

NP

and

within

c

log

n

for

any

c

unless

NP

DTIME

-LRB-

nlog

log

n

-RRB-

even

when

restricted

to

homogeneous

systems

with

ternary

coefficients

in

f

Proof

We

proceed

by

cost

preserving

reduction

from

the

known

NP-complete

problem

Min

Dominating

Set

which

is

defined

as

follows

-LSB-

-RSB-

Given

an

undirected

graph

G

-LRB-

V

E

-RRB-

find

a

minimum

cardinality

set

V

V

that

dominates

all

nodes

of

G

i.e.

for

all

v

V

n

V

there

exists

V

such

that

-LSB-

v

-RSB-

E.

Let

G

-LRB-

V

E

-RRB-

be

an

arbitrary

instance

of

Min

Dominating

Set

For

each

node

vi

V

i

n

we

consider

the

homogeneous

inequality

xi

-LRB-

-RRB-

and

the

inhomogeneous

inequality

xi

X

-LRB-

vi

-RRB-

xj

-LRB-

-RRB-

where

j

is

included

in

N

-LRB-

vi

-RRB-

if

and

only

if

vj

is

adjacent

to

vi

Thus

we

have

a

system

with

inequalities

and

n

variables

We

claim

that

there

exists

a

dominating

set

in

G

of

size

at

most

s

if

and

only

if

there

exists

a

solution

x

that

violates

at

most

s

inequalities

of

the

corresponding

system

Given

a

dominating

set

V

V

of

size

s

the

solution

x

defined

by

xi

-LRB-

if

vi

V

otherwise

satisfies

all

inhomogeneous

inequalities

and

n

s

homogeneous

ones

Conversely

given

a

solution

vector

x

that

violates

s

inequalities

we

can

always

satisfy

every

inhomogeneous

inequality

that

is

not

already

satisfied

by

making

one

variable

xi

in

the

inequality

negative

enough

This

operation

yields

a

solution

that

satisfies

at

least

as

many

inequalities

as

x.

Consider

the

set

of

nodes

V

V

containing

all

nodes

vi

such

that

xi

V

is

clearly

a

dominating

set

of

size

s

because

xi

P

-LRB-

vi

-RRB-

xj

only

when

at

least

one

of

the

variables

is

negative

which

corresponds

to

the

case

where

at

least

one

of

the

nodes

is

in

the

dominating

set

Since

the

above

reduction

is

cost

preserving

and

without

amplification

we

have

exactly

the

same

nonapproximability

bounds

for

Min

ULR

and

Min

ULR

as

for

Min

Dominating

Set

This

reduction

can

easily

be

adapted

to

Min

ULR

restricted

to

homogeneous

systems

It

suffices

to

replace

by

and

the

in

the

right

hand

side

of

the

second

type

of

inequalities

by

There

is

a

standard

way

to

transform

an

inhomogeneous

Min

ULR

or

Min

ULR

system

to

a

homogeneous

one

Given

a

system

consisting

of

p

relations

we

first

multiply

all

constant

right

hand

sides

by

a

new

variable

Then

we

only

have

to

ensure

that

this

constant

is

nonzero

-LRB-

for

the

equality

case

-RRB-

or

positive

-LRB-

for

the

inequality

case

-RRB-

Therefore

we

introduce

new

relations

k

k

or

k

for

k

-LSB-

:p

-RSB-

where

is

a

new

variable

In

every

solution

that

satisfies

more

than

p

relations

at

least

one

of

the

new

relations

must

be

satisfied

and

since

all

the

new

relations

are

satisfied

simultaneously

will

become

the

variable

occurring

in

the

largest

number

of

satisfied

equations

Hence

is

nonzero

For

the

inequality

case

this

implies

that

Since

the

same

trick

can

also

be

applied

to

all

variants

of

Min

ULR

or

Min

ULR

we

will

not

give

this

reduction

every

time

it

is

used

Clearly

for

large

n

and

small

a

factor

of

n

is

larger

than

c

log

n

with

any

c

but

NP

DTIME

-LRB-

npolylog

n

-RRB-

is

more

likely

to

be

true

than

NP

DTIME

-LRB-

nlog

log

n

-RRB-

Furthermore

the

above

proof

is

much

simpler

than

those

given

in

-LSB-

-RSB-

Notice

that

the

basic

versions

of

Min

ULRR

with

R

f

g

can

be

approximated

within

n

where

n

is

the

number

of

variables

This

is

a

consequence

of

Helly

's

theorem

for

convex

sets

which

states

-LRB-

when

applied

to

linear

systems

-RRB-

that

every

infeasible

system

with

p

inequalities

or

equations

and

n

variables

contains

an

infeasible

subsystem

with

at

most

n

relations

-LSB-

-RSB-

The

question

of

whether

a

logarithmic

performance

ratio

c

log

n

for

some

c

can

be

guaranteed

in

polynomial

time

unless

P

NP

is

still

open

Complexity

of

a

class

of

closely

related

problems

In

this

section

we

consider

a

class

of

problems

pertaining

to

feasible

systems

but

which

are

closely

related

to

Min

ULR

Given

any

feasible

system

of

linear

relations

one

seeks

a

solution

satisfying

all

relations

with

as

few

nonzero

variables

as

possible

Since

the

purpose

is

to

minimize

the

number

of

Relevant

Variables

in

Linear

Systems

these

problems

are

referred

to

as

Min

RVLSR

with

R

f

g.

Min

RVLS

is

called

minimum

weight

solution

to

linear

equations

in

the

list

of

NP-complete

problems

in

-LSB-

-RSB-

There

is

a

simple

relation

between

the

approximability

of

Min

ULR

and

that

of

Min

RVLS

Proposition

Min

ULRR

with

R

f

g

is

at

least

as

hard

to

approximate

as

Min

RVLSR

with

the

same

type

of

relations

Proof

It

suffices

to

note

that

for

any

instance

of

Min

RVLS

given

by

Ax

b

one

can

construct

an

equivalent

instance

of

Min

ULR

This

is

simply

achieved

by

considering

for

each

variable

xi

with

i

n

the

equation

xi

-LRB-

-RRB-

and

by

eliminating

variables

in

this

system

using

the

set

of

equations

Ax

b

that

must

be

satisfied

Similarly

by

replacing

each

equation

-LRB-

-RRB-

by

two

nonstrict

inequalities

one

shows

that

Min

ULR

is

at

least

as

hard

to

approximate

as

Min

RVLS

Since

by

proposition

Min

ULR

is

equivalent

to

Min

ULR

for

systems

with

integer

coefficients

Min

ULR

is

also

at

least

as

hard

as

Min

RVLS

Thus

any

nonapproximability

bound

for

Min

RVLSR

with

R

f

g

implies

an

equivalent

bound

for

the

corresponding

Min

ULRR

Proposition

Min

RVLS

can

not

be

approximated

within

any

constant

factor

unless

P

NP

Proof

By

reduction

from

a

variant

of

Set

Covering

with

disjoint

sets

In

Set

Covering

given

a

collection

C

Cng

of

subsets

of

a

finite

set

S

one

seeks

a

subcollection

C

of

minimum

cardinality

such

that

-LSB-

mi

S

with

m

n.

Any

such

is

a

cover

of

S

If

all

the

sets

in

are

pairwise

disjoint

it

is

an

exact

cover

The

proof

is

based

on

the

following

result

by

Bellare

et

al.

on

exact

covers

-LSB-

-RSB-

For

every

c

there

exists

a

polynomial

time

reduction

that

transforms

any

instance

OE

of

the

satisfiability

problem

Sat

-LRB-

see

-LSB-

-RSB-

-RRB-

into

an

instance

of

Set

Covering

with

a

positive

integer

K

such

that

ffl

if

OE

is

satisfiable

there

exists

an

exact

cover

of

size

K

ffl

if

OE

is

unsatisfiable

no

set

cover

has

size

less

than

bc

Kc

For

any

instance

-LRB-

S

C

-RRB-

of

set

cover

we

can

construct

a

system

with

n

variables

and

jSj

equations

Ax

-LRB-

-RRB-

where

aij

if

the

ith

element

of

S

belongs

to

Cj

and

otherwise

denotes

the

jSjdimensional

vector

with

all

components

Clearly

the

nonzero

variables

in

any

solution

x

of

the

above

system

define

a

set

cover

Conversely

given

any

exact

cover

of

cardinality

K

the

vector

x

given

by

xj

-LRB-

if

Cj

otherwise

satisfies

all

equations

and

has

K

nonzero

variables

Thus

the

minimum

number

of

nonzero

variables

in

a

solution

of

Ax

is

either

K

or

greater

or

equal

to

bc

Kc

Note

that

while

there

is

a

one

-LCB-

to

-LCB-

one

correspondence

between

the

exact

covers

and

the

solutions

of

-LRB-

-RRB-

with

or

components

minimum

cardinality

covers

do

not

necessarily

correspond

to

a

solution

For

example

in

the

set

covering

instance

associated

to

B

CA

x

B

CA

all

subcollections

of

cardinality

are

minimum

-LRB-

nonexact

-RRB-

covers

but

the

system

has

the

unique

solution

-LRB-

-RRB-

with

three

nonzero

variables

For

infeasible

systems

like

B

CA

x

B

CA

the

minimum

number

of

nonzero

variables

can

be

considered

as

larger

than

n.

According

to

proposition

the

above

result

directly

implies

that

Min

ULR

can

not

be

approximated

within

any

constant

factor

unless

P

NP

which

was

established

in

-LSB-

-RSB-

Furthermore

propositions

and

show

that

Min

ULR

and

Min

ULR

are

at

least

as

hard

to

approximate

In

fact

Min

RVLS

and

Min

RVLS

with

inequalities

are

not

approximable

within

a

factor

of

n

for

any

In

order

to

prove

this

strong

nonapproximability

bound

we

consider

a

natural

variant

of

the

Label

Cover

problem

studied

in

the

above

reference

Definition

Let

G

-LRB-

E

-RRB-

be

a

bipartite

graph

and

two

sets

of

possible

labels

for

the

nodes

in

and

respectively

and

a

relation

consisting

of

admissible

pairs

of

labels

for

each

edge

A

labelling

of

G

is

a

function

P

-LSB-

-LSB-

that

assigns

a

set

of

possible

labels

to

each

node

of

G

We

say

that

a

labelling

P

covers

an

edge

e

-LSB-

-RSB-

if

the

two

sets

of

labels

P

-LRB-

-RRB-

and

P

-LRB-

-RRB-

are

nonempty

and

for

every

P

-LRB-

-RRB-

there

exists

an

admissible

label

P

-LRB-

-RRB-

such

that

-LRB-

e

-RRB-

A

labelling

that

covers

all

edges

of

G

is

called

a

total

cover

Furthermore

the

cost

of

a

cover

P

is

defined

as

P

-LSB-

jP

-LRB-

v

-RRB-

j

The

Total

Label

Cover

problem

is

then

to

find

a

minimum

cost

cover

The

size

of

an

instance

of

Total

Label

Cover

is

the

total

length

of

a

description

of

the

sets

G

and

The

only

difference

between

Total

Label

Cover

and

Label

Cover

is

in

the

objective

function

In

Total

Label

Cover

the

cost

is

equal

to

the

total

number

of

labels

needed

to

label

all

the

nodes

of

the

bipartite

graph

-LRB-

instead

of

only

considering

those

in

-RRB-

Lemma

For

any

fixed

there

exists

a

quasi

-LCB-

polynomial

that

reduces

an

instance

OE

of

Sat

of

size

s

to

an

instance

of

Total

Label

Cover

of

size

with

s

such

that

ffl

if

OE

is

satisfiable

there

is

a

total

cover

with

cost

ffl

if

OE

is

unsatisfiable

then

any

labelling

covering

more

than

half

of

the

edges

has

cost

greater

or

equal

to

-LRB-

-RRB-

Proof

By

construction

the

bipartite

graphs

derived

from

Feige

's

and

Lov?az

's

-LCB-

prover

-LCB-

round

interactive

proof

systems

have

the

following

properties

-LRB-

see

-LSB-

-RSB-

-RRB-

The

graphs

are

regular

-LRB-

every

node

has

the

same

degree

-RRB-

and

for

any

e

and

there

is

at

most

one

such

that

-LRB-

e

-RRB-

Furthermore

we

may

assume

that

the

relation

is

restricted

to

have

only

valid

labels

for

any

node

in

where

is

valid

for

if

for

every

reduction

is

said

quasi

-LCB-

polynomial

if

it

has

an

O

-LRB-

spolylog

s

-RRB-

time

-LCB-

complexity

edge

e

incident

to

there

is

a

label

such

that

-LRB-

e

-RRB-

In

fact

the

total

label

covers

corresponding

to

any

satisfiable

OE

use

only

one

label

per

node

of

the

graph

Thus

one

easily

verify

that

the

gap

for

Label

Cover

implies

an

equivalent

gap

for

Total

Label

Cover

Theorem

Min

RVLSR

with

R

f

g

can

not

be

approximated

within

a

factor

of

n

for

any

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

where

n

is

the

number

of

variables

Proof

For

any

Total

Label

Cover

instance

-LRB-

G

-RRB-

arising

from

a

Feige-Lov

az

interactive

proof

system

one

can

construct

a

particular

instance

of

Min

RVLS

with

exactly

the

same

objective

function

Therefore

we

can

use

the

first

part

of

the

construction

described

in

-LSB-

-RSB-

The

idea

is

to

introduce

one

variable

for

each

pair

-LRB-

vi

bi

-RRB-

where

vi

Vi

and

bi

Bi

i

which

satisfy

the

following

conditions

While

every

pair

-LRB-

-RRB-

with

and

is

considered

only

the

pairs

-LRB-

-RRB-

where

and

is

valid

are

retained

For

each

pair

e

the

unique

label

such

that

-LRB-

e

-RRB-

is

denoted

by

-LSB-

e

-RSB-

We

construct

an

inhomogeneous

system

Ax

-LRB-

-RRB-

with

jEj

-LRB-

-RRB-

equations

namely

for

each

edge

e

E.

For

each

column

of

A

the

components

corresponding

to

a

given

e

are

called

its

e

-LCB-

projection

The

columns

are

defined

as

follows

For

each

pair

the

e

-LCB-

projection

is

uj

-LRB-

the

-LCB-

dimensional

vector

in

which

the

jth

component

is

and

all

the

others

-RRB-

and

otherwise

where

denotes

the

trivial

vector

with

all

zero

components

For

each

valid

pair

the

e

-LCB-

projection

is

-LSB-

e

-RSB-

if

e

is

incident

to

and

otherwise

Thus

we

have

a

linear

system

with

jEj

-LRB-

-RRB-

equations

and

at

most

variables

It

is

easily

verified

that

the

nonzero

variables

of

any

solution

x

of

-LRB-

-RRB-

define

a

total

cover

of

the

original

bipartite

graph

G.

Thus

according

to

lemma

Min

RVLS

can

not

be

approximated

within

the

given

factor

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

By

inspecting

the

construction

one

observes

that

the

same

bound

holds

even

for

homogeneous

systems

with

binary

coefficients

Since

any

equation

can

be

replaced

by

two

nonstrict

inequalities

the

result

is

also

valid

for

Min

RVLS

Using

the

equivalence

between

systems

with

nonstrict

and

strict

inequalities

we

obtain

the

same

bound

for

Min

RVLS

This

nonapproximability

result

can

be

still

strengthened

Theorem

Min

RVLSR

with

R

f

g

is

not

approximable

in

polynomial

time

within

a

factor

of

n

for

any

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

where

n

is

the

number

of

variables

Proof

We

proceed

by

self

-LCB-

improvement

as

in

-LSB-

-RSB-

The

idea

is

to

consider

the

reduction

in

proposition

from

set

cover

to

Min

RVLS

with

a

fixed

constant

gap

c

between

the

satisfiable

and

unsatisfiable

cases

and

to

increase

it

recursively

For

any

particular

set

covering

instance

we

start

with

the

corresponding

system

-LRB-

-RRB-

whose

coefficients

are

or

and

we

construct

the

squared

system

-LRB-

-RRB-

obtained

by

replacing

each

coefficient

of

A

by

the

whole

matrix

A

and

each

coefficient

by

the

p?n

matrix

with

all

Thus

is

a

matrix

of

size

and

is

a

vector

with

components

all

equal

to

Clearly

if

the

corresponding

instance

OE

of

the

Sat

is

satisfiable

there

is

a

solution

x

of

-LRB-

-RRB-

with

variables

equal

to

and

all

the

other

to

On

the

contrary

if

OE

is

unsatisfiable

any

solution

x

that

satisfies

all

equations

of

-LRB-

-RRB-

has

at

least

nonzero

variables

By

applying

this

construction

t

times

recursively

we

obtain

a

system

with

equations

and

variables

Let

us

take

t

blog

-LRB-

log

n

-RRB-

c

where

n

is

the

number

of

variables

in

the

Min

RVLS

instance

corresponding

to

the

considered

Sat

instance

OE

and

is

a

positive

real

number

The

construction

requires

O

-LRB-

npolylog

n

-RRB-

time

because

the

system

has

pO

-LRB-

log

n

-RRB-

equations

and

nO

-LRB-

log

n

-RRB-

-LRB-

log

n

-RRB-

variables

Since

log

O

-LRB-

log

n

-RRB-

we

obtain

a

gap

of

clog

n

-LRB-

-RRB-

=-LRB-

-RRB-

The

theorem

follows

by

contradiction

Suppose

there

exists

a

polynomial

time

algorithm

that

approximates

Min

RVLS

instances

with

n

variables

within

a

factor

n

for

any

=-LRB-

-RRB-

By

applying

it

to

the

resulting

instance

of

Min

RVLS

one

could

decide

in

O

-LRB-

npolylog

n

-RRB-

time

whether

any

given

instance

OE

of

Sat

is

satisfiable

But

this

would

imply

NP

DTIME

-LRB-

npolylog

n

-RRB-

The

same

bound

is

also

valid

for

strict

and

nonstrict

inequalities

This

result

for

Min

RVLS

immediately

implies

the

nonapproximability

of

Min

ULRR

with

R

f

g

within

the

same

factor

In

fact

there

is

a

reduction

from

homogeneous

Min

ULRR

to

Min

RVLSR

with

R

f

g.

Proposition

Min

ULRR

with

R

f

g

is

equally

hard

to

approximate

as

Min

RVLSR

with

the

same

type

of

relations

Proof

Min

RVLSR

with

R

f

g

can

be

reduced

to

Min

ULRR

by

proposition

and

Min

ULRR

can

be

transformed

to

homogeneous

Min

ULRR

as

shown

in

the

proof

of

theorem

Thus

it

suffices

to

show

that

homogeneous

Min

ULRR

can

be

reduced

to

Min

RVLSR

for

R

f

g.

Let

Pnj

aijxj

with

i

p

be

an

instance

of

homogeneous

Min

ULR

We

construct

an

instance

of

Min

RVLS

consisting

of

p

-LRB-

n

-RRB-

equations

of

the

following

type

nX

j

aijxj

yik

where

i

p

k

n

and

yik

are

p

-LRB-

n

-RRB-

new

variables

If

s

equations

of

Ax

are

satisfied

then

we

will

get

a

solution

of

Min

RVLS

with

between

s

-LRB-

n

-RRB-

and

s

-LRB-

n

-RRB-

n

variables

equal

to

zero

Conversely

a

solution

of

Min

RVLS

with

between

s

-LRB-

n

-RRB-

and

s

-LRB-

n

-RRB-

n

variables

equal

to

zero

will

give

us

a

solution

of

Min

ULR

with

s

satisfied

equations

The

reduction

is

an

S-reduction

with

size

amplification

O

-LRB-

pn

-RRB-

The

same

construction

can

be

used

for

Min

RVLS

just

by

substituting

with

Note

that

due

to

the

size

amplification

the

nonapproximability

bound

for

Min

ULR

established

in

-LSB-

-RSB-

leads

to

a

smaller

bound

for

Min

RVLS

than

that

of

theorems

and

Using

the

same

reduction

the

complementary

maximization

problem

Max

IVLS

Maximum

number

of

Irrelevant

Variables

in

Linear

Systems

can

be

shown

in

its

homogeneous

version

to

be

equally

hard

to

approximate

as

homogeneous

Max

FLS

i.e.

not

approximable

within

p

for

some

unless

P

NP

-LSB-

-RSB-

Interestingly

Max

IVLS

and

Max

IVLS

are

much

harder

to

approximate

than

Max

FLS

and

Max

FLS

respectively

It

is

easily

proved

that

the

former

problems

are

harder

than

the

maximum

independent

set

problem

-LRB-

which

is

not

approximable

within

n

for

some

unless

P

NP

where

n

is

the

number

of

nodes

-RRB-

while

the

latter

ones

can

be

approximated

within

-LSB-

-RSB-

It

suffices

to

construct

for

each

edge

e

-LSB-

vi

vj

-RSB-

the

inequality

xi

xj

or

xi

xj

and

to

observe

that

there

is

a

correspondence

between

the

independent

sets

of

cardinality

at

least

s

and

the

solutions

with

at

least

s

zero

components

Approximability

of

weighted

and

constrained

Min

ULR

In

many

practical

situations

different

relations

may

have

different

importances

This

can

be

modeled

by

assigning

a

weight

to

each

relation

and

by

seeking

a

solution

that

minimizes

the

total

weight

of

the

unsatisfied

relations

Interesting

special

cases

of

weighted

Min

ULR

include

the

constrained

versions

where

some

relations

are

mandatory

while

the

others

are

optional

CMin

with

f

g

denotes

the

variant

where

the

mandatory

relations

are

of

type

and

the

optional

ones

of

type

When

the

problem

can

be

seen

as

a

weighted

Min

problem

in

which

the

weight

of

every

mandatory

relation

is

larger

than

the

total

weight

of

all

optional

ones

Proposition

Weighted

Min

ULRR

with

R

f

g

and

positive

integer

weights

that

are

bounded

in

size

by

a

polynomial

in

the

size

of

the

whole

input

are

equally

hard

to

approximate

as

the

basic

versions

Proof

Basic

Min

ULRR

is

clearly

a

special

case

of

weighted

Min

ULRR

where

all

weights

are

equal

to

one

For

proving

the

other

direction

it

suffices

to

note

that

any

weighted

instance

can

be

associated

to

an

equivalent

unweighted

one

This

is

simply

achieved

by

making

for

each

relation

a

number

of

copies

equal

to

the

corresponding

weight

The

number

of

relations

will

still

be

polynomial

since

the

weights

are

polynomially

bounded

This

equivalence

between

weighted

and

unweighted

variants

holds

also

for

the

constrained

versions

where

the

optional

relations

and

mandatory

relations

are

of

the

same

type

To

make

sure

that

the

mandatory

relations

are

satisfied

a

large

enough

number

of

equivalent

relations

is

required

namely

more

than

the

number

of

optional

ones

Thus

the

approximability

of

constrained

Min

ULR

can

be

very

well

characterized

using

the

unconstrained

versions

It

is

worth

noting

that

no

such

relation

exists

between

weighted

and

unweighted

versions

of

the

complementary

problems

Max

FLS

As

we

proved

in

-LSB-

-RSB-

just

enforcing

some

mandatory

relations

makes

Max

FLS

with

inequalities

harder

to

approximate

While

Max

FLS

and

Max

FLS

can

be

approximated

within

a

factor

the

constrained

variants

are

at

least

as

hard

as

the

maximum

independent

set

problem

and

hence

can

not

be

approximated

within

a

factor

n

for

some

where

n

is

the

instance

size

Any

instance

of

a

constrained

problem

CMin

ULR

R

with

R

f

g

can

be

transformed

into

an

equivalent

instance

of

Min

ULRR

by

eliminating

variables

in

the

set

of

optional

relations

using

the

set

of

mandatory

ones

All

the

problems

C

Min

ULRR

with

R

f

g

are

trivial

for

the

same

reason

as

Min

see

section

Proposition

C

Min

ULR

is

equally

hard

to

approximate

as

Min

ULR

Proof

Proposition

tells

us

that

Min

ULR

can

be

reduced

to

Min

ULR

with

nonnegative

variables

This

can

be

expressed

as

a

CMin

ULR

problem

where

the

mandatory

inequalities

ensure

that

all

variables

are

nonnegative

Conversely

CMin

ULR

can

be

reduced

to

CMin

ULR

by

substituting

each

optional

equation

by

two

inequalities

as

in

the

proof

of

proposition

We

can

then

use

the

standard

reduction

from

C

Min

ULR

to

Min

ULR

Similarly

we

can

show

that

C

Min

ULR

is

at

least

as

hard

to

approximate

as

Min

ULR

Proposition

C

Min

is

Min

Dominating

Set-hard

Therefore

it

can

not

be

approximated

within

any

constant

unless

P

NP

and

not

within

c

log

n

for

any

c

unless

NP

DTIME

-LRB-

nlog

log

n

-RRB-

even

when

restricted

to

homogeneous

systems

with

binary

coefficients

in

Proof

The

proof

is

by

reduction

from

Min

Dominating

Set

Let

G

-LRB-

V

E

-RRB-

be

an

arbitrary

instance

of

Min

Dominating

Set

For

each

node

vi

V

i

n

we

consider

the

optional

equation

xi

-LRB-

-RRB-

and

the

mandatory

inhomogeneous

relation

xi

X

-LRB-

vi

-RRB-

xj

-LRB-

-RRB-

where

j

is

included

in

N

-LRB-

vi

-RRB-

if

and

only

if

vj

is

adjacent

to

vi

We

claim

that

there

exists

a

dominating

set

in

G

of

size

at

most

s

if

and

only

if

there

exists

a

solution

that

violates

at

most

s

relations

of

the

corresponding

linear

system

Given

a

dominating

set

V

V

of

size

s

the

solution

x

defined

by

xi

-LRB-

if

vi

V

otherwise

satisfies

all

mandatory

relations

and

all

but

s

of

the

optional

equations

Conversely

given

a

solution

vector

x

that

satisfies

all

mandatory

relations

and

violates

s

of

the

optional

relations

we

consider

the

set

of

nodes

V

V

containing

all

nodes

vi

such

that

xi

V

is

clearly

a

dominating

set

of

size

s

because

xi

P

-LRB-

vi

-RRB-

xj

only

when

at

least

one

of

the

variables

is

nonzero

which

corresponds

to

the

case

where

at

least

one

of

the

nodes

is

in

the

dominating

set

Hardness

of

approximation

for

bounded

discrete

variables

In

this

section

we

determine

the

approximability

of

Min

ULRR

and

Min

RVLSR

with

R

f

g

when

the

variables

are

restricted

to

take

a

finite

number

of

discrete

values

In

particular

we

consider

the

cases

with

binary

variables

in

and

bipolar

ones

in

f

The

corresponding

variants

of

Min

ULR

and

Min

RVLS

are

referred

to

as

Bin

Min

ULR

Bip

Min

ULR

Bin

Min

RVLS

and

Bip

Min

RVLS

respectively

Theorem

The

following

results

are

valid

for

every

combination

of

f

g.

Bin

Min

and

C

Bin

Min

are

NPO

PB-complete

Assuming

NP

C

Bin

Min

can

not

be

approximated

within

for

any

and

Bin

Min

can

not

be

approximated

within

for

any

where

s

maxfn

pg

i.e.

the

maximum

of

the

number

of

variables

and

the

number

of

relations

Proof

We

first

show

the

result

for

the

problem

C

Bin

Min

ULR

and

then

extend

the

result

to

the

other

variants

We

proceed

by

reduction

from

Min

Ind

Dom

Set

which

is

defined

as

follows

-LSB-

-RSB-

Given

an

undirected

graph

G

-LRB-

V

E

-RRB-

find

a

minimum

cardinality

independent

set

V

V

that

dominates

all

nodes

of

G.

Let

G

-LRB-

V

E

-RRB-

be

an

arbitrary

instance

of

Min

Ind

Dom

Set

For

each

node

vi

V

i

n

we

consider

the

optional

inequality

xi

-LRB-

-RRB-

and

the

inhomogeneous

mandatory

inequality

xi

X

-LRB-

vi

-RRB-

xj

-LRB-

-RRB-

where

j

is

included

in

N

-LRB-

vi

-RRB-

if

and

only

if

vj

is

adjacent

to

vi

Furthermore

we

construct

for

each

edge

-LSB-

vi

vj

-RSB-

E

the

inhomogeneous

mandatory

inequality

xi

xj

-LRB-

-RRB-

Thus

we

have

a

system

with

n

variables

n

optional

inequalities

and

n

jEj

mandatory

ones

We

claim

that

there

exists

an

independent

dominating

set

in

G

of

size

at

most

s

if

and

only

if

there

exists

a

solution

x

that

violates

s

relations

of

the

corresponding

linear

system

Given

an

independent

dominating

set

V

V

of

size

s

the

solution

x

defined

by

xi

-LRB-

if

vi

V

otherwise

satisfies

all

mandatory

inequalities

and

n

s

of

the

optional

ones

Conversely

given

a

solution

vector

x

that

violates

s

of

the

relations

we

consider

the

set

of

nodes

V

V

containing

all

nodes

vi

such

that

xi

V

is

clearly

of

size

s

independent

-LRB-

because

of

the

mandatory

relations

-LRB-

-RRB-

-RRB-

and

dominating

-LRB-

because

of

the

mandatory

relations

-LRB-

-RRB-

-RRB-

The

theorem

follows

because

Min

Ind

Dom

Set

is

NPO

PB-complete

and

can

not

be

approximated

within

for

any

where

n

is

the

sum

of

the

number

of

nodes

and

edges

in

the

graph

For

the

other

constrained

problems

C

Bin

Min

we

use

the

same

reduction

as

above

but

the

right

hand

side

of

the

three

types

of

relations

must

be

substituted

according

to

the

following

table

type

type

type

type

type

-LRB-

-RRB-

type

-LRB-

-RRB-

jN

-LRB-

vi

-RRB-

j

jN

-LRB-

vi

-RRB-

j

X

j

yij

type

-LRB-

-RRB-

zij

In

the

case

of

mandatory

equations

we

need

to

introduce

n

additional

slack

variables

yij

and

jEj

additional

slack

variables

zij

Thus

the

total

number

of

variables

will

be

that

is

still

a

linear

number

in

n

and

jEj

To

get

the

hardness

results

for

the

unconstrained

problems

we

add

jV

j

copies

of

each

mandatory

relation

so

that

there

are

more

valuable

than

the

optional

ones

Since

such

a

reduction

has

a

quadratic

size

amplification

we

get

a

weaker

nonapproximability

bound

than

for

Min

Ind

Dom

Set

It

is

worth

noting

that

Bin

Min

and

C

Bin

Min

with

f

g

remain

NPO

PB-complete

for

homogeneous

systems

In

the

above

reduction

from

Min

Ind

Dom

Set

we

multiply

each

nonzero

constant

in

the

right

hand

side

of

a

relation

by

a

new

variable

which

should

be

viewed

as

the

constant

In

order

to

prevent

from

being

zero

we

introduce

new

mandatory

relations

or

with

a

new

variable

depending

on

the

type

of

relations

In

the

case

of

nonstrict

inequalities

and

equalities

we

introduce

-LRB-

as

in

the

proof

of

theorem

-RRB-

a

lot

of

copies

of

the

relations

in

order

to

make

the

variable

occurring

the

most

frequently

in

the

satisfied

relations

Corollary

The

following

results

are

valid

for

every

combination

of

f

g.

Bip

Min

and

C

Bip

Min

are

NPO

PB-complete

Assuming

NP

C

Bip

Min

can

not

be

approximated

within

for

any

and

Bip

Min

can

not

be

approximated

within

for

any

where

s

maxfn

pg

Proof

By

cost

preserving

transformation

from

Bin

Min

and

C

Bin

Min

For

any

relation

nX

j

aijxj

R

bi

with

binary

variables

xj

and

i

p

we

can

construct

an

equivalent

relation

nX

j

aij

yj

R

nX

j

aij

with

bipolar

variables

yj

f

using

the

variable

substitution

yj

Min

RVLS

restricted

to

binary

or

bipolar

variables

turns

out

to

be

also

very

hard

to

approximate

Proposition

Bin

Min

RVLSR

with

R

f

g

is

NPO

PB-complete

Assuming

NP

neither

Bin

Min

RVLS

nor

Bin

Min

RVLS

can

be

approximated

within

for

any

and

Bin

Min

RVLS

can

not

be

approximated

within

for

any

where

n

is

the

number

of

variables

Proof

As

before

we

proceed

by

reduction

from

Min

Ind

Dom

Set

The

reduction

is

very

similar

to

the

one

used

in

theorem

for

proving

the

NPO

PB-hardness

of

C

Bin

Min

ULRR

R

with

R

f

g

The

Bin

Min

RVLSR

instance

is

simply

composed

of

the

mandatory

relations

constructed

in

the

C

Bin

Min

ULRR

R

instance

The

number

of

violated

optional

relations

in

the

proof

exactly

corresponds

to

the

number

of

nonzero

variables

This

implies

that

Bin

Min

RVLS

and

Bin

Min

RVLS

are

NPO

PB-hard

For

Bin

Min

RVLS

we

still

have

to

deal

with

the

slack

variables

yij

and

zij

that

have

been

added

Suppose

there

is

a

total

number

of

N

slack

variables

In

order

to

make

the

value

of

the

x

variables

more

important

than

the

values

of

the

N

slack

variables

altogether

we

introduce

for

each

variable

xi

N

new

variables

xiN

and

the

N

additional

equations

xi

xij

for

j

-LSB-

N

-RSB-

In

any

solution

x

of

the

resulting

instance

we

will

have

for

each

variable

xi

that

xi

xiN

Consider

the

set

of

nodes

V

V

containing

all

nodes

vi

such

that

xi

V

is

clearly

independent

and

dominating

If

t

variables

in

x

are

equal

to

the

size

of

the

independent

set

will

be

bt

-LRB-

N

-RRB-

c.

Conversely

an

independent

dominating

set

containing

s

nodes

corresponds

to

a

solution

of

the

Bin

Min

RVLS

instance

with

between

s

-LRB-

N

-RRB-

and

s

-LRB-

N

-RRB-

N

variables

equal

to

Thus

the

reduction

is

an

S-reduction

with

size

amplification

O

-LRB-

nN

-RRB-

and

we

get

the

nonapproximability

bound

for

any

where

n

is

the

number

of

variables

Using

the

cost

preserving

transformation

in

the

proof

of

corollary

one

can

show

the

same

nonapproximability

results

for

Bip

Min

RVLSR

with

R

f

g.

Note

that

the

Bin

Min

RVLS

problem

is

exactly

the

same

problem

as

Min

Polynomially

Bounded

Programming

which

was

shown

to

be

NPO

PB-complete

in

-LSB-

-RSB-

Moreover

the

corresponding

maximization

problem

Bin

Max

IVLS

is

the

same

problem

as

Max

Polynomially

Bounded

Programming

which

is

NPO

PB-complete

-LSB-

-RSB-

and

can

not

be

approximated

within

for

any

where

n

is

the

number

of

variables

unless

P

NP

-LSB-

-RSB-

Relation

between

Min

ULR

with

real

and

binary

variables

The

question

naturally

arises

as

to

whether

the

strong

nonapproximability

results

for

the

binary

versions

of

Min

ULR

have

any

consequences

for

the

basic

versions

The

answer

is

positive

if

we

slightly

modify

the

definition

of

the

objective

function

by

introducing

a

threshold

Indeed

we

obtain

a

set

of

problems

that

are

at

least

as

hard

as

the

binary

versions

The

threshold

variant

of

Min

ULR

named

Threshold

Min

ULR

with

any

type

of

relations

is

defined

as

follows

The

input

consists

of

a

linear

system

of

relations

together

with

a

threshold

T

which

is

a

lower

bound

on

the

minimum

number

of

unsatisfiable

relations

of

the

system

Given

a

solution

vector

x

that

violates

s

relations

the

objective

function

is

defined

as

s

T

Since

at

least

T

relations

are

violated

this

value

is

always

nonnegative

Theorem

Threshold

Min

ULRR

with

R

f

g

is

NPO

PB-complete

Assuming

NP

threshold

Min

ULRR

can

not

be

approximated

within

for

any

where

n

is

the

number

of

variables

and

not

within

for

any

where

p

is

the

number

of

relations

Proof

We

proceed

by

reduction

from

Bin

Min

ULRR

with

R

f

g.

Given

an

input

instance

of

Bin

Min

ULRR

with

n

binary

variables

xn

and

p

relations

we

construct

an

instance

of

Threshold

Min

ULRR

by

extending

the

system

with

relations

which

ensure

that

the

variables

just

take

zero

or

one

values

In

the

equality

case

we

add

for

every

variable

xi

p

copies

of

the

equations

xi

and

xi

If

all

variables

in

a

solution

vector

take

values

in

exactly

n

-LRB-

p

-RRB-

of

the

added

equations

will

be

satisfied

and

n

-LRB-

p

-RRB-

violated

For

each

variable

that

is

not

either

zero

or

one

p

additional

equations

will

be

violated

which

are

more

than

the

number

of

equations

in

the

original

system

If

we

choose

n

-LRB-

p

-RRB-

as

the

threshold

we

will

get

an

S-reduction

from

Bin

Min

ULR

to

threshold

Min

ULR

without

variable

amplification

and

with

a

relation

amplification

of

O

-LRB-

np

-RRB-

In

the

proof

of

the

nonapproximability

bound

of

Bin

Min

ULRR

in

theorem

the

number

of

relations

is

about

the

same

as

the

number

of

variables

and

consequently

Threshold

Min

ULR

can

not

be

approximated

within

for

any

where

p

is

the

number

of

relations

For

threshold

Min

ULR

we

use

the

same

construction

but

instead

of

the

two

equations

xi

and

xi

we

include

the

four

inequalities

xi

xi

xi

and

xi

If

all

components

of

a

solution

vector

are

either

zero

or

one

just

one

of

these

four

inequalities

will

be

violated

Otherwise

two

of

the

inequalities

will

be

violated

The

threshold

and

the

rest

of

the

proof

is

the

same

as

for

the

equality

case

Finally

for

threshold

Min

ULR

we

just

use

the

equivalence

between

Min

ULR

and

Min

ULR

described

in

proposition

Two

special

cases

of

Min

ULR

and

Min

RVLS

In

this

section

we

discuss

two

special

cases

of

Min

ULR

and

Min

RVLS

with

inequalities

which

arise

when

designing

two-class

linear

classifiers

and

when

training

perceptrons

Given

a

set

of

vectors

S

k

<=p

ae

Rn

labelled

as

positive

or

negative

examples

we

look

for

a

hyperplane

H

specified

by

a

normal

vector

x

Rn

and

a

bias

such

that

all

the

positive

vectors

lie

on

the

positive

side

of

H

while

all

the

negative

ones

on

the

negative

side

In

other

words

we

seek

a

discriminant

hyperplane

H

which

separates

the

examples

in

the

first

class

from

those

in

the

second

class

This

problem

is

referred

to

as

Min

H-Misclassifications

A

hyperplane

H

is

said

to

be

consistent

with

an

example

ak

if

akx

or

akx

depending

on

whether

ak

is

positive

or

negative

In

the

general

case

where

S

is

nonlinearly

separable

a

natural

objective

is

to

minimize

the

number

of

input

vectors

that

are

-LRB-

see

-LSB-

-RSB-

and

the

included

references

-RRB-

approximability

of

the

complementary

problem

where

one

looks

for

a

hyperplane

which

is

consistent

with

as

many

ak

S

as

possible

has

been

studied

in

-LSB-

-RSB-

Note

that

the

symmetric

variant

of

Min

H-Misclassifications

is

also

frequently

considered

where

we

ask

for

akx

for

positive

ak

and

akx

for

negative

ones

In

-LSB-

-RSB-

a

way

of

extending

the

nonapproximability

bounds

for

Min

ULR

to

the

symmetric

version

of

Min

H-Misclassifications

is

suggested

Although

the

argument

used

does

not

suffice

to

complete

the

proof

it

can

easily

be

fixed

The

problem

is

related

to

the

fact

that

starting

with

any

instance

of

Min

ULR

we

must

construct

a

system

with

strict

inequalities

with

a

particular

variable

playing

the

role

of

the

threshold

As

mentioned

in

-LSB-

-RSB-

one

can

easily

associate

to

any

considered

instance

of

Min

ULR

an

equivalent

inhomogeneous

instance

of

Min

ULR

Therefore

each

equation

is

replaced

by

two

nonstrict

inequalities

and

then

a

new

slack

variable

ffi

is

added

to

change

each

nonstrict

inequality

into

a

strict

one

More

precisely

every

relation

ax

is

replaced

by

ax

ffi

Now

in

order

to

make

sure

that

the

two

systems

are

equivalent

we

must

have

ffi

L

where

L

is

a

given

positive

integer

This

can

of

course

be

guaranteed

by

introducing

a

large

enough

number

of

copies

of

this

strict

inequality

but

then

the

resulting

system

is

not

an

instance

of

symmetric

Min

H-Misclassifications

Indeed

if

ffi

is

considered

as

the

threshold

the

inequalities

ensuring

ffi

L

are

not

homogeneous

Fortunately

there

exists

a

simple

and

general

technique

to

construct

for

any

instance

of

inhomogeneous

Min

ULR

an

equivalent

instance

of

symmetric

Min

H-Misclassifications

Observation

Suppose

we

have

a

system

akx

bk

with

k

p

where

all

bk

are

nonzero

Multiply

each

inequality

by

an

appropriate

constant

so

that

all

right

hand

sides

are

equal

to

By

replacing

all

right

hand

sides

constants

by

a

variable

we

get

a

system

with

either

akx

type

or

akx

type

inequalities

Clearly

any

solution

of

this

new

system

such

that

gives

a

solution

of

the

original

system

Thus

by

adding

a

large

enough

number

of

copies

of

the

two

problems

are

guaranteed

to

be

equivalent

In

order

to

complete

the

reduction

in

-LSB-

-RSB-

we

just

apply

this

technique

to

the

system

consisting

of

ax

ffi

-LRB-

-RRB-

inequalities

and

enough

copies

of

ffi

L

It

is

worth

noting

that

the

same

argument

can

be

used

to

show

that

-LRB-

nonsymmetric

-RRB-

Min

H-Misclassifications

can

not

be

approximated

within

n

for

any

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

A

special

case

of

Min

RVLS

and

Min

RVLS

is

also

of

particular

interest

in

discriminant

analysis

and

machine

learning

The

problem

occurs

when

given

a

linearly

separable

set

S

of

positive

and

negative

examples

one

wants

to

minimize

the

number

of

parameters

that

are

required

to

correctly

classify

all

examples

in

S

-LSB-

-RSB-

In

other

words

we

look

for

a

solution

vector

x

with

as

few

nonzero

components

as

possible

This

objective

which

is

related

to

the

concept

of

parsimony

is

crucial

because

the

number

of

nonzero

parameters

has

a

strong

impact

on

the

performances

of

the

classifier

for

unseen

data

In

-LSB-

-RSB-

the

symmetric

variant

with

strict

inequalities

was

proved

to

be

at

least

as

hard

to

approximate

as

Min

Dominating

Set

Moreover

it

was

shown

that

an

approximate

algorithm

minimizing

the

number

of

nonzero

parameters

within

a

factor

of

O

-LRB-

log

p

-RRB-

where

p

is

the

number

of

examples

would

require

far

fewer

examples

to

achieve

a

given

level

of

accuracy

than

any

algorithm

which

does

not

minimize

this

quantity

Finally

it

was

left

as

an

open

question

whether

this

number

could

be

approximated

within

a

factor

of

O

-LRB-

log

p

-RRB-

We

will

show

elsewhere

that

the

nonapproximability

bounds

for

Min

RVLS

established

in

section

lead

to

a

negative

answer

to

this

question

unless

NP

DTIME

-LRB-

ppolylog

p

-RRB-

The

consequences

of

this

result

on

the

hardness

of

designing

compact

feedforward

neural

networks

will

be

also

discussed

Conclusions

The

various

versions

of

Min

ULRR

with

R

f

g

that

we

have

considered

are

obtained

by

placing

constraints

on

the

coefficients

on

the

variables

and

by

assigning

a

possibly

different

importance

to

each

relation

Table

summarizes

the

nonapproximability

results

that

hold

for

Min

ULR

variants

unless

P

NP

As

seen

in

the

table

the

approximability

of

similar

variants

of

Min

ULR

differs

real

variables

binary

variables

Min

ULR

not

within

any

constant

-LSB-

-RSB-

Min

ULR

Min

Dominating

Set-hard

Min

ULR

Min

trivial

CMin

ULR

as

hard

as

Min

ULR

CMin

ULR

harder

than

Min

ULR

CMin

Min

Dominating

Set-hard

CMin

ULR

as

hard

as

Min

ULR

NPO

PB-complete

CMin

ULR

harder

than

Min

ULR

CMin

harder

than

Min

ULR

CMin

ULR

harder

than

Min

ULR

CMin

ULR

as

hard

as

Min

ULR

CMin

harder

than

Min

ULR

CMin

ULR

CMin

ULR

trivial

CMin

Table

Main

approximability

results

for

Min

ULR

variants

which

hold

provided

NP

The

constrained

versions

CMin

ULR

R

with

mandatory

equations

and

R

f

g

have

the

same

approximability

as

Min

ULRR

depending

on

the

type

of

relations

The

results

are

valid

for

inhomogeneous

systems

with

integer

coefficients

and

no

pairs

of

identical

relations

and

some

of

them

are

still

valid

for

homogeneous

systems

with

ternary

and

even

binary

coefficients

In

order

to

avoid

trivial

solutions

for

relations

that

are

equalities

or

nonstrict

inequalities

we

required

the

variables

occurring

most

frequently

in

the

satisfied

relations

to

be

nonzero

Min

ULRR

with

R

f

g

have

recently

been

shown

not

to

be

approximable

within

any

constant

unless

P

NP

and

not

within

n

for

any

unless

NP

DTIME

-LRB-

npolylog

n

-RRB-

Using

a

simple

reduction

from

Min

Dominating

Set

we

obtained

a

weaker

but

more

likely

logarithmic

bound

for

Min

ULR

as

well

as

Min

ULR

Moreover

we

got

for

free

the

constant

factors

results

In

the

process

of

studying

Min

ULR

we

proved

that

Min

RVLSR

with

R

f

g

can

not

be

approximated

within

a

constant

factor

and

within

n

under

the

usual

assumptions

These

nonapproximability

results

which

are

of

interest

by

their

own

directly

implies

the

bounds

established

in

-LSB-

-RSB-

for

the

basic

versions

of

Min

ULR

Interestingly

both

problems

with

equations

or

nonstrict

inequalities

are

equivalent

to

approximate

The

weighted

variants

of

Min

ULR

are

not

harder

to

approximate

than

the

unweighted

ones

if

the

weights

are

polynomially

bounded

Restricting

the

variables

to

binary

-LRB-

or

bipolar

-RRB-

values

makes

these

problems

harder

to

approximate

namely

NPO

PB-complete

As

to

Min

while

the

basic

version

is

trivial

some

of

the

constrained

variants

are

hard

to

approximate

and

all

binary

versions

even

turn

out

to

be

NPO

PB-complete

The

nonapproximability

bounds

such

as

for

any

makes

the

existence

of

any

nontrivial

approximation

algorithm

extremely

unlikely

It

is

worth

noting

that

the

overall

situation

for

Min

ULR

differs

considerably

from

that

for

the

complementary

Max

FLS

problems

-LSB-

-RSB-

Here

all

basic

versions

can

be

approximated

within

a

factor

n

where

n

is

the

number

of

variables

while

Max

FLS

can

not

be

approximated

within

p

for

some

where

p

is

the

number

of

equations

Some

interesting

questions

related

to

the

approximability

of

Min

ULRR

with

R

f

g

are

still

open

Assuming

NP

can

we

guarantee

a

logarithmic

performance

ratio

c

log

n

for

some

c

Do

these

problems

admit

polynomial

time

algorithms

providing

approximate

solutions

within

n

for

any

References

-LSB-

-RSB-

E.

Amaldi

On

the

complexity

of

training

perceptrons

In

T.

Kohonen

et

al.

editor

Artificial

Neural

Networks

pages

-LCB-

Amsterdam

Elsevier

science

publishing

company

-LSB-

-RSB-

E.

Amaldi

and

C.

G.

Diderich

On

the

probabilistic

and

thermal

perceptron

training

algorithms

Technical

Report

Department

of

Mathematics

Swiss

Federal

Institute

of

Technology

Lausanne

-LSB-

-RSB-

E.

Amaldi

and

V.

Kann

The

complexity

and

approximability

of

finding

maximum

feasible

subsystems

of

linear

relations

Technical

Report

Department

of

Mathematics

Swiss

Federal

Institute

of

Technology

Lausanne

and

Technical

Report

ISSN

Department

of

Numerical

Analysis

and

Computing

Science

Royal

Institute

of

Technology

Stockholm

To

be

published

in

Theoretical

Computer

Science

-LSB-

-RSB-

E.

Amaldi

and

V.

Kann

On

the

approximability

of

finding

maximum

feasible

subsystems

of

linear

systems

In

Proc

Ann

Symp

on

Theoretical

Aspects

of

Comput

Sci.

pages

-LCB-

Springer-Verlag

Lecture

Notes

in

Comput

Sci

-LSB-

-RSB-

S.

Arora

L.

Babai

J.

Stern

and

Z.

Sweedyk

The

hardness

of

approximate

optima

in

lattices

codes

and

systems

of

linear

equations

In

Proc

of

Ann

IEEE

Symp

on

Foundations

of

Comput

Sci.

pages

-LCB-

-LSB-

-RSB-

S.

Arora

L.

Babai

J.

Stern

and

Z.

Sweedyk

The

hardness

of

approximate

optima

in

lattices

codes

and

systems

of

linear

equations

Manuscript

-LSB-

-RSB-

S.

Arora

C.

Lund

R.

Motwani

M.

Sudan

and

M.

Szegedy

Proof

verification

and

hardness

of

approximation

problems

In

Proc

of

Ann

IEEE

Symp

on

Foundations

of

Comput

Sci.

pages

-LCB-

-LSB-

-RSB-

M.

Bellare

S.

Goldwasser

C.

Lund

and

A.

Russell

Efficient

probabilistically

checkable

proofs

and

applications

to

approximation

In

Proc

Twenty

fifth

Ann

ACM

Symp

on

Theory

of

Comp.

pages

-LCB-

ACM

-LSB-

-RSB-

K.

P.

Bennett

and

O.

L.

Mangasarian

Robust

linear

programming

discrimination

of

two

linearly

inseparable

sets

Optimization

Methods

and

Software

-LCB-

-LSB-

-RSB-

P.

Berman

and

G.

Schnitger

On

the

complexity

of

approximating

the

independent

set

problem

Inform

and

Comput.

-LCB-

-LSB-

-RSB-

J.

Chinneck

and

E.

Dravnieks

Locating

minimal

infeasible

constraint

sets

in

linear

programs

ORSA

Journal

on

Computing

-LCB-

-LSB-

-RSB-

V.

Chvatal

Linear

programming

Freeman

and

Co.

New

York

-LSB-

-RSB-

P.

Crescenzi

V.

Kann

and

L.

Trevisan

On

the

structure

of

the

classes

NPO

and

APX

Manuscript

in

preparation

-LSB-

-RSB-

P.

Crescenzi

and

A.

Panconesi

Completeness

in

approximation

classes

Inform

and

Comput.

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

R.

O.

Duda

and

P.

E.

Hart

Pattern

classification

and

scene

analysis

Wiley

New

York

-LSB-

-RSB-

U.

Feige

and

L.

Lov?asz

Two

-LCB-

prover

one

round

proof

systems

Their

power

and

their

problems

In

Proc

Twenty

fourth

Ann

ACM

Symp

on

Theory

of

Comp.

pages

-LCB-

-LSB-

-RSB-

M.

R.

Frean

A

thermal

perceptron

learning

rule

Neural

Computation

-LCB-

-LSB-

-RSB-

S.

I.

Gallant

Perceptron-based

learning

algorithms

IEEE

Trans

on

Neural

Networks

-LCB-

-LSB-

-RSB-

M.

R.

Garey

and

D.

S.

Johnson

Computers

and

Intractability

a

guide

to

the

theory

of

NP-completeness

W.

H.

Freeman

and

Company

San

Francisco

-LSB-

-RSB-

J.

Gleeson

and

J.

Ryan

Identifying

minimally

infeasible

subsystems

of

inequalities

ORSA

Journal

on

Computing

-LCB-

-LSB-

-RSB-

H.

J.

Greenberg

Consistency

redundancy

and

implied

equalities

in

linear

systems

Manuscript

November

-LSB-

-RSB-

H.

J.

Greenberg

and

F.

H.

Murphy

Approaches

to

diagnosing

infeasible

linear

programs

ORSA

Journal

on

Computing

-LCB-

-LSB-

-RSB-

R.

Greer

Trees

and

Hills

Methodology

for

Maximizing

Functions

of

Systems

of

Linear

Relations

volume

of

Annals

of

Discrete

Mathematics

Elsevier

science

publishing

company

Amsterdam

-LSB-

-RSB-

M.

M.

Halld?orsson

Approximating

the

minimum

maximal

independence

number

Inform

Process

Lett.

-LCB-

-LSB-

-RSB-

K-U

Hoffgen

H-U

Simon

and

K.

van

Horn

Robust

trainability

of

single

neurons

Technical

Report

Computer

Science

Department

Brigham

Young

University

Provo

-LSB-

-RSB-

D.

S.

Johnson

and

F.

P.

Preparata

The

densest

hemisphere

problem

Theoretical

Computer

Science

-LCB-

-LSB-

-RSB-

V.

Kann

On

the

Approximability

of

NP-complete

Optimization

Problems

PhD

thesis

Department

of

Numerical

Analysis

and

Computing

Science

Royal

Institute

of

Technology

Stockholm

-LSB-

-RSB-

V.

Kann

Polynomially

bounded

minimization

problems

which

are

hard

to

approximate

In

Proc

of

International

Colloquium

on

Automata

Languages

and

Programming

pages

-LCB-

Springer-Verlag

Lecture

Notes

in

Comput

Sci

-LSB-

-RSB-

N.

Karmarkar

A

new

polynomial

time

algorithm

for

linear

programming

Combinatorica

-LCB-

-LSB-

-RSB-

G.

J.

Koehler

Linear

discriminant

functions

determined

by

genetic

search

ORSA

Journal

on

Computing

-LCB-

-LSB-

-RSB-

G.

J.

Koehler

and

S.

S.

Erenguc

Minimizing

misclassifications

in

linear

discriminant

analysis

Decision

Sciences

-LCB-

-LSB-

-RSB-

P.

G.

Kolaitis

and

M.

N.

Thakur

Logical

definability

of

NP

optimization

problems

Technical

Report

Board

of

Studies

in

Computer

and

Information

Sciences

University

of

California

at

Santa

Cruz

To

be

published

in

Information

and

Computation

-LSB-

-RSB-

C.

Lund

and

M.

Yannakakis

On

the

hardness

of

approximating

minimization

problems

In

Proc

Twenty

fifth

Ann

ACM

Symp

on

Theory

of

Comp.

pages

-LCB-

-LSB-

-RSB-

M.

Marchand

and

M.

Golea

On

learning

simple

neural

concepts

from

halfspace

intersections

to

neural

decision

lists

Network

Computation

in

Neural

Systems

-LCB-

-LSB-

-RSB-

A.

Panconesi

and

D.

Ranjan

Quantifiers

and

approximation

Theoretical

Computer

Science

-LCB-

-LSB-

-RSB-

C.

H.

Papadimitriou

and

K.

Steiglitz

Combinatorial

Optimization

Algorithms

and

Complexity

Prentice

Hall

-LSB-

-RSB-

M.

Parker

and

J.

Ryan

Covering

IIS

's

in

preparation

-LSB-

-RSB-

K.

S.

van

Horn

and

T.

Martinez

The

minimum

feature

set

problem

Technical

Report

Computer

Science

Department

Brigham

Young

University

Provo

-LSB-

-RSB-

R.

E.

Warmack

and

R.

C.

Gonzalez

An

algorithm

for

optimal

solution

of

linear

inequalities

and

its

application

to

pattern

recognition

IEEE

Trans

on

Computers

-LCB-

approximability
computational complexity
feasible subsystems
linear relations
training perceptrons

designing linear classifiers
min dominating set-hard problems
npo pb-complete problems