Project: ArangoDB (App Definition and Development / Database)
File: cochet-terrasson98numerical.pdf

1
Papersubmitted totheIFACConference onSystem Structure andControl (Nantes, France, July8-10 1998).
Nov14,1997. Revised Feb.10,1998.
Numericalcomputationofspectralelements
inmax-plusalgebra
JEANCOCHET-TERRASSON
GUYCOHEN

STÂEPHANEGAUBERT

MICHAELMCGETTRICK

JEAN-PIERREQUADRAT

Abstract
Wedescribethespecialization tomax-plusalgebraof
Howard'spolicyimprovement scheme,whichyieldsanal-
gorithmtocomputethesolutionsofspectralproblemsinthe
max-plussemiring.Experimentally ,thealgorithmshowsa
remarkable(almostlinear)averageexecutiontime.
I.Introduction
Themax-plussemiring	
istheset,equipped
withmax,writtenadditively(  "!$#&%')(*,+ ),and -,writ-
tenmultiplicatively ( ./0-/ ).Thezeroelementwillbe
denotedby 1( 123),theunitelementwillbedenotedby 4
( 45/6).Wewilladopttheusualalgebraicconventions,writing
forinstance 7for 8., 1forthezerovectororzeromatrix(the
dimensionbeingclearfromthecontext),etc.
Thespectralproblemforamatrix9;:<%=
>
?+A@CB7@canbe
writenas9DEGF)D3( (1)
whereD:H%I
>
+J@LKMN1OandF:
>
,i.e.withtheusual
notationPOQ:RTSU(WVXVWV(ZY[7( !#\Z]7^]@
%_9`
^-D
^+> Fa-D&`?( (2)
where Db:c%=deN+J@ hasatleastone®niteentry,andF:2fUg .Asusual,wewillcall Faneigenvalue,and Dan
associatedeigenvector.Whereasthemax-plusspectraltheorem,
whichcharacterizes thesolutionsof(1),isoneofthemoststud-
iedmax-plusresults1,comparatively littlecanbefoundabout
thenumericalsolvingof(1).Unlikeinusualalgebra,themax-
plusspectralproblemcanbesolvedexactlyina®nitenumber
ThisworkwaspartiallysupportedbytheEuropeanCommunityFrameworkIV
programthroughtheresearchnetworkALAPEDES(ªTheAlgebraicApproach
toPerformanceEvaluationofDiscreteEventSystemsº).hBPRE,ÂEtatMajordel'ArmÂeedel'Air,24Bd.Victor,75015Paris,Francei
CentreAutomatique etSystÁemes,ÂEcoledesMinesdeParis,35rueSaint-
HonorÂe,77305Fontainebleau Cedex,France.email:cohen@cas.ensmp.frj
Corresponding author.k
INRIA,DomainedeVoluceau,B.P.105,78153LeChesnayCedex,France.
email:Stephane.Gaubert@inria.frl
CentreAutomatique etSystÁemes.email:gettrick@cas.ensmp.frm
INRIA.email:Jean-Pierre.Quadrat@inria.frn
See[24,26,6,15,16]forhistoricalreferences.Recentpresentations canbe
foundin[1,
3.2.4,
3.7],[7],[14,
3.7].See[22,21]forgeneralizations tothe
in®nitedimensioncase.ofsteps.Thecommonlyreceivedmethodtosolve(1)relieson
Karp'salgorithm[20],whichcomputesthe(unique)eigenvalue
ofanirreducible2matrix 9in oE%pYOqW+time3(infact, oE%pYsrfta+
time,where tisthenumberofnon- 1entriesof 9),and4ou%'YC+
space q.Then,someadditionalmanipulations allowonetoobtain
ageneratingfamilyoftheeigenspace,tocomputeotherinter-
estingspectralcharacteristics suchasthespectralprojector,the
cyclicity,etc.(see[1,v3.7]).Agoodbibliographyonthemaxi-
malcyclemeanproblem,andacomparisonofKarp'salgorithm
withotherclassicalalgorithms,canbefoundin[9].
Thepurposeofthispaperistodescribeaverydifferentalgo-
rithm,whichseemsmoreef®cient,inpractice.
Wewillshowhowthespecialization tothemax-pluscase
ofHoward'smultichainpolicyimprovement algorithm(see
e.g.[10],or[23]forasurvey),whichiswellknowninstochas-
ticcontrol,runsintime5 wxou%'ta+andspace oE%pYC+,where
wyxis
thenumberofiterationsofthealgorithm.Although
w5x,which
dependsonbothYandthenumericalvaluesoftheentriesof9,
seemsdif®culttoevaluate,itsaveragevalueissmall(experimen-
taltestsonfullmatricessuggest
wzx oE%_{=|U}~YC+?+.
Inotherwords,itseemsexperimentally possibletosolveinan
almostlinear(i.e.almostoE%'ta+)averagetimeafamilyofcom-
binatorialproblemsforwhichthebeststandardalgorithmsrun
in oE%pYrta+time.
Weconjecturethattheworstcasevalueofthenumberofiter-
ations
wxispolynomialin t.Examplesshowthatitisatleast
oforder Y.
Themax-plusversionofHoward'salgorithmoutperforms
otherknownmethodswithgoodaverageexecutiontime,such
aslinearprogramming. Theonlyotherfastmethodknowntous
isCuninghame-Green andYixun'salgorithm[8],whichrunsin
time
wzCoE%'tz+,wheretheaveragevalueofthenumberofiter-
ations
wzCisexperimentally oE%'Y W+forfullmatrices,accord-
ingto[8].
Irreducibility isde®nedin
IIIbelow.
Throughoutthepaper,ªtimeºandªspaceºrefertotheexecutiontime(ona
sequentialmachine)andtothememoryspacerequiredbythealgorithm,respec-
tively.
Thenaturalimplementation ofKarp'salgorithm,describedin[20],needs=
,
space.However,itiseasytodesignatwopassesvariant,whichneeds
adoubletime,andrunsinonly
=

space.Asdetailedin[9],itisalsopossible
tooptimizeKarp'salgorithmusingthesometimessparsecharacterofthematrix
thatitbuilds.
ThefamilyofHoward'salgorithmsworksonlyforªnon-degenerateº matri-
ceswithatleastonenon- entryperrow.Forsuchmatrices,
d,and=
&=M

.2
Somepartsofthepresentworkwereinitiatedin[3],andde-
velopedinadifferentdirectionin[4,12].Itisremarkablethat
Howard'spolicyimprovement schemenotonlyprovidesef®-
cientalgorithms,butalsosimpleexistenceproofs.Inparticular,
theexistenceofgeneralizedeigenmodesformax-pluslineardy-
namicalsystemswithseveralincommensurable delays,whichis
statedinvIIIbelow,seemsnew.Asimilarprooftechniquewas
appliedtomin-maxfunctionsin[12].
Thepaperisorganizedasfollows.
InsectionII,wemotivatethemax-plusspectralproblems,by
showinghowfamiliarproblemsinDiscreteEventSystemsthe-
oryandOperationsResearchreducetothespectralproblem(1),
andtosomeofitsextensions.
InsectionIII,webrie¯yrecalltheveryclassicalcharacter-
izationofeigenvaluesofmax-plusmatrices.Wediscusstheir
relationwithcycletimevectors,whichgoverntheasymptotic
behaviorofmax-pluslineardynamicalsystems.Weshowhow
thesecycletimescanbycomputedfromgeneralized eigen-
modes,whichareanonclassicalusefulextensionofthenotion
ofeigenvector,alreadyusedin[12].
InsectionIV,wedescribethemax-plusversionofHoward's
policyiterationalgorithm,whichcomputesgeneralizedeigen-
modes,andwhichinfactshowsthatsucheigenmodesexist.
Theonlynoticeableoriginality,bycomparisonwiththeclassi-
calstochasticcontrolcase,isthatavaluedetermination stepcan
beperformedintime oE%'YC+,usingaspecialgraphexplorational-
gorithmthatwepresentindetail.
InsectionV,illustrativeexamplesandsystematicalnumerical
testsarepresented.
Asmallprototype,writteninC,whichimplementsthemax-
pluspolicyiterationalgorithmdescribedherecanbefoundcur-
rentlyonthewebpagehttp://amadeus.inria.fr/gaubert. Thispro-
totypewillbeintegratedinthemax-plustoolboxofSCILAB6
whichisunderdevelopment.
II.Whatthemax-plusspectraltheorycandoforyou
Inthissection,welistseveralbasicproblemsthatreduceto
thespectralproblem(1)andtosomeofitsextensions.Other
applicationsofthemax-plusspectralproblemcanbefounde.g.
in[22,14],andinthereferencestherein.
PROBLEM1(MAXIMALCIRCUITMEAN).Givenadirected
graph7 %G( +,equippedwithavaluationmap 
	 ,
computethemaximalcircuitmean "!#
a%X+
S
( (3)
wherethe "!$#istakenoverallthecircuits of
,andthesums
aretakenoveralltheedgesof.
Thedenominatorof(3)isthelengthofcircuit.Thenumer-
atoristhevaluationorweightofcircuit.
AfreeopenMATLAB-analogue software,developedatINRIA.Thecurrent
versionofSCILAB(withoutthemax-plustoolbox)canbefoundonhttp://www-
rocq.inria.fr/scilab. 
A(®nite,directed)graphcanbedescribedbya®nitesetofnodes andaset
of(oriented)edges .Inthesequel,wewillusethefamiliarnotions
of(directed)path,(directed)circuit,etc.,withoutfurthercomments.ByTheoremIII.1below,when
isstronglyconnected,coincideswiththe(unique)eigenvalueofmatrix 9 :%=M
*+! uB" ,de®nedasfollows:9 `
^
#a%
Q(%$+if %
Q(%$+ :
,1 %'e +otherwise.(4)
Conversely8,withanymatrix 9 : %=
>
?+A@CB7@,wewillasso-
ciatethegraph
'&withsetofnodes   7S(XVWVXV?(*Y[andset
ofedges 0b7%
Q(($+*) 9 `
^,+b1),equippedwiththevalua-
tion a%
Q(%$$+  9 `
^.Thisbijectivecorrespondence betweenval-
uedgraphs,ontheonehand,andmax-plusmatrices,ontheother
hand,willbeusedsystematically inthesequel.
PROBLEM2(CYCLETIME).Givenamatrix93:/%=
>
+J@OB @
withatleastone®niteentryperrow,computethecycletimevec-
tor-%_9+  {.=/1032
S4rD[%
4+ ( (5)
whereD`
%
4+8 "!$#\*]T^$]@
%_9`
^-MD
^%
4	SX+?+(
PS65
Q5Y8(
P74:98LK)N6Us(
(6)
andtheinitialcondition D[%'6U+ :L	@isarbitrary.
Ofcourse,(6)isnothingbutalinearsysteminthemax-plus
semiring:D %
4+>9D[%
4 SX+,(
P:4:;8 K 67 V(7)
Inotherwords,thecycletimevector
-% 9+determinesthelin-
eargrowthrateofthetrajectoriesofthemax-pluslineardynam-
icalsystem(7).Thefactthat
-% 9+exists,thatitisindepen-
dent9oftheinitialconditionD[%'6U+ :L	@,andthatitcanbecom-
putedfromtheeigenvaluesofthesubmatricesassociatedwith
thestronglyconnectedcomponentsofthegraphof9,willbede-
tailedinProp.III.2below.
Wenextdescribeausefulgeneralization ofthemax-plusspec-
tralproblem,whichrequiresthede®nitionofmax-polynomials.
A(formal,generalized)max-polynomial intheindeterminate <
issimplyaformalsum =>
@?BADC><
>,where
Cisamap FE	
>
(%GH 	
C>,suchthat
C> 1forallbut®nitelymanyvalues
of G:2IE.Wedenoteby 
>
AJ<thesetofsuchpolynomials.
Thegeneralized spectralproblemforapolynomialmatrixK:s%=
>
@<CN+J@OB7@canbewrittenas:K%'FBL
\+?DE D ( (8)M
Notethataccordingto(4)andthroughoutthepaper,thereisanarcfromNtoO
if PRQ SUT
.Thisªdirectºconvention,whichisstandardincombinatorial matrix
theoryandautomatatheory,wasalreadyusedin[14].Theªinverseºconvention
(with PBS%QDT
insteadof PRQ SVT
)wasusedin[1].Thisªinverseºconventionis
standardandpreferablefordiscreteeventsystemapplications,unlessoneaccepts
todealwithlinearsystemsoftheformW
YX
 W
YXZ;[
P,W
\X

beingarow
vector,and Pasquarematrix,insteadofthemorefamiliar W
YX
 PDW
YX]Z^[

,W
\X

beingacolumnvector.Aconsequenceofthecompromisemadeinthis
paper(choosingtheªdirectºconvention,whileconsideringdynamicalsystems
ofthesecondform)isthattheaccessibilityrelation,inProp.III.2andIII.4below,
istheinverseoftheoneusede.g.in[1,18]._
IfsomeentriesofW
a`

arein®nite,thelimitin(5)neednotexist,seee.g.[11,
Remark1.1.10,Chap.VI]and[14,Th.17].TheconditionthatalltheentriesofW
`

are®nite,andthat Phasatleastone®niteentryperrow(whichguarantees
that Psends bdcto bdc,i.e.thattheimageby Pofacolumnvectorwith®nite
entrieshas®niteentries)isfrequentlyusedsinceitseemspracticallyrelevantfor
discreteeventsystemsandmakeslifesimpler.3
where DR:/%=
>
A+A@5KyN1), F :R,and
K%'FL
\+ :/%=
>
+J@OB @
denotesthematrixobtainedbyreplacingeachoccurrenceofthe
indeterminate<byFL
\
( F,withtheusualnotation)inthe
formalexpressionof
K.If
K =>
@?
A9
><
>,with9
>:%=M
,+A@CB7@,thespectralproblem(8)canberewrittenmoreex-
plicitlyas>
@?
A
9
>F
L
>DEGD3( (9)
wherethesumisindeeda®niteone,since 9
>is 1forallbut
®nitelymanyvaluesof G.When
K/9U<,(9)specializesto(1).
Forthisreason,wewillcall Dageneralizedeigenvectorof
KandFageneralizedeigenvalue.
Theappropriategraphicalobjecttobeassociatedwithapoly-
nomialmatrix
K: %=	
J<C+
@OB @isnotavalueddirected
graph,butthebi-valueddirectedmultigraph10 ,withsetof
nodes   7S(XVWVXV?(*Y[,setofedges R T%
Q(%GW(($+ :  rFEur )a% 9
>+J`
^ + 1),initialnodemapIn %
Q(%GW(%$$+8
Q,terminalnode
mapOut %
Q(%GW(($+ $,®rstvaluation  6z%
Q((GW(%$+";%_9
>+J`
^,
andsecondvaluation  C%
Q(%GW(($+5 G.Then,thegeneralized
spectralproblem(8)becomesPOQ: G( D&`  "!$#`
>
^	

%!a%
Q(%GW(($+[ F rC%
Q(%GW(%$$+C-D
^+RV
(10)
WewillseeinTheoremIII.3thatthesolution Fof(10)(which
isuniqueundernaturalconditions)yieldsthesolution
ofthe
followingproblem.
PROBLEM3(MAXIMALCIRCUITMEAN
).Givenamulti-
graph
 %YG(  (In (Out +,equippedwithtwovaluations   	 ,   	 IE,suchthat
C%X+ 6,for
allcircuitsof
,computethe(generalized) maximalcircuit
mean:
"!$#

a%!W+C%X+
( (11)
wherethe"!$#istakenoverallthecircuitsof
.
AsshowninProp.III.4below,thegeneralizedspectralprob-
lem(8)isalsousefulintheeffectivecomputationofcycletimes
ofsomemax-pluslineardynamicalsystems,thatarein®nitedi-
mensional(multi-delay) versionsof(7).
Wewillsaythat
K
=,>
@? A9
><
>: %=
>
J<C+A@OB @isa
goodpolynomialmatrixifithasatleastonenon- 1entryperrow,
andiftherearenocircuitsinthegraphof 9.
PROBLEM4(CYCLETIME
).GivenagoodpolynomialmatrixK:s%=5>
,@<CN+J@OB7@ ,computethecycletimevector-%
K+>;{. /102
S4rD %
4+( (12)
wherethetrajectoryDisnowgivenbythedynamicsD `Z%
4+>;"!$#\*]T^$]@
!#>
@?
A%?%_9
>+A`
^-D
^%
4 GZ+?+,(
P:46G((13)
and %'D %
4+?+
/Lisagiven(bounded)initialcondition,with
"!#  G :LIE ) 9
>
+G1).n
Looselyspeaking,amultigraphisagraphinwhichseveraledgescanlinkthe
samepairofnodes.Formally,a(®nite)multigraphcanbede®nedbya(®nite)
setofnodes ,a(®nite)setofedges ,andtwomapsIn  andOut ! ,whichgivetheinitialnodeandterminalnodeofanedge,respectively.Morealgebraically,(13)canberewrittenasfollows:D[%
4+8
>
@?BA
9
>D[%
4 GZ+,(
P:4"6/V(14)
RemarkII.1.Problems4and2areinfacttwospecialversionsofa
moregeneralproblem(seee.g.[19]).If #isanormedvectorspaceand$%#'&(#isanon-expansive map(i.e.)
$*+-,.$*0/,)213)
+4./)),
thelimit5
*0$-,687:9<;>=@?BABC"DE$
=*+-,,ifitexists,isindependentofthe
initialpoint
+.Problem2dealswiththecasewhen #isequalto F
c,
equippedwiththesupnorm,and
$*+-,26HG2+.InProblem4, #isthe
setofboundedfunctionsfrom I
.JBKML
,to F
c,equippedwiththesup
norm,and
$istheevolutionoperatorwhichwiththepieceoftrajectoryN+O*CP,MQR-SUT
=V(initialcondition),associatesthetrajectoryobtained
afteroneunitoftime:
N+*CXWY?B,MQZR-SUT
=V.Theevolutionoperatoris
obviouslywellde®nedsincetherearenocircuitsinthegraphof
G.Itis
clearlymonotoneandhomogeneous, hence,byasimpleresult[5],itis
non-expansive forthesup-norm.Thus,theexistenceofthelimit(12)
foraparticularboundedfunction
N+*C[,MQ\R-SUT
=V,impliestheexis-
tenceof5
*0$-,,whichisequalto5
*^]_,.Conversely,theexistenceof5
*0$-,clearlyimpliesthatthelimit(12)exists,with5
*^]_,`65
*0$a,.
III.Someclassicalandlessclassicalelementsofmax-plus
spectraltheory
Inallthissection,withamatrix9 :s%=
>
+J@OB @weassociate
thegraph
 & %G( +,equippedwiththevaluation,asde-
®nedinthediscussionfollowingEqn4.Thestronglyconnected
componentsofthegraphof9arecalledclasses.Amatrixisir-
reducibleifitsgraphisstronglyconnected,i.e.ifithasasingle
class.Thefollowingresultisclassical[24,26,6,15,16].See
e.g.[1,7]forrecentpresentations andproofs.
THEOREMIII.1(MAX-PLUSSPECTRALTHEOREM).Anirre-
duciblematrix9 :%IE>
J +J@OB @hasauniqueeigenvalue,given
by(3).
Ingeneral,thereareseveralnon-proportion aleigenvectors
(seee.g.[1]or[14]).Areduciblematrix9hasingeneralsev-
eraldistincteigenvalues,andthemaximalcircuitmean(3)yields
preciselythemaximaleigenvalue(seee.g.[11,Ch.IV],[14],[2]
forcharacterizations ofthespectrumofreduciblematrices).
Wesaythat
Qhasaccessto $ifthereisapathfrom
Qto $in
thegraphof 9.Wesaythat
Qhasaccesstoaclass bifithas
accesstoany $:cb(thispropertyisobviouslyindependentof
thechoiceof $5:db,byde®nitionofaclass).Byªeigenvalueof
aclass bº,wemeantheeigenvalueofthe b3rebsubmatrixof9,whichisuniquebyTheoremIII.1.
Thefollowingresultappearedin[18,Prop.7],and,ina
stochasticcontext,in[1,Th.7.36].
PROPOSITIONIII.2(CYCLETIMEFORMULA).Let 9 :%=

+A@OB @,withatleastone®niteentryperrow.The
Q-thentry
-`*%_9+ofthecycletimevectorisequaltothemaximumofthe
eigenvaluesoftheclassestowhich
Qhasaccess.
Thenextstatementusesthecorrespondence betweenpolyno-
mialmatricesandmultigraphs,describedin vIIabove.Wewill
saythatapolynomialmatrix
Kisirreducibleifitsmultigraph
isstronglyconnected.Moregenerally,wewillnaturallyextend
thenotionsofaccessibility,classes,etc.topolynomialmatrices
(thesenotionsarede®nedasinthecaseofordinarymatrices,but4
replacingthegraph
'&bythemultigraph
_).Thefollowing
resultistakenfrom[1,Th.3.28]
THEOREMIII.3(SPECTRALTHEOREM
).Anirreducible
polynomialmatrix
K
=>
@?BA9
><
>: %I
>
@<CN+J@OB @,
suchthatthegraphof9hasnocircuits11,admitsaunique
generalizedeigenvalueF,givenby(11).
ThefollowingextensionofProp.III.2isimmediate.
PROPOSITIONIII.4(CYCLETIMEFORMULA
).Let
Kdenote
agoodpolynomialmatrix.The
Q-thentry
-`*%
K+ofthecycletime
vectorisequaltothemaximumofthegeneralizedeigenvaluesof
theclassestowhich
Qhasaccess .
Sincethedecomposition ofadirectedgraphormultigraphin
stronglyconnectedcomponentscanbedoneinlineartimeus-
ingTarjan'salgorithm[25],Prop.III.2andProp.III.4reducein
lineartimethecomputationofthecycletimevectortothecom-
putationofthe(possiblygeneralized)eigenvaluesofirreducible
(possiblypolynomial)matrices.Inparticular,thetraditionalway
tocomputethecycletimevector
-%_9+istocomputetheeigen-
valuesoftheclassesof9viaKarp'salgorithm[20],andthen
toapplyProp.III.2.Thismethoddoesnotworkforthegen-
eralizeddynamics(13),sinceKarp'salgorithmcannotcompute
generalizedeigenvalues.Therearetwotraditionalwaystoover-
comethisdif®culty.ÐWhen9
>iszeroexceptforintegerval-
uesof G,aneliminationoftheimplicitpartandafamiliaraug-
mentationofstatereducesthegeneralizedspectralproblemforKtoanordinaryspectralproblemforalargermatrix 9
.This
method,whichispresentedin[1, v2.5.3, v2.5.4],isnotsoex-
pensivewhenthenumberofvaluesof Gforwhich 9
>
+b1is
small,particularlyifitisimplementedwithsomere®nements,as
in[13],for
K/9
R9
\<.ÐThesecondmethodreliesonthe
generaltechniquespresentedin[17,AppendixV],whichallow
onetomaximizeinpseudo-polynomial timearatiooftheforma%!,+C%+forina®niteset,providedthatforanyvalueofF :,weknowhowtomaximizeinpolynomialtimetheratioa%!,+[FO%,+ for inthesameset .
Wewillnotdiscussindetailthesetwomoreorlessclassical
approaches,butrathershowhowadifferentgeneralization ofthe
spectralproblemallowsustodeterminedirectlyandinfullgen-
eralitycycletimevectors.Alltheremainingpartofthispaper,
andinparticular,themax-plusversionofHoward'spolicyim-
provementalgorithm,willbebasedonthisnewspectralprob-
lem.
Weconsideragoodpolynomialmatrix
K.Wesaythat%)(*D + : %I5@+isageneralized eigenmode12ifthereexistsnAn
In[1,Th.3.28],itisonlyrequiredthatthecircuitsofthegraphof P
have
negativeweights.Wewillnotneedthisdegreeofgeneralityhere.Intermsofthe
associateddynamicalsystems(13),theconditionofthetheoremsimplymeans
thattherearenocircuitsinvolvingzero-delaycausalityrelations.n_
Thisspectralnotionisobtainedbytwosuccessivegeneralizations ofor-
dinaryspectralproblems.The®rstgeneralization consistsinreplacingordi-
narydynamicalsystemsoftheform(7)(withunitarydelays)bysystemsofthe
form(14)(withmultipledelays).Theordinaryspectralproblem(1)anditsgen-
eralization(9)areobtainedbylookingforsolutionsoftheform W
YX
>
=W,
where

isascalarandW	
b

c
`.Butthede®nitionofcycle-time
vectorsrequires Wtohave®nitecoordinates.Then,inthegeneralcase,asim-
pleaf®neregime W
\X

=W
X

Wneednotexist,but
amoregeneralaf®neregime W
\X
 
=W,where

isadiagonalmatrix,is
expected.Inotherwords,weexpectthedifferententriesof W
\X

tohavedif-
:2suchthat4:L(
4  !
/DE
K%
!L
\+
!
/D3((15)
where
!defdiag%
\(WVXVXVZ("@
+and
!
/diag%
4r#
\(XVWVXV*(
4r@
+.
When
K96<,(15)becomes4:2"(
4"$ !
/DE/9
!
/L
\D3V(16)
Thatis,theactionof 9coincideswiththeactionof
!onthe
orbit 
!
/D
/ L
\.Asdetailedinfootnote12,theeigenmode
equation(15)isobtainedbylookingforanultimatelyaf®neso-
lutionof(13), D %
4+E
!
/D/
4r%5- D.Ifsuchasolution
exists,
-%
K+5 {.=
/
\/rsD %
4+M&.Thenextlemmafollows
readilyfromthisobservation,andfromthefact,mentionedin
RemarkII.1above,thatthelimit{a.=
/
\/r	D[%
4+ isindepen-
dentoftheparticularboundedinitialcondition.
LEMMAIII.5.Ifagoodpolynomialmatrix
Khasageneralized
eigenmode%)(ZD)+,then
-%
K+>'.
Inparticular,if
Kisirreducible,Prop.III.4impliesthat 
-%
K+ %pF (XVWVXV*(ZF)+,whereFisthegeneralizedeigenvalueofK.Therefore,(15)reducestothe(generalized) spectralprob-
lem(8),andDisa(generalized) eigenvectorof
K.I.e.,forirre-
duciblematrices,®ndinggeneralizedeigenmodesisequivalent
to®ndinggeneralizedeigenvectors.
Theexistenceofgeneralizedeigenmodeswasprovedin[12]
when
K96<,asaspecialcaseofamoregeneralresultformin-
maxfunctions.Inthenextsection,wewillshowhowthemax-
plusversionofHoward'spolicyimprovement algorithmallows
ustocomputegeneralizedeigenmodes.Inparticular,thetermi-
nationofthealgorithmwillprovetheexistenceofsucheigen-
modes,forgoodpolynomialmatrices.
IV.Themax-pluspolicyimprovementalgorithm
Inthissection,
Kwillbeagoodpolynomialmatrix.We
willusesystematically themultigraph
4 %YG(  (In(Out+
equippedwiththevaluationsa( ,canonicallyassociatedwithKin vII.
Itcanbecheckedthattheeigenmodeequation(15)which
seemsdeceivinglytoinvolveanin®nitenumberofconditions,
isequivalenttothefollowing®nitesystem:`  "!$#`
>
^ 


^(17)D `c "!$#`
>
^ 
 
%z%
Q((GW(%$$+ C%
Q(%GW(%$$+ r(
^-D
^+R((18)
where 7%
Q((GW(%$$+ :9 ))`
*
^V
Inlooseterms,themultichainpolicyiterationalgorithmwill
solvethissystembytryingtoguessthearcsthatattainthemax-
imum.Aprecisestatementofthisideaneedsthede®nitionof
ferent(linear)growthrates,givenbythediagonalentriesof

.Hence,thesec-
ondgeneralization consistsinsubstitutingW
YX
 +
=Wfor
Xlargeenough(i.e.X-,(./+.)in(14):then,oneobtainspreciselythegeneralizedeigenmode
equation(15).Contrarytothecasewhen
/
,

Rn
neednotcommutewith
thematrices P10,andthus,therelation W
32

RnJW
=05476
AP0

R0W
neednotimplythat

=W
82

Rn?9
=W,for
X:, `.Thisiswhy(15)hasto
bestatedforalllarge
X.5
policy,whichisamap@ 	 ~(suchthatIn%
%
Q+?+ 
Q(
POQ:  V
Thatis,apolicyisjustamapwhichwithanodeassociatesan
edgestartingfromthisnode.
Withapolicy
,weassociatethespecialpolynomialmatrixK =>
@?
A9
><
>:% 9
>+J`
^
z%
%
Q+?+if $yOut %
%
Q+?+and G'C%
%
Q+?+1otherwise.
Hence,thematrix
Khasexactlyonenon-zeroentryperrow,
whichcorrespondstotheedgeselectedby
,i.e.inthemulti-
graphof
K,
%
Q+istheuniqueedgestartingfrom
Q.Ithas
thesamevaluations and asintheoriginalmultigraphof
K.
E.g.,wehavedepictedinFig.2belowthemultigraph(infact,
thegraph)of
K
,where
\isthepolicy S*	 SU(
Q	 ,forQ	&(
&(and
K96<,where9isdisplayedinFig.1.
We®rstshowhowageneralizedeigenmode%)(ZD)+ofamatrix
oftheform
Kcanbecomputedintimeou%'YC+.
ALGORITHMIV.1(VALUEDETERMINATION).Input:agood
polynomialmatrix
Kandapolicy
.Output:ageneralized
eigenmodeof
K.
1.Findacircuit inthemultigraphof
K
2.SetM
a%X+C%!W+
V (19)
3.Selectanarbitrarynode
Qin ,set `
 andset D`to
anarbitraryvalue,say D`
 6.
4.Visitingallthenodes $thathaveaccessto
Qinbackward
topologicalorder,set
^  (20)D
^ z%
%Y$$++> Er C%
%$+?+ -DOut

^	(21)
5.Ifthereisanonemptysetbofnodes$thatdonothave
accessto
Q,repeatthealgorithmusingthe br bsubmatrix
of
Kandtherestrictionof
to b.
Thealgorithmshouldbespeci®edasfollows.
Step1isveryeasytoimplement:wecanstartfromanar-
bitrarynode
Q,movetonode$ Out%
%
Q+?+,thenpossiblyto
nodeOut%
%Y$$++,etc.,untilanodethatisalreadyvisitedisfound.
Then,acircuithasbeenfound.Thisrequiresalineartime.
Eqn(21)requiresvisitingthenodesinbackwardtopological
order,startingfrom
Q,sincethevalueof DOut

^	mustbeal-
ready®xedwhenwevisitnode $andset D
^.Thiscompletevisit
canbedoneinlineartime,atthepriceofanaprioritabulationof
the(multi-valued)-inv erseofthemap  	 G(D$'H 	Out %
%Y$$++.
Computingtheinverseofthismapalsorequiresalineartime.
Thehandlingofthisinverseisinfacttheonlypartofthealgo-
rithmwhichrequiresmorere®neddatatypesthansimplearrays
(inourimplementation, weusedlinearlychainedlists).
Step5isformulatedinarecursivewayonlytosimplifythe
statementofthealgorithm,whichisessentiallynonrecursive.
Theaboveconsiderations justifythefollowingtheorem.
THEOREMIV.2.AlgorithmIV.1computesageneralizedeigen-
modeof
KintimeandspaceoE%pYC+.ThesecondingredientofHoward'salgorithmisapolicyim-
provementroutine,whichgivenapolicy
andageneralized
eigenmode%)(*D +of
K,either®ndsanewpolicy
`suchthat
-%
K+

-%
K+,orprovesthat%)(ZD)+isageneralizedeigen-
modeof
K.
ALGORITHMIV.3(POLICYIMPROVEMENT).Input:agood
polynomialmatrix
K,apolicy
,togetherwithageneral-
izedeigenmode%)(*D +of
K.Output:apolicy
 ,suchthat
-%
K
+

-%
K+.
1.Let13 
Q);"!$#`
>
^ 


^ $`A%
Q+  !?}>!#`
>
^	


^(for
Q S>VXVXV Y, 
Q) "!#
`
>
^	

`

%!a%!W+ C%!W+
^-D
^+  D`
%
Q+  !}>"!$#
`
>
^ 

`

%z%X+8 C%X+
^- D
^+(
for
Q S VWVXVAY.
2.If

, %)(*D +isageneralizedeigenmodeof
K.
Stop.
3.(a)If
+	,weset:
%
Q+ 
#
anyin
%
Q+if
Q:
,%
Q+if
Q+:
.
(b)If
	but
+,weset
%
Q+ 
#
any :
%
Q+if
Q:
,%
Q+if
Q+:
.
Thepolicyimprovement rules3aand3bsimplymeanthat
oneselectsforthenewpolicytheedgeswhichrealizethemaxi-
muminEqns(17),(18).Thismaximumistakenhierarchically:
Eqn(17)haspriorityonEqn(18)inapolicyimprovement step.
OnlywhenEqn(17)issatis®edEqn(18)isusedtodeterminethe
newpolicy.Theotherconditionsinsteps3aand3bsimplymean
thattheprecedingvaluesof
shouldbekeptin
 ,whenever
possible.Thesetechnicaltrickswillguaranteethetermination
ofthepolicyiterationalgorithmbelow,evenwhenªdegenerateº
policyimprovements inwhich
-%
K
+8
-%
K+occur.
Thesets
%
Q+and
%
Q+,whichareintroducedtosimplify
thestatementofthealgorithm,neednotbeexplicitlytabulated.
Clearly,AlgorithmIV.3runsin oE% ) ) +time14and oE%pYC+space15
Wenextstatethemax-plusversionofHoward'spolicyitera-
tionalgorithm.
ALGORITHMIV.4(MAX-PLUSPOLICYITERATION).Input:
agoodpolynomialmatrix
K.Output:ageneralizedeigenmode
of
K.
1.Initialization.Selectanarbitrarypolicy
\.Com-
puteageneralizedeigenmode %
\(*D
\+of
K
,usingAlgo-
rithmIV.1.Set
4S.n_
Recallthatby !#"#$4&%('
*)

,wemeanasusualthesetofelements+suchthat'
+
&!#"
$4&%'
,)

.n_.-
-
simplydenotesthenumberofedgesofthemultigraph.n_
Thealgorithmneedslessinternalmemory(
=

space)thanthecodingof
theinputitself,whichrequires

-
-

space.6
2.Policyimprovement.Improvethepolicy
/,usingAl-
gorithmIV.3withinput

/("s&
/(*D D
/
.Ifthe
stoppingconditionofAlgorithmIV.3issatis®ed,%
/(*D
/+
isageneralizedeigenmodeof
K.Stop.Otherwise,set/E
\
(theoutputofAlgorithmIV.3).
3.Valuedetermination .Findageneralizedeigenmode%
/E
\(ZD
/E
\+of
K
AusingAlgorithmIV.1,takingthe
specialvalue D
/E
\`
GD
/`instep3,IV.1.
4.Increment
4byoneandgotostep2.
Thealgorithmbuildsasequenceofgeneralizedeigenmodes%
/(ZD
/+thatisstrictlyincreasingforthelexicographic orderon%=M@+,de®nedby %'D[(
+lex
%pD
(
+if D D
or D D
and

Z.Thefactthat D
/E
\`mustbesetto D
/`instep3
ofAlgorithmIV.1isaconservativetrickanalogoustothefact
thatthevaluesof
arekeptin
`wheneverpossible,inAlgo-
rithmIV.3.Thistechnicalconditionisessentialtoguaranteethe
strictmonotonicity ofthesequence %
/(ZD
/+,whichisneededin
theproofthatthealgorithmterminates.
Theproofofthefollowingresultissimilartotheproofofthe
maintheoremof[10].Itreliesessentiallyonaversionofthe
maximumprinciplefortransientMarkovchains.Amorealge-
braicversionofthisproof,usinggermsofaf®nefunctions,ap-
pearsin[4],andintheproofoftheresultsannouncedtherein.
THEOREMIV.5.Themax-pluspolicyiterationalgorithmter-
minatesinanumberofiterations
wzxwhichislessthanthenum-
berofpolicies.Oneiterationrequires
\ou% ) ) +time.Thealgo-
rithmrequires oE%pYC+space
\	
.
Indeed,thesamepolicyisneverselectedtwice.Bounding
wx
bythenumberofpolicieswhichis®nitebutexponentialisvery
coarse.Onexperimentalrandomexamples,
wxisverysmall,as
detailedinsectionVbelow.Thefollowingresultisanimmediate
consequenceoftheterminationofthepolicyiterationalgorithm
andofLemmaIII.5.
COROLLARYIV.6.Agoodpolynomialmatrix
Khasagener-
alizedeigenmode %)(*D +.Inparticular,thecycletime
-%
K+ 
exists.
RemarkIV.7.Howard'salgorithmisnotlimitedtospectralproblems.
Itispossibletodesignpolicyiterationalgorithmsfor®xedpointsequa-
tionsoftheform
+ 6eG2+
,where
Gisasquarematrixwithmaximal
eigenvaluestrictlylessthan,and
acolumnvector.Thiswillbede-
tailedelsewhere.
V.Examplesandnumericaltests
A.Illustrativeexample
Weapplythemax-pluspolicyiterationalgorithmtodetermine
theeigenvalueofthematrixdisplayedinFig.1.Thiscorre-
spondstothecasewhere
K 9U<,and  S.Inparticular,
themultigraphof
Kwillbeidenti®edwiththegraphof 9.
1
34
8
2127
23
5
43
P


[   


Fig.1.AmatrixanditsgraphThefollowingrunofthealgorithmisvisualizedinFig.2.We
choosetheinitialpolicy
\: S 	 S,
Q	 ,for
Q  ( 
 (.
ApplyingAlgorithmIV.1,we®nda®rstcircuit
\&S 	 S,with a%!
\+\C%!
\+  S.Weset
\\ S,D
\\36.SinceSisthe
onlynodewhichhasaccesstoS,weapplyAlgorithmIV.1tothe
subgraphof
withnodes&(
&(.We®ndthecircuit
 	 
andsetG z%
+C%
+ 
,
\
 
andD
\
 6.Since
&(haveaccessto ,weset 
\`
 
for
Q 
 (.Moreover,an
applicationof(21)yields D
\q
  
>-RD
\, D
\	  
>-RD
\.To
summarize:
\ ES 
 
 
"!	#( D
\$a6 6 S aS%!&#cV
WeimprovethepolicyusingAlgorithmIV.3.Since
 TS
+,wehaveatype3aimprovement. Thisyields


Q	 ,forQ3S(&(
&(.Onlytheentry Sof D
\
and 
\
hastobemodi®ed,
whichyields


 
 
 
!#"(sD

aS 6 S aS
!# V
Wenexttabulatewithlessdetailstheendoftherunofthealgo-
rithm.AlgorithmIV.3,type3bpolicyimprovement.
q
S9	 (	 
&(
F	  (F	 
.AlgorithmIV.1.Valuedetermination.
Circuitfound, 9 
9	 9	 
, f %z% ( 
U+ -a%
 ( U+?+ L' .
q$)(
(
(
(
!*#"(fD
q 
\Z\
6 
\

+!*# V
AlgorithmIV.3,type3bpolicyimprovement.Theonlychangeis%
U+> .AlgorithmIV.1.Valuedetermination. Circuitfound,U 
 	 	 
,a%z%
&(T+C- a%  (
+?+   SUS .
$
\*\
\*\
\Z\
\Z\
!*#"(sD
,  
\
6

!*# V
AlgorithmIV.3.Stop. SUS isaneigenvalueof 9,and D

isan
eigenvector.
1
12 34
32 1
2 34
2
3442
1
2 34
8
457
1
2 34
8
537
-n
Z [`
-Z [
Z [[
[`[ [/.0
Z [/.12
Q
43.0
N
[

-
-`
2
Q
[ [/.0
N
[

2
Q

N
[

2
nn
[,
2
nQ

N


.0Z [/.1 `
`
W
nn
Fig.2.Thesequenceofpoliciesbuiltbythemax-pluspolicyiteration
algorithm,forthematrix
GdisplayedinFig.1.Thevaluationsof
thenodesindicatethevectors
+
=
,
C_6?65&5&5K7.
B.NumericalTests
Theresultsofthenumericalexperiments displayedin
Fig.3,4,5shouldbeself-explanatory .7
0204060
0200 400 600 8001000 1200 1400
Fig.3.Numberofiterations ofHoward'salgorithmasafunction
ofthedimension,forfullrandommatrices,withi.i.dentriesdis-
tributeduniformlyonaninterval.
050100150200
0 500000 1e+06 1.5e+06 2e+06
Fig.4.Cputime(insec.)ofHoward'salgorithm(inred)vsKarp's
algorithm(inblack)onapentium200Mhzwith500MbofRAM,
asafunctionofthenumberofarcsforfullrandommatrices(same
probabilisticcharacteristics asinFig.3).
References
[1]F.Baccelli,G.Cohen,G.J.Olsder,andJ.P.Quadrat.Synchronizationand
Linearity.Wiley,1992.
[2]R.B.Bapat,D.Stanford,andP.vandenDriessche.Patternpropertiesand
spectralinequalitiesinmaxalgebra.SIAMJournalofMatrixAnalysisand
Applications,16(3):964±97 6,1995.
[3]J.Cochet-Terrasson.ÂEtudeetmiseenúuvredelalgorithmedeHoward
sousdeshypothÁesesfaiblesd'accessibilitÂe.Rapportdestage,ENSTAet
DEAªModÂelisationetMÂethodesMathÂematiquesenÂEconomieº,Univer-
sitÂedeParisI,July1996.
[4]J.Cochet-Terrasson,S.Gaubert,andJ.Gunawardena. Dynamicsofmin-
maxfunctions.1997.Submittedforpublication.AlsoTechnicalReport
HPL-BRIMS-97-13.
050100150200
020000 40000 60000 80000 100000 120000 140000
Fig.5.Cputime(insec.)ofHoward'salgorithm(inred)vsKarp's
algorithm(inblack),onapentium200Mhzwith500MbofRAM,
asafunctionofthenumberofnodes,forsparserandommatrices,
withexactly5arcsstartingfromeachnode.The5successorsofa
givennodearedrawnrandomlyfrom
N?K
5&5&5K
Qwiththeuniform
distribution.Thecorresponding valuationsofthearcsarei.i.d.,with
anuniformdistributiononaninterval.[5]M.G.CrandallandL.Tartar.Somerelationsbetweennonexpansiveand
orderpreservingmaps.ProceedingsoftheAMS,78(3):385±39 0,1980.
[6]R.A.Cuninghame-Green .MinimaxAlgebra.Number166inLecturenotes
inEconomicsandMathematical Systems.Springer,1979.
[7]R.ACuninghame-Green. Minimaxalgebraandapplications. Advancesin
ImagingandElectronPhysics,90,1995.
[8]R.A.Cuninghame-Green andLinYixun.Maximumcycle-means of
weighteddigraphs.Appl.Math.JCU,11B:225±234, 1996.
[9]A.DasdanandR.Gupta.Fastermaximumandminimummeancycleal-
gorithmsforsystemperformanceanalysis.Technicalreport97-07,Univ.
ofCalifornia,Irvine,1997.SubmittedtoIEEE-TransactionsonComputer-
AidedDesign.
[10]E.V.DenardoandB.L.Fox.MultichainMarkovrenewalprograms.SIAM
J.Appl.Math,16:468±487,1968.
[11]S.Gaubert.ThÂeoriedessystÁemeslinÂeairesdanslesdioÈõdes.ThÁese,ÂEcole
desMinesdeParis,July1992.
[12]S.GaubertandJ.Gunawardena. Thedualitytheoremformin-maxfunc-
tions.C.R.A.S,1997.Accepted.
[13]S.GaubertandJ.Mairesse.Modellingandanalysisoftimedpetrinetsus-
ingheapsofpieces.1997.Submittedforpublication.Alsotechnicalreport
LITP/97-14.AnabridgedversioncanbefoundintheProceedingsofthe
ECC'97,Bruxells,1997.
[14]S.GaubertandM.Plus.Methodsandapplicationsof(max,+)linearalge-
bra.InR.ReischukandM.Morvan,editors,STACS'97,number1200in
LNCS,LÈubeck,March1997.Springer.
[15]M.GondranandM.Minoux.ValeurspropresetvecteurspropresenthÂeorie
desgraphes.InProblÁemescombinatoiresetthÂeoriedesgraphes,number
260inColloquesinternationaux CNRS,Orsay,1976.
[16]M.GondranandM.Minoux.Valeurspropresetvecteurspropresdansles
dioÈõdesetleurinterprÂetationenthÂeoriedesgraphes.EDF,Bulletindela
DirectiondesEtudesetRecherches,SerieC,MathÂematiquesInformatique,
2:25±41,1977.
[17]M.GondranandM.Minoux.Graphesetalgorithmes.Eyrolles,Paris,
1979.Engl.transl.GraphsandAlgorithms,Wiley,1984.
[18]J.Gunawardena. Cycletimeand®xedpointsofmin-maxfunctions.In
G.CohenandJ.P.Quadrat,editors,11thInternational ConferenceonAnal-
ysisandOptimization ofSystems,number199inLNCIS,pages266±272.
Springer,1994.
[19]J.Gunawardena andM.Keane.Ontheexistenceofcycletimesforsome
nonexpansive maps.TechnicalReportHPL-BRIMS-95-003, Hewlett-
PackardLabs,1995.
[20]R.M.Karp.Acharacterization oftheminimummean-cycleinadigraph.
DiscreteMaths.,23:309±311,1978.
[21]V.KolokoltsovandVMaslov.Idempotentanalysisandapplications.
KluwerAcad.Publisher,1997.
[22]V.MaslovandS.SamborskiÆõ,editors.Idempotentanalysis,volume13of
Adv.inSov.Math.AMS,RI,1992.
[23]M.L.Puterman.Markovdecisionprocesses.Handbboksinoperationsre-
searchandmanagement science,2:331±434,1990.
[24]I.V.RomanovskiÆõ.Optimization andstationarycontrolofdiscretedeter-
ministicprocessindynamicprogramming. Kibernetika,2:66±78,1967.
Engl.transl.inCybernetics3(1967).
[25]R.E.Tarjan.Depth®rstsearchandlineargraphalgorithms.SIAMJ.Com-
put,1:146±160,1972.
[26]N.N.Vorobyev.Extremalalgebraofpositivematrices.Elektron.Informa-
tionsverarbeitung undKybernetik,3,1967.inRussian.