The 19th Annual Conference of the Japanese Society for Artificial Intelligence, 2005

                    EHBSA "! #+-, !-."!-.0/21 3-.54*6"7$&%('*)

2F1-02

Study on the Strategies and Parameters for the Probabilistic Model-Building Genetic Algorithm EHBSA∗1

Daisuke Sakai∗1∗2

8 9 % !*.*:";-.-<

Kazunori Ueda
∗2

Graduate School of Informatics, Kyoto University

Science and Engineering, Waseda University

We studied strategies and parameters for the TSP solutions of EHBSA (edge histogram based sampling algorithm), which is one of the probabilistic-model genetic algorithms, mainly from the perspective of accuracy. As
the result of systematic experiments, we found some facts. For example, in the stage of population selection andparallelization, the elite strategy performed best.
For the implementation language, we chose a strongly-typed functional language OCaml, because of richness oflibraries and efficiency of programming, and implemented EHBSAWO (EHBSA without template) and ENBSAWT(EHBSA with template) for a single processor and multiple processors,respectively. For multiprocessors, we usedocamlmpi, which is an interface module to the MPI message-passing library. By the parallelization, the accuracy(reciprocal of traversal distance) improved by 1.3 to 1.6 times than that of the single-processor version.

Dgih_EHGi= O]GJPcIL> Q_KN?ASjMF@CTiOJVFBPRGlQNkmSU^JTWniV oWprq_[iGYsFXWtrZAuw[]vJ\_xW^ayJ`cdebFzl{}Ge|mdef~FF¢¡lH ^xWeywwJzmL}i£¥F¤ee¦}}§NPi[¨_gie©F lªeP «]`e`YDe¬Ecl­FG WGl}¦e§iG®i¯cGi[arzY(mi}} 
Pm´ f}Y°W±  `cbFG W PczJ²³L^  ¤m¦F§R[lgW©~ YL f
rLi P rµF `»Ww¼Ä ¼e½Nf ´¨¾¿rÅmº ÆmÇ vrx¿¶_Sw·rti¸ziÈ}¹ ÀAÉwÊÁ Ëi`eÌcDe¢ÍeEcÎº GxwP´ ÅF Ï}GJ`cÂitaÃNÉJbFÊGM
bÔiÕ iGÐ ^ mzrÄ  ÑW¼ F^ Ór ` G }WÄ m ¼ }F ½HP ¾RºjzÒ ²mÁ ³eL^   ^   ´ za£ÖÑe`JD}^ E_ GG¶LÝe·wÞL¸Ff¹ ß ]H¨àN `cáJGeO

[

×
Ø

k
e
^
}
Ù
c
Ú
J
t
i
Û

Ü
W
P
w


ï¬mPc­JQc ÔeSâT}cVW^ º G]zw òôãL^ Jä óU}G]måèc»}éilêaæmPxWçyl`JËaèõc éiöD}ê}]E_æLxFG çy ae÷}ërðøìHí  ñ ®`mzá lzlî Á `Glù` úYûFü}ªmzL[a¼ý  ]bi GþLpiJ ÿNz}ßHãLó äF~  Má ]æ[ ç· `iØ è é }ê}xWý y}a[ L~  á G     ®z ´ _ · ØY`  î ï [

1.

(GA)

GA

GA

GA

GA
PBIL, compactGA, ECGA (extended compact GA) ([5][6])
toy problem
GA§cG; GWOYPHQHS TeVmG<1=  ; L> Gz mY
¦e§A[i¡ét xL?JVmyFz][ ge¼ ©m^a`~ B/ Ce¬­[JÑ(ó @BACF¬­ à Á[²_ Iz]¤W ¦e m§Nr[ [ë DefJ©m` ~ Å  KÚ J~ LN M à Á Ow[_ór£ "W~ 
E/F GH
9(TSP

GA
EHBSA (

TSP
[4])

EHBSA

GA

GA

EHBSA

1.
2.
3.
4. EHM
5.

GA

TSP

3.

EHBSA

EHBSA

EHM t
t+1

samplingz

EHBSAWO, EHBSAWT

EHBSA
EHBSAWO(EHBSA without
template)
MPQ
EHBSAWT (EHBSA with template)
T1U WVR
EHBSAWO
2. S
WX
EHBSAWT
template
YWZ [
cut]\_^
segment
samplingJsegment
d


EHBSA

matrix (EHM)

EHM´ SUTeYV ì a`   ¬_f ­WG ÓaFà ½ Á ^Rbe G ëFcë GWGeeOJPA QbFbeÑWGG GëiÓ ëG HsiÑmuJ[ xLG  £ ìc]  b á  ì  ÁwäWz[w[ w^¿ý ó 3Ñ ` à aG W bc zG e]ý   > ®ciäµÂ `zG  IeKF3 ~CNj áY ` FÂ}iÂz àiàL efg  hiH maNºÍ¼wÎW^ x]]PlrÅ}mÏ ] irt]mÉlP}Ê }Iic KcPaº IeGMrO]K_PcMiÉlQ_OlÊPHSjTizQFV Sâ~T}¬VW­eGG][¨ni¦}WoF§if ®i~ ´ ¯c £ `]Gi[ ÌzF PLfr°F±L~ ^R r` Wf  f   Ñ á ë  £ [ ` eÂ Gl¬­r~´_ý £%_$á ^ ßµfJ`F` [áenFiÂ oLà ~ Á Y Gl¬m­m[][]ÑÑ¢óór]£ á áJG[ !(¼ ^ #£ "  zL£ `&'(
ÿ +e` ,-eÿ .i/ÿ .10324. î ï 56 p 24.7
8L` )*2.

1: :edge histogramtsampling:sakai@kuicr.kyoto-u.ac.jp

1

e!b GN£#" ~}G `L Î  Hä m  `cé_b /t  ?r[V¬cÈz ­WMG]Ñ Ô @qì Í ä `{/ ]a [ @A ~  /ºwxrTif 3Ä ½FGÐµÑLG æç [éLt?JVmz[@Ñ» Ä É z Ä¿ñ @A ~ å @A ~  [ ~ /$_ºr^ xi TFGf [ ½WÒG MUÑeGæç [{/ Yw @WA ém t ?wVmz 3 Ä[Ñ@» Ä É z Ä ñ @/A ~ [DF©~  WºrxiTFf Ä ½m
G MÑG æ_ç [{ Ja {/­ wWÓ f  á   å(ÓW GJÿ¿ßa^ÔdWy}xì ý w
G Õ ¸ z²m³i^ w P_3y ÖÔ

[¢
l
G Øú}³LäF[    [ D×©L~ üG æmeç ^[¯/{W °NÙ M vw]±3x¢w² S LÅ KÚ F_J d Å [ ¢ô à ^¢¢F ü De EN^ G à ~G M%z Q¢
 [¢([ ¢Wü ^´WµHG æ_çRem ´/µF` 1µHz Ä ½WNG O/Qm«LÑ{  ]weÛÚ ¦N[é_t ?rVm3z ´WWµ~» /Ä É z Äñ ´/µ~  Gm[ @ Ñ~  å¢ » Ä É Gk^  G ÜjcÁ W Ô ì ä ´µ£ÞØ  » Ä/É G å¿ÓÝ ^  Gm[ /µ ] G [[ ´/µL~ Jé t ?JVmz MµÑ @ÔÍ bFGN ¥ » Ä
É Glk_^  Gm] GL[ ßà  ` `LWém t^ ?JcVLz3ßrWG}Ñ úJ@N/ Í ám`c~ b J¿ /µfzWÑc^  G <=Az âNHãHt}~ yi1x W¶Rºr·ex ¸cÅ ¹f
S 
´é är£ ` taÈi[ mØ GK]L  c ÈeéaVWdef[ ¼ ^  [á  ¼ zmJ_£  D}E L½ccµ¾È
º [ ¼ ^  mØ  Rmº lziz ÀR Á £FÅ  dH^ S  Læ åè~çL éc G ÈFéwê VcJ[lf `{ åAsrÿ¿ìx ß ë [ ïWí ð_  á  îlE Y f ñNG ò A m /G ó ´à /ÔÁ µ\ ëîfE ´ôc£  ^R^  ¬_ß¨ìwW`  ô £HØ º  f `  õö L c [È}LéYVic Î3 ÷Å
YHz L a`f

The 19th Annual Conference of the Japanese Society for Artificial Intelligence, 2005?
«iÑ i[µnF© oFl~Y biG ÈrzréYVd [wGib GLO]OJPcJPAQ_QHS SµT}Te¿VWVcG [ ¼ ¿^r` ®G H[ G
 [ ó  b    ÁU~Fzl`w è_[µ{éiWê}~xFy xaYÚ Pc_[ SjNÚ ß¨`W`b ð õG G]è
Úé}ê}HxF§NS Úy Meez [ w{ P_G_[ Y_w ä   ôèA£ éWêix[ y _MrrzFPAÈ}[ é  rzJ_f~  ¼ x~ Y

kmlonqpsrutwvmx qy{z4.1 |~}s3qEHBSAWO, L EHBSAWT
MQ
c
W 3>    ~TSP
 3D
^ 
e  $ #jK  U  1 Kef
/   [  1_ N /
D/>• population
select tournament¥ Ï :

4.

4.2.2• choose template randomly :• choose template elite :4.2.3

makeparameter

4.2.6

5.

• send randomly, receive randomly :sampling

EHM• send roulette, receive roulette :template, sampling

EHBSAWT
1• send elite :• recive elite :• send tournament

template, sampling node

EHM

CPU

4.2.1

sampling
2opt

4.3

ocamlmpi

ocamlmpi[1][2][3]) ocamlmpi
OCaml
MPI
OCaml
MPI
ocamlmpi
C, C++, Fortran
OCaml

CPU

|~
® }s3KÀÂÁ WWº@Aocamlmpi• no population select :• population select half ::• receive torunament :ç [{ ]Ýea¬c ­WG G ax}ó TzÃF[^ Wü 3Ä ^ ½WGNO]ÑG æ

: @AðHñ ¶Ç_G  GrÈMUÑi« D`LÁ _»   dÅGÆWÊF®Aû_[z ¢ô Ñ ¤ë Ë £
Ä/É
Ä ñ
[ ÌW~


4.2

GA

• non use twoopt : 2opt

î G ï f  ` ® Ým¬NG­_æG ç @`A ¯/` °HM ±² WG  @KGA B1` ³e`´WµG æç G z]@®_A ä  Jæç [{  lG} úef áY´ 1G £Ö¶` ` ´/µcG æmçm   G}úWG]@{A mf ´   9 zFbiG §_G;  G 9  ´ ]3·^ ¸_Graz è_¹ ëYéieêrxL^yÖ`  ¼1GrÐ ½W z¨ ^ ºJ xa´Tm: [¨ÃGaJ {` »de]lz æmç^ GlÔ_´ zLY® Ýe¬­[ BCF¬­[ D J@G
A ¾û 
[ @WA
~BCFJ¬ ­ à Á [ De©  `(W æç zYùú¦ ëwW ^  `  f3~ ¯/J°H M ±²A[
G ¾eû  ¨ à Á ^ G ¿ ¬­LG YZH[   ´µm~ J
3.
4.

2opt

• use twoopt : 2opt

ocaml str.cma unix.cma
read_ehbsawo.ml graph makeparameter

1.
2. EHBSAWT

EHM

4.2.5

3 × 1 × 2 × 1 × 1 × 2 = 12

node

EHM

• make ehm weight :áYG  m¨zw[ ~KJP¨P_zªG«LYZ~ GldF}¬iGJèû_éiGlÔêrxFì yä¨`K©_JPWMrfz{ E{­/ümý  ^ [__Hr¨Mi zLbiÈ}G é/ Í me ~  ÒY~    ^ ¬ z ß L >JGmë}L ` z e¤Ñ
(sampling node

• make ehm delta : EHBSA

ocaml str.cma unix.cma
execute_ehbsawo.ml makeparameternode

sampling node

• choose node roulette :4.2.4

EHBSAWO

template

template• choose node randomly :áYG1 JP/¡_[f e ] ^ ßaz ~ > ]G á  z[ ¤~¢¥ ^J¦` § £m Øtemplate• choose template roulette :pop_num;20,30,40
city_num;50
max_generation;30,40
population_select;no_population_selectmake_ehm_strategy;make_ehm_weight:0.04
use_twoopt;use_twoopt,non_use_twoopt

template

(

)

send
receivefloat int

ocamlmpi• population select roulette :2(

The 19th Annual Conference of the Japanese Society for Artificial Intelligence, 2005
ehbsawo
30000"plotdataehbsawo15020050result412""plotdataehbsawo15020050result312""plotdataehbsawo15020050result212""plotdataehbsawo15020050result112"

29000

28000

min distance

27000

26000

25000

24000

23000

22000

9

0

5

10

15

2: EHBSAWO

20

25
generation

30

35

40

45

50

zrãä ® Ýi¬_­FG @/A
9

3: EHMz  ß à

mpiehbsawo
30000

if myrank=0 then (let a = 1 and b = 2.0 in
send (a,b) 1 0 comm_world) else (let a,b = (receive 0 0 comm_world) in
Printf.printf "int:%d, float:%f\n" a b
)

ø/ù

25000

min distance20000

Hºe </L =Lc zÈiîéwE V_][Fúmëaú}ËF> ~ôGLójm~ ¨z]¬¿`  îlµFE ~ J ßJzòH[

15000

10000

5000

0

cut

zrãä  ´µ æç©_ ÿAzwãLß ä:æL; ç [ Y<R{ý   ]^ laæL ç G Ó f  `  G É z  ¨wåÅz´(Ò£U` ®H[ cGY¼ ä ¬c^ [ ­mG  =Ó / f mb åf    [K» ´jWÄ W®É µ_GlGYzr¬mkcã­F^ ä G lÓ Gæf [çåR^ z  KG» ¨LÄÉcL ÎcG ÝÎÅ^ YZ ">m ÞØ c ^A2 * Sµæx å æç  å¢ B _ì ý  9
G@ É½ëeG `/#W ´dF1,G µÒ ~ÉHÁ  Ô$  ó Ã -` 1F>KÔ¿Rà¢3z º ß1z3 `ßWU U/ îßïàR?frMYW{Õà g Á GJ¾W û?

F
G
]
O
c
P
_
Q
â
S
}
T
V
A ùrþ_z$/eÑBL Ò Á L Ã

}
ù
N
þ
[
2
m
<
Ñ
_

´  ]`F ãLm`  eA    OlPHQ_SjTiV  á ià ÁG C}ÊLf
PÏ t~ilëU[¢^3dr f  ý w^Y`DFëi[ E ¼ G_` ^aze`R_sL t\ è ÍWrG}P ú}` f H´ I YW Ç vYxs

EHBSAWO

6.

let x : int * float = (receive 0 0 comm_world) in
let a,b = x in

9

{ ­AküG cûþ £ ýmÓ`}`mÿ {Ke?  ^ÉwÄ Há  [   ^  GaG w½ Ã  fa`Ná
1
[YdW[ ÒcÍmý  ^ G¬ É¿` ^AÔH [ ZmWËf F ~ {   h ó L Wb¬mGl®[­JÔLJGJÿN É ßl~ W~   Gá Éeër à  ØWØAý  Á LW -~ } Ô_¬Hz º  µÒ^[ máJGN¥` z NÔ_[}émGYë3t _?Jý VLz ^mz @ ^ Ñ prªWGþm^}ÉJfrk`^cWGá $~ zó É]Ê_^wza BÑG
_µ "!{ Fü ~LLJ ` bá]Gfa` É å¿G  #r» diÄ É CGYj kJ^ à  ` GL[ Ø]{  zr`fr` ½Nl¾¢º G ØNú} ³Lmüä}G Á eà wý Õ a  ó / ¯¿ý á NJ ·ë Já ~ zlÿ áß  ÔA [ J  cz´ r~ a· AØH©ºÁ ^mG]F aý3  ÷YHz LJ 3G ±²_z $ÍFG î ï f  `  G É z%}ZH['&cý  ^  Gaf Ä ½rf G}Diú©F[~ [ i£U)d( rJ¨F ` *¢SUìx å æmç ì` å ,x +mêBCe¬m­m
[tå æmç  ».- f ´  á w® à ý  9 Fë} - ® G/02[ 13 £ `cá W ^ G æmç z ½c¨¾A"º 4 ÷ }à ý w Ñ ë £^ G æmç `wè_éeêaxWyY º GmL  Gaf ´  ` ® Ýr¬­FG
zrãe^ W *¢SUxæå æç `ìå x,+mêLtqå æmç $L^YþWp[2@A ]w
D}c©L£ ~  `¨Gæm-Diç5©G WWGÎ G, 6$ºw7¢xw^][ TWþL· zapØ ãe2[  ^ ØcLú9a 8N äm9 Pc[wy ãNÖáÔAáJ G[ ^ ¼ ^ G DG[

5.

EHBSAWOSAWT

EHBSAWO

EHB1
EHBSAWT

20

25
generation

30

35

40

45

50

EHBSAWO

1.3

(

EHBSA

1.6

2)

(

template

3)

cut

SGI Altix 350 (Itanium2 × 8 , memory:40GB)
OCaml-3.08
ocamlmpi-1.01
ocamlopt

EHBSAWT

3)

15

(

EHM

(

  

10

4)

cut
template
template

2)

4:

5

MPI

cut

(

0

$R_JLGCrKNÊ MPrOP¨FQ  OlPcQ_SjTeVFG±$W ´ l EHBSA LÂiLÂ_z àeà_ ) ST G
H£ *  hcS¨^ixÛ}å æ_` ç G,[ ¼ É^
GA z
µ§ ST G
,
z
U
á
Y


m

J


æ
ç
Øúi³LGA
[
Y
ä
~
R
^

"VXiW zlneoYFAZ [lª  j 
£


F
à

EHMG
]
j
`


¯
°
F
z
[
w
p
æZRç[3±3²~   z  nLo] \. ['ÿ¿ ßRóþL[ Jck ^w_ªW ^iaf æ_´ ç ]`W èHáJGéêixWëeye [LËWW ~ J÷NY ß}´ z J G, É aÿ^ zF# di~  á  ´


EHM

EHM
EHM

3

The 19th Annual Conference of the Japanese Society for Artificial Intelligence, 2005ÃÔ _e ]Y`ãc Í  G É¬­wÔ å]_ 67 ] ¬­JÔ å_ 67 ]
1: cut//(s)
150/8823.9/16.6
150/9792.9/16.7
150 /11824.8/7.63
150/13729.6/5.35
150/14436.0/4.43
150/15950.1 /4.17

EHBSAWOEHBSAWT/1
EHBSAWT/2
EHBSAWT/3
EHBSAWT/4
EHBSAWT/5

]eÔ
-

ô

200
400
600
800
1000

`alÔ
7
8
6
8
6

Ã

2:

  

iÔ

´Wµ

5
10
10
10
20

G

EHBSAWO  

4864.18273.515428.0
17800.1
23639.5//(s)
150/8823.9/16.6
150/9792.9/16.7
300/12462.9/15.4
450/14720.0/16.0
600/17208.3 /17.9
750/19280.9/21.3

G]{/­/
>W

\

G

7657.113525.5
19317.1
24701.030812.7

>W

  

/
1.57
1.63
1.25
1.38
1.30

ôc GY£ >F¿]bà¢fá  ß`fY newî oF`ï \eá faWr{/  :HiYÿWód c~  W  wá àF E.¥aGlf GL¬mrOY­FµPRGY·QcZNØNSµT[ÁV óµlGwå B»  è_ó `éiêrÉ xmGyi$L[ ^  ØA[ Á Ò LÑL ä G/ H G}-z i» Ô3 A e º f5Ð àh xÈÁ"g ëGî]Ôï Ciz ÊHjiz k mþ Y] Ò ÑWä_Á  Wrà ý Y áYGYZG± ²  $RG
[

EHBSA
(

Ã
-]eÔ
200
200
200
400
400
400
600
600
600
800
800
800
1000
1000
1000

3:

l

YG {­W M
L c Î Å Ô´µiÔ   G  

EHBSAWO

6
7
8
6
7
8
6
7
8
6
7
8
6
7
8

5
5
5
10
10
10
10
10
10
10
10
10
20
20
20

Pmon

[1] Xavier Leroy : OCamlMPI: interface with the MPI
message-passing interface
http://cristal.inria.fr/ xleroy/software.html#ocamlmpi

8284.64864.15562.414291.7
10221.98273.515428.015757.1
18500.4
18160.2
18956.0
17800.1
23639.530047.3
28848.7[2]pq]rstu]v: ocaml
http://www.ueda.info.waseda.ac.jp/
˜sakai/ocaml/ocamlmemo.html[3] Xavier Leroy : The Objecttive Caml system release 3.08
Documentation and user’s manual, 2004
http://caml.inria.fr/pub/docs/
manual-ocaml/index.html[4] Shigeyoshi Tsutsui : Probabilistic Model-Building Genetic Algorithms in Permutation Representation Domain Using Edge Histogram, Proc. of the 7th Parallel
Problem Solving from Nature - PPSN VII, pp.224-233,
2002.
[5] Georges Harik : Linkage Learning via Probabilistic
Modeling in the ECGA, Technical report, IIIiGALTechnical Report, No. 99010, 1999.
[6] Georges R. Harik, G.Lobo, David E. Goldberg : The
Compact Genetic Algorithm, Proceeding of the 1998
IEEE Conference on Evolutionary Computation, pp.
523-528, 1998

4