ᚑくᄢቇℂᎿቇ⎇ⓥႎ๔
J. Fac. Sci. Tech., Seikei Univ.
Vol.47 No.2 (2010) pp.53-58
㧔․೎⎇ⓥ⾌ߦଥࠆ⺰ᢥ㧕㔚ജ࡮ᕈ⢻ߩേ⊛ߥᦨㆡൻࠍⴕ߁ 㪧㪚 ࠢ࡜ࠬ࠲ߩ᭴▽㐳ฬ ଻▸㧖㧘ᩙᨋ િ৻㧖

Construction of a PC cluster with dynamic power/performance optimization featureYasunori OSANA㧖1, Shin-ichi KURIBAYASHI㧖2ABSTRACT㧦 Integrated power and performance optimization of network and computers is an
important issue in this cloud computing age. This paper proposes a dynamic relocation method of virtualcomputing nodes to optimize and reduce power consumption of datacenters. The experiment system
consists a network attached storage and 2 host PCs with a quad-core CPU, to run 16 virtual computingnodes. From the early evaluation of the method, the method is expected to enable dynamic optimizationof power and performance.
Keywords㧦PC cluster, virtualization, green computing
㧔Received September 22, 2010㧕㧝㧚ߪߓ߼ߦNFS ⚻↱ߢ mount ߒߡ߅ࠅ㧘઒ᗐ⸘▚ࡁ࡯࠼ߩ࠺ࠖࠬࠢࠗࡔ࡯ࠫࠍ౒᦭ߒߡ޿ࠆ‫ޕ‬ ᖱႎㅢାᛛⴚ߇␠ળߩ㓈‫ߢ߹ޘ‬ᶐㅘߒߟߟ޽ࠆ੹ᣣ㧘
GS116v2 Gigabit copper switchࡀ࠶࠻ࡢ࡯ࠢᯏེߣࠦࡦࡇࡘ࡯࠲ߩਔᣇࠍ฽߼ߚ㧘ᖱႎ
ಣℂ࡮ᖱႎㅢାᛛⴚ✚૕ߢߩᶖ⾌㔚ജシᷫ߇ᄢ߈ߥᛛⴚ
⊛⺖㗴ߣߥߞߡ޿ࠆ‫ᧄޕ‬Ⓜߢߪ․ߦ㧘ࠢ࡜࠙࠼ࠦࡦࡇࡘ
࡯࠹ࠖࡦࠣᛛⴚߩ᥉෸ߦ઻ߞߡᕆㅦߦჇടߒߟߟ޽ࠆ࠺

Network storage:
ReadyNAS Pro (8TB)࡯࠲࠮ࡦ࠲ߩᶖ⾌㔚ജᦨㆡൻࠍⷞ㊁ߦ౉ࠇ㧘઒ᗐൻᛛⴚ
ࠍ↪޿ߡⒿ௛ਛߩ࠲ࠬࠢࠍ⒖േߔࠆߎߣߢേ⊛ߦᶖ⾌㔚

Computing nodes:
Dell PowerEdge T110 x 2ജ߿ᕈ⢻ߩᦨㆡൻࠍ࿑ࠆᚻᴺߩᬌ⸛ߣ㧘ߘߩ੍஻⹏ଔߩ
࿑㧝 ታ㛎ࠪࠬ࠹ࡓߩ᭴ᚑ᭎ⷐ
⚿ᨐࠍ␜ߔ‫ޕ‬ᧄᚻᴺߪ㧘ࡑ࡞࠴ࠦࠕࡊࡠ࠮࠶ࠨࠍ៞タߒߚ‛ℂ⸘▚એਅߢ㧘᭴ᚑߩ⹦⚦ࠍㅀߴࠆ‫ޕ‬ࡁ࡯࠼਄ߦⶄᢙߩ઒ᗐ⸘▚ࡁ࡯࠼ࠍ㈩⟎ߒ㧘ⶄᢙߩ‛ℂ⸘▚ࡁ࡯࠼㑆ߢ઒ᗐ⸘▚ࡁ࡯࠼ࠍ⒖േߔࠆߎߣߦࠃࠅ㧘㧚⸘▚ࡁ࡯࠼ታ㓙ߩ PC ߢ޽ࠆ‛ℂ⸘▚ࡁ࡯࠼ߩ਄ߦ㧘઒ᗐൻ࠰ࡈ‛ℂ⸘▚ࡁ࡯࠼㑆ߩ⽶⩄ಽᢔࠍⴕߞߡᕈ⢻߿ᶖ⾌㔚ജࠍᦨㆡൻߒߚࠅ㧘઒ᗐ⸘▚ࡁ࡯࠼ࠍⒿ௛⁁ᘒߦ߅޿ߚ߹߹

࠻࠙ࠚࠕߢ޽ࠆ VirtualBox OSE 1) ࠍ↪޿ߡ઒ᗐ⸘▚ࡁ࡯ਇⷐߥ‛ℂ⸘▚ࡁ࡯࠼ࠍ஗ᱛߔࠆߎߣߢߐࠄߦᶖ⾌㔚ജ࠼ࠍ᭴ᚑߔࠆ‫ޕ‬
VirtualBox OSE ߪࠝ࡯ࡊࡦ࠰࡯ࠬߩ઒ᗐൻ࠰ࡈ࠻࠙ࠚࠍᛥ೙ߔࠆ߽ߩߢ޽ࠆ‫ޕ‬ࠕߢ޽ࠅ㧘ࠥࠬ࠻ OS ߩ userland ߩࠦ࡯࠼ࠍ Intel ࠕ࡯ࠠ㧞㧚ታ㛎ࠪࠬ࠹ࡓߩ᭴ᚑ

࠹ࠢ࠴ࡖߦ߅ߌࠆ ring 3 ߢߘߩ߹߹ታⴕߒ㧘ㅢᏱ ring 0ߢታⴕߐࠇࠆ kernel space ߩࠦ࡯࠼ߪ ring 1 ߢታⴕߔࠆ‫ޕ‬࿑㧝ߦ㧘ታ㛎ࠪࠬ࠹ࡓߩ᭴ᚑࠍ␜ߔ‫‛ޕ‬ℂ⸘▚ࡁ࡯࠼ᔅⷐߦᔕߓߡㅒࠕ࠮ࡦࡉ࡞ߦࠃࠆࠛࡒࡘ࡟࡯࡚ࠪࡦ߇ⴕ

ߪ 2 บߢ㧘หߓࡀ࠶࠻ࡢ࡯ࠢߦធ⛯ߐࠇߚࠬ࠻࡟࡯ࠫࠍ

ࠊࠇࠆ႐ว߽޽ࠆ߇㧘ring 1 ߢ kernel ߩࠦ࡯࠼ࠍน⢻ߥ

*1㧦ᖱႎ⑼ቇ⑼ഥᢎ (osana@st.seikei.ac.jp)㒢ࠅታⴕߔࠆߎߣߢ㧘߶߷ࡀࠗ࠹ࠖࡉߦࠦ࡯࠼ࠍታⴕߔ

*2ࠆߎߣ߇ߢ߈㧘㜞޿ታⴕㅦᐲࠍታ⃻ߒߡ޿ࠆ‫ޕ‬㧦ᖱႎ⑼ቇ⑼ᢎ᝼

㧙㧙 ‛ℂ⸘▚ࡁ࡯࠼࡚ࠫࡉࠍ▤ℂߔࠆߎߣ߇ߢ߈ࠆ‫ޕ‬

2 บߩ‛ℂ⸘▚ࡁ࡯࠼ߪหߓ᭴ᚑߢ㧘એਅߩࠃ߁ߦߥ

ᧄⓂߢขࠅᛒ߁ታ㛎ࠪࠬ࠹ࡓߢߪ㧘઒ᗐࡁ࡯࠼ 1㨪16ߩ߁ߜ㧘઒ᗐࡁ࡯࠼ 1 ߇▤ℂࡁ࡯࠼౗↪ߦߥߞߡ߅ࠅ㧘ߞߡ޿ࠆ‫ޕ‬• Model: Dell PowerEdge T110

ߎߎߢ qsub ࠦࡑࡦ࠼ߦࠃࠅᛩ౉ߐࠇߚ࡚ࠫࡉߪ㧘઒ᗐ

ࡁ࡯࠼ 1ޯ16 ߩ޿ߕࠇ߆ߢታⴕߐࠇࠆࠃ߁ߦߥߞߡ޿ࠆ‫ޕ‬• CPU: Intel Xeon X3440 2.53GHz (4 cores x 2 threads)• RAM: DDR3 1333MHz ECC, 2GBx4• OS: FreeBSD 8.1-RELEASE (amd64)

 ࠢ࡜ࠬ࠲▤ℂ࠷࡯࡞ߎߎ߹ߢߢㅀߴߚࠪࠬ࠹ࡓࠍ㧘ipmitool ߿ SSH ࠍ↪޿OS ߪࡠ࡯ࠞ࡞ߩࡂ࡯࠼࠺ࠖࠬࠢ߆ࠄ⿠േߔࠆ‫ࠄࠇߎޕ‬ߩ ‛ ℂ ⸘ ▚ ࡁ ࡯ ࠼ ߪ IPMI(Intelligent Platform2)ߡ◲නߦ▤ℂߔࠆߎߣߩߢ߈ࠆ▤ℂ࠷࡯࡞ࠍ㐿⊒ਛߢ޽

Management Interface) ߦኻᔕߒߡ߅ࠅ㧘ࡀ࠶࠻ࡢ࡯ࠢ⚻ࠆ‫⃻ޕ‬࿷ߩࡃ࡯࡚ࠫࡦߢߪ㧘• ‛ℂ⸘▚ࡁ࡯࠼ߩ⿠േ࡮஗ᱛ࡮⁁ᘒ⏕⹺↱ߢ㔚Ḯᛩ౉ߥߤߩ೙ᓮࠍⴕ߁ߎߣ߇ߢ߈ࠆ‫ޕ‬
VirtualBox OSE ࠍ↪޿ߡᄙᢙߩ઒ᗐ⸘▚ࡁ࡯࠼ࠍേ૞• ઒ᗐ⸘▚ࡁ࡯࠼ߩ⿠േ࡮஗ᱛ࡮⁁ᘒ⏕⹺ߐߖࠆߚ߼㧘ࠞ࡯ࡀ࡞ᄌᢙ kern.ipc.semmni ߅ࠃ߮• ‛ℂ⸘▚ࡁ࡯࠼㑆ߢߩ઒ᗐ⸘▚ࡁ࡯࠼ߩ⒖േ

kern.ipc.semmns ࠍⷙቯ୯ࠃࠅჇടߐߖ㧘ߘࠇߙࠇ 30ࠍⴕ߁ߎߣ߇ߢ߈ࠆ‫ޕ‬㧟㧚ᕈ⢻⹏ଔ

ߣ 300 ߦ⸳ቯߒߡ޽ࠆ‫ޕ‬ ઒ᗐ⸘▚ࡁ࡯࠼઒ᗐ⸘▚ࡁ࡯࠼ߪ 16 บሽ࿷ߒ㧘ߔߴߡએਅߩࠃ߁ߥ᭴

㧚 ⸘▚ಣℂᕈ⢻หᤨߦታⴕߔࠆࡊࡠ࠮ࠬᢙ߿⸘▚ࡁ࡯࠼ߩ઒ᗐൻߦࠃ

ᚑߦߥߞߡ޿ࠆ‫ޕ‬• Virtualization software: VirtualBox OSE 3.2.8

ࠆᕈ⢻߳ߩᓇ㗀߿㧘㔚ജല₸ࠍ᷹ቯߔࠆߚ߼ߦએਅߩ߰

• CPU: 1 coreߚߟߩࡌࡦ࠴ࡑ࡯ࠢࠍⴕߞߚ‫ޕ‬• MEncoder 4) ߦࠃࠆേ↹ࠛࡦࠦ࡯࠼

• RAM: 512MB• ౣ↢ᤨ㑆: 15 ಽ

• OS: FreeBSD 8.1-RELEASE (amd64)• ⸃௝ᐲ: 720x480, 24fps• Network: Bridged• ᤋ௝: H264 900kbps

࠺ࠖࠬࠢࠗࡔ࡯ࠫߪࡀ࠶࠻ࡢ࡯ࠢࠬ࠻࡟࡯ࠫ਄ߦ⟎߆ࠇ• 㖸ჿ: AAC stereo 192kbps

ߡ߅ࠅ㧘ߎࠇࠍ‛ℂ⸘▚ࡁ࡯࠼߇ NFS ⚻↱ߢ mount ߔࠆ• ClustalW 5) ߦࠃࠆ DNA ㈩೉ߩࡑ࡞࠴ࡊ࡞ࠕ࡜ࠗࡔߎߣߢേ૞ߔࠆ‫ޕ‬ೋᦼ⁁ᘒߢߪฦ‛ℂ⸘▚ࡁ࡯࠼ߦߟ߈
8 บߩ઒ᗐ⸘▚ࡁ࡯࠼߇⿠േߒ㧘VirtualBox ߩ teleport ᯏࡦ࠻• ࠺࡯࠲࠮࠶࠻: BioBench 6) ߆ࠄขᓧ⢻ࠍ↪޿ߡേ૞⁁ᘒߩ߹߹ઁᣇߩ‛ℂ⸘▚ࡁ࡯࠼ߦ⒖േ

• 430 ㈩೉ߔࠆߎߣ߇ߢ߈ࠆ‫ޕ‬• ᐔဋ㈩೉㐳 350bp

VirtualBox ߪ ৻ ⥸ ⊛ ߦ GUI ߆ ࠄ ೑ ↪ ߐ ࠇ ࠆ ߇ 㧘VBoxManage ߥߤߩࠦࡑࡦ࠼ࠍ↪޿ߡࠦࡑࡦ࠼࡜ࠗࡦ߆

એਅߦߎࠇࠄߩ⚿ᨐࠍ␜ߔ‫߅ߥޕ‬㧘૶↪ߒߚ‛ℂ⸘▚ࡁ࡯࠼ߩ CPU ߇ 4 ࠦࠕ2 ࠬ࡟࠶࠼หᤨታⴕߢ޽ࠆߚ߼㧘ࠄ઒ᗐࡑࠪࡦߩᠲ૞ࠍⴕ߁ࠗࡦ࠲ࡈࠚ߽ࠗࠬ↪ᗧߐࠇߡ߅ࠅ㧘ᧄⓂߢᛒ߁ታ㛎ࠪࠬ࠹ࡓߢߪ 1 บ⋡ߩ઒ᗐ⸘▚ࡁ઒ᗐ⸘▚ࡁ࡯࠼߿ࡀࠗ࠹ࠖࡉߦታⴕߔࠆࡊࡠ࠮ࠬߩᢙߪ

࡯࠼ߩ࠮࠶࠻ࠕ࠶ࡊࠍ㒰߈㧘GUI ࠍ૶ࠊߕߦ VirtualBox

ᦨᄢߢ 8 ߣߒ㧘એਅߦ␜ߔߔߴߡߩ᷹ቯ⚿ᨐߪ 10 ࿁ߩ᷹ࠍㆇ↪ߒߡ޿ࠆ‫ޕ‬ቯߩᐔဋ୯ߢ޽ࠆ‫ޕ‬㧚 ࠰ࡈ࠻࠙ࠚࠕ ࡝࠰࡯ࠬ▤ℂ࠷࡯࡞
16 บߩ઒ᗐ⸘▚ࡁ࡯࠼ߦߪ OpenPBS/Torque 3) ࠍ↪޿ߡ⸘▚࡚ࠫࡉࠍᛩ౉ߔࠆ‫ޕ‬OpenPBS/Torque ߪ㧘PC ࠢ࡜ࠬ࠲਄ߩ⸘▚࡚ࠫࡉࠍ▤ℂߔࠆߚ߼ߩࡃ࠶࠴ಣℂࠪࠬ࠹ࡓߢ޽ࠅ㧘࡚ࠫࡉߩࠬࠤࠫࡘ࡯࡝ࡦࠣ߿ታⴕࠍⴕ߁‫࡙ޕ‬࡯ࠩߪ࡚ࠫࡉࠍᛩ౉ߔࠆ qsub㧘࡚ࠫࡉࠍ೥㒰ߔࠆ qdel㧘࡚ࠫࡉߩ⁁ᘒࠍෳᾖߔࠆ qstat ߥߤߩࠦࡑࡦ࠼ࠍ૶ߞߡ㧙㧙

160

700

140

600

120500100Power (W)Time (sec)800

400300MEncoder vbox
MEncoder native
ClustalW vbox
ClustalW native

200
10001234568060
40
207

MEncoder native
ClustalW native

08

123

10.8
0.6
0.4

0

MEncoder native
MEncoder vbox
ClustalW native
ClustalW vbox
12345675678࿑㧠 หᤨታⴕࡊࡠ࠮ࠬᢙߣᶖ⾌㔚ജ

Power efficiency (normalized to 1 proc)

Throughput (normalized to 1 proc)࿑㧞 หᤨታⴕࡊࡠ࠮ࠬᢙߣታⴕᚲⷐᤨ㑆

0.2

4# processes# processes

85432
10

MEncoder native
ClustalW native
123࿑㧟 หᤨታⴕࡊࡠ࠮ࠬᢙߣࠬ࡞࡯ࡊ࠶࠻678 㔚ജኻᕈ⢻Ყ਄⸥ߩࡌࡦ࠴ࡑ࡯ࠢࠍታⴕߒߚ㓙ߩታⴕᤨ㑆ࠍ࿑ 2 ߦ㧘ࡠ࠮ࠬታⴕᤨߩ⋧ኻታⴕㅦᐲࠍ࿑ 3 ߦ㧘ߘࠇߙࠇ␜ߔ‫ޕ‬

5࿑㧡 หᤨታⴕࡊࡠ࠮ࠬᢙߣ㔚ജല₸ ታⴕㅦᐲ
1 ࡊࡠ࠮ࠬߩ႐วߩታⴕㅦᐲࠍ 1 ߣߒߚ႐วߩ㧘ⶄᢙࡊ

4# processes# processes‛ℂ⸘▚ࡁ࡯࠼਄ߢⶄᢙߩ MEncoder ߅ࠃ߮ ClustalWࠍታⴕߒߚ႐วߩᶖ⾌㔚ജࠍ࿑ 4 ߦ㧘ታⴕㅦᐲߩว⸘ߢ

ᱜⷙൻߒߚ㔚ജല₸ࠍ࿑ 5 ߦ␜ߔ‫ޕ‬ᶖ⾌㔚ജߪߔߴߡߩ

ߎߩ⚿ᨐ߆ࠄ㧘‛ℂ⸘▚ࡁ࡯࠼਄ߢࡀࠗ࠹ࠖࡉߦታⴕ

CPU ࠦࠕ߇Ⓙ௛ߔࠆ 4 ࡊࡠ࠮ࠬߦ㆐ߔࠆߣჇട߇⧯ᐓ✭ߒߚ႐วߦߪ㧘3 ࡊࡠ࠮ࠬ߹ߢߪ߶ߣࠎߤㅦᐲߩૐਅ߇

߿߆ߦߥࠆ᭽ሶ߇ࠊ߆ࠆ‫ߚ߹ޕ‬㧘ᶖ⾌㔚ജߩჇടࠃࠅ߽

⿠߈ߕ㧘4 ࡊࡠ࠮ࠬ߆ࠄࠬ࡞࡯ࡊ࠶࠻߇ૐਅߒᆎ߼ࠆߩ

ࡊࡠ࠮ࠬᢙߩჇടߦ઻߁ಣℂ⢻ജߩว⸘ߩჇടߩિ߮ߩ

ߦኻߒߡ㧘઒ᗐ⸘▚ࡁ࡯࠼਄ߢߩಣℂߪ߶߷⋥✢⊛ߦࠬ߶߁߇ㅦ޿ߚ߼㧘ߥࠆߴߊᄙߊߩࡊࡠ࠮ࠬࠍหᤨታⴕߒ

࡞࡯ࡊ࠶࠻߇ૐਅߔࠆߎߣ߇ࠊ߆ࠆ‫࡯ࠦߦࡉࠖ࠹ࠗࡀޕ‬ߚ߶߁߇㔚ജല₸߇ᡷༀߔࠆߎߣ߇ࠊ߆ࠆ‫ޕ‬࠼ࠍታⴕߔࠆ႐วߦߪ㧘3 ࡊࡠ࠮ࠬหᤨߩታⴕ߹ߢߪ 4ߟߩࡊࡠ࠮࠶ࠨࠦࠕߩ߁ߜ 1 ߟࠍᏱߦ OS ߩ౉಴ജߦ૶㧚 ㅢାᕈ⢻↪ߔࠆߎߣ߇ߢ߈ࠆߩߦኻߒߡ㧘઒ᗐ⸘▚ࡁ࡯࠼ߢߪᏱ

MPI ߥߤࠍ↪޿ߡਗ೉ൻߐࠇߚࡊࡠࠣ࡜ࡓߩታⴕㅦᐲ

ߦ߭ߣߟߩࡊࡠ࠮࠶ࠨࠦࠕߢ౉಴ജߣ⸘▚ߩߔߴߡࠍⴕߦߪ⸘▚ࡁ࡯࠼㑆ߩㅢାㅦᐲ߇ᄢ߈ߊᓇ㗀ߔࠆߚ߼㧘ㅢࠊߥߌࠇ߫ߥࠄߥ޿ߚ߼ߛߣ⠨߃ࠄࠇࠆ‫ޕ‬ାㅦᐲߩࡌࡦ࠴ࡑ࡯߽ࠢታᣉߒߚ‫ޕ‬

߹ߚ㧘ClustalW ߪᲧセ⊛࠺ࠖࠬࠢߩ౉಴ജߥߤߩ I/Oಣℂ߇ᄙ޿ߚ߼㧘઒ᗐ⸘▚ࡁ࡯࠼ࠍ↪޿ߚ႐วߩࠝ࡯ࡃ࡯ࡋ࠶࠼߇ᄢ߈޿ߎߣ߽ࠊ߆ࠆ‫ޕ‬dd ߅ࠃ߮ nc ࠦࡑࡦ࠼ࠍ↪޿ߡ㧘/dev/zero ߆ࠄ⺒ߺ಴ߒߚ 1 ࡉࡠ࠶ࠢ 1024 ࡃࠗ࠻ߩ࠺࡯࠲ࠍォㅍߒߚ㓙ߩታലࠬ࡞࡯ࡊ࠶࠻ࠍ࿑ 6 ߦ␜ߔ‫ޕ‬᷹ቯߪ (1) ห৻ߩ‛ℂ⸘▚ࡁ࡯࠼਄ߩ⇣ߥࠆ઒ᗐ⸘▚ࡁ࡯࠼㑆㧘(2) ⇣ߥࠆ‛ℂ⸘▚ࡁ࡯࠼਄ߩ઒ᗐ⸘▚ࡁ࡯࠼㑆㧘(3) ‛ℂ⸘▚ࡁ࡯࠼㧙㧙

100

㧠㧚߹ߣ߼

Throughput (Mbps)

80એ਄ߩ⚿ᨐ߆ࠄ㧘⸘▚ㅦᐲࠍఝవߔࠆ႐วߦߪหᤨߦ

60ታⴕߔࠆࡊࡠ࠮ࠬᢙߪዋߥ޿߶ߤࠃߊ㧘㔚ജല₸ࠍఝవߔࠆ႐วߦߪᦨૐߢ߽‛ℂࡊࡠ࠮࠶ࠨࠦࠕᢙߣหᢙߩࡊ

40ࡠ࠮ࠬࠍ⿛ࠄߖࠆߩ߇ℂᗐ⊛ߢ޽ࠆߎߣ߇ࠊ߆ࠆ‫ޕ‬
200Same physical node
Different physical node
Physical node to physical node
1

2

3

4

56߹ߚ㧘⸘▚ࡁ࡯࠼ߩ઒ᗐൻߦ઻߁ࡍ࠽࡞࠹ࠖߪਥߦ౉
಴ജߦ߅޿ߡ⊒↢ߔࠆ߽ߩߩ㧘ዋߥߊߣ߽੹࿁↪޿ߚࡌ
78ࡦ࠴ࡑ࡯ࠢߩ▸࿐ߢߪ⥌๮⊛ߥᕈ⢻ૐਅࠍ߽ߚࠄߔ߽ߩ

# processesߢߪߥ޿ߎߣ߇⏕⹺ߢ߈ߚ‫ޕ‬ߎࠇߦട߃ߡ㧘઒ᗐ⸘▚ࡁ࡯࠼ߩ migration ߪలಽߦ㜞࿑㧡 ㅢାࠬ࡞࡯ࡊ࠶࠻ㅦߢ޽ࠆߚ߼㧘േ⊛ߥ‛ℂ⸘▚ࡁ࡯࠼㑆ߢߩౣ㈩⟎ߦࠃ㑆㧘ߩ 3 ㅢࠅߢⴕߞߚ‫߅ߥޕ‬㧘(1) ߅ࠃ߮ (2) ߦ߅޿ߡࠆᕈ⢻࡮ᶖ⾌㔚ജߩᦨㆡൻߪ᦭ലߥᚻᲑߢ޽ࠆߣ⠨߃ࠄߪ㧘ฦ઒ᗐ⸘▚ࡁ࡯࠼ߢታⴕߐࠇࠆㅍା߅ࠃ߮ฃାߩࡊࠇࠆ‫ޕ‬ࡠ࠮ࠬߪߘࠇߙࠇ㧘ᦨᄢߢ 1 ߟߢ޽ࠅ㧘޿ߕࠇߩ႐วߦ
߽ᦨᄢ 8 ࡝ࡦࠢߩ࠺࡯࠲ォㅍࠍⴕߞߡ㧘1 ࡝ࡦࠢ޽ߚࠅ㧡㧚੹ᓟߩ⺖㗴ߩࠬ࡞࡯ࡊ࠶࠻ࠍ᷹ቯߒߚ‫ޕ‬ߎߩ⚿ᨐ߆ࠄ㧘ห৻ߩ‛ℂ⸘▚ࡁ࡯࠼਄ߩ઒ᗐ⸘▚ࡁ⃻࿷ߩࠢ࡜ࠬ࠲▤ℂ࠷࡯࡞ߦߪ㧘ฦ‛ℂ࡮઒ᗐ⸘▚ࡁ࡯࠼㑆ߩㅢାߪ㧘⇣ߥࠆ‛ℂ⸘▚ࡁ࡯࠼਄ߩߘࠇࠃࠅ߽࡯࠼ߩ⽶⩄⁁ᘒࠍ⋙ⷞߒߡౣ㈩⟎ࠍⴕ߁ᯏ⢻߇ሽ࿷ߒߥ

10㨪30% ߶ߤ㜞ㅦߢ޽ࠆߎߣߣ㧘઒ᗐ⸘▚ࡁ࡯࠼਄ߢ 4޿ߚ߼㧘੹ᓟߪߎߩᯏ⢻ࠍ㐿⊒ߒߡ޿ߊ੍ቯߢ޽ࠆ‫ߎޕ‬࡝ࡦࠢࠍ⿧߃ࠆ࠺࡯࠲ォㅍࠍหᤨߦⴕ߁႐วߦߪ‛ℂ⸘ࠇߦߪ OpenPBS/Torque ߩ࡚ࠫࡉߩ⁁ᴫ߿ฦ⸘▚ࡁ࡯࠼▚ࡁ࡯࠼਄ߢࡀࠗ࠹ࠖࡉߦ࠺࡯࠲ォㅍࠍⴕߞߚ႐วߣᲧ

ߩ CPU ⽶⩄㧘ㅢା㊂ߥߤࠍ৻᜝ߒߡ⋙ⷞߔࠆߎߣ߇ᔅⷐ

ߴߡ㆖⦡ߥ޿ォㅍㅦᐲ߇ᓧࠄࠇࠆߎߣ߇ࠊ߆ࠆ‫ޕ‬8 ࡝ࡦߦߥࠆ‫ޕ‬ࠢߩ႐วߦߪ⇣ߥࠆ‛ℂ⸘▚ࡁ࡯࠼਄ߩࠬ࡞࡯ࡊ࠶࠻߇઒ᗐ⸘▚ࡁ࡯࠼ߩ▤ℂߦߟ޿ߡ߽㧘⃻⁁ߩ㕒⊛ߥ᭴ᚑะ਄ߒߡ޿ࠆ߇㧘ߎࠇߪߔߴߡߩ઒ᗐ⸘▚ࡁ࡯࠼ߩ CPU߇ᔅߕߒ߽ᦨㆡߢߪߥ޿‫ߪߢ⁁⃻ޕ‬Ⓙ௛⁁ᘒ߿⽶⩄⁁ᘒ૶↪₸߇ 100% ߦㄭߠ߈㧘ࡎࠬ࠻ OS ߩࠬࠤࠫࡘ࡯࡜߇ࠍ໧ࠊߕ㧘หߓࠨࠗ࠭ߩࡔࡕ࡝ࠍഀࠅᒰߡߡ߅ࠅ㧘ߎࠇฦ઒ᗐ⸘▚ࡁ࡯࠼ࠍ৻ቯߩ‛ℂ CPU ࠦࠕߦഀࠅᒰߡ⛯ߪ‛ℂ⸘▚ࡁ࡯࠼਄ߢߪࠬࡢ࠶ࡊࠕ࠙࠻ߐࠇߥ޿ߚ߼㧘ߌࠆߚ߼㧘ታⴕല₸߇ะ਄ߔࠆߚ߼ߢߪߥ޿߆ߣ⠨߃ࠄᧄᒰߦࡔࡕ࡝߇ᔅⷐߥ઒ᗐ⸘▚ࡁ࡯࠼ߦᄙߊߩࡔࡕ࡝ኈࠇࠆ‫ޕ‬㊂ࠍഀࠅᒰߡࠆߎߣ߇ߢ߈ߥ޿‫ߦࠄߐޕ‬㧘‛ℂ޽ࠆ޿ߪ ᧄታ㛎ࠪࠬ࠹ࡓߢߪേ⊛ߦ઒ᗐ⸘▚ࡁ࡯࠼ࠍ‛ℂ⸘▚઒ᗐߩࡀ࠶࠻ࡢ࡯ࠢࠗࡦ࠲ࡈࠚࠗࠬࠍ⚻↱ߒߚㅢାߪ౒ࡁ࡯࠼㑆ߢ⒖േߢ߈ࠆߚ߼㧘ਗ೉ࡊࡠࠣ࡜ࡓߩേ૞ⅣႺ᦭ࡔࡕ࡝ࠍ↪޿ߚㅢାߣᲧߴߡᕈ⢻߇ૐ޿ߚ߼㧘MPI ߿ߩേ⊛ߥᦨㆡൻࠍⴕ߁ߎߣ߽น⢻ߢ޽ࠆ‫ޕ‬pthread ࠍ↪޿ߡਗ೉ൻߐࠇߚࡊࡠࠣ࡜ࡓࠍല₸⊛ߦࡂࡦ࠼࡝ࡦࠣߔࠆߚ߼ߦߪ㧘઒ᗐ⸘▚ࡁ࡯࠼ߩ CPU ᢙࠍ 1㧚 ઒ᗐ⸘▚ࡁ࡯࠼ߩ⒖േߦ㒢ቯߖߕ㧘ᔅⷐߦᔕߓߡჇᷫߔࠆߎߣ߽⠨߃ࠄࠇࠆ‫ޕ‬߹ߚ㧘੹࿁ߪ LAN ធ⛯ߐࠇ㧘࠺ࠖࠬࠢࠗࡔ࡯ࠫࠍ౒‛ℂ⸘▚ࡁ࡯࠼㑆ߢ઒ᗐ⸘▚ࡁ࡯࠼ࠍ⒖േߔࠆߩߦᚲ
ⷐߔࠆᤨ㑆ߪᐔဋߒߡ 6.60 ⑽ߢ޽ߞߚ‫‛ޕ‬ℂ⸘▚ࡁ࡯࠼᦭ߔࠆ‛ℂ⸘▚ࡁ࡯࠼㑆ߢߩ઒ᗐ⸘▚ࡁ࡯࠼ߩ⒖േ߇ల㑆ߢ࠺ࠖࠬࠢࠗࡔ࡯ࠫߪ౒᦭ߐࠇߡ߅ࠅ㧘⒖േߦ޽ߚߞ

ಽߦ㜞ㅦߢ޽ࠆߎߣࠍ␜ߒߚ߇㧘㆙㓒࿾㑆ߢห᭽ߩᯏ⢻ߡߪ CPU ࠦࡦ࠹ࠠࠬ࠻ߣࡔࡕ࡝ߩౝኈࠍォㅍߔࠇ߫ࠃ

ࠍታ⃻ߔࠆ㓙ߦߪ࠺ࠖࠬࠢࠗࡔ࡯ࠫߩ౒᦭ߪ⃻ታ⊛ߢߥㅍߔࠆߩߦᦨૐ㒢ᔅⷐߥᤨ㑆ߪ⚂ 5 ⑽ߢ޽ࠆߩߢ㧘లಽ

TCP/IP ߩ઀⚵ߺ 7)ࠍᔕ↪ߔࠆߥߤߒߡ㧘ߎࠇߦኻᔕߔࠆ

޿ߚ߼㧘1000Mbps ߩធ⛯ߢ 512MB ߩࡔࡕ࡝ߩౝኈࠍォ޿ߚ߼㧘FreeBSD ߢᦨㄭ㐿⊒ߐࠇߚ remote replication overߦᅷᒰ߆ߟ㜞ㅦߦ઒ᗐ⸘▚ࡁ࡯࠼ߩ migration ߇ታ⃻ߢ

ᚻᴺࠍ㐿⊒ߔࠆᔅⷐ߇޽ࠆ‫ޕ‬߈ߡ޿ࠆߎߣ߇ࠊ߆ࠆ‫ޕ‬㧙㧙߁ߦߥࠆ‫ޕ‬ෳ⠨ᢥ₂
1㧕Oracle Corportation: “VirtualBox website”,

kern.ipc.semmni=40

http://virtualbox.org/

kern.ipc.semmns=300

2㧕Intel Corporation: “Intelligent platform managementinterface”,

ipmi_load="YES"

http://www.intel.com/design/servers/ipmi/

3㧕Cluter Resources Inc.: “Torque resource manager”,

console="comconsole,vidconsole"

http://www.clusterresources.com/

boot_multicons="YES"

4㧕MPlayer Project web site, http://www.mplayerhq.hu/

hint.uart.0.disabled="1"5㧕Larkin MA, Blackshields G, Brown NP, Chenna R,

hint.uart.1.flags="0x10"

McGettigan PA, McWilliam H, Valentin F, Wallace IM,

hw.bge.allow_asf="1"

Wilm A, Lopez R, Thompson JD, Gibson TJ, Higgins DG:“Clustal W and Clustal X version 2.0.” Bioinformatics,VirtualBox ߩࠥࠬ࠻ OS ߣߒߡ 64bit

ߩ FreeBSD ࠍ↪

Vol.23, pp 2947-2948, 2007.

޿ࠆߚ߼㧘BIOS ⸳ቯߢ CPU ߩ VT-x ᜛ᒛࠍ᦭ലߦߒߡ

Tseng CW, and Yeung D: “BioBench: A benchmark suite߅ߊᔅⷐ߇޽ࠆὐߦ߽ᵈᗧ߇ᔅⷐߢ޽ࠆ‫ޕ‬BIOS ߢߪߐ

6㧕Albayraktaroglu K, Jaleel A, Wu X, Franklin M, Jacob B,ࠄߦ㧘console redirection ߥߤߩ⸳ቯࠍⴕߞߡ߅ߌ߫㧘㔚of bioinformatics applications.”, In prceedigs of 2005Ḯᛩ౉⋥ᓟ߆ࠄ IPMI ߩ serial over LAN ⚻↱ߢߩᠲ૞߇

of Systems and Software, pp.2-9, Mar.2005.

น⢻ߢ޽ࠆ‫ޕ‬IEEE International Symposium on Performance Analysis
7㧕FreeBSD Project: “HAST - Highly available storage”,
http://wiki.freebsd.org/HASTB. ઒ᗐ⸘▚ࡁ࡯࠼ߩ૞ᚑ࡮ⶄ⵾
VirtualBox ߪ઒ᗐࡑࠪࡦߩⶄ⵾ࠍ߅ߎߥ߁ᯏ⢻ࠍᜬߚߥ޿߇㧘઒ᗐࡂ࡯࠼࠺ࠖࠬࠢߩⶄ⵾ߪࠨࡐ࡯࠻ߐࠇࠆߚ

ઃ ㍳߼㧘ࠦࡑࡦ࠼࡜ࠗࡦ߆ࠄᄙᢙߩ઒ᗐࡑࠪࡦࠍ૞ᚑߒ㧘ⶄ

A. ‛ℂ⸘▚ࡁ࡯࠼ߩ⸳ቯ
VirtualBox ߪ System V IPC ࠍ૶↪ߒߡࡈࡠࡦ࠻ࠛࡦ࠼ߣㅢାࠍⴕ߁ߚ߼㧘ᄙᢙߩ VirtualBox ࠍหᤨߦታⴕߔࠆ⵾ߒߚࡂ࡯࠼࠺ࠖࠬࠢࠍഀࠅᒰߡࠆߎߣߢ㧘ห᭽ߩ઒ᗐࡑࠪࡦࠍᄙᢙ૞ᚑߔࠆߎߣ߇ߢ߈ࠆ‫ޕ‬ ߚߣ߃߫㧘ࡂ࡯࠼࠺ࠖࠬࠢ vmbsd01.vdi ࠍⶄ⵾ߒߡvmbsd02.vdi ࠍ૞ᚑߔࠆߥࠄ߫એਅߩࠃ߁ߦߔࠇ߫ࠃ޿‫ޕ‬႐วߦߪ㧘ߘߩᢙએ਄ߩ System V semaphore ࠍ⏕଻ߒߡ% VBoxManage clonehd ¥߅ߊᔅⷐ߇޽ࠆ‫ߩߎޕ‬୯ߪ sysctl ࠦࡑࡦ࠼ߢขᓧߔࠆ

vmbsd01.vdi vmbsd02.vdi --registerߎ ߣ ߇ ߢ ߈ ࠆ ߇ 㧘 ᄌ ᦝ ߪ ߢ ߈ ߥ ޿ ߚ ߼ 㧘/boot/loader.conf ߢ⸳ቯߔࠆᔅⷐ߇޽ࠆ‫ޕ‬ߐࠄߦᣂߒ޿઒ᗐࡑࠪࡦߩ઀᭽ࠍਅ⸥ߩࠃ߁ߦቯ߼ࠆ‫ޕ‬߹ߚ㧘‛ℂ⸘▚ࡁ࡯࠼ߣߒߡ૶↪ߒߚ PowerEdge T110

• Guest OS: FreeBSD (amd64)

ߦߪ Ethernet controller ߣߒߡ Broadcom BCM5722 (bge• RAM: 512MB, ‛ℂࠕ࠼࡟ࠬ᜛ᒛ off, VRAM 10MB

• Audio: OSS (Open sound system: /dev/dsp)

࠼࡜ࠗࡃߢേ૞) ߇៞タߐࠇߡ޿ࠆ߇㧘࠺ࡈࠜ࡞࠻ߩ⸳ቯ ߢ ߪ bge ࠼ ࡜ ࠗ ࡃ ߩ ೋ ᦼ ൻ ᤨ ߦ BMC (Baseboard• Network: bge0 ߦࡉ࡝࠶ࠫManagement Controller) ߩ㧘Ethernet ࠍ⚻↱ߒߚㅢାᯏ⢻• PIIX4 compatible IDE controller• Primary master: HDDࠍ஗ᱛߒߡߒ߹߁ߚ߼㧘IPMI ࠍ೑↪ߒߚ㔚Ḯ೙ᓮ߿
serial over LAN ߥ ߤ ߩ ᯏ ⢻ ࠍ ૶ ߁ ߚ ߼ ߦ ߪ• Secondary master: DVD (࠼࡜ࠗࡉߪⓨ)hw.bge.allow_asf ࠍ 1 ߦ⸳ቯߒߡ߅ߊᔅⷐ߇޽ࠆ‫ޕ‬ߘߩ߶߆㧘serial over LAN ߢࠦࡦ࠰࡯࡞ࠍ૶߁ߚ߼ߩ⸳ቯࠍ฽߼ࠆߣ㧘/boot/loader.conf ߩ⸥ㅀߪએਅߩࠃߎߩ઀᭽ߩ઒ᗐࡑࠪࡦࠍ૞ᚑߒߡߐ߈߶ߤߩ઒ᗐࡂ࡯࠼࠺ࠖࠬࠢࠍ attach ߔࠆ߹ߢߩᚻ㗅ߪએਅߩࠃ߁ߦߥࠆ‫ޕ‬㧙㧙

% VBoxManage createvm ¥--name vmbsd02 ¥--ostype "FreeBSD_64" --register% VBoxManage modifyvm vmbsd02 ¥--memory 512 --pae off --vram 10 ¥ ૞ᚑߒߚ઒ᗐࡑࠪࡦࠍ⿠േ࡮஗ᱛߔࠆߦߪ㧘% VBoxManage startvm vmbsd02 ¥--type=headless% VBoxManage controlvm vmbsd02 ¥acpipowerbutton--audio oss ¥--nic1 bridged --bridgeadapter1 bge0
% VBoxManage storagectl vmbsd02 ¥--name "IDE Controller" ¥--add ide --controller PIIX4ߩࠃ߁ߦߔࠆ‫⿠ޕ‬േᤨߦ“--type=headless” ࠍߟߌߥߌࠇ߫㧘࠺ࠖࠬࡊ࡟ࠗߣࠠ࡯ࡏ࡯࠼ࠍ↪޿ߡࠦࡦ࠰࡯࡞ߦࠕࠢ࠮ࠬߔࠆߎߣ߇ߢ߈ࠆ‫ޕ‬% VBoxManage storageattach vmbsd02 ¥--storagectl "IDE Controller" ¥--port 0 --device 0 ¥--type hdd --medium vmbsd02.vdi% VBoxManage storageattach vmbsd02 ¥--storagectl "IDE Controller" ¥--port 1 --device 0 ¥ ߥ߅㧘࠺ࠖࠬࠢࠗࡔ࡯ࠫߘߩઁࠍ౒᦭ߒߡ޿ࠆ⁁ᘒߢteleport ᯏ⢻ߦࠃࠅ઒ᗐࡑࠪࡦߩ migration ࠍⴕ߁႐วߦߪ㧘ห৻᭴ᚑߢหߓ࠺ࠖࠬࠢࠗࡔ࡯ࠫߩ attach ߐࠇߚ߰ߚߟߩ઒ᗐࡑࠪࡦࠍ↪ᗧߒߡ߅߈㧘వߦ⿠േߒߚߤߜ

ࠄ߆৻ᣇ߆ࠄઁᣇ߳ߩ teleport ߣ޿߁ᒻࠍߣࠆߎߣߦߥࠆὐߦᵈᗧ߇ᔅⷐߢ޽ࠆ‫ޕ‬--type dvddrive --medium emptydrive㧙㧙