別紙41 (課程博士(英文))Dale of sub加Ssion (month day, year)

July 6t,',2021

Depa11ment ofComputer science andStudent Φ Number

D189301

Engineerin音

Jun Miura,Supervisors

Shi曾eru Kuriyama

ChandraKusumaDewa

Applicant's name

Abstract (DoctⅢ')
Tj{1e ofThesis

Policy hlte即、alion for person・F0ⅡOwing RobolTrainino usino Deep Reinforcement Leaming

APPIOX 80o words

Recently, Tobots are widely used in numerous aelds to support humans in manytasks. Thanks to the rapid gTowth of techn010gy, the automated machines arebecomin号 mの'e powerful and more beneficialin their Toles to help performin号
Some tasks that hun〕ans even cannot do. Likewise, a particular type of partner
Tobots which can interact closely to humans is also required to support the辻 dailyneeds.1n such cases, there are also demands for specialized xobots which have themain abi1北y to f0110w and a此end specified humans il) a safe close distanceContinuously. However,1naking them able to perform tl〕at kind of skiⅡS is nottnvial since person、f0ⅡOwing cannot be considered as a simple task to be
Performed by robots

In this thesis, we tried to train a mobile robot for perfonnil)曾 thePerson、f0ⅡOwin晉 task. Here, we see that the task is a complex task which can be
broken down into several simpler sub tasks.＼入7hen the robot is away from thetar牙et person, it should be able to perform the navigation task safely in order tomake its position is close enoU牙h to him. Afterwards, the robot should also be ableto peTform the attending task properly once 北S position is close to tl)e target
Pel'son.1n our study, we consider sevexalprevious studies which tTied to n〕ake therobot able to accompany the tar宮et person at his left side or at his right side
instead offoHowin曾 hin〕 from behind.1n order to n].ake the robot able to master theComplex pel'son、f0110wln宮 task appropriately, 叉入7e Utilize deep reinf01'cement

Iearnin牙(DRL) approach for both obtaining the optimal policy for each sub task
and integrating aⅡ those optimal policies into one stron今 Optilnal person・f0110訊7ing
meta'policy

To obtaln the optimal navigation policy, we employ the s0丘 actの・・critic(SAC) 1earnin牙 a1目orithm for making the robot able to approach the tar牙et person
We11. Moreover, we propose a specific framework which is intended to train amobile robot to navi宮ate quickly but safely. The 丘amework atilizes a novel statetransition checking method to ensure that the tTainin号 environment ptovided forthe robot always f0ⅡOws the Markov decision process properly. FUTthermore, italso employs a novel velocity increment schedulin牙 technique durin今 the trainin曾Process. The technique f0110ws a curriculum learnin晉 Sね'ategy by settin旦 a sma11
Value of veloC北y for the robot at the beginnin牙 of the trainin牙 episode. As thenumber of episodes increases, the robot's velocity is increased gradua11y so thatthe robot can 今radua11y learn the complex task of fast but safe naV喰ation in thetrainin号 environment form the easiest level, such as the one with the slown〕ovement, to the most difficult level, such as the one with the fast movement.

To obtain the optimal attendin今 Policies, we also use the sAc a1牙01'ithln toInake the robot able to attend the taτ今et person when its position is close to thetar号et person. DUI'in牙 the trainlng process, we propose the u'shaped reward
function which can guide the robot to attend the target person at his left side or at
his right side.八10reover, we pl,opose a novel wei牙ht、scheduled action sl〕〕oot}〕ing
technique so that the robot can 曾enerate slnooth and safe trajectories for theattending task. To make the robot can better portl'ay the surroundil〕牙S we also

Propose a novel policy network architecture which employs one din〕ensional
Convolutional neural netw01.k to extract features h'om laser scans auton〕atjca11y

Fina11y, to inte目rate a11 the optin〕al navi曾ation and attend11〕g policies, we
also propose a framework whicl〕 employs the double deep Q、network whicl) canD〕凡ke the robot leaxn to cl)oose the l〕].ost approprlate policy 今iven the current stateOf tl)e person・f0110圦7in今 environn)ent.1nside our framework, we introduce t・heaction 今enerator lnodule which can adjust the state of the pel'son、f0110win今
environment f01・ each sub policy apl)王Oprlately. Furthel'more, the module is alsoal)1e to sn〕ooth the actions generated by the robot usin目 the action smootl〕11〕gSI,rate号y to prevent tl〕e robot h北ting the target person when lt is close t0 1〕11n andW}〕en the robot's actions are generated 丘om cl)an部ng policies

From a11 experilnents tl)at we conduct ln our study, we can conclude thata〕e proposed navigation trahung fran)ework is able to make the Tobot navigateal)1辻oaching the target person faster W北1〕 1()WQr c0Ⅱision rate compared to other
DRL・based navigation baseline n念̲1new(旺']ζS.＼Ne conflrn〕 tl〕at the u'shapedroward function and the weigl)t・schedulcd aclion sn〕oothln牙 that we propose cann〕ake the robot attend tl)e targ'et pe玲on bot'1〕 at his le丘 and n牙ht side with smoothand safe trajectohes. we also confirm u〕at oU王 Proposed fran〕ewoflく Can lntegxate

a11 the navigation and attendin曾 Policies for obta11〕1ng the n)eta、1)olicy for thePerson・f0110wing task. F01・ future work, we consider uslng dynamic envlronments

for the training of the navi今ation and the attendln宮 tasks so tl)at ln01'e robust
Policies can be obtained.＼入7e also plan to propose another method so that the robot
Can switch 北S attendin晉 Position easily 訊7hen lt is close to the tar今et, person
Furthermore, we also wi11 Use computer vision techniques for detecting andt〕、acking the tar晉et pe王Son so that the policies can be deployed for real robots