The effect of behavioral preferences on skill acquisition in
determining unspecified, suitable action patterns to control
humanoid robots
Taiki Takayama and Tetsuyou Watanabe, Member, IEEE

Abstract— This research investigated the effect of behavioral
preferences on learning efficiency when attempting to determine
unspecified, but suitable action sequences for unfamiliar tasks.
The goal of this research was to develop a skill acquisition
support system for the elderly to aid them in using unfamiliar IT
products, particularly those of welfare systems. Here, behavioral
preference is defined as the type of action sequences that people
would prefer to adopt for completing unfamiliar tasks. To
achieve this goal, this research investigated the action sequencesof participants when they attempt to control the posture of an
unfamiliar humanoid robot with an unfamiliar controller. The
participants were assigned the task of making the humanoid
stand on one foot. Machine-learning-based methods were
presented for analyzing the behavioral preferences. The analysis
results indicate that participants having behavioral preferencesof adopting random action sequences can complete the task in amuch shorter time, compared to participants having abehavioral preference of adopting action sequences similar to
those of previous actions.

desired task on his/her own. However, it is difficult for theelderly and middle-aged people to learn these required IT
skills, as reported in [1], resulting in the so-called digital divide.
In order to make the best use of IT products, including thoseof welfare systems, systems that facilitate skill acquisition may
be required. However, it is difficult to develop such a support
system because a suitable sequence, which the user has todetermine, is unspecified. First, this research investigated theprocesses involved in learning or determining a suitablesequence in the operation of IT products. Controlling theposture of a humanoid robot was set as the target operation, as
this task would be unfamiliar to all participants, and theywould need to acquire this skill of controlling the posture. A
significant outcome of this study is that a humanoid robot
could one day help or support the elderly to carry out everyday
tasks. Machine-learning based methods were also presented inorder to extract the behavioral preferences obtained fromexperimental results; these preferences were then analyzed.

I. INTRODUCTION

Plenty of research has been conducted on skill acquisition.
One typical example of skill acquisition for determining
unspecified sequences for task completion is found in the
production of traditional handicrafts. To facilitate the transferof skills from an expert to a beginner (next generation), several
supporting systems were developed [2, 3]. If a person learning
Japanese calligraphy takes a bamboo brush attached to a robot,
the robot moves along a trajectory (sequence) recorded withan expert; the person learning can then follow the expert’s
trajectory. In those studies [2, 3], the key to dealing withunspecified sequences was to specify that the suitabletrajectory was that of the expert’s, with the assumption that theexpert’s trajectory is perfect. However, there is no guaranteethat following the expert’s trajectory/sequence would be
effective for beginners to acquiring that certain skill. There isa support system that displays the expert’s eye movements [4].
The eye movement of the expert could provide hints for taskcompletion. It would be useful if the sequence for the task were
specified. Another example of skill acquisition for determining
unspecified sequences for tasks is found in sports. In the fieldof sports engineering, effective methods for athletes or experts
to polish their skills have been studied [5, 6]. However, thetarget for acquisition is different, and hence, the methods
employed in that field cannot be used in this study. In the fieldsof neuroscience and psychology, a large amount of research on
skill acquisition has been conducted. Random patterns canprovide better learning and retention [7]. It is associated withthe concept of the contextual interference effect in psychology

This research investigated the behavioral preferences in thelearning process of participants when they attempt todetermine unspecified, but suitable action sequences forcompleting unfamiliar tasks: controlling the posture ofhumanoid robots that are expected to help the elderly in the
future. In this study, “unspecified” indicates that the numberof suitable sequences is infinite. For example, when operating
humanoid robots, different command sequences are available
for achieving the same final posture. It is necessary to
determine the types of action sequences that people prefer toadopt during the learning process. Some people tend to repeat
similar action sequences while other people tend to adopt
different action sequences. Behavioral preference is defined as
the pattern of a selected action sequence. This research
investigated the effect of behavioral preferences on learning
time and efficiency.
This research was motivated by the need for elderly people
to acquire skills that will help them use IT products. There are
numerous welfare and healthcare systems that require the useof smart phones and tablet PCs. They are useful and have highpotential in supporting societies with aging populations. The
elderly, as well as middle-aged people who support the elderly,
therefore need to learn to use IT products such as smartphones
and tablet PCs. Current IT products come with little or no
instructions for use, and therefore, the user has to determine anunspecified, but suitable action sequence for completing a*Research partly supported by JSPS Kakenhi 26540143.
T. Takayama is with the Graduated school of Natural science andTechnology, Kanazawa University, Kakuma-machi, Kanazawa, 9201192,
Japan (e-mail: t.taiki@stu.kanazawa-u.ac.jp).

T. Watanabe is with the Institute of Science and Engineering, Kanazawa
University, Kakuma-machi, Kanazawa, 9201192 Japan (e-mail: tewatanabe@ieee.org).

[8]. Some learning mechanisms [9] such as motor chunking[10, 11] and associative learning [12, 13] have been presented.
However, the sequences to be learned were specified.
Therefore, another type of study is required to understand thelearning process when acquiring skills associated withunspecified sequences.
Robot
FailureCommandController

Upright
standing

Success(One-foot standing)Commandhistories

Target task

Figure 1. Schematic view of experimental setup with target task:
controlling a humanoid to shift from an upright standing posture (standing
with both feet on the ground) to a one-foot standing posture

Figure 2. Controller display on the tablet PC
TABLE I.
ASSOCIATION BETWEEN BUTTON NUMBERS AND JOINTSWITH THEIR ROTATIONAL DIRECTION (THE INDIVIDUAL JOINTS FOR UPPER
BODY ARE NOT SHOWN BECAUSE THEY WERE NOT USED)
Button number
Joint
Joints for upper body
Right thigh (roll)
Right thigh (pitch)
Left thigh (pitch)
Left thigh (roll)
Right knee
Left knee
Right ankle (roll)
Right ankle (pitch)
Left ankle (pitch)
Left ankle (roll)

Negative/inverse
rotation

Positive rotation

1–14
15
17
19
21
23
25
27
29
31
33

16
18
20
22
24
26
28
30
32
34

II. METHODOLOGY
A. Participants
The participants who took part in the experiment wereeleven healthy students consisting of four males and sevenfemales (aged between 18 and 22 years, with an average ageof 20.5 years). The experiment was approved by the Medical
Ethics Committee of Kanazawa University.
B. Task
The task was to control a humanoid robot, to make it shift
from its initial upright standing posture to a single-footstanding posture, as shown in Fig. 1. A single-foot standing
posture is defined as being able to stand with either foot off theground, without falling under a steady-state condition. If aparticipant succeeded in achieving a single-foot standing

posture three times, the experiment was stopped. The
experiment was also stopped if one hour had elapsed withoutcompletion of the task.
The operation of a humanoid robot was used as the target
task in this study; this operation is unfamiliar to the generalpublic. Therefore, the participants of this study would notknow how to perform the operation in advance, and theywould attempt to determine unspecified and suitable action
sequences for completing this unfamiliar task. In addition, thecontroller for the robot, which was a self-made application
written for tablet PCs, was also unfamiliar to the participants.
C. Apparatus
Fig. 1 shows the schematic view of the experimental setup.
The humanoid robot was a Kondo Kagaku KHR-3HV, Ver. 2.
The tablet PC was a Sony VAIO Duo 11. The participants
controlled the humanoid robot through a controller displayedon the tablet PC. The controller was created in Microsoft
Visual Studio 2012 based on the Rcb4 library presented by
Kondo Kagaku. The program for the controller was written in
the Visual C# language.
The controller is shown in Fig. 2. If a participant touches
or pushes the button displayed on the touch screen, thecorresponding motor rotates at a constant angle of 72/23
degrees. The name of the associated joint is displayed on the
button. The button numbers were used to simplify therecording and analysis. Table 1 lists the association between
the button numbers and joints with their rotational direction. If
the button is touched or pushed, the associated number as wellas the time it was pushed is recorded in the command history.
D. Procedure
The task (mentioned above) and the operation of thecontroller were explained to the participants. The association
between the buttons and the joints with rotational direction
was explained to the participants by illustrating how the robot
moved in response to the commands provided. In a preliminary
experiment, it took a very long time if a participant attempted
to control all joints (at least 1 h was required even whenexperts tried). The participants were then asked to control only
the joints of the lower body.
The participants carried out the robot operation. Every trial
started from an initial state where the robot had an uprightstanding posture. The participant then tried to achieve therequired single-foot standing posture. If the robot fell down or
the participants determined that the robot could not attain the
one-foot standing posture (for example, in cases where therobot had postures indicating that it was about to fall), the trial
was stopped and designated a failure. After every trial, we
asked the participants to push button 35 if the trial was a failure
and push button 36 if the trial was a success. The experiments
were completed or stopped when the participant succeededwith the trials three times or when one hour had elapsed.
III. RESULTS
A. Total operation time
Fig. 3 shows the total operation time for each participant,which is the total time required for successful completion of
the trial three times. In the figure, “Par.” refers to participant.
Note that the total operation time for Par. 6 and Par. 8 was onehour. In the case of Par. 6, none of the trials were successful,and in the case of Par. 8, only one trial was successful in the
stipulated 1-hour limit.

(No success) (1 success)60

𝑑𝑖𝑗 was calculated for all pairs.

50
40

20
10
0

Trial 2

40
35
30
25
20
15
10
5
0

Trial 1

Trial 2AS vector
0
・
・

8
・
・
・

4
1 6 11 16 21 26 31 36 41 46 51 56

Operation number from start(Par.2)

・・

1 6 11 16 21 26 31 36 41 46 51 56

3

Operation number from start(Par.2)

・0

Figure 4. Command history for Par.2

Figure 5. Example representationof trial data as a AS vector

IV. MACHINE LEARNING BASED BEHAVIOR PREFERENCEANALYSIS

The presence of many similar action sequences indicatesthat the participant preferred those certain action sequences.
Therefore, in order to observe the behavioral preference ateach trial, we studied the similarities among action sequences.
Here, machine learning [14]–based methods for analyzing thesimilarities (behavioral preferences) are presented.
A. Similarity among action sequences
Each trial was represented by the vector
𝒙 = [𝑥1 , 𝑥2 , ⋯ 𝑥34 ]T ∈ 𝑅34
where 𝑥𝑘 represents the number of times button k was pushedduring the trial. We call the vector 𝒙 an AS (action sequence)
vector, as shown in Fig. 5. The final posture of the robot did
not depend on the order a button was pushed, but rather on thenumber of times a button was pushed. For example, thefollowing two sequences resulted in a similar rotation angle forthe knee joint: (1) 4 times pushing button for knee joint,
pushing button for other joints, 4 times pushing button for knee
joint; and (2) 8 times pushing button for knee joint. Therefore,
the command order information was ignored for this analysis.
In order to better understand how the participants control the
joints, the numbers associated with the positive andnegative/inverse rotations for the same joint were separately

k-means clustering (k = 2) was performed by setting themean and median values for 𝑑𝑖𝑗 as feature values. Note thatmedian value was considered for taking distribution form into
account. Fig. 6 shows clustered groups: Groups 1 and 2. Group
1 has relatively small median and mean values of dissimilarity,
which indicates that the participants belonging to Group 1
performed or repeated similar action sequences to determine asuitable sequence for completing the given task. On the other
hand, the participants belonging to Group 2 performed
relatively randomized action sequences. These results are due
to behavioral preferences.
Fig. 7 shows the mean total operation time with SD. foreach group. It is observed that the participants belonging toGroup 2 completed the sequence in a short time, whereas theparticipants belonging to Group 1 required a long time.
However, there was not a statistically significant difference
between the groups when applying Welch’s t-test. In order todetermine the reason for this, similarities to suitable sequences
for completion are focused on and investigated in the next
subsection.

Group1
Group2
Mean of dissimilarity[-]

Figure 6. k-means clustering (k = 2)
result for dissimilarity
between each trial

Total operation time[min]30

Button number [-]Button number [-]Trial 1

𝑑𝑖𝑗 = |𝒙i − 𝒙j | (𝑖 ≠ 𝑗)(𝑖, 𝑗 = 1, ⋯ 𝑁)

70

Figure 3. Total operation time for each participant: Par. stands forparticipant.
40
35
30
25
20
15
10
5
0

By utilizing the AS vector 𝒙, dissimilarities between action
sequences were analyzed. Let N be the number of trials for acertain participant. Let 𝒙i be 𝒙 for the ith trial. The
dissimilarity between 𝒙i and 𝒙j can thus be defined by

Median of dissimilarity[-]

Total operation time[min]B. Command history
Fig. 4 shows part of the command history for Par. 2 as an
example. The horizontal axis shows the number of commands
that the participant executed, and the vertical axis shows the
button number associated with a joint with rotational direction.
Note that button 35 indicates failure and button 36 indicates
success. Using these commands, the operation can be divided
into each trial, as seen in Fig. 4.

treated. For example, if the participant pushed the button
numbers associated with both positive and negative/inverse
rotations, it indicates that the participant tried to control the
joint precisely. On the other hand, if the participant pushed the
button number associated with either one rotation, it indicatesrough or brave control.

60
50
40
30
20

10
0

Group1

Group2

Figure 7. Mean total operation
time for clusteredgroups with SD.

B. Similarity between suitable sequence for completion andtrial sequences for participants in Group 1
The similarity between suitable sequences for completionand the trial sequences for the participants in Group 1 wasinvestigated. The number of suitable sequences is more thanone. Therefore, the set of suitable sequences should becompared with action sequences of the trials. The method
based on nearest neighborhood algorithm [14] was then used
for the analysis. Because the suitable sequence is obtained at
the final posture, the final posture vector 𝒙𝑓 was used insteadof the AS vector 𝒙 whereT

𝒙𝑓 = [𝑥𝑓1 , 𝑥𝑓2 , ⋯ 𝑥f17 ] ∈ 𝑅17

𝜌𝑖𝑗 = min |𝒙fij − 𝒙fk | 
𝒙f ∈𝑆skwhere 𝒙fk ∈ 𝑆s is the kth 𝒙𝑓 in 𝑆s . The k-means clustering(K=2) was conducted by setting the following mean andmedian maximum values of 𝜌𝑖𝑗 for every participant as feature
values:𝜌𝑚𝑒𝑎𝑛𝑖 = mean 𝜌𝑖𝑗 ,j𝜌𝑚𝑎𝑥𝑖 = max 𝜌𝑖𝑗 jHere we saw how far from stable one-foot standing
postures participant tried to control the robot to, from theviewpoint of “maximally” and “on average”. Fig. 8 shows theclustering result, where Group 1A and Group 1B are the twoclustered groups. There are two participants in Group 1. The
participants in Group 1A repeated similar action sequencesthat were quite different from suitable sequences, whereas theparticipants in Group 1B repeated similar action sequencesthat were similar to suitable sequences. Fig. 9 shows the mean
total operation time with SD. for Group 1A, Group 1B, andGroup 2. The task completion times for Group 1A were long,whereas those for Group 1B times were short, shorter thanGroup 2 even. Statistically significant differences wereobserved between Group 1A and Group 2 and between Group
1A and Group 1B, when applying Tukey’s test. The reason for
the non-significant differences between Group 1 and Group 2
is considered to be due to the large difference between Group
1A and Group 1B.
V. DISCUSSION
This research investigated the effect of behavioral
preferences on learning time and efficiency, with the aim ofdeveloping systems to facilitate skill acquisition for the elderly
to aid them in using unfamiliar IT products, particularly thoseof welfare care systems. To achieve this goal, the learning
process when participants tried to control the posture of an
unfamiliar humanoid robot was investigated. The task was tobring the robot to a stable one-foot standing posture from astable, upright initial standing posture. The machine learning–based method was also presented for analyzing behavioral
preferences. The analysis results indicate that the participantsof Group 2, who have a behavioral preference for random
action sequences, were able to complete the task in a very shorttime. Conversely, the participants of Group 1, who have a
behavioral preference for action sequences similar to those ofprevious actions required longer periods for completion. Notethat the required times for task completion for the participantsin Group 1 were widely ranged. Similar to simple optimization

Group1B

Group1A

Mean value of dissimilarity[-]

Total operation time[min]Let 𝑆s be the set of final postures and their laterally inverted
postures for the successful trials for all participants. Because
the robot is bilaterally symmetric, postures laterally inverted
to those under stable conditions are also considered stable. Let
𝑆t𝑖 be the set of 𝒙𝑓 vectors corresponding to all AS vectors forthe ith participant. The dissimilarity between 𝒙fij ∈ 𝑆t𝑖 (where𝒙fij corresponds to the jth AS vector for the ith participant) andthe set of stable final postures 𝑆s was then calculated asfollows:

methods such as the steepest descent method, if participants(fortunately) could adopt action sequences close to the suitablesequence, the required time was short (Group 1B). If they
(unfortunately) could not adopt action sequences close to thesuitable sequence, the required time was very long (Group 1A).
It is interesting that behavioral preferences are analogous to
optimization methods. This suggests the possibility ofdeveloping a system for facilitating skill acquisition based on
optimization methods. In summary, random behavioral
preferences are essential for achieving higher learning
efficiencies when attempting to determine unspecified suitableaction sequences for completing unfamiliar tasks.
Max of dissimilarity[-]𝑥𝑓𝑖 = 𝑥2𝑖−1 − 𝑥2𝑖

60

＊

＊＊

50
＊＊：p＜0.05＊：p＜0.01

40
30
20
10
0

Group1A Group1B

Group2

Figure 8. k-means clustering (k = 2) Figure 9. Total operation time forresult for dissimilarity between
clustered groups based onsuitable and trial sequencesdissimilarity between suitablefor participants in Group 1
and trial sequences
for participants in Group 1

REFERENCES[1] Hokuriku life care system study group, “Report for co-creation ofhealthy aged society utilizing ICT,” JST report, 2015.

[2] K. Henmi and T. Yoshikawa, “Virtual Lesson and Its Application to
Virtual Calligraphy System,” Proc. of the IEEE Int. Conf. on Robotics
and Automation, 1998, pp.1275-1280.
[3] Y. Yokokura, S. Katsura, K. Ohishi, “Stability Analysis andExperimental Validation of a Motion-Copying System,” IEEE Trans.
Industrial Electronics, vol. 56, No. 10, pp. 3906-3913, July 2009.
[4] T. Nagamatsu, Y. Kaieda, J. Kamahara, and H. Shimada,“Development of a Skill Acquisition Support System Using Expert's
Eye Movement,” in Human Interface and the Management ofInformation. Interacting in Information Environments, M. J. Smithand G. Salvendy, Ed. Springer Berlin Heidelberg, 2007, pp. 430-439.
[5] A. M. Williams and J. H. Nicola eds. Skill acquisition in sport:
Research, theory and practice. Routledge, 2004.
[6] K. Lohse , D. E. Sherwood and A. F. Healy, “ How changing the
focus of attention affects performance, kinematics, andelectromyography in dart throwing,” Human Movement Science,
vol.29, pp.542-555, 2010[7] T. S. Lemieux and V. B. Penhune, “The effect of practice pattern on
the acquisition, consolidation, and transfer of visual-motor
sequences”, Exp. Brain Res., vol.204, pp.271-281, 2010[8] R. A. Magill and K. G. Hall, “A review of the contextual interference
effect in motor skill acquisition”, Human movement science, vol.9,
No.3, pp. 241-289, 1990.
[9] W. B. Verwey and E. L. Abrahamse, “Distinct modes of executing
movement sequences: Reacting, associating, and chunking” Acta
Psychologica, Vol. 140, pp.274-282, 2014.
[10] J. A. Adams, “A closed-loop theory of motor learning,” Journal ofMotor Behavior, Vol. 3, pp. 111–150, 1971.
[11] W. James, The principles of psychology, New York: H. Holt & Co.,
1890.
[12] J. Paillard, “The pattern of skilled movements,” in Handbook ofphysiology. A critical comprehensive presentation of physiologicalknowledge and concepts, H. W. Magoun Ed., Washington, DC:
American Physiological Society, 1960, pp. 1679–1708.
[13] G. A. Miller, E. Galanter, and K. H. Pribram, Plans and the structureof behavior, New York: Holt, 1960
[14] C. M. Bishop, Pattern Recognition and Machine Learning, Springer,
2006.