# How Can Brain Learn to Control a Nonholonomic System?

Authors: Noriyasu Homma; Shinpei Kato; Takakuni Goto; Ivo  Bukovsky; Ryuta Kawashima; Makoto Yoshizawa
Journal: Journal of Robotics
 (2010)

---

## Abstract

Humans can often conduct both linear and nonlinear control tasks after a sufficient number of trials, even if they initially do not have sufficient knowledge about the system's dynamics and the way to control it. Theoretically, it is well known that some nonlinear systems cannot be stabilized asymptotically by any linear controllers and we have reported by an f-MRI experiment that different types of information may be involved in linear and nonlinear control tasks, respectively, from a brain function mapping point of view. In this paper, from a controllability analysis, we still show a possibility that human may use a linear control scheme for such nonlinear control tasks by switching the linear controllers with a virtual constraint. It is suggested that the proposed virtual constraint can play an important role to overcome a limitation of the linear controllers and to mimic human control behavior.


---

## Body

## 1. Introduction

In the fields of control engineering, system engineering, and brain sciences, the excellent human’s control abilities have been widely studied [1–6]. Among such studies, Wolpert and Kawato [7] have proposed a new model of human control mechanism called Modular Selection And Identification Control (MOSAIC) in which many elemental prediction and control system modules are combined in order to conduct the desired motion. Imamizu et al. [8] have evaluated the MOSAIC model using linear control tasks. They used a functional MRI (f-MRI) scanner to observe brain activities during the control tasks and reported plausibility of the model.However, human ability is not limited to the linear control case. In fact, humans can often achieve difficult control tasks after a sufficient number of trials, even if they initially do not have sufficient knowledge about the system’s dynamics and the way to control it. For example, Goto et al. [9] have reported manual control ability to operate a 2-link planar under actuated manipulator (2PUAM). The 2PUAM is a nonlinear system with a nonholonomic constraint and cannot be stabilized asymptotically by any linear controller. In this case, to control such nonholonomic manipulator, the operators have to plan the object’s trajectory first and then control the system to follow the trajectory. To do so, the operator must use shape information of the manipulator while the operator does not need such shape information to control linear manipulators.To verify this difference between linear and nonlinear control tasks, we have conducted another f-MRI experiment using the 2PUAM [10]. The results suggest that the difference can be observed through the brain activities. In fact, a specific area involving the shape information processing was activated with a significant difference compared to the linear task only when subjects control the 2PUAM.In this paper, to further clarify the experimental results, a new model-based analysis is conducted. The model is a multiple model-based reinforcement learning (MMRL) [11] for nonlinear controls but it is a linear controller. Using the MMRL, we attempt to explain the f-MRI results of the human information processing for the 2PUAM control task from a control theory point of view.


## 2. 2-Link Planer Under Actuated Manipulator (2PUAM)

In this study, we use the 2PUAM shown in Figure1 as the control object. The 2PUAM moves in a horizontal plane. Thus, the 2PUAM is free from gravity, and arbitrary positions are equilibrium points. Motor is mounted on the first joint. Then the dynamic equations of the 2PUAM are(1)M11θ̈1+M12θ̈2+C1=T1+Tfr1,M21θ̈1+M22θ̈2+C2=T2+Tfr2,
where(2)M11=(m1+m2)l22+m2l22+2m2l1l2cosθ2,M12=M21=m2l22+m2l1l2cosθ2,M22=m2l22,C1=-m2l1l2θ̇1(2θ̇1+θ̇2)sinθ2+c1θ̇,C2=m2l1l2θ̇12sinθ2+c2θ̇2.
Here, as shown in Figure 1, m1 and m2 are the mass of the first and second links, l1 and l2 are the length, c1 and c2 are the resistance coefficient, respectively, θ1  and θ2  are angles, θ̇1  and  θ̇2  are angular velocities, and θ̈1  and θ̈2 are angular accelerations of joints, respectively. The right-hand sides of (1) are the input and friction torques. In this study, we simplify the model as follows:(3)m1=m2=m,l1=l2=l,c1=c2=0,T2=Tfr1=Tfr2=0.Figure 1
2-link planner under actuated manipulator (2PUAM).It is known that the 2PUAM is not controllable by the standard continuous feedback. This implies that reaching the 2PUAM and stopping it at some point is a difficult task. Although some indirect methods have been proposed to control the 2PUAM from the control engineering point of views [9], it is still unclear how human can control the nonlinear object such as the 2PUAM.


## 3. Summary of f-MRI Experiments

### 3.1. Control Tasks and Training

Here, human operator is required to feed an input torqueτ∈[-1,1] so that the end effector of the arm will be driven to the goal point and kept at the point. Since the position of the first joint is fixed, there are at least two objective positions of the second joint as shown in Figure 2: the upper position of the second joint p1 and the lower p2.Figure 2
Positioning task.The manual control experiment was conducted by 6 neurologically normal subjects (19–24 years of age; six males) participated in the experiments. All subjects were right-handed. Informed written consent was obtained from each participant. They had no knowledge about the dynamic response of the system before the experiment. They observed the virtual manipulator’s states (positions, angles, and velocities of the 2PUAM) through visual data displayed on an LCD monitor and fed the torque by using a joystick. The time limit was 30 seconds for each trial, and 300 training trials have been conducted outside of the MRI scanner before the following scanning sessions. The duration of 300 training trials depended on the subject’s willingness and tiredness, but all the subjects completed them for 2-3 days.For each trial, the performance index defined by the following equation was recorded:(4)J=∑k=0T(x(k)-xG)2+(y(k)-yG)2.
Here (x(k),y(k)) denotes the position of the end effector at time kΔt(k=0,1,2,…,T), Δt is the interval of time step, (xG,yG) is the position of the goal point, and T is the maximum steps of each trial. The performance index J has been displayed on the monitor to guide subjects’ criterion.


### 3.2. MRI Scanning Sessions

In scanning sessions, the trained subjects control two kinds of virtual manipulators whose shapes are projected on a screen in the MRI scanner as shown in Figure3, by using an optical (magnet-free) joystick [10]. Both manipulators are the same in shape, but the first manipulator is the 2PUAM and the other is a manipulator which has a linear input and output relation (subjects can control it like a PC mouse). Controlling the 2PUAM is the main task while the linear control task is a comparative one, and thus the former is called test trial and the latter is baseline trial, and their duration are called test and baseline periods, respectively. In other words, subjects can directly operate the coordinate of the end effector of the manipulator by the joystick in baseline trial. On the other hand, subjects operate only the torque of the first joint of the manipulator by the joystick and indirectly control the coordinate of the end effector in test trial. In each trial, subjects try to move the end effector of the manipulator to the goal point and keep it at the goal. The coordinates of the goal points are chosen randomly and displayed on the screen when trial starts.Figure 3
Experimental system.MRI scanner (1.5T SIEMENS: Symphony 1.5T) was used to obtain blood oxygen level-dependant contrast functional images. Images weighted with the apparent transverse relaxation time were obtained with an echo-planner imaging sequence (repetition time: 3.0 s; echo time: 50 ms; flip angle: 90; field of view:256×256 mm). High-resolution anatomical images of all subjects were also acquired with a T1-weighted sequence.


### 3.3. Results

Figure4 indicates regions significantly more activated during test periods than baseline periods (t>5.89, P<.001). Activities in the primary motor cortex, the somatosensory cortex, the somatosensory association cortex, the prefrontal cortex, the inferior temporal gyrus, and the fusiform gyrus were observed [10].Significant activated areas of test period compared to baseline period (for more details, see [10]).
(a)(b)In the nonlinear control task, significant activities of the inferior temporal gyrus and the fusiform gyrus were observed. These areas are known to have an intimate involvement in recognizing the characteristic (color and shape) of the object. On the other hand, the prefrontal cortex is known as a region that receives the information that is necessary for action planning from both the temporal association area and the occipital association area and assembles complicated action plan. These suggest that in operating 2PUAM, subjects use the information about shape or position of 2PUAM and make trajectory planning of the positioning task based on that information. From a viewpoint of control theory, it may be worth to mention that such information is not needed for the linear control, but it is necessary for the nonholonomic systems control.


## 3.1. Control Tasks and Training

Here, human operator is required to feed an input torqueτ∈[-1,1] so that the end effector of the arm will be driven to the goal point and kept at the point. Since the position of the first joint is fixed, there are at least two objective positions of the second joint as shown in Figure 2: the upper position of the second joint p1 and the lower p2.Figure 2
Positioning task.The manual control experiment was conducted by 6 neurologically normal subjects (19–24 years of age; six males) participated in the experiments. All subjects were right-handed. Informed written consent was obtained from each participant. They had no knowledge about the dynamic response of the system before the experiment. They observed the virtual manipulator’s states (positions, angles, and velocities of the 2PUAM) through visual data displayed on an LCD monitor and fed the torque by using a joystick. The time limit was 30 seconds for each trial, and 300 training trials have been conducted outside of the MRI scanner before the following scanning sessions. The duration of 300 training trials depended on the subject’s willingness and tiredness, but all the subjects completed them for 2-3 days.For each trial, the performance index defined by the following equation was recorded:(4)J=∑k=0T(x(k)-xG)2+(y(k)-yG)2.
Here (x(k),y(k)) denotes the position of the end effector at time kΔt(k=0,1,2,…,T), Δt is the interval of time step, (xG,yG) is the position of the goal point, and T is the maximum steps of each trial. The performance index J has been displayed on the monitor to guide subjects’ criterion.


## 3.2. MRI Scanning Sessions

In scanning sessions, the trained subjects control two kinds of virtual manipulators whose shapes are projected on a screen in the MRI scanner as shown in Figure3, by using an optical (magnet-free) joystick [10]. Both manipulators are the same in shape, but the first manipulator is the 2PUAM and the other is a manipulator which has a linear input and output relation (subjects can control it like a PC mouse). Controlling the 2PUAM is the main task while the linear control task is a comparative one, and thus the former is called test trial and the latter is baseline trial, and their duration are called test and baseline periods, respectively. In other words, subjects can directly operate the coordinate of the end effector of the manipulator by the joystick in baseline trial. On the other hand, subjects operate only the torque of the first joint of the manipulator by the joystick and indirectly control the coordinate of the end effector in test trial. In each trial, subjects try to move the end effector of the manipulator to the goal point and keep it at the goal. The coordinates of the goal points are chosen randomly and displayed on the screen when trial starts.Figure 3
Experimental system.MRI scanner (1.5T SIEMENS: Symphony 1.5T) was used to obtain blood oxygen level-dependant contrast functional images. Images weighted with the apparent transverse relaxation time were obtained with an echo-planner imaging sequence (repetition time: 3.0 s; echo time: 50 ms; flip angle: 90; field of view:256×256 mm). High-resolution anatomical images of all subjects were also acquired with a T1-weighted sequence.


## 3.3. Results

Figure4 indicates regions significantly more activated during test periods than baseline periods (t>5.89, P<.001). Activities in the primary motor cortex, the somatosensory cortex, the somatosensory association cortex, the prefrontal cortex, the inferior temporal gyrus, and the fusiform gyrus were observed [10].Significant activated areas of test period compared to baseline period (for more details, see [10]).
(a)(b)In the nonlinear control task, significant activities of the inferior temporal gyrus and the fusiform gyrus were observed. These areas are known to have an intimate involvement in recognizing the characteristic (color and shape) of the object. On the other hand, the prefrontal cortex is known as a region that receives the information that is necessary for action planning from both the temporal association area and the occipital association area and assembles complicated action plan. These suggest that in operating 2PUAM, subjects use the information about shape or position of 2PUAM and make trajectory planning of the positioning task based on that information. From a viewpoint of control theory, it may be worth to mention that such information is not needed for the linear control, but it is necessary for the nonholonomic systems control.


## 4. Controllability Analysis

The difference between linear and nonlinear control tasks observed through the significant brain activities can be a reflection of the human control mechanism that can cognize the target nonlinear dynamics and use an appropriate piece of information. However, this is not sufficient to conclude that human does not use any linear control scheme. Instead, the hypothesis proposed in this paper is that human can use a linear control scheme by switching linear controllers responsible for specific regions where the linear approximation can work well for the target nonlinear task. The following linear model can then be employed to verify the hypothesis.


### 4.1. Multiple Model-Based Reinforcement Learning (MMRL)

MMRL has multiple modules that are pairs of prediction model which predicts future state of the controlled objects and reinforcement learning controller which learns the control output. “Responsibility signals” are calculated from the softmax function of the prediction errors. The prediction model which outputs the more accurate prediction has the larger responsibility signal. By weighting control signal and learning of the each module with responsibility signals, these modules are adapted to the corresponding specific situations, respectively.In this study, we use a multiple linear quadratic controller (MLQC) by using multiple linear prediction and quadratic reward models as an efficient implementation of the MMRL [11]. Change in the state vector of the target system, x∈R4, is given by(5)ẋ(t)=f(x(t),u(t)),
where x is the state vector of the system, and u is the control output. Each variable of the vector for the 2PUAM can be defined as(6)x1=θ1,x2=θ̇1,x3=θ2,x4=θ̇2.Linear prediction models of the MMRL can be represented as follows:(7)ẋ̂i(t)=fi(x(t),u(t))=Aix(t)+Biu(t),i=1,2,…,n.
where n denotes the number of modules. State prediction ẋ̂ is given by a weighting sum of prediction models with responsibility signals λi [11]:(8)ẋ̂(t)=∑i=1nλi(Ei(t))ẋ̂i(t)=∑i=1nλi(Ei(t))(Aix(t)+Biu(t)),
where Ei(t) is a short-time average of the prediction error ẋ(t)-ẋ̂(t). Learning of each prediction model in (7), i∈{1,2,…,n}, is conducted by changing its parameter vector wi consisting of all the elements of the matrices Ai and Bi as follows: (9)ẇi(t)=ηwλi(Ei(t))(∂fi∂wi)T(ẋ(t)-ẋ̂(t)),
where ηw (>0) is an update coefficient. Schematic diagram of the multiple predictor-controller pair architecture is shown in Figure 5.Figure 5
Schematic diagram of multiple predictor-controller pair architecture [11].


### 4.2. Controllability of 2PUAM

We obtain the following state equation of the 2PUAM from (1) and (6) under the condition in (3):(10)ẋ1=x2,ẋ2=12-cos2x3{(3+cosx3)x22sinx3+x2x4sinx3+u/ml2},ẋ3=x4,ẋ4=-(1+cosx3)ẋ2-x22sinx3.Equation (10) can be described using the vector form as(11)[ẋ1ẋ2ẋ3ẋ4]=f([x1x2x3x4],u).
Here, we define the microscopic fluctuation at some positions xa in phase space as δx=x-xa, δu=u-ua. Calculating the Taylor series of (11) and ignoring the higher-order terms from the second order, we get(12)δẋ=Aδx+Bδu.
Here matrices A and B are(13)A=∂f∂x=(01000∂ẋ2∂x2∂ẋ2∂x3∂ẋ2∂x400010∂ẋ4∂x2∂ẋ4∂x3∂ẋ4∂x4),B=∂f∂u=1aml2(010-(1+cosx3)),
where(14)∂ẋ2∂x2=(2x2(3+cosx3)+x4)sinx3a,∂ẋ2∂x3=(6x2+x4)x2cos2x3-2x22a+usin2x3ml2a,∂ẋ2∂x4=x2sinx3a,∂ẋ4∂x2=-(1+cosx3)∂ẋ2∂x2-2x2sinx3,∂ẋ4∂x3=ẋ2sinx3-(1+cosx3)∂ẋ2∂x3-x22cosx3,∂ẋ4∂x4=-(1+cosx3)x2sinx3a,a=2-cos2x3.Since the MMRL is a linear controller, it cannot control the 2PUAM. Indeed, the rank of the controllability matrixUc=[BABA2BA3B] at any equilibrium position x0 becomes 2, which is not the full rank, 4. That is, let us denote an equilibrium point by using arbitrary values θ10 and θ20(15)x0=(θ100θ200)T
with no input u = 0. Then, from (14), (16)A=∂f∂x|u=0x=x0=(0100000000010000),B=∂f∂x|u=0x=x0=1a0ml2(010-(1+cosθ20)),
where a0=2-cos2θ20. In this case, the rank of the Uc is not equal to the full rank, 4, but is 2 as follows:(17)rank[Uc]=rank[BABA2BA3B]=2.


### 4.3. Possible Control Strategy

The rank of the controllability matrix can, however, be the full rank, 4, at some positionsxs in the phase space where angular velocity of the first joint is not exactly zero, θ̇1≠0, even if the θ̇1 is very close to zero, as shown in Figure 6.Figure 6
Rank of the controllability matrixUc at some positions xs(θ̇2=0,u=0).To verify the controllability, let us denote positionsxs in the phase space by using arbitrary values θ1s, θ2s, and θ̇2s:(18)xs=(θ1sθ̇1sθ2sθ̇2s)T,θ̇1s≠0.
Then, from (14),(19)As=∂f∂x|u=usx=xs=(01000∂ẋ2∂x2∂ẋ2∂x3∂ẋ2∂x400010∂ẋ4∂x2∂ẋ4∂x3∂ẋ4∂x4),Bs=∂f∂x|u=usx=xs=1asml2(010-(1+cosθ2s)),
where as=2-cos2θ2s. In this case, the rank of the controllability matrix is 4, the full rank:(20)rank[Ucs]=rank[BsAsBsAs2BsAs3Bs]=4.
Thus, it is confirmed that 2PUAM cannot be stabilized at any equilibrium point, but if θ̇1s≠0, the end effector of the 2PUAM can approach any geometrical point with any slow speed.


### 4.4. Discussions

The slow speed approach verified above might be an interesting result because, nevertheless human subjects can control the 2PUAM very well, it is often very hard even for human subjects to stop the 2PUAM completely in the manual control tasks [9, 10]. The linear controller can be responsible only for a small region in which the linear approximation can work well. Thus, by switching multiple linear models, there is a possibility to control the 2PUAM with the MMRL.Different from (16), we assumed the condition in (19) where ∂ẋi/∂xj≠0, i=2,4,andj=2,3,4 to make the rank of controllability matrix be the full rank. The condition implies a virtual constraint of the manipulator’s shape (joint movement) because, for example, ∂ẋ2/∂x3≠0 implies the virtual existence of relation between θ̈1 and θ2 that makes a constraint on the angular acceleration depended on the angle of the manipulator shape. Interestingly, if human subjects could feel and realize such virtual constraint on the 2PUAM control (joint movement), the control task can sometimes be achieved relatively easier [9].According to the f-MRI results, subjects may use the shape and position information of the 2PUAM. The virtual constraint discussed above can further be created in order to make the control easier. Unfortunately, the controllability analysis does not prove this hypothesis, but there is a possibility that in operating the 2PUAM, subjects use the shape information to switch the controllers [10] and create the virtual constraint to make the control easier [9]. In this sense, MMRL can be regarded as a linear model for controlling the 2PUAM. Then, if human’s superior learning ability to control the complex nonlinear system could be based on such linear control schemes, its implementation on a robot system might be easier than we expected.


## 4.1. Multiple Model-Based Reinforcement Learning (MMRL)

MMRL has multiple modules that are pairs of prediction model which predicts future state of the controlled objects and reinforcement learning controller which learns the control output. “Responsibility signals” are calculated from the softmax function of the prediction errors. The prediction model which outputs the more accurate prediction has the larger responsibility signal. By weighting control signal and learning of the each module with responsibility signals, these modules are adapted to the corresponding specific situations, respectively.In this study, we use a multiple linear quadratic controller (MLQC) by using multiple linear prediction and quadratic reward models as an efficient implementation of the MMRL [11]. Change in the state vector of the target system, x∈R4, is given by(5)ẋ(t)=f(x(t),u(t)),
where x is the state vector of the system, and u is the control output. Each variable of the vector for the 2PUAM can be defined as(6)x1=θ1,x2=θ̇1,x3=θ2,x4=θ̇2.Linear prediction models of the MMRL can be represented as follows:(7)ẋ̂i(t)=fi(x(t),u(t))=Aix(t)+Biu(t),i=1,2,…,n.
where n denotes the number of modules. State prediction ẋ̂ is given by a weighting sum of prediction models with responsibility signals λi [11]:(8)ẋ̂(t)=∑i=1nλi(Ei(t))ẋ̂i(t)=∑i=1nλi(Ei(t))(Aix(t)+Biu(t)),
where Ei(t) is a short-time average of the prediction error ẋ(t)-ẋ̂(t). Learning of each prediction model in (7), i∈{1,2,…,n}, is conducted by changing its parameter vector wi consisting of all the elements of the matrices Ai and Bi as follows: (9)ẇi(t)=ηwλi(Ei(t))(∂fi∂wi)T(ẋ(t)-ẋ̂(t)),
where ηw (>0) is an update coefficient. Schematic diagram of the multiple predictor-controller pair architecture is shown in Figure 5.Figure 5
Schematic diagram of multiple predictor-controller pair architecture [11].


## 4.2. Controllability of 2PUAM

We obtain the following state equation of the 2PUAM from (1) and (6) under the condition in (3):(10)ẋ1=x2,ẋ2=12-cos2x3{(3+cosx3)x22sinx3+x2x4sinx3+u/ml2},ẋ3=x4,ẋ4=-(1+cosx3)ẋ2-x22sinx3.Equation (10) can be described using the vector form as(11)[ẋ1ẋ2ẋ3ẋ4]=f([x1x2x3x4],u).
Here, we define the microscopic fluctuation at some positions xa in phase space as δx=x-xa, δu=u-ua. Calculating the Taylor series of (11) and ignoring the higher-order terms from the second order, we get(12)δẋ=Aδx+Bδu.
Here matrices A and B are(13)A=∂f∂x=(01000∂ẋ2∂x2∂ẋ2∂x3∂ẋ2∂x400010∂ẋ4∂x2∂ẋ4∂x3∂ẋ4∂x4),B=∂f∂u=1aml2(010-(1+cosx3)),
where(14)∂ẋ2∂x2=(2x2(3+cosx3)+x4)sinx3a,∂ẋ2∂x3=(6x2+x4)x2cos2x3-2x22a+usin2x3ml2a,∂ẋ2∂x4=x2sinx3a,∂ẋ4∂x2=-(1+cosx3)∂ẋ2∂x2-2x2sinx3,∂ẋ4∂x3=ẋ2sinx3-(1+cosx3)∂ẋ2∂x3-x22cosx3,∂ẋ4∂x4=-(1+cosx3)x2sinx3a,a=2-cos2x3.Since the MMRL is a linear controller, it cannot control the 2PUAM. Indeed, the rank of the controllability matrixUc=[BABA2BA3B] at any equilibrium position x0 becomes 2, which is not the full rank, 4. That is, let us denote an equilibrium point by using arbitrary values θ10 and θ20(15)x0=(θ100θ200)T
with no input u = 0. Then, from (14), (16)A=∂f∂x|u=0x=x0=(0100000000010000),B=∂f∂x|u=0x=x0=1a0ml2(010-(1+cosθ20)),
where a0=2-cos2θ20. In this case, the rank of the Uc is not equal to the full rank, 4, but is 2 as follows:(17)rank[Uc]=rank[BABA2BA3B]=2.


## 4.3. Possible Control Strategy

The rank of the controllability matrix can, however, be the full rank, 4, at some positionsxs in the phase space where angular velocity of the first joint is not exactly zero, θ̇1≠0, even if the θ̇1 is very close to zero, as shown in Figure 6.Figure 6
Rank of the controllability matrixUc at some positions xs(θ̇2=0,u=0).To verify the controllability, let us denote positionsxs in the phase space by using arbitrary values θ1s, θ2s, and θ̇2s:(18)xs=(θ1sθ̇1sθ2sθ̇2s)T,θ̇1s≠0.
Then, from (14),(19)As=∂f∂x|u=usx=xs=(01000∂ẋ2∂x2∂ẋ2∂x3∂ẋ2∂x400010∂ẋ4∂x2∂ẋ4∂x3∂ẋ4∂x4),Bs=∂f∂x|u=usx=xs=1asml2(010-(1+cosθ2s)),
where as=2-cos2θ2s. In this case, the rank of the controllability matrix is 4, the full rank:(20)rank[Ucs]=rank[BsAsBsAs2BsAs3Bs]=4.
Thus, it is confirmed that 2PUAM cannot be stabilized at any equilibrium point, but if θ̇1s≠0, the end effector of the 2PUAM can approach any geometrical point with any slow speed.


## 4.4. Discussions

The slow speed approach verified above might be an interesting result because, nevertheless human subjects can control the 2PUAM very well, it is often very hard even for human subjects to stop the 2PUAM completely in the manual control tasks [9, 10]. The linear controller can be responsible only for a small region in which the linear approximation can work well. Thus, by switching multiple linear models, there is a possibility to control the 2PUAM with the MMRL.Different from (16), we assumed the condition in (19) where ∂ẋi/∂xj≠0, i=2,4,andj=2,3,4 to make the rank of controllability matrix be the full rank. The condition implies a virtual constraint of the manipulator’s shape (joint movement) because, for example, ∂ẋ2/∂x3≠0 implies the virtual existence of relation between θ̈1 and θ2 that makes a constraint on the angular acceleration depended on the angle of the manipulator shape. Interestingly, if human subjects could feel and realize such virtual constraint on the 2PUAM control (joint movement), the control task can sometimes be achieved relatively easier [9].According to the f-MRI results, subjects may use the shape and position information of the 2PUAM. The virtual constraint discussed above can further be created in order to make the control easier. Unfortunately, the controllability analysis does not prove this hypothesis, but there is a possibility that in operating the 2PUAM, subjects use the shape information to switch the controllers [10] and create the virtual constraint to make the control easier [9]. In this sense, MMRL can be regarded as a linear model for controlling the 2PUAM. Then, if human’s superior learning ability to control the complex nonlinear system could be based on such linear control schemes, its implementation on a robot system might be easier than we expected.


## 5. Conclusions

In this paper, by using a controllability analysis, we have revised the previous f-MRI experimental results that reveal significant activation areas for the nonlinear control task compared to the linear one. Even the useful pieces of information for the linear task may be different from nonlinear ones, the analysis suggests some possibility to control the 2PUAM with a set of linear control models in a similar way by which human subjects can control it. In fact, to stop the 2PUAM at an equilibrium point completely seems very hard or almost impossible, but to approach there with any arbitrary slow speed seems an easier task. The additional information of shape and position of the 2PUAM can then be used for switching the linear controllers. Although the internal relation between the virtual constraint and the controllability of the nonlinear task is still unclear and should be clarified further from both computational and brain sciences point of views, the hypothesis proposed in this paper implies that it could be possible to design or realize robots with the human-level learning ability in an easier way.



---

*Source: 919306-2010-06-03.xml*