A Cognitive Robot Architecture based on Tactile

Abstract

1
1
Introduction
Robotic soccer is a new common task for artificial intelligence (AI) and robotics research [12, 2]. The robotic soccer provides a good testbed for evaluation of various theories and algorithms like ma- chine learning and agent architectures.  A number of technical break- throughs for AI and robotics are expected to be discovered through the research for accomplishing this task. Through robotic soccer issue, we focus on "perception" and "sit- uation and behavior” problem among RoboCup physical agent challenges [2].  So far, we have implemented some behaviors for playing soccer by combining four primitive processes (motor con- trol, camera control, vision, and behavior generation processes) [13]. Such behaviors were not sophisticated because they were fully imple- mented by human programmers. An actual cognitive agent should organize its own internal structure autonomously in order to adapt to its environment through the direct interaction between the agent's body and its environment. We need the basic architecture for such agents. So far, many researchers in robotics and AI have proposed methodologies for realizing the autonomous agent system. Brooks proposed the so-called "behavior-based" approach [6]. He advocated that the agents need to employ no representations at all [7]. He and researchers of behavior-based approach (ex. [9]) attempt to build up agents through networks of simple fully functional behaviors map- ping sensors to actuators, without explicitly representing the world model. However, all behavior modules and their subsumption hier- archy are explicitly designed by the programmer, therefore it seems difficult to cope with the dynamic changes of the environment with- out the capability of learning. Aloimonos and et.al [1, 14, 8] proposed a so-called "purposive ac- tive vision paradigm."  In this paradigm, it is considered to be desir- able that the vision system can be greatly simplified by computing only the information needed by the robot to perform its immediate tasks. Moreover, purposive vision does not consider vision in isola- tion but as a part of complex system that interacts with world in specific ways [1]. Although tight coupling between visual informa- tion and motor commands is regarded as an important issue, there have been little achievement to solve this issue and only a few con-
2
ceptual proposals [3, 4].  In most cases, the relationship between the visual information and actions of the agent are designed by the programmer. Therefore, such agents can not organize their own internal representation. To cope with these issues in building a vision-based cognitive agent, we should resolve the following two key issues; (1) how to discover important visual features for accomplishing a given task and (2) how to organize its internal representation through the di- rect interaction between the agent and its environment. A philosophical theory offers an important suggestion for realizing such cognitive agent.  Berkeley [5], an English philosopher, suggested (1)that the object of sight have nothing in common with the object of touch, and (2)that the connection of sight and touch is arbitrary and learned by only experience not by calculation. This concept is supported by many case studies for behavior patterns and visual re- habilitation after successful operations for congenital blind patients. According to this concept, if the cognitive agent can acquire tightly coupling between modalities for visual, tactile and spatial recogni- tion, the agent would recognize the object by associating information from even one of these modalities with the coupling. Furthermore, based on the result of such object recognition, the agent could take appropriate behavior for accomplishing the given tasks. In this paper, we propose an architecture for the cognitive robot in which tactile information has primary role to organize internal repre- sentation to adapt its body and environment. Extending Berkeley's theory [5], we insist such cognitive agents should think recognizing objects based on visual information as taking appropriate behaviors with respect to such objects through direct interaction between the agent's body and environment. To support our methodology, we develop a cognitive robot system with visual and tactile sensors and a method for constructing a state space for vision-based reinforce- ment learning agents including a mechanism for selection of visual features based on the tactile sensors. This method would resolve two key issues (1) and (2) described above. There may be a situation that the visual information is used to characterize the tactile information. However we think that such situation is possible when the relation between the visual informa-
し
tion and the tactile information is already acquired. So we employ an assumption that the visual information does not make sense by itself and that only the tactile information gives physical meanings of environment to the visual information. This paper is organized as follows:  The next section gives an overview of the architecture for cognitive robot based on our method- ology. In the section 3, we explain our methods for selecting visual features based on discounted tactile stimulus and for constructing a state space of the cognitive agent. In the section 4, we show some experimental results. We summarize this paper in the section 5.
2
Cognitive Robot Architecture
Figure 1 shows the cognitive architecture of our robot.  Based on Berkeley's theory we propose an architecture in which visual infor- mation is characterized by tactile information.  In his theory, it is ar- gued that for recognition of an object it is important to discriminate the physical property of the object and tactile senor is only sensor for measuring physical contact with the object. Furthermore, he argued that it is the tactile sensor that can give physical meanings to the object recognized by the visual sensor. In this sense, only the tactile information can characterize the visual information. The architec- ture can realize a visual object recognition without prior knowledge. The key issue of visual object recognition is to acquire the relation between physical object and visual information. In other word it is to know how to be represented a physical object in images.  While an image contains various features such as color, edge, area of colored region, most of these features are irrelevant for object recognition to achieve the given task. In the architecture tactile information plays a key role in selection of visual features and discritization of selected features. The robot receives a tactile stimulus from an object at the end of an action series. The tactile stimulus is employed to represent a distance to the object by propagating a discounted one to actions in the action series. During the action series an image sequence is also observed, and we can see a change of a value of each feature in it. A discounted tactile stimulus is used to find relevant visual
4
features to the action series from the image sequence. If a value of certain feature changes as the robot moves to a object, the feature is relevant to the action series, moreover, it is significant feature to represent the object. Discritization of feature is carried out by calculating distribution of discounted tactile stimulus values. A state space of representing an action series to an object consists of discritized features. By utilizing this state space to reinforcement learning algorithm, an appropriate behavior to an object is acquired.
Our Method
3
In this section we describe a method of finding features which repre- sent the tactile stimulus from a image and a method of discritization of selected feature.
3.1
Feature Selection
Features that we should select are not just for object representation when tactile stimulus occurs but also for representation of distance
5
to objects when tactile stimulus does not occur.  The key issue is how to extract appropriate visual features to represent both a physical object and a distance to the object. The primary relation between features and the objects should be obtained at the moment that the robot touches the physical objects. Without looking at a moment when the body touches to the physical object and furthermore with- out knowing which feature would represent the body of the robot in images, it is hard to detect features from an image captured at a moment when the robot senses a tactile stimulus. A key idea to ex- tract features is that a change of distance is represented as a change of value of certain feature. Consequently, we use correlation coef- ficient between values of features and action series in order to find appropriate features. F is the given feature set { f 1 , f 2 , · · · , f n } . The
robot can select an action a i ∈ A = { a 1 , a 2 , · · · , a m } in time step t . At the end of an action series { a t , · · · , α 1 , α0( end )} , the robot should receive a tactile stimulus s 0 like a reward in reinforcement learn- ing algorithm. Then the discounted tactile stimulus { st, ･ ･ ･ , s1, s0} are calculated for each time step, where st = 0.9 t . We represent the value of F in each time step as { f(n,t), . . , f(n,1), f(n,0)}.  The correlation coefficient rate r is;
We select features which r value exceeds the r − threshold as sig- nificant features to represent physical objects.
3.2
Discritization Feature
In previous subsection, we showed the method of feature selection. But it is not enough for object perception because we need measure- ment for each selected feature to use this feature for decision making for the agent. The rest problem is how to divide the value of the selected feature into a small number of intervals. In order to solve this problem we employ ChiMerge algorithm that uses the χ 2 statis- tic to discretize numeric attributes proposed by Kerber[11]. The ChiMerge algorithm consists of an initialization step and a bottom- up merging process, where intervals are continuously merged until
6
a termination condition is met. ChiMerge is first initialized by sort- ing the training examples according to their values for the attribute being discretized and the constructing the initial discretization, in which each example is put into its own interval, i.e., place an in- terval boundary before and after each example. The interval merg- ing process contains two steps. Which are repeated continuously: (1) compute the χ 2 value for each pair of adjacent intervals, (2) merge(combine) the pair of adjacent intervals with lowest χ 2 value. Merging continues until all pairs of intervals have χ 2 values exceed- ing the parameter χ 2 — threshold ; that is, all adjacent intervals are considered significantly different by the χ 2 independence test. The value for χ 2 – threshold is determined by selecting a desired significance level and then using a table or formula to obtain the corresponding χ 2 value (obtaining the χ 2 value also requires spec- ifying the number of degrees of freedom , which will be 1 less than the number of classes). The formula for computing the χ 2 value is:
Where: m = 2 (the 2 intervals being compared) k = number of classes A ij = number of examples in ith interval, j th class R i = number of examples in ith interval = ∑ k j=1 Aij C j = number of examples in j th class = ∑ i=1 A ij N = total number of examples = ∑ k –1 E ij = expected frequency of A ij = R i ×O
Experimental Result
4
Based on methods described above we have an experiment on a mobile robot. The robot is our RoboCup robot [13] and the envi- ronment is the RoboCup field.  A main feature of the field is that objects in the environment are colored according to physical prop- erties respectively. Table1 shows the relation among object name, colors and physical properties except two goals. 7
Figure2 shows an overall view of our simulator and an image of scene which the robot in the field is looking at. The robot can take an action from a1 to a5, i.e. a1(0,1), a2(0.5,1), a3(1,1), a4(1,0.5), and a5(1,0) (See Figure 3). A distribution and arrangement of tactile sensors is important so that a robot can sense whole interaction between the physical world and its body during accomplishment of certain task.  Accordingly tactile sensors must exist everywhere in the body where a contact
8

Table 2: The variance of F
Table 3: The correlation coefficient rate with physical world occurs.  A qualification of a distribution and arrangement of tactile sensors relates following 3 points.  (1) The robot can move in only 2D plane. (2) A wall of the filed is a vertical plane. (3) The equator of the ball is the height of h. (1) and (2) implies that tactile sensors should be distributed horizontally and (3) implies the height of the position of tactile sensors should be h . Consequently we put 16 tactile sensors in front of our robot.
4.1
Feature selection
We prepare 9 features for captured images. We use two attributes in image, i.e., colors and the center of gravity for each colored area. We define F as f 1 =(red x), f 2 =(red y), f 3 =(red a), f 4 =(green x), f 5 =(green y), f 6 =(green a), f 7 =(white x), f 8 =(white y), f9 =(white a). After executing 58,223 random actions, 459 action series which terminated by touching ball were observed. For each action series the discounted tactile stimulus { s t , . . , s 1 , s 0 } are calculated, where s t = 0.9 t , and the value of s t is divided into 10 classes; class1=[0 → 1), class2=[1 → 2) … class10=[9 → 10]. Table 2 shows the variance of each F. Table 3 shows the correlation coefficient rate between the value of discounted tactile stimulus and the value of feature. We select a feature f 3 to represent the ball because the correlation coefficient rate is high. The correlation coefficient rate of f2 is also high, although the variance is only 2.9 (See Table 3). This means
9
4.2
Discretization of Feature
Figure 4 shows the histogram for feature 3. The discretization in Table4 shows the final result produced by ChiMerge at the 0.95 significance level (χ 2 = 16.92). The arrows in figure 4 represent the boundary of each interval.
4.3
Behavior Learning
In order to test validity of constructed state space, the learning of behavior of reaching ball is achieved by Q-learning. Q-learning [15] is based on estimating the value of Q (s,a), which is the expected future discounted reward for taking an action a in the input state s
and continuing with the optimal policy. The discounted reward is the sum of all future reward weighted by how close they are. The value of Q(s,a) is renewed.
Where t is the present time, γ is a discount factor (0 < γ < 1), and r t is the reward received at time t . Figure 5 shows an action series of reaching ball.
5
Conclusion and Discussion
Experimental result shows that the robot can acquire the relation between color and physical property of objects. By means of rein- forcement learning an appropriate behavior to physical property is acquired and the robot can behave by visual input. The importance
11
of the result is that the robot can acquire such actions without any predefined information on visual features such as colors. The robot could learn what it should focus on in visual input by its experiences of moving and touching. We showed a new architecture for cogni- tive robots that can organize its internal representation by physical experiences. It need less predefined information on environments so that it is applicable to flexible and dynamic environments. If we think the tactile stimulus corresponds to rewards given by a teacher, there was the study [10] similar to our method. They uses principle component analysis to select visual features and segment state space based on the distribution of rewards in a incremental manner. Our method also analyzed the distribution of rewards for segmenting the state space. Only the difference is how to process input data. That is, their method is incremental and our method is one-shot processing. We are not sure which manner is appropriate for cognitive robots. In this point, we should do further investiga- tion. [1] Y. Aloimonos. "Reply: What i have learned". CVGIP: Image Understanding, 60:1:74–85, 1994. [2] M. Asada, Y. Kuniyoshi, A. Drogoul, and et.al. "The RoboCup Physical Agent Challenge:Phase I(Draft)". In Proc. of The First International Workshop on RoboCup , pages 51–56, 1997. [3] R. Bajczy. "Active perception". Proc. of IEEE, 76:8:996–1005, 1988. [4] Dana H. Ballard. “Animate vision”.  Artificial Intelligence, 48:1:57–86, 1991. [5] George Berkeley. Essay Towards a New Theory of Vision. Ev- eryman's Library, No. 483. Dent., 1709. [6] R. A. Brooks. “A robust layered control system for a mobile robot".  IEEE J. Robotics and Automation, RA-2:14–23, 1986. [7] R. A. Brooks. "Intelligence without representation". Artificail Intelligence, 47:139–160, 1991. [8] S. Edelman.  "Reply: Representatin without reconstruction". CVGIP: Image Understanding, 60:1:92–94, 1994. [9] Ian Horswill. PhD Specialization of perceptual processes". thesis, Masachusetts Institute of Technology, Cambridge, May 1993. [10] H. Ishiguro, R. Sato, and T. Ishida. Robot oriented state space construction. In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems 1996 (IROS96) , pages 1496– 1501, 1996. [11] R. Kerver. Chimerge: Discretization of numeric attributes. In AAAI-92, Proceedings Ninth National Conference on Artificial Intelligence, pages 123–128, 1992. [12] H. Kitano, M. Tambe, Peter Stone, and et.al. "The RoboCup Synthetic Agent Challenge 97". In Proc. of The First Interna- tional Workshop on RoboCup , pages 45–50, 1997. [13] T. Nakamura, K. Terada, and et al. "Development of a Cheap On-board Vision Mobile Robot for Robotic Soccer Research". In Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems 1998 (IROS '98), pages 431–436, 1998. [14] G. Sandini and E. Grosso. "Reply: Why purposive vision". CVGIP: Image Understanding, 60:1:109–112, 1994. [15] C.J.C.H. Watkins and P. Dayan. Technical note: Q-learning. Machin Learning, 8:39–46, 1992.