ABSTRACT

1. INTRODUCTION
One of the features of virtual reality technology is a di- rect manipulation of an object using fingers. Touching an object increases the perceived information by the user. Displaying the physical property of virtual objects in addition to color and shape is expected to increase the presence of the virtual world. Num- bers of haptic devices have been developed in order to display surface roughness, softness and other material properties. Two groups of physiological sensations are involved in the wide meaning of haptic sense. The first is the tactile sense where the receptors are located under the dermis. Tactile sense detects the information from the skin surface such as contact pres- sure or vibration. Another is the proprioceptive sense where the receptors exist in the muscle or in the tendon. Proprioceptive sense provides the inner information of the body, such as joint angle or muscle contractile force. Various types of display devices to stimulate tactile re- ceptors have been developed, such as OPTACON for blinds, pin array driven by shape-memory-alloy [1], vibrating pin array to display textures [2], tapered membrane for display of elastic wave as texture [3], a device that stimulates the each type of receptors selectively [4]. The devices that stimulate proprioceptive recep- tors have been developed as "force display devices". Force dis- play devices such as PHANToM [5] are  getting started being used for design of 3D object. A glove shaped force display de- vice to display the reaction force to each fingertip is also com- mercially available as CyberGrasp [6]. The display of the softness is an important matter to dis- play the feel of a material. The display of the softness of a de- formable virtual object has been investigated by using force dis- play systems. In the studies that display the softness in direct manipulation using force feedback systems, the softness was dis- played to the fingertip as the displacement of the object surface, in order to represent the object deformation caused by the ap- plied force [7]. From the viewpoint of the human perception, the dis- placement is perceived through the proprioceptive sensation, such as muscle spindle. On the other hand, the softness of the real object is perceived from the tactile sensation, as the dynamic change of the contact area at the fingertip. For instance, the con- tact area while touching a soft object is larger than the contact area while touching a hard object, even if the same forces were applied to these objects. Srinivasan revealed that the perception of the tactile information is more important in the softness cogni- tion than the proprioceptive information that is displayed by force feedback device [8]. The authors have also performed the ex- periment to quantify the role of the proprioceptive and tactile sensation in the softness cognition. The greater role of the tactile information to perceive the dynamic change of the contact area as the deformation information than the proprioceptive informa- tion was experimentally demonstrated [9]. It was hypothesized based on the previous studies that the softness will be better displayed by dynamically controlling the contact area, when the human touches the softness display device using their finger. If this hypothesis is proved, it becomes possible to display the softness in "passive touch" action that is impossible using force display devices. This study was performed to confirm the hypothesis through the preliminarily system de- velopment and the psychophysiological experiments. When human touches a deformable object by using his fingertip, the touched object deforms and the fingertip sinks into the object as shown in figure 1(a). That increases the contact area between the fingertip and the object. In this case, two kinds of information are involved in the perception about the object deformation. The first is the displacement of the fingertip that is perceive by the muscle spindle, that is one of the proprioceptive receptors. Another is the information about the contact area in- crease that is perceived as the increase of the number of the fir- ing tactile receptors. However, the conventional force display de- vices only represent the displacement as shown in figure 1(b) and have no ability to display the contact area increase between the object and the fingertip. It had been revealed experimentally that the information perceived by the tactile receptor is more important in softness perception [8,9]. Therefore, the develop- ment of the device to display the deformation through tactile sen- sation has been desired for more realistic softness representa- tion. On the other hand, an experimental law had been ob- served that the peak contact area is independent from the object softness in human pinch when human pinches an object with unknown softness [10]. It appears that human perceives the ob- ject softness from the pinch force by pinching an object until the contact area reaches the target level. This experimental law al- lows an assumption that the softness can be displayed by detect- ing the contact force and controlling the contact area increasing rate. Therefore, a device that displays the contact area increase as shown in figure 1(c) was developed. The sinking displace- ment was intentionally eliminated to prove the possibility of the softness display by dynamic contact area control. Figure 1  (a)Object deformation and figertipn contact area in- crease in touch of real object. (b)Display of softness with force display device (The increase of the finger- tip contact area is generated by only the fingertip de- formation). (c)Display of softness by the proposed dy- namic contact area control method.
3.1 Contact area control system
The structure of the softness display system is shown in figure 2. The system consists of a  softness display device includ- ing a loadcell for contact force detection, a servo controlled pump for fluid volume control, a DC servo controller and a contact area control software that is programmed on a PC. There are potentially several mechanisms that could con- trol the contact area between fingertip and a device surface such as pin-array system. However, the present tactile display devices are not designed for displaying the dynamic change of the con- tact area and they do not have enough spacial resolution. In this study, a fluid-driven vertically moving cylinder that has rubber sheet at its top surface was utilized, because of the simplicity of development and the spacial resolution as shown in figure 3. The piston of the softness display device was installed on a loadcell for contact force detection. The inside of the piston was designed as empty, and fluid was pumped into the piston through the pipe at the side wall of the piston. The pumped fluid flaws out from twelve holes at the top of the piston, and the fluid push-up the rubber-top cylinder. Because the center of the rubber is pushed by the fingertip, the peripheral part is mainly pushed up. There- fore, the contact area between the fingertip and the rubber in- creases. The pressure distribution within the contact area be- comes constant because of the intervention of the fluid. The soft- ness was represented as the increase rate of the contact area. The point is that the displacement of the cylinder is opposite to the virtual material deformation. That means the developed system displays the object deformation through the tactile information only, not through the proprioceptive information. The fluid volume control pump consists of a motor-driven piston, a cylinder and a potentiometer to detect the piston posi- tion. The fluid volume in the softness display device is indirectly measured and controlled by controlling the piston position of the pump. A DC servo control circuit was utilized for the pump con- trol. In order to attain the dynamic control of the contact area, a "numerical material model" must be constructed and stored on the PC in advance. The numerical material model is a property between the contact force and the contact area while a user touches the material by his fingertip. By using this model the required contact area can be calculated from the detected contact force. The required fluid volume was calculated from the re- quired contact are. That was attained by using a "numerical de- vice model", which is the measured property among the fluid volume, the contact force and the contact area of the softness display part in advance. The required fluid volume was sent as the command to the servo pump controller through D/A converter. The control program was written as C program and executed on a PC with Pentium III 450MHz at 0.09ms sampling interval.
3.2 NUMERICAL MATERIAL MODEL
Four kinds of materials, a gelatin, two kinds of silicon rubber and acrylic polymer disk plates, were modeled for the evaluation of the device. The cylindrical shape material models as shown in figure 4 were utilized. The diameter of the model was forty-five millimeters and the thickness was twelve milli- meters. The property between the contact force and the contact area was measured by touching the material installed on the loadcell by the fingertip while the surface of the material is cov- ered with a polyvinyl sheet and the fingertip is colored with paint. The contact area was measured by counting the pixels of the binary converted image from the scanned contact area image. The contact area is shown in figure 5 while the contact force was varied between one to six newtons. The numerical material models are the logarithmic approximation of the measured existing ma- terial models, as shown in the figure 5.
3.3 NUMERICAL DEVICE MODEL
The numerical device model was also constructed by the similar experimental method to the experiment for the numeri- cal material model. The measured property is shown as figure 6. The property was changed by controlling the fluid volume be- tween zero to one point five milliliters. By using the developed softness display system, any softness property between these lines can be represented by the interpolation. The hard object can be displayed by exhausting the fluid in the rubber-top cylinder. The contact area increase observed in the bottom line of figure 6 is because of the fingertip deformation. The most soft object that can be displayed is limited by the rubber property. The gelatine was the most soft object with the developed prototype device. Figure 7 is the relationship between the required contact and the actual contact area measured by the similar method to the previous experiments for constructing the numerical models. The all four kinds of material models were represented in this experiment. The average actual contact area was 3.6 percent greater than the required contact area. It was confirmed that the developed softness display system can re-display the softness of the soft materials such as gelatin as well as the hard material such as acrylic acid resin.
4. EVALUATION METHOD
Four series of psychophysiological experiments were per- formed with ten normal volunteers. The relative and absolute softness cognition tasks were performed. The cognitive rate of the four level softness was measured. The softness cognitive rate in passive touch was also examined as well as the cognitive rate in active touch, in order to demonstrate the performance to dis- play the softness in passive touch that the force feedback devices do not have in principle.
4.1 Relative cognitive rate of softness in active touch
In the relative cognition of the softness, paired compari- son method was utilized. The subject was required to touch the two softness using their index finger, and answer the softer one. The two softness were displayed sequentially using the devel- oped device. The pair of the softness was randomized. Totally 18 trials were performed per subject, that means three times repeti- tion of the same pair. The visual and the auditory information were masked during the experiment to avoid the influence of the device action and the mechanical noise on the cognitive rate. The subject sat on a chair and the subject's hand was put on a desk to limit the hand motion. The only one time of the retrial was allowed for the subject. The cognitive rate of the original materials was also ex- amined in the same subjects. The experimental procedure was the same as the experiment with the device but the softness was displayed by using the actual object.
4.2 Absolute cognitive rate of softness in active touch
The absolute softness cognitive rate was also measured in the same subjects as the subjects in the relative cognition. The subjects were required to touch the displayed virtual softness at first, and then required to touch the original materials and to answer the material with the same softness. The four softness were displayed three times each in random sequence, then the total trial was 12 per subject. The experiment was performed in blind condition. The experiments to examine the absolute softness cogni- tion of the real materials were also carried out in the same sub- jects.
4.3 Cognitive rate of softness in passive touch
The relative and absolute softness cognitive rate was also measured in the same experimental procedure as the experiment in active touch, except the touch motion. The softness display device was installed on a screw-driven stage. The device was pushed up to touch the index finger of the subject that is fixed by a frame.  The touch force was monitored and the peak contact force was controlled constant.
5. RESULTS OF PSYCHOPHYSICAL EXPERIMENT
The relative softness cognitive rate in active touch is shown in figure 8. The average cognitive rate was 98 percent while the cognitive rate of the original material was 100 percent. It was demonstrated experimentally that the "difference of two softness" is possible by controlling the fingertip contact area even the displacement is not displayed. The absolute cognitive rate of the virtual softness displayed by the developed system is shown in figure 9. The average cognitive rate was 85 percent with the developed device and 98 percent with the original material. The cognitive rate was higher in softer materials with the device. And, in the 70 percent cases of the incorrect answers, the subject chose the softer materials than the displayed softness. It appears this mistake was because of the error of the displayed contact area that is 4% larger than the required area as already shown in figure 7. The average rate is higher more than three times of the chance level, because the task was the choice among four mate- rials that means the chance level is 25 percent. It was concluded that the "softness of an object" can be displayed by the dynamic control of the fingertip contact area. The relative softness cognitive rate in the passive touch is shown in figure 10. The average cognitive rate was 94 percent with the device and  99 percent with the original objects. It can be confirmed obviously that the softness was perceived without the displacement because of the object deformation. That indi- rectly proves the relatively higher functional importance of the tactile sense in softness perception [9,10]. The absolute cogni- tive rate was 75 percent with the device and  97 percent with the original object.  The cognitive rate is noticeably higher than the chance level again. The softness display performance of the dy- namic contact area control in passive touch was demonstrated.
6. CONCLUSIONS
A new softness display method based on the dynamic control of the fingertip contact area was proposed. The effective- ness of the contact area control in softness display was experi- mentally demonstrated through the psychophysical experiments using the developed system. The softness display in passive touch was attained that is impossible with force feedback device in principle. For more detailed representation of deformable mate- rials, development of contact area control device without using fluid and integration of the system with a force feedback device is required.
REFERENCES
[1]R.D.Howe, D.A.Kontarinis, W.J.Peine: Shape Memory Alloy Actuator Controller Design for Tactile Displays, in Proc. 34th IEEE Conference on Decision and Control, pp.13-15(1995) [2] Y.Ikei, K.Wakamatsu, S.Fukuda: Vibratory tactile display of Relative softness cognitive rate (rate of correct answer in paired comparison task) in active touch. image-based textures, IEEE CG&A, Vol.17,No.6, pp.53- 61(1997) [3]T.Nara, Y.Yanagida, T.Maeda, S.Tachi: A tactile display us- ing elastic waves in a tapered membrane, Trans. Virtual Re- ality Soc. Japan, Vol.4, No.2, pp.467-474(1999)(in Japanese) [4]N.Asamura, N.Yokoyama, H.Shinoda: Selectively Stimulat- ing Skin Receptors for Tactile Display, IEEE Computer Graph- ics and Applications, Vol. 18, No. 6, pp.32-37(1998). [5]http://www.sensable.com/ [6]http://www.virtex.com/ [7] K.Fujita: Development and evaluation of force feedback glove for hardness display of virtual object, Trans. IEICE,Vol.J81- D-II, No.10, pp.2394-2400(1997)(in Japanese) [8]M.A.Srinivasan, R.H.Lamotte: Tactual Discrimination of Soft- ness, Journal of Neurophysiology, Vol.73, No.1, pp.88- 101(1995) [9]H.Sasaki, K.Fujita: Experimental analysis of role of visual information in hardness cognition displayed by a force dis- play system and effect of altered visual information, Trans. Virtual Reality Soc. Japan, Vol.5, No.1, pp.795-802(2000)(in Japanese) [10]K.Fujita, Y.Oyama: Control strategies in human pinch mo- tion to detect the hardness of an object, in Proc. IEEE Int. Conf. on Systems, Man and Cybernetics,  pp.II-682-686 (1999) Figure10 Relative softness cognitive rate (rate of correct answer in paired comparison task) in passive touch.