Patent Publication Number: US-11654353-B2

Title: Control apparatus and control program

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This is a continuation application of U.S. patent application Ser. No. 16/942,092, accorded a filing date of Jul. 29, 2020, which is a continuation application of U.S. patent application Ser. No. 16/075,265, accorded a filing date of Aug. 3, 2018 (U.S. Pat. No. 10,850,192, issued Dec. 1, 2020); which is a national stage application of PCT/JP2017/004763, filed Feb. 9, 2017; which claims priority to JP2016-042403, filed Mar. 4, 2016, the entire disclosures of which are hereby incorporated by reference. 
    
    
     TECHNICAL FIELD 
     The present invention relates to a control apparatus and a control program. 
     BACKGROUND ART 
     A controller device for use in home game machines and so on detects operations of depressing a button by a user and shaking the controller device and the like and sends the information indicative of these detected user operations to the main body of the game machine or the like. 
     Some controller devices are known in which an acceleration sensor and so on are built so as to detect the timing and strength of the shaking of the controller device by a user and output the detected information. Also known is a controller device designed to be worn on the hand of a user like a glove, thereby detecting finger movements. 
     SUMMARY 
     Technical Problems 
     However, the related-art controller devices are not gripped in each of the hands of a user, making it difficult to present the sense of force for such a movement of each finger of the left and right hands of the user as a gripping operation. Further, although a controller device shaped like a glove can present the sense of force, wearing the glove causes the sense of unnaturalness, thereby possibly deteriorating the sense of immersion in game playing or the like. 
     Therefore, it is an object of the present invention to solve the above-mentioned problems by providing a control apparatus and a control program that are configured to control a controller device capable of presenting the sense of force for the movement of each finger of the left and right hands of a user while mitigating the sense of unnaturalness caused by wearing the controller device. 
     Solution to Problems 
     The present invention for solving problems of the above-mentioned relate-art technologies is a control apparatus for a device with one of left and right sides of the device fixed as pressed against a palm of a user. The device includes a button that is depressible within a predetermined movable range and a sense-of-force presentation block that presents a sense of force against depressing of the button within a range in which any of fingers of the user is reachable. The control apparatus includes means of receiving an input of a sense-of-force presentation start position in the predetermined movable range, means of acquiring a depress amount of the button, and means of instructing the sense-of-force presentation block to present a sense of force against depressing of the button when the depress amount reaches the inputted sense-of-force presentation start position. 
     Advantageous Effect of Invention 
     According to the present invention, the sense of force can be presented for the movement of each finger of the left and right hands of a user while mitigating the sense of unnaturalness caused by wearing of the controller device. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG.  1    is a perspective view illustrating, from a front side, a device controlled by a control apparatus practiced as one embodiment of the present invention. 
         FIG.  2    is a perspective view illustrating, from a rear side, the device controlled by the control apparatus practiced as one embodiment of the present invention. 
         FIG.  3    is a block diagram illustrating an example of a configuration of the control apparatus practiced as one embodiment of the present invention. 
         FIG.  4    is a block diagram illustrating another example of the control apparatus practiced as one embodiment of the present invention. 
         FIG.  5    is a diagram for describing a setting example of the presentation of the sense of force that is executed for the control apparatus practiced as one embodiment of the present invention. 
         FIG.  6    is a diagram for describing an example of a sense-of-force amount table that is held in the control apparatus practiced as one embodiment of the present invention. 
     
    
    
     DESCRIPTION OF EMBODIMENT 
     The following describes an embodiment of the present invention with reference to drawings. It should be noted that, in the following description, the sizes of components, the ratios between these sizes, and the arrangement of these components are illustrative only; therefore, the examples of the present embodiment are not limited to the sizes, ratios, and arrangement illustrated and otherwise indicated in the drawings. 
     A device  20  to be controlled by a control apparatus  10  related with one example of the embodiment of the present invention is a controller device that is illustrated in  FIG.  1    and  FIG.  2   , for example.  FIG.  1    is a perspective view illustrating the device  20  seen from the front side and  FIG.  2    is a perspective view illustrating the device  20  seen from the rear side. In one example of the present embodiment, the device  20  is configured by a device main body  210  and a fixture  220 . The fixture  220  is one of accessories. 
     The device main body  210  includes a holding block  211  that is held by a user and a manipulation block  212 . The holding block  211  has a left-side surface  221  and a right-side surface  222  and is substantially formed into a polygonal column in one example of the present embodiment. The manipulation block  212  is formed continuously from the holding block  211  and, in the examples illustrated in  FIGS.  1  and  2   , includes a thumb sensor block  231  and a button manipulation block  232  on the front side and a first finger sensor  233 , a depress button  234 , and a display block  235  on the rear side. In addition, on the rear side of the holding block  211 , a second finger sensor  241  and a third finger sensor  242  are arranged. 
     On one of the left-side surface  221  and the right-side surface  222  of the device main body  210 , the fixture  220  that is an accessory is fixed. Further, on the other side of the left-side surface  221  or the right-side surface  222 , a battery  240 , another accessory, may be fixed. In the example of the present embodiment, the fixture  220  may be made by forming a flexible belt into a ring. Also, the fixture  220  may be made by forming stop means such as a buckle into a ring, the diameter of which is adjustable. 
     The device  20  is used as fixed at a position where the device main body  210  abuts against the base of the thumb of the user with the other fingers of the user, the index finger to the little finger, being passing through the fixture  220  such as a ring-formed belt. In the examples of  FIGS.  1  and  2   , at this moment, the device  20  is fixed with the right-side surface  222  of the device  20  pressed against the palm of the user. Further, the device  20  is approximately as large as that, when the user naturally hold the device main body  210 , the distal end of the thumb of the user reaches the button manipulation block  232  on the front side of the device  20  and the proximal side of the thumb of the user comes close to the thumb sensor block  231 . Still further, the index finger as one example of any one of the fingers of the user shall be positioned so as to touch the depress button  234  on the rear side of the device  20 , thereby making the user grip the holding block  211  with the palm, the middle finger, the ring finger, and the little finger. Obviously, if the user opens his or her hand from this state, the device  20  is fixed to the hand of the user with the fixture  220 , so that the user is able to close or open his or her fingers without feeling the resistance to the fingers. 
     When the user naturally holds the device main body  210 , the thumb sensor block  231  is arranged at a position that the proximal side of the thumb of the user comes close, this position being somewhat below a position reached by the distal side of the thumb of the user, for example. The thumb sensor block  231  detects the thumb of the user within a detectable range that is a comparatively wide angle range of the left-side surface side of the front of the device  20  to the right-side surface side around the normal-line direction of the surface of the device  20  that is the front side of the device  20 . Then, the position (the angle within the above-mentioned angle range) at which the thumb concerned is detected and a distance from the thumb sensor block  231  to the detected thumb of the user are detected so as to be output to the control apparatus  10  as detection result information. This detection result information includes the information related to the position at which the thumb was detected and the information related to the distance up to the thumb. The thumb sensor block  231  may be any device that is capable of detecting the position of the thumb and the distance up to the thumb, such as a camera, a pyroelectric sensor, a capacitance sensor or the like, for example. The button manipulation block  232  includes at least one button and, when depressed by the user, outputs the information for identifying the depressed button to the control apparatus  10 . 
     The depress button  234  is a button having a button knob that is movable between a first position projecting to the rear side of the device  20  and a second position depressed into the side of the device  20 . In  FIGS.  1  and  2   , an example in which the depress button  234  is a rocking switch (a rocking button) is illustrated. The depress button  234  is depressingly operated with any one of the fingers, such as the index finger, of the user, thereby outputting the information indicative of the depress amount (the travel amount of the button knob) of this operation to the control apparatus  10 . To be more specific, the depress button  234  can be made up of a potentiometer or the like; however, as long as the depress amount can be detected, any other configurations are usable. 
     The display block  235  has a light emitting diode (LED) that can emit light in multiple colors, for example. In accordance with a command entered from the control apparatus  10 , the display block  235  makes the LED emit light in an indicated light emitting mode (a mode in which the LED is turned in an indicator color or blinked with indicated intervals, for example). 
     Further, in the present embodiment, the first finger sensor  233  is amounted on the depress button  234 . The first finger sensor  233  detects whether or not the depress button  234  is touched by the index finger and outputs the information indicative of the result of this detection to the control apparatus  10 . The first finger sensor  233  can be made up of a capacitance sensor or the like, for example. 
     Still further, in one example of the present embodiment, the depress button  234  may have a sense-of-force presentation apparatus. In accordance with a command (a magnitude of force to be presented F) entered from the control apparatus  10 , the depress button  234  having the sense-of-force presentation apparatus presents a force against a force with which the user depresses the depress button  234 . To be more specific, the depress button  234  having this sense-of-force presentation apparatus can be made up of a motor having a rotor that rotates when the user executes an operation of depressing the depress button  234 , a direct-acting actuator in which a driven body linearly travels when the depress button  234  is depressed, and so on. When such a motor and so on are used, if the rotation by depressing or the travel of a driven body takes place, the sense-of-force presentation apparatus of the depress button  234  controls the motor or the actuator so as to apply force F in the direction reverse to this rotation or the travel. Thus, for the sense-of-force presentation apparatus arranged on a depress button so as to present a sense-of-force upon reception of a command for identifying magnitude of force to be presented F, any widely known products can be employed, so that the description of further details is skipped here. 
     Yet further, in the present embodiment, the device  20  may vibrate the device main body  210  with a vibrator, a haptic device, a speaker, or the like to as to transmit the vibration to the palm of the user on the device  20  is fixed, thereby presenting the sense of touch. The detail description of a method of the presentation of the sense of touch is also skipped here. 
     The second finger sensor  241  detects whether or not the holding block  211  of the device  20  is in the proximity of (or touched by) the middle finger of the user. Then, the information indicative of the result of this detection is output to the control apparatus  10 . Likewise, the third finger sensor  242  detects whether or not the holding block  211  of the device  20  is in the proximity of (or touched by) the ring finger or the little finger of the user. Then, the information indicative of the result of this detection is output to the control apparatus  10 . These second finger sensor  241  and third finger sensor  242  can each be made up of a capacitance sensor, an optical sensor, or the like. Here, optical sensors include those each having a light-emitting block for radiating infrared light and a photoreception block for detecting infrared light, thereby measuring a ratio with which the light radiated from the light-emitting block is reflected from an object into the photoreception block, for example. According to such an optical sensor, as an object gets closer in the proximity of the optical sensor, ratio (specific ratio) r obtained by dividing the intensity of the incident light into the photoreception block by the intensity of the light radiated from the light-emitting block gets greater, so that the approach of fingers or the like can be detected. 
     In addition, in the present embodiment, the device  20  may have a sensor (a fixture sensor) for detecting to which of the left-side surface  221  and the right-side surface  222  the fixture  220  that is a predetermined accessory is fixed. This fixture sensor, to be more specific, can be realized by arranging, on the left-side surface  221  and the right-side surface  222  individually, a micro switch or the like that is turned on upon amounting of the fixture  220 . For a configuration in which, when a predetermined accessory such as above is mounted, the mounted side is detected, widely known method can be used, so that the detail description thereof is skipped. In the present embodiment, the result of the detection by this fixture sensor is inputted in the control apparatus  10 . 
     The control apparatus  10  is built in the device main body  210  of the device  20 , for example. As illustrated in  FIG.  3   , the control apparatus  10  is configured by a control block  11 , a storage block  12 , a device interface  13 , and a communication block  14 . The control block  11  is a program control device such as a central processing unit (CPU) or the like, operating as instructed by a program stored in the storage block  12 . In the present embodiment, the control block  11  detects on which side, the right side or the left side of the device  20 , the accessory of a predetermined type is mounted and, on the basis of the result of this detection, determines whether the device  20  is worn on the left hand or the right hand of the user. The specific contents of the control block  11  will be described later in detail. 
     The storage block  12  is a memory device or the like that holds a program to be executed by the control block  11 . This program may be computer-readable, provided as stored in a non-temporary storage medium, and copied to the storage block  12 . Further, the storage block  12  operates also as a work memory for the control block  11 . 
     The device interface  13  is connected to the thumb sensor block  231  of the device  20 , the button manipulation block  232 , the first finger sensor  233 , the depress button  234 , the second finger sensor  241 , the third finger sensor  242 , and an sense-of-force presentation apparatus and outputs the information output from the thumb sensor block  231 , the button manipulation block  232 , the first finger sensor  233 , the depress button  234 , the second finger sensor  241 , and the third finger sensor  242  to the control block  11 . In addition, the device interface  13  outputs a command inputted from the control block  11  to the sense-of-force presentation apparatus. 
     The communication block  14  is configured by a serial interface such as a universal serial bus (USB) or the like or a wireless communication interface such as Bluetooth (registered trademark) or the like. The communication block  14  is communicably connected to a device (hereafter referred to as a main body equipment) that operates on the basis of the movement or the like of user&#39;s hand or fingers detected by the device  20 , receives a command from this main body equipment, and outputs the received command to the control block  11 . Further, the information output from the control block  11  is sent to the main body equipment. 
     The following describes an operation of the control block  11  of the control apparatus  10 . In the present embodiment, the control block  11  is functionally configured by a sense-of-force command receiving block  41 , a command storage block  42 , a depress amount acquisition block  43 , and a sense-of-force presentation processing block  44  as illustrated in  FIG.  4   . 
     The sense-of-force command receiving block  41  receives the input of the setting related with the sense of force from the main body equipment. This setting is executed by indicating a sense-of-force presentation amount in accordance with a depress amount of the depress button  234 . To be more specific, from the main body equipment, depress amount (sense-of-force presentation start position) Ds for starting sense-of-force presentation is specified. The sense-of-force command receiving block  41  outputs the information of the sense-of-force presentation amount received here to the command storage block  42 . The command storage block  42  stores into the storage block  12  the sense-of-force presentation start position Ds that is the information of the sense-of-force presentation amount received by the sense-of-force command receiving block  41 . 
     The depress amount acquisition block  43  repetitively acquires information D of the depress amount of the depress button  234  from the depress button  234  every predetermined timing ( 1/1000 second, for example). 
     The sense-of-force presentation processing block  44  determines whether or not the information D of the depress amount acquired by the depress amount acquisition block  43  is in excess of the sense-of-force presentation start position Ds stored in the storage block  12  (whether or not D≥Ds). If the information D of the depress amount acquired by the depress amount acquisition block  43  is found to be in excess of the sense-of-force presentation start position Ds stored in the storage block  12  (if D≥Ds), then the sense-of-force presentation processing block  44  controls the sense-of-force presentation apparatus of the depress button  234  so as to present the sense of force against the depressing of the depress button  234  by the user. It should be noted that the magnitude of the sense of force to be presented is a force that is enough for pushing back by −ΔD when the depress amount is D+ΔD. 
     The control apparatus  10  according to the present embodiment has the configuration as described above and operates as follows. The user determines on which of the left and right hands the device  20  is to be worn. If the device  20  is to be worn on the right hand, then the user mounts the fixture  220  onto the right-side surface  222  of the device  20 . If the device  20  is to be worn on the left hand, then the user mounts the fixture  220  onto the left-side surface  221  of the device  20 . Also, at this moment, a battery  240  may be mounted on the side opposite to the side on which the fixture  220  is mounted. In this case, it is assumed that a terminal for carrying the power supply from the battery  240  to the internal circuit be arranged on the left-side surface  221  and the right-side surface  222  of the device  20  individually. 
     Next, the user pass the index finger, the middle finger, the ring finger, and the little finger of the left hand or the right hand through the fixture  220  mounted on the device  20  and wears the device  20  by abutting the device  20  to the base of the thumb. Then, the user holds the holding block  211  with the middle finger, the ring finger, and the little finger. 
     The control apparatus  10  receives the input of depress amount (sense-of-force presentation start position) Ds for starting sense-of-force presentation as the setting related with the sense of force from the side of the main body equipment. The control apparatus  10  stores the received depress amount (sense-of-force presentation start position) Ds for staring sense-of-force presentation into the storage block  12 . 
     Further, when the user depresses the depress button  234  with the index finger, the control apparatus  10  acquires repetitively acquires information D of this depress amount every 1/1000 second. 
     The control apparatus  10  determines whether or not the sense-of-force presentation start position Ds stored in the storage block  12  is exceeded (whether or not D≥Ds). While the information D of the depress amount acquired by the control apparatus  10  does not exceed the sense-of-force presentation start position Ds stored in the storage block  12  (while D&lt;Ds), the control apparatus  10  does not control the sense-of-force presentation apparatus. Therefore, the control apparatus  10  controls the sense-of-force presentation apparatus of the depress button  234  such that the sense of force against the depressing of the depress button  234  is not presented. Hence, the user is able to further depress the depress button  234  without resistance. 
     If, in the course of time, the information D of the depress amount acquired by the control apparatus  10  exceeds the sense-of-force presentation start position Ds stored in the storage block  12  (if D≥Ds), then the control apparatus  10  controls the sense-of-force presentation apparatus of the depress button  234  to present the sense-of-force against the depressing of the depress button  234 . This makes the user recognize as if the depressing beyond the sense-of-force presentation start position Ds is disabled. 
     Consequently, on the side of game software, for example, sense-of-force presentation start position Ds may be set in accordance with the size of a virtual object in a game, thereby making the user recognize the depressible amount (the depress amount of the depress button  234 ) according to the size of the virtual object. 
     According to one example of the present embodiment, also while the user is closing or opening fingers without feeling the resistance to the fingers, the finger movement and the depress amount of the depress button  234  can be detected and, at the same time, executing the sense-of-force presentation on the depress operation of the depress button  234  with any one of the fingers (the index finger in the example of the present embodiment) allows the user to feel the size and texture of a virtual object gripped in the user hand. 
     Other Examples of Sense-of-Force to be Presented 
     It should be noted that the setting related with the sense-of-force executed from the side of the main body equipment described so far is assumed to be the depress amount (sense-of-force presentation start position) Ds for starting sense-of-force presentation. That is, in the example described above, a force against the depress amount in excess of the depress amount Ds is presented so as to present a sense-of-force that may disable the depressing beyond this sense-of-force presentation start position. However, the embodiment of the present invention is not limited to this configuration. 
     For example, for the setting related with sense-of-force, the control apparatus  10  may receive the setting of the magnitude of force F against the depressing of the depress button  234  relative to the depress amount D of the depress button  234  within a predetermined movable range. In this example, in accordance with the depress amount, the sense-of-force presentation amount (force F necessary for depressing by ΔD) is set from the side of the main body equipment. The setting in this case may only indicate the sense-of-force presentation start position Ds and the variable ΔF of the magnitude of the maximum sense of force presented per subsequent predetermined unit depress amount ΔD. In this example, if the depress amount is D=Ds+kΔD (k&gt;0), the force F necessary for further depressing by ΔD becomes F=kΔF ( FIG.  5   ). 
     Here, the counter force ΔF that increases for every unit depress amount varies in a linear manner; however, this force may be set as varying in a logarithmic manner or an exponential manner. In addition, the maximum force Fmax to be presented may be set. If the maximum force Fmax is set and the force F=kΔF and so on to be computed exceeds Fmax, then the control apparatus  10  instructs the sense-of-force presentation apparatus of the depress button  234  to present the force Fmax with the force F to be presented being Fmax. 
     In this example, force F necessary for the user to depress the depress button  234  further from the depress amount D, relative to the depress amount D of the depress button  234 , by ΔD is set as illustrated in  FIG.  5   . Then, as illustrated in  FIG.  6   , a sense-of-force value table in accordance with the variation (the variation in force F relative to depress amount D) illustrated in  FIG.  5    is generated by the control apparatus  10  and the generated table is stored in the storage block  12 . 
     Subsequently, when the user depresses the depress button  234  with the index finger, the control apparatus  10  repetitively acquires information D of the depress amount every 1/1000 second, for example. 
     The control apparatus  10  references the sense-of-force value table stored in the storage block  12  so as to acquire force value F to be presented corresponding to the information D of the acquired depress amount. Then, the control apparatus  10  controls the sense-of-force presentation apparatus of the depress button  234  to present the sense-of-force of the force F against the depressing of the depress button  234 . 
     In this example, the counter force at force F is presented when the user depresses the depress button  234  by D with force F, thereby putting the user into a state where the user cannot further depress the button once. When the user depresses the button with a force greater than the force F, the depressing by ΔD is enabled (at this moment, unless force F′ stored in the sense-of-force value table in accordance with the information D+ΔD is further exceeded, counter force F′ disabling further depressing is presented). This allows the user to feel as if the user is gripping an object having a restorative force. 
     It should be noted that, on the side of the main body equipment of the home game machines and so on, the settings of sense-of-force presentation start position Ds described so far, variation amount ΔF in the magnitude of the maximum sense of force to be presented per predetermined unit depress amount ΔD, and so on may be values obtained as a result of the physical computation (the computation taking physical laws into consideration) done on a virtual object, for example. To be more specific, a parameter related with the elastic modulus of an object may be used so as to compute the restorative force for a depress amount by Hooke&#39;s law, thereby obtaining variation amount ΔF in the magnitude of the maximum sense of force to be presented per predetermined unit depress amount ΔD. 
     Example in which the Depress Button is Retracted in Advance 
     In the description so far, if sense-of-force presentation start position Ds is inputted, the control apparatus  10  may control the sense-of-force presentation apparatus of the depress button  234  so as to put the depress button  234  into a state where the depress button  234  is retracted up to the sense-of-force presentation start position Ds in advance. 
     In this example, unless the user does not move the index finger up to the sense-of-force presentation start position Ds in accordance with sense-of-force presentation start position Ds, the user cannot touch the depress button  234 . Hence, varying sense-of-force presentation start position Ds allows the user to recognize as if the user is touching a virtual object having a different size. 
     Example in which a Sense of Force is Presented to a Finger Other than the Index Finger 
     In the description of the present embodiment so far, the depress button  234  is manipulated with the index finger; however, the present embodiment is not limited thereto. For example, when the user naturally holds the device main body  210 , the depress button  234  may be arranged at a position where any one of the middle finger, the ring finger, and the little finger reaches. 
     The above-mentioned configuration allows the presentation of sense of force to the middle finger, the ring finger, or the little finger, instead of the index finger, relative to the finger movement. 
     Further, in the description so far, the examples in which there is only one depress button  234  (only one finger to which sense of force is presented) are described; however, the present embodiment is not limited thereto. For example, two or more depress buttons  234  may be arranged corresponding to two or more fingers of the user, the index finger and the middle finger, for example. In this case, the settings of the sense-of-force presentation start position Ds of a sense of force to be presented at each depress button  234 , subsequent variation amount ΔF in the magnitude of the maximum sense of force to be presented per predetermined unit depress amount ΔD, and so on may be different from each other or the same as each other. 
     Still further, in the examples described so far, the depress button  234  has a sense-of-force presentation apparatus and a sense of force is presented for a manipulation of depressing the depress button  234 . However, the present embodiment is not limited to this configuration; for example, instead of or in addition to the arrangement of a sense-of-force presentation apparatus on the depress button  234 , a button, a switch, or the like arranged at the button manipulation block  232  (a button or the like manipulated by the thumb) may have a sense-of-force presentation apparatus, thereby presenting (presenting a counter force against depressing or tilting) a sense of force to a button depressing manipulation or a switch tilting manipulation. 
     According to the above-mentioned example, a sense of force can be presented to the thumb and, if a sense-of-force presentation apparatus is arranged on the depress button  234 , a sense of force can be presented to the thumb and other fingers. 
     REFERENCE SIGNS LIST 
       10  Control apparatus,  11  Control block,  12  Storage block,  13  Device interface,  14  Communication block,  20  Device,  41  Sense-of-force command receiving block,  42  Command storage block,  43  Depress amount acquisition block,  44  Sense-of-force presentation processing block,  210  Device main body,  211  Holding block,  212  Manipulation block,  220  Fixture,  221  Left-side surface,  222  Right-side surface,  231  Thumb sensor block,  232  Button manipulation block,  233  First finger sensor,  234  Depress button,  235  Display block,  240  Battery,  241  Second finger sensor,  242  Third finger sensor