Abstract:
A passive prop for controlling the operation of a video game through visually detecting and tracking the prop. One aspect of the invention is a spherical prop of principally one color, but with a spot of another color located on the sphere. This provides control of a video system through tracking of position and orientation of the prop. Another aspect of the invention is a planar offset prop consisting of two color regions, a first color region being offset from a second color region. A third aspect of the invention allows the provision of a push-button control for a simple, passive prop, the push-button being effective to cause a color region of a prop to change color.

Description:
FIELD OF THE INVENTION 
     The present invention relates to control of a video system through tracking of a visual pattern. More particularly, the invention relates to methods and apparatus implementing a variety of multicolored props which can be easily tracked by a color recognition system, and whose movements are translatable into control inputs to control the operation of apparatus such as a video game or other video controlled device. 
     BACKGROUND OF THE INVENTION 
     A variety of computer and video systems exist today, which employ input devices, such as props. The props are held and manipulated by the user, and carry a color or pattern which is easily tracked by a color recognition system, and whose motion is easily followed by the color recognition system. The motion of the prop is translated into control inputs for a video or computer device. For example, in a video game or virtual reality system, a prop can be wielded by the user, and the motions of the prop can be used to direct motions of a character displayed on a screen. In contrast to wired props, which are expensive and which limit the user&#39;s movement, passive props can allow the user to move anywhere within the field of view of a camera which is part of the video system. Also, in contrast to wired, radio frequency (“RF”) or infrared (“IR”) control devices, which are relatively expensive, and which contain a certain amount of electronic circuitry, passive props can be inexpensive to produce, simply being made of plastic or paper with a design printed on the prop. Also, in contrast to the traditional wired joystick, which is a nuisance to attach and remove from the video system or computer, the passive prop is freely interchangeable. The user need merely cast one prop aside and take up another in order to make the change. Also, because of the simplicity and ease of manufacture of the prop, props can be manufactured and offered very inexpensively, enabling the user to afford a whole collection of props. 
     Various difficulties exist in the present use of passive props. For example, not every prop offers all desirable degrees of freedom. It may be desirable, for example, to allow a prop to be used to control a cursor on a television screen. In such a prop, five degrees of freedom would be desirable. Movement of the prop in the x, y and z directions can control gross movements of the cursor, and adjusting the pitch and yaw of the prop can be used to control fine movements of the cursor. 
     One difficulty with the use of passive props is the difficulty with disambiguation that occurs in some prop designs. Tracking of a passive prop may be performed through conic solutions. The design of many props results in two similar or identical conic solutions. It is difficult to disambiguate these solutions to accurately track the motion of the prop. A further difficulty is that presently existing props do not provide a sufficient range of control options. Thus, there exists a need in the art for improved passive prop designs which suffer from minimal ambiguity, which provide a convenient and precise method for controlling a cursor, and which provide an increased range of control. 
     SUMMARY OF THE INVENTION 
     One embodiment of the present invention is a sphere of a known color and size with a large dot of another color upon it. The x and y position and movement are established from the outline of the spherical surface. The z position and movement are established from the apparent area of the spherical surface. Since the actual size of the sphere is known, it is a simple matter to compare the apparent size of the sphere with the actual size to establish the z position. The pitch and yaw of the sphere are determined from the dot relative to the position of the sphere. 
     Another embodiment of the present invention is used to increase the ability to disambiguate the potential conic solutions. In this embodiment, the prop consists of a square within a circle. The square is moved off the plane of the circle. This can be done by placing a larger circle and a smaller square parallel to one another along a cylinder. The general shape of the prop can be that of a bowl, or of an inverted bowl, depending on the particular design choice made. A small offset can significantly help address the problem of disambiguation. 
     Another advantageous prop design in accordance with the present invention is a push-button design. In addition to controlling a video or computer device through the motion of the prop, a push-button would permit the user an additional degree of control by adding an additional binary choice. The addition of a push-button feature can make a passive prop usable in the same way as a computer mouse, for example. The user could use the prop to move a cursor or pointer over a radio button and use the push-button to activate the radio button. Multiple buttons of course could be employed. 
     To this end, in a further additional embodiment of the present invention, a prop has a pattern that faces the camera of a video system. The user holds a handle or other structure on the back side of the prop. The prop has a hole in the center. The hole is occluded by a non-recognized background color or by a swatch of the surrounding color. The color of the center point is changed when a button is pressed by the user. This color change is recognized by the prop detection of the video system. 
     A more complete understanding of the present invention, as well as further features and advantages of the invention, will be apparent from the following Detailed Description and the accompanying drawings. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is an “eyeball” prop according to an embodiment of the present invention; 
     FIG. 2 is front view of a planar offset prop according to another embodiment of the present invention, consisting of a square within and offset from a circle, the square being offset forward of or in front of the circle; 
     FIG. 2A is a side view of the planar offset prop of FIG. 2, showing additional details of the prop illustrated in FIG. 2; 
     FIG. 2B is a front view of a planar offset prop according to another embodiment of the present invention, consisting of a square within and offset from a circle, the square being offset rearward of or in front of the circle; 
     FIG. 2C is a side view of the planar offset prop of FIG. 2B, showing additional details of the prop illustrated in FIG. 2B; 
     FIG. 2D is a front view of a bowl-shaped prop according to the present invention; 
     FIG. 2E is a side view of the prop of FIG. 2D, showing additional details of the prop illustrated in FIG. 2D; 
     FIG. 3 is a front view of a push-button prop according to another embodiment of the present invention, with the push-button not actuated; 
     FIG. 3A is a front view of the push-button prop of FIG. 3 with the push-button actuated; and 
     FIG. 3B is a back view of the push-button prop of FIG. 3 showing additional details of the prop. 
    
    
     DETAILED DESCRIPTION 
     FIG. 1 shows a prop  10  in accordance with one embodiment of the present invention with the prop  10  shown in relation to a video game system  100  with which it can suitably be used. Video game system  100  consists of a camera  112 , a video game processor  114 , and a video display unit  122 . Video game processor  114  includes a color detector  116 , a color tracker  118 , and a game processor  120 . Prop  10  is preferably designed to be easily detected and tracked by a video game system such as the video game system  100 . Prop  10  preferably consists of a sphere  20  of known size and of uniform color. Sphere  20  preferably contains indentations  22  and  24  or tend to direct which induce the user to hold it in a preferred orientation relative to camera  112 . A circular dot  30  of a different color is located on the surface of the sphere  20 . Sphere  20  is preferably of a distinctive color which is readily distinguishable from the colors of background objects, and which is easily detectable by a color detection device. Dot  30  is likewise preferably chosen of a color readily distinguishable from those of background objects. 
     Prop  10  offers several advantages. First, as a sphere, it is relatively immune to highlighting. Unlike a planar prop which has very different reflection characteristics as it is angled toward or away from a light source, spherical prop  10  does not significantly change its reflective characteristics with changes in orientation. Secondly, prop  10  can be grasped in such a way as to allow the user fine control, as of a cursor, as well as gross control. Thirdly, the dot  30  on prop  10  allows the orientation to be conveniently detected. The colors of sphere  20  and dot  30  can be distinguished by color detection and processing circuitry or a suitable combination of hardware and software. The relative positions of the dot  30  and the sphere  20  can be computed and used to provide control inputs. 
     Spherical prop  10  is able to provide control in the x, y and z directions. Control in the x and y directions is provided by tracking the position of prop  10  within a frame of video data. Control in the z direction is provided by comparing the apparent area of prop  10  within a frame of video data with the known size of the prop  10 . Control of pitch and yaw are provided by tracking the position of dot  30  relative to sphere  20 . 
     The indentations  22  and  24  tend to induce a user to hold the prop  10  such that dot  30  is not aligned with an axis through the sphere  20  and pointing at camera  112  when prop  10  is normally held and utilized. 
     The images of sphere  20  and prop  30  are processed by a video system, such as the video system  100 , into conic sections. These conic sections remain conic sections even after distortion by a perspective transform. The mathematics of the conic sections provides a way to find the perspective distortion, and thus the location of prop  10  relative to a camera, from the statistics of the apparent shape of prop  10 . The presence of dot  30  on prop  10  also allows a video system to determine the orientation of prop  10  by projecting separate conic sections for sphere  20  and dot  30 . The relative orientations and positions of conic sections processed from sphere  20  and dot  30  can be determined by a video system and used to determine the position and orientation of prop  10 . 
     FIG. 2 shows a planar offset prop  32  according to another embodiment of the present invention. Prop  32  preferably includes a large circular color region  36  and a smaller square color region  38 . Each of the circular color region  36  and the square color region  38  is generally planar. Depending on the application to which prop  32  is to be put, circular color region  36  and square color region  38  may or may not share a common center. 
     FIG. 2A is a side view of prop  32 , shown in a representative orientation and position with respect to a video camera  124 . Video camera  124  may be used with the prop  32  in a video system, such as the video game system  100 . FIG. 2A shows a handle  34  by which the prop  32  can be held. From FIG. 2A, it can be seen that circular color region  36  and square color region  38  each rest on a substrate  36   a  and  38   a  respectively, each of the substrates  36   a  and  38   a  having a finite and significant thickness. The thickness of substrate  38   a  serves to offset square color region  38  toward the camera  124  when prop  32  is held in a typical position and orientation relative to camera  124 . This greatly simplifies disambiguation of different conic sections projected by different orientations of prop  32 . This embodiment of the present invention causes some occlusion problems, in that light from square color region  38  obstructs light from circular color region  36 . This difficulty is offset by the significant disambiguation provided by this embodiment, particularly since the offset between circular color region  36  and square color region  38  can be very slight, thus minimizing occlusion, and still provide significant disambiguation. 
     FIG. 2B shows a second planar offset prop  33  according to another embodiment of the present invention. Prop  33  preferably includes a large circular color region  42  and a smaller square color region  44 . Both the circular color region  42  and the square color region  44  are preferably generally planar. 
     FIG. 2C shows a side view of the prop  33 , shown in a representative orientation and position with respect to a video camera  124 . FIG. 2C shows a handle  35  by which prop  33  can be held. From FIG. 2C it can be seen that the circular color region  42  rests on a substrate  42   a,  the substrate  42   a  having a significant thickness. A section is cut out of the substrate  42   a  for a square color region  44 , which is placed at the bottom of the cut out section. Square color region  44  is thus offset away from camera  124  when prop  33  is held in a typical position and orientation relative to the camera  124 . This greatly simplifies disambiguation of different conic sections projected by different orientations of prop  33 . 
     This embodiment of the present invention causes a slight occlusion and a gap  49  between the circular color region  42  and the square color region  44 . This gap may slightly complicate run-length processing typically used by a video system to track a moving prop. These complications are offset by the increased disambiguation provided by the prop  33 , particularly since, as with prop  32 , a slight offset provides significant disambiguation. 
     The occlusion issue raised by props  32  and  33  can be eliminated by another embodiment of the present invention. FIG. 2D shows a bowl-shaped prop  70  according to this embodiment of the present invention. Prop  70  consists of a concave, or bowl, shape  80  of a predetermined size, such as the size of a handguard of a fencing foil, and uniform color. A square  90  of a known, easily distinguishable color, is located on the bowl shape  80 . The colors of bowl shape  80  and square  90  are chosen so as to be easily distinguishable from one another and from background objects. 
     FIG. 2E is a side view of the prop  70 , more clearly showing its shape, and showing a representative position and orientation relative to camera  124 . FIG. 2E also shows a handle  72 , by which a user can hold and manipulate the prop  70 . Placing square  90  at the bottom of bowl shape  80  provides an offset, which simplifies disambiguation as described above with respect to the props  32  and  33 , but without the occlusion which would result if square  90  were separated from bowl shape  80 . Prop  70  causes an intensity gradient, but this is more than compensated by the simplification of disambiguation which the design of prop  70  provides. 
     FIG. 3 is a front view of a push-button prop  100  according to another embodiment of the present invention. Prop  100  gives a user an additional degree of control by providing the equivalent of a binary switch. The push-button of prop  100  can be used, for example, as the equivalent of a mouse button. Alternatively, for example in a role-playing computer game, the push-button can be used to enable the player&#39;s character to pick up or drop objects, by using the prop  100  to move a cursor over the object and pressing the push-button to pick it up. FIG. 3 shows a generally planar first color region  102 , a generally planar square color region  104 , and a hole  106  in the center of square color region  104 . While a square region  104  is shown, it will be recognized that a circle or other predetermined shape might also be employed. FIG. 3 shows prop  100  in the condition in which a button  110  shown in FIG. 3B is not pushed. With the button  110  not pushed, hole  106  is black, as illustrated, but it will be recognized that it could be any of a number of predetermined recognizable colors. 
     FIG. 3A is a front view of prop  100  with the button pushed. Hole  106  is now white, as illustrated, but it will be recognized that it could be any of a number of predetermined recognizable colors different than the color selected to appear when button  110  (shown in FIG. 3B) is not pushed. 
     FIG. 3B is a rear view of the prop  100 , showing further details of the prop  100 . In addition to the circular color region  102  and the hole  106 , FIG. 3B additionally shows a handle  108  by which the prop  100  can be held by a user, the button  110  is located on the handle  108 , and provides one suitable mechanism by which hole  106  is made to change color. A slider bar assembly  112  is controlled by operation of the button  110 . Slider bar assembly  112  is of a desired, easily distinguishable color, preferably distinct from the colors of circular color region  102  and square color region  104 . When button  110  is not pressed, slider bar assembly  112  remains above hole  106 . When the user presses button  110 , the slider bar assembly  112  moves down, covering hole  110 . For the purposes of illustration, the slider bar assembly is white. Therefore, when slider bar assembly  112  covers hole  110 , the hole  110  appears white. Handle  108  may also suitably include grip preferably shaped so that the user may comfortably and conveniently hold and manipulate the prop  100 . While a simple mechanical assembly is described above, it will be recognized that other arrangement may suitably be employed. 
     While the present invention has been disclosed above in the context of a number of presently preferred embodiments, it will be recognized that a wide variety of implementations may be employed by persons of ordinary skill in the art consistent with the above teachings and the claims which follow below.