Source: http://www.google.com/patents/US6890262?dq=5537618
Timestamp: 2017-10-23 06:55:43
Document Index: 416620779

Matched Legal Cases: ['art 71', 'art 70', 'art 70', 'art 70', 'art 70', 'art 7211', 'art 7214']

Patent US6890262 - Video game apparatus, method and recording medium storing program for ... - Google Patents
A 3-dimensional video game apparatus comprises display 40 for displaying an image; image-processing means 16 for creating a 3-dimensional image from the viewpoint of a simulated camera and displaying that image on a display screen; and game controller 14 which conducts the game in response to reactions...http://www.google.com/patents/US6890262?utm_source=gb-gplus-sharePatent US6890262 - Video game apparatus, method and recording medium storing program for controlling viewpoint movement of simulated camera in video game
Publication number US6890262 B2
Application number US 10/197,514
Also published as EP1279425A2, EP1279425A3, US20030017872
Publication number 10197514, 197514, US 6890262 B2, US 6890262B2, US-B2-6890262, US6890262 B2, US6890262B2
Inventors Toshimitsu Oishi, Toru Okubo, Hideyuki Domitsu, Tomoya Yamano
Patent Citations (8), Referenced by (193), Classifications (17), Legal Events (7)
US 6890262 B2
image capture means which can be arranged in such an orientation that a field of view is a play area in front of the display screen, and said image capture means periodically capturing an image of the play area in a state arranged;
a main memory having a first buffer storing the captured images by the image capture means and a second buffer storing a position of a reference region of the player;
said image capture means including:
an averaging processor generating an average image based on a plurality of the captured images stored in the first buffer except a last captured image;
a first differentiator/binary converter generating a difference regarding color information between the last captured image and the averaged image;
an area determinator checking whether an area of a region corresponding to said difference is below a threshold value or not;
position detecting means for detecting from the captured images a position of a player's reference region in case said area is found to be greater than the threshold value and for storing the position of the detected reference region into the second buffer;
movement detecting means for detecting two-dimensional movement of said position of the reference region detected by said position detecting means; and
viewpoint changing means for moving the simulated camera viewpoint to a new viewpoint in accordance with a change direction and change amount of the position of the detected reference region and displaying on the screen a view from said new viewpoint.
2. A video game apparatus according to claim 1, further comprising initial play position setup means for performing initial calibration processing to normalize the player's standing position.
information updating means for updating position data, said position data being indicative of the positions of the player's reference regions in said second buffer;
a movement amount calculator for calculating each movement amount between a present position of the reference region and a last position of the reference region and for calculating an average value of said movement amounts; and
buffer size determining means for determining a size of the second buffer according to the averaged value of the movement amounts.
11. The video game apparatus according to claim 1, wherein said image capture means further includes an image extractor which extracts an image of a specified radius centered on the reference region of the player from the difference of the image obtained by said first differentiator/binary converter and said area determinator checks whether an area of the reference region corresponding to said difference in said extracted image is below the threshold value.
periodically capturing images of a play area by means of an image capturing means positioned in a desired position so that an orientation is such that a field of view is a play area in front of the display screen including the player;
generating an averaged image based on averaging a plurality of the captured images stored in an image buffer except a last captured image;
generating a difference regarding color information between the last captured image and the averaged image;
checking whether an area of a region corresponding to said difference is below a threshold value or not;
detecting a player's reference region from the last captured image; detecting two-dimensional movement of a position of the reference region; and
moving said simulated camera viewpoint to a new viewpoint in accordance with a direction of change and amount of change of the two-dimensional movement of the position of the detected reference region when said difference is greater than said threshold value and displaying on said screen a view from the new viewpoint.
13. The method according to claim 12, wherein the player's reference region is the player's head.
detecting a player's reference region from the last captured image;
detecting two-dimensional movement of a position of the reference region; and
In contrast, unlike the case of coin operated games, it is difficult to achieve home video games which provide game scenes having a feeling of realism, through linking the viewpoint of the simulated camera to the player's movement. For example, a fixed and invariable play area cannot be assured, as it can in the case of coin-operated games, so there is the problem of processing the varied background images included in the captured image according to place and time. Furthermore, there is a limit to how strongly a player can be urged to buy the special “detectable object” which, as mentioned above, is easily detectable in a captured image.
FIG. 8 is a diagram illustrating an example of the “player standing position guide” image included in the title screen;
FIG. 9 is a diagram showing a variation of the “player standing position guide” image included in the title screen;
FIGS. 21A and 21B are diagrams showing the memory content of the “recognized position” buffer.
With family-use video game machine 46, a 3-dimensional game space (virtual 3-dimensional space) is built in memory and it is arranged that the appearance of viewing the 3-dimensional space from the viewpoint of a simulated camera located at a specified spatial position is displayed on home-use TV receiver 40. At this time, home-use game machine 46 confirms the position of player P's head from the player image and performs viewpoint change processing to link above-mentioned viewpoint position to player's head position. As a result, when player P moves his body in front of camera unit 42, the game scene displayed on home-use TV receiver 40 changes in response to that, and player P can get a sense just as though he himself had entered into the 3-dimensional game space (a sense of “being there.”) In this way, with home-use game machine 46, a game is achieved wherein player P, using gun controller 48, shoots the enemy characters who appear in the 3-dimensional game space and player P, by moving his body (specifically his head position), can dodge the incoming attack (shots) of the enemy characters.
FIG. 2 is a diagram showing the hardware configuration of game apparatus 10. As shown in that diagram, game apparatus 10 is configured by mounting DVD (Digital Versatile Disk, trademark) unit 25, which serves as an information storage medium, on home-use game machine 46, which is connected to monitor 18 and speaker 22, which are part of home-use TV receiver 40. Here, for supplying game programs and game data to home-use game machine 46, DVD unit 25 is used, but any other kind of information storage medium, such as CD-ROM (Compact Disk—Read Only Memory, trademark) or ROM (Read Only Memory) card, etc. could be used. In addition, it is possible to supply game programs and game data to home-use game machine 46 from a distant location, via a data communication network, such as the Internet.
Image processor 16 calculates the coordinate positions, etc. of each character (enemy characters, physical “characters” such as the various buildings positioned in the game space, etc.) viewed from the viewpoint of the simulated camera within the simulated 3-dimensional space. From the calculated coordinate positions in the simulated 3-dimensional space, image processor 16 also performs the calculations for converting to coordinate positions in 2-dimensional space and performs positioning processing of the polygons which constitute the images to be plotted in the display buffer of main memory 26. In addition, image processor 16 performs texture mapping in regard to each polygon. As information on the viewpoint of the simulated camera for calculation of character coordinate positions, position information on the player's marked region for example his head, is transmitted from position recognizing part 71 which will be explained later. Consequently, the viewpoint of the simulated camera substantially matches the eye of the player, and in this embodiment, a character corresponding to the player does not appear on the monitor screen.
Initial play position setup part 70 is a means for performing initial calibration processing to normalize the player's standing position. Thus, as shown in FIG. 6, in a specified position on monitor 18, it displays “player standing position guide image” 50 and in a specified position in that image 50, which in the present embodiment is slightly above the center, it displays “head reference position marker” 52 shown, for example, by a circular marker. It also displays, superimposed on the above image, player image 56, image-captured by camera unit 42, and “present recognized position marker” 54, obtained from that image 56 and shown, for example, as a round dot. By changing the standing position or stance of player P, or the photographing direction or photographing position of camera unit 2, present recognized position marker 54 moves into circularly-shaped head reference position marker 52. Whereupon the display appearance of head reference position marker 52 is changed (for example, turning ON or OFF a colored or flashing display) indicating that the reference position (initial play position) has been normalized.
Next, in FIGS. 4A through D, is shown a series of game scenes displayed on home-use TV receiver 40 for the case that player P's head moves in an up-down direction. In FIGS. 4A through 4D of the same figure, four scenes are shown in the order of their display for the case that player P, in front of camera unit 42, rises from a crouched stance gradually to an almost fully upright stance—in other words, for the case that player P's head moves from a lower position to a higher one. In the game scene shown in FIG. 4A of the same figure, table-shaped barrier B exists immediately in front of the viewpoint, expressing the situation that player P is hiding in front of the lower part of this barrier B. In this case, only a part of the head of enemy character AC1, who is behind barrier B holding a gun in his hand, appears in the game scene. Next, the game scene shown in FIG. 4B of the same figure shows the situation where player P has raised his head a small amount, so that the level of his head is about the same as that of the upper surface of barrier B. In this case, enemy character AC1 becomes visible down to his chest while behind him the heads of two more enemy characters, AC2 and AC3, newly appear in the game scene. The game scene shown in FIG. 4C of the same figure shows the situation where player P has straightened up more and raised his head. In this case, player P's head is in a slightly higher position in the game scene than the barrier upper surface Bs. The upper bodies of the three enemy characters, AC1 through AC3, are visible and the barrier upper surface Bs also appears. The game scene shown in FIG. 4D of the same figure shows the situation where player P is essentially in a fully upright stance. In this case, the head and shoulders of player P are exposed above barrier B and barrier upper surface Bs is visible to a greater extent than in the game scene shown in FIG. 4C of the same figure. As shown in these figures, as the position of player P's head rises, the display position of barrier B moves lower, and a more complete view of barrier upper surface Bs and enemy characters AC1 through AC3 gradually appears in the game scene.
FIG. 6 is a diagram showing an example of the game title screen (start screen) supplied by home-use game machine 46. The title screen shown in the same figure is displayed on home-use TV receiver 40 after start up of home-use game machine 46 or after game completion and before the start of a new game. As shown in the same figure, on both sides of the screen in this title screen, images of menu buttons are displayed and in the upper part of the center, game title text line 60 is displayed. In addition, under title text line 60, player “standing position guide image” 50 is displayed. In addition, menu buttons 62 are objects to be operated to select various games or set options, and when player P, by means of controller 48 shoots menu button image 62, the game or option setting corresponding to that menu button image 62 is started.
Player “standing position guide image” 50 is displayed by means of initial play position setup part 70 and is an image for assisting player to normalize his standing position prior to game start. Prior to start of the game, player P, looking at this same “standing position guide image” 50, corrects his standing position or stance in front of camera unit 42 or corrects the picture-taking angle or position of camera unit 42. Initial play position setup part 70, based on real-time images acquired by camera unit 42 from image capture controller 7111, in other words, based on player image 56, aligns and displays head reference position marker 52 and present recognized position marker 54. Head reference position marker 52 is a circular mark displayed in a fixed position of player image 56 (for example in the center of player image 56 at a position 15% from the top, etc.) Its purpose is to show the location in player image 56 where the image position of player P's head (top part of his head) should be aligned, prior to start of the game. Likewise, with this game apparatus 10, it is arranged that the present position of player P's head can be recognized from player image 56 and the circular dot indicating that recognized position is displayed superimposed as “present recognized position marker” 54 on player image 56. Player P, by moving his own body (standing position or stance) or changing the picture-taking direction or position of camera unit 42, can make present recognized position marker 54 move into circular head reference position marker 52. Initial play position setup part 70 determines whether present recognized position marker 54 is displayed inside head reference position marker 52. It is made so that, depending on that result, it changes the display appearance of head reference position marker 52 (for example, turning ON or OFF a color or flashing display). In this way, it is arranged that player P, while viewing player standing position guiding image 50, can readily normalize the positional relationship of himself and camera unit 42, prior to start of the game.
FIG. 7 is a flow diagram describing game title screen display processing by home-use game machine 46. As shown in the same figure, with home-use game machine 46, first, microprocessor 14 plots the background image of the title screen, title text line 60, menu button images 62, copyright indication, etc. into VRAM which is included in image processor 16 (S101). Next, the image to be captured by camera unit 42 (here, the object is to photograph player P, so this is called a “player image”) is captured (S102). Then, the player image captured in S102 is plotted into VRAM (S103), so that the player image is positioned in the “player standing position guide image” 50 position in the title screen.
Further, based on the player image captured in S102, the position of player P's head (head position) is recognized and information after filter processing the recognition results (filter processed head position) is acquired (S104). The position recognition processing and filter processing mentioned here will be explained in detail later. Afterward, microprocessor 14 draws the “present recognized position marker” 54 at the VRAM coordinates corresponding to the filter processed head position (S105), so that a marker (red point) is displayed at head position of the player image included in the title screen.
If the above is done, in the case that the positional relationship between player P and camera unit 42 is not appropriate, as shown in FIG. 8, “present recognized position” marker 54 will be displayed outside circular head reference position marker 52 in the title screen and head reference position marker 52 will be displayed in blue. On the other hand, in the case that the positional relationship between player P and camera unit 42 is appropriate, as shown in FIG. 6, “present recognized position” marker 54 will be displayed inside circular head reference position marker 52 in the title screen and head reference position marker 52 will be displayed in red. In this way, by changing his standing position or changing the mounting position or camera-direction of camera unit 42, so that head reference position marker 52 is made to be displayed in red in the title screen, player P can readily normalize the positional relationship between himself and camera unit 42.
In addition it is acceptable if the arrangement is that, according to the relationship between the present recognized position of player P's head and the reference position, a message, for example “Please move to the right,” or “Please move to the left” or “Please move forward,” or “Please move backward,” is displayed. Also, the shapes of present recognized position marker 54 and head reference position marker 52 are not limited to those described above, but a variety of kinds of shapes can be adopted. Furthermore, the guidance for player P can be provided prior to start of the game. To do that, the guidance for player P may be provided, for example, on a different screen from the title screen.
Next, in the above-described filter processing, as shown in FIG. 21A, a head position buffer is provided in main memory 26, and, by means of information updating part 7211, this buffer stores a maximum of 16 head positions (Pn through Pn−15; where the appended characters correspond to the recognition time). This head position buffer is for storing the head positions of player P, recognized every specified time period, and in the buffer are stored in time sequence a maximum of 16 head positions, including the newest head position. Then, when the newest head position is recognized, that head position is additionally stored in the head position buffer and, at the same time, the oldest head position is discarded. For example, in the situation of FIG. 21A of the same figure, when head position Pn+1 is recognized, that head position Pn+1 is additionally stored in head position buffer and the oldest head position, Pn−15 is discarded from the head position buffer. In addition, with this head position buffer, the buffer size (in other words, the number of stored head position “units”) is variable. In the present embodiment, for example, as shown in FIG. 21B of the same figure, the states of 2 units, 4 units, and 8 units can be arranged, while as shown in FIG. 21B of the same figure, the state of 16 units can be arranged, thus providing a total of four possible storage states.
Next, buffer size determining part 7214 compares the current buffer size n and buffer size N (S308). If current buffer size n is larger than buffer size N, head positions stored in the head position buffer are discarded in order from the oldest ones, so that current buffer size n becomes buffer size N (S309). On the other hand, if current buffer size n is equal to or less than N, S309 is skipped. Afterward, the head positions stored in the head position buffer are averaged and those values are stored in main memory 26 as “previous output” (S311) and are outputted as filter processed head positions (S312). As explained above, the filter processed head positions are used in determining viewpoint position when a 3-dimesional space is plotted. In addition, they are used when determining the display position of present recognized position marker 54 when displaying player standing position guide image 50 in the title screen. Note that, here, the average value of head positions stored in the head position buffer is considered to be the filter processed head position. However, it would also be acceptable to attach weight to the head positions stored in the head position buffer, for example making it such that the influence of the newest head position in determining the filter processed head position would be greater.
US6176782 * Dec 22, 1997 Jan 23, 2001 Philips Electronics North America Corp. Motion-based command generation technology
JPH09138637A Title not available
US7452275 * Jun 25, 2002 Nov 18, 2008 Konami Digital Entertainment Co., Ltd. Game device, game controlling method and program
US7833100 * Apr 24, 2006 Nov 16, 2010 Nintendo Co., Ltd. Video game program and video game system
US7976380 Aug 10, 2006 Jul 12, 2011 Avago Technologies General Ip (Singapore) Pte. Ltd. System and method for using wavelet analysis of a user interface signal for program control
US8619030 * Nov 9, 2010 Dec 31, 2013 Blackberry Limited Method and apparatus for controlling an output device of a portable electronic device
US8659590 * Dec 17, 2008 Feb 25, 2014 Nvidia Corporation System, method, and computer program product for modifying signals of a three-dimensional graphics application program based on a tracking algorithm
US9220976 * Aug 31, 2009 Dec 29, 2015 Nintendo Co., Ltd. Storage medium storing game program, and game device
US9333424 * Aug 17, 2012 May 10, 2016 Apple Inc. Interactive gaming with co-located, networked direction and location aware devices
US20060105842 * Apr 19, 2003 May 18, 2006 Yong-Hwan Kim Shooting game machine and method for performing it
US20060252474 * May 6, 2006 Nov 9, 2006 Zalewski Gary M Method and system for applying gearing effects to acoustical tracking
US20070265076 * Aug 9, 2006 Nov 15, 2007 Pixart Imaging Inc. Interactive wireless game apparatus and wireless peripheral module
US20080045328 * Aug 10, 2006 Feb 21, 2008 Nobutaka Itagaki System and method for using wavelet analysis of a user interface signal for program control
US20120113002 * Nov 9, 2010 May 10, 2012 Research In Motion Limited Method and apparatus for controlling an output device of a portable electronic device
US20120315994 * Aug 17, 2012 Dec 13, 2012 Apple Inc. Interactive gaming with co-located, networked direction and location aware devices
WO2007130582A3 * May 4, 2007 Dec 4, 2008 Xiadong Mao Computer imput device having gearing effects
WO2007130792A3 * Apr 19, 2007 Sep 12, 2008 Sony Comp Entertainment Inc System, method, and apparatus for three-dimensional input control
WO2008070178A1 * Dec 6, 2007 Jun 12, 2008 Actionairgun Llc System, method, and apparatus for organizing and implementing a real-life, physical activity
WO2012148853A3 * Apr 23, 2012 Jan 10, 2013 Microsoft Corporation Manual and camera-based game control
U.S. Classification 463/31, 463/36, 463/7
International Classification G06F3/0488, A63F13/00, G06F3/01, A63F13/10
Cooperative Classification A63F13/213, A63F2300/6676, A63F2300/1012, A63F13/5258, A63F13/10, A63F2300/8076, A63F2300/6661, A63F13/837, A63F2300/1093
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OISHI, TOSHIMITSU;OKUBO, TORU;DOMITSU, HIDEYUKI;AND OTHERS;REEL/FRAME:013124/0242