Abstract:
Locality based morphing in a computer graphics system provides efficient techniques for simulating or animating position-based changes including but not limited to deformations. Two models are provided: a model unaffected by a destructive or other process, and a further model that has been totally affected by the process. Position information is used to specify particular parts of the model to morph. Morphing is selectively applied on a spatially localized basis so that only parts of the object are morphed whereas other parts of the object remain unmorphed. Such techniques can be used for example to simulate or animate progressive damage to a target such as an enemy robot.

Description:
FIELD 
       [0001]    The technology herein relates to computer graphics, and more particularly to localized morphing between computer generated models. Still more particularly, the technology herein relates to systems, methods, techniques and stored programs that provide real time interactive computer imagery between less deformed and more deformed models of an object, wherein only parts of the object initially appear deformed based on user input. 
       BACKGROUND AND SUMMARY 
       [0002]    Changes tend to visually stimulate. Staring at a static painting or other fixed image in a gallery can be interesting and contemplative, but most people tend to prefer to view images that exhibit changes. In the real world, wind blowing through a tree causes every leaf to move and deform. Dropping an object onto the surface of water causes the surface to deform with waves and ripples. The flickering flames of a fire create a continuously changing pattern and also slowly transform the shape of the fuel that is being consumed. A car in a demolition derby will become progressively damaged and dented as it encounters one collision after another. These kinds of visual changes captivate and fascinate us and keep our interest. 
         [0003]    Anthropologists and behavioral scientists theorize that we humans have evolved in a way that makes us notice changes in our environment. A slight change in our surroundings might indicate the presence of a predator or an enemy. Whatever the reason, people are generally interested in the way a visual changes. For example, when viewing a vista or landscape, we tend to focus in on what is changing (e.g., smoke rising, cars moving, lights flashing, etc.). The changing parts of the scene tend to interest us more than the static parts. 
         [0004]    Computer graphics scientists and video game developers have long sought to realistically model changes in computer graphics generated scenes. Almost anything in the real world can be visually modeled using computer graphics given sufficient computation power and computation time. However, generating computer graphics images interactively in real time based on user input typically has significant constraints with respect to efficiency and often computation power. For example, video game developers are often constrained by relatively low capability computer graphics hardware and the need to produce images that rapidly respond (e.g., once every 1/30 or 1/60 or a second) to changing user inputs. 
         [0005]    One approach to changing or deforming a computer graphic model of an object is to mathematically perturb the object&#39;s shape in particular desired ways. For example, it is possible to model an object surface using a wire frame that can be mathematically perturbed using various techniques to provide realistic changes or deformations. Such techniques have been used successfully in the past to realistically model a variety of objects such as flags flying in the breeze, ripples propagating across the surface of a pond, and other interesting visual effects. Fractals have been used, for example, as a way to create interesting perturbations that appear to accurately model real world effects such as wind, flames and other influences. While some such techniques have been used successfully in video games, such mathematical techniques can be relatively computationally intensive and may therefore not be readily accessible by those developing interactive images displayable on low-capability computer graphics systems such as home and portable video game platforms. 
         [0006]    Another technique that has been widely used in video games and other computer graphics is called “morphing.” The basic idea behind morphing is to interpolate between two different computer graphic models. For example, to realistically model a person&#39;s face, one approach is to provide a number of different models with different facial expressions. One model might show the face with no expression. Another model might show the face with a smile. A third model might show the face with a frown. Rather than attempting to calculate mathematically how to change from one model to another, a “morphing” technique is used to generate a range of intermediate images between the two models. Such interpolation between a pair of different position, color or other values can be done relatively efficiently. Any number of progressive changes between the two models can be used depending upon the precision desired. A smooth, gradual interpolation between two different computer models such as facial expressions can be performed interactively with excellent results using low capability computer graphics hardware. See for example Haber et al., SIGGRAPH 2004 Course Notes “Facial Modeling and Animation”: (incorporated herein by reference). 
         [0007]    Generally, morphing in the past has been performed with respect to entire models. Dramatic effects such as seen in science fiction motion pictures have been generated in this way. For example, it is possible to “morph” one person&#39;s shape into that of another to make it appear as if the person is changing into another person, an animal or other object. 
         [0008]    However, there are certain kinds of computer graphics simulations and animations where it would be desirable to change only parts of a model. Take, for example, the example above of a demolition derby car. New dents will appear in various positions on the car&#39;s body depending upon where collisions occur. At the end of the demolition derby, the car may be completely damaged and dented. However, if one wished to show progressive denting and deformation of the demolition derby car, it would be necessary to provide selective deformation of different parts of the car&#39;s body with each new collision. It is not readily apparent how morphing could be used to provide such an effect. A more common approach might be to selectively calculate perturbations or deformations due to collision impacts at each collision position and apply such deformations to the wireframe model of the car. 
         [0009]    We have developed efficient techniques that allow computer graphics objects appear to be progressively deformed while giving the artist complete control over how the final deformed object would appear. In one exemplary illustrative non-limiting implementation, we created two 3-D models: an original undeformed model, and a final, completely deformed model. We then generated data for performing locality based morphing between the models. During interactive run time, we create a new model that will be the drawn model. We also create an internal data structure indicating how much to morph between each vertex of the 3D model. If the object is acted upon during user interaction, we calculate or otherwise determine where the object was acted upon and we also determine how much force or impact was applied to the object at that position. We use this position and force information to perform a locality based morph. It is possible to apply deformation amount to morphing blend values for specific vertices that will be affected. Using the morphing blend values, we morph geometry by interpolating, in a spatially localized way, between the original non-deformed model and the completely deformed model. We then update vertex normals so that the lighting will be corrected. 
         [0010]    Using such a technique, we can provide very efficient transformation between the two objects with only a part of the displayed model showing deformations based for example on user interaction. It is thus possible to selectively morph only certain portions of the model while allowing other portions of the model to remain undeformed or otherwise unchanged. We can provide such effects while using and storing only two initial models—a completely undeformed model and a completely deformed model. 
         [0011]    One exemplary illustrative non-limiting specific implementation is to simulate or animate damage to an object based on an attack. For example, in a fighting video game, a user can target different portions of an object such as a robot, a building or other enemy and apply projectiles, hammer blows or other forms of attack can cause the object to become progressively damaged in different places depending upon where the attack lands. Our locality-based morphing technique provides an efficient way to model such progressive damage on a low-capability computer graphics platform such as a home or portable video game machine. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0012]    These and other features and advantages will be better and more completely understood by referring to the following detailed description of exemplary non-limiting illustrative embodiments in conjunction with the drawings of which: 
           [0013]      FIG. 1  shows an exemplary external view of a non-limiting interactive computer graphics system in the form of a home video game apparatus for executing a game program; 
           [0014]      FIG. 2  is a block diagram showing an internal structure of the game apparatus; 
           [0015]      FIGS. 3A ,  3 B and  4  show different views of an exemplary illustrative non-limiting handheld controller for the video game system of  FIG. 1 ; 
           [0016]      FIG. 5  is a block diagram of an exemplary illustrative non-limiting implementation of the handheld controller; 
           [0017]      FIG. 6  is an exemplary flowchart of an illustrative non-limiting method for performing a localized morphing; 
           [0018]      FIG. 7  shows an exemplary flow of a model creation process; 
           [0019]      FIG. 8  shows an exemplary flow of a model morph-data generation process; 
           [0020]      FIG. 9  shows an exemplary flow of a morph parameter detection process; 
           [0021]      FIG. 10  shows an exemplary flow a localized morph application process; 
           [0022]      FIG. 11  shows an exemplary flow of a lighting correction process; 
           [0023]      FIGS. 12A ,  12 B and  12 C show different exemplary illustrative non-limiting localized morphing deformation profiles; 
           [0024]      FIG. 13  graphically illustrates exemplary illustrative non-limiting correlations between amount of applied force and amount of deformation; 
           [0025]      FIG. 14  graphically illustrates exemplary illustrative undeformed and deformed 3D polygon-based wireframe models; 
           [0026]      FIGS. 15 and 15A  show an exemplary illustrative non-limiting undeformed model of an object; 
           [0027]      FIGS. 16 and 16A  show an exemplary illustrative non-limiting completely deformed model of the object; and 
           [0028]      FIGS. 17 and 17A  show exemplary illustrative non-limiting additional views of undeformed and completely deformed 3D polygon based models. 
       
    
    
     DETAILED DESCRIPTION 
       [0029]    Techniques described herein can be performed on any type of computer graphics system including a personal computer, a home video game machine, a portable video game machine, a networked server and display, a cellular telephone, a personal digital assistant, or any other type of device or arrangement having computation and graphical display capabilities. One exemplary illustrative non-limiting implementation includes a home video game system such as the Nintendo Wii 3D video game system, a Nintendo DS or other 3D capable interactive computer graphics display system. One exemplary illustrative non-limiting implementation is described below, but other implementations are possible. 
       Exemplary Video Game Platform 
       [0030]      FIG. 1  shows a non-limiting example game system  10  including a game console  100 , a television  102  and a controller  107 . 
         [0031]    Game console  100  executes a game program or other application stored on optical disc  104  inserted into slot  105  formed in housing  110  thereof. The result of the execution of the game program or other application is displayed on display  101  of television  102  to which game console  100  is connected by cable  106 . Audio associated with the game program or other application is output via speakers  109  of television  102 . While an optical disk is shown in  FIG. 1  for use in storing video game software, the game program or other application may alternatively or additionally be stored on other storage media such as semiconductor memories, magneto-optical memories, magnetic memories and the like and/or downloaded over a network or by other means. 
         [0032]    Controller  107  wirelessly transmits data such as game control data to the game console  100 . The game control data may be generated using an operation section of controller  107  having, for example, a plurality of operation buttons, a key, a stick and the like. Controller  107  may also wirelessly receive data transmitted from game console  100 . Any one of various wireless protocols such as Bluetooth (registered trademark) may be used for the wireless transmissions between controller  107  and game console  100 . 
         [0033]    As discussed below, controller  107  also includes an imaging information calculation section for capturing and processing images from light-emitting devices  108   a  and  108   b . Preferably, a center point between light-emitting devices  108   a  and  108   b  is aligned with a vertical center line of television  101 . The images from light-emitting devices  108   a  and  108   b  can be used to determine a direction in which controller  107  is pointing as well as a distance of controller  107  from display  101 . By way of example without limitation, light-emitting devices  108   a  and  108   b  may be implemented as two LED modules (hereinafter, referred to as “markers”) provided in the vicinity of a display screen of television  102 . The markers each output infrared light and the imaging information calculation section of controller  107  detects the light output from the LED modules to determine a direction in which controller  107  is pointing and a distance of controller  107  from display  101  as mentioned above. As will become apparent from the description below, various implementations of the system and method for simulating the striking of an object described herein do not require use such markers. 
         [0034]    Although markers  108   a  and  108   b  are shown in  FIG. 1  as being above television  100 , they may also be positioned below television  100  or in other configurations. 
         [0035]    With reference to the block diagram of  FIG. 2 , game console  100  includes a RISC central processing unit (CPU)  204  for executing various types of applications including (but not limited to) video game programs. CPU  204  executes a boot program stored in a boot ROM (not shown) to initialize game console  100  and then executes an application (or applications) stored on optical disc  104  which is inserted in optical disk drive  208 . User-accessible eject button  210  provided on housing  110  of game console  100  may be used to eject an optical disk from disk drive  208 . 
         [0036]    In one example implementation, optical disk drive  208  receives both optical disks of a first type (e.g., of a first size and/or of a first data structure, etc.) containing applications developed for execution by CPU  204  and graphics processor  216  and optical disks of a second type (e.g., of a second size and/or a second data structure) containing applications originally developed for execution by a different CPU and/or graphics processor. For example, the optical disks of the second type may be applications originally developed for the Nintendo GameCube platform. 
         [0037]    CPU  204  is connected to system LSI  202  that includes graphics processing unit (GPU)  216  with an associated graphics memory  220 , audio digital signal processor (DSP)  218 , internal main memory  222  and input/output (IO) processor  224 . 
         [0038]    IO processor  224  of system LSI  202  is connected to one or more USB ports  226 , one or more standard memory card slots (connectors)  228 , WiFi module  230 , flash memory  232  and wireless controller module  240 . 
         [0039]    USB ports  226  are used to connect a wide variety of external devices to game console  100 . These devices include by way of example without limitation game controllers, keyboards, storage devices such as external hard-disk drives, printers, digital cameras, and the like. USB ports  226  may also be used for wired network (e.g., LAN) connections. In one example implementation, two USB ports  226  are provided. 
         [0040]    Standard memory card slots (connectors)  228  are adapted to receive industry-standard-type memory cards (e.g., SD memory cards). In one example implementation, one memory card slot  228  is provided. These memory cards are generally used as data carriers. For example, a player may store game data for a particular game on a memory card and bring the memory card to a friend&#39;s house to play the game on the friend&#39;s game console. The memory cards may also be used to transfer data between the game console and personal computers, digital cameras, and the like. 
         [0041]    WiFi module  230  enables game console  100  to be connected to a wireless access point. The access point may provide internet connectivity for on-line gaming with players at other locations (with or without voice chat capabilities), as well as web browsing, e-mail, file downloads (including game downloads) and many other types of on-line activities. In some implementations, WiFi module may also be used for communication with other game devices such as suitably-equipped hand-held game devices. Module  230  is referred to herein as “WiFi”, which is generally used in connection with the family of IEEE 802.11 specifications. However, game console  100  may of course alternatively or additionally use wireless modules that conform with other wireless standards. 
         [0042]    Flash memory  232  stores, by way of example without limitation, game save data, system files, internal applications for the console and downloaded data (such as games). 
         [0043]    Wireless controller module  240  receives signals wirelessly transmitted from one or more controllers  107  and provides these received signals to IO processor  224 . The signals transmitted by controller  107  to wireless controller module  240  may include signals generated by controller  107  itself as well as by other devices that may be connected to controller  107 . By way of example, some games may utilize separate right- and left-hand inputs. For such games, another controller (not shown) may be connected to controller  107  and controller  107  could transmit to wireless controller module  240  signals generated by itself and by the other controller. 
         [0044]    Wireless controller module  240  may also wirelessly transmit signals to controller  107 . By way of example without limitation, controller  107  (and/or another game controller connected thereto) may be provided with vibration circuitry and vibration circuitry control signals may be sent via wireless controller module  240  to control the vibration circuitry. By way of further example without limitation, controller  107  may be provided with (or be connected to) a speaker (not shown) and audio signals for output from this speaker may be wirelessly communicated to controller  107  via wireless controller module  240 . By way of still further example without limitation, controller  107  may be provided with (or be connected to) a display device (not shown) and display signals for output from this display device may be wirelessly communicated to controller  107  via wireless controller module  240 . 
         [0045]    Proprietary memory card slots  246  are adapted to receive proprietary memory cards. In one example implementation, two such slots are provided. These proprietary memory cards have some non-standard feature such as a non-standard connector or a non-standard memory architecture. For example, one or more of the memory card slots  246  may be adapted to receive memory cards developed for the Nintendo GameCube platform. In this case, memory cards inserted in such slots can transfer data from games developed for the GameCube platform. In an example implementation, memory card slots  246  may be used for read-only access to the memory cards inserted therein and limitations may be placed on whether data on these memory cards can be copied or transferred to other storage media such as standard memory cards inserted into slots  228 . 
         [0046]    One or more controller connectors  244  are adapted for wired connection to respective game controllers. In one example implementation, four such connectors are provided for wired connection to game controllers for the Nintendo GameCube platform. Alternatively, connectors  244  may be connected to respective wireless receivers that receive signals from wireless game controllers. These connectors enable players, among other things, to use controllers for the Nintendo GameCube platform when an optical disk for a game developed for this platform is inserted into optical disk drive  208 . 
         [0047]    A connector  248  is provided for connecting game console  100  to DC power derived, for example, from an ordinary wall outlet. Of course, the power may be derived from one or more batteries. 
         [0048]    GPU  216  performs image processing based on instructions from CPU  204 . GPU  216  includes, for example, circuitry for performing calculations necessary for displaying three-dimensional (3D) graphics. GPU  216  performs image processing using graphics memory  220  dedicated for image processing and a part of internal main memory  222 . GPU  216  generates image data for output to television  102  by audio/video connector  214  via audio/video IC (interface)  212 . 
         [0049]    Audio DSP  218  performs audio processing based on instructions from CPU  204 . The audio generated by audio DSP  218  is output to television  102  by audio/video connector  214  via audio/video IC  212 . 
         [0050]    External main memory  206  and internal main memory  222  are storage areas directly accessible by CPU  204 . For example, these memories can store an application program such as a game program read from optical disc  104  by the CPU  204 , various types of data or the like. 
         [0051]    ROM/RTC  238  includes a real-time clock and preferably runs off of an internal battery (not shown) so as to be usable even if no external power is supplied. ROM/RTC  238  also may include a boot ROM and SRAM usable by the console. 
         [0052]    Power button  242  is used to power game console  100  on and off. In one example implementation, power button  242  must be depressed for a specified time (e.g., one or two seconds) to turn the consoled off so as to reduce the possibility of inadvertently turn-off. Reset button  244  is used to reset (re-boot) game console  100 . 
         [0053]    With reference to  FIGS. 3 and 4 , example controller  107  includes a housing  301  on which operating controls  302   a - 302   h  are provided. Housing  301  has a generally parallelepiped shape and is sized to be conveniently holdable in a player&#39;s hand. Cross-switch  302   a  is provided at the center of a forward part of a top surface of the housing  301 . Cross-switch  302   a  is a cross-shaped four-direction push switch which includes operation portions corresponding to the directions designated by the arrows (front, rear, right and left), which are respectively located on cross-shaped projecting portions. A player selects one of the front, rear, right and left directions by pressing one of the operation portions of the cross-switch  302   a . By actuating cross-switch  302   a , the player can, for example, move a character in different directions in a virtual game world. 
         [0054]    Cross-switch  302   a  is described by way of example and other types of operation sections may be used. By way of example without limitation, a composite switch including a push switch with a ring-shaped four-direction operation section and a center switch may be used. By way of further example without limitation, an inclinable stick projecting from the top surface of housing  301  that outputs signals in accordance with the inclining direction of the stick may be used. By way of still further example without limitation, a horizontally slidable disc-shaped member that outputs signals in accordance with the sliding direction of the disc-shaped member may be used. By way of still further example without limitation, a touch pad may be used. By way of still further example without limitation, separate switches corresponding to at least four directions (e.g., front, rear, right and left) that output respective signals when pressed by a player may be used. 
         [0055]    Buttons (or keys)  302   b  through  302   g  are provided rearward of cross-switch  302   a  on the top surface of housing  301 . Buttons  302   b  through  302   g  are operation devices that output respective signals when a player presses them. For example, buttons  302   b  through  302   d  are respectively an “X” button, a “Y” button and a “B” button and buttons  302   e  through  302   g  are respectively a select switch, a menu switch and a start switch, for example. Generally, buttons  302   b  through  302   g  are assigned various functions in accordance with the application being executed by game console  100 . In an exemplary arrangement shown in  FIG. 3 , buttons  302   b  through  302   d  are linearly arranged along a front-to-back centerline of the top surface of housing  301 . Buttons  302   e  through  302   g  are linearly arranged along a left-to-right line between buttons  302   b  and  302   d . Button  302   f  may be recessed from a top surface of housing  701  to reduce the possibility of inadvertent pressing by a player grasping controller  107 . 
         [0056]    Button  302   h  is provided forward of cross-switch  302   a  on the top surface of the housing  301 . Button  302   h  is a power switch for remote on-off switching of the power to game console  100 . Button  302   h  may also be recessed from a top surface of housing  301  to reduce the possibility of inadvertent pressing by a player. 
         [0057]    A plurality (e.g., four) of LEDs  304  is provided rearward of button  302   c  on the top surface of housing  301 . Controller  107  is assigned a controller type (number) so as to be distinguishable from the other controllers used with game console  100  and LEDs may  304  may be used to provide a player a visual indication of this assigned controller number. For example, when controller  107  transmits signals to wireless controller module  240 , one of the plurality of LEDs corresponding to the controller type is lit up. 
         [0058]    With reference to  FIG. 3B , a recessed portion  308  is formed on a bottom surface of housing  301 . Recessed portion  308  is positioned so as to receive an index finger or middle finger of a player holding controller  107 . A button  302   i  is provided on a rear, sloped surface  308   a  of the recessed portion. Button  302   i  functions, for example, as an “A” button which can be used, by way of illustration, as a trigger switch in a shooting game. 
         [0059]    As shown in  FIG. 4 , an imaging element  305   a  is provided on a front surface of controller housing  301 . Imaging element  305   a  is part of an imaging information calculation section of controller  107  that analyzes image data received from markers  108   a  and  108   b . Imaging information calculation section  305  has a maximum sampling period of, for example, about 200 frames/sec., and therefore can trace and analyze even relatively fast motion of controller  107 . The techniques described herein of simulating the striking of an object can be achieved without using information from imaging information calculation section  305 , and thus further detailed description of the operation of this section is omitted. Additional details may be found in Application Nos. 60/716,937, entitled “VIDEO GAME SYSTEM WITH WIRELESS MODULAR HANDHELD CONTROLLER,” filed on Sep. 15, 2005; 60/732,648, entitled “INFORMATION PROCESSING PROGRAM,” filed on Nov. 3, 2005; and application No. 60/732,649, entitled “INFORMATION PROCESSING SYSTEM AND PROGRAM THEREFOR,” filed on Nov. 3, 2005. The entire contents of each of these applications are incorporated herein. 
         [0060]    Connector  303  is provided on a rear surface of controller housing  301 . Connector  303  is used to connect devices to controller  107 . For example, a second controller of similar or different configuration may be connected to controller  107  via connector  303  in order to allow a player to play games using game control inputs from both hands. Other devices including game controllers for other game consoles, input devices such as keyboards, keypads and touchpads and output devices such as speakers and displays may be connected to controller  107  using connector  303 . 
         [0061]    For ease of explanation in what follows, a coordinate system for controller  107  will be defined. As shown in  FIGS. 3 and 4 , a left-handed X, Y, Z coordinate system has been defined for controller  107 . Of course, this coordinate system is described by way of example without limitation and the systems and methods described herein are equally applicable when other coordinate systems are used. 
         [0062]    As shown in the block diagram of  FIG. 5 , controller  107  includes a three-axis, linear acceleration sensor  507  that detects linear acceleration in three directions, i.e., the up/down direction (Z-axis shown in  FIGS. 3 and 4 ), the left/right direction (X-axis shown in  FIGS. 3 and 4 ), and the forward/backward direction (Y-axis shown in  FIGS. 3 and 4 ). Alternatively, a two-axis linear accelerometer that only detects linear acceleration along each of the Y-axis and Z-axis may be used or a one-axis linear accelerometer that only detects linear acceleration along the Z-axis may be used. Generally speaking, the accelerometer arrangement (e.g., three-axis or two-axis) will depend on the type of control signals desired. As a non-limiting example, the three-axis or two-axis linear accelerometer may be of the type available from Analog Devices, Inc. or STMicroelectronics N.V. Preferably, acceleration sensor  507  is an electrostatic capacitance or capacitance-coupling type that is based on silicon micro-machined MEMS (micro-electromechanical systems) technology. However, any other suitable accelerometer technology (e.g., piezoelectric type or piezoresistance type) now existing or later developed may be used to provide three-axis or two-axis linear acceleration sensor  507 . 
         [0063]    As one skilled in the art understands, linear accelerometers, as used in acceleration sensor  507 , are only capable of detecting acceleration along a straight line corresponding to each axis of the acceleration sensor. In other words, the direct output of acceleration sensor  507  is limited to signals indicative of linear acceleration (static or dynamic) along each of the two or three axes thereof. As a result, acceleration sensor  507  cannot directly detect movement along a non-linear (e.g. arcuate) path, rotation, rotational movement, angular displacement, tilt, position, attitude or any other physical characteristic. 
         [0064]    However, through additional processing of the linear acceleration signals output from acceleration sensor  507 , additional information relating to controller  107  can be inferred or calculated (i.e., determined), as one skilled in the art will readily understand from the description herein. For example, by detecting static, linear acceleration (i.e., gravity), the linear acceleration output of acceleration sensor  507  can be used to determine tilt of the object relative to the gravity vector by correlating tilt angles with detected linear acceleration. In this way, acceleration sensor  507  can be used in combination with micro-computer  502  of controller  107  (or another processor) to determine tilt, attitude or position of controller  107 . Similarly, various movements and/or positions of controller  107  can be calculated through processing of the linear acceleration signals generated by acceleration sensor  507  when controller  107  containing acceleration sensor  307  is subjected to dynamic accelerations by, for example, the hand of a user, as will be explained in detail below. 
         [0065]    In another embodiment, acceleration sensor  507  may include an embedded signal processor or other type of dedicated processor for performing any desired processing of the acceleration signals output from the accelerometers therein prior to outputting signals to micro-computer  502 . For example, the embedded or dedicated processor could convert the detected acceleration signal to a corresponding tilt angle (or other desired parameter) when the acceleration sensor is intended to detect static acceleration (i.e., gravity). 
         [0066]    Returning to  FIG. 5 , image information calculation section  505  of controller  107  includes infrared filter  528 , lens  529 , imaging element  305   a  and image processing circuit  530 . Infrared filter  528  allows only infrared light to pass therethrough from the light that is incident on the front surface of controller  107 . Lens  529  collects and focuses the infrared light from infrared filter  528  on imaging element  305   a . Imaging element  305   a  is a solid-state imaging device such as, for example, a CMOS sensor or a CCD. Imaging element  305   a  captures images of the infrared light from markers  108   a  and  108   b  collected by lens  309 . Accordingly, imaging element  305   a  captures images of only the infrared light that has passed through infrared filter  528  and generates image data based thereon. This image data is processed by image processing circuit  520  which detects an area thereof having high brightness, and, based on this detecting, outputs processing result data representing the detected coordinate position and size of the area to communication section  506 . From this information, the direction in which controller  107  is pointing and the distance of controller  107  from display  101  can be determined. 
         [0067]    Vibration circuit  512  may also be included in controller  107 . Vibration circuit  512  may be, for example, a vibration motor or a solenoid. Controller  107  is vibrated by actuation of the vibration circuit  512  (e.g., in response to signals from game console  100 ), and the vibration is conveyed to the hand of the player holding controller  107 . Thus, a so-called vibration-responsive game may be realized. 
         [0068]    As described above, acceleration sensor  507  detects and outputs the acceleration in the form of components of three axial directions of controller  107 , i.e., the components of the up-down direction (Z-axis direction), the left-right direction (X-axis direction), and the front-rear direction (the Y-axis direction) of controller  107 . Data representing the acceleration as the components of the three axial directions detected by acceleration sensor  507  is output to communication section  506 . Based on the acceleration data which is output from acceleration sensor  507 , a motion of controller  107  can be determined. 
         [0069]    Communication section  506  includes micro-computer  502 , memory  503 , wireless module  504  and antenna  505 . Micro-computer  502  controls wireless module  504  for transmitting and receiving data while using memory  503  as a storage area during processing. Micro-computer  502  is supplied with data including operation signals (e.g., cross-switch, button or key data) from operation section  302 , acceleration signals in the three axial directions (X-axis, Y-axis and Z-axis direction acceleration data) from acceleration sensor  507 , and processing result data from imaging information calculation section  505 . Micro-computer  502  temporarily stores the data supplied thereto in memory  503  as transmission data for transmission to game console  100 . The wireless transmission from communication section  506  to game console  100  is performed at a predetermined time interval. Because game processing is generally performed at a cycle of 1/60 sec. (16.7 ms), the wireless transmission is preferably performed at a cycle of a shorter time period. For example, a communication section structured using Bluetooth (registered trademark) technology can have a cycle of 5 ms. At the transmission time, micro-computer  502  outputs the transmission data stored in memory  503  as a series of operation information to wireless module  504 . Wireless module  504  uses, for example, Bluetooth (registered trademark) technology to send the operation information from antenna  505  as a carrier wave signal having a specified frequency. Thus, operation signal data from operation section  302 , the X-axis, Y-axis and Z-axis direction acceleration data from acceleration sensor  507 , and the processing result data from imaging information calculation section  505  are transmitted from controller  107 . Game console  100  receives the carrier wave signal and demodulates or decodes the carrier wave signal to obtain the operation information (e.g., the operation signal data, the X-axis, Y-axis and Z-axis direction acceleration data, and the processing result data). Based on this received data and the application currently being executed, CPU  204  of game console  100  performs application processing. If communication section  506  is structured using Bluetooth (registered trademark) technology, controller  107  can also receive data wirelessly transmitted thereto from devices including game console  100 . 
         [0070]    The exemplary illustrative non-limiting system described above can be used to execute software stored on optical disk  104  or in other memory that controls it to interactive generate displays on display  101  of a progressively deformed object in response to user input provided via controller  107 . Exemplary illustrative non-limiting software controlled techniques for generating such displays will now be described. 
       Exemplary Localized Morphing 
       [0071]    According to one exemplary illustrative non-limiting implementation shown in  FIG. 6 , the developer of such software can, through a series of steps prepare a computer generated model for morphing and allow the end user to apply one or more localized morphs to achieve realistic in-game interactions with an object to be progressively deformed. In one exemplary illustrative non-limiting implementation, an object such as a robot (see  FIG. 13  and following) can be progressively dented by striking it repeatedly with a weapon such as a hammer. The techniques herein can be used to provide such progressive deformation in a computationally efficient and economic manner. 
         [0072]    According to one exemplary illustrative non-limiting implementation, the developer generates (block  101 ) an initial model and a final model of an object. The initial model is used to display a version of the object as the player earlier encounters the model, e.g., before the player has interacted with the object. The final model represents the object in a fully altered state—what the designer would have the model look like once the player has completely interacted with the model. 
         [0073]    As one example, if a player were going to strike an object with a hammer or other weapon repeatedly and dent the object in various places, then the initial model might be a smooth, unbroken, undented model and the final model might be a completely dented model, or at least dented to a degree that the developer desires. Other implementations could provide the initial model as being fully deformed and the final model as being fully undeformed if a different process (e.g., dent removal) was being performed on the object. The techniques herein are applicable to any process that changes the shape and/or appearance of an object in any way by deformation or any other shape or appearance altering process or other mechanism. 
         [0074]    In one exemplary illustrative non-limiting implementation, the software developer uses a 3D authoring tool program of conventional design to create the two models. Such models may be 3D models of conventional design constructed of polygons having vertices and textures. 
         [0075]    In addition to creating the models, the developer may generate (block  103 ) additional data that can be useful for locality based morphing. This data can include, for example:
       for each vertex in a model, a list of all triangles that reference the vertex;   for each triangle, a list of the three vertices that are referenced by that triangle; and   the normal of each triangle.       
 
         [0079]    The developer may save, in this data structure, the vertex normal indices for each vertex. The developer may also compare all of the vertex normals and flag the ones that are the same. See  FIG. 8  for a more detailed flowchart of an exemplary data structure generation process. This optimization can be used to simplify calculations during run time. According to this implementation, these processes, both the model creation  101  and the data generation  103 , may be done at authoring time prior to game initialization but in other implementations could be performed in whole or in part in real time. The results are stored on optical disk  104  or other memory. 
         [0080]    Upon game initialization, when the software is read into CPU  204  for execution, the software may control the  FIG. 1  system to generate an additional “draw” model ( 105 ) for display purposes. Initially, the “draw model” will start out as being a copy of the initial (e.g., undeformed or otherwise unaltered) model. This is the model that is displayed. An array may be generated (block  107 ) that will specify the degree of morphing between various corresponding vertices of the first and second models created at step  101 . 
         [0081]    The player may then play the game, and the process detailed in  FIG. 6  will wait until the player encounters an object to be affected. The game may, during this time, check (block  109 ) to see if the object has been interacted with or affected. For example, if the object is to be struck by a game character wielding a hammer, the game may continue to check  109  to see if the game character has in fact struck the object. 
         [0082]    Once the game character has interacted with the object, the game may then determine  111  the location and degree of affect the interaction will have on the object. For example, if the object is a large block that must be destroyed, then the game may display, at the appropriate time, a game character encountering a large, unbroken, undented block. If the character then strikes the upper left corner of the block with a hammer, the game, at step  111 , may determine that the game character has hit the upper left corner of the block with a certain degree of force. According to this implementation, once the area of effect and the degree of affect have been determined  111 , the game may then perform a localized morph of the geometry and texture of the “draw” model created at block  105  to interpolate both the geometry and the surface appearance (shading and texture) of the draw model between the initial (unaffected) model and the completely effected model. Such interpolation is localized in that only a selected subset of the vertices of the draw model are morphed. Selection of which vertices to morph is based in one exemplary illustrative implementation on conventional collision detection techniques indicating which portions of the object have been collided with or otherwise interacted with by other objects in the 3D world. 
         [0083]    In on exemplary illustrative non-limiting implementation, when the localized morph is performed, the appearance of the affected game object is changed based on the locality (position) of the interaction and the amount of force or other effect applied at that locality. For example, collision detection canb used to detect a medium powered strike with a hammer to a certain portion of an object. This may cause a corresponding area of that object located around the strike to partially or fully morph between the original model and the final, fully deformed model. Because the morph is a localized shift between an unaffected model and a fully deformed model, the player will get the impression that the object has been realistically affected in a manner consistent with user interaction. 
         [0084]    If the developer desires, the degree of morphing performed can be greater at the center point of the strike or other interaction and can trail off further out from the center of the strike. This can create a very realistic “dent” in the object. 
         [0085]      FIG. 7  shows an exemplary flow of a model creation process  101 . As previously noted, the game developer may want to both make  121  an initial model that shows an object in its unaffected state and make  123  a final model that shows an object in a completely altered state. If, for example, a game character is going to destroy a 3D car with a hammer, then the developer may want a car in pristine condition as the initial model. The final model can be a car with shattered or missing windows, dented paneling and flat tires. As the game character destroys the car with a hammer or other weapon, the initial model will morph, based on the strength of the strikes, the type of weapon, the location of the strikes, etc., into the dented model in the locality of the weapon strikes.  FIG. 14  illustrates an initial model in the form of an undeformed cube and a final model in the form of a fully deformed cube.  FIGS. 15 and 15A  shown an exemplary illustrative non-limiting example of an initial unaffected model, and  FIGS. 16 and 16A  show an exemplary illustrative non-limiting example of a fully affected (deformed) model.  FIGS. 16 and 16A  show an additional example, respectively. 
         [0086]    In one exemplary illustrative non-limiting implementation, localized morphing between two models is performed by correlating vertices of the two models and interpolating between the corresponding vertices in a spatially selective way. As the alteration process progresses over increasing portions of the surface area of the object, the appearance of the displayed object shifts in a spatially progressively way from the original model to the final model. 
         [0087]    According to a further illustrative exemplary non-limiting implementation, when a player affects an object to which a localized morph is to be applied, the program determines which coordinates have been affected and to what degree they have been affected. An exemplary flow for such a determination process  111 , is shown in  FIG. 9 . The program may first determine  109  whether or not the object has been affected using for example conventional collision detection or other algorithms. In an example where the game character is striking an object with a hammer, this would be a determination as to whether or not the game character has struck the object with a hammer. Once the program has determined  109  that the game character has affected the object, the program may determine  161  where the object was affected. In the example of a character wielding a hammer, the program would determine where the hammer strike fell. In addition to the affected coordinates, the program may transform  163  the affected coordinates into model space so they can be correlated with the models created as described above. Also, the program may determine  165  the degree of the affect caused by the game character&#39;s interaction with the object. For example, based on how hard a game character strikes an object with a hammer, the program may determine  165  how complete of a morph (percentage of interpolation) between the initial model and the final model to apply at that locale 
         [0088]    After calculating the locality and degree of change to an object based on player interaction or other effects, the game system will then in one exemplary illustrative non-limiting implementation apply the localized morph to the object.  FIG. 10  shows an exemplary flow of a localized morph application process  113 . According to an illustrative exemplary non-limiting implementation, the process may use the degree of affect determined at step  165  and apply  171  that value to the morph blend or interpolate values of affected vertices. According to this implementation, the affected vertices were previously determined at step  161 . Applying the degree of affect to the morph blend values allows the game to determine how complete of a morph should be performed for that part of the object. Once the degree of affect has been applied, the process may then morph  173  the affected vertices to the extent determined by the morph blend values. This morph will leave the affected portion of the object in a displayed state somewhere on a spectrum from the original model and the final model. Areas that have not been affected retain characteristics of the initial model. Areas that have been affected are progressively morphed toward the final model. The amount of interpolation between the initial and final models is determined in one exemplary illustrative implementation by the amount of force applied to the area. Some areas of the object may be completely morphed, other areas of the object may not be morphed at all (yet), and still other areas may be partially morphed. 
         [0089]    For each such progressive morphing transformation, the software may also update  175  the vertex normals, once the morph has been processed, to ensure that the lighting on the newly displayed model is still correct. If the object does not have any transparency, an alpha channel may be used to blend textures. Otherwise, texture blending may occur using other conventional means to provide well-known texture morphing as well as geometry morphing. 
         [0090]    According to this implementation, to update  175  the vertex normals to ensure appropriate lighting, the software may perform the steps in the exemplary normal update process  175  shown in  FIG. 11 . The program determines  181  whether or not there are any triangles remaining that have been changed by the morph. If there is a remaining triangle to which the correction steps have not been applied, the program calculates  183  the new normal for that triangle. Then, the program may calculate the angle between the original triangle normal and the newly calculated triangle normal. The program may also calculate the axis of rotation to go from the old triangle normal to the new triangle normal. Using the axis of rotation and the angle between the old and new triangle normals, the program may then rotate  189  the vertex normals for that triangle. This process can then be continued for all of the triangles that were affected by the morph. Other lighting/shading adjustment and blending algorithms can be used as desired. 
         [0091]    The exemplary illustrative non-limiting implementations allow games and other graphics display generation processes to selectively alter specific portions of models in reaction to player interaction with those models. Since the locality based morphing done by a game according to the exemplary illustrative non-limiting implementations can be dynamically determined and applied, the player can dynamically determine on an interactive basis what change is done to a model and where, and the player can, through different actions, have a different game experience each time the player interacts with a model. Meanwhile, the author has a great degree of freedom in determining how a player&#39;s interactions will affect a model. Interaction with a model while using different developer provided tools or implements may have different results. For example, a player striking a game object with a piece of wood may not do as much damage to the object as a player striking the same object with a metal hammer. 
         [0092]    One of the things that a developer may control on an object is the way that the morph is applied to the object at distances further out from the center point of the strike.  FIGS. 12A ,  12 B, and  12 C show some exemplary damage application curves suggesting several possible ways that a morph could be applied. 
         [0093]    For example,  FIG. 12A  shows damage being applied in a curve-like fashion out from the center of a point of impact  191 . The developer may wish to use a damage application like this if the player hits an object with a curved implement. Once example of an implement that might make such a dent would be a ball peen hammer. A baseball bat might also make such a dent in an object. The degree of morphing between the original varies over the length of the dent. An area  193  closer to the point of impact  191  has a greater degree of morphing applied to it than an area  195  further out from the point of impact  191 .  FIG. 12B , on the other hand, shows damage being applied equally at all affected portions of an impact point  197 . If a player struck a pliable material heavily with a solid, flat object, such as a mallet, then a dent like the one shown in  FIG. 9B  might be made. According to the exemplary transform of  FIG. 12B , the degree of morphing in an area  199  close to the point of impact  197  is the same as the degree of morphing at the affected area  201  farthest from the point of impact  197 .  FIG. 12C , as a third example, shows a wedge-like dent being made in an object, such as might occur if a player struck an object with an axe or a wedge. In this exemplary transform, like in  FIG. 12A , the degree of morphing in an area  205  closer to the point of impact  203  is greater than the degree of morphing further from the point of impact. Thus, in these examples, the localized morphing interpolation factors are determined by the shape of a virtual weapon used to interact with the object as well as by the location the virtual weapon strikes the object. Of course, it is not necessary to apply the damage in relation to the type of tool or weapon used. For example, the developer could use any other function than the ones shown to determine damage, the figures shown here are just three examples which suggest possible regular damage application transforms. 
         [0094]    In addition to determining how the damage is applied over the affected area, the developer may also need to determine how great the degree of change is at the initial point of impact. Using this determination, the developer can then apply a gradient of change out from that point based on transforms such as the exemplary transforms shown in  FIGS. 12A ,  12 B and  12 C.  FIG. 13  shows some exemplary determinations of degree of change at the point of impact based on power of a character&#39;s swing and a second factor. For example, the line that rapidly slopes upward  211  could be an example of the change applied at an impact point where a character is wielding a heavy hammer. Since there is a maximum amount of change that can be applied, that is, when the morph has fully switch over to the final image, a character with a heavy hammer may only need to strike an object with a medium degree of power to reach that point. A light tap with a hammer, such as shown at point  217 , may not do a lot of damage, but by the time the strike reaches 50% of the maximum power 100% of the change is being applied at the point of impact. Alternatively, say the game character is wielding a piece of wood with spikes. A light strike from a spiky piece such as shown at point  219  might do more damage than a light hammer strike, due to the spikes, but no matter how hard the game character swings the wood, it cause no more than 50% of a degree of change with a single strike as is shown by the line  213 . This could be because the wood breaks, because it is just not strong enough, etc. Of course, the designer may just want damage to uniformly equate to power, and a straight line  215  would provide this sort of calculation, where the more power the game character uses the more damage is done. Any determination of power to change may be used, and factors other than power may also be used to determine damage. For example, power could be replaced on the graph by strength of the tool being used, and then the line chosen could be determined by the malleability of the object being struck or affected. Line  211  would be the damage done by any tool used on a very malleable material, tin or lead, for example, such that any tool of 50% or greater of the maximum tool strength would cause maximum damage. Line  213 , on the other hand, would be a very resilient material, thick steel or diamond, for example, such that even the strongest tools would affect the material only a marginal amount. The game developer can apply any desired factors to achieve a determination of how the degree of change is calculated at the initial impact point. 
         [0095]    An additional detailed non-limiting example of how a method of the exemplary illustrative non-limiting implementations might be performed is as follows:
       1. Two models are created in a 3D Authoring Tool Program
           a. The original un-dented model   b. The final completely dented model   
           2. Generate some data used to perform locality based morphing
           a. For each vertex store all triangles that reference this vertex   b. For each triangle store the 3 vertices that will be referenced   c. For each triangle generate the normal   d. For each vertex save the vertex normal indices   e. For each vertex go through all the vertex normals and flag ones that are the same.   
           3. In-game at init create a new model that will be the drawn model and also create an internal array to say how much to morph between each vertex   4. During the game, if the object is hit, calculate where the object was hit and then transform this position into model space. Also, calculate how “hard” this object was hit. (this step does not have to be based on an object being “hit”, it could be based on other factors).   5. Next, take this position and damage amount and perform the locality based morph.
           a. Apply the damage amount to the morphing blend values for the specific vertices that will be affected.   b. Using the morphing blend values morph the affected vertices between the original un-dented model and the dented model.   c. Update the vertex normals so that the lighting will still be correct
               i. For each changed triangle calculate the new triangle normal.   ii. Calculate the angle between the original triangle normal and the new triangle normal   iii. Calculate the axis of rotation to go from the original triangle normal to the new triangle normal   iv. Using the axis of rotation and the angle between the two triangle normals rotate the vertex normals   
               
           d. Also, if the object doesn&#39;t contain any transparency, use the alpha component of the color to control a texture operation that will combine two textures (the original model&#39;s texture and the dented model&#39;s texture).       
 
         [0116]    While the technology herein has been described in connection with exemplary illustrative non-limiting embodiments, the invention is not to be limited by the disclosure. The invention is intended to be defined by the claims and to cover all corresponding and equivalent arrangements whether or not specifically disclosed herein.