Source: http://www.freepatentsonline.com/6256044.html
Timestamp: 2020-06-01 23:26:48
Document Index: 782060772

Matched Legal Cases: ['application No. 08', 'application No. 60', 'application No. 08', 'application No. 08', 'application No. 08', 'application No. 09', 'application No. 08', 'application No. 08', 'application No. 08', 'application No. 09', 'application No. 09', 'art 105']

United States Patent 6256044
Ensor, James Robert (Middletown, NJ)
09/098218
345/419, 345/473, 345/660, 345/672
G06T15/20; (IPC1-7): G06T11/00
345/419, 345/433, 345/428, 345/435, 345/438, 345/439, 345/113, 345/127, 345/131, 345/473
Download PDF 6256044 PDF help
U.S. application No. 08/938,304, Alfred Vaino Aho, filed Sep. 26, 1997.
U.S. application No. 60/055,384, Alfred Vaino Aho, filed Aug. 4, 1997.
U.S. application No. 08/869,056, Gianpaolo U. Carraro, filed Jun. 4, 1997.
U.S. application No. 08/906,214, Gianpaolo U. Carraro, filed Aug. 4, 1997.
U.S. application No. 08/906,215, Gianpaolo U. Carraro, filed Aug. 4, 1997.
U.S. application No. 09/107,566, Gianpaolo U. Carraro, filed Jun. 29, 1998.
U.S. application No. 08/405,062, Sudhir Raman Ahuja, filed Mar. 16, 1995.
U.S. application No. 08/052,492, Sudhir Raman Ahuja, filed Apr. 22, 1993.
U.S. application No. 08/906,216, James Robert Ensor, filed Aug. 4, 1997.
U.S. application No. 09/107,059, John T. Edmark, filed Jun. 30, 1998.
U.S. application No. 09/108,029, John T. Edmark, filed Jun. 30, 1998.
DOCKET ADMINISTRATOR (RM 3C-5120 (LUCENT TECHNOLOGIES INC 600 MOUNTAIN AVENUE P O BOX 636, MURRAY HILL, NJ, 079740636, US)
A limitation of such a world occurs when an object represented by computer graphics undergoes a trajectory that takes it to a location in the world that is not represented as computer graphics but instead is within the field represented by the video. If the object is generated at its proper location using computer graphic techniques, without further processing such an object becomes invisible to a viewer in front of the video screen, because the object is blocked from a viewer's view by the video screen. In U.S. patent application Ser. No. 08/906215 it was recognized that, when a computer graphic object undergoes a trajectory that takes it to a location in the world that is not represented as computer graphics, but instead is within the field represented by the video, that such an object should be represented as video on the screen, rather than computer graphics. Doing so prevents the object from becoming invisible as it would if it was generated at its proper location using computer graphic techniques, since it would be blocked from view by the video screen. Thus, the computer graphics object "goes into the video" as video and remains visible to a viewer in front of the video screen.
We have recognized that when a computer graphic object is located at, or undergoes a trajectory that takes it to, a location in the world that is not represented as computer graphics, but instead is within a field represented by video, one may wish to maintain the object as computer graphics, yet still have the object be visible to a viewer and appear as if the object was at its proper location within the video. This can be achieved, in accordance with the principles of the invention, by a) determining when a computer graphic object is located at, or undergoes a trajectory that takes it to, a location in the world that is not represented as computer graphics, but instead is within the field represented by the video, and b) representing such an object as a computer graphic which is in front of the screen and to which at least one transformation, e.g., a perspective transformation, is applied. The perspective transformation may be at least a function of the distance beyond the video screen at which the object would actually be located if it were to be drawn using the same representation employed prior to its crossing the video screen. In other words, instead of the computer graphics object "going into the video" as video, the object as computer graphics is made to appear as if it went into the video by using perspective techniques. Thus, the object remains visible to a viewer in front of the video screen rather than becoming invisible because it is blocked from view by the video screen as would have occurred had the object been generated at its proper location.
Note that the viewpoint of the user is actually behind the representation of both bicyclists 107 and 109. Also note that bicyclist 107 is moving slower than bicyclist 109. If this continues, eventually, bicyclist 109 will reach screen 103 while bicyclist 107 lags behind. This occurs in FIG. 2, which shows world 101 of FIG. 1, but at a later time and from a side perspective. If the relative motion difference continues between bicyclists 107 and 109, part, and then all, of bicyclist 109 will intersect with, or go past, video screen 103. If this happens, part, and then all, of bicyclist 109 will become occluded by video screen 103 from the view of a user in front of video screen 103, and so bicyclist 109 will no longer be visible to such a user. Therefore, in accordance with the principles of the invention, the appearance of bicyclist 109 is transformed into transformed bicyclist 309, e.g., using perspective techniques, so that it gives the illusion that bicyclist 309 appears to be within the video that is displayed on video screen 103 rather than being represented using the same appearance as is used for bicyclist 109 within CG Part 105, as shown in FIG. 3. Note that by perspective techniques it is meant techniques that apply at least one of scaling, translation, and orientation manipulation to the bicyclist. Also note that transformed bicyclist 309 is referred to as a so-called "trace" of bicyclist 109.
A corresponding side view of the world as it appears in FIG. 3 is shown in FIG. 4. Also shown in FIG. 4 is "ghost" bicyclist 409, which represents the actual location and appearance of bicyclist 109 from which transformed bicyclist 309 is generated. Although not done so for FIG. 4, which is merely for pedagogical purposes, preferably, but not necessarily, both transformed bicyclist 309 and the untransformed bicyclist 409 should not visible to a viewer at the same time.
As described in connection with FIGS. 1-4, the transformed representation, e.g., bicyclist 301, is displayed immediately adjacent to video screen 103. However, doing so is not necessary. Instead, the transformed representation may be placed anywhere between video screen 103 and the viewer's viewpoint--provided the transformation is properly conformed to the location of the transformed object. However, an advantage of placing the transformed representation adjacent to video screen 103 is that by doing so any items that are between video screen 103 and the view's viewpoint that should occlude a portion of the transformed object will do so. Otherwise, if the transformed object is placed closer to the user's viewpoint, although the object will look correct to a viewer, objects between the location of the transformed object and the screen will not occlude the transformed object as they would if the untransformed object could be rendered and remain visible at its actual expected location.
If the test result in step 605 is NO, control passes to step 607 in which the object is added to a so-called "display set" for use in step 503. Control next passes to conditional branch point 609, which tests to determine if there remains more unchecked objects. If the test result in step 609 is YES, control passes back to step 603 to check the next object. If the test result in step 609 is NO, control passes to step 611 and the process is exited.
If the test result in step 605 is YES, indicating that the object has passed beyond the video screen and will otherwise become occluded from the viewer's view, a process is undertaken in steps 613 through 621 to develop the transformed representation of the object, in accordance with the principles of the invention. More specifically, in step 613, a duplicate of the object is developed, and in step 615 a reference point for the object is selected. The point chosen is at the discretion of the implementer. It may correspond to the predesignated point of object that was employed in step 605. Typically, the point chosen will--although it is not required to--be within the bounding box of the object. The remaining steps are more easily understood in connection with the example shown in FIG. 7.
<- Previous Patent (Three dimensional vi...) | Next Patent (Device and method fo...) ->