Source: http://www.google.com/patents/US6396491?ie=ISO-8859-1&dq=6859936
Timestamp: 2014-04-16 14:37:50
Document Index: 384712597

Matched Legal Cases: ['art, 202', 'art, 203', 'art, 204', 'art, 205', 'art, 206', 'art, 207', 'art, 208', 'art, 209', 'art, 210', 'art 207', 'art 208', 'art, 302', 'art, 303', 'art, 304', 'art, 305', 'art, 306', 'art, 307', 'art, 308', 'art 307']

Patent US6396491 - Method and apparatus for reproducing a shape and a pattern in a three ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA method and apparatus for reproducing a shape and a pattern in a three-dimensional scene with good precision, based on a plurality of images, by realizing processing that considers a three-dimensional model from the beginning of the processing. A plurality of images are captured, a plurality of frames...http://www.google.com/patents/US6396491?utm_source=gb-gplus-sharePatent US6396491 - Method and apparatus for reproducing a shape and a pattern in a three-dimensional sceneAdvanced Patent SearchPublication numberUS6396491 B2Publication typeGrantApplication numberUS 09/781,328Publication dateMay 28, 2002Filing dateFeb 13, 2001Priority dateAug 20, 1998Fee statusPaidAlso published asUS20010005425Publication number09781328, 781328, US 6396491 B2, US 6396491B2, US-B2-6396491, US6396491 B2, US6396491B2InventorsMasaki Watanabe, Shuichi Shiitani, Masaaki Oota, Susumu Endo, Hiroaki HaradaOriginal AssigneeFujitsu LimitedExport CitationBiBTeX, EndNote, RefManPatent Citations (8), Non-Patent Citations (4), Referenced by (8), Classifications (6), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetMethod and apparatus for reproducing a shape and a pattern in a three-dimensional sceneUS 6396491 B2Abstract A method and apparatus for reproducing a shape and a pattern in a three-dimensional scene with good precision, based on a plurality of images, by realizing processing that considers a three-dimensional model from the beginning of the processing. A plurality of images are captured, a plurality of frames to which processing is applied are selected from the captured images, correspondence between a plurality of images are extracted, a three-dimensional model to be a base is input, a three-dimensional model that defines geometric properties in a target object in the images is selected, correspondence between the images and the three-dimensional model is specified, the three-dimensional model is deformed while satisfying both the correspondence between the images and the correspondence between the images and the three-dimensional model, a pattern image to be attached to a surface is generated based on a shape of the three-dimensional model and the correspondence between the three-dimensional model and the images, and the final three-dimensional model is output.
What is claimed is: 1. A method for reproducing a shape and a pattern in a three-dimensional scene forming a three-dimensional model used for computer graphics based on a plurality of images, comprising:
capturing the plurality of images, storing the captured images, selecting a plurality of frames to which processing is applied from the stored images, and extracting correspondence between the plurality of images; inputting the three-dimensional model to be a base, storing the input three-dimensional model, selecting the three-dimensional model that defines geometric properties of a target object in the images, and specifying correspondence between the images and the three-dimensional model; deforming the three-dimensional model while satisfying both the obtained correspondence between the images and the obtained correspondence between the images and the three-dimensional model; generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained by the deforming, and the specified correspondence between the final three-dimensional model and the images; and outputting the completed final three-dimensional model. 2. A method for reproducing a shape and a pattern in a three-dimensional scene according to claim 1, wherein, in the extracting of the correspondence between the plurality of images, a user can specify the correspondence.
capturing the plurality of images, storing the captured images, and selecting a plurality of frames to which processing is applied from the stored images; inputting the three-dimensional model to be a base, storing the input three-dimensional model, selecting the three-dimensional model that defines geometric properties of a target object in the images, and specifying correspondence between the images and the three-dimensional model; deforming the three-dimensional model while investigating portions, between the images, which satisfy the obtained correspondence between the images and the three-dimensional model; generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained by the deforming, and the specified correspondence between the final three-dimensional model and the images; and outputting the completed final three-dimensional model. 7. A method for reproducing a shape and a pattern in a three-dimensional scene according to claim 6, wherein, in the deforming of the three-dimensional model and in the generating of a pattern image to a surface of the three-dimensional model, a user can specify a partial change in geometric properties of the three-dimensional model.
an image input part for capturing the plurality of images; an image storing part for storing the images captured from the image input part; an image selecting part for selecting a plurality of frames to which processing is applied from the images stored in the image storing part; an inter-image correspondence obtaining part for extracting correspondence between the plurality of images; a three-dimensional model input part for inputting the three-dimensional model to be a base; a three-dimensional model storing part for storing the three-dimensional model input from the three-dimensional mode input part; a three-dimensional model selecting part for selecting the three-dimensional model that defines geometric properties of a target object in the images; an image�three-dimensional model correspondence obtaining part for specifying correspondence between the images and the three-dimensional model; a three-dimensional information reproducing part for deforming the three-dimensional model while satisfying both the correspondence between the images obtained in the inter-image correspondence obtaining part and the correspondence between the images and the three-dimensional model obtained in the image�three-dimensional model correspondence obtaining part; a pattern image generating part for generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained in the three-dimensional information reproducing part, and the specified correspondence between the final three-dimensional model and the images; and a generated model output part for outputting the completed final three-dimensional model. 10. An apparatus for reproducing a shape and a pattern in a three-dimensional scene according to claim 9, wherein, in the inter-image correspondence obtaining part, a user can specify the correspondence.
an image input part for capturing the plurality of images; an image storing part for storing the images captured from the image input part; an image selecting part for selecting a plurality of frames to which processing is applied from the images stored in the image storing part; a three-dimensional model input part for inputting the three-dimensional model to be a base; a three-dimensional model storing part for storing the three-dimensional model input from the three-dimensional mode input part; a three-dimensional model selecting part for selecting the three-dimensional model that defines geometric properties of a target object in the images; an image�three-dimensional model correspondence obtaining part for specifying correspondence between the images and the three-dimensional model; a part for simultaneously executing investigation of inter-image correspondence and reproduction of three-dimensional information for deforming the three-dimensional model while investigating portions, between the images, which satisfy the correspondence between the images and the three-dimensional model obtained in the image�three-dimensional model correspondence obtaining part; a pattern image generating part for generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained in the part for simultaneously executing investigation of inter-image correspondence and reproduction of three-dimensional information, and the specified correspondence between the final three-dimensional model and the images; and a generated model output part for outputting the completed final three-dimensional model. 15. An apparatus for reproducing a shape and a pattern in a three-dimensional scene according to claim 14, wherein, in the part for simultaneously executing investigation of inter-image correspondence and reproduction of three-dimensional information, and in the pattern image generating part, a user can specify a partial change in geometric properties of the three-dimensional model.
capturing the plurality of images; storing the captured images; selecting a plurality of frames to which processing is applied from the stored images; extracting correspondence between the plurality of images; inputting the three-dimensional model to be a base; storing the input three-dimensional model; selecting the three-dimensional model that defines geometric properties of a target object in the images; specifying correspondence between the images and the three-dimensional model; deforming the three-dimensional model while satisfying both the obtained correspondence between the images and the obtained correspondence between the images and the three-dimensional model; generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained by the deforming, and the specified correspondence between the final three-dimensional model and the images; and outputting the completed final three-dimensional model.
This application is a continuing application, filed under 35 U.S.C. �111( a), of International Application PCT/JP99/03437, filed Jun. 25, 1999, it being further noted that priority is based upon Japanese Patent Application 10-234759, filed Aug. 20, 1998.
TECHNICAL FIELD The present invention relates to processing for reproducing a partial or entire three-dimensional scene, in which a plurality of digitized videos or animation appear, into a three-dimensional shape, and converting the three-dimensional shape into a three-dimensional model to be displayed in computer graphics.
BACKGROUND ART In order to reproduce a three-dimensional shape by using a three-dimensional scene that is a two-dimensional image, in which a three-dimensional object such as a plurality of videos or animation appears, it is required to appropriately grasp the correspondence between a plurality of videos or between a plurality of frames (hereinafter, referred to as �between images�).
DISCLOSURE OF INVENTION The present invention overcomes the above-mentioned problem, and its object is to provide a method and apparatus for realizing processing in which a three-dimensional model is considered from the beginning of the processing in reproduction of a shape and a pattern in a three-dimensional scene based on a plurality of images, and reproducing the shape and the pattern with good precision.
Next, in order to solve the above-mentioned problem, the apparatus for reproducing a shape and a pattern in a three-dimensional scene of the present invention is an apparatus for reproducing a shape and a pattern in a three-dimensional scene forming a three-dimensional model used for computer graphics based on a plurality of images, including: an image input part for capturing the plurality of images; an image storing part for storing the images captured from the image input part; an image selecting part for selecting a plurality of frames to which processing is applied from the images stored in the image storing part; an inter-image correspondence obtaining part for extracting correspondence between the plurality of images; a three-dimensional model input part for inputting the three-dimensional model to be a base; a three-dimensional model storing part for storing the three-dimensional model input from the three-dimensional mode input part; a three-dimensional model selecting part for selecting the three-dimensional model that defines geometric properties of a target object in the images; an image�three-dimensional model correspondence obtaining part for specifying correspondence between the images and the three-dimensional model; a three-dimensional information reproducing part for deforming the three-dimensional model while satisfying both the correspondence between the images obtained in the inter-image correspondence obtaining part and the correspondence between the images and the three-dimensional model obtained in the image�three-dimensional model correspondence obtaining part; a pattern image generating part for generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained in the three-dimensional information reproducing part, and the specified correspondence between the final three-dimensional model and the images; and a generated model output part for outputting the completed final three-dimensional model.
Next, in order to solve the above-mentioned problem, the apparatus for reproducing a shape and a pattern in a three-dimensional scene of the present invention is an apparatus for reproducing a shape and a pattern in a three-dimensional scene forming a three-dimensional model used for computer graphics based on a plurality of images, including: an image input part for capturing the plurality of images; an image storing part for storing the images captured from the image input part; an image selecting part for selecting a plurality of frames to which processing is applied from the images stored in the image storing part; a three-dimensional model input part for inputting the three-dimensional model to be a base; a three-dimensional model storing part for storing the three-dimensional model input from the three-dimensional mode input part; a three-dimensional model selecting part for selecting the three-dimensional model that defines geometric properties of a target object in the images; an image�three-dimensional model correspondence obtaining part for specifying correspondence between the images and the three-dimensional model; a part for simultaneously executing investigation of inter-image correspondence and reproduction of three-dimensional information for deforming the three-dimensional model while investigating portions, between the images, which satisfy the correspondence between the images and the three-dimensional model obtained in the image�three-dimensional model correspondence obtaining part; a pattern image generating part for generating a pattern image to be attached to a surface of the three-dimensional model, based on a shape of the final three-dimensional model obtained in the part for simultaneously executing investigation of inter-image correspondence and reproduction of three-dimensional information, and the specified correspondence between the final three-dimensional model and the images; and a generated model output part for outputting the completed final three-dimensional model.
BRIEF DESCRIPTION OF DRAWINGS FIG. 1 is a flow chart of processing in a conventional method for reproducing a shape and a pattern in a three-dimensional scene.
BEST MODE FOR CARRYING OUT THE INVENTION (Embodiment 1) Hereinafter, an apparatus for reproducing a shape and a pattern in a three-dimensional scene in Embodiment 1 of the present invention will be described with reference to the drawings. FIG. 2 is a block diagram of an apparatus for reproducing a shape and a pattern in a three-dimensional scene in Embodiment 1 of the present invention.
In FIG. 2, reference numeral 201 denotes an image input part, 202 denotes an image storing part, 203 denotes an image selecting part, 204 denotes a three-dimensional model input part, 205 denotes a three-dimensional model storing part, 206 denotes a three-dimensional model selecting part, 207 denotes an inter-image correspondence obtaining part, 208 denotes an image�three-dimensional model correspondence obtaining part, 209 denotes a three-dimensional information reproducing part, 210 denotes a pattern image creating part, and 211 denotes a generated model output part, respectively.
In the inter-image correspondence obtaining part 207, the user specifies portions corresponding to each other between a plurality of images, and in the image�three-dimensional model correspondence obtaining part 208, the user also specifies portions corresponding to each other between each image and the three-dimensional model. For example, this corresponds to that the user specifies the identical position determined based on a pattern, fine-adjusts the positions of vertexes in accordance with the three-dimensional model, and the like.
When the perspective projection process is expressed by a mathematical expression, vertexes (x, y, z) of a three-dimensional model are projected onto points (r, c) on an image as expressed by Equation 1. ( r c ) = ( f  Y Z f  X Z ) , ( X Y Z ) = A   ( x y z ) + 0 ( 1 ) Where r and c are a row number and a column number of an image, respectively, A is a rotating matrix of 3�3, and O is an origin vector representing movement on a screen. Thus, a rotation matrix is expressed by three rotation angles and the order of the origin vector is 3; therefore, it is clear that r and c are composed of three variables. Further, f represents a focal length.
F(x k ,y k ,z k ;f 1h.1,A1h,1,O1k,1)=(r k,1 ,c k,1)(k=0, . . . , l−1), F(x k ,y k ,z k ;f 1h 2,A1h.2,O1h 2)=(rk,2 , c k,2)(k=0, . . . , l−1) (2) Further, as restricting conditions, assuming that an equation of an S-th plane is expressed as S (x, y, z; s), Equation 3 holds.
S(x k , y k ,z k ;s k)=0(k=0, . . . , l−1) (3) By solving the above-mentioned equations, shape parameters of the three-dimensional model can be determined, and incidentally, a projection equation of a three-dimensional mode to an image frame, and coordinates of the corresponding point can be obtained.
Embodiment 2 Embodiment 2 of the present invention will be described with reference to the drawings.
FIG. 10 is a block diagram of an apparatus for reproducing a shape and a pattern in a three-dimensional scene in Embodiment 2 of the present invention. In FIG. 10, reference numeral 301 denotes an image input part, 302 denotes an image storing part, 303 denotes an image selecting part, 304 denotes a three-dimensional model input part, 305 denotes a three-dimensional model storing part, 306 denotes a three-dimensional model selecting part, 307 denotes an image�three-dimensional model correspondence obtaining part, 308 denotes a part for simultaneously executing investigation of inter-image correspondence and reproduction of three-dimensional information, 309 denotes a pattern image creating part, and 310 denotes a generated model output part, respectively.
In the image�three-dimensional model correspondence obtaining part 307, the user specifies portions corresponding to each other between the selected respective images and the three-dimensional model. For example, this corresponds to that the user specifies the identical position determined based on a pattern, fine-adjusts the position of a vertex in accordance with the three-dimensional model, and the like.
In the case where a point (ri, ci) in the i-th (0≦i≦n−1) frame is present on a plane s, coordinates (x, y, z) on the three-dimensional model can be obtained by solving Simultaneous Equations 4. { ( r i c i ) = ( f i   Y Z f i   X Z ) , ( X Y Z ) = A i   ( x y z ) + O i , S  ( x ,  y ,  z ;  s )  =  0 ( 4 ) Further, an image is projected onto a point (rj, cj) that satisfies Simultaneous Equations 5 in a j-th (j≠i, 0≦j≦n−1) frame. { ( r j c j ) = ( f j   Y Z f j   X Z ) , ( X Y Z ) = A j   ( x y z ) + O j , ( 5 ) Where, it is assumed that a color in the i-th frame (r, c) is expressed by a vector Ii (r, c), a matching condition of the previous pattern can be expressed by Ii (ri, ci)=C (x, y, z; ai), Ij (rj, cj)=C (x, y, z; aj). By setting these equations with respect to all the points in a range contained in a three-dimensional model on a screen, simultaneous equations can be formulated. Then, by solving these simultaneous equations simultaneously with Simultaneous Equations 2, the following unknown numbers of a three-dimensional model and a projection can be obtained without determining the correspondence between the respective images.
INDUSTRIAL APPLICABILITY As described above, according to the method and apparatus for reproducing a shape and a pattern in a three-dimensional scene of the present invention, the shape of a three-dimensional model can be deformed while the consistency of a pattern on the surface of the three-dimensional model is being taken into consideration. Therefore, it becomes possible to reproduce a three-dimensional model that is close to an entity of a three-dimensional scene.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5727093Feb 6, 1996Mar 10, 1998Canon Kabushiki KaishaImage processing method and apparatus thereforUS6256036 *Sep 8, 1998Jul 3, 2001Sanyo Electric Co., Ltd.Three-dimensional modeling apparatus for producing three-dimensional model of object of interest and method thereforUS6304263 *Sep 4, 1998Oct 16, 2001Hyper3D Corp.Three-dimensional display system: apparatus and methodUS6331851 *May 15, 1998Dec 18, 2001Matsushita Electric Industrial Co., Ltd.Graphic display apparatus, synchronous reproduction method, and AV synchronous reproduction apparatusUS6342887 *Nov 18, 1998Jan 29, 2002Earl Robert MunroeMethod and apparatus for reproducing lighting effects in computer animated objectsJPH0969170A Title not availableJPH0997344A Title not availableJPH09147147A Title not available* Cited by examinerNon-Patent CitationsReference1"Latest CG Software Purchase Guide '97 to '98", Oct. 1997.2"Nikkei Computer Graphics", Mar. 1997.3Hotoge, Keiichi et al., "System Applied VRML Technology" in Fujitsu, Mar. 1997.4Watanabe, Masaki et al., "User Supporting Function of Sketch Vision, CG Modeling Tool Based on Video-Images in Graphics and CAD Symposium", Fujitsu Laboratories Ltd., Oct. 1995.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS6580429 *Aug 16, 2000Jun 17, 2003Mitsubishi Denki Kabushiki KaishaMethod of generating data for three-dimensional graphic recording medium and three-dimensional graphic systemUS6819318 *Jul 17, 2000Nov 16, 2004Z. Jason GengMethod and apparatus for modeling via a three-dimensional image mosaic systemUS6954217 *Jun 30, 2000Oct 11, 2005Pentax CorporationImage processing computer system for photogrammetric analytical measurementUS7289662 *Dec 8, 2003Oct 30, 2007Hrl Laboratories, LlcMethod and apparatus for apparatus for generating three-dimensional models from uncalibrated viewsUS8078436Oct 16, 2008Dec 13, 2011Eagle View Technologies, Inc.Aerial roof estimation systems and methodsUS8145578Apr 17, 2008Mar 27, 2012Eagel View Technologies, Inc.Aerial roof estimation system and methodUS8170840May 15, 2009May 1, 2012Eagle View Technologies, Inc.Pitch determination systems and methods for aerial roof estimationUS8209152May 15, 2009Jun 26, 2012Eagleview Technologies, Inc.Concurrent display systems and methods for aerial roof estimation* Cited by examinerClassifications U.S. Classification345/419International ClassificationG06T17/00, G06T11/00, G06T1/00Cooperative ClassificationG06T17/00European ClassificationG06T17/00Legal EventsDateCodeEventDescriptionJan 3, 2014REMIMaintenance fee reminder mailedOct 28, 2009FPAYFee paymentYear of fee payment: 8Nov 4, 2005FPAYFee paymentYear of fee payment: 4Feb 13, 2001ASAssignmentOwner name: FUJITSU LIMITED, JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WATANABE, MASAKI;SHIITANI, SHUICHI;OOTA, MASAAKI;AND OTHERS;REEL/FRAME:011555/0929Effective date: 20010205Owner name: FUJITSU LIMITED NAKAHARA-KU, KAWASAKI-SHI 1-1, KAMFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WATANABE, MASAKI /AR;REEL/FRAME:011555/0929RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google