Source: http://www.google.com/patents/US6593938?dq=5,371,548
Timestamp: 2016-09-29 12:09:49
Document Index: 543033685

Matched Legal Cases: ['art 304', 'art 307', 'art 304', 'art 307', 'art 304', 'art 401', 'art 402', 'art 403', 'art 404', 'art 104', 'art 304', 'art 2104', 'art 2304', 'art 2303', 'art 304', 'art 307', 'art 304', 'art 307', 'art 304', 'art 2401', 'art 2402', 'art 2403', 'art 2404', 'art 1802']

Patent US6593938 - Image processing apparatus, method and computer-readable recording medium ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsAn image processing apparatus is provided with a display device to display images on a display screen so that the images displayed on the display device can be edited and processed for joining of the images. The image processing apparatus includes an image specifying device to specify two or more images...http://www.google.com/patents/US6593938?utm_source=gb-gplus-sharePatent US6593938 - Image processing apparatus, method and computer-readable recording medium with program recorded thereon, for joining images together by using visible joining points and correcting image distortion easilyAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS6593938 B1Publication typeGrantApplication numberUS 09/290,553Publication dateJul 15, 2003Filing dateApr 12, 1999Priority dateApr 10, 1998Fee statusPaidAlso published asCN1214614C, CN1238637A, DE19916349A1, DE19916349B4Publication number09290553, 290553, US 6593938 B1, US 6593938B1, US-B1-6593938, US6593938 B1, US6593938B1InventorsNorihiko Sakata, Toshinori Takaki, Minoru HasegawaOriginal AssigneeRicoh Company, Ltd.Export CitationBiBTeX, EndNote, RefManPatent Citations (9), Referenced by (92), Classifications (6), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetImage processing apparatus, method and computer-readable recording medium with program recorded thereon, for joining images together by using visible joining points and correcting image distortion easily
US 6593938 B1Abstract
An image processing apparatus is provided with a display device to display images on a display screen so that the images displayed on the display device can be edited and processed for joining of the images. The image processing apparatus includes an image specifying device to specify two or more images to be joined together on the display device. An image arranging device arranges the images specified by the image specifying device in such an order as to join the images together, and a display control device controls the display device to display the images arranged by the image arranging device. A joining point specifying device specifies any one joining point for each image so that vertically or laterally adjacent images displayed by the display control device can be joined together by referring to the joining points. A joining device joins adjacent images together by referring to the joining points specified by the joining point specifying device. This makes it possible to easily join the images together by such a way as to merely specify one joining point for each image, and hence to execute image joining easily and efficiently.
It is known that an image photographed by a digital still camera or video camera generally contains geometrical distortion produced by distortion aberration of the lens system forming the image in a position deviated from the original position. To correct distortion of such an image, a method is known such as one disclosed in Japanese patent application laid-open publication No. 9-294225, entitled “Method for Deciding Parameter for Image Distortion Correction and Image Pickup Device.” This publication discloses a technique in which plural images including a common pattern picked up from one point are used to detect plural sets of corresponding observing points from these images and measure angles of the observation points with respect to an optical axis of the image pickup system, thus estimating a parameter for distortion correction based on the angle information obtained.
In order to solve the above conventional problems, an object of the present invention is to provide an image processing apparatus and method capable of joining plural images in an easy operation even for beginners, and a computer-readable recording medium with a program recorded thereon for making the computer execute the method.
FIG. 1 is an illustration functionally showing the general structure of an illustrated document creating system including an image processing apparatus according to the present invention commonly used for joining plural images together and correcting distortion of the images;
Referring to appended drawings, the present invention will be described in detail below with respect to preferred embodiments of an image processing apparatus, an image processing method and a computer-readable recording medium with a program recorded thereon for making the computer execute the method.
In the embodiment in which plural picture images are edited into a picture, the arrangements of the image altering part 304 and the display control part 307 are first described in more detail. FIG. 4 is a functional block diagram showing the arrangement of the image altering part 304, the display control part 307 and the surroundings in the image processing apparatus according to the first embodiment. As shown in FIG. 4, the image altering part 304 includes an image specifying part 401, an image arranging part 402, a joining point specifying part 403 and a joining part 404.
If a predetermined key word is registered for each image data file, such as “character” or “landscape,” “business” or “private”, the reduced images can be displayed by arranging the display sequence of the reduced images based on the key words. Since the reduced images are arranged based on the key words, a desired image data file can be retrieved at high speed even if the number of registered image data files is large.
After specifying the reduced images, the operator moves the cursor to an “IMAGE CORRECTION” menu 604 on the menu bar in FIG. 6, clicks the button of the mouse 212 or the like, and further clicks an “IMAGE JOINING” menu 605 displayed as one of pull-down menus to select the image joining function. Then, the joining window 700 is popped up and displayed as shown in FIG. 7.
FIG. 12 is an illustration for explaining an example of such a window displayed on the display part of the image processing apparatus according to the first embodiment. In FIG. 12, if an “yes” button 1201 is pressed, the image resulting from the joining processing is assigned a file name and stored into the image data storage part 104. On the other hand, if a “no” button 1202 is pressed, the joining result is canceled and the operating procedure returns to the display screen just before the start button 706 is pressed, so that joining points can be specified again.
Although in the first embodiment one joining point is specified at a point, i.e., by clicking the button of the mouse 212 or the like, it may be specified using a line, i.e., by specifying the joint between adjacent images in a drag-and drop operation of the mouse 212 or the like, as discussed in the following second embodiment.
The set of joining points already specified may be released when a new set of joining points is specified for a pair of adjacent images, as described in the following third embodiment. Since the general structure of the illustrated document creating system including an image processing apparatus according to the third embodiment of the present invention, and the hardware configuration of the image processing apparatus 100 are the same as those of the first embodiment, description thereof is omitted here. Further, since the image processing apparatus 100 includes the same parts as those of the first embodiment except the image altering part 304, the parts common to those of the first embodiment are also not described here.
Although the above first to third embodiments have been made for images photographed with a digital still camera 213 having a fixed focal length, i.e., these embodiments assume that the focal length is invariable, the focal length may be set for proper joining of images even if the images have been photographed with a digital still camera 213 having variable focal lengths, as discussed in the following fourth embodiment.
The zoom switching part 2104 is to switch the mode related to the zoom depending on whether the image to be joined has been photographed in the zoom mode or not. When no zoom is used for the photograph. the mode is switched to “standard,” i.e., the standard focal length for the specified camera. On the other hand, when the zoom is used for the photograph, the mode is switched to “maximum,” i.e., maximum zoom for the camera specified.
Since the above first to fourth embodiments assume a case where a maximum of three images can be arranged both in the vertical direction and in the lateral direction, such a number of images (two, three, four, six or nine) as to shape the joined image into a square or rectangle can only be joined together. However, the number of images to be arranged vertically or laterally can be so increased that further different combinations of images (such as eight images, arranged two in length and four in width, or four in length and two in width) can be arranged. This embodiment is discussed below on the assumption that such a number of images as to make the joined image into a rectangular shape can be set, but the present invention is not limited by these cases, and such combinations of images as to form unique shapes other than rectangles may be set. Further, plural images such as four images, five images, six images, seven images, eight images, nine images and so on may be so joined that a panorama of landscape type can be formed.
Selectable zoom switching alternatives related to whether an image to be joined with another has been photographed in the zoom mode or not are displayed on the zoom switching part 2304. When no zoom is used for the photograph, the mode is switched to “standard” 2313. On the other hand, when the zoom is used for the photograph, the mode is switched to “maximum” 2314. The zoom mode is thus switched.
FIG. 25a shows displayed modes on the arrangement switching part 2303 when two images are specified. When two images are specified, there are two arrangement modes, a mode for arranging the two images laterally (2 laterally�1 vertically), and a mode for arranging the two images vertically (1 laterally�2 vertically). FIG. 25b shows an image arrangement in the joining work area 2301 when one arrangement alternative “2 laterally�1 vertically” is specified. FIG. 25c shows another image arrangement in the joining work area 2301 when the other arrangement alternative “1 laterally�2 vertically” is specified.
For landscape type images in FIGS. 27a to 32 a, the word “panorama” is specially added so that the operator can see it at a glance. Further, since in the embodiment the image arrangement is decided based on the case where a maximum of three images can be arranged both in the vertical direction and in the lateral direction, arrangement alternatives for arranging four or more images in the vertical direction or in the lateral direction are not displayed except in respective cases of panorama. For example, when eight images are specified, arrangement alternatives “4 laterally�2 vertically” and “2 laterally�4 vertically” are not displayed in FIG. 31a. Such arrangement alternatives, however, may be displayed as required.
First, the arrangements of the image altering part 304 and the display control part 307 are described in more detail. FIG. 34 is a functional block diagram showing the arrangements of the image altering part 304, the display control part 307 and the surroundings in the image processing apparatus according to the present invention for correcting image distortion. As shown in FIG. 34, the image altering part 304 includes an image specifying part 2401, an image arranging part 2402, an identical point specifying part 2403 and a distortion correcting part 2404.
After completion of specifying the reduced images, the operator moves the cursor to an “image correction” menu 603 on the menu bar in FIG. 35, clicks the button of the mouse 212 or the like, and further clicks an “distortion correction” menu 604 displayed as one of pull-down menus to select the distortion correcting function. Thus, the distortion correcting window 700 is popped up and displayed as shown in FIG. 36.
After completion of specifying the identical points. the operator presses the start button 706 to start execution to the distortion correction processing. A window, though not shown here, is opened with a message indicating “Distortion correction processing is completed”. When an “OK” button in the window is clicked, the distortion correction processing is ended.
Although in the sixth embodiment an identical point in each image is specified by moving a cursor to the point and then clicking the button of the mouse 212 or the like, the identical point may be specified using a line, i.e., by specifying identical points on two or more adjacent images in a drag-and drop operation of the mouse 212 or the like, as discussed in the following seventh embodiment.
Although the above sixth and seventh embodiments do not describe a case where a set of identical points once specified is altered, the set of identical points already specified may be released when a set of identical points is specified for images for which another set of identical points has already been specified, as described in the following eighth embodiment.
Although the above sixth to eighth embodiments for correcting image distortion have been made for images photographed with a digital still camera 213 having a fixed focal length, i.e., the above embodiments assume that the focal length is invariable, when the images for correcting image distortion are those photographed with a digital still camera 213 having variable focal lengths, the focal length may be set for proper distortion correction of images, as discussed in the following ninth embodiment.
The zoom switching part 1802 displays selectable zoom switching, alternatives related to whether an image to be joined with another has been photographed in the zoom mode or not. When no zoom is used for the image, the mode is switched to “standard” S1813. On the other hand, when the zoom is used for the image, the mode is switched to “maximum” 1814. The zoom mode is thus switched.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5469536 *May 21, 1993Nov 21, 1995Imageware Software, Inc.Image editing system including masking capabilityUS5982951 *May 23, 1997Nov 9, 1999Canon Kabushiki KaishaApparatus and method for combining a plurality of imagesUS5986670 *Sep 13, 1996Nov 16, 1999Dries; Roberta L.Method and apparatus for producing a computer generated display that permits visualization of changes to the interior or exterior of a building structure shown in its actual environmentUS6064399 *Apr 3, 1998May 16, 2000Mgi Software CorporationMethod and system for panel alignment in panoramasUS6078701 *May 29, 1998Jun 20, 2000Sarnoff CorporationMethod and apparatus for performing local to global multiframe alignment to construct mosaic imagesUS6128108 *Sep 3, 1997Oct 3, 2000Mgi Software CorporationMethod and system for compositing imagesUS6157747 *Aug 1, 1997Dec 5, 2000Microsoft Corporation3-dimensional image rotation method and apparatus for producing image mosaicsUS6167167 *Jul 2, 1997Dec 26, 2000Canon Kabushiki KaishaImage extractions apparatus and methodJPH09294225A Title not available* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7103516 *Dec 8, 2000Sep 5, 2006Fujitsu LimitedApparatus and method for optimizing three-dimensional modelUS7119841 *Apr 19, 2000Oct 10, 2006Canon Kabushiki KaishaImage processing apparatus method and storage medium thereforUS7256799 *Sep 10, 2002Aug 14, 2007Sanyo Electric Co., Ltd.Image synthesizer, image synthesis method and computer readable recording medium having image synthesis processing program recorded thereonUS7424490 *Jun 24, 2003Sep 9, 2008Ricoh Company, Ltd.System for document management and information processingUS7496369 *Jul 30, 2002Feb 24, 2009Canon Kabushiki KaishaImaging apparatusUS7538812Feb 22, 2005May 26, 2009Canon Kabushiki KaishaImage processing apparatus having plural display unitsUS7620906 *Nov 17, 2009Canon Kabushiki KaishaDisplay apparatus and method for displaying screen where dragging and dropping of object can be executed and program stored in computer-readable storage mediumUS7982896Sep 29, 2009Jul 19, 2011Silverbrook Research Pty LtdPrinter module logically and physically connectable to further modulesUS8045226Oct 25, 2011Silverbrook Research Pty LtdMulti-module device including a printer moduleUS8068170 *Nov 29, 2011Silverbrook Research Pty LtdModular digital cameraUS8111314 *Dec 21, 2006Feb 7, 2012Samsung Electronics Co., Ltd.Camera-enabled portable terminal and method for taking multi-image photograph using the sameUS8225210 *Jul 17, 2012Canon Kabushiki KaishaImage processing apparatus and control methodUS8249909 *Aug 21, 2012Hitachi-Ge Nuclear Energy, Ltd.System and method for visualizing the construction progress of scaffolding utilizing 3D CAD modelsUS8300989Oct 30, 2012Brother Kogyo Kabushiki KaishaImage processing device and storage medium storing image processing programUS8370739Mar 27, 2009Feb 5, 2013Brother Kogyo Kabushiki KaishaCombining multiple images from different display areas using a plurality of reference positionsUS8611702Oct 26, 2012Dec 17, 2013Brother Kogyo Kabushiki KaishaImage processing device and medium storing image processing programUS8789939Sep 4, 2011Jul 29, 2014Google Inc.Print media cartridge with ink supply manifoldUS8823823Sep 15, 2012Sep 2, 2014Google Inc.Portable imaging device with multi-core processor and orientation sensorUS8836809Sep 15, 2012Sep 16, 2014Google Inc.Quad-core image processor for facial detectionUS8866923Aug 5, 2010Oct 21, 2014Google Inc.Modular camera and printerUS8866926Sep 15, 2012Oct 21, 2014Google Inc.Multi-core processor for hand-held, image capture deviceUS8896720Sep 15, 2012Nov 25, 2014Google Inc.Hand held image capture device with multi-core processor for facial detectionUS8896724May 4, 2008Nov 25, 2014Google Inc.Camera system to facilitate a cascade of imaging effectsUS8902324Sep 15, 2012Dec 2, 2014Google Inc.Quad-core image processor for device with image displayUS8902333Nov 8, 2010Dec 2, 2014Google Inc.Image processing method using sensed eye positionUS8902340Sep 15, 2012Dec 2, 2014Google Inc.Multi-core image processor for portable deviceUS8902357Sep 15, 2012Dec 2, 2014Google Inc.Quad-core image processorUS8908051Sep 15, 2012Dec 9, 2014Google Inc.Handheld imaging device with system-on-chip microcontroller incorporating on shared wafer image processor and image sensorUS8908069Sep 15, 2012Dec 9, 2014Google Inc.Handheld imaging device with quad-core image processor integrating image sensor interfaceUS8908075Apr 19, 2007Dec 9, 2014Google Inc.Image capture and processing integrated circuit for a cameraUS8913137Sep 15, 2012Dec 16, 2014Google Inc.Handheld imaging device with multi-core image processor integrating image sensor interfaceUS8913151Sep 15, 2012Dec 16, 2014Google Inc.Digital camera with quad core processorUS8913182Sep 15, 2012Dec 16, 2014Google Inc.Portable hand-held device having networked quad core processorUS8922670Sep 15, 2012Dec 30, 2014Google Inc.Portable hand-held device having stereoscopic image cameraUS8922791Sep 15, 2012Dec 30, 2014Google Inc.Camera system with color display and processor for Reed-Solomon decodingUS8928897Sep 15, 2012Jan 6, 2015Google Inc.Portable handheld device with multi-core image processorUS8934027Sep 15, 2012Jan 13, 2015Google Inc.Portable device with image sensors and multi-core processorUS8934053Sep 15, 2012Jan 13, 2015Google Inc.Hand-held quad core processing apparatusUS8936196Dec 11, 2012Jan 20, 2015Google Inc.Camera unit incorporating program script scannerUS8937727Sep 15, 2012Jan 20, 2015Google Inc.Portable handheld device with multi-core image processorUS8947592Sep 15, 2012Feb 3, 2015Google Inc.Handheld imaging device with image processor provided with multiple parallel processing unitsUS8947679Sep 15, 2012Feb 3, 2015Google Inc.Portable handheld device with multi-core microcoded image processorUS8953060Sep 15, 2012Feb 10, 2015Google Inc.Hand held image capture device with multi-core processor and wireless interface to input deviceUS8953061Sep 15, 2012Feb 10, 2015Google Inc.Image capture device with linked multi-core processor and orientation sensorUS8953178Sep 15, 2012Feb 10, 2015Google Inc.Camera system with color display and processor for reed-solomon decodingUS9055221Sep 15, 2012Jun 9, 2015Google Inc.Portable hand-held device for deblurring sensed imagesUS9060128Sep 15, 2012Jun 16, 2015Google Inc.Portable hand-held device for manipulating imagesUS9083829Sep 15, 2012Jul 14, 2015Google Inc.Portable hand-held device for displaying oriented imagesUS9083830Sep 15, 2012Jul 14, 2015Google Inc.Portable device with image sensor and quad-core processor for multi-point focus image captureUS9088675Jul 3, 2012Jul 21, 2015Google Inc.Image sensing and printing deviceUS9100516Sep 15, 2012Aug 4, 2015Google Inc.Portable imaging device with multi-core processorUS9106775Sep 15, 2012Aug 11, 2015Google Inc.Multi-core processor for portable device with dual image sensorsUS9124736Sep 15, 2012Sep 1, 2015Google Inc.Portable hand-held device for displaying oriented imagesUS9124737Sep 15, 2012Sep 1, 2015Google Inc.Portable device with image sensor and quad-core processor for multi-point focus image captureUS9131083Sep 15, 2012Sep 8, 2015Google Inc.Portable imaging device with multi-core processorUS9137397Jul 3, 2012Sep 15, 2015Google Inc.Image sensing and printing deviceUS9137398Sep 15, 2012Sep 15, 2015Google Inc.Multi-core processor for portable device with dual image sensorsUS9143635Sep 15, 2012Sep 22, 2015Google Inc.Camera with linked parallel processor coresUS9143636Sep 15, 2012Sep 22, 2015Google Inc.Portable device with dual image sensors and quad-core processorUS9148530Sep 15, 2012Sep 29, 2015Google Inc.Handheld imaging device with multi-core image processor integrating common bus interface and dedicated image sensor interfaceUS9167109Apr 4, 2013Oct 20, 2015Google Inc.Digital camera having image processor and printerUS9168761Dec 11, 2012Oct 27, 2015Google Inc.Disposable digital camera with printing assemblyUS9177216Dec 16, 2013Nov 3, 2015Brother Kogyo Kabushiki KaishaImage processing device and medium storing image processing programUS9179020Sep 15, 2012Nov 3, 2015Google Inc.Handheld imaging device with integrated chip incorporating on shared wafer image processor and central processorUS9185246Sep 15, 2012Nov 10, 2015Google Inc.Camera system comprising color display and processor for decoding data blocks in printed coding patternUS9185247Sep 15, 2012Nov 10, 2015Google Inc.Central processor with multiple programmable processor unitsUS9191529Sep 15, 2012Nov 17, 2015Google IncQuad-core camera processorUS9191530Sep 15, 2012Nov 17, 2015Google Inc.Portable hand-held device having quad core image processorUS9197767Apr 4, 2013Nov 24, 2015Google Inc.Digital camera having image processor and printerUS9219832Sep 15, 2012Dec 22, 2015Google Inc.Portable handheld device with multi-core image processorUS9237244Sep 15, 2012Jan 12, 2016Google Inc.Handheld digital camera device with orientation sensing and decoding capabilitiesUS9338312Sep 15, 2012May 10, 2016Google Inc.Portable handheld device with multi-core image processorUS9432529Sep 15, 2012Aug 30, 2016Google Inc.Portable handheld device with multi-core microcoded image processorUS20010000963 *Dec 8, 2000May 10, 2001Masahira DeguchiApparatus and method for optimizing three-dimensional modelUS20030027603 *Jul 30, 2002Feb 6, 2003Atsushi TakasakiImaging apparatusUS20030058255 *Sep 17, 2002Mar 27, 2003Yoichi YamagishiImage management systemUS20030076361 *Sep 10, 2002Apr 24, 2003Haruo HatanakaImage synthesizer, image synthesis method and computer readable recording medium having image synthesis processing program recorded thereonUS20040088281 *Jun 24, 2003May 6, 2004Takaya MatsuishiDocument management system, information processing apparatus, document management method and computer-readable recording mediumUS20050078052 *Oct 13, 2004Apr 14, 2005Casio Computer Co., Ltd.Display image generating device comprising converting function of resolutionUS20060048069 *Aug 24, 2005Mar 2, 2006Canon Kabushiki KaishaDisplay apparatus and method for displaying screen where dragging and dropping of object can be executed and program stored in computer-readable storage mediumUS20070263071 *Dec 21, 2006Nov 15, 2007Samsung Electronics Co., Ltd.Camera-enabled portable terminal and method for taking multi-image photograph using the sameUS20070291337 *Jun 8, 2007Dec 20, 2007Canon Kabushiki KaishaImage processing apparatus and control methodUS20080292138 *May 22, 2008Nov 27, 2008Brother Kogyo Kabushiki KaishaImage processing device and storage medium storing image processing programUS20080297619 *Aug 10, 2008Dec 4, 2008Silverbrook Research Pty LtdModular digital cameraUS20090244094 *Mar 27, 2009Oct 1, 2009Brother Kogyo Kabushiki KaishaImage processing apparatus and image processing programUS20100114635 *Oct 28, 2009May 6, 2010Hitachi-Ge Nuclear Energy, Ltd.Construction progress visualization system and method of visualizing construction progressUS20100149356 *Oct 27, 2009Jun 17, 2010Samsung Electronics Co., LtdDisplay method and photographing apparatus and display apparatus using the sameUS20100296116 *Aug 4, 2010Nov 25, 2010Silverbrook Research Pty LtdMulti-Module Device including a Printer ModuleUS20130326419 *May 24, 2013Dec 5, 2013Toru HaradaCommunication terminal, display method, and computer program productUS20140085511 *Sep 11, 2013Mar 27, 2014Olympus Imaging Corp.Image processing device, method for processing image, and recording mediumEP1850583A2 *Apr 24, 2007Oct 31, 2007Samsung Electronics Co., Ltd.Method of photographing panoramic imageWO2006032015A2 *Sep 14, 2005Mar 23, 2006E.I. Dupont De Nemours And CompanyComputer-implemented methods for organizing image data produced by resonant mie scattering of light from microparticles and resulting data structures* Cited by examinerClassifications U.S. Classification345/629, 382/284, 348/584International ClassificationH04N1/387Cooperative ClassificationH04N1/3875European ClassificationH04N1/387C2BLegal EventsDateCodeEventDescriptionAug 14, 2000ASAssignmentOwner name: RICOH COMPANY, LTD., JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SAKATA, NORIHIKO;TAKAKI, TOSHINORI;HASEGAWA, MINORU;REEL/FRAME:011046/0295Effective date: 19990506Dec 26, 2006FPAYFee paymentYear of fee payment: 4Jan 7, 2011FPAYFee paymentYear of fee payment: 8Jan 8, 2015FPAYFee paymentYear of fee payment: 12RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services