Abstract:
An imaging apparatus includes: an imaging unit for acquiring a plurality of view point images imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; a storage unit for storing an error of the imaging optical system; an optical zoom magnification specifying unit for receiving specification instruction of an optical zoom magnification; a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification; and a correction unit for setting an electronic zoom magnification corresponding to the position of the zoom lens, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image to eliminate an object point shift amount corresponding to the error from the magnified viewpoint image.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS  
       [0001]    This application is a Continuation of copending application Ser. No. 13/142,749 filed on Jun. 29, 2011, which is a 371 national phase application of PCT/JP2010/053926 filed on Mar. 3, 2010, which claims priority to Japanese Application No. 2009-058244 filed in Japan, on Mar. 11, 2009. The entire contents of all of the above applications are hereby incorporated by reference. 
     
    
     TECHNICAL FIELD  
       [0002]    The presently disclosed subject matter relates to an imaging apparatus capable of acquiring images from a plurality of viewpoints (multi-viewpoints). 
       BACKGROUND ART  
       [0003]    PTL 1 discloses a stereoscopic imaging apparatus including a first lens barrel  1 R having a CCD  16 R for obtaining imaging information for a right eye; a second lens barrel  1 L having a CCD  16 L for obtaining the imaging information for a left eye; a camera detection circuit  43  for detecting a focal length of the first lens barrel  1 R and a focal length of the second lens barrel  1 L; a ROM  47  including an EEPROM for preliminarily storing a shift amount of the first lens barrel  1 R and the second lens barrel  1 L from the respective centers of optical axes in the respective focal lengths; and a CPU  46  for controlling an image extraction area in at least one of the CCDs  16 R and  16 L at the respective focal lengths based on an output of the ROM  47 . 
         [0004]    PTL 2 discloses a stereoscopic camera  2  including: first and second potentiometers  25  and  35  for detecting positions of first and second variable power lenses  21  and  31  respectively; a correction data storage unit  52  for storing a power difference D 1  between the first and second variable power lenses  21  and  31  for each predetermined position; an electronic variable power circuit  53  for electronically changing a magnification of second image data (performing a digital zoom (electronic zoom) on second image data) based on the power difference D 1 ; and a CPU  40  for reading the power difference D 1  corresponding to the detected positions of the potentiometers  25  and  35  from the correction data storage unit  52  and setting the read power difference D 1  in the electronic variable power circuit  53  so as to match an imaging range of the first image data with an imaging range of the second image data. The stereoscopic camera  2  further includes a coordinate conversion circuit for storing an optical axis coordinate difference between the first and second variable power lenses  21  and  31  in the correction data storage unit  52  and converting the coordinates of the second image data based on the optical axis coordinate difference. 
         [0005]    In PTL 3, AF drive circuits  2  and  13  control focusing of left and right imaging lenses  1  and  12  respectively; outputs of a CCD  3  and a CCD  14  are written to electronic zoom memories  6  and  17 ; a microcomputer  23  controls the extraction position of an electronic zoom extraction frame based on the AF data proportional to the subject distance; and thereby the convergence angle is electronically controlled. In addition, other related arts related to the presently disclosed subject matter are included in the following PTLs 4 to 8. 
       CITATION LIST  
     Patent Literature 
       [0006]    PTL 1: Japanese Patent Application Laid-Open No. H08-317424 
         [0007]    PTL 2: Japanese Patent Application Laid-Open No. 2006-162991 
         [0008]    PTL 3: Japanese Patent Application Laid-Open No. H07-95623 
         [0009]    PTL 4: Japanese Patent Application Laid-Open No. H06-296252 
         [0010]    PTL 5: Japanese Patent Application Laid-Open No. 2006-251683 
         [0011]    PTL 6: Japanese Patent Application Laid-Open No. 2005-20606 
         [0012]    PTL 7: Japanese Patent Application Laid-Open No. H09-037302 
         [0013]    PTL 8: Japanese Patent Application Laid-Open No. H08-317429 
       SUMMARY OF INVENTION  
     Technical Problem 
       [0014]    PTLs 1 and 2 each discloses a stereoscopic imaging apparatus which can electronically correct a shift of an optical axis due to an individual difference between the two variable power imaging lenses to provide an appropriate three dimensional viewing of objects. However, PTL 1 has a problem in that the image extraction position and the area change amount are calculated according to the amount of the optical axis shift between the two lenses with respect to each zoom position, and thus a different angle of view is set depending on the difference (variation) in error amount even at the same zoom position of the individual apparatus. 
         [0015]    In PTL 2, a wide-angle lens is used as a lens included in a second imaging unit in which the electronic zoom is performed, and the angle of view is set by the lens (i.e. optical zoom) included in a first imaging unit in which the electronic zoom is not performed. Therefore, angles of view set for the first and second imaging units are set to be identical to each other. However, PTL 2 has a problem in that the structures of the two lenses are not identical to each other, and thus it is difficult to match the characteristics of the lenses such as a distortion and a chromatic aberration, and thereby an appropriate three dimensional viewing may not be provided. 
         [0016]    PTL 3 discloses an apparatus for electronically correcting the convergence angle according to the subject distance. However, PTL 3 does not disclose means for correcting a parallactic shift when the subject matter disclosed in PTL 3 is applied to an imaging apparatus including variable power lenses. In addition, PTL 3 does not disclose means for correcting an optical axis shift between the two lenses. 
         [0017]    In PTL 4, when the digital zoom (electronic zoom) are performed, a region of pixels from which image signals are not actually read out are occurs in proportion to the magnification of zooming up by the electronic zoom. And, the region of pixels from which image signals are not actually read out does not occur when a zoom lens is positioned at a wide end. Accordingly, in the conventional arts, when two variable power imaging lenses are located at the wide ends, the optical axis shift due to the individual difference between the two lenses cannot be electronically corrected. 
         [0018]    In order to correct manufacturing errors and parallaxes of a plurality of variable power lenses having identical characteristics, the presently disclosed subject matter sets a magnification of an electronic zoom in a uniformed manner according to an optical zoom position (position of the variable power lenses) so as to obtain an appropriate three dimensional image and to maximize the angle of view on the wide angle side. 
       Solution to Problem 
       [0019]    An imaging apparatus according to the presently disclosed subject matter can include: an imaging unit for acquiring a plurality of viewpoint images imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; a storage unit for storing an error of the imaging optical system; an optical zoom magnification specifying unit for receiving a specification instruction of an optical zoom magnification; a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification received by the optical zoom magnification specifying unit; and a correction unit for setting an electronic zoom magnification corresponding to the position of the zoom lens moved by the zoom lens driving unit, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image to eliminate an object point shift amount corresponding to the error stored in the storage unit from the magnified viewpoint image. 
         [0020]    An imaging apparatus according to the presently disclosed subject matter can include: an imaging unit for acquiring a plurality of viewpoint images imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; a storage unit for storing a parallax corresponding to a distance to a subject of each viewpoint image acquired by each imaging optical system; an optical zoom magnification specifying unit for receiving a specification instruction of an optical zoom magnification; a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification received by the optical zoom magnification specifying unit; a distance measuring unit for measuring a distance to a subject; and a correction unit for setting an electronic zoom magnification corresponding to the distance to the subject measured by the distance measuring unit and the position of the zoom lens moved by the zoom lens driving unit, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image for securing a parallax stored in the storage unit from the magnified linage. 
         [0021]    An imaging apparatus according to the presently disclosed subject matter can include: an imaging unit for acquiring a plurality of viewpoint linages imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; a parallax measuring unit for measuring a parallax by detecting a pair of corresponding point from the viewpoint images acquired by each imaging optical system; an optical zoom magnification specifying unit for receiving a specification instruction of an optical zoom magnification; a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification received by the optical zoom magnification specifying unit; and a correction unit for setting an electronic zoom magnification corresponding to the position of the zoom lens moved by the zoom lens driving unit, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image for securing a predetermined applied parallax range from the magnified image. 
         [0022]    The electronic zoom magnification set by the correction unit can increase as the position of the zoom lens moves from a wide angle side to a telephoto side. 
         [0023]    The imaging apparatus can further include a three dimensional image processing unit for generating a three dimensional image based on the image extracted by the correction unit and the viewpoint image other than the viewpoint image to be corrected, and outputting the three dimensional image onto a predetermined display device. 
         [0024]    An image correction method according to the presently disclosed subject matter can cause an imaging apparatus including: an imaging unit for acquiring a plurality of viewpoint images imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; an optical zoom magnification specifying unit for receiving a specification instruction of an optical zoom magnification; and a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification received by the optical zoom magnification specifying unit, to execute: a step of storing an error of the imaging optical system; and a step of setting an electronic zoom magnification corresponding to the position of the zoom lens moved by the zoom lens driving unit, magnifying a desired viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image to eliminate an object point shift amount corresponding to the error from the magnified viewpoint image. 
         [0025]    An image correction method according to the presently disclosed subject matter can cause an imaging apparatus including: an imaging unit for acquiring a plurality of viewpoint images imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; an optical zoom magnification specifying unit for receiving a specification instruction of an optical zoom magnification; and a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification received by the optical zoom magnification specifying unit, to execute: a step of measuring a distance to a subject; and a step of setting an electronic zoom magnification corresponding to the measured distance to the subject and the position of the zoom lens moved by the zoom lens driving unit, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image for securing the stored parallax from the magnified image. 
         [0026]    An image correction method according to the presently disclosed subject matter can cause an imaging apparatus including: an imaging unit for acquiring a plurality of viewpoint images imaged from a plurality of viewpoints by a plurality of imaging optical systems each including a zoom lens; an optical zoom magnification specifying unit for receiving a specification instruction of an optical zoom magnification; and a zoom lens driving unit for moving the zoom lens to a position corresponding to the instruction of the optical zoom magnification received by the optical zoom magnification specifying unit, to execute: a step of measuring a parallax by detecting a pair of corresponding point from the viewpoint images acquired by each imaging optical system; and a step of setting an electronic zoom magnification corresponding to the position of the zoom lens moved by the zoom lens driving unit, magnifying a desired viewpoint image to be corrected from within the plurality of viewpoint images based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image for securing a predetermined applied parallax range from the magnified image. 
         [0027]    A computer-readable recording medium including instructions stored thereon, such that when the instructions are read and executed by a processor, the processor is configured to perform the steps of: receiving an instruction for specifying an optical zoom magnification; moving zoom lenses included in a plurality of imaging optical systems to positions corresponding to the instruction; and setting an electronic zoom magnification corresponding to the position of the zoom lens, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images imaged by using the imaging optical systems based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image to eliminate an object point shift amount corresponding to an error of the imaging optical systems stored in a storage unit from the magnified viewpoint image. 
         [0028]    A computer-readable recording medium including instructions stored thereon, such that when the instructions are read and executed by a processor, the processor is configured to perform the steps of: receiving an instruction for specifying an optical zoom magnification; moving zoom lenses included in a plurality of imaging optical systems to positions corresponding to the instruction; measuring a distance to a subject; and setting an electronic zoom magnification corresponding to the measured distance to the subject and the position of the zoom lens, magnifying a viewpoint image to be corrected from within the plurality of viewpoint images by using the imaging optical systems based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image for securing a parallax corresponding to the distance to the subject of the viewpoint images stored in a storage unit from the magnified image. 
         [0029]    A computer-readable recording medium including instructions stored thereon, such that when the instructions are read and executed by a processor, the processor is configured to perform the steps of: receiving an instruction for specifying an optical zoom magnification; moving zoom lenses included in a plurality of imaging optical systems to positions corresponding to the instruction; measuring a parallax by detecting a pair of corresponding point from a plurality of viewpoint images acquired by the imaging optical systems; and setting an electronic zoom magnification corresponding to the position of the zoom lens, magnifying a desired viewpoint image to be corrected from within the plurality of viewpoint images by using the imaging optical systems based on the electronic zoom magnification, and extracting a part of the magnified viewpoint image for securing a predetermined applied parallax range from the magnified image. 
       Advantageous Effects of Invention 
       [0030]    According to the presently disclosed subject matter, maximum values of an optical axis shift and a parallactic shift in the manufacturing process of an individual imaging apparatus are estimated, and an image shift correction region for correcting an image shift by controlling an electronic zoom magnification corresponding to (proportional to) the position of the optical zoom lens (an optical zoom position) is uniformly set for an imaging element included in the imaging apparatus. Thereby, a correction region (patt) can be set on an imaging device according to each optical zoom position. Since the electronic zoom magnification increases as the position of the zoom lens moves from the wide angle side (wide angle end side) to the telephoto side (telephoto end side), the angle of view on the wide angle side can be maximized. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0031]      FIG. 1  is a block diagram of a camera according to a first embodiment; 
           [0032]      FIG. 2  is a flowchart of a correction process according to the first embodiment; 
           [0033]      FIG. 3  illustrates an example of an allowable limit value emax of an optical axis shift; 
           [0034]      FIGS. 4A to 4H  illustrate an example of electronic zoom extraction range; 
           [0035]      FIG. 5  illustrates a relationship between electronic zoom magnification data y and a current zoom position x; 
           [0036]      FIG. 6  is a block diagram of a camera according to a second embodiment; 
           [0037]      FIG. 7  illustrates an example of a shortest photographing distance (J., and a convergence angle setting distance  13 ; 
           [0038]      FIG. 8  is a flowchart of a correction process according to the second embodiment; 
           [0039]      FIGS. 9A to 9H  illustrate an example of electronic zoom extraction range; 
           [0040]      FIG. 10  is a block diagram of a camera according to a third embodiment; 
           [0041]      FIGS. 11A and 11B  illustrate an example of corresponding points; and 
           [0042]      FIG. 12  is a flowchart of a correction process according to the third embodiment. 
       
    
    
     DESCRIPTION OF EMBODIMENTS  
     First Embodiment  
       [0043]      FIG. 1  illustrates an electrical configuration of a camera  10  according to a first embodiment. 
         [0044]    Each of the imaging optical systems  14 A and  14 B includes a zoom lens and a focus lens. The convergence angle formed between the optical axes of the imaging optical systems  14 A and  14 B is assumed to be mechanically fixed. Although a binocular parallax formed between the imaging optical systems  14 A and  14 B changes depending on a distance to a focused subject from the camera  10 , in the first embodiment, it is assumed that the focused subject is located at a predetermined position and the focus lens does not move. 
         [0045]    In response to an operation input from a zoom button of the operation unit  74 , a lens motor  24 A move the zoom lens of the imaging optical systems  14 A along a lens optical axis of the imaging optical systems  14 A to a telephoto side (zoom-in side) or a wide angle side (zoom-outside) to change a focal length (imaging magnification) of the imaging optical systems  14 A, and a lens motor  24 B move the zoom lens of the imaging optical systems  14 B along a lens optical axis of the imaging optical systems  14 B to a telephoto side (zoom-in side) or a wide angle side (zoom-outside) to change a focal length (imaging magnification) of the imaging optical systems  14 B. 
         [0046]    Two imaging elements  50 A and  50 B each configured of an area CCD (charge coupled device) sensor, for example. The imaging elements  50 A and  50 B are disposed in a position corresponding to the respective focus positions of the imaging optical systems  14 A and  14 B inside the main body  12  (not illustrated) of the camera  10 . Light reflected by a subject enters the imaging optical systems  14 A and  14 B, and the light is focused on light receiving surfaces of the imaging elements  50 A and  50 B respectively. The imaging elements  50 A and  50 B each include a plurality of photoelectric conversion elements disposed in a matrix on the respective light receiving surface thereof. 
         [0047]    The imaging elements  50 A and  50 B each output an analog signal indicating an amount of light received by each of photoelectric conversion elements as an image signal. The imaging elements  50 A and  50 B each output the image signal in a timing synchronized with a timing signal generated by a timing signal generation unit connected to a drive circuit (not illustrated) respectively. 
         [0048]    An aperture diaphragm (not illustrated) is disposed between the imaging optical systems  14 A and  14 B and the imaging elements  50 A and  50 B respectively. The aperture diaphragm may be configured of a single aperture which can be changed consecutively, or may be configured such that a plurality of apertures having different aperture amount are switched. 
         [0049]    The timing signal generation unit is also connected to a flash control circuit for controlling lighting of the flash (not illustrated). When a low light intensity is detected or when a user instructs lighting of the flash, the lighting of the flash is controlled in a timing synchronized with a timing signal generated by the timing signal generation unit. 
         [0050]    Each of the signal output ends of the imaging elements  50 A and  50 B is connected to analog signal processing circuits  56 A and  56 B, A/D converters (analog-to digital converters)  58 A and  58 B, image signal processing circuits  60 A and  60 B, memories  62 A and  62 B, and compression/expansion processing circuit  64  in that order, each of which is connected to a system bus  68  and is integrally controlled by a CPU (central processing unit)  70  connected to the system bus  68 . 
         [0051]    The analog signal processing circuits  56 A and  56 B takes the samples of the image signals outputted from the imaging elements  50 A and  50 B respectively in a timing synchronized with a timing signal generated by the timing signal generation unit. The analog signal processing circuits  56 A and  56 B amplifies the image signals outputted from the imaging elements  50 A and  50 B respectively and outputs the amplified image signals to the respective A/D converters  58 A and  58 B respectively. The analog signal processing circuit  56 A and  56 B each include a correlated double sampling (CDS) unit (not illustrated). The CDS unit includes a clamp circuit which uses, for example, a CCD-type imaging element to clamp various kinds of noise basically generated by the imaging element in response to a timing signal from the timing signal generation unit; and a sample hold circuit which holds an analog voltage signal in response to the timing signal. The CDS units of the analog signal processing circuits  56 A and  56 B each remove noise components included in the image signals, and outputs the image signal as an analog output signal to the A/D converters  58 A and  58 B respectively. The image signals outputted from the analog signal processing circuits  56 A and  56 B are converted to digital image data by the A/D converters  58 A and  58 B to be inputted to the image signal processing circuits  60 A and  60 B respectively. The image signal processing circuits  60 A and  60 B perform various kinds of image processing on the inputted image data such as a color correction, a gamma correction and a Y/C conversion. The image data outputted from the image signal processing circuits  60 A and  60 B is temporarily stored in the memories  62 A and  62 B each made of a RAM (random access memory) or the like respectively. Then, the image data is compressed by the compression/expansion processing circuit  64  and stored in a memory card  80  inserted into a memory card slot (not illustrated) of the camera  10 . 
         [0052]    A display driver  27  is also connected to the system bus  68 . The display driver  27  drives a display  26  so that an image based on image data obtained by photographing can be displayed on the display  26 . The display driver  27  performs display control so as to display various display modes. 
         [0053]    The image data in the memories  62 A and  62 B is converted to three-dimensional image data by the three-dimensional image processing circuit  34  and then is displayed as a reproduced image on the display  26  driven by the display driver  27 . 
         [0054]    The detailed structure of the display  26  is not illustrated. The display  26  includes a parallax barrier display layer on the surface thereof. For three-dimensional display (3D display), the display  26  generates a parallax barrier made of patterns each having a light-transmitting portion and a light-shielding portion arranged alternately at a predetermined pitch on the parallax barrier display layer as well as displays strip-shaped image pieces indicating images for left and right eyes arranged alternately on the image display surface on a lower layer thereof, thereby enabling pseudo-3D view. Note that the right eye and the left eye of a viewer views the same plane image not by reconstructing the plane images obtained from the imaging elements SOA and SOB into strip-shaped image pieces to be arranged alternately, but by reconstructing only the right or left image obtained from one of the imaging elements SOA and SOB into strip-shaped image pieces to be arranged alternately. A device for 3D display in the presently disclosed subject matter may not be limited to the above described embodiment using parallax barrier. 
         [0055]    The operation unit  74  is connected to the system bus  68 . The operation unit  74  includes a release switch, a zoom button, a menu switch, an execution/screen switching switch, a selection switch, a cancel switch and an imaging mode switching switch. The CPU performs control according to the operation of the operation unit  74 . 
         [0056]    More specifically, when the release switch is operated to instruct the image data to be stored in the memory card  80 , the CPU  70  reads the image data temporarily stored in the memories  62 A and  62 B and transfers the image data to thy compression/expansion processing circuit  64 . Then, the image data is compressed by the compression/expansion processing circuit  64  and then stored in the memory card  80 . Note that the image data may be stored in the memory card  80  without being compressed depending on the imaging mode. 
         [0057]    When an instruction is made to reproduce (display) an image represented by the image data stored in the memory card  80 , the image data is read from the memory card  80 , and the read image data is decompressed (expanded) by the compression/expansion processing circuit  64  and temporarily stored in the memories  62 A and  62 B. Then, the image data temporarily stored in the memories  62 A and  62 B is used to display (reproduce) the image on the display  26 . 
         [0058]    The zoom lens positions (zoom positions, optical zoom positions) of the imaging optical systems  14 A and  14 B are detected by the zoom position detection units  76 A and  76 B respectively. The detection signals detected by the zoom position detection units  76 A and  76 B are inputted into the CPU  70  as variable power information of the respective zoom lenses. 
         [0059]    Electronic zoom magnification data is stored in the ROM (read-only memory)  71  and optical axis difference data is stored in the ROM  72 . 
         [0060]    The extraction position setting unit  73  determines the extraction position based on the zoom position, the electronic zoom magnification data, and the optical axis difference data received from the CPU  70  and outputs an instruction to the image signal processing circuits  60 A and  60 B. The image signal processing circuits  60 A and  60 B each extract the range based on the instruction from the image subjected to various kinds of image processing to be outputted to the memories  62 A and  62 B. 
         [0061]      FIG. 2  is a flowchart of a correction process whose execution is controlled by CPU  70 . 
         [0062]    In step S 1 , a determination is made as to whether the zoom button is operated to change the zoom position. In the case of Yes, the process moves to step S 2 . 
         [0063]    In step S 2 , according to the operation of the zoom button, the motors  24 A and  24 B are controlled so as to move the zoom lenses of the imaging optical systems  14 A and  14 B respectively to the telephoto (T) end side or the wide (W) end side. 
         [0064]    In step S 3 , the current positions of the zoom lenses of the imaging optical systems  14 A and  14 B are acquired by the zoom position detection units  76 A and  76 B respectively. 
         [0065]    In step S 4 , a determination is made as to whether the current positions of the zoom lenses of the imaging optical systems  14 A and  14 B are located in the respective positions instructed by the operation of the zoom button. In the case of Yes, the process moves to step S 5 . In the case of No, the process returns to step S 2 . 
         [0066]    In step S 5 , the electronic zoom magnification data corresponding to the respective current positions of the zoom lenses of the imaging optical systems  14 A and  14 B are read from the ROM  71 . The electronic zoom magnification data indicates a margin of the electronic zoom extraction range of each zoom lens position sufficiently required to correct a shift amount d of the left and right viewpoint images generated by a predetermined allowable limit value (maximum optical axis shift angle) θmax (see  FIG. 3 ) of the optical axis shift of the respective imaging optical systems  14 A and  14 B. 
         [0067]    Assuming that the allowable limit value θmax of the optical axis shift is constant even when the zoom lens moves from the W end to the T end, the object point shift amount d which indicates a shift amount of an identical object (a delta of the positions of the object) in the two viewpoint images gradually increases as the zoom lens moves from the W end to the T end. 
         [0068]      FIGS. 4A to 4H  are diagrams illustrating a shift amount of the subject SUB. Referential marks SL w  and SR w  of  FIGS. 4A and 4B  are images imaged from the viewpoints corresponding to the left eye and the right eye respectively when the zoom lenses are positioned at the wide end (W end). A referential mark S w  in  FIG. 4C  is a synthesized image generated by synthesizing the images SL w  and SR w . In  FIGS. 4A  to  4 C, referential marks SUB L  and SUB R  represent positions of the subject SUB in the images SL w  and SR w  respectively. A referential mark Aw in  FIG. 4D  indicates an electronic zoom extraction range for absorbing the horizontal object point shift amount (horizontal shift amount of the subject SUB or a delta dw of the positions SUB L  and SUB R ). 
         [0069]    Referential marks SL T  and SR T  of  FIGS. 4E and 4F  are images imaged from the viewpoints corresponding to the left eye and the right eye respectively when the zoom lenses are positioned at the telephoto end (T end). A referential mark S T  in  FIG. 4G  is a synthesized image generated by synthesizing the images SL T  and SR T . In  FIGS. 4E to 4G , referential marks SUB L  and SUB R  represent positions of the subject SUB in the images SL T  and SR T  respectively. A referential mark A T  in  FIG. 4H  indicates an electronic zoom extraction range for absorbing the horizontal object point shift amount (horizontal shift amount of the subject SUB or a delta dt of the positions SUB L  and SUB R ). 
         [0070]    More specifically, an assumption is made such that the imaging optical systems  14 A and  14 B are arranged side by side along the horizontal direction, the optical axis shift of the imaging optical systems  14 A and  14 B is equal to the allowable limit value θmax, and at this time the shift amount of the subject SUB of the left and right viewpoint images by the imaging optical systems  14 A and  14 B (a delta of the position SUB L  and SUB R ) is “dw” at the W end and “dt” at the T end respectively as illustrated in  FIGS. 4E and 4F . In this case, assuming that the current zoom position is x, the position at the W end is W, and the position at the T end is T, and the horizontal length of the original viewpoint image is L, the horizontal length V of the electronic zoom extraction range for absorbing the horizontal object point shift amount d is expressed as V=L−dw−(dt−dw)/(T−W)×(x−W). Note that if x=W, V=L−dw, and if x=T, V=L−dt (see  FIGS. 4D and 4H ). Since the aspect ratio of the electronic zoom extraction range (indicated by the referential marks Aw and AT in  FIGS. 4D and 4H  respectively) conforms to that of the original viewpoint image, the electronic zoom magnification y is expressed as y=L/V. Even if the imaging optical systems  14 A and  14 B are arranged side bay side along the vertical direction, the electronic zoom magnification y can be obtained based on the vertical shift amount of the subject SUB in the same manner. 
         [0071]    Here, specific numerical examples are given. If the maximum optical zoom magnification at the T end is 5 times and the shift amount dw at the W end is 2% of the image length L in the horizontal direction thereof, the maximum shift amount dt at the T end is 2×5=10% of the image length L in the horizontal direction thereof. Accordingly, the electronic zoom magnification at the Wend is 1.02 times, and the electronic zoom magnification at the T end is 1.1 times. Since the electronic zoom magnification for correction increases with an increase in optical zoom magnification, the magnification of the image theoretically exceeds the optical zoom magnification specified by the user, but the apparent magnification of the image is adjusted so as to match the optical zoom magnification specified by the user. 
         [0072]    As illustrated in  FIG. 5 , electronic zoom magnification data y increases proportionally as the current zoom position x increases. As illustrated in  FIG. 5 , if x=W (=x o ), y=Y 1 =L/(L−dw), and if x=T (=X 1 ), y:: Y 2 =L/(L−dt). In other words, V becomes smaller with an increase in y. Even if the zoom position is at the W end, V is smaller than L, and the electronic zoom magnification exceeds “1”. Note that the position of the W end may be set to the reference position as W=O. The above equation is stored in ROM  71 , and the actual calculation of the electronic zoom magnification data y may be performed by the CPU  70 . 
         [0073]    If the optical axis shift of an individual product of the camera  10  is assumed not to exceed the allowable limit value θmax in quality control, it can be determined that the object point shift amount of the individual product of the camera  10  does not exceed d. Accordingly, in the following description, the range of performing electronic zooming using uniform electronic zoom magnification data y is determined so as to absorb the object point shift amount of any products. If the imaging optical systems  14 A and  14 B are arranged side by side along the horizontal direction, a parallax is formed in the horizontal direction. Thus, a horizontal shift control is performed on the extraction range so as not to destroy the parallax to determine the extraction range. Note that in the same manner as disclosed in the PTLs 6 and 7, the image extraction range may be determined so that the displayed 3D image falls within the binocular fusion range of the viewer. 
         [0074]    In step S 6 , optical axis difference data is read from the ROM  72 . The optical axis difference data refers to an actual optical axis shift angle of each of the imaging optical systems  14 A and  14 B. The data is a value specific to each product of the camera  10  and stored in the ROM  72  at manufacturing and shipment. 
         [0075]    In step S 7 , based on the optical axis difference data and the electronic zoom magnification data y read by the extraction position setting unit  73 , the extraction range (range of performing electronic zooming) of eliminating the object point shift due to an optical axis shift is determined. The size of the extraction range conforms to the uniform electronic zoom magnification data y, but the place of the extraction changes depending on the positional relationship of the object point shift. The aspect ratio of the extraction range is assumed to be the same as that of the original viewpoint image. 
         [0076]    Then, the image contained in the determined extraction range is extracted from the images stored in preliminarily specified one of the memories  62 A and  62 B, and the extracted image is electronically magnified by magnification y (electronic zoom). Then, the magnified image is stored as a new viewpoint image in one of the memories  62 A and  62 B. If the object point shift can be removed, the extraction and the electronic magnification may be performed on both viewpoint images. The three-dimensional image processing circuit  34  generates a three-dimensional image (3D image) S from the new viewpoint image and the other viewpoint image not subjected to electronic zooming. Note that the viewpoint image subjected to this electronic zooming and the viewpoint image not subjected to electronic zooming may be associated with each other and stored in the memory card  80 . 
         [0077]    As described above, maximum values of an optical axis shift in the manufacturing process of an individual camera  10  are estimated, and an image shift correction region is uniformly set to every camera  10  using an electronic zoom magnification proportional to the optical zoom lens position. Accordingly, a uniformed correction region of individual product can be set according to each optical zoom position and the angle of view on the wide angle side can be maximized. 
       Second Embodiment  
       [0078]      FIG. 6  illustrates an electrical configuration of a camera  10  according to a second embodiment. The camera  10  includes the configuration similar to that of the first embodiment and similar reference characters or numerals refer to similar elements. The second embodiment assumes that the focused subject is located at any distance from the camera  10  and the focus lens can be moved to focus the subject. The CPU  70  performs a well-known automatic focusing technique such as a contrast AF (automatic focus) and a passive AF using a triangular measurement to determine the amount of movement and instructs the motors  24 A and  24 B with the focus lens position of the focus lenses. 
         [0079]    The focus measurement unit  77  measures the distance to the focused subject from the camera  10  based on the lens position of the focus lens or using a well-known distance measurement technique such as the triangular measurement. 
         [0080]    The ROM  72  stores a shortest imaging distance a which is a shortest distance from the camera  10  to the subject SUB at which an image of the subject SUB which can be viewed as a three-dimensional image can be imaged by the camera  10 ; and a convergence angle setting distance ˜ which is a distance from the camera  10  to the intersecting point PI of the optical axes L 1  and L 2  of the imaging optical systems SOA and SOB each with a predetermined convergence angle (see  FIG. 7 ). The parallax with respect to an object at the shortest imaging distance a is called a marginal parallax. The object located at a distance closer than the shortest imaging distance a; is blurred and cannot be viewed three dimensionally. The parallax with respect to an object located at the shortest imaging distance a or farther (from a to infinity) is called an appropriate parallax. If the distance to the focused subject measured by the focus measurement unit  77  is a or more, the CPU  70  determines that the camera  10  can image a  3 D image. Then, the CPU  70  performs shift control on the horizontal position of the extraction range to compensate for the expansion of the parallax by changing the optical zoom magnification. The determination of the extraction range is similar to that of the PTL  8  for compensating for the expansion of the parallax by changing the electronic zoom magnification. 
         [0081]      FIG. 8  is a flowchart of a correction process executed by CPU  70 . 
         [0082]    Steps S 11  to S 14  are the same as steps  81  to  84  in  FIG. 2  respectively. 
         [0083]    In step S 14 , focused subject distance to the focused subject from the camera  10  is acquired via the focus measurement unit  77 . 
         [0084]    In step S 16 , a determination is made as to whether the acquired focused subject distance is the shortest imaging distance a of the ROM  72  or farther. In the case of Yes, the process moves to step  817 . In the case of No, the process moves to step  820 . Steps S 17  to S 18  are the same as steps  85  to  86  in  FIG. 2  respectively. 
         [0085]    In step S 19 , the extraction position setting unit  73  performs shift control on the horizontal position of the extraction range with a size determined by the electronic zoom magnification y so as to compensate for the expansion of the parallax by changing the optical zoom magnification (see  FIGS. 9A to 9H ). 
         [0086]    In step S 20 , a warning message is displayed on the display  26 , indicating that the subject is too close to generate a 3D image. Note that the warning message may be vocally reproduced on a speaker (not illustrated) included in the camera  10 . 
         [0087]    The above process assures a uniform correction of a viewpoint image shift due to a change in optical zoom magnification within a range in which the camera  10  can image and generate a three-dimensional image of the subject located at any position. 
       Third Embodiment  
       [0088]      FIG. 10  illustrates an electrical configuration of a camera  10  according to a third embodiment. The camera  10  includes configuration similar to that of the first and second embodiments and similar reference characters or numerals refer to similar elements. 
         [0089]    The corresponding point detection unit  7  uses stereo matching technique to obtain mutually corresponding points on the images  8 R and  8 L acquired respectively by the imaging elements SOA and SOB. A well-known technique may be applied to obtain the corresponding points. For example, the corresponding point detection unit  7  extracts parts as matrices (e.g., 3×3 pixels) from the images SR and SL respectively, calculates a correlation value thereof and obtains a pair of the corresponding points on the images SR and SL based on the correlation value. More specifically, the corresponding point detection unit  7  detects a boundary portion (edge component) where luminance and color difference are changed, from one of the images SR and SL to obtain a characteristic point, and detects the portion in another image having an edge component same as or similar to the edge component at the characteristic point as a corresponding point corresponding to the characteristic point. Alternatively, facial detection is performed on the images SR and SL and each of the head top and the jaw end of the detected face area may be detected as the characteristic point or the corresponding point of the respective images. The methods of extracting characteristic points and corresponding points need not be mutually dependent. 
         [0090]    The parallax measuring circuit  8  calculates the parallax based on the difference in position coordinate between the characteristic point and the corresponding point in the images SR and SL.  FIGS. 11A and 11B  illustrate an example of corresponding points. 
         [0091]    Referential marks SL and SR are images imaged from the viewpoints corresponding to the left eye and the right eye respectively. In  FIGS. 11A and 11B , the coordinate (coordinate along the horizontal direction (x direction)) of the characteristic point PL of the image SL is X 1 , the coordinate of the corresponding point PR of the image SR corresponding to the characteristic point PL is X 2 . Then, the parallax measuring circuit  8  calculates the parallax as d=X 1 −X 2 . 
         [0092]      FIG. 12  is a flowchart of a correction process executed by CPU  70 . 
         [0093]    Steps S 21  to S 24  are the same as steps S 1  to S 4  in  FIG. 2  respectively. 
         [0094]    In step S 25 , the corresponding point detection unit  7  obtains the characteristic point and the corresponding point of the images SR and SL. 
         [0095]    In step S 26 , the parallax measuring circuit  8  calculates the parallax d. 
         [0096]    Steps S 27  to S 30  are similar to the steps S 16 , S 19  and S 20  in  FIG. 8  respectively. Note that in step S 27 , a determination is made as to whether the parallax d calculated by the parallax measuring circuit  8  is the shortest imaging distance a. of the ROM  72  or farther. In the case of Yes, the process moves to step S 28 . In the case of No, the process moves to step S 30 . In step S 29 , the extraction range is set so that the parallax given to the viewer by the synthesized image generated from the corrected viewpoint image falls within the appropriate parallax range. Thus, the parallax data needs not be preliminarily stored in the ROM  72 . 
         [0097]    It should be noted that the correction processing described in the present description can be applied not only the optical axis shift but also the magnification error and the rotation error. 
         [0098]    In the above embodiments, the camera  10  includes two imaging optical systems for imaging two viewpoint images. However, the number of viewpoint images is not limited to two. For example, the above described process can be applied to a case where the number of the viewpoint images is more than two by selecting one of the plurality of viewpoint images as a standard, and performing an electronic zoom for the viewpoint images other than the standard images. 
         [0099]    The presently disclosed subject matter can be provided as a computer-readable program code for causing a device (such as an electronic camera, a stereoscopic camera or a computer which can obtain images imaged (photographed) from a plurality of viewpoints) to execute the above described process, a computer-readable recording medium on which the computer-readable program code is stored or a computer program product including the computer-readable program code. 
       REFERENCE SIGNS LIST  
       [0100]      7  . . . corresponding point detection unit,  14 A and  14 B . . . imaging optical system,  24 A and  24 B . . . zoom motor,  34  . . . three-dimensional image processing circuit,  50  . . . imaging element,  70  . . . CPU,  71  . . . ROM,  72  . . . , ROM,  73  . . . extraction position setting unit.