Source: https://patents.google.com/patent/CN103069814B/en
Timestamp: 2019-12-09 05:00:53
Document Index: 529669442

Matched Legal Cases: ['art 12', 'art 18', 'art 12', 'art 12', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 18', 'art 22', 'art 26', 'art 22', 'art22', 'art 26', 'art 22', 'art 26', 'art 26', 'art 28', 'art 26', 'art 28', 'art 26', 'art 28', 'art 28', 'art 26', 'art 28', 'art 28', 'art 54', 'art 58', 'art 54', 'art54', 'art 54', 'art 54', 'art 58', 'art 58', 'art 58', 'art58', 'art 58', 'art 58', 'art 54', 'art 58', 'art 58', 'art 54', 'art 58', 'art 12', 'art 18', 'art 12', 'art 18', 'art 54', 'art 54', 'art 58', 'art, 20']

CN103069814B - Image processing apparatus, image processing method, display unit, display methods and program - Google Patents
Image processing apparatus, image processing method, display unit, display methods and program Download PDF
CN103069814B
CN103069814B CN201180039376.8A CN201180039376A CN103069814B CN 103069814 B CN103069814 B CN 103069814B CN 201180039376 A CN201180039376 A CN 201180039376A CN 103069814 B CN103069814 B CN 103069814B
CN201180039376.8A
CN103069814A (en
太平博久
山垣浩司
2010-08-10 Priority to JP179144/2010 priority Critical
2010-08-10 Priority to JP179143/2010 priority
2010-08-10 Priority to JP2010179144A priority patent/JP5556494B2/en
2010-08-10 Priority to JP2010179143A priority patent/JP5556493B2/en
2011-08-04 Application filed by 株式会社尼康 filed Critical 株式会社尼康
2011-08-04 Priority to PCT/JP2011/004423 priority patent/WO2012020558A1/en
2013-04-24 Publication of CN103069814A publication Critical patent/CN103069814A/en
2016-05-11 Publication of CN103069814B publication Critical patent/CN103069814B/en
The invention provides image processing apparatus, image processing method, display unit, display methods and program, for strengthening the distance sense of photographic images. Image processing apparatus comprises: detecting element, for detection of the subject distance going out in the each region of photographic images; And image processing part, the distance perspective for the subject corresponding to detecting for each region of photographic images apart from the image in this region of change. As an example, image processing part can improve and is contained in the subject distance detecting and is greater than the blue component ratio in the region of a reference value. Or as an example, image processing part can will be arranged in the subject distance detecting and be greater than image blurringization in the region of a reference value.
Image processing apparatus, image processing method, display unit, display methods and program
The present invention relates to image processing apparatus, image processing method, display unit, display methods and program.
The right and left eyes that is beholder by stereoscopic display device provides the image with parallax, thereby makes to seeThe person of seeing watches the image with far and near sense. In addition, the third dimension tool of watching the people of stereo-picture to experienceThere is individual difference. That is to say, even if in the time showing same 3D content, somebody can experience very consuminglyThird dimension, and somebody just almost experiences less than third dimension.
Patent documentation 1: Unexamined Patent 6-324620 communique
Patent documentation 2: JP 2004-289527
The problem that invention will solve:
But, in stereoscopic display device, do not carry out following processing: to the image from outside inputFurther strengthen or weaken the processing of far and near sense after image is shown. In addition, show stereo-pictureThe third dimension that preferably can suitably experience with each individual of display unit be suitable for stereo-picture. But, eachThe third dimension that individual experiences there are differences, and therefore, being difficult to commonly provides and experience suitably everyoneRelief image.
Therefore,, in side of the present invention, object is to provide the image that can address the above problem and processes dressPut, image processing method, display unit, display methods and program. This object by claims solelyThe Feature Combination of vertical claim is reached. In addition, to define the present invention further effective for dependent claimsConcrete example.
The scheme of dealing with problems:
In order to address the above problem, in the present invention's the first form, provide image processing apparatus and image processingMethod and program, comprising: detecting element, for detection of the subject distance going out in the each region of photographic images; WithAnd image processing part, change for the subject distance corresponding to detecting for each region of above-mentioned photographic imagesBecome the distance perspective of the image in this region.
In the present invention's the second form, provide display unit, display methods and program, comprising: display part, forBe shown as beholder and give relief image; And changing unit, for the setting corresponding to beholder, changeThe relief reinforcement becoming in the above-mentioned image being shown by above-mentioned display part is measured.
In addition, foregoing invention content does not list whole essential feature of the present invention, the feature of these featuresThe sub-portfolio of group also can form invention.
Fig. 1 represents the structure of image processing apparatus 10 described in the first embodiment.
Fig. 2 represents that carrying out image by image processing apparatus described in the first embodiment 10 processes previous original batTake the photograph an example of image.
Fig. 3 represents that carrying out image processing shooting afterwards by image processing apparatus described in the first embodiment 10 schemesOne example of picture.
Fig. 4 represents an example of the structure of detecting element 14 described in the first embodiment.
Fig. 5 represents the structure of image processing apparatus 10 described in the variation of the first embodiment.
Fig. 6 represents the structure of display unit 50 described in the second embodiment.
Fig. 7 represents the structure of display unit 50 described in the variation of the second embodiment.
Fig. 8 represents an example of the hardware configuration of computer 1900 described in present embodiment.
Below by invention embodiment, the present invention will be described, but following embodiment is not wanted rightAsk the related invention of book to limit. And the Feature Combination illustrating in embodiment is all not alsoEssential feature of the present invention.
Fig. 1 represents the structure of image processing apparatus 10 described in the first embodiment. Described in the first embodiment, schemeAs treating apparatus 10 to the photographic images real-time image processing being photographed by camera. As an example, image processingDevice 10 is arranged at the image input stage of display unit. In addition, image processing apparatus 10 also can be arranged atThe video output stage of camera.
Image processing apparatus 10 comprises: input part 12, detecting element 14, range information obtaining section 16, imageHandling part 18, efferent 20. Input part 12 is for inputting the photographic images of being taken by camera. As an example,Input part 12 can be read the photographic images being stored in storage medium, also can be to from transmissions such as broadcasting organizationsPhotographic images carry out demodulation.
Detecting element 14 is for detection of the subject distance going out in each region of photographic images. Subject is apart from expressionDistance in the time taking from camera to subject.
As an example, detecting element 14 calculates this district corresponding to the blue component ratio in each region in photographic imagesSubject distance in territory. For example, take the distant subject of subject when landscape due to be subject to fromCamera, to the air layer impact between this subject, can show more blueness than the color of subject itselfComponent ratio. Therefore, detecting element 14 can be corresponding to the blue composition comprising in each region in photographic imagesRatio is inferred the subject distance in this region.
As an example, detecting element 14 calculates each region average color separately in photographic images, to what calculateThe color of average color is resolved, and calculates the blue component ratio that this average color comprises. And, asOne example, the blue component ratio of regional is more, and detecting element 14 is judged subject apart from just far away,Infer accordingly the subject distance in this region.
In addition, detecting element 14 also can be corresponding to the space of the profile of the subject demonstrating in photographic images frequentlyRate calculates the subject distance of this subject. For example, while, the taking landscape distant subject of subjectDue to the impact from camera to the air layer this subject, it is heavy that fog-level can become. Therefore, detecting element14 can infer this district corresponding to the spatial frequency of the profile of the subject comprising in each region in photographic imagesSubject distance in territory.
As an example, detecting element 14 extracts the marginal portion of subject separately, each region in photographic images, meterThe spatial frequency at the edge extracting. And as an example, the spatial frequency of regional is lower,It is just far away that detecting element 14 is judged subject distance, infers accordingly the subject distance in this region.
Range information obtaining section 16 obtains and is illustrated in each region of being measured by camera when photographic images is takenThe range information of subject distance. As an example, range information obtaining section 16 obtain be called as depth map,The two-dimensional signal of the subject surface distance in representing from camera to picture.
Image processing part 18 corresponding to the subject detecting for the each region of photographic images apart from change this regionThe distance perspective of image.
As an example, image processing part 18 is corresponding to by detecting element 14, the each region of photographic images being detectedSubject is apart from changing the blue component ratio in each region, thereby changes the distance perspective of the image in this region. ExampleAs, in each region of photographic images, image processing part 18 increases with the subject detecting apart from respective amountBlue composition.
More specifically, image processing part 18 can preset a reference value of subject distance, and raising detectsSubject distance be greater than the blue component ratio comprising in the region of a reference value. For example, image processing part 18Set these two a reference values of 1km, 10km. And image processing part 18 arrives for 1km in subject distanceIn the region of 10km, blue component ratio is improved to 20%, in the region more than 10km by blue composition ratioExample improves 40%, improves by this way blue component ratio.
In addition, as an example, image processing part 18 is corresponding to by detecting element 14, the each region of photographic images being examinedThe subject distance of measuring is the obfuscation of each region, thereby changes the distance perspective of the image in this region. For example,In each region of photographic images, image processing part 18 with the corresponding fuzzy quantity of the subject distance detectingBy image blurringization.
More specifically, image processing part 18 can preset a reference value of subject distance, by what detectSubject distance is greater than image blurringization in the region of a reference value. For example, image processing part 18 set 1km,These two a reference values of 10km. And, subject distance be 1km in the region of 10km, image placeReason portion 18 with the first fuzziness by image blurringization, in the region more than 10km, fuzzy to be greater than firstThe second fuzziness of degree is by image blurringization, by this way by image blurringization.
By carrying out such processing, image processing part 18 can improve subject close together region and being shotDistance perspective between the distant region of body poor. Thereby make image processing part 18 can strengthen photographic imagesFar and near sense.
In addition, further, image processing part 18 can further use by range information obtaining section 16 and getThe range information obtaining, the distance perspective of the image in the change each region of photographic images. For example,, by range informationThe distance that obtaining section 16 obtains is than in the farther region of preset distance, and image processing part 18 is corresponding to by detectingThe subject that portion 14 detects is apart from the distance perspective that changes the image in this region. For example, get at range informationPortion 16 detects in the region into infinity, image processing part 18 is corresponding to being detected by detecting element 14Subject is apart from the distance perspective that changes the image in this region.
Then the subject distance region farther unlike preset distance of, being obtained by range information obtaining section 16In, image processing part 18 corresponding to the subject being obtained by range information obtaining section 16 apart from change this regionThe distance perspective of image. For example, detect the region of the distance that is not infinity in range information obtaining section 16In, image processing part 18 corresponding to the subject being detected by range information obtaining section 16 apart from change this districtThe distance perspective of the image in territory. Accordingly, image processing part 18 can the subject distance based on wider scope changeBecome the distance perspective of image, thereby further strengthen far and near sense.
Efferent 20 will carry out image photographic images after treatment by image processing part 18 and output to outside. ExampleAs, if this image processing apparatus 10 is arranged in display unit, efferent 20 can be felt quilt by distanceStrengthened photographic images outputs to monitor. In addition, for example, if image processing apparatus 10 is arranged at phaseOn machine, the photographic images of efferent 20 after distance sense can being reinforced is kept in storage medium.
Fig. 2 represents that carrying out image by image processing apparatus described in the first embodiment 10 processes previous original batTake the photograph an example of image. Fig. 3 represents that carrying out image by image processing apparatus described in the first embodiment 10 processes itAfter an example of photographic images.
In the photographic images shown in Fig. 2 and Fig. 3, for example comprise personage's (close shot), family (middle scape), mountain(distant view) and sky (super distant view). In addition, in Fig. 3, × mark represents to make blue component ratio to improveAfter region.
As an example, image processing apparatus 10 make to be contained in for example mountain (distant view) in image shown in Fig. 2 andSky (super distant view) improves blue component ratio. Further, image processing apparatus 10 is to subject distanceBlue component ratio is further improved in the region of sky (super distant view) that also will be far away than the region of mountain (distant view).
In addition, image processing apparatus 10 can or be got and generation on the basis of having improved blue component ratioGround, make mountain (distant view) and sky (super distant view) obfuscation. Now, image processing apparatus 10 also canThe region of sky (super distant view) that also will be far away than the region of mountain (distant view) to subject distance is with larger mouldPaste amount is by image blurringization.
So, image processing apparatus 10 can make people experience more consumingly subject in photographic imagesRange difference between territory, Yu compare far field, close together region. Accordingly, by image processing apparatus 10, Neng GoujiaDistance sense in strong photographic images, and do not change the profile of the subject in the photographic images being photographed by cameraAnd composition.
Fig. 4 represents an example of the structure of detecting element 14 described in the first embodiment. As an example, as Fig. 4 instituteShow, detecting element 14 has: judging part 22, blue composition detecting element 24, blue composition storage part 26 and fortuneCalculation portion 28.
Judging part 22 is for judging the type of subject in the each region of photographic images. As an example, judging part22 compare the subject comprising in the picture pattern of registration in advance and photographic images, judge photographic imagesIn the subject that comprises be which kind of type.
Blue composition detecting element 24 is for detection of the blue component ratio that goes out each region in photographic images. As oneExample, to regional, blue composition detecting element 24 calculates the average color in this region, and it is flat that analytical Calculation goes outThe color of homochromatic, and calculate the blue component ratio that this average color comprises.
In blue composition storage part 26, there is the blue composition that should comprise by the type stores of subject in imageRatio. As an example, for each type of the subject that can be detected by judging part 22, blue compositionStorage part 26 all stores the original contained blue component ratio of color on the subject surface of the type. DoBe an example, blue composition storage part 26 can be stored by user and wait the value of registration in advance.
For the each region in photographic images, operational part 28 reads out and is judged from blue composition storage part 26Portion 22 judges the blue component ratio of the subject of type. Next, operational part 28 calculates by indigo plantThe portion that is judged 22 in the blue component ratio of every kind of subject that colour content storage part 26 is stored judges classThe blue component ratio of the subject of type is poor with the blue component ratio being detected by blue composition detecting element 24Value. Then, the difference of operational part 28 based on calculating calculated the subject distance in this region.
For example, operational part 28 calculates the blue component ratio detecting from blue composition detecting element 24 and deductsWhether the difference obtaining after the blue component ratio of every kind of subject of being stored by blue composition storage part 26 is baseMore than accurate value. Then,, in the time that difference is less than a reference value, operational part 28 is judged subject distance for benchmarkBelow distance. In addition, when the difference calculating is a reference value when above, operational part 28 is judged subjectDistance is more than reference range.
By this detecting element 14, in the time having registered in advance the subject of type and be included in photographic images, energyEnough subject distances that correctly detects.
Fig. 5 represents the structure of image processing apparatus 10 described in the variation of the first embodiment. This variation instituteState image processing apparatus 10 and adopt structure and the function roughly the same with image processing apparatus 10 shown in Fig. 1,Therefore,, for the parts of the structure roughly the same with the parts shown in Fig. 1 and function mark identical symbol, removeBeyond following difference, all carry out that description thereof is omitted.
Described in this variation, image processing apparatus 10 further comprises: positional information obtaining section 32 and meteorological barPart obtaining section 34. Positional information obtaining section 32 is for obtaining the positional information of the camera site that represents photographic images.For example, there is the phase function of GPS (GlobalPositioningSystem, global positioning system) receiverEnough output is accompanied with the photographic images of the positional information that represents camera site. Positional information obtaining section 32 obtain byThe positional information comprising in the photographic images of this camera output.
The shooting date of meteorological condition obtaining section 34 based on photographic images and time and by positional information obtaining section32 positional informations that obtain, obtain the meteorological condition of photographic images in the time taking. For example, camera can be exportedBe accompanied with the photographic images of shooting date and time. Positional information obtaining section 32 obtains to be exported by this cameraThe shooting date comprising in photographic images and time. Then, meteorological condition obtaining section 34 is for example got by networkMust be in obtained shooting date and time, place shown in the positional information being obtained by positional information obtaining section 32Meteorological condition.
Described in this variation, detecting element 14 detects corresponding to the meteorological condition being obtained by meteorological condition obtaining section 34Go out the subject distance in the each region of photographic images. For example,, when the meteorological condition of photographic images in the time takingDuring for " fine ", due to fresh air, therefore can clearly photograph the wheel of the distant subject of subjectWide. On the contrary, when the meteorological condition of photographic images in the time taking is while being " rain " etc., due to stuffy, because ofThis can not clearly photograph the profile of the distant subject of subject. Therefore, as an example, in inspectionGoing out the spatial frequency of portion 14 based on subject profile infers in the processing of subject distance, even the same spaceFrequency, farther when the subject distance that meteorological condition is inferred when " fine " is " rain " etc. than meteorological condition.
In addition, described in this variation, detecting element 14 can or be got on the basis of obtained meteorological conditionAnd the ground in generation, corresponding to the positional information being obtained by positional information obtaining section 32, detects the each district of photographic imagesSubject distance in territory. For example, according to the difference of the spot for photography of photographic images, some places air is clearNewly, and some places are stuffy. The quilt that (for example, beyond big city) photographs in place with fresh airThe profile of taking the photograph the distant subject of body is more clear. On the contrary, in stuffy place (for example, in big cityIn city) profile of the distant subject of the subject that photographs is unintelligible. Therefore, as an example, inspectionGoing out portion 14 can infer in the processing of subject distance in the spatial frequency based on subject profile, even withOne spatial frequency, the subject distance that infer in (for example, beyond big city) in spot for photography with fresh airAlso can be for example, than stuffy spot for photography (big city) more farther.
By above content, image processing apparatus 10 can detect subject more accurately described in this variationDistance, thus can strengthen exactly the distance sense in photographic images.
In addition, described in this variation, detecting element 14 can be taken same subject by setting different focal lengthsMultiple photographic images, and detect the subject distance in each region of multiple photographic images based on this. Now,As an example, detecting element 14, to each of multiple photographic images, detects the focal zone in picture. SoAfter, detecting element 14 is inferred the subject distance in picture inner focusing region according to multiple photographic images focal length separately.Accordingly, for example, even if do not comprise range information in photographic images, detecting element 14 also can generate and the depth of fieldScheme identical range information.
In addition, detecting element 14 can be taken multiple photographic images by setting different apertures described in this variation,And detect the subject distance in the each region of multiple photographic images based on this. Now, in multiple photographic imagesPicture in the expansion successively along with moving to aperture closed condition from aperture open mode of the scope of focusing.Therefore, as an example, detecting element 14 is to each focusing range in picture relatively in multiple photographic images,Thereby the subject distance in each region in deduction picture. Accordingly, for example, even do not comprise in photographic imagesWhen range information, detecting element 14 also can generate the range information identical with depth map.
Fig. 6 represents the structure of display unit 50 described in the second embodiment. Display unit 50 is for being shown asBeholder gives relief image. In the second embodiment, display unit 50 demonstrations offer beholderThe right eye of right eye is with image and offer the left eye image of left eye, as give relief figure for beholderPicture. Right eye is the image at left and right directions with parallax with image and left eye image, watches by offeringPerson's right eye and left eye, can make beholder experience third dimension.
Display unit 50 comprises: input part 54, image rib 56, display part 58, changing unit 60. DefeatedEnter portion 54 and give relief image for be input as beholder from outside. As an example, input part 54 is defeatedEnter from the image of storage medium playback or to broadcast singal and carry out the image demodulation. In this example, input part54 inputs offer the right eye image of observer's right eye and offer the left eye image of observer's left eye.
The image real-time image processing that image rib 56 is inputted input part 54, strengthens beholder and is viewing and admiringTime the third dimension experienced, or weaken third dimension. In addition, in the time that setting does not add intensity, image rib56 images that can not input input part 54 implement that any image is processed and output image intactly.
Display part 58 for show export from image rib 56 give relief image for beholder.In the present embodiment, display part 58 show right eye image and left eye image vertical as giving for beholderThe image of body sense.
As an example, display part 58 has monitor and special eyeglasses. Monitor is used for showing right eye imageAnd left eye image. Special eyeglasses is for only offering beholder's by the right eye showing on monitor with imageRight eye, and the left eye showing on monitor is only offered to beholder's left eye with image. Accordingly, display part58 can make beholder experience third dimension.
Changing unit 60 is shown the third dimension of image by display part 58 for the setting changing corresponding to beholderReinforcement amount. Particularly, changing unit 60 changes the parameter of image rib 56 for image processing, therebyControl relief reinforcement amount.
Herein, as an example, figure is strengthened or weakened to image rib 56, by carrying out following image processing,Third dimension in picture.
When taking when landscape, the distant subject of subject is owing to being subject to from camera to this subjectThe impact of air layer, can show than the more blue component ratio of the color of subject itself. Now,In the time strengthening third dimension, image rib 56 is the blue component ratio apart from raising image corresponding to subject.On the contrary, in the time weakening third dimension, image rib 56 becomes apart from the blueness that reduces image corresponding to subjectDivide ratio. Now, image rib 56 is for subject distance subject far away, just by blue composition ratioExample improves more significantly or reduces. Accordingly, the solid of photographic images can be strengthened or weaken to image rib 56Sense.
And now, changing unit 60, corresponding to beholder's setting, changes by image rib 56 quiltTake the photograph the reinforcement amount of the blue component ratio of body. , changing unit 60 is corresponding to beholder's the blue one-tenth of setting changingDivide raising amount or the reduction of ratio. Accordingly, changing unit 60 can be taken corresponding to beholder's setting changingThe third dimension of image.
In addition, when taking when landscape, the distant subject of subject is because meeting is subject to from camera to this quiltTake the photograph the impact of the air layer of body, can seem fuzzyyer than the profile of subject self. Therefore, three-dimensional when strengtheningWhen sense, image rib 56 can be corresponding to subject apart from the fog-level that increases image. On the contrary, whenWhile weakening third dimension, image rib 56 can reduce corresponding to subject distance the fog-level of image. ThisTime, image rib 56 is for subject distance subject far away, just fuzzy quantity is improved more significantly orReduce. Accordingly, the third dimension of photographic images can be strengthened or weaken to image rib 56.
Then, now, changing unit 60 corresponding to beholder's setting changing by image rib 56 to being shotThe reinforcement amount of the fog-level of body. , changing unit 60, corresponding to beholder's setting, changes image blurring amountRaising amount or reduction. What accordingly, changing unit 60 can be corresponding to beholder's setting changing photographic imagesThird dimension.
In addition, if expanded image pitch between right eye image and left eye use image from, beholderConverge like the spokes of a wheel at the hub approach parallel, thereby make people think that subject is far. Therefore,, in the time strengthening third dimension, image addsStrong portion 56 can be corresponding to the subject of this subject apart from increasing the demonstration of image for the right eye of same subjectPosition and left eye with the image pitch between the display position of image from. On the contrary, in the time weakening third dimension,Image rib 56 can corresponding to subject distance reduce image pitch from. Now, image rib 56For subject distance subject far away, just by image pitch from raising more significantly or minimizing. Accordingly,The third dimension of photographic images can be strengthened or weaken to image rib 56.
And now, changing unit 60, corresponding to beholder's setting, changes by image rib 56 quiltThe image pitch of taking the photograph body from reinforcement amount. , changing unit 60, corresponding to beholder's setting, changes between imageRaising amount or the reduction of distance. Accordingly, changing unit 60 can, corresponding to beholder's setting, change and takeThe third dimension of image.
As mentioned above, display unit 50 can freely change the relief degree that beholder experiences. According toThis, can be provided and give suitable relief stereo-picture for each individual by display unit 50.
Herein, in the photographic images shown in Fig. 2 and Fig. 3, for example comprise personage's (close shot), family (middle scape),Mountain (distant view) and sky (super distant view). Image shown in Fig. 2 is to be added by image described in the second embodimentStrong portion 56 carries out an example of the original photographic images before image processing. In addition, image is as shown in Figure 3Carried out an example of image processing photographic images afterwards by image rib described in the second embodiment 56. In addition,In Fig. 3, × mark represents the region that blue component ratio has been enhanced.
As an example, image rib 56 is to for example mountain (distant view) and the sky that comprise in image shown in Fig. 2(super distant view) improves blue component ratio. Further, image rib 56 is (far away than mountain to subject distanceScape) blue component ratio improves get Geng Gao by farther sky (super distant view) region, region.
In addition, image rib 56 can or instead make on the basis of improving blue component ratioMountain (distant view) and sky (super distant view) obfuscation. Now, image rib 56 to subject distance than mountainThe region of the farther sky (super distant view) in the region of (distant view) with larger fuzzy quantity by image blurringization.
So, image rib 56 can make people experience more consumingly subject distance in photographic imagesImmediate area and compared with the range difference between territory, far field. Accordingly, can strengthen shooting figure according to image rib 56The distance sense of picture, thereby the third dimension of reinforcement image.
Fig. 7 represents the structure of display unit 50 described in the variation of the second embodiment. Aobvious described in this variationShowing device 50 adopts structure and the function roughly the same with display unit 50 shown in Fig. 6, therefore to Fig. 6Shown in the roughly the same structure of parts and the parts of function mark identical symbol, except following difference,All carry out that description thereof is omitted.
Described in this variation, display unit 50 further has detecting element 70. Detecting element 70 is for detection of going out to answerWhen the subject distance of the image regional being shown by display part 58.
As an example, detecting element 70 detects display position and the left eye of the right eye image in same subjectWith the image pitch between the display position of image from. In the time that people observes object far away, right eye and left eyeConvergence angle can be narrower than observing convergence angle when closer objects. Therefore, the right eye image in same subjectDisplay position and left eye by the image pitch between the display position of image from along with the change of subject distance is largeAnd become large. Therefore, detecting element 70 can be for image and left eye figure for the right eye of being inputted by input part 54Picture detect in picture the image pitch in each region from, thereby infer the subject distance in each region.
Then, in this variation, the subject distance that image rib 56 detects for detecting element 70The larger part of difference, the difference of the image range of a signal in the image just display part 58 being shown increases moreGreatly, to increase relief reinforcement amount. Accordingly, image rib 56 can carry out corresponding to subject distanceReinforcement progressively.
In addition, in this variation, changing unit 60 can be calibrated before showing image. Particularly,Changing unit 60 is presented at the image that produces different stereo effects in display part 58, selects third dimension by beholderReinforcement measure to carry out calibration.
Then, after calibration is carried out, image rib 56 is measured according to the relief reinforcement of being selected by beholder,The image of exporting from input part 54 is strengthened. Then, display part 58 is to being strengthened by image rib 56After image show. Accordingly, changing unit 60 can make observer observe image with suitable third dimension.
In addition, described in the second embodiment, display unit 50 can further have described in the first embodiment and schemesThe function of picture treating apparatus 10. Accordingly, described in the second embodiment, display unit 50 can be strengthened shooting figureThe distance sense of picture, can provide and give the relief stereo-picture that is suitable for each individual simultaneously.
Fig. 8 represents an example of the hardware configuration of computer 1900 described in present embodiment. Electricity described in present embodimentBrain 1900 comprises: have by the interconnective CPU2000 of console controller 2082, RAM2020, figureThe CPU periphery of controller 2075 and display unit 2080, has by IOC 2084 and connectsBe connected to communication interface 2030, hard disk drive 2040 and the CD-ROM drive of console controller 20822060 input and output portion, has the ROM2010, the disk drive that are connected in IOC 2084Tradition (legacy) the input and output portion of device 2050 and I/O chip 2070.
Console controller 2082 is by RAM2020 and the CPU2000 that accesses RAM2020 with high transfer rateAnd graphics controller 2075 is connected. The journey of CPU2000 based on being stored in ROM2010 and RAM2020Order is carried out work, and each several part is controlled. Graphics controller 2075 is obtained CPU2000 etc. to arrangeThe view data generating in frame buffer in RAM2020 is also presented in display unit 2080. Alternatively,Graphics controller 2075 also can comprise in inside for storing the view data that generated by CPU2000 etc.Frame buffer.
IOC 2084 is using console controller 2082 and communicating by letter as high speed input/output unitInterface 2030, hard disk drive 2040, CD-ROM drive 2060 are connected. Communication interface 2030Communicate by network and other devices. Hard disk drive 2040 is stored by the CPU2000 in computer 1900The program and the data that use. CD-ROM drive 2060 is from CD-ROM2095 fetch program or data,Offer hard disk drive 2040 through RAM2020.
In addition, on IOC 2084, be connected with ROM2010 and floppy disk 2050And the low speed input/output unit of I/O chip 2070. When ROM2010 stores computer 1900 and startsThe start-up routine of carrying out and/or depend on the program etc. of the hardware of computer 1900. Floppy disk 2050 is from softCoil 2090 fetch programs or data, offer hard disk drive 2040 through RAM2020. I/O chipFloppy disk 2050 is connected in IOC 2084 by 2070, and by for example LPT, stringRow mouth, keyboard mouth, mouse mouth etc. are connected various input/output units with IOC 2084.
The program that offers hard disk drive 2040 through RAM2020 be stored in floppy disk 2090,In the storage medium such as CD-ROM2095 or IC-card, provided by user. Program is read from storage medium,In hard disk drive 2040 in RAM2020 is installed on computer 1900, in CPU2000, carry out.
Be installed on computer 1900 and make computer 1900 bring into play image processing apparatus 10 described in the first embodimentThe program of function comprise: input module, check-out module, range information obtain module, image processing module,Output module. These programs or module impel CPU2000 etc. to make computer 1900 bring into play input part 12, detectPortion 14, range information obtaining section 16, image processing part 18 and efferent 20 function separately.
The information processing of recording in these programs is read into computer 1900, thereby as software and above-mentioned various hardThe concrete device that part resource cooperates mutually, performance input part 12, detecting element 14, range information obtaining section 16, figureThe function of picture handling part 18 and efferent 20. Then, specifically install by these, realize and this enforcement sideThe calculation that described in formula, the application target of computer 1900 carries out information accordingly or processing, build accordingly and useThe corresponding distinctive image processing apparatus 10 of object.
In addition, be installed on computer 1900 and make computer 1900 bring into play display unit described in the second embodimentThe program of 50 function comprises: input module, image reinforcing module, display module, change module. TheseProgram or module impel CPU2000 etc. to make computer 1900 bring into play input part 54, image rib 56, showPortion 58 and changing unit 60 function separately.
The information processing of recording in these programs is read into computer 1900, thereby as software and above-mentioned various hardThe concrete device that part resource cooperates mutually, performance input part 54, image rib 56, display part 58 and changeThe more function of portion 60. Then, specifically install by these, realize and computer 1900 described in present embodimentApplication target calculation or the processing accordingly information carried out, build with application target accordingly peculiar accordinglyDisplay unit 50.
As an example, in the time communicating between computer 1900 and external device (ED) etc., CPU2000 carries out and addsBe loaded in the signal procedure on RAM2020, and based on being recorded in contents processing in signal procedure to communication interface2030 instruction communication process. Communication interface 2030 is subject to the control of CPU2000, by RAM2020, hardThe transmission buffering area arranging on the storage devices such as disk drive 2040, floppy disk 2090 or CD-ROM2095 etc.The transmission data reading of middle storage also sends to network, or the reception data that receive from network are write and establishedBe placed in reception buffer zone on storage device etc. Thus, communication interface 2030 can be passed through DMA(DirectMemoryAccess, direct memory access) mode transmits transmitting-receiving between itself and storage deviceData, alternatively, CPU2000 can be from the storage device of transfer source or communication interface 2030 sense datas,Write data into and transmit in the communication interface 2030 or storage device of destination, thereby realize transceiving dataTransmit.
In addition, CPU2000 will be all by modes such as DMA transmit or necessary part from hard disk drive2040, CD-ROM drive 2060 (CD-ROM2095), floppy disk 2050 (floppy disk 2090)Be read into RAM2020 Deng the file of storing in external memory or database, on RAM2020Data are carried out various processing. Then, CPU2000 by modes such as DMA transmit by the number after end processAccording to writing back in external memory. In this processing, RAM2020 can regard interim preservation outside as and depositThe content of storage device, therefore in the present embodiment, is referred to as RAM2020 and external memory etc. to depositReservoir, storage part or storage device etc. Various programs in present embodiment, data, form, databaseBe stored on such storage device etc. various information, as the object of information processing. In addition, CPU2000A part of RAM2020 can be stored in buffer storage, in buffer storage, read and write. ?Make in this mode, buffer storage is also born the partial function of RAM2020, therefore in present embodimentIn, except the situation that difference represents, buffer storage be also contained in RAM2020, memory and/orIn storage device.
In addition, CPU2000 carries out and is specified and wrapped by the command line of program the data of reading from RAM2020Each containing the various calculations of recording in present embodiment, information processing, condition judgment, information retrieval replacement etc.Plant and process, and write back RAM2020. For example, CPU2000 is in the time carrying out condition judgment, by this enforcement sideVarious variablees shown in formula are compared with its dependent variable or constant, judge whether to meet large and small, above,Below, phase equal conditions, in the time that condition is set up while being false (or), be branched off into different command lines or tuneGo out subprogram.
In addition, CPU2000 can retrieve the file or the database information that are stored in storage device.For example, when the property value of the second attribute and the property value of the first attribute respectively associated multiple entries be stored inIn storage device time, CPU2000 retrieves and specifies the first attribute in the multiple entries that are stored in storage deviceThe consistent entry of condition of property value, and read the property value of the second attribute that is stored in this entry, justCan obtain the property value of the second attribute being associated with the first attribute that meets institute's fixed condition.
Program above or module can be stored in exterior storage medium. As storage medium, exceptBeyond floppy disk 2090, CD-ROM2095, can also use the optical storage medium such as DVD or CD, MOEtc. semiconductor memories such as photomagnetism storage medium, tape-shaped medium's, IC-card etc. In addition, can be by special logicalThe storage device such as hard disk or RAM arranging in the server system connecting on letter net or internet is as storageMedium, and can program be offered to computer 1900 by network.
Above, use embodiment to carry out explanation of the present invention, but technical scope of the present invention is not limited toState the scope that embodiment is recorded. It will be apparent to those skilled in the art that the basis at above-mentioned embodimentOn various changes or improvement in addition. From the record of claim, this change or improved realityThe mode of executing is also included in the technical scope of the present invention.
It should be noted in the discussion above that the device shown in claims, description and accompanying drawing, system, program withAnd action in method, sequentially, the execution sequence of each processing such as step and stage, as long as no bright especiallyShow " more early ", " early than " etc., and as long as in the not postpose processing of output of previous processed, canTo realize with random order. About the motion flow in claims, description and accompanying drawing, for convenience of andUse " first ", " then " etc. to be illustrated, but and do not mean that and must implement in this order.
10 image processing apparatus, 12 input parts, 14 detecting elements, 16 range information obtaining sections, 18 imagesHandling part, 20 efferents, 22 judging parts, 24 blue composition detecting elements, 26 blue composition storage parts,28 operational parts, 32 positional information obtaining sections, 34 meteorological condition obtaining sections, 50 display unit, 54 inputsPortion, 56 image ribs, 58 display parts, 60 changing units, 70 detecting elements, 1900 computers, 2000CPU,2010ROM, 2020RAM, 2030 communication interfaces, 2040 hard disk drives, 2050 floppy disks,2060CD-ROM driver, 2070 I/O chips, 2075 graphics controllers, 2080 show dressPut, 2082 console controllers, 2084 IOCs, 2090 floppy disks, 2095CD-ROM.
Detecting element, detects the subject distance in the each region of photographic images; And
Image processing part, corresponding to the subject detecting for each region of described photographic images apart from changeThe distance perspective of the image in this region;
Described detecting element has:
Judging part, judges the type of the subject in each region in described photographic images;
Blue composition detecting element, detects the blue component ratio in each region in described photographic images;
Blue composition storage part, by the blue component ratio that should comprise in the type stores image of subject;And
Operational part, in the each region in described photographic images, based on depositing in described blue composition storage partThe blueness of the subject of the type of being judged by described judging part in the blue component ratio of each subject of storageDifference between component ratio and the blue component ratio that detected by described blue composition detecting element, calculatesThe subject distance in this region.
2. image processing apparatus according to claim 1, wherein, described image processing part improves in inspectionThe subject distance of measuring is greater than the blue component ratio comprising in the region of a reference value.
3. image processing apparatus according to claim 1, wherein, described image processing part will be positioned at inspectionThe subject distance of measuring is greater than image blurringization in the region of a reference value.
4. according to the image processing apparatus described in any one in claim 1～3, wherein, described detecting elementCalculate the quilt of this subject corresponding to the spatial frequency of the profile of the subject showing in described photographic imagesTake the photograph body distance.
5. according to the image processing apparatus described in any one in claim 1～3, wherein, further comprise:Meteorological condition obtaining section, the shooting date based on described photographic images and time are obtained described photographic images and are clappingMeteorological condition while taking the photograph;
Described detecting element detects being shot in the each region of described photographic images corresponding to obtained meteorological conditionBody distance.
6. according to the image processing apparatus described in any one in claim 1～3, wherein, further comprise:Positional information obtaining section, obtains the positional information of camera site that represents described photographic images;
Described detecting element detects in the each region of described photographic images corresponding to obtained described positional informationSubject distance.
7. image processing apparatus according to claim 6, wherein, further comprises: meteorological condition is gotThe portion of obtaining, the described positional information based on described photographic images obtains the meteorological bar of described photographic images in the time takingPart;
Described detecting element detects in the each region of described photographic images corresponding to obtained described meteorological conditionSubject distance.
8. according to the image processing apparatus described in any one in claim 1～3, wherein, further comprise:Range information obtaining section, obtains and is illustrated in each region of being measured by camera when described photographic images is takenThe range information of subject distance;
Described image processing part is for be distal to preset distance by the obtained distance of described range information obtaining sectionRegion, the distance perspective corresponding to the subject being detected by described detecting element apart from the image in this region of change;
For the region that is not distal to preset distance by the obtained distance of described range information obtaining section, corresponding toChanged the distance perspective of the image in this region by the obtained distance of described range information obtaining section.
9. image processing apparatus according to claim 8, wherein, described detecting element is set based on passing throughDifferent focal and multiple photographic images of photographing, detect being shot in the each region of described multiple photographic imagesBody distance.
10. image processing apparatus according to claim 8, wherein, described detecting element is based on by establishingMultiple photographic images of determining different apertures and photograph, detect the quilt in the each region of described multiple photographic imagesTake the photograph body distance.
11. image processing apparatus according to claim 1, wherein, further comprise:
Meteorological condition obtaining section, obtains the meteorological condition of described photographic images in the time taking;
Described detecting element is inferred in the processing of subject distance, in the spatial frequency of the profile based on subjectMaking is the same space frequency, and the subject distance of inferring when described meteorological condition is " fine " is than described meteorologyCondition is far away while being " rain ".
12. 1 kinds of image processing apparatus, comprising:
Detecting element, detects the subject distance in the each region of photographic images;
Range information obtaining section, obtains and is illustrated in each region of being measured by camera when described photographic images is takenIn the range information of subject distance; And
Image processing part, for the district that is distal to preset distance by the obtained distance of described range information obtaining sectionTerritory, the distance perspective corresponding to the subject being detected by described detecting element apart from the image in this region of change,For the region that is not distal to preset distance by the obtained distance of described range information obtaining section, corresponding to by instituteState the distance perspective that the obtained distance of range information obtaining section changes the image in this region;
13. 1 kinds of image processing methods, wherein, comprise the steps:
Detect the subject distance in the each region of photographic images by detecting element; And
Image corresponding to the subject detecting for each region of described photographic images apart from this region of changeDistance perspective;
The step that detects described subject distance comprises:
Judge the type of the subject in each region in described photographic images;
Detect the blue component ratio in each region in described photographic images;
By the blue component ratio that should comprise in the type stores image of subject; And
In each region in described photographic images, the blue composition ratio of the each subject based on described storageThe blue component ratio of the subject of the type of judging described in example with described in the blue component ratio that detectsBetween difference, calculate the subject distance in this region.
14. 1 kinds of display unit, wherein, comprising:
Display part, is shown as beholder and gives relief image;
Changing unit, corresponding to beholder's setting, changes standing in the described image being shown by described display partThe reinforcement amount of body sense; Judging part, judges the type of the subject in each region in photographic images;
Blue composition storage part, by the blue component ratio that should comprise in the type stores image of subject;
Operational part, in the each region in described photographic images, based on depositing in described blue composition storage partThe blueness of the subject of the type of being judged by described judging part in the blue component ratio of each subject of storageDifference between component ratio and the blue component ratio that detected by described blue composition detecting element, calculatesThe subject distance in this region; And
Image processing part, corresponding to the subject detecting for each region of described photographic images apart from changeThe distance perspective of the image in this region.
15. display unit according to claim 14, wherein, further comprise: image rib,For subject distance subject far away, blue component ratio is improved more significantly, thereby described in strengtheningThird dimension in image;
Described changing unit, corresponding to described beholder's setting, changes and by described image rib, subject is executedThe reinforcement amount of the blue component ratio adding.
16. display unit according to claim 14, wherein, further comprise: image rib,For subject distance subject far away, fuzziness is improved more significantly, thereby strengthen in described imageThird dimension;
Described changing unit, corresponding to described beholder's setting, changes and by described image rib, subject is executedThe reinforcement amount of the fuzziness adding.
17. display unit according to claim 14, wherein,
Described display part, shows the right eye image of the right eye that offers described beholder and offers left eyeLeft eye image, as giving relief image for described beholder; This display unit further comprises:Image rib, for subject distance subject far away, by described right eye with the display position of image withImage pitch between the described left eye use display position of image is from improving more significantly, thus reinforcement third dimension;
Described changing unit, corresponding to described beholder's setting, changes and by described image rib, subject is executedThe image pitch adding from reinforcement amount.
18. according to the display unit described in any one in claim 15～17, and wherein, this display unit is enteredOne step comprises: detecting element, detects the subject in each region of the image that should be shown by described display partDistance;
Described image rib is for the larger part of described subject distance, makes to be shown by described display partRelief reinforcement amount in described image is larger.
19. according to the display unit described in any one in claim 15～17, and wherein, described display part is aobviousShow the right eye image of the right eye that offers described beholder and offer the left eye image of left eye, as beingDescribed beholder gives relief image;
This display unit further comprises: detecting element, detects same subject at described right eye imageDisplay position and described left eye with the image pitch between the display position of image from;
Described image rib, makes to be shown by described display part from larger part for described image pitchRelief reinforcement amount in described image is larger.
20. according to the display unit described in any one in claim 14～17, wherein:
Described changing unit is presented at the image with different stereo effects in described display part, carries out and makes described sightThe person of seeing selects the calibration of relief reinforcement amount;
Described display part shows image according to the relief reinforcement amount of being selected by described beholder.
21. 1 kinds of display methods, wherein, comprise the steps:
Be shown as beholder and given relief image;
Corresponding to beholder's setting, change the relief reinforcement amount in shown described image;
Judge the type of the subject in each region in photographic images;
By the blue component ratio that should comprise in the type stores image of subject;
In each region in described photographic images, the blue composition ratio of the each subject based on described storageThe blue component ratio of the subject of the type of judging described in example with described in the blue component ratio that detectsBetween difference, calculate the subject distance in this region; And
Image corresponding to the subject detecting for each region of described photographic images apart from this region of changeDistance perspective.
CN201180039376.8A 2010-08-10 2011-08-04 Image processing apparatus, image processing method, display unit, display methods and program CN103069814B (en)
JP179144/2010 2010-08-10
JP179143/2010 2010-08-10
JP2010179144A JP5556494B2 (en) 2010-08-10 2010-08-10 Image processing apparatus, image processing method, and program
JP2010179143A JP5556493B2 (en) 2010-08-10 2010-08-10 Display device, display method, and program
PCT/JP2011/004423 WO2012020558A1 (en) 2010-08-10 2011-08-04 Image processing device, image processing method, display device, display method and program
CN201610243296.9A CN105915880B (en) 2010-08-10 2011-08-04 Image processing apparatus and image processing method
CN201610243296.9A Division CN105915880B (en) 2010-08-10 2011-08-04 Image processing apparatus and image processing method
CN103069814A CN103069814A (en) 2013-04-24
CN103069814B true CN103069814B (en) 2016-05-11
ID=45567533
CN201180039376.8A CN103069814B (en) 2010-08-10 2011-08-04 Image processing apparatus, image processing method, display unit, display methods and program
US (2) US9488841B2 (en)
CN (2) CN103069814B (en)
WO (1) WO2012020558A1 (en)
CN104093015B (en) 2014-06-23 2016-01-06 京东方科技集团股份有限公司 A system and control method thereof
WO2017022303A1 (en) * 2015-08-03 2017-02-09 ソニー株式会社 Information processing device, information processing method, and program
CN108076492B (en) * 2017-11-20 2018-10-30 张天勇 A method of promoting the image quality that camera shoots crowd
CN107784295B (en) * 2017-11-20 2018-12-21 惠州太初科技有限公司 The camera camera system stretched based on human body distance
JPH06324620A (en) 1993-05-17 1994-11-25 Patoroma Res Kk Artificial environment forming device
JPH10191397A (en) * 1996-12-27 1998-07-21 Sanyo Electric Co Ltd Intention adaptive device for converting two-dimensional video into three-dimensional video
JP2001016609A (en) * 1999-06-05 2001-01-19 Soft Foo Deii:Kk Stereoscopic video image generator and its method using mpeg data
EP1762881A1 (en) * 2004-06-30 2007-03-14 Matsushita Electric Industrial Co., Ltd. Three-dimensional image displaying system
JP2007110360A (en) * 2005-10-13 2007-04-26 Ntt Comware Corp Stereoscopic image processing apparatus and program
JP2007264722A (en) * 2006-03-27 2007-10-11 Victor Co Of Japan Ltd Image processor
JP5183277B2 (en) * 2008-04-03 2013-04-17 三菱電機株式会社 Stereoscopic image display device
CN101567082B (en) * 2009-06-04 2011-07-13 中华映管股份有限公司 Image processing method and device thereof
CN102959973B (en) 2010-06-25 2015-01-21 富士胶片株式会社 Stereoscopic image playback device, stereoscopic image playback system, and stereoscopic image playback method
JP2013197797A (en) * 2012-03-19 2013-09-30 Seiko Epson Corp Image display device and image display method
2011-08-04 CN CN201180039376.8A patent/CN103069814B/en active IP Right Grant
2011-08-04 WO PCT/JP2011/004423 patent/WO2012020558A1/en active Application Filing
2011-08-04 CN CN201610243296.9A patent/CN105915880B/en active IP Right Grant
2013-02-08 US US13/762,391 patent/US9488841B2/en active Active
2016-10-06 US US15/286,588 patent/US10462455B2/en active Active
US9488841B2 (en) 2016-11-08
CN103069814A (en) 2013-04-24
CN105915880A (en) 2016-08-31
WO2012020558A1 (en) 2012-02-16
US20170026639A1 (en) 2017-01-26
US10462455B2 (en) 2019-10-29
US20130148859A1 (en) 2013-06-13
CN105915880B (en) 2018-02-23