Source: https://patents.google.com/patent/JP5251215B2/en
Timestamp: 2020-02-18 05:06:17
Document Index: 35013351

Matched Legal Cases: ['art 4', 'art 5', 'art 6', 'art 7', 'art 8', 'art 10']

JP5251215B2 - Digital camera - Google Patents
JP2008276214A (en
2008-11-13 Publication of JP2008276214A publication Critical patent/JP2008276214A/en
2013-07-31 Publication of JP5251215B2 publication Critical patent/JP5251215B2/en
Conventionally, a contrast method is known as an autofocus (AF) method for digital cameras. In this contrast method, a subject is imaged by an image sensor such as a CCD to obtain an image signal, a component in a predetermined spatial frequency band is extracted from the image signal in a predetermined AF area in the image, and the absolute value is integrated. To calculate the focus evaluation value. This focus evaluation value is a value corresponding to the contrast of the focus detection area, and the higher the contrast, the higher the value. Based on the characteristic that the contrast of the image increases as the focus lens is closer to the in-focus position, a position where the focus evaluation value reaches a peak (peak position) is found and determined as the in-focus position, and the focus lens is placed at that position. Drive (Patent Document 1).
JP 2003-315665 A
However, in order to detect the peak position of the contrast, which is the in-focus position, the focus evaluation value is calculated at predetermined intervals while moving the focus lens along the optical axis, and the focus evaluation value is analyzed and the peak position is analyzed. It takes a long time to focus on the subject, and there is a problem that it cannot focus on a moving subject.
An object of the present invention is to provide a digital camera that can more accurately focus and photograph a moving subject.
The digital camera according to claim 1 recognizes a feature region of the subject using an imaging unit that receives and captures light from a subject that has passed through a photographing optical system, and an image that is captured and acquired by the imaging unit. A recognition unit that detects the size of the feature region recognized by the recognition unit, and predicts the distance to the subject after a predetermined time based on the size of the feature region. A control unit that controls the photographing optical system to focus, a distance calculation unit that calculates a distance to the subject based on a size of the feature region, and a time change of the distance to the subject, A speed calculation unit that calculates a moving speed of a subject; an aperture control unit that controls an aperture; a release button that is operated by a user to perform a half-press operation and a full-press operation as a shooting execution operation; Before The distance to the subject calculated by the distance calculation unit, and the moving speed of the subject calculated by the speed calculation unit predicts the distance to the subject, prior Symbol aperture control unit, the release button is the In response to the half-pressing operation, the aperture is controlled so as to achieve pan focus, and the release button is in the half-pressed state and the release button is not fully pressed. In some cases, when the feature region is recognized by the recognition unit, the aperture is controlled so as to achieve pan focus, and when the feature region is not recognized by the recognition unit, the pan focus is released. The aperture is controlled as described above .
According to a second aspect of the present invention, in the digital camera according to the first aspect, the aperture control unit is configured such that an image used when the recognition unit recognizes the feature area of the subject is pan-focused. The diaphragm is controlled.
According to a third aspect of the present invention, in the digital camera according to the first or second aspect, the diaphragm control unit controls the diaphragm to a minimum aperture.
According to a fourth aspect of the present invention, in the digital camera according to any one of the first to third aspects, when the photographing execution operation is performed, the aperture control unit performs appropriate exposure on the subject. In order to achieve this, the diaphragm is controlled.
The digital camera according to claim 5 is the digital camera according to any one of claims 1 to 4, wherein the distance calculation unit is configured from position information of a lens constituting the photographing optical system to the subject. And calculating the distance to the subject from the positional information of the lenses constituting the photographing optical system, the distance to the subject is calculated based on the calculated distance to the subject and the size of the feature region. The distance is calculated.
The digital camera according to claim 6 is the digital camera according to any one of claims 1 to 5, wherein the control unit is configured to calculate a time from when the imaging unit performs imaging until the imaging unit performs imaging. A distance to the subject at the time of imaging is predicted, and the imaging optical system is controlled so that the subject is focused at the time of imaging by the imaging unit.
The digital camera according to claim 7 is the digital camera according to any one of claims 1 to 6, wherein the distance to the subject in at least one of the feature regions recognized by the recognition unit. A registration unit that selects the feature region of the subject for predicting the feature and registers the feature information of the feature region of the selected subject, and after registering the feature information of the feature region of the subject, the recognition unit Is characterized in that the feature area of the subject is recognized based on feature information of the feature area of the subject.
The digital camera according to claim 8, further comprising a recording control unit that records an image acquired by imaging with the imaging unit on a recording medium in the digital camera according to claim 7, wherein the registration unit includes the recording unit The feature information of the feature region of the subject is registered based on the image recorded on the medium.
The digital camera according to claim 9 is the digital camera according to claim 7 or claim 8, wherein the feature information of the feature region includes position information of a lens constituting the photographing optical system, a distance to the subject, It is at least one of the sizes of the feature regions.
The digital camera according to claim 10 is the digital camera according to any one of claims 1 to 9, wherein the digital camera according to any one of the distance calculation unit and the speed calculation unit It is characterized in that the photographing condition when photographing is changed based on the photographing execution operation.
The digital camera according to an eleventh aspect is the digital camera according to the tenth aspect, wherein the photographing condition is one of a shutter speed and an ISO sensitivity.
The digital camera according to claim 12 is the digital camera according to any one of claims 1 to 11, wherein the control unit is present in a plurality of images acquired in time series by the imaging unit. The distance to the subject after the predetermined time is predicted based on the sizes of the plurality of feature regions.
ADVANTAGE OF THE INVENTION According to this invention, the digital camera which can focus and shoot more accurately with respect to the to-be-photographed object can be provided.
FIG. 1 is a block diagram showing an electrical configuration of a digital camera 1 in the present embodiment.
The lens 2 includes a focus lens 2a and a zoom lens 2b, and constitutes a photographing optical system. The focus lens 2 a is a lens for performing focus adjustment on the subject, and is moved in the optical axis direction by the focus lens driving unit 3. The zoom lens 2 b is a lens for changing the focal length of the lens 2, and is moved in the optical axis direction by the zoom lens driving unit 4. The focus lens driving unit 3 and the zoom lens driving unit 4 are each composed of, for example, a stepping motor, and are controlled based on a command from the control unit 5. The focus lens position detection unit 6 detects the position of the focus lens 2 a on the optical axis and sends a detection signal to the control unit 5. The zoom lens position detector 7 detects the position of the zoom lens 2 b on the optical axis and sends a detection signal to the controller 5.
Light from the subject is imaged on the image sensor 8 by the lens 2. The image sensor 8 is a solid-state image sensor such as a CCD or a CMOS, for example, and outputs an image signal obtained by converting a subject image into an electrical signal by photoelectric conversion to the analog signal processing unit 9. An imaging signal which is an analog signal input to the analog signal processing unit 9 is subjected to processing such as correlated double sampling (CDS) and input to an analog-to-digital converter (ADC) 10. The imaging signal is converted from an analog signal to a digital signal by the ADC 10 and stored in the memory 11. The memory 11 includes a buffer memory that temporarily stores image signals and a built-in memory that records captured image data. The image data stored in the memory 11 is sent to the digital signal processing unit 13 via the bus 12. The digital signal processor 13 is a digital signal processor (DSP), for example, and performs known image processing such as white balance processing, interpolation processing, and gamma correction on the image data, and then stores the image data in the memory 11 again.
The image data subjected to the image processing is subjected to a known compression process such as JPEG by the compression / decompression unit 14 and is recorded on a memory card 15 that is detachable from the digital camera 1. When playing back and displaying an image recorded on the memory card 15, the image recorded on the memory card 15 is read into the memory 11, and the digital image is converted into an analog video signal by the digital-analog converter (DAC) 16. Then, an image is displayed on the display unit 17. The display unit 17 is, for example, a liquid crystal display, and reproduces and displays an image recorded on the memory card 15 or displays an image captured by the image sensor 8 at the time of image capturing as a through image. The image data recording destination may be the memory card 15 or a built-in memory provided in the memory 11. However, the memory card 15 is not used when the internal memory is used.
The control unit 5 is connected to the operation unit 18. The control unit 5 includes a CPU, for example, and controls the operation of the digital camera 1 in accordance with a signal input from the operation unit 18. The operation unit 18 includes a power button 19, a release button 20, a menu button 21, a cross key 22, a determination button 23, an AF mode selection switch 24, and the like.
The power button 19 is a button for switching between a power-on (ON) state and a disconnection (OFF) state of the digital camera 1.
The release button 20 is a button that is pressed when the user gives an instruction to capture an image. When the release button 20 is half-pressed, the half-press switch SW1 is turned on (ON) and an ON signal is output. When the release button 20 is not half-pressed, the half-press switch SW1 is disconnected (OFF) and an OFF signal is output. A signal output from the half-press switch SW1 is input to the control unit 5. When the release button 20 is fully pressed (pressed further deeper than the half-press operation), the full-press switch SW2 is turned on (ON) and an ON signal is output, and if not fully pressed, the full-press switch SW2 is disconnected (OFF). And outputs an OFF signal. A signal output from the full-press switch SW2 is input to the control unit 5.
The menu button 21 is a button for displaying a menu corresponding to the mode selected by the user.
The cross key 22 is a button for selecting an operation that the user wants to perform, such as moving a cursor for selecting an item displayed on the display unit 17 up, down, left, or right.
The enter button 23 is a button for determining the operation selected by the cross key 22.
The AF mode selection switch 24 is a switch for selecting whether or not to take an image in the moving object prediction AF mode. Here, the moving object prediction AF mode is a shooting mode executed when the AF mode selection switch 24 is turned on (ON), and is an operation as shown in FIG. Details of FIG. 3 and the moving object prediction AF process will be described later. When the AF mode selection switch 24 is turned on (ON), the mode is switched to the moving object prediction AF mode. When the AF mode selection switch 24 is disconnected (OFF), the mode is switched to the conventional contrast AF mode as shown in the background art, for example.
The feature region recognition calculation unit 25 recognizes the feature region from the image data, and outputs the coordinates representing the position and size of the recognized feature region to the control unit 5 when the recognition is successful. When the coordinates representing the position and size of the feature area are input, the control unit 5 creates an image in which a frame (feature area mark) representing the size of the feature area is superimposed on the live view display image based on the coordinates. Then, the image is displayed on the display unit 17. The calculation for recognizing the feature area may be performed by the control unit 5.
The digital camera 1 according to the first embodiment of the present invention recognizes a feature region from an image captured by the image sensor 8, continuously detects the size of the feature region specified by the user, and determines the size of the feature region. The movement of the subject is calculated from the change of the object. Based on this result, the distance to the subject at the time of imaging is predicted, and the drive position of the focus lens 2a is controlled so as to focus on the subject.
A method for recognizing a feature region will be described. For example, when the subject is a person, the face of the person is recognized as a feature region. The feature area recognition calculation unit 25 detects whether or not there is a human face in the through image displayed on the display unit 17. As a method for detecting a human face, for example, skin color detection is performed from an image (Japanese Patent Laid-Open No. 2004-037733), a face candidate area corresponding to a face shape is extracted, and a face area is determined from the area ( JP-A-8-063597). In addition, as a person authentication method, for example, an image obtained by extracting facial feature points such as eyes, nose and mouth from an image is compared with a dictionary image of each registrant registered in advance to identify the person. (Japanese Patent Laid-Open No. 9-251534). If the person's face is successfully recognized using such a known method, coordinates representing the position and size of the recognized face area are output to the control unit 5.
When a plurality of persons are recognized, the person who wants to focus is specified from the plurality of persons as will be described later. The control unit 5 controls the display unit 17 based on the coordinates input from the feature region recognition calculation unit 25, and displays a frame (face region mark) representing the face region on the through image as shown in FIG. To do. When one face is detected by the feature area recognition calculation unit 25, a face area mark is displayed for the face area. When there are a plurality of faces (three faces in FIG. 2) detected by the feature area recognition calculation unit 25 as shown in FIG. 2, face area marks M1 to M3 are displayed for the plurality of face areas, respectively.
A feature region registration method and a subject distance prediction method will be described later.
FIG. 3 is a flowchart showing a photographing procedure in the moving object prediction AF mode. The process shown in FIG. 3 is executed by the control unit 5. In the present embodiment, a case will be described in which a plurality of persons are present in a through image, and a person whose focus is to be continuously selected is selected and photographed.
When the power button 19 of the digital camera 1 is in the ON state and the AF mode selection switch 24 is in the ON state, a moving object prediction AF program that performs an operation as shown in FIG. 3 is executed.
First, steps S101 to S105 are steps related to the recognition of the feature region.
In step S101, when the AF mode selection switch 24 is turned on, a through image is displayed on the display unit 17. In this case, images repeatedly captured by the image sensor 8 are sequentially updated and displayed on the display unit 17 as a through image.
In step S102, when the menu button 21 is pressed in a state where the through image is displayed on the display unit 17, the control unit 5 sends a command to the display unit 17 so that a screen for selecting the type of subject to be recognized is displayed as the through image. Overlapping is displayed on the display unit 17. As the type of the subject, for example, a subject, a soccer ball, a car or the like that the subject itself moves is displayed on the selection screen. The user operates the cross key 22 to select the type of subject on the selection screen and presses the enter button 23 to make a decision. If the determination button 23 is not turned on, the determination in S102 is repeated until the determination button 23 is turned on. If the determination button 23 is turned on, the process proceeds to step S103. Here, since the subject of the present embodiment is a person, a case where a person is selected as the type of subject will be described below.
When the type of the subject to be recognized is selected, in step S103, the control unit 5 sends a command for starting the feature region recognition processing for the through image to the feature region recognition calculation unit 25. Here, since a person is selected as the type of subject in step S102, face area recognition processing for recognizing a person's face as a feature area is started.
In step S <b> 104, the control unit 5 receives the recognition result of the face area at that time from the feature area recognition calculation unit 25 and determines whether the face area is recognized. If the face area does not exist in the through image, or if the face area is not recognized for some reason such as the face area is too small even if the face area exists in the through image, the process returns to step S103 and the face area recognition process is performed again. To do. If the face area is recognized, the process proceeds to step S105, where the control unit 5 sends a command to the display unit 17, and displays the face area marks M1 to M3 on the through image as shown in FIG. To do.
At this time, although details will be described later, a cross mark M4 for selecting a face area is displayed on the display unit 17. The cross mark M4 is displayed only inside the face area mark M1 closest to the center of the through image displayed on the display unit 17 among the face area marks M1 to M3. That is, when there is one face area mark, a cross mark is displayed inside the face area mark. When there are a plurality of face area marks, a cross mark is displayed only inside the face area mark closest to the center of the through image displayed on the display unit 17.
The next steps S106 to S109 are steps relating to registration of the face area.
In step S106, the face area of the subject to be photographed is selected. In a state where a plurality of face area marks M1 to M3 are displayed on the through image as shown in FIG. 2, the user first operates the cross key 22 to select a face area mark to be registered from the face area marks M1 to M3. select. At this time, the cross mark M4 indicates the face area mark selected at that time (in FIG. 2, the face area mark M2 is selected). When the user operates the cross key 22 up, down, left, or right, the face area mark on which the cross mark M4 is displayed jumps to another face area mark to move. For example, when the user presses the left side of the cross key 22 with the face area mark M2 selected as shown in FIG. 2, the cross mark M4 jumps and moves from the face area mark M2 to the face area mark M1. .
The user decides by pressing the decision button 23 in a state where the cross mark M4 is aligned with the face area mark of the person to be photographed. When the face area mark is selected, the feature area recognition calculation unit 25 extracts feature points such as eyes, nose, and mouth of the selected face area. The neighborhood area including the feature points (feature point neighborhood area including the eye area, the nose area, the mouth area, etc.) is registered in the memory 11 as a template. When the face area mark is selected, a through image is displayed on the display unit 17. Then, the feature area recognition calculation unit 25 extracts a feature point neighborhood area for the face area recognized in the through image. The control unit 5 compares the feature point neighborhood extracted from the through image with the template registered in the memory 11, that is, calculates the similarity.
Based on the calculation result of the similarity, the control unit 5 issues a command to display the face area mark in the feature point vicinity area determined to have a high similarity with the template, while the similarity with the template is low. A command to stop the display of the face area mark is sent to the display unit 17 in the area near the feature point determined to be.
Therefore, after the face area mark is selected, the face area mark is displayed only in the face area that matches the selected face area in the face area in the through image, and the face area mark is not displayed in the other face areas. Note that the method of selecting the feature area is not limited to this. For example, in the case of the display unit 17 having a touch panel on the surface, instead of the user operating the cross key 22 to select the face area mark, the face area to be selected The face area mark may be selected by pressing the mark with a finger or the like. If no face area is selected in step S106, the process returns to step S105.
When the face area is selected, the process proceeds to step S107. In step S107, when the ON signal of the half-press switch SW1 by the user's half-press operation of the release button 20 is input to the control unit 5, the AF process is performed on the face area selected in step S106. This AF process is a conventional contrast AF as shown in the background art. When the face area is focused, it is determined in step S108 whether or not the enter button 23 has been pressed in a state where the face area is focused. If the enter button 23 is not pressed, the process returns to step S107, and AF processing is performed again. If the enter button 23 is pressed, the process proceeds to step S109.
In step S109, the control unit 5 registers information on the subject determined in step S108 in the memory 11. The information on the subject includes the position information of the lens 2 when the face region is determined in step S108, the distance (subject distance) to the subject (face region) calculated based on the position information of the lens 2, and the face region mark. It is a size. The position information of the lens 2 is the position of the focus lens 2a and the zoom lens 2b on the optical axis, and is acquired by the focus lens position detection unit 6 and the zoom lens position detection unit 7, and the detection signal is sent to the control unit 5. Is output. When the detection signal is input, the control unit 5 calculates the subject distance based on the detection signal. The size of the face area mark is the length of one of the vertical and horizontal sides of the face area mark displayed in a rectangular shape, or a combination thereof. Thereby, the relationship between the predetermined subject distance and the size of the face area mark is determined. When registration of information about the subject is completed, a through image is displayed on the display unit 17.
The next steps S110 to S120 are steps relating to photographing.
In step S110, it is determined whether or not the half-press switch SW1 of the release button 20 is in the ON state. If the half-press switch SW1 of the release button 20 is in the OFF state, the determination in step S110 is repeated until the half-press switch SW1 is in the ON state. If the half-press switch SW1 of the release button 20 is in the ON state, the process proceeds to step S111.
When the half-push switch SW1 is turned on, in step S111, the aperture value is set so that the aperture (not shown) is close to the minimum aperture or the minimum aperture. That is, the pan focus is set. This is because a moving subject, particularly a face region, is recognized in the subject distance calculation process in step S116 described later. By increasing the depth of field, a moving subject, particularly the face region, is moved with respect to the optical axis direction. This is so that it can be recognized over a wide range. At this time, the focus lens 2a is driven so as to correspond to the hyperfocal distance. The hyperfocal distance is the shortest subject distance among subject distances included in the depth of field in pan-focus imaging. At this time, the depth of field can be set according to the composition of the subject to be photographed by the user, and the focus lens 2a may be driven so as to correspond to this.
In step S112, it is determined whether or not the face area registered in step S109 exists in the through image. If the registered face area does not exist in the through image, the process proceeds to step S113. In step S113, the control unit 5 resets the aperture value set in step S110, cancels the pan focus setting, and sets the aperture value so that the subject existing in the through image has an appropriate exposure. In step S114, the moving object prediction AF mode is switched to the normal AF mode, for example, the conventional contrast AF as shown in the background art. For example, when a landscape such as a mountain is displayed as a through image, the focus lens 2a is driven to focus on infinity. In step S115, it is determined whether or not the full press switch SW2 of the release button 20 is in the ON state.
If the full-press switch SW2 is in the OFF state, the process returns to step S111, and the aperture value is set to a minimum aperture diameter or an aperture value close to the minimum aperture diameter (not shown). If the full push switch SW2 is in the ON state, the process proceeds to step S120.
On the other hand, if the registered face area exists in the through image in step S112, a face area mark is displayed for the registered face area, and then the process proceeds to step S116 to start subject distance calculation processing. The subject distance at this time is calculated by substituting the parameters of the size of the face area mark and the focal length of the lens 2 into a predetermined arithmetic expression. In addition, a table in which the relationship between the size of the face area mark and the focal length of the lens 2 is associated with the subject distance is created in advance and stored in the memory 11, and the subject distance is calculated by referring to this table. It may be.
As long as the face area is detected, the face area mark is tracked and displayed so as to overlap the face area even if the face area moves. Here, since the focus lens 2a is driven so as to correspond to the hyperfocal distance in step S110, the focus lens 2a is not driven even when the half-press switch SW1 of the release button 20 is turned on. However, the focus lens 2a may be driven so as to be focused in accordance with the movement of the subject. This process will be described in the fourth embodiment. The setting of the pan focus described above is continued until it is determined in step S112 that a face area exists, until the pan focus setting is canceled in step S118.
In step S116, information on the size of the face area mark and the focal length of the lens 2 is acquired for each image captured in time sequence by the image sensor 8 (for example, 30 frames / second), and the subject distance is calculated. If the focal length of the lens 2 is the same as when the face area is registered in step S109, the size of the face area mark displayed in the through image is larger than the face area mark size when the face area is registered. If it is smaller, it is recognized that the distance is longer than the subject distance at the time of face area registration.
On the other hand, if the size of the face area mark displayed in the through image is larger than the size of the face area mark at the time of face area registration, it is recognized that the distance is shorter than the subject distance at the time of face area registration. The obtained subject distance is stored in the memory 11. The subject distance stored in the memory 11 is for a plurality of frames, and the subject distance in the memory 11 is also updated sequentially each time images are sequentially picked up by the image sensor 8.
Further, the moving speed of the subject is calculated from the temporal change in the subject distance for a plurality of frames stored in the memory 11, and the subsequent subject distance is calculated. In FIG. 4, the person who is the subject will be described as person A. It is assumed that when the time t = 0 seconds, the vertical length of the face area mark is a and the calculated subject distance is 5 m. After that, the subject moved at a constant speed. When the time t = 5/30 seconds, the vertical length of the face area mark was b larger than a, and the calculated subject distance was 4.83 m. . If the focal length of the lens 2 does not change during this period, the moving speed of the subject is 1 m / sec. Therefore, when t = 6/30 seconds, it can be predicted that the subject is at a subject distance of 4.80 m. The calculation of the subject distance described above is repeated until the release button 20 is fully pressed.
In step S117, it is determined whether or not the full press switch SW2 of the release button 20 is in the ON state. If the full-press switch SW2 of the release button 20 is in the OFF state, the process returns to step S112, and it is determined again whether or not the registered face area exists in the through image. If the full press switch SW2 of the release button 20 is in the ON state, the process proceeds to step S118.
In step S118, the aperture value set in step S111 is reset to cancel the pan focus setting, and the aperture value is set so that an appropriate exposure is obtained for the subject.
In step S119, the moving object prediction AF process is executed on the subject. In a camera having an AF function, there may be a problem of a time difference (release time lag) from when the release button 20 is fully pressed to when shooting is actually performed. In particular, when the subject moves, the focus position with respect to the subject changes during the release time lag, and there is a problem that the photographed image is out of focus. Here, the focus position with respect to the subject after the release time lag is predicted from the moving speed of the subject based on the result of the subject distance calculation process in step S116, and the focus lens 2a is moved so as to focus on this focus position. The focus state on the subject at the time of imaging was optimized. In the present embodiment, the release time lag is set to 0.01 seconds, and the position of the subject 0.01 seconds after the release button 20 is fully pressed is predicted, and the image is taken after focusing on that position.
FIGS. 5A and 5B are diagrams illustrating an example of the moving object prediction AF process in step S119. Here, the movement of the person B as the subject is shown.
FIG. 5A is a diagram illustrating a change in the size of the face area mark of the person B over time. Here, the size of the face area mark is the length of the vertical side of the face area mark. The horizontal axis is a number assigned to each image captured in time sequence by the image sensor 8 (through image I1 to through image I7), and these images are captured at 30 frames / second. That is, one scale on the horizontal axis is 1/30 second. The vertical axis represents the size of the face area mark.
FIG. 5B is a diagram illustrating a change in the subject distance of the person B over time. Similarly to FIG. 5A, the horizontal axis is a number assigned to each image captured in time sequence by the image sensor 8, and one scale on the horizontal axis is 1/30 second. The vertical axis represents the subject distance of the person B. As described above, the subject distance is calculated based on the size of the face area mark and the focal length of the lens 2.
The movement of the person B will be described. The size of the face area mark is a at the time of the through image I1 (FIG. 5A), and the subject distance of the person B is 5.0 m (FIG. 5B). Similarly, in the through image I2 and the through image I3, since the size of the face area mark remains a (FIG. 5A), the subject distance of the person B remains unchanged at 5.0 m (FIG. 5). 5 (b)). In the through image I4, since the size of the face area mark has changed to b larger than a (FIG. 5A), the subject distance of the person B is shortened to 4.9 m (FIG. 5B). )). In the through image I5, the through image I6, and the through image I7, the size of the face area mark increases proportionally with time, c, d, and e (FIG. 5A), and the subject distance of the person B is It was 4.6 m at the time of the through image I7. From this, it is calculated that the person B is approaching the camera side at a moving speed of 3.0 m / sec.
Here, it is assumed that the release button 20 is fully pressed at the time of the through image I7. In response to the full-press signal, the control unit 5 calculates the position of the person B at the time of imaging, that is, 0.01 seconds after the release time lag, from the subject distance and moving speed of the person B. Therefore, the position of the person B at the time of imaging can be predicted to be 4.6 m + (− 3.0 m / sec) × 0.01 sec = 4.57 m. Based on the calculation result of the subject distance, the control unit 5 sends a command to the focus lens driving unit 3 to drive the focus lens 2a so as to focus on the subject distance 4.57m. Then, the focus lens driving unit 3 that has received a command from the control unit 5 drives the focus lens 2a.
In step S120, an image is picked up by the image pickup device 8. At this time, the exposure condition of the digital camera 1 may be changed according to the movement of the subject. For example, when the moving speed of the subject is fast, the shutter speed is increased or the ISO sensitivity is increased.
The distance to the subject at the time of imaging is predicted by calculating the distance to the subject from the size of the feature region and the focal length of the lens 2 for each image captured in time sequence by the imaging device 8, and focus on the subject. Thus, the focus lens 2a is driven. As a result, it is possible to shoot with a more accurate focus on a moving subject.
The feature region selected by the user among at least one feature region recognized from the image is registered. Thus, even when there are a plurality of subjects at the time of imaging, the moving object prediction AF is performed on the subject having the registered feature region, so that the registered feature region is not focused on other subjects that are not registered. It is possible to always focus on a subject having
When displaying a through image at the time of shooting, the aperture value is set so that the aperture (not shown) is the minimum aperture or close to the minimum aperture, and the focus lens 2a is driven to a position corresponding to the hyperfocal distance. As a result, the depth of field is deepened, and focused image data can be acquired over a wide range even for a moving subject (characteristic region). Further, since it is not necessary to drive the lens 2, it is possible to save the power of the digital camera 1.
The focus lens 2a is fixed after being driven to a position corresponding to the hyperfocal distance, and is driven to a focus position with respect to the subject at the time of imaging. Thereby, the lens 2 is efficiently driven to the in-focus position, and the AF processing can be speeded up.
At the time of imaging, the exposure condition of the digital camera 1 is changed according to the movement of the subject. Thereby, it is possible to capture an image of the subject under appropriate exposure conditions.
Note that the present embodiment can be modified as follows.
In step S102 of FIG. 3, the example in which a person is selected as the type of subject has been described, but here, a case in which a soccer ball is selected will be described. Here, a method for recognizing and determining a soccer ball as a feature region will be described. Other parts are as described in this embodiment.
As a method for recognizing a soccer ball, there are a method for extracting a circular area candidate corresponding to the shape of a soccer ball from image data and determining a soccer ball from the area, and a method for detecting a color from image data. Moreover, you may make it recognize combining these methods.
Here, a method for recognizing a soccer ball by detecting a color from image data will be described. The colors that form the soccer ball are black and white. Therefore, a soccer ball is recognized by extracting an area formed from two colors of black and white from the image data. Further, since the black and white area ratio forming the soccer ball is not significantly different from any angle, the black and white area ratio is registered in advance. Then, an area corresponding to a black / white area ratio registered in advance is extracted from the image data.
Further, the user may be able to set the shape of the feature area. When a soccer ball is selected as the type of subject as in step S102 of FIG. 3, a selection tool corresponding to the shape of the soccer ball, for example, a circular frame, is displayed superimposed on the through image. The size of the selection tool can be adjusted by operating the cross key 22, and the user adjusts the selection tool to approximately the same size as the size of the soccer ball displayed in the through image. After adjusting the size of the selection tool, the size of the selection tool is fixed by pressing the enter button 23. The selection tool having a fixed size can be moved up, down, left and right by operating the cross key 22, and the user superimposes the selection tool on the soccer ball displayed in the through image. When the position of the selection tool is adjusted, the soccer ball is registered as a feature region by pressing the enter button 23.
According to the modification described above, the following operational effects can be obtained.
As a method for recognizing a soccer ball from an image, in addition to extracting a circular area and detecting a specific color from the image data, a color area ratio peculiar to the characteristic area is registered in advance, and this color area ratio is The corresponding feature area was detected. Thereby, the accuracy of recognizing the feature region from the image data can be improved.
A selection tool corresponding to the type of the selected subject is displayed, and the user can adjust the selection tool according to the size and position of the feature region. As a result, even if it is difficult to recognize the feature area from the image data, the feature area can be designated.
In the first embodiment, the feature area to be focused is set from the feature areas recognized in the through image. However, in this embodiment, the feature area to be focused is set in advance from image data stored in the memory card 15 or the like. Can be set.
The basic configuration of the digital camera in the second embodiment is the same as that of the first embodiment described above. Hereinafter, parts different from the first embodiment will be described. FIG. 6 is a flowchart showing a processing procedure when a feature region is set from image data stored in the memory card 15 or the like in the moving object prediction AF mode. The process shown in the flowchart of FIG. 6 is executed by the control unit 5 or the like.
First, when the power button 19 of the digital camera 1 is ON and the user selects a setup mode by operating a mode dial (not shown), a setup menu screen is displayed on the display unit 17. Various menu items relating to shooting and reproduction are displayed on the setup menu screen, and among these, there is an item of “moving object prediction AF” for performing various settings of the moving object prediction AF mode. When the user operates the cross key 22 to select the “moving object prediction AF” item and presses the determination button 23 to determine the “moving object prediction AF” item, the display unit 17 displays the moving object prediction AF. The AF menu screen is displayed.
Various menu items related to the moving object prediction AF mode are displayed on the moving object prediction AF menu screen. Among these, a “feature area setting” for designating a characteristic region when shooting in the moving object prediction AF mode is displayed. There are items. When the user operates the cross key 22 to select the item “characteristic region setting” and presses the determination button 23 to determine the item “characteristic region setting”, the process proceeds to step S201, and the display unit A feature area setting screen 17 is displayed.
A list of images stored in the memory card 15 is displayed on the feature area setting screen. As a list display method, for example, thumbnail images of images stored in the memory card 15 are displayed. When the memory card 15 is not used, thumbnail images of images stored in the built-in memory provided in the memory 11 may be displayed.
In step S202, it is determined whether a thumbnail image including a feature area to be focused on has been determined from the thumbnail images displayed on the display unit 17. The thumbnail image is determined by selecting the thumbnail image by the user operating the cross key 22 and pressing the determination button 23 to determine the thumbnail image. If the thumbnail image is not determined, the determination in S202 is repeated until the thumbnail image is determined. If a thumbnail image is determined, the process proceeds to step S203.
When the thumbnail image is determined, an image corresponding to the thumbnail image is first reproduced and displayed on the display unit 17 in step S203. At this time, a screen for selecting the type of subject to be recognized is displayed on the display unit 17 so as to overlap the through image. When the type of the subject is selected, a selection tool corresponding to the shape of the subject is displayed over the through image. For example, when a person is selected as the type of subject, a selection tool having an elliptical shape that is long in the vertical direction is displayed. Then, the user operates the cross key 22 and the determination button 23 to set the feature region by adjusting the size and position of the selection tool. Details of the setting method of the selection tool are the same as in the modification of the first embodiment, and thus the description thereof is omitted here.
In step S204, it is determined whether there is an instruction to end the feature area setting screen. If the feature area setting screen is not terminated, the process returns to step S201, and the feature area setting screen is displayed again on the display unit 17. For example, when the user selects the end of the feature area setting screen by operating the operation unit 18 and ends the feature area setting screen, the setting of the feature area is ended. When photographing a subject after setting the feature region, the process proceeds to the process shown in the flowchart of FIG.
In the present embodiment, only one image data is used when setting the feature area. However, the feature area may be set using a plurality of image data for the same subject. For example, when the type of subject is a person, the face is a feature area, but not only from image data that includes a face that faces the front, but also from image data that includes a face that faces an angle such as a side face. A feature region can be set. When a feature region is set for a single subject using a plurality of image data, the plurality of image data are related image data. As a method of making related image data, for example, there is a method in which a user inputs and saves the same keyword in the image data.
Further, the number of subjects for which feature areas are set is not limited to one, and a plurality of feature regions for different subjects may be set.
FIG. 7 is a flowchart showing a photographing procedure in the moving object prediction AF mode when a feature region has already been set from image data.
When the power button 19 of the digital camera 1 is in the ON state and the AF mode selection switch 24 is in the ON state, it is determined in step S205 whether or not there is one feature region set from the image data. If there is one feature region set from the image data, the process proceeds to step S208. If there are a plurality of feature regions set from the image data, the process proceeds to step S206, and a list is displayed so that the set feature regions can be seen. As a list display method, a thumbnail image of an image including the set feature area may be displayed, or a keyword registered in the image including the set feature area may be displayed.
In step S207, it is determined whether one feature region has been selected from the set feature region list. If the feature region is not selected, the determination in step S207 is repeated until the feature region is selected. If a feature region is selected, the process proceeds to S208.
In step S208, information regarding the set subject is registered in the memory 11. The information on the subject includes the position information of the lens 2 when the image is taken, the distance to the subject (subject distance) calculated based on the position information of the lens 2, and the size of the feature region. Is recorded on the image in Exif format. Note that the size of the feature region is the size of the selection tool when the size of the selection tool is determined in step S203.
For example, when the shape of the selection tool is an ellipse, the size of the feature area is the length of the major axis of the ellipse (the line segment with the intersection of the ellipse between the straight line connecting the two focal points of the ellipse) The length of the minor axis (the line segment with the intersection of the straight line and the ellipse perpendicular to the major axis and the center of the ellipse), or the combination of the major axis length of the ellipse and the minor axis length of the ellipse . As a result, the control unit 5 reads information on the subject from the image and stores it in the built-in memory provided in the memory 11.
When the information about the subject is registered in step S208, the process proceeds to step S110 in FIG. 3, but the subsequent steps are the same as those in the first embodiment, and thus description thereof is omitted.
The feature area to be focused can be set from the image data stored in the memory card 15 or the like. Thus, the user can set a feature area to be focused in advance before shooting, and can take a picture immediately after the digital camera 1 is activated.
A feature region can be set for a single subject using a plurality of image data. Thereby, the accuracy of recognizing the subject can be improved.
In the first embodiment, the feature area to be focused is set and subject information is registered in accordance with the operation of the decision button 23 by the user. In this embodiment, however, the feature area setting and subject information registration are automatically performed. To do.
The basic configuration of the digital camera in the third embodiment is the same as that of the first embodiment described above. Hereinafter, parts different from the first embodiment will be described. A shooting procedure in the moving object prediction AF mode in the third embodiment will be described with reference to the flowchart of FIG. The process shown in the flowchart of FIG. 8 is executed by the control unit 5 or the like.
In step S301, when the AF mode selection switch 24 is turned on, a through image is displayed on the display unit 17. In step S302, the type of subject is selected. Hereinafter, a case where a person is selected as the type of subject will be described.
When the type of subject to be recognized is selected, in step S303, the control unit 5 sends a command for starting the feature region recognition processing for the through image to the feature region recognition calculation unit 25. Here, since a person is selected as the type of subject in step S302, face area recognition processing for recognizing a person's face as a feature area is started.
In step S304, the control unit 5 receives the recognition result of the face area at that time from the feature area recognition calculation unit 25, and determines whether the face area is recognized. If the face area is not recognized, the process returns to step S303 and the face area recognition process is performed again. If the face area is recognized, the process proceeds to step S305. Here, when a plurality of face areas are recognized, the largest face area is automatically selected from the recognized face areas, and the process proceeds to step S305. Or you may select automatically the thing arrange | positioned in the position nearest to the screen center among several face area | regions.
In step S305, a face area mark indicating the recognized face area is displayed on the display unit 17 so as to overlap the through image. This face area mark indicates a face area where subject information is to be registered. The face area mark is, for example, a rectangular frame as shown in FIG.
When the face area to be registered is set, the process proceeds to step S307. In step S307, contrast AF processing is performed on the set face area in response to the ON signal of the half-press switch SW1. In subsequent step S308, it is determined whether or not the set face area is focused. If it is determined that the face area is in focus, the process proceeds to step S309. In step S309, face recognition processing is performed again with the subject accurately focused, and subject information in this state is registered in the memory 11. When the set face area is focused, the display color of the face area mark is changed so as to indicate that the focused face area has been focused. For example, the face area mark displayed in white is changed to green after focusing. Alternatively, the face area mark may be blinked after focusing.
On the other hand, if it is determined in step S308 that the face area is not focused, the process proceeds to step S321 to notify the user that subject information cannot be registered. Here, as a warning that registration is impossible, for example, a display for warning is performed on the display unit 17 or a warning lamp is turned on.
As described above, after face area recognition and subject information registration are automatically performed, the process proceeds to step S311. Since the processing in steps S311 to S320 is the same as the processing in steps S111 to S120 in the first embodiment described above, description thereof will be omitted.
According to the third embodiment described above, it is possible to easily perform setting of a feature area and registration of subject information.
In the first embodiment described above, pan focus setting is performed when a through image is displayed in accordance with the operation of the half-press switch SW1. In the fourth embodiment, when the selected face area exists in the through image, the moving object prediction AF process is performed so that the movement of the face area is predicted and focused.
The basic configuration of the digital camera in the fourth embodiment is the same as that in the first embodiment described above. Hereinafter, parts different from the first embodiment will be described. An imaging procedure in the moving object prediction AF mode in the fourth embodiment will be described with reference to the flowchart of FIG. The process shown in the flowchart of FIG. 9 is executed by the control unit 5 or the like.
Since the processing in steps S401 to S409 is the same as the processing in steps S101 to S109 in the first embodiment described above, description thereof is omitted.
In step S410, it is determined whether or not the half-press switch SW1 of the release button 20 is in the ON state. If the half-press switch SW1 of the release button 20 is in the OFF state, the determination in step S410 is repeated until the half-press switch SW1 is in the ON state. If the half-press switch SW1 of the release button 20 is in the ON state, the process proceeds to step S412.
In step S412, it is determined whether or not the face area registered in step S409 exists in the through image. If the registered face area does not exist in the through image, the process proceeds to step S414. In step S414, a conventional contrast AF process is performed. In step S415, it is determined whether or not the full press switch SW2 of the release button 20 is in the ON state. If the full push switch SW2 is in the OFF state, the process returns to step S412. When the full push switch SW2 is in the ON state, the process proceeds to step S420.
On the other hand, if the registered face area is present in the through image in step S412, the face area mark is displayed for the registered face area, and then the process proceeds to step S416 to perform subject distance calculation processing. The control unit 5 calculates the current subject distance based on the information on the size of the face area mark and the focal length of the lens 2. Further, the subject distance after a predetermined time is predicted as in the first embodiment described above.
In the subsequent step S416A, the previous subject distance calculated in the previous cycle and recorded in the memory 11 is compared with the subject distance after the predetermined time calculated in step S416. For the predetermined time, an appropriate value is set in advance in consideration of a control delay or the like in the control unit 5. For example, the predetermined time may be set to the same value as the release time lag described above.
If it is determined in step S416B that the difference between the previous subject distance and the predicted subject distance (= previous subject distance−predicted subject distance) is greater than or equal to the threshold value, the process proceeds to step S416C. On the other hand, if it is determined that (previous subject distance−predicted subject distance) is less than the threshold value, the process proceeds to step S417. Here, the threshold value is appropriately set in advance as a value that prevents the face of the subject corresponding to the registered face area from being blurred on the through image even if the subject distance changes. As the threshold value, a change rate of the subject distance may be set. Note that when the subject distance is a short distance, the threshold value may be set smaller than in the case of a long distance.
In step S416C, the moving object prediction AF process is performed on the subject. Here, the focus position with respect to the subject after the predetermined time is predicted using the subject distance after the predetermined time calculated in step S416, and the focus lens 2a is moved so as to focus on this focus position. In step S416D, the subject distance of the current cycle calculated in step S416A is recorded in the memory 11.
In step S417, it is determined whether or not the full-press switch SW2 of the release button 20 is in the ON state. If the full-press switch SW2 of the release button 20 is in the OFF state, the process returns to step S412 to determine again whether the registered face area exists in the through image. If the full push switch SW2 of the release button 20 is in the ON state, the process proceeds to step S418.
In step S418, subject distance calculation processing is performed on the registered face area, and in step S419, moving object prediction AF processing is performed on the subject. Thereafter, in step S420, an image is picked up by the image pickup device 8.
According to the fourth embodiment described above, when displaying a through image, the previous subject distance is compared with the predicted subject distance, and the face image of the subject corresponding to the face area set on the through image is blurred. If it is predicted, the moving object prediction AF process is performed on the set face area. As a result, it is possible to display a through image that is accurately focused on a moving subject as necessary.
Note that the second embodiment described above can be combined with the third or fourth embodiment. Alternatively, the third embodiment and the fourth embodiment can be combined.
1 is a block diagram showing an electrical configuration of a digital camera 1 in a first embodiment of the present invention. In the first embodiment of the present invention, a face area mark is displayed on the face of a person who is a subject. 6 is a flowchart illustrating a photographing procedure in a moving object prediction AF mode in the first embodiment of the present invention. In the first embodiment of the present invention, it is a diagram showing an example of a relationship between a temporal change in subject distance for a plurality of frames and a subsequent subject distance. FIG. 5 is a diagram illustrating an example of a relationship between a through image, a size of a face area mark, and a subject distance in the first embodiment of the present invention. 9 is a flowchart illustrating a procedure for setting a feature region from image data in the second embodiment of the present invention. 9 is a flowchart showing a shooting procedure in a moving object prediction AF mode when a feature region is set from image data in the second embodiment of the present invention. It is the flowchart which showed the imaging | photography procedure in the moving body prediction AF mode in 3rd Embodiment of this invention. It is the flowchart which showed the imaging | photography procedure in the moving body prediction AF mode in 4th Embodiment of this invention.
DESCRIPTION OF SYMBOLS 1 Digital camera 2 Lens 2a Focus lens 2b Zoom lens 3 Focus lens drive part 4 Zoom lens drive part 5 Control part 6 Focus lens position detection part 7 Zoom lens position detection part 8 Imaging element 9 Analog signal processing part 10 Analog digital converter 11 Memory 12 Bus 13 Digital signal processing unit 14 Compression / decompression unit 15 Memory card 16 Digital / analog converter 17 Display unit 18 Operation unit 19 Power button 20 Release button 21 Menu button 22 Cross key 23 Enter button 24 AF mode selection switch 25 Feature area recognition Calculation unit
An imaging unit that receives and images light from a subject that has passed through the imaging optical system;
A recognition unit for recognizing a feature region of the subject using an image captured and acquired by the imaging unit;
A detection unit for detecting the size of the feature region recognized by the recognition unit;
A control unit that predicts a distance to the subject after a predetermined time based on the size of the feature region and controls the photographing optical system to focus on the subject;
A distance calculation unit that calculates a distance to the subject based on the size of the feature region;
A speed calculating unit that calculates a moving speed of the subject based on a time change of the distance to the subject;
An aperture controller for controlling the aperture ;
A release button that allows the user to perform a half-press operation and a full-press operation as a shooting execution operation ,
The control unit predicts the distance to the subject from the distance to the subject calculated by the distance calculation unit and the moving speed of the subject calculated by the speed calculation unit,
Before SL aperture control unit, in response to the release button is the half-pressed, and controls the aperture so that the pan-focus, a state where the release button is the half-pressed the When the release button is not fully pressed, and when the feature region is recognized by the recognition unit, the aperture is controlled so that pan focus is achieved, and the feature region is detected by the recognition unit. A digital camera characterized by controlling the aperture so as to cancel pan focus when it is not recognized .
The digital camera according to claim 1, wherein the aperture control unit controls the aperture so that an image used when the recognition unit recognizes the feature region of the subject is pan-focused.
The aperture control unit controls the aperture to a minimum aperture.
A digital camera characterized by
The digital camera according to any one of claims 1 to 3,
When the shooting execution operation is performed, the aperture control unit controls the aperture to obtain an appropriate exposure for the subject.
The digital camera according to any one of claims 1 to 4,
The distance calculation unit calculates the distance to the subject from the position information of the lens constituting the photographing optical system, and after calculating the distance to the subject from the position information of the lens constituting the photographing optical system, Calculating the distance to the subject based on the calculated distance to the subject and the size of the feature region
The digital camera according to any one of claims 1 to 5,
The control unit predicts a distance to the subject at the time of imaging from a time from the imaging execution operation to imaging by the imaging unit, and focuses on the subject at the time of imaging by the imaging unit. Controlling the system
The digital camera according to any one of claims 1 to 6,
A registration unit that selects a feature region of the subject for predicting a distance to the subject from at least one feature region recognized by the recognition unit, and registers feature information of the selected feature region of the subject Further comprising
After registering the feature information of the feature region of the subject, the recognition unit recognizes the feature region of the subject based on the feature information of the feature region of the subject.
The digital camera according to claim 7, wherein
A recording control unit for recording an image captured by the imaging unit on a recording medium;
The registration unit registers feature information of the feature region of the subject based on an image recorded on the recording medium.
The digital camera according to claim 7 or 8,
The feature information of the feature region is at least one of position information of lenses constituting the photographing optical system, a distance to the subject, and a size of the feature region.
The digital camera according to any one of claims 1 to 9,
Changing shooting conditions when shooting is performed based on the shooting execution operation according to the calculation result of either the distance calculation unit or the speed calculation unit
The shooting condition is one of shutter speed and ISO sensitivity.
The digital camera according to any one of claims 1 to 11,
The control unit predicts the distance to the subject after the predetermined time based on the sizes of the plurality of feature regions present in a plurality of images acquired in time series by the imaging unit.
JP2008094974A 2007-04-04 2008-04-01 Digital camera Active JP5251215B2 (en)
CN200810090579.XA CN101281290B (en) 2007-04-04 2008-04-03 Digital camera
JP2008276214A JP2008276214A (en) 2008-11-13
JP5251215B2 true JP5251215B2 (en) 2013-07-31
JP2008094974A Active JP5251215B2 (en) 2007-04-04 2008-04-01 Digital camera
JP (1) JP5251215B2 (en)
CN (1) CN101281290B (en)
JP2010130590A (en) * 2008-11-28 2010-06-10 Samsung Yokohama Research Institute Co Ltd Imaging apparatus and imaging method
JP5377092B2 (en) * 2009-06-05 2013-12-25 キヤノン株式会社 Imaging apparatus and distance measuring method
JP6032879B2 (en) * 2011-10-03 2016-11-30 キヤノン株式会社 Imaging information output device and lens device having the same
JP5953187B2 (en) * 2011-10-11 2016-07-20 オリンパス株式会社 Focus control device, endoscope system, and focus control method
JP5913902B2 (en) * 2011-10-23 2016-04-27 キヤノン株式会社 Automatic focusing device and automatic focusing method
JP6253454B2 (en) * 2014-03-05 2017-12-27 オリンパス株式会社 Imaging device and imaging device control method
CN106331518A (en) * 2016-09-30 2017-01-11 北京旷视科技有限公司 Image processing method and device and electronic system
WO2019058691A1 (en) * 2017-09-20 2019-03-28 富士フイルム株式会社 Imaging control device, imaging device, imaging control method, and imaging control program
JP6465322B2 (en) * 2017-11-28 2019-02-06 オリンパス株式会社 Imaging device and imaging device control method
JPH05216093A (en) * 1992-01-31 1993-08-27 Canon Inc Camera with function for initializing operation mode
JP4557758B2 (en) * 2005-03-14 2010-10-06 三洋電機株式会社 Imaging device
2008-04-01 JP JP2008094974A patent/JP5251215B2/en active Active
2008-04-03 CN CN200810090579.XA patent/CN101281290B/en active IP Right Grant
CN101281290B (en) 2014-07-30
JP2008276214A (en) 2008-11-13
CN101281290A (en) 2008-10-08