Apparatus, method for apparatus, and storage medium

A method for an apparatus includes setting, as a first setting, a first item relating to a type of a detection target, setting, as a second setting, a second item relating to an AF area, executing object detection processing on an object of the type set in the first setting, executing focus detection processing based on information about the AF area set in the second setting, and storing the settings as a combination of the set first item and the set second item, wherein the stored settings are called by a user through a specific operation, and the object detection processing and the focus detection processing are executed based on the called settings.

BACKGROUND

Technical Field

The aspect of the embodiments relates to an apparatus, a method for the apparatus, and a storage medium.

Description of the Related Art

With the recent improvements in auto exposure performance, autofocus (AF) performance, and object detection performance of image capturing apparatuses, almost any type of image capturing can be performed in an automatic mode. However, in some image capturing scenes, it may be desirable to individually set image capturing conditions. In this case, it may be desirable to immediately change settings.

Japanese Patent Application Laid-Open No. 2009-71433 discusses an image capturing apparatus that includes a setting unit configured to set a plurality of setting items, a designation unit configured to designate a registration target item to be registered from among the plurality of setting items, and a registration control unit configured to resister registered image capturing conditions. The image capturing apparatus has such a feature that the registered image capturing conditions are called according to a call operation by a user and new image capturing conditions in which the registered image capturing conditions are reflected are generated.

This enables the user to preliminarily register image capturing settings and call the settings, thereby making it possible to immediately change the settings.

However, the method discussed in Japanese Patent Application Laid-Open No. 2009-71433 fails to describe a technique for registering settings for object detection, and also fails to describe a method for fully utilizing the advancements in the detection performance.

Many of the related-art image capturing apparatuses register only settings for exposure and AF in the operation of calling image capturing functions, and do not take into consideration the conditions for object detection.

SUMMARY

According to an aspect of the embodiments, a method for an apparatus includes setting, as a first setting, a first item relating to a type of a detection target, setting, as a second setting, a second item relating to an AF area, executing object detection processing on an object of the type set in the first setting, executing focus detection processing based on information about the AF area set in the second setting, and storing the settings as a combination of the set first item and the set second item, wherein the stored settings are called by a user through a specific operation, and the object detection processing and the focus detection processing are executed based on the called settings.

DESCRIPTION OF THE EMBODIMENTS

The disclosure will now be described in detail below with reference to the accompanying drawings showing embodiments thereof. Configurations described in the following exemplary embodiments are merely examples, and the scope of the disclosure is not limited by the configurations described in the exemplary embodiments.

Configuration of Digital Camera

FIGS.1A and1Bare external views of a digital camera100as an example of an apparatus (electronic apparatus) according to an exemplary embodiment of the disclosure.FIG.1Ais a front perspective view of the digital camera100, andFIG.1Bis a rear perspective view of the digital camera100. As illustrated inFIG.1B, a display unit28is provided on the back side of the digital camera100and displays images and various information. A touch panel70ais configured to detect a touch operation on a display surface (operation surface) of the display unit28. An outer finder display unit43is provided on the top surface of the digital camera100, and displays various setting values of the digital camera100, including a shutter speed and an aperture. A shutter button61is an operation unit for issuing an image capturing instruction. A mode selection switch60is an operation unit for changing various modes. A terminal cover40is a cover for protecting a connector (not illustrated) that connects the digital camera100to a connection cable with an external apparatus.

A main electronic dial71is a rotational operation member. For example, setting values, such as the shutter speed and the aperture, can be changed by turning the main electronic dial71. A power switch72is an operation member to switch on and off the power supply of the digital camera100. A sub-electronic dial73is a rotational operation member. For example, movement of a selection frame and image feeding can be performed by turning the sub-electronic dial73. A cross key74is a cross-key operation member (four-direction key) including buttons corresponding to an upper part, a lower part, a left part, and a right part of the cross key74, respectively, that can be pressed in the four directions, respectively, and processing corresponding to the pressed part of the cross key74can be performed. A SET button75is a push button and is mainly used, for example, to determine a selection item. A video button76is used to instruct to start or stop capturing (recording) a moving image.

By pressing an auto exposure (AE) lock button77in an image capturing standby state, an exposure state can be fixed. A zoom button78is an operation button to switch on and off a zoom mode in a live view (LV) display of an image capturing mode. By operating the main electronic dial71after the zoom mode is turned on, an LV image can be enlarged or reduced. In a reproduction mode, the zoom button78functions as an operation button to enlarge a reproduced image or to increase the magnification ratio thereof. A reproduction button79is an operation button to switch between the image capturing mode and the reproduction mode. When the reproduction button79is pressed in the image capturing mode, the mode transitions to the reproduction mode, so that the latest image out of the images recorded on a recording medium200can be displayed on the display unit28. When a menu button81is pressed, a menu screen on which various settings can be made is displayed on the display unit28. A user can intuitively perform various settings using the menu screen displayed on the display unit28, the cross key74, the SET button75, or a multi-controller (MC)65. The MC65is configured to issue an orientation instruction in eight directions and to receive a press operation at a central portion of the MC65.

A communication terminal10is a communication terminal for the digital camera100to communicate with a (detachable) lens unit150to be described below. An eyepiece16is an eyepiece of an eyepiece finder (peep-type finder), and the user can view a video image displayed on an internal electronic view finder (EVF)29through the eyepiece16. An eye approach detection unit57is an eye proximity detection sensor that detects whether an eye of the user is in proximity to the eyepiece16. A cover202is a cover of a slot to store the recording medium200.

A grip portion90is a holding portion having such a shape that the user can easily hold the digital camera100with his or her right hand. When the user holds the digital camera100by gripping the grip portion90with the little finger, ring finger, and middle finger of his or her right hand, the shutter button61and the main electronic dial71are disposed in positions where the user can operate the digital camera100with the index finger of his or her right hand. In this state, the sub-electronic dial73is disposed in a position where the user can operate the digital camera100with the thumb of his or her right hand.

FIG.2is a block diagram illustrating a configuration example of the digital camera100according to the present exemplary embodiment. As illustrated inFIG.2, the lens unit150is a lens unit equipped with a replaceable image capturing lens. A lens103is generally composed of a plurality of lenses, but is illustrated as a single lens inFIG.2for simplification. A communication terminal6is a communication terminal for the lens unit150to communicate with the digital camera100. The lens unit150communicates with a system control unit50via the communication terminal6and the communication terminal10, and controls an aperture1via an aperture drive circuit2by using an internal lens system control circuit4. Then, the lens unit150performs focusing by displacing the lens103via an AF drive circuit3.

A shutter101is a focal plane shutter for freely controlling an exposure time of an image capturing unit22under the control of the system control unit50.

The image capturing unit22is an image sensor that converts an optical image into an electrical signal, and is composed of a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) device, or the like. An analog-to-digital (A/D) converter23is used to convert an analog signal output from the image capturing unit22into a digital signal.

An image processing unit24performs predetermined processing, such as pixel interpolation, resizing processing, including reduction, or color conversion processing, on data supplied from the A/D converter23or data supplied from a memory control unit15. The image processing unit24also performs predetermined arithmetic processing using the captured image data. The system control unit50performs exposure control and focus adjustment control based on the arithmetic processing result obtained by the image processing unit24. Thus, through-the-lens (TTL) autofocus (AF) processing, AE processing, flash preliminary emission (EF) processing, and the like are performed. The image processing unit24further performs predetermined arithmetic processing using the captured image data, and performs TTL auto white balance (AWB) processing based on the obtained arithmetic processing result.

The memory control unit15controls data transmission and reception between the A/D converter23, the image processing unit24, and a memory32. Data output from the A/D converter23is written into the memory32via the image processing unit24and the memory control unit15, or is directly written into the memory32via the memory control unit15without using the image processing unit24.

The memory32stores the image data that is obtained by the image capturing unit22and is converted into the digital data by the A/D converter23. The memory32has a storage capacity that is sufficient to store a predetermined number of still images and a predetermined duration of moving images and sounds. The memory32is also used as a memory for image display (video memory). The image data for display that is written into the memory32is displayed on the display unit28or the EVF29via the memory control unit15. The display unit28and the EVF29perform display in response to signals from the memory control unit15on a display such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display. The data that is subjected to A/D conversion by the A/D converter23and stored in the memory32is sequentially transferred to and displayed on the display unit28or the EVF29, thereby making it possible to perform LV display. An image displayed on the LV display is referred to as a “LV image”.

An infrared light-emitting diode166is a light-emitting element to detect the line-of-sight position of the user in a finder screen, and emits infrared light to an eyeball (eye)161of the user that is in proximity to the eyepiece16. The infrared light emitted from the infrared light-emitting diode166is reflected by the eyeball (eye)161, and this infrared reflected light reaches a dichroic mirror162. The dichroic mirror162reflects the infrared light and allows visible light to pass. The infrared reflected light optical path of which has been changed forms an image on an imaging surface of a line-of-sight detection sensor164via an image-forming lens163. The image-forming lens163is an optical member that constitutes a line-of-sight detection optical system. The line-of-sight detection sensor164is composed of an image capturing device such as a CCD image sensor.

The line-of-sight detection sensor164photoelectrically converts the incident infrared reflected light into an electrical signal, and outputs the electrical signal to a line-of-sight detection circuit165. The line-of-sight detection circuit165includes at least one processor. The line-of-sight detection circuit165detects the line-of-sight position of the user from an image or motion of the eyeball (eye)161of the user based on the output signal from the line-of-sight detection sensor164, and outputs detected information to the system control unit50. Thus, the dichroic mirror162, the image-forming lens163, the line-of-sight detection sensor164, the infrared light-emitting diode166, and the line-of-sight detection circuit165constitute a line-of-sight detection block160.

In the present exemplary embodiment, the line-of-sight detection block160is used to detect a line-of-sight based on a method called a cornea reflection method. The cornea reflection method is a method for detecting the orientation and position of a line of sight based on a positional relationship between reflected light, which is infrared light emitted from the infrared light-emitting diode166and reflected especially by a cornea of the eye (eyeball)161, and a pupil of the eye (eyeball)161. Other various methods for detecting the orientation and position of a line of sight include a sclera reflection method that utilizes a difference in light reflectance between black and white eye regions. Line-of-sight detection methods other than the above-described methods can also be used as long as the orientation and position of a line of sight can be detected.

The outer finder display unit43displays various setting values, including the shutter speed and the aperture, for the digital camera100via an outer finder display unit drive circuit44.

A nonvolatile memory56is an electrically erasable recordable memory, such as a flash read-only memory (ROM). The nonvolatile memory56stores constants, programs, and the like for operations of the system control unit50. Programs stored in the nonvolatile memory56refer to computer programs for executing various flowcharts to be described below according to the present exemplary embodiment.

The system control unit50is a control unit including at least one processor or circuit, and controls the overall operation of the digital camera100. Each processing according to the present exemplary embodiment to be described below is implemented such that the system control unit50executes the above-described programs recorded in the nonvolatile memory56. A system memory52is, for example, a random access memory (RAM). Constants and variables for operations of the system control unit50and programs read from the nonvolatile memory56are loaded into the system memory52. The system control unit50also performs display control by controlling the memory32, the display unit28, and the like.

A system timer53is a time measurement unit for measuring time to be used for various control operations and time of a built-in clock.

Various operation members each serving as an input unit that receives an operation from the user include at least the following operation units as an operation unit70. The operation units includes the shutter button61, the MC65, the touch panel70a, the main electronic dial71, the sub-electronic dial73, the cross key74, the SET button75, the video button76, the AE lock button77, the zoom button78, the reproduction button79, and the menu button81. The mode selection switch60and the power switch72are also operation members that receive an operation from the user.

The operation unit70, the mode selection switch60, and the power switch72each function as an operation means for inputting various operation instructions to the system control unit50.

The mode selection switch60changes an operation mode of the system control unit50to any one of a still image capturing mode, a moving image capturing mode, and the like. The still image capturing mode includes an automatic image capturing mode, an automatic scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). The still image capturing mode further includes various scene modes as image capturing settings each for a different image capturing scene, and a custom mode. The mode selection switch60enables the user to directly select any one of these modes. Alternatively, the mode selection switch60may switch the screen to an image capturing mode list screen, and the user may select any one of the plurality of displayed modes and change the mode using other operation members. Similarly, the moving image capturing mode may also include a plurality of modes.

The shutter button61is a two-stage switch including a first shutter switch62and a second shutter switch64.

The first shutter switch62turns on in the middle of an operation of the shutter button61provided on the digital camera100, which is called halfway pressing (image capturing preparation instruction), to generate a first shutter switch signal SW1. The first shutter switch signal SW1causes the system control unit50to start an imaging capturing preparation operation, such as AF processing, AE processing, AWB processing, and EF processing.

The second shutter switch64turns on upon completion of an operation of the shutter button61, which is called full pressing (image capturing instruction), to generate a second shutter switch signal SW2. The second shutter switch signal SW2causes the system control unit50to start a series of image capturing processing operations ranging from reading of a signal from the image capturing unit22to writing a captured image as an image file into the recording medium200.

A power supply control unit80includes a battery detection circuit, a direct-current (DC) to DC (DC-DC) converter, and a switch circuit for selecting a block to be supplied with power, and detects an attachment of a battery, the battery type, and the remaining battery capacity. The power supply control unit80also controls the DC-DC converter based on the detection result and an instruction from the system control unit50to supply required voltages to the units, including the recording medium200, for a required period of time. A power supply unit30includes a primary battery, such as an alkaline battery or a lithium (Li) battery, a secondary battery, such as a NiCd battery, a NiMH battery, or a Li battery, and an alternating current (AC) adaptor.

A recording medium interface (I/F)18is an interface to the recording medium200, such as a memory card or a hard disk. The recording medium200is, for example, a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

A communication unit54establishes a wireless connection or a wire cable connection to transmit and receive a video image signal and an audio signal. The communication unit54is connectable to a wireless local area network (LAN) and the Internet. The communication unit54can communicate with an external apparatus using Bluetooth® or Bluetooth® Low Energy. The communication unit54is configured to transmit images (including an LV image) captured by the image capturing unit22and images stored in the recording medium200, and to receive images and other various information from an external apparatus.

An attitude detection unit55detects an attitude of the digital camera100in the gravity direction. Based on the attitude detected by the attitude detection unit55, the system control unit50can determine whether the image captured by the image capturing unit22is an image captured with the digital camera100horizontally held or an image captured with the digital camera100vertically held. The system control unit50can append orientation information corresponding to the attitude detected by the attitude detection unit55to an image file of an image captured by the image capturing unit22, or can rotate the image before recording. An acceleration sensor, a gyroscope sensor, and the like can be used as the attitude detection unit55. Motions (e.g., pan, tilt, raising, and stand still) of the digital camera100can also be detected using the acceleration sensor and the gyroscope sensor serving as the attitude detection unit55.

The eye approach detection unit57is an eye proximity detection sensor that detects (eye approach detection) a state where the eye (object)161approaches (comes into contact with) the eyepiece16of the finder (eye-proximity state) and a state where the eye (object)161is being detached from (comes out of contact with) the eyepiece16of the finder (eye out-of-proximity state). The system control unit50turns the display of the display unit28and the EVF29on (display state) or off (not display state) depending on the state detected by the eye approach detection unit57. More specifically, at least in a case where the digital camera100is in the image capturing standby state and an automatic changeover setting is made as a changeover setting for the display destination of an LV image captured by the image capturing unit22, the display unit28is turned on as the display destination in the out-of-proximity state, and the EVF29is not displayed. In the proximity state, the EVF29is turned on as the display destination, and the display unit28is not displayed. For example, an infrared proximity sensor can be used as the eye approach detection unit57. The eye approach detection unit57can detect that some object is approaching the eyepiece16of the finder incorporating the EVF29. If the object has approached the eyepiece16, infrared light emitted from a light projecting portion (not illustrated) of the eye approach detection unit57is reflected and then received by a light-receiving portion (not illustrated) of the eye approach detection unit57. The eye approach detection unit57can also determine a distance (eye proximity distance) from the eyepiece16to the object based on the amount of the received infrared light. In this manner, the eye approach detection unit57performs eye proximity detection for detecting the eye proximity distance from the object to the eyepiece16. In the present exemplary embodiment, the light projecting portion and the light receiving portion of the eye approach detection unit57are devices different from the infrared light-emitting diode166and the line-of-sight detection sensor164, respectively. However, the infrared light-emitting diode166may also serve as the light projecting portion of the eye approach detection unit57, and the line-of-sight detection sensor164may serve as the light receiving portion. In a case where an object approaching the eyepiece16at a predetermined distance or less is detected in the eye out-of-proximity state (non-approach state), the eye approach detection unit57determines that the eye has approached the eyepiece16. In a case where an object in the eye-proximity state (approach state) has been detached from the eyepiece16by a predetermined distance or more, the eye approach detection unit57determines the eye has separated from the eyepiece16. Different values may be used as a threshold for detecting the eye-proximity state and a threshold for detecting the eye out-of-proximity state, respectively, for example, by setting a hysteresis. Once the eye-proximity state is detected, the eye-proximity state continues until the eye out-of-proximity state is detected. Once the eye out-of-proximity state is detected, the eye out-of-proximity state continues until the eye-proximity state is detected. The infrared proximity sensor is an example of the eye approach detection unit57. Other sensors configured to detect an approach of an eye or object with which the eye-proximity state can be determined may be employed as the eye approach detection unit57.

The system control unit50can detect the following operations or states based on the output from the line-of-sight detection block160.A state where a new line of sight of the user's eye in proximity to the eyepiece16is input (detected), i.e., the user starts the line-of-sight input.A state where the line of sight of the user's eye in proximity to the eyepiece16is being input.A state where the user's eye in proximity to the eyepiece16is gazing.A state where the user's eye in proximity to the eyepiece16shifts the line of sight, i.e., the user ends the line-of-sight input.A state where the user's eye in proximity to the eyepiece16is not performing the line-of-sight input.

The term “gaze” used herein refers to a state where the line-of-sight position of the user does not exceed a predetermined movement amount within a predetermined period of time.

The touch panel70aand the display unit28can be integrally formed. For example, the touch panel70ais configured to have a light transmittance that does not interfere with the display on the display unit28, and is disposed on an upper layer of the display surface of the display unit28. Input coordinates on the touch panel70aare associated with display coordinates on the display screen of the display unit28. This makes it possible to provide such a graphical user interface (GUI) that virtually allows the user to directly operate the screen displayed on the display unit28. The system control unit50can detect the following operations on the touch panel70a, or the states thereof.An operation to newly touch the touch panel70awith a finger or a pen that has not been in contact with the touch panel70a, i.e., to start touching (hereinafter referred to as a “touch-down”)A state where the finger or the pen is in contact with the touch panel70a(hereinafter referred to as a “touch-on”)An operation to move the finger or the pen while being in contact with the touch panel70a(hereinafter referred to as a “touch-move”)An operation to detach the finger or the pen that has been in contact with the touch panel70afrom the touch panel70a, i.e., to end touching (hereinafter referred to as a “touch-up”)A state where the finger or the pen is out of contact with the touch panel70a(hereinafter referred to as a “touch-off”)

When a touch-down is detected, a touch-on is also detected at the same time. After the touch-down, the touch-on is normally kept being detected until a touch-up is detected. A touch-move is detected in a state where the touch-on is detected. Even when the touch-on is detected, a touch-move is not detected if the touch position is not moving. After a touch-up is detected for all fingers and the pen that have been in contact with the touch panel70a, a touch-off is detected.

A notification about the above-described operations and states and position coordinates of a position where a finger or a pen contacts the touch panel70ais provided to the system control unit50via an internal bus. Based on the notification information, the system control unit50determines what kind of operation (touch operation) has been performed on the touch panel70a. For a touch-move, the moving direction of the finger or the pen moving on the touch panel70acan be determined for each of vertical and horizontal components on the touch panel70abased on changes of the position coordinates. In a case where a touch-move by a predetermined distance or more is detected, the system control unit50determines that a slide operation has been performed. An operation to quickly move the finger by a certain distance while being in contact with the touch panel70aand then release the finger from the touch panel70ais referred to as a flick. In other words, a flick is an operation to quickly flip the surface of the touch panel70awith the finger. In a case where a touch-move at a predetermined speed or higher by a predetermined distance or more is detected and then a touch-up is subsequently detected, it can be determined that a flick has been performed (it can be determined that a flick has been performed following a slide operation). A touch operation to simultaneously touch a plurality of positions (e.g., two positions) and bring these positions close to each other is referred to as a “pinch-in”. A touch operation to move these positions away from each other is referred to as a “pinch-out”. A pinch-out and a pinch-in are collectively referred to as a pinch operation (or simply referred to as a “pinch”). The touch panel70amay be any one of various types of touch panels, including a resistive touch panel, a capacitive touch panel, a surface acoustic wave touch panel, an infrared touch panel, an electromagnetic induction touch panel, an image recognition touch panel, and a photosensor touch panel. Examples of the touch detection method may include a method of detecting a touch when the finger or the pen comes into contact with the touch panel70a, and a method of detecting a contact when the finger or the pen approaches the touch panel70a.

When a touch-move operation is performed in the eye-proximity state, the user can set one of absolute position designation and relative position designation as a method for designating the position of a position index according to the touch-move operation. For example, assuming that an AF frame is set as the position index, when the absolute position designation is set and the touch panel70ais touched, an AF position associated with the touched position (position where coordinates are input) is set. In other words, position coordinates where the touch operation is performed are associated with position coordinates of the display unit28. On the other hand, when the relative position designation is set, position coordinates where the touch operation is performed are not associated with position coordinates of the display unit28. In the relative position designation, the touch position is moved by a distance corresponding to the movement amount of the touch-move operation in the movement direction of the touch-move operation from the AF position currently set, regardless of the touch-down position on the touch panel70a.

Setting Screen

FIG.3Aillustrates a menu setting screen including setting items301to305indicating setting values, respectively.

The setting item301indicates an AF operation in which “one shot AF” for locking the focus after AF is activated (after the image capturing preparation instruction is issued) and “servo AF” for tracking the focus can be set.

The setting item302indicates an AF area in which the size of the AF area can be set. In the case of setting a large number of setting values, a dedicated screen (FIG.3B) is used to set the setting values in a deeper layer. As illustrated inFIG.3B, the user selects the setting value306from among the displayed setting values.

The setting item303indicates a tracking setting for setting whether to implement tracking after AF is activated (after the image capturing preparation instruction is issued). An object to be tracked is determined from the AF area before AF is activated (before the image capturing preparation instruction is issued), and tracking is performed on the entire screen by increasing the size of the AF area to the entire area after AF is activated (after the image capturing preparation instruction is issued). According to this setting, when a main object is detected, a detection frame is displayed for the main object to provide a notification about the detected main object to the user. The main object is determined based on the setting of the object to be detected in the setting item304.

The setting item304indicates the setting of the object to be detected. A type of a detection target to be prioritized to determine the main object is selected in the setting item304. In the present exemplary embodiment, as illustrated inFIG.3D, any one of types310, that is, “person”, “animal priority”, “vehicle priority”, and “OFF” can be set. As indicated by item311inFIG.3D, advanced settings can be made depending on the setting values. In this case, the image processing unit24can execute object detection processing for detecting a specific object using captured image data. In the present exemplary embodiment, examples of the specific object to be detected include a person, an animal, such as a dog or wild bird, a vehicle, and a part corresponding to a main area in the object (spot detection). For example, when spot detection is performed on a person, the body, head, pupil, and face of the person are detected. For example, when spot detection is performed on an animal, the pupil, face, and body of the animal are detected. In the spot detection, for example, when “vehicle priority” is selected, local spots can be set and a driver in a vehicle, a first vehicle of a train (railway), and a cockpit in an aircraft are detected. For these detection methods, a learning method using machine learning, recognition processing by image processing, and the like are used.

For example, the following types of machine learning are used.(1) Support Vector Machine(2) Convolutional Neural Network(3) Recurrent Neural Network

As an example of recognition processing, in the case of detecting a face, for example, a skin color area may be extracted from gradation colors in each pixel represented by image data and the face is detected based on the degree of matching with a prepared face contour plate. In addition, a face detection method in which face feature points corresponding to the eyes, nose, mouse, and the like are extracted by a known pattern recognition technique can also be employed. Further, the main area detection method according to the present exemplary embodiment is not limited to the above-described methods, and any other methods can also be employed.

As illustrated inFIG.3C, a screen on which AF area setting309and tracking setting308are simultaneously made can also be set. In this case, an LV through image is displayed on the background and the user can change the settings while viewing the display of an AF area307.

Button Customization Function

FIGS.4A to4Eeach illustrate a menu for setting a button customization function for tracking activation and function call to be displayed on the display unit28or the EVF29.

To customize a tracking activation button, for example, as indicated by an item401inFIG.4A, when the user moves a cursor (index) indicating a current processing target position to an operation member (e.g., a button) on the screen to which a function is to be allocated, the allocatable functions are displayed in a selectable manner as indicated by an item402inFIG.4B. InFIG.4B, “start/stop tracking” is allocated to the SET button. The function to “start/stop tracking” corresponds to a function of starting tracking based on the position of the AF area and a function of stopping the tracking that has been started, regardless of the tracking setting. This function can be activated in the image capturing standby state (SW0), during activation of AF (first shutter switch signal SW1is held or ON), and during servo AF continuous shooting (SW2ON).

For example, as indicated by an item403inFIG.4C, when the user moves the cursor to the button on the screen to which a function is to be allocated, the allocatable functions are displayed in a selectable manner as indicated by an item404inFIG.4D. InFIG.4D, “call registration function” is allocated to an AE lock (AEL) button.

The function of “call registration function” is a function for calling a function that is preliminarily registered by the user. When an item405illustrated inFIG.4Dis pressed by the user, an advanced settings screen illustrated inFIG.4Eopens. A checkbox406illustrated inFIG.4Eenables the user to register a function to be called by setting a setting item407and a setting value408to be called and by checking the checkbox406. InFIG.4E, “one shot” is set as an AF operation and “home position (HP)” is registered as a call position. A gray-out display may be performed to indicate that an item cannot be registered during setting of another item. For example, the AF area illustrated inFIG.4Eis grayed out to indicate that the AF area cannot be registered. For example, like in the tracking setting and spot detection illustrated inFIG.4E, if the user cannot check the corresponding checkbox, the current setting value cannot be changed.

Image Capturing Information Display and Icons Indicating Setting Values

FIG.5Aillustrates an example of a display screen of the display unit28. The display screen includes an icon501indicating an AF area and a tracking setting, an icon502indicating an AF operation setting, an icon503indicating a setting of an object to be detected, an icon504indicating a pupil detection setting, and an icon505indicating a tracking activation state. The user can check the current settings and states by checking the icons501to505.

FIG.5Billustrates an example of an icon display list. As indicated by an item506, the icon501is represented by a combination of an AF area and a tracking setting. As indicated by items507,508, and509, the icons502to504that are prepared according to the setting values are displayed. As indicated by an item510, the icon505indicating a tracking control state is displayed.

Frame Display Before and After Activation of AF

FIG.6illustrates a display of each frame before and after activation of AF to be displayed on the display unit28or the EVF29.

FIG.6illustrates an example where the display of the frame before AF is activated is different from that after AF is activated. However, the frame may be displayed in the same manner before and after AF is activated. The display of the frame may be changed using colors such that, for example, green is used when “one shot” is set as the AF operation, and blue is used when “servo AF” is set.

An AF area frame601represents a narrow area such as a spot, one point, or a zoom area. The AF area frame601is represented by a rectangular shape before AF is activated, and is represented by a bold rectangular shape after AF is activated as indicated by an AF area frame602inFIG.6.

An AF area frame603represents a wide area such as a zone or an entire area. The AF area frame603is represented by an area square brackets before the AF is activated. After AF is activated, a small rectangular frame604is displayed in an in-focus portion within the area of the AF area frame603. Additionally, a plurality of rectangular frames604may be displayed within the area of the AF area frame603. If the AF area frame603indicates the entire area of the screen, for example, when “entire area” is set, the display of the AF area frame (brackets)603may be omitted before the AF is activated.

A frame605represents a detection frame. The frame605is displayed for an automatically detected specific object by setting the specific object as a target to be detected. InFIG.6, “person” is set and the frame is displayed at a position corresponding to the face of the person. The detection frame605can also be displayed on the pupil area in the pupil detecting setting. If “animal” and “vehicle” are set, the detection frame605is displayed for the entire body and face of the animal and the vehicle. After AF is activated, a rectangular frame display606corresponding to the detection frame605is performed. In the present exemplary embodiment, the frame display606is represented by a dotted line so that the frame display606can be distinguished from the AF area frame602. Each detection frame can be updated and the object can be tracked. However, the frame position is fixed at the in-focus position after one-shot AF is activated.

A frame607represents a tracking frame. The frame607is displayed when the user selects the detection frame605, or when a tracking target is selected by a user operation. Examples of various types of user selection operation include selection based on an operation position on the touch panel70a, selection by a tracking start operation, and selection of the detection frame605using a cross button. After AF is activated, a dotted-line double rectangular frame608is displayed. Like the detection frame, each tracking frame can be updated and the object can be tracked. However, the frame position is fixed at the in-focus position after one-shot AF is activated.

A frame609represents an HP frame. The user stores the HP frame609by setting the AF area frames601and603at desired positions to be registered and by performing a registration operation. The HP frame609can be called by selecting “call HP” or “call image capturing function” on the button customization screen. The HP frame609can be called, for example, during an operation, or during a continuous period of time until the call is performed again. After AF is activated, the HP frame609is represented by a bold rectangular shape610, like the AF area.

Representation of Frame Based on Combination of AF Area and Tracking Setting

FIG.7illustrates an example of frame representation based on a combination of an AF area and a tracking setting.

When tracking is “OFF”, states7-A to7-I each indicating the tracking setting and the AF area are displayed, and when tracking “ON”, states7-J to7-R each indicating the tracking setting and the AF area are displayed.FIG.7illustrates conditions when “AF area frame”, “detection frame”, and “tracking frame” are active.

A case where tracking is “OFF” in the states7-A to7-I will now be described.

When the AF area frame is active, the AF area frame, such as a one-point AF area frame701or a zone AF area frame702, is displayed according to each setting (7-A,7-B,7-C).

When tracking is “OFF”, object detection is not performed and thus the detection frame is not active. Accordingly, no frame is displayed in the states7-D,7-E, and7-F. For convenience of explanation, the screen area corresponding to the states7-D,7-E, and7-F illustrated inFIG.7is grayed out and LV display is performed.

When tracking is selected and performed by the user, a tracking frame703is displayed (7-G,7-H,7-I). In this case, the tracking frame corresponds to, for example, a tracking frame set when tracking is started based on an operation position on the touch panel70a, or a tracking frame set when tracking is started based on a tracking start operation position.

Next, a case where tracking is “ON” in the states7-J to7-R will be described.

In the states7-J to7-O, when tracking is “ON” and a main object is detected, both the AF area frame and the detection frame are displayed and a frame to be actually focused during activation of AF is active. If the main object is not detected, the AF area frame is displayed, like in the case where tracking is “OFF”.

The frame to actually focus on during activation of AF when the main object is detected is selected in the following manner.

First, in a condition where the detection frame does not overlap the AF area frame, the AF area frame is active (7-J,7-K,7-L). If the AF area corresponds to the entire area of the screen, based on the condition that the detection frame does not overlap the AF area frame even when the object is not detected in the AF area, the AF area entire area is active (7-L). In this case, however, since the entire screen area is set as the AF area, the frame display is not performed. Accordingly, for convenience of explanation, the screen area corresponding to the AF area7-L is grayed out, but LV display is performed. When the AF area frame is active, the AF area frames704and706are represented by a solid line and the detection frame is represented by a transparent line705. During activation of AF, a focus adjustment is performed on the AF area frame.

Next, in a condition where the detection frame overlaps the AF area frame, the detection frame is active (7-M,7-N,7-O). For example, in the case of pupil detection or spot detection, if the face or entire body constituting the object overlaps the AF area frame even when the pupil or spot area of the object does not overlap the AF area, the detection frame can be activated in some cases. When the detection frame is active, the detection frame is represented by a solid line707and the AF area frame is represented by a transparent line708.

In a case where tracking is selected and to be performed by the user and tracking is “ON”, when a main object is detected, the tracking frame is active and the tracking frame is displayed (709) during execution of tracking (7-P,7-Q,7-R). When the main object is not detected, the tracking frame is displayed, like in the case where tracking is “OFF”.

State Transition Among Image Capturing Standby State, Image Capturing Preparation State, and Image Capturing State

FIG.8Aillustrates state transition among an image capturing standby state (8-A,8-D), an image capturing preparation state (8-B,8-E), and an image capturing state (8-C,8-F).

An icon801indicates a setting value for AF area×tracking setting. An icon802indicates a setting value for AF operation. An icon803indicates a setting value for an object to be detected. An icon804indicates a setting value for pupil detection. An icon805indicates a tracking state. A frame display806indicates an AF area for zone AF.

A frame display807indicates a detection frame for an object to be detected. In the example illustrated inFIG.8A, the object is detected in the AF area, and thus the detection frame is active and the AF area is not active. In the state of8-A (seeFIG.7), when an operation8aof AF start (SW1) is performed, the state transitions to8-B. In this case, an AF operation is executed in the active frame area.

In the state of8-B, in-focus display is performed by a frame808. Since one shot is set as the AF operation, the frame position is locked. A notification about an in-focus state is provided to the user by changing colors such that green is used for the in-focus state and red is used for a not-in-focus state. When an operation8bof image capturing start (SW2) is performed, the state transitions to8-C.

In the state of8-C, image capturing is performed. A frame809indicates that image capturing is being executed. In the present exemplary embodiment, the frame809is not displayed in some cases because it can be determined that image capturing is being executed based on image capturing sound. However, during silent image capturing, the frame809is displayed in many cases. Since the focus is locked during continuous shooting in the one-shot AF image capturing operation, the frame808performed at SW1is not performed.

The state transition in the basic image capturing operation has been described above. Next, function call patterns will be described.

In the state of8-A, when an operation8cof image capturing function call is performed, the state transitions to8-D. In this case, the icon801changes to “entire area×tracking”, the icon802changes to “servo AF”, the icon803changes to “animal priority”, and the icon804changes to “pupil detection OFF”. After the call operation, animal detection is performed on the entire area. Accordingly, a little bird located at the lower right position is detected as a main object, and a detection frame810is displayed. Since the entire area is set as the AF area, the detection frame810is an active frame.

In the state of8-A, when an operation8dof image capturing function call is performed, the state transitions to8-E. The display and the like in the operation8dare similar to those in the operation8c. The operation8ddiffers from the operation8cin that the AF operation is simultaneously executed and an in-focus frame811is displayed after AF is executed. The operation8dcan also be executed in the states8-B and8-C, and the states transition as illustrated inFIG.8A.

In the state of8-D, when the AF operation is executed by the operation8a, the state transitions to8-E. In a case where the tracking setting is “ON” and “servo AF” is set as the AF operation, the AF area is expanded to the entire area and then the object is tracked during a period in which the AF operation is held. If object tracking is to be cancelled, the AF operation is stopped or the “stop tracking” operation is performed on the button customization screen.

In the state of8-E, when image capturing is started by the operation8b, the state transitions to8-F. In servo continuous shooting, image capturing is repeated by continuously focusing on the object while tracking the object.

FIG.8Billustrates a menu set by the button customization function described above with reference toFIGS.4A to4E. A menu called by the operation8cis displayed on the left side ofFIG.8B, and a menu called by the operation8dis displayed on the right side ofFIG.8B.

Call Patterns

FIGS.9A to9Cillustrate three call patterns, respectively, according to the present exemplary embodiment.

FIG.9Aillustrates a pattern in which “zone×tracking OFF×detection OFF” is called from “entire area×person detection”. Before the call operation, image capturing is performed by prioritizing “person” over the other types on the entire area. Since the object moves with a large motion, image capturing is performed in the servo mode. In the case of capturing an image mainly of an object rather than a person, the detection setting is turned off and auto selection is performed in a specific area. At the same time, AF can be started and image capturing can be performed in a procedure, for example, image capturing can be performed immediately after a one-shot call. The display illustrated inFIG.9Ais effective when flowers displayed on the frontmost side in the area are focused. This display is effective, for example, when the object is switched to a bouquet tossed by a bride in bouquet toss during a wedding ceremony.

FIG.9Billustrates a pattern in which “one point (HP)×tracking ON×animal detection” is called from “entire area×person detection”. Before the call operation, image capturing is performed by prioritizing “person” over the other types on the entire area. In case an object with a large motion appears, one point is registered as HP in the vicinity of the center as a point where servo AF is started. In this state, even if an object with a large motion, such as an animal, appears, this function is called to catch the object from the area in the vicinity of the center and start tracking. Even if a rapidly moving dog appears as illustrated inFIG.9A, the object can be caught at the center, thereby enabling tracking of the object. The size (one point, zone) and position of HP can be stored. This increases options for image capturing by calling the size and position of HP simultaneously with the detection setting.

FIG.9Cillustrates a pattern in which “entire area×vehicle detection” is called from “zone×person detection”. This is effective when image capturing is performed, before the call operation, prioritizing “person” over the other types on the entire area, then performing image capturing by changing the main object to a vehicle. Not only an automobile, but also a motorcycle, a train, an aircraft, and the like can be detected as a vehicle. This pattern is also effective in ceremonial image capturing, circuit image capturing, and event image capturing. Also, pinpoint focusing in the vehicle can be performed using the spot detection settings described above with reference toFIGS.3A to3D.

There are other effective scenes in addition to the above-described effective scenes. When a combination of an AF area and a detection setting is called, user's customization performance can be enhanced and image capturing can be performed without missing a shot.

Control Flow to Call Combination of Object Detection Type and AF Area

FIG.10Ais a flowchart illustrating a control operation according to the present exemplary embodiment. Each process is implemented such that the system control unit50loads programs stored in the nonvolatile memory56into the system memory52and executes the programs.

In the following flowchart, processing and control operation are performed by the system control unit50.

In step S1001, it is determined whether a function call is performed. If the function call is performed (YES in step S1001), the processing proceeds to step S1002. If the function call is not performed (NO in step S1001), the processing proceeds to step S1003.

In step S1002, control for the function call is started.

In step S1003, it is determined whether an AF operation (SW1) is performed. If the AF operation is performed (YES in step S1003), the processing proceeds to step S1004. If the AF operation is not performed (NO in step S1003), the processing returns to step S1001. During this process, if normal image capturing preparation settings (exposure change, AF area movement, and other settings) are made, the settings are changed.

In step S1004, the AF operation is executed. In the one shot AF setting, the focus position is fixed after completion of AF, and the focus is tracked also after completion of AF in the servo AF setting.

In step S1005, it is determined whether a function call is performed. If the function call is performed (YES in step S1005), the processing proceeds to step S1006. If the function call is not performed (NO in step S1005), the processing proceeds to step S1008.

In step S1006, control for the function call is started.

In step S1007, the AF operation is executed. If the setting value is not changed in step S1006, the AF operation in step S1007is skipped.

In step S1008, it is determined whether an image capturing operation (SW2) is performed. If the image capturing operation is performed (YES in step S1008), the processing proceeds to step S1010. If the image capturing operation is not performed (NO in step S1008), the processing proceeds to step S1009.

In step S1009, it is determined whether the AF operation (SW1) is held. If the AF operation is held (YES in step S1009), the processing returns to step S1005. If the AF operation is not held (NO in step S1009), the processing returns to step S1001.

In step S1010, an image capturing operation is executed.

In step S1011, it is determined whether a function call is performed. If the function call is performed (YES in step S1011), the processing proceeds to step S1012. If the function call is not performed (NO in step S1011), the processing proceeds to step S1017.

In step S1012, it is determined whether a servo AF operation is being executed. If the servo AF operation is being executed (YES in step S1012), the processing proceeds to step S1014. If the servo AF operation is not being executed (NO in step S1012), the processing proceeds to step S1013.

In step S1013, it is determined in advance whether the AF operation is changed in the function call. If the AF operation is changed (YES in step S1013), the processing proceeds to step S1015. If the AF operation is not changed (NO in step S1013), the processing proceeds to step S1017.

In steps S1014and S1015, control for the function call is started.

In step S1016, the AF operation is executed. If the setting values are not changed in steps S1014and S1015, the AF operation in step S1016is skipped.

In step S1017, it is determined whether to perform continuous image capturing. If continuous image capturing is to be performed (YES in step S1017), the processing proceeds to step S1018. If continuous image capturing is not to be performed (NO in step S1017), the processing proceeds to step S1019.

In step S1018, the image capturing operation is executed. In this case, since continuous image capturing is executed, image capturing is performed according to the current driving setting.

In step S1019, it is determined whether image capturing is completed. If image capturing is completed (YES in step S1019), the processing in the flowchart is terminated. If image capturing is not completed (NO in step S1019), the processing returns to step S1011.

Function Reading Control Flow

FIG.10Bis a flowchart illustrating details of the function reading processes in steps S1002, S1006, S1014, and S1015illustrated inFIG.10A. Each process is implemented such that the system control unit50loads programs stored in the nonvolatile memory56into the system memory52and executes the programs.

In the following flowchart, processing and control operation are performed by the system control unit50.

In step S1020, it is determined whether the setting value is changed. If the setting value is changed (YES in step S1020), the processing proceeds to step S1021. If the setting value is not changed (NO in step S1020), the function call sub-flow ends.

In step S1021, it is determined whether HP is called. If HP is called (YES in step S1021), the processing proceeds to step S1024. If HP is not called (NO in step S1021), the processing proceeds to step S1022.

In step S1022, it is determined whether the AF area is changed. If the AF area is changed (YES in step S1022), the processing proceeds to step S1023. If the AF area is not changed (NO in step S1022), the processing proceeds to step S1025.

In step S1023, the AF area is changed.

In step S1024, the AF position is set as HP.

In step S1025, it is determined whether the AF operation is changed. If the AF operation is changed (YES in step S1025), the processing proceeds to step S1026. If the AF operation is not changed (NO in step S1025), the processing proceeds to step S1027.

In step S1026, the AF operation is changed.

In step S1027, it is determined whether the tracking setting is changed. If the tracking setting is changed (YES in step S1027), the processing proceeds to step S1028. If the tracking setting is not changed (NO in step S1027), the processing proceeds to step S1029.

In step S1028, the tracking setting is changed.

In step S1029, it is determined whether the setting of the object to be detected is changed. If the setting of the object to be detected is changed (YES in step S1029), the processing proceeds to step S1030. If the setting of the object to be detected is not changed (NO in step S1029), the processing proceeds to step S1033.

In step S1030, it is determined whether advanced settings in the setting of the object to be detected are changed. If the advanced settings are changed (YES in step S1030), the processing proceeds to step S1032. If the advanced settings are not changed (NO in step S1030), the processing proceeds to step S1031.

In step S1031, the setting of the object to be detected is changed.

In step S1032, the setting of the object to be detected and the advanced settings are changed.

In step S1033, it is determined whether the pupil detection setting is changed. If the pupil detection setting is changed (YES in step S1033), the processing proceeds to step S1034. If the pupil detection setting is not changed (NO in step S1033), the processing proceeds to step S1035.

In step S1034, the pupil detection setting is changed.

In step S1035, it is determined whether a tracking operation is to be performed. If the tracking operation is to be performed (YES in step S1035), the processing proceeds to step S1036. If the tracking operation is not to be performed (NO in step S1035), the function call sub-flow ends.

In step S1036, it is determined whether each frame is active based onFIG.7, and the main object is determined based on the AF setting and detection setting after the call operation.

The above-described state transitions and flowcharts based on the exemplary embodiments enable the user to perform comfortable image capturing by calling various combinations of image capturing settings and detection settings, which leads to an improvement in operability.

Especially, the present exemplary embodiment is effective as a method for determining image capturing conditions in a digital camera, and this method enables the user to perform comfortable image capturing without missing a shot.

While the present exemplary embodiment illustrates an example where a display frame is represented by a rectangle as an index for indicating a position or size on a screen, the index is not limited to a rectangle, but instead may be any shape such as a circle or hexagonal shape. Alternatively, a target display such as a cross indicating a position may be used.

The system control unit50can generate metadata based on image capturing conditions as a combination of the image capturing setting and the detection setting, and can record the generated metadata by adding it to image data captured by the image capturing unit22on the recording medium200. In the case of displaying recorded images, the metadata recorded in association with the image data may be displayed. The image data, metadata, and the like to be recorded are recorded based on standards such as an Exchangeable Image File Format (Exif).

The above-described various control operations that have been described to be performed by the system control unit50may be performed by one piece of hardware or by controlling the entire apparatus by sharing processing among a plurality of pieces of hardware.

While the disclosure has been described in detail above based on exemplary embodiments, the disclosure is not limited to these specific exemplary embodiments. Various modifications made without departing from the scope of the invention are also included in the disclosure. Further, the above-described exemplary embodiments are merely examples of the disclosure, and the exemplary embodiments can be combined as appropriate.

While the exemplary embodiments described above illustrate an example where the disclosure is applied to the digital camera100, the disclosure is not limited to this example. The disclosure can be applied to any display control apparatus, as long as the display control apparatus can perform control for image processing. Specifically, the disclosure can be applied to a mobile phone terminal, a portable image viewer, a personal computer (PC), a printer apparatus including a finder, a home electric appliance including a display unit, a digital photo frame, a projector, a tablet PC, a music player, a game console, an electronic book reader, and the like.

According to an exemplary embodiment, it is possible to immediately change settings on the basis of object detection.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2021-098472, filed Jun. 14, 2021, which is hereby incorporated by reference herein in its entirety.