Patent Description:
Patent Document <NUM> proposes an imaging device that detects a pupil area, which is an area of a pupil of a person, and focuses on the detected pupil area.

Patent Document <NUM>: International Publication No. <CIT>.

<CIT> discloses an imaging apparatus, an imaging method, and a program that are capable of easily setting a type of a subject as a focusing target in each imaging. An image sensor acquires an image. An operation unit selects a type of a subject as a focusing target in each imaging of the image acquired by the image sensor. In the case where the type of the subject as a focusing target is selected by the operation unit, a microcomputer detects an area of the subject of that type from the image and sets the detected area as an in-focus area of the image. The present disclosure can be applied to an imaging apparatus, for example.

<CIT> discloses an apparatus where the user focuses on the target object to obtain an image. An image capturing apparatus includes an object detection unit which detects an object from an image based on a signal output from an image sensor, a display unit which displays a marker corresponding to the position of the object detected by the object detection unit, and a change unit which changes the position of the marker displayed on the display unit. The change unit moves the marker to the position of an object detected by the object detection unit, that is different from the current position of the marker displayed on the display unit.

<CIT> discloses an imaging device capable of taking digital images, comprising a facial organ detection section for detecting facial organs within a face from an image that has been taken by an imaging section; an eye selection section for comparing sizes of eyes that have been detected by the facial organ detection section and selecting the largest eye; an eye size comparative determination section for comparatively determining the size of the eye that has been selected by the eye selection section with respect to a specified value; an AF region setting section for setting an AF region in accordance with size and position of the selected eye when it has been determined by the eye size comparative determination section that the size of the eye is larger than a specified value, and setting an AF region in accordance with position and size of a face to which the selected eye belongs when it has been determined that the size of the eye is smaller than a specified value; and an AF processing section for carrying out an AF operation based on the AF region set by the AF region setting section.

<CIT> discloses a focusing control device, an imaging device, a focusing control method, and a focusing control program capable of enabling focusing on an eye region and obtaining an intended focusing result. The focusing control device includes a focusing position determination unit which determines a focusing position based on a captured image signal obtained by imaging a subject with an imaging element while moving a focus lens, and an AF region determination unit which determines an AF region based on a face region detected from a captured image signal obtained through imaging with the imaging element at an arbitrary time and an eye region detected from a captured image signal obtained through imaging at a time before the arbitrary time. The focus lens is driven to the focusing position determined by the focusing position determination unit based on a signal of the AF region.

Optional embodiments of the invention are described in the dependent claims. The claims do not cover embodiments described in relation to <FIG>, <FIG>, <FIG> describing a human pupil detection mode or a human as the subject.

The imaging device disclosed in Patent Document <NUM> detects only a pupil area of a person, and it has been difficult to detect a pupil area of an animal.

The present technology has been conceived in view of such a situation, and it aims to easily perform imaging focusing on a subject or a specific part of the subject.

Hereinafter, an embodiment for implementing the present technology will be described. Descriptions will be given in the following order.

An imaging device according to an embodiment of the present technology has a detection mode for each specific part, which detects an area of a specific part of a subject and uses the detected area of the specific part for focusing. Examples of the detection mode for each specific part include a human pupil detection mode and an animal pupil detection mode.

<FIG> is a diagram illustrating an exemplary process of detecting a pupil of a person.

<FIG> illustrates a screen displayed on a display unit of the imaging device in a case where the human pupil detection mode is set. The screen on the left side in <FIG> is a screen displayed before an operation for instructing a start of focusing is performed by a user. The screen on the right side is a screen displayed after the operation for instructing a start of focusing is performed by the user. Note that a face of a person is displayed on the screen.

In a case where the human pupil detection mode is set, as illustrated on the left side in <FIG>, a face area that overlaps a focus frame F is detected among the face areas that are areas of faces, and a face preliminary-notice frame PF that gives notice of the detected face area is displayed. In the example on the left side in <FIG>, the face preliminary-notice frame PF is displayed to partially overlap the focus frame F. The focus frame F is a focus setting frame for setting a focus.

In a case where the user performs the operation for instructing a start of focusing in this state, as illustrated on the right side in <FIG>, detection of a pupil area, which is an area of a pupil, is performed on the face area, and a pupil frame AE indicating the detected pupil area is displayed in place of the face preliminary-notice frame PF. Furthermore, focusing is performed on the pupil area surrounded by the pupil frame AE.

<FIG> is a diagram illustrating an exemplary process of detecting a pupil of an animal.

<FIG> illustrates a screen displayed on the display unit of the imaging device in a case where the animal pupil detection mode is set. The screen on the left side in <FIG> is a screen displayed before the operation for instructing a start of focusing is performed by the user. The screen on the right side is a screen displayed after the operation for instructing a start of focusing is performed by the user. Note that a face of an animal (cat) is displayed on the screen.

In a case where the animal pupil detection mode is set, as illustrated on the left side in <FIG>, detection of the pupil area is performed on the inside and outside of the focus frame F, and a pupil preliminary-notice frame PE that gives notice of the detected pupil area is displayed. Note that, at this time, the detection of the pupil area is performed with priority given to the inside of the focus frame F.

In a case where the operation for instructing a start of focusing is performed by the user in this state, as illustrated on the right side in <FIG>, the pupil frame AE indicating the pupil area is displayed in the pupil area in place of the pupil preliminary-notice frame PE. Furthermore, focusing is performed on the pupil area surrounded by the pupil frame AE.

For example, the pupil frame AE is displayed according to a display method different from the pupil preliminary-notice frame PE such as the pupil preliminary-notice frame PE is displayed as a white frame and the pupil frame AE is displayed as a green frame. Note that the face preliminary-notice frame PF and the pupil frame AE are displayed in a similar manner.

As described above, in a case where the human pupil detection mode is set, a face area is detected and the face preliminary-notice frame PF is displayed in the detected face area before the operation for instructing a start of focusing is performed by the user.

Meanwhile, in a case where the animal pupil detection mode is set, a pupil area is detected and the pupil preliminary-notice frame PE is displayed in the detected pupil area before the operation for instructing a start of focusing is performed by the user.

With the preliminary-notice frame displayed before the user performs the operation for instructing a start of focusing, it is known in advance that the intended position or an unintended position has been detected, whereby the user is enabled to select not performing automatic focusing. This makes it possible to easily perform imaging focusing on a specific part of the subject.

<FIG> and <FIG> are diagrams illustrating a method of instructing a start of focusing in a case where the animal pupil detection mode is set.

With the imaging device, the user can make an instruction for a focusing start by pressing a pupil autofocus (AF) button P, pressing an AF-ON button Q, pressing a shutter button R halfway, or the like.

As illustrated on the left side in <FIG>, the pupil AF button P is configured as, for example, a center button located at the center of a cross button provided on the back surface of the imaging device. The pupil AF button P is a dedicated button for instructing a start of focusing on the pupil, which is a specific part of the subject.

As described above, the pupil preliminary-notice frame is displayed in the detected pupil area in the image of the subject on the display unit of the imaging device before the operation for instructing a start of focusing is performed by the user.

In a case where the user presses the pupil AF button P in this state, the display unit displays a screen as illustrated on the right side in <FIG>.

As illustrated on the right side in <FIG>, the focus frame F and the pupil frame AE are displayed on the screen in a manner of overlapping the image in which a cat is facing the front. On the right side in <FIG>, the focus frame F is displayed at the center of the screen, which is around the back of the cat.

In a case where the user presses the pupil AF button P, a range up to approximately <NUM>% of the screen is set as a detection range W1 of the pupil area with reference to the center of the focus frame F. Note that the detection range W1 is not displayed in actuality. Note that the focus frame F itself may be set in a wide range like the detection range W1.

When the pupil area is detected in the detection range W1 in a case where the user presses the pupil AF button P, the pupil frame AE is displayed in the pupil area in place of the pupil preliminary-notice frame having been displayed before the pressing of the pupil AF button P.

Furthermore, as illustrated on the left side in <FIG>, the AF-ON button Q is provided on the upper side of the back surface of the imaging device, for example. The AF-ON button Q is a button for instructing a start of focusing on the pupil in the focus frame F.

As illustrated on the left side in <FIG>, the shutter button R is provided on the upper surface of the imaging device, for example. In a similar manner to the AF-ON button Q, the shutter button R is a button for instructing a start of focusing on the pupil in the focus frame F in the case of being pressed halfway by the user, and is a button for instructing a shutter in the case of being pressed fully by the user.

In a case where the user presses the AF-ON button Q in this state, or in a case where the user presses the shutter button R halfway, a screen as illustrated on the right side in <FIG> is displayed on the display unit.

As illustrated on the right side in <FIG>, the focus frame F and the pupil frame AE are displayed on the screen in a manner of overlapping the image in which a dog is sleeping next to the cat. On the right side in <FIG>, the focus frame F is displayed on the left pupil of the cat.

In a case where the user presses the AF-ON button Q, a range from the center of the focus frame F to the vicinity of the focus frame F is set as a detection range W2 of the pupil area. Note that the detection range W2 is not displayed in actuality.

When the pupil area is detected in the detection range W2 in a case where the user presses the AF-ON button Q, the pupil frame AE is displayed in the pupil area in place of the pupil preliminary-notice frame having been displayed before the pressing of the AF-ON button Q.

As described above, in the case of the animal pupil detection mode, the pupil preliminary-notice frame that gives notice of the pupil area is displayed in the imaging device before the operation for instructing a start of focusing is performed by the user. When the user performs the operation for instructing a start of focusing, the pupil frame AE indicating the pupil area is displayed, and the focusing is performed on the pupil area.

Therefore, it becomes possible to promptly grasp the position of the animal pupil in the image. As a result, the user can easily perform imaging focusing on the pupil of the animal.

Note that the position at which each button is provided on the imaging device is not limited to the position illustrated in <FIG> and <FIG>, and may be another position.

Furthermore, although an exemplary case of detecting a pupil as a specific part of the subject has been described above, the subject itself may be detected in a case where the subject is a small animal such as a bird.

<FIG> is a block diagram illustrating an exemplary main configuration of the imaging device.

An imaging device <NUM> illustrated in <FIG> has a detection mode for each specific part of the subject, which includes the human pupil detection mode and the animal pupil detection mode. Note that a detection mode according to a specific part can be provided in the imaging device <NUM>, not limited to a pupil of a person or a pupil of an animal. The user can select and set a desired mode from each detection mode for the subject or a specific part of the subject.

As illustrated in <FIG>, the imaging device <NUM> includes a lens <NUM>, a diaphragm <NUM>, an imaging element <NUM>, an analog signal processing unit <NUM>, an analog/digital (A/D) converter <NUM>, and a digital signal processing unit <NUM>. The imaging device <NUM> includes a lens driver <NUM>, a timing generator (TG) <NUM>, a gyroscope <NUM>, and a system controller <NUM>.

Furthermore, the imaging device <NUM> includes a display unit <NUM>, a storage <NUM>, an input unit <NUM>, an output unit <NUM>, a communication unit <NUM>, an operation unit <NUM>, and a drive <NUM>.

The lens <NUM> adjusts a focus to the subject, and collects the light from the focused position. The diaphragm <NUM> adjusts exposure.

The imaging element <NUM> images the subject to obtain a captured image. That is, the imaging element <NUM> photoelectrically converts the light from the subject to output it to, as image signals, the analog signal processing unit <NUM>. On the basis of such photoelectric conversion, the imaging element <NUM> may capture a still image, or may capture a moving image.

The analog signal processing unit <NUM> performs analog signal processing on the image signals obtained by the imaging element <NUM>. The A/D converter <NUM> performs A/D conversion on the image signals having been subject to the analog signal processing to obtain image data that are digital signals.

The digital signal processing unit <NUM> performs digital signal processing on the image data obtained by the A/D converter <NUM>. As the digital signal processing, the digital signal processing unit <NUM> at least detects an area of the subject or a specific part of the subject from the moving image supplied as the image data, and performs processing such as setting a focus area. Hereinafter, a specific part of the subject will be simply referred to as a specific part.

Furthermore, the digital signal processing unit <NUM> also performs processing for controlling display of a frame or the like indicating the area of the subject or the specific part on the basis of the detection result of the area of the subject or the specific part. Details of those processes will be described later.

Note that the contents of the digital signal processing are optional, and processing other than the above may be performed. For example, the digital signal processing unit <NUM> may perform, as the digital signal processing, color mixture correction, black level correction, white balance adjustment, demosaic processing, matrix processing, gamma correction, YC conversion, and the like. Furthermore, the digital signal processing unit <NUM> may perform codec processing, which is processing associated with encoding and decoding of image data, as the digital signal processing.

The lens driver <NUM> drives the lens <NUM> and the diaphragm <NUM>, and controls a focal length, exposure, and the like. The TG <NUM> generates and supplies synchronization signals to the imaging element <NUM> to drive the imaging element <NUM>, and controls imaging. The gyroscope <NUM> is a sensor that detects a position and an attitude of the imaging device <NUM>. The gyroscope <NUM> outputs detected sensor information to the A/D converter <NUM>.

The system controller <NUM> includes, for example, a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), and the like, and controls each processing unit of the imaging device <NUM> by executing a program or processing data. Furthermore, on the basis of signals supplied from the operation unit <NUM>, the system controller <NUM> receives operation input made by the user, and performs processing or control corresponding to the operation input.

For example, the system controller <NUM> can control the focal length, exposure, and the like on the basis of, for example, the detection result of the area of the subject or the specific part supplied from the digital signal processing unit <NUM>.

The display unit <NUM> is configured as, for example, a liquid crystal display or the like, and displays an image corresponding to the image data stored in the memory of the digital signal processing unit <NUM>. For example, the display unit <NUM> can display a taken image obtained by the imaging element <NUM>, a saved captured image, and the like.

The storage <NUM> stores image data stored in the memory of the digital signal processing unit <NUM>. At that time, the storage <NUM> stores coded data coded by the digital signal processing unit <NUM> to reduce the data volume. The coded data stored in the storage <NUM> is read out and decoded by the digital signal processing unit <NUM>, and is displayed on the display unit <NUM>, for example.

The input unit <NUM> includes an external input interface such as an external input terminal, and outputs, to the digital signal processing unit <NUM>, various kinds of data (e.g., image data and coded data) supplied from the outside of the imaging device <NUM> via the external input interface.

The output unit <NUM> includes an external output interface such as an external output terminal, and outputs various kinds of data supplied via the digital signal processing unit <NUM> to the outside of the imaging device <NUM> via the external output interface.

The communication unit <NUM> performs, with another device, predetermined communication that is at least one of wired communication or wireless communication, and exchanges data with the another device through the predetermined communication. For example, the communication unit <NUM> outputs various kinds of data (e.g., image data and coded data) supplied from the digital signal processing unit <NUM> to the another device through the predetermined communication. Furthermore, the communication unit <NUM> obtains various kinds of data from the another device through the predetermined communication, and outputs the obtained data to the digital signal processing unit <NUM>.

The operation unit <NUM> includes an optional input device such as a key, a button, or a touch panel, for example. The operation unit <NUM> includes the pupil AF button P, the AF-ON button Q, or the shutter button R described above with reference to <FIG> or <FIG>. The operation unit <NUM> receives operation input made by the user, and outputs signals corresponding to the operation input to the system controller <NUM>.

The drive <NUM> reads out information (programs, data, etc.) stored in a removable recording medium <NUM> such as a semiconductor memory, which is attached to the drive itself, for example. The drive <NUM> supplies the information read from the removable recording medium <NUM> to the system controller <NUM>. Furthermore, in a case where a writable removable recording medium <NUM> is attached to the drive <NUM>, the drive <NUM> can cause the removable recording medium <NUM> to store the information (image data, coded data, etc.) supplied via the system controller <NUM>.

Note that the lens <NUM>, the diaphragm <NUM>, and the lens driver <NUM> described above may be formed as an interchangeable lens <NUM> that is detachable (replaceable) from the imaging device <NUM> and has a housing separate from the imaging device <NUM>.

<FIG> is a block diagram illustrating an exemplary configuration of the digital signal processing unit <NUM>.

The digital signal processing unit <NUM> includes a memory <NUM>, a subject detection unit <NUM>, an area setting unit <NUM>, a display control unit <NUM>, and a codec processing unit <NUM>.

The memory <NUM> stores image data supplied from the A/D converter <NUM>. The image data is, for example, image data of each frame of a moving image or image data of a still image.

The subject detection unit <NUM> detects an area of a subject or a specific part from image data stored in the memory <NUM> on the basis of the signals corresponding to the operation input made by the user, which are supplied from the system controller <NUM>. The subject detection unit <NUM> outputs a detection result of the area of the subject or the specific part to the area setting unit <NUM> and the display control unit <NUM>.

The subject detection unit <NUM> includes a person detection unit <NUM>-<NUM>, an animal detection unit <NUM>-<NUM>, and an animal detection unit <NUM>-<NUM>.

In a case where a detection mode for each specific part of the subject is the human pupil detection mode, the person detection unit <NUM>-<NUM> detects a face area of a person, and outputs a detection result of the detected face area to the area setting unit <NUM> and the display control unit <NUM>. In a case where an operation for instructing a start of focusing is performed by the user, the person detection unit <NUM>-<NUM> detects a pupil area of the person on the basis of the detection result of the face area, and outputs a detection result of the pupil area to the area setting unit <NUM> and the display control unit <NUM>.

The animal detection units <NUM>-<NUM> and <NUM>-<NUM> differ in the type of animal to be detected. In a case where a detection mode for each specific part of the subject is the animal pupil detection mode, the animal detection units <NUM>-<NUM> and <NUM>-<NUM> detect a pupil area of a target animal, and output a detection result of the detected pupil area of the animal to the area setting unit <NUM> and the display control unit <NUM>.

Techniques such as deep learning are used to detect a pupil of an animal, for example. In a case where an operation for instructing a start of focusing is performed by the user, the animal detection units <NUM>-<NUM> and <NUM>-<NUM> detect a pupil of an animal according to a focus frame, and outputs a detection result of the detected pupil area of the animal to the area setting unit <NUM> and the display control unit <NUM>.

For example, the animal detection unit <NUM>-<NUM> detects a pupil area of an animal such as a dog or a cat. The animal detection unit <NUM>-<NUM> detects a pupil area of an animal such as a lizard or a frog. Another animal detection unit may be provided depending on a type of the animal having the same characteristics at the time of detection, for example, not limited to only the animal detection units <NUM>-<NUM> and <NUM>-<NUM>.

Depending on the detection mode for each specific part of the subject, the area setting unit <NUM> sets, as a focus area, either the area of the specific part of the subject detected by the subject detection unit <NUM> or the area indicated by the focus frame. The area setting unit <NUM> supplies information associated with the set focus area to the system controller <NUM>.

According to the signals corresponding to the operation input made by the user, which are supplied from the system controller <NUM>, the display control unit <NUM> generates a focus frame to cause the display unit <NUM> to superimpose and display it on the image from the memory <NUM>. The information associated with the focus frame is output to the subject detection unit <NUM>.

Furthermore, the display control unit <NUM> generates a predetermined frame (face frame, preliminary-notice frame, or pupil frame) according to the face or pupil area detected by the subject detection unit <NUM> on the basis of the signals corresponding to the operation input made by the user, which are supplied from the system controller <NUM>. The display control unit <NUM> superimposes the generated predetermined frame on the image from the memory <NUM>, and causes the display unit <NUM> to display it. Information associated with the face frame, the preliminary-notice frame, or the pupil frame is output to the subject detection unit <NUM> as necessary.

Note that the display control unit <NUM> generates an image of a graphical user interface (GUI) such as a menu, a button, or a cursor, and displays it together with a taken image, a captured image, or the like.

The codec processing unit <NUM> performs processing related to encoding and decoding of image data of moving images or still images stored in the memory <NUM>.

<FIG> is a diagram illustrating a selection method in a case where a plurality of pupils is detected in the human pupil detection mode.

A screen displayed before the user performs an operation for instructing a start of focusing is illustrated in A of <FIG>. A screen displayed after the user performs the operation for instructing a start of focusing is illustrated in B of <FIG>.

In a case where the human pupil detection mode is set, as illustrated in A of <FIG>, a face area that overlaps the focus frame F is detected among the face areas, and a face preliminary-notice frame PF indicating the detected face area is displayed.

In a case where the user performs the operation for instructing a start of focusing in this state, as illustrated in B of <FIG>, detection of a pupil area is performed on the face area, and a pupil frame AE indicating the detected pupil area is displayed. Furthermore, focusing is performed on the pupil area surrounded by the pupil frame AE.

At this time, in the detected face area, the pupil area located on the near side of the imaging device <NUM> is detected. Therefore, in a case where the left pupil is located on the near side of the imaging device <NUM>, the pupil frame AE is displayed on the left pupil even if the user sets the focus frame F on the right pupil, as illustrated on the left side in B of <FIG>. Furthermore, in a case where the right pupil is located on the near side of the imaging device <NUM>, the pupil frame AE is displayed on the right pupil even if the user sets or does not set the focus frame F on the right pupil, as illustrated on the right side in B of <FIG>.

In this manner, in the human pupil detection mode, the user simply selects a face to focus on the front pupil in the selected face area, regardless of the position of the focus frame F.

<FIG> is a diagram illustrating a selection method in a case where a plurality of pupils is detected in the animal pupil detection mode.

In a case where the animal pupil detection mode is set, as illustrated in A of <FIG>, detection of a pupil area is performed on the inside and outside of the focus frame F, and the pupil preliminary-notice frame PE that gives notice of the detected pupil area is displayed. Note that, at this time, the detection of the pupil area is performed with priority given to the inside of the focus frame F.

However, in a case where a plurality of pupil areas is detected, the pupil preliminary-notice frame PE is displayed in the pupil area located on the near side of the imaging device <NUM> and closer to the center (center position) of the focus frame F. Note that details of the selection of the pupil area will be described later with reference to <FIG>.

In a case where the user performs the operation for instructing a start of focusing, as illustrated in B of <FIG>, the pupil frame AE is displayed in the selected pupil area, and focusing is performed on the pupil area indicated by the pupil frame AE.

Note that it is also possible to set the pupil preliminary-notice frame PE not to be displayed. In this case, it becomes possible to suppress hiding of a facial expression of the subject caused by the display of the pupil preliminary-notice frame PE.

<FIG> is a diagram illustrating a relationship between the orientation of a face of a subject and the number of pupil areas.

<FIG> illustrates images P1 to P7 with a fox as a subject. The foxes appearing in the images P1 to P7 have different face orientations (angles). A rectangle of a solid line indicates a front pupil area detected in each image, and a rectangle of a broken line indicates a back pupil area detected in each image.

The image P1 shows a fox facing to the left. The image P2 shows a fox facing to the diagonally forward left. The image P3 shows a fox facing to the slightly diagonally forward left. The image P4 shows a fox facing forward. The image P5 shows a fox facing to the slightly diagonally forward right. The image P6 shows a fox facing to the diagonally forward right. The image P7 shows a fox facing to the right.

Of those images, each of the images P1, P2, P6, and P7, which shows a fox facing diagonally forward or sideways (right and left), indicates a case where only a pupil area located on the near side of the imaging device <NUM> is detected.

In each of the images P3 and P5, which shows a fox facing slightly diagonally forward, two pupil areas are detected, and it is easy to determine which one of the pupil areas is located on the near side of the imaging device <NUM> and the other pupil area is located on the far side.

In the image P4, which shows a fox facing forward, it is difficult to determine which of the left pupil area and the right pupil area is located on the near side. In this case, as described above, a preliminary-notice frame or a pupil frame is displayed in the pupil area located on the near side of the imaging device <NUM> and closer to the center of the focus frame.

<FIG> is a flowchart illustrating an imaging process of the imaging device <NUM> in the human pupil detection mode.

The imaging process in the human pupil detection mode of <FIG> starts when the power is turned on by a power button being operated, for example. A detection mode for each specific part of the subject is set in advance as a human pupil detection mode from a setting screen or the like.

In step S11 of <FIG>, the system controller <NUM> determines whether or not to terminate the process, for example, whether or not the power button has been operated.

In a case where the process is determined to be terminated in step S11, the imaging process is terminated.

In a case where the process is determined not to be terminated in step S11, the process proceeds to step S12.

In step S12, the imaging element <NUM> photoelectrically converts the light from the subject collected through the lens <NUM> and the diaphragm <NUM> in pixel units to obtain an electric signal of each pixel of the image. An image signal, which is an electric signal of each pixel of the image, is output to the memory <NUM> of the digital signal processing unit <NUM> via the analog signal processing unit <NUM> and the A/D converter <NUM>.

In step S13, the display control unit <NUM> causes the display unit <NUM> to display an image based on the image data stored in the memory <NUM> as a live view image.

In step S14, the person detection unit <NUM>-<NUM> detects a face area from the image data stored in the memory <NUM>. The person detection unit <NUM>-<NUM> supplies information associated with the detected face area to the area setting unit <NUM> and the display control unit <NUM>.

The user provides an instruction for starting focusing by performing an operation such as pressing the pupil AF button, pressing the AF-ON button, or pressing the shutter button halfway. Note that the instruction for starting focusing is provided for each image capturing unit. The operation unit <NUM> receives operation input made by the user, and outputs signals corresponding to the operation input to the system controller <NUM>.

In step S15, the system controller <NUM> determines whether or not the user has pressed the pupil AF button. In a case where the pupil AF button is determined to have been pressed in step S15, the process proceeds to step S16.

In step S16, the system controller <NUM> forcibly changes the focus frame to a wide range.

In step S17, the person detection unit <NUM>-<NUM> detects a pupil area with respect to the face area in the focus frame under the control of the system controller <NUM>. Information associated with the detected pupil area is output to the area setting unit <NUM> and the display control unit <NUM>.

In step S18, the area setting unit <NUM> determines whether or not the pupil area has been detected. In a case where the pupil area is determined to have been detected in step S18, the process proceeds to step S19.

In step S19, the area setting unit <NUM> sets the pupil area detected by the person detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is supplied to the system controller <NUM>.

In a case where the pupil area is determined not to have been detected in step S18, the process proceeds to step S20.

In step S20, the area setting unit <NUM> sets the face area detected by the person detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is output to the system controller <NUM>.

Meanwhile, in a case where the pupil AF button is determined not to have been pressed in step S15, the process proceeds to step S21.

In step S21, the system controller <NUM> determines whether or not the user has pressed the shutter button halfway or whether or not the user has pressed the AF-ON button.

In a case where it is determined in step S21 that the shutter button has been pressed halfway or the AF-ON button has been pressed, the process proceeds to step S22.

In step S22, the system controller <NUM> does not change the focus frame.

In step S23, the person detection unit <NUM>-<NUM> detects a pupil area with respect to the face area in the focus frame under the control of the system controller <NUM>. Information associated with the detected pupil area is output to the area setting unit <NUM> and the display control unit <NUM>.

In step S24, the area setting unit <NUM> determines whether or not the pupil area has been detected. In a case where the pupil area is determined to have been detected in step S24, the process proceeds to step S25.

In step S25, the area setting unit <NUM> sets the pupil area detected by the person detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is supplied to the system controller <NUM>.

In a case where the pupil area is determined not to have been detected in step S24, the process proceeds to step S26.

In step S26, the area setting unit <NUM> sets the focus frame or the face area detected by the person detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is output to the system controller <NUM>.

Furthermore, in a case where it is determined in step S20 that the shutter button has not been pressed halfway or the AF-ON button has not been pressed, the process proceeds to step S27.

In step S27, the area setting unit <NUM> sets the face area detected by the person detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is output to the system controller <NUM>.

In step S28, the display control unit <NUM> generates a face preliminary-notice frame in the face area detected by the person detection unit <NUM>-<NUM>, superimposes the face preliminary-notice frame on the live view image, and causes the display unit <NUM> to display it.

Subsequent to steps S19, S20, and S25, the process proceeds to step S29 of <FIG>.

In step S29, the system controller <NUM> controls the lens driver <NUM> to drive an optical system such as the lens <NUM> and the diaphragm <NUM> in such a manner that the focus area is in focus. Thereafter, the process proceeds to step S30.

The process also proceeds to step S30 subsequent to step S27 of <FIG>.

In step S30, the system controller <NUM> determines whether or not the focusing has been complete.

In a case where the focusing is determined to have been complete in step S30, the process proceeds to step S31.

In step S31, the display control unit <NUM> causes the display unit <NUM> to display an image based on the image data stored in the memory <NUM> as a live view image. Furthermore, the display control unit <NUM> superimposes an in-focus frame (pupil frame, face frame, or focus frame), which is a frame surrounding the set focus area, on the live view image, and causes the display unit <NUM> to display it.

In a case where the focusing is determined not to have been complete in step S30, the process skips step S31 and proceeds to step S32.

Furthermore, the process proceeds to step S32 subsequent to step S29 of <FIG>.

In step S32, the system controller <NUM> determines whether or not the shutter button has been fully pressed on the basis of the signals corresponding to the operation input from the operation unit <NUM>. In a case where the shutter button is determined to have been fully pressed in step S32, the process proceeds to step S33.

In step S33, the imaging element <NUM> photoelectrically converts the light from the subject collected through the optical system such as the lens <NUM> and the diaphragm <NUM> in pixel units to obtain an electric signal of each pixel of the image. An image signal, which is an electric signal of each pixel of the image, is output to the memory <NUM> of the digital signal processing unit <NUM> via the analog signal processing unit <NUM> and the A/D converter <NUM>.

In step S34, the display control unit <NUM> causes the display unit <NUM> to display an image based on the image data stored in the memory <NUM> as a captured image.

In step S35, the codec processing unit <NUM> encodes the image data stored in the memory <NUM>. The codec processing unit <NUM> supplies the encoded image data to the storage <NUM>.

In step S36, the codec processing unit <NUM> causes the storage <NUM> to record the encoded image data. Thereafter, the process returns to step S11, and the subsequent processing is repeated.

Furthermore, also in a case where the shutter button is determined not to have been fully pressed in step S32, the process returns to step S11, and the subsequent processing is repeated.

<FIG> is a flowchart illustrating an imaging process of the imaging device <NUM> in the animal pupil detection mode.

The imaging process in the animal pupil detection mode of <FIG> starts when the power is turned on by the power button being operated, for example. An animal pupil detection mode is set in advance as a detection mode for each specific part of the subject from a setting screen or the like. <FIG> illustrates an exemplary case where the animal detection unit <NUM>-<NUM> that detects a pupil of a cat or a dog detects a pupil of an animal.

In step S111 of <FIG>, the system controller <NUM> determines whether or not to terminate the process, for example, whether or not the power button has been operated.

In a case where the process is determined to be terminated in step S111, the imaging process is terminated.

In a case where the process is determined not to be terminated in step S111, the process proceeds to step S112.

In step S112, the imaging element <NUM> photoelectrically converts the light from the subject collected through the lens <NUM> and the diaphragm <NUM> in pixel units to obtain an electric signal of each pixel of the image. An image signal, which is an electric signal of each pixel of the image, is output to the memory <NUM> of the digital signal processing unit <NUM> via the analog signal processing unit <NUM> and the A/D converter <NUM>.

In step S113, the display control unit <NUM> causes the display unit <NUM> to display an image based on the image data stored in the memory <NUM> as a live view image.

In step S114, the animal detection unit <NUM>-<NUM> detects a pupil area from the image data stored in the memory <NUM>. The animal detection unit <NUM>-<NUM> supplies information associated with the detected pupil area to the area setting unit <NUM> and the display control unit <NUM>.

In step S115, the display control unit <NUM> performs processing of selecting the pupil area detected by the animal detection unit <NUM>-<NUM>. According to the processing of selecting the pupil area, a pupil area in which a preliminary-notice frame is displayed is selected from a plurality of the detected pupil areas.

In step S116, the system controller <NUM> determines whether or not the user has pressed the pupil AF button.

In a case where the pupil AF button is determined to have been pressed in step S116, the process proceeds to step S117.

In step S117, the system controller <NUM> forcibly changes the focus frame to a wide range.

In step S118, the animal detection unit <NUM>-<NUM> detects a pupil area within the focus frame changed to the wide range under the control of the system controller <NUM>. Information associated with the detected pupil area is output to the area setting unit <NUM> and the display control unit <NUM>.

In step S119, the area setting unit <NUM> determines whether or not the pupil area has been detected. In a case where the pupil area is determined to have been detected in step S119, the process proceeds to step S120.

In step S120, the area setting unit <NUM> sets the pupil area detected by the animal detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is supplied to the system controller <NUM>.

Meanwhile, in a case where the pupil AF button is determined not to have been pressed in step S116, the process proceeds to step S121.

In step S121, the system controller <NUM> determines whether or not the user has pressed the shutter button halfway or whether or not the user has pressed the AF-ON button.

In a case where it is determined in step S121 that the shutter button has been pressed halfway or the AF-ON button has been pressed, the process proceeds to step S122.

In step S122, the system controller <NUM> does not change the focus frame.

In step S123, the animal detection unit <NUM>-<NUM> detects a pupil area within the focus frame under the control of the system controller <NUM>. Information associated with the detected pupil area is output to the area setting unit <NUM> and the display control unit <NUM>.

In step S124, the area setting unit <NUM> determines whether or not the pupil area has been detected. In a case where the pupil area is determined to have been detected in step S124, the process proceeds to step S125.

In step S125, the area setting unit <NUM> sets the pupil area detected by the animal detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is supplied to the system controller <NUM>.

In a case where the pupil area is determined not to have been detected in step S125, the process proceeds to step S126.

In step S126, the area setting unit <NUM> sets the focus frame as a focus area as another condition. Information associated with the set focus area is output to the system controller <NUM>.

Furthermore, in a case where it is determined in step S121 that the shutter button has not been pressed halfway or the AF-ON button has not been pressed, the process proceeds to step S127.

In step S127, the area setting unit <NUM> determines whether or not the pupil area has been detected. In a case where the pupil area is determined to have been detected in step S127, the process proceeds to step S128.

In step S128, the area setting unit <NUM> sets the pupil area detected by the animal detection unit <NUM>-<NUM> as a focus area. Information associated with the set focus area is supplied to the system controller <NUM>.

In step S129, the display control unit <NUM> generates a pupil preliminary-notice frame in the pupil area detected by the animal detection unit <NUM>-<NUM>, superimposes the pupil preliminary-notice frame on the live view image, and causes the display unit <NUM> to display it.

Subsequent to steps S120 and S125, the process proceeds to step S130 of <FIG>.

In step S130, the system controller <NUM> controls the lens driver <NUM> to drive the optical system such as the lens <NUM> and the diaphragm <NUM> in such a manner that the focus area is in focus. Thereafter, the process proceeds to step S131.

The process also proceeds to step S131 subsequent to step S126 of <FIG>.

In step S131, the system controller <NUM> determines whether or not the focusing has been complete.

In a case where the focusing is determined to have been complete in step S131, the process proceeds to step S132.

In step S132, the display control unit <NUM> causes the display unit <NUM> to display an image based on the image data stored in the memory <NUM> as a live view image. Furthermore, the display control unit <NUM> superimposes an in-focus frame (pupil frame or focus frame), which is a frame surrounding the set focus area, on the live view image, and causes the display unit <NUM> to display it.

In a case where the focusing is determined not to have been complete in step S131, the process skips step S132 and proceeds to step S133.

Furthermore, in a case where the pupil area is determined not to have been detected in step S119 or S127 of <FIG>, the focusing is not performed, and the process proceeds to step S133 of <FIG>. The process also proceeds to step S133 subsequent to step S129.

In step S133, the system controller <NUM> determines whether or not the shutter button has been fully pressed on the basis of the signals corresponding to the operation input from the operation unit <NUM>. In a case where the shutter button is determined to have been fully pressed in step S133, the process proceeds to step S134.

In step S134, the imaging element <NUM> photoelectrically converts the light from the subject collected through the optical system such as the lens <NUM> and the diaphragm <NUM> in pixel units to obtain an electric signal of each pixel of the image. An image signal, which is an electric signal of each pixel of the image, is output to the memory <NUM> of the digital signal processing unit <NUM> via the analog signal processing unit <NUM> and the A/D converter <NUM>.

In step S135, the display control unit <NUM> causes the display unit <NUM> to display an image based on the image data stored in the memory <NUM> as a captured image.

In step S136, the codec processing unit <NUM> encodes the image data stored in the memory <NUM>. The codec processing unit <NUM> supplies the encoded image data to the storage <NUM>.

In step S137, the codec processing unit <NUM> causes the storage <NUM> to record the encoded image data. Thereafter, the process returns to step S111, and the subsequent processing is repeated.

Furthermore, also in a case where the shutter button is determined not to have been fully pressed in step S133, the process returns to step S111, and the subsequent processing is repeated.

Note that, in the process described above, descriptions have been given such that the focusing processing is not executed or the focusing processing is executed under other conditions such as the focus frame in a case where the pupil area has not been detected. The operation in a case where the pupil area has not been detected may be performed properly according to the setting made by the user or the like. At that time, the operation in the case of not having been detected may be changed depending on the target subject or specific part that has not been detected.

For example, by performing the setting such that the focusing processing is not carried out in the case of not having been detected, it becomes possible to suppress further deterioration of the focusing precision in a case where motion of the target subject or specific part is small. On the other hand, by performing the setting such that the focusing processing is carried out at the position instructed by the user in the case of not having been detected, it becomes possible to perform preliminary focusing processing in a case where motion of the target subject or specific part is large.

<FIG> is a flowchart illustrating the pupil area selection process in step S114 of <FIG>.

In step S151, the display control unit <NUM> determines whether or not there are two or more pupil areas in the focus frame. For example, in a case where two animals are present at a position on the near side of the imaging device <NUM>, pupils of the number of any of one to four may be detected.

In a case where it is determined in step S151 that there are two or more pupil areas in the focus frame, the process proceeds to step S52.

In step S152, the display control unit <NUM> determines whether or not there are two or more front pupil areas.

In a case where it is determined in step S152 that there are two or more front pupil areas, the process proceeds to step S153.

In step S153, the display control unit <NUM> calculates a distance between each pupil area and the center of the focus frame.

In step S154, the display control unit <NUM> selects the pupil area having the shortest distance from the center of the focus frame.

On the other hand, in a case where it is determined in step S151 that there are not two or more detected pupil areas in the focus frame, the process proceeds to step S155.

In step S155, the display control unit <NUM> selects one pupil area.

Furthermore, in a case where it is determined in step S152 that there are not two or more front pupil areas, the process proceeds to step S156.

In step S156, the display control unit <NUM> selects the front pupil area.

Subsequent to steps S154 to S156, the pupil area selection process is terminated, and the process returns to step S114 of <FIG>.

As described above, according to the present technology, a preliminary-notice frame that gives notice of a specific area to be focused is displayed on the image obtained by the imaging unit according to a type of the subject.

For example, in a case where the preliminary-notice frame is not displayed, the user performs an operation for instructing a start of focusing to carry out automatic focusing and imaging, which may lead to a result unintended by the user depending on conditions of the subject.

According to the present technology, with the preliminary-notice frame displayed, it is known in advance that the intended position or an unintended position has been detected, whereby the user is enabled to select not performing automatic focusing.

As described above, the user can easily focus on a specific part such as a pupil area or an animal itself according to the animal that is a type of the subject.

Note that, while the process of detecting a pupil of an animal, such as a dog and cat, has been described in the descriptions above, the present technology can be applied to a specific part of a subject, such as a pupil, face, part of a face, neck, and head of all forms of life, such as birds, fish, reptiles, and amphibians, or the entire body (subject). Furthermore, the present technology can also be applied to combinations of those specific parts of the subject or the subjects.

Moreover, the present technology can be applied not only to a living being but also to a specific part of a subject such as a headlight, front emblem, front window, or driver seat of a vehicle, or a headlight or helmet of a motorcycle.

In those cases, a detection mode for detecting a specific part of the subject is preset and used. With this arrangement, it becomes possible to give, to the imaging device, the user's intention such as which detection result or detection method is prioritized among a plurality of detection results or detection methods, or which subject is prioritized among a plurality of subjects.

Note that, unlike the case of a person, in the case of an animal having long hair, hair on a pupil may be focused instead of the pupil in an attempt to focus on the pupil. In this case, the position of the focal point is adjusted to the rear, or the imaging device is set in advance that the subject is likely to be in focus on the hair and the imaging device performs control on the basis of the setting, whereby an imaging result more suitable for the user's intention can be obtained.

However, even in the case of a subject having short hair, performing the same control may result in an out-of-focus state. Therefore, it becomes possible to further improve the convenience by adjusting what type of adjustment is to be reflected using a subject or specific part to be detected and other detection results.

Note that, while a pupil of a person and a pupil of an animal are detected by different detection processes in the descriptions above, a pupil of a person may also be detected by the detection process same as a pupil of an animal.

Furthermore, the series of processing described above can be executed by hardware or by software. In a case where the series of processing described above is executed by software, a program constituting the software is installed from a network or a recording medium.

As illustrated in <FIG>, for example, the recording medium includes the removable recording medium <NUM> on which the program is recorded, which is distributed to deliver the program to the user separately from the device main body. The removable recording medium <NUM> includes a magnetic disk (including a flexible disk) and an optical disk (including a CD-ROM and a DVD). Moreover, a magneto-optical disc (including a mini disc (MD)), a semiconductor memory, and the like are also included.

In this case, the program may be installed in the storage <NUM> by the removable recording medium <NUM> mounted in the drive <NUM>.

Furthermore, the program may also be provided through a wired or wireless transmission medium such as a local area network, the Internet, and digital satellite broadcasting. In this case, the program may be received via the communication unit <NUM>, and may be installed in the storage <NUM>.

In addition, the program may be installed in advance in the storage <NUM>, a read only memory (ROM) in the system controller <NUM>, or the like.

Note that the program to be executed by the computer may be a program in which processing is executed in a time-series manner according to the order described in the present specification, or may be a program in which processing is executed in parallel or at a necessary timing such as a calling is performed.

Note that, in the present specification, a system indicates a set of a plurality of constituent elements (devices, modules (parts), etc.), and it does not matter whether or not all the constituent elements are in the same housing. Therefore, a plurality of devices housed in separate housings and connected through a network, and one device in which a plurality of modules is housed in one housing are both systems.

An embodiment of the present technology is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the invention as defined by the appended claims.

For example, the present technology may employ a configuration of cloud computing in which one function is shared and jointly processed by a plurality of devices via a network.

Furthermore, each step described in the flowcharts described above may be executed by one device or shared by a plurality of devices.

Moreover, in a case where a plurality of processes is included in one step, the plurality of processes included in the one step may be executed by one device or shared by a plurality of devices.

Claim 1:
An imaging device (<NUM>) comprising:
a display control unit (<NUM>) configured to cause a preliminary-notice frame (PE) that gives notice of a specific area to be focused to be displayed on an image obtained by an imaging unit according to a specific part of a subject,
wherein:
the subject is an animal being a dog or cat, and
the specific part is a pupil and the specific area is a pupil area,
the imaging device (<NUM>) further comprising:
an area detection unit configured to detect, in a captured image, at least one front specific area including the specific area to be focused, wherein the at least one front specific area indicates the pupil area of the animal located on a near side of the imaging device (<NUM>), wherein
in a case where a plurality of front specific areas has been detected, the display control unit (<NUM>) is configured to control display of the preliminary-notice frame (PE) according to the front specific area being closer to a center position of a focus setting frame for setting a focus area.