IMAGING DEVICE, CONTROL METHOD FOR CONTROLLING IMAGING DEVICE, AND STORAGE MEDIUM

An imaging device includes one or more memories, and one or more processors in communication with the one or more memories. The one or more processors and the one or more memories are configured to capture an image, display a live view image obtained by capturing the image on a display unit, and in a case where a piece of camerawork used to capture a moving image is selected from among a plurality of pieces of camerawork indicating imaging methods for capturing a moving image, then based on attribute information regarding the selected piece of camerawork, perform control to display on the display unit a guide that instructs a user to make a motion according to the selected piece of camerawork.

BACKGROUND

Field

The present disclosure relates to an imaging device, a control method for controlling an imaging device, and a storage medium.

Description of the Related Art

In recent years, more and more people have captured moving images using cameras, such as digital still cameras or digital video camera. Capturing moving images with appropriate camerawork involves many experiences and techniques to achieve a video expression intended by a user. Japanese Patent Application Laid-Open No. 2013-128251 discusses an imaging device that provides advice regarding setting values of a camera to a user of the camera for the user to capture better still images.

For example, beginners who newly purchase cameras do not understand what video expression techniques are available. Even if there is a video expression intended by a beginner, the beginner does not know how to use the camera to capture an image. Thus, it is difficult for the beginner to perform moving image capturing with appropriate camerawork. In Japanese Patent Application Laid-Open No. 2013-128251, even if advice for still image capturing can be provided, appropriate information regarding camerawork for moving image capturing in which it is necessary to change the relationship between the camera and the user of the camera and an object, as appropriate, cannot be provided to the user of the camera, particularly beginners.

SUMMARY

The present disclosure is directed to providing an imaging device that assists a user of a camera in performing moving image capturing with appropriate camerawork, and a control method for controlling an imaging device.

According to an aspect of the present disclosure, an imaging device includes one or more memories, and one or more processors in communication with the one or more memories. The one or more processors and the one or more memories are configured to capture an image, display a live view image obtained by capturing the image on a display unit, and in a case where a piece of camerawork used to capture a moving image is selected from among a plurality of pieces of camerawork indicating imaging methods for capturing a moving image, then based on attribute information regarding the selected piece of camerawork, perform control to display on the display unit a guide that instructs a user to make a motion according to the selected piece of camerawork.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments will be described with reference to the attached drawings. The following exemplary embodiments do not limit the disclosure according to the appended claims. In the specification of the present disclosure, an “image” is a concept including not only a “still image” but also a “moving image”.

A first exemplary embodiment will be described.FIG.1illustrates the hardware configuration of a digital camera. A digital camera100illustrated inFIG.1is an example of an imaging device. Hereinafter, the digital camera100will occasionally be referred to simply as the “camera”. The digital camera100includes a central processing unit (CPU)101, a primary storage unit102, a communication control unit103, an input unit104, a display unit105, an imaging unit106, a secondary storage unit107, a camera control unit108, and a lens control unit109. The CPU101is connected to these components via a bus110. In some embodiments, the digital camera100is or includes a mobile device, such as a smartphone having a camera function or a tablet.

The CPU101controls the entirety of the digital camera100. The processing of flowcharts described below is achieved by the CPU101reading a program stored in the secondary storage unit107, loading the program into a work area formed in the primary storage unit102, and operating according to the loaded program. The primary storage unit102forms a work area that temporarily stores various programs run by the CPU101and data related to these programs.

The communication control unit103is connected to an external electronic device (not illustrated), such as a personal computer (PC), using a communication cable, and controls the transmission and reception of data on an image. In other embodiments, the connection to the external electronic device is a wireless connection using infrared light or Wi-Fi, instead of a wired connection using a communication cable. In other embodiment, the communication control unit103is also connected to a mobile device, such as a smartphone. In yet other embodiments, the communication control unit103is also connected to a network using a communication cable and control the transmission and reception of data to and from a cloud server on a network.

The input unit104includes a power key, a shutter key, a movement key for giving an instruction to move the position of a cursor on a display screen of the display unit105, and a determination key for giving an instruction to set the content selected on the display screen. The input unit104also includes a mode change switch (not illustrated) for switching the operation mode of the digital camera100, such as a normal imaging mode, an imaging assistance mode, an end mode, and a reproduction mode. Then, the input unit104outputs operation signals according to operations on the keys and an operation on the switch by a user. Further, the input unit104and the display unit105each can include a touch panel display having an input function and a display function. In this case, the keys and the switch corresponding to the various operations can be displayed on the touch panel display. The input unit104can output an operation signal according to a touch operation on the touch panel display to the CPU101.

The display unit105includes a monitor (a display screen), such as a liquid crystal display (LCD). According to an instruction of a display signal input from the CPU101, the display unit105displays a live view image captured by the imaging unit106or an image stored in the secondary storage unit107on the display screen.

The imaging unit106includes an imaging element that receives object light incident through an optical lens device, converts the object light into an imaging signal, and outputs the imaging signal, and an analog-to-digital (A/D) conversion unit that converts the imaging signal output from the imaging element into image data and outputs the image data. The imaging unit106writes the image data output from the A/D conversion unit directly or indirectly to the secondary storage unit107. The imaging unit106is an example of an imaging unit.

The secondary storage unit107stores various programs, camerawork information, and images captured by the imaging unit106. The camerawork information will be described in detail below.

The camera control unit108causes a camera main body to operate. Specifically, the camera control unit108causes a driving mechanism for moving the camera main body in at least one of a total of six axis directions, namely x, y, and z axis directions in a three-dimensional space and rotational directions about the x, y, and z axes, to operate according to a control signal input from the CPU101, changing the position and orientation of the camera main body. The camera control unit108includes a driving device, such as a motor and a driving circuit. The camera control unit108also includes an acceleration sensor and an angular velocity sensor for detecting the position and orientation of the camera main body.

The lens control unit109causes a lens to operate. Specifically, the lens control unit109causes the optical lens device of the imaging unit106to operate according to a control signal input from the CPU101, changing the zoom, the focus, or the aperture of the lens. The lens control unit109includes a driving device, such as a motor and a driving circuit.

The functional configuration of the digital camera100according to the present exemplary embodiment will now be described with reference toFIG.2.

The digital camera100functions as functional units illustrated inFIG.2by the CPU101running programs stored in the secondary storage unit107. A description is given of the functions of a camerawork information holding unit201, an object recognition unit202, a matching determination unit203, a camerawork notification unit204, and a camerawork selection unit205included in the digital camera100.

The camerawork information holding unit201holds one or more pieces of camerawork information in association with pieces of information regarding a plurality of objects. An example is illustrated where the type of each object is associated with pieces of camerawork information suitable for the type of the object. Examples of the type of the object include the entire body of a person, the face of a person, a scenery, and an animal. In some embodiments, the size and the position of each object and movement information regarding the object (whether the object is a static object or a dynamic object and the moving velocity and the moving direction of the object) are associated with pieces of camerawork information suitable for the size, the position, and the movement information.

FIG.3illustrates the configuration of data held in the camerawork information holding unit201according to the present exemplary embodiment. As illustrated inFIG.3, the camerawork information holding unit201holds a data table illustrating the association of pieces of camerawork with the type of each object. Examples of the pieces of camerawork include follow, dolly in, dolly out, circle, fix, pan, tilt, zoom in, zoom out, rack focus (in), rack focus (out), and tilt follow.

Follow, dolly in, dolly out, and circle is each an imaging method in which the user performs moving image capturing while moving the camera main body by holding the camera main body. Follow is an imaging method in which the user performs moving image capturing while following the motion of the object. Dolly in and dolly out are each an imaging method in which the user performs moving image capturing while moving close to or away from the object. Circle is an imaging method in which the user performs moving image capturing while moving by drawing a circle about the object. Fix is an imaging method for performing moving image capturing with the camera main body fixed. Pan is an imaging method for performing moving image capturing while moving the camera main body in the horizontal directions. Tilt is an imaging method for performing moving image capturing while moving the camera main body in the vertical directions. Zoom in and zoom out are each an imaging method for performing moving image capturing while causing a zoom function of the imaging unit106to operate. Rack focus (in) and rack focus (out) are each an imaging method for performing moving image capturing while causing a focus function of the imaging unit106to operate. Tilt follow is an imaging method for performing moving image capturing while simultaneously moving the user and moving the camera main body. The types of the pieces of camerawork are not limited to the above.

FIG.4illustrates an example of the configuration of camerawork information regarding each of the pieces of camerawork. The camerawork information is created in advance according to a template. The camerawork information includes attribute information to perform moving image capturing with the piece of camerawork, such as information regarding how the user moves the camera main body, the path of the camera main body, and the motion of the camera itself. Specifically, the camerawork information includes the information in formats, such as characters, a schematic diagram or an image representing a composition during moving image capturing, and numerical values (the moving velocity, the moving direction, the moving time, and the moving angle of the camera main body and the setting values of the camera). In some embodiment, the user can appropriately change the various numerical values included in the camerawork information by a touch operation or a key operation through the input unit104. In these cases, the camerawork information can further include an example of a moving image captured with the piece of camerawork (an example moving image) and a moving image obtained by recording a method for capturing an image with the piece of camerawork from a third person's point of view (an imaging point moving image).FIG.4illustrates an example of the camerawork information regarding follow and the camerawork information regarding pan.

The information held in the camerawork information holding unit201is stored in the secondary storage unit107. In other embodiments, a part of the information can be stored in an external storage device connected to the digital camera100via the communication control unit103. For example, the example moving image or the moving image regarding the method for capturing an image can be stored in the external storage device or the cloud server.

First, the object recognition unit202acquires a group of images captured by the imaging unit106until a predetermined time before the current moment as currently captured moving image data from the secondary storage unit107. Then, the object recognition unit202analyzes the acquired moving image data and recognizes an object appearing in the moving image data. Specifically, the object recognition unit202acquires the type, the size, and the position of the object and the movement information regarding the object using information including the color distribution and the exposure distribution in an image obtained from the moving image data and information regarding changes over time in the color distribution and the exposure distribution. The movement information is information regarding whether the object is a static object or a dynamic object, and information regarding the moving velocity and the moving direction of the object if the object is a dynamic object. Examples of the type of the object recognized by the object recognition unit202include a person, the entire body of a person, the face of a person, the front surface of a person, the back surface of a person, a pupil, an animal, a scenery (without a person), and a building. If the object recognition unit202recognizes a plurality of objects from the moving image data, the object recognition unit202can acquire information regarding each of the plurality of objects. Further, the object recognition unit202can identify a main object from the plurality of objects and acquire information regarding the main object alone.

The matching determination unit203compares the information regarding the object acquired by the object recognition unit202and the pieces of information regarding the objects associated with the pieces of camerawork information held in the camerawork information holding unit201and determines whether there are pieces of camerawork information in which the information regarding the object acquired by the object recognition unit202and the pieces of information regarding the objects associated with the pieces of camerawork information match each other. If there are pieces of camerawork information in which the information regarding the object acquired by the object recognition unit202and the pieces of information regarding the objects associated with the pieces of camerawork information match each other, the matching determination unit203determines at least one of the pieces of camerawork information associated with the information regarding the same object to be the information regarding the object acquired by the object recognition unit202as recommended camerawork. If there are no pieces of camerawork information in which the information regarding the object acquired by the object recognition unit202and the pieces of information regarding the objects associated with the pieces of camerawork information match each other, the matching determination unit203determines at least one piece of camerawork information determined in advance as recommended camerawork. In the example ofFIG.3, since the pieces of camerawork information are associated with the type of each object, recommended camerawork is determined using the type of the object. However, recommended camerawork can be determined using the size and the position of the object and the movement information regarding the object.

The camerawork notification unit204displays a notification based on the recommended camerawork determined by the matching determination unit203on the display unit105. Instead of displaying the notification on the display unit105of the digital camera100, the camerawork notification unit204can communicate with an external electronic device via the communication control unit103and display the notification on, for example, a display screen of a PC or a mobile terminal. Furthermore, the camerawork notification unit204can give the notification by another method, such as using a sound output unit (not illustrated), instead of the method for displaying the notification.

With reference toFIG.3, a specific description is given of the operation of the camerawork notification unit204when the object recognition unit202recognizes the entire body of a person. In this case, the matching determination unit203determines follow, dolly in, circle, tilt, zoom in, zoom out, and tilt follow associated with a person (the entire body) among the plurality of pieces of camerawork held in the camerawork information holding unit201as pieces of recommended camerawork. Then, the camerawork notification unit204gives a notification based on the determined pieces of recommended camerawork. If there is a plurality of pieces of camerawork information associated with the same type of object as the type of the object acquired by the object recognition unit202, the camerawork notification unit204can give a notification by narrowing down the plurality of pieces of camerawork information to some of the plurality of pieces of camerawork information. In this case, the camerawork notification unit204can give a notification based on pieces of camerawork information set in advance by the user. Alternatively, the camerawork notification unit204can give a notification based on a predetermined number of pieces of camerawork information having high priorities.

FIG.5illustrates an example of a recommended camerawork notification screen. A recommended camerawork notification screen300is displayed on the display unit105. The recommended camerawork notification screen300inFIG.5displays a recommended camerawork notification301regarding four pieces of recommended camerawork, namely circle, dolly in, follow, and pan. The camerawork notification unit204acquires the pieces of camerawork information regarding the pieces of recommended camerawork from the camerawork information holding unit201and displays the recommended camerawork notification301using the acquired pieces of camerawork information. In the example ofFIG.5, the camerawork notification unit204displays the recommended camerawork notification301by superimposing an arrow and description text on a thumbnail image of a schematic diagram or an example moving image included in each of the pieces of camerawork information regarding circle, dolly in, follow, and pan. Each arrow illustrated inFIG.5schematically represents the user of the camera or the motion of the camera itself during moving image capturing.

The recommended camerawork notification screen300also displays a camerawork selection area302. In the camerawork selection area302, a user interface (UI) is provided for selecting a single piece of recommended camerawork from among the pieces of recommended camerawork of which the notification is given. If a piece of recommended camerawork is selected in the camerawork selection area302by an operation of the user through the input unit104, the camerawork notification unit204displays a detailed description of the selected piece of recommended camerawork on the display unit105.

FIG.6illustrates an example of a recommended camerawork details screen for the currently selected piece of recommended camerawork. A recommended camerawork details screen400is a screen displaying the detailed description of the piece of recommended camerawork currently selected inFIG.5and is displayed on the display unit105. The recommended camerawork details screen400displays an example moving image401, an overview description402, and points of imaging403as description information. Information displayed as the description information is included in the pieces of camerawork information and is held in the camerawork information holding unit201as described above. The recommended camerawork details screen400also displays a camerawork selection key404. If the camerawork selection key404is selected by an operation of the user through the input unit104, the camerawork selection unit205selects the currently selected piece of recommended camerawork as camerawork for use in moving image capturing.

A description will now be given of the functions of a camerawork setting unit206, a camera instruction unit207, a user instruction unit208, an image capturing unit209, and an image recording unit210included in the digital camera100.

The camerawork setting unit206acquires the camerawork information regarding the camerawork selected by the camerawork selection unit205from the camerawork information holding unit201and provides the camerawork information to the camera instruction unit207or the user instruction unit208. Further, the camerawork setting unit206instructs the camera instruction unit207or the user instruction unit208to start controlling the operation of the camerawork selected by the camerawork selection unit205.

For example, if the selected camerawork is pan, the camerawork setting unit206notifies the camera instruction unit207of the angle and the time of panning. Information regarding the angle and the time of panning is included in the camerawork information regarding pan held in the camerawork information holding unit201. In some embodiments, the user can appropriately change the angle and the time of panning by a touch operation or a key operation through the input unit104.

If the selected camerawork is follow, the camerawork setting unit206notifies the user instruction unit208of the information regarding the object acquired by the object recognition unit202and the camerawork information regarding follow held in the camerawork information holding unit201. The information regarding the object of which the user instruction unit208is notified includes the size and the position of the object and the movement information regarding the object.

If the camera instruction unit207is instructed to start controlling the operation of the camerawork, the camera instruction unit207instructs the camera control unit108or the lens control unit109to start operating. In the example of pan, the camera instruction unit207outputs a control signal based on the angle and the time of panning of which the camera instruction unit207is notified by the camerawork setting unit206to the camera control unit108. The camera control unit108performs a panning operation on the camera according to the control signal from the camera instruction unit207. The start of the operation of the camera main body or the lens and the start of the control of the operation of the camerawork can be performed at different timings.

If the user instruction unit208is instructed to start controlling the operation of the camerawork, the user instruction unit208performs control to display on the display unit105a guide that instructs the user to move or perform a camera operation.

FIG.7illustrates an example of a display screen that displays a guide with follow as the selected camerawork. The image capturing unit209displays an image captured by the imaging unit106in real time on the display unit105, whereby the display unit105displays a live view image. A live view image500inFIG.7represents a scene where a person is walking away from the user of the camera along the shore in the evening. Based on the information provided by the camerawork setting unit206, the user instruction unit208displays a guide that instructs the user to move according to the selected camerawork in a superimposed manner on the live view image500.

Specifically, the user instruction unit208displays on the live view image500a guide frame502that guides the position of a main object501(the entire body of the person) in the live view image500.

The guide frame502indicates to the user where in the screen and in what size the main object501should be maintained during moving image capturing with follow. The position and the size of the guide frame502can be set in advance. In other embodiments, the user instruction unit208can automatically set the position and the size of the guide frame502according to the size and the position of the object in the live view image500before or when the control of the operation of the selected camerawork starts.

A message503and an arrow504indicate details, such as how to move while holding the camera during the moving image capturing and a note of caution in the movement, to the user. The size and the direction of the arrow504can be set in advance. In other embodiments, the user instruction unit208can automatically set the size and the direction of the arrow504according to the moving velocity and the moving direction of the object in the live view image500before or when the control of the operation of the camerawork starts. The guide frame502, the message503, and the arrow504are examples of the guide. The user captures the object while moving according to the guide, and then can perform appropriate moving image capturing with the selected camerawork. The user instruction unit208also has the function of updating the guide based on the information regarding the object acquired by the object recognition unit202during the control of the operation of the camerawork. The display unit105also displays a start video recording key505with the live view image500.

If the start video recording key505is selected by an operation of the user through the input unit104, the image recording unit210receives an instruction to start video recording.

The image capturing unit209captures the object using the imaging unit106.

If the image recording unit210receives an instruction to start video recording, the image recording unit210records moving image data captured by the image capturing unit209from the reception of the instruction to start video recording to the end of video recording as a moving image file in the secondary storage unit107.

FIG.8is a flowchart illustrating main processing of moving image capturing. The processing of this flowchart is achieved by the CPU101loading a program stored in the secondary storage unit107into the primary storage unit102and running the program.

First, in step S601, the CPU101analyzes the current operation mode. The current operation mode can be switched by operating the mode change switch (not illustrated) of the input unit104.

In step S602, the CPU101determines whether the current operation mode is the normal imaging mode. If it is determined that the current operation mode is the normal imaging mode (Yes in step S602), the processing proceeds to step S603. The normal imaging mode is an imaging mode for performing imaging by processing set in advance as standard in the digital camera100. If, on the other hand, it is determined that the current operation mode is not the normal imaging mode (No in step S602), the processing proceeds to step S604.

In step S603, the CPU101executes processing in the normal imaging mode. Then, the processing returns to step S601.

In step S604, the CPU101determines whether the current operation mode is the imaging assistance mode. If it is determined that the current operation mode is the imaging assistance mode (Yes in step S604), the processing proceeds to step S605. If, on the other hand, it is determined that the current operation mode is not the imaging assistance mode (No in step S604), the processing proceeds to step S606.

In step S605, the CPU101executes the processing flow of the imaging assistance mode. The processing flow of the imaging assistance mode will be described in detail with reference toFIG.9. Then, the processing returns to step S601.

In step S606, the CPU101determines whether the current operation mode is the end mode. If it is determined that the current operation mode is not the end mode (No in step S606), the processing proceeds to step S607. If, on the other hand, it is determined that the current operation mode is the end mode (Yes in step S606), the processing of the flowchart inFIG.8ends.

In step S607, the CPU101executes processing according to the current operation mode. Then, the processing returns to step S601.

For example, if the current operation mode is the reproduction mode, the CPU101displays a list of images recorded in the secondary storage unit107on the display screen of the display unit105and displays an image selected from the list of images.

FIG.9illustrates the processing flow of the imaging assistance mode according to the present exemplary embodiment. The processing in the imaging assistance mode is the process executed in step S605inFIG.8. During the execution of the processing in the imaging assistance mode, the image capturing unit209performs imaging, and a group of images captured by the image capturing unit209until a predetermined time before the current moment is stored in the secondary storage unit107as currently captured moving image data. The currently captured moving image data is sequentially updated. The display unit105displays a live view image.

In step S701, the object recognition unit202analyzes the currently captured moving image data and acquires the type of an object appearing in the moving image data and information regarding whether the object is a dynamic object or a static object.

In step S702, the matching determination unit203compares the information regarding the object acquired in step S701and the pieces of information regarding the objects associated with the pieces of camerawork information held in the camerawork information holding unit201and determines whether there are pieces of camerawork information that match the information acquired in step S701. If it is determined that there are pieces of camerawork information that match the information acquired in step S701(Yes in step S702), the matching determination unit203determines the pieces of camerawork information that match the information acquired in step S701as pieces of recommended camerawork. Then, the processing proceeds to step S703. If, on the other hand, it is determined that there are no pieces of camerawork information that match the information acquired in step S701(No in step S702), the processing proceeds to step S704.

In step S703, the camerawork notification unit204gives a notification based on the determined pieces of recommended camerawork. For example, the camerawork notification unit204displays the recommended camerawork notification screen300(FIG.5) on the display unit105.

In step S704, the camerawork notification unit204gives a notification based on the pieces of camerawork information regarding pieces of camerawork determined in advance (pieces of camerawork corresponding to “except for above” inFIG.3).

In step S705, if the user operates the input unit104, specifying the user's favorite camerawork from among the pieces of camerawork based on which the notification is given, the camerawork selection unit205selects the specified camerawork. The camerawork selection unit205acquires the camerawork information regarding the selected camerawork from the camerawork information holding unit201and notifies the camerawork setting unit206of the camerawork information.

In step S706, the camerawork setting unit206starts controlling the operation of the camerawork based on the camerawork information regarding the camerawork selected in step S705. Thus, moving image capturing according to the selected camerawork starts. In step S707, the camerawork setting unit206determines whether the camerawork selected in step S705is camerawork that involves the movement of the user or camerawork that can be handled by controlling the operation of the camera. In the present exemplary embodiment, if the camerawork selected in step S705is camerawork that can be handled under control of the camera control unit108and the lens control unit109, it is determined that the camerawork selected in step S705is camerawork that can be handled by controlling the operation of the camera. For example, the camerawork that can be handled by controlling the operation of the camera is pan, tilt, zoom in, zoom out, or rack focus. Further, the operation of the camerawork during the moving image capturing can be performed by a manual operation of the user. That is, the camerawork as a target of during-imaging assistance is not limited to camerawork that involves the movement of the user. In the during-imaging assistance regarding such camerawork, a guide that gives an instruction to perform a camera operation (e.g., a zoom operation) of the user is displayed on the live view image.

If it is determined that the camerawork selected in step S705is camerawork that can be handled by controlling the operation of the camera (“operation of camera” in step S707), the processing proceeds to step S708. If, on the other hand, it is determined that the camerawork selected in step S705is camerawork that involves the movement of the user (“movement of user” in step S707), the processing proceeds to step S711.

In step S708, the camera instruction unit207controls the camera control unit108or the lens control unit109according to the camerawork information regarding the camerawork selected in step S705, controlling the operation of the camera main body or the lens, such as pan, tilt, zoom, or focus. Simultaneously with the start of the control of the operation of the camerawork or when an instruction to start video recording is received from the user, the image recording unit210starts recording the moving image data captured by the image capturing unit209as a moving image file.

In step S709, the camera instruction unit207detects the end of the operation of the camera main body or the lens from the camera control unit108of the lens control unit109.

In step S710, the image recording unit210ends the recording of the moving image data. Then, the image capturing unit209ends the imaging. Then, the processing in the imaging assistance mode ends.

In step S711, the CPU101executes the processing of the during-imaging assistance. Then, if the processing of the during-imaging assistance ends, the processing in the imaging assistance mode ends.

FIG.10illustrates the processing flow of the during-imaging assistance according to the present exemplary embodiment. The processing of the during-imaging assistance is the process executed in step S711inFIG.9. Simultaneously with the start of the processing of the during-imaging assistance or when an instruction to start video recording is received from the user (when the start video recording key505is pressed), the image recording unit210starts recording the moving image data captured by the image capturing unit209as a moving image file.

In step S800, based on the camerawork information regarding the camerawork selected in step S705, the user instruction unit208displays a guide that instructs the user to move in a superimposed manner on the live view image. For example, the user instruction unit208displays the display screen inFIG.7on the display unit105.

In step S801, the image capturing unit209determines whether the control of the operation of the camerawork is continuing. If it is determined that the control of the operation of the camerawork is continuing (Yes in step S801), the processing proceeds to step S802. If it is determined that the control of the operation of the camerawork is completed (No in step S801), the processing proceeds to step S808. For example, if the object comes out of the frame of the currently captured moving image data, the camerawork setting unit206instructs the user instruction unit208to end the control of the operation of the camerawork.

In step S802, the object recognition unit202analyzes the currently captured moving image data and calculates the size and the position of the object appearing in the moving image data and the movement information regarding the object (the moving velocity and the moving direction of the object).

In step S803, based on the information regarding the object calculated in step S802, the user instruction unit208determines whether the operation of the camerawork selected in step S705is appropriately performed. If it is determined that the operation of the camerawork is not appropriately performed (No in step S803), the processing proceeds to step S804. If it is determined that the operation of the camerawork is appropriately performed (Yes in step S803), the processing returns to step S800. In step S800, the user instruction unit208can update the display of the guide based on the camerawork information regarding the camerawork selected in step S705and the size and the position of the object and the movement information regarding the object calculated in step S802. Specifically, the user instruction unit208can change the size or the position of a guide frame or change the direction of an arrow. This allows instruction given to the user to move so that an appropriate operation of the camerawork according to the position and the motion of the object at the current moment is performed during the moving image capturing.

For example, suppose that the display screen illustrated inFIG.7is displayed during the processing of the during-imaging assistance. The instruction of the guide illustrated inFIG.7involves the user performing imaging while moving so that the main object501in the live view image500is included in an appropriate size within the guide frame502. That is, the movement of the user according to the instruction of the guide during the moving image capturing is an appropriate operation of the camerawork.

FIG.11is an example of a display screen displayed after a while since the start of display of the display screen inFIG.7. A main object901in a live view image900inFIG.11is shifted from a guide frame902in the left direction of the screen and becomes too small relative to the size of the guide frame902. As described above, in some cases, the object protrudes from the guide frame in the left-right direction of the screen, or the object becomes too small (or large). This indicates that the movement of the user according to the instruction of the guide is not performed.

In step S804, based on the information regarding the object calculated in step S802and the camerawork information regarding the camerawork selected in step S705, the camera instruction unit207calculates the amount of correction of the camera to achieve an appropriate operation of the camerawork. Specifically, taking into account the focal length of the lens, the angle of view, and the amount of shift of the object from a desired position, the camera instruction unit207calculates the amount of correction of the position and orientation or the enlargement ratio of the camera.

In the example ofFIG.11, from the angle of view (angle) of the lens and the proportion of the amount of shift to the screen, the camera instruction unit207calculates the amount of correction indicating to what extent (at what degrees) the camera should be panned in the left-right directions to correct the shift of the object in the left-right directions. Regarding the state where the object becomes too small, the camera instruction unit207can correct the size of the object by zooming in. The camera instruction unit207calculates the amount of correction of the size of the object from the size of the object at both the current time and the time when the during-imaging assistance starts, and the amount of change in the angle of view (the enlargement ratio) by zooming.

In step S805, the camera instruction unit207compares the amount of correction calculated in step S804and a threshold set in advance and determines whether the correction can be handled by controlling the operation of the camera main body or the lens. If it is determined that the amount of correction is smaller than the threshold and the correction can be handled by controlling the operation of the camera main body or the lens (Yes in step S805), the processing proceeds to step S806. If, on the other hand, it is determined that the amount of correction is greater than or equal to the threshold and the correction cannot be handled by controlling the operation of the camera main body or the lens (No in step S805), the processing proceeds to step S807.

In the above example where the object becomes too small, the size of the object can be corrected by zooming in. If, however, zooming is performed, the appearance range of the background as well as the size of the object changes. If this change is too great, the moving image is not a smooth moving image on the whole. Thus, there is an acceptable range for the correction of the size of the object by zooming. Moreover, the enlargement ratio has a limit value. Thus, a threshold can be provided in advance for the amount of correction that can be handled by controlling the operation of the camera main body or the lens. The threshold can be fixed or be appropriately changed by the user. If the threshold is0(if the threshold is not provided), the camera instruction unit207does not control the operation of the camera main body or the lens.

In step S806, based on the amount of correction calculated in step S804, the camera instruction unit207controls the operation of the camera main body by driving the camera main body to the left and right or zooming in or out using the camera control unit108or the lens control unit109. Then, the processing returns to step S800.

In step S807, based on the amount of correction calculated in step S804, the camera instruction unit207controls the operation of the camera main body by driving the camera main body to the left and right or zooming in or out using the camera control unit108or the lens control unit109. Simultaneously with this control, the user instruction unit208displays correction information that gives an instruction to correct the movement of the user on the live view image. While both the camera main body and the lens are controlled in steps S806and S807, a configuration can be employed in which either the camera main body or the lens is controlled.

FIG.12illustrates an example of a display screen that displays a guide that gives an instruction to correct the movement of the user according to the display of the display screen inFIG.11. InFIG.12, a live view image1000, a main object1001, and a guide frame1002correspond to the live view image900, the main object901, and the guide frame902, respectively, inFIG.11. The user instruction unit208displays correction text1003on the live view image1000. The correction text1003displays a comment that gives an instruction to correct the movement of the user. For example, in a correction instruction “chase object faster”, the user instruction unit208highlights an arrow1004by causing the arrow1004to blink, changing the color of the arrow1004, or changing the size of the arrow1004. For example, in a correction instruction “include object within frame”, the user instruction unit208highlights the guide frame1002by causing the guide frame1002to blink or changing the color of the guide frame1002similarly to the arrow1004. The correction text1003and the highlighting of the guide frame1002and the arrow1004are examples of the correction information that gives an instruction to correct the movement of the user.

In step S808, the image capturing unit209ends the moving image capturing. Then, the processing of the during-imaging assistance ends. The processes of steps S800to S807are continuously repeated during the moving image capturing. As described above, according to the detection that the motion of a mover according to camerawork is not performed during moving image capturing, the camera instruction unit207displays a guide that gives an instruction to correct the motion of the mover. The camera instruction unit207not only displays the guide that gives an instruction to correct the motion of the mover, but also controls the operation of the camera main body or the lens to correct a shift from an appropriate operation of the camerawork.

According to the present exemplary embodiment as described above, a user of a camera can be assisted by a guide being displayed that instructs the user to move or perform a camera operation in such a manner that allows the user to perform moving image capturing according to camerawork intended by the user. The user can also be provided with camerawork suitable for capturing a moving image of a currently captured object.

A second exemplary embodiment will be described. In the first exemplary embodiment, the method for determining recommended camerawork according to information regarding an object has been described. In the present exemplary embodiment, a method will be described for determining recommended camerawork taking into account a currently captured scene in information regarding an object. The differences from the first exemplary embodiment are mainly described below, and portions similar to those in the first exemplary embodiment are omitted.

FIG.13illustrates a functional configuration of a digital camera100according to the present exemplary embodiment. The functional configuration illustrated inFIG.13is different from the functional configuration inFIG.2in that a scene recognition unit1303is provided between an object recognition unit1302(equivalent to the object recognition unit202) and a matching determination unit1304(equivalent to the matching determination unit203). Moreover, the configuration of data held in a camerawork information holding unit1301is different from the configuration of data held in the camerawork information holding unit201according to the first exemplary embodiment.

FIGS.14A to14Dillustrate the configuration of data held in the camerawork information holding unit1301according to the present exemplary embodiment. As illustrated inFIGS.14A to14D, the camerawork information holding unit1301holds a data table illustrating the association of pieces of camerawork with the type of each object with respect to each of a plurality of scenes. For example, if an object is “person (entire body)” and a scene is “blue sky”, the matching determination unit1304determines circle, tilt, zoom out, and tilt follow among the plurality of pieces of camerawork as pieces of recommended camerawork. If there is a plurality of pieces of recommended camerawork, a camerawork notification unit1305(equivalent to the camerawork notification unit204) can give a notification based on pieces of camerawork information alone set in advance by the user so that a notification of the pieces of camerawork information is given as pieces of recommended camerawork information.

The scene recognition unit1303analyzes currently captured moving image data and recognizes a scene as an imaging target. Specifically, the scene recognition unit1303recognizes the type of the scene using information including the color distribution and the exposure distribution, in an image obtained from the moving image data and information regarding changes over time in the color distribution and the exposure distribution. Examples of the type of the scene recognized by the scene recognition unit1303include bright, dark, a blue sky, an evening view, and a spotlight.

The matching determination unit1304compares the information regarding the object acquired by the object recognition unit1302and the type of the scene recognized by the scene recognition unit1303and the pieces of information associated with the pieces of camerawork information held in the camerawork information holding unit201.

Then, as a result of the comparison, the matching determination unit1304determines whether there are pieces of camerawork information that match the information regarding the object acquired by the object recognition unit1302and the type of the scene recognized by the scene recognition unit1303.

FIG.15illustrates the processing flow of the imaging assistance mode according to the present exemplary embodiment.

Steps S1401and S1403to S1412in the processing flow illustrated inFIG.15are similar to steps S701to S711in the processing flow inFIG.9. That is, the processing flow of the imaging assistance mode according to the present exemplary embodiment is obtained by adding a scene recognition step (step S1402) to the processing flow of the imaging assistance mode according to the first exemplary embodiment.

In step S1402, the scene recognition unit1303analyzes the currently captured moving image data to recognize a scene.

In step S1403, based on the information regarding the object and the result of the recognition of the scene acquired in steps S1401and S1402, the matching determination unit203determines pieces of recommended camerawork. The following processing steps are similar to those in the first exemplary embodiment except for the processing of during-imaging assistance (step S1412).

FIG.16illustrates the processing flow of the during-imaging assistance according to the present exemplary embodiment. The processing flow of the during-imaging assistance corresponds to the process executed in step S1412inFIG.15. Steps S1500, S1501, and S1503to S1509in the processing flow illustrated inFIG.16are similar to steps S800to S808in the processing flow inFIG.10. That is, the processing flow of the during-imaging assistance according to the present exemplary embodiment is obtained by adding a camera shake correction step (step S1502) to the processing flow of the during-imaging assistance according to the first exemplary embodiment.

In step S1502, the camera instruction unit207receives signals from the acceleration sensor and the angular velocity sensor included in the camera control unit108and corrects the position and orientation of the camera main body by changing the position and orientation of the camera main body to cancel out changes in the position and orientation of the camera main body. Thus, when the amount of correction of the camera is calculated in step S1505at a subsequent stage, it is possible to more actively correct the influence of camera shake caused by the user or small blur associated with the movement of the user on a captured moving image.

In the present exemplary embodiment as described above, camerawork according to the scene can be recommended.

Further, moving image capturing can be performed by appropriately correcting blur in a camera during the moving image capturing.

According to the present disclosure, an imaging device can be provided that assists a user of a camera in performing moving image capturing with appropriate camerawork, and a control method for controlling the imaging device.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2023-108615, filed Jun. 30, 2023, which is hereby incorporated by reference herein in its entirety.