Electronic apparatus and control method thereof

An electronic apparatus according to the present invention, includes at least one memory and at least one processor which function as: an acquisition unit configured to acquire a captured image; and a control unit configured to control so as to extract a partial range of an image acquired by the acquisition unit and record a still image that is not a VR image in a storage, and control so as to record a moving image that is a VR image in the storage based on the acquired image.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an electronic apparatus, a control method thereof, a program, and a storage medium. In particular, the present invention relates to a method for capturing a moving image and a still image.

Description of the Related Art

There has been known an imaging apparatus capable of capturing a VR image having a wide field of view of, for example, 180 degrees or 360 degrees, which is wider than a normal field angle (Japanese Patent Application Publication No. 2015-115778). There has also been known an imaging apparatus capable of capturing still images and moving images at various timings (Japanese Patent Application Publication No. 2012-182730).

There are cases where a user wishes to capture both a VR image and an image with a normal field angle. However, in such a case, the user needs to use both an imaging apparatus capable of capturing VR images and an imaging apparatus capable of capturing images with a normal field angle. This is inconvenient for the user. Also, the user could miss a photo opportunity if taking too much time to switch between the two imaging apparatuses.

SUMMARY OF THE INVENTION

The present invention provides an electronic apparatus that enables capturing of both a VR image and an image with a normal field angle.

An electronic apparatus according to the present invention, includes at least one memory and at least one processor which function as: an acquisition unit configured to acquire a captured image; and a control unit configured to control so as to extract a partial range of an image acquired by the acquisition unit and record a still image that is not a VR image in a storage, and control so as to record a moving image that is a VR image in the storage based on the acquired image.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings.FIGS.1A and1Billustrate external views of an imaging apparatus100as an example of an electronic apparatus to which the present invention is applicable. The imaging apparatus100is a lens-integrated digital camera. The imaging apparatus100may as well be a single-lens reflex camera or a mirrorless single-lens camera.

FIG.1Aillustrates an external view of a front face of the imaging apparatus100. A display141displays various kinds of images and various kinds of information. The imaging apparatus100can cause the display141to perform a live-view display (a display of a live-view image in which an object is displayed in almost real time) (display control). In addition, the user can issue various instructions related to imaging capturing (various operations related to image capturing) while checking the object with the live-view display on the display141. A touch panel13provided on a display surface of the display141can receive touch operations. A release button11is a switch that can be half-pressed or fully-pressed. The user can issue at least one of an instruction to prepare for image capturing, an instruction to start moving-image capturing, an instruction to end moving-image capturing, an instruction to capture a still image, etc. by half-pressing or fully-pressing the release button11. A lens101is a wide-angle lens and forms an image of a luminous flux incident into the lens101on an image sensor102(not illustrated inFIG.1A).

FIG.1Billustrates an external view of a rare face of the imaging apparatus100. The user can issue at least one of an instruction to prepare for image capturing, an instruction to start moving-image capturing, an instruction to end moving-image capturing, an instruction to capture a still image, etc. by pressing a button12or releasing a finger pressing the button12therefrom. The lens111is a wide-angle lens similar to the lens101and forms an image of a luminous flux incident into the lens111on an image sensor112(not illustrated inFIG.1B).

Alternatively, the display141may be a separate device (an external device such as a smartphone or a tablet terminal connected to the imaging apparatus100by wire or wirelessly) from the imaging apparatus100. For example, the display141may be mounted on a smartphone or the like, which is a separate body from the imaging apparatus100. In this case, since the imaging apparatus100can cause the external device to perform a live-view display, the user can issue the various instructions related to image capturing while checking an object on the live-view display of the external device. In addition, an operation to issue the various instructions related to image capturing may be performed by using the separate device from the imaging apparatus100. For example, the user may perform the operation on a smartphone, which is a separate body from the imaging apparatus100, and the smartphone may issue an instruction corresponding to this operation to the imaging apparatus100.

FIG.2illustrates an internal configuration of the imaging apparatus100. WhileFIG.2separately illustrates each function as a constituent element, each function may be realized by hardware such as a single or a plurality of ASICs or programmable logic arrays (PLAs). Alternatively, each function may be realized by a programmable processor such as a CPU or an MPU that executes software.

The lens101is a single focus lens including an aperture mechanism, and the focal position of the lens101is set to obtain deep focus. However, the lens101is not necessarily be a single focus lens.

The image sensor102includes a CCD element and a CMOS element and has a configuration in which a plurality of pixels including a photoelectric conversion element is two-dimensionally arranged. The image sensor102photoelectrically converts an object image (optical image) formed by an imaging optical system including the lens101per pixel. Namely, the image sensor102captures an image, generates an analog signal of the image, converts the generated analog signal into a digital signal by an A/D conversion circuit, and outputs the image data on a pixel basis.

A memory I/F unit106writes image data for all the pixels output from the image sensor102into memory107. In addition, the memory I/F unit106reads out image data held in the memory107and outputs the read-out image data to an image processing unit108. The memory107is a volatile storage medium storing image data for several frames (each frame includes image data for all the pixels).

The image processing unit108performs processing such as gamma correction, color separation, and color difference matrix on image data (image data for all the pixels) from the memory I/F unit106, adds a synchronization signal to the image data, and outputs the image data as video data for recording or display. In addition, the image processing unit108obtains a photometric value from a luminance value of each pixel of the image data that is input to the image processing unit108and provides the photometric value to a camera control unit120via a bus121.

The lens111, the image sensor112, a memory I/F unit116, memory117, and an image processing unit118have functions similar to those of the lens101, the image sensor102, the memory I/F unit106, the memory107, and the image processing unit108, respectively. The lens111, the image sensor112, the memory I/F unit116, the memory117, and the image processing unit118perform processing similar to that performed by the lens101, the image sensor102, the memory I/F unit106, the memory107, and the image processing unit108, respectively.

The camera control unit120is configured by a CPU or the like and controls the entire operation of the imaging apparatus100. A RAM160is used as a work area (work memory) of the camera control unit120. In the RAM160, constants and variables to be used for the operation of the camera control unit120are recorded, and a program, etc. are expanded. A ROM180stores a computer program or the like for operating the camera control unit120. For example, the camera control unit120controls the entire operation of the imaging apparatus100by expanding the computer program stored in the ROM180in the RAM160and executing the computer program.

A gain control unit103, a shutter control unit104, and an aperture control unit105are used for exposure control. The camera control unit120controls the above control units103to105based on photometric values provided from the image processing unit108and the image processing unit118or an operation parameter manually set by the user. The gain control unit103controls gain of the image sensor102and the image sensor112. The shutter control unit104controls shutter speed of the image sensor102and the image sensor112. The aperture control unit105controls aperture mechanisms of the lens101and the lens111.

A storage medium I/F unit130is an interface for connecting a storage medium131to the imaging apparatus100. The storage medium I/F unit130records video data that is input from the image processing unit108, the image processing unit118, and a GPU142in the storage medium131and reads out the recorded video data from the storage medium131. The storage medium131is a storage unit configured by a semiconductor memory or the like.

The display I/F unit140outputs video data from the image processing unit108and the image processing unit118and image data rendered in a video RAM (VRAM)143by the GPU142to the display141.

The GPU142is a rendering engine that renders, based on an instruction from the camera control unit120, video data output from the image processing unit108and the image processing unit118in the VRAM143by performing image conversion processing as well as image cut-out processing, enlargement processing, distortion correction, etc. In addition, the GPU142performs processing for rendering various kinds of information about the imaging apparatus100and a menu screen in the VRAM143in a superimposed manner. For example, touch-panel buttons for receiving various instructions from the user are rendered as a graphic user interface (GUI) by the GPU142and displayed on the display141via the display I/F unit140.

A communication I/F unit150connects to a smartphone, a PC, and the like (not illustrated) located outside the imaging apparatus100via a communication unit151and transfers image data displayed on the display141thereto. In addition, the communication I/F unit150receives a control command, etc. to the imaging apparatus100from a smartphone, a PC, and the like and notifies the camera control unit120of the received control command, etc. The communication unit151is a wireless LAN interface for performing wireless communication and performs hardware processing on communication performed by the communication I/F unit150.

The release button11, the button12, and the touch panel13have functions described above and output operation information corresponding to an operation (an instruction to prepare for image capturing, an instruction to start moving-image capturing, an instruction to end moving-image capturing, an instruction to capture a still image, etc.) to an IO controller170. The IO controller170outputs the operation information from the various operational units to the camera control unit120, and the camera control unit120performs processing that corresponds to the operation information received from the IO controller170. For example, the touch panel13is superimposed on the display141and outputs operation information including coordinate information about a position and a region where a touch operation is performed.

FIG.3illustrates an internal configuration of the imaging apparatus100.FIG.3illustrates the configuration in a simplified manner so that characteristic processing of the present embodiment can be represented. Processing performed by a cut-out processing unit301and a VR processing unit302is realized by, for example, at least one constituent element including the camera control unit120inFIG.2.

The cut-out processing unit301is used to generate a still image that is not a virtual reality (VR) image (non-VR still image). The VR image will be described below. The cut-out processing unit301acquires an image (image data output from the image sensor102) captured by the image sensor102. The cut-out processing unit301performs cut-out processing for extracting at least a partial range of the acquired image. The cut-out processing unit301stores the extracted range of the image (non-VR still image; image data) in the memory107(still image buffering). The image processing unit108performs compression encoding processing on the still image stored in the memory107and records the still image (non-VR still image) obtained by the compression encoding processing in the storage medium131.

The range to be extracted in the cut-out processing may be a predetermined range such as a central portion (central part) of a range of the image captured by the image sensor102or may be a different range. Since the image captured by the image sensor102sometimes has distortion, the cut-out processing may include distortion correction to reduce the distortion of the image. By performing the distortion correction, the image with little distortion can be displayed or recorded. Alternatively, a non-VR still image may be recorded in the storage medium131by extracting a partial range of an image captured by the image sensor112.

The VR processing unit302is used to capture a moving image which is a VR image (VR moving image). The VR processing unit302acquires an image captured by the image sensor102and an image captured by the image sensor112and performs VR processing by using these two images. The VR processing is not performed when a non-VR still image is captured.

The VR processing is image processing performed on an image captured so as to process the image to be treated as a VR image (an image in a VR format). For example, the VR processing is image processing for obtaining an image in a celestial-sphere format (celestial-sphere image format). In the present embodiment, the VR processing unit302performs the following processing in the VR processing. First, by using one of the two images as a reference image and the other as a comparison image, the VR processing unit302calculates an amount of deviation between the reference image and the comparison image per area by pattern matching processing and detects a connection position at which the two images are connected, based on the amount of the deviation per area. Next, taking the detected connection position and lens characteristics of each optical system into consideration, the VR processing unit302corrects distortion of each image by performing geometric conversion. In this way, each of the images is converted into an image in the celestial-sphere format. Next, the VR processing unit302generates a single image in the celestial-sphere format by synthesizing (blending) the two images in the celestial-sphere format. The image (VR image) obtained by the VR processing is an image using equidistant cylindrical projection, for example. Namely, an individual direction from a position (imaging position) at which the image is captured can be associated with an individual position (an individual pixel) in the image. In the present embodiment, each of the image sensors102and112captures an image that corresponds to a field angle of 180 degrees. Next, by performing the VR processing, a 360-degree image (an omnidirectional image; a celestial-sphere image) that corresponds to a field angle of 360 degrees is obtained from the two images captured by the two image sensors102and112. The above synthesis may be performed when a VR image is VR-displayed (which will be described below), separately from the VR processing. In addition, a 180-degree image (a half-celestial-sphere image) may be generated by cutting out an arbitrary range corresponding to a field angle of 180 degrees from the 360-degree image.

The VR processing unit302stores the VR image obtained by the VR processing in the memory117so that the VR moving image is temporarily stored in the memory107(moving-image buffering). The image processing unit118performs compression encoding processing on the moving image stored in the memory117and records the moving image (VR moving image) obtained by the compression encoding processing in the storage medium131. For example, the image processing unit118records an image file of the VR moving image in the storage medium131with VR information. The VR information is information for associating an individual direction from an imaging position with an individual position in the image. For example, the VR information includes at least one of information indicating a position (pixel) that corresponds to a zenith direction, information indicating a position (pixel) that corresponds to a nadir direction, and information indicating that this moving image is a VR image. The image processing unit118may record the image file of the VR still image in the storage medium131with a character string added to its file name, the character string indicating that this image file includes the VR image. For example, the file name of the VR moving image may include a three-character string “.vr” in front of the extension “.mp4”, such as “123456.vr.mp4”. These additions of the VR information and the character string indicating a VR image are not applied when a non-VR still image is captured.

The VR image will be described next. The VR image is an image that can be VR-displayed (displayed in a display mode “VR view”). For example, the VR image is an image in a celestial-sphere format, in which an individual direction from an imaging position can be associated with an individual position (an individual pixel). In the VR display, a part of the VR image is displayed as a display range. The display range can be freely changed in accordance with an instruction (operation) from the user, a change in the orientation of the display device that performs the VR display, or the like. In the VR display, a display is performed (a display range is determined or changed) such that the user can feel as if the user were in a virtual space (VR space) of the place where the image is captured (the user can get sense of realism and immersion). In other words, the VR display provides a display in which a view from the imaging position is reproduced so that the user can obtain feeling of overlooking every direction from the imaging position.

Example 1 of the present invention will be described. In example 1, a VR moving image up to a time position at which a non-VR still image is captured is recorded in the storage medium131.FIG.4is a flowchart illustrating an example of image-capturing processing of the imaging apparatus100. The image-capturing processing inFIG.4is realized by the camera control unit120that expands a program stored in the ROM180in the RAM160and executes the program. For example, the image-capturing processing inFIG.4starts in response that a still image capturing mode is set on the imaging apparatus100. Of a plurality of processing steps inFIG.4, processing steps performed by a constituent element other than the camera control unit120are performed in accordance with instructions from the camera control unit120.

In step S401, the camera control unit120initializes an elapsed time MSPT from the start of the still image capturing mode to zero seconds.

In step S402, the camera control unit120updates the elapsed time MSPT. In step S403, the image sensors102and112perform image capturing. Next, the cut-out processing unit301acquires the image captured by the image sensor102, and the VR processing unit302acquires the image captured by the image sensor102and the image captured by the image sensor112. In step S404, by performing VR processing by using the two images acquired in step S403, the VR processing unit302generates a VR image (a 360-degree image). In step S405, the VR processing unit302stores the VR image obtained in step S404in the memory117as a part of the VR moving image (moving-image buffering). While the moving image is being buffered, the VR moving image is temporarily stored in the memory117. In step S406, the cut-out processing unit301extracts a partial range of the individual image acquired in step S403(cut-out processing). In step S407, the display141displays the image extracted in step S406(live-view display).

Step S404may be performed after step S405(before step S413described below). Specifically, the VR processing unit302may directly store the two images captured by the image sensors102and112in the memory117(without performing the VR processing). Subsequently, the VR processing unit302may generate a VR image by reading out the two images stored in the memory117and performing the VR processing on the read-out images.

Step S407performed for live-view display may be omitted. However, if the live-view display is performed, the user can capture an image at desired timing while checking the live-view display (the object). In addition, the user can check a brightness level, etc. of the live-view display and make an adjustment to obtain a desired imaging parameter so that the user can capture an image having a desired brightness level, etc. As described above, the live-view display may be performed on a separate device from the imaging apparatus100(an external device such as a smartphone or a tablet terminal connected to the imaging apparatus100).

In step S408, the camera control unit120determines whether an instruction to capture a still image (an instruction to perform still-image capturing) has been received from the user. If the instruction to capture a still image has been received, the processing proceeds to step S409, and if not, the processing proceeds to step S415.

In step S409, the cut-out processing unit301stores the range of the image extracted in step S406in the memory107as a non-VR still image (still-image buffering). In step S410, the image processing unit108performs compression encoding processing on the non-VR still image stored in the memory107. In step S411, the image processing unit108generates a file of the non-VR still image obtained by the compression encoding processing in step S410and records the file in the storage medium131. For example, if the compression encoding processing is performed based on a JPEG format, the still-image file in the non-VR format can be reproduced by using a reproduction viewer corresponding to the JPEG format, there will be no need to use a reproduction viewer corresponding to the VR format.

In example 1, the range (the partial range of the captured image) to be recorded in the storage medium131is the same as the range of the live-view display. Alternatively, these ranges may be different from each other. However, if these ranges are the same, the user can capture a desired range of the still image that the user has checked on the live-view display. Thus, the user can capture a still image more comfortably.

In example 1, a minimum duration MSRT and a maximum duration MSQT of the VR moving image are previously determined. The minimum duration MSRT and the maximum duration MSQT are longer than zero seconds, and the maximum duration MSQT is longer than the minimum duration MSRT. For example, the minimum duration MSRT is 3 seconds or 5 seconds, and the maximum duration MSQT is 5 seconds or 10 seconds. The minimum duration MSRT may be previously determined by a manufacturer or the like, may be automatically determined in the imaging apparatus100, or may be specified by the user. The minimum duration MSRT may be automatically or manually changeable. The same applies to the maximum duration MSQT.

In step S412, the camera control unit120determines whether the elapsed time MSPT is longer than the minimum duration MSRT. If the elapsed time MSPT is longer than the minimum duration MSRT, the processing proceeds to step S413, and if not, the processing proceeds to step S415.

In step S413, the image processing unit118sequentially reads out the VR image stored in the memory117and performs the compression encoding processing on the read-out VR image (compression encoding processing on the moving image). The compression encoding processing in step S413is performed on the VR moving image for a duration of the elapsed time MSPT up to the time position at which the non-VR still image is captured (for example, the time position in the period from step S409to step S413) or more and the maximum duration MSQT or less. For example, if the elapsed time MSPT is longer than the maximum duration MSQT, the image processing unit118reads out the VR moving image corresponding to the maximum duration MSQT and performs the compression encoding processing thereon. If the elapsed time MSPT is shorter than the maximum duration MSQT, the image processing unit118reads out the VR moving image corresponding to the elapsed time MSPT and performs the compression encoding processing thereon. In step S414, the image processing unit118records an image obtained by the compression encoding processing in step S413in the storage medium131as a part (frame) of the VR moving image. Specifically, a file of the VR moving image including the image obtained by the compression encoding processing is generated in the storage medium131(creation of a new file). Subsequently, an image obtained by the compression encoding processing is sequentially stored in the existing file (updating of the file). In step S414, VR information may be stored in the file of the VR moving image, or a character string indicating that the file includes a VR image may be added to a file name of the VR moving image.

In step S415, the camera control unit120determines whether an instruction to end (cancel) the still-image capturing mode has been received from the user. If the instruction has been received, the processing proceeds to step S416, and if not, the processing proceeds to step S402.

In step S416, the VR processing unit302stops the moving-image buffering in step S405.

In step S417, the camera control unit120determines whether the recording of the VR moving image (the VR moving image for a duration of the elapsed time MSPT up to the time position at which the non-VR still image is captured or more and the maximum duration MSQT or less) into the storage medium131has been completed. If the recording has been completed, the image-capturing processing inFIG.4ends, and if not, the processing proceeds to step S418.

In step S418, as in step S413, the image processing unit118performs the compression encoding processing on the moving image. In step S419, as in step S414, the image processing unit118records an image obtained by the compression encoding processing in step S413in the storage medium131as a part (frame) of the VR moving image. Steps S417to S419are repeated until the recording of the VR moving image is completed.

FIG.5illustrates an example of image capturing (capturing of the non-VR still image and the VR moving image) according to the image-capturing processing inFIG.4. The horizontal axis inFIG.5represents a lapse of time. In the image-capturing processing inFIG.4, every time an instruction to capture a still image is issued, a non-VR still image is captured. When a plurality of non-VR still images is captured, each of the non-VR still images captured is recorded in the storage medium131as an individual file. InFIG.5, the instruction to capture a still image has been issued twice. Accordingly, a file of the non-VR still image503and a file of the non-VR still image504are separately recorded in the storage medium131. In addition, in the image-capturing processing inFIG.4, every time a non-VR still image is captured, a VR moving image up to a time position at which the non-VR still image is captured is recorded as a file in the storage medium131. If the elapsed time MSPT is longer than the maximum duration MSQT, a VR moving image corresponding to the maximum duration MSQT is recorded, and if the elapsed time MSPT is longer than the minimum duration MSRT and shorter than the maximum duration MSQT, a VR moving image corresponding to the elapsed time MSPT is recorded. If the elapsed time MSPT is shorter than the minimum duration MSRT, a VR moving image is not recorded. InFIG.5, a file of a VR moving image501is recorded in the storage medium131in response to the capturing of the non-VR still image503, and a file of a VR moving image502is recorded in the storage medium131in response to the capturing of the non-VR still image504. Even when the instruction to capture the non-VR still image504is issued during the recording of the VR moving image501, the file of the non-VR still image503and the file of the non-VR still image504are separately recorded, and also, the file of the VR moving image501and the file of the VR moving image502are separately recorded. When the files are recorded, the VR moving image501and the non-VR still image503may be recorded in the storage medium131in association with each other, and the VR moving image502and the non-VR still image504may be recorded in the storage medium131in association with each other. By doing so, the VR moving image and the non-VR still image that have captured the same object can easily be switched to be displayed as well as can easily be managed. Thus, the convenience is improved.

Example 2 of the present invention will be described. In example 2, a VR moving image from a time position at which a non-VR still image is captured is recorded in the storage medium131.FIG.6is a flowchart illustrating an example of image-capturing processing of the imaging apparatus100. The image-capturing processing inFIG.6is realized by the camera control unit120that expands a program stored in the ROM180in the RAM160and executes the program. For example, the image-capturing processing inFIG.6starts in response that a still image capturing mode is set on the imaging apparatus100. Of a plurality of processing steps inFIG.6, processing steps performed by a constituent element other than the camera control unit120are performed in accordance with instructions from the camera control unit120.

In step S601, the camera control unit120deletes data recorded by buffering from the memories107and117(buffer clearing). In step S602, the camera control unit120sets a parameter FALSE which indicates that a moving image is not being recorded to a flag fRecordingMovie. In step S603, the camera control unit120initializes an image-capturing time ST of a still image to, for example, the start time of the still-image capturing mode.

In step S604, the image sensors102and112perform image capturing. Next, the cut-out processing unit301acquires the image captured by the image sensor102, and the VR processing unit302acquires the image captured by the image sensor102and the image captured by the image sensor112. In step S605, the cut-out processing unit301extracts a partial range of the individual image acquired in step S604(cut-out processing). In step S606, the display141displays the image extracted in step S605(live-view display).

In step S607, the camera control unit120determines whether a parameter TRUE which indicates that a moving image is being recorded is set to the flag fRecordingMovie. If the parameter TRUE is set, the processing proceeds to step S608, and if not, the processing proceeds to step S611.

In step S608, the camera control unit120determines whether an elapsed time from the image-capturing time ST of the still image has reached a predetermined moving-image capturing time period. If the moving-image capturing time period has been reached, the processing proceeds to step S610, and if not, the processing proceeds to step S609. The moving-image capturing time period is 3 seconds, 5 seconds, or 10 seconds, for example. The moving-image capturing time period may be previously determined by a manufacturer or the like, may be automatically determined in the imaging apparatus100, or may be specified by the user. The moving-image capturing time period may be automatically or manually changeable.

In step S609, the VR processing unit302stores each of the two images acquired in step S604(the two images captured by the image sensors102and112) in the memory117as a part of the moving image (moving-image buffering).

In step S610, the VR processing unit302stops the moving-image buffering in step S609, and the camera control unit120sets a parameter FALSE to the flag fRecordingMovie.

In step S611, the camera control unit120determines whether data to be read out (at least a part of the moving image corresponding to the moving-image capturing time period from the image-capturing time ST of the still image) still remains in the memory117. If the data to be read out still remains, the processing proceeds to step S612, and if not, the processing proceeds to step S614.

In step S612, the image processing unit118sequentially reads out the images stored in the memory117(reading-out of the moving images). Next, the image processing unit118performs VR processing by using the read-out images (the two images captured by the image sensors102and112) (VR processing on the moving images) and performs compression encoding processing on a VR image obtained by the VR processing (compression encoding processing on the moving image). In step S613, the image processing unit118records an image obtained by the compression encoding processing in step S612in the storage medium131as a part (frame) of the VR moving image.

In step S614, the camera control unit120determines whether an instruction to end (cancel) the still-image capturing mode has been received from the user. If the instruction has been received, the processing proceeds to step S621, and if not, the processing proceeds to step S615.

In step S615, the camera control unit120determines whether an instruction to capture a still image (an instruction to perform still-image capturing) has been received from the user. If the image-capturing instruction has been received, the processing proceeds to step S616, and if not, the processing proceeds to step S604.

In step S616, the camera control unit120updates the image-capturing time ST of the still image with the current time. In step S617, the camera control unit120sets a parameter TRUE to the flag fRecordingMovie. In step S618, the cut-out processing unit301stores the range of the image extracted in step S605in the memory107as a non-VR still image (still-image buffering). In step S619, the image processing unit108performs the compression encoding processing on the non-VR still image stored in the memory107. In step S620, the image processing unit108generates a file of the non-VR still image obtained by the compression encoding processing in step S619and stores the file in the storage medium131.

In step S621, the VR processing unit302stops the moving-image buffering in step S609, and the camera control unit120sets a parameter FALSE to the flag fRecordingMovie.

In step S622, as in step S611, the camera control unit120determines whether data to be read out still remains in the memory117. If the data to be read out still remains, the processing proceeds to step S623, and if not, the image-capturing processing inFIG.6ends.

In step S623, as in step S612, the image processing unit118performs the reading-out of the moving images, the VR processing, and the compression encoding processing. In step S624, as in step S613, the image processing unit118records an image obtained by the compression encoding processing in step S623in the storage medium131as a part (frame) of the VR moving image. Steps S622to S624are repeated until the reading-out of the data (recording of the VR moving image) from the memory117is completed.

FIGS.7and8illustrate examples of the image capturing (capturing of non-VR still images and VR moving images) according to the image-capturing processing inFIG.6. The horizontal axis in each ofFIGS.7and8represents a lapse of time. In the image-capturing processing inFIG.6, every time an instruction to capture a still image is issued, a non-VR still image is captured. When a plurality of non-VR still images is captured, each of the non-VR still images is recorded in the storage medium131as an individual file. InFIG.7, the instruction to capture a still image has been issued twice. Accordingly, a file of the non-VR still image703and a file of the non-VR still image704are separately recorded in the storage medium131. InFIG.8, too, the instruction to capture a still image has been issued twice. Accordingly, a file of the non-VR still image803and a file of the non-VR still image804are separately recorded in the storage medium131.

In addition, in the image-capturing processing inFIG.6, every time a non-VR still image is captured, a VR moving image corresponding to the moving-image capturing time period from a time position at which the non-VR still image is captured is recorded as a file in the storage medium131. InFIG.7, a file of a VR moving image701is recorded in the storage medium131in response to the capturing of the non-VR still image703, and a file of a VR moving image702is recorded in the storage medium131in response to the capturing of the non-VR still image704.

However, if an instruction to capture a still image is issued while a VR moving image is being captured, the capturing of the VR moving image is cancelled, and a VR moving image is started to be captured from a time position at which a non-VR still image is captured last. InFIG.8, capturing of a VR moving image801corresponding to the moving-image capturing time period from a time position at which the non-VR still image803is captured is cancelled, and only a file of a VR moving image802corresponding to the moving-image capturing time period from a time position at which the non-VR still image804is captured is recorded in the storage medium131. When the file is recorded, the VR moving image802may be associated with only the non-VR still image804or may be associated with both the non-VR still image803and the non-VR still image804.

Alternatively, in the case ofFIG.8, a file of the VR moving image801and a file of the VR moving image802may be recorded in the storage medium131without cancelling the capturing of the VR moving image801. Instead of the file of the VR moving image801corresponding to the moving-image capturing time period, a file of a VR moving image for a duration from a time position at which the non-VR still image803is captured to a time position at which the non-VR still image804is captured may be recorded in the storage medium131. The method for recording the VR moving image is changeable as appropriate.

Example 3 of the present invention will be described. In example 3, a VR moving image for a duration from before a time position at which a non-VR still image is captured to after the time position is recorded in the storage medium131.FIG.9is a flowchart illustrating an example of image-capturing processing of the imaging apparatus100. The image-capturing processing inFIG.9is realized by the camera control unit120that expands a program stored in the ROM180in the RAM160and executes the program. For example, the image-capturing processing inFIG.9starts in response that a still image capturing mode is set on the imaging apparatus100. Of a plurality of processing steps inFIG.9, processing steps performed by a constituent element other than the camera control unit120are performed in accordance with instructions from the camera control unit120.

In step S901, the camera control unit120deletes data recorded by buffering from the memories107and117(buffer clearing). In step S902, the camera control unit120initializes an image-capturing time ST of a still image to the start time of the still-image capturing mode. In step S903, the camera control unit120initializes an elapsed time MSPT from the start of the still image capturing mode to zero seconds. In step S904, the camera control unit120sets a parameter FALSE which indicates that a moving image is not being recorded to a flag fRecordingMovie. In step S905, the camera control unit120initializes a start time MBS of moving-image capturing to, for example, the start time of the still-image capturing mode.

In step S906, the image sensors102and112capture images. Next, the cut-out processing unit301acquires the image captured by the image sensor102, and the VR processing unit302acquires the image captured by the image sensor102and the image captured by the image sensor112. In step S907, the VR processing unit302stores each of the two images acquired in step S906(the two images captured by the image sensors102and112) in the memory117as a part of the moving image (moving-image buffering). In step S908, the cut-out processing unit301extracts a partial range of the individual image acquired in step S906(cut-out processing). In step S909, the display141displays the image extracted in step S908(live-view display).

In step S910, the camera control unit120determines whether a parameter TRUE which indicates that a moving image is being recorded is set to the flag fRecordingMovie. If the parameter TRUE is set, the processing proceeds to step S911, and if not, the processing proceeds to step S916.

In step S911, the image processing unit118sequentially reads out the images captured after the start time MBS of the moving-image capturing from the memory117(reading-out of the moving images). Next, the image processing unit118performs VR processing by using the read-out images (the two images captured by the image sensors102and112) (VR processing on the moving images) and performs compression encoding processing on a VR image obtained by the VR processing (compression encoding processing on the moving image). In step S912, the image processing unit118records an image obtained by the compression encoding processing in step S911in the storage medium131as a part (frame) of the VR moving image.

In step S913, the camera control unit120determines whether an elapsed time from the image-capturing time ST of the still image has reached a predetermined reference time period MT. If the reference time period MT has been reached, the processing proceeds to step S914, and if not, the processing proceeds to step S916. The reference time period MT is 3 seconds, 5 seconds, or 10 seconds, for example. The reference time period MT may be previously determined by a manufacturer or the like, may be automatically determined in the imaging apparatus100, or may be specified by the user. The reference time period MT may be automatically or manually changeable.

In step S914, the VR processing unit302stops the moving-image buffering in step S907, and the camera control unit120sets a parameter FALSE to the flag fRecordingMovie. In step S915, the VR processing unit302closes the file of the VR moving image recorded in the storage medium131in step S912.

In step S916, the camera control unit120determines whether an instruction to end (cancel) the still-image capturing mode has been received from the user. If the instruction has been received, the processing proceeds to step S925, and if not, the processing proceeds to step S917.

In step S917, the camera control unit120determines whether an instruction to capture a still image (an instruction to perform still-image capturing) has been received from the user. If the image-capturing instruction has been received, the processing proceeds to step S918, and if not, the processing proceeds to step S906.

In step S918, the camera control unit120updates the image-capturing time ST of the still image with the current time. In step S919, the camera control unit120sets a parameter TRUE to the flag fRecordingMovie.

In step S920, the camera control unit120determines whether an elapsed time from the start time MBS of the moving-image capturing has reached a predetermined reference time period MT. If the reference time period MT has been reached, the processing proceeds to step S921, and if not, the processing proceeds to step S922. The start time MBS is a set value, and a moving-image capturing is not necessarily performed at the time in step S920. For example, when step S920is performed for the first time, the start time MBS of the moving-image capturing is set to the start time of the still-image capturing mode (the initial value).

In step S921, the camera control unit120updates the start time MBS of the moving-image capturing with a time that is the reference time period MT before the image-capturing time ST of the still image.

In step S922, the cut-out processing unit301stores the range of the image extracted in step S908in the memory107as a non-VR still image (still-image buffering). In step S923, the image processing unit108performs compression encoding processing on the non-VR still image stored in the memory107. In step S924, the image processing unit108records (generates) a file of the non-VR still image obtained by the compression encoding processing in step S923in the storage medium131.

In step S926, the VR processing unit302stops the moving-image buffering in step S907, and the camera control unit120sets a parameter FALSE to the flag fRecordingMovie.

In step S927, the camera control unit120determines whether data to be read out (at least a part of the moving image captured after the start time MBS of the moving-image capturing) still remains in the memory117. If the data to be read out still remains, the processing proceeds to step S928, and if not, the processing proceeds to step S930.

In step S928, as in step S911, the image processing unit118performs the reading-out of the moving images, the VR processing, and the compression encoding processing. In step S929, as in step S912, the image processing unit118records an image obtained by the compression encoding processing in step S928in the storage medium131as a part (frame) of the VR moving image. Steps S927to S929are repeated until the reading-out of the data (recording of the VR moving image) from the memory117is completed.

In step S930, the VR processing unit302closes the file of the VR moving image recorded in the storage medium131in step S929.

FIGS.10and11illustrate examples of image capturing (capturing of non-VR still images and VR moving images) of the image-capturing processing inFIG.9. The horizontal axis in each ofFIGS.10and11represents a lapse of time. In the image-capturing processing inFIG.9, every time an instruction to capture a still image is issued, a non-VR still image is captured. When a plurality of non-VR still images is captured, each of the non-VR still images is recorded in the storage medium131as an individual file. InFIG.10, the instruction to capture a still image has been issued twice. Accordingly, a file of the non-VR still image1003and a file of the non-VR still image1004are separately recorded in the storage medium131. InFIG.11, too, the instruction to capture a still image has been issued twice. Accordingly, a file of the non-VR still image1104and a file of the non-VR still image1105are separately recorded in the storage medium131.

In addition, in the image-capturing processing inFIG.9, every time the non-VR still image is captured, a VR moving image is captured for a duration from a time MBS as a start time to a time that is the reference time period MT after the image-capturing time ST of the non-VR still image as an end time. Hereinafter, an image-capturing time period from a time MBS as a start time to a time that is the reference time period MT after the image-capturing time ST of the non-VR still image as an end time is referred to as a “basic image-capturing time period”. In the image-capturing processing inFIG.9, for example, a file of a VR moving image for a duration from a time that is the reference time period MT before an image-capturing time ST to a time that is the reference time period MT after the image-capturing time ST is recorded in the storage medium131as a file of the VR moving image for a basic image-capturing time period. InFIG.10, a file of a VR moving image1001is recorded in the storage medium131in response to the capturing of the non-VR still image1003, and a file of a VR moving image1002is recorded in the storage medium131in response to the capturing of the non-VR still image1004.

However, if the basic image-capturing time period of a VR moving image overlaps the basic image-capturing time period of another VR moving image, the plurality of basic image-capturing time periods overlapping with each other is synthesized into one image-capturing time period, and one VR moving image is captured for the one image-capturing time period, rather than that VR moving images are captured for the respective basic image-capturing time periods. InFIG.11, the basic image-capturing time period of the VR moving image1101corresponding to the capturing of the non-VR still image1104and the basic image-capturing time period of the VR moving image1102corresponding to the capturing of the non-VR still image1105overlap with each other. Accordingly, only a file of a VR moving image1103for an image-capturing time period from the start time of the basic image-capturing time period of the VR moving image1101to the end time of the basic image-capturing time period of the VR moving image1102is recorded in the storage medium131. When the file is recorded, the VR moving image1103may be associated with only the non-VR still image1105or may be associated with both the non-VR still image1104and the non-VR still image1105.

Alternatively, in the case ofFIG.11, instead of recording the file of the VR moving image1103in the storage medium131, a file of the VR moving image1101and a file of the VR moving image1102may be recorded in the storage medium131. Instead of the file of the VR moving image1101, a file of a VR moving image for an image-capturing time period from the start time of the basic image-capturing time period of the VR moving image1101to the start time of the basic image-capturing time period of the VR moving image1102may be recorded in the storage medium131. Instead of the file of the VR moving image1102, a file of a VR moving image for an image-capturing time period from the end time of the basic image-capturing time period of the VR moving image1101to the end time of the basic image-capturing time period of the VR moving image1102may be recorded in the storage medium131. The method for recording the VR moving image is changeable as appropriate.

As described above, according to the present embodiment (including examples 1 to 3), both a non-VR still image (a still image with a normal field angle) and a VR moving image can be captured. While the present embodiment has been described with the example in which the VR moving image is a 360-degree image, the filed angle of the VR moving image may be smaller than 360 degrees or smaller than 180 degrees. However, the field angle of the VR moving image is wider than that of the non-VR still image.

In addition, while the present embodiment has been described with the example in which the VR moving image is obtained by using the two image sensors102and112, a VR moving image may be generated from an image captured by a single image sensor, and the imaging apparatus100may include only one image sensor. Alternatively, a plurality of images may be captured by using three or more image sensors, and the VR processing may be performed on the captured images to generate a VR moving image. In addition, instead of a non-VR still image, a VR still image (a still image which is a VR image) may be generated. However, in the present embodiment, the field angle of the still image is smaller than that of the moving image.

In addition, the various kinds of control operations, which have been described above to be performed by the camera control unit120, may be performed by a single hardware unit, or alternatively, a plurality of hardware units (for example, a plurality of processors or circuits) may share the processing and control the entire apparatus.

While the present invention has been described in detail based on the preferred embodiment, the present invention is not limited to this specific embodiment, and various modes are included in the present invention without departing from the gist of the present invention. The embodiment described above is merely an exemplary embodiment of the present invention, and it is also possible to combine the embodiment with another embodiment as appropriate.

While the above embodiment has been described with the exemplary case where the present invention is applied to the digital camera, the present invention is not limited to this exemplary case and is applicable to any apparatus (electronic device) capable of acquiring a captured image. For example, the present invention is applicable to a personal computer, a PDA, a mobile phone terminal, a portable image viewer, a printer, a digital photo frame, a music player, a game machine, an electronic book reader, a video player, and the like. The present invention is also applicable to a television device, a projection device, a tablet terminal, a smartphone, an AI speaker, a household electric appliance, an on-vehicle device, medical equipment, and the like.

In addition, the present invention is applicable not only to an imaging apparatus itself but also to a control apparatus that communicates with an imaging apparatus (including a network camera) via wired or wireless communication and remotely controls the imaging apparatus. Examples of the apparatus that remotely controls the imaging apparatus include a smartphone, a tablet PC, and a desktop PC. On the basis of operations and processing performed on the control apparatus side, the control apparatus can remotely control the imaging apparatus by notifying the imaging apparatus of a command that causes the imaging apparatus to perform various operations and settings. Alternatively, the control apparatus may receive a live-view image captured by the imaging apparatus via wired or wireless communication and display the received live-view image on the control apparatus side.

According to the present disclosure, both a VR image and an image with a normal field angle can be captured.

OTHER EMBODIMENTS

This application claims the benefit of Japanese Patent Application No. 2019-091446, filed on May 14, 2019, which is hereby incorporated by reference herein in its entirety.