Electronic apparatus enabling a user to recognize gradation of a raw image with high accuracy by checking the displayed raw histogram

There is provided a technology for enabling a user to recognize gradation of a captured image with high accuracy, when the user checks a substitute image displayed in substitution for the captured image. An electronic apparatus divides a captured image into a plurality of partial areas, generates each of histograms for the respective partial areas, the histograms each representing a number of pixels of each gradation, and outputs the generated histogram to an apparatus that displays a substitute image having a data size smaller than a data size of the captured image.

BACKGROUND

Field of the Disclosure

The present disclosure generally relates to an electronic apparatus that can present an output to outside or execute display control of a substitute image that substitutes for a captured image.

Description of the Related Art

There is an image capturing apparatus that has a wireless communication function in conformity with a Wireless Fidelity (Wi-Fi) standard. This image capturing apparatus can wirelessly transmit a captured image which is recorded in a memory card or built-in memory to an external apparatus such as a smartphone, and store the transmitted image. In general, a RAW image has a data size (an information amount) which is relatively larger than the data size of an image in a general-purpose image format such as Joint Photographic Experts Group (JPEG). Therefore, in a case where an image to be transmitted is a RAW image, the RAW image itself may not be transmitted and a JPEG image substituting for the RAW image may be transmitted. In this case, a reduction in transmission time and battery saving in an external apparatus can be expected. For example, Japanese Patent Application Laid-Open No. 2005-80049 discusses a technology for converting the image format of a captured image and transmitting the image after the conversion.

In general, information about color and color tone captured by an image sensor (an image capturing element) of an image capturing apparatus is recorded in a RAW image as much as possible, and the color depth of a JPEG image is lower than the color depth of the RAW image. Therefore, if the RAW image is converted into the JPEG image, the color depth may be reduced. As a result, gradation collapse (whiteout or blackout) not present in the RAW image can occur in the JPEG image. Even if a user can visually recognize the JPEG image displayed in substitution for the RAW image, the user cannot recognize the gradation of the RAW image with high accuracy. For example, the user cannot determine whether whiteout or blackout occurring in the JPEG image is also present in the RAW image.

SUMMARY

The present disclosure is related to a technology for enabling a user to recognize gradation of a captured image with high accuracy when the user checks a substitute image displayed in substitution for the captured image.

According to a first aspect of the present disclosure, an electronic apparatus includes an acquisition unit configured to acquire a histogram from a captured image, the histogram having gradation values in a partial area that is an area forming a part of the captured image, and an output unit configured to output the histogram, area information about the partial area, and a substitute image that has a data size smaller than a data size of the captured image and substitutes for the captured image, to outside.

According to a second aspect of the present disclosure, an electronic apparatus includes an acquisition unit configured to acquire a histogram from a captured image, the histogram having gradation values in a partial area that is an area forming a part of the captured image, and a control unit configured to perform control for displaying a substitute image that has a data size smaller than a data size of the captured image and substitutes for the captured image, in such a manner that a correspondence area of the substitute image and corresponding to the partial area is distinguishable, wherein the control unit performs control for displaying the histogram in a case where the correspondence area is designated by a user.

According to a third aspect of the present disclosure, a control method for an electronic apparatus includes acquiring a histogram from a captured image, the histogram having gradation values in a partial area that is an area forming a part of the captured image, and outputting the histogram, area information about the partial area, and a substitute image that has a data size smaller than a data size of the captured image and substitutes for the captured image, to outside.

According to a fourth aspect of the present disclosure, a control method for an electronic apparatus includes acquiring a histogram from a captured image, the histogram having gradation values in a partial area that is an area forming a part of the captured image, and performing control for displaying a substitute image that has a data size smaller than a data size of the captured image and substitutes for the captured image, in such a manner that a correspondence area of the substitute image and corresponding to the partial area is distinguishable, wherein the control is performed to display the histogram in a case where the correspondence area is designated by a user.

According to a fifth aspect of the present disclosure, a computer readable storage medium stores a program that causes a computer to function as each of the units in the above-described electronic apparatuses.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the drawings.

An exemplary embodiment of the present disclosure will be described below. In the present exemplary embodiment, a Joint Photographic Experts Group (JPEG) image (a substitute image, or a substitute JPEG image) that substitutes for a RAW image (a captured image) can be displayed. Further, a RAW histogram of a partial area that forms a part of the RAW image can also be displayed. The RAW histogram is a histogram that represents the number of pixels for RAW gradation values (the gradation of the brightness of the RAW image) in the partial area. The user can thereby check not only the substitute JPEG image but also the RAW histogram. As a result, the user can recognize the gradation of the RAW image with high accuracy by checking the displayed RAW histogram. Further, the user can determine whether gradation collapse (whiteout or blackout) occurring in the substitute JPEG image is also present in the RAW image by checking the substitute JPEG image and the RAW histogram that are displayed. The captured image is not limited to the RAW image, and the substitute image is not limited to the JPEG image.

In the present exemplary embodiment, an example of a display control system that has an image capturing apparatus serving as a first electronic apparatus, and a display control apparatus serving as a second electronic apparatus, will be described. The image capturing apparatus may be any type of apparatus if the apparatus can perform image capturing (image sensing). Examples of the image capturing apparatus include a digital camera and various mobile terminals (such as a smartphone, a tablet terminal, and a laptop computer). The display control apparatus may be any type of apparatus if the apparatus can control display. Examples of the display control apparatus include various mobile terminals, a personal computer, various reproduction apparatuses, and a server.

FIG. 1is a block diagram illustrating a configuration example of an image capturing apparatus100according to the present exemplary embodiment. The image capturing apparatus100has a control unit101, a first storage unit102, a second storage unit103, an image capturing unit104, a storage unit105, an operation unit106, a display unit107, and a communication unit108.

The control unit101, which may include one or more processors, one or more memories, circuitry, or a combination thereof, may control various kinds of processing performed by the image capturing apparatus100. For example, a central processing unit (CPU) can be used for the control unit101. One piece of hardware may be used or a plurality of pieces of hardware may be used for the control unit101. The processing by the image capturing apparatus100may be controlled based on the shared execution of the processing by the plurality of pieces of hardware.

The first storage unit102is a unit that temporarily stores various kinds of data, and is used for processing by the control unit101. For example, a volatile memory (such as a random access memory (RAM)) can be used for the first storage unit102.

The second storage unit103is a unit that non-temporarily stores various kinds of data. The second storage unit103stores, for example, various kinds of program (including firmware) to be used by the image capturing apparatus100(such as the control unit101), and various kinds of setting information. For example, a nonvolatile memory (such as a read only memory (ROM)) can be used for the second storage unit103.

The image capturing unit104generates a RAW image (RAW image data) by performing image capturing. The image capturing unit104has a lens, a shutter, an iris diaphragm, and an image capturing element (a complementary metal-oxide-semiconductor (CMOS) or a charge-coupled device (CCD)).

The storage unit105is a unit that non-temporarily stores various kinds of data. The storage unit105stores, for example, a RAW image, and a RAW file including the RAW image. For example, a semiconductor memory (a memory card), a magnetic disk (a hard disk), or an optical disc (a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc) can be used for the storage unit105. A card-type medium such as a magnetic card, an optical card, or an integrated circuit (IC) card can be also used for the storage unit105. The storage unit105may be or may not be detachably attached to the image capturing apparatus100. The image capturing apparatus100only needs to have a function of reading data from and writing data into the storage unit105and deleting data stored in the storage unit105by accessing the storage unit105.

The operation unit106receives a user operation for the image capturing apparatus100. The operation unit106outputs an operation signal based on the user operation to the control unit101. The control unit101performs processing based on the operation signal. For example, an input device such as a physical button, a pointing device (such as a touch panel, a touchpad, or a mouse), or a keyboard can be used for the operation unit106. The operation unit106may be or may not be detachably attached to the image capturing apparatus100. The image capturing apparatus100only needs to have a function of receiving an electrical signal based on a user operation performed using the input device.

The communication unit108transmits data (such as a control command and image data) to an external apparatus provided outside the image capturing apparatus100, and receives data from the external apparatus. In the present exemplary embodiment, the communication unit108wirelessly transmits data to a display control apparatus (a display apparatus)200illustrated inFIG. 2and wirelessly receives data from the display control apparatus200(wireless communication). Examples of the wireless communication include wireless communication in conformity with a standard such as Wireless Fidelity (Wi-Fi) or Bluetooth. The communication between the image capturing apparatus100and the display control apparatus200may be wired communication using a cable such as a Universal Serial Bus (USB) cable. The communication between the image capturing apparatus100and the display control apparatus200may be communication via a network such as the Internet and a server, or may be communication without the intervention of these.

FIG. 2is a block diagram illustrating a configuration example of the display control apparatus200. The display control apparatus200has a control unit201, a first storage unit202, a second storage unit203, a storage unit204, an operation unit205, a display unit206, and a communication unit207.

The control unit201, which may include one or more processors, one or more memories, circuitry, or a combination thereof, may control various kinds of processing by the display control apparatus200. For example, a CPU can be used for the control unit201. One piece of hardware may be used or a plurality of pieces of hardware may be used for the control unit201. The processing by the display control apparatus200may be controlled based on the shared execution of the processing by the plurality of pieces of hardware.

The first storage unit202is a storage unit that temporarily stores various kinds of data, and is used for processing (work) by the control unit201. For example, a volatile memory (such as a RAM) can be used for the first storage unit202.

The second storage unit203is a storage unit that non-temporarily stores various kinds of data. The second storage unit203stores, for example, various kinds of program (including firmware) to be used by the display control apparatus200(such as the control unit201), and various kinds of setting information. For example, a nonvolatile memory (such as a ROM) can be used for the second storage unit203.

The storage unit204is a storage unit that non-temporarily stores various kinds of data. The storage unit204stores, for example, data received from the image capturing apparatus100(the communication unit108). For example, a semiconductor memory, a magnetic disk, an optical disc, a magnetic card, an optical card, or an IC card can be used for the storage unit204. The storage unit204may be or may not be detachably attached to the display control apparatus200. The display control apparatus200only needs to have a function of reading data from and writing data into the storage unit204and deleting data stored in the storage unit204by accessing the storage unit204.

The operation unit205receives a user operation for the display control apparatus200. The operation unit205outputs an operation signal based on the user operation to the control unit201. The control unit201performs processing based on the operation signal. For example, an input device such as a physical button, a pointing device, or a keyboard can be used for the operation unit205. The operation unit205may be or may not be detachably attached to the display control apparatus200. The display control apparatus200only needs to have a function of receiving an electrical signal based on a user operation performed using the input device.

The display unit206displays various images. The display unit206displays, for example, data received from the image capturing apparatus100, and a graphic image for an interactive operation. For example, a liquid crystal panel and a backlight module, or an organic EL panel can be used for the display unit206. The display unit206may be or may not be detachably attached to the display control apparatus200. The display control apparatus200only needs to have a function of controlling the display by the display unit206.

The communication unit207transmits data (such as a control command and image data) to an external apparatus provided outside the display control apparatus200, and receives data from the external apparatus. In the present exemplary embodiment, the communication unit207wirelessly transmits data to the image capturing apparatus100(the communication unit108) and wirelessly receives data from the image capturing apparatus100.

FIG. 3is a flowchart illustrating an example of a processing flow in the image capturing apparatus100. The processing flow inFIG. 3starts, for example, in response to a user operation for providing an instruction for transmission or display of a substitute JPEG image. The processing flow inFIG. 3may be performed such that a substitute JPEG image corresponding to a RAW image generated by image capturing is output from the image capturing apparatus100to the display control apparatus200based on this image capturing.

In step S301, the control unit101determines a partial area that is an area forming a part of the RAW image that is a processing target (a target RAW image). In the present exemplary embodiment, the control unit101determines four partial areas in two rows and two columns that form the entire area of the target RAW image. Any number, layout, and shape of the plurality of partial areas forming the entire area of the target RAW image may be employed. The entire area of the target RAW image may not be formed by the plurality of partial areas. A plurality of partial areas away from each other may be determined.

In step S302, for each of the partial areas, the control unit101generates (acquires) a RAW histogram of the partial area (a histogram of gradation values of the target RAW image) from the target RAW image.

In step S303, the control unit101controls the communication unit108to transmit (output) the following data (information) to outside. The communication unit108thereby transmits the following data to the display control apparatus200(the communication unit207).Substitute JPEG image (substitute JPEG image data) for target RAW imageArea information about each partial areaEach RAW histogram (RAW histogram data) generated in step S302

The substitute JPEG image to be transmitted in step S303is, for example, a JPEG image that is stored in a RAW file together with the target RAW image. The substitute JPEG image may be a JPEG image obtained by converting the target RAW image at the timing of the transmission thereof. The substitute JPEG image may not be a JPEG image obtained by converting the target RAW image. The substitute JPEG image may be a JPEG image generated by other image capturing performed at substantially the same timing as the timing of the capturing of the target RAW image (“substantially” including “perfectly”). The data size (information amount) of the substitute JPEG image is smaller than the data size of the target RAW image.

FIG. 4is a flowchart illustrating an example of a processing flow in the display control apparatus200. The processing flow inFIG. 4is performed after step S303inFIG. 3.

In step S401, the control unit201detects reception (acquisition) of data by the communication unit207. This data is the data (the substitute JPEG image, each of the RAW histograms, and the area information) transmitted from the image capturing apparatus100in step S303.

In step S402, the control unit201performs control for displaying the substitute JPEG image at the display unit206such that a plurality of correspondence areas corresponding to the respective plurality of partial areas is distinguishable, based on the substitute JPEG image and the area information received in step S401. The correspondence area is an area forming a part of the substitute JPEG image. The area information may be information indicating the partial area, or information indicating the correspondence area. The area information may be any kind of information if the correspondence area and the partial area corresponding to the correspondence area can be determined using the area information.

For example, a screen illustrated inFIG. 5is displayed at the display unit206by the process in step S402. Any method may be employed to make the correspondence areas distinguishable. InFIG. 5, four correspondence areas501to504corresponding to the respective four partial areas can be distinguished by a broken line505on a substitute JPEG image500. The broken line505can also be “the boundary of each of four correspondence areas501to504”. A person506that is a main object and a cloud507that is a part of a background appear in the substitute JPEG image500. Here, assume that whiteout is occurring in a part of the cloud507.

In step S403, the control unit201determines whether the correspondence area is designated by a user. A user operation for designating the correspondence area can be performed, for example, using the operation unit205. If the control unit201determines that the correspondence area is designated (YES in step S403) after having waited for the designation of the correspondence area, the processing proceeds to step S404.

In step S404, the control unit201performs control for displaying the RAW histogram of the correspondence area designated in step S403(specifically, the RAW histogram of the partial area corresponding to the designated correspondence area), at the display unit206.

For example, a screen illustrated inFIG. 6is displayed at the display unit206by the process in step S404. The screen illustrated inFIG. 6includes an R-histogram601corresponding to red, a G-histogram602corresponding to green, and a B-histogram603corresponding to blue. The horizontal axis of the R-histogram601indicates gradation values corresponding to red, the horizontal axis of the G-histogram602indicates gradation values corresponding to green, and the horizontal axis of the B-histogram603indicates gradation values corresponding to blue. The vertical axis of each of the R-histogram601, the G-histogram602, and the B-histogram603indicates a frequency (the number of pixels having the corresponding gradation value).

The user can recognize the gradation of the target RAW image in the partial area corresponding to the correspondence area501with high accuracy, by checking the R-histogram601, the G-histogram602, and the B-histogram603. The screen illustrated inFIG. 6is a screen in a case where the correspondence area501illustrated inFIG. 5is designated. In this case, the frequency reaches an upper limit in a gradation range of gradation values, in all of the R-histogram601, the G-histogram602, and the B-histogram603. Therefore, the user can determine that the whiteout occurring in the cloud507of the substitute JPEG image500is also present in the RAW image. Similarly, the user can determine whether the whiteout or blackout of the person506is occurring in the target RAW image, by designating the target area504and checking the RAW histogram of the designated target area504.

As described above, according to the present exemplary embodiment, not only the substitute JPEG image but also the RAW histogram can be displayed. The user who checks the substitute JPEG image can thereby recognize the gradation of the RAW image with high accuracy. In addition, according to the present exemplary embodiment, the RAW histogram of the partial area is used. Therefore, in a case where a permissible part in which the whiteout and blackout of the RAW image are permissible and a non-permissible part in which the whiteout and blackout of the RAW image are not permissible are present, the RAW histogram of the partial area corresponding to the non-permissible part can be displayed. As a result, the user can determine whether the whiteout or blackout of the RAW image is occurring in the non-permissible part, with high accuracy. Further, according to the present exemplary embodiment, the plurality of RAW histograms corresponding to the respective partial areas can be individually displayed. The user can thereby determine whether the whiteout or blackout of the RAW image is occurring in a plurality of objects, individually. In general, in the display of a JPEG image, a processing load is smaller and processing time is shorter than in the display of a RAW image. According to the present exemplary embodiment, since the JPEG image is displayed in substitution for the RAW image, a reduction in time before starting image display as well as battery saving can be expected.

The present disclosure is described in detail above based on the exemplary embodiment, but the present disclosure is not limited to this specific exemplary embodiment, and various forms in a range not deviating from the subject matter of the present disclosure are also included in the present disclosure.

For example, the first electronic apparatus is not limited to the image capturing apparatus. Any type of apparatus can be used as the first electronic apparatus as long as the apparatus can read out a captured image. For example, an information processing apparatus such as a personal computer that can read out a captured image from a storage medium may be used as the first electronic apparatus.

Any method may be employed to determine the partial area. For example, an area of a predetermined size (a predetermined width and a predetermined height) independent of the RAW image may be determined as the partial area. In this case, as the image size of the RAW image increases, more partial areas are determined as the plurality of partial areas forming the entire area of the RAW image. A number of partial areas independent of the RAW image may be predetermined. In this case, as the image size of the RAW image increases, a larger partial area is determined as each of the partial areas forming the entire area of the RAW image. An area designated by the user may be determined as the partial area. The user may designate the size and the number of partial areas.

The control unit101may determine the partial area based on the data format of the RAW image (the RAW image data). For example, in a case where a RAW-image encoding format is a tile format for encoding data for each of tiles forming the entire area of the RAW image, the control unit101may determine each of the tiles as the partial area. The tile is thereby encoded to generate the RAW histogram of this tile. Therefore, a reduction in processing load (memory saving) can be expected, as compared with a configuration for generating a RAW histogram by encoding an entire RAW image. Moreover, a reduction in processing time can also be expected, because processes such as decoding, generation of the RAW histogram, and transmission of the RAW histogram can be executed in parallel.

The control unit101may determine the partial area, based on information from an external apparatus (a smartphone). For example, this external apparatus outputs information about the partial area, in response to a user operation performed on the external apparatus. Specifically, a user operation is performed to designate at least any one of a partial area, the layout (e.g., two rows and two columns, or three rows and three columns) of a partial area, and the number of partial areas. The external apparatus then outputs information indicating at least any one of the partial area, the layout of the partial area, and the number of partial areas. The control unit101determines the partial area, based on the information output from the external apparatus. The external apparatus (specifically, such as an application to be executed in the external apparatus) may automatically generate information based on a situation as the information about the partial area, and output the generated information.

The example in which the plurality of RAW histograms corresponding to the respective partial areas is described, but the exemplary embodiment is not limited to the example. For example, the control unit201of the display control apparatus200may acquire the RAW histogram for only a part of the plurality of partial areas, in step S401inFIG. 4. Specifically, the control unit201may acquire the RAW histogram for only the partial area where the whiteout or blackout are occurring in the correspondence area. The control unit201may acquire the RAW histogram for only the partial area designated by the user. Specifically, the control unit201may perform control to send a request for only the RAW histogram of the designated partial area to the image capturing apparatus100. The image capturing apparatus100may transmit only the RAW histogram of the designated partial area to the display control apparatus200, in response to the request. This also applies to the process (step S302inFIG. 3) in which the control unit101of the image capturing apparatus100acquires the RAW histogram. Only the partial area with respect to which the RAW histogram is acquired may be determined in step S301inFIG. 3.

Any method may be employed to determine the partial area where the whiteout or blackout is occurring in the correspondence area. For example, the control unit201can detect the partial area where the whiteout or blackout is occurring in the correspondence area from the plurality of partial areas, based on the substitute JPEG image. Information about the partial area where the whiteout or blackout is occurring in the correspondence area may be acquired from the outside of the display control apparatus200. Then, the control unit201may determine the partial area where the whiteout or blackout is occurring in the correspondence area, using the information acquired from the outside. The control unit101of the image capturing apparatus100can also determine the partial area where the whiteout or blackout is occurring in the correspondence area, by a similar method. Because the RAW image is present within the image capturing apparatus100, the control unit101can detect the partial area where the whiteout or blackout is occurring in the correspondence area from the plurality of partial areas, based on the RAW image.

Any color element of the RAW histogram may be employed. For example, colors such as cyan, magenta, and yellow may be used for the color element of the RAW histogram. In step S401inFIG. 4, the control unit201of the display control apparatus200may acquire the plurality of RAW histograms corresponding to the respective color elements, or may acquire the RAW histogram for only a part of the plurality of color elements. For example, the control unit201may acquire only one of the R-histogram601, the G-histogram602, and the B-histogram603illustrated inFIG. 6. The control unit201may acquire only two of the R-histogram601, the G-histogram602, and the B-histogram603. The control unit201may acquire a histogram for a luminance value (a Y value) as the RAW histogram, instead of the histogram for the color element. This also applies to the process (step S302inFIG. 3) in which the control unit101of the image capturing apparatus100acquires the RAW histogram.

The image capturing apparatus100and the display control apparatus200may be integrated. In this case, for example, the processes in step S402to step404inFIG. 4are performed after the processes in step S301and step302inFIG. 3are performed (the processes in step S303and step S401is omitted). In addition, in this configuration, the user can recognize the gradation of the RAW image with high accuracy, by checking the displayed RAW histogram.

The units described throughout the present disclosure are exemplary and/or preferable modules for implementing processes described in the present disclosure. The term “unit”, as used herein, may generally refer to firmware, software, hardware, or other component, such as circuitry or the like, or any combination thereof, that is used to effectuate a purpose. The modules can be hardware units (such as circuitry, firmware, a field programmable gate array, a digital signal processor, an application specific integrated circuit or the like) and/or software modules (such as a computer readable program or the like). The modules for implementing the various steps are not described exhaustively above. However, where there is a step of performing a certain process, there may be a corresponding functional module or unit (implemented by hardware and/or software) for implementing the same process. Technical solutions by all combinations of steps described and units corresponding to these steps are included in the present disclosure.

Other Embodiments

This application claims the benefit of priority from Japanese Patent Application No. 2018-037174, filed Mar. 2, 2018, which is hereby incorporated by reference herein in its entirety.