Patent Description:
In <CIT>, a technique, in which a second gradation conversion curve is generated and gradation conversion of image data is performed by use of the second gradation conversion curve, is described, the second gradation conversion curve resulting from adjustment of a first gradation conversion curve corresponding to a dynamic range of the image data, the adjustment being done so that luminance changes in a bright portion and a dark portion of the image data become visually equivalent to each other. In <CIT>, a signal conversion device having a converting unit and a calculating unit is described, the converting unit including a control function having hybrid log gamma characteristics, the calculating unit calculating a parameter value for adjusting a coefficient of a control function that converts an input signal of a bright area set beforehand.

<CIT> discloses a video brightness conversion apparatus including: frame-by-frame gain calculating means for converting an HLG video signal into a linear signal and calculating the gain for each frame; and gain adjusting means for performing gain adjustment. The frame-by-frame gain calculating means includes: current frame brightness calculating means for calculating the brightness level of the current frame of the linear signal; gain calculating means for calculating a gain according to a level difference between the brightness level of the current frame and the brightness level of the past frame stored in past frame brightness storage means; and past frame brightness calculation means for calculating the brightness level of the past frame stored in the past frame brightness storage means from the gain and the brightness level of the current frame.

High Dynamic Range (HDR) has less whiteouts, for example, and is thus higher in image quality than Standard Dynamic Range (SDR). There is a demand for even less whiteouts, for example, and thus even higher image quality for HDR of the hybrid log gamma type.

The present disclosure is made in view of the above and provides an imaging control device, an imaging control method, and a program that achieve higher image quality.

According to the present disclosure, an effect of achieving higher image quality is exerted.

An embodiment of an imaging control device, an imaging control method, and a program according to the present disclosure is hereinafter described in detail by reference to the appended drawings. The present invention is not limited by the following embodiment. An imaging system converts imaging data to a video signal and outputs the video signal, the conversion using an HLG curve according to a representative value related to luminance values in the imaging data. With respect to the following embodiment, a vehicle recording device <NUM> will be described as an example of the imaging system.

<FIG> is a block diagram illustrating an example of a configuration of the vehicle recording device <NUM> according to the embodiment. The vehicle recording device <NUM> is, for example, a so-called driving recorder. In addition to being a device that is installed in a vehicle, the vehicle recording device <NUM> may be a device that is portable and able to be used in a vehicle. The vehicle recording device <NUM> includes a camera (imaging unit) <NUM> and an imaging control device <NUM>.

The camera <NUM> is arranged to capture an image in at least one of a traveling direction of the vehicle or a direction opposite to the traveling direction. With respect to this embodiment, the camera <NUM> is described as a camera that captures an image in the traveling direction of the vehicle. With respect to this embodiment, the camera <NUM> is described by use of a single camera oriented in one direction that is a forward direction of the vehicle, but the camera <NUM> may be a group of plural cameras. The camera <NUM> is arranged at, for example, a position that is in a front region of the interior of the vehicle and that enables imaging of a region in front of the vehicle. In this embodiment, the camera <NUM> continually captures a video while an accessory power source of the vehicle is on. The camera <NUM> outputs imaging data (image data) that have been captured, to an imaging data obtaining unit <NUM> of the imaging control device <NUM>.

The imaging control device <NUM> is an arithmetic processing device (control device) formed of, for example, a central processing unit (CPU). The imaging control device <NUM> loads a stored program into a memory and executes commands included in the program. The imaging control device <NUM> includes an internal memory not illustrated in the drawings, and the internal memory is used for temporarily storing therein data in the imaging control device <NUM>. The imaging control device <NUM> includes the imaging data obtaining unit <NUM>, a frame setting unit <NUM>, a luminance value obtaining unit <NUM>, a representative value calculating unit <NUM>, and a converting unit <NUM>.

The imaging data obtaining unit <NUM> obtains imaging data capturing therein surroundings of the vehicle. More particularly, the imaging data obtaining unit <NUM> obtains imaging data output by the camera <NUM>. The imaging data are data through which images are output consecutively timewise, the images each being formed of a predetermined number of pixels along horizontal and vertical directions.

The frame setting unit <NUM> sets a predetermined frame (obtainment range) in the imaging data obtained by the imaging data obtaining unit <NUM> and capturing therein the surroundings of the vehicle, the predetermined frame being for selecting and extracting some of pixels on an image (screen) so that the predetermined frame is appropriate as a range of obtainment of luminance values. The position, shape, and size of the frame are set so that the frame becomes appropriate as the range of obtainment of the luminance values from the imaging data. More specifically, the frame setting unit <NUM> sets a frame having predetermined position, shape, and size, on an image forming the imaging data from the camera <NUM> arranged to capture the image in at least one of the traveling direction of the vehicle or the direction opposite to the traveling direction. In other words, a frame for selection and extraction of some of pixels of an image is provided. The position, shape, and size of the frame are set so that the frame becomes appropriate as the range of obtainment of luminance values from the imaging data. With respect to this embodiment, the frame is described to be in an oblong shape rectangular in horizontal and vertical directions, but without being limited to this oblong shape, the frame may be circular or oval, or may have a combination of any of these shapes. According to the description of this embodiment, one frame is set, but without being limited to one frame, plural frames may be set.

The frame set by the frame setting unit <NUM> may include a range in which a road surface is captured in the imaging data, the road surface being in at least one of the traveling direction of the vehicle or the direction opposite to the traveling direction. The position, shape, and size of the frame on the image are set to include the range in which the road surface is captured. In this case, the frame setting unit <NUM> may set the frame in a range including a region lower than a central portion of the imaging data. The range in which the road surface is captured accounts for a predetermined proportion or more of the imaging data, and is suitable as the range where the frame for obtainment of the luminance values is set because road surfaces do not have much change in color.

The frame set by the frame setting unit <NUM> preferably includes, in addition to the road surface, a range in which a surrounding landscape is captured, the surrounding landscape being, for example, the sky, the sea, or a river. The position, shape, and size of the frame on the image are set to include the surrounding landscape, in addition to the road surface. Including the surrounding landscape will include luminance values of a region other than the road surface. More appropriate luminance values are thereby obtained.

The frame setting unit <NUM> may have an image recognizing unit <NUM>. The image recognizing unit <NUM> recognizes imaging data to set the position, shape, and size of the frame. More specifically, the image recognizing unit <NUM> recognizes, for example, a road surface in at least one of the traveling direction of the vehicle or the direction opposite to the traveling direction, in the imaging data. By performing image processing of the imaging data, the image recognizing unit <NUM> may recognize, as the road surface, for example, a portion in a region lower than the central portion of the imaging data, the portion being black or gray in color. The image recognizing unit <NUM> may recognize, as the road surface, a predetermined range on an image in the imaging data, for example, a region lower than the central portion.

By performing image processing of the imaging data, for example, the image recognizing unit <NUM> may recognize presence or absence of the sun and the position of the sun. In a case where the image recognizing unit <NUM> performs image processing of the imaging data, and the imaging data are bright overall and has a portion that is in a region above the central portion of the imaging data and that is locally brighter than the surroundings; the image recognizing unit <NUM> may recognize that portion as the sun. Furthermore, the image recognizing unit <NUM> may recognize the time of day and weather at the time of imaging from the presence or absence of the sun and position of the sun that have been recognized.

By performing image processing of the imaging data, the image recognizing unit <NUM> may recognize, for example, a portion illuminated by headlights of the vehicle of its own or another vehicle. In a case where the image recognizing unit <NUM> performs image processing of the imaging data, the imaging data is dark overall, and there is a portion brighter than the surroundings, the portion being only in a region lower than the central portion of the imaging data; the image recognizing unit <NUM> may recognize that portion as the portion illuminated by the headlights. In a case where an imaging time saved with the imaging data is night-time and there is a portion brighter than the surroundings, the portion being only in the region lower than the central portion of the imaging data; the image recognizing unit <NUM> may recognize that portion as the portion illuminated by the headlights.

By performing image processing of the imaging data, for example, the image recognizing unit <NUM> may recognize the sky. By performing image processing of the imaging data, the image recognizing unit <NUM> may recognize, as the sky, for example, a portion above the central portion of the imaging data, the portion being blue, white, or gray, in color. The image recognizing unit <NUM> may recognize, as the sky, a predetermined range in the imaging data, the predetermined range being, for example, above the central portion. Furthermore, on the basis of, for example, information on an imaging time saved with the imaging data, the image recognizing unit <NUM> may recognize the night-time sky.

By performing image processing of the imaging data, for example, the image recognizing unit <NUM> may recognize the sea and a river. By performing image processing of the imaging data, the image recognizing unit <NUM> may recognize, as the sea or a river, for example, a portion lower than the central portion of the imaging data, the portion being blue in color.

By performing image processing of the imaging data, for example, the image recognizing unit <NUM> may recognize a surrounding landscape, such as a tunnel.

On the basis of a result of recognition by the image recognizing unit <NUM>, the frame setting unit <NUM> may set a frame having predetermined position, shape and size, on an image of the imaging data.

A frame set by a frame setting unit <NUM> will be described by use of <FIG>. <FIG> is an explanatory diagram illustrating a group of cells resulting from lengthwise and crosswise divisions of imaging data, according to the embodiment. As illustrated in <FIG>, the frame setting unit <NUM> may set a frame having any position, shape, and size by dividing the imaging data into a group of cells of <NUM> cells lengthwise by <NUM> cells crosswise, for example.

<FIG> is an explanatory diagram illustrating an example of a frame set in imaging data at a normal time, according to the embodiment. The normal time refers to, for example, the daytime in fine weather. As illustrated in <FIG>, the frame setting unit <NUM> sets a frame in a range of the imaging data, the range being lower than a central portion of a screen so that the frame includes a road surface more than anything else. The frame is set in a rectangular shape having a size of, for example, about <NUM>% lengthwise and about <NUM>% crosswise of the imaging data.

<FIG> is an explanatory diagram illustrating an example of a frame set in imaging data in the daytime, according to the embodiment. Depending on the time of day, weather, or imaging direction, strong light, such as direct sunlight, may be captured in the daytime and may generate a locally bright portion in the imaging data. Being locally bright corresponds to, for example, pixels having values that are <NUM>% or more of the largest value of luminance (for example, <NUM> that is <NUM>% of the largest value, <NUM>, for eight bits). On the basis of a result of recognition by the image recognizing unit <NUM>, the frame setting unit <NUM> sets a position of the frame by avoiding any locally bright portion in the imaging data. In a case where there are any bright pixels in the frame, the frame setting unit <NUM> moves the position of the frame in a direction opposite to the positions of the bright pixels in the frame. In a case where the imaging data have been divided lengthwise and crosswise into four and there are bright pixels in the upper right, for example, the frame setting unit <NUM> moves the position of the frame to the bottom left. The frame setting unit <NUM> may change and set the size of the frame by avoiding the locally bright portion in the imaging data. In a case where the frame setting unit <NUM> is unable to move the position of the frame by avoiding the bright pixels, the frame setting unit <NUM> may reduce the size of the frame.

<FIG> is an explanatory diagram illustrating an example of a frame set in imaging data in the night-time, according to the embodiment. In the night-time, the direction in which a vehicle travels is bright due to headlights of the vehicle and a peripheral dark portion where light from the headlights does not reach becomes dark. In a case where the frame setting unit <NUM> has set a frame including a range illuminated by the headlights, the peripheral dark portion where the light from the headlights does not reach becomes dark. On the basis of a result of recognition by the image recognizing unit <NUM>, the frame setting unit <NUM> may set a position of the frame such that the frame includes the peripheral dark portion or a night-time sky portion. On the basis of the result of recognition by the image recognizing unit <NUM>, the frame setting unit <NUM> may change and set the size the frame such that the frame includes the peripheral dark portion or the night-time sky portion. The imaging data are able to be determined to be of those in the night time by: the imaging time; or the image recognizing unit <NUM> recognizing whether the headlights are on or off.

<FIG> is an explanatory diagram illustrating an example of a frame set in imaging data that are dark overall, according to the embodiment. In a case where it is dark in whole, like in a tunnel, for example, the size of the frame may be enlarged for obtainment of luminance values from a wider range. The imaging data are able to be determined to be dark overall by: the luminance mean of pixels in the frame; or the number of pixels having luminance values equal to or larger than a predetermined value or equal to or smaller than a predetermined value, for example.

The luminance value obtaining unit <NUM> obtains luminance values of imaging data in the range of the frame. More specifically, the luminance value obtaining unit <NUM> obtains a luminance value of each pixel included in the range of the frame.

The representative value calculating unit <NUM> calculates a representative value from the luminance values obtained by the luminance value obtaining unit <NUM>. For example, the representative value is the median of the luminance values obtained from the imaging data in the range of the frame. The representative value is not necessarily the median of the luminance values, and may be the mode of the luminance values or any of various averages of the luminance values.

The converting unit <NUM> performs gamma conversion of the imaging data obtained by the imaging data obtaining unit <NUM>. More specifically, on the basis of the representative value calculated by the representative value calculating unit <NUM>, the converting unit <NUM> normalizes the imaging data by setting the peak luminance that is the upper limit value of luminance values and performs gamma conversion of the normalized imaging data by means of a hybrid log gamma transfer function according to the peak luminance. The smaller the representative value is, for example, the larger this peak luminance may be set.

In this embodiment, the peak luminance is set in a range of <NUM> or larger and <NUM> or smaller. The larger the peak luminance is, the wider the dynamic range of HDR becomes.

HLG curves representing hybrid log gamma transfer functions according to the peak luminance will be described by use of <FIG>. <FIG> is a diagram for explanation of an HLG curve in a case where the peak luminance is <NUM>. In <FIG>, the horizontal axis represents scene luminance E that has been normalized so that the reference white color in SDR has a luminance level of <NUM>, and the vertical axis represents the signal level E' of an electric signal corresponding to the horizontal axis. The hybrid log gamma transfer function has been adjusted so that when white in SDR has reference luminance of <NUM>, the signal for HDR is at <NUM>.

<FIG> is a diagram for explanation of an HLG curve in a case where the peak luminance is <NUM>. Similarly to <FIG>, in <FIG>, the horizontal axis represents scene luminance E that has been normalized so that the reference white color in SDR has a luminance level of <NUM>, and the vertical axis represents the signal level E' of an electric signal corresponding to the horizontal axis.

<FIG> is a diagram illustrating correlations between normalized scene luminance E and normalized electric signal levels E'. In <FIG>, the horizontal axis represents E/maxE and the vertical axis represents E'/maxE'. <FIG> illustrates HLG curves that have been normalized in ranges of <NUM> or larger and <NUM> or smaller along the horizontal axes and vertical axes of <FIG>. For peak luminance values of <NUM> or larger and <NUM> or smaller, HLG curves normalized similarly are obtained, although these HLG curves are not illustrated in <FIG>. As described above, the HLG curves used in gamma conversion by the converting unit <NUM> are drawn in curves that differ according to the peak luminance.

By using an HLG curve according to the representative value calculated by the representative value calculating unit <NUM>, the converting unit <NUM> converts the imaging data into an image signal and outputs the image signal, the HLG curve being one of HLG curves that have been normalized. More specifically, the converting unit <NUM> converts an image signal that has been normalized, into an image signal, by means of a hybrid log gamma transfer function expressed by Equation <NUM>. Herein, E is scene luminance and has been normalized with a reference white level. Herein, E' is signal level. Herein, γ is signal level corresponding to the reference white level and y = <NUM>. Herein, a, b, and c are constants and are defined for each HLG curve. For example, a = <NUM>, b = <NUM>, and c = <NUM>.

A flow of processing at the imaging control device <NUM> will be described next by use of <FIG> is a flowchart illustrating the flow of the processing at the imaging control device <NUM> according to the embodiment. In this embodiment, while the vehicle recording device <NUM> is in operation, the camera <NUM> is continually imaging and continually executing the processing illustrated in <FIG>. The processing illustrated in <FIG> may be executed per frame or may be executed every few frames or at predetermined time intervals, for example.

Firstly, the imaging control device <NUM> obtains, by means of the imaging data obtaining unit <NUM>, imaging data output by the camera <NUM> (Step S1). The processing by the imaging control device <NUM> proceeds to Step S2 after Step S1.

After the processing at Step S1, the imaging control device <NUM> sets, by means of the frame setting unit <NUM>, a predetermined frame in the imaging data (Step S2). The imaging control device <NUM> sets, by means of the frame setting unit <NUM>, the frame at a predetermined position and having predetermined shape and size, in the imaging data from the camera <NUM>. The position, shape, and size of the frame are set so that the frame becomes appropriate as a range of obtainment of luminance values from the imaging data. The processing by the imaging control device <NUM> proceeds to Step S3 after Step S2.

After the processing at Step S2, the imaging control device <NUM> obtains, by means of the luminance value obtaining unit <NUM>, luminance values from imaging data in the range of the frame (Step S3). The processing by the imaging control device <NUM> proceeds to Step S4 after Step S3.

After the processing at Step S3, the imaging control device <NUM> calculates, by means of the representative value calculating unit <NUM>, a representative value from the luminance values obtained by the luminance value obtaining unit <NUM> (Step S4). The imaging control device <NUM> calculates, by means of the representative value calculating unit <NUM>, as the representative value, the median of the luminance values obtained from the imaging data in the range of the frame, for example. The processing by the imaging control device <NUM> proceeds to Step S5 after Step S4.

After the processing at Step S4, the imaging control device <NUM> normalizes, by means of the converting unit <NUM>, the imaging data by setting an upper limit value of luminance, on the basis of the representative value (Step S5). The processing by the imaging control device <NUM> proceeds to Step S6 after Step S5.

After the processing at Step S5, the imaging control device <NUM> performs, by means of the converting unit <NUM>, gamma conversion of the normalized imaging data by using a hybrid log gamma transfer function corresponding to the peak luminance set at Step S5, and outputs the converted imaging data (Step S6). The processing by the imaging control device <NUM> is ended after Step S6.

As described above, in this embodiment, on the basis of a representative value related to luminance values from imaging data, the imaging data are subjected to gamma conversion by use of a hybrid log gamma transfer function according to the peak luminance of the imaging data, and an image signal is output. This embodiment enables conversion of the imaging data by use of the hybrid log gamma transfer function according to the imaging data. The embodiment enables setting of a dynamic range correspondingly to the imaging data. The embodiment enables more minimization of whiteouts in HDR images of the HLG type than in conventional HDR images. The embodiment enables improvement in image quality of output data.

This embodiment enables a frame to be set on the basis of a result of recognition by the image recognizing unit <NUM>, the frame being suitable for each set of imaging data. The embodiment enables more appropriate calculation of a representative value and thus enables improvement in image quality of output data.

In this embodiment, a frame at a predetermined position and having predetermined shape and size is set in any one of sets of imaging data from the camera <NUM> arranged to capture an image in at least one of a traveling direction of a vehicle or a direction opposite to the traveling direction. This embodiment enables the preferred frame to be set in the imaging data by recognition of the imaging data from the camera <NUM> arranged to capture the image in at least one of the traveling direction of the vehicle or the direction opposite to the traveling direction.

In this embodiment, a frame set by the frame setting unit <NUM> includes a road surface in imaging data, the road surface being in at least one of a traveling direction of a vehicle or a direction opposite to the traveling direction. This embodiment enables the frame including the road surface to be set in the imaging data by recognition of the road surface from the imaging data. The embodiment enables output of output data high in image quality and suitable for driving recorders.

In this embodiment, a frame is set by recognition of: presence or absence of the sun and a position of the sun; a portion illuminated by headlights; the sky; the sea and a river; and/or any surrounding landscape. This embodiment enables an appropriate frame to be set according to the imaging data. The embodiment enables output of output data high in image quality according to the imaging data.

Each component of the vehicle recording device <NUM> are functionally and/or conceptually illustrated in the drawings, and is not necessarily configured physically as illustrated in the drawings. That is, the specific form of each device is not limited to the one illustrated in the drawings, and all or part of each device may be functionally or physically separated or integrated in any units according to the processing load on the device and the use situation of the device.

The configuration of the vehicle recording device <NUM> is implemented as, for example, software, by means of a method that causes the vehicle recording device <NUM> to operate in the vehicle recording device <NUM> or a program loaded into a memory and executed by a computer that causes the vehicle recording device <NUM> to operate. With respect to the embodiment, functional blocks implemented by cooperation of these pieces of hardware or pieces of software have been described above. That is, these functional blocks may be implemented in any of various forms, by hardware only, software only, or a combination of hardware and software.

An example where the present disclosure is applied to the imaging system that is the vehicle recording device <NUM>, such as a driving recorder, has been described above, but the present disclosure is not limited to this example. The imaging system is also applicable to, for example, a video camera, a smartphone, or broadcasting equipment.

Claim 1:
An imaging processing device (<NUM>), comprising:
an imaging data obtaining unit (<NUM>) configured to obtain imaging data;
a frame setting unit (<NUM>) configured to set a predetermined frame on an image, that is formed of a predetermined number of pixels along horizontal and vertical directions, in the imaging data obtained by the imaging data obtaining unit (<NUM>) as a range for obtaining the luminance value of the imaging data, the frame having an oblong shape;
a luminance value obtaining unit (<NUM>) configured to obtain luminance values of pixels included in the predetermined frame set by the frame setting unit (<NUM>);
a representative value calculating unit (<NUM>) configured to calculate a representative value from the luminance values obtained by the luminance value obtaining unit (<NUM>), the representative value being a median of the luminance values obtained; and
a converting unit (<NUM>) configured to perform gamma conversion of the imaging data obtained by the imaging data obtaining unit (<NUM>) and to output the converted imaging data, wherein
the converting unit (<NUM>) is configured to normalize the imaging data by setting an upper limit value of luminance values on the basis of the representative value calculated by the representative value calculating unit (<NUM>) and is configured to perform, by means of a hybrid log gamma transfer function according to the upper limit value of luminance values, the gamma conversion of the imaging data that have been normalized.