Patent Description:
In general, the dynamic range of an image sensor used for a digital camera and other apparatuses is known to be narrower than the dynamic range in the natural world. For this reason, a method of expanding the dynamic range of an image sensor has been studied. <CIT> discusses a technique for expanding the dynamic range of an imaging sensor by determining an exposure time for each pixel from information obtained from a preliminary image capturing and then performing a main image capturing.

In <CIT>, a gain calculation is performed on each region of an exposure image, based on an exposure amount map, to perform correction processing on an image. However, in a case where such exposure correction processing on an image is performed outside the imaging apparatus using the exposure information for each region generated in the imaging apparatus, following issues arise. In a case where exposure information for an entire region is obtained after image data for an exposure image is obtained, since the correction processing cannot be started until the exposure information is obtained, all the image data of the exposure image has to be once held. Accordingly, a large capacity buffer is necessary. Since the exposure information for each region is obtained after the exposure image, the correction processing for the image data is delayed for a time period corresponding to the delay time caused thereby. As a result, delay occurs until the desired image data is obtained from the imaging result. This is not suitable for a use, for example, in monitoring for which a premium is placed on real time responsiveness. <CIT> discloses an imaging apparatus according to the precharacterising portion of claim <NUM>. <CIT>, <CIT> and <CIT> all disclose imaging apparatuses in which an exposure setting map is output before processing the corresponding image region data.

The present invention is directed to a technique for obtaining a high dynamic range image with a less delay without using a large capacity buffer, in a case where an exposure image, and exposure information for each region are output outside and exposure correction processing is performed outside.

According to a first aspect of the present invention, there is provided an imaging apparatus as specified in claims <NUM> to <NUM>. According to a second aspect of the present invention, there is provided an image processing system as specified in claims <NUM> and <NUM>. According to a third aspect of the present invention, there is provided a control method for an imaging apparatus as specified in claim <NUM>.

Hereinbelow, embodiments of the present invention will be described with reference to the accompanying drawings. Configurations described in the following embodiments are merely examples and the present invention is not limited thereto. The same components and processes are described with the same reference numerals assigned.

<FIG> is block diagram illustrating a schematic configuration of an imaging apparatus <NUM> according to a first embodiment.

While the imaging apparatus <NUM> according to the present embodiment includes various components which typical imaging apparatuses have, only main components of the imaging apparatus <NUM> according to the present embodiment are illustrated in <FIG>, to simplify the drawings and descriptions. The components of the imaging apparatus <NUM> will be schematically described, starting from an image sensor unit <NUM>.

The image sensor unit <NUM>, the imaging region of which includes a plurality of regions, is configured to be capable of being driven dividedly for each region, and has a function of performing an exposure operation (accumulation of charge) with a different exposure time for each region. Each region is referred to as a pixel block. Details of the pixel block will be described below. In the case of the present embodiment, an exposure time of the image sensor unit <NUM> is set for each region by an exposure control signal <NUM> supplied from an exposure time control unit <NUM> (described below), and the exposure is performed with the exposure time set for each region. The exposure control signal <NUM> is a signal for setting an exposure time for each region of the image sensor unit <NUM>. The image sensor unit <NUM> reads out, as a pixel potential <NUM>, the charge accumulated in each pixel in the exposure time controlled for each region by the exposure control signal <NUM>. The pixel potential <NUM> is then output to an analog to digital (A/D) conversion unit <NUM>.

The A/D conversion unit <NUM> performs A/D conversion of the pixel potential <NUM> read out from the image sensor unit <NUM> to convert the analog signal into a digital value. In the present embodiment, an analog gain <NUM> corresponding to each region is set in the A/D conversion unit <NUM> by a gain control unit <NUM>. The A/D conversion unit <NUM> applies the analog gain <NUM> for each region to the corresponding pixel potential <NUM> output from the image sensor unit <NUM>, and then converts the pixel potential <NUM> into a digital value. Hereinbelow, an image formed by the digital signal generated by being subjected to the A/D conversion with the analog gain <NUM> applied to the A/D conversion unit <NUM> for each region, is referred to as an exposure image <NUM>. The exposure image <NUM> output from the A/D conversion unit <NUM> is transmitted to an exposure condition calculation unit <NUM> and a data superimposition unit <NUM>.

The exposure condition calculation unit <NUM> calculates, based on the exposure image <NUM>, an exposure time <NUM> and an analog gain value <NUM> for each region so that the imaging condition is optimized, and updates the imaging condition. The value of the exposure time <NUM> for each region is transmitted to the exposure time control unit <NUM>, and the analog gain value <NUM> for each corresponding region is transmitted to the gain control unit <NUM>. The exposure time <NUM> and the analog gain value <NUM> for each region are transmitted also to the data superimposition unit <NUM>.

A synchronization control unit <NUM> generates an exposure time output pulse <NUM> and a gain output pulse <NUM> synchronized with each other, outputs the exposure time output pulse <NUM> to the exposure time control unit <NUM>, and outputs the gain output pulse <NUM> to the gain control unit <NUM>.

The exposure time control unit <NUM> generates the exposure control signal <NUM> for each region based on the exposure time output pulse <NUM> and a value of the exposure time <NUM> for the corresponding region, and outputs the generated exposure control signal <NUM> to the image sensor unit <NUM>. In this way, the exposure time corresponding to the exposure time <NUM> for each region is set to the image sensor unit <NUM>.

The gain control unit <NUM> generates the analog gain <NUM> for each region to be applied to the pixel potential <NUM> of the corresponding region of the image sensor unit <NUM>, based on the gain output pulse <NUM> and the analog gain value <NUM> for the corresponding region, and outputs the generated analog gain <NUM> for each region to the A/D conversion unit <NUM>. In this way, the A/D conversion unit <NUM> performs A/D conversion after the analog gain <NUM> for each region is applied to the pixel potential <NUM> of the corresponding region. The data having been subjected to the A/D conversion is transmitted to the exposure condition calculation unit <NUM> and the data superimposition unit <NUM> as the exposure image <NUM> for each region.

The data superimposition unit <NUM> receives the exposure time <NUM> and the analog gain value <NUM> for each region and performs packing of the exposure time <NUM> and the analog gain value <NUM> as exposure information. The data superimposition unit <NUM> subsequently outputs the exposure information and the exposure image <NUM> to an image output unit <NUM> in an appropriate order in which an order relationship with the exposure image <NUM> is reflected.

The image output unit <NUM> receives the exposure information and the exposure image <NUM> from the data superimposition unit <NUM>, and outputs the exposure information and the exposure image <NUM> to an external source of the imaging apparatus <NUM>. In the example of the present embodiment, connected to the imaging apparatus <NUM> is a controller <NUM> serving as a processing module for receiving the image data from the imaging apparatus <NUM>. In the present embodiment, a signal line connecting the image output unit <NUM> of the imaging apparatus <NUM> and the controller <NUM> is a low voltage differential signaling (LVDS) signal line having <NUM> data channels. The type and the data channel width of the signal line are not limited thereto. In the present embodiment, the image output unit <NUM> is merely an example of a first output unit and a second output unit, and the controller <NUM> is merely an example of a processing apparatus.

<FIG> is a diagram illustrating a configuration of the image sensor unit <NUM>.

The imaging region of the image sensor unit <NUM> includes a plurality of pixel blocks <NUM>, and each of the pixel blocks <NUM> includes a plurality of pixels <NUM>. In the example of the present embodiment, the number of pixels in the imaging region of the image sensor unit <NUM> in a width <NUM> direction (horizontal line direction) is <NUM> pixels, and <NUM> pixels in a height <NUM> direction, which corresponds to the number of horizontal lines in a vertical direction <NUM> lines. The number of pixels in the pixel block <NUM> in a width <NUM> direction (horizontal line direction) is <NUM> pixels, and <NUM> pixels in a height <NUM> direction, which corresponds to the number of horizontal lines in a vertical direction <NUM> lines. In this case, the number of pixel blocks <NUM> in the imaging region of the image sensor unit <NUM> is <NUM> in the horizontal direction, and <NUM> in the vertical direction. Each of pixel blocks [<NUM>, <NUM>] to [<NUM>, <NUM>] illustrated in the pixel block <NUM> in <FIG> indicates a position of each pixel block <NUM> in the imaging region, and values described in the brackets, i.e., [ , ], indicate indexes of each pixel block <NUM> in the horizontal and the vertical directions in the imaging region. For example, the pixel block <NUM> at the upper right in the image sensor unit <NUM> in <FIG> is the pixel block [<NUM>, <NUM>]. A group of pixel blocks that can be indicated by a same index in the vertical direction is referred to as a block row (i.e., pixel block row). More specifically, a block row N includes pixel blocks [<NUM>, N] to [<NUM>, N]. For example, a block row <NUM> includes pixel blocks [<NUM>, <NUM>] to [<NUM>, <NUM>]. The sizes of the image sensor unit <NUM> and the pixel block <NUM> (the number of pixels in the vertical and horizontal directions) are not limited to the above-descried example. The shape and the aspect ratio of the pixel <NUM> are not limited to the above-descried example, and, for example, the pixel <NUM> may not be square and may be rectangular. Further, the pixel block <NUM> may include only one pixel <NUM>.

In the present embodiment, each of the pixel blocks <NUM> is a unit for which the exposure time and the analog gain can be controlled.

Herein, the exposure time corresponds to a time period during which charge is accumulated in each pixel (light-sensitive element) of the image sensor unit <NUM> at a time of image capturing. Thus, for example, if a light amount incident on the image sensor unit <NUM> is constant and no pixel saturation occurs, the pixel potential <NUM> is higher as the exposure time is longer (i.e., brighter image can be captured). More specifically, in a case where the incident light amount is constant and the saturation of the pixel is not considered, for example, a brighter image can be obtained when the exposure time is <NUM>/<NUM> second than when the exposure time is <NUM>/<NUM> second.

The analog gain <NUM> is a gain to be applied to the pixel potential <NUM> in the A/D conversion unit <NUM> at a time of image capturing. Thus, the digital pixel value (digital value obtained through the A/D conversion after the gain is applied to the pixel potential <NUM>) output from the A/D conversion unit <NUM> is larger as the value of the analog gain is larger.

Referring bock to <FIG>, the configuration and the operation of the imaging apparatus <NUM> according to the present embodiment will be described.

The exposure time for the image sensor unit <NUM> is controlled in unit of pixel block <NUM> based on the exposure control signal <NUM>, and the image sensor unit <NUM> performs image capturing. The image sensor unit <NUM> outputs the pixel potential <NUM> based on the charge accumulated in each pixel.

The A/D conversion unit <NUM> applies the analog gain <NUM> set for each pixel block of the image sensor unit <NUM> to the corresponding pixel potential <NUM> output from the image sensor unit <NUM>, and then performs digital conversion of the pixel potential <NUM> to output the exposure image <NUM>. In the present embodiment, assume that the exposure image <NUM> is expressed by digital values of <NUM> bits. The analog gain <NUM> can take any of four gain values of <NUM> time, <NUM> times, <NUM> times, and <NUM> times, for example.

Next, the exposure time <NUM> and the analog gain value <NUM> will be described with reference to <FIG>, <FIG>, and <FIG>.

With reference to <FIG>, the exposure time <NUM> set for each pixel block <NUM> will be described.

The exposure time <NUM> includes exposure time IDs, values of exposure times (second), and exposure correction coefficients as illustrated in <FIG>. The exposure time ID is an index indicating an exposure time (second). The exposure time IDs [<NUM>, <NUM>] to [<NUM>, <NUM>] illustrated in the pixel blocks <NUM> in <FIG> indicate the exposure time IDs of the pixel blocks [<NUM>, <NUM>] to [<NUM>, <NUM>] illustrated in <FIG>, respectively. In the case of the present embodiment, each index value of the exposure time ID is any one of values of <NUM> to <NUM>. In the example of <FIG>, a case where the index value of the exposure time ID [<NUM>, <NUM>] for the pixel block [<NUM>, <NUM>] on upper right in the imaging region is four, is illustrated. The actual exposure time (second) and the exposure correction coefficient corresponding to each exposure time ID will be described below with reference to <FIG>.

Next, with reference to <FIG>, the analog gain value <NUM> will be described. As illustrated in <FIG>, the analog gain value <NUM> includes gain IDs, values of analog gains, and gain correction coefficients. Each of the gain IDs is an index indicating an analog gain. The gain IDs [<NUM>, <NUM>] to [<NUM>, <NUM>] illustrated in the pixel blocks <NUM> in <FIG> indicate the gain IDs for the pixel blocks [<NUM>, <NUM>] to [<NUM>, <NUM>] illustrated in <FIG>, respectively. In the case of the present embodiment, each index value of the gain ID is any one of <NUM> to <NUM>. In the example of <FIG>, a case where the index value of the gain ID [<NUM>, <NUM>] for the pixel block [<NUM>, <NUM>] on upper right in the imaging region is two, is illustrated. The actual analog gain and the gain correction coefficient corresponding to each gain ID will be described below with reference to <FIG>.

Next, with reference to <FIG>, each of the exposure time IDs, the corresponding exposure time, and the exposure correction coefficient will be described. As described above, the exposure time ID takes a value from <NUM> to <NUM> as an index value. The index value <NUM> of the exposure time ID corresponds to <NUM>/<NUM> second of the exposure time [s]. Similarly, the index value <NUM> of the exposure time ID corresponds to <NUM>/<NUM> second of the exposure time [s], the index value <NUM> corresponds to <NUM>/<NUM> second, the index value <NUM> corresponds to <NUM>/<NUM> second, and the index value <NUM> corresponds to <NUM>/<NUM> second. The exposure time is one of parameters related to the imaging condition, and in the present embodiment, the index value of the exposure time ID that is a condition under which the brightest image is obtainable is set to <NUM>. If a light amount incident on the image sensor unit <NUM> is constant and no pixel saturation occurs, when the exposure time changes from <NUM>/<NUM> second to <NUM>/<NUM> second, the brightness at the time of image capturing is <NUM> time to <NUM>/<NUM> times with the exposure time <NUM>/<NUM> second as a reference, with which the brightest image is obtainable. For example, the brightness at a time of image capturing when the exposure time is <NUM>/<NUM> second corresponding to the index value <NUM> of the exposure time ID becomes <NUM>/<NUM> times (= (<NUM>/<NUM>) second ÷ (<NUM>/<NUM>) second) compared with that when the exposure time is <NUM>/<NUM> second corresponding to the index value <NUM> of the exposure time ID.

The exposure correction coefficient is used to match levels of pixel values with each other in a case where the image capturing is performed with each exposure time corresponding to the respective exposure time ID, as described above. In the case of the present embodiment, the exposure correction coefficient is used so as to match the level of each pixel value at a corresponding exposure time (any one of <NUM>/<NUM> second to <NUM>/<NUM> second) to a level of the pixel value in a case where the image capturing is performed with the exposure time <NUM>/<NUM> second at which the brightest image is obtainable, as the reference. Thus, an inverse number of the ratio of the brightness at the image capturing is used as the exposure correction coefficient. As described above, with the exposure time <NUM>/<NUM> second at which the brightest image is obtainable as the reference, since the brightness obtained with the exposure time <NUM>/<NUM> second to <NUM>/<NUM> second changes from <NUM> time to <NUM>/<NUM> times, the exposure correction coefficient changes from <NUM> time to <NUM> times, each of which is the inverse number of the brightness illustrated in <FIG>.

Next, with reference to <FIG>, each of the gain IDs, the corresponding analog gain, and the gain correction coefficient will be described. As described above, the gain ID takes a value from <NUM> to <NUM> as an index value. The index value <NUM> of the gain ID corresponds to the analog gain of <NUM> times. Similarly, the index value <NUM> of the gain ID corresponds to the analog gain of <NUM> times, the index value <NUM> corresponds to the analog gain of <NUM> times, and the index value <NUM> corresponds to the analog gain of <NUM> time. The analog gain is one of the parameters related to the imaging condition as in the exposure time described above, and in the present embodiment, the index value of the gain ID that is a condition to be able to obtain the brightest image is set to <NUM>.

As described above, the gain correction coefficient is used to match the levels of pixel values with each other when each analog gain corresponding to the gain ID is applied. In the case of the present embodiment, the gain correction coefficient is set so that the level of each pixel value is matched with the corresponding analog gain (any one of <NUM> times to <NUM> time) with the level of the pixel value as a reference, in a case where the analog gain is <NUM> times with which the brightest image is obtainable. Thus, as illustrated in <FIG>, the gain correction coefficients are from <NUM> time to <NUM> times with respect to the analog gains <NUM> times to <NUM> time, which are inverse to each other.

Next, with reference to <FIG>, a combination of an exposure time (second) and an exposure correction coefficient corresponding to each exposure time ID, and an analog gain and a gain correction coefficient corresponding to each gain ID will be described.

The exposure time and the analog gain are the parameters related to the imaging condition as described above, and in the case of the present embodiment, each of the index values for the exposure time ID and the gain ID that is a condition under which the brightest image is obtainable is set to zero. Thus, for example, a combination of the index value zero of the exposure time ID (exposure time <NUM>/<NUM> second) and the index value zero of the gain ID (analog gain <NUM> times) indicated by A in <FIG> is a condition under which the brightest image is captured. Hereinbelow, the setting of the imaging condition with this combination is referred to as an imaging condition setting A.

On the other hand, a combination of the largest index values of the exposure time ID and the gain ID is a condition under which the darkest image is captured, which is indicated by C in <FIG>. Hereinbelow, the setting of the imaging condition with this combination is referred to as an imaging condition setting C. Each combination of the analog gain and the exposure time is merely an example and it is not limited thereto. A combination example indicated by B in <FIG> (referred to as an imaging condition setting B) will be described below.

The image output unit <NUM> illustrated in <FIG> outputs the exposure image <NUM> for each region. The image is output with the brightness based on the exposure information for the corresponding region. The exposure information for each region is the combination of the exposure time ID and the gain ID described in conjunction with <FIG> and <FIG>. Thus, the controller <NUM> that has received the exposure image <NUM> needs to correct the exposure condition for each region to generate an image desired by the user. For example, in a case where the user wants an image in which the brightness smoothly changes in the entire captured image region, exposure correction processing for each region is to be performed on the entire image region of the captured image using the exposure information corresponding to the region. Hereinbelow, the method of performing the exposure correction processing for each region by the controller <NUM> in such a case will be described in detail using, as an example, the imaging condition settings A, B, and C illustrated in <FIG>, with reference to <FIG>, <FIG>, and <FIG>.

With reference to <FIG>, a description will be provided of exposure correction processing for each region in a case of the setting with which the brightest image is captured (imaging condition setting A in <FIG>).

<FIG> is a diagram illustrating a brightness (illuminance) of an object, a pixel potential, an exposure image, a gain corrected image, an exposure corrected image, and a gradation extended image, on respective axes each indicating a brightness direction. <FIG> illustrates processes from when an image is captured to when the gradation extended image is output. The gain corrected image, the exposure corrected image, and the gradation extended image are obtained through the processing of the controller <NUM> side. In the example of <FIG>, the setting for capturing the brightest image of the object (imaging condition setting A in <FIG>) is set as a reference setting, and in this setting, each axis is normalized with the lowest illuminance value (indicated by circles in <FIG>) and the highest illuminance value (indicated by triangles in <FIG>) as references. The values on the axes for the object, the pixel potential, and the exposure image are different in unit. However, in order to simplify the description in a case where the settings are changed in <FIG> and <FIG> described below, values corresponding to the lowest illuminance and the highest illuminance are illustrated to align in the horizontal direction in <FIG>.

Hereinbelow, a description will be provided of shifts of each value in the processes from when the image of the object is captured to when the gradation extended image is output.

As described above, the imaging condition setting A illustrated in <FIG> is the combination for capturing a brightest image. In the case of the imaging condition setting A, the image sensor unit <NUM> captures an image of an object with the exposure time of <NUM>/<NUM> second, and the A/D conversion unit <NUM> applies the gain of <NUM> times to the pixel potential output from the image sensor unit <NUM> to perform A/D conversion. In the following descriptions, the brightness with which an image can be captured with the imaging condition setting A that is a condition to obtain the brightest image is referred to as a "reference brightness". The exposure image obtained by performing A/D conversion on the pixel potential is of <NUM> bit digital value as described above.

The A/D conversion unit <NUM> performs A/D conversion using the analog gain of <NUM> times according to the above-described imaging condition setting A. Since the gain correction coefficient in a case where the analog gain of <NUM> times is applied is one, the gain corrected image in the controller <NUM> becomes an image to which the gain correction coefficient of <NUM> time is applied. In the example of <FIG>, the image sensor unit <NUM> performs image capturing with the exposure time <NUM>/<NUM> second according to the imaging condition setting A. Since the exposure correction coefficient is one in a case where the exposure time is <NUM>/<NUM> second, the exposure corrected image in the controller <NUM> is an image to which the exposure correction coefficient (<NUM> time) is applied.

The exposure image is obtained by capturing an image for each region of the image sensor unit <NUM> with the combination of various imaging conditions illustrated in <FIG> described above. Thus, the controller <NUM> matches the levels of the pixel values of the images for respective regions with each other. However, to match the levels of the regions of the image with each other, <NUM> bits corresponding to each exposure time described above are further necessary in addition to the number of bits (<NUM> bits) for the exposure image, and <NUM> bits corresponding to each analog gain is further necessary. More specifically, as illustrated in <FIG>, since the exposure time ranges from <NUM>/<NUM> second to <NUM>/<NUM> second, for example, in order to match the brightness of an image captured with the exposure time <NUM>/<NUM> second to the reference brightness of the image captured with the exposure time <NUM>/<NUM> second, the pixel value needs to be increased <NUM> times. This corresponds to +<NUM> bits (<NUM> = <NUM><NUM>). Similarly, as illustrated in <FIG>, since the analog gain has a width of <NUM> times to <NUM> time, for example, in order to match the brightness obtained with the analog gain of <NUM> time to the reference brightness with the analog gain of <NUM> times, the pixel value needs to be increased <NUM> times. This corresponds to +<NUM> bits (<NUM> = <NUM><NUM>). Thus, the controller <NUM> generates a gradation extended image of <NUM> bits (= <NUM> + <NUM> + <NUM>) obtained by performing bit extension processing on the exposure corrected image (<NUM> bits) to match the levels of the regions with each other.

As in the example of <FIG>, it can be understood that the processing from capturing the object image to outputting the gradation extended image in the setting for capturing the brightest image (imaging condition setting A) corresponds to the processing of mapping the dark side of the object onto the lower bit side of the gradation extended image, and is suitable for capturing the object in the dark area.

Next, with reference to <FIG>, an example of a case where the exposure time is <NUM>/<NUM> second and the analog gain is <NUM> times (i.e., imaging condition setting B in <FIG>) will be described. <FIG> is illustrated in a manner similar to <FIG>.

In <FIG>, the exposure time of <NUM>/<NUM> second is a time corresponding to <NUM>/<NUM> times of the reference exposure time (<NUM>/<NUM> second) described above. Accordingly, in a case where the brightness (illuminance) of the object is the same as that when the image capturing is performed with the reference exposure time (<NUM>/<NUM> second), the pixel potential obtained in a case where the object is captured with the exposure time of <NUM>/<NUM> second becomes <NUM>/<NUM> times of the pixel potential in a case where the object is captured with the reference exposure time (<NUM>/<NUM> second). In the imaging condition setting B, the analog gain is <NUM> times, which is the gain of <NUM>/<NUM> times of the reference analog gain (<NUM> times). Accordingly, the exposure image obtained in a case where the analog gain is <NUM> times becomes an image with the <NUM>/<NUM> times level of the exposure image obtained in the case of the reference analog gain (<NUM> times). As a result, in the case of the imaging condition setting B illustrated in <FIG>, the value of the object on the bright side is mapped into the <NUM> bits of the exposure image.

Next, the controller <NUM> matches the value of the exposure image for each region with the level of that in the case of the reference imaging condition (exposure time <NUM>/<NUM> second and analog gain <NUM> times). In the case of the imaging condition setting B (exposure time <NUM>/<NUM> second and analog gain <NUM> times), as illustrated in <FIG> described above, the gain correction coefficient is four (= <NUM> times ÷ <NUM> times) and the exposure correction coefficient is <NUM> (= (<NUM>/<NUM>) second ÷ (<NUM>/<NUM>) second). Accordingly, when the gain correction coefficient of four and the exposure correction coefficient of <NUM> are applied to the exposure image, the exposure corrected image is mapped onto the upper bit side by <NUM> bits (<NUM> × <NUM> = <NUM><NUM>) with respect to the exposure image. As a result, the exposure image is mapped from the 6th bit to the 15th bit in the <NUM> bits for the gradation extended image.

In other words, as in the example of <FIG>, in the case of the imaging condition setting B (exposure time <NUM>/<NUM> second and the analog gain <NUM> times), relatively bright side of the object is mapped in the gradation extended image.

Next, with reference to <FIG>, an example case where the exposure time is <NUM>/<NUM> second and the analog gain is <NUM> time (i.e., imaging condition setting C for obtaining the darkest image in <FIG>) will be described. <FIG> is also illustrated in a manner similar to <FIG> and <FIG>.

As described above in conjunction with <FIG>, since the exposure time of <NUM>/<NUM> second is a time of <NUM>/<NUM> times of the reference exposure time <NUM>/<NUM> second, the pixel potential in the case of the exposure time of <NUM>/<NUM> second is <NUM>/<NUM> times of the pixel potential in the case of the reference exposure time of <NUM>/<NUM> second. In the case of the imaging condition setting C, the analog gain is <NUM> time and is the gain of <NUM>/<NUM> times of the reference analog gain of <NUM> times. Accordingly, the exposure image obtained in a case where the analog gain is <NUM> time becomes an image with the level of <NUM>/<NUM> times of the exposure image obtained in the case of the reference analog gain (<NUM> times).

Also in the example of <FIG>, the controller <NUM> matches the value of the exposure image for each region to the level in the case of the reference imaging condition (exposure time <NUM>/<NUM> second and analog gain <NUM> times). In the case of the imaging condition setting C (exposure time <NUM>/<NUM> second and analog gain <NUM> time), as illustrated in <FIG> described above, the gain correction coefficient is eight (= <NUM> times ÷ <NUM> time) and the exposure correction coefficient is <NUM> (= (<NUM>/<NUM>) second ÷ (<NUM>/<NUM>) second).

Thus, as in the example of <FIG>, in the case of the imaging condition setting C (exposure time <NUM>/<NUM> second and analog gain <NUM> time), the bright side of the object is mapped into the most upper bit side (7th to 16th bit) of the gradation extended image.

As described above in conjunction with <FIG>, the controller <NUM> in <FIG> performs processing of converting the exposure image <NUM> for each region (<NUM> bits) into the gradation extended image (<NUM> bits).

The controller <NUM> further performs gradation conversion processing such as a gamma conversion based on the bit depth of the image to be used at subsequent stages. However, details are not described herein.

As described in conjunction with <FIG>, in the case where the user wants an image in which brightness smoothly changes in the entire captured image region, the exposure correction processing is to be performed on the entire image region of the captured image for each region using the exposure information for the corresponding region. In this case, the exposure information is to be appropriately applied for the region to the exposure image of the corresponding region. The exposure information to be applied to a certain region to perform correction processing for each region is desirably transmitted to the controller <NUM> before the image data of the entire region to be subjected to the correction processing with the exposure information is transmitted to the controller <NUM>. In this way, the controller <NUM> side is only to store the image data of the region to be corrected with the exposure information. Since the correction processing can be started each time the image data of the region to be corrected with the exposure information becomes completed, the delay of the image to be generated by the controller <NUM> can be reduced. Hereinbelow, the transmission of the exposure information for each region from the imaging apparatus <NUM> to the controller <NUM> side according to the present embodiment will be described with reference to <FIG>.

<FIG> is a diagram illustrating an arrangement order of pieces of the image data when the pieces of the image data are output from the image output unit <NUM> of the imaging apparatus <NUM> to the controller <NUM>. The data superimposition unit <NUM> generates the pieces of image data arranged in the order as illustrated in <FIG>, and transmits the image data to the image output unit <NUM>, and then the image output unit <NUM> transmits the image data to the controller <NUM>. All the transmission data of the image in one frame to be transmitted in synchronization with the vertical synchronization signal is illustrated as frame data <NUM>. The frame data <NUM> includes pieces of pixel block data <NUM> each constituting the exposure image for each region, optical black (OB) region data <NUM>, a blanking region <NUM>, and a synchronization code/identification code <NUM>. Exposure information <NUM> corresponding to the exposure image for each region is located at a position illustrated in <FIG> in each block row by the data superimposition unit <NUM>. The frame data <NUM> is output from the image output unit <NUM> to the controller <NUM> in a raster scan order of left to right, and upper to lower. Hereinbelow, each piece of data will be described.

The pixel block data <NUM> obtained by the exposure with the exposure time controlled for each region is illustrated in <FIG> for each region corresponding to the pixel block <NUM> in <FIG>. For example, the image data captured in the region indicated by the pixel block [<NUM>, <NUM>] in <FIG> is output as the pixel block data [<NUM>, <NUM>].

The OB region data <NUM> is image data corresponding to the OB region, which is shielded from light, of the image sensor unit <NUM>. The OB region data <NUM> is used, for example, for detecting a dark current component of a sensor pixel and detecting an amount of offset overlapped on the image.

The synchronization code/identification code <NUM> is a particular data string attached to indicate a beginning of valid data and/or a position of a top row when the data is transmitted from the image output unit <NUM> to the controller <NUM> through an output signal line. The controller <NUM> analyzes the arrangement order of the pieces of data obtained through the output signal line and starts interpreting the subsequent data as significant data, by detecting the data indicating the beginning of the valid data. The controller <NUM> detects the beginning of the frame data by detecting the identification code indicating the top row, and determines the operation thereafter.

In the blanking region <NUM>, blank data indicating a blank period that does not include meaningful data is output. In the frame data <NUM> obtained by the image output unit <NUM>, the blank data is output for a data portion other than the pixel block data <NUM>, the OB region data <NUM>, and the synchronization code/identification code <NUM>.

The exposure information <NUM> is exposure information used for performing correction processing of the exposure image for each region by the controller <NUM>. In the example illustrated in <FIG>, the exposure information <NUM> is superimposed on the blank period (blanking region) before the pixel block data for each row in the exposure image. More specifically, the exposure information <NUM> is superimposed on the blanking region <NUM> between the synchronization code/identification code <NUM> and the OB region data <NUM>. In <FIG>, the exposure information <NUM> to which one of numerals <NUM> to <NUM> is added is exposure information applied to the pixel blocks included in the corresponding one of block rows <NUM> to <NUM>. The exposure information <NUM> for the entire one frame is not transmitted together, but the exposure information <NUM> is divided to correspond to each block row and arranged. The exposure information <NUM> is superimposed over several pixel rows at the head position of each block row.

In <FIG>, an example of the exposure information <NUM> is illustrated. As described above, in the present embodiment, the signal line connecting the image output unit <NUM> and the controller <NUM> is the LVDS signal line having <NUM> data channels. As described above, the exposure image <NUM> for each region is set to <NUM> bit digital value and the image data for <NUM> pixel is transmitted and received using <NUM> cycles through each data channel. Through each channel, the data is output from the most significant bit (MSB) to the least significant bit (LSB), from bit <NUM> to bit <NUM>. In the case of the present embodiment, the image data for <NUM> pixels is transmitted through <NUM> data channels in parallel.

As illustrated in <FIG>, the exposure information for each region is transmitted using each channel. Through channels <NUM>, <NUM>, <NUM>, and <NUM>, a valid flag <NUM>, which indicates that the exposure information to be transmitted is valid, a pixel block horizontal direction ID <NUM>, and an inverted value <NUM> of the LSB in the pixel block horizontal direction ID <NUM> are packed in <NUM> bits and transmitted. The valid flag <NUM> is set to one in a case where the information to be transmitted is valid (i.e., valid exposure information exists).

Through the channels <NUM>, <NUM>, <NUM>, and <NUM>, a pixel block vertical direction ID <NUM> and an inverted value <NUM> of the LSB in the pixel block vertical direction ID <NUM> are packed and transmitted.

Through the channels <NUM>, <NUM>, <NUM>, and <NUM>, an exposure time ID <NUM> of <NUM> bits, a gain ID <NUM> of <NUM> bits, and an inverted value <NUM> of the LSB in the gain ID <NUM> are packed and transmitted. Through the channels <NUM>, <NUM>, <NUM>, and <NUM>, additional information <NUM> of <NUM> bits and an inverted value <NUM> of the LSB in the additional information <NUM> are packed and transmitted. The additional information <NUM> can include, for example, a value indicating a relationship of information about the current frame with that of a previous frame (whether it is the same data or how the value changes). In <FIG>, other bits are dummy data, and they can be any values of <NUM> or <NUM>. These packings are performed by the data superimposition unit <NUM>.

Each channel has any one of the inverted values <NUM>, <NUM>,<NUM>, and <NUM>, because if all the values of the <NUM> bits become zero, there is a possibility that the exposure information cannot be distinguished from the synchronization code and the like. In this example, since the exposure image <NUM> for each region is of <NUM>-bit digital value, the packing is performed in the format as illustrated in <FIG>. However, the format of packing, for example, each ID data position, is not limited to the example described above. The additional information includes <NUM> bits, but the number of bits is not limited to <NUM> bits.

The arrangement of the exposure information indicated by the exposure information <NUM> in <FIG> will be described taking the exposure information regarding the block row zero and the data therearound as an example, with reference to <FIG> illustrates data sequentially from the left end data beginning with the synchronization code one to the OB region data in the image data arrangement in the 0th row (i.e., 0th pixel row) and the subsequent rows consisting of <NUM> pixel rows, i.e., from 0th row to 99th row. Each of a synchronization code <NUM> (<NUM>), a synchronization code <NUM> (<NUM>), a synchronization code <NUM> (<NUM>), an identification code <NUM>, and an exposure information/blank code <NUM> illustrated in <FIG> corresponds to data transmitted and received in one cycle.

The synchronization code <NUM> (<NUM>), the synchronization code <NUM> (<NUM>), and the synchronization code <NUM> (<NUM>) indicate the beginning of the data row using these three particular data arrangement. The subsequent identification code <NUM> indicates that it is the beginning of a frame row, a block row, or the like. The exposure information/blank code <NUM> is arranged at the position subsequent thereto. The pieces of exposure information corresponding in number to pixel blocks included in the block row (block row <NUM> in this case) are sequentially packed at this position from the top of the block row, from the 0th row of the block row <NUM>.

Here, the pieces of exposure information for the pixel blocks with the horizontal direction IDs <NUM> to <NUM> in the block row <NUM> are arranged in the 0th row of the block row <NUM>. More specifically, the pieces of exposure information for pixel blocks [<NUM>, <NUM>], [<NUM>, <NUM>], [<NUM>, <NUM>], and [<NUM>, <NUM>] are arranged therein. In this case, the exposure information for one pixel block is expressed using data corresponding to <NUM> pixels. Accordingly, for example, the exposure information for the pixel block [<NUM>, <NUM>] is arranged at the position corresponding to the channels <NUM> to <NUM> in the <NUM> data channels, and the exposure information for the pixel block [<NUM>, <NUM>] is arranged at the position corresponding to the channels <NUM> to <NUM>. The exposure information for the pixel block [<NUM>, <NUM>] is arranged at the position corresponding to the channels <NUM> to <NUM>, and the exposure information for the pixel block [<NUM>, <NUM>] is arranged at the position corresponding to the channels <NUM> to <NUM>. Similarly, in the 1st row of the block row <NUM>, the pieces of exposure information for the pixel blocks with the horizontal direction IDs <NUM> to <NUM> in the block row <NUM> are disposed.

In the present embodiment, since the number of pixel blocks included in one block row is <NUM> blocks, the exposure information for each pixel block is included in the 0th row to the 4th row of each block row. In the present embodiment, the respective pieces of exposure information is one illustrated in <FIG>, and the information for the channels <NUM> to <NUM> (CH <NUM> to CH <NUM>) is embedded at a position indicated by [<NUM>, <NUM>] in <FIG>. Similarly, information for channels <NUM> to <NUM> (CH <NUM> to CH <NUM>) illustrated in <FIG> is embedded at a position indicated by [<NUM>, <NUM>] in <FIG>. In the 5th row of the block row and the subsequent rows, no exposure information to be transmitted is present, so that a blank code is embedded therein. Since one block row includes <NUM> pixel rows, a blank code, not the exposure information, is embedded in each of the 5th to 99th rows of the block row. From the beginning (0th row) of the next block row, the pieces of exposure information for the pixel blocks in the next block row are embedded at these positions. In the example of the present embodiment, while the OB region data <NUM> is arranged immediately subsequent to the exposure information, a plurality of pieces of blank data may be present between the exposure information and the OB data.

Next, timing at which the exposure information is generated will be described with reference to <FIG> illustrates the timing at which the exposure information is generated. <FIG> schematically illustrates processing timings regarding the image data of the Nth frame and the (N + <NUM>)th frame. In <FIG>, the horizontal direction indicates a flow of time. The vertical direction corresponds to vertical direction regions in the image.

When a vertical synchronization signal <NUM> is input, the pixel data is sequentially read out from the upper direction pixel (from upper direction in the image) in the image sensor unit <NUM>, in accordance with the pixel reading timing of the Nth frame indicated by a diagonal solid line arrow <NUM> in <FIG>. In the present embodiment, since the rolling shutter method is used, the vertical OB region (VOB) is read out first, and then pixel data for each pixel is sequentially read out from the 0th row to the 99th row in the pixel blocks of the block row <NUM>.

At a time <NUM>, reading of the pixel data of the pixel blocks in the block row <NUM> in the Nth frame is completed. Thus, all the pixel data is read out from the pixel blocks in the block row <NUM>, so that the exposure information to be used for the (N + <NUM>)th frame starts to be generated.

The exposure information generated using the pixel data of the exposure images in the Nth frame, which is one frame previous to the (N + <NUM>)th frame, is the exposure information to be applied to the image capturing of the (N + <NUM>)th frame. Thus, the time <NUM> is a generation start timing of the exposure information using the exposure images in the block row <NUM> in the Nth frame. This means that, with reference to <FIG>, all the pixel data in the block row <NUM> in the exposure image <NUM> is input to the exposure condition calculation unit <NUM>. The exposure condition calculation unit <NUM> generates the exposure information to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame from the exposure image of block row <NUM> in the Nth frame, using a time period <NUM> illustrated in <FIG>.

At a time <NUM>, at which the time period <NUM> has elapsed since the time <NUM>, the exposure condition calculation unit <NUM> completes generating the exposure information using the exposure images of the block row <NUM> in the Nth frame. At the timing of the time <NUM>, the exposure information <NUM> to be applied to the pixel blocks of the block row <NUM> in the next (N + <NUM>)th frame is generated. The exposure information generated based on the Nth frame exposure image is schematically illustrated in such a manner that the number indicating each block row is enclosed by a square in <FIG>. The exposure information generated by the exposure condition calculation unit <NUM> is accumulated, as the exposure time <NUM> and the analog gain value <NUM>, for each region each time the generation is completed.

As understood from <FIG>, the exposure information <NUM> to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame has been generated before the pixel readout timing of the (N + <NUM>)th frame indicated by a diagonal solid line arrow <NUM>. Thus, the data superimposition unit <NUM> can read out in advance the generated exposure information <NUM> before the exposure images of the block row <NUM> in the (N + <NUM>)th frame is received from the A/D conversion unit <NUM>, as indicated by a dotted curved line in <FIG>. Reading out in advance the exposure information <NUM> thus generated enables transmission of the exposure information together with the exposure image <NUM> through the LVDS signal line for transmitting image data as illustrated in <FIG>.

As for other block rows, the exposure condition calculation unit <NUM> generates the exposure information, as in the block row <NUM>, to sequentially accumulate the generated exposure information as the exposure time <NUM> and the analog gain value <NUM> for each region.

<FIG> is block diagram illustrating a schematic configuration of the controller <NUM> according to the first embodiment. The data transmitted from the imaging apparatus <NUM> through the LVDS signal line is received by an image input interface (I/F) <NUM>. The image input I/F <NUM> analyzes the synchronization code and the identification code included in the received signal to detect the beginning of the frame data, and extracts the subsequent input data. In response to detecting the exposure information, the image input I/F <NUM> transmits the detected exposure information to an exposure information holding unit <NUM>. The image input I/F <NUM> transmits the OB region data and the exposure image for each region to a block row buffer <NUM>.

The block row buffer <NUM> holds the OB region data and the exposure image for each region input from the image input I/F <NUM> for the number of pixel rows included in the block row. In the present embodiment, since one block row includes <NUM> rows, the block row buffer <NUM> holds the pixel data for <NUM> rows and then transmits, in the input order, the pixel data to a gain calculation unit <NUM> to subject the pixel data to the exposure correction processing.

The exposure information holding unit <NUM> holds the exposure information input from the image input I/F <NUM>. In the present embodiment, since the exposure information is superimposed in the 0th row to the 4th row of the block row as illustrated in <FIG>, in a case where the vertical OB region includes, for example, <NUM> rows, the exposure information for the block row <NUM> is input to the exposure information holding unit <NUM> with the image data in the subsequent <NUM> rows. As described above, the exposure image for each region to be input to the gain calculation unit <NUM> delays for a time corresponding to the <NUM> pixel rows. Accordingly, the gain calculation unit <NUM> can perform the gain calculation illustrated in <FIG> using the exposure information for the corresponding block row already stored in the exposure information holding unit <NUM>, when the gain calculation unit <NUM> performs processing on the pixel at the beginning position in each block row. The image data having been subjected to the gain calculation is transmitted to an image processing unit <NUM> located at the subsequent stage, and subsequent image processing is performed.

As described above, the imaging apparatus <NUM> is enabled to output the exposure conditions through the image output signal line as the exposure information without waiting for the exposure conditions each for a region becoming completed for one frame, before completing outputting the exposure image for each region captured using the exposure condition applied to the corresponding region. The imaging apparatus <NUM> completes outputting the exposure information for the pixel blocks in the block row before completing outputting the image data of the block row, in the exposure image, to be subjected to the exposure correction processing. In this way, the controller <NUM> is enabled to perform the gain calculation of the exposure image for each received region and the exposure correction processing on the image without having a buffer for holding the image data of one frame and with a little delay. As a result, it is possible to obtain an image with a little delay and a high dynamic range in the image processing system including the imaging apparatus <NUM> and the controller <NUM>.

In the present embodiment, since the correction processing is not performed by the imaging apparatus <NUM> side and the image is output, the controller <NUM> can select the pixel blocks to be used in the subsequent stage processing in performing the correction processing on the image, and can perform the correction processing. In this case, it is possible to implement the correction processing by selectively using, depending on the pixel block, a path <NUM> for bypassing the image data without going through the gain calculation unit <NUM>.

In the above description, as illustrated in <FIG>, the exposure information <NUM> is superimposed in the blank period before the pixel data for each row in the exposure image, more specifically, in the blanking region <NUM> between the synchronization code/identification code <NUM> and the OB region data <NUM>. However, the exposure information only needs to be received by the controller <NUM> side before the image data for one block row to be processed using the exposure information becomes completed on the controller <NUM> side, and the exposure information needs not necessarily be superimposed as illustrated in <FIG>.

For example, as illustrated in <FIG>, the exposure information may be superimposed in the blank period (blanking region) located after the pixel data for each row in the exposure image. In the example illustrated in <FIG>, in frame data <NUM>, exposure information <NUM> is superimposed over several rows from the top row, but the exposure information <NUM> is positioned in a blanking region <NUM> located after pixel block data <NUM>, which is an exposure image for each region. In <FIG>, the exposure information <NUM> to which one of the numerals <NUM> to <NUM> is added is the one applied to the pixel blocks included in the corresponding block row in the block rows <NUM> to <NUM>. The exposure information <NUM> is not provided before the pixel block data <NUM> as in the <FIG>, and data of the OB region data <NUM> is provided immediately after the synchronization code/identification code <NUM>. The exposure information <NUM> to be applied to the block row <NUM> is arranged from the top row at a position subsequent to the pixel block data [<NUM>, <NUM>]. In a similar manner, the exposure information <NUM> to be applied to the block row is disposed at the top row at a position subsequent to the pixel block data of each block row in a sequential order.

Hereinbelow, an alternative configuration not within the scope of the claims will be described with reference to <FIG>. In the first embodiment, the exposure information is transmitted to the controller <NUM> in the superimposed manner on the image data output signal line, but it is also possible to transmit the exposure information to the controller <NUM> using an I/F other than the image data output signal line. Hereinbelow, an example of such a case will be described.

<FIG> is block diagram illustrating a schematic configuration of an imaging apparatus <NUM>. The imaging apparatus <NUM> includes various functions which general imaging apparatuses have, but in <FIG>, only main components are illustrated to simplify the drawings and the descriptions. In <FIG>, common components as those in <FIG> are assigned the same reference number and duplicate descriptions thereof are omitted.

In the imaging apparatus <NUM> illustrated in <FIG>, the exposure condition calculation unit <NUM> calculates the exposure time <NUM> and the analog gain value <NUM> for each region, and transmits the calculated exposure time <NUM> and the analog gain value <NUM> to the exposure time control unit <NUM> and the gain control unit <NUM>, respectively. The exposure information including the value of the exposure time <NUM> and the analog gain value <NUM> for each region is transmitted also to an internal memory <NUM>. The exposure condition calculation unit <NUM> stores the exposure information in the internal memory <NUM>, and instructs an interrupt I/F <NUM>, each time an amount of the stored exposure information reaches a predetermined value, to output an interrupt pulse. In response to receiving the instruction to output the interrupt pulse from the exposure condition calculation unit <NUM>, the interrupt I/F <NUM> outputs an interrupt pulse to an interrupt signal line <NUM>.

A serial input and output (SIO) I/F <NUM> transmits to and receives from a controller <NUM> various types of information through a serial signal line <NUM>. For example, the SIO I/F <NUM> transmit to and receives from the controller <NUM> the exposure information and the readout request therefor through the serial signal line <NUM>. The image output unit <NUM> is an example of a first output unit, and the SIO I/F <NUM> is an example of a second output unit. The interrupt I/F <NUM> is an example of a notification unit, and the controller <NUM> is an example of a processing apparatus.

The internal memory <NUM> stores the exposure information and the related information in a format, for example, as illustrated in <FIG> is a diagram illustrating an example of the storage format of the exposure information and the related information in the internal memory <NUM>. As illustrated <FIG>, the information stored in the internal memory <NUM> has almost the same contents as the information illustrated in <FIG> in the first embodiment. In the present configuration, access to the internal memory <NUM> is performed in <NUM> bit unit. Thus, the exposure information and the like is packed in <NUM> bits and stored in the internal memory <NUM>.

A pixel block horizontal direction ID <NUM> is stored in bit <NUM> to bit <NUM> and a pixel block vertical direction ID <NUM> is stored in bit <NUM> to bit <NUM>. An exposure time ID <NUM> of <NUM> bits is stored in bit <NUM> to bit <NUM>, and a gain ID <NUM> of <NUM> bits is stored in bit <NUM> to bit <NUM>. Additional information <NUM> of <NUM> bits is stored in bit <NUM> to bit <NUM>. A frame number <NUM> of <NUM> bits is stored in bit <NUM> to bit <NUM>. The frame number <NUM> is information to indicate a correspondence relationship with the frame, which is not present in <FIG> and is described in conjunction with the first embodiment. A value of the frame number <NUM> is assigned in such a manner that the frame number increases one by one each time data corresponding to one frame is transmitted, and returns to <NUM> when the frame number reaches a maximum value. It is particularly effective when the exposure information is held in the internal memory <NUM> in a double buffer format.

Timings and the like at which the exposure information is generated and stored in the internal memory <NUM> will be described with reference to <FIG> is a diagram illustrating operation timings of the imaging apparatus <NUM>. Common portions with those illustrated in <FIG> according to the first embodiment are assigned the same reference number in <FIG>, and only the different portions are described.

The exposure condition calculation unit <NUM> generates exposure information to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame from the exposure images of the block row <NUM> in the Nth frame, using the time period <NUM> illustrated in <FIG>. At the time <NUM>, at which the time period <NUM> has elapsed from the time <NUM>, the exposure condition calculation unit <NUM> completes generating the exposure information using the exposure images of the block row <NUM> in the Nth frame. At the timing of the time <NUM>, the exposure information <NUM> to be applied to the pixel blocks of the block row <NUM> in the next (N + <NUM>)th frame is generated. The exposure information generated based on the exposure images in the Nth frame is schematically illustrated in a manner that the number indicating each block row is enclosed by a square in <FIG>. The exposure information generated by the exposure condition calculation unit <NUM> is accumulated as the exposure time <NUM> and the analog gain value <NUM> for each region each time the generation is completed. The exposure information is also stored in the internal memory <NUM>. Then, each time exposure images of a new block row are input, an exposure condition corresponding to the new block row is sequentially generated by the exposure condition calculation unit <NUM> and the exposure information is stored in the internal memory <NUM>.

At a time <NUM>, the exposure condition calculation unit <NUM> completes generating the exposure information to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame to be generated using the exposure images of the block row <NUM> in the Nth frame. At a timing of the time <NUM>, the exposure condition calculation unit <NUM> instructs the interrupt I/F <NUM> to output an interrupt pulse after writing in the internal memory <NUM> the exposure information to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame. In response to receiving the instruction, the interrupt I/F <NUM> output an interrupt pulse to the interrupt signal line <NUM> as indicated by an interrupt signal <NUM> illustrated in <FIG>.

Similarly, at a time <NUM>, the exposure condition calculation unit <NUM> completes generating the exposure information to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame to be generated using the exposure images of the block row <NUM> in the Nth frame. Further, at a time <NUM>, the exposure condition calculation unit <NUM> completes generating the exposure information to be applied to the pixel blocks of the block row <NUM> in the (N + <NUM>)th frame to be generated using the exposure images of the block row <NUM> in the Nth frame. At the timings of the time <NUM> and the time <NUM>, the exposure condition calculation unit <NUM> instructs the interrupt I/F <NUM> to output an interrupt pulse as well. In response to receiving the instruction, the interrupt I/F <NUM> outputs an interrupt pulse to the interrupt signal line <NUM> as indicated by the interrupt signal <NUM>. The operation is similarly performed in the next (N + <NUM>)th frame, and interrupt pulses are generated at timings of a time <NUM>, a time <NUM>, and a time <NUM>.

In this way, the interrupt I/F <NUM> that has received the instruction from the exposure condition calculation unit <NUM> outputs an interrupt pulse to the interrupt signal line <NUM> and notifies the controller <NUM> that the exposure information is generated and stored. In response to detecting the interrupt pulse input through the interrupt signal line <NUM>, the controller <NUM> attempts to read out the exposure information from the internal memory <NUM> in the imaging apparatus <NUM> through the serial signal line <NUM>. The imaging apparatus <NUM> receives a readout request from the controller <NUM> by the SIO I/F <NUM>, and outputs the exposure information from the internal memory <NUM>. In this example, the exposure information is output in a manner divided into three times without waiting for all pieces of the exposure information to be applied to one frame becoming completed.

<FIG> is block diagram illustrating a schematic configuration of the controller <NUM>. In <FIG>, common components as those illustrated in <FIG> are assigned the same reference number and duplicate descriptions thereof are omitted. In response to an interrupt pulse being input from the imaging apparatus <NUM> via an interrupt I/F <NUM>, a control central processing unit (CPU) <NUM> in the controller <NUM> instructs an SIO I/F <NUM> to read out the exposure information from the imaging apparatus <NUM>. The SIO I/F <NUM> that has received the instruction outputs a readout request to read out the exposure information, to the imaging apparatus <NUM>. At this time, the SIO I/F <NUM> outputs the readout request to acquire the exposure information for the block row <NUM> to the block row <NUM> in response to the interrupt pulse input at the timing of the time <NUM> illustrated in <FIG>. Similarly, the SIO I/F <NUM> outputs a readout request to acquire the exposure information for the block row <NUM> to the block row <NUM> in response to the interrupt pulse input at the timing of the time <NUM> illustrated in <FIG>. The SIO I/F <NUM> outputs a readout request to acquire the exposure information for the block row <NUM> to the block row <NUM> in response to the interrupt pulse input at the timing of the time <NUM> illustrated in <FIG>.

The transfer speed of the SIO is set in such a manner that the time to transfer the exposure information for <NUM> block row is shorter than the time to transfer the image data for one block row from the imaging apparatus <NUM> to the controller <NUM>. Which exposure information for the block row is to be obtained in response to which interrupt pulse is determined in advance using the parameter settings through communication of settings between the controller <NUM> and the imaging apparatus <NUM>. The generation timing of the interrupt pulse and the number of pieces of exchanged exposure information accompanying the generation of the interrupt pulse in the example are merely examples.

In this way, the controller <NUM> that has received the exposure information from the imaging apparatus <NUM> sequentially transfers the exposure information from the SIO I/F <NUM> to the exposure information holding unit <NUM>. The gain calculation unit <NUM> reads out from the exposure information holding unit <NUM> the exposure information for the block row corresponding to the position of the pixel data output from the block row buffer <NUM>, and performs the gain calculation.

As described above, the imaging apparatus <NUM> can output the exposure conditions as the exposure information without waiting for the exposure conditions, each for a region, for one frame becoming completed before completion of outputting the exposure image for each region captured using the exposure condition applied to each region. The imaging apparatus <NUM> completes outputting the exposure information for the pixel blocks in the block row before completion of outputting the image data of the block row to be subjected to the exposure correction processing in the exposure image. In this way, the controller <NUM> can perform the gain calculation of the received exposure image for each region and the exposure correction processing on the image, without having a buffer for holding the image data for one frame and with a little delay. As a result, it is possible to obtain an image with a little delay and a high dynamic range in the image processing system including the imaging apparatus <NUM> and the controller <NUM>.

The present invention can be realized by processing of supplying a program for implementing one or more functions of the above-described embodiment to a system or an apparatus via a network or a storage medium, and one or more processors in the system or the apparatus reading and executing the program. The present invention can also be realized by a circuit (e.g., application specific integrated circuits (ASIC)) that can implement one or more functions.

The above-described embodiments are merely examples of the present invention and shall not be construed as limiting the technical range of the present invention. The present invention can be realized in diverse ways so long as it is in accordance with the technological thought or main features of the present invention.

Claim 1:
An imaging apparatus (<NUM>) comprising:
a first output means (<NUM>) configured to output, outside the imaging apparatus (<NUM>), a first image (<NUM>) that an image sensor (<NUM>), an imaging region of which is divided into a plurality of regions, has captured while an exposure condition is controlled for each of the plurality of regions, wherein the first image comprises frame data (<NUM>) including pixel block data (<NUM>) and a blanking region (<NUM>); and
a second output means (<NUM>) configured to output, outside the imaging apparatus (<NUM>), exposure information for each of the plurality of regions, the exposure information (<NUM>) indicating the exposure condition to be applied to the corresponding one of the plurality of regions when the first image (<NUM>) is captured;
wherein the second output means is configured to complete outputting exposure information for a region to be subjected to exposure correction processing in the first image, out of the plurality of regions, before the first output means completes outputting an image of the region to be subjected to exposure correction processing;
characterized by a superimposition means (<NUM>),
wherein the first output means and the second output means are configured as one output means (<NUM>); and
wherein the superimposition means (<NUM>) is configured to insert the exposure information (<NUM>) in the blanking region (<NUM>) before pixel block data (<NUM>) for each row in the first image when the first image is output.