Patent Description:
For example, a viewing system in which a camera that is an image-capturing apparatus is installed in the rear of a vehicle such as an automobile and an image of a region behind the vehicle that is captured using the camera is displayed, is proposed.

<NUM><NUM><NUM><NUM> Examples of the image of a region behind a vehicle that is provided by the viewing system include an image of a region situated further rearward than a region just behind the rear of the vehicle, and an image of the region just behind the rear of the vehicle.

<NUM><NUM><NUM><NUM> Here, the image of a region situated further rearward than a region just behind the rear of a vehicle is, for example, an image that corresponds to an image seen in a Class I mirror, which is a so-called interior rearview mirror, and is hereinafter also referred to as a back mirror (BM) image. Further, the image of a region just behind the rear of a vehicle is an image of the rear of the vehicle and a region just behind the rear of the vehicle, and is hereinafter also referred to as a rearview (RV) image.

<NUM><NUM><NUM><NUM> The specifications, that is, angles of view and optical-axis directions that are necessary to capture a BM image and an RV image are different. Thus, the BM image and the RV image are respectively captured using different cameras. Therefore, in order to display a BM image and an RV image in the viewing system, there is a need to provide, in the rear of a vehicle, two cameras that are a camera used to capture the BM image and a camera used to capture the RV image.

<NUM><NUM><NUM><NUM> In recent years, there is a need to capture a BM image and a RV image using a single camera in order to, for example, reduce costs.

<NUM><NUM><NUM><NUM> For example, Patent Literature <NUM> discloses a camera in which an angle of view and an optical-axis direction are changed by a PTZ (pan, tilt, and zoom) operation being performed. In the camera disclosed in Patent Literature <NUM>, the PTZ operation makes it possible to switch between capturing of a distant image of a range distant from a vehicle and capturing of a nearby image of a range close to the vehicle.

<CIT> discloses a display control apparatus including a memory and a controller. The memory stores a source picture taken by a rear view camera attached to a vehicle. The controller detects a road gradient and calculates a first point on a road that is located at a predetermined distance from the vehicle, based on the detected road gradient. The controller then crops out a partial area of the source picture such that the partial area includes a second point on a straight line between the rear view camera and the first point on the road, and displays a picture of the partial area.

<CIT> discloses a vision system for a vehicle including an exterior structure mounted at an exterior portion of a body side of the equipped vehicle, with an imaging sensor disposed at the exterior structure and having a field of view exterior and sideward and rearward of the vehicle. A video display screen is disposed in an interior cabin of the equipped vehicle and operable to display video images. Responsive to detection of another vehicle at or near the side of the equipped vehicle, said video display screen displays a graphic overlay at the display screen. The graphic overlay includes a semitransparent overlay having a longitudinally extending line segment that extends along a displayed body side portion of the equipped vehicle and a plurality of laterally extending line segments that extend laterally outwardly from the longitudinally extending line segment at spaced apart intervals along the longitudinally extending line segment.

<NUM> The camera disclosed in Patent Literature <NUM> makes it possible to switch between output of a distant image corresponding to a BM image and output of a nearby image corresponding to an RV image. However, it is difficult to simultaneously output the distant image and the nearby image using the camera disclosed in Patent Literature <NUM>.

<NUM> Further, there is a restriction on a vehicle transmission bandwidth that can be used to transmit data in a vehicle. Thus, even if it is possible to simultaneously output a distant image corresponding to a BM image and a nearby image corresponding to an RV image, it will be difficult to simultaneously transmit the distant image and the nearby image to display those images.

<NUM> Although an increase in vehicle transmission bandwidth makes it possible to simultaneously transmit a distant image and a nearby image, the increase in vehicle transmission bandwidth results in increasing the costs for the viewing system.

The present technology has been made in view of the circumstances described above, and is intended to make it possible to suppress an increase in costs.

The claimed invention is set out in the appended claims.

In an image-capturing apparatus, an image processing method, and a program according to the present technology, an extraction control for extracting a first image and a second image from a captured image captured by an image sensor that performs image-capturing is performed, the first image being displayed on a first display section, the second image being displayed on a second display section. Then, data amounts of the first image and the second image are adjusted according to vehicle information acquired from a vehicle.

<NUM> Note that the image-capturing apparatus may be an independent apparatus or an internal block included in a single apparatus.

<NUM> Further, the program can be provided by being transmitted through a transmission medium or by being stored in a recording medium.

<NUM><NUM><NUM><NUM> <FIG> is a perspective view illustrating an example of a configuration of an appearance of a vehicle <NUM> that includes a viewing system to which the present technology is applied.

<NUM><NUM><NUM><NUM> For example, a camera unit <NUM> that is an image-capturing apparatus used to capture an image of a region behind the (four-wheeled) vehicle <NUM> is installed in the rear of the vehicle <NUM>. In <FIG>, the camera unit <NUM> is installed above a rear window of the vehicle <NUM>.

<NUM><NUM><NUM><NUM> The camera unit <NUM> is a wide-angle camera unit such that it is possible to capture an image in which both a range corresponding to a BM image (a first image) and a range corresponding to an RV image (a second image) appear. Further, the camera unit <NUM> is a high-resolution camera unit such that a distant subject is apparent in the BM image. Thus, the camera unit <NUM> is capable of capturing a wide-angle, high-resolution image.

<NUM><NUM><NUM><NUM> Note that, in the camera unit <NUM>, a BM image and an RV image are extracted from an image captured by the camera unit <NUM> (hereinafter also referred to as a captured image). This will be described later.

<NUM><NUM><NUM><NUM> The camera unit <NUM> is installed in a state in which the orientation of the optical axis has been adjusted, such that the BM image includes an image of a state of a region situated further rearward than a region just behind the rear of the vehicle <NUM>, and such that the RV image includes an image of states of the rear of the vehicle <NUM> and the region just behind the rear of the vehicle <NUM>, the state of the region situated further rearward than the region just behind the rear of the vehicle <NUM> being a state that can be observed using an interior rearview mirror (a Class I mirror in Regulation No. <NUM> defined by the United Nations Economic Commission for Europe <UNECE>) when the interior rearview mirror is installed in the vehicle <NUM>.

<NUM><NUM><NUM><NUM> Thus, the BM image is an image of a state of a region situated further rearward than a region just behind the rear of the vehicle <NUM>, the state being a state that can be observed using an interior rearview mirror when the interior rearview mirror is installed in the vehicle <NUM>. Further, the RV image is an image of states of the rear of the vehicle <NUM> and the region just behind the rear of the vehicle <NUM>. The RV image is particularly useful when the vehicle <NUM> is traveling backward, since a region just behind the rear of the vehicle <NUM> that is a blind spot of the interior rearview mirror appears in the RV image. Further, the RV image can be used to generate an overhead image obtained when the vehicle <NUM> is viewed from above.

<NUM><NUM><NUM><NUM> Note that the camera unit <NUM> is not limited to being installed above the rear window of the vehicle <NUM> as long as it is possible to capture a captured image from which the BM image and the RV image described above can be extracted. For example, in addition to being installed above the rear window of the vehicle <NUM>, the camera unit <NUM> may be installed at, for example, a position P11 above a license plate situated in the rear of the vehicle <NUM>.

<NUM><NUM><NUM><NUM> <FIG> is a perspective view illustrating an example of a configuration of the interior of the vehicle <NUM> of <FIG>.

<NUM><NUM><NUM><NUM> A BM display section <NUM> that displays thereon a BM image is provided at a position, in the vehicle <NUM>, at which the interior rearview mirror is installed. The BM display section <NUM> is a display section that is an alternative to the interior rearview mirror.

<NUM><NUM><NUM><NUM> An RV display section <NUM> that displays thereon an RV image is provided at a center position of a dashboard in the vehicle <NUM>.

<NUM><NUM><NUM><NUM> Note that an in-vehicle camera <NUM> used to capture an image of a driver is provided on the side of a driver's seat of the dashboard in the vehicle <NUM>. An image of the driver is captured to be output by the in-vehicle camera <NUM>. In the vehicle <NUM>, positions of the line of sight and the head of the driver are detected from the image of the driver.

<NUM><NUM><NUM> Here, the in-vehicle camera <NUM> used to capture an image of the driver may be provided at any position other than a position on the dashboard, such as a position P21 above the BM display section <NUM>.

<NUM><NUM><NUM><NUM> <FIG> is a block diagram illustrating a first example of a configuration of the viewing system included in the vehicle <NUM>.

<NUM><NUM><NUM><NUM> The viewing system includes the camera unit <NUM>, the BM display section <NUM>, and the RV display section <NUM> described with reference to <FIG> and <FIG>.

<NUM><NUM><NUM><NUM> The camera unit <NUM> includes an optical system <NUM>, an image sensor <NUM>, a data amount adjuster <NUM>, an output section <NUM>, an acquisition section <NUM>, and a controller <NUM>.

<NUM><NUM><NUM><NUM> The optical system <NUM> includes optical components such as a condenser and a diaphragm, and collects light entering the optical system <NUM> onto the image sensor <NUM>.

<NUM><NUM><NUM><NUM> The image sensor <NUM> receives light from the optical system <NUM> and performs photoelectric conversion to capture a captured image. Then, in accordance with control performed by the controller <NUM>, the image sensor <NUM> extracts the BM image and the RV image from the captured image to output the read images. The BM image and the RV image that are output by the image sensor <NUM> is supplied to the data amount adjuster <NUM>.

<NUM><NUM><NUM><NUM> In accordance with control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of the BM image and the RV image that are output by the image sensor <NUM>, and supplies, to the output section <NUM>, the BM image and the RV image of which the respective data amounts have been adjusted.

The output section <NUM> is an output interface (IF) that transmits, to the outside of the camera unit <NUM>, the BM image and the RV image from the data amount adjuster <NUM>. The output section <NUM> transmits the BM image to the BM display section <NUM>, and transmits the RV image to the RV display section <NUM>. The BM image from the output section <NUM> is displayed on the BM display section <NUM> in accordance with the specification of the BM display section <NUM>, and the RV image from the output section <NUM> is displayed on the RV display section <NUM> in accordance with the specification of the RV display section <NUM>. The output section <NUM> is capable of performing a format conversion and other image processing on the BM image and the RV image as necessary.

The acquisition section <NUM> acquires, from the vehicle <NUM>, information that can be acquired from the vehicle <NUM> (hereinafter also referred to as vehicle information), and supplies the acquired vehicle information to the <NUM><NUM><NUM><NUM> controller <NUM>.

Here, examples of the vehicle information include traveling information, the specifications of the BM display section <NUM> and the RV display section <NUM>, positions of the line of sight and the head of a driver of the vehicle <NUM>, and gyroscopic information.

<NUM><NUM><NUM><NUM> The traveling information is information that indicates a traveling state of the vehicle <NUM>, and specifically indicates a vehicle speed and a traveling direction (forward or backward). For example, it is possible to acquire the vehicle speed from output of a speed sensor when the vehicle <NUM> includes the speed sensor. For example, it is possible to acquire the traveling direction from a state of the transmission. <NUM><NUM><NUM><NUM>.

<NUM><NUM><NUM><NUM> For example, the specifications of the BM display section <NUM> and the RV display section <NUM> are the resolution of the BM display section <NUM> and the resolution of the RV display section <NUM>, and can be acquired from the BM display section <NUM> and the RV display section <NUM>.

The positions of the line of sight and the head of a driver of the vehicle <NUM> are obtained from an image captured by the in-vehicle camera <NUM><NUM><NUM><NUM>.

<NUM><NUM><NUM><NUM> The gyroscopic information is information that indicates a pose of the vehicle <NUM> (/ an angle of the inclination of the vehicle). It is possible to obtain the gyroscopic information from output of a gyroscope when the vehicle <NUM> includes the gyroscope. The use of the gyroscopic information makes it possible to recognize whether the vehicle <NUM> is on a hill.

The controller <NUM> controls the image sensor <NUM> and the data amount adjuster <NUM> according to the vehicle information supplied by the acquisition section <NUM>.

<NUM><NUM><NUM><NUM> In other words, for example, the controller <NUM> performs, according to the vehicle information, an extraction control for controlling extraction of a BM image and an RV image from a captured image captured by the image sensor <NUM>. Further, according to the vehicle information, the controller <NUM> performs an adjustment control for controlling adjustment of data amounts of the BM image and the RV image that is performed by the data amount adjuster <NUM>.

<NUM><NUM><NUM><NUM> Thus, it can be said that the image sensor <NUM> extracts a BM image and an RV image from a captured image according to vehicle information, and the data amount adjuster <NUM> adjusts data amounts of the BM image and the RV image according to the vehicle information.

<NUM><NUM><NUM><NUM> <FIG> is a diagram describing an example of a control of extraction of a BM image and an RV image from a captured image, the control being performed by the controller <NUM>.

<NUM><NUM><NUM><NUM> Regarding a positional relationship between the optical system <NUM> and a captured image captured by the image sensor <NUM>, the captured image (a light-receiving surface of the image sensor <NUM>) includes an image circle of (a lens included in) the optical system <NUM>, as illustrated in, for example, <FIG>.

<NUM><NUM><NUM><NUM> In the extraction control, the controller <NUM> controls reading of data (a pixel signal) from the image sensor <NUM> such that a specified region R11 is extracted from a captured image as a BM image, the specified region R11 being a region in which a region situated further rearward than a region just behind the rear of the vehicle <NUM> appears (a region observed using an interior rearview mirror if the interior rearview mirror is installed in the vehicle <NUM>).

<NUM><NUM><NUM><NUM> Further, in the extraction control, the controller <NUM> controls reading of data from the image sensor <NUM> such that a specified region R12 is extracted as an RV image from (an image in the image circle from among) the captured image, the specified region R12 being a region in which the rear of the vehicle <NUM> and the region just behind the rear of the vehicle <NUM> appear.

<NUM><NUM><NUM><NUM> In accordance with control performed by the controller <NUM>, the image sensor <NUM> reads, from a captured image obtained by performing image-capturing, data (a pixel signal) of the region R11 that corresponds to (data of) the BM image, and reads, from the captured image, data of the region R12 that corresponds to the RV image.

<NUM><NUM><NUM><NUM> Note that the sizes of the regions R11 and R12 can be set according to the specifications of the BM display section <NUM> and the RV display section <NUM>.

Further, in the control of extraction of a BM image, the controller <NUM> controls (a position of) the region R11 extracted as the BM image, according to one of positions of the line of sight and the head of a driver, or according to both of them. <NUM><NUM><NUM><NUM>.

In other words, if an interior rearview mirror is installed in the vehicle <NUM>, a range that appears in an image that can be seen by a driver using the interior rearview mirror will be changed as the driver moves his/her line of sight or his/her head. In the control of extraction of a BM image, the controller <NUM> changes a position of the region R11 extracted as the BM image, according to the positions of the line of sight and the head of the driver, such that the driver can see a BM image of a range similar to the range in an image that can be observed when the interior rearview mirror is installed in the vehicle <NUM>.

<NUM><NUM><NUM><NUM> Moreover, in the control of extraction of a BM image, the controller <NUM> can control the region R11 extracted as the BM image, according to gyroscopic information included in vehicle information.

<NUM><NUM><NUM><NUM> <FIG> is a diagram describing the control of extraction of (the region R11 that is) a BM image, the control being performed according to gyroscopic information.

Here, the range of a three-dimensional space in a BM image is referred to as a BM range. Further, it is assumed that positions of the line of sight and the head of a driver are fixed in order to simplify the description. Therefore, it is assumed that (a position of) the region R11 extracted as a BM image from a captured image, and thus a BM range are not changed due to the driver moving the position of his/her line of sight or his/her head.

<NUM><NUM><NUM><NUM> Further, it is assumed that, when the vehicle <NUM> is on a flat road, an object situated rearward a specified distance from the vehicle <NUM> appears in a BM image.

<NUM><NUM><NUM><NUM> In this case, when the vehicle <NUM> is on an uphill, there is no change in BM range, but the optical axis of the camera unit <NUM> installed in the rear of the vehicle <NUM> is downwardly inclined (toward the road), compared to when the vehicle <NUM> is on a flat road.

<NUM><NUM><NUM><NUM> Consequently, an object situated rearward a specified distance from the vehicle <NUM> that is within a BM image and appears in a BM image when the vehicle <NUM> is on a flat road, is not within the BM range and does not appears in the BM image when the vehicle <NUM> is on an uphill. In other words, an object situated rearward a specified distance from the vehicle <NUM> appears in a BM image when the vehicle <NUM> is on a flat road, and does not appear in the BM image when the vehicle <NUM> is on an uphill.

<NUM><NUM><NUM><NUM> In this case, when the vehicle <NUM> is on an uphill, a rearward range, on a road, that can be confirmed by a driver of the vehicle <NUM> using a BM image is smaller, compared to when the vehicle <NUM> is on a flat road.

<NUM><NUM><NUM><NUM> Thus, in the control of extraction of a BM image, the controller <NUM> can control the region R11 extracted as the BM image, according to gyroscopic information included in vehicle information.

<NUM><NUM><NUM><NUM> In other words, according to gyroscopic information, the controller <NUM> can change the position of the region R11 extracted as a BM image to an upper position in a captured image when the front side of the vehicle <NUM> is upwardly inclined.

<NUM><NUM><NUM><NUM> Accordingly, an object situated rearward a specified distance from the vehicle <NUM> appears in a BM image when the vehicle <NUM> is on a flat road. Further, it is possible to prevent the object from not appearing in the BM image when the vehicle <NUM> is on an uphill. Note that, according to gyroscopic information, the controller <NUM> can change the position of the region R11 extracted as a BM image to a lower position in a captured image when the vehicle <NUM> is on a downhill.

<FIG> illustrates an example of a configuration of the image sensor <NUM> of <FIG>.

<NUM><NUM><NUM><NUM> The image sensor <NUM> includes a plurality of pixels <NUM>, an analog-to-digital (AD) converter, and a line memory <NUM>.

<NUM><NUM><NUM><NUM> The plurality of pixels <NUM> is arranged in a two-dimensional plane. In <FIG>, the plurality of pixels <NUM> is arranged in a grid. A vertical signal line (VSL) is routed with respect to each column of the pixels <NUM> arranged in a grid, and the pixels <NUM> in each column are connected to a VSL routed with respect to the column. The pixel <NUM> converts light entering the pixel <NUM> into a pixel signal that is an electrical signal corresponding to an amount of the light. The pixel signal is read from the pixel <NUM> to be output to a VSL routed with respect to a column of that pixel <NUM>.

<NUM><NUM><NUM><NUM> For example, an AD converter <NUM> includes, for each column of the pixels <NUM> arranged in a grid, an AD converter (ADC) (not illustrated) that is responsible for AD conversion performed on a pixel signal of the pixel <NUM> in the column. The ADC for each column is connected to a VSL routed with respect to the column, and performs AD conversion on a pixel signal read from the pixel <NUM> connected to the VSL. The pixel signal on which AD conversion has been performed by the ADC of the AD converter <NUM> for each column is supplied to the line memory <NUM>.

<NUM><NUM><NUM><NUM> The line memory <NUM> stores therein pixel signals of at most the pixels <NUM> included in one row and supplied by the AD converter <NUM>, and sequentially transfers the stored pixel signals in a row direction to output the pixel signals to the outside of the image sensor <NUM>.

<NUM><NUM><NUM><NUM> Here, in <FIG>, the pixel <NUM> in the yth row from the top and in the xth column from the left, and a pixel signal obtained from the pixel <NUM> are also hereinafter respectively represented by a pixel yx and a pixel signal yx as appropriate.

<NUM><NUM><NUM><NUM> A captured image is an image that is made up of pixel signals obtained from all of the pixels (the effective pixels) <NUM> of the light-receiving surface of the image sensor <NUM>. In the image sensor <NUM>, a pixel signal is read for each row in a direction of a lower row from the pixel <NUM> in an uppermost row a when the captured image is output.

<NUM><NUM><NUM><NUM> The pixel signals of the pixels <NUM> in one row are supplied to the AD converter <NUM> through a VSL. In the AD converter <NUM>, AD conversion is performed on the pixel signals of the pixels <NUM> in one row at the same time, and the pixel signal on which AD conversion has been performed is supplied to be stored in the line memory <NUM>.

<NUM><NUM><NUM><NUM> In the line memory <NUM>, pixel signals of the pixels <NUM> in one row that are supplied by the AD converter <NUM> are stored and sequentially transferred in a row direction to be read into the outside of the image sensor <NUM>.

<NUM><NUM><NUM><NUM> The processing described above is performed with respect to all of the rows of the pixels <NUM>, and this results in (pixel signals for) a captured image for a single screen being read into the outside of the image sensor <NUM>.

<NUM><NUM><NUM><NUM> A BM image and an RV image correspond to (images of) regions that are portions of a captured image. Thus, in order to obtain the BM image and the RV image, there is no need to read the entire captured image from the image sensor <NUM>, and it is sufficient if only (pixel signals for) regions that are portions of the captured image that respectively correspond to the BM image and the RV image, are read.

<NUM><NUM><NUM><NUM> Here, reading of an entire captured image from the image sensor <NUM> is also referred to as full reading, and reading of a region that is a portion of the captured image from the image sensor <NUM> is also referred to as partial reading.

<NUM><NUM><NUM><NUM> In the extraction control for controlling reading of data from the image sensor <NUM> such that a BM image and an RV image are extracted from a captured image, the image sensor <NUM> is controlled by the controller <NUM> such that partial reading is performed.

<NUM><NUM><NUM> O For example, when the region R11 corresponding to a BM image is made up of the pixels <NUM> in columns <NUM> to <NUM> from among the pixels <NUM> in rows d to h, as illustrated in <FIG>, the controller <NUM> controls the image sensor <NUM> such that pixel signals of the pixels <NUM> in columns <NUM> to <NUM> from among the pixels <NUM> in rows d to h are read.

<NUM><NUM><NUM><NUM> In this case, in the image sensor <NUM>, the pixel signals of the pixels <NUM> in the row d are read, AD conversion is performed by the AD converter <NUM> with respect to the read pixel signals, and the pixel signals on which AD conversion has been performed are supplied to the line memory <NUM>. Then, the line memory <NUM> stores therein the pixel signals of the pixels <NUM> in the row d, the pixel signals being pixel signals on which AD conversion has been performed. Then, in the line memory <NUM>, pixel signals d1, d2, d3, d4, and d5 of the pixels <NUM> in the rows <NUM> to <NUM> from among the pixels <NUM> in the row d, are sequentially transferred to be read into the outside of the image sensor <NUM>.

<NUM><NUM><NUM><NUM> AD conversion is performed by the AD converter <NUM> with respect to pixel signals of the pixels <NUM> in a next row e while the pixel signals d1 to d5 of the pixels <NUM> in the rows <NUM> to <NUM> from among the pixels <NUM> in the row d are being transferred in the line memory <NUM>. Subsequently, pixel signals of the pixels <NUM> in the columns <NUM> to <NUM> from among the pixels <NUM> in the rows d to h are similarly read into the outside of the image sensor <NUM>.

<NUM><NUM><NUM><NUM> As described above, in the image sensor <NUM>, the partial reading makes it possible to extract a BM image and an RV image from a captured image.

<NUM><NUM><NUM><NUM> Note that, in <FIG>, a column-parallel-based AD conversion approach in which a single ADC is responsible for AD conversion performed on a pixel signal of the pixel <NUM> in a single column, is adopted as an approach for AD conversion performed by the AD converter <NUM>. However, the approach for AD conversion performed by the AD converter <NUM> is not limited to the column-parallel-based AD conversion approach. For example, an area-based AD conversion approach in which a single ADC is responsible for AD conversion performed on a pixel signal of the pixel <NUM> in an area that includes a signal pixel or a plurality of pixels, may be adopted as the approach for AD conversion performed by the AD converter <NUM>.

<NUM><NUM><NUM><NUM> <FIG> illustrates an example of an image that can be output by the image sensor <NUM>.

<NUM><NUM><NUM><NUM> Here, a highest-resolution captured image that can be output by the image sensor <NUM> is referred to as a highest-resolution image. It is assumed that, for example, the image sensor <NUM> has the ability to output a highest-resolution image of a resolution (the number of pixels) Rmax at (a frame rate of) <NUM> fps (frame per second) or more.

<NUM><NUM><NUM><NUM> Here, it is assumed that a resolution RBM for a highest-resolution BM image (a BM image of a largest number of pixels) that is extracted from the highest-resolution image is equal to or less than <NUM>/<NUM> of the resolution Rmax for the highest-resolution image. It is also assumed that a resolution RRV for a highest-resolution RV image that is extracted from the highest-resolution image is equal to or less than the resolution RBM for the BM image.

<NUM><NUM><NUM><NUM> In the present embodiment, it is assumed that, for example, a sum RBM+RRV of the resolution RBM for a BM image and the resolution RRV for a RV image is equal to or less than <NUM>/<NUM> of the resolution Rmax for the highest-resolution image. In this case, the use of the image sensor <NUM> capable of outputting a highest-resolution image of the resolution Rmax at <NUM> fps (or more) makes it possible to output both the BM image of the resolution RBM and the RV image of the resolution RRV at <NUM> fps, the BM image and the RV image being obtained by partially reading the high-resolution image.

<NUM><NUM><NUM><NUM> <Example of Vehicle Transmission Bandwidth That Can be Used for Data Transmission in Vehicle <NUM>>.

<NUM><NUM><NUM><NUM> <FIG> is a diagram describing an example of a vehicle transmission bandwidth that can be used for data transmission in the vehicle <NUM>.

<NUM><NUM><NUM><NUM> In other words, <FIG> illustrates examples of a BM image and an RV image that can be output by the camera unit <NUM> without data amounts being adjusted by the data amount adjuster <NUM>.

<NUM><NUM><NUM><NUM> For example, the camera unit <NUM> can output, as a BM image, a color image of the resolution RBM in a YUV <NUM>:<NUM>:<NUM> format in which the number of bits per pixel is eight (with respect to each of the brightness and a difference in color). Further, for example, the camera unit <NUM> can output, as an RV image, a color image of the resolution RRV in the YUV <NUM>:<NUM>:<NUM> format in which the number of bits per pixel is eight.

The BM image of the resolution RBM in the YUV <NUM>:<NUM>:<NUM> format in which the number of bits per pixel is eight is referred to as a highest-quality BM image, and the RV image of the resolution RRV in the YUV <NUM>:<NUM>:<NUM> format in which the number of bits per pixel is eight is referred to as a highest-quality RV image. <NUM><NUM><NUM><NUM>.

<NUM><NUM><NUM><NUM> In the present embodiment, it is assumed that the vehicle transmission bandwidth, which is a bandwidth in which data is transmitted from the camera unit <NUM>, is a transmission bandwidth in which, for example, two screens of a highest-quality BM image at <NUM> fps can be transmitted (in real time). In the present embodiment, the vehicle transmission bandwidth in which two screens of a highest-quality BM image at <NUM> fps can be transmitted, makes it possible to transmit, for example, two screens of a highest-quality RV image at <NUM> fps, since RBM ≥ RRV. Further, the vehicle transmission bandwidth makes it possible to transmit, for example, two screens in total that are a single screen of the highest-quality BM image at <NUM> fps, and a single screen of the highest-quality RV image at <NUM> fps.

<NUM><NUM><NUM><NUM> As described with reference to <FIG> and <FIG>, the camera unit <NUM> is capable of outputting both a highest-quality BM image and a highest-quality RV image at up to <NUM> fps.

<NUM><NUM><NUM><NUM> However, in the present embodiment, the vehicle transmission bandwidth only makes it possible to transmit two screens of a highest-quality BM image (or RV image) at <NUM> fps.

<NUM><NUM><NUM><NUM> The increase in vehicle transmission bandwidth makes it possible to transmit both a highest-quality BM image and a highest-quality RV image at <NUM> fps that can be output by the camera unit <NUM>. However, the increase in vehicle transmission bandwidth results in increasing the costs for the viewing system.

<NUM><NUM><NUM><NUM> In the present technology, the camera unit <NUM> appropriately adjusts data amounts of a BM image and an RV image to transmit the BM image and the RV image in the vehicle transmission bandwidth, in order to suppress an increase in the costs for the viewing system.

<NUM><NUM><NUM><NUM> <Control of Adjustment of Data Amounts of BM Image and RV Image When Two Screens of Highest-Quality BM Image at <NUM> fps Can be Transmitted in Vehicle Transmission Bandwidth>.

<NUM><NUM><NUM><NUM> <FIG> is a diagram describing a first example of a control of adjustment of data amounts of a BM image and an RV image, the control being performed by the controller <NUM>.

<NUM><NUM><NUM><NUM> A of <FIG> illustrates the adjustment control performed when the vehicle <NUM> is traveling forward, and is traveling backward at a high speed that is a speed equal to or greater than a first threshold for speed.

<NUM><NUM><NUM><NUM> In this case, the controller <NUM> performs an adjustment control that is a control of the data amount adjuster <NUM>, such that a BM image of the resolution RBM at <NUM> fps is output and output of an RV image is restricted. In accordance with the adjustment control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of a BM image and an RV image from the image sensor <NUM> to output the BM image of the resolution RBM at <NUM> fps and to restrict output of the RV image.

<NUM><NUM><NUM><NUM> Thus, in this case, the BM image of the resolution RBM at <NUM> fps is output from the camera unit <NUM>, and the RV image is not output from the camera unit <NUM>. Consequently, the BM image of the resolution RBM at <NUM> fps is displayed on the BM display section <NUM>, and the RV image is not displayed on the RV display section <NUM>.

<NUM><NUM><NUM><NUM> As described above, when the vehicle <NUM> is traveling forward, and is traveling backward at the high speed, a driver can confirm a region situated further rearward than a region just behind the rear of the vehicle <NUM>, using a BM image of the resolution RBM at <NUM> fps, that is, a high-resolution BM image at a high frame rate.

<NUM><NUM><NUM><NUM> Note that, when the vehicle <NUM> is traveling forward, and is traveling backward at the high speed, an RV image that includes an image of the region just behind the rear of the vehicle <NUM> is not displayed.

<NUM><NUM><NUM><NUM> Further, a BM image of the resolution RBM at <NUM> fps that is output by the camera unit <NUM> can be transmitted in the vehicle transmission bandwidth in which two screens of a highest-quality BM image (of the resolution RBM) at <NUM> fps can be transmitted.

<NUM><NUM><NUM><NUM> Here, examples of a data amount adjusting method for adjusting a data amount of an image that is performed by the data amount adjuster <NUM> include a method for reducing resolution (the number of pixels), a method for reducing gradation (the number of bits per pixel), a method for reducing a frame rate, and a compression method using a specified compression-encoding approach, in addition to the method for restricting output of an image (not outputting an image) as described above.

<NUM><NUM><NUM><NUM> In addition to being performed by the data amount adjuster <NUM>, a restriction of output of an image, a reduction in resolution, a reduction in gradation, and a reduction in frame rate from among the data amount adjusting methods can be performed by the image sensor <NUM> by the controller <NUM> performing control such as a control of extraction performed by the image sensor <NUM>.

<NUM><NUM><NUM><NUM> In other words, the restriction of output of an image such as a restriction of output of an RV image can be performed by the controller <NUM> controlling reading of data from the image sensor <NUM> and not reading a pixel signal corresponding to the RV image from the pixel <NUM> (<FIG>), such that extraction of an RV image from a captured image is restricted. When the vehicle <NUM> is traveling forward, and is traveling backward at the high speed, the controller <NUM> can control, according to vehicle information, extraction performed by the image sensor <NUM> such that extraction of an RV image from a captured image is restricted. Of course, output of an RV image may be restricted by the data amount adjuster <NUM>, not by the image sensor <NUM>.

<NUM><NUM><NUM><NUM> The resolution of an image can be reduced by, for example, the controller <NUM> controlling the image sensor <NUM> such that the number of pixels <NUM> from which a pixel signal is read is reduced, or such that binning for adding pixel signals of a plurality of pixels <NUM> is performed by performing, for example, a so-called source follower (SF) addition or floating diffusion (FD) addition.

<NUM><NUM><NUM><NUM> The gradation can be reduced by, for example, the controller <NUM> controlling the image sensor <NUM> such that the number of bits necessary for AD conversion performed by the AD converter <NUM> (<FIG>) is reduced.

<NUM><NUM><NUM><NUM> The frame rate can be reduced by, for example, the controller <NUM> controlling the image sensor <NUM> such that a rate at which a pixel signal is read from the pixel <NUM>, or a rate at which AD conversion is performed by the AD converter <NUM> is reduced.

<NUM><NUM><NUM><NUM> B of <FIG> illustrates the adjustment control performed when the vehicle <NUM> is traveling backward at a medium speed that is a speed that is less than the first threshold and is equal to or greater than a second threshold less than the first threshold.

<NUM><NUM><NUM><NUM> In this case, the controller <NUM> performs an adjustment control that is a control of the data amount adjuster <NUM> such that a BM image of a resolution RBMM at <NUM> fps is output and an RV image of a resolution RRVM at <NUM> fps is output, the resolution RBMM being less than the resolution RBM, the resolution RRVM being less than the resolution RRV. In accordance with the adjustment control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of a BM image and an RV image from the image sensor <NUM> to output the BM image of the resolution RBMM at <NUM> fps and to output the RV image of the resolution RRVM at <NUM> fps.

<NUM><NUM><NUM><NUM> Thus, in this case, the BM image of the resolution RBMM at <NUM> fps and the RV image of the resolution RRVM at <NUM> fps are output by the camera unit <NUM>. Consequently, the BM image of the resolution RBMM at <NUM> fps is displayed on the BM display section <NUM>, and the RV image of the resolution RRVM at <NUM> fps is displayed on the RV display section <NUM>.

<NUM><NUM><NUM><NUM> As described above, when the vehicle <NUM> is traveling backward at the medium speed, a driver can confirm a region situated further rearward than a region just behind the rear of the vehicle <NUM>, using a BM image of the resolution RBMM at <NUM> fps, that is, a medium-resolution BM image at the high frame rate. Further, the driver can confirm the region just behind the rear of the vehicle <NUM>, using an RV image of the resolution RRVM at <NUM> fps, that is, a medium-resolution RV image at a low frame rate.

<NUM><NUM><NUM><NUM> Note that it is assumed that a transmission bandwidth necessary to transmit a RV image of the resolution RRVM at <NUM> fps is equal to or less than a transmission bandwidth for a difference between a maximum transmission rate (here, a transmission rate necessary to transmit a BM image of the resolution RBM at <NUM> fps without compressing the BM image) and a transmission rate (a first transmission rate) necessary to transmit the resolution RBMM at <NUM> fps. In this case, both a BM image of the resolution RBMM at <NUM> fps and an RV image of the resolution RRVM at <NUM> fps that are output by the camera unit <NUM> can be transmitted in the vehicle transmission bandwidth in which two screens of a highest-quality BM image at <NUM> fps can be transmitted.

<NUM><NUM><NUM><NUM> Here, when the transmission bandwidth necessary to transmit an RV image of the resolution RRVM at <NUM> fps is not equal to or less than the transmission bandwidth for the difference between the maximum transmission rate and the first transmission rate, one of the BM image and the RV image or both of them can be compressed (compression-encoded) by the data amount adjuster <NUM> such that the transmission bandwidth necessary to transmit the RV image of the resolution RRVM at <NUM> fps is equal to or less than the transmission bandwidth for the difference between the maximum transmission rate and the first transmission rate (such that the RV image of the resolution RRVM can be transmitted). For example, with respect to a BM image of the resolution RRVM at <NUM> fps, a portion of or all of the BM image is compressed (compression-encoded). This results in being able to reduce a data amount of the BM image. With respect to an RV image of the resolution RRVM at <NUM> fps, the RV image is compression-encoded in a state of remaining a color image, or is converted into a black-and-white image to be compression-encoded. This results in being able to reduce a data amount of the RV image.

<NUM><NUM><NUM><NUM> C of <FIG> illustrates the adjustment control performed when the vehicle <NUM> is traveling backward at a low speed that is a speed that is less than the second threshold less than the first threshold.

<NUM><NUM><NUM><NUM> In this case, the controller <NUM> performs an adjustment control that is a control of the data amount adjuster <NUM> such that a BM image of the resolution RBM at <NUM> fps is output and an RV image of the resolution RRV at <NUM> fps is output. In accordance with the adjustment control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of a BM image and an RV image from the image sensor <NUM> to output the BM image of the resolution RBM at <NUM> fps and to output the RV image of the resolution RRV at <NUM> fps.

<NUM><NUM><NUM><NUM> Thus, in this case, the BM image of the resolution RBM at <NUM> fps and the RV image of the resolution RRV at <NUM> fps are output by the camera unit <NUM>. Consequently, the BM image of the resolution RBM at <NUM> fps is displayed on the BM display section <NUM>, and the RV image of the resolution RRV at <NUM> fps is displayed on the RV display section <NUM>.

As described above, when the vehicle <NUM> is traveling backward at the low speed, a driver can confirm a region situated further rearward than a region just behind the rear of the vehicle <NUM>, using a BM image of the resolution RBM at <NUM> fps, that is, a high-resolution BM image at a medium frame rate. Further, the driver can confirm the region just behind the rear of the vehicle <NUM>, using an RV image of the resolution RRV at <NUM> fps, that is, a high-resolution RV image at the medium frame rate.

<NUM><NUM><NUM><NUM> The case in which the vehicle <NUM> is traveling backward at the low speed is, for example, a case in which a driver is about to park the vehicle <NUM>, and it is important to confirm a region just behind the rear of the vehicle <NUM> that is a blind spot as viewed from the driver. Thus, when the vehicle <NUM> is traveling backward at the low speed, an RV image is displayed at a higher resolution and at a higher frame rate, compared to when the vehicle <NUM> is traveling backward at the high speed or the medium speed. This makes it possible to easily confirm a region of a blind spot and to easily control the vehicle according to a state of the blind spot.

Note that both a BM image of the resolution RBM at <NUM> fps and an RV image of the resolution RRV at <NUM> fps that are output by the camera unit <NUM> can be transmitted in the vehicle transmission bandwidth in which two screens of a highest-quality BM image at <NUM> fps can be transmitted.

<NUM><NUM><NUM><NUM> <FIG> is a diagram describing a second example of the control of adjustment of data amounts of a BM image and an RV image, the control being performed by the controller <NUM>.

<NUM><NUM><NUM><NUM> Here, in the second example of the control of adjustment of data amounts of a BM image and a RV image, it is assumed that the vehicle transmission bandwidth is a transmission bandwidth in which, for example, a single screen of a highest-quality BM image (of the resolution RBM) at <NUM> fps (or more) can be transmitted.

<NUM><NUM><NUM><NUM> A of <FIG> illustrates the adjustment control performed when the vehicle <NUM> is traveling forward, and is traveling backward at the high speed that is a speed equal to or greater than the first threshold for speed.

In this case, the controller <NUM> performs an adjustment control that is a control of the data amount adjuster <NUM>, such that a BM image of the resolution RBM at <NUM> fps (or more) is output and output of an RV image is restricted. In accordance with the adjustment control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of a BM image and an RV image from the image sensor <NUM> to output the BM image of the resolution RBM at <NUM> fps and to restrict output of the RV image.

Thus, in this case, the BM image of the resolution RBM at <NUM> fps is output from the camera unit <NUM>, and the RV image is not output from the camera unit <NUM>. Consequently, the BM image of the resolution RBM at <NUM> fps is displayed on the BM display section <NUM>, and the RV image is not displayed on the RV display section <NUM>.

As described above, when the vehicle <NUM> is traveling forward, and is traveling backward at the high speed, a driver can confirm a region situated further rearward than a region just behind the rear of the vehicle <NUM>, using a BM image of the resolution RBM at <NUM> fps, that is, a high-resolution BM image at the medium frame rate.

Note that, when the vehicle <NUM> is traveling forward, and is traveling backward at the high speed, an RV image that includes an image of the region just behind the rear of the vehicle <NUM> is not displayed, as described with reference to A of <FIG>.

Further, a BM image of the resolution RBM at <NUM> fps that is output by the camera unit <NUM> can be transmitted in the vehicle transmission bandwidth in which a single screen of a highest-quality BM image (of the resolution RBM) at <NUM> fps can be transmitted.

<NUM><NUM><NUM><NUM> B of <FIG> illustrates the adjustment control performed when the vehicle <NUM> is traveling backward at the medium speed that is a speed that is less than the first threshold and is equal to or greater than the second threshold less than the first threshold.

In this case, the controller <NUM> performs an adjustment control that is a control of the data amount adjuster <NUM> such that a BM image of the resolution RBMM at <NUM> fps is output and an RV image of the resolution RRVM at <NUM> fps is output, the resolution RBMM being less than the resolution RBM, the resolution RRVM being less than the resolution RRV. In accordance with the adjustment control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of a BM image and an RV image from the image sensor <NUM> to output the BM image of the resolution RBMM at <NUM> fps and to output the RV image of the resolution RRVM at <NUM> fps.

Thus, in this case, the BM image of the resolution RBMM at <NUM> fps and the RV image of the resolution RRVM at <NUM> fps are output by the camera unit <NUM>. Consequently, the BM image of the resolution RBMM at <NUM> fps is displayed on the BM display section <NUM>, and the RV image of the resolution RRVM at <NUM> fps is displayed on the RV display section <NUM>.

As described above, when the vehicle <NUM> is traveling backward at the medium speed, a driver can confirm a region situated further rearward than a region just behind the rear of the vehicle <NUM>, using a BM image of the resolution RBMM at <NUM> fps, that is, a medium-resolution BM image at the medium frame rate. Further, the driver can confirm the region just behind the rear of the vehicle <NUM>, using an RV image of the resolution RRVM at <NUM> fps, that is, a medium-resolution RV image at the low frame rate.

Note that, when a transmission bandwidth (hereinafter also referred to as a necessary transmission bandwidth) necessary to transmit both a BM image of the resolution RBMM at <NUM> fps and an RV image of the resolution RRVM at <NUM> fps that are output by the camera unit <NUM>, is not within the vehicle transmission bandwidth in which a single screen of a highest-quality BM image at <NUM> fps can be transmitted, the BM image may be compressed at a first compression rate for medium speed, and the RV image may be compressed in a state of remaining a color image at a second compression rate for medium speed that provides a higher compression than the first compression rate for medium speed, or the RV image may be converted into a black-and-white image to be compressed at the second compression rate for medium speed, such that the necessary transmission bandwidth is within the vehicle transmission bandwidth.

<NUM><NUM><NUM><NUM> C of <FIG> illustrates the adjustment control performed when the vehicle <NUM> is traveling backward at the low speed that is a speed that is less than the second threshold less than the first threshold.

In this case, the controller <NUM> performs an adjustment control that is a control of the data amount adjuster <NUM> such that a BM image of a resolution RBML at <NUM> fps is output and an RV image of the resolution RRVM at <NUM> fps is output, the resolution RBML being less than the resolution RBMM. In accordance with the adjustment control performed by the controller <NUM>, the data amount adjuster <NUM> adjusts data amounts of a BM image and an RV image from the image sensor <NUM> to output the BM image of the resolution RBML at <NUM> fps and to output the RV image of the resolution RRVM at <NUM> fps.

Thus, in this case, the BM image of the resolution RBML at <NUM> fps and the RV image of the resolution RRV at <NUM> fps are output by the camera unit <NUM>. Consequently, the BM image of the resolution RBML at <NUM> fps is displayed on the BM display section <NUM>, and the RV image of the resolution RRVM at <NUM> fps is displayed on the RV display section <NUM>.

As described above, when the vehicle <NUM> is traveling backward at the low speed, a driver can confirm a region situated further rearward than a region just behind the rear of the vehicle <NUM>, using a BM image of the resolution RBML at <NUM> fps, that is, a low-resolution BM image at the medium frame rate. Further, the driver can confirm the region just behind the rear of the vehicle <NUM>, using an RV image of the resolution RRVM at <NUM> fps, that is, a medium-resolution RV image at the low frame rate.

Note that, when the necessary transmission bandwidth necessary to transmit both a BM image of the resolution RBML at <NUM> fps and an RV image of the resolution RRVM at <NUM> fps that are output by the camera unit <NUM>, is not within the vehicle transmission bandwidth in which a single screen of a highest-quality BM image at <NUM> fps can be transmitted, the BM image may be compressed at a first compression rate for low speed, and the RV image may be compressed (in a state of remaining a color image) at a second compression rate for low speed that provides a higher compression than the first compression rate for low speed, such that the necessary transmission bandwidth is within the vehicle transmission bandwidth.

<NUM><NUM><NUM><NUM> Here, a rate that provides a higher compression than the first compression rate for medium speed can be adopted as the first compression rate for low speed. The same compression rate can be adopted as the second compression rate for medium speed and the second compression rate for low speed. In this case, the following is the relationship among the first and second compression rates for low speed, and the first and second compression rates for medium speed: the second compression rate for low speed = the second compression rate for medium speed > the first compression rate for low speed > the first compression rate for medium speed. However, here, it is assumed that a compression rate exhibiting a larger value provides a higher compression. The following is the relationship among data amounts respectively obtained by performing compression at the first compression rate for low speed, by performing compression at the second compression rate for low speed, by performing compression at the first compression rate for medium speed, and by performing compression at the second compression rate for medium speed: the data amount obtained by the compression at the second compression rate for low speed = the data amount obtained by the compression at the second compression rate for medium speed < the data amount obtained by the compression at the first compression rate for low speed < the data amount obtained by the compression at the first compression for medium speed.

<NUM><NUM><NUM><NUM> The adjustment control for adjusting data amounts of a BM image and an RV image according to the vehicle speed and the traveling direction (forward or backward) of the vehicle <NUM> has been described above. The adjustment control method is not limited to the methods described with reference to <FIG> and <FIG>. In other words, the adjustment control method may be set as appropriate according to, for example, the vehicle transmission bandwidth, the ability of the image sensor <NUM>, and the specifications of the BM display section <NUM> and the RV display section <NUM>. The data amounts are respectively related to the quality of a displayed BM image and a displayed RV image. Thus, it can be said that the quality of a BM image and the quality of an RV image are changed according to the vehicle speed and the traveling direction (forward or backward) of the vehicle <NUM>.

<NUM><NUM><NUM><NUM> Further, in the adjustment control, the resolution of the BM image and the resolution of the RV image can be reduced by reducing the number of pixels included in the BM image and the RV image, or by (irreversibly) compressing the BM image and the RV image without changing the number of pixels.

<NUM><NUM><NUM><NUM> When the resolution of a BM image and the resolution of a RV image are reduced by compressing the BM image and the RV image, it is possible to adopt, as compression rates at which the BM image and the RV image are compressed, compression rates such that a BM image and a RV image that are obtained by performing compression and decompression and from which a portion of frequency components such as a high frequency component has been lost, each have a substantial resolution (a highest frequency component) that is equivalent to the resolution (the number of pixels) described with reference to <FIG> or <FIG>.

<NUM><NUM><NUM><NUM> <FIG> is a flowchart describing an example of display processing of displaying a BM image and an RV image that is performed in the viewing system of <FIG>.

<NUM><NUM><NUM><NUM> In Step S11, the image sensor <NUM> captures a captured image, and the process moves on to Step S12.

<NUM><NUM><NUM><NUM> In Step S12, the acquisition section <NUM> acquires vehicle information from the vehicle <NUM>, and supplies the vehicle information to the controller <NUM>. Then, the process moves on to Step S13.

<NUM><NUM><NUM><NUM> In Step S13, the controller <NUM> controls extraction performed by the image sensor <NUM> according to the vehicle information from the acquisition section <NUM>. The image sensor <NUM> extracts a BM image and an RV image from the captured image in accordance with the extraction control performed by the controller <NUM>. Then, the image sensor <NUM> supplies the BM image and the RV image to the data amount adjuster <NUM>, and the process moves on to Step S14 from Step S13.

<NUM><NUM><NUM><NUM> In Step S14, the controller <NUM> controls adjustment performed by the data amount adjuster <NUM> according to the vehicle information from the acquisition section <NUM>. The data amount adjuster <NUM> adjusts data amounts of the BM image and the RV image from the image sensor <NUM> in accordance with the adjustment control performed by the controller <NUM>. Then, the data amount adjuster <NUM> supplies, to the output section <NUM>, the BM image and the RV image of which the respective data amounts have been adjusted, and the process moves on to Step S15 from Step S14.

<NUM><NUM><NUM><NUM> In Step S15, the output section <NUM> outputs, to the outside of the camera unit <NUM>, the BM image and the RV image that are supplied from the data amount adjuster <NUM> and of which the respective data amounts have been adjusted, transmits the BM image to the BM display section <NUM>, and transmits the RV image to the RV display section <NUM>. Then, the process moves on to Step S16.

<NUM><NUM><NUM><NUM> In Step S16, the BM display section <NUM> displays thereon the BM image from the output section <NUM> in accordance with the specification of the BM display section <NUM>, and the RV display section <NUM> displays thereon the RV image from the output section <NUM> in accordance with the specification of the RV display section <NUM>.

<NUM><NUM><NUM><NUM> <FIG> is a block diagram illustrating a second example of the configuration of the viewing system included in the vehicle <NUM>.

Note that, in the figure, a portion corresponding to that in <FIG> is denoted by the same reference numeral as <FIG>, and a description thereof is omitted below.

<NUM><NUM><NUM><NUM> In <FIG>, the viewing system includes the camera unit <NUM>, the BM display section <NUM>, and the RV display section <NUM>, and the camera unit <NUM> includes the optical system <NUM>, the image sensor <NUM>, the data amount adjuster <NUM>, the output section <NUM>, the acquisition section <NUM>, the controller <NUM>, and a memory <NUM>.

Thus, the viewing system of <FIG> is similar to the viewing system of <FIG> in including the camera unit <NUM>, the BM display section <NUM>, and the RV display section <NUM>, and in that the camera unit <NUM> includes the components from the optical system <NUM> to the controller <NUM>. <NUM><NUM><NUM><NUM>.

However, the viewing system of <FIG> is different from the viewing system of <FIG> in that the camera unit <NUM> newly includes the memory <NUM>.

<NUM><NUM><NUM><NUM> In the viewing system of <FIG>, the image sensor <NUM> supplies a highest-resolution image (a highest-resolution captured image) to the memory <NUM>, and the memory <NUM> stores therein the highest-resolution image from the image sensor <NUM>. Then, the controller <NUM> performs, as an extraction control, a control of reading of data (a pixel signal) from the memory <NUM> with respect to the memory <NUM>, not with respect to the image sensor <NUM>. This results in extracting a BM image and an RV image from a highest-resolution image, as in the case of <FIG>.

<NUM><NUM><NUM><NUM> The BM-image and the RV-image that are extracted from the highest-resolution image and stored in the memory <NUM> are supplied to the data amount adjuster <NUM>.

<NUM><NUM><NUM><NUM> Note that, with respect to the restriction of output of an RV image that is performed when the vehicle <NUM> is traveling forward, and is traveling backward at the high speed, as described with reference to A of <FIG> and <FIG>, the restriction of output of an RV image may be performed in the viewing system of <FIG> by restricting extraction of the RV image from a highest-resolution image (by not reading the RV image) in a control of extraction from the memory <NUM>, in addition to being performed by the data amount adjuster <NUM> restricting output of the RV image.

<NUM><NUM><NUM><NUM> Further, display processing of displaying a BM image and an RV image in the viewing system of <FIG> is similar to the display processing described with reference to <FIG>, and thus a description thereof is omitted.

<NUM><NUM><NUM><NUM> However, in the viewing system of <FIG>, the image sensor <NUM> captures a captured image and then the captured image is stored in the memory <NUM> in Step S11. Further, in Step S13, the controller <NUM> controls extraction from the memory <NUM>, not from the image sensor <NUM>, according to vehicle information from the acquisition section <NUM>. This results in extracting a BM image and an RV image from the captured image stored in the memory <NUM>.

<NUM><NUM><NUM><NUM> <FIG> is a block diagram illustrating a third example of the configuration of the viewing system included in the vehicle <NUM>.

<NUM><NUM><NUM><NUM> In <FIG>, the viewing system includes the camera unit <NUM>, the BM display section <NUM>, the RV display section <NUM>, and an extraction section <NUM>, and the camera unit <NUM> includes the optical system <NUM>, the image sensor <NUM>, the data amount adjuster <NUM>, the output section <NUM>, the acquisition section <NUM>, and a controller <NUM>.

Thus, the viewing system of <FIG> is similar to the viewing system of <FIG> in including the camera unit <NUM>, the BM display section <NUM>, and the RV display section <NUM>, and in that the camera unit <NUM> includes the components from the optical system <NUM> to the acquisition section <NUM>.

<NUM> However, the viewing system of <FIG> is different from the viewing system of <FIG> in newly including the extraction section <NUM>, and in that the camera unit <NUM> includes the controller <NUM> instead of the controller <NUM>.

<NUM> Note that the extraction section <NUM> may be provided within the camera unit <NUM>, although the extraction section <NUM> is provided outside of the camera unit <NUM> in <FIG>.

<NUM><NUM> Vehicle information is supplied by the acquisition section <NUM> to the controller <NUM>. Examples of the vehicle information include traveling information, the specifications of the BM display section <NUM> and the RV display section <NUM>, and gyroscopic information. However, in this example, the vehicle information supplied by the acquisition section <NUM> does not include positions of the line of sight and the head of a driver. Note that the positions of the line of sight and the head of the driver of the vehicle <NUM> are input to the extraction section <NUM> as a portion of the vehicle information.

<NUM><NUM> As in the case of the controller <NUM>, the controller <NUM> controls extraction performed by the image sensor <NUM> and adjustment performed by the data amount adjuster <NUM>, according to the vehicle information supplied by the acquisition section <NUM>.

<NUM> However, in the extraction control, the controller <NUM> causes a region larger in size than the region R11 to be extracted as a BM image instead of controlling (the position of) the region R11 extracted as the BM image, according to one of the positions of the line of sight and the head of the driver, or according to both of them.

<NUM> Thus, the BM image output by the output section <NUM> in <FIG> is larger in size than the BM image output by the output section <NUM> in <FIG> and <FIG>.

<NUM> In <FIG>, the BM image larger in size than the region R11 output by the output section <NUM> is supplied to the extraction section <NUM>.

<NUM> The positions of the line of sight and the head of the driver from among the vehicle information are supplied to the extraction section <NUM>, in addition to the BM image larger in size than the region R11 being supplied by the output section <NUM> to the extraction section <NUM>.

<NUM> According to one of the positions of the line of sight and the head of the driver, or according to both of them, the extraction section <NUM> extracts, as a final BM image to be displayed on the BM display section <NUM>, a region that is a portion of the BM image larger in size than the region R11 from the output section <NUM>, that is, a region having the same size as the region R11, and the extraction section <NUM> supplies the BM image to the BM display section <NUM>.

<NUM> <FIG> is a diagram describing an example of a control of extraction of a BM image and an RV image from a captured image, the control being performed by the controller <NUM>.

<NUM> As in the case of <FIG>, <FIG> illustrates an image circle of the optical system <NUM> and a captured image captured by the image sensor <NUM> (the light-receiving surface of the image sensor <NUM>).

<NUM> In the extraction control, the controller <NUM> controls reading of data from the image sensor <NUM> such that the region R12 is extracted as an RV image, as in the case of the controller <NUM>.

Further, <NUM> in the extraction control, the controller <NUM> controls reading of data from the image sensor <NUM> such that not the region R11 but a region R31 that is larger in size than the region R11 is extracted from the captured image as a BM image.

<NUM> The region R31 is a region that includes a largest range that a driver can see using an interior rearview mirror by moving his/her line of sight or his/her head, if the interior rearview mirror is installed in the vehicle <NUM>. The region R11 is a variable region of which the position is changed according to positions of the line of sight and the head of the driver, whereas the region R31 is a fixed region.

<NUM> A region that is situated at a position depending on one of the positions of the line of sight and the head of the driver or depending on both of them, and has the same size as the region R11, is extracted by the extraction section <NUM> from the region R31 described above as a final BM image to be displayed on the BM display section <NUM>. In other words, the region R11 to be observed by the driver using an interior rearview mirror if the interior rearview mirror is installed in the vehicle <NUM>, is extracted by the extraction section <NUM> from the region R31 as a BM image.

<NUM> <FIG> is a flowchart describing an example of display processing of displaying a BM image and an RV image that is performed in the viewing system of <FIG>.

In Step S21, the image sensor <NUM> captures a captured image, and the process moves on to Step S22.

In Step S22, the acquisition section <NUM> acquires vehicle information from the vehicle <NUM>, and supplies the vehicle information to the controller <NUM>. Then, the process moves on to Step S23.

In Step S23, the controller <NUM> controls extraction performed by the image sensor <NUM>. As described with reference to <FIG>, in accordance with the extraction control performed by the controller <NUM>, the image sensor <NUM> extracts, from the captured image, the regions R31 and R12 as a BM image and an RV image, respectively. Then, the image sensor <NUM> supplies the BM image and the RV image to the data amount adjuster <NUM>, and the process moves on to Step S24 from Step S23.

In Step S24, the controller <NUM> controls adjustment performed by the data amount adjuster <NUM> according to the vehicle information from the acquisition section <NUM>. The data amount adjuster <NUM> adjusts data amounts of the BM image and the RV image from the image sensor <NUM> in accordance with the adjustment control performed by the controller <NUM>. Then, the data amount adjuster <NUM> supplies, to the output section <NUM>, the BM image and the RV image of which the respective data amounts have been adjusted, and the process moves on to Step S25 from Step S24.

In Step S25, the output section <NUM> outputs, to the outside of the camera unit <NUM>, the BM image and the RV image that are supplied from the data amount adjuster <NUM> and of which the respective data amounts have been adjusted, and the process moves on to Step S26. Consequently, in <FIG>, 0163the BM image is supplied to the extraction section <NUM>, and the RV image is transmitted to the RV display section <NUM>.

<NUM> In Step S26, the extraction section <NUM> acquires, from the vehicle <NUM>, positions of the line of sight and the head of a driver that are included in the vehicle information, and the process moves on to Step S27.

<NUM> In Step S27, the extraction section <NUM> extracts, from the BM image from the output section <NUM>, a region that is situated at a position depending on the positions of the line of sight and the head of the driver and has the same size as the region R11, the region being extracted as a final BM image to be displayed on the BM display section <NUM>. Then, the extraction section <NUM> transmits the final BM image to the BM display section <NUM>, and the process moves on to Step S28 from Step S27.

<NUM> In Step S28, the BM display section <NUM> displays thereon the BM image from the extraction section <NUM> in accordance with the specification of the BM display section <NUM>, and the RV display section <NUM> displays thereon the RV image from the output section <NUM> in accordance with the specification of the RV display section <NUM>.

<NUM> <FIG> is a block diagram illustrating a fourth example of the configuration of the viewing system included in the vehicle <NUM>.

Note that, in the figure, a portion corresponding to that in <FIG> or <FIG> is denoted by the same reference numeral as <FIG> or <FIG>, and a description thereof is omitted below. Examples of vehicle information acquired by the acquisition section <NUM> include traveling information, the specifications of the BM display section <NUM> and the RV display section <NUM>, and gyroscopic information. Note that the positions of the line of sight and the head of a driver of the vehicle <NUM> are input to the extraction section <NUM> as a portion of the vehicle information.

<NUM> In <FIG>, the viewing system includes the camera unit <NUM>, the BM display section <NUM>, the RV display section <NUM>, and the extraction section <NUM>, and the camera unit <NUM> includes the optical system <NUM>, the image sensor <NUM>, the data amount adjuster <NUM>, the output section <NUM>, the acquisition section <NUM>, the memory, and the controller <NUM>.

Thus, the viewing system of <FIG> is similar to the viewing system of <FIG> in including the camera unit <NUM>, the BM display section <NUM>, the RV display section <NUM>, and the extraction section <NUM>, and in that the camera unit <NUM> includes the components from the optical system <NUM> to the acquisition section <NUM>, and the controller <NUM>.

However, the viewing system of <FIG> is different from the viewing system of <FIG> in newly including the memory <NUM> of <FIG>.

<NUM> In the viewing system of <FIG>, processing similar to the processing performed in the viewing system of <FIG> is performed, except that the memory <NUM> stores therein a highest-resolution image (a highest-resolution captured image) from the image sensor <NUM> and an extraction control is performed with respect to the memory <NUM>, not with respect to the image sensor <NUM>, as in the case of the viewing system of <FIG>.

<NUM> Next, the series of processes performed by the controllers <NUM> and <NUM>, the data amount adjuster <NUM>, and the extraction section <NUM> described above can be performed using hardware or software. When the series of processes is performed using software, a program included in the software is installed on, for example, a general-purpose computer.

<NUM> <FIG> is a block diagram illustrating an example of a configuration of an embodiment of a computer on which a program used to perform the series of processes described above is installed.

<NUM> The program can be recorded in advance in a hard disk <NUM> or a read only memory (ROM) <NUM> that is a recording medium included in the computer.

<NUM> Alternatively, the program can be stored (recorded) in a removable recording medium <NUM> driven by a drive <NUM>. Such a removable recording medium <NUM> can be provided as so-called package software. Here, examples of the removable recording medium <NUM> include a flexible disk, a compact disc read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disc (DVD), a magnetic disk, and a semiconductor memory.

<NUM> Note that, in addition to being installed on the computer from the removable recording medium <NUM> described above, the program can be downloaded to the computer through a communication network or a broadcast network to be installed on the hard disk <NUM> included in the computer. In other words, for example, the program can be wirelessly transferred to the computer from a download site through a satellite for digital satellite broadcasting, or can be transferred to the computer by wire through a network such as a local area network (LAN) or the Internet.

<NUM> The computer includes a central processing unit (CPU) <NUM>, and an input/output interface <NUM> is connected to the CPU <NUM> through a bus <NUM>.

<NUM> When a command is input by a user operating an input section <NUM> through the input/output interface <NUM>, the CPU <NUM> executes a program stored in the ROM <NUM> according to the input command. Alternatively, the CPU <NUM> loads a program stored in the hard disk <NUM> into a random access memory (RAM) <NUM> and executes the program.

<NUM> This results in the CPU <NUM> performing the processing according to the flowcharts described above or the processing performed on the basis of the configurations of the block diagrams described above. Then, for example, as necessary, the CPU <NUM> outputs a result of the processing using an output section <NUM> or transmits the processing result using a communication section <NUM> through the input/output interface <NUM>, and the CPU <NUM> further records the processing result in the hard disk <NUM>.

<NUM> Note that the input section <NUM> includes, for example, a keyboard, a mouse, and a microphone. Further, the output section <NUM> includes, for example, a liquid crystal display (LCD) and a speaker.

<NUM> Here, in the specification, the processes performed by a computer in accordance with a program does not necessarily have to be chronologically performed in the order of the descriptions in the flowcharts. In other words, the processes performed by a computer in accordance with a program include processes performed in parallel or individually (for example, parallel processing or processing performed using an object).

<NUM> Further, the program may be a program on which processing is performed by a single computer (processor) or may be a program on which distributed-processing is performed by a plurality of computers. Furthermore, the program may be transferred to a remote computer to be executed by the remote computer.

Further, the system as used herein refers to a collection of a plurality of components (such as apparatuses and modules (parts)) and it does not matter whether all of the components are in a single housing. Thus, a plurality of apparatuses accommodated in separate housings and connected to one another via a network, and a single apparatus in which a plurality of modules is accommodated in a single housing are both the system.

Note that the embodiment of the present technology is not limited to the examples described above, and various modifications may be made thereto without departing from the scope of the appended claims.

For example, the present technology may also have a configuration of cloud computing in which a single function is shared to be cooperatively processed by a plurality of apparatuses via a network.

Further, the respective steps described using the flowcharts described above may be shared to be performed by a plurality of apparatuses, in addition to being performed by a single apparatus.

Claim 1:
An image-capturing apparatus, comprising:
an image sensor (<NUM>) that performs image-capturing;
a controller (<NUM>) that performs an extraction control for extracting a first image (BM) and a second image (RV) from a captured image captured by the image sensor (<NUM>), the first image (BM) being displayed on a first display section (<NUM>) while the second image (RV) is being simultaneously displayed on a second display section (<NUM>); and
a data amount adjuster (<NUM>) that adjusts data amounts of the first image (BM) and the second image (RV) according to vehicle information acquired from a vehicle (<NUM>), wherein the first image is an image of a region situated further rearward than a region just behind the rear of the vehicle (<NUM>), the second image (RV) is an image of a region just behind the rear of the vehicle, and wherein the vehicle information includes a traveling direction or a vehicle speed of the vehicle (<NUM>), wherein
the data amount adjuster (<NUM>) adjusts the data amount of the second image (RV) by restricting output of the second image, or
the data amount adjuster (<NUM>) adjusts the data amount of the first image (BM) by compressing the first image (BM), or the data amount of the second image (RV) by compressing the second image (RV).