Patent Description:
MIPI CSI-<NUM> (Camera Serial Interface <NUM>) is known as a widely adopted, high-speed protocol for transmitting still and video images from image sensors to application processors. Further, technologies for cutting a partial region of an image captured by an imaging device and transferring data of the cut region have been developed. The technology described in PTL <NUM> below, for example, can be listed among the aforementioned technologies. In addition, the technologies described in Patent Literatures <NUM> to <NUM> below, for example, can be listed among the technologies for cutting a partial region of an image.

In a case in which the technology described in PTL <NUM> is used, for example, it is possible to cut a partial region from an image and transfer data of the cut region. Therefore, since the amount of data related to the transfer is smaller in a case in which the image of the region cut by using the technology described in PTL <NUM> is transferred than in a case in which the entire image is transferred, there are advantages, such as an advantage that a transfer time is shortened.

However, the cut region according to the technology described in PTL <NUM> is a rectangular region, and transferring data in a region with another shape is not particularly taken into consideration. In addition, cut regions are also rectangular regions according to the technologies of cutting partial regions of images as described in PTLS <NUM> to <NUM>. Therefore, it is difficult to transfer data in a region with an arbitrary shape set in an image using the existing technologies in PTLS <NUM> to <NUM>.

The present disclosure will propose a novel and improved transmission device capable of transmitting data in a region with an arbitrary shape set in an image.

According to an embodiment of the present disclosure, there is provided a transmission device, a reception device, a communication system, and method as defined in the appended claims.

According to an embodiment of the present disclosure, it is possible to transmit data in a region with an arbitrary shape set in an image.

In addition, the order shown below will be used in the following description.

In the following description, a case in which a communication scheme between devices that are included in the communication system according to the embodiment is a communication scheme in accordance with Mobile Industry Processor Interface (MIPI) Camera Serial Interface <NUM> (CSI-<NUM>) standards will be exemplified. Note that the communication scheme between the devices that are included in the communication system according to the embodiment is not limited to the communication scheme in accordance with the MIPI CSI-<NUM> standards. For example, the communication between the devices that are included in the communication system according to the embodiment may be another standard defined by the MIPI alliance, such as a communication scheme in accordance with MIPI CSI-<NUM> standards or a communication scheme in accordance with MIPI Display Serial Interface (DSI). In addition, it is needless to say that the communication scheme to which the transmission method according to the embodiment can be applied is not limited to the communication scheme related to the standards defined by the MIPI alliance.

<FIG> is an explanatory diagram illustrating an example of a configuration of a communication system <NUM> according to the embodiment. As the communication system <NUM>, a communication device such as a smartphone, a drone (equipment capable of performing operations through remote operations or autonomous operations), a moving body such as a vehicle, and the like are exemplified, for example. Note that the application examples of the communication system <NUM> are not limited to the examples described above. Other application examples of the communication system <NUM> will be described later.

The communication system <NUM> has an image sensor <NUM>, a processor <NUM>, a memory <NUM>, and a display device <NUM>, for example.

The image sensor <NUM> has an imaging function and a transmission function and transmits data indicating an image generated by image capturing. The processor <NUM> receives data transmitted from the image sensor <NUM> and processes the received data. That is, the image sensor <NUM> serves as a transmission device while the processor <NUM> serves as a reception device, in the communication system <NUM>.

Note that although <FIG> illustrates the communication system <NUM> that has one image sensor <NUM>, the number of the image sensors <NUM> that the communication system according to the embodiment has is not limited to the example illustrated in <FIG>. For example, the communication system according to the embodiment may have two or more image sensors <NUM>.

In addition, although <FIG> illustrates the communication system <NUM> that has one processor <NUM>, the number of the processors <NUM> that the communication system according to the embodiment has is not limited to the example illustrated in <FIG>. For example, the communication system according to the embodiment may have two or more processors <NUM>.

In a communication system that has a plurality of image sensors <NUM> and a plurality of processors <NUM>, the image sensors <NUM> and the processors <NUM> may have one-to-one correspondence, or one processor <NUM> may have correspondence with a plurality of image sensors <NUM>. Alternatively, in the communication system that has a plurality of image sensors <NUM> and a plurality of processors <NUM>, a plurality of processors <NUM> may have correspondence with one image sensor <NUM>.

Even in the communication system that has a plurality of image sensors <NUM> and a plurality of processors <NUM>, communication is performed between the image sensors <NUM> and the processors <NUM> similarly to the communication system <NUM> illustrated in <FIG>.

The image sensor <NUM> and the processor <NUM> are electrically connected to each other by a data bus B1. The data bus B1 is a transfer path of one signal for connecting the image sensor <NUM> and the processor <NUM>. For example, data indicating an image transmitted from the image sensor <NUM> (hereinafter referred to as "image data" in some cases) is transferred from the image sensor <NUM> to the processor <NUM> via the data bus B1.

A signal transferred via the data bus B1 in the communication system <NUM> is transferred according to a communication scheme in accordance with predetermined standards, such as MIPI CSI-<NUM> standards, for example.

<FIG> are explanatory diagrams illustrating formats of packets defined by the MIPI CSI-<NUM> standards. <FIG> illustrates a format of a short packet defined by the MIPI CSI-<NUM> standards, and <FIG> illustrates a format of a long packet defined by the MIPI CSI-<NUM> standards.

The long packet is data including a packet header ("PH" illustrated in <FIG>), a payload ("Payload Data" illustrated in <FIG>), and a packet footer ("PF" illustrated in <FIG>). The short packet is data that has a structure similar to the packet header ("PH" illustrated in <FIG>) as illustrated in <FIG>.

Virtual channel (VC) numbers ("VC" illustrated in <FIG>; VC values) are recorded in header portions of both the short packet and the long packet, and arbitrary VC numbers can be applied to each packet. Packets to which the same VC number is applied are handled as packets that belong to the same image data.

In addition, data type (DT) values ("Data Type" illustrated in <FIG>) are recorded in the header portions of both the short packet and the long packet. Therefore, it is also possible to handle packets to which the same DT value has been applied as packets that belong to the same image data similarly to the VC numbers.

The end of the packet is recorded as the number of words in Word Count in the header portion of the long packet. An error correcting code is recorded in ECC in the header portions of the short packet and the long packet.

According to the MIPI CSI-<NUM> standards, a high-speed differential signal is used in a period during which a data signal is transferred, and a low-power signal is used in a blanking period of the data signal. In addition, the period during which the high-speed differential signal is used is called a high speed state (HPS) period while the period during which the low power signal is used is called a low power state (LPS) period.

<FIG> is an explanatory diagram illustrating an example of a signal waveform related to transmission of a packet in accordance with the MIPI CSI-<NUM> standards. A in <FIG> illustrates an example of transfer of a packet, and B in <FIG> illustrates another example of transfer of a packet. "ST", "ET", "PH" "PF", 'SP", and "PS" illustrated in <FIG> have the following meanings.

As illustrated in <FIG>, it is possible to recognize that an amplitude of a differential signal in the LPS period ("LPS" illustrated in <FIG>) and a differential signal in the HPS period (other than "LPS" illustrated in <FIG>) are different from each other. Therefore, it is desirable that as little of the LPS period as possible be included in view of improving transfer efficiency.

The image sensor <NUM> and the processor <NUM> are electrically connected to each other by a control bus B2 that is different from the data bus B1, for example. The control bus B2 is a transfer path for other signals, and connects the image sensor <NUM> to the processor <NUM>. For example, control information output from the processor <NUM> is transferred from the processor <NUM> to the image sensor <NUM> via the control bus B2.

The control information includes information for control and processing commands, for example. As the information for control, data for controlling functions of the image sensor <NUM>, such as one or two or more of data indicating an image size, data indicating a frame rate, and data indicating the amount of an output delay from reception of an image output command to an image output is exemplified. In addition, the control information may include identification information indicating the image sensor <NUM>. As the identification information, arbitrary data with which it is possible to specify the image sensor <NUM>, such as an ID set for the image sensor <NUM> is exemplified.

Note that the information transferred from the processor <NUM> to the image sensor <NUM> via the control bus B2 is not limited to the example described above. For example, the processor <NUM> may transfer region designation information designating a region in an image via the control bus B2. As the region designation information, data in an arbitrary form with which a region can be specified, such as data indicating a position of a pixel included in the region (for example, coordinate data representing a position of a pixel included in the region using coordinates), is exemplified.

Although <FIG> illustrates an example in which the image sensor <NUM> and the processor <NUM> are electrically connected to each other by the control bus B2, the image sensor <NUM> and the processor <NUM> may not be connected by the control bus B2. For example, the image sensor <NUM> and the processor <NUM> may transmit and receive control information and the like through wireless communication of an arbitrary communication scheme.

Hereinafter, the respective devices that are included in the communication system <NUM> illustrated in <FIG> will be described.

The memory <NUM> is a recording medium that the communication system <NUM> has. As the memory <NUM>, a volatile memory such as a random access memory (RAM) or a nonvolatile memory such as a flash memory is exemplified. The memory <NUM> operates using electrical power supplied from an internal power source (not illustrated) that is included in the communication system <NUM>, such as a battery, or electrical power supplied from an external power source of the communication system <NUM>.

An image output from the image sensor <NUM>, for example, is stored in the memory <NUM>. The recording of the image in the memory <NUM> is controlled by the processor <NUM>, for example.

The display device <NUM> is a display device that the communication system <NUM> has. As the display device <NUM>, a liquid crystal display and an organic electro-luminescence display (organic EL display), also known as an organic light emitting diode display (OLED display) are exemplified. The display device <NUM> operates using electrical power supplied from an internal power source (not illustrated) that is included in the communication system <NUM>, such as a battery, or electrical power supplied from an external power source of the communication system <NUM>.

Various images and screens, such as an image output from the image sensor <NUM>, a screen related to an application that is executed by the processor <NUM>, and a screen related to a user interface (UI), are displayed on a display screen of the display device <NUM>. The display of the image and the like on the display screen of the display device <NUM> is controlled by the processor <NUM>, for example.

The processor <NUM> receives data transmitted from the image sensor <NUM> and processes the received data. As described above, the processor <NUM> serves as a reception device in the communication system <NUM>. An example of a configuration related to processing of data transmitted from the image sensor <NUM> (a configuration for serving as the reception device) will be described later.

The processor <NUM> includes one or two or more processors, various processing circuits, and the like that include arithmetic circuits such as a micro processing unit (MPU), for example. The processor <NUM> operates using electrical power supplied from an internal power source (not illustrated) that is included in the communication system <NUM>, such as a battery, or electrical power supplied from an external power source of the communication system <NUM>.

The processor <NUM> performs various kinds of processing such as processing related to control of recording of image data in the recording medium such as the memory <NUM>, processing related to control of display of an image on the display screen of the display device <NUM>, and processing of executing arbitrary application software, for example. As the processing related to the control of recording, "processing of delivering control data including a recording command and data that is caused to be recorded in the recording medium to the recording medium such as the memory <NUM>" is exemplified. In addition, as the processing related to the control of the display, "processing of delivering control data including a display command and data that is caused to be displayed on the display screen to the display device such as a display device <NUM>" is exemplified.

In addition, the processor <NUM> may control functions of the image sensor <NUM> by transmitting control information to the image sensor <NUM>, for example. The processor <NUM> can also control data to be transmitted from the image sensor <NUM> by transmitting region designation information to the image sensor <NUM>, for example.

The image sensor <NUM> has an imaging function and a transmission function and transmits data indicating an image generated by image capturing. As described above, the image sensor <NUM> serves as a transmission device in the communication system <NUM>.

The image sensor <NUM> includes "an imaging device such as a digital still camera, a digital video camera, or a stereo camera" or an image sensor device of an arbitrary scheme capable of generating an image, such as "an infrared sensor" and "a distance image sensor" and has a function of transmitting the generated image. The image generated by the image sensor <NUM> corresponds to data indicating a sensing result of the image sensor <NUM>. An example of a configuration of the image sensor <NUM> will be described later.

The image sensor <NUM> transmits data corresponding to a region set in an image (hereinafter referred to as "region data") according to the transmission method according to the embodiment, which will be described later. The control related to the transmission of the region data is performed by a component (which will be described later) that functions as an image processing unit in the image sensor <NUM>, for example. The region set in an image is referred to as a region of interest (ROI) in some cases. Hereinafter, the region set in the image is represented as an "ROI" in some cases.

As the processing related to the setting of a region in the image, arbitrary processing with which it is possible to specify a partial region in the image (or arbitrary processing capable of cutting a partial region from the image) such as "processing of detecting an object in the image and setting a region including the detected object," or "processing of setting a region designated by an operation performed on an arbitrary operation device or the like" is exemplified.

The processing related to the setting of the region in the image may be performed by the image sensor <NUM> or may be performed by an external device such as the processor <NUM>. In a case in which the image sensor <NUM> performs the processing related to the setting of the region in the image, the image sensor <NUM> specifies the region in accordance with a result of the processing related to the setting of the region in the image. Alternatively, in a case in which the processing related to the setting of the region in the image is performed by the external device, for example, the image sensor <NUM> specifies the region on the basis of region designation information acquired from the external device.

The amount of data related to the transfer becomes smaller than the data amount when the entire image is transferred because the region data, that is, partial data of the image, is transmitted. Accordingly, various advantages achieved by the reduction of the data amount are achieved by the image sensor <NUM> transmitting the region data, and for example, a transfer time is shortened and a load related to the transfer is reduced in the communication system <NUM>.

Note that the image sensor <NUM> can also transmit data indicating the entire image.

In a case in which the image sensor <NUM> has a function of transmitting region data and a function of transmitting data indicating the entire image, the image sensor <NUM> can perform the transmission of the region data and the transmission of the data indicating the entire image in a selectively switched manner.

The image sensor <NUM> transmits the region data or transmits the data indicating the entire image depending on a set operation mode, for example. The setting of the operation mode is performed by an operation performed on an arbitrary operation device, for example.

In addition, the image sensor <NUM> may selectively switch between the transmission of the region data and the transmission of the data indicating the entire image on the basis of region designation information acquired from an external device. When the region designation information is acquired from the external device, for example, the image sensor <NUM> transmits the region data of the region corresponding to the region designation information. When the region designation information is not acquired from the external device, the image sensor <NUM> transmits data indicating the entire image.

The communication system <NUM> has the configuration illustrated in <FIG>, for example. Note that the configuration of the communication system according to the embodiment is not limited to the example illustrated in <FIG>.

For example, although the image sensor <NUM> has been exemplified as the device that serves as the transmission device in <FIG>, a device that serves as the transmission device is not limited to the image sensor <NUM>. For example, in a case in which the communication system according to the embodiment has a configuration that has an image sensor device such as an imaging device and a transmitter that is electrically connected to the image sensor device, the transmitter may serve as the transmission device.

In addition, although the processor <NUM> has been exemplified as the device that serves as the reception device in <FIG>, the device that serves as the reception device is not limited to the processor <NUM>. For example, an arbitrary device that has a function of receiving data can serve as the reception device in the communication system according to the embodiment.

In addition, the communication system according to the embodiment may not have the memory <NUM> in a case in which the image transmitted from the image sensor <NUM> is stored in a recording medium outside the communication system, in a case in which the image transmitted from the image sensor <NUM> is stored in a memory provided in the processor <NUM>, in a case in which the image transmitted from the image sensor <NUM> is not recorded, or the like.

In addition, the communication system according to the embodiment can employ a configuration in which the communication system does not have the display device <NUM> illustrated in <FIG>.

Further, the communication system according to the embodiment may have an arbitrary configuration in accordance with functions of an electronic device to which the communication system according to the embodiment is applied as will be described later.

Although the communication system has been described above as an example in the embodiment, the embodiment is not limited to such an example. The embodiment can be applied to various electronic devices such as a communication device such as a smartphone, a drone (equipment capable of performing operations through remote operations or autonomous operations), a moving body such as a vehicle, a computer such as a personal computer (PC), a tablet-type device, and a game console.

Next, the transmission method according to the embodiment will be described.

Hereinafter, a case in which the transmission method according to the embodiment is applied to the image sensor <NUM> (the transmission device according to the embodiment) illustrated in <FIG> will be described as an example. The processing related to the transmission method according to the embodiment is performed by an IC chip (which will be described later) that functions as an image processing unit, for example, in the image sensor <NUM>. Note that the processing related to the transmission method according to the embodiment may be performed by another component that functions as an image processing unit in the image sensor <NUM>.

<FIG> san explanatory diagram illustrating an example of regions set in the image. <FIG> illustrates four regions, namely a region <NUM>, a region <NUM>, a region <NUM>, and a region <NUM> as an example of the regions. Note that it is needless to say that the region set in the image is not limited to the example illustrated in <FIG>.

As described above, it is possible to transfer data in a rectangular region such as the region <NUM> in <FIG> in a case in which the existing technique is used.

However, it is difficult to transfer data in a region with an arbitrary shape other than the rectangular shape set in the image, such as the region <NUM>, the region <NUM>, and the region <NUM> in <FIG> even if the existing technology is used as described above. In addition, in a case in which a plurality of regions are set in a row, such as the region <NUM> and the region <NUM> in <FIG>, and in a case in which the set regions are overlaid, such as the region <NUM> and the region <NUM> in <FIG>, it is difficult to transfer data in the region even if the existing technology is used. Here, the row in the image means that the y coordinates are the same when the positions of the pixels are represented by a two-dimensional plane coordinates (x, y).

Thus, the image sensor <NUM> sets region information corresponding to the region set in the image for each row in the image.

Then, the image sensor <NUM> causes the set region information and the region data corresponding to the region to be transmitted for each row. The image sensor <NUM> causes the region information and the region data for each row to be transmitted in accordance with a predetermined order, such as an ascending order a descending order of the values of the y coordinates, for example. In addition, the image sensor <NUM> may cause the region information and the region data for each row to be transmitted in a random order.

The region information according to the embodiment is data (data group) for specifying, on the side of the reception device, the region set in the image. The region information includes, for example, information indicating the position of the row, identification information of the region included in the row, information indicating the position of a column of the region included in the row, and information indicating the size of the region included in the row.

Note that the information included in the region information can change depending on a transmission scheme, which will be described later. For example, one or both of the identification information of the region included in the row and the information indicating the size of the region included in the row may not be included in a second transmission scheme and a third transmission scheme, which will be described later. In a case in which the region is divided by using a VC number, for example, the VC number may serve as the identification information of the region included in the row according to the second transmission scheme and the third transmission scheme, which will be described later. In addition, in the case in which he region is divided by using the VC number, it is possible to use the payload length instead as the information indicating the size of the region included in the row according to the second transmission scheme and the third transmission scheme, which will be described later. Hereinafter, "a case in which the region information includes the information indicating the position of the row, the identification information of the region included in the row, the information indicating the position of the column of the region included in the row, and the information indicating the size of the region included in the row" will be exemplified.

As the information indicating the position of a row, arbitrary data with which it is possible to specify a row, such as data indicating a number of a y coordinate or data indicating a value of a y coordinate calculated by incrementing an initial value (for example, "<NUM> (zero)" is exemplified. Hereinafter, the information indicating the position of the row will be referred to as "Y" in some cases.

As the identification information of a region, arbitrary data with which it is possible to uniquely specify the region, such as data indicating an ID of a region such as a number applied to the region is exemplified. Hereinafter, the identification information of the region will be referred to as "ROI ID" or "I" in some cases.

As the information indicating the position of a column of the region, arbitrary data with which it is possible to specify the column, such as data indicating a number of an x coordinate, is exemplified. Hereinafter, the information indicating the position of the column in the region will be referred to as "ROI X" or "X" in some cases.

As the information indicating the size of a region, arbitrary data with which it is possible to specify the size of the region in the row, such as data indicating the number of pixels in the region at the row is exemplified. Hereinafter, the size of the region at the row will be referred to as the "ROI length" "ROI LEN", or "L" in some cases.

The reception device that has received the region information by the information indicating the position of the row, the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region being set in the region information as described above, for example, can perform specification of the region, specification of the position of the region, and specification of the ROI length. In addition, it is possible to uniquely specify at which position and with which ROI length the region has been set in the row by the specification of the region, the specification of the position of the region, and the specification of the RIO length being performed.

Accordingly, the communication system <NUM> can be respectively compatible with the transfer of "region data in the rectangular region, such as the region <NUM> in <FIG>" and "region data in regions with arbitrary shapes other than the rectangular shape, such as the region <NUM>, the region <NUM>, and the region <NUM> in <FIG>" by the information indicating the position of the row, the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region being set in the region information.

In addition, the image sensor <NUM> may collectively set, in the region information, the information to be included in the region information for each region included in the row. The image sensor <NUM> collectively sets, in the region information, the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region, for example, for each region included in the row. The collective setting of the region information for each region means setting of the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region, for example (an example of the information included in the region information) as a group of data in the region information.

Here, since the image sensor <NUM> sets the region information for each row, the position of the row is constant even if a plurality of regions are included. Therefore, the image sensor <NUM> sets one piece of information indicating the position of the row in the region information. Note that it is needless to say that the image sensor <NUM> can collectively set the information indicating the position of the row, the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region for each region included in the row.

For example, the reception device that has received the region information can more easily perform the specification of the region, the specification of the position of the region, and the specification of the ROI length for each region by the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region being collectively set for each region included in the row as described above. In addition, it is possible to uniquely specify which region has been set at which position with which ROI length in the row by the specification of the region, the specification of the position of the region, and the specification of the ROI length being performed.

Therefore, the communication system <NUM> can address "the transfer of the region data in the case in which a plurality of regions are set in in one row, such as the region <NUM> and the region <NUM> in <FIG>" and "the transfer of the region data in the case in which the set regions are overlaid, such as the region <NUM> and the region <NUM> in <FIG>" by the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region being collectively set for each region included in the row.

In addition, a plurality of same regions can be present in one row depending on a set region, as illustrated as the region <NUM> in <FIG>, for example. Thus, in the case in which the plurality of same regions are present in one row, the image sensor <NUM> may set the region information corresponding to the number of the same regions. In the case in which the plurality of same regions are present in one row, the image sensor <NUM> sets the identification information of the region, the information indicting the position of the column of the region, and the information indicating the size of the region corresponding to the number of the present regions, for example, in the region information.

The reception device that has received the region information can perform the specification of the region, the specification of the position of the region, and the specification of the ROI length for each of the plurality of same regions by the identification information of the region, the information indicating the position of the column of the region, and the information indicating the size of the region corresponding to the number of the same regions that are present in one row being set in the region information, for example.

Therefore, the communication system <NUM> can address the transfer of the region data in the region <NUM> in <FIG> by the region information corresponding to the number of the plurality of same regions that are present in one row being set in the region information.

Therefore, the reception device that has received the region information can uniquely specify the region set in the row by the region information that includes various kinds of information as described above, for example, being set for each row.

Note that the information included in the region information is not limited to the example described above. For example, the region information may include other data such as data indicating the number of regions included in the row (hereinafter, referred to as "Num of ROI" in some cases) and the like. Specific examples of the information included in the region information will be described later.

Note that the processing related to the transmission method according to the embodiment is not limited to the above basic processing.

For example, the image sensor <NUM> may not set, in the region information, information that does not change from the information included in the region information in a row that is caused to be transmitted immediately before, in the information included in the region information. That is, the image sensor <NUM> causes the region information, which includes information that has changed from the row from which the region information is caused to be transmitted immediately before, to be transmitted.

It is possible to cause the data amount of the region information to be further reduced by the information, which has not changed from the information included in the region information in the row to be caused to be transmitted immediately before, being not transmitted as described above. The reduction of the data amount of the region information contributes to shortening of a transfer time and reduction of a load related to the transfer in the communication system <NUM>.

Note that there may be information set in the region information in accordance with a change in other information even if the information does not change from the information included in the region information in the row that is caused to be transmitted immediately before.

For example, the image sensor <NUM> sets the identification information of the region in the region information when the information indicating the position of the column of the region is set in the region information. In addition, the image sensor <NUM> sets the identification information of the region in the region information when the information indicating the size of the region is set in the region information. That is, the image sensor <NUM> sets the identification information of the region in the region information in a case in which one or both the x coordinate and the ROI length of the region are changed from the content included in the region information in the row that is caused to be transmitted immediately before.

The reception device that has received the region information can uniquely specify the region corresponding to the content of the change by the image sensor <NUM> setting the identification information of the region in the region information in a case in which one of or both the x coordinate and the ROI length of the region are changed.

As described above, the image sensor <NUM> can cause the region information and the region data for each row to be transmitted in a predetermined order. In a case in which the row from which the region information and the region data are caused to be transmitted is the row in accordance with the predetermined order at this time, the image sensor <NUM> may not set the information indicating the position of the row in the region information.

The reception device can specify in which row the received region information and region data are included, by following the predetermined order even in a case in which the image sensor <NUM> does not set the information indicating the position of the row in the region information.

The image sensor <NUM> transmits the region information and the region data for each row by performing the basis processing described in "above <<NUM>-<NUM>>" or "the basis processing described in "above <<NUM>-<NUM>> and the exception processing described in above <<NUM>-<NUM>>". Next, the transmission scheme in which the region information and the region data are transmitted will be described.

As the transmission scheme of the region information and the region data, a first transmission scheme described in (<NUM>) below, a second transmission scheme described in (<NUM>) below, and a third transmission scheme described in (<NUM>) below are exemplified.

The image sensor <NUM> stores the region information and the region data in a payload of a packet and causes the region information and the region data to be transmitted for each row.

<FIG> is an explanatory diagram illustrating an example of data that is transmitted in a first transmission scheme related to the transmission method according to the claimed invention. <FIG> illustrates "an example in which the region information and the region data respectively corresponding to the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> are stored in a payload of a long packet of MIPI illustrated in <FIG> and are transmitted for each row".

"FS" illustrated in <FIG> represents a frame start (FS) packet in accordance with the MIPI CSI-<NUM> standard, and "FE" illustrated in <FIG> is a frame end (FE) packet in accordance with the MIPI CSI-<NUM> standards (the same applies to the other diagrams).

"Embedded Data" illustrated in <FIG> is data that can be embedded in a header or a footer of the data to be transmitted. As "Embedded Data", additional information that is additionally transmitted by the image sensor <NUM> is exemplified. Hereinafter, Embedded Data will be referred to as "EBD" in some cases.

As the additional information according to the embodiment, one or two or more of information indicating the data amount of the region, information indicating the size of the region, and information indicating priority of the region are exemplified.

As the information indicating the data amount of the region, data in an arbitrary form with which it is possible to specify the data amount of the region such as "the number of pixels included in the region (or the data amount of the region) and data indicating the data amount of the header" is exemplified. The reception device can specify the data amount of each region by the information indicating the data amount of the region being transmitted as "Embedded Data" illustrated in <FIG>. That is, it is possible to cause the reception device to specify the data amount of the region even in a case in which the reception device does not have a function of calculating the data amount of each region on the basis of the region information, by the information indicating the data amount of the region being transmitted as "Embedded Data" illustrated in <FIG>.

As the information indicating the size of the region, data in an arbitrary form with which it is possible to specify the size of the region, such as "the data indicating the rectangular region including the region (for example, data indicating the number of pixels in the horizontal direction and the number of pixels in the vertical direction in the rectangular region) is exemplified.

The information indicating priority of regions is data used in processing of region data, for example. In one example, the priority indicated by the information indicating priority of regions is utilized for an order in which the regions are processed, in processing performed in a case in which the set regions are overlaid such as the region <NUM> and the region <NUM> in <FIG>, and the like.

Note that the additional information according to the embodiment is not limited to the example described above. As the additional information according to the embodiment, various kinds of data such as exposure information indicating an exposure value or the like of the image sensor device and gain information indicating a gain of the image sensor device are exemplified. Each of the exposure value indicated by the exposure information and the gain indicated by the gain information is set in the image sensor device under control performed by the processor <NUM> via the control bus B2.

<FIG> is an explanatory diagram for describing an example of Embedded Data transmitted according to the first transmission scheme related to the transmission method according to the embodiment. <FIG> illustrates an example in which the information indicating the size of the region is transmitted as "Embedded Data" illustrated in <FIG>" and the transmitted information indicating the size of the region is data indicating a minimum rectangular region including the region. In addition, <FIG> illustrates an example in which four regions, namely the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> have been set similarly to <FIG>.

The reception device can specify the minimum rectangular region including the region <NUM> illustrated as R1 in <FIG>, the minimum rectangular region including the region <NUM> illustrated as R2 in <FIG>, the minimum rectangular region including the region <NUM> illustrated as R3 in <FIG>, and the minimum rectangular region including the region <NUM> illustrated as R4 in <FIG> by the information indicating the size of the region being transmitted as "Embedded Data" illustrated in <FIG>. That is, it is possible to cause the reception device to specify the minimum rectangular region including each region on the basis of the region information even in the case in which the reception device does not have a function of specifying the minimum rectangular region including each region on the basis of the region information, by the information indicating the size of the region being transmitted as "Embedded Data" illustrated in <FIG>. Note that it is needless to say that the information indicating the size of the region is not limited to the data indicating the minimum rectangular region including each region.

As the information indicating the priority of regions, data in an arbitrary form with which it is possible to specify priority of regions such as data in which ROI IDs are aligned in an order from the highest priority or data in which ROI IDs are aligned in an order from the lowest priority is exemplified. The reception device can specify the processing order of the regions or which region is to be processed with priority, for example, by the information indicating the priority of the regions being transmitted as "Embedded Data" illustrated in <FIG>. That is, it is possible to control the processing performed by the reception device on the regions by the information indicating the priority of the regions being transmitted as "Embedded Data" illustrated in <FIG>.

Note that it is needless to say that the respective examples of the information indicating the data amount of the region, the information indicating the size of the region, and the information indicating the priority of the region transmitted as "Embedded Data" illustrated in <FIG> are not limited to the examples described above.

"PH" illustrated in <FIG> is a packet header of a long packet. Here, the packet header of the long packet related to the first transmission scheme may also function as data (change information) indicating whether or not the information included in the region information has changed from the region information included in a packet that is caused to be transmitted immediately before. That is, it is possible to state that "PH" illustrated in <FIG> is data indicating a data type of the long packet.

In one example, the image sensor <NUM> sets "0x38" in "PH" in a case in which the information included in the region information has changed from the region information included in the packet that is caused to be transmitted immediately before. In this case, the image sensor <NUM> stores the region information in the payload of the long packet.

In another example, the image sensor <NUM> sets "0x39" in "PH" in a case in which the information included in the region information has not changed from the region information included in the packet that is caused to be transmitted immediately before. In this case, the image sensor <NUM> does not store the region information in the payload of the long packet. That is, in the case in which the information included in the region information has not changed from the region information included in the packet that is caused to be transmitted immediately before, the image sensor <NUM> does not transmit the region information.

Note that it is needless to say that the data set in "PH" is not limited to the examples described above.

"Info" illustrated in <FIG> is region information stored in the payload (the same applies to the other diagrams). As illustrated in <FIG>, the region information is stored in the head portion of the payload. Hereinafter, region information will be referred to as "ROI Info" in some cases.

Each of "<NUM>", "<NUM>", "<NUM>", and "<NUM>" illustrated in <FIG> corresponds to the region data of the region <NUM>, the region data of the region <NUM>, the region data of the region <NUM>, and the region data of the region <NUM> that are stored in the payload (the same applies to the other drawings). Note that although the respective pieces of region data are illustrated in a divided manner in <FIG>, this represents the division for convenience, and there is no division in the data to be stored in the payload (the same applies to the other drawings). Hereinafter, the region data will be referred to as "ROI DATA" in some cases.

<FIG> is an explanatory diagram illustrating an example of a configuration of a long packet transmitted according to the first transmission scheme related to the transmission method according to the embodiment.

"ROI Info Type" illustrated in <FIG> is data indicating a transfer scheme of "ROI Info Parameter". The reception device that has received the region information can uniquely specify the content of "ROI Ingo Parameter" by "ROI Info Type" being set.

In addition, "ROI Info Type" corresponds to other data indicating a data type of the long packet. That is, for the long packet related to the first transmission scheme, two items, namely "PH" and "ROI Info Type" are defined as data types of the long packet.

"ROI Info Type" is <NUM>-byte data, for example, and has the following meaning when represented by Verilog-HDL description. Note that it is needless to say that "ROI Info Type" is not limited to <NUM>-byte data.

In addition, data in accordance with "ROI Info Type" is stored in "ROI Info Parameter" illustrated in <FIG>.

Note that the region information related to the first transmission scheme may include a flag (hereinafter, referred as "NO CHANGE flag") indicating that both the x coordinate and the ROI length of the region have not changed from the region information included in the packet that is caused to be transmitted immediately before, for example. In a case in which the NO CHANGE flag indicates that there has been no change (for example, in a case in which the NO CHANGE flag is <NUM>'b1), [<NUM>] and [<NUM>] described above of "ROI Info Type" may be omitted.

As the NO CHANGE flag, a most significant bit (MSB) of the ROI ID is exemplified. Note that the NO CHANGE flag may be data that is separate from the ROI ID.

<FIG> is an explanatory diagram illustrating an example of data transmitted according to the first transmission scheme related to the transmission method according to the embodiment and is a diagram specifically illustrating the data illustrated in <FIG> in accordance with the configuration of the long packet illustrated in <FIG>.

In the first transmission scheme, the region information and the region data corresponding to each of the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> are stored in the payload of the long packet of MIPI as illustrated in <FIG>, for example, and are transmitted for each row.

As described above, the image sensor <NUM> can transmit, as Embedded Data" illustrated in <FIG>, one or two or more of the information indicating the data amount of the region, the information indicating the size of the region, and the information indicating the priority of the region (one example of the additional information).

The first transmission scheme related to the transmission method according to the embodiment has features described below, for example. Note that it is needless to say that the features that the first transmission scheme related to the transmission method according to the embodiment has are not limited to the examples described below.

In addition, the region information that is stored in the head portion of the payload and is then transmitted has the features as described below, for example, in the first transmission scheme. Note that it is needless to say that the features that the region information related to the first transmission scheme has are not limited to the examples described below.

In addition, the first transmission scheme has the advantages described below, for example.

Next, an example of a configuration of the image sensor <NUM> that is compatible with the aforementioned first transmission scheme and an example of a configuration of the processor <NUM> that is compatible with the first transmission scheme (an example of a configuration related to processing of the data transmitted from the image sensor <NUM>) will be described.

<FIG> is a hardware block diagram illustrating an example of a configuration of the image sensor <NUM> that is compatible with the first transmission scheme related to the transmission method according to the embodiment. The image sensor <NUM> includes an image sensor device <NUM> and an IC chip <NUM>, for example. The image sensor <NUM> illustrated in <FIG> operates using electrical power supplied from an internal power source (not illustrated) that is included in the communication system <NUM>, such as a battery, or electrical power supplied from an external power source of the communication system <NUM>.

The image sensor device <NUM> is "an imaging device such as a digital still camera" or an image sensor device of an arbitrary scheme capable of generating an image, such as an "infrared sensor" or a "distance image sensor".

In one example, the imaging device that functions as the image sensor device <NUM> includes lens/imaging elements and a signal processing circuit.

The lens/imaging elements include for example, an optical system lens and an image sensor using a plurality of imaging elements such as complementary metal oxide semiconductors (CMOSs) or charge coupled devices (CCDs).

The signal processing circuit includes an automatic gain control (AGC) circuit or an analog to digital converter (ADC), for example, and converts an analog signal generated by the imaging elements into a digital signal (image data). In addition, the signal processing circuit performs various kinds of processing related to RAW development, for example. Further, the signal processing circuit may perform various kinds of signal processing such as white balance correction processing, color tone correction processing, gamma correction processing, YCbCr conversion processing, and edge emphasis processing, for example.

In addition, the signal processing circuit may perform processing related to setting of a region in the image and deliver the region designation information to the IC chip <NUM>. Further, the signal processing circuit may deliver a variety of kinds of data such as exposure information and gain information to the IC chip <NUM>.

The signal indicating the image generated by the image sensor device <NUM> is delivered to the IC chip <NUM>. Note that in a case in which the signal indicating the image to be delivered from the image sensor device <NUM> to the IC chip <NUM> is an analog signal, the IC chip <NUM> converts the analog signal to a digital signal by using an ADC provided therein and processes the image data obtained by the conversion, for example. Hereinafter, a case in which image data is delivered from the image sensor device <NUM> to the IC chip <NUM> will be described as an example.

The IC chip <NUM> is an integrated circuit (IC) with a circuit related to a function of transmitting data according to the first transmission scheme formed into a chip shape, processes the image data delivered from the image sensor device <NUM>, and transmits data corresponding to the generated image. The data corresponding to the image is image data (that is, data indicating the entire image) delivered from the image sensor device <NUM> or region information and region data. Note that the circuit related to the function of transmitting data according to the first transmission scheme is not limited to implementation in the form of one IC chip and may be formed of a plurality of IC chips.

The IC chip <NUM> includes, for example, an image processing circuit <NUM>, a LINK control circuit <NUM>, an ECC generation circuit <NUM>, a PH generation circuit <NUM>, an EBD buffer <NUM>, an image data buffer <NUM>, a synthesis circuit <NUM>, and a transmission circuit <NUM>.

The image processing circuit <NUM> is a circuit that has a function of performing processing related to the transmission method according to the embodiment. In a case in which the processing related to the transmission method according to the embodiment is performed, the image processing circuit <NUM> sets region information for each row in the image, and causes the LINK control circuit <NUM>, the ECC generation circuit <NUM>, the PH generation circuit <NUM>, the EBD buffer <NUM>, the image data buffer <NUM>, the synthesis circuit <NUM>, and the transmission circuit <NUM> to transmit the set region information and the region data corresponding to the region according to the first transmission scheme. In addition, the image processing circuit <NUM> also can cause the image data delivered from the image sensor device <NUM> (that is, data indicating the entire image) to be transmitted for each row.

As the image processing circuit <NUM>, a processor such as an MPU is exemplified.

Functions that the image processing circuit <NUM> has will be described separately for functional blocks. As illustrated in <FIG>, the image processing circuit <NUM> has a region cutting unit <NUM>, an image processing control unit <NUM>, and an encoding unit <NUM>, for example.

The region cutting unit <NUM> plays a role in performing processing related to setting of a region in the image and sets the region (ROI) in the image indicated by the image data delivered from the image sensor device <NUM>. The region cutting unit <NUM> performs processing related to setting of the region in the image in accordance with a set operation mode, for example. In a case in which the operation mode is an operation mode in which the region data is transmitted, for example, the region cutting unit <NUM> performs the processing related to the setting of the region in the image. In addition, in a case in which the operation mode is an operation mode in which the data indicating the entire image is transmitted, the region cutting unit <NUM> does not perform the processing related to the setting of the region in the image.

The region cutting unit <NUM> detects an object by performing arbitrary object detection processing on the image, for example, and sets a region including the detected object for each detected object. In addition, the region cutting unit <NUM> may set a region designated through an operation performed on an arbitrary operation device or the like. As the region set by the region cutting unit <NUM>, a rectangular region as illustrated as the region <NUM> in <FIG> and regions with arbitrary shapes other than the rectangular shape set in the image, such as the region <NUM>, the region <NUM>, and the region <NUM> in <FIG>, can be included.

In a case in which the region has been set, the region cutting unit <NUM> delivers the region designation information indicating the set region to the image processing control unit <NUM>, for example. In addition, in a case in which the region has not been set, the region cutting unit <NUM> does not deliver the region designation information to the image processing control unit <NUM>.

In addition, the region cutting unit <NUM> delivers the image data, which has been delivered from the image sensor device <NUM>, to the encoding unit <NUM>.

The image processing control unit <NUM> plays a role in performing the processing related to the transmission method according to the embodiment, sets the region information for each row in the image, and delivers the set region information to the encoding unit <NUM> and the PH generation circuit <NUM>.

The image processing control unit <NUM> specifies the region included in each row in the image on the basis of the region designation information acquired from the region cutting unit <NUM> and the region designation information (not illustrated) acquired form an external device, for example. Then, the image processing control unit <NUM> sets the region information for each row on the basis of the specified region. At this time, the image processing control unit <NUM> may not set the information, which has not changed from the information included in the region information in the row that is caused to be transmitted immediately before, in the region information as described above as the exception processing.

In addition, in a case in which the region designation information has not been acquired, the image processing control unit <NUM> does not set the region information.

Note that the processing performed by the image processing control unit <NUM> is not limited to the examples described above.

For example, the image processing control unit <NUM> may generate frame information, for example, and deliver the generated frame information to the LINK control circuit <NUM>. As the frame information, a VC number applied to each frame is exemplified. In addition, the frame information may include data indicating a data type such as YUV data, RGB data, or RAW data.

In addition, the image processing control unit <NUM> may perform processing of setting the additional information and deliver the set additional information to the EBD buffer <NUM>.

As the processing of setting the additional information, processing of generating the additional information is exemplified. As the processing of generating the additional information, one or two or more of processing of generating information indicating the data amount of the region, processing of generating information indicating the size of the region, and processing of generating information indicating priority of the region are exemplified.

Note that the processing of setting the additional information is not limited to the processing of generating the additional information. For example, the image processing control unit <NUM> may set, as the additional information, information acquired from the image sensor device <NUM> such as the exposure information and the gain information. In addition, the image processing control unit <NUM> may set, as the additional information, a variety of data items related to the region, such as data indicating a physical region length, data indicating an output region length, data indicating an image format, and data indicating the total data amount, for example. As the physical region length, the number of pixels of the image sensor device <NUM> is exemplified. As the output region length, the number of pixels (the length on the image) of the image output from the image sensor device <NUM> is exemplified.

The encoding unit <NUM> encodes the image data delivered from the image sensor device <NUM> in a predetermined scheme such as a scheme that is compatible with Joint Photographic Experts Group (JPEG), for example.

In a case in which the region information has not been acquired from the image processing control unit <NUM>, the encoding unit <NUM> delivers the encoded image data to the image data buffer <NUM>. Hereinafter, the encoded image data, that is, the data indicating the entire encoded image will be referred to as "ordinary data" in some cases.

In addition, in a case in which the region information is acquired from the image processing control unit <NUM>, the encoding unit <NUM> delivers the acquired region information and the encoded region data corresponding to the region to the image data buffer <NUM>.

The image processing circuit <NUM> performs processing related to the transmission method according to the embodiment by having the region cutting unit <NUM>, the image processing control unit <NUM>, and the encoding unit <NUM>. Note that the functional blocks of the image processing circuit <NUM> illustrated in <FIG> are obtained by dividing the functions that the image processing circuit <NUM> has for convenience, and the dividing of the functions of the image processing circuit <NUM> is not limited to the example illustrated in <FIG>.

The LINK control circuit <NUM> delivers the frame information to the ECC generation circuit <NUM>, the PH generation circuit <NUM>, and the synthesis circuit <NUM> for each row, for example.

The ECC generation circuit <NUM> sets an error correction code for each row. The ECC generation circuit <NUM> generates an error correction code of one row in the frame information (for example, the VC number. the data type, or the like) on the basis of data of the row. The ECC generation circuit <NUM> delivers the generated error correction code to the PH generation circuit <NUM> and the synthesis circuit <NUM>, for example. In addition, the ECC generation circuit <NUM> may generate the error correction code in conjunction with the PH generation circuit <NUM>.

The PH generation circuit <NUM> generates a packet header for each row by using the frame information.

The PH generation circuit <NUM> may generate the packet header on the basis of the region information delivered from the image processing circuit <NUM> (the image processing control unit <NUM> in the example in <FIG>). Specifically, the PH generation circuit <NUM> sets "the data indicating whether or not the information included in the region information has changed from the region information included in the packet that is caused to be transmitted immediately before" (change information) as described above in the packet header on the basis of the region information.

The EBD buffer <NUM> is a buffer that temporarily holds the additional information delivered from the image processing circuit <NUM> (the image processing control unit <NUM> in the example in <FIG>). The EBD buffer <NUM> outputs the additional information as "Embedded Data" to the synthesis circuit <NUM> at a predetermined timing. Note that "Embedded Data" output from the EBD buffer <NUM> may be delivered to the synthesis circuit <NUM> via the image data buffer <NUM>, which will be described later.

The image data buffer <NUM> is a buffer that temporarily hold data (the ordinary data or the region information and the region data) delivered from the image processing circuit <NUM> (the encoding unit <NUM> in the example in <FIG>). The image data buffer <NUM> outputs the held data to the synthesis circuit <NUM> for each row at a predetermined timing.

The synthesis circuit <NUM> generates a packet to be transferred on the basis of data respectively acquired from the ECC generation circuit <NUM>, the PH generation circuit <NUM>, the EBD buffer <NUM>, and the image data buffer <NUM>, for example.

The transmission circuit <NUM> transmits the packet delivered from the synthesis circuit <NUM> via the data bus B1 for each row (an example of a signal transfer path; the same applies to the following description). For example, the transmission circuit <NUM> transmits the packet by a high-speed differential signal as illustrated in <FIG>.

In a case in which the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> are set, for example, the transmission circuit <NUM> transmits the set region information and the region data corresponding to the region in a long packet for each row as illustrated in <FIG>.

In addition, in a case in which no region is set, that is, in a case in which ordinary data is output from the image data buffer <NUM>, the transmission circuit <NUM> transmits the long packet in which the data corresponding to each row is stored in the payload, for each row. Note that the transmission circuit <NUM> can transmit the additional information as "Embedded Data" even in this case.

The image sensor <NUM> corresponding to the first transmission scheme has a hardware configuration illustrated in <FIG>, for example. Note that it is needless to say that the hardware configuration of the image sensor <NUM> corresponding to the first transmission scheme is not limited to the example illustrated in <FIG>.

<FIG> is a functional block diagram illustrating an example of a configuration of the processor <NUM> that is compatible with the first transmission scheme related to the transmission method according to the embodiment. The processor <NUM> has a header separation unit <NUM>, a header interpretation unit <NUM>, a payload separation unit <NUM>, an image processing unit <NUM>, and a transmission unit <NUM>, for example.

The header separation unit <NUM> separates header data corresponding to the header portion and payload data corresponding to the payload portion from the received data. The header separation unit <NUM> separates the header data from the received data in accordance with a rule defined in advance by standards or the like, for example. In addition, the header separation unit <NUM> may separate the payload data from the received data in accordance with the rule defined in advance by standards or the like or may separate the payload data from the received data on the basis of a result of processing performed by the header interpretation unit <NUM>, for example.

The header interpretation unit <NUM> interprets content indicated by the header data.

In one example, the header interpretation unit <NUM> interprets that the information included in the region information has changed from the region information included in the packet that is caused to be transmitted immediately before in a case in which the header data indicates "0x38", for example. In addition, the header interpretation unit <NUM> may interpret that the region information is included in the payload data in this case.

In another example, the header interpretation unit <NUM> interprets that the information included in the region information has not changed from the region information included in the packet that is caused to be transmitted immediately before in a case in which the header data indicates "0x39", for example. In addition, the header interpretation unit <NUM> may interpret that the region information is not included in the payload data in this case.

Note that the example of the processing performed by the header interpretation unit <NUM> is not limited to the example described above. For example, the header interpretation unit <NUM> may specify the position of the payload data and deliver the specified position to the header separation unit <NUM>. In addition, the header interpretation unit <NUM> can also interpret whether or not the payload data is "Embedded Data".

The payload separation unit <NUM> separates the additional information, the region information, and the image data (the ordinary data or the region data; the same applies to the description of <FIG> below) from the payload data on the basis of the result of the interpretation performed by the header interpretation unit <NUM>.

For example, in a case in which the header interpretation unit <NUM> determines that the payload data is "Embedded Data", the payload separation unit <NUM> regards the payload data as the additional information.

In addition, in a case in which the header interpretation unit <NUM> does not interpret that the payload data is "Embedded Data", the payload separation unit <NUM> separates the region information and the image data from the payload data. As described above with reference to <FIG>, <FIG>, and <FIG>, for example, the region information and the region data are stored in the payload while the region information is stored in the head portion of the payload. In addition, in the case in which the ordinary data is transmitted from the image sensor <NUM>, the region information is not stored in the payload. Accordingly, the payload separation unit <NUM> can separate the region information and the image data by processing the payload from the head portion.

The image processing unit <NUM> processes the image data for each row on the basis of the additional information, the region information, and the image data delivered from the payload separation unit <NUM> and obtains data indicating the image corresponding to the region or the data indicating the entire image.

The image processing unit <NUM> may determine whether or not the data acquired via the data bus B1 is data indicating the image corresponding to the region or data indicating the entire image on the basis of the information delivered from the payload separation unit <NUM>, for example, and perform processing in a processing mode in accordance with the result of the determination.

In a case in which the region information is delivered when the processing targeted at an image of a certain frame is performed for the first time (that is, when the processing corresponding to the first row is performed), for example, the image processing unit <NUM> determines that the data acquired via the data bus B1 is data indicating the image corresponding to the region. Then, the image processing unit <NUM> proceeds to the processing mode in which the region data is processed and obtains data indicating the image corresponding to the region on the basis of the region information and the region data for each row.

In a case in which the region information is not delivered when the processing targeted at the image of a certain frame is performed for the first time, for example, the image processing unit <NUM> determines that the data acquired via the data bus B1 is data indicating the entire image. Then, the image processing unit <NUM> proceeds to the ordinary processing mode (the processing mode in which the ordinary data is processed) and obtains the data indicating the entire image on the basis of the ordinary data for each row.

Note that the method of determining the data acquired via the data bus B1 is not limited to the example described above. For example, the image processing unit <NUM> may determine that the data acquired via the data bus B1 is data indicating the image corresponding to the region in a case in which the information related to the region (for example, one or two or more of the information indicating the data amount of the region, the information indicating the size of the region, and the information indicating priority of the region).

The image processing unit <NUM> has a decoding unit <NUM> and an image generation unit <NUM>, for example.

The decoding unit <NUM> decodes the image data by a predetermined scheme corresponding to the encoding performed by the image sensor <NUM>, for example. At this time, the decoding unit <NUM> may change content of the processing on the basis of the additional information.

The image generation unit <NUM> generates the data indicating the image corresponding to the region or the data indicating the entire image from the image data for each row decoded by the decoding unit <NUM> on the basis of the region information.

In a case in which the region information is delivered when the processing targeted at the image of a certain frame is performed for the first time, for example, the image generation unit <NUM> determines that the image data delivered from the decoding unit <NUM> is region data. Then, in a case in which it is determined that the image data is the region data, the image generation unit <NUM> divides the region data for each row into the data for each region on the basis of the region information. Here, the region information that the image generation unit <NUM> uses for the processing can include the region information corresponding to the row as a target of the processing and region information corresponding to a row for which the processing has already been performed.

Here, since the region information basically includes the information indicating the position of the row, the identification information of the region included in the row, the information indicating the position of the column of the region included in the row, and the information indicating the size of the region included in the row, the image generation unit <NUM> can uniquely specify the region data included in each row. Accordingly, the image generation unit <NUM> can generate the data indicating the image corresponding to the region set in the image of a certain frame, for each region.

In addition, in a case in which the region information is not delivered when the processing targeted at the image of a certain frame is performed for the first time, the image generation unit <NUM> determines that the delivered image data is ordinary data. In a case in which the image data is determined to be ordinary data, the image generation unit <NUM> generates data indicating the entire image from the image data for each row delivered from the decoding unit <NUM>.

The image processing unit <NUM> obtains the data indicating the image corresponding to the region or the data indicating the entire image by being provided with the decoding unit <NUM> and the image generation unit <NUM>, for example.

Note that the processing performed by the image processing unit <NUM> is not limited to the example described above. For example, the image processing unit <NUM> can perform a variety of kinds of processing such as processing related to control of recording of image data in a recording medium such as the memory <NUM>, processing related to control of display of an image on a display screen of the display device <NUM>, and processing related to setting of a region in an image, for example, as described above.

The transmission unit <NUM> transmits various kinds of data (for example, image data, control information, and the like) to an external device via the control bus B2, various signal lines, and the like. As the transmission unit <NUM>, communication devices that are provided in the processor <NUM> and follow predetermined standards such as inter-integrated circuits (I2C) or improved inter-integrated circuits (I3C) are exemplified.

The communication performed by the transmission unit <NUM> is controlled by the image processing unit <NUM>, for example. The recording of the image data in the recording medium such as the memory <NUM>, the display of the image on the display screen of the display device <NUM>, and the transmission of the region designation information indicating the set region o the image sensor <NUM> are realized by the image processing unit <NUM> causing the transmission unit <NUM> to transmit the various kinds of data (for example, the image data, the control information, and the like).

The processor <NUM> can obtain the data indicating the image corresponding to the region or the data indicating the entire image on the basis of the data received from the image sensor <NUM> by being provided with the header separation unit <NUM>, the header interpretation unit <NUM>, the payload separation unit <NUM>, the image processing unit <NUM>, and the transmission unit <NUM>, for example. Note that the functional blocks of the processor <NUM> illustrated in <FIG> is obtained by dividing the functions that the processor <NUM> has for convenience, and how to divide the functions of the processor <NUM> is not limited to the example illustrated in <FIG>. In addition, in a case in which the function of transmitting various kinds of data to the external device is not provided, the processor <NUM> may not have the transmission unit <NUM>.

The image sensor <NUM> stores the region data in a payload of a first packet and stores each piece of information included in the region information in a second packet that is different from the first packet. Then, the image sensor <NUM> causes the first packet and the second packet to be transmitted for each row.

A long packet of MIPI is exemplified as the first packet related to the second transmission scheme while a short packet of MIPI is exemplified as the second packet related to the second transmission scheme. Note that the examples of the first packet and the second packet according to the embodiment are not limited to the examples described above. Other examples of the first packet and the second packet according to the embodiment will be described as a third transmission scheme, which will be described later. The case in which the first packet is a long packet of MIPI and the second packet is a short packet of MIPI will be exemplified for the second transmission scheme described below.

<FIG> is an explanatory diagram illustrating an example of data transmitted according to the second transmission scheme related to the transmission method according to the embodiment. <FIG> illustrates "an example in which each piece of information included in the region information corresponding to the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> is transmitted for each row by a short packet of MIPI illustrated in <FIG>". In addition, <FIG> illustrates "an example in which the region data corresponding to each of the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> is stored in a payload of a long packet of MIPI illustrated in <FIG> and is transmitted for each row".

"Y" illustrated in <FIG> illustrates a short packet in which the information indicating the position of the row is stored. "I1", "I2", "I3", and "I4" illustrated in <FIG> illustrate short packets in which the identification information of the regions corresponding to the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> are respectively stored. "X1", "X2", "X3", and "X4" illustrated in <FIG> illustrate short packets in which the information indicating the sizes of the regions corresponding to the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> are respectively stored. "L1", "L2", "L3", and "l4" illustrated in <FIG> represent short packets in which information indicating the sizes of the regions respectively corresponding to the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> is stored.

In the case in which the first transmission scheme described above in (<NUM>) is used as illustrated in <FIG>, the region information and the region data are stored in a payload of one packet, such as a long packet of MIPI, for example, and are transmitted for each row.

Meanwhile, in a case in which the second transmission scheme is used as illustrated in <FIG>, the region information and the region data are stored separately in different packets and are transmitted for each row.

In addition, it is possible to transmit "Embedded Data" even in the case in which the second transmission scheme is used as illustrated in <FIG> similar to the case in which the first transmission scheme is used. That is, the image sensor <NUM> can transmit, as "Embedded Data" illustrated in <FIG>, one or two or more (an example of the additional information) of the information indicating the data amount of the region, the information indicating the size of the region, and the information indicating priority of the region.

The second transmission scheme related to the transmission method according to the embodiment has the features described below, for example. Note that it is needless to say that the features that the second transmission scheme related to the transmission method according to the embodiment has are not limited to the examples described below.

In addition, the region information transmitted by the short packet has features described below, for example, in the second transmission scheme. Note that it is needless to say that the features that the region information related to the second transmission scheme has are not limited to the examples described below.

In addition, the second transmission scheme has advantages described below, for example.

Next, an example of a configuration of the image sensor <NUM> that is compatible with the aforementioned second transmission scheme and an example of a configuration of the processor <NUM> that is compatible with the second transmission scheme (an example of a configuration related to the processing of data transmitted from the image sensor <NUM>) will be described.

<FIG> is a hardware block diagram illustrating an example of a configuration of the image sensor <NUM> that is compatible with the second transmission scheme related to the transmission method according to the embodiment. The image sensor <NUM> includes an image sensor device <NUM> and an IC chip <NUM>, for example. The image sensor <NUM> illustrated in <FIG> operates using electrical power or the like supplied from an internal power source (not illustrated) that is included in the communication system <NUM> similarly to the image sensor <NUM> illustrated in <FIG>.

The image sensor device <NUM> illustrated in <FIG> has a function and a configuration that are similar to those of the image sensor device <NUM> illustrated in <FIG>.

The IC chip <NUM> is obtained by forming an IC, on which circuits related to the function of transmitting data according to the second transmission scheme are integrated, into a chip shape, processes the image data delivered from the image sensor device <NUM>, and transmits data corresponding to the generated image. Note that the circuits related to the function of transmitting data according to the second transmission scheme is not limited to implementation in the form of one IC chip and may be formed of a plurality of IC chips.

The IC chip <NUM> includes an image processing circuit <NUM>, a LINK control circuit <NUM>, an ECC generation circuit <NUM>, a PH generation circuit <NUM>, an EBD buffer <NUM>, an image data buffer <NUM>, a synthesis circuit <NUM>, and a transmission circuit <NUM>, for example.

The link control circuit <NUM>, the ECC generation circuit <NUM>, the EBD buffer <NUM>, the image data buffer <NUM>, and the transmission circuit <NUM> illustrated in <FIG> have functions and configurations that are similar to those of the LINK control circuit <NUM>, the ECC generation circuit <NUM>, the EBD buffer <NUM>, the image data buffer <NUM>, and the transmission circuit <NUM> illustrated in <FIG>, respectively.

The image processing circuit <NUM> is a circuit that has a function of performing the processing related to the transmission method according to the embodiment. In a case in which the processing related to the transmission method according to the embodiment is performed, the image processing circuit <NUM> sets the region information for each row in the image and causes the LINK control circuit <NUM>, the ECC generation circuit <NUM>, the PH generation circuit <NUM>, the EBD buffer <NUM>, the image data buffer <NUM>, the synthesis circuit <NUM>, and the transmission circuit <NUM> to transmit the set region information and the region data corresponding to the region for each row according to the second transmission scheme. In addition, the image processing circuit <NUM> can also cause the image data delivered from the image sensor device <NUM> (that is, data indicating the entire image) to be transmitted for each row.

The functions that the image processing circuit <NUM> has will be divided into functional blocks and will be described. As illustrated in <FIG>, the image processing circuit <NUM> has a region cutting unit <NUM>, an image processing control unit <NUM>, and an encoding unit <NUM>, for example.

The functions of the region cutting unit <NUM> and the image processing control unit <NUM> illustrated in <FIG> are similar to those of the region cutting unit <NUM> and the image processing control unit <NUM> illustrated in <FIG>.

The encoding unit <NUM> encodes the image data delivered from the image sensor device <NUM> by a predetermined scheme similarly to the encoding unit <NUM> illustrated in <FIG>.

In a case in which the region information is not acquired from the image processing control unit <NUM>, the encoding unit <NUM> delivers the encoded image data to the image data buffer <NUM> similarly to the encoding unit <NUM> illustrated in <FIG>.

In addition, in a case in which the region information is acquired from the image processing control unit <NUM>, the encoding unit <NUM> specifies the region on the basis of the acquired region information and delivers the encoded region data corresponding to the specified region to the image data buffer <NUM>.

The image processing circuit <NUM> performs the processing related to the transmission method according to the embodiment by being provided with the region cutting unit <NUM>, the image processing control unit <NUM>, and the encoding unit <NUM>. Note that the functional blocks of the image processing circuit <NUM> illustrated in <FIG> are obtained by dividing the functions that the image processing circuit <NUM> has for convenience, and how to divide the functions of the image processing circuit <NUM> is not limited to the example illustrated in <FIG>.

The PH generation circuit <NUM> generates a short packet in which each piece of information included in the region information is stored, on the basis of the region information delivered from the image processing circuit <NUM> (the image processing control unit <NUM> in the example in <FIG>).

In addition, the PH generation circuit <NUM> generates a packet header of a long packet for each row by using frame information similarly to the PH generation circuit <NUM> illustrated in <FIG>.

The synthesis circuit <NUM> generates, as a packet to be transferred, "a short packet in which the region information is stored" and "a long packet in which the image data (the ordinary data or the region data) or each row is stored in the payload" on the basis of the data acquired from each of the ECC generation circuit <NUM>, the PH generation circuit <NUM>, the EBD buffer <NUM>, and the image data buffer <NUM>.

The transmission circuit <NUM> transmits the packet delivered from the synthesis circuit <NUM> via the data bus B1 for each row.

In a case in which the region <NUM>, the region <NUM>, the region <NUM>, and the region <NUM> illustrated in <FIG> have been set, for example, the transmission circuit <NUM> stores each piece of information included in the region information in a short packet, stores the region data in the payload of the long packet, and causes the information and the data to be transmitted for each row as illustrated in <FIG>.

In addition, in a case in which no region is set, that is, in a case in which ordinary data is output from the image data buffer <NUM>, the transmission circuit <NUM> transmits the long packet in which the data corresponding to each row is stored in the payload, for each row similarly to the transmission circuit <NUM> illustrated in <FIG>. Note that the transmission circuit <NUM> can transmit the additional information as "Embedded Data" similarly to the transmission circuit <NUM> illustrated in <FIG>.

The image sensor <NUM> that is compatible with the second transmission scheme has a hardware configuration illustrated in <FIG>, for example. Note that it is needless to say that the hardware configuration of the image sensor <NUM> that is compatible with the second transmission scheme is not limited to the example illustrated in <FIG>.

<FIG> is a functional block diagram illustrating an example of a configuration of the processor <NUM> that is compatible with the second transmission scheme related to the transmission method according to the embodiment. The processor <NUM> has a packet separation unit <NUM>, a short packet interpretation unit <NUM>, a long packet interpretation unit <NUM>, a header interpretation unit <NUM>, a payload interpretation unit <NUM>, an image processing unit <NUM>, and a transmission unit <NUM>, for example.

The packet separation unit <NUM> separates the short packet and the long packet from the received data. The packet separation unit <NUM> separates the short packet and the long packet by detecting the division of the short packet, for example.

The short packet interpretation unit <NUM> extracts various kinds of information included in the region information from the short packet delivered from the packet separation unit <NUM> and delivers the region information to the image processing unit <NUM>.

The long packet interpretation unit <NUM> separates header data corresponding to the header portion and payload data corresponding to the payload portion from the long packet delivered from the packet separation unit <NUM> similarly to the header separation unit <NUM> illustrated in <FIG>.

The header interpretation unit <NUM> interprets content indicated by the header data. For example, the header interpretation unit <NUM> interprets whether or not the payload data is "Embedded Data". In addition, the header interpretation unit <NUM> may specify the position of the payload data and deliver the specified position to the long packet interpretation unit <NUM>.

The payload interpretation unit <NUM> extracts the additional information or the image data (the ordinary data or the region data; the same applies to the description of <FIG> below) from the payload data on the basis of the result of the interpretation performed by the header interpretation unit <NUM>.

In a case in which the header interpretation unit <NUM> interprets that the payload data is "Embedded Data", for example, the payload interpretation unit <NUM> regards the payload data as the additional information.

In addition, in a case in which the header interpretation unit <NUM> does not interpret that payload data is "Embedded Data", the payload interpretation unit <NUM> regards the payload data as image data.

The image processing unit <NUM> illustrated in <FIG> processes the image data for each row on the basis of the additional information, the region information, and the image data and obtains the data indicating the image corresponding to the region or the data indicating the entire image similarly to the image processing unit <NUM> illustrated in <FIG>.

In addition, the image processing unit <NUM> illustrated in <FIG> can perform a variety of kinds of processing such as processing related to control of recording of the image data in the recording medium such as the memory <NUM>, processing related to control of display of an image on the display screen of the display device <NUM>, and processing related to setting of the region in the image similarly to the image processing unit <NUM> illustrated in <FIG>.

The transmission unit <NUM> illustrated in <FIG> transmits variety of kinds of data (for example, the image data, the control information, and the like) to an external device via the control bus B2 or various signal lines similarly to the transmission unit <NUM> illustrated in <FIG>.

The processor <NUM> can obtain data indicating an image corresponding to the region or data indicating the entire image on the basis of the data received from the image sensor <NUM> by being provided with the packet separation unit <NUM>, the short packet interpretation unit <NUM>, the long packet interpretation unit <NUM>, the header interpretation unit <NUM>, the payload interpretation unit <NUM>, the image processing unit <NUM>, and the transmission unit <NUM>. Note that the functional blocks of the processor <NUM> illustrated in <FIG> are obtained by dividing the functions that the processor <NUM> has for convenience, and how to divide the functions of the processor <NUM> is not limited to the example illustrated in <FIG>. In Addition, in a case in which the function of transmitting various kinds of data to the external device is not provided, the processor <NUM> may not have the transmission unit <NUM> similarly to the processor <NUM> illustrated in <FIG>.

The image sensor <NUM> stores the region data in the payload of the first packet and stores the information included in the region information in the second packet that is different from the first packet similarly to the second transmission scheme described above in (<NUM>). Then, the image sensor <NUM> causes the first packet and the second packet to be transmitted for each row.

Along packet of MIPI is exemplified as the first packet related to the third transmission scheme, and a packet header is exemplified as the second packet related to the third transmission scheme. That is, the region information is transmitted by the short packet of MIPI, for example, in the second transmission scheme while the region information is transmitted by the packet header in the third transmission scheme.

Here, the short packet and the packet header have similar structures as illustrated in <FIG>. Accordingly, the image sensor <NUM> using the third transmission scheme can transmit the region information and the region data for each row similarly to the image sensor <NUM> using the second transmission scheme described above in (<NUM>).

Note that the second packet related to the third transmission scheme is not limited to the packet header that has a structure similar to that of the packet header illustrated in <FIG>. For example, the second packet related to the third transmission scheme may be a packet header (for example, a packet header obtained by expanding "Data Field" from <NUM> bytes to <NUM> bytes or more) that has a structure in which "Data Field" illustrated in <FIG> has been expanded. In a case in which the second packet related to the third transmission scheme is a packet header that has the aforementioned expanded structure, the information included in the region information can be stored in one second packet.

In addition, the region in which the information included in the region information is stored may be a part of the first packet in the third transmission scheme. That is, it is possible to regard that the aforementioned first packet and the aforementioned second packet as one packet in the third transmission scheme.

The third transmission scheme related to the transmission method according to the embodiment has features that are similar to that of the second transmission scheme described above in (<NUM>).

In addition, the third transmission scheme has advantages described below, for example.

The image sensor <NUM> that is compatible with the third transmission scheme employs a configuration similar to that (including the modification examples) of the image sensor <NUM> that is compatible with the second transmission scheme described above in (<NUM>) as illustrated in <FIG>, for example. In addition, the processor <NUM> that is compatible with the third transmission scheme employs a configuration that is similar to that (including the modification examples) of the processor <NUM> that is compatible with the second transmission scheme described above in (<NUM>) as illustrated in <FIG>, for example.

Advantages described below are achieved by the transmission method according to the embodiment being used. Note that it is needless to say that the advantages achieved by the transmission method according to the embodiment being used are not limited to the examples described below.

It is possible to transmit data of any region with an arbitrary shape set in an image by executing, by a processor or the like in a computer, a program that causes the computer to function as the transmission device according to the embodiment (for example, a program that causes the computer to execute the processing related to the transmission method according to the embodiment).

In addition, effects that are achieved by the aforementioned transmission method according to the embodiment being used can be achieved by the program that causes the computer to function as the transmission device according to the embodiment being executed by a processor or the like in the computer.

Although the above description has illustrated that the program (computer program) that causes the computer to function as the transmission device according to the embodiment is provided, the embodiment can further provide a recording medium with the aforementioned program stored therein together.

The aforementioned configurations are examples of the embodiment, and it is a matter of course that the aforementioned configuration belongs to a technical scope of the present disclosure.

Claim 1:
A transmission device included in a communication system in accordance with MIPI standards comprising:
an image processing circuitry having a Region-Of-Interest (ROI) mode, in which Region-Of-Interest (ROI) information of one or more Regions-Of-Interest (ROIs) in an image is transmitted by the image processing circuitry, and a non-Region-Of-Interest (ROI) mode, in which the Region-Of-Interest (ROI) information of the one or more Regions-Of-Interest (ROIs) is not transmitted,
the image processing circuitry, in the ROI mode, is configured to
transmit a first frame of data of an image, the first frame of data including embedded data, Region-Of-Interest (ROI) information for each image row of a first plurality of image rows of the image, and Region-Of-Interest (ROI) data for each image row of a second plurality of image rows of the image, the second plurality of image rows including the first plurality of image rows and image rows for which the information included in the Region-Of-Interest (ROI) information has not changed from the Region-Of-Interest (ROI) information included in the image row that is caused to be transmitted immediately before, and
the image processing circuitry, in the non-ROI mode, is configured to
transmit a second frame of data of the image, the second frame of data including all payload data for each image row of a third plurality of image rows of the image, the third plurality of image rows including the second plurality of image rows, and the second frame of data does not include the ROI information and the ROI data, wherein
the Region-Of-Interest (ROI) data for each image row of the second plurality of image rows indicate an image sensing result corresponding to the Region-Of-Interest (ROI) to be transmitted, and
the Region-Of-Interest (ROI) information for each image row of the first plurality of image rows includes information indicating a position of the image row and a position of a column of one or more Regions-Of-Interest (ROIs) included in the each image row, and wherein
the Region-Of-Interest (ROI) information further includes identification information of the one or more Regions-Of-Interest (ROIs) included in the image row and information indicating a size of the one or more Regions-Of-Interest (ROIs) included in the image row.