Patent Description:
For Camera Serial Interface <NUM> (CSI-<NUM>) ver. <NUM>, which is currently being standardized, amendments and studies are underway for the purpose of wider use not only for mobile devices but also for various applications such as on-board items and Internet of Things (IoT).

By the way, requirements placed on on-board cameras are different from those on mobile devices, for example, on-board cameras are required to comply with functional safety stipulated by the ISO <NUM> standard. In particular, in an autonomous driving vehicle, a computer recognizes an environment on the basis of an image imaged by an on-board camera to perform autonomous traveling. Therefore, it is indispensable to have enough reliability to prevent an error from occurring in application as an on-board camera.

For example, even if a vehicle collides and some error occurs such as a disconnection of wiring which connects an on-board camera and a computer, continuous image transmission is required to be performed using another wiring which is not disconnected. This enables the computer to perform self-driving, and to evacuate the vehicle to the shoulder to stop the vehicle safely.

Furthermore, Patent Document <NUM> proposes a system capable of reducing the number of data buses when connecting a processing device and a plurality of image sensors by using the CSI-<NUM> standard.

Patent Document <NUM> discloses a System, methods and an apparatus that support multimode operation of a data communication interface. A method includes initializing a scrambler with a first pseudo-random binary sequence seed word after receiving a first sync word, the first sync word preceding a first packet, using the scrambler and the first pseudo-random binary sequence seed word to scramble a first copy of a packet header that succeeds the first sync word in the first packet, initializing the scrambler with a second pseudo-random binary sequence seed word after scrambling the first copy of the packet header, the second sync word succeeding the first copy of the packet header in the first packet, using the scrambler and the second pseudo-random binary sequence seed word to scramble a second copy of the packet header that succeeds the second sync word in the first packet.

Patent Document <NUM> discloses a redundant transfer unit of the transmitter that redundantly transfers redundant transfer data, which has the same contents as those of data to communication paths. A data division transfer unit adds a class identifier to division data obtained by dividing data and distributes the divided division data into the communication paths. Data transmitting units transmit the redundant transfer data or the division data through the communication paths. Data receiving units of the receiver receives the transmitted data. Data analyzing units determine the classes of the received data. A data selecting unit selects one of the redundant transfer data, and outputs the selected data. A data merging unit merges the division data and outputs data.

However, a packet structure in the existing CSI-<NUM> standard is specialized for use in mobile devices, so that it is difficult to ensure the reliability required for application as on-board cameras such as those described above.

The present disclosure has been made in view of such a situation, and is intended to make it possible to achieve higher reliability.

According to a first aspect the present invention provides a communication apparatus in accordance with independent claim <NUM>. According to a second aspect the present invention provides a communication method in accordance with independent claim <NUM>. According to a third aspect the present invention provides a program in accordance with pseudo dependent claim <NUM>. According to a fourth aspect the present invention provides a communication apparatus in accordance with independent claim <NUM>. According to a fifth aspect the present invention provides a communication method in accordance with independent claim <NUM>. According to a sixth aspect the present invention provides a program in accordance with pseudo dependent claim <NUM>. Further aspects are set forth in the dependent claims, the drawings and the following description.

Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

<FIG> is a block diagram illustrating an example configuration of a first embodiment of a communication system to which the present technology is applied.

As illustrated in <FIG>, a communication system <NUM> includes an image sensor <NUM>, an application processor <NUM>, a serializer <NUM>, and a deserializer <NUM>. Then, the image sensor <NUM> and the serializer <NUM> are connected via a bus <NUM>-<NUM>, the application processor <NUM> and the deserializer <NUM> are connected via a bus <NUM>-<NUM>, and the serializer <NUM> and the deserializer <NUM> are connected via a bus <NUM>. For example, the communication system <NUM> is used for connection in an existing on-board camera.

The image sensor <NUM> is configured by incorporating an extension mode-supporting CSI-<NUM> transmitter <NUM> together with, for example, a lens and an imaging element (neither thereof illustrated). For example, the image sensor <NUM> transmits image data of an image acquired by imaging by the imaging element to the application processor <NUM> by the extension mode-supporting CSI-<NUM> transmitter <NUM>.

The application processor <NUM> is configured by incorporating an extension mode-supporting CSI-<NUM> receiver <NUM> together with a large scale integration (LSI) which performs a process depending on various applications executed by a mobile device including the communication system <NUM>. For example, the application processor <NUM> can receive image data transmitted from the image sensor <NUM> by the extension mode-supporting CSI-<NUM> receiver <NUM>, and can perform a process on the image data depending on the application by the LSI.

The serializer <NUM> includes a CSI-<NUM> receiver <NUM> and a serializer deserializer (SerDes) transmitter <NUM>. For example, the serializer <NUM> acquires a bit parallel signal transmitted from the image sensor <NUM> by the CSI-<NUM> receiver <NUM> communicating with the extension mode-supporting CSI-<NUM> transmitter <NUM> in conformity with the normal CSI-<NUM> standard. Then, the serializer <NUM> converts the acquired signal into a bit serial signal, and the signal is transmitted to the deserializer <NUM> by the SerDes transmitter <NUM> communicating with a SerDes receiver <NUM> in one lane.

The deserializer <NUM> includes the SerDes receiver <NUM> and a CSI-<NUM> transmitter <NUM>. For example, the deserializer <NUM> acquires a bit serial signal transmitted by the SerDes receiver <NUM> communicating with the SerDes transmitter <NUM> in one lane. Then, the deserializer <NUM> converts the acquired signal into a bit parallel signal, and the signal is transmitted to the application processor <NUM> by the CSI-<NUM> transmitter <NUM> communicating with the extension mode-supporting CSI-<NUM> receiver <NUM> in conformity with the normal CSI-<NUM> standard.

The buses <NUM>-<NUM> and <NUM>-<NUM> are communication paths which transmit signals in conformity with the CSI-<NUM> standard, and for example, a transmission distance which is a distance at which signals can be transmitted is about <NUM>. Furthermore, as illustrated in the figure, the bus <NUM> connects the image sensor <NUM> and the application processor <NUM> with a plurality of signal lines (HS-GPIO, I2C or I3C, CLKP/N, D0P/N, D1P/N, D2P/N, and D3P/N).

The bus <NUM> is a communication path which transmits signals in conformity with a SerDes standard other than the CSI-<NUM>, such as flat panel display (FPD) -link III, and for example, a transmission distance which is a distance at which signals can be transmitted is about <NUM>.

Here, in the communication system <NUM>, the extension mode-supporting CSI-<NUM> transmitter <NUM> and the extension mode-supporting CSI-<NUM> receiver <NUM> support communication in the extension mode in which the CSI-<NUM> standard is extended, and can transmit and receive, for example, packets used in the extension mode (hereinafter referred to as extended packets) as illustrated in <FIG> as described later. For example, the extension mode-supporting CSI-<NUM> receiver <NUM> has an error recognition function of recognizing an error which occurs on the bus <NUM>-<NUM> on the basis of an extended packet transmitted via the bus <NUM>-<NUM>. Note that detailed configurations of the extension mode-supporting CSI-<NUM> transmitter <NUM> and the extension mode-supporting CSI-<NUM> receiver <NUM> will be described later with reference to <FIG> and <FIG>.

Furthermore, in the communication system <NUM>, the CSI-<NUM> receiver <NUM> has an error recognition function of recognizing an error which occurs on the bus <NUM>-<NUM> on the basis of an extended packet transmitted via the bus <NUM>-<NUM>. Note that a detailed configuration of the CSI-<NUM> receiver <NUM> will be described later with reference to <FIG>.

By the way, in a case where the communication system <NUM> is adopted in on-board cameras such as those described above, higher reliability is required, for example, continuous communication can be performed quickly even if a defect occurs, and therefore, it is necessary to take measures to deal with two points described below.

Conventionally, a packet transmitted from the image sensor <NUM> to the application processor <NUM> has a packet structure as illustrated in <FIG>, for example, and is distributed to a plurality of lanes and transmitted.

An upper side of <FIG> illustrates, as an example of a packet transmitted from the image sensor <NUM> to the application processor <NUM>, a packet structure of a CSI-<NUM> long packet for D-PHY when D-PHY is used for a physical layer. Furthermore, a lower side of <FIG> illustrates a state where the CSI-<NUM> long packet for D-PHY is subjected to distribution to N lanes (lane distribution) and transmitted.

Then, as illustrated in <FIG>, both a packet header (PH) and a packet footer (PF) are conventionally distributed to the plurality of lanes and transmitted in parallel. Therefore, in a case where an error occurs in any of these lanes, it may not be possible to identify the lane in which the error has occurred.

For example, in a case where a <NUM>-bit error occurs in packet data, a CRC error is detected in accordance with the existing CSI-<NUM> standard. However, since the packet structure has only one CRC for the entire packet, it has not been possible for a receiving side to identify in which lane the <NUM>-bit error has occurred. Therefore, it has not been possible for the application processor <NUM> to separate a specific lane in which an error occurs frequently and to perform continuous communication again only in normal lanes other than the lane.

Accordingly, in a case where the communication system <NUM> is adopted in on-board cameras, it is necessary to take measures which make it possible to identify a lane in which an error has occurred.

Furthermore, for example, in a case where a signal line D1P/N marked with a cross in <FIG> is disconnected, it has not been possible for the receiving side to immediately detect which lane has been disconnected. Similarly, in a case where SYNC is lost due to a bit error or the like occurring in the signal line D1P/N, it has not been possible for the receiving side to immediately detect in which lane the bit error has occurred to cause the SYNC to be lost.

For example, the serializer <NUM> is configured on the premise that in a case where a CRC error occurs in a payload, data can be output from the SerDes transmitter <NUM> even with the error.

On the other hand, in a case where any of the lanes is disconnected or a case where the SYNC is lost due to a bit error occurring in any of the lanes, the serializer <NUM> cannot receive the payload itself and cannot output data from the SerDes transmitter <NUM>. That is, while the <NUM>-bit error can be corrected by a Hamming code, in a case where a <NUM>-bit error is detected (SECDED), the serializer <NUM> cannot correctly receive word count (WC: data length of payload) in the packet header. Therefore, the serializer <NUM> does not know the length of the payload, and cannot output data from the SerDes transmitter <NUM> similarly to a case of a disconnection.

Then, in a case where Sync loss simply due to garbled bits occurs, or a case where an error of <NUM> bits or more occurs in the packet header, it is not possible to determine in which lane packet loss has occurred even if communication is restored in the next packet. Therefore, a separate inquiry is required using the I2C or the like, and a delay occurs before continuous communication is performed while avoiding a lane with a high error frequency.

Furthermore, in a case where a disconnection occurs, an application processor <NUM> side cannot receive any communication thereafter. Therefore, the application processor <NUM> in a subsequent stage considers the possibility of a disconnection after the elapse of a certain period of time, and reads a registry setting of the serializer <NUM> using the I2C or the like, and thereby a disconnection error can be detected, but a delay will occur by then. In on-board cameras, this delay is expected to be fatal.

Accordingly, in a case where the communication system <NUM> is adopted in on-board cameras, it is necessary to take measures which make it possible to immediately detect the occurrence of a defect and to perform continuous communication without delay.

For this reason, the communication system <NUM> is configured so that a lane in which an error has occurred can be identified, and the occurrence of a defect can be immediately detected to perform continuous communication without delay.

For example, in the communication system <NUM>, an extended packet (CSI-<NUM> long packet for D-PHY) having a packet structure such as that illustrated in an upper side of <FIG> is transmitted from the extension mode-supporting CSI-<NUM> transmitter <NUM> to the extension mode-supporting CSI-<NUM> receiver <NUM>. Then, in the communication system <NUM>, as illustrated in a lower side of <FIG>, a state where the extended packet is subjected to distribution to N lanes (lane distribution) and transmitted is illustrated.

For example, in the extended packet, the packet header and the packet footer have the same packet structure as that in the existing CSI-<NUM> standard. For example, the packet header stores virtual channel (VC) indicating the number of virtual channel lines, data type indicating a type of data, word count (WC) indicating a data length of a payload, and VCX/ECC. Furthermore, the packet footer stores cyclic redundancy check (CRC).

Here, regarding the data type transmitted in the packet header, the existing CSI-<NUM> standard defines 0x38 to 0x3F as reserved. For this reason, in the extended packet for D-PHY, setting information for identifying the extension mode on the receiving side is newly defined by using data type defined as reserved in the existing CSI-<NUM> standard.

That is, among data types 0x38 to 0x3F defined as reserved in the existing CSI-<NUM> standard, for example, DataType [<NUM>:<NUM>] is defined as extension mode setting information, and DataType [<NUM>:<NUM>] is defined as extension type setting information. The extension mode setting information indicates whether or not it is the extension mode, and indicates that it is the extension mode, for example, in a case where DataType [<NUM>:<NUM>] is <NUM>'b111. Furthermore, when four types of extension mode, i.e., extension mode <NUM>, extension mode <NUM>, extension mode <NUM>, and extension mode <NUM> are prepared as types of extension modes, the extension type setting information indicates which type of extension mode it is among the four types of extension mode. For example, in a case where DataType [<NUM>:<NUM>] is <NUM>'b00, it is indicated that the type of extension mode is extension mode <NUM>.

Then, in extension mode <NUM> (DataType [<NUM>:<NUM>] = <NUM>'b00), for example, a packet structure in which a payload is separated into four is defined. That is, as illustrated in <FIG>, the payload in extension mode <NUM> is separated into an extended payload header (ePH), an optional extended payload header (OePH), a legacy payload, and an optional extended payload footer (OePF).

The extended payload header is placed at the head corresponding to a payload of the existing CSI-<NUM> standard, and is required to be transmitted without fail in the extension mode. For example, the extended payload header includes setting information such as SROI identification flag, extended virtual channel (VC), extended data type, OePH selection flag, and OePF selection flag, as illustrated in the figure. Here, the extended VC extends the VC, which was <NUM> bits in the existing CSI-<NUM> standard, to <NUM> bits, and the extended data type extends the data type, which was <NUM> bits in the existing CSI-<NUM> standard, to <NUM> bits.

For example, in a packet for D-PHY, the VC of the existing packet header already exists in <NUM> bits, and by defining the extended VC of the extended payload header as <NUM> bits, the total thereof can be <NUM> bits. Specifically, definitions can be given as OePH [<NUM>:<NUM>] = {<NUM>'h00, RSID, XY_POS, MC} and OePF [<NUM>:<NUM>] = {<NUM>'h0, pCRC}, and it is possible to control ON/OFF of various packet transmissions required for respective applications.

The optional extended payload header and the optional extended payload footer are selectively transmitted depending on the applications.

The legacy payload corresponds to the same payload as that in the existing CSI-<NUM> standard.

As described above, by setting the extended payload header, the optional extended payload header, and the optional extended payload footer as needed, it becomes possible to transmit data corresponding to various applications. Furthermore, data transmitted with the extended payload header, the optional extended payload header, and the optional extended payload footer is <NUM> bits + <NUM>-bit error correction code (ECC). Therefore, it is possible to divert an existing payload header circuit to suppress an increase in a circuit scale and to improve error tolerance.

Then, in the communication system <NUM>, the packet header is copied depending on the number of multiple lanes without performing lane distribution of the packet header, and the packet header is transmitted independently in each lane. Then, in order to allow the payload to be reproduced in each lane on the receiving side, only the word count (WC) included in the packet header is changed from the existing CSI-<NUM> standard to be newly defined so as to indicate the number of bytes of the payload in each lane.

Furthermore, in the communication system <NUM>, CRC to be a packet footer is calculated independently for each payload in each lane without performing lane distribution of the packet footer, and the packet footer is transmitted independently in each lane.

Moreover, in the communication system <NUM>, lane distribution of message count for GLD (MC) included in the payload is not performed and the MC is transmitted independently in each lane, similarly to the packet header. At that time, in the communication system <NUM>, a lane number (LN) indicating via which lane the MC has been transmitted is also transmitted by incrementing (+<NUM>) each time the MC is transmitted. Note that as illustrated in <FIG>, the MC is placed in the optional extended payload header (OePH). Note that transmission of the LN is not essential, and the LN is only required to be transmitted as needed.

Furthermore, <FIG> illustrates an example in which the packet header (PH), the extended payload header (ePH), the OePH, the optional extended payload footer (OePF), and the packet footer (PF) are copied and transmitted in each lane. Then, in this example, lane distribution is performed for only the legacy payload on a transmitting side.

Note that as described with reference to <FIG>, the setting for independently transmitting the packet header, the WC, and the packet footer in each lane (hereinafter referred to as parallel transmission setting) is substantially an essential function in a case of adopting the communication system <NUM> in on-board cameras, and therefore, there is no need to perform dynamic on/off switching. Accordingly, in the communication system <NUM>, the application processor <NUM> is configured to turn on/off this parallel transmission setting with respect to a registry of the image sensor <NUM> at the start of communication.

Accordingly, the communication system <NUM> can have an error recognition function of recognizing in which lane an error has occurred and the type of error that has occurred. Furthermore, in a case where the CSI-<NUM> receiver <NUM> recognizes an error, the CSI-<NUM> receiver <NUM> notifies the application processor <NUM> of error information associated therewith.

<FIG> illustrates a packet structure of a CSI-<NUM> short packet used to notify the application processor <NUM> of the error information from the CSI-<NUM> receiver <NUM>.

For example, as illustrated in <FIG>, data type (= 0x00-0x0F) for notification of occurrence of an error is assigned to a data ID of the CSI-<NUM> short packet.

Furthermore, a lane in which the error has occurred and details of the error {Error_Lane [<NUM>:<NUM>], Error_Type [<NUM>:<NUM>]} are stored in a data field of the CSI-<NUM> short packet. For example, Error_Lane [<NUM>:<NUM>] indicates the lane in which the error has occurred. As an example, Error _Lane [<NUM>] indicates that an error has occurred in Lane <NUM>, and Error _Lane [<NUM>] indicates that an error has occurred in Lane <NUM>. Furthermore, Error _Type [<NUM>:<NUM>] indicates the type of error which has occurred in any lane. As an example, Error_Type [<NUM>] indicates Sync loss and Error_Type indicates PH error.

Specifically, in a case where normal reception can be performed in Lane <NUM>, Sync loss has occurred in Lane <NUM>, normal reception can be performed in Lane <NUM>, and PH error has occurred in Lane <NUM>, {<NUM>'b0000_1010, <NUM>'b0000_0011} is stored in the data field of the CSI-<NUM> short packet.

<FIG> is a block diagram illustrating an example configuration of the image sensor <NUM> including the extension mode-supporting CSI-<NUM> transmitter <NUM>.

As illustrated in <FIG>, in addition to the extension mode-supporting CSI-<NUM> transmitter <NUM>, the image sensor <NUM> includes a pixel <NUM>, an AD converter <NUM>, an image signal processor <NUM>, a pixel CRC calculation unit <NUM>, a physical layer processing unit <NUM>, an I2C/I3C slave <NUM>, and a register <NUM>. Furthermore, the extension mode-supporting CSI-<NUM> transmitter <NUM> includes a packing unit <NUM>, a packet header generation unit <NUM>, a payload header generation unit <NUM>, a payload footer generation unit <NUM>, selectors <NUM> and <NUM>, a CRC calculation unit <NUM>, a lane distribution unit <NUM>, a CCI slave <NUM>, and a controller <NUM>.

The pixel <NUM> outputs an analog pixel signal depending on the amount of received light, and the AD converter (analog-to-digital converter (ADC)) <NUM> digitally converts the pixel signal output from the pixel <NUM> and supplies the pixel signal to the image signal processor <NUM>. The image signal processor (ISP) <NUM> supplies, to the pixel CRC calculation unit <NUM> and the packing unit <NUM>, image data obtained by performing various image processes on an image based on the pixel signal. Furthermore, the image signal processor <NUM> supplies, to the packing unit <NUM> and the controller <NUM>, a data enable signal data_en indicating whether or not the image data is valid.

The pixel CRC calculation unit <NUM> calculates and obtains CRC for each pixel in the image data supplied from the image signal processor <NUM>, and supplies the CRC to the payload footer generation unit <NUM>.

The physical layer processing unit <NUM> can execute both C-PHY and D-PHY physical layer processes. For example, the physical layer processing unit <NUM> executes the C-PHY physical layer process in a case where a C-layer enable signal cphy_en supplied from the controller <NUM> is valid, and executes the D-PHY physical layer process in a case where the C-layer enable signal cphy_en is invalid. Then, the physical layer processing unit <NUM> transmits a packet divided into four lanes by the lane distribution unit <NUM> to the application processor <NUM>.

On the basis of the inter-integrated circuit (I2C) or improved inter-integrated circuit (I3C) standard, the I2C/I3C slave <NUM> performs communication following the lead of an I2C/I3C master <NUM> (<FIG>) of the application processor <NUM>.

Various settings transmitted from the application processor <NUM> are written to the register <NUM> via the I2C/I3C slave <NUM> and the CCI slave <NUM>. Here, as the settings to be written to the register <NUM>, for example, a communication setting in conformity with the CSI-<NUM> standard, an extension mode setting indicating whether or not the extension mode is used, and a fixed communication setting required for communication in the extension mode are exemplified.

The packing unit <NUM> performs a packing process of storing the image data supplied from the image signal processor <NUM> in a payload of the packet, and supplies the payload to the selector <NUM> and the lane distribution unit <NUM>.

When the packet header generation unit <NUM> is instructed to generate a packet header in accordance with a packet header generation instruction signal ph_go supplied from the controller <NUM>, the packet header generation unit <NUM> generates a packet header including WC indicating the number of bytes of the payload in each lane, and supplies the packet header to the selector <NUM> and the lane distribution unit <NUM>.

That is, in conformity with the existing CSI-<NUM> standard, the packet header generation unit <NUM> generates a packet header for storing setting information indicating conditions set for data to be transmitted in the packet, for example, data type indicating the type of data. Furthermore, the packet header generation unit <NUM> stores, in an unused region defined as unused in the existing CSI-<NUM> standard in the data type which is setting information indicating the type of data to be transmitted in the packet, extension mode setting information indicating whether or not the extension mode uses an extended header. Moreover, the packet header generation unit <NUM> stores, in the unused region, extension type setting information indicating which type of extension mode it is among the plurality of types of extension modes prepared as the extension mode.

The payload header generation unit <NUM> generates each of an extended payload header and an optional extended payload header in accordance with an extended payload header generation instruction signal eph_go and an extended payload header enable signal ePH_en supplied from the controller <NUM>, and supplies the extended payload header and the optional extended payload header to the selector <NUM> and the lane distribution unit <NUM>. Furthermore, the payload header generation unit <NUM> is supplied with an on-board line number, a source identification (ID), and the like depending on the application of the image sensor <NUM>, and if necessary, stores the on-board line number, the source identification (ID), and the like in the extended payload header or the optional extended payload header.

That is, the payload header generation unit <NUM> generates, for example, an extended payload header such as that illustrated in <FIG>, separately from the packet header generated by the packet header generation unit <NUM>. Moreover, in a case of transmitting the optional extended header, the payload header generation unit <NUM> stores, in the extended header, as optional extended header setting information (OePH [<NUM>:<NUM>]) indicating whether or not to transmit the optional extended header, optional extended header setting information indicating that the optional extended header is transmitted, and generates the extended header followed by the optional extended header.

The payload footer generation unit <NUM> generates an optional extended payload footer in accordance with an extended payload footer generation instruction signal epf_go and an extended payload header enable signal ePF_en supplied from the controller <NUM>, and supplies the optional extended payload footer to the selector <NUM> and the lane distribution unit <NUM>.

That is, in a case where the packet transmitted in the extension mode is an extended long packet which stores data transmitted as a payload in the existing CSI-<NUM> standard, the payload footer generation unit <NUM> generates an optional extended payload footer placed subsequent to the legacy payload in which the data is stored.

Furthermore, the C-layer enable signal cphy_en is supplied from the controller <NUM> to the packet header generation unit <NUM>, the payload header generation unit <NUM>, and the payload footer generation unit <NUM>. Then, in a case where the C-layer enable signal cphy_en is valid, the packet header generation unit <NUM> generates a packet header for C-PHY, the payload header generation unit <NUM> generates an extended payload header and an optional extended payload header for C-PHY, and the payload footer generation unit <NUM> generates an optional extended payload footer for C-PHY. On the other hand, in a case where the C-layer enable signal cphy_en is invalid, the packet header generation unit <NUM> generates a packet header for D-PHY, the payload header generation unit <NUM> generates an extended payload header and an optional extended payload header for D-PHY, and the payload footer generation unit <NUM> generates an optional extended payload footer for D-PHY.

In accordance with the C-layer enable signal cphy_en supplied from the controller <NUM>, in a case where the C-layer enable signal cphy_en is valid, the selector <NUM> selects a packet header supplied from the packet header generation unit <NUM> and supplies the packet header to the selector <NUM>. On the other hand, in a case where the C-layer enable signal cphy_en is invalid, the selector <NUM> selects a payload supplied from the packing unit <NUM> and supplies the payload to the selector <NUM>.

In accordance with the data selection signal data_sel supplied from the controller <NUM>, the selector <NUM> selects any of the packet header or the payload selectively supplied via the selector <NUM>, the extended payload header and the optional extended payload header supplied from the payload header generation unit <NUM>, and the optional extended payload footer supplied from the payload footer generation unit <NUM>, and supplies the selected one to the CRC calculation unit <NUM>.

The CRC calculation unit <NUM> calculates and obtains CRC of the packet header, the payload, the extended payload header, the optional extended payload header, or the optional extended payload footer selectively supplied via the selector <NUM>, and supplies the CRC to the lane distribution unit <NUM>.

In accordance with the control of the controller <NUM>, the payload supplied from the packing unit <NUM>, the packet header supplied from the packet header generation unit <NUM>, the extended payload header and the optional extended payload header supplied from the payload header generation unit <NUM>, the optional extended payload footer supplied from the payload footer generation unit <NUM>, and the CRC supplied from the CRC calculation unit <NUM> are distributed by the lane distribution unit <NUM> to four lanes in conformity with the CSI-<NUM> standard and supplied to the physical layer processing unit <NUM>.

On the basis of the CSI-<NUM> standard, the camera control interface (CCI) slave <NUM> performs communication following the lead of a CCI master <NUM> (<FIG>) of the application processor <NUM>.

The controller <NUM> reads various settings stored in the register <NUM> and controls each block constituting the extension mode-supporting CSI-<NUM> transmitter <NUM> in accordance with the settings. For example, the controller <NUM> controls switching between transmission of a packet having a packet structure in conformity with the existing CSI-<NUM> standard and transmission of a packet having a packet structure in the extension mode depending on the content of the data to be transmitted.

Then, in the image sensor <NUM>, the parallel transmission setting, which is a communication setting for setting the parallel transmission such as that described with reference to <FIG> described above, is written from the application processor <NUM> to the register <NUM>. Therefore, the controller <NUM> refers to the parallel transmission setting of the register <NUM>, and supplies a parallel transmission enable signal para_en indicating whether or not the parallel transmission is valid to the packet header generation unit <NUM>, the payload header generation unit <NUM>, the CRC calculation unit <NUM>, the lane distribution unit <NUM>, and the physical layer processing unit <NUM>.

For example, in a case where the parallel transmission enable signal para_en is valid, the packet header generation unit <NUM> generates (copies) a packet header for each lane, and the payload header generation unit <NUM> generates an MC and an LN for each lane. Furthermore, in this case, the CRC calculation unit <NUM> calculates CRC from a payload in each lane to generate a packet footer, and the lane distribution unit <NUM> appropriately distributes objects to be transmitted to each of these lanes.

The image sensor <NUM> is configured as described above, and can transmit packets to the application processor <NUM> by parallel transmission such as that described with reference to <FIG>.

Furthermore, the image sensor <NUM> can generate an extended packet having a packet structure such as that illustrated in <FIG> and transmit the extended packet to the application processor <NUM>.

<FIG> is a block diagram illustrating an example configuration of the application processor <NUM> including the extension mode-supporting CSI-<NUM> receiver <NUM>.

As illustrated in <FIG>, in addition to the extension mode-supporting CSI-<NUM> receiver <NUM>, the application processor <NUM> includes a physical layer processing unit <NUM>, the I2C/I3C master <NUM>, a register <NUM>, and an error processing unit <NUM>. Furthermore, the extension mode-supporting CSI-<NUM> receiver <NUM> includes a packet header detection unit <NUM>, a lane merging unit <NUM>, an interpretation unit <NUM>, selectors <NUM> and <NUM>, a CRC calculation unit <NUM>, an unpacking unit <NUM>, and the CCI master <NUM>.

The physical layer processing unit <NUM> can execute both C-PHY and D-PHY physical layer processes. As described above, the physical layer processing unit <NUM> of the image sensor <NUM> performs a physical layer process of either C-PHY or D-PHY, and the physical layer processing unit <NUM> executes the same physical layer process as that executed in the physical layer processing unit <NUM>.

The I2C/I3C master <NUM> takes the lead in communicating with the I2C/I3C slave <NUM> (<FIG>) of the image sensor <NUM> on the basis of the I2C or I3C standard.

In the register <NUM>, various settings to be written to the register <NUM> of the image sensor <NUM> are recorded.

The error processing unit <NUM> is supplied with an error notification short packet reception flag output from the packet header detection unit <NUM>, a crc error detection signal output from the CRC calculation unit <NUM>, and message count for GLD (MC) and an LN output from the interpretation unit <NUM>. Then, the error processing unit <NUM> can perform an error process as described later with reference to the flowchart of <FIG> to identify a target lane in which a disconnection occurs or an error occurs frequently, and can write a setting which stops use of the target lane to the register <NUM> Furthermore, the error processing unit <NUM> sets, for example, a decrease in communication speed as a result of stopping the use of the target lane with respect to the CCI master <NUM>, or causes the CCI master <NUM> to perform communication for causing a restoration process of the target lane to be performed.

The packet header detection unit <NUM> detects a packet header from a packet supplied from the physical layer processing unit <NUM>, and checks a data type stored in the packet header. Then, in a case where the extension mode setting information indicates that it is the extension mode in the data type of the packet header (DataType [<NUM>:<NUM>] = <NUM>'b111), the packet header detection unit <NUM> supplies an extension mode detection flag indicating the extension mode to the interpretation unit <NUM>, the selector <NUM>, and the selector <NUM>. Furthermore, the packet header detection unit <NUM> supplies, to the lane merging unit <NUM>, a merging enable signal mrg_en indicating whether or not merging of divided four lanes is valid on the basis of the packet header.

That is, in conformity with the existing CSI-<NUM> standard, the packet header detection unit <NUM> detects a packet header which stores the setting information (data type and the like) indicating conditions set for data to be transmitted in the packet. At that time, the packet header detection unit <NUM> outputs the extension mode detection flag in accordance with extension mode setting information indicating whether or not the extension mode uses an extended header, the extension mode setting information being stored in the unused region defined as unused in the existing CSI-<NUM> standard in the data type which is the setting information indicating the type of data to be transmitted in the packet, thereby causing switching to be performed between reception of a packet having a packet structure in conformity with the existing CSI-<NUM> standard and reception of a packet having a packet structure in the extension mode. Furthermore, in accordance with extension mode type information stored in the unused region of the data type defined as unused in the existing CSI-<NUM> standard, the packet header detection unit <NUM> recognizes which type of extension mode it is among the plurality of types of extension modes prepared as the extension mode.

In a case where the merging enable signal mrg_en supplied from the packet header detection unit <NUM> is valid, the lane merging unit <NUM> receives and merges the packet divided into four lanes supplied from the physical layer processing unit <NUM>. Then, the lane merging unit <NUM> supplies the packet of one lane to the interpretation unit <NUM>, the selector <NUM>, and the selector <NUM>.

In a case where the extension mode detection flag supplied from the packet header detection unit <NUM> indicates that it is the extension mode, the interpretation unit <NUM> reads the extended payload header, the optional extended payload header, and the optional extended payload footer from the packet supplied from the lane merging unit <NUM> on the basis of a packet structure in the extension mode. Then, the interpretation unit <NUM> interprets the setting information stored in the extended payload header, the optional extended payload header, and the optional extended payload footer.

That is, the interpretation unit <NUM> receives, as an extended header, an extended payload header placed at the head of a payload in conformity with the existing CSI-<NUM> standard, and interprets setting information stored in the extended payload header. Furthermore, in a case where the optional extended header setting information stored in the extended header indicates that the optional extended header which is selectively transmitted depending on the application is transmitted, the interpretation unit <NUM> receives the extended header followed by the optional extended header and interprets setting information stored in the optional extended header. Moreover, in a case where the packet transmitted in the extension mode is an extended long packet which stores data transmitted as a payload in the existing CSI-<NUM> standard, the interpretation unit <NUM> receives an optional extended payload footer placed subsequent to a legacy payload in which the data is stored, and interprets the optional extended payload footer.

Then, the interpretation unit <NUM> reads and outputs, for example, an on-board line number and a source ID stored in the optional extended payload header to an LSI (not illustrated) in a subsequent stage.

Note that in a case where the extension mode detection flag supplied from the packet header detection unit <NUM> does not indicate that it is the extension mode, that is, in a case where a packet having an existing packet structure is supplied, the interpretation unit <NUM> stops and does not perform the processes such as those described above.

In accordance with the extension mode detection flag supplied from the packet header detection unit <NUM>, the selector <NUM> selectively supplies data to the unpacking unit <NUM> on the basis of the packet structure of the existing packet or the packet structure of the extended packet.

In accordance with the extension mode detection flag supplied from the packet header detection unit <NUM>, the selector <NUM> selectively supplies data to the CRC calculation unit <NUM> on the basis of the packet structure of the existing packet or the packet structure of the extended packet.

The CRC calculation unit <NUM> calculates CRC of the packet header, the payload, the extended payload header, the optional extended payload header, or the optional extended payload footer selectively supplied via the selector <NUM>. Then, in a case where a CRC error is detected, the CRC calculation unit <NUM> outputs a crc error detection signal indicating to that effect to the LSI (not illustrated) in the subsequent stage.

The unpacking unit <NUM> performs an unpacking process of extracting image data stored in a payload selectively supplied via the selector <NUM>, and outputs the acquired image data to the LSI (not illustrated) in the subsequent stage.

The CCI master <NUM> takes the lead in communicating with the CCI slave <NUM> (<FIG>) of the image sensor <NUM> on the basis of the CSI-<NUM> standard.

Then, in the application processor <NUM>, the parallel transmission setting is written to the register <NUM> similarly to the register <NUM> of the image sensor <NUM>. Then, the packet header detection unit <NUM>, the lane merging unit <NUM>, the interpretation unit <NUM>, and the CRC calculation unit <NUM> refer to the parallel transmission setting of the register <NUM> and receive a packet transmitted in parallel from the image sensor <NUM>.

For example, the packet header detection unit <NUM> detects a packet header transmitted in each lane, and the interpretation unit <NUM> interprets an MC and an LN transmitted in each lane. Furthermore, the CRC calculation unit <NUM> calculates CRC on the basis of a payload in each lane and detects a CRC error.

Moreover, in the application processor <NUM>, the packet header detection unit <NUM> can detect an error notification packet (CSI-<NUM> short packet illustrated in <FIG>) transmitted from an error notification packet generation unit <NUM> of <FIG>, which will be described later. Then, in a case where the packet header detection unit <NUM> detects the error notification packet, the packet header detection unit <NUM> outputs a flag indicating to that effect (error notification short packet reception flag) to the LSI (not illustrated) in the subsequent stage.

The application processor <NUM> is configured as described above, and can receive packets transmitted from the image sensor <NUM> by parallel transmission such as that described with reference to <FIG>.

Furthermore, the application processor <NUM> can receive an extended packet transmitted from the image sensor <NUM>, and can interpret setting information stored in the extended payload header, the optional extended payload header, and the optional extended payload footer to obtain image data. At that time, even if any of the lanes is disconnected, the CSI-<NUM> receiver <NUM> can receive a payload of a packet transmitted by a normal lane among the plurality of lanes on the basis of the WC indicating the number of bytes of the payload in each lane.

<FIG> is a block diagram illustrating an example configuration of the serializer <NUM> including the CSI-<NUM> receiver <NUM>.

Note that in the serializer <NUM> illustrated in <FIG>, regarding configurations common to those in the application processor <NUM> of <FIG>, the same reference numerals are given thereto, and detailed descriptions thereof will be omitted. That is, the serializer <NUM> has a configuration common to that of the application processor <NUM> of <FIG> in the following point: the serializer <NUM> includes the physical layer processing unit <NUM>, the I2C/I3C master <NUM>, and the register <NUM>, and the CSI-<NUM> receiver <NUM> includes the packet header detection unit <NUM>, the lane merging unit <NUM>, the CRC calculation unit <NUM>, and the CCI master <NUM>.

On the other hand, the serializer <NUM> has a configuration different from that of the application processor <NUM> of <FIG> in the following point: the CSI-<NUM> receiver <NUM> includes the error notification packet generation unit <NUM>.

The error notification packet generation unit <NUM> generates and outputs the error notification packet illustrated in <FIG> in accordance with the flag (error notification short packet reception flag) supplied from the packet header detection unit <NUM>.

As described above, in the communication system <NUM>, the packet footer (CRC) is independently transmitted in each lane. This makes it possible to identify in which lane an error has occurred. Then, in a case where a lane is detected in which a certain number or larger number of occurrences of CRC error is identified, it is determined that the lane has high error frequency, and the application processor <NUM> sets the registry of the image sensor <NUM> so as not to use the lane, changes the mode, and then resumes communication.

Furthermore, in the communication system <NUM>, the packet header is transmitted independently in each lane, and the definition of the WC is changed to the number of bytes to be sent in each lane, instead of the total length of the payload. Therefore, even if there is a lane in which there occurs a defect such as a disconnection, Sync loss, or a <NUM>-bit error occurring in the packet header, reception can be continued in normal lanes other than the lane.

Furthermore, in the communication system <NUM>, the MC is transmitted independently in each lane and incremented in each lane. Moreover, the lane number may be transmitted at the same time to clearly indicate via which lane the transmission has been performed. Note that the MC is cleared to zero at the head of a frame. Therefore, the CSI-<NUM> receiver <NUM> can easily detect a disconnection and Sync loss only by a link layer (CSI-<NUM>) without changing a PHY layer (D-PHY or C-PHY). Furthermore, the CSI-<NUM> receiver <NUM> can determine a disconnection or Sync loss only by the lack of the MC in the link layer.

Moreover, in the communication system <NUM>, in a case where there occurs a serious error (Sync loss, disconnection, error of <NUM> bits or more in PH) which prevents data from being output from the SerDes transmitter <NUM>, error information can be transmitted to the application processor <NUM> in a subsequent stage. At that time, the error information can be transmitted at high speed by transmitting the error notification packet (CSI-<NUM> short packet illustrated in <FIG>) in accordance with a CSI-<NUM> format. Therefore, the application processor <NUM> in the subsequent stage can immediately know the error in the SerDes transmitter <NUM>. Then, in a case where the application processor <NUM> receives the error information in a specific lane a predetermined number of times or more, the application processor <NUM> can instruct the image sensor <NUM> and the SerDes transmitter <NUM> to stop using the lane.

As described above, in the communication system <NUM>, the application processor <NUM> side can quickly identify in which lane the disconnection, the SYNC loss, or the frequent occurrence of the error occurs. This makes it possible for the application processor <NUM> to detour around a specific lane and to perform continuous communication using normal lanes only, and therefore reliability can be improved and securer communication such as that for on-board items can be supported.

A communication process performed by the image sensor <NUM> and the application processor <NUM> will be described with reference to <FIG>.

<FIG> is a flowchart explaining a process in which the image sensor <NUM> transmits a packet.

For example, when the image sensor <NUM> is connected to the application processor <NUM> via the bus <NUM>, the process is started. In step S11, the controller <NUM> determines whether or not to use the extension mode when starting communication with the application processor <NUM>. For example, the controller <NUM> checks the extension mode setting stored in the register <NUM>, and determines to use the extension mode in a case where an extension mode setting indicating the use of the extension mode is written by the application processor <NUM>.

If the controller <NUM> determines in step S11 not to use the extension mode, the process proceeds to step S12.

In step S12, the I2C/I3C slave <NUM> receives a transmission start command of image data transmitted from the application processor <NUM> (in step S54 of <FIG> as described later). Moreover, the I2C/I3C slave <NUM> receives a communication setting in conformity with the CSI-<NUM> standard transmitted together with the transmission start command, and writes the communication setting to the register <NUM> via the CCI slave <NUM>.

In step S13, the image sensor <NUM> executes a conventional packet transmission process of transmitting a packet having a packet structure in conformity with the existing CSI-<NUM> standard to the application processor <NUM> on the basis of the communication setting stored in the register <NUM>.

On the other hand, if the controller <NUM> determines in step S11 to use the extension mode, the process proceeds to step S14.

In step S14, the I2C/I3C slave <NUM> receives a fixed communication setting (for example, a copy of PH/PF for each lane during GLD) required for communication in the extension mode and writes the fixed communication setting to the register <NUM> via the CCI slave <NUM>.

In step S15, the I2C/I3C slave <NUM> receives a transmission start command of image data transmitted from the application processor <NUM> (in step S57 of <FIG> as described later). Moreover, the I2C/I3C slave <NUM> receives a communication setting in conformity with the CSI-<NUM> standard transmitted together with the transmission start command, and writes the communication setting to the register <NUM> via the CCI slave <NUM>.

In step S16, the controller <NUM> determines whether or not to start transmitting a packet, and stands by without starting a process until it is determined to start transmitting the packet.

Then, if it is determined in step S16 to start transmitting the packet, the process proceeds to step S17, and the controller <NUM> determines whether or not it is data to be transmitted in the extension mode. Here, depending on the content of the data to be transmitted, the controller <NUM> determines that it is data to be transmitted in the extension mode, for example, in a case where it is data to be transmitted in a predetermined use case.

If the controller <NUM> determines in step S17 that it is data to be transmitted in the extension mode, the process proceeds to step S18, and the extension mode transmission process (see <FIG>) of transmitting an extended packet supporting the extension mode is performed.

On the other hand, if the controller <NUM> determines in step S17 that it is data not to be transmitted in the extension mode, the process proceeds to step S19.

In step S19, the controller <NUM> determines whether or not to transmit a short packet. For example, the controller <NUM> determines to transmit the short packet at start of frame and end of frame.

If the controller <NUM> determines in step S19 to transmit the short packet, the process proceeds to step S20. In step S20, the packet header generation unit <NUM> generates a packet header and transmits a short packet having a conventional packet structure to the application processor <NUM>.

On the other hand, if the controller <NUM> determines in step S19 not to transmit the short packet (that is, to transmit a long packet), the process proceeds to step S21. In step S21, the packing unit <NUM> stores image data in a payload, and the CRC calculation unit <NUM> obtains the CRC, and thereby a long packet having a conventional packet structure is generated and transmitted to the application processor <NUM>.

After the process of step S18, step S20, or step S21, the process proceeds to step S22, and the controller <NUM> ends the packet transmission process. Thereafter, the process returns to step S16, and afterward the process of similarly transmitting a packet is repeatedly performed for the next packet.

<FIG> is a flowchart explaining the extension mode transmission process performed in the process of step S18 of <FIG>.

In step S31, the packet header generation unit <NUM> generates a packet header storing VC, data type, WC, and the like, and transmits the packet header to the application processor <NUM>. At that time, the packet header generation unit <NUM> writes the extension mode setting information (DataType [<NUM>:<NUM>] = <NUM>'b111) indicating that it is the extension mode and the extension type setting information (DataType [<NUM>:<NUM>] = <NUM>'b00) identifying that the mode setting of the extension mode is extension mode <NUM>, to the data type of the packet header. Furthermore, the packet header is transmitted independently in each lane.

In step S32, the application processor <NUM> determines whether or not to transmit an extended short packet. For example, the controller <NUM> determines to transmit the extended short packet at start of frame and end of frame.

If the application processor <NUM> determines in step S32 to transmit the extended short packet, the process proceeds to step S33.

In step S33, the payload header generation unit <NUM> transmits an extended payload header in which data type (DataType [<NUM>:<NUM>]) is set to a short packet in a first byte of the payload. At that time, the payload header generation unit <NUM> makes various settings (for example, OePH [<NUM>:<NUM>], OePF [<NUM>:<NUM>], and the like) stored in the extended payload header.

In step S34, the payload header generation unit <NUM> stores and transmits frame number (FN) in a second byte of the payload.

In step S35, the payload header generation unit <NUM> generates and transmits an optional extended payload header (OePH) in accordance with the setting (OePH [<NUM>:<NUM>]) made in step S33.

In step S36, the CRC calculation unit <NUM> obtains CRC and transmits the CRC as a packet footer. At that time, the packet footer is transmitted independently in each lane.

On the other hand, if the application processor <NUM> determines in step S32 not to transmit the extended short packet (that is, to transmit a long packet), the process proceeds to step S37.

In step S37, the payload header generation unit <NUM> transmits an extended payload header in which the data type (DataType [<NUM>:<NUM>]) is set to other type than a short packet in the first byte of the payload. At that time, the payload header generation unit <NUM> makes various settings (for example, OePH [<NUM>:<NUM>], OePF [<NUM>:<NUM>], and the like) stored in the extended payload header.

In step S38, the payload header generation unit <NUM> generates and transmits an optional extended payload header (OePH) in accordance with the setting (OePH [<NUM>:<NUM>]) made in step S37.

In step S39, the packing unit <NUM> packs image data supplied from the image signal processor <NUM>, generates and transmits a legacy payload.

In step S40, the payload footer generation unit <NUM> generates and transmits an optional extended payload header (OePH) in accordance with the setting (OePF [<NUM>:<NUM>]) made in step S37.

In step S41, the CRC calculation unit <NUM> obtains CRC and transmits the CRC as a packet footer. At that time, the packet footer is transmitted independently in each lane.

Then, after the process of step S36 or S41, the extension mode transmission process is ended.

As described above, the image sensor <NUM> can generate and transmit an extended short packet or an extended long packet.

<FIG> is a flowchart explaining a process in which the application processor <NUM> receives a packet.

For example, when the image sensor <NUM> is connected to the application processor <NUM> via the bus <NUM>, the process is started. In step S51, an initial setting of the image sensor <NUM> (for example, which of C-PHY or D-PHY is used as a physical layer) is written to register <NUM> and transmitted by the I2C/I3C master <NUM> to the image sensor <NUM> via the CCI master <NUM>. Therefore, the initial setting is written to the register <NUM> of the image sensor <NUM>.

In step S52, it is recognized whether or not the image sensor <NUM> supports the extension mode. For example, whether or not the image sensor <NUM> supports the extension mode can be recognized by acquiring a set value (for example, extended PH/PF-supporting capability) stored in the register <NUM> of the image sensor <NUM> by the I2C/I3C master <NUM>. Alternatively, it is possible to recognize in advance whether or not the image sensor <NUM> supports the extension mode on the basis of manual input or the like, for example.

In step S53, it is determined whether or not the image sensor <NUM> supports the extension mode and whether or not the application executed by the application processor <NUM> requires the use of the extension mode.

If it is determined in step S53 that the image sensor <NUM> does not support the extension mode or that the use of the extension mode is not required, the process proceeds to step S54.

In step S54, the I2C/I3C master <NUM> transmits a transmission start command of image data to the image sensor <NUM>. At that time, a communication setting in conformity with the CSI-<NUM> standard is also transmitted.

In step S55, the application processor <NUM> performs a conventional packet reception process of receiving a packet having a packet structure in conformity with the existing CSI-<NUM> standard on the basis of the communication setting transmitted in step S54.

On the other hand, if it is determined in step S53 that the image sensor <NUM> supports the extension mode and that the application executed by the application processor <NUM> requires the use of the extension mode, the process proceeds to step S56.

In step S56, the I2C/I3C master <NUM> transmits a fixed communication setting required for communication in the extension mode before the communication in the extension mode is started. Therefore, the fixed communication setting is written to the register <NUM> of the image sensor <NUM> (step S14 of <FIG>).

In step S57, the I2C/I3C master <NUM> transmits a transmission start command of image data to the image sensor <NUM>. At that time, a communication setting in conformity with the CSI-<NUM> standard is also transmitted.

In step S58, the packet header detection unit <NUM> determines whether or not reception of the packet has started by checking data supplied from the physical layer processing unit <NUM>, and stands by without starting a process until it is determined that the reception of the packet has started. For example, in a case where a packet header is detected from the data supplied from the physical layer processing unit <NUM>, the packet header detection unit <NUM> determines that the reception of the packet has started. At that time, the packet header is transmitted independently in each lane.

If the packet header detection unit <NUM> determines in step S58 that the reception of the packet has started, the process proceeds to step S59.

In step S59, the packet header detection unit <NUM> checks the data type of the packet header detected in step S58, and determines whether or not the packet of which reception has started is an extended packet supporting the extension mode. For example, in a case where the extension mode setting information indicates that it is the extension mode in the data type of the packet header (DataType [<NUM>:<NUM>] = <NUM>'b111), the packet header detection unit <NUM> determines that the packet of which reception has started is an extended packet.

If the packet header detection unit <NUM> determines in step S59 that the packet of which reception has started is an extended packet, the process proceeds to step S60, and an extension mode reception process (see <FIG>) of receiving an extended packet is performed.

On the other hand, if the packet header detection unit <NUM> determines in step S59 that the packet of which reception has started is not an extended packet, the process proceeds to step S61.

In step S61, the packet header detection unit <NUM> checks the data type (DataType [<NUM>:<NUM>]) of the packet header detected in step S58, and determines whether or not the packet of which reception has started is a short packet.

If the packet header detection unit <NUM> determines in step S61 that the packet of which reception has started is a short packet, the process proceeds to step S62. In step S62, the packet header detection unit <NUM> receives a short packet having a conventional packet structure transmitted from the image sensor <NUM>.

On the other hand, if the packet header detection unit <NUM> determines in step S61 that the packet of which reception has started is not a short packet (that is, reception of a long packet has started), the process proceeds to step S63. In step S63, the unpacking unit <NUM> receives a payload of the long packet having a conventional packet structure transmitted from the image sensor <NUM> and extracts image data, and the CRC calculation unit <NUM> receives WC + first byte transmitted following the packet header as CRC.

After the process of step S60, step S62, or step S63, the process proceeds to step S64, and the packet reception process is ended. Thereafter, the process returns to step S58, and afterward the process of similarly receiving a packet is repeatedly performed for the next packet.

<FIG> is a flowchart explaining the extension mode reception process performed in the process of step S60 of <FIG>.

In step S71, the packet header detection unit <NUM> determines whether or not the mode setting of the extension mode is extension mode <NUM>. For example, in a case where the extension type setting information indicates that it is extension mode <NUM> in the data type of the packet header (DataType [<NUM>:<NUM>] = <NUM>'b00), the packet header detection unit <NUM> determines that the mode setting of the extension mode is extension mode <NUM>.

If the packet header detection unit <NUM> determines in step S71 that the mode setting of the extension mode is extension mode <NUM>, the process proceeds to step S72. In step S72, the interpretation unit <NUM> receives a first byte of the payload as the extended payload header.

In step S73, the interpretation unit <NUM> checks the data type (DataType [<NUM>:<NUM>]) of the extended payload header received in step S72, and determines whether or not the packet of which reception has started is an extended short packet.

If the interpretation unit <NUM> determines in step S73 that the packet is the extended short packet, the process proceeds to step S74. In step S74, the interpretation unit <NUM> receives an optional extended payload header in accordance with the setting (OePH [<NUM>:<NUM>]) stored in the extended payload header received in step S72.

In step S75, the CRC calculation unit <NUM> receives the WC + first byte transmitted following the optional extended payload header as CRC. At that time, the CRC, which is a packet footer, is transmitted independently in each lane.

On the other hand, if the interpretation unit <NUM> determines in step S73 that the packet is not an extended short packet (that is, reception of an extended long packet has started), the process proceeds to step S76. In step S76, the interpretation unit <NUM> receives an optional extended payload header in accordance with the setting (OePH [<NUM>:<NUM>]) stored in the extended payload header received in step S72.

In step S77, the unpacking unit <NUM> receives the legacy payload of the extended long packet transmitted from the image sensor <NUM> and extracts image data.

In step S78, the interpretation unit <NUM> receives an optional extended payload footer in accordance with the setting (OePF [<NUM>:<NUM>]) stored in the extended payload header received in step S72.

In step S79, the CRC calculation unit <NUM> receives the WC + first byte transmitted following the optional extended payload footer as CRC. At that time, the CRC, which is a packet footer, is transmitted independently in each lane.

Then, if it is determined in step S71 that the mode setting of the extension mode is not extension mode <NUM>, the extension mode reception process is ended after the process of step S75 or the process of step S79.

As described above, the application processor <NUM> can receive the extended short packet or the extended long packet and acquire the data.

<FIG> is a flowchart explaining an error process performed by the error processing unit <NUM> of <FIG>.

In step S81, the error processing unit <NUM> determines whether or not a CRC error has been detected, that is, whether or not a crc error detection signal output from the CRC calculation unit <NUM> indicates that a CRC error has been detected.

If the error processing unit <NUM> determines in step S81 that the CRC error has been detected, the process proceeds to step S82. In step S82, the error processing unit <NUM> increments (+<NUM>) with respect to an error count register for a lane in which the CRC error has been detected.

In step S83, the error processing unit <NUM> determines whether or not a value of the error count register incremented in step S82 has exceeded a predetermined number of times.

If the error processing unit <NUM> determines in step S83 that the value of the error count register has not exceeded the predetermined number of times, the process proceeds to step S84. In step S84, the application processor <NUM> discards reception data in which the CRC error has been detected, then the process returns to step S81, and afterward a similar process is repeatedly performed.

On the other hand, if the error processing unit <NUM> determines in step S81 that the CRC error has not been detected, the process proceeds to step S85. In step S85, the error processing unit <NUM> determines whether or not an error notification short packet has been received. For example, the error processing unit <NUM> can perform the determination in accordance with the error notification short packet reception flag output from the packet header detection unit <NUM>.

If the error processing unit <NUM> determines in step S85 that the error notification short packet has been received, the process proceeds to step S86.

In step S86, the error processing unit <NUM> stops using a target lane for which it is indicated in the error notification short packet that an error has occurred. Then, the error processing unit <NUM> transmits a notification to the effect that the use of the target lane has been stopped from the CCI master <NUM> to the image sensor <NUM>, the serializer <NUM>, and the deserializer <NUM> via the I2C/I3C master <NUM>, to cause a restoration process of the target lane to be performed.

That is, in this case, since it has been possible to receive the error notification short packet, it can be recognized that the bus <NUM> and the bus <NUM>-<NUM> are normal and the bus <NUM>-<NUM> is disconnected. However, in a case where both the bus <NUM>-<NUM> and the bus <NUM>-<NUM> do not have the same number of lanes, the communication speeds cannot be balanced. Therefore, when a disconnection is detected in the bus <NUM>-<NUM>, even if it is a lane usable in the bus <NUM>-<NUM>, a process is performed which makes the numbers of lanes in the bus <NUM>-<NUM> and the bus <NUM>-<NUM> the same.

Thereafter, the process returns to step S81, and afterward a similar process is repeatedly performed.

On the other hand, if the error processing unit <NUM> determines in step S85 that the error notification short packet has not been received, the process proceeds to step S87. In step S87, the error processing unit <NUM> determines whether or not a state where some lane cannot be received has occurred a predetermined number of times or more.

If the error processing unit <NUM> determines in step S87 that the state where some lane cannot be received has occurred the predetermined number of times or more, the process proceeds to step S88.

In step S88, the error processing unit <NUM> identifies the target lane in which the error has occurred on the basis of the message count for GLD (MC) and the lane number (LN) transmitted independently in each lane by the interpretation unit <NUM> interpreting and outputting the setting information stored in the optional extended payload header. After the process of step S88, the process proceeds to step S86, and afterward a process similar to that described above is performed.

That is, in this case, in view of the fact that the MC is successively missing, it can be recognized that the bus <NUM>-<NUM> and the bus <NUM> are normal, and the bus <NUM>-<NUM> is disconnected. However, in a case where both the bus <NUM>-<NUM> and the bus <NUM>-<NUM> do not have the same number of lanes, the communication speeds cannot be balanced. Therefore, the disconnected lane of the bus <NUM>-<NUM> is identified, and the use of a corresponding lane of the bus <NUM>-<NUM> is prohibited.

On the other hand, if the error processing unit <NUM> determines in step S87 that the state where some lane cannot be received has not occurred the predetermined number of times or more, the process proceeds to step S89.

In step S89, the error processing unit <NUM> determines whether or not an interrupt notifying the disconnection has been received.

Here, for example, in a case where the deserializer <NUM> detects the disconnection of the bus <NUM>, the deserializer <NUM> can notify the error processing unit <NUM> that the disconnection of the bus <NUM> has been detected by using one of the HS-GPIO signal lines of the bus <NUM>-<NUM> as a disconnection error detection interrupt line. Accordingly, the error processing unit <NUM> can determine in step S87 whether or not the interrupt notifying the disconnection has been received by using the HS-GPIO signal line as the disconnection error detection interrupt line. Furthermore, in addition thereto, the disconnection of the bus <NUM> among the detected errors may be detected by using one of the HS-GPIO signal lines of the bus <NUM>-<NUM> as an error detection interrupt line, and reading an internal register of the deserializer <NUM> via the I2C/I3C master <NUM> regarding specific details of the errors.

If the error processing unit <NUM> determines in step S89 that the interrupt notifying the disconnection has been received, the process proceeds to step S90. In step S90, the error processing unit <NUM> sets the CCI master <NUM> so as to stop using the target lane in which the disconnection has occurred and reduce the communication speed of the communication by the CSI-<NUM>.

That is, in this case, the disconnection has occurred in the bus <NUM>, and in a case where the bus <NUM> includes a plurality of lanes, the use of the target lane in which the disconnection has occurred is prohibited and continuous communication is performed. Then, in line with the decrease in the bandwidth of the bus <NUM>, the communication speeds of the buses <NUM>-<NUM> and <NUM>-<NUM> are reduced in order to maintain the balance of the communication speeds.

After the process of step S90, or if it is determined in step S89 that the interrupt notifying the disconnection has not been received, the process returns to step S81, and afterward a similar process is repeatedly performed.

As described above, the error processing unit <NUM> can quickly identify a lane in which many CRC errors are detected, a disconnected lane notified by the error notification short packet, a lane in which unsuccessful reception has frequently occurred, and a lane notified of an interrupt notifying disconnection by receiving the interrupt, and the like. Therefore, the application processor <NUM> can stop using the lane identified by the error processing unit <NUM> and perform continuous communication using normal lanes only. Accordingly, it is possible for the communication system <NUM> to improve the reliability and to support securer communication such as that for on-board items.

An example configuration of a second embodiment of a communication system to which the present technology is applied will be described with reference to <FIG>.

With <FIG>, the communication system <NUM> according to the first embodiment and a communication system 11A according to the second embodiment will be compared and described.

For example, as illustrated in A of <FIG>, the communication system <NUM> has a configuration in which the CSI-<NUM> transmitter is connected to a physical layer (A-PHY Tx) on an image sensor <NUM> side, and a physical layer (A-PHY Rx) is connected to the CSI-<NUM> receiver on an application processor <NUM> side.

Accordingly, it is necessary in the communication system <NUM> to change a circuit configuration with respect to the existing CSI-<NUM> receiver and CSI-<NUM> transmitter.

On the other hand, as illustrated in B of <FIG>, the communication system 11A has a configuration in which the CSI-<NUM> transmitter and the physical layer (A-PHY Tx) are connected via an adaptation layer on the image sensor <NUM> side, and the physical layer (A-PHY Rx) and the CSI-<NUM> receiver are connected via an adaptation layer on the application processor <NUM> side.

By providing the adaptation layers as described above, circuit configurations of the existing CSI-<NUM> receiver and CSI-<NUM> transmitter can be directly used in the communication system 11A. That is, it is possible to avoid changing the circuit configurations of the existing CSI-<NUM> receiver and CSI-<NUM> transmitter by providing adaptation layers having functions necessary for parallel transmission such as that described with reference to <FIG> described above.

That is, as illustrated in <FIG>, processes of transmitting a packet header and an MC independently in each lane, calculating CRC from a payload in each lane and transmitting the CRC independently in each lane, and the like can be additionally performed by adaptation layers. Furthermore, a payload similar to the conventional CSI-<NUM> can be processed in the existing CSI-<NUM> receiver and CSI-<NUM> transmitter.

<FIG> illustrates a schematic example configuration of each of an image sensor 21A and an application processor 22A constituting the communication system 11A.

As illustrated in <FIG>, the image sensor 21A includes a non-volatile memory <NUM>, a CSI-<NUM> transmitter <NUM>, an adaptation layer <NUM>, a selector <NUM>, and a physical layer processing unit <NUM>. The application processor 22A includes a non-volatile memory <NUM>, a physical layer processing unit <NUM>, an adaptation layer <NUM>, a selector <NUM>, and a CSI-<NUM> transmitter <NUM>.

For example, the physical layer processing unit <NUM> and the physical layer processing unit <NUM> can selectively switch any of the physical layers of A-PHY, D-PHY, and C-PHY. Then, a selection signal for selecting any of the physical layers is supplied from the non-volatile memory <NUM> to each block of the image sensor 21A, and is supplied from the non-volatile memory <NUM> to each block of the application processor 22A.

Accordingly, for example, in a case where the selection signal indicates that A-PHY is selected, in the image sensor 21A, the CSI-<NUM> transmitter <NUM> supplies a packet to the adaptation layer <NUM>, and the adaptation layer <NUM> generates a packet header and an MC for each lane, or calculates CRC from a payload in each lane. Then, the selector <NUM> supplies an output from the adaptation layer <NUM> to the physical layer processing unit <NUM>, and a process using the A-PHY as the physical layer is performed in the physical layer processing unit <NUM>.

Then, in the application processor 22A, the process using the A-PHY as the physical layer is performed in the physical layer processing unit <NUM>, a received packet is supplied to the adaptation layer <NUM>, and a process is performed on an object transmitted in each lane. Then, the selector <NUM> supplies an output from the adaptation layer <NUM> to the CSI-<NUM> transmitter <NUM>, and a process is performed on a packet in the CSI-<NUM> transmitter <NUM>.

On the other hand, in a case where the selection signal indicates that D-PHY or C-PHY is selected, in the image sensor 21A, the CSI-<NUM> transmitter <NUM> supplies a packet to the physical layer processing unit <NUM> via the selector <NUM>, and a process using the D-PHY or the C-PHY as the physical layer is performed in the physical layer processing unit <NUM>.

Then, in the application processor 22A, the process using the D-PHY or the C-PHYY as the physical layer is performed in the physical layer processing unit <NUM>, and a received packet is supplied to the CSI-<NUM> transmitter <NUM> via the selector <NUM>.

<FIG> is a block diagram illustrating an example configuration of hardware of a computer caused to execute the series of processes described above by a program.

In the computer, a central processing unit (CPU) <NUM>, a read only memory (ROM) <NUM>, a random access memory (RAM) <NUM>, and an electronically erasable programmable read only memory (EEPROM) <NUM> are connected to one another by a bus <NUM>. An input/output interface <NUM> is further connected to the bus <NUM>, and the input/output interface <NUM> is connected to the outside.

In the computer with the above configuration, the CPU <NUM> loads, for example, programs stored in the ROM <NUM> and the EEPROM <NUM> to the RAM <NUM> via the bus <NUM> to execute the programs, and thereby the above-described series of processes is performed. Furthermore, such a program executed by the computer (CPU <NUM>) can be written in advance in the ROM <NUM>, or can be installed or updated in the EEPROM <NUM> from the outside via the input/output interface <NUM>.

Here, the processes performed herein by the computer in accordance with the program do not necessarily have to be performed in time series in the order described as the flowcharts. That is, in the processes performed by the computer in accordance with the program, a process executed in parallel or individually (for example, a parallel process or an object-based process) is also included.

Furthermore, the program may be processed by one computer (processor) or may be processed by a plurality of computers in a distributed manner. Moreover, the program may be transferred to a distant computer and executed.

Moreover, a system as used herein means a collection of multiple components (such as an apparatus and a module (part)), irrespective of whether or not all components are included in the same housing. Accordingly, both of multiple apparatuses accommodated in separate housings and connected through a network, and one apparatus in which multiple modules are accommodated in one housing are systems.

Furthermore, for example, the configuration described in the above as one apparatus (or processing unit) may be divided and configured as multiple apparatuses (or processing units). On the contrary, the configuration described in the above as multiple apparatuses (or processing units) may be integrated and configured as one apparatus (or processing unit). Furthermore, as a matter of course, a configuration other than those described above may be added to the configuration of each apparatus (or each processing unit). Moreover, as long as a configuration or an operation of the system as a whole is substantially the same, a part of a configuration of an apparatus (or processing unit) may be included in a configuration of another apparatus (or another processing unit).

Furthermore, for example, the present technology may have a cloud computing configuration in which multiple apparatuses share one function through a network, and jointly perform a process.

Furthermore, for example, the above-described program can be executed in any apparatus. In that case, it is enough for the apparatus to have necessary functions (functional blocks and the like) so as to be able to obtain necessary information.

Furthermore, for example, each step described with the above flowcharts can be executed by one apparatus, and in addition, can be shared and executed by multiple apparatuses. Moreover, in a case where multiple processes are included in one step, the multiple processes included in the one step can be executed by one apparatus, and in addition, can be shared and executed by multiple apparatuses. In other words, multiple processes included in one step can be also executed as processes of multiple steps. On the contrary, processes described as multiple steps can be collectively executed as one step.

Note that regarding the program executed by the computer, processes of a step of describing the program may be executed in time series in the order described herein, or may be executed in parallel or individually at a necessary timing, for example, when a call is made. That is, as long as there occurs no contradiction, processes of respective steps may be executed in an order different from the above-described order. Moreover, the processes of the step of describing the program may be executed in parallel with a process of another program, or may be executed in combination with a process of another program.

Note that each of the multiple present technologies described herein can be independently implemented as long as there occurs no contradiction. As a matter of course, any multiple present technologies can be used in combination. For example, some or all of the present technology described in any of the embodiments can be combined with some or all of the present technology described in other embodiments and implemented. Furthermore, some or all of any present technology described above can be implemented in combination with another technology which has not been described above.

Claim 1:
A communication apparatus (<NUM>, <NUM>), for communicating based on the Camera Serial Interface <NUM> specification, comprising:
a transmission unit (<NUM>, <NUM>) configured to, when a packet is distributed to a plurality of lanes and transmitted to a communication apparatus (<NUM>, <NUM>) on a receiving side, transmit a packet header and a packet footer of the packet parallelly in each of the lanes, and
a payload header generation unit (<NUM>) configured to generate a payload header placed at a head of a payload of the packet,
wherein the transmission unit (<NUM>, <NUM>) is configured to transmit the payload header parallelly in each of the lanes, and
the payload header includes message count that is incremented in each of the lanes, and includes, a lane number that identifies each of the lanes to be transmitted.