A sensor unit utilized in an autonomous driving vehicle, the sensor unit comprising: a sensor interface to be coupled to a plurality of sensors (510, 510A, 510B, 510C) mounted on a plurality of locations of the Autonomous Driving Vehicle, ADV, the plurality of sensors being configured to detect information about an environment in which the ADV operates; a host interface to be coupled to a host system, so that the host system perceives a driving environment surrounding the ADV based on sensor data obtained from the sensors and plans a path to autonomously drive the ADV; and a plurality of data transfer modules (502, 502A, 502B, 502C) corresponding to the plurality of sensors (510, 510A, 510B, 510C), wherein each of the data transfer modules is capable of being configured to operate in one of a plurality of operating modes to transfer data between a corresponding sensor and the host system, dependent upon a type of the corresponding sensor, the operating modes including a low latency mode and a high bandwidth mode; wherein each of the plurality of sensors (510A, 510B, 510C) is associated with a respective specific channel (channel 1, channel 2, channel N) for processing sensor data and transferring the processed sensor data between the host system and the corresponding sensor, wherein each channel includes a respective specific data transfer module of the plurality of data transfer modules; wherein when operating in the low latency mode, each data transfer module transfers data with a minimum delay; and when operating in the high bandwidth mode, each of the data transfer modules is configured to accumulate the data up to a predetermined amount, prior to sending the accumulated data in a burst to the host system to maximize a bandwidth of a connection between the data transfer module and the host system. a sensor interface to be coupled to a plurality of sensors (510, 510A, 510B, 510C) mounted on a plurality of locations of the Autonomous Driving Vehicle, ADV, the plurality of sensors being configured to detect information about an environment in which the ADV operates; a host interface to be coupled to a host system, so that the host system perceives a driving environment surrounding the ADV based on sensor data obtained from the sensors and plans a path to autonomously drive the ADV; and a plurality of data transfer modules (502, 502A, 502B, 502C) corresponding to the plurality of sensors (510, 510A, 510B, 510C), wherein each of the data transfer modules is capable of being configured to operate in one of a plurality of operating modes to transfer data between a corresponding sensor and the host system, dependent upon a type of the corresponding sensor, the operating modes including a low latency mode and a high bandwidth mode; wherein each of the plurality of sensors (510A, 510B, 510C) is associated with a respective specific channel (channel 1, channel 2, channel N) for processing sensor data and transferring the processed sensor data between the host system and the corresponding sensor, wherein each channel includes a respective specific data transfer module of the plurality of data transfer modules; wherein when operating in the low latency mode, each data transfer module transfers data with a minimum delay; and when operating in the high bandwidth mode, each of the data transfer modules is configured to accumulate the data up to a predetermined amount, prior to sending the accumulated data in a burst to the host system to maximize a bandwidth of a connection between the data transfer module and the host system.