Patent application EP20180636A (EP), filed 2020-06-17
Publication number: EP3926360A1
Title: NEURAL NETWORK BASED METHODS AND SYSTEMS FOR OBJECT DETECTION USING CONCATENATED LIDAR, RADAR AND CAMERA DATA SETS
Abstract: [0001]    A computer implemented method for object detection comprises the following steps carried out by computer hardware components: acquiring a plurality of lidar data sets from a lidar sensor; acquiring a plurality of radar data sets from a radar sensor; acquiring at least one image from a camera; determining concatenated data based on casting the plurality of lidar data sets and the plurality of radar data sets to the at least one image; and detecting an object based on the concatenated data.
IPC codes: G01S  13/    08            A I
G01S  13/    86            A I
G01S  17/    08            A I
G01S  17/    86            A I
G01S  17/   931            A I
G01S   7/    41            A I
Claims: number: 1
text: Computer implemented method for object detection,
the method comprising the following steps carried out by computer hardware components:
number: 2
text: The computer implemented method of claim 1, further comprising the following step carried out by the computer hardware components:
determining a plurality of camera residual blocks.
number: 3
text: The computer implemented method of at least one of claims 1 or 2,
wherein the camera data is processed using a first artificial neural network.
number: 4
text: The computer implemented method of at least one of claims 1 to 3,
wherein the casting comprises aligning a plurality of sweeps of the lidar data sets.
number: 5
text: The computer implemented method of at least one of claims 1 to 4,
further comprising the following step carried out by the computer hardware components:
carrying out linear depth completion of the lidar data sets.
number: 6
text: The computer implemented method of at least one of claims 1 to 5,
further comprising the following step carried out by the computer hardware components:
determining a plurality of lidar residual blocks based on the linear depth completed lidar data.
number: 7
text: The computer implemented method of at least one of claims 1 to 6,
wherein the lidar data sets are processed using a second artificial neural network.
number: 8
text: The computer implemented method of at least one of claims 1 to 7,
wherein the casting comprises aligning a plurality of sweeps of the plurality of radar data sets.
number: 9
text: The computer implemented method of at least one of claims 1 to 8,
further comprising the following step carried out by the computer hardware components:
determining a plurality of radar residual blocks.
number: 10
text: The computer implemented method of at least one of claims 1 to 9,
wherein the radar data is processed using a third artificial neural network.
number: 11
text: The computer implemented method of at least one of claims 1 to 10, further comprising the following step carried out by the computer hardware components:
concatenating the plurality of camera residual blocks, the plurality of lidar residual blocks, and the plurality of radar residual blocks
number: 12
text: Computer system, the computer system comprising a plurality of computer hardware components configured to carry out steps of the computer implemented method of at least one of claims 1 to 11.
number: 13
text: Vehicle, comprising a radar sensor, a lidar sensor and a camera, wherein the vehicle is configured to detect objects according to the computer implemented method of at least one of claims 1 to 11.
number: 14
text: Non-transitory computer readable medium comprising instructions for carrying out the computer implemented method of at least one of claims 1 to 11.
Office action: search_report, dated 2021-12-22
Rejection reasons: novelty_epc_54
Prior art cited: ref_id: US2018314253A1
ref_type: patent
severity: background
ref_id: EP3525000A1
ref_type: patent
severity: background
ref_id: US2020160559A1
ref_type: patent
severity: background
ref_id: - CHANG SHUO ET AL: "Spatial Attention Fusion for Obstacle Detection Using MmWave Radar and Vision Sensor", SENSORS, vol. 20, no. 4, 11 February 2020 (2020-02-11), CH, pages 956, XP093195499, ISSN: 1424-8220, Retrieved from the Internet <URL:https://www.mdpi.com/1424-8220/20/4/956> [retrieved on 20250915], DOI: 10.3390/s20040956
ref_type: npl
severity: background
ref_id: US2018314253
ref_type: patent
severity: novelty_destroying
Final disposition: pending (2021-12-22)