Patent application EP20465534A (EP), filed 2020-06-09
Publication number: EP3923200A1
Title: PREDICTION OF FUTURE SENSORY OBSERVATIONS OF A DISTANCE RANGING DEVICE
Abstract: [0001]    The present invention is related to a method, a computer program code, and an apparatus for predicting future sensory observations of a distance ranging device of an autonomous or semi-autonomous vehicle. The invention is further related to an autonomous driving controller using such a method or apparatus and to an autonomous or semi-autonomous vehicle comprising such an autonomous driving controller. In a first step, a sequence of previous sensory observations and a sequence of control actions are received (S1). The sequence of previous sensory observations and the sequence of control actions are then processed (S2) with a temporal neural network to generate a sequence of predicted future sensory observations. Finally, the sequence of predicted future sensory observations is output (S3) for further use.
IPC codes: G05D   1/    02            A I
G06N   3/    04            A I
G06N   3/    08            A I
Claims: number: 1
text: A method for predicting future sensory observations (Ω̂<<t,t+P>) of a distance ranging device (42) of an autonomous or semi-autonomous vehicle (40), the method comprising:
number: 2
text: The method according to claim 1, wherein the temporal neural network (10) uses a gated recurrent unit (15).
number: 3
text: The method according to claim 1 or 2, wherein an observation input layer of the temporal neural network (10) is a first multi-layer perceptron.
number: 4
text: The method according to claim 3, wherein the first multi-layer perceptron uses three dense layers (11) and rectified linear unit activations.
number: 5
text: The method according to one of the preceding claims, wherein an action input layer of the temporal neural network (10) is a second multi-layer perceptron.
number: 6
text: The method according to claim 5, wherein the second multi-layer perceptron uses rectified linear unit activations.
number: 7
text: The method according to claim 5 or 6, wherein the temporal neural network (10) uses lambda layers (14) for splitting the action input layer at a desired timestep.
number: 8
text: The method according to one of the preceding claims, wherein predictors (17) of the temporal neural network (10) are multi-layer perceptrons with two layers.
number: 9
text: The method according to one of the preceding claims, wherein the temporal neural network (10) uses batch normalization layers (13) for performing a normalization of previous activations of a layer.
number: 10
text: The method according to one of the preceding claims, wherein the distance ranging device (42) is one of an ultrasonic sensor, a laser scanner, a lidar sensor, a radar sensor, and a camera.
number: 11
text: A computer program code comprising instructions, which, when executed by at least one processor, cause the at least one processor to perform a method according to any of claims 1 to 10 for predicting future sensory observations (Ω̂<<t,t+P>) of a distance ranging device (42) of an autonomous or semi-autonomous vehicle (40).
number: 12
text: An apparatus (20) for predicting future sensory observations (Ω̂<<t,t+P>) of a distance ranging device (42) of an autonomous or semi-autonomous vehicle (40), the apparatus (20) comprising:
number: 13
text: An autonomous driving controller (41), characterized in that the autonomous driving controller (41) comprises an apparatus (20) according to claim 12 or is configured to perform a method according to any of claims 1 to 10 for predicting future sensory observations (Ω̂<<t,t+P>) of a distance ranging device (42) of an autonomous or semi-autonomous vehicle (40).
number: 14
text: An autonomous or semi-autonomous vehicle (40), characterized in that the autonomous or semi-autonomous vehicle (40) comprises an autonomous driving controller (41) according to claim 13.
Office action: search_report, dated 2021-12-15
Rejection reasons: novelty_epc_54
Prior art cited: ref_id: WO2015144410A1
ref_type: patent
severity: background
ref_id: - M. BOJARSKI ET AL.: "End to end learning for self-driving cars", ARXIV: 1604.07316, 2016, Retrieved from the Internet <URL:https://arxiv.org/abs/1604.07316>
ref_type: npl
severity: background
ref_id: - V. MNIH ET AL.: "Human-level control through deep reinforcement learning", NATURE, vol. 518, 2015, pages 529 - 533, XP055283401, DOI: 10.1038/nature14236
ref_type: npl
severity: background
ref_id: - R. HADSELL ET AL.: "Learning long-range vision for autonomous off-road driving", JOURNAL OF FIELD ROBOTICS, vol. 26, 2009, pages 120 - 144, XP055134670, DOI: 10.1002/rob.20276
ref_type: npl
severity: background
ref_id: - W. LOTTER ET AL.: "Deep predicting coding networks for video prediction and unsupervised learning", ARXIV: 1605.08104, 2016, Retrieved from the Internet <URL:https://arxiv.org/abs/1605.08104>
ref_type: npl
severity: background
ref_id: - M. G. AZAR ET AL.: "World discovery models", ARXIV:1902.07685, 2019, Retrieved from the Internet <URL:https://arxiv.org/abs/1902.07685>
ref_type: npl
severity: background
ref_id: - K. CHO ET AL.: "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation", ARXIV: 1406.1078, 2014, Retrieved from the Internet <URL:https://arxiv.org/abs/1406.1078>
ref_type: npl
severity: background
ref_id: - D. P. KINGMA ET AL.: "Adam: A Method for Stochastic Optimization", ARXIV: 1412.6980, 2014, Retrieved from the Internet <URL:https://arxiv.org/abs/1412.6980>
ref_type: npl
severity: background
ref_id: WO2018211141A1
ref_type: patent
severity: background
ref_id: US2020148215A1
ref_type: patent
severity: background
ref_id: WO2018211141
ref_type: patent
severity: novelty_destroying
ref_id: US2020148215
ref_type: patent
severity: background
Final disposition: pending (2021-12-15)