Patent application EP22154048A (EP), filed 2022-01-28
Publication number: EP4095812A1
Title: METHOD FOR PREDICTING A TRAJECTORY OF AN AGENT IN A VICINITY OF A SELF-DRIVING VEHICLE BASED ON RANKING
Abstract: [0001]    A method (600) of operating a self-driving vehicle is provided. The method includes predicting a trajectory of an agent in a vicinity of the self-driving vehicle. This is done by: receiving sensor data indicative of a current situation in the vicinity of the self-driving vehicle; generating a current situation feature vector based, at least in part, on the sensor data; searching a pre-built bank of library feature vectors to identify a result feature vector from the pre-built bank that is most relevant to the current situation feature vector, wherein each library feature vector in the pre-built bank is associated with an observed trajectory; and predicting the trajectory of the agent based, at least in part, on the result feature vector. The method (600) also includes planning an action of the vehicle based, at least in part, on the predicted agent trajectory, and operating the vehicle according to the planned action.
IPC codes: G06V  20/    56            A I
Claims: number: 1
text: A method (600) of operating a self-driving vehicle (302), the method comprising:
number: 2
text: The method (600) of claim 1, wherein generating the current situation feature vector comprises using a pre-trained neural network to generate the current situation feature vector.
number: 3
text: The method (600) of claim 2, wherein using the pre-trained neural network comprises providing a feature map representing aspects of a birds-eye view of the current situation in the vicinity of the self-driving vehicle (302), based, at least in part, on the sensor data as input to the pre-trained neural network.
number: 4
text: The method (600) of claim 3, wherein the feature map is centered on the agent, and has an orientation based on an orientation of the agent.
number: 5
text: The method (600) of any of claims 1-4, wherein searching the pre-built bank of library feature vectors comprises performing an approximate nearest neighbor search.
number: 6
text: The method (600) of claim 5, wherein the pre-built bank of library feature vectors includes an index based on a hierarchical navigable small world graph, and wherein performing the approximate nearest neighbor search comprises traversing the hierarchical navigable small world graph.
number: 7
text: The method (600) of any of claims 1-6, wherein the result feature vector comprises the library feature vector that is closest to the current situation feature vector.
number: 8
text: The method (600) of any of claims 1-4, wherein searching the pre-built bank of library feature vectors comprises generating a ranked list of feature vectors from the pre-built bank.
number: 9
text: The method (600) of claim 8, wherein generating the ranked list of feature vectors comprises ranking the library feature vectors based, at least in part, on a measure of relevance of the library feature vectors to the current situation feature vector.
number: 10
text: The method (600) of any of claims 1-9, further comprising generating the pre-built bank of library feature vectors , the generating the pre-built bank of library feature vectors comprising:
number: 11
text: The method (600) of claim 10, wherein sampling, on the server, at least the first portion of the plurality of observation samples, to obtain the library of agent trajectories comprises:
number: 12
text: The method (600) of claim 10, wherein assembling, on the server, the bank of library feature vectors further comprises generating an index of library feature vectors in the bank of library feature vectors, and optionally wherein generating the index of library feature vectors comprises generating a proximity graph of the library feature vectors.
number: 13
text: The method (600) of claim 10, wherein generating, on the server, the library feature vector for each trajectory in the library of agent trajectories comprises using a neural network to generate the library feature vector from each trajectory in the library of agent trajectories.
number: 14
text: The method (600) of claim 10, wherein each observation sample further includes data on a scene surrounding the observed agent, and wherein the method further comprises:
number: 15
text: A self-driving vehicle (302) comprising:
Office action: search_report, dated 2022-11-30
Rejection reasons: novelty_epc_54
Prior art cited: ref_id: - Y. A. MALKOVD. A. YASHUNIN: "Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 42, no. 4, 1 April 2020 (2020-04-01), pages 824 - 836
ref_type: npl
severity: background
ref_id: US2021150350A1
ref_type: patent
severity: background
ref_id: - YURIY BIKTAIROV ET AL: "PRANK: motion Prediction based on RANKing", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 22 October 2020 (2020-10-22), XP081795889
ref_type: npl
severity: background
ref_id: - MALKOV YU A ET AL: "Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY, USA, vol. 42, no. 4, 28 December 2018 (2018-12-28), pages 824 - 836, XP011775565, ISSN: 0162-8828, [retrieved on 20200304], DOI: 10.1109/TPAMI.2018.2889473
ref_type: npl
severity: background
ref_id: - ANDREY RUDENKO ET AL: "Human Motion Trajectory Prediction: A Survey", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 15 May 2019 (2019-05-15), XP081371049
ref_type: npl
severity: background
ref_id: US2021150350
ref_type: patent
severity: background
ref_id: - YURIY BIKTAIROV ET AL, "PRANK: motion Prediction based on RANKing", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20201022), XP081795889 [X] 1-5,15 * the whole document * [I] 1-15
ref_type: npl
severity: novelty_destroying
ref_id: - MALKOV YU A ET AL, "Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY, USA, vol. 42, no. 4, doi:10.1109/TPAMI.2018.2889473, ISSN 0162-8828, (20181228), pages 824 - 836, (20200304), XP011775565 [A] 6 * the whole document *
ref_type: npl
severity: background
ref_id: - ANDREY RUDENKO ET AL, "Human Motion Trajectory Prediction: A Survey", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20190515), XP081371049 [A] 1-15 * the whole document *
ref_type: npl
severity: background
Final disposition: pending (2022-11-30)