Patent ID: 11971712
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G  B | IPC B  G

Claim 10:
11. A method of training a neural model at a computing device of a mobile automated system for monitoring navigation progress, the method comprising:
receiving, at the computing device, training data comprising one or more training navigation instruction samples;
generating, by a first attention module, an instruction grounding from the training navigation instruction sample for instructing the mobile automated system to navigate an environment in which the mobile automated system is located;
generating, by a second attention module, a visual grounding from one or more training images observed for the environment as the mobile automated system is navigating therethrough at the first time instance;
generating, by a neural model, a progress monitor probability in response to an input of the instruction grounding and the visual grounding;
generating an action probability of navigation directions based on a hidden state of the neural model and the instruction grounding;
training the neural model based at least in part on a first cross entropy between the progress monitor probability and a target progress label, and a second cross-entropy between the action probability and a ground-truth navigable direction; and
generating a navigational action for the mobile automated system to navigate the environment by the trained neural model.