Patent ID: 11971712
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G  B | IPC B  G

Claim 19:
20. A non-transitory machine-readable medium comprising executable code which when executed by one or more processors associated with a computing device of a mobile automated system are adapted to cause the one or more processors to perform a method comprising:
receiving, at the computing device, training data comprising one or more training navigation instruction samples;
generating, by a first attention module, an instruction grounding from the training navigation instruction sample for instructing the mobile automated system to navigate an environment in which the mobile automated system is located;
generating, by a second attention module, a visual grounding from one or more training images observed for the environment as the mobile automated system is navigating therethrough at the first time instance;
generating, by a neural model, a progress monitor probability in response to an input of the instruction grounding and the visual grounding;
generating an action probability of navigation directions based on a hidden state of the neural model and the instruction grounding;
train the neural model based at least in part on a first cross entropy between the progress monitor probability and a target progress label, and a second cross-entropy between the action probability and a ground-truth navigable direction; and, generating a navigational action for the mobile automated system to navigate the environment by the trained neural model.