Patent ID: 11971712
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G  B | IPC B  G

Claim 0:
1. A computing device of a mobile automated system comprising:
a memory containing machine readable medium storing machine executable code; and
one or more processors coupled to the memory and configurable to execute the machine executable code to cause the one or more processors to:
receive training data comprising one or more training navigation instruction samples;
generate, by a first attention module, an instruction grounding from the training navigation instruction sample for instructing the mobile automated system to navigate an environment in which the mobile automated system is located;
generate, by a second attention module, a visual grounding from one or more training images observed for the environment as the mobile automated system is navigating therethrough at the first time instance;
generate, by a neural model, a progress monitor probability in response to an input of the instruction grounding and the visual grounding;
generate an action probability of navigation directions based on a hidden state of the neural model and the instruction grounding;
train the neural model based at least in part on a first cross entropy between the progress monitor probability and a target progress label, and a second cross-entropy between the action probability and a ground-truth navigable direction; and
generate a navigational action for the mobile automated system to navigate the environment by the trained neural model.