A computer-implemented method, comprising: obtaining, by a computing system (100; 195) comprising one or more computing devices, sensor data associated with a surrounding environment of a first vehicle, wherein the sensor data comprises a sequence of image frames, each image frame corresponding to one of a plurality of time steps; determining, by the computing system and via one or more machine-learned models (740; 780), one or more regions of interest (230; 300) including at least a second vehicle and associated with the sensor data that comprises the sequence of image frames; determining, by the computing system and via one or more machine-learned models, one or more spatial features associated with at least one of the one or more regions of interest, wherein at least one of the one or more spatial features are indicative of an orientation of the second vehicle and wherein at least one of the one or more spatial features are indicative of one or more occluding objects that include any object within the one or more regions of interest (230; 300) other than the second vehicle; determining, by the computing system and via one or more machine-learned models, one or more temporal features associated with at least one of the one or more regions of interest, wherein the one or more temporal features are indicative of one or more states of at least one signal light of the second vehicle; determining, by the computing system and via one or more machine-learned models, an intention indicating a predicted movement of the second vehicle based on the one or more spatial features and the one or more temporal features associated with the sensor data including the sequence of image frames, wherein determining the intention comprises inputting the one or more spatial features and the one or more temporal features into at least one of the one or more machine-learned models configured to determine the intention; and initiating, by the computing system, one or more actions based on the intention. obtaining, by a computing system (100; 195) comprising one or more computing devices, sensor data associated with a surrounding environment of a first vehicle, wherein the sensor data comprises a sequence of image frames, each image frame corresponding to one of a plurality of time steps; determining, by the computing system and via one or more machine-learned models (740; 780), one or more regions of interest (230; 300) including at least a second vehicle and associated with the sensor data that comprises the sequence of image frames; determining, by the computing system and via one or more machine-learned models, one or more spatial features associated with at least one of the one or more regions of interest, wherein at least one of the one or more spatial features are indicative of an orientation of the second vehicle and wherein at least one of the one or more spatial features are indicative of one or more occluding objects that include any object within the one or more regions of interest (230; 300) other than the second vehicle; determining, by the computing system and via one or more machine-learned models, one or more temporal features associated with at least one of the one or more regions of interest, wherein the one or more temporal features are indicative of one or more states of at least one signal light of the second vehicle; determining, by the computing system and via one or more machine-learned models, an intention indicating a predicted movement of the second vehicle based on the one or more spatial features and the one or more temporal features associated with the sensor data including the sequence of image frames, wherein determining the intention comprises inputting the one or more spatial features and the one or more temporal features into at least one of the one or more machine-learned models configured to determine the intention; and initiating, by the computing system, one or more actions based on the intention.