Patent ID: 8494223

Claim:
A method of annotating video images associated with an environmental situation based on detected actions of a human interacting with the environmental situation, the method comprising: receiving, with an information processing system, a set of real-time video images captured by at least one video camera associated with an environment presenting one or more environmental situations to a human utilizing the environment; determining, based on one or more images in the set of real-time video images, that the human is encountering a given environmental situation; storing, based on the determining, the one or more images; monitoring, with the information processing system and based on the determining, one or more user actions made by the human with respect to the environmental situation; determining, based on the monitoring, that the human has one of performed and failed to perform at least one action associated with the environmental situation; and annotating, with the information processing system, the one or more images with a set of annotations based on the at least one action that has been one of performed and failed to be performed by the human.