Functional units and working algorithms for real time visual recognition of human pick and place action sequences are presented. It generates a symbolic description of the observed action sequence. Given a different initial state, the system re-instantiates the recognized action sequence to carry out an equivalent assembly task. Experimental results on several assembly tasks support the effectiveness of the method. Action concepts are analyzed into attention, context, and change.
Temporal segmentation is defined as a context switch detected by a switching of attention.
Validity of the indexicality depends on the attention and the context. An action recognizer built upon the framework successfully recognized human assembly action sequences in real time and output qualitative descriptions of the tasks.