Modeling Context using Petri Nets for Activity Recognition

This work considers the problem of recognizing activities in surveillance video. Activities are high-level non-atomic semantic concepts which may have complex temporal structure. Activities are not easily identifiable using image features, but rather by the recognition of their composing events. Unfortunately, these composing events may, generally, only be observed up to a particular certainty. Approaches to classification/recognition in computer vision rely on the availability of large corpora of training data. This data can be input into learning algorithms, yielding a classifier for future examples. The domain of activity recognition in surveillance video, generally does not have this luxury. Interesting activities are usually rare, and few (if any) training examples are available for training. This limitation limits the usefulness of probabilistic state space models such as Hidden Markov Models(HMM), for activity recognition. These type of models generally require a number of examples on the order of the state space. For these reasons, The activity recognition community has thus turned to using formal specifications of domain knowledge to model activities in video. Thus, the leading approaches to activity recognition construct a model describing the activity model using some choice of formal methods(e.g. Petri Nets, Propagation Networks, Temporal Constraint Logic). This formal specification is then translated into a Bayesian framework in which the activity progress can be estimated using (uncertain) observations of events. ∗Corresponding author. Tel: +972-50-405-6074; fax: +972-4-829-3900. E-mail address: gallavee@cs.technion. ac.il(G. Lavee). 1 T ec hn io n C om pu te r Sc ie nc e D ep ar tm en t T eh ni ca l R ep or t C IS -2 01 207 2 01 2 One major failing of current approaches is that they fail to separate the physical constraints of the scene (which we call context) and the constraints of the activity. The context constraints must then be modeled explicitly within the specification of each activity. This leads to an increase in the size of the model, and thus state space over which we must perform probabilistic estimation. This paper describes FPFPN (Factored Particle Filter Petri Net), an activity recognition process which addresses the issue described above. That is, the context state and activity state are separately modeled. This intuition yields simpler activity models, that are able to achieve better recognition results (as we demonstrate experimentally). Furthermore, the factored nature of our approach, results in a reduced state space. The particle filter literature has shown that less particles are required to perform estimation when the state space is smaller [5]. In our experiments we validate this result, showing that indeed a smaller number of particles can be used to achieve a the maximal recognition rate using our proposed approach. As the complexity of particle filter approaches is a (linear) function of the number of particles used, our construction yields a more efficient recognition algorithm. Another, benefit of our approach is the modularity of its design. Unlike previous approaches, which rely on an ad hoc construction of activity models and require an in-depth understanding of the activity formalism, our approach affords a structured activity model construction which only requires knowledge of the temporal structure of the activity.