Patent ID: 8589315

Claim:
A behavior recognition system for detecting the behavior of objects in a scene, the system comprising a processor and a memory coupled with the processor, wherein the processor includes: a semantic object stream module for receiving a video stream having at least two frames and detecting objects in the video stream; and a group organization module for utilizing the detected objects from the video stream to detect a behavior of the detected objects, including both static and moving objects, where the group organization module detects the behavior using a hierarchical event detection scheme further comprising: a spatial organization layer comprising an object group stream module, the object group stream module being configured to spatially organize the detected objects to have relative spatial relationships; and a temporal organization layer comprising a group action stream module, the group action stream module being configured to model a temporal structure of the detected objects, the temporal structure being an action of the detected objects between the two frames, whereby through detecting, organizing and modeling actions of objects, a user can detect the behavior of the objects; and wherein the group organization module is operative for using fuzzy attributed relational graphs (FARGs) for modeling spatial and temporal aspects of the behavior of the object to generate a scene FARG from the detected objects in the scene; wherein the FARGs include nodes and edges, with the nodes representing different objects in the scene and the edges representing relationships between the objects; wherein the processor is further configured to receive a user query to detect a desired behavior, and based on the query and objects in the video stream, identify if the objects exhibit the desired behavior; wherein the processor is further configured to associate the user query with a behavior FARG, with the behavior FARG modeling the spatial and temporal aspects of a behavior the user is searching for, and once associated, the processor matches the scene FARG with a behavior FARG to identify an object exhibiting the desired behavior; and wherein the scene FARG is matched with the behavior FARG by determining compatibilities between nodes in the FARGs and using the compatibilities in a fuzzy graph matching (FGM) algorithm to match the scene FARG with the behavior FARG.