A computer-implemented method for analyzing a sequence of video frames depicting a scene captured by a video camera, the method comprising: passing each of a plurality of kinematic data vectors derived from analyzing a foreground object being a vehicle detected in the sequence of video frames to a plurality of adaptive resonance theory networks, wherein each of the kinematic data vectors comprises kinematic data describing a position, a movement, and a speed of the foreground object; each ART network modeling a subset of kinematic data selected from the kinematic data vectors, the modeled subsets of kinematic data being distinct with respect to each other; identifying, in each ART network, a cluster to which each respective subset of kinematic data parsed from the plurality of kinematic data vectors is mapped; generating, for each of the ART networks, a respective primitive trajectory based on the clusters identified in each of the ART networks, the primitive trajectory being a sequence of cluster labels of the identified clusters; generating a symbol trajectory from the primitive trajectories by combining the primitive trajectories into a symbol trajectory including symbols defined as respective sets of cluster labels of the primitive trajectories; and determining whether the symbol trajectory is anomalous comprising computing a probability of observing the symbol trajectory in the scene, and upon determining that a probability of observing the symbol trajectory falls below a threshold value, alerting a user upon of the video surveillance system of an occurrence of an anomalous trajectory, wherein the probability of observing the symbol trajectory in the scene is computed based on a plurality of symbol trajectories previously observed in the scene. passing each of a plurality of kinematic data vectors derived from analyzing a foreground object being a vehicle detected in the sequence of video frames to a plurality of adaptive resonance theory networks, wherein each of the kinematic data vectors comprises kinematic data describing a position, a movement, and a speed of the foreground object; each ART network modeling a subset of kinematic data selected from the kinematic data vectors, the modeled subsets of kinematic data being distinct with respect to each other; identifying, in each ART network, a cluster to which each respective subset of kinematic data parsed from the plurality of kinematic data vectors is mapped; generating, for each of the ART networks, a respective primitive trajectory based on the clusters identified in each of the ART networks, the primitive trajectory being a sequence of cluster labels of the identified clusters; generating a symbol trajectory from the primitive trajectories by combining the primitive trajectories into a symbol trajectory including symbols defined as respective sets of cluster labels of the primitive trajectories; and determining whether the symbol trajectory is anomalous comprising computing a probability of observing the symbol trajectory in the scene, and upon determining that a probability of observing the symbol trajectory falls below a threshold value, alerting a user upon of the video surveillance system of an occurrence of an anomalous trajectory, wherein the probability of observing the symbol trajectory in the scene is computed based on a plurality of symbol trajectories previously observed in the scene.