  In autonomous driving, perceiving the driving behaviors of surrounding agents
is important for the ego-vehicle to make a reasonable decision. In this paper,
we propose a neural network model based on trajectories information for driving
behavior recognition. Unlike existing trajectory-based methods that recognize
the driving behavior using the hand-crafted features or directly encoding the
trajectory, our model involves a Multi-Scale Convolutional Neural Network
(MSCNN) module to automatically extract the high-level features which are
supposed to encode the rich spatial and temporal information. Given a
trajectory sequence of an agent as the input, firstly, the Bi-directional Long
Short Term Memory (Bi-LSTM) module and the MSCNN module respectively process
the input, generating two features, and then the two features are fused to
classify the behavior of the agent. We evaluate the proposed model on the
public BLVD dataset, achieving a satisfying performance.
