## 1 Introduction  
In recent years, automatic human activity recognition (HAR) based on computer vision has drawn much attention of researchers around the globe due to its promising results. The major applications of HAR include: Human Computer Interaction (HCI), intelligent video surveillance, ambient assisted living, human-robot interaction, entertainment, and content-based video search. In HCI, the activity recognition systems observe the task carried out by the user and guide him/her to complete it by providing feedback. In video surveillance, the activity recognition system can automatically detect a suspicious activity and report it to the authorities for immediate action. Similarly, in entertainment, these systems can recognize the activities of different players in the game. Depending on the complexity and duration, activities fall into four categories, i.e., gestures, actions, interactions, and group activities [1] as shown in Figure 1.  
Gesture: A gesture is defined as a basic movement of the human body parts that carry some meaning. 'Head shaking', 'hand waving', and 'facial expression' are some good examples of gestures. Usually, a gesture takes a very short amount of time and its complexity is the lowest among the four mentioned categories. Action: An action is a type of an activity that is performed by a single person. In fact, it is a combination of multiple gestures (atomic actions). 'Walking' 'running', 'jogging', and 'punching' are some good examples of human actions. Interaction: It is a type of an activity performed by two actors. One actor must be a human and the other one may be a human or an object. Thus, it could be human-human interaction or human-object interaction. 'Fighting between two persons', 'hand shaking', and 'hugging each other' are examples of human-human interaction, while 'a person using an ATM', 'a person using a computer', 'and a person stealing a bag' are examples of human-object interaction. Group Activity: This is the most complex type of activity. Certainly, it is a combination of gestures, actions, and interactions. It involves more than two humans and a single or multiple objects. A 'group of people protesting', 'two teams playing a game', and a 'group meeting', are good examples of group activities. Since the 1980s, researchers have been working on human action recognition from images and videos. One of the important directions that researchers have been following for action recognition is similar to the working of the human vision system. At low level, the human vision system can receive the series of observations regarding the movement and shape of the human body in a short span of time. Then, these observations are passed to the intermediate human perception system for further recognition of the class of these observations, such as walking, jogging, and running. In fact, the human vision and perception system is robust and very accurate in recognition of observed movement and human activities. In order to achieve a similar level of performance by a computer-based recognition system, researchers have carried out a lot of efforts during the past few decades. However, unfortunately, due to many challenges and issues involved in HAR such as environmental complexities, intra-class variations, viewpoint variations, occlusions, and non-rigid shape of the humans and objects, we are still very far from the level of the human vision system. What we have achieved so far may be a fraction of what a mature human vision system can do. Based on the comprehensive investigation of the literature, vision-based human activity recognition approaches can be divided into two major categories. (1) The traditional handcrafted representation-based approach, which is based on the expert designed feature detectors and descriptors such as Hessian3D, Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), Enhanced Speeded-Up Robust Features (ESURF), and Local Binary Pattern (LBP).