A method of recognizing a human body orientation based on a two-lens camera, comprising: a receipt step of receiving a grayscale or color image and a corresponding depth map including the human body, obtained by the two-lens camera; an obtainment step of conducting foreground recognition to get foreground pixels so as to carry out XYZ-dimensional reconstruction with respect to a depth map of the foreground pixels in a world coordinate system to acquire a three-dimensional image, and then, projecting the three-dimensional image onto the XZ plane in the world coordinate system so as to obtain a top view, wherein, X represents width, Y denotes height, and Z refers to depth; a tracking step of tracking a movement trajectory of the human body based on the top view; a determination step of determining whether a moving velocity of the human body acquired according to the movement trajectory of the human body is greater than a predetermined threshold; a first recognition step of recognizing, if the moving velocity of the human body is greater than the predetermined threshold, a direction of the movement trajectory of the human body as the human body orientation; a first acquirement step of conducting, if the moving velocity of the human body is less than or equal to the predetermined threshold, classification with respect to the top view by using a perpendicular-to-shoulder direction classifier so as to acquire an initial pair of relative directions from plural predetermined pairs of relative directions; a second acquirement step of conducting back projection onto the plane XY with respect to the three-dimensional image so as to acquire two-dimensional information of a head region; and a second recognition step of conducting classification with respect to the two-dimensional information of the head region by utilizing a face binary classifier so as to select one direction from the initial pair of relative directions which corresponds to an orientation of a facial portion in the head region to serve as the human body orientation. a receipt step of receiving a grayscale or color image and a corresponding depth map including the human body, obtained by the two-lens camera; an obtainment step of conducting foreground recognition to get foreground pixels so as to carry out XYZ-dimensional reconstruction with respect to a depth map of the foreground pixels in a world coordinate system to acquire a three-dimensional image, and then, projecting the three-dimensional image onto the XZ plane in the world coordinate system so as to obtain a top view, wherein, X represents width, Y denotes height, and Z refers to depth; a tracking step of tracking a movement trajectory of the human body based on the top view; a determination step of determining whether a moving velocity of the human body acquired according to the movement trajectory of the human body is greater than a predetermined threshold; a first recognition step of recognizing, if the moving velocity of the human body is greater than the predetermined threshold, a direction of the movement trajectory of the human body as the human body orientation; a first acquirement step of conducting, if the moving velocity of the human body is less than or equal to the predetermined threshold, classification with respect to the top view by using a perpendicular-to-shoulder direction classifier so as to acquire an initial pair of relative directions from plural predetermined pairs of relative directions; a second acquirement step of conducting back projection onto the plane XY with respect to the three-dimensional image so as to acquire two-dimensional information of a head region; and a second recognition step of conducting classification with respect to the two-dimensional information of the head region by utilizing a face binary classifier so as to select one direction from the initial pair of relative directions which corresponds to an orientation of a facial portion in the head region to serve as the human body orientation.