A computer-implemented method for real-time video processing, the method comprising: receiving a video including a sequence of images; identifying at least one face as an object of interest in one or more of the images; detecting feature reference points of the object of interest; generating a CANDIDE-3 virtual face mesh which is a parameterized three-dimensional face mesh controlled by shape units, action units and a position vector, the shape units controlling a face shape of the mesh, the action units controlling facial mimics that vary with facial expression, and the position vector corresponding to a rotation of the face around three axes and translation along the axes; tracking the object of interest in the video, wherein the tracking comprises aligning said virtual face mesh to the object of interest in at least some of the images based at least in part on the feature reference points, a state of the aligned mesh being described by an intensity vector of the shape units, an intensity vector of the action units and said position vector; identifying a background in each of the images by separating the object of interest from each image based at least in part on the virtual face mesh; modifying the background in each of the images, thereby generating a modified background; and generating a modified video which includes the object of interest and the modified background wherein the identifying of the background comprises, assuming that the observed face is neutral and frontal viewed in the image, projecting the mesh points of the generated virtual face on the image plane by scaled orthographic projection using the equation R x i y i + ∑ j = 1 m X ij ∗ b j ∑ j = 1 m Y ij ∗ b j + x y = z x ^ i y ^ i, where R = cos θ − sin θ sin θ cos θ is a rotation matrix corresponding to a found yaw θ, b j is j-th shape unit intensity; x, y, z are mesh translational coordinates; xi and yi are i-th mesh vertex model coordinates; x̂ i and ŷ i are i-th mesh vertex image coordinates obtained as weighted combinations of reference feature points; and Xij, Yij are coefficients, which denote how the i-th mesh vertex model are changed by j-th shape unit. receiving a video including a sequence of images; identifying at least one face as an object of interest in one or more of the images; detecting feature reference points of the object of interest; generating a CANDIDE-3 virtual face mesh which is a parameterized three-dimensional face mesh controlled by shape units, action units and a position vector, the shape units controlling a face shape of the mesh, the action units controlling facial mimics that vary with facial expression, and the position vector corresponding to a rotation of the face around three axes and translation along the axes; tracking the object of interest in the video, wherein the tracking comprises aligning said virtual face mesh to the object of interest in at least some of the images based at least in part on the feature reference points, a state of the aligned mesh being described by an intensity vector of the shape units, an intensity vector of the action units and said position vector; identifying a background in each of the images by separating the object of interest from each image based at least in part on the virtual face mesh; modifying the background in each of the images, thereby generating a modified background; and generating a modified video which includes the object of interest and the modified background wherein the identifying of the background comprises, assuming that the observed face is neutral and frontal viewed in the image, projecting the mesh points of the generated virtual face on the image plane by scaled orthographic projection using the equation R x i y i + ∑ j = 1 m X ij ∗ b j ∑ j = 1 m Y ij ∗ b j + x y = z x ^ i y ^ i, where R = cos θ − sin θ sin θ cos θ is a rotation matrix corresponding to a found yaw θ, b j is j-th shape unit intensity; x, y, z are mesh translational coordinates; xi and yi are i-th mesh vertex model coordinates; x̂ i and ŷ i are i-th mesh vertex image coordinates obtained as weighted combinations of reference feature points; and Xij, Yij are coefficients, which denote how the i-th mesh vertex model are changed by j-th shape unit.