A computer-implemented method of training a multi-task deep convolutional neural network, CNN for a multi-task deep learning of fine-grained classification and hyper-class classification, the CNN comprising low-level feature layers, configured to receive an input image, and high-level feature layers, and subsequent classifier layers, the method comprising: supplementing fine-grained training images, which are labeled by fine-grained classes, with auxiliary images labeled by two hyper-classes by identifying hyper-classes in the fine-grained training images and acquiring a number of hyper-class-labeled auxiliary images from external sources, the two hyper-classes being a super-class that subsumes a set of fine-grained classes and a factor-class that explains intra-class variance; and performing a multi-task deep learning using the fine-grained training images and the auxiliary images to learn a fine-grained classification model, a factor-class classification mode, and a super-class classification model by exploiting i) a first regularization between factor-class specific weights u v of classifier layers configuring the fine-grained classification model and weights w v,c of classifier layers configuring the factor-class classification model, and ii) a second regularization between super-class specific weights u vc of classifier layers configuring the fine-grained classification model and weights w vc,c of classifier layers configuring the super-class classification model, wherein the fine-grained classification model and the factor-class classification model share and learn weights of the preceding low-level and high-level feature layers, and the fine-grained classification model and the super-class classification model share and learn weights of the preceding low level and high-level feature layers, the first regularization is represented by β 2 ∑ v = 1 K Σ c = 1 C ‖ w v,c − u v ‖ 2 2, and the second regularization is represented by β 2 ∑ c = 1 C ‖ w vc,c − u vc ‖ 2 2. supplementing fine-grained training images, which are labeled by fine-grained classes, with auxiliary images labeled by two hyper-classes by identifying hyper-classes in the fine-grained training images and acquiring a number of hyper-class-labeled auxiliary images from external sources, the two hyper-classes being a super-class that subsumes a set of fine-grained classes and a factor-class that explains intra-class variance; and performing a multi-task deep learning using the fine-grained training images and the auxiliary images to learn a fine-grained classification model, a factor-class classification mode, and a super-class classification model by exploiting i) a first regularization between factor-class specific weights u v of classifier layers configuring the fine-grained classification model and weights w v,c of classifier layers configuring the factor-class classification model, and ii) a second regularization between super-class specific weights u vc of classifier layers configuring the fine-grained classification model and weights w vc,c of classifier layers configuring the super-class classification model, i) a first regularization between factor-class specific weights u v of classifier layers configuring the fine-grained classification model and weights w v,c of classifier layers configuring the factor-class classification model, and ii) a second regularization between super-class specific weights u vc of classifier layers configuring the fine-grained classification model and weights w vc,c of classifier layers configuring the super-class classification model, wherein the fine-grained classification model and the factor-class classification model share and learn weights of the preceding low-level and high-level feature layers, and the fine-grained classification model and the super-class classification model share and learn weights of the preceding low level and high-level feature layers, the first regularization is represented by β 2 ∑ v = 1 K Σ c = 1 C ‖ w v,c − u v ‖ 2 2, and the second regularization is represented by β 2 ∑ c = 1 C ‖ w vc,c − u vc ‖ 2 2.