A processor-implemented method for hand-gesture recognition, the method comprising: estimating a plurality of network-implicit 3D articulation priors having a plurality of key points of sample user's hands from sample RGB images using a deep learning network, wherein the plurality of key-points includes 21 hand key-points, comprising four key points per finger and one key point close to the wrist of the user's hand, wherein the deep learning network is pre-trained in order to estimate a plurality of key points, the deep learning network being trained by using a large scale 3D hand pose dataset having a plurality of training sample RGB images based on synthetic hand models, receiving, via one or more hardware processors, a plurality of frames of a media stream of a scene captured from a first person view (FPV) of a user using at least one RGB sensor communicably coupled to a wearable Augmented reality (AR) device, the media stream comprising RGB image data associated with the plurality of frames of the scene, the scene comprising a dynamic hand gesture performed by the user; estimating, via the one or more hardware processors, a temporal information associated with the dynamic hand gesture from the RGB image data by using the deep learning network, the temporal information being the coordinate values of the 21 key-points on the user's hand detected in each of the plurality of frames, wherein the estimated temporal information is associated with hand poses of the user and wherein the 21 key points of user's hand are detected by the deep learning network based on the plurality of network-implicit 3D articulation priors; and classifying, by using a multi-layered Long Short Term memory (LSTM) classification network, the dynamic hand gesture into at least one predefined gesture class based on the coordinate values of the 21 key-points on the user's hand detected in each of the plurality of frames, via the one or more hardware processors. estimating a plurality of network-implicit 3D articulation priors having a plurality of key points of sample user's hands from sample RGB images using a deep learning network, wherein the plurality of key-points includes 21 hand key-points, comprising four key points per finger and one key point close to the wrist of the user's hand, wherein the deep learning network is pre-trained in order to estimate a plurality of key points, the deep learning network being trained by using a large scale 3D hand pose dataset having a plurality of training sample RGB images based on synthetic hand models, receiving, via one or more hardware processors, a plurality of frames of a media stream of a scene captured from a first person view (FPV) of a user using at least one RGB sensor communicably coupled to a wearable Augmented reality (AR) device, the media stream comprising RGB image data associated with the plurality of frames of the scene, the scene comprising a dynamic hand gesture performed by the user; estimating, via the one or more hardware processors, a temporal information associated with the dynamic hand gesture from the RGB image data by using the deep learning network, the temporal information being the coordinate values of the 21 key-points on the user's hand detected in each of the plurality of frames, wherein the estimated temporal information is associated with hand poses of the user and wherein the 21 key points of user's hand are detected by the deep learning network based on the plurality of network-implicit 3D articulation priors; and classifying, by using a multi-layered Long Short Term memory (LSTM) classification network, the dynamic hand gesture into at least one predefined gesture class based on the coordinate values of the 21 key-points on the user's hand detected in each of the plurality of frames, via the one or more hardware processors. <EOS>