A processor implemented method for an on-device classification of fingertip motion patterns into gestures in real-time, the method comprising: receiving in real-time, in a Cascaded Deep Learning Model CDLM executed via the one or more hardware processors of a mobile communication device, a plurality of Red, Green and Blue RGB input images from a live feed or a video from an image capturing device, wherein the CDLM comprises a MobileNetV2 as an object detector, a Fingertip regressor and a Bidirectional Long Short Term Memory Bi-LSTM Network, for accurate gesture recognition, and wherein the CDLM is ported on the mobile communication device and removes hand gesture recognition framework dependence on a remote server; detecting in real-time, using the MobileNetV2 comprised in the CDLM executed via the one or more hardware processors on the mobile communication device, a plurality of hand candidate bounding boxes from the received plurality of RGB input images, wherein each of the plurality of hand candidate bounding boxes is specific to a corresponding RGB image from the received plurality of RGB input images, wherein each of the plurality of hand candidate bounding boxes comprises a hand candidate, and wherein each of the plurality of hand candidate bounding boxes comprising the hand candidate depicts a pointing gesture pose; downscaling, the hand candidate from each of the plurality of hand candidate bounding boxes to obtain a set of down-scaled hand candidates; detecting in real-time, using the Fingertip regressor comprised in the CDLM executed via the one or more hardware processors on the mobile communication device, a spatial location of a fingertip from each down-scaled hand candidate from the set of down-scaled hand candidates, wherein a first coordinate and a second coordinate are derived from the spatial location of the fingertip, wherein the spatial location of the fingertip from the set of down-scaled hand candidates represents a fingertip motion pattern, wherein the Fingertip regressor implemented based on a Convolutional Neural Network CNN architecture to localise the first coordinate and the second coordinate of the fingertip, wherein the CNN consists of two convolutional blocks and three fully connected layers to regress over the fingertip spatial location, wherein each of the two convolutional blocks have three convolutional layers followed by a max-pooling layer; and classifying in real-time, via the Bi-LSTM Network comprised in the CDLM executed via the one or more hardware processors on the mobile communication device, using the first coordinate and the second coordinate from the spatial location of the fingertip, the fingertip motion pattern into the one or more hand gestures. receiving in real-time, in a Cascaded Deep Learning Model CDLM executed via the one or more hardware processors of a mobile communication device, a plurality of Red, Green and Blue RGB input images from a live feed or a video from an image capturing device, wherein the CDLM comprises a MobileNetV2 as an object detector, a Fingertip regressor and a Bidirectional Long Short Term Memory Bi-LSTM Network, for accurate gesture recognition, and wherein the CDLM is ported on the mobile communication device and removes hand gesture recognition framework dependence on a remote server; detecting in real-time, using the MobileNetV2 comprised in the CDLM executed via the one or more hardware processors on the mobile communication device, a plurality of hand candidate bounding boxes from the received plurality of RGB input images, wherein each of the plurality of hand candidate bounding boxes is specific to a corresponding RGB image from the received plurality of RGB input images, wherein each of the plurality of hand candidate bounding boxes comprises a hand candidate, and wherein each of the plurality of hand candidate bounding boxes comprising the hand candidate depicts a pointing gesture pose; downscaling, the hand candidate from each of the plurality of hand candidate bounding boxes to obtain a set of down-scaled hand candidates; detecting in real-time, using the Fingertip regressor comprised in the CDLM executed via the one or more hardware processors on the mobile communication device, a spatial location of a fingertip from each down-scaled hand candidate from the set of down-scaled hand candidates, wherein a first coordinate and a second coordinate are derived from the spatial location of the fingertip, wherein the spatial location of the fingertip from the set of down-scaled hand candidates represents a fingertip motion pattern, wherein the Fingertip regressor implemented based on a Convolutional Neural Network CNN architecture to localise the first coordinate and the second coordinate of the fingertip, wherein the CNN consists of two convolutional blocks and three fully connected layers to regress over the fingertip spatial location, wherein each of the two convolutional blocks have three convolutional layers followed by a max-pooling layer; and classifying in real-time, via the Bi-LSTM Network comprised in the CDLM executed via the one or more hardware processors on the mobile communication device, using the first coordinate and the second coordinate from the spatial location of the fingertip, the fingertip motion pattern into the one or more hand gestures.