A processor implemented method comprising the steps of: receiving, by one or more processors, an image from a temporal set of images of at least a portion of one or more crops being monitored for a pre-defined event, wherein the pre-defined event relates to manifestation of diseases, manifestation of pests, count of one or more parts of the crop and growth stages; detecting one or more Regions Of Interest (ROIs), by the one or more processors, as localized bounding boxes in the received image using the pre-defined event and one or more context sensitive pre-trained models of the pre-defined event for the one or more crops being monitored, wherein the one or more ROIs correspond to a detected event; performing unsupervised segmentation of the one or more ROIs, by the one or more processors, using a Convolutional Neural Network to obtain a segmentation map having predicted labels derived from spatial continuity of pixels comprised within each of the ROIs; post processing the segmentation map using Conditional Random Fields (CRFs) to refine the boundaries of the one or more segments comprised in the segmentation map; obtaining superpixels from the received image, by the one or more processors, using a superpixel generating algorithm; and iteratively auto-correcting, by the one or more processors, the obtained segmentation map by: updating labels of the pixels in the one or more ROIs using boundaries of the obtained superpixels, by using unsupervised segmentation and computing features on one or more regions from the received images to obtain disjoint partitions of the received image; computing a cross entropy loss between the predicted labels by a last convolution layer of the CNN and the updated labels; and back propagating the computed cross entropy loss to the CNN, until a change in the cross entropy loss from a previous iteration is less than a predetermined threshold. receiving, by one or more processors, an image from a temporal set of images of at least a portion of one or more crops being monitored for a pre-defined event, wherein the pre-defined event relates to manifestation of diseases, manifestation of pests, count of one or more parts of the crop and growth stages; detecting one or more Regions Of Interest (ROIs), by the one or more processors, as localized bounding boxes in the received image using the pre-defined event and one or more context sensitive pre-trained models of the pre-defined event for the one or more crops being monitored, wherein the one or more ROIs correspond to a detected event; performing unsupervised segmentation of the one or more ROIs, by the one or more processors, using a Convolutional Neural Network to obtain a segmentation map having predicted labels derived from spatial continuity of pixels comprised within each of the ROIs; post processing the segmentation map using Conditional Random Fields (CRFs) to refine the boundaries of the one or more segments comprised in the segmentation map; obtaining superpixels from the received image, by the one or more processors, using a superpixel generating algorithm; and iteratively auto-correcting, by the one or more processors, the obtained segmentation map by: updating labels of the pixels in the one or more ROIs using boundaries of the obtained superpixels, by using unsupervised segmentation and computing features on one or more regions from the received images to obtain disjoint partitions of the received image; computing a cross entropy loss between the predicted labels by a last convolution layer of the CNN and the updated labels; and back propagating the computed cross entropy loss to the CNN, updating labels of the pixels in the one or more ROIs using boundaries of the obtained superpixels, by using unsupervised segmentation and computing features on one or more regions from the received images to obtain disjoint partitions of the received image; computing a cross entropy loss between the predicted labels by a last convolution layer of the CNN and the updated labels; and back propagating the computed cross entropy loss to the CNN, until a change in the cross entropy loss from a previous iteration is less than a predetermined threshold.