###### Abstract  
Image understanding plays a pivotal role in various computer vision tasks, such as extraction of essential features from images, object detection, and segmentation. At a higher level of granularity, both semantic and instance segmentation are necessary for fully grasping a scene. In recent times, the concept of panoptic segmentation has emerged as a field of study that unifies semantic and instance segmentation. This article sheds light on the pivotal role of panoptic segmentation as a visualization tool for understanding scene components, including object detection, categorization, and precise localization of scene elements. Advancements in achieving panoptic segmentation and suggested improvements to the predicted outputs through a top-down approach are discussed. Furthermore, datasets relevant to both scene recognition and panoptic segmentation are explored to facilitate a comparative analysis. Finally, the article outlines certain promising directions in image recognition and analysis by underlining the ongoing evolution in image understanding methodologies. convolutional neural networks; image segmentation; computer vision; instance segmentation; semantic segmentation; panoptic segmentation; scene recognition; artificial intelligence; machine learning; deep learning; pre-trained networks +
Footnote †: journal: _Algorithms_  
0000-0001-999-4000-0001-999-4000-0001-999-4000-0001-999-40001-9999-4000-0001-9999-40001-9999-40001-9999-40001-9999-40001-  
the case of U-Net [3], the approach involves augmenting the deconvolutional layers and efficiently mapping the data to higher resolutions. Conversely, the SegNet network [4] employs an encoder-decoder model to refine deconvolutional layers, by extracting indices from the max-pool layers. However, it is worth noting that many networks share architectures that are comparable on the encoder side but mainly differ in their decoder configurations. Most recently, a probabilistic graphical model known as the fully connected random field (CRF) has been adopted by Deep Lab [5] as a substitution for the deconvolutional layers. To address the issue of information loss, the literature has explored numerous semantic segmentation approaches. These methods concentrate on feature extraction, employing either multi-scale feature aggregation [6] or end-to-end structured prediction [7]. Instance segmentation on the other hand, segments objects and identifies the object boundaries irrespective of whether the objects belong to the same category. In situations where multiple objects fall in a similar category, instance segmentation promises to differentiate each object from the other by drawing object boundaries whereas semantic segmentation relies on segmenting the collective information from these objects. Two important strategies, namely, a segment-first and instance-first strategy, have been proposed [8]. The segment-first strategy segments each object by classification results whereas the instance-first strategy identifies the ROIs for every instance and performs classification and segmentation task in parallel to each ROI. By incorporating this idea, it is inferred that Mask-RCNN outperforms other models on the COCO based instance segmentation [9]. Authors in [10] hold the view that, although semantic and instance segmentation may seem similar at first glance, the metrics and datasets associated with these two visual recognition tasks exhibit substantial differences. These differences are akin to the difference between \"stuff\" and \"things\". To gain a better understanding between semantic and instance segmentation models, [PERSON] et al. [10] emphasized the need of distinguishing between \"stuff\" and \"things\", a crucial aspect in various visual recognition tasks, \"stuff\" referring to countable objects within an image (instance segmentation), including entities like people, animals, trucks, sky, road, and grass, while \"things\" (semantic segmentation), account for identification of regions with similar textures.