## 6 Results and Analysis
### Qualitative Analysis  
Figures 14-20 present several examples of the proposed approach. In each figure, we can see an image in the top left corner: color-coded bounding boxes correspond to the detection by the YOLO-V3 model; the regions are cropped and passed through Show, Attend, and Tell to obtain the image captions; the captions are written below the image with relevant color codes as per the boxes. Below that, we can see the dependency parse trees of the image captions and regional graph generated by our approach. In the top right corner, we can see the combined graph generated by merging redundant nodes: each regional graph is bounded by a rectangle with the same color code as before. Even in the bounding box, dotted polygons mark which component is obtained from which region. Moreover, merged nodes are written in bold font with thick borders; these nodes are shared by two or more region graphs. Figure 14 demonstrates a simple image with a single object of interest. The caption corresponding to the full image is enough; however, the regional captions add more attributes about the shirt and tie. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No. of Words** & **No. of Captions** & **No. of Entities** & **No. of Actions** & **No. of Attributes** & **No. of Relations** \\ \hline
7 & 3 & \(3.33\pm 0.58\) & \(2.00\pm 0.00\) & \(2.00\pm 0.00\) & \(4.00\pm 0.00\) \\
8 & 339 & \(3.91\pm 0.59\) & \(2.02\pm 1.04\) & \(2. +++ ==WARNING: Truncated because of repetitions==
07\pm 0.96\) & \(4.03\pm 1.10\) \\  
++  
Figure 15 demonstrates three very similar captions, thus creating a tightly combined graph with many merged nodes. Redundant actions, such as \"sit\", are also merged, as they appear in the same phrase in both captions. Figure 16 shows an example of the box overlap principle. There are two captions with the \"plant\" word. However, they are not merged in the combined graph as their corresponding bounding boxes do not overlap, thus signifying that the plants are separate entities. Moreover, nodes such as the \"window\" correspond to a specific detection of the \"window\", thus demonstrating that the graph generation module is not limited by the object detection limits of YOLO_V3. Figure 14: Holistic scene graph from a simple image with a single object of interest. Figure 13: Visualization of the dependency between the number of words and the number of graph elements. The dotted line demonstrates the linear regression of the scatter plot of a specific graph element. Figure 15: Holistic scene graphs showing three very similar captions, hence demonstrating several merged nodes and several WordNet-based “is” edges. Figure 17 shows an erroneous node detection (\"tree\") due to inaccurate captioning. Additionally, the spatial relation \"on_top_of\" is not appropriate for this case. In Figure 18, also, smaller regions incorrectly recognize the wooden carriage as a wooden bench or wooden fence in different region captions. There is another error as the person is treated as the agent for the action of drawing the carriage and riding the horses, possibly due to statistical bias. The correct caption would have been \"person ride carriage\" and \"horse draw carriage\". Figure 19 demonstrates how regional captions can mistakenly classify the gender of the person beyond the frame. Moreover, the \"other\" node mistakenly detected as the phrase \"each other\" is not aptly represented. Here, the action should have been associated with both the \"man\" and the \"woman\" node. Figure 16: Holistic scene graphs showing distinct instances of plants in disjoint bounding boxes. Figure 17: Holistic scene graph showing a captioning error leading to an erroneous node “tree”. Figure 18: Holistic scene graphs demonstrating statistical bias forcing erroneous edges. Finally, Figure 20 shows how partially visible objects can be misclassified. Here, the \"armchair\" is recognized as a \"bed\". The proposed algorithm does not utilize any graph-based ground truths to generate the holistic scene graphs.