See More for Scene: Pairwise Consistency Learning for Scene Classification
Authors: Gongwei Chen
Xinhang Song
Bohan Wang
Shuqiang Jiang
Venue: NeurIPS 2021 Poster (2021)
Keywords: pairwise learning
scene classification
convolution neural networks

Abstract: Scene classification is a valuable classification subtask and has its own characteristics which still needs more in-depth studies. Basically, scene characteristics are distributed over the whole image, which cause the need of “seeing” comprehensive and informative regions. Previous works mainly focus on region discovery and aggregation, while rarely involves the inherent properties of CNN along with its potential ability to satisfy the requirements of scene classification. In this paper, we propose to understand scene images and the scene classification CNN models in terms of the focus area. From this new perspective, we find that large focus area is preferred in scene classification CNN models as a consequence of learning scene characteristics. Meanwhile, the analysis about existing training schemes helps us to understand the effects of focus area, and also raises the question about optimal training method for scene classification. Pursuing the better usage of scene characteristics, we propose a new learning scheme with a tailored loss in the goal of activating larger focus area on scene images. Since the supervision of the target regions to be enlarged is usually lacked, our alternative learning scheme is to erase already activated area, and allow the CNN models to activate more area during training. The proposed scheme is implemented by keeping the pairwise consistency between the output of  the erased image and its original one. In particular, a tailored loss is proposed to keep such pairwise consistency by leveraging category-relevance information. Experiments on Places365 show the significant improvements of our method with various CNNs. Our method shows an inferior result on the object-centric dataset, ImageNet, which experimentally indicates that it captures the unique characteristics of scenes.

See More for Scene: Pairwise Consistency Learning for Scene Classification
Gongwei Chen Xinhang Song! Bohan Wang" 12, and Shuqiang Jiang 2,3 Institute of Computing Technology. Chinese Academy of Sciences University of Chinese Academy of Sciences, Beijing
IIntttttte of Intelligent Computing Technology, Suzhou, CAS {gongwei, chen, xinhang. song, bohan. wang) bohan.wang)00i. iiticcia. ac cn
Abstract
Scene classification is valuable classification subtask and has its own characteris- tics which still needs more in-depth studies. Basically, scene characteristics are distributed over the whole image, which cause the need of "seeing" comprehensive and informative regions. Previous works mainly focus on region discovery and aggregation, while rarely involves the inherent properties of CNN along with its potential ability to satisfy the requirements of scene classification. In this paper, we propose to understand scene images and the scene classification CNN models terms of the focus area. From this new perspective, we find that large focus area is preferred in scene classification CNN models scene characteristics. Mcanwhile, the analysis about existing training schemes helps us understand the effects of focus area, and also raises the question about optimal training method for scene classification. Pursuing the better usage scene characteristics,
consequence of learning
propose new learning scheme with tailored loss in the goal of activating larger focus area on scene images. Since the supervision of the target regions to be enlarged is usually lacked, our alternative learning scheme is to erase already activated area, and allow the CNN models to activate more area during training. The proposed scheme is implemented by keeping the pairwise consistency between the output of the erased image and its original one. In partic- ular, tailored loss is proposed to keep such pairwise consistency by leveraging category-relevance relevance information. Experiments on Places365 show the significant improvements of our method with various CNNs. Our method shows an inferior result on the object-centric dataset, ImageNet, which experimentally indicates that it captures the unique characteristics of scenes.
Introduction
Image classification is one of the fundamental tasks of computer vision, and attracts lot of attention as popular task for facilitating the improvement of convolution neural network (CNN) [16] 23 26 14]. As valuable classification subtask, scene classification has its own characteristics which still needs further study. comparison with objects, scenes are more complex and have many differences. One of main differences is that the characteristics of scenes are basically distributed over the whole image, while those objects are confined within a clear boundary. This phenomenon inspires core idea of scene classification, "seeing" comprehensive and informative regions in the image.
Most scene classification methods aim to extract unspecific 99] 25 or specific regions (2 33
then aggregate them via statistical models 9 6 29 or relation modeling approaches 25 3 The main challenge is that the extraction and aggregation are partially independent from the main CNN backbone. It induces some issues as follows. 1) some sort of incompatibility caused by the separated
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

ways to extract and represent regions. 2) inevitable computational consumption from extra region extraction and aggregation processes. is less studied to investigate the inherent properties of CNN and make them adaptive to the needs of scene classification in a more organic and effective way.
In this paper, we explore new way to understand scene images and the CNN models for scene classification in terms of the focus area. It is defined as the region consisting of pixels with large aggregated activation values in the feature maps. From this perspective, we obtain some observations that reveal the differences in the characteristics of CNNs between scene and object classification, which also correlates well with the properties of these two kinds of images, i.e., scenes are more complex with richer objects. One of the interesting findings is that some properties of training scheme could drive the development of large focus area of the corresponding CNN model, which raises a question about how to optimize the training strategy by considering the scene characteristies.
In order take advantages of the scene characteristics, we propose a new learning scheme with a tailored loss for scene classification. Our goal is to inspire the CNN models to focus on more regions in the feature maps. However, the supervision of potentially useful regions difficult to be obtained. In alternative, we attempt to erase the already activated regions (from the images) and require the consistent outputs, thus, the CNN models themselves could expand the focus area with further optimizing. This is also motivated by our preliminary observations that a scene image can retain its semantic meaning even when some regions are masked. Specifically, the modified image is generated by partially erasing the input images with the guidance of model preference. The tailored loss is implemented as pairwise consistency of predictions, which can leverage the category-relevane information.
The principle behind proposed method is the adversarial learning mechanism under the setting of pairwise consistency. Compared to the original image allows the unconfined exploration, the modified image is used for discovering additional parts which are previously ignored the original branch. The consistency constraint with the shared parameters unifies these two modes of region explorations, leveraging the underlying opposing operations to perform a more comprehensive and robust information extraction process. In addition, instance- pairwise consistency shares the excellent quality of knowledge distillation, and yields the unchanged models with superior performances.
Experiments on one of the largest scene datasets, Places365 38 demonstrate the effectiveness of the proposed method with various CNNs by showing the significant performance improvements. Some statistical analyses about the focus area indicate that our proposed method can enhance the capability of exploring larger focus area in CNNs and fit the characteristics of scene images. We also evaluate our method on ImageNet (15), and the inferior result can be the evidence reveal the inner working mechanism that is specifically designed for scene images.
2 Related work
Scene classification. In the era of CNN, many works 9 5
18
31 explore various region
extraction and encoding methods based on CNN representations to improve scene classification. They generally need lots of extra overhead compared to single CNN classifier and are not easy to apply on large scale datasets. A few researches focus on specific deep models designed for scene classification, including Dictionary Learning CNN 199 spatial unstructured layer [10], and contextual relation learning (40). However, we aim to investigate the inherent properties of scene CNN model, and propose an advanced training method to enhance it without any change of architecture.
Comparisons of scene and object networks. A growing number of works have been proposed to understand the discrepancy between these two kinds of networks, especially in terms of the learned representations. In [39]. the mean image method is used to visualize the units of CNN layers, and visually exhibits the relevance between the unit structures and the differences of the training data in qualitative way. Subsequently, a series of works tried to align the units of the CNN with some semantic concepts by measuring the empirical receptive fields [36] or using the feature activation as the segmentation results [I] and found that the networks trained to recognize scenes generally have more interpretable units related to certain concepts, especially objects. In addition, Herranz et al. 13 investigated the representations of CNNs trained on ImageNet and Places365 by comparing the object scale ranges these two datasets, and suggested the transfer performances could be improved when choosing the suitable image sizes. In contrast, analyze the feature maps by estimating the

---

ResNer18
Comptox
Inageffi
0.35
3.50
0.45
0:30
0.55
0.60
RedNet50
Complex
Sinple
ImagaNet
0.35
0.40
0.45
0.50
0.55
0.60
Figure I: The distribution of the coverage ratio of focus area across various datasets (Places365 and ImageNet) and training schemes (Simple and Complex). The bottom and top of the box show the 25th and 75th percentiles. The center line and triangle in the box denote the median and mean of the data.
aggregated activation values and providing the new perspective of the focus area. Moreover, the training schemes are also introduced and compared in terms of the focus area for better understanding of the difference the learned CNN models.
Erasing methods in CNN. Erasing regions or pixels in images or feature maps is an effective method weakly supervised object localization [24 28 32 segmentation (17), due the observation that the models often only concentrate on small and most discriminative regions of object when supervised by only image-level labels. Our idea is inspired by this but study further the properties of CNN from the perspective of the focus area, which changes previous empirical conclusion based on some examples to robust and reliable statistical analysis. In addition, we explore the erasing method with a tailored loss and extending them into classification task with superior performances, which enriches the understanding of the intrinsic correlations between the erasing method and image classification.
Comparisons of scene and object networks via the focus area
large number of previous image classification works propose various technical changes of CNN, and evaluate them ImageNet [15), which is an object-centric dataset. Despite the proposed large-scale scene-centric datasets, Places205 (39 and Places365 38), there is still lack researches on them, due to the underlying assumption that effective models on object classification could be generalized to scene classification. However, there are always some obvious differences between object and scene images. In this case, scene classification should be investigated individually compared to object classification.
Here, we provide new perspective to analyze the differences between scene and object networks by estimating the focus area of model on the input images. When the CNN models are trained, is possible to find out the regions, which the model concentrates on and extracts information from, to make final decision. Various techniques have been developed to achieve this goal by using occluded images 39, gradient information 22). architecture tweaks (2). The success of these attempts motivates us to analyze the distributions focus area models trained on scene and object datasets, and figure out the possible similarities and differences between them.

---

Table 1: The classification performance (Top-1 accuracy %) of ImageNet and Places365 with two training schemes, simple and complex ways.
ResNet18
ResNet50
Simple
Complex
Simple
Complex
ImageNet Places365
68.80
69.93
74.72 55.53
75.78
54.36
54.30
55.75
Based on the feature maps from last convolution layer, adopt simple adaptive thresholding method to discover the focus area of model on an input image. The channel-wise average pooling are performed on the feature maps to generate heatmap. Then the mean value of this heatmap is used as the threshold to segment it into a binary map. Finally, the focus area is composed of the pixels with positive value in the binary map. The underlying assumption is that the feature maps can serve an indicator to point out the regions which models desired [30] (27) and the pixels with high activation values in the feature maps have more valuable information
The two variables, considered in our analyses about focus area, are dataset and training scheme. In terms of dataset, ImageNet and Places365 are used as object-centric and scene-centric datasets, respectively. Additionally, we introduce two training schemes, including simple and complex ways. The main difference between them is the crop percent in the input processing operations. The former fixes the percent to 76% (224x224) / 256 x 256), while the latter chooses a random percent in range (8%. 100%) and is widely used in modern CNN training process. The detailed description of these training schemes could be found in appendix. Given a focus area, we compute the coverage ratio of it to the whole image an analytical metric. As the focus area on a single image may be too specific and not informative, we gather the validation datasets of ImageNet and Places365 to form an evaluation set for analyzing the distribution of the coverage ratio.
We present the classification performance and the visualization of the distribution of the coverage ratio across datasets and training schemes. Referring to the box plots in Figure it is clearly shown that the model learned on Places365 tend to focus more parts of the input image than ImageNet when comparing the median and mean of the data distribution, and the result holds for both training schemes. is likely that the model trained on Places365 learns the mechanism of comprehensive exploration and activation as necessary condition for recognizing a scene.
Recalling the definition of the focus area, the pixels all have the aggregated activation values than the mean value, which indicates that they either have large activation in a few channels or be frequently activated in many channels. Some evidences unit) in feature maps could be regarded as a part full of detector for semantic concept, and the concept regions emerge as the pixels with large activation values. In this way, the focus area roughly represents the group of detected semantic concepts in each image, suggesting that the size of the focus area can be accepted as a quantitative index of the regions with semantic concepts. By this correlation, the preference for large focus seems to be natural consequence of the scene attributes (containing more semantic concepts).
have shown that each channel (or
Another intriguing finding is the effect of training scheme. In contrast with complex training, simple way presents the capability of enhancing the region exploration of model and yielding large focus area on more examples, as shown in Figure T We also show the classification performances of two training schemes on ImageNet and Places365 in Table [ It concludes that simple training is a competitive choice for Places365, while it works worse on ImageNet. We infer that complex training adopts the cropped patches with various seales and yields more diverse training examples, which is better for object classification. However, the similar performances of them Places365 indicate that the ability of simple training to explore regions is useful for improving the performances, and could bridge the gap caused by the stronger data augmentation. This performance difference also raises question about the optimal training method considering scene characteristics, and motivates us to design& new learning framework the effective response.
4 Advanced training via pairwise consistency
Some earlier works 28
32 4 have effectively demonstrated that erasing certain regions in the image
or feature maps can force CNN to cover the full extent of object in the context of object classification

---

Input Image
PC
Giobal Poolling
Pairwise Consistency
112x112
56656
28s28
CNN
CAM Module
Modified Image
Category-level
Instance-lever
1ll
1
s
Thrnsholding
Minimize Devergence
Wixe
Figure 2: The overview of our proposed method.
along with the classification performance degradation. We conclude that erasing regions can useful way to help the CNN models focus more information of the image, which is reflected by more activated positions in the feature maps.
Based on this, propose new learning scheme and design tailored loss for scene classification. Our design also correlates well with an observation about scene images, which that if some regions of an image are hiding, we still have high probability to classify this image as the same category or even the same image. In this way, we decide to make the outputs of an input image and its modified version consistent. The main components of this learning process are the modifying way and the consistency function.
Modifying image. Here, we will discuss which regions need to be erased. Normally, It is intuitive to randomly choose regions, which is independent the model and may not fit well. What is expected the kind of regions that could reflect the characteristics of images models and are relevant the demands of classification.
that the convolution feature maps can indicate the model preferences in terms of spatial positions. simple way 330 4 to leverage this property is to average the feature maps over channel dimensions, which will present the model-relevant regions as kind learned knowledge. Furthermore, class activation map (CAM) (37 is proposed direct the attention to class-specific regions, which reveals the contents that provide evidences for making the corresponding prediction. In our design, class-specific regions (produced by CAM) can present more targeted information, and perform better in deriving region exploration desired.
Fortunately, it is widely recognized 30 27
The generation of CAM depends a specific module, termed CAM module, that sequentially contains a convolution layer (1 x in our setting), global average pooling layer, and fully connected layer (as classifier). Then, the output M of CAM computed follows.
M ==J(AA = Σ Ae
where A E RCxHxW the output from the convolution layer, and W is the weights in the classifier generate the prediction of class Note, the normalization is performed on the output map as 31 M--ii((A)
M max(M))min(MA) Most modern CNN architectures can meet the demands CAM generation. So is quite straightforward to use the original linear classifier of CNN to produce CAM.
However, we attach CAM module to the input of last down-sampling layer (normally, spatial dimension 14 x 14) for two reasons. One that larger resolution feature maps may contain more and better detail information. The other is that decoupling the classifier and the generation of CAM would reduce the performance loss caused by erasing more relevant regions derived from the original classifier. In addition, we use the prediction (category with the maximum output from the original classifier) as class I, not the ground truth, because the focus point the model's characteristics, not the properties of the input images.
5

---

Guided by the output M, we can use a threshold y to convert it to a binary mask Mg. which indicates the regions that need to be erased. Then, modified image I* is yielded by multiplying the input image 7 and the binary mask MB. The formulation of this process is shown as follows,
= MB @ I, MB = T(M, 7),
(2)
where O means element-wise multiplication, T denotes the thresholding function.
Pairwise consistency. Given the imput image I and its modified version 1*, we propose a tailored loss function to measure the similarity of their outputs so to achieve pairwise consistency. Specifically, two consistency forms can be considered, category-level and instance-level. In category-leve consistency, assume that the pair of images belongs to the same category. In instance-level consistency, a stricter constraint is created by enforcing the same prediction of these two images.
In practice, we keep the main component of CNN shared, and only create specific linear classifier for two images, separately.
inference, the classifier of the modified image discarded and only the original image is used as input to make prediction. For category-level consistency, two cross-entropy loss with the same ground truth are used. For instance-level consistency, we use Kullback- divergence as the metric to measure similarity between the output predictions, and also keep supervising the main branch with the ground truth. Our final loss combines these two consistency forms, and is formulated as
L = main + Leide + almodd + Blas
K
Ib log(pk)
(nlog((r)
km
kml
K
9l log(gx) ) + Σ qk -x log( 9k
+ 0
(3)
Pk
where p. q denote the predictions of the original and modified images respectively, g is the ground truth (represented as one-hot vector). The item side indicates the classification loss when applying CAM module and yielding the prediction r. We empirically set and B 2 in all experiments.
Discussion. The erasing idea is the fundamental element of driving the region exploration of the CNN model. It could be thought as a paradigm that can spawn various specific technical approaches. When applying it on scene classification, core point is the balance between the loss of important information and the enhancement of exploring regions. Compared with previous alternatively erasing our pairwise learning method has the advantage of handling these two opposing situations, due to the available complete features in the original image and the drive capability of the erased image. Instance-level pairwise consistency maximizes the benefits by using the category correlation, which sufficiently helps the model expand the focus area and further meets the scene demands. Our method only influences the training stage, and shows no change in inference. Although the proposed method needs double memory usage and computational time during training, we believe it is tolerable as an one-off process.
way 24 28
Experiments
In this section, we empirically evaluate the effectiveness of the proposed method and conduct some ablation studies for the deeper analyses.
Datasets. Places365 [38 is one of largest scene-centric datasets with over 10 million labeled images of scenes, which consists of 365 scene categories and two versions of training sets, Places365- challenge and -aaaess.S-ttandardd standard. We choose Places365-standee experiments, which provides 1.8 million images. The validation set has 36,500 images, where 100 images per category. We report Top-1 1 and Top-5 accuracy (single crop) on the validation set. Meanwhile, for fast evaluation, we also randomly choose 1000 images per category from Places365- standard set, and construct a new training set, termed Places365-small, We also reported some experimental results on object dataset, ImageNet [15) which contains 1000 object categories with 1.3 million training images and 50,000 validation images.
as the training set in following

---

Table 2: Top-1 and Top-5 accuracy (%) on Places365-small and Places365 dataset. "RF" is short for receptive field, which computed according to the CNN architectures. "Param." denotes the number of parameters.
Baseline
Our Method
Dataset
Model
Param,
RF
Top-1 1
Top-5
Top-1
Top-5
ShuffleNeV2
2.85M
527
47.67
78.68
48.91
79.76
8
11.36M
435
48.21
79.22
49.70
80.62
Places365 -small
ResNet34
21.47M
899
48.52
79.58
50.72
81.31
ResNet50
24.26M
427
49.66
80.67
50.92
81.70
DenseNet121
7.33M
2071
49.48
80.87
51.55
82.20
ShuffleNetV2
2.85M
527
54.01
84.05
54.95
84.56
ResNet18
11.36M
435
54.36
84.50
55.12
84.94
Places365
ResNet34
21.47M
899
54.84
84.72
56.23
85.79
ResNet50
24.26M
427
55.53
85,85
56.61
86.15
DenseNet121
7.33M
2071
55.40
85.50
56.84
86.44
Implementation details. We evaluate the proposed method several CNN architectures (imple- mented based on torchvision model:") including ResNet [T]]. ShuffleNetV2 21]. and DenseNet [14]. We use simple data augmentation with random cropping (from 256 256 to 224 224) and random horizontal flipping during training. In inference, we apply center-cropping in resized 256 x 256 images. All models trained from scratch by synchronous SGD with momentum 0.9 and a mini-bated size of 256, starting from learning rate 0.1 and decreasing it by cosine annealing strategy Titan RTX devices. The weight decay is 2e- for ResNet and DenseNet on Places365, le for ShuffleNetV2 on Places365, and 5e- for all models on Places365-small. We conduct experiments on Places365 with 40 epochs and Places365 small with 30 epochs. We simply evaluate the seper-parameter in range {0.5, 0.7, 0.9}. and find y 0.7 is better choice. Although, we believe more detailed evaluation of y may lead to better performances.
20 All experiments are conducted with the PyTorch framework
5.1 Main results
Comparison with the baseline. The main results are shown in Table 2 We compare the baseline and our method with five CNNs (ShuffleNett2, ResNet18 / 34 50, and DenseNet121). It can be observed that the proposed method significantly improves the performances. For example, when applied on ResNet34, the proposed method achieves the largest improvements of 2.20%, 1.39% on Places365-smi small and Places365 datasets, respectively. Note that these improvements are very significant, because there are less performance differences (e.g. less than 1.5% for ResNet18 and ResNet50) between various models with the standard training scheme.
Interestingly, we also find the positive correlation between the receptive field and the improvement of the proposed method. For example, ResNer34 and DenseNet121 have larger receptive field sizes (899 and 2071) along with bigger relative improvements (4.53% and 4. 4.18), compared to ResNet50 with smaller receptive field size (427) and relative improvement (2.54%). It suggests that the proposed method benefits much from the large receptive field, as it facilitates the ability of our method to explore more possible useful regions. Another interesting finding is that the performance ranking of ResNet50 and 1enseNet121 1 is different between baseline and our method. Considering the large difference in the receptive field size, we infer that the proposed method is able leverage the potential value of big receptive field, which may be limited in the standard training scheme.
Comparison with other erasing methods. We compare the proposed method with other related erasing methods, HaS (24). ACoL (32). and ADL [4) All three methods are proposed for weakly supervised object localization task, which concentrate on the characteristics of object images, and suffer degraded classification performances (excluding HaS). While ACoL and ADL achieve the comparable or worse classification performances in WSOL, they clearly outperform the baselines in
'hapss//gitubbbco/yyrrrrbbbiiiii

---

Table 3: Comparison with other erasing methods,
Table 4: The position of CAM module.
HaS
24
ACoL 33
and ADL [4].
Methods HaS
ADL
ACoL
Ours
Model
ResNet18
ResNet34
DenseNet
ResNet18
49.25
48.70
49.03
49.70
stage4 stage3
49.62:10.2
49.77::0.08 50.72::0.05
51.29::0.08 51.55:0..17
ResNet34
49.91
48.68
49.55
50.72
49.70:10.10
Table Comparison with object classification. All experiments are based on ResNet18. and trained with the complex training settings. Top-1 accuracy % (single crop) is reported.
Dataset
Places365-small
Places365
ImageNet
Baseline
47.80
54.30
69.93
Ours
48.68
54.78
69.57
the context of scene classification, as shown in Table Naturally, as data augmentation method, HaS still show superior performance when applied on scene images.
The superior results of HaS verify the effectiveness of erasing idea. Compared it, ADL and ACoL apply the erasing operations on the middle layers, which results in the worse performances. However, all these methods are inferior our method. This can be attributed the tailored pairwise consistency loss along with the effective region generation method.
Comparison with object classification. We also evaluate the proposed method ImageNet [15), the most popular object dataset. All comparison experiments in Table use the bascline training procedure in 12 with total batch size of 256, and 90 epochs, also named complex training scheme in section 3 It can be observed that our method has negative effect on object classification and show inferior performance compared the baseline. Note that the complex data augmentation strategy adopt many cropped patches with various scales, which yields more training examples, but also noise especially scene images. In this way, many small scale patches are hard to be recognized scenes, leading to the inappropriate situations under the setting of pairwise consistency. Despite this bad impact, the proposed method still obtain better performances. This comparison clearly indicates that our method is specifically designed for scene images and leverage their unique attributes.
5.2 Ablation studies
The position of CAM module. We evaluate two positions for inserting the CAM module, stage3 (convolution layers operated on x feature maps) and stage4 (convolution layers operated on 7 feature maps). Although we assume that the erased image will have the same category distribution as the original input, it may be hard to hold when the erased regions cover lots discriminative parts. As result, insert the CAM module stage3, which is little far away from the final classifier and produces the less discriminative regions. As shown in Table 4 it can be seen that this strategy works well on the models with large receptive field, like ResNet34 and DenseNet121, and retain the similar performances on the models with small receptive field, such as ResNet18. Limited by the computation resource, only report the error bar in Table 4 with repeated runs. The results show that the random deviations are relatively small and do not interfere with our conclusions.
Table 6: Consistent or not? "AM" means Attention Mining loss (17), which aims to minimize the predic- tion value the modified image at the ground truth.
Table 7: The effect of two kinds of pairwise consistency and side loss from CAM module. "CPC", "IPC" are short for the category-level and instance-level pairwise consistency, respectively.
ResNet
Side
CPC
IPC
Loss
AM
CPC
18
34
50
ResNet18
48.85
49.12
48.75
48.76
49.52
ResNet34
48.89
49.92
49.12
49.92
50.43
49.65
50.36
50.85
/
49.70
50.72
50.92
Baselines
48.21
48,52
49.66
8

---

640
040
030
ShuffleNetV2
ResNet18
ResNet34
ResNet50
DenseNet121
Figure 3: Comparison with the baseline. The focus area is computed with the convolution maps from last convolution layer. "Green", "Red" and "Blue" boxes correspond to the baseline and the proposed method with CAM module inserted into stage3 and stage4 separately.
Pairwise consistency is the key factor of the propose method, and motivated by the specific properties of scenes. Firstly, we want to discuss the value of the consistency idea. Li et al. [17] proposed different loss (attention mining loss) to minimize the prediction value of the modified image the ground truth. Table shows the results from the attention mining loss and the category-leve pairwise consistency loss. We loss, especially on ResNet34. infers that the consistency target provides a stronger driving force for the model, and is a better way to explore more region information for scenes.
that proposed consistency loss outperforms the attention mining
Beyond the category-level pairwise consistency (CPC), we also introduce the instance-level pairwise consistency (IPC)
powerful tool. As shown in Table 7 It be observed that IPC significantly outperforms CPC on various CNNs. is not surprising because IPC provides a clearer and more precise guidance by investigating the releeanceeee information from the prediction distribution. Finally, we combine these two losses to produce a slightly better and robust model.
5.3 Analyses of the focus area
Figure shows the distribution of coverage ratio of the focus area from our proposed method with CAM module in stage3 (red box) and stage4 (blue box), and the baseline (green box). The evaluation results are based on Places365 validation dataset and the feature maps from last convolution layer of CNNs trained on Places365- small dataset. It is clearly shown that the proposed method could change the distribution coverage ratio of the focus area, and force the model to have large focus area on more images, especially when CAM module is inserted into stage4. In terms of the mean and median, the proposed method present significantly larger values than baseline, except ResNet50 with CAM stage3 that shows a small difference. It can be attributed to the complex influence factors under the training process including the focus area. In other words, the proposed method may need to find balance of the enhancement of region exploration, the limitation of network architectures, and even the effects of various losses.
Dassline
5000
CPC
Figure 4: Comparisons of dif- ferent loss items. The results are evaluated with ResNet34 backbone.
In addition, we also present a detailed comparison of different items in our final loss (described in equation 33 by investigating their ef- fects on the distribution of the focus area in Figure 4 Benefiting from the stronger consistency constraint, instance-level pairwise consistency loss indicates a better driving force of region ex- ploration, compared to cetegory-level pairwise consistency loss. All loss items show the superior results of the focus area to the bascline, which demonstrate the working mechanism of erasing idea

---

and our pairwise consistency learning method. Unfortunately, there does not seem to be linear association between the differences in the mean or median of the distribution of different methods and the corresponding classification performances.
6 Conclusion
In this work, we propose to investigate the CNN classification model in terms of focus area. Two variables, image datasets and training schemes, is considered when measuring the focus area, and help to analyze the difference between various classification models. An interesting finding that the focus area in the effect of training schemes on the focus area is also significant and inspired us to think the optimal training methods on scene images. Based this, we propose new learning scheme and a tailored loss to force CNN to activate more positions in the feature maps for improving scene classification. Extensive experiments on large scale scene dataset, Places365, demonstrate the effectiveness of the proposed method. Mcanwhile, the worse performance on object dataset, ImageNet, indicates that our method is specifically designed for scenes and captures their unique attributes. Some comparison analyses also verify the ability of our method to enlarge the focus area.
networks generally has bigger size than that in object networks. Meanwhile,
Acknowledgements
This work was supported in part by Beijing Natural Science Foundation under Grant L182054 and Z190020, in part by the National Natural Science Foundation of China under Grant 62032022, 61902378 and U1936203, in part by the Lenovo Outstanding Young Scientists Program, in part by the National Postdoctoral Program for Innovative Talents under Grant BX201700255.
References
[1] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network Dissection: Quantifying Interpretability Deep Visual Representations. In CVPR, pages 3319-3327. IEEE, jul 2017.
3
[2] Wieland Brendel and Matthias Bethge. Approximating CNNs with Bag-of-local-Feature models works surprisingly well on ImageNet. In ICLR, 2019.
[3] Gongwei Chen, Xinhang Song, Haitao Zeng, and Shuqiang Jiang. Scene Recognition with Prototype-Agnostic Scene Layout. IEEE Transactions on Image Processing, 29:5877-5888, 2020.
[4] Junsuk Choe and Hyunjung Shim. Attention-based dropout layer for weakly supervised object localization. In CVPR, pages 2214-2223 2019. 2 5.1
[5] Mandar Dixit, Si Chen, Dashan Gao, Nikhil Rasiwasia, and Nuno Vasconcelos. Scene classifi- cation with semantic Fisher vectors. In CVPR, pages 2974-2983. IEEE, jun 2015. 2
[6] Mandar Dixit, Yunsheng Li, and Nuno Vasconcelos. Semantic Fisher Scores for Task Transfer: Using Objects to Classify Scenes. IEEE Transactions Pattern Analysis and Machine Intelligence, 11 2019.
[7] Mandar D. Dixit and Nuno Vasconcelos. Object based Scene Representations using Fisher Scores of Local Subspace Projections. In NeurlPS, pages 2811-2891 1-119, 2016. 2
[8] Ruth Fong and Andrea Vedaldi. Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks. In CVPR, 2018.
[9] Yunchao Gong, Liwei Wang, Ruiqi Guo, and Svetlana Lazebnik. Multi-scale Orderless Pooling of Deep Convolutional Activation Features. In ECCV, pages 1-17, 2014. 1
[10] Munawar Hayat, Salman H. Khan, Mohammed Bennamoun, and Senjian An. A Spatial Layout and Scale Invariant Feature Representation for Indoor Scene Classification. IEEE Transactions on Image Processing, 25(10):4829-44. 2016. 2
10

---

[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In CVPR, pages 770-778. IEEE, jun 2016.
[12] Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. Bag of Tricks for Image Classification with Convolutional Neural Networks. In CVPR, pages
2019.
[13] Luis Herranz, Shuqiang Jiang, and Xiangyang Li. Scene Recognition with CNNs: Objects, Scales and Dataset Bias. In CVPR, pages 571-579. IEEE, jun 2016. 2
[14] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely Con- nected Convolutional Networks. In CVPR, pages 2261-2269. IEEE, jul 2017. D
[15] Jia Deng, Wei Dong, R. Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.
T 5.1
[16] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks. In NeurlPS, pages 1-9, 2012. !
[17] Kunpeng Li, Ziyan Wu, Kuan-Chuan Peng, Jan Ernst, and Yun Fu. Tell Me Where to Look: Guided Attention Inference Network. In CVPR, 2018. 2 5.2
[18] Yao Li, Lingqiao Liu, Chunhua Shen, and Anton van den Hengel. Mining Mid-level Visual Patterns with Deep CNN Activations. International Journal of Computer Vision, 121(3)3344 364, 2017. 2
[19] Yang Liu, Qingchao Chen, Wei Chen, and Ian Wassell. Dictionary Learning Inspired Deep Network for Scene Recognition. In AAAI, pages 7178-7185, 2018. 2
[20] Iyya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In ICLR, pages 1-16, 2017. 5
[21] Ningning Ma. Xiangyu Zhang, Hai- Tao Zheng, and Jian Sun. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. In ECCV, pages 122-138, 2018.
[22] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual Explanations from Deep Networks via Gradient- Based Localization. In ICCV, volume 128, pages 336-359, 2017. 3
[23] Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR, pages 1-14, 2015.
[24]
Krishna Kumar Singh and Yong Jae Lee. Lide-and-Seekk Forcing Network to be Meticulous for Weakly-Supervie Object and Action Localization. In ICCV, pages 3544-3553, 2017. 2
5.1
[25] Xinhang Song, Shuqiang Jiang, and Luis Herranz. Multi-Scale Multi-Feature Context Modeling for Scene Recognition in the Semantic Manifold. IEEE Transactions Image Processing, (6(6):2721-2732 jun 2017. [
[26] Christian Szegedy, Wei Liu, Yangging Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR, pages 1-9, 2015. [
[27] Xiu-Shen Wei, Jian-Hao Luo, Jianxin Wu, and Zhi-Hua Zhou. Selective Convolutional Descrip- tor Aggregation for Fine- Grained Image Retrieval. IEEE Transactions on Image Processing, 26(6):2868 2282888888881,, jun 2017.
[28] Yunchao Wei, Jiashi Feng, Xiaodan Liang, and Ming-ming Cheng. Object Region Mining with Adversarial Erasing: A Simple Classification to Object Region Mining with Adversarial. In CVPR, pages 1568-1576, 2017.
[29] Ruobing Wu, Baoyuan Wang, and Wenping Wang. Harvesting Discriminative Meta Objects with Deep CNN Features for Scene Classification. In ICCV, pages 1287-1295, 2015. |

---

[30] Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance convolutional neural networks via attention transfer. ICLR, pages 1-13, 2017.
[31] Haitao Zeng, Xinhang Song, Gongwei Chen, and Shuqiang Jiang. Learning Scene Attribute for Scene Recognition. IEEE Transactions on Multimedia, pages 1-1, 2019. 2
[32] Xiaolin Zhang, Yunchao Wei, Jiashi Feng. Yi Yang, and Thomas Huang. Adversarial Comple- mentary Learning for Weakly Supervised Object Localization. CVPR, pages 1325-1334, 2018.
3
[33] Zhengyu Zhao and Martha Larson. From Volcano to Toyshop: Adaptive Discriminative Region Discovery for Scene Recognition. In ACM MM, 2018. [
[34] Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo. Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition, In ICCV, pages 5219-5227. IEEE, oct
2017.
[35] Bolei Zhou, David Bau, Aude Oliva, and Antonio Torralba. Interpreting Deep Visual Rep- resentations via Network Dissection. IEEE Transactions Puttern Analysis and Machine Intelligence, 11(0):2111-2145, sep 2019.
[36] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene CNNs. ICLR, 2015,
[37] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning Deep Features for Diseriminative Localization. In CVPR, pages 2921-2929, 2016. 4
[38] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 Million Image Database for Scene Recognition. JEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6):1452-1 1464, 2018.
[39] Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. Learning Deep Features for Scene Recognition using Places Database. In NIPS, pages 487-495, 2014. 2
3
[40] Zhen Zuo, Bing Shuai, Gang Wang, Xiao Liu, Xingxing Wang. Bing Wang, and Yushi Chen. Learning Contextual Dependence With Convolutional Hicrarchical Recurrent Neural Networks. IEEE Transactions Image Processing, 25(7):2983-2996, jul 2016.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [No] The data is publicly available, and code easy to implement by referring to above description and open resources.
12

---

(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section [
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] We have enough resource complete error bars for all experiments. But we did some test and found the random error can be neglected in comparisons, shown in 5.2
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section S
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [No] It is not stated in the corresponding
website and paper.
(c) Did you include any new assets either in the supplemental material or URL? [No] The assets are publicly available.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No] The related information can found the original paper that produced the dataset.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "wvEafa_vKX9", "reviewer": "Reviewer_DYcV", "summary": "To explore the potential recognition ability of CNN for scenes, this paper presents a scene recognition framework based on the analysis of effective receptive field (ERF). This paper first analyzes the correlation between ERF and CNN models of different tasks. Then the authors propose a framework to erase the useful regions while emphasizing other regions and expanding receptive field during training. Experiments are conducted on Places365 and ImageNet. Results show that the proposed method is more effective for scenes than objects.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Image classification is a classical and fundamental task in computer vision. Many excellent models have been proposed for object classification (on ImageNet), but the typical researches of scene classification are relatively few. Therefore, it’s interesting to design typical models to exploit the characteristic of scene classification rather than following the same architecture of object recognition.\n\nOne interesting insight provided by this paper is that a scene image could be correctly classified even when some regions are covered. And based on this observation, the authors propose a framework to erase the “easier” regions so more other regions could be emphasized during training. Also, the receptive field could be expanded.\n\nSome concerns:\n-ImageNet dataset is not introduced. \n-As the reviewer claimed in Line 61, “while is not applicable to object classification,” is this an advantage of the proposed method or a disadvantage? And what’s the reason for this.\n-In Table 1, which factor is more important to the performance, the number of parameters, or RF size? Also, it seems the gains on Top-1 are larger than Top-5. What’s the reason?\n-There are many related results in different tables. Therefore, the differences between them also should be clarified. For instance, ResNet in Table 1 is 54.36, in Table 4 is 54.12. What are the differences between the two settings? \n\nMinor issues: \n- Line 30, “as follows.”-> “as follows:”\n- It’s better to avoid repeated sentences, such as Line 50 and 177.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have discussed the limitations in section 4.2, such as no change in inference and high cost in training.", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical issue.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "SjwgdWP2TfE", "reviewer": "Reviewer_PqQU", "summary": "In this work, the authors enlarge effective receptive fields (ERF) of CNNs for scene image classification. To this goal, the authors propose a new learning scheme and a tailored loss. The proposed learning target is to make the outputs of an input image and its modified version to be consistent. The tailored loss is implemented as pairwise consistency of prediction to leverage category-relevance information. Experiments on Places365 partly verify the effectivenss of the proposed methdods.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In this work, the authors should address the following concerns.\n\n1.Contributions\n\nIn fact, the conceptions of ERF and pairwise learning are not new in current CV field, especically in image classification. Section 3 only list the existing works on ERF and most of the theories are from existing works. The authors only use Histogram to compute the distribution information. In  my view, this is not a contribution, but only a computing method. Besides, the proposed pairwise learning is more like a data augmentation method with erasing modules. In fact, the whole training method is very similar to existing weakly-supervised classification or segmentation methods. In the network structure, the CAM and region erasing methods have already been used in previous works. Thus, the techinical contributions are very limited, considering most of the used modules are from existing works. If there are any insigtful contributions, the authors can highlight them and compare with previous works.\n\n2. Motivations\n\nAs far as we known, large ERF is helpful for many CV tasks. The authors aims to enlarge the ERF. However, the basic idea is just using existing methods. What are your essential insights? In my view, there are many methods to modify the image to enlarge ERF, for example recurrent cropping, multi-scale, pyramid structure,etc. How to ensure the effectiveness of the proposed histogram computation and erasing design? Besdies, the basic module is still CAM without any changes.\n\n3.Experiments\n\nFirst, through the whole experiments, I find that there are no comparisons with recent scene classification methods. In fact, the authors only compare with basic methods with different backbone networks as shown in Tab.1. The performance is not comparable to current SOTA results. Second, there are no full analysis on the key modules. For example, how about using ResNet-101/152 backbone? Why very different ERFs (435 vs 899) show very similar results in Tab.2? If the claims are correct, is it consistent to the experiments? Besides, there are no significant differences in performance of Tab. 6, considering the CPC and IPC. Third, there are no visual comparisions to verify the proposed modules, especially CPC, IPC, Erasing effects, etc. In overall, experiments are not very convincing for the claims.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have not adequately addressed the limitations and potential negative societal impact of their work.  I suggest the authors add more computation comparisons such as FLOPS/SPEED. Besides, the proposed method should show some failture examples in classification and analysed why the model can predict the true labels under larger ERF.", "review_text": "", "extra_scores": {"ethical_concerns": "no", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "EYiQ6V-P3tg", "reviewer": "Reviewer_BQ7K", "summary": "This paper investigated the difference between object classification and scene classification in terms of the effective receptive field. This paper further proposed an erasing scheme guided by the class activation map (CAM) to help enlarge the effective receptive field of classification models during training. Both the original image and the erased image are used to produce the traditional cross-entropy training signal. Also, a KL-divergence term is used to ensure the pairwise consistency between the original image and the erased image. Experiments show minor improvements over baseline methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nLimited novelty. This paper is the first to investigate the difference between object classification and scene classification in terms of the effective receptive field. But how effective receptive field affects the performance of scene classification is not well studied, neither theoretically nor empirically. The proposed erasing-based training scheme has been used in other lines of research such as multimodal community. Hence the novelty of the proposed approach is limited.\n\nQuality: \nSome of the main claims of this paper are simply based on intuition. More theoretical or empirical analysis is required.\n1)\tAbout the histograms of impact maps in Figure 2. First to speak, it is confusing to use different numbers of bins. Why is there a difference between 90 bins and 70 bins? Also, some of the charts don’t agree with the claim that ERF of models trained on scene images is larger, e.g. Row 1 Column 3. Lastly, the AUCs of Figure 2 are not consistent, why so?\n2)\tThis paper is based on an intuition that larger ERF will improve the performance of scene classification. But this correlation is well studied. For example, how and to what extent will larger ERF improve performance?\n3)\tThe correlation between the performance gain of the erasing-based training scheme and the enlarged ERF is not studied. This paper merely intuitively provided this chain of causation: erasing scheme during training -> more activated ReLU units -> larger ERF. More comprehensive analysis is required to draw a connection between ERF and scene classification.\n\n\nClarity:\nThis paper is generally easy to follow. However, Section 4.2 about the CAM-guided erasing scheme is quite confusing. How to multiply classification weights with CAM?\n\nSignificance:\nModerately significant. The proposed method outperforms various baselines. But the improvements are minor.\n\nMain Weakness:\nAs mentioned earlier, the erasing-based training scheme is of limited novelty. More analysis is required to draw connections between larger ERF and better scene classification performance.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a systematic study on the potential recognition ability of CNN for scenes in terms of the effective receptive field (ERF).  It is interesting to see that the proposed model is based on exploiting the characteristic of scene classification rather than following the same architecture of object recognition. Moreover, all the reviewers agree that the proposed method provides a key contribution to the fundamental CV tasks.  


Decision: Accept (Poster)