SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[95] Thibaut Durand, Nazanin Mehrasa, and Greg Mori. Learning a deep convnet for multi-label classification with partial labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 647-657, 2019.
[96] Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified framework for multi-label image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2285-2294, 2016.
[97] Pramuditha Perera, Ramesh Nallapati, and Bing Xiang. Ocgan: One-class novelty detection using gans with constrained latent representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2898-2906, 2019.
[98] Ryota Yoshihashi, Wen Shao, Rei Kawakami, Shaodi You, Makoto Iida, and Takeshi Naemura. Classification-reconstruction learning for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4016-4025, 2019.
[99] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. Mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
[100] Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in Neural Information Processing Systems, 32, 2019.
[101] Jorge Ramón Fonseca Cacho, Ben Cisneros, and Kazem Taghva. Building a wikipedia n-gram corpus. In Proceedings of SAI Intelligent Systems Conference, pages 277–294. Springer, 2021.
A Datasets Details
A.1 WebVision1k
It contains 2.4M web images collected from Google and Flickr, which share the same 1k category names with ImageNet1k [1]. For each example, we use all available description, title, and tag in its metadata for raw text preparation. Besides, we follow [22, 78] to use the subset of WebVision-Google500 for ablation studies in consideration of lower GPU resource and time consumption without losing generalization. It contains 0.48M images from Google with randomly chosen 500 categories. The testing set of ImageNet1k and its subset ImageNet500 are involved as well for evaluation.
A.2 NUS-WIDE (Web)
It contains 0.26M web images from Flickr with 5k unique user tags. Each example is manually annotated with multiple labels within 81 concepts that are filtered out of the 5k tags. It also provides weak labels (an official web version) by checking if each of the 81 category name appears in user tags of every example. Almost  $50\%$  of the web labels are wrong and  $50\%$  of the true labels are missing in tags [88]. Since tags contain phrases without delimiters, we split phrases based on unigram frequencies [101] for raw text preparation. We follow [78] to train models with weakly-labeled training set (a.k.a., NUS-WIDE Web) and validate them on the clean testing set.
B Mathematical Notations
In this section, we present the description of all math notations in the manuscript (see Table 5).
C Implementation and Training Details
C.1 General Settings
In consideration of performance and efficiency, we adopt ResNet-50 [89] and MiniLM-L6 [30] as image and text encoders by default. The training settings are listed in Table 6. In general, we refer to [28] for: batch size is 256; optimizer is SGD; learning rate linearly rises with 10 warm-up epochs and decays by cosine schedule. Specifically, although the benefit of an increased batch size (e.g., 1024) has been validated [78, 58], we follow the standard batch size setting of 256 on WebVision1k for two reasons: 1) comparability with most of the previous methods; 2) limited computing resources with 8 GPUs (a mini-batch size of 32 on each GPU for a batch size of 256 in total).
We empirically set  $\lambda^{prj} = 1$ ,  $\lambda^{pro} = 1$ ,  $\lambda^{ins} = 1$ ,  $\lambda^{bts} = 0.1$ ,  $m_p = 0.999$ ,  $d_p = 128$ ,  $\tau = 0.1$ ,  $\alpha = 0.5$ , and  $K = 50$  by default. Their optimal values require meticulous fine-tuning of each hyper-parameter, which is beyond consideration of the present study.
Data augmentation (random cropping, resacing, and horizontal flipping) is applied on the inputs to the query encoder while stronger augmentation, including color jittering and blurring [50]), is added on those to the key encoder.
Experiments are conducted on a CentOS 7 workstation with an Intel 8255C CPU, 377 GB Mem, and 8 NVIDIA V100 GPUs. The training of CAPro on WebVision1k, Google500, and NUS-WIDE respectively costs about ten days, three days, and one day under the environment mentioned above.
C.2 WebVision1k-only Implementation
We refer to [28] for  $Q = 8192$  and  $\gamma = 0.6$ . The learning rate is 0.1 and the number of epochs is 120. Given the complexity of graph construction, we use  $k = 5$  for both text denoising and matching.
C.3 NUS-WIDE (Web)-only Implementation
In view of the dataset scale, we set  $Q = 2048$  with a learning rate of  $2 \times 10^{-3}$  and a total of 100 epochs.  $k = 10$  is chosen since user tags are noisier and sparser in NUS-WIDE.
Table 5: List of symbols.
Symbol	Description
xi	a web image indexed with i
ti	textual metadata associated with xi
yi	web label associated with xi
yi*	ground-truth label associated with xi
N	the total number of images in a web dataset
C	the total number of categories
D	a web dataset
θe	parameters of image encoder
θc	parameters of classifier
F(θe; θc)	a deep model with image encoder and classifier
dv	dimension of the visual features
vi	visual features of xi extracted by image encoder, vi ∈ IRdv
dt	dimension of the textual embeddings
si	textual embeddings of ti extracted by text encoder, si ∈ IRdt
tc	category definition of the c-th class
sc	textual embeddings of tc extracted by text encoder, sc ∈ IRdt
pi	predictions on vi from classifier, pi ∈ IRC, pi(k) denotes its k-th element
dp	dimension of the visual embeddings
zi	low-dimensional embeddings of vi after projection, zi ∈ IRdp
tili	reconstructed visual features from zi,tili ∈ IRdv
qi	predictions on zi from auxiliary classifier, qi ∈ IRC
Q	the size of dictionary, namely the length of queue
k	number of nearest neighbors (NN) in k-NN and k-reciprocal-NN
V	vertices, nodes
ε	edges
G	graph, G = {V, ε}
A	adjacency matrix
N(xi,k)	k-NN sets of xi
R(xi,k)	k-reciprocal-NN sets of xi
d(vi,vj)	distance between vi and vj
d*(vi,vj)	refined distance between vi and vj
Vvi,vj	k-reciprocal feature encoding the distance between vi and vj
S	concatenated textual embeddings, S = (s1, s2, ..., sN), S ∈ IRN×dt
IN	identity matrix
A	adjacency matrix A with self-connection IN
Dii	degree matrix of A
S	denoised S after smoothing
si	denoised si after smoothing
K	top-K selected examples with visual-semantic alignment
σKc	the K-th smallest distance d*(si, sc) in the c-th class
DKc	top-K set of the c-th class
DK	sets of top-K examples from all classes
zc	unnormized visual prototype of the c-th class
zc	normalized visual prototype of the c-th class
τ	temperature coefficient
α	weight between self-prediction and prototype-instance similarity
oi	fused, comprehensive output for label correction, oi ∈ IRC
ri	prototype-instance similarity, ri ∈ IRC, ri(k) denotes its k-th element
γ	threshold for label correction
mp	momentum coefficient
bi	collective bootstrapping target on zi
wij	weight of contribution from zi' in the dictionary for bi
λbts	weight for collective bootstrapping loss
λprj	weight for projection and reconstruction losses
λpro	weight for prototypical contrastive loss
λins	weight for instance contrastive loss
Table 6: List of hyper-parameters for training settings.
Settings	WebVision1k/Google500	NUS-WIDE
Optimizer	SGD
Optimizer momentum	0.9
Optimizer weight decay	\( 1 \times 10^{-4} \)
Batch size	256
Step1 pre-training scheduler	Cosine decay with linear warm-up
Step1 pre-training warm-up epochs	5/10	10
Step1 pre-training learning rate	\( 1 \times 10^{-1} \)	\( 2 \times 10^{-3} \)
Step1 pre-training epochs with encoders frozen	0
Step1 pre-training epochs in total	120	100
Step2 training scheduler	Cosine decay with linear warm-up
Step2 training warm-up epochs	5/10	10
Step2 training learning rate	\( 1 \times 10^{-1} \)	\( 2 \times 10^{-3} \)
Step2 training epochs with encoders frozen	5/10	10
Step2 training epochs in total	60/120	100
Step3 fine-tuning scheduler	Cosine decay
Step3 fine-tuning warm-up epochs	0
Step3 fine-tuning learning rate	\( 1 \times 10^{-4} \)	\( 2 \times 10^{-5} \)
Step3 fine-tuning epochs with encoders frozen	15/20	20
Step3 fine-tuning epochs in total	15/20	20
Image encoder (by default)	ResNet-50 [89]
Text encoder (by default)	MiniLM-L6 [30]
\( λ^{prj} \)	1
\( λ^{pro} \)	1
\( λ^{ins} \)	1
\( λ^{bts} \)	0.1
\( m_p \)	0.999
\( d_p \)	128
τ	0.1
α	0.5
top-K	50
Q	8192	2048
γ	0.6	0.9
k-NN/k-reciprocal-NN	5	10
To support multi-label learning, it is necessary, but not yet enough, to simply replace the softmax-based cross-entropy losses with sigmoid-based binary cross-entropy losses. The premise of prototypical contrastive learning does not hold true anymore because one instance can be simultaneously engaged in formation of multiple clusters, which violates the exclusivity inherited in softmax activation. Our experiments demonstrate that, only by projecting  $\mathbf{v}_i$  into compact subspaces specific to each class, can we properly learn the decision boundary to continue prototypical learning. Technically, we set  $C$  additional fully-connected (FC) layers after the projector to respectively map  $\mathbf{z}_i$  into  $\tilde{\mathbf{z}}_{i,c} \in \mathbb{R}^{d_p}, c = 1,2,\dots,C$ . For the  $c$ -th class, both positive and negative prototypes  $\tilde{\mathbf{z}}^{c+}, \tilde{\mathbf{z}}^{c-}$  are shaped accordingly for the contrast against  $\tilde{\mathbf{z}}_{i,c}$ . Such operation can be viewed as magnifying class-indicative contents via recombination of the shared embedding  $\mathbf{z}_i$ . Another minor modification is required on noise removal, where the output  $\mathbf{o}_{i,c}$  is fused independently for the  $c$ -th class for binary separation. Considering the overwhelming negative instances, we set  $\gamma = 0.9$  to avoid deviation by majorities in label rectification. Discussion on the hyper-parameter  $\gamma$  can be found in Sec. D.2.
C.4 Training Steps
CAPro adopts a three-step training pipeline. It employs a pre-training step to learn common visual patterns from the original web dataset  $D$ . Then, the training starts with instance-prototype and instance-wise contrastive learning, collective bootstrapping, and on-line noise removal. Finally, given the trained model, off-line noise removal is performed on  $D$  for data cleaning and we fine-tune the classifier alone with the cleaned dataset.
Step1 pre-training We perform visual pre-training on ResNet-50 with cross-entropy and projection-reconstruction losses. At this time, we only use original web labels to train the model to learn common visual description, which lays foundation for the subsequent prototype initialization in step2. Note that the pre-trained model is also the vanilla method in our experiments.
Step2 training All components are initialized with the pre-trained parameters in step1. As shown in Table 6, we keep the encoders frozen with warm-up epochs. This helps stabilize training to avoid the prototypical embeddings, which are initialized at the beginning by averaging top-  $K$  semantically-correct examples, being perturbed drastically. In this step, we re-train the model with all losses. Apart from classification, we perform instance-wise and instance-prototype contrastive learning, collective bootstrapping, noise removal, and prototype update.
Step3 fine-tuning We follow MoPro [28] to perform noise removal on the training dataset  $D$ . The trained model in step2 is used to correct labels or discard OOD examples with our control flow. Then, we keep encoders frozen and fine-tune the classifier alone with the cleaned set for better performance. Note that such fine-tuned model is exactly the CAPro in experiments.
Algorithm 1: CAPro's training procedure.
Data: Web images and their associated texts and labels  $D = \{(\mathbf{x}_i,\mathbf{t}_i,y_i)\}_{i = 1}^N$    
Step1 pre-training   
for  $(\mathbf{x}_i,y_i)\in D$  do  $\mathcal{L}_i = \mathcal{L}_i^{\mathrm{cls}} + \mathcal{L}_i^{\mathrm{prj}};$  Update image encoder, classifier, projector, and reconstructor to minimize  $\mathcal{L}_i$    
end   
Step2 training   
for  $(\mathbf{x}_i,\mathbf{t}_i,y_i)\in D$  do Extract  $\mathbf{v}_i$  from  $\mathbf{x}_i$  via the image encoder; Extract  $\mathbf{s}_i$  from  $\mathbf{t}_i$  via the text encoder;   
end   
Build  $k$  -reciprocal-NN graph  $\mathcal{G} = \{\mathcal{V},\mathcal{E}\}$  with  $\{\mathbf{v}_i\}_{i = 1}^N$    
Enhance text embeddings from  $\mathbf{s}_i$  to  $\hat{\mathbf{s}}_i$  via graph convolution on  $\mathcal{G}$    
for  $c\in \{1,2,\dots,C\}$  do Extract  $\mathbf{s}^c$  from  $\mathfrak{t}^c$  via the text encoder; for  $i\in \{1,2,\dots,N|y_i = c\}$  do Match textual instances  $\mathbf{s}_i$  to prototypes  $\mathbf{s}^c$  to obtain visual anchors  $D_K^c$  end   
end   
Initialize visual prototypes with  $D_K$  .   
for  $(\mathbf{x}_i,y_i)\in D$  do  $\mathcal{L}_i = (1 - \lambda^{\mathrm{bts}})\mathcal{L}_i^{\mathrm{cls}} + \lambda^{\mathrm{bts}}\mathcal{L}_i^{\mathrm{bts}} + \lambda^{\mathrm{prj}}\mathcal{L}_i^{\mathrm{prj}} + \lambda^{\mathrm{pro}}\mathcal{L}_i^{\mathrm{pro}} + \lambda^{\mathrm{ins}}\mathcal{L}_i^{\mathrm{ins}};$  Update image encoder, classifier, and projector to minimize  $\mathcal{L}_i$  Refine  $y_{i}$  to  $\hat{y}_i$  to remove noise;   
end   
Step3 fine-tuning   
for  $(\mathbf{x}_i,\hat{y}_i)\in D$  do  $\mathcal{L}_i = \mathcal{L}_i^{\mathrm{cls}};$  Update classifier to minimize  $\mathcal{L}_i$    
end
We also prepare Algo. 1 to explicitly explain the entire training process.
Figure 5: Top-matched WebVision1k instances are chosen: (a) without text enhancement, (b) with text enhancement in VSGraph [78], and (c) with our text enhancement.
Airport (airdrome)
Bridge (span)
Car (auto)
(a)
Figure 6: Top-matched NUS-WIDE (Web) instances are chosen: (a) without text enhancement, (b) with text enhancement in VSGraph [78], and (c) with our text enhancement.
D Ablation Study
D.1 Effect of Text Enhancement
Figs. 5 and 6 present additional qualitative comparison for selecting instances with potentially-correct semantics. For WebVision, noiser categories are chosen to validate the effectiveness of text enhancement by smoothing and reranking. We can observe that due to the problem of polysemy, a majority of the retrieved images are irrelevant to the correct semantics and simple  $k$ -NN-based smoothing in [78] can hardly handle such situation. In contrast, our text enhancement helps pinpoint, not perfect but comparatively reliable, web instances that share similar semantics (e.g., metalwork in Nail). Besides, we also sample three categories from the noiser NUS-WIDE to double-check the effectiveness of text enhancement. For example, in the category of airport, direct matching of user tag embeddings to the textual prototype returns a few close-up images of warcrafts, which has nothing to do with Airport. On the contrary, our text enhancement helps to select the truly matched instances.
Table 7: Effect of  $\gamma$  on CAPro without collective bootstrapping.
γ	Reference Provider	Google500	ImageNet500	NUS-WIDE
Top1	Top5	Top1	Top5	C-F1	O-F1	mAP
0.6	×	72.0	88.0	66.9	85.4	8.3	9.1	6.9
0.8	×	71.2	87.7	65.9	84.8	-	-	-
0.9	×	-	-	-	-	39.2	44.4	46.8
D.2 Effect of  $\gamma$  on Noise Removal
Table 7 reports the influence of  $\gamma$  when collective bootstrapping is not applied. We follow MoPro [28] to validate two  $\gamma$  candidates:  $\gamma = 0.6$  and  $\gamma = 0.8$ . We find that  $\gamma = 0.6$  works the best on Google500. As  $\gamma$  increases, it becomes more difficult for the model to correct labels and thereafter label-flipping errors may still exist. As suggested by MoPro, the optimum  $\gamma$  is related to the percentage of noise in web datasets. For noiser datasets,  $\gamma$  should be decreased for the model to correct wrong labels at an early stage. Otherwise, overfitting might occur and weaken prototypical representation learning. The fine-tuning of  $\gamma$  requires elaborate experiments, which is beyond the scope of the present study.
For multi-label learning on NUS-WIDE, the optimum  $\gamma$  is not only related to the noise level but also to the ratio of the number of positive examples to that of negative examples. Since the negative instances in each class exceeds positive ones by more than one order of magnitude, decreasing  $\gamma$  will easily make the model to classify any instance as negative. Once the model overfits the overwhelming negative examples, valid positive supervision sources would only come from the top- $K$  matched examples with cross-modality alignment, which degrades generalizability greatly. In this case, we should keep a stricter threshold  $\gamma = 0.9$  to only allow confident label rectification.
Table 8: Effect of prototype update frequency on CAPro. By default, we update visual prototypes every epoch using high-quality examples in each mini-batch. For 0-epoch per update, we do not introduce additional high-quality web examples to polish prototypes, but only update them with the top- $K$  matched semantically-correct examples with their latest visual embeddings.
# Epochs per update	Google500	ImageNet500	NUS-WIDE
Top1	Top5	Top1	Top5	C-F1	O-F1	mAP
0	75.5	91.1	71.6	88.8	39.2	44.4	47.2
1 (by default)	76.0	91.3	72.0	89.2	39.3	45.4	48.0
5	75.9	91.2	71.8	89.2	39.6	45.0	47.6
10	76.0	91.2	71.7	89.1	39.3	45.8	48.2
D.3 Effect of Prototype Update Frequency
By default, we update prototypes by examples in a mini-batch for every epoch. We additionally perform ablation study on the update frequency where comparison is conducted between: 1) 0-epoch, 2) 1-epoch (by default), 3) 5-epoch, and 4) 10-epoch. For 0-epoch update, we do not update prototypes with embeddings from other high-quality web examples. Instead, prototypes are renewed
with the latest embeddings only from the top- $K$  matched examples to avoid being out-of-date. For 5-epoch and 10-epoch update, we polish prototypes every 5 and 10 epochs, respectively.
Table 8 reports the effect of prototype update frequency on CAPro. On the Google500 and NUS-WIDE datasets, if prototypes are only formed by limited clean examples, their generalization across domain becomes poor and thus causes performance drop of  $0.45\%$  (top1) and  $0.6\%$  on average, respectively. For Google500, reduced update frequency generally causes lower performance, meaning that the prototypes should keep refreshed for large-scale datasets. For NUS-WIDE, if the update frequency decreases a bit (5-epoch), the model improves on C-F1 but underperforms a little on O-F1 and mAP. With the 10-epoch frequency, we surprisingly find that results are improved on all evaluation metrics. One possible explanation is that the delayed prototype update can help stabilize training at an early stage, but the optimal frequency might be subject to dataset scale and noise level.
Table 9: Effect of noise removal policy on CAPro. We compare with MoPro to show the effectiveness of keeping labels of top-  $K$  matched semantically-correct examples unchanged.
Noise Removal policy	Google500	ImageNet500	NUS-WIDE
Top1	Top5	Top1	Top5	C-F1	O-F1	mAP
MoPro [28]	75.8	91.1	71.7	89.0	38.8	42.2	47.2
CAPro (ours)	76.0	91.3	72.0	89.2	39.3	45.4	48.0
D.4 Effect of Noise Removal Policy
Table 9 reports the effect of noise removal policy on CAPro. Our label adjustment policy is inspired from the Eq. (5) of MoPro [28] but differs in that we keep labels of the top-  $K$  matched examples selected in cross-modality alignment unchanged. Therefore, if we replace our noise removal policy with the MoPro one, we actually allow the deep model to get rid of the guidance from top-  $K$  examples. These selected top-  $K$  examples can be altered or even discarded by the MoPro-style policy.
It turns out that without such enforcement, the performance dropped under both single-label and multi-label scenarios. The possible reason behind is that, due to the overwhelming noise (e.g., the semantic-misalignment noise) in certain categories, the model itself cannot keep representation learning robust to noise even with a good start. The labels of top-  $K$  samples will be prone to the noisy majority, which invalidates prototypical learning. Besides, the superiority of CAPro over MoPro-style update also substantiates the purity and correctness of the selected examples.
E Analysis on Open-Set Recognition
We provide detailed analysis on CAPro for open-set recognition. Fig. 7 presents the per-class F1 (C-F1), per-class Precision (C-P), and per-class Recall (C-R). We compare five methods for ablation study including: 1) the vanilla method, 2) CAPro without text enhancement (TE) & collective bootstrapping (CB), 3) CAPro without CB, 4) CAPro, and 5) CAPro with mix-up (MU).
We vary the confidence threshold from 0 to 1 with an interval of 0.01 to comprehensively measure the performance of CAPro. For each example, if the highest model prediction confidence is below the threshold, the example will be classified as the open-set category. Otherwise, the example will be classified into one of the known categories. We train methods on the Google500 training set and validate them on WebVision1k and ImageNet1k testing sets. Examples from the remaining 500 novel categories should all fall into the open-set category.
It can be observed that compared with vanilla method, CAPro enjoys a much higher recall but a relatively lower precision. It means that our CAPro is more confident about its prediction and a threshold around 0.6 would end up with an optimal C-F1 over  $66\%$ . The precision-recall curve reflects that each key component does improve open-set recognition. Note that due to the limited sampling of confidence threshold, the precision-recall curve is not spanning across the entire axes. However, the tendency of curves confirms the effectiveness of our components.
(b)
Figure 7: Effect of threshold for open-set recognition on (a) WebVision1k and (b) ImageNet1k. TE, CB, and MU respectively refer to our text enhancement, collective bootstrapping, and mix-up. Examples with prediction confidence lower than the threshold will be classified as open-set category.
F Guidelines on Tuning of Hyper-Parameters
Loss weights of  $\lambda^{\mathrm{bts}}$ ,  $\lambda^{\mathrm{prj}}$ ,  $\lambda^{\mathrm{pro}}$ , and  $\lambda^{\mathrm{ins}}$  First, for the total objective, we follow MoPro [28] to use  $\lambda^{\mathrm{pro}} = 1$  and  $\lambda^{\mathrm{ins}} = 1$ . Out of simplicity, we also use  $\lambda^{\mathrm{prj}} = 1$  as default.
Second, we would like to explain the effect of  $\lambda^{\mathrm{pro}}$ ,  $\lambda^{\mathrm{ins}}$ , and  $\lambda^{\mathrm{prj}}$  on regularization. A larger  $\lambda^{\mathrm{pro}}$  may pull instances too close to their prototypes, which "shrinks" class clusters in the embedding space. A larger  $\lambda^{\mathrm{ins}}$  will enforce stronger visual discriminability between two instances. It may cause two examples from the same category to differ greatly and thereafter downgrades the visual prototype update and class cluster regularization. A larger  $\lambda^{\mathrm{prj}}$  improves the reconstruction quality of  $\tilde{\mathbf{v}}_i$ , which encourages  $\mathbf{z}_i$  to retain more information of  $\mathbf{v}_i$  in the embedding space. The projection-reconstruction loss is only involved in the pre-training stage (see Algo. 1), and therefore  $\lambda^{\mathrm{prj}}$  will not affect the prototypical and instance-wise contrastive learning in the following stage.
Third, for one's custom web datasets, we suggest that  $\lambda^{\mathrm{pro}}$ ,  $\lambda^{\mathrm{ins}}$ , and  $\lambda^{\mathrm{prj}}$  should be tuned according to the performance results under three settings: 1)  $\lambda^{\mathrm{pro}} = 0$  vs.  $\lambda^{\mathrm{pro}} = 1$ ; 2)  $\lambda^{\mathrm{ins}} = 0$  vs.  $\lambda^{\mathrm{ins}} = 1$ ; 3)  $\lambda^{\mathrm{prj}} = 0$  vs.  $\lambda^{\mathrm{prj}} = 1$ .