SlowGuess/NAACL_2022_OCR huggingface dataset:
To see the difficulty of negative samples constructed by various generation strategies, we plot the value distribution of samples. To evaluate the difficulty, we compute the similarity gap between the positive pair  $\mathrm{ITM}(I_i,T_i)$  and the negative one  $\mathrm{ITM}(I_i,T_t^-)$ . We plot the value of negative pair minus positive one with respect to training steps (X-axis). In general, higher value means higher difficulty. The result is shown in Figure 3 where the darker color means more samples. The overall values of TAGS w/ SG (Figure 3 (a)) are higher than TAGS w/ DG (Figure 3 (b)). This implies that the static generator fails to provide negative sentences close to the image for ITM during training while our generator with dynamic generating strategy is effective.
(a) Dynamic
(b) Static
Figure 3: Value  $\{\mathrm{ITM}(I_i, T_t^-) - \mathrm{ITM}(I_i, T_i)\}$  distribution of triples generated by dynamic and static generators respectively during the training. X-axis is training steps.
4.2 Quality Evaluation of Synthetic Sentences
We evaluate the quality of generated synthetic sentences in terms of automatic metrics and human evaluation.
Fluency We utilize the pre-trained language model GPT-2 (Radford et al., 2019) to compute the perplexity of synthetic negative sentences for the measurement of their fluency. We use positive sentences in the test set of Flickr30K as original ones and generate negative samples by TAGS and VSE-C. Furthermore, we look into sentences after correction. The overall results are shown in Table 3. Compared with sentences produced by VSE-C, our synthetic sentences have much smaller perplexity. After correction, the fluency of synthetic sentences can be improved.
Human Evaluation We perform a human evaluation to see whether all negative sentences generated are true negative. We randomly sample 200 sentences generated by TAGS and ask two annotators to determine whether the synthetic sentences are mismatched to the corresponding images. The result shows that  $96.5\%$  of synthetic sentences generated are true negative.
Positive	Synthetic	Corrected	VSE-C
Perplexity	51.13	87.63	70.87	292.76
4.3 Negative Sentences Discrimination
In this section, we explore to see if the generator can discriminate positive sentences from synthetic ones. We compare UNITER and TAGS. For a pair of sentences (one is positive and the other is a synthetic negative one), the generator should assign a higher score to the positive one. We report the accuracy of discrimination. We utilize two negative sentence generators TAGS and VSE-C (Shi et al., 2018). Two versions of TAGS with different seeds are used for cross-validation. Results are shown in Table 4. We have several findings as follows. (1) TAGS2 is trained with a different seed with TAGS1, but the performance of TAGS1 almost makes no difference in discriminating their generated sentences. (2) Although the synthetic sentences of VSE-C are constructed with human efforts, TAGS also outperforms UNITER by about  $9\%$ . (3) Three generators
produce negative sentences with different distributions, but TAGS performs better than UNITER consistently. These facts validate the robustness of TAGS.
Table 3: Perplexity of synthetic negative sentences.
Generator	Discriminator	Accuracy
TAGS1	TAGS1	98.7%
UNITER	2.3%
TAGS2	TAGS1	99.7%
UNITER	2.8%
VSE-C	TAGS1	96.3%
UNITER	87.5%
4.4 Effectiveness of Two Auxiliary Tasks
Table 4: Accuracy of TAGS1 and UNITER in discriminating the negative sentences constructed by TAGS1, TAGS2 and VSE-C (Shi et al., 2018).
Image	Type	Sentence	U	T
Positive
Synthetic
Corrected	A man wearing a helmet, floating in the water
A man carrying white helmet, swimming in the water
A man wearing a helmet, swimming in the water	92.35
93.17
-	99.90
98.92
-	-
Positive
Synthetic
Corrected	A young man about to throw a football
A man playing playing to catch a ball
A man player about to throw a ball	89.54
90.61
-	99.90
75.39
-	-
Figure 4: Examples of TAGS-DC. The second column is the sentence type including positive one, synthetic one and corrected one. The third column is the corresponding sentence of the second column. The fourth and fifth columns are the UNITER(U) and TAGS-DC(T) scores for the sentence in the third column, respectively. The word color in synthetic sentences from green to yellow means the increase of the word mismatching scores. Words with underline mean the regenerated words are different from the original ones.
We show the performance of our model in two auxiliary tasks, namely, word discrimination and correction in the testing set of Flickr30K. In word discrimination, we use a threshold of 0.5 to split the positive and negative ones in terms of probability. The accuracy of word discrimination is  $66.5\%$ . In word correction, the accuracy is  $87.3\%$ . With the probability, we can provide additional support information accompanied to the final decision of our model.
Two examples are presented in Figure 4. (1) TAGS-DC assigns lower scores for synthetic negative sentences than positive ones, but UNITER
fails. (2) Color of "carrying" and "playing playing" are yellow which means that our word discrimination successfully detects these mismatched words. Our model finds the local alignment in word-level and grammatical errors, then generates "wearing" and "man player" for correction. In the examples, word discrimination marks the mismatched components and word correction provides reasons for mismatching. (3) Our model fails to identify two mismatched words, "swimming", and "ball". Considering they are partially related to the image, our model is less effective in determining the relevance of these fuzzy words.
Image-Text Retrieval Most works in image-text retrieval focus on better feature extraction and cross-modal interaction. Nam et al. (2017) and Ji et al. (2019) represent the image by semantics gathered from block-based attention. A line of research (Lee et al., 2018; Li et al., 2019; Wang et al., 2020; Wei et al., 2020; Li et al., 2021; Chen et al., 2022; Zheng et al., 2021; Fan et al., 2019, 2021b) detects features by pre-trained Faster R-CNN (Ren et al., 2015). Some other methods also focus on enhancing cross-modality relationship modeling, such as the dual attention network (Nam et al., 2017), the stacked cross attention (Lee et al., 2018; Liu et al., 2019; Hu et al., 2019), the graph structure attention (Liu et al., 2020), and the multimodal transformer modeling (Wei et al., 2020; Fan et al., 2021a). UNITER (Chen et al., 2020b), Unicoder (Li et al., 2020) and ERNIE-ViL (Yu et al., 2020) follow BERT (Devlin et al., 2019) to pretrain the vision-language transformer model on the large-scale image-text datasets, and finetune in image-text retrieval.
Negative Samples in Contrastive Learning Selection strategies for negative samples have been widely studied in metric learning (Schroff et al., 2015; Oh Song et al., 2016; Harwood et al., 2017; Suh et al., 2019; Zhang et al., 2020; Chen et al., 2020a). Wu et al. (2017) employ distance weighted sampling to select more informative and stable examples. Ge (2018) present a novel hierarchical triplet loss capable of automatically collecting informative training samples. In image-text retrieval, early works (Kiros et al., 2014; Karpathy and Fei-Fei, 2015; Socher et al., 2014) utilize random negative samples for training. VSE++ (Faghri et al., 2018) incorporates difficult negative ones in the
multi-modal embedding learning. The method is widely applied in the following works (Lee et al., 2018; Wei et al., 2020), and achieves significant performance improvement. UNITER (Chen et al., 2020b) randomly samples a portion of texts ( $\sim$ 512) from the dataset and picks up the hardest ones. AOQ (Chen et al., 2020a) selects these hard-to-distinguish cases from the whole dataset through a pre-trained ITM model and assigns hierarchical and adaptive penalties for samples with different difficulties. UNITER+DG (Zhang et al., 2020) samples hard negative sentences according to the structure relevance based on denotation graph (Plummer et al., 2015). These methods are retrieval-based and inspire us to find more difficult negative sentences through generation. Chuang et al. (2020) propose a method for debiasing, i.e., correcting for the fact that some negative pairs may be false negatives. In our work, we mask keywords (objects, attributes, and relationships) in the positive sentence then refilling, and exclude these sentences of which each token is included in image annotated sentences. This method introduces new keywords and alleviates the generation of false negative samples. Kalantidis et al. (2020) consider applying mixup to produce hard negatives in latent space. In our work, we directly rewrite the positive sentences that is missing in the latent space based method, and this improves the robustness and faithfulness. The most similar work is VSE-C (Shi et al., 2018) that attacks the VSE++ (Faghri et al., 2018) through replacing the nouns, numerals, and relations according to language priors of human and the WordNet knowledge base. Compare with VSE-C (Shi et al., 2018), our method has three advantages. (1) Our model does not depend on rules. (2) Our model is more flexible and can generate negative sentences with any number, but this is intractable for VSE-C. (3) The generated sentences of our model are more fluent than these of VSE-C as the results in Table 4.
In this paper, we focus on the image-text retrieval task and find that retrieve-based negative sentence construction methods are limited by the dataset scale. To further improve the performance, we propose TAiloring neGative Sentences (TAGS). It utilizes masking and refilling to produce synthetic negative sentences as negative samples. We also set up the word discrimination and word correction
to introduce word-level supervision to better exploit the synthetic negative sentences. Our model shows competitive performance in MS-COCO and Flickr30k compared with current state-of-the-art models. We also demonstrate the behavior of our model is robust and faithful.
This work is partially supported by Natural Science Foundation of China (No.6217020551), Science and Technology Commission of Shanghai Municipality Grant (No.20dz1200600,21QA1400600,GWV-1.1,21511101000) and Zhejiang Lab (No.2019KD0AD01).
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. Spice: Semantic propositional image caption evaluation. In ECCV.
Tianlang Chen, Jiajun Deng, and Jiebo Luo. 2020a. Adaptive offline quintuplet loss for image-text matching. In European Conference on Computer Vision, pages 549-565. Springer.
Yangdong Chen, Zhaolong Zhang, Yanfei Wang, Yuejie Zhang, Rui Feng, Tao Zhang, and Weiguo Fan. 2022. Ae-net: Fine-grained sketch-based image retrieval via attention-enhanced network. Pattern Recognition, 122:108291.
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020b. Uniter: Universal image-text representation learning. In ECCV.
Ching-Yao Chuang, Joshua Robinson, Lin Yen-Chen, Antonio Torralba, and Stefanie Jegelka. 2020. Debiased contrastive learning. arXiv preprint arXiv:2007.00224.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Fartash Faghri, David J Fleet, Jamie Ryan Kiros, and Sanja Fidler. 2018. Vse++: Improving visual-semantic embeddings with hard negatives. In Proceedings of the British Machine Vision Conference (BMVC).
Zhihao Fan, Zhongyu Wei, Zejun Li, Siyuan Wang, Hai-jun Shan, Xuanjing Huang, and Jianqing Fan. 2021a.
Constructing phrase-level semantic labels to form multi-grained supervision for image-text retrieval. arXiv preprint arXiv:2109.05523.
Zhihao Fan, Zhongyu Wei, Siyuan Wang, and Xuan-Jing Huang. 2019. Bridging by word: Image grounded vocabulary construction for visual captioning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6514-6524.
Zhihao Fan, Zhongyu Wei, Siyuan Wang, Ruize Wang, Zejun Li, Hajjun Shan, and Xuanjing Huang. 2021b. Tcic: Theme concepts learning cross language and vision for image captioning. arXiv preprint arXiv:2106.10936.
Weifeng Ge. 2018. Deep metric learning with hierarchical triplet loss. In Proceedings of the European Conference on Computer Vision (ECCV), pages 269-285.
Ben Harwood, Vijay Kumar B G, Gustavo Carneiro, Ian Reid, and Tom Drummond. 2017. Smart mining for deep metric learning. In Proceedings of the IEEE International Conference on Computer Vision (ICCV).
Zhibin Hu, Yongsheng Luo, Jiong Lin, Yan Yan, and Jian Chen. 2019. Multi-level visual-semantic alignments with relation-wise dual attention network for image and text matching. In IJCAI, pages 789-795.
Tri Huynh, Simon Kornblith, Matthew R Walter, Michael Maire, and Maryam Khademi. 2020. Boosting contrastive self-supervised learning with false negative cancellation. arXiv preprint arXiv:2011.11765.
Zhong Ji, Haoran Wang, Jungong Han, and Yanwei Pang. 2019. Saliency-guided attention network for image-sentence matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5754-5763.
Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. 2020. Hard negative mixing for contrastive learning. arXiv preprint arXiv:2010.01028.
Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In CVPR.
Ryan Kiros, Ruslan Salakhutdinov, and Richard S Zemel. 2014. Unifying visual-semantic embeddings with multimodal neural language models. CoRR, abs/1411.2539.
Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. 2018. Stacked cross attention for image-text matching. In Proceedings of the European Conference on Computer Vision (ECCV), pages 201-216.
Gen Li, Nan Duan, Yuejian Fang, Ming Gong, and Daxin Jiang. 2020. Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence.
Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu. 2019. Visual semantic reasoning for image-text matching. In ICCV.
Zejun Li, Zhongyu Wei, Zhihao Fan, Hajunj Shan, and Xuanjing Huang. 2021. An unsupervised sampling approach for image-sentence matching using document-level structural information. arXiv preprint arXiv:2104.02605.
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dólar, and C Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In ECCV.
Chunxiao Liu, Zhendong Mao, An-An Liu, Tianzhu Zhang, Bin Wang, and Yongdong Zhang. 2019. Focus your attention: A bidirectional focal attention network for image-text matching. In Proceedings of the 27th ACM International Conference on Multimedia, pages 3-11.
Chunxiao Liu, Zhendong Mao, Tianzhu Zhang, Hongtao Xie, Bin Wang, and Yongdong Zhang. 2020. Graph structured network for image-text matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10921-10930.
Hyeonseob Nam, Jung-Woo Ha, and Jeonghee Kim. 2017. Dual attention networks for multimodal reasoning and matching. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 299-307.
Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. 2016. Deep metric learning via lifted structured feature embedding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4004-4012.
Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pages 2641-2649.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823.
Haoyue Shi, Jiayuan Mao, Tete Xiao, Yuning Jiang, and Jian Sun. 2018. Learning visually-grounded semantics from contrastive adversarial samples. arXiv preprint arXiv:1806.10348.
Richard Socher, Andrej Karpathy, Quoc V Le, Christopher D Manning, and Andrew Y Ng. 2014. Grounded compositional semantics for finding and describing images with sentences. Transactions of the Association for Computational Linguistics, 2:207-218.
Yumin Suh, Bohyung Han, Wonsik Kim, and Kyoung Mu Lee. 2019. Stochastic class-based hard example mining for deep metric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7251-7259.
Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, and Jingjing Liu. 2021. Lightningdot: Pre-training visual-semantic embeddings for real-time image-text retrieval. arXiv preprint arXiv:2103.08784.
Sijin Wang, Ruiping Wang, Ziwei Yao, Shiguang Shan, and Xilin Chen. 2020. Cross-modal scene graph matching for relationship-aware image-text retrieval. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1508-1517.
Xi Wei, Tianzhu Zhang, Yan Li, Yongdong Zhang, and Feng Wu. 2020. Multi-modality cross attention network for image and sentence matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10941-10950.
Chao-Yuan Wu, R Manmatha, Alexander J Smola, and Philipp Krahenbuhl. 2017. Sampling matters in deep embedding learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 2840-2848.
Hong Xuan, Abby Stylianou, Xiaotong Liu, and Robert Pless. 2020. Hard negative examples are hard, but useful. In European Conference on Computer Vision, pages 126-142. Springer.
Xu Yang, Kaihua Tang, Hanwang Zhang, and Jianfei Cai. 2019. Auto-encoding scene graphs for image captioning. In CVPR.
Fei Yu, Jiji Tang, Weichong Yin, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2020. Ernie-vil: Knowledge enhanced vision-language representations through scene graph. arXiv preprint arXiv:2006.16934.
Bowen Zhang, Hexiang Hu, Vihan Jain, Eugene Ie, and Fei Sha. 2020. Learning to represent image and text with denotation graph. arXiv preprint arXiv:2010.02949.
Yi Zheng, Yuejie Zhang, Rui Feng, Tao Zhang, and Weiguo Fan. 2021. Stacked multimodal attention network for context-aware video captioning. IEEE Transactions on Circuits and Systems for Video Technology, 32(1):31-42.
We have two settings, base and large. The base setting of model has 12-layers, 768 hidden size and 12 attention heads and the large one has 24-layers, 1024 hidden size and 16 attention heads.
We utilize grid search to determine the hyperparameters. In retrieval-based matching, we randomly samples 399 negative sentence (image) from the whole dataset for the query image (sentence), and pick out the top 31 ones from them according to the matching scores. In the masked language modeling, we utilize the scene graph parser in SPICE to extract the phrases of objects, relationships and attributes from the positive sentence, and take these phrases as a whole to sample and mask. The mask probability is 0.15. In the generation enhanced matching, the temperature  $\tau \in \{1.0, 1.5\}$ , and we set  $K = L = 20$  and  $|\mathbb{T}_i^-| = 31/23$  for the base and large settings.  $\lambda_{ITM}$ ,  $\lambda_{MLM}$ ,  $\lambda_{ISTM}$ ,  $\lambda_{WoD}$  and  $\lambda_{WoC}$  is sampled from  $\{1.0\}$ ,  $\{5e-2, 1e-1\}$ ,  $\{1e-4, 5e-4, 1e-3\}$ ,  $\{5e-4, 1e-3\}$  and  $\{5e-4, 1e-3\}$ , where we set  $\lambda_{WoD} = \lambda_{WoC}$ .
Our training is composed of two steps, (1) we train with ITM, MLM and ISTM with 5,000 steps as NSG; (2) we further train the model with the whole loss function as NSGDC with 1,500 steps. The learning rate  $lr$  is sampled from  $\{5\mathrm{e} - 5,4\mathrm{e} - 5,1\mathrm{e} - 5\}$ . We use a linear learning rate scheduler with  $10\%$  warmup proportion. The Adam with  $\beta_{1} = 0.9$  and  $\beta_{2} = 0.98$  is taken as the optimizer. The dropout is 0.1.
Our code is implemented with pytorch. For base setting in Flickr30K, we utilize 8 V100 for training and the computation time is about 8 hours.
A.2 Algorithm of TAGS-DC
Algorithm 1 Training step of TAGS-DC
Input: A positive image-text pair  $(I_i, T_i)$ .
Parameter: Backbone  $M_{\theta}$ , the head of masked language model  $H_{MLM}$ , image-text matching  $H_{ITM}$ , word discrimination  $H_{WoD}$  and word correction  $H_{WoC}$ .
1: # negative sentence generation.
2: Initializing  $\widehat{\mathbb{T}}_i^{-} := \{\}$ .
3: for  $k$  in  $I, \ldots, K$  do
4: Randomly masking  $T_{i}$  to get the masked one  $T_{i}^{(k)}$ .
5: Computing  $\mathcal{L}_{MLM}$  in Eq. (1) with  $M_{\theta}$  and  $H_{MLM}$ .
6: for  $l$  in  $I,\ldots ,L$  do
7: Refilling  $T_{i}^{(k)}$  to generate a synthetic sentence  $T_{j}^{(k,l)}$  following Eq. (2).
8: if  $T_{i}^{(k,l)}$  satisfies criteria C1 then
9: Adding  $T_{i}^{(k,l)}$  to  $\widehat{\mathbb{T}}_i^-$  and computing its matching score with  $I_{i}$ .
10: end if
11: end for
12: end for
13: #image text matching.
14: Sampling negative image  $I_{i}^{-}$  and negative sentence  $T_{i}^{-}$  to compute  $\mathcal{L}_{IRTM}$  in Eq. (5) with  $M_{\theta}$  and  $H_{ITM}$ .
15: Picking out top- $m$  synthetic sentences from  $\widehat{\mathbb{T}}_i^-$  by the matching scores to constitute  $\mathbb{T}_i^-$ .
16: Utilizing  $\mathbb{T}_i^-$  and  $I_{i}$  to compute  $\mathcal{L}_{ISTM}$  in Eq. (7) with  $M_{\theta}$  and  $H_{ITM}$ .
17: # word discrimination and word correction.
18: for  $T_{t}^{-}$  in  $\mathbb{T}_i^-$  do
19: Utilizing  $T_{t}^{-}$  and  $I_{i}$  to compute  $\mathcal{L}_{WoD}$  in Eq. (8) with  $M_{\theta}$  and  $H_{WoD}$ .
20: Utilizing  $T_{t}^{-}$  and  $I_{i}$  to compute  $\mathcal{L}_{WoC}$  in Eq. (9) with  $M_{\theta}$  and  $H_{WoC}$ .
21: end for
Dataset	Model	lr	α	τ	|T_i^-|	λITM	λMLM	λISTM	λWoD	λWoC
Flickr30k	NSG(B)	5e-5	0.2	1.5	31	1.0	1e-1	1e-3	-	-
NSGDC(B)	1e-5	0.2	1.5	31	1.0	1e-1	1e-3	1e-3	1e-3
NSG(L)	4e-5	0.2	1.5	23	1.0	1e-1	5e-4	-	-
NSGDC(L)	1e-5	0.2	1.5	23	1.0	1e-1	5e-4	5e-4	5e-4
MS-COCO	NSG(B)	5e-5	0.2	1.5	31	1.0	5e-2	1e-4	-	-
NSGDC(B)	1e-5	0.2	1.5	31	1.0	5e-2	1e-4	5e-4	5e-4
NSG(L)	4e-5	0.2	1.5	23	1.0	5e-2	5e-4	-	-
NSGDC(L)	1e-5	0.2	1.5	23	1.0	5e-2	5e-4	5e-4	5e-4
Table 5: Hyper-parameters