Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models
Authors: Adyasha Maharana
Amita Kamath
Christopher Clark
Mohit Bansal
Aniruddha Kembhavi
Venue: Accepted by TMLR ()

Abstract: As general purpose vision models get increasingly effective at a wide set of tasks, it is imperative that they be consistent across the tasks they support. Inconsistent AI models are considered brittle and untrustworthy by human users and are more challenging to incorporate into larger systems that take dependencies on their outputs. Measuring consistency between very heterogeneous tasks that might include outputs in different modalities is challenging since it is difficult to determine if the predictions are consistent with one another. As a solution, we introduce a benchmark dataset, CocoCON, where we create contrast sets by modifying test instances for multiple tasks in small but semantically meaningful ways to change the gold label and outline metrics for measuring if a model is consistent by ranking the original and perturbed instances across tasks. We find that state-of-the-art vision-language models suffer from a surprisingly high degree of inconsistent behavior across tasks, especially for more heterogeneous tasks. To alleviate this issue, we propose a rank correlation-based auxiliary training objective, computed over large automatically created cross-task contrast sets, that improves the multi-task consistency of large unified models while retaining their original accuracy on downstream tasks.

Published in Transactions on Machine Learning Research (02/2024)
Exposing and Addressing Pross-Task Inconsistency in Unified Vision-Language Models
Adyasha Maharana
adyasha@es..ncc.ooo unc.edu
University of North Carolina, Chapel Hill
Amita Kamath
kamatha@cs.uccke eduu
University of California, Los Angeles
Christopher Clark Allen Institute for AT
chrisc@aallenai.org
Mohit Bansal
mbansal@cs. unc. edu
University of North Carolina, Chapel Hall
Aniruddha Kembhavi
onibaallenaioorg
Allen Institute for AT
Reviewed on OpenReview: o/pnreeveeeeeeeee net/ forum? id= TDDL2
Abstract
As general purpose vision models get increasingly effective at wide set of tasks, it im- perative that they be consistent across the tasks they support. Inconsistent AI models are considered brittle and untrustworthy by human users and are more challenging to incorpo- rate into larger systems that take dependencies on their outputs. Mcasuring consistency between very heterogeneous tasks that might include outputs in different modalities is chal- lenging since is difficult to determine the predictions are consistent with one another. As a solution, we introduce a benchmark dataset, CocoCoN, where we create contrast sets by modifying test instances for multiple tasks in small but semantically mcaningful ways to change the gold label and outline metrics for measuring if a model is consistent by ranking the original and perturbed instances across tasks. We find that state-of-the tere--of-fhe-attt vision-language models suffer from a surprisingly high degrec of inconsistent behavior across tasks, especially for more heterogeneous tasks. To alleviate this issue, we propose a rank correlation-bann auxiliary training objective, computed over large automatically created tross-task contrast sets, that improves the multi-task consistency of large unified models while retaining their original accuracy on downstream tasks. Data and code are available Inttps://adymaharann github. io/ cococon/
1
Introduction
General Purpose Vision (GPV) models (Gupta al., 2022a Kamath et al. 2022 Cho al. 2021 Lu al. 2022 Wang al. 2022 are trained perform many diverse multimodal tasks ranging from visual question answering (VQA) and referring expression grounding to semantic segmentation and image generation. A fundamental requirement and intuitive expectation of such systems is that they provide consistent results across the tasks they support. For example, if a system produces the caption two jaguars are sitting on a tree branch then one would expect it to answer the question Whal animals are these? with jaguars and to return two bounding boxes if asked to locate the jaguars.
While the latest GPV models (Lu et 2022 Wang al. 2022 Huang al. 2023) perform impressively on multi-task benchmarks (Gupta et al.) 2022b), find that these models provide surprisingly inconsistent

---

Published in Transactions on Machine Learning Research (02/2024)
Image Captioning
A cat eating a bird it has caught. A eating a rabbit it caught.
2in mobi
Inconsistent
Visual Question Answering
What is the cal eating? bird What the cat nating? rabbit
Unified
Vision-Language Model
Consistent
Soundra Box
Localization
Localize the bird image. Localize the rabbit in the image.
Consistent
Text-to-ima image Generation A cat eating a bird it has caught. A cat eating rahbit it caught.
thupe
10004
Figure 1: Examples of consistent and inconsistent predictions from Unified-TOxz (Tuu et al. 2022
answers for simple images and lasks. Fig. 1 ssows an example where Unified-TOxx (Lu et al. 2022 prefers the caption: cat eating a rabbit it has cought, but then answers bind when asked What is the cat eating? Solving multiple tasks for one image may require some degree of specialized reasoning, but they necessitate semantic interpretation of the imput image which should be common across tasks. When models demonstrate such trivial inconsistencies, it is hard for end users to trust them, particularly in important applications, because it is harder to understand and predict their behavior. From a practical standpoint, it is challenging to incorporate such models into larger systems, because it's hard to calibrate for them. Finally, from a philosophical view, having different interpretations of an image depending on the target task defies how we intuitively think unified models should behave.
In computer vision, cross-task consistency has been of some interest for classical tasks Zamir al. 2020), while in natural language processing past work has studied consistency for tasks like question- answering (Kassner et al. 2021). However, in vision-and-language research, much work has focused on within-task consistency for visual question answering (Shah et al. 2019 Ribeiro et al. 2019 Dharur et al.
Ray et al., 2019 Bitton et al., 2021). Semantic consistency of multi-modal models across tasks has remained unexplored, partly due to the absence of models (until recently) that can perform various tasks simultaneously and effectively.
2021
With recent advances in GPV rescarch, we can now probe models for cross-task consistency. A simple and straightforward method is to compute the semantic overlap between model's predictions for the same image across tasks. While possible for related tasks like captioning and VQA, measuring semantic overlap between outputs from different modalities can be ill-defined (e.g. it is unclear how to quantify the overlap between bounding boxes for localization and an answer for VQA). Additionally, models may perform well by producing simple outputs for tasks. For example, if model generates short captions about single subject, this method can only probe consistency for that narrow set of visual elements. Instead, we choose to utilize human-defined outputs for tasks that cover a wide range of semantic elements for complete evaluation of consistency. For given pair of tasks, we perturb the test instances in similar but meaningful ways that change the gold label, in order to create contrast sets (Gardner et al., 2020). More likely perturbations (e.g. keyboard + laptop in Fig. b)) lead to harder contrast sets whereas less likely perturbations (e.g. keyboard + earbuds) lead to easier contrast sets. Then, we measure a model's likelihood of predicting the ground truths as well as their contrast counterparts for both tasks. a model is more likely to predict the contrast output for one task and the ground truth output for the other task or vice-versa, it implics that the model has contradicting interpretations of the same input for the two tasks. In the example shown in Fig. 2 the model favors the caption with computer keyboard, but is more likely to answer laptop in response to the question: What is the object in the lower right-hand corner?, leading to cross-task inconsistency. Operating with likelihoods also allows us to overcome the challenges of comparing outputs from two different modalities.
For this purpose, we present CoCoCoN, a benchmark dataset with contrast sets for four commonly used multimodal tasks. Each sample in CocoCon contains up to five contrast sets of varying difficulty for each of the tasks. We use image captioning as an anchor task because captions contain semantic elements used by most other tasks and evaluate it against VQA which has textual outputs, localization which has bounding
2

---

Published in Transactions on Machine Learning Research (02/2024)
Object Localization on
Local koyboa in
Humen-Obi Interaction
Local la the kptop in the image
condistent
Anchor Task
Referring Expression
Image Generation
Image Captioning
keyboard siting on
Menerateeccompall
Tncons stert
Acomputar keybbard sittngg on toa of table.
Situation Recognition
kotop
Alapoop siting top
siting table
Multigle Contrast Sets
Region Classification
Visual Question Answering
keyboard mouse
What e the object in the lower righthand comee??
keyboard video game controller
beyboardd earbuds
keyboardd ii
keybcard tablet
keyboard
Image Retrieval
laptop
Our evaluation framework for probing cross-task consistency in unified models via contrast sets.
Figure 2: Illustration of our method for probing inconsistencies across tasks. We build candidate answers for multiple tasks that correspond to different semantic understandings of an image (e.g., keyboard vs. laptop), and check if the model's preferred answers across lasks match the same semantie understanding.
box outputs, and image generation with image outputs. This covers task pairs with outputs in the same output modalities well as different output modalities. We mcasure consistency % well as Spearman's 's rank correlation coelficient between the ranking of contrast sets.
We evaluate two recent GPV models, Unified-10 (Lu et al. 2022) and OFA (Wang et al. 2022), both of which support all four tasks in CocoCoN. Additionally, we evaluate Kosmos-2 (Pegg et al. 2024) and GILL [Koh et al. 2023) which support three out of four lasks CocoCoN. We show that cross-lask inconsistency is a surprisingly significant phenomenon in these models across all tasks in CocoCon and various model sizes. Inconsistency increases with the heterogencity between output modalities within pair of tasks well with the complexity of the tasks themselves. Moreover, consistency improves with casier contrast sets, yet. remains significantly less than 100% for all tasks. We also find that larger models are more consistent by virtue of being more accurate at the tasks. Finally, our evaluation suggests that multi-task models capable of performing a larger set of tasks are more inconsistent.
Cross-task inconsistency undesirable a unified model, and it paramount that we work toward mit- igating it. To this end, we propose using a consistency objective utilizing large automatically generated cross-task contrast sets and rank correlation loss objective via soft ranking (Blondel et al.) 2020). Our experiments show that continued training of models using this auxiliary consistency- based objective can lead to consistency improvements when evaluated on CocoCon while preserving or improving the accuracy of the model on the original test sets. In summary, our contributions include:
(a) highlighting the issue of cross-task inconsistency in multi-modal models,
(b) introducing the use of contrast sets and benchmark dataset, CocoCon, to measure cross-task inconsistency amongst four popular multimodal tasks,
(c) demonstrating the inconsistent behavior state-of-tho-ar languael models, and
(d) a consistency-based training objective to improve consistency without compromising accuracy.
2 Related Work
To our knowledge, no existing work evaluates cross- task consistency for multi-modal models. In this section, we discuss studies that evaluate and enforce consistency for individual or multiple tasks within one modality.
Consistency for VQA. Shah et al. (2019) revealed that VQA models are inconsistent across linguistic variations of a visual question, then improved consistency using automatic data augmentation; an approach which further improved in Kant et al. (2021) using an additional contrastive loss. Ribeiro et al. 2019
3

---

Published in Transactions on Machine Learning Research (02/2024)
Ray et al. (2019) evaluated consistency across the original QA data and automatically generated QA pairs implied by this data. [Selvaraju et al. (020)) collected buman-annotated sub-questions to evaluate model reasoning capabilities through the lens of consistency. Dharur et al. (2021) trained models to rank the sub-questions proposed by SQUiNT (Selvaraju et al. 2020) higher than unrelated questions from the same image, making models more consistent. across both sub-questions and rephrasings of the question. Contrast sets have also been used to measure and improve consistency for VQA (Ribeiro et al. 2019 Bitton et al. 2021). Unlike these works, our approach evaluates and improves consistency across multiple tasks.
Consistency for NLP. Consistency has also been discussed in NLP, primarily in the single-task setting. Elazar ct al. (2021) evaluated and improved factual consistency of pre-trained LMs across paraphrasings of factual statements. Kassner et al. (2021) considered the responses of a pre-trained LM to a stream of questions, and evaluated and improved the consistency and accuracy of its answers over time. Kaushik et al. (2019) collected counterfactual instances evaluate the overreliance of NLP models on spurious attributes. Gardner ct al. (2020) mamually created contrast sets for 10 individual NLP tasks to evaluate single-task consistent responses across meaning-allering perturbations. In comparison to these works, we evaluate consistency across multiple tasks, without the need for human annotations used in Gardner et al.) 22020 Nishino et al. (2019) used multi-task learning with a hicrarchical consistency objective to prodict the headlines, key phrases, and categories of articles; however, the model uses separate decoders per task. Our work studies cross-task consistency of General Purpose Vision (GPV) models with unified output decoders.
Cross- task Consistency for Vision. Cross- task relationships among classic vision tasks have been studied by Zamir al. (2018). [u et al. (2021) used geometry and physics to identify consistency constraints between such tasks, and use them to improve performance in low data regimes. Zaaiir et al. (2020) enforced cross-task consistency for vision tasks using inference-path invariance and demonstrate their method for tasks in the pixel space (like depth and surface normals). It is not straightforward to extend this approach to vision and language tasks which are often conditioned not just on an image but also on a language input and where one task's output may not casily be transformed into another's output.
3 Contrast Sets for Cross- Task Consistency
In this section, we describe the problem of inconsistency across tasks in unified models, motivate the use of contrast sets to evaluate consistency, and outline our framework for measuring cross-task consistency.
The Problem. In the pursuit of developing task- and moddaltty-gnoosii unified systems, models like Unified- I0 (Lu et al.) 2022) are trained on a variety of tasks geared towards learning robust semantic representations of the input. Each task is designed to strengthen the model's understanding of a distinct perspective of the ground truth. For instance, a visuo-linguistic model is simultancously trained to generate a caption for the entire image as well as answer questions about subjects in the image. The popular and effective training paradigm for such models is to learn probability distribution over the space of possible outputs and maximize the likelihood of the target output. This lcads to an inherent ranking of possible outputs based on their probabilities, which can be used to rank outputs that reflect distinct semantic understandings of the input. For a reliable and truly unified model, the ranked space of such probable outputs should also be aligned across tasks. However, (see Fig. D we find that unified models can interpret inputs differently for different tasks, leading to misalignment between these spaces and inconsistency in predictions. We measure this inconsistency with the help of rast sets.
Contrast Sets. Model performances on the i.i.d. test data are often treated as an absolute measurement of its abilities. However, when the test data has systematic gaps like annotation artifacts (Gururangan et al, 2018), the model can learn simple decision boundaries to solve the dataset and result in misleading high performances. Gardner et al. (2020) introduce contrast sets to close such systematic gaps in evaluation. Contrast sets are created by perturbing Lest. instances in meaningful ways that change the gold label. This allows for the evaluation of a model's local decision boundary around a pivot test instance and measurement of how well it aligns with the correct decision boundary. Models with simple docision boundarics fail to perform well on contrast sets. Using the same intuition, we can create equivalent perturbations on test instance for pair of tasks and evaluate whether the unified model performs similarly on the contrast set.
4

---

Published in Transactions on Machine Learning Research (02/2024)
for either task. In this manner, we leverage the framework of contrast sets to measure how well model's decision boundaries for two distinct tasks align with each other.
Consider a model with parameters 0 and two tasks to: 4 that can be performed by the model. To construct a contrast set, we first pick a test instance and the respective ground truth annotations for each task i.e. (Ttg Vta), ( )t termed as the pivot instances. We define the space of contrast. outputs for an instance x as the set of outputs ÿ that are within some distance of y. That is, C(x) = {(r d(y, V) < e), where d(.) is some distance function. Let (yl)z) be the likelihood of model 0 for predicting the output y in response to imput Now, we define the model 0 to be consistent across tasks to, & with respect to the pivots 2 Zt, if the model is more likely to predict the gold outputs Uto Ut, in both tasks, compared to their respective contrast outputs Mre Mh.. The model is also considered consistent if it assigns a larger likelihood to the contrast outputs than the gold outputs of both tasks because, even if the model answers wrongly for both tasks, as long as it reflects a common understanding of the input, the model consistent. Mathematically,
1 1 if fo(yta > f0 (ato)) A fe(ve, > (o
C=
1 if So (2rto) < So (o))) A lo, < So otherwise
(1)
where it. E C(1to), Bt E C(a,,) and C is the consistency score. This framework can be casily extended to more than two tasks. For the scenario of > 2 tasks, we define an anchor task 4n, that contains semanttc elements common to each of the remaining tasks. Then, we compute pairwise consistency scores for the anchor and the rest of the tasks {41, (,, i.e. we have T pairwise scores for T tasks.
Difficulty (k). Contrast sets can be of varying difficulty, depending the likelihood of the perturbations used to create the contrast sets For example, basketball is likelier substitute for the semantic concept football whereas kile is much less likely. Hence, the contrast set containing basketball is a hard contrast set and the one containing kite is an easy contrast set. We rank all contrast sets for a given instance and use the rank &o to indicate the difficulty i.c. lower h implics harder contrast sets.
Evaluation Metrics. We introduce two metrics for calculating the consistency of a model over a dataset of N samples, containing K contrast sets each, for T tasks. Each sample consists of pivot instances for the T tasks and the corresponding sets of up to K contrast outputs. We first rank the K contrast sets by difficulty according to the model's likelihoods for the anchor task, (il..... V6 J. For each task t, and at each k we compute % consistency @ k (C&) as the % of samples for which the model is consistent i.e.,
N
C+ = 1 E Clyt Mi Vhg gf.)
where consistency C(.) is computed as per Equ. ! Higher values for Cs suggest that the model more consistent across to and f,. This metric mcasures consistency with respect to the ground truth annotations, which are used as pivots in our setup. We also compute spearmanr (Prant); the Spearman's rank correla- tion coefficient over the ranked contrast outputs for both tasks, in order to measure the global alignment between the two output spaces. We observe these metrics in tandem with task-specific accuracies to avoid overestimating a model with degenerate but consistent solutions.
4 The CocoCon Benchmark
In this section, we detail the construction and composition of our benchmark dataset CocoCon, developed as per the framework in Sec. 3 Then, we discuss the statistics and evaluation framework of CocoCon.
4.1 Dataset Construction
The COCO dataset (Lin et al. 2014) contains annotations for many tasks in vision and language, which makes it very suitable for evaluating cross-task consistency in a multimodal model. CocoCon created from the validation splits of the four tasks i.. image captioning (anchor task), VQA (Antol et al. 2015; Goyal et al. 2017), localization, and text-to-image generation. The dataset creation pipeline is as follows.
5

---

Published in Transactions on Machine Learning Research (02/2024)
The jumping in the air to catch the frisbee.
Describe the image.
What is playing?
GPV-2 Model
The women is jumping the to catch the
T5
What is she playing?
(Rt)
foothall (3ts
frisbee [ak.
Detect the frishee. (
(2tt (6)
Vtc
football, valleyball, teeball, hide-and-seek
Uku
football.
teeball.
hide-and-seek.
X
Generate the frisbee. 3
Generate football.
Detect the football
(
Step Selection of Pivot Instance for Anchor Task (fo) l.e. Image Caplioning
Step 2. Get Answer
Slep 3. Select High-Quality
Step 4. Add Evaluation Tasks i.e. VQA.
Candidates for Creating Contrast Sets
Contrest Sets by Filtering
Localization (ta), and Text-lo-mmage
Caplions
Generation(ss.
Figure 3: Step-by-step demonstration of the automated pipeline for generating contrast sets. Contrast sets generated from this pipeline are manually filtered to prepare the CocoCon benchmark.
(Step 1) Selection of Pivot Instances. First, select pivot instances for the captioning and VQA tasks since it is easy to compute semantic overlap between the outputs of these tasks. Existing captioning annotations for the COCO dataset were filtered to retain ones that had semantic overlap with at least one question-ansee pair from VQAv2 annotations. For instance, the caption: The womon is jumping in the air eatch the frisbee. from COCO overlaps with the VQA sample: What is she playing? frishee (see Fig. 3 Step 1) and was retained in our method. The semantic overlap was computed using series of text-processing steps including lemmatization and word overlap.
(Step 2) Contrast Set Candidates. Next, we need to substitute the overlapping semantic concept with other likely concepts to create contrast sets. There are many ways to perform this step. For instance, these perturbations can be written by human annotators, which might result in undesirable systematic biases in the contrast sets /urrrangaa et al.) 2018). Adversarial methods advocate gradient- craseent-based methods to get hard negatives for such perturbations (Alzantot et al., 2018), however, we want to avoid integrating the biases of the models we are evaluating into a benchmark dalaset.
In contrast, we choose to use probable answers to the VQA questions from an off-the-shed VQA model, GPV-2 (Kamath et al. 2022). to create a large set of perturbations (sce Fig. 3 Step 2). GPV-2 is trained on the WeblOK dataset (Kamath et al. 2022) that contains semantic concepts beyond COCO. This makes the contrast sets in CocoCon diverse and additionally challenging for unified models. Note that we do not evaluate GPV-2 on CocoCon since it can perform only a subset of the tasks present in it (see Fig. 2
(Step 3) Filtering. The perturbations obtained from the previous step are filtered to retain high-quality candidates only, by creating contrast captions and retaining captions (and the corresponding contrast VQA samples) with high scores from the T5 language model i.e., the ungrammatical and nonsensical captions are filtered out. For instance, in Fig. 3 (see Step 3), the GPV-2 answer score, because cotch the hide-and-seek is an unlikely phrase.
is filtered out using T5
(Step 4) Heterogeneous Evaluation Tasks. The next step is to add evaluation tasks with heterogeneous output modalities i.e., localization and text-to-image generation (see Fig. 3 Step 4). For the localization task, the automatically generated dataset from the last step is merged with the COCO localization annotations. Annotations for localization in COCO images pertain to the narrow set of pre-defined COCO objects, which may may not appear in the caption. Only those objects which appear in the caption and VQA answer are retained in CocoCon for the localization task. The contrast outputs created for the VQA task are used as contrast inputs for the localization task. For instance, in Fig. 3 the contrast outputs frisbee and football selected in Step 3 for the VQA task are used as localization queries (contrast inputs) in Step 4.
Finally, since image captioning the task of gencrating a natural language description from an image, and text-to-image generation is the reverse process, one can reuse the ground truth annotations and contrasting annotations of captions for the task of image generation by simply reversing them. Similar to localization, the contrast outputs created for the image captioning task are used as contrast inputs for this task, and we measure the models' likelihood of generating the ground truth image in response to the contrast inputs.
(Step 5) Manual Filtering. This generated dataset was then subject to manual filtering and editing to ensure the high quality of the contrast sets. In this step, contrast sets that were synonyms, holonyms, hypernyms, or meronyms were removed from the dataset, in addition to other invalid perturbations. We conducted a study for inter-annotator agreement between two expert annotators on 200 samples and found
6

---

Published in Transactions on Machine Learning Research (02/2024)
Category
Input Image
Original Instances
Contrast Sets
V VIGIL
Caption: child bed striped sweeter and colorful blanket. VQA: What is the baby sleeping with? bla
stuffed an mal pillow
x
Object
Image Generation: Generate image with the text "A child bed with striped sweeter and colorful lalankes.
teddy bear
yellow black
Caption: A brown and white cat lying on the bed. VQA, What color
spots? white
Attribute
Image Generation: Generate an image with the text 'A brown and white cat y ng the bed."
gray
orange
Caption: Apples hanging from
that has hardly any leaves it.
VOA What type tree is this? apple
Food
Image Generation: Generate an image with the text "Appee hanging from tree that hardly any leaves on
orange cantaloupe
Localization: Which region does the text "appls" describe?
Caption: Office space cat on
work.
VQA: What is playing on the TV? cat
squirrel
Animal
Image Generation: Generate image with cat on the felev-sion and work."
"Office space
baseball
butterfly
Localization: Which region does the text "cat" descrios?
bowling
Caption: A woman eating piece of pizza near the kitchen counter.
living room dining room
VQA Where the person standing? Kitchen
Location
Image Generation: Generate an image with the next woman eating the kitchen counter."
Caption: A man holding out to catch baseball
tee ball
VOA What sport the playing? basebal.
softball
Image Generation: Generate an image with the text "A man holding out to catch a basehell"
football
Role
basketball
Caption: A herd of zebra standing on top of dry gress field.
running
VQA What the two zebra doing the laft? standiin
Action
Image Generation: Generate an image with the next "A hard of zebra
standing of city grass field."
Caption: young boy getting ready blow out candles for his birthday.
girl
Person
VOA: is this child boy girl? boy
Image Generation: Generate an image getting ready to blow out cand les for his birthday."
text "A young boy
Caption: A giant colgate clock sits on the shore next to water. VOA: What the sponsor named? colpate
walgreens
billboard
OCR
Image Generation Generate an image with the text "A giant clock sits on the shore next water."
state farm
wrigley's yves saint lauren
Caption: There sheep grazing field.
Misc.
Image Generation: Generate image with the text "There are sheep grazing in field."
VOA: How many sheep can be seen?
6
5
7
Figure 4: Examples of contrast sets used in CocoCon. For each example, we show the relevant image (left)), the ground truth caption, VQA question, image generation prompt for the image with the perturbed concept in green (middle), the set of perturbations used generate alternative answers and predictions from Unified-IO XL for VQA (V), image generation (G) and localization (L) (right columns). / and indicate scenarios where the model predictions for captioning and the corresponding task for that particular contrast set are consistent and inconsistent respectively. '_' denotes a lack of localization annotations for the sample.
agreement for 98% of the data, indicating the high quality of the dataset. We prioritized the collection of clean, expert annotations over size for this probing dataset. Note that the contrast sets were manually filtered to ensure high quality at test, but at training time we only use automatically generated data.
Note: Our contrast set generation pipeline precedes recent advances in multimodal large language models LLu et al. 2023a by that can be used to significantly ease the generation of contrast sets (see Appendix B.1
4.2 Dataset Categories & Statistics
Each sample in the CocoCon dataset contains of ground truth annotations and a semantic concept within the original caption is replaced with multiple contrast. sets. The ground truth annotations comprise

---

Published in Transactions on Machine Learning Research (02/2024)
those for image captioning, VQA, and text-to-image generation, and 30% of the CocoCon samples also contain annotations for localization In total, the CocoCon dataset contains 4789 contrast sets for 1,500 samples from the COCO validation split, with an average of 3.2 contrast sets per sample. The semantic concepts used for perturbing the pivot instances in this dataset range from a large variety of semantic, syntactic, and grounding phenomena. We labeled each sample from CoCoCon for these phenomena, see examples in Fig. andd a brcakdown of the categories as well as additional examples in Appendix B Attributes (color, height, material, etc.), inanimate objects, and food are the most frequent semantic concept categories in CoCoCON, followed by animals, roles, actions, and location.
4.3 Evaluation
We measure the consistency between the captioning task (anchor) and each of the evaluation tasks inde- pendently. To evaluate consistency between captioning and VQA tasks, we compare the models' likelihoods of generating the caption and the VQA answer for both, pivot and contrast. instances. For the localization and text-to-image generation tasks, the outputs are common to both, pivot and contrast instances, whereas the inputs contain semantic perturbations (see Fig, 3 Step 4). Hence, we compare the models" likelihood of generating the output in response to the imput from the pivot instance (It, V1) vs. the input from the contrast instance (āt, Mt) i.e, we replace So Jo (Tt,) in Equ. [ with Sa(Vio (Tt,), [o (Tx,) respec- tively. For example, we compare models' likelihood of generating the ground truth image in response to the gold caption and the contrast caption (c.g. caption containing frisbee vs. football in Fig. 3 for the text-to-ing to-ema generation task.
5 Consistency-band based Training
Algorithm 1 Cross-Taas Consistency-based Training
A unified model exhibiting inconsistent predic- tions suggests that the model has learned weak semantic representations that are sensitive to task variations. It is undesirable work with a model that is susceptible such frailties. Moreover, consistency constraints can provide useful infor- mation for learning well-rounded semantic repre- sentations (Lu et al. 2021) and reduce the nood for training data (Zamir et al. 2020). Hence, we propose to train unified models in a way that pre- serves consistency across their predictions (see Al- gorithm D. Given pair of train instances 24, St, for the tasks to, t1, let {yro {3t, } be the spaces of K probable and semantically equivalent out- puts. fo(.) is the scoring function for a model with parameters and R(.) is some ranking function. Since ranking a non-differentiable operation, we use soft ranking via regularization (Blondel et al., 2020) as the differentiable ranking function R(.). We formulate the consistency-batt loss objective
1: 7 - ratio of conssstency-based updates to total updates
2: A wewighh co-efficient for consistency-based loss
3: to, <- anchor task (e.g. captioning), evaluation tasks
4: (Pt,,1,, (8t,3) }] imput, gold output, contrast outputs for &. do
5: for epoch = 1,2
6: for step = =, randon (0, 1)
M do
7:
8:
if r 5 then i r-ndoo (1, 2, 3)
9:
Auchor task: (Xey. Yig: (1ip)) 1) t- (Ftg Vao (We))
11:
Evaluation task: (Xe,i Yeo (Y..) & (Fi, Mees (8r,
Cross-cntropy losses: (L2.) (Laz)
13:
Ranks: Ro, Ri + rank (L2e rank( (( (L
14:
I.const + -spearmanr( Ro, Ra)
15:
1 X Leonst + Lice
16:
else
17:
Standard pretraining data: (X,Y) + {2.0} Cross-entropy loss: {Lee}
18:
19:
end if
20:
Compute backward pass end for
21:
22: Evaluate updated model for cross-task consistency
23:
end for
using Spcarman's correlation as follows:
Loonst = II 1 - R(Jo(ty,
Within a space of l probable outputs for either task, if an output for task to is ranked at l: 2 while the equivalent output for task L ranked at + 2, the gradients from this objective are designed to push the two misaligned outputs towards a common rank ls, which increases consistency as per the definition of C2 in This can affect the task-specific accuracy of an inconsistent model, especially when the more probable
Sec. 3.
Localization annotations are present when a COCO object appears in the gold caption and VQA answer.
8

---

Published in Transactions on Machine Learning Research (02/2024)
Table 1: Summary of the CocoCon tasks supported by the various models used in our experiments.
Model
Image Captioning Visnal QA (VQA)
Localization
Text-to-Image Gen.
Unified-10 (Lu et al. 2022
OFA (Wang et all 2022
finetune
Kosmos-2 Peng el 2024
zero-shot zero-shot
zero-shot zero-shot
GILL Koh ct al 2023
X
output is the gold label. Hence, we minimize our proposed consistency objective in addition to the standard cross-entropy loss during training i.e.
L x Leanst +
Lee
(4)
where Lee is the cross-entropy loss and À is the weighting factor for the consistency objective. See Algorithm
6 Experimental Setup
Vision-Language Models. Unified-1O (Lu et al. 2022) and OFA (Wagg et al. 2022) are two recent publicly released models that perform a wide variety of tasks, including all tasks supported in the CocoCon benchmark. Unified-IO is pre-trained on all tasks in CoCoCON, as well as multiple other vision-only, language-only and vision-language tasks. OFA models are pretrained on image captioning. VQA, image- infilling, and language-only tasks. Hence, we finetune the pretrained OFA models on the tasks supported in CocoCon for two epochs to support tcxt-to-image generation P models. Additionally, we evaluate Kosmos-2 (Peng et al. 2024) and GILL (Koh al. 2023) which support localization and text-to-image generation tasks respectively. Besides, both models support zero-shot image captioning and VQA tasks. See a summary of these models' capabilities in Tab.
We evaluate all size variations of both
Evaluation Metrics. As outlined in Sec. 3 we compute consistency % (C&) and spearmanr (Prank) for evaluating cross-task consistency. Additionally, we mcasure the following task-spocific metrics: CIDEr score (Vedantam et al. 2015) for image captioning, accuracy for VQA (Goyal et al. 2017), IOU score (Padilla et al. 2020 for localization, and FID score (Heusel et al. 2017) for text-to-image generation.
Consistency-based Training. We begin with the finctuned checkpoint for the OFALARGE model and continue training with the objective proposed in Sec. 5 We adapt the automated pipeline introduced in Sec. 4 to generate nearly 84K contrast sets from the training split of COCO Captioning, VQA, and localization datasets. We performed a manual analysis this dataset and found that nearly 85% of the contrast sets are valid, which is of sufficient quality for large-scale training purposes. We use the cross-entropy loss as the score fo(-) function for each sample. The models subjected to continued pretraining for one epoch and trained on the combination of contrast sets and original datasets for the four tasks in CocoCoN. We set X 0.25 and use learning rate of le-6, Additional hyperparameters can be found in Appendix C This finetuned model is referred to as OFACON in the rest of the paper.
7 Results
In this section, we discuss our findings from the evaluation of pretrained vision-language models, calibration of model likelihoods, common failure modes across models, and consistency-based training (see Sec.
7.1 Evaluation of Pretrained Models
Models are more inconsistent across tasks of diverse modalitics. We wish to study how the semantic understanding of a unified model varies with lasks. We evaluate the best (and largest) OFA and Unified-IO models on CoCoCon and compare % consistency across the 3 tasks i.e., VQA, localization, and text-to- image generation, with respect to the anchor task, i.e. image captioning. Results are shown in Fig. 5 a). For VQA (blue plots), OFA HUGE and Unified-IO X models exhibit 78% and 68% top-1 consistency respectively.
2The FID score of our finetuned OFA models on the tott-to-mmage generation task higher (worse performance) than that. reported in Wang et 2022 because the latter model finetuned on the to-iage generation task only.

---

Published in Transactions on Machine Learning Research (02/2024)
Task & Contrast Set Difficulty
Accuracy Consistency
Model Size
100
VQA
Localizatio
% Consistency
Unificd-OO
OFA
Kosmos-2
GILL
OFAcow
241 Parameters Unified-IO Model
Difficulty (k)
Accuracy
Figure 5: Results from evaluation on the CocoCon benchmark. (a) % Consistency of Unified-IO XL: OFA Kosmos-2 and GILL models for varying difficulty (k) and all tasks in CocoCoN, (b) comparison of % accuracy with % consistency (k=1)) values for all models evaluated in this paper and our OFAco model (see Sec. 5 and (c) % consistency (I=1) values for different sizes of Unified-OO models.
This number changes to 68% and 65% top-1 consistencies for localization (red plots), respectively, suggesting that unified models are especially prone to variation in semantic understanding when the outputs belong to different modalities. This is further supported by results for image generation (green plots) with 48% and 50% top-1 consistencies. Text-to-image generation is more complex than localization, because of the high dimensional output and rigorous semantic understanding required for the task. These results also suggest that cross-task inconsistency increases with the complexity of the task as well.
Models are inconsistent at hard as well as casy contrast scts. The contrast sets used for evaluating top-1 % consistency are hord negatives and we observe low consistency for these samples (sce Fig. 3 a)). For easier contrast sets i.e. in k > 1 scenarios, the % consistency increases significantly (yet remains 100%) for tasks with outputs of the same modality as the anchor task, as seen for VQA in Fig, 5(a). The % consistency for VQA increases from k=1 to k=2 by 12%, 8%, and 4% for Unified-IO, OFA, and Kosmos-2 respectively, and then by smaller margins with increasing k. Similarly, the % consistency for localization increases from k=1 to 1=2 by 12%, 1,, and 5% for Unified-IO, OFA, and Kosmos-2 respectively, and then by smaller (or negative) margins. However, we see almost no improvement consistency for text-to-image generation with increasing k implying that the unification of modalities within model is a non-trivial challenge.
Models are more accurate than consistent. We compare the top-1 % consistency scores with the task-specific accuracies of models the CoCoCon dataset in Fig. 5)),, and observe that consistency and accuracy are positively correlated. Unified-IO models feature well below the x y line while other models, i.., OFA and Kosmos-2, lie close to the x = line. Unified-10 models are more accurate at the VQA and the localization tasks than the other models, without demonstrating a similar gain in consistency. This suggests that when the Unified-TO models make mistakes for one task, they rarely make the same kind of mistakes on the other tasks, which is what would allow model achieve high consistency independently of accuracy, On the other hand, OFA and Kosmos-2 models show tight correlation between accuracy and consistency. Additionally, observe that the models gradually transition over the x = y line with increasing is i.e., easier contrast sets (see Fig. 13.. Notably, OFACON consistently ranks the highest in terms of consistency at all k. Ideally, we want models to be highly consistent across tasks despite being inaccurate i.e., lie well above the x=y line, and theoretically, it is possible with a unified semantic backbone in the model (sce discussion in Appendix A). Instead, existing models appear to be consistent mostly by virtue of being accurate. This has the worrying implication that harder or more ambiguous tasks will lead to severe inconsistencics, and high consistency on easy (asks does not necessarily mean models are parsing inputs in a unified way across tasks. It also highlights the importance of studying hard tasks like image generation when evaluating consistency.
Models capable of performing more tasks are more inconsistent. Unificd-IO models are trained on 90 diverse datasets from vision and language domains and can perform all 7 tasks on the GRIT benchmark Gupta et al. 2022a]. In contrast, OFA models are pretrained on a subset of the tasks that can be performed by TO. Interestingly, we observe that OFA models are more consistent than Unificd-IO across all three tasks in the CoCoCon benchmark. Additionally, Kosmos-2 and GILL (see rows I,J in Tab. 2 are more consistent than any Unified-TO or OFA models at their specialized tasks ie.. localization and text-to-image
10

---

Published in Transactions on Machine Learning Research (02/2024)
Table 2: Results from evaluation on the CocoCon benchmark. Metrics are task-specific accuracies,
% consistency (A: 1) and Spearman's 's rank correlation coefficient (Prank). Higher is better, except for FID.
Caption CIDEr
VQA
Localization
Text-to-Image Gen.
Model
Param
Acc.
C1
Prank
Acc.
C1
Prank
FID
G
Prank
A
Unified-10 Small smmll
71M
111.8
75.3
28.1
-0.06
50.6
36.3
-0.09
93.45
49.5
0.05
B
Unified-10 Base
241M
140.5
87.8
36.2
0.12
61.59
41.6
0.13
91.56
50.4
0.02
C
Unified-1O Large
776M
227.1
90.0
65.1
0.36
68.8
56.2
0.03
85.04
48.5
-0.01
D
Unified-10xz
2.9B
269.9
92.3
68.7
0.48
72.1
65.9
0.20
70.23
50.8
-0.0
edium
93M
83.4
72.7
72.1
0.67
55.6
52.3
0.21
110.3
49.1
-0.02
OFA
182M
100.7
77.8
77.0
0.65
62.3
59.4
0.19
105.7
50.1
0.04
Lurge
472M
113.5
82.6
80.7
0.64
71.3
64.7
0.28
103.4
52.3
0.02
OFA
930M
110.3
82.7
78.8
0.62
70.1
68.5
0.33
107.3
48.3
-0.01
I
Rosmas-2
1.6B
65.8
44.8
70.9
0.62
70.8
70.1
0.60
J
GILL
8B
45.6
35.7
51.9
0.41
25.4
57.6
0.10
Consistencyy based
Training
G
OFAge
472M
113.5
82.6
80.7
0.64
71.3
64.7
0.28
103.4
52.3
0.02
K
Cont. Pretrain
472M
118.8
82.7
81.1
0.63
73.5
65.9
0.27
98.5
51.7
0.04
L
Hinge Loss
472M
117.5
83.0
82.9
0.64
73.8
67.7
0.29
99.5
53.2
0.05
M
OFAcON (ours)
472M
119.4
82.4
83.8
0.67
74.1
69.5
0.35
99.1
53.8
0.09
Unified-IOsmau
Unified-IOppss
Uniffed-1Oranee
Unified-IO
Cumulaaivv frac. Average
of dalnset CLIPScoree
6.75
100% data
0.70
95% data
0.65
2.0
10c
0.5
Image like lood
Image Ikelihood
Image likelihaod
Image likelhoad
Reliability maps of model calibration for the text-to-image generation task
Figure 6: Reliability maps (Guo et al., 2017) for likelihood scores from Unified-IO. A positive correlation between likelihoods and output accuracy i.e. CLIPScore (Hessel et al. 2021) indicates that the likelihood scores can be reliable stand-ins for model confidence in our evaluation of the CocoCon benchmark.
generation respectively. This suggests that massive multi-tasking can lead to larger misalignment between models' semantic understanding across tasks, especially those with heterogencous output modalities.
Larger multi-task models that are more accurate are more consistent. We evaluate various sized Unified-10 and OFA models (see Fig. ((c)) and Tab. 2). We see that the top-1 % consistency values increase gencrously with the scale of the model for VQA and localization, up to 20% increase from Unified-IO SML-IISMMAALL to Unified-TOxx: x on VQA. Improvements are modest for image generation with model size. We see similar trends in OFA models barring a small drop in accuracy well as consistency in the largest model.
7.2 Calibration of Model Likelihoods for Text-to-image Generation
In our evaluation of multimodal models on the CoCoCon benchmark, we compare the likelihood of a model for predicting one output vs. another and observe meaningful, intuitive trends for the VQA and localization tasks. However, the results are less sensitive to variations in model size and contrast set difficulty for the text-to-image generation task. Hence, in this section, we examine the calibration of the likelihoods obtained from Unified-10 and OFA models for text-to-image generation outputs, to find out whether they can be used reliably for evaluation on the CocoCon benchmark. Guo et al. (2017) state that the class probabilities predicted by a well-calibrated classifier should reflect the ground truth correctness likelihood of the model. They demonstrate this behavior through reliability maps, where first, the range of class probabilities predicted by a model is divided into equally spaced bins. In each bin, the average predicted probability of the samples belonging to the bin is compared to the average accuracy of the samples. This plot should align with the y line for well-calibrated model, indicating that the model confidence is exact estimate of how
11

---

Published in Transactions on Machine Learning Research (02/2024)
likely it is for the prediction to be correct. We adapt this principle for the text-to-image generation task in multimodal models and plot reliability maps for all model sizes of Unified-TO by comparing the likelihood of predicted images with their average CLIPScore (Hessel et al. 2021 See results in Fig. 6
The ideal scenario of x = y for classification tasks is ill-defined for the text-to-image generation task since likelihood scores do not have a finite lower bound. Hence, we only seek a positive correlation between the likelihood scores and CLIPScore values as an indication that the model is well-calibrated. We find that Unified-IO models' likelihoods are indeed positively correlated with CLIPScore values for all model sizes (see linear regression fit over data points in Fig. 6 The correlation sometimes falters towards the upper bound of the distribution which can be attributed to the small sample size in those bins, as seen in the distribution of the cumulative fraction of the dataset over the range of likelihood scores (see Fig, G bottom). A linear regression fit over 95% of the samples yields an even stronger positive correlation, especially for the smaller Unified-TO models. This analysis suggests that the low values of cross-task consistency as seen for the text-to-image generation task in Fig. 5 primarily stems from the complexity of the task itself as well the gross misalignment between text and image output modalities in multimodal models. Further, we perform temperature calibration (Guo et al. 2017) the likelihoods from Unified-10sMan ALL and observe no significant improvement in correlation (sec discussion and supporting figures in Appendix D.I.. We also perform this analysis on likelihoods of text-to-image generation outputs from OFA models and find that they are similarly reliable for evaluation on the CocoCon benchmark.
7.3 Common Failure Modes
CocoCon
Unifled-IDxz Errors
We analyze the contrast sets in CocoCon for which Unified-TOxI. OFA Huge; Kosmos-2 models are in- consistent for all tasks and categorize the errors into the tags defined in Tab. 3 We find that all three of the models perform worst at recognizing al- tributes correctly, i.e., 39.7%, 34.2%, 25.5% of er- rors from Unified-10xt, OFA Kosmos-2 re- spectively pertain to attributes, which are signifi- cantly higher than the category's 20.9% distribution in the dataset. The other prevalent error categories are commensurate with the distribution in Coco- CoN i.., object, food, animal, and location. See ex-
30-
OFAnae Errrs
Kasmos Emors
25
20
15
10
5
Objec
Atribute
Food
Animal
Location
Role
Actinn
Person
OCR
Mac
Figure Comparison of categorical distribution the CocoCon benchmark with that of errors from evalu- ation of Unifted-IOxL, OFA Kosmos-2 models.
amples of errors from Unified-IOxx in Fig. 4
7.4 Consistency-basel Training
As outlined in Sec. 5 we continue training OFA via the use of a cross-task consistency-basee loss objective. Results for the finetuned model, OFAcom, are shown in Tab. 2 (see rows K-M in Consistency- -based Training). Since OFAcon (row M) is finetuned for an additional epoch, we also provide a baseline where OFA is finetuned for an additional epoch with just the original cross entropy objective (row K). We find that our proposed loss objective improves consistency along both metrics i.e. top-1 % consistency and rank correlation. The top-1 % consistency improves by 2% for VQA and text-to-image generation, and a larger margin i.c. 4%, for localization. Importantly, see that this preserves the accuracy for VQA, lipping the model over the x y line in Fig. bb) improves localization by +0.6 points, and preserves the FID for mext-to-image generation. These results show the benefits of incorporating consistency-based objectives while training GPV modols.
8 Conclusion
We present a benchmark dataset, CoCoCON, to probe cross-task inconsistency in unified multimodal models and loss objective improve the same. Our results demonstrate that cross-task inconsistency a significant issue in such models and can be mitigated with our proposed loss objective. We hope that CoCoCon serves useful resource for probing the reliability of unified multimodal models in the future.
12

---

Published in Transactions on Machine Learning Research (02/2024)
Limitations & Future Work
Out-of-domain evaluation. The CocoCon benchmark is the first step (to the best of our knowledge) lowards measuring consistency across lasks in multimodal models. It is derived from the COCO dataset (Lin et al., 2014] which has been widely used for pretraining multimodal models, hence, the data samples in CocoCon are in-domain for these modcls. Despite CocoCon being sampled from the same distribution as the training data of such models, we are seeing significant inconsistencies across lasks, especially those of different output modalities. We expect that these incousistencies will aggravate when the samples are drawn from an out-of-domain distribution. Our work does not conduct out-of-domain evaluation, nevertheless, our automated dataset creation pipeline and evaluation framework provide the groundwork for preparing benchnarks similar to CocoCon for out-of--oomain--vllulliii in future research.
Additional tasks. Our evaluation and analysis are carried out three tasks that do not represent the full spectrum of tasks in the multimodal space. However, as shown in Fig. 2 this evaluation framework can be easily extended to more Aision-and-lanuuee
tasks. For instance, the consistency between image captioning and referring expression comprehension (Kazemzadeh et al.) 2014) can be computed in the same way as done for localization. In this case, the semantic concepts appearing in the referring expression should appear in the caption, and one or more of those concepts should be replaced with contrast scts. The ground truth bounding box of the referring expression can be used estimate likelihoods (as outlined for localization in Sec. 4.3) Similarly, to compute cross-task consistency between image captioning and visual entailment (Xie et al. 2019 the hypothoses that contain one or more semantic concepts overlapping with the caption should be perturbed with contrast sets Further, our work capitalized on creating contrast sets based on text perturbations since they are easy to generate at scale and can be implemented for several vision-and- language tasks (Hu et al. 2022 Gupta et al. 2022bb. However, contrast sets based on image perturbations can provide important, additional insights into cross-task consistency of unified vision-language models.
Advances in text-guided image-editing (Zhang et al. 2023 Choi et al. 2023) can be leveraged to create such contrast sets at scale in future work.
Availability of cross-task annotations. We chose the COCO dataset as starting point for our bench- mark because it is popular dataset that has been widely used for the training and evaluation of vision- language models. The availability of annotations for multiple tasks served as convenient starting point as it allowed us to create annotations for multiple tasks with the same pivot instance. However, cross-task consistency is defined for a pair of tasks and is evaluated independently of the other tasks (see Sec. 3) Hence, to evaluate cross-task consistency, we do not require multiple task annotations for the same pivot instance. Instead, we can have different sets of pivol instances with annotations for a different pair of tasks, which can be easily scaled to multiple tasks. Further, many contemporary models super-specialize in one or more tasks, such as Segment Anything for image segmentation (Kirillov et al., 2023). Kosmos-2 for localization (Peng et al. b. BLIP-2 for captioning and VQA (Li el al. 2023). These models can be used to rapidly create annotations for a pair of tasks, and coupled with manual filtering, can be used for scaling to other datasets as well as tasks.
Aggregation of likelihood over semantically similar outputs. Ideally, when computing likelihood for a textual output such as the caption A woman playing with her hair while silfing on a bed, we should consider likelihoods for semantically similar captions as well e.g., those with invariant edits like her hair- and a bed->bed, paraphrases etc. This issue of considering highly similar outputs for likelihood estimation is subset of a broader, open challenge of aggregating over semantically equivalent outputs during the evaluation of free-form natural language generations. We were mindful of this issue when creating the CocoCon benchmark. To maintain uniformity in the evaluation setting, all contrast sets in CocoCon are made of noun, verb, adjective, or preposition words only. Additional pronouns like her in her hair are neither replaced in the original caption nor added in the contrast sets if they are not siguificant semannti concepts. Some recent works propose methods for query or output expansion via mining, paraphrasing (Kuhn et al., 2022), multiple generations (Jiang et al.) 2020), etc., and then ensemble over a pool of similar outputs to get an aggregate score estimate. Since this is an active area of research, we leave their integration with cross-task consistency evaluation to future work.
13

---

Published in Transactions on Machine Learning Research (02/2024)
Broader Impact Statement
The CocoCon benchmark is designed to test the cross-task consistency of unified multimodal models. Our evaluation exposes inconsistencies in such models, indicating that the model outputs are not sufficiently reliable for real-world deployment. We anticipate that our work will influence further research on the important topic of stress testing of unified vision-language models in the community.
References
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai- Chang. Generating natural language adversarial examples. In Proceedings of the 2018 Conference Empirioal Methods in Natural Language Processing, pp. 2890-2896, 2018.
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruy Batra, C Lawrence Zitnick, and Devi Parikh. Vqu: Visual question answering. In Proceedings of the IEEE international conference computer vision, pp. 2425-2433, 2015.
Yonatan Bitton, Gabriel Stanovsky, Roy Schwartz, and Michael Elhadad. Automatic generation of contrast sets from scene graphs: Probing the compositional consistency of gqa. In Proceedings of the 2021 Confer- ence of the North American Chapter of the Association for Computational Linguistios: Human Language Technologies, pp. 94 105, 2021.
Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference on Machine Learning, pp. 950-9550 1500959. PMLR, 2020.
Jaemin Cho, Jie Lei, Hao Tan, and Mohit Bansal. Unifying vision-and-language tasks via text generation. In International Conference on Machine Learning, pp. 1931 1942. PMLR, 2021.
Jooyoung Choi, Yunjey Choi, Yunji Kim, Junho Kim, and Sungroh Yoon. Custom-edit: Text-guided image editing with customized diffusion models. CVPR 2023 AT4CC Workshop, 2023.
Sameer Dharur, Purva Tendulkar, Dhruv Batra, Devi Paríkh, and Ramprasoath R Selvaraju. Sort-ing vqa models: Contrastive gradient learning for improved consistency. In Proccedings of the 2021 Conference of the North American Chapter of the Associalion for Compulational Linguisties: Human Language Tech- nologies, pp. 1103-3111, 2021.
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 111012-101,, 2021.
doi: 10.1162/tacl_a a0010_____0001..._.11__1___11111 URL
https://acllathhl..og wwwchhhooogy.oogooooooo orgg/0221 tac1-1.60
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannanch Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou. Evaluating models' local decision boundaries via contrast sets. In Pindings of the Associalion for Compulational Lin- guistics: EMNLP 2020, pp. 1307-1323, Online, November 2020. Association for Computational Linguistics. URL
doi: 10.18653/v1
wrgp:0000000/000000000000 iindings-emnn
117
Yash Goyal, Tejas Khot, Douglas Summers Summers-Stay, Dhruv Batra, and Devi Parikh. Making the in vqa matter: Flevating the role of image understanding visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6904 6913, 2017.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330 PMTRR 2017.
14

---

Published in Transactions on Machine Learning Research (02/2024)
Tanmay Gupta, Amita Kamath, Aniruddha Kembhavi, and Derek Hoiem. Towards general purpose vision systems: An end-to-end task-agnostic vision-language architecture. In Proceedingg of the IEEE/ IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 16399 16409, 2022a.
Tanmay Gupta, Ryan Marten, Aniruddha Kembhavi, and Derek Hoiem. Grit: General robust image task benchmark. arXiv preprint #xXt:2204.13655 2022b.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah A Smith. Annotation artifacts in natural language inference data. In 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techmologies, NAACL HI.T 2018, pp. 107 112. Association for Computational Linguistics (ACL), 2018.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipseore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7514-7528, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in newral infor- mation processing systems, 30, 2017.
Hezhen Hu, Weilun Wang, Wengang Zhou, and Houqiang Li. Hand-object interaction image generation. Advances in Newral Information Processing Systems, 35:23805 23817, 2022.
Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengehao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, et al. Language not all you need: Aligning perception with language models. Advances in Neural Information Processing Systems, 36, 2023.
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know. Transactions of the Association for Computational Linguistics, 8:423-438, 2020.
Amita Kamath, Christopher Clark, Tanmay Gupta, Eric Kolve, Derek Hoiem, and Aniruddha Kembhavi. Webly supervised concept expansion for general purpose vision models. In European Conference on Com- puter Vision, pp. 662 681. Springer, 2022.
Yash Kant, A. Moudgil, Dhruv Batra, Devi Parikh, and Harsh Agrawal. Contrast and classify: Training robust vqa models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1584-1593, 2021.
Nora Kassner, Oyvind Tafjord, Hinrich Schütze, and Peter Clark. Beliefbank: Adding memory to pre- trained language model for a systematic notion of belief. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8849 8861, 2021.
Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. Learning the difference that makes difference with counterfactually-aanneee
data. In International Conference Learning Representations, 2019.
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects photographs of natural scenes. In Proceedings of the 2014 conference on empirical methods in natural languoge processing (EMNLP), pp. 787 787-798, 2014.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Ven Lo, et al. Segment. anything. arXio preprinl arXiv:2304.022 026443,, 2023.
Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2023.
Lorenz Kuhn, Yarin Gal, and Scbastian Farquhar. Semantic uncortainty: Linguistic invariances for uncer- tainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations, 2022.
15

---

Published in Transactions on Machine Learning Research (02/2024)
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: bootstrapping language- image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, 2023.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C' Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740 755. Springer, 2014.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, In NewrIPS 2023 Workshop on Instruction Tuning and Instruction Following. 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36, 2023b.
Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kembhavi. Unified-io: A unified model for vision, language, and multi-modal tasks, In The Eleventh International Conference Learning Representations, 2022.
Yao Lu, Soren Pirk, Jan Dlabal, Anthony Brohan, Ankita Pasad, Zhao Chen, Vincent Casser, Anelia An- gelova, and Ariel Gordon. Taskology: Utilizing task relations scale. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 8700-8709, 2021.
Toru Nishino, Shotaro Misawa, Ryuji Kano, Tomoki Taniguchi, Yasuhide Miura, and Tomoko Ohkuma. Keeping consistency of sentence generation and document classification with multi-task learning. Tn Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the Ith International Joind Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3195 3205, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/DD 1315. URL httpps://aclanthooogg. hww.:aaathoooooooo///l--------
R. Padilla, S. L. Netto, and E. A. B. da Silva. A survey on performance metrics for object-detection algorithms. In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 237-242, 2020.
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. The Thirleenth International Conference on Learning Representations, 2024.
Arijit Ray, Karan Sikka, Ajay Divakaran, Stefan Lee, and Giedrius Burachas. Sunny and dark outside?! improving answer consistency in vqa through entailed question generation. In Proceedings of the 2019 Con- ference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNL)), pp. 5860-5865, 2019.
Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. Are red roses red? evaluating consistency of question-answering models. In Association for Computational Linguistics (ACL), 2019.
Ramprasaath R. Selvaraju, Purva Tendulkar, Devi Parikh, Eric Horvitz, Marco Tulio Ribeiro, Besmira Nushi, and Ece Kamar. Squinting at vqa models: Introspecting vqa models with sub-questions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10000 10000-10008, 2020.
Meet Shah, Xinlei Chen, Marcus Rohrbach, and Devi Parikh. Cycle-consistency for robust visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 6649-66588 6658, 2019.
Ramakrishna Vedantam, C LLwrencce Zitnick, and Devi Parikh. Cider: Consensus-basod image description ovaluation. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 4566- 4575, 2015.
16

---

Published in Transactions on Machine Learning Research (02/2024)
Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unilying architectures, tasks, and modalities through simple sequence-to- sequence learning framework. In International Conference on Machine Learning, pp. 23318 23340. PMLR, 2022.
Ning Xie, Farley Lai, Derek Doran, and Asim Kadav. Visual entailment: A novel task for fine-grained image understanding. ar preprint orXiv:1901.06706 2019.
Amir Zamir, Alexander Sax, Nikhil Cheerla, Rohan Suri, Zhangjie Cao, Jitendra Malik, and Leonidas J Guíbas. Robust learning through cross-task consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11197 11206, 2020.
Amir Roshan Zamir, Alexander Sax, Bokui (William) Shen, Leonidas J. Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. 2018 IEEE/CVP Conference on Computer Vision and Pattern Recognition, pp. 3712-3722 3722, 2018
Zhixing Zhang, Ligong Han, Arnab Ghosh, Dimitris N Metaxas, and Jian Ren. Sine: Single image editing with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6027 6037, 2023.
Overview
The Appendix is organized as follows:
Section A: Discussion on the relationship between cross-task consistency and task-specific accuracies, in- cluding results from simulation experiments.
Section B: Definitions of the categories in CocoCoN, additional examples from CocoCon and examples of contrast sets generated using Multimodal Large Language Models (MLLMs). Section C: Hyperparameters for training OFACON model. Section D: Additional results model calibration, ablations from training of OFAcON and supporting figures.
A Relationship between Consistency and Accuracy
To understand the relationship between cross-task consistency and task-specific accuracies, we run simu- lation experiment where generate predictions from hypothetical model for anchor task and target task under three different scenarios where the model makes: (A) independent crrors on both tasks, (B) same errors on both tasks and (C) different errors on both tasks. See results in Fig. and a discussion below.
When a model has 100% accuracy on the anchor task (see Sec. 3), the task consistency of the model (with respect the anchor task and target task) is equal the model's accuracy on the target task, irrespective of the model's semantic alignment across tasks (see Fig. B However, when anchor task accuracy is less than 100%, the outcome of model's cross-task consistency (Ci) is closely tied with the nature of errors made by the model across tasks. If the model makes independent errors on anchor and target task, the cross-task consistency tends to stay close to 50% (see Fig. [ A). If the model makes the same errors in both tasks, then the model's consistency generally remains high (see Fig. g B) and it features above the x x= y line in Fig. b). If the model makes different errors for both tasks, the model's consistency remains low (see Fig. S C) and it features below the x = y line in Fig. 5((b). The exceptions to these scenarios are when the anchor and target accuracies are either 0% or 100%. In such cases, the cross-task consistency is 100% if the accuracies of both tasks are the same, and 0% otherwise.
B The CocoCon Benchmark
See selection of additional examples from the CoCoCon benchmark in Fig. and an explanation as well breakdown of the various contrast set categories in CocoCon in Tab.
17

---

Published in Transactions on Machine Learning Research (02/2024)
At 100% pivot task acc., G is identical to target task acc.
When model makes same errors both tasks, C lies above x-y line
When model makes different errors both tasks,
C lies below x-y line
1.0
1.0
1.0
Target Task Accuracy
Target Task Accuracy
Target Task Accuracy
0.6
0.6
0.6
Consistency
0.4
0.4
0.4
0.2
0.2
0.0
0.0.0
0.8.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.8
1.0
0.2
0.4
0.6
0.8
1.0
Anchor Task Accuracy
Anchor Task Accuracy
Anchor Task Accuracy
A. Independent errors
B. Same errors
C. Different errors
Figure 8: Simulation of the relationship between pivot task accuracy and target task accuracy under the scenarios where model makes (a) independent errors in both tasks, (b) same errors in both tasks, and (c) different errors in both tasks.
Table 3: Definition of CocoCon categories and dataset statistics.
#
Unique
Category
Description
Samples
contrast sets
Object
All inanimate objects excluding food items.
388
648
Attribute
Adjectives used modifiers to e.g., color (red chair), height (tall building), (small), material (filed wall), etc. Food items including fruits, vegetables, and other cooked items. Includes mentions animals, predominantly those featured in COCO objects.
314
221
Food
231
409
Animal
177
Location
Includes broadly defined areas (e.g., bathroom, hotel, library), finer visual elements (c.g., floor, sidewalk), and spatial references (c.g., inside, outside, on table).
Role
Includes professional roles such as chef, basehall player, etc. Comprises transitive (e-g. flying kite) as well as intransitive ac- tions (e.g. sitting, standing) performed by persons and animals. Concepts from one woman/ feann//emmle/mmn
109
74
Action
117
Person
the following: man/ maan/male/ae guol//gu.. boy, girl.
47
16
OCR
Texts present. in the image e.g. writing on cake, numbers on digital clock, billboard, etc.
43
132
Misc.
All other minor sub-categories e.g., weather, direction, etc.
55
116
Overall
1500
1820
B.1
Generating Contrast Sets using Multimodal Large Language Models
As note in Sec. work the automated pipeline for generating contrast sets scale precedes recent advances in open-source multimodal large language models (MLLMs) like LLaVA115 (Liu et al., 2023a). Fortunately, the multi-step process outlined in Sec. g can now be replaced with single inference step using state-of-art MLLM as we show below. To compare the diversity of contrast sets generated using our method to those generated using MLLMs, we perform a small experiment using LLaVA 1.5 (Liu et al, 2023a) where we generate contrast set candidates for the examples demonstrated in Fig. by prompting the model to replace semantic concepts within captions directly. We use the following prompt and provide the corresponding image input:
18

---

Published in Transactions on Machine Learning Research (02/2024)
Category
Input Image
Original Instances
Contrast Sets
Caption: surfer is surfing the waves the ocean.
lake
VQA: Where is the man surfing? grean
river
Location
Image Generation: Generate image with the text 'A surfer surfing the waves in ocean."
badminton ping-pong polo
Caption: A gorgeous young lady holding tennis racquet on court. VQA: What sport is this? tennis
Role
Image Generation: Generate an image with the text "A gorgeous young lady holding tennis racquet on a court.
Caption: A man is flying kite on the beach.
woman
VQA; Who is flying the kites? man
girl
Person
Image Generation: Generate an image with the text "A man is flying kite on the beach."
boy
couple
dog
Caption: A large white kitchen range with gold towels the front.
silver
VQA: What color is the stove in this room? white
black
Attribute
Image Generation: Generate an image with the text "A large white kitchen range with gold towels on the front:"
sliver and blue
black
tan
Caption: A cup with three pairs scissors sitting on table. VQA: What is in this cup? scissors
pliers
utensils
Object
Image Generation:
Generate an image with the text "A cup with three pairs of scissors sitting on a table."
pens
Localization: Which region does the text "scissors" describe?
Caption: A person cutting up soccer ball cake.
chocolate cake
VOA: Which one has been nibbled on? soccer ball cake
Food
Image Generation: Generate an image with the text "A person cutting up a soccer ball cake,"
Localization Which region does the text "soccer ball cake" describe?
Caption: A professional tennis player about to serve.
hit ball
Action
VQA: What is the man about to do? serve
swing
Image Generation: Generate an image with the text "A professional tennis player about to serve."
return ball
throw ball
Caption: Three elephants walking along river near a jungle. VQA: What animals are they? elephants
lions
babies
Animal
Image Generation: Generate an image with the text "Three elephants wa walking along a river near jungle."
apes
Localization:
Which region does the text "elephant" describe?
Caption: yellow diamond shape sign indicating duck crossing. VQA What the sign portraying? duck crossing
goose crossing snake crossing
OCR
Image Generation: Generate an image with the text yellow diamond shape sign indicating duck crossing."
Caption: A winter scene of park bench covered in snow among the trees.
summer fall spring
Misc.
VQA: Is it summer winter? winter
Image Generation: Generate image with the text "A winter scene of a park bench covered in snow among the trees.
Figure 9: Additional examples of contrast sets in CocoCon. For each example, show the relevant image (left), the ground truth caption, VQA question, image generation prompt for the image with the perturbed concept. in green (middle) and the set of perturbations used to generale alternative answers.
Based on the image, write different answers for filling the blank the following image caption: [caption with blank placeholder J. For each answer, write your confidence in how well the anawer fits into the caption while staying true to the image.
19

---

Published in Transactions on Machine Learning Research (02/2024)
Table 4: Examples of contrast sets generated using LLaVA1.5 (Liu al., 2023a). Comparison of
contrast sets gencrated using our method in Soc. f to those generated using a single-step inference from the VL1.51.5 model. Concepts replaced in the caption are emphasized. The comparable diversity of both of contrast sets suggests that our multi-step pipeline can be significantly simplified with the use of MLLMs.
CocoCon contrast sets [using VQA)
LLaVA contrast
Original Caption
(using caption)
A child in bed with a striped sweater and a
stuffed animal, pillow, teddy bear
stuffed animal, blanket, pillow, teddy bear
colorful blanket.
A brown and white cat lying on the bed. Appies are hanging from a tree that has hardly any leaves on it.
yellow, black, grey, orange
black, orange, tan, cream
pear, olive, orange, cantaloupe
oranges, pears, peaches, plums
Office space with out on the television and work.
cartoon, squirrel, baseball, butter- fly, bowling
A man holding out to eatch baseball.
tee ball, basket ball, softball, foot- ball walgreens, billboard, state farm, wrigloys, yves saint lauren
frisbee, soccerball, tennis ball, golf ball coca-cola, clocktower, clockface
A giant colgate clock sits the shore next water.
Table Hyperparameters for training OFAcon
Hyperparameter
Value
Proportion ranking updates (+)
0.5
Weight co-efficient of ranking loss (A)
0.25
Regularization strength of soft ranking
Learning rale
1e-6
Max. train epochs
Batch Size
Warmup ratio
0.1
Label smoothing
0.0
The results are presented in Tab. Overall, we observe similar diversity in the contrast sets generated using VQA vs. image caption. This suggests that the multi-step contrast set generation pipeline in Sec. 4 can significantly simplified with the use of MLLMs and suitable prompt engineering, and can be potentially used for extending the CocoCon benchmark to other vision-language tasks in scalable manner.
C Training Hyperparameters
The complete hyperparamcters for training OF using the rank correlation-base based loss objective are available in Tab. 5
D Additional Results
D.1 Calibration of Model Likelihoods for Text-to-image Generation
note that perform temperature calibration (Guo et al. 2017 of Unified-10 to examine whether it improves the calibration of the model likelihoods for outputs from the text-to-image generation task. See results in Fig. 10 We observe that the correlation does not improve significantly with different temperature values. We also compare the distribution of output likelihoods for various tasks and all model sizes of Unified-1O in Fig. [I] While we see different distributions for outputs from pivot instances and contrast instances for the VQA and localization tasks, we observe nearly complete overlap the distributions of outputs for the text-to-image generation task, further supporting our conclusion that unified vision-language models do not have unified semantic backbone across all supported tasks.
In Sec. 7.2
20

---

Published in Transactions on Machine Learning Research (02/2024)
Temperature 0.25
Temperature 0.5
Temperature 1.0
Temperature 2.0
Cumulative frac. Average of dataset CLIPScore
0..11 ASE = 0.002
m..0.112 NSE - 0.002
21 0.110
Po. 0.115.
MSE 0.003
MSE - 0.003
0.5
0.0
0.2
0.4
0.6 0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
Norm. Likelihood
Norm. Likelihood
Norm. Likelihood
Norm. Likelihood
Temperature calibration of Unified-1O for the text-to-image generation task
Figure 10: Temperature calibration (Guo et al., 2017) of the likelihood scores from Unified- I0 SALL model for the text-to-image generation task. We measure the slope (m) and mean squared error (MSE) of the best linear regression fit to the temperaturo-calibratod likelihoods, and compare for different values of lemperature (t). We do not observe significant differences in correlation strength (m) linear fit. (MSE) at values of other than 1, which is the default in evaluation of the CocoCon benchmark.
Image Captioning
VQA
Localization
Text-lo-lmage
Small
pivat contrast
Ba58
0.0
Large
0.0
XL
Model Likelihood
-10
-5
-100
Model Likelihood
-75 -50 -2.
8.0
Model Likelihood
-10
-5
Model Likelihood
-2
10
Distribution of Unified-IO model likelilhoods for different tasks and model sizes
Figure 11: Comparison of distribution of Unified-IO model likelihoods for pivol and contrast in- stances, for various tasks in the CocoCon benchmark and all model sizes.
D.2 Ablation Results & Examples
We present results from the ablation of the weight co-efficient (A) hyperparameter for the consistency-basel loss objective in Tab. 0 We observe that higher À hurts accuracy while lower À does not improve consistency. We also present examples where OFACON is more consistent than the pretrained OFALARGE
in Fig. 12
Table 6: Results from ablation of the weight co-efficient (A) for training of OFACON. Metrics are task-spocific accuracies, % consistency (k = and Spearman's rank correlation cocfficient (Prank). Higher is better for all metrics except FID.
Captioning CIDEr
VQA
Localization
Text-to-Image -maat Gen.
Model
Params
Acc.
C1
Prank
Acc.
C1
FID C₁
Consistency-beet based Training
OFAcoN (A = 0.0)
472M
118.8
82.7
81.1
0.63
73.5
65.9
0.27
98.5
51.7
0.04
OFACON (A 0.25)
472M
119.4
82.4
83.8
0.67
74.1
69.5
0.35
99.1
53.8
0.09
OFACON (A50)
472M
117.8
81.8
84.2
0.70
73.1
69.9
0.35
99.3
54.1
0.08
21

---

Published in Transactions on Machine Learning Research (02/2024)
OFA VIGIL
OFAcON VIG
Category
Input Image
Original Instances
Contrast Sets
Caption: child bed with striped sweater and colorful blanket. VOA: What baby sleeping blanket
stuffed animal pillow
Location
Image Generation: Generate an image with the text 'A child in a bed striped
teddy bear
x
colorful blanket""
Caption: white and grean bus driiing
blue gray tan
36:
33:
VOA: What color is this school bus? grean
Animal
Image Generation: Generate image with the text "A white and green bus driving down a street.
Caption: sitting table putting spoon
spatula
< x
Action
VQA: What the boy holding? spoon
fork
< x x
Image Generation: Generate image with the "A child sitting table putting spoon to a bowl."
scoop
funnel
V
Localization: Which region does the text "spoon" describe?
sponge
Figure 12: Examples from the CocoCon benchmark where OFAcON is more consistent than pretrained OFALARGE For each example, we show the relevant image (left), the ground truth caption, VQA question, or image generation prompt for the image with the perturbed concept in green (middle), the set of perturbations used to generate alternative answers and prodictions from OFALARGE and OFAcoN for VQA (V), image generation (G) and localization (L) (right columns). / and x indicate scenarios where the model predictions for captioning and the corresponding task for that particular contrast set are consistent and inconsistent respectively. '_' denotes a lack of localization annotations for the given sample.
Difficulty of contrast sets
k=2
k=3
k=4
k=5
VQA
00
Localization
Consistency
20
70
OFA
50
Unifled-IO
50-
Kosmos-2
GILL
OF Acon
Accuracy
Accuracy
Accuracy
Accuracy
Comparison of % accuracy with % consistency values for all models evaluated in this paper and our OFAcon model (sce Sec. 5) for decrcasing difficulty of contrast sets (k-2,33,,5). See Fig, 5 b)) for performance on the hardest contrast sets i.e. 1.
Figure 13:
D.3 Consistency vs. Accuracy at Various Difficulty (k)
We compare the target task accuracies of all models evaluated in our experiments with the cross-task consis- tency at difficulty = (C1) in Fig. (b) in the main lext. To further support our analysis, we also present similar plots at decreasing difficulty levels ie. = 2, 4, 5, in Fig. 13 With decreasing difficulty or for casier contrast sets, models gradually transition over the y line which implics that models are generally more consistent than accurate al easier contrast sets.
22


Reviews:
[{"review_id": "36tyRSlPrs", "reviewer": "Reviewer_Wuuv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n### Important Problem\n\n- The paper addresses a significant issue: measuring cross-task consistency in multi-task V&L models.\n- Given the trend of training a single V&L model for various tasks, ensuring consistent behaviour across all tasks is crucial.\n\n### Benchmark\n\n- The CocoCon benchmark utilizes images from the COCO dataset's test set.\n- The data collection involves both automatic and manual filtering steps, ensuring a high-quality benchmark.\n- The benchmark evaluates models across tasks with diverse output modalities, such as text, bounding boxes, and images, making it adaptable to other tasks.\n\n### Extensive Experiments\n\n- The paper conducts experiments on four leading V&L models, highlighting their inconsistencies.\n- The experiments explore factors like model parameters, task nature and count, training task count, and contrast set quality.\n- A rank correlation-based auxiliary training objective is proposed to enhance model consistency across all tasks.\n\n### Presentation\n\n- The paper is articulate and comprehensible.\n- The data collection pipeline is clearly explained.\n\n---\n\n## Weaknesses\n\n### Image Perturbation\n\n- The benchmark primarily uses contrast sets derived from perturbing ground truth textual answers.\n- To assess the model's understanding of images without language bias, the contrast sets could also be generated by perturbing images, as indicated in [1].\n\n### Candidate Answers in Step 2\n\n- Step 2 of the data collection pipeline generates alternatives (candidate answers) solely based on the ground truth answer, limiting contrast set diversity.\n- Creating candidate answers for objects or visual elements mentioned in captions might yield more diverse and challenging contrast sets. The rationale behind relying solely on the ground truth answer needs clarification.\n\n### Comparing Highly Similar Outputs\n\n- How are the confidence scores of highly similar answers like \"hair\", \"her hair\" (id 1441) or \"caution\" \"use caution\" (id 1446) in the dataset compared? In such cases both the answers might be correct but the confidence score will slightly differ which might result in models performing lower on the benchmark when evaluted for cross consistency.\n\n### Consistency between VQA and Captioning\n\n- As the contrast sets are generated based upon the answers should'nt the VQA task be the anchor task? \n- Furthermore, measuring cross consistency between Captioning and VQA seems to make no sense as the input remains the same for VQA (the image and question are unchanged), only the output label (answer in this case) is changed. Hence, if the model assigns a high likelihood to a certain answer correct or incorrect the cross consistency is solely dependent on the captioning. It seems one can evaluate cross consistency of VQA and Captioning based upon some relationship between the accuracies of the respective tasks. More explanation is required to demonstrate how the cross consistency is relevant between VQA and Captioning. The benchmark can either include VQA or Captioning as anchor task and other tasks can remain as such.\n\n### Relationship with Accuracy\n\n- Figure 8 in the appendix illustrates the relationship between consistency and accuracy without specifying the task. Is this relationship computed across all tasks? The details behind its computation remains are not in the paper. A dedicated figure for each pair of tasks would enhance the clarity and interpretation of the relationship.\n\n### Extending to Other Tasks\n\n- The paper could benefit from discussing how the benchmark and evaluation metrics might apply to other V&L tasks, such as referring expressions and visual entailment.\n\n[1] Evaluating models' local decision boundaries via contrast sets.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. The paper proposes a novel benchmark (CocoCon) and evaluation metrics for measuring cross task consistency in vision and language (V&L) models. \n2. It selects image captioning as an anchor task and measures pairwise cross consistency with VQA, Image Generation and object detection.\n3. CocoCon is built upon the COCO test set created by modifying the test instances for multiple tasks in small but semantically meaningful way to create contrast sets that changes the ground truth label. For each image it selects questions that have semantic overlap with the captions and generates contrast sets of the ground truth answers i.e similar but wrong answers. These contrast sets are then substituted in the caption, filtered out and used for evaluating consistency on other tasks. \n4. The paper outline metrics for measuring if a model is consistent by ranking the original and contrast instances across tasks.  \n5. The results shows that although V&L models are accurate they are still inconistent across tasks and models trained on diverse tasks are more inconsistent with larger multi-task models that are more accurate being more consistent.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Mentioned above", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GIderuGgZ9", "reviewer": "Reviewer_9JFw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\tThe paper is well-structured and easy to follow, providing clarity for readers.\n\n\tGiven the surge in recent unified model releases, there is a pressing need for evaluating task-wise consistency, making this paper particularly relevant and timely.\n\n\tThe mechanism for creating the dataset is commendably automated, suggesting the feasibility of generating new contrast sets using other suitable datasets in the future.\n\n\tBy employing the proposed framework, the authors adeptly illuminate the existence of inherent inconsistency in current unified models. The analysis outlines the tendencies displayed by such inconsistencies.\n\n\tA simple yet effective consistency-based loss demonstrates potential improvements in model consistency.\n\n### Weaknesses\n\tThe paper relies heavily on the COCO dataset. For the creation of the proposed contrast sets, it is imperative that captions, localization, and VQA labels exist for the same image. However, datasets fulfilling these criteria are scarce.\n\n\tThere are certain ambiguities in understanding the correlation between accuracy and consistency. More details can be found in the \"Requested Changes\" section below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper offers a comprehensive analysis on the cross-task inconsistency in unified vision-language models. The authors bring attention to the prevalent issue of inconsistency in the existing models (i.e., Unified-IO, OFA), and introduce CocoCon, a benchmark framework employing contrast sets, tailored to measure inconsistency across four multi-modal tasks. The paper showcases the inconsistent behaviors manifested by leading vision-language models and proposes a novel consistency-based training objective, which enhances consistency while preserving model accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### On the correlation between C1 and task accuracy.\n\tAccording to the equations presented in the paper, C1 is a pseudo accuracy of the target task calculated by utilizing the prediction result of the pivot task as the true values. If there is a model that perfectly executes the pivot task, then C1 for each task will be identical with the accuracy of each target task, showing a perfect x=y correlation. This equilibrium between C1 and the target task accuracy starts to distort as the accuracy of the pivot task decreases.\n\n\tIf the pivot task and target task make the same errors, then the model would lie above the x=y line.\n\n\tConversely, if the target task errs a sample that the pivot task does not, then the model would lie below the x=y line.\n\n\tIf both the pivot task and target task err independently, the model would lie around the x=y line.\n\n\tIf the pivot task completely guesses randomly, then the C1 value of the target task would converge to 0.5 regardless of the target accuracy.\n\n\tIn essence, it might be more appropriate to understand that as accuracy increases, consistency also increases. For instance, the claim that an easier contrast set shows better consistency might be because it is naturally easier to discern, leading to a higher accuracy.\n\nTo conclude, further analysis and validation are required based on the assumption that accuracy and consistency naturally correlate.\n\n\n### Observation on text-to-Image generation task\n\tFor the Text-to-Image generation, almost all models, except for the GILL model, show a C1 around 50%. This seems to indicate that for this task, random guess is taking place. While the GILL model does show noteworthy numbers, its performance in all other tasks, including the pivot task, is significantly behind the baseline. This makes it ambiguous if the actual consistency has whether improved or not.\n\n\tThe method employed by the authors to evaluate this task might be questionable. Given the diversity in images, measuring the probability of generating the correct image when given a contrast label is uncertain.\n\nTo conclude, there exist room for debate regarding whether the evaluation method the authors suggest is valid, and thus a deeper exploration is necessary regarding the assessment method employed for the Text-to-Image generation task.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "MDeZQ5HifN", "reviewer": "Reviewer_pVDY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\nThe paper poses an interesting research question, which is how well aligned are model predictions across output spaces in terms of the scores they assign to similar labels. This question is well-motivated (indeed human-computer interaction requires a certain degree of reliability) and increasingly more relevant as more general models are being developed that are capable of multiple different vision tasks. Furthermore:\n\n* The paper is reasonably well written, and includes many visualizations, which are really helpful for giving an intuition of the dataset generation procedure and how evaluation works.\n\n* The design of the dataset is quite innovative, leveraging pre-trained VQA models and language models to help generate contrast sets, while using human evaluating and filtering for quality control. The likelihood-based evaluating using the ground-truth image for t2i eval is quite clever, though it is not quite clear to me how well calibrated model likelihoods are for this task.\n\n* The insight that the considered models are often not consistent is not very surprising, but it is useful to have some quantitative measure of this. There is a small increase in consistency as a function of contrast-difficulty (Figure 5a), which is encouraging to see. Figure 5c, which shows that VQA and Localization consistency (but not T2I performance) improves as a function of model scale is quite interesting as well. \n\n* I certainly expect follow-up works to make use of this dataset.\n\n## Weakenesses\n\n* I don’t quite agree with some of the analysis presented in the paper. For example, it is written that “in k > 1 scenarios, the % consistency increases steeply” in response to Figure 5(a), though only a modest improvement of about 2-3% per step can be observed. Or “Most models feature below the x = y line ” for Figure 5b. While strictly speaking this is certainly true, barring Unified-IO it is more accurate to speak of most models falling “on” the line in my view, as the accuracy is quite a good predictor of consistency, barring a small intercept.\n\n* The scatter plot in Figure 5b is only concerned with k=1, while I would like to see this analysis extended to greater values of k. It seems to be that this data is already available from Figure 5a and it is just a matter of plotting.\n\n* Though the text-to-image setting is cleverly evaluated, it is questionable how well the models considered support this kind of evaluation. In particular, there is close to no sensitivity with regards to difficulty or model size in Figure 5, and the absolute consistency is very low. One explanation is that this task is the hardest and model struggle generally with this. Another, is that this kind of likelihood-based evaluation is ill-suited to evaluate model consistency. In particular, it is unclear to me how well calibrated likelihoods are for these models. Some follow-up analysis, such as looking at the distribution of likelihoods the model assigns in response to captions with substituted concepts could be informative. \n\n\n## Questions\n\n* What is the standard error of these models on this task? Is it safe to compare mean performance across samples or are there significant deviations between samples? Please update to plots to provide an indication of this (Figure 5a, 5c, 6)\n\n* The effect of training with consistency based regularization is quite small, which is unexpected. What is the hypothesis for why this supervised training procedure does not fully close the gap?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes CocoCon, a benchmark dataset for measuring consistency of General Purpose Vision (GPV) model predictions across multiple related tasks. Example tasks include image captioning, visual question answering, localization, and text-to-image generation. CocoCon uses annotations from the validation split of the COCO dataset, and is constructed as follows (see Figure 3 for a nice overview):\n\n1. The dataset is filtered to contain just “pivot instances” by comparing annotations for captioning and VQA. In this way it is ensured that the retained annotations concern the same concept, and thus predictions for them can be compared.\n\n2.  Second, a number of “contrast sets” are created, which provide substitutes for a concept in the annotation. Suggestions for these are obtained from a pre-existing VQA model by considering its answer distribution for the remaining questions for each image.\n\n3. A subsequent filtering step with a language model (T5) ensures that answers are valid completions to the question in terms of grammar, etc.\n\n4. The detection labels for the localization tasks are filtered to retain only those that are present in the caption and corresponding VQA annotations. In this way, the concepts in the contrast set obtained in the previous step can be used as queries for the detection task, Here the models’ likelihood of generating the ground-truth bounding box in response to any of the queries is used for evaluation. Similarly for the generation task, the likelihood of generating the ground-truth image is measure for evaluation in response to an input (having substituted the concept with one from the contrast set).\n\n5. A manual filtering steps was used to remove invalid concepts for contrasting, such as hypernyms. A study with two expert annotators on 200 samples revealed high consistency between their predictions. The manual filtering was only done for test sets, not for training.\n\nThe paper proceeds by evaluating two GPV models on CocoCon: Unified-IO and OFA. Two additional models are considered (Kosmos-2 and GILL), which only support a subset of the four tasks. Evaluation proceeds by comparing the consistency of the predictions of the model across tasks using substitutions from the contrast set. A model is deemed consistent if its preference for a particular label on the anchor task (here captioning) persists on the other tasks. Metrics include “% consistency @ k”, which is the fraction of the tasks the model is consistent for using the kth element of the contrast set (ordered by difficulty), and rank correlation over the ranked elements of the contrast set for two tasks. The latter matters global consistency of the output spaces.\n\nFinally, a regularizer is proposed for training models to be more consistent. It works by using a soft ranker (that can be differentiated through) and a consistency-based loss that measures how well aligned the predictions are between two output spaces (tasks).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I listed a number of questions regarding the analysis that I would like to see addressed in the previous section. Additionally, please consider the following questions for securing my recommendation:\n\n* Could you please add additional examples of the dataset to the appendix, maybe 10 or 20 randomly selected or so? This would be helpful to review the quality of it.\n\n* Please include a discussion of limitations of the proposed benchmark and analysis carried out.  \n\n* What is the process for releasing this dataset, and how will it be licensed?", "broader_impact_concerns": "no concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I agree with the reviewers to accept the paper.

I think the paper provides a very interesting benchmark and evaluation to increase our understanding of multimodal models, by evaluating the consistency of models across different multimodal tasks.
Decision: Accept (poster)