Original: W HAT DOES A PLATYPUS LOOK LIKE ? G ENERATING CUSTOMIZED PROMPTS FOR ZERO - SHOT IMAGE CLASSIFICATION
Revised: W HAT DOES A PLATYPUS LOOK LIKE ? G ENERATING CUSTOMIZED PROMPTS FOR ZERO - SHOT IMAGE CLASSIFICATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Open vocabulary models are a promising new paradigm for image classification.
Revised: Open vocabulary models are a promising new paradigm for image classification.

Original: Unlike traditional classification models, open vocabulary models classify among any arbitrary set of categories specified with natural language during inference.
Revised: Unlike traditional classification models, open vocabulary models classify among any arbitrary set of categories specified with natural language during inference.

Original: This natural language, called “prompts”, typically consists of a set of hand-written templates (e.g., “a photo of a {} ”) which are completed with each of the category names.
Revised: This natural language, called “prompts”, typically consists of a set of hand-written templates (e.g., “a photo of a {} ”) which are completed with each of the category names.

Original: This work introduces a simple method to generate higher accuracy prompts, without relying on any explicit knowledge of the task domain and with far fewer hand-constructed sentences.
Revised: This work introduces a simple method to generate higher accuracy prompts, without relying on any explicit knowledge of the task domain and with far fewer hand-constructed sentences.

Original: To achieve this, we combine open vocabulary models with large language models (LLMs) to create Customized Prompts via Language models (CuPL, pronounced “couple”).
Revised: To achieve this, we combine open vocabulary models with large language models (LLMs) to create Customized Prompts via Language models (CuPL, pronounced “couple”).

Original: In particular, we leverage the knowledge contained in LLMs in order to generate many descriptive sentences that are customized for each object category.
Revised: In particular, we leverage the knowledge contained in LLMs in order to generate many descriptive sentences that are customized for each object category.

Original: We find that this straightforward and general approach improves accuracy on a range of zero-shot image classification benchmarks, including over one percentage point gain on ImageNet.
Revised: We find that this straightforward and general approach improves accuracy on a range of zero-shot image classification benchmarks, including over one percentage point gain on ImageNet.

Original: Finally, this simple baseline requires no additional training and remains completely zero-shot.
Revised: Finally, this simple baseline requires no additional training and remains completely zero-shot.

Original: [Figure]
Revised: [Figure]

Original: Standard Zero-shot
Revised: Standard Zero-shot

Original: Cu stomized P rompts via L anguage models (CuPL)
Revised: Cu stomized P rompts via L anguage models (CuPL)

Original: [Figure]
Revised: [Figure]

Original: method of CuPL.
Revised: method of CuPL.

Original: First, an LLM generates descriptive captions for given class categories.
Revised: First, an LLM generates descriptive captions for given class categories.

Original: Next, an open vocabulary model uses these captions as prompts for performing classification.
Revised: Next, an open vocabulary model uses these captions as prompts for performing classification.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Open vocabulary models (Pham et al., 2021; Jia et al., 2021; Radford et al., 2021; Yu et al., 2022a) achieve high classification accuracy across a large number of datasets without labeled training data for those tasks.
Revised: Open vocabulary models (Pham et al., 2021; Jia et al., 2021; Radford et al., 2021; Yu et al., 2022a) achieve high classification accuracy across a large number of datasets without labeled training data for those tasks.

Original: To accomplish this, these models leverage the massive amounts of image-text pairs available on the internet by learning to associate the images with their correct caption, leading to greater flexibility during inference.
Revised: To accomplish this, these models leverage the massive amounts of image-text pairs available on the internet by learning to associate the images with their correct caption, leading to greater flexibility during inference.

Original: Unlike standard models, these models classify images by providing a similarity score between an image and a caption.
Revised: Unlike standard models, these models classify images by providing a similarity score between an image and a caption.

Original: To perform inference, one can generate a caption or “prompt” associated with each of the desired categories, and match each image to the best prompt.
Revised: To perform inference, one can generate a caption or “prompt” associated with each of the desired categories, and match each image to the best prompt.

Original: This means that categories can be selected ad hoc and adjusted without additional training.
Revised: This means that categories can be selected ad hoc and adjusted without additional training.

Original: However, this new paradigm poses a challenge:
Revised: However, this new paradigm poses a challenge:

Original: How can we best represent an image category through natural language prompts?
Revised: How can we best represent an image category through natural language prompts?

Original: The standard approach is to hand write a number of prompts templates (Radford et al., 2021)
Revised: The standard approach is to hand write a number of prompts templates (Radford et al., 2021)

Original: (e.g.,“a photo of a {} ”), compile a natural language label for each category in the dataset, and create a set of prompts for each category by filling in each of these templates with the natural language labels.
Revised: (e.g.,“a photo of a {} ”), compile a natural language label for each category in the dataset, and create a set of prompts for each category by filling in each of these templates with the natural language labels.

Original: Then, image embeddings are matched to the nearest set of prompt embeddings and labelled with the category associated with that set of prompts (more details in Section 2).
Revised: Then, image embeddings are matched to the nearest set of prompt embeddings and labelled with the category associated with that set of prompts (more details in Section 2).

Original: This method has three major drawbacks.
Revised: This method has three major drawbacks.

Original: Firstly, each prompt template has to be hand-written, so having twice as many prompts for a category requires twice as much human effort.
Revised: Firstly, each prompt template has to be hand-written, so having twice as many prompts for a category requires twice as much human effort.

Original: This can become costly as each new dataset typically has a different set of prompt templates (Radford et al., 2021).
Revised: This can become costly as each new dataset typically has a different set of prompt templates (Radford et al., 2021).

Original: Secondly, the prompt templates must be general enough to apply to all image categories.
Revised: Secondly, the prompt templates must be general enough to apply to all image categories.

Original: For example, a prompt for the ImageNet (Deng et al., 2009) category “platypus” could only be as specific as “a photo of a { platypus } ”, and could not be something like “a photo of a { platypus } , a type of aquatic mammal” as that template would no longer be relevant for other image categories.
Revised: For example, a prompt for the ImageNet (Deng et al., 2009) category “platypus” could only be as specific as “a photo of a { platypus } ”, and could not be something like “a photo of a { platypus } , a type of aquatic mammal” as that template would no longer be relevant for other image categories.

Original: Lastly, writing high performing prompt templates currently requires prior information about the contents of the dataset.
Revised: Lastly, writing high performing prompt templates currently requires prior information about the contents of the dataset.

Original: For example, the list of hand-written ImageNet prompts (Radford et al., 2021) includes “a black and white photo of the {} .”
Revised: For example, the list of hand-written ImageNet prompts (Radford et al., 2021) includes “a black and white photo of the {} .”

Original: , “a low resolution photo of a {} .”, and “a toy {} .”
Revised: , “a low resolution photo of a {} .”, and “a toy {} .”

Original: all of which demonstrate prior knowledge about the type of representations present in the dataset.
Revised: all of which demonstrate prior knowledge about the type of representations present in the dataset.

Original: This information is not generalizable to other datasets, as ImageNet contains “black and white” and “toy” representations of its categories, but other datasets do not (e.g., FVGC Aircraft (Maji et al., 2013)).
Revised: This information is not generalizable to other datasets, as ImageNet contains “black and white” and “toy” representations of its categories, but other datasets do not (e.g., FVGC Aircraft (Maji et al., 2013)).

Original: To overcome these challenges, we propose Customized Prompts via Language models (CuPL).
Revised: To overcome these challenges, we propose Customized Prompts via Language models (CuPL).

Original: In this algorithm, we couple a large language model (LLM) with a zero-shot open vocabulary image classification model.
Revised: In this algorithm, we couple a large language model (LLM) with a zero-shot open vocabulary image classification model.

Original: We use the LLM to generate prompts for each of the image categories in a dataset.
Revised: We use the LLM to generate prompts for each of the image categories in a dataset.

Original: Using an LLM allows us to generate an arbitrary number of prompts with a fixed number of hand-written sentences.
Revised: Using an LLM allows us to generate an arbitrary number of prompts with a fixed number of hand-written sentences.

Original: Additionally, these prompts are now customized to each category and can contain rich visual descriptions while still remaining zero-shot (e.g., “A platypus looks like a beaver with a duck’s bill” – a sentence generated by an LLM).
Revised: Additionally, these prompts are now customized to each category and can contain rich visual descriptions while still remaining zero-shot (e.g., “A platypus looks like a beaver with a duck’s bill” – a sentence generated by an LLM).

Original: We find these customized prompts outperform the hand-written templates on 15 zero-shot image classification benchmarks, including a greater than 1 percentage point gain on ImageNet (Deng et al., 2009)
Revised: We find these customized prompts outperform the hand-written templates on 15 zero-shot image classification benchmarks, including a greater than 1 percentage point gain on ImageNet (Deng et al., 2009)

Original: Top-1 accuracy and a greater than 6 percentage point gain on Describable Textures
Revised: Top-1 accuracy and a greater than 6 percentage point gain on Describable Textures

Original: Dataset (Cimpoi et al., 2014), with fewer hand-written prompts when compared to the standard method used in Radford et al. (2021).
Revised: Dataset (Cimpoi et al., 2014), with fewer hand-written prompts when compared to the standard method used in Radford et al. (2021).

Original: Finally, this method requires no additional training or labeled data for either model.
Revised: Finally, this method requires no additional training or labeled data for either model.

Original:  M ETHODS
Revised:  M ETHODS

Original: The CuPL algorithm consists of two steps: (1) generating customized prompts for each of the categories in a given dataset and (2) using these prompts to perform zero-shot image classification.
Revised: The CuPL algorithm consists of two steps: (1) generating customized prompts for each of the categories in a given dataset and (2) using these prompts to perform zero-shot image classification.

Original: 2. G ENERATING C USTOMIZED P ROMPTS
Revised: 2. G ENERATING C USTOMIZED P ROMPTS

Original: This step consists of generating prompts using an LLM.
Revised: This step consists of generating prompts using an LLM.

Original: For clarity, we distinguish between two different kind of prompts.
Revised: For clarity, we distinguish between two different kind of prompts.

Original: The first are the prompts which cue the LLM to generate the descriptions of the dataset categories.
Revised: The first are the prompts which cue the LLM to generate the descriptions of the dataset categories.

Original: These prompts do not describe an object, but rather prompt the description of an object (e.g., “What does a platypus look like?”).
Revised: These prompts do not describe an object, but rather prompt the description of an object (e.g., “What does a platypus look like?”).

Original: We will refer to these as “LLM-prompts”.
Revised: We will refer to these as “LLM-prompts”.

Original: Secondly, there are the prompts to be matched with images in the zero-shot image classification model.
Revised: Secondly, there are the prompts to be matched with images in the zero-shot image classification model.

Original: These are the prompts that describe a category (e.g., “A platypus looks like ...”).
Revised: These are the prompts that describe a category (e.g., “A platypus looks like ...”).

Original: We call them “image-prompts.”
Revised: We call them “image-prompts.”

Original: These are the output of the LLM, as examplified in Figure 2.
Revised: These are the output of the LLM, as examplified in Figure 2.

Original: In this work, we use GPT-3 (Brown et al., 2020) as our LLM.
Revised: In this work, we use GPT-3 (Brown et al., 2020) as our LLM.

Original: To generate our image-prompts, we must first construct a number of LLM-prompt templates.
Revised: To generate our image-prompts, we must first construct a number of LLM-prompt templates.

Original: While this does require some engineering
Revised: While this does require some engineering

Original: [Figure]
Revised: [Figure]

Original: Viaduct Lorikeet
Revised: Viaduct Lorikeet

Original: Papillon LLM “A lorikeet is a small to medium-sized parrot with a brightly colored plumage.”
Revised: Papillon LLM “A lorikeet is a small to medium-sized parrot with a brightly colored plumage.”

Original: “A marimba is a large wooden percussion instrument that looks like a xylophone.”
Revised: “A marimba is a large wooden percussion instrument that looks like a xylophone.”

Original: “A viaduct is a bridge composed of several spans supported by piers or pillars.”
Revised: “A viaduct is a bridge composed of several spans supported by piers or pillars.”

Original: “A Papillon is a small, spaniel-type dog with a long, silky coat and fringed ears”
Revised: “A Papillon is a small, spaniel-type dog with a long, silky coat and fringed ears”

Original: [Figure]
Revised: [Figure]

Original: by hand, it is significantly less than the amount of hand-engineered sentences used in the standard method of creating image-prompt templates for CLIP.
Revised: by hand, it is significantly less than the amount of hand-engineered sentences used in the standard method of creating image-prompt templates for CLIP.

Original: For example, in our ImageNet experiments, we construct 5 LLM-prompt templates compared to the 80 image-prompts used by CLIP for zeroshot ImageNet classification.
Revised: For example, in our ImageNet experiments, we construct 5 LLM-prompt templates compared to the 80 image-prompts used by CLIP for zeroshot ImageNet classification.

Original: After constructing these LLM-prompts, we generate 10 different image-prompts for each of the
Revised: After constructing these LLM-prompts, we generate 10 different image-prompts for each of the

Original: LLM-prompts.
Revised: LLM-prompts.

Original: This means for ImageNet we use an LLM to generate a total of 50 customized image-prompts for each image category.
Revised: This means for ImageNet we use an LLM to generate a total of 50 customized image-prompts for each image category.

Original: For each of these, we generate a maximum of 50 tokens, but halt a generation early if it produces a period.
Revised: For each of these, we generate a maximum of 50 tokens, but halt a generation early if it produces a period.

Original: Additionally, we generate with a high temperature of 0.99, which encourages more diversity among the 10 generated image-prompts.
Revised: Additionally, we generate with a high temperature of 0.99, which encourages more diversity among the 10 generated image-prompts.

Original: We also clean each generated sentences by deleting any blank lines and adding a period at the end.
Revised: We also clean each generated sentences by deleting any blank lines and adding a period at the end.

Original: 2. U TILIZING C USTOMIZED P ROMPTS
Revised: 2. U TILIZING C USTOMIZED P ROMPTS

Original: After generating image-prompts for each of the categories, we then perform zero-shot image classification.
Revised: After generating image-prompts for each of the categories, we then perform zero-shot image classification.

Original: While there are a number of open vocabulary models (Pham et al., 2021; Jia et al., 2021;
Revised: While there are a number of open vocabulary models (Pham et al., 2021; Jia et al., 2021;

Original: Radford et al., 2021; Yu et al., 2022a), we report our results using CLIP (Radford et al., 2021) as this is the most popular publicly available open vocabulary model.
Revised: Radford et al., 2021; Yu et al., 2022a), we report our results using CLIP (Radford et al., 2021) as this is the most popular publicly available open vocabulary model.

Original: CLIP consists of a text encoder and and image encoder (schematic on the left side of Figure 1).
Revised: CLIP consists of a text encoder and and image encoder (schematic on the left side of Figure 1).

Original: In the standard setting, there are a number of hand-written templates which can be completed with the relevant category names (e.g. “A photo of a {} ”, “A photo of many {} ”).
Revised: In the standard setting, there are a number of hand-written templates which can be completed with the relevant category names (e.g. “A photo of a {} ”, “A photo of many {} ”).

Original: To classify the images in a dataset, each of these templates is filled in with a given category name.
Revised: To classify the images in a dataset, each of these templates is filled in with a given category name.

Original: Then each of these sentences is embedded via the text encoder, and all sentences completed with the same category name are averaged and normalized.
Revised: Then each of these sentences is embedded via the text encoder, and all sentences completed with the same category name are averaged and normalized.

Original: This results in n embeddings where n is the number of categories in the dataset.
Revised: This results in n embeddings where n is the number of categories in the dataset.

Original: Each of these n embeddings is the mean of many different sentence embeddings.
Revised: Each of these n embeddings is the mean of many different sentence embeddings.

Original: Then each image in the dataset is embedded using the image encoder.
Revised: Then each image in the dataset is embedded using the image encoder.

Original: This embedding is compared to each of the n text embeddings using cosine similarity and is labeled with the most similar one.
Revised: This embedding is compared to each of the n text embeddings using cosine similarity and is labeled with the most similar one.

Original: CuPL requires only a small adjustment from this standard practice.
Revised: CuPL requires only a small adjustment from this standard practice.

Original: Instead of filling in the handwritten templates for each category, we simply replace these altogether with the sentences output by
Revised: Instead of filling in the handwritten templates for each category, we simply replace these altogether with the sentences output by

Original: GPT-3.
Revised: GPT-3.

Original: This means for CuPL, hand-written templates are only used as input for the LLM, while the prompts for CLIP are entirely generated text.
Revised: This means for CuPL, hand-written templates are only used as input for the LLM, while the prompts for CLIP are entirely generated text.

Original: We present 2 different setting of CuPL (as shown in
Revised: We present 2 different setting of CuPL (as shown in

Original: Table 1), each representing a different trade-off between accuracy and hand-engineering.
Revised: Table 1), each representing a different trade-off between accuracy and hand-engineering.

Original: CuPL (base) .
Revised: CuPL (base) .

Original: This setting uses three hand-written sentence across all 15 examined datasets.
Revised: This setting uses three hand-written sentence across all 15 examined datasets.

Original: We do this by constructing general LLM-prompt templates which are filled in with the category
Revised: We do this by constructing general LLM-prompt templates which are filled in with the category

Original: names for each dataset.
Revised: names for each dataset.

Original: Our three general templates are as follows:
Revised: Our three general templates are as follows:

Original: Describe what a/the looks like: Describe a/the : What are the identifying characteristics of a/the ?
Revised: Describe what a/the looks like: Describe a/the : What are the identifying characteristics of a/the ?

Original: The blank portion of this template is either filled in with the category type plus the category name (e.g. “pet” + {} for the Oxford Pets dataset (Parkhi et al., 2012) or “aircraft” + {} for FGVC Aircraft
Revised: The blank portion of this template is either filled in with the category type plus the category name (e.g. “pet” + {} for the Oxford Pets dataset (Parkhi et al., 2012) or “aircraft” + {} for FGVC Aircraft

Original: (Maji et al., 2013)) or just the category name for more general datasets like ImageNet (Deng et al., 2009).
Revised: (Maji et al., 2013)) or just the category name for more general datasets like ImageNet (Deng et al., 2009).

Original: Type specification is necessary because of words that have multiple meanings.
Revised: Type specification is necessary because of words that have multiple meanings.

Original: For example “boxer” from the Oxford Pets dataset can also mean a person who boxes, as opposed to a dog breed, so it is necessary to specify “Describe a pet boxer:”.
Revised: For example “boxer” from the Oxford Pets dataset can also mean a person who boxes, as opposed to a dog breed, so it is necessary to specify “Describe a pet boxer:”.

Original: Similarly, “Tornado” from the FGVC Aircraft dataset can be a type of aircraft or a type of weather.
Revised: Similarly, “Tornado” from the FGVC Aircraft dataset can be a type of aircraft or a type of weather.

Original: CuPL (full) .
Revised: CuPL (full) .

Original: In this setting we use different LLM-prompt templates for each dataset, just as
Revised: In this setting we use different LLM-prompt templates for each dataset, just as

Original: Radford et al.
Revised: Radford et al.

Original: (2021) uses different image-prompt templates for each dataset.
Revised: (2021) uses different image-prompt templates for each dataset.

Original: However, we use fewer hand-written templates overall and also contain less specific information about each dataset in the templates.
Revised: However, we use fewer hand-written templates overall and also contain less specific information about each dataset in the templates.

Original: For this work, each dataset has between 2 and 9 LLM-prompts which generate between 20 and 90 image-prompt per category (10 generated sentences per LLM-prompt).
Revised: For this work, each dataset has between 2 and 9 LLM-prompts which generate between 20 and 90 image-prompt per category (10 generated sentences per LLM-prompt).

Original: For
Revised: For

Original: ImageNet, we use the following 5 LLM-prompts: (1) “Describe what a(n) {} looks like”, (2) “How can you identify a(n) {} ?”, (3) “What does a(n) {} look like?”, (4) “A caption of an image of a(n) {} ”, (5) “Describe an image from the internet of a(n) {} ”.
Revised: ImageNet, we use the following 5 LLM-prompts: (1) “Describe what a(n) {} looks like”, (2) “How can you identify a(n) {} ?”, (3) “What does a(n) {} look like?”, (4) “A caption of an image of a(n) {} ”, (5) “Describe an image from the internet of a(n) {} ”.

Original: Example generations for each of these
Revised: Example generations for each of these

Original: LLM-prompts are given for two ImageNet categories in Figure 3.
Revised: LLM-prompts are given for two ImageNet categories in Figure 3.

Original: Full LLM-prompts for all datasets as well as example image-prompts are given in Sections A and D of the Appendix.
Revised: Full LLM-prompts for all datasets as well as example image-prompts are given in Sections A and K of the Appendix.

Original:  E XPERIMENTS AND R ESULTS
Revised:  E XPERIMENTS AND R ESULTS

Original: We first discuss the details of our experimental setup.
Revised: We first discuss the details of our experimental setup.

Original: We next show improvements on a wide range of image classification benchmarks.
Revised: We next show improvements on a wide range of image classification benchmarks.

Original: We then examine the scaling behavior with respect to the model size and report observations regarding hyperparameters such as the LLM sampling temperature.
Revised: We then examine the scaling behavior with respect to the model size and report observations regarding hyperparameters such as the LLM sampling temperature.

Original: Finally, we consider and compare with other methods of obtaining descriptive captions, and provide analysis of CuPl’s improvements over the standard method.
Revised: Finally, we consider and compare with other methods of obtaining descriptive captions, and provide analysis of CuPl’s improvements over the standard method.

Original: 3. S ETUP
Revised: 3. S ETUP

Original: Unless specified otherwise, we use CLIP with a backbone of ViT-L/14 (Dosovitskiy et al., 2020) and the GPT-3 DaVinci-002 model.
Revised: Unless specified otherwise, we use CLIP with a backbone of ViT-L/14 (Dosovitskiy et al., 2020) and the GPT-3 DaVinci-002 model.

Original: Additionally, in order to perform open vocabulary image classification, each image category needs a natural language label.
Revised: Additionally, in order to perform open vocabulary image classification, each image category needs a natural language label.

Original: This is sometimes provided by the dataset, but not always (e.g. ImageNet categories are described by an id number which can map to multiple synonyms).
Revised: This is sometimes provided by the dataset, but not always (e.g. ImageNet categories are described by an id number which can map to multiple synonyms).

Original: For this work, we use the same natural language labels specified in Radford et al.
Revised: For this work, we use the same natural language labels specified in Radford et al.

Original: We report our findings on 15 zero-shot image recognition benchmarks: ImageNet (Deng et al., 2009), Describable Textures Dataset (DTD) (Cimpoi et al., 2014), Stanford Cars (Krause et al., 2013), Scene UNderstanding (SUN397) (Xiao et al., 2010), Food101 (Bossard et al., 2014), FGVC
Revised: We report our findings on 15 zero-shot image recognition benchmarks: ImageNet (Deng et al., 2009), Describable Textures Dataset (DTD) (Cimpoi et al., 2014), Stanford Cars (Krause et al., 2013), Scene UNderstanding (SUN397) (Xiao et al., 2010), Food101 (Bossard et al., 2014), FGVC

Original: Aircraft (Maji et al., 2013), Oxford Pets (Parkhi et al., 2012), Caltech101 (Fei-Fei et al., 2004),
Revised: Aircraft (Maji et al., 2013), Oxford Pets (Parkhi et al., 2012), Caltech101 (Fei-Fei et al., 2004),

Original: Slide rules are rectangular, and have a number of lines and markings of di ﬀ erent lengths on them.
Revised: Slide rules are rectangular, and have a number of lines and markings of di ﬀ erent lengths on them.

Original: Slide rules have a linear scale on one edge of the rule and a logarithmic scale on the other edge.
Revised: Slide rules have a linear scale on one edge of the rule and a logarithmic scale on the other edge.

Original: A slide rule is a straight ruler with two scales that can slide past one another.
Revised: A slide rule is a straight ruler with two scales that can slide past one another.

Original: A slide rule is a mechanical analog computer.
Revised: A slide rule is a mechanical analog computer.

Original: This slide rule belongs to my grandfather.
Revised: This slide rule belongs to my grandfather.

Original: The platypus is an odd-looking animal with a duck-like bill, beaver-like tail, and otter-like body.
Revised: The platypus is an odd-looking animal with a duck-like bill, beaver-like tail, and otter-like body.

Original: The easiest way to identify a platypus is by its beak, which is wide and ﬂat like a duck’s.
Revised: The easiest way to identify a platypus is by its beak, which is wide and ﬂat like a duck’s.

Original: A platypus looks like a beaver that has a duck bill.
Revised: A platypus looks like a beaver that has a duck bill.

Original: The image shows a platypus swimming in a body of water.
Revised: The image shows a platypus swimming in a body of water.

Original: This adorable creature is a platypus, a unique mammal found in
Revised: This adorable creature is a platypus, a unique mammal found in

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Flowers 102 (Nilsback & Zisserman, 2008), UCF101 (Soomro et al., 2012), Kinetics-700 (Carreira et al., 2019), Remote Sensing Image Scene Classification (RESISC45) (Cheng et al., 2017), CIFAR10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), and Birdsnap (Berg et al., 2014).
Revised: Flowers 102 (Nilsback & Zisserman, 2008), UCF101 (Soomro et al., 2012), Kinetics-700 (Carreira et al., 2019), Remote Sensing Image Scene Classification (RESISC45) (Cheng et al., 2017), CIFAR10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009), and Birdsnap (Berg et al., 2014).

Original: For the two video datasets, we extract the middle frame of the video, as is done in Radford et al.
Revised: For the two video datasets, we extract the middle frame of the video, as is done in Radford et al.

Original: 3. R ESULTS
Revised: 3. R ESULTS

Original: Our results for the base prompts setting and the full prompts setting are in Table 1.
Revised: Our results for the base prompts setting and the full prompts setting are in Table 1.

Original: We present our method’s performance on 15 different image classification benchmarks, comparing both the classification accuracy and the number of hand-written sentence templates needed for each method.
Revised: We present our method’s performance on 15 different image classification benchmarks, comparing both the classification accuracy and the number of hand-written sentence templates needed for each method.

Original: Note that for the standard method (Radford et al., 2021), the hand-written sentences refer to the image-prompts, while for CuPL the hand-written sentences refer to the LLM-prompts, with which image-prompts are generated.
Revised: Note that for the standard method (Radford et al., 2021), the hand-written sentences refer to the image-prompts, while for CuPL the hand-written sentences refer to the LLM-prompts, with which image-prompts are generated.

Original: CuPL (base).
Revised: CuPL (base).

Original: In this setting, we see performance gains in 13 out of the 15 examined datasets.
Revised: In this setting, we see performance gains in 13 out of the 15 examined datasets.

Original: Note
Revised: Note

Original: this setting uses just three hand-constructed sentence across all datasets .
Revised: this setting uses just three hand-constructed sentence across all datasets .

Original: This is in comparison to the nearly 175 unique image-prompt templates that are hand-written across all of these datasets in the standard setting.
Revised: This is in comparison to the nearly 175 unique image-prompt templates that are hand-written across all of these datasets in the standard setting.

Original: Additionally, in the standard setting these hand-constructed prompts must be very specific to the dataset (e.g., “a black and white photo of a {} .”
Revised: Additionally, in the standard setting these hand-constructed prompts must be very specific to the dataset (e.g., “a black and white photo of a {} .”

Original: , “a plastic {} .”).
Revised: , “a plastic {} .”).

Original: In comparison,
Revised: In comparison,

Original: CuPL (base) requires only the category type of the overall dataset and still outperforms the handwritten, domain specified baseline in almost all cases.
Revised: CuPL (base) requires only the category type of the overall dataset and still outperforms the handwritten, domain specified baseline in almost all cases.

Original: Thus, we present this base prompt setting as a simple standard that matches or exceeds prompt engineering open vocabulary models.
Revised: Thus, we present this base prompt setting as a simple standard that matches or exceeds prompt engineering open vocabulary models.

Original: CuPL (full prompts).
Revised: CuPL (full prompts).

Original: Here we see improvements on all examined datasets.
Revised: Here we see improvements on all examined datasets.

Original: This includes large (over 1 percentage point) gains on ImageNet Top-1, DTD (texture classification), SUN (scene classification), FGVC Aircraft (fine-grained aircraft classification), and Flowers 102 (flower classification).
Revised: This includes large (over 1 percentage point) gains on ImageNet Top-1, DTD (texture classification), SUN (scene classification), FGVC Aircraft (fine-grained aircraft classification), and Flowers 102 (flower classification).

Original: While this setting requires more hand-written prompts than setting (1), it still requires significantly fewer than the baseline method (5 sentences versus 80 sentence for ImageNet), and does not include knowledge about the image domain.
Revised: While this setting requires more hand-written prompts than setting (1), it still requires significantly fewer than the baseline method (5 sentences versus 80 sentence for ImageNet), and does not include knowledge about the image domain.

Original: The full list of hand-constructed sentences for
Revised: The full list of hand-constructed sentences for

Original: CuPL (full prompts) and the baseline method (Radford et al., 2021) can be found in Section A of the Appendix.
Revised: CuPL (full prompts) and the baseline method (Radford et al., 2021) can be found in Section A of the Appendix.

Original: 3. A NALYSIS AND A BLATIONS
Revised: 3. A NALYSIS AND A BLATIONS

Original: Model Size.
Revised: Model Size.

Original: In Figure 4, we show CuPL (full prompts) at different model scales.
Revised: In Figure 4, we show CuPL (full prompts) at different model scales.

Original: As there are two different zero-shot models in the CuPL algorithm, we show the effects of varying each model individually.
Revised: As there are two different zero-shot models in the CuPL algorithm, we show the effects of varying each model individually.

Original: On the left hand side, we vary the CLIP model used while holding the LLM constant.
Revised: On the left hand side, we vary the CLIP model used while holding the LLM constant.

Original: We see consistent gains across all model sizes.
Revised: We see consistent gains across all model sizes.

Original: On the right hand side, we vary the size of the LLM.
Revised: On the right hand side, we vary the size of the LLM.

Original: We plot the accuracy of the baseline as well, which does not vary as it does not utilize an LLM.
Revised: We plot the accuracy of the baseline as well, which does not vary as it does not utilize an LLM.

Original: We find larger models lead to higher accuracy, though the 2nd and 3rd largest models perform similarly.
Revised: We find larger models lead to higher accuracy, though the 2nd and 3rd largest models perform similarly.

Original: Number of Prompts.
Revised: Number of Prompts.

Original: In Figure 6, we present ablations on the number of LLM-prompts and image-prompts for CuPL (full prompts).
Revised: In Figure 6, we present ablations on the number of LLM-prompts and image-prompts for CuPL (full prompts).

Original: On the left side, we show ImageNet accuracy as we increase the number of LLM-prompts.
Revised: On the left side, we show ImageNet accuracy as we increase the number of LLM-prompts.

Original: This also corresponds to the number of sentences that have to be hand-written.
Revised: This also corresponds to the number of sentences that have to be hand-written.

Original: Notably, this methods outperforms the baseline even when using prompts generated from a single handwritten sentence.
Revised: Notably, this methods outperforms the baseline even when using prompts generated from a single handwritten sentence.

Original: On the right hand side, we hold the number of LLM-prompts constant atand adjust how many image-prompts we generate per LLM-prompt.
Revised: On the right hand side, we hold the number of LLM-prompts constant atand adjust how many image-prompts we generate per LLM-prompt.

Original: We plot the accuracy given the total number of image-prompts (sogenerated image-prompt per LLM-prompt corresponds to 50 total image-prompts).
Revised: We plot the accuracy given the total number of image-prompts (sogenerated image-prompt per LLM-prompt corresponds to 50 total image-prompts).

Original: We see that CuPL begins to outperform the baseline at just 25 image-prompts, well below the 80 imageprompts used in the baseline.
Revised: We see that CuPL begins to outperform the baseline at just 25 image-prompts, well below the 80 imageprompts used in the baseline.

Original: Diversity of Prompts.
Revised: Diversity of Prompts.

Original: We also examine the impact of the diversity of image-prompts on ImageNet accuracy.
Revised: We also examine the impact of the diversity of image-prompts on ImageNet accuracy.

Original: We adjust this parameter by changing the temperature of the GPT-3 model.
Revised: We adjust this parameter by changing the temperature of the GPT-3 model.

Original: This value changes the likelihood of selecting lower probability tokens and makes sentences more diverse from each other.
Revised: This value changes the likelihood of selecting lower probability tokens and makes sentences more diverse from each other.

Original: As demonstrated in Figure 5, more diverse prompts lead to higher ImageNet accuracy.
Revised: As demonstrated in Figure 5, more diverse prompts lead to higher ImageNet accuracy.

Original: Note these comparisons are done with a single LLM-prompt to save computational cost.
Revised: Note these comparisons are done with a single LLM-prompt to save computational cost.

Original: WordNet Definitions and Wikipedia Descriptions.
Revised: WordNet Definitions and Wikipedia Descriptions.

Original: We also consider two additional methods of obtaining descriptive sentences for each ImageNet category, other than using an LLM.
Revised: We also consider two additional methods of obtaining descriptive sentences for each ImageNet category, other than using an LLM.

Original: Firstly, we compare CuPL (full) image-prompts with image-prompts generated using definitions of each ImageNet category.
Revised: Firstly, we compare CuPL (full) image-prompts with image-prompts generated using definitions of each ImageNet category.

Original: Because each ImageNet category is derived from the WordNet database (Miller, 1995), we can use the WordNet definition of each word.
Revised: Because each ImageNet category is derived from the WordNet database (Miller, 1995), we can use the WordNet definition of each word.

Original: We preprocess these definitions so they are of the form “A(n) {} is a ...” as not all WordNet definitions contain the name of the word itself.
Revised: We preprocess these definitions so they are of the form “A(n) {} is a ...” as not all WordNet definitions contain the name of the word itself.

Original: We also add a period to the end of each definition, as we find this increases performance.
Revised: We also add a period to the end of each definition, as we find this increases performance.

Original: As shown in Table 2, ImageNet Top-1 accuracy with WordNet definition prompts is below that of CuPL or standard prompts.
Revised: As shown in Table 2, ImageNet Top-1 accuracy with WordNet definition prompts is below that of CuPL or standard prompts.

Original: In addition to lower accuracy, this method uses significantly more hand-constructed sentences as it requires 1000 unique hand-written definitions compared to 175 unique hand-written image-prompt templates for the standard method and 45 unique hand-written LLM-prompt templates for CuPL (full).
Revised: In addition to lower accuracy, this method uses significantly more hand-constructed sentences as it requires 1000 unique hand-written definitions compared to 175 unique hand-written image-prompt templates for the standard method and 45 unique hand-written LLM-prompt templates for CuPL (full).

Original: [Table]
Revised: [Table]

Original: Secondly, we compare against prompts generated from Wikipedia articles corresponding to each ImageNet category, as collected in Bujwid & Sullivan (2021b).
Revised: Secondly, we compare against prompts generated from Wikipedia articles corresponding to each ImageNet category, as collected in Bujwid & Sullivan (2021b).

Original: Note that the Wikipedia article does not always exactly match the natural language name of the class used by Radford et al.
Revised: Note that the Wikipedia article does not always exactly match the natural language name of the class used by Radford et al.

Original: Additionally, 80 categories map to more than one Wikipedia article (e.g. the category associated with the natural language word “patio” is mapped to the articles for “patio” and “terrace”).
Revised: Additionally, 80 categories map to more than one Wikipedia article (e.g. the category associated with the natural language word “patio” is mapped to the articles for “patio” and “terrace”).

Original: In this case, we select the first associated article.
Revised: In this case, we select the first associated article.

Original: We preprocess these by removing the first line (the name of the article), and then extracting the first sentence, including the final period.
Revised: We preprocess these by removing the first line (the name of the article), and then extracting the first sentence, including the final period.

Original: We find both of these preprocessing steps lead to increase in accuracy.
Revised: We find both of these preprocessing steps lead to increase in accuracy.

Original: We also truncate this sentence to the maximum allowed input length of CLIP.
Revised: We also truncate this sentence to the maximum allowed input length of CLIP.

Original: For the 24 ImageNet categories that do not have an associated Wikipedia page, we use the name of the category as the image-prompts.
Revised: For the 24 ImageNet categories that do not have an associated Wikipedia page, we use the name of the category as the image-prompts.

Original: As shown in Table 2, we find Wikipedia to be less effective than standard prompts, CuPL prompts, or
Revised: As shown in Table 2, we find Wikipedia to be less effective than standard prompts, CuPL prompts, or

Original: WordNet definitions.
Revised: WordNet definitions.

Original: Ensembling with Standard Prompts.
Revised: Ensembling with Standard Prompts.

Original: We also consider using LLM generated prompts from CuPL (full) in addition to hand-written prompts.
Revised: We also consider using LLM generated prompts from CuPL (full) in addition to hand-written prompts.

Original: We do this by averaging together all the text embeddings of the CuPL prompts and hand-written prompts.
Revised: We do this by averaging together all the text embeddings of the CuPL prompts and hand-written prompts.

Original: As shown in Table 3, we find that for some datasets, ensembling both types of prompts outperforms CuPL prompts on their own, while for others CuPL prompts perform better.
Revised: As shown in Table 3, we find that for some datasets, ensembling both types of prompts outperforms CuPL prompts on their own, while for others CuPL prompts perform better.

Original: For all datasets, this ensemble performs better than standard prompts alone.
Revised: For all datasets, this ensemble performs better than standard prompts alone.

Original: However, this ensembling method requires all the hand-written effort and domain knowledge of the standard approach.
Revised: However, this ensembling method requires all the hand-written effort and domain knowledge of the standard approach.

Original: Analysis of Accuracy Gains.
Revised: Analysis of Accuracy Gains.

Original: In addition to total accuracy gains, we present the per class accuracy shift between the image-prompts used in Radford et al.
Revised: In addition to total accuracy gains, we present the per class accuracy shift between the image-prompts used in Radford et al.

Original: (2021) and CuPL, shown in Figure 7.
Revised: (2021) and CuPL, shown in Figure 7.

Original: As demonstrated, the accuracy gains seen in CuPL are not distributed uniformly through the ImageNet classes, with some classes seeing ∼ 40 percentage point accuracy gains, and others seeing ∼percentage point accuracy losses when compared against class accuracy with standard prompts.
Revised: As demonstrated, the accuracy gains seen in CuPL are not distributed uniformly through the ImageNet classes, with some classes seeing ∼ 40 percentage point accuracy gains, and others seeing ∼percentage point accuracy losses when compared against class accuracy with standard prompts.

Original: In other words, while CuPL sees a higher accuracy overall when compared to the standard method, the images which are correctly predicted by the standard prompts are not a subset of the images  which are correctly predicted by CuPL.
Revised: In other words, while CuPL sees a higher accuracy overall when compared to the standard method, the images which are correctly predicted by the standard prompts are not a subset of the images [Table] which are correctly predicted by CuPL.

Original: In fact CuPL sees just over a 1 percentage point gain when 
Revised: In fact CuPL sees just over a 1 percentage point gain when compared to standard prompts

Original: [Table]
Revised: [Table]

Original: compared to standard prompts but differs in it’s predictions from the standard method for 11.50% of predictions (with CuPL correct for 4.48% of these, standard correct for 3.32%, and neither correct for 3.70%).
Revised: but differs in it’s predictions from the standard method for 11.50% of predictions (with CuPL correct for 4.48% of these, standard correct for 3.32%, and neither correct for 3.70%).

Original: Figure 7 also shows the classes with the 20 greatest accuracy gains and losses when comparing class accuracy with standard image-prompts (Radford et al., 2021) versus with CuPL image-prompts.
Revised: Figure 7 also shows the classes with the 20 greatest accuracy gains and losses when comparing class accuracy with standard image-prompts (Radford et al., 2021) versus with CuPL image-prompts.

Original: Interestingly, for many of the classes which see a large accuracy gain, we see a corresponding class with a large accuracy loss that is either similar to the initial class or likely to co-occur with it (e.g. agaric/mushroom, academic gown/graduation cap, military uniform/Pickelhaube, desk/monitor).
Revised: Interestingly, for many of the classes which see a large accuracy gain, we see a corresponding class with a large accuracy loss that is either similar to the initial class or likely to co-occur with it (e.g. agaric/mushroom, academic gown/graduation cap, military uniform/Pickelhaube, desk/monitor).

Original:  R ELATED W ORK 4. N ATURAL L ANGUAGE D ESCRIPTIONS FOR I MAGE C LASSIFICATION
Revised:  R ELATED W ORK 4. N ATURAL L ANGUAGE D ESCRIPTIONS FOR I MAGE C LASSIFICATION

Original: Several prior works use text-based knowledge of image categories to improve classification accuracy.
Revised: Several prior works use text-based knowledge of image categories to improve classification accuracy.

Original: Elhoseiny et al.
Revised: Elhoseiny et al.

Original: (2017) extract visual information from unstructured text descriptions collected from the internet to recognize parts of object and classify them in a zero-shot way.
Revised: (2017) extract visual information from unstructured text descriptions collected from the internet to recognize parts of object and classify them in a zero-shot way.

Original: Reed et al.
Revised: Reed et al.

Original: (2016) and He & Peng (2017)
Revised: (2016) and He & Peng (2017)

Original: use natural language descriptions of bird types to train a multimodal classification model.
Revised: use natural language descriptions of bird types to train a multimodal classification model.

Original: Huang et al.
Revised: Huang et al.

Original: (2021) use hand-collected attribute tags to attend over relevant features in images.
Revised: (2021) use hand-collected attribute tags to attend over relevant features in images.

Original: Paz-Argaman et al.
Revised: Paz-Argaman et al.

Original: (2020) extract visual information from Wikipedia descriptions to enable zero-shot bird classification.
Revised: (2020) extract visual information from Wikipedia descriptions to enable zero-shot bird classification.

Original: Additional works (Shen et al., 2022; Bujwid & Sullivan, 2021a) show improvements on large datasets (e.g., ImageNet) using external information from external databases such as Imagenet-wiki and Wordnet.
Revised: Additional works (Shen et al., 2022; Bujwid & Sullivan, 2021a) show improvements on large datasets (e.g., ImageNet) using external information from external databases such as Imagenet-wiki and Wordnet.

Original: While these works show the effectiveness of augmenting zero-shot models with descriptive text, all of these prior works rely on external natural language databases for descriptions.
Revised: While these works show the effectiveness of augmenting zero-shot models with descriptive text, all of these prior works rely on external natural language databases for descriptions.

Original: This often limits the possible categories that can be classified and can require extensive preprocessing to extract visual descriptions from noisy natural language.
Revised: This often limits the possible categories that can be classified and can require extensive preprocessing to extract visual descriptions from noisy natural language.

Original: 4. G ENERATED T EXT FOR D OWNSTREAM T ASKS
Revised: 4. G ENERATED T EXT FOR D OWNSTREAM T ASKS

Original: Recent work has utilized text generated from LLMs in a number of ways.
Revised: Recent work has utilized text generated from LLMs in a number of ways.

Original: Santurkar et al.
Revised: Santurkar et al.

Original: (2022) use an LLM to paraphrase existing image captions to use as data augmentation for CLIP.
Revised: (2022) use an LLM to paraphrase existing image captions to use as data augmentation for CLIP.

Original: Liu et al.
Revised: Liu et al.

Original: (2022) use GPT-3 to generate knowledge on a topic when given a number of demonstrations, which is then used to improve accuracy on common sense reasoning questions.
Revised: (2022) use GPT-3 to generate knowledge on a topic when given a number of demonstrations, which is then used to improve accuracy on common sense reasoning questions.

Original: Hu et al.
Revised: Hu et al.

Original: (2022) use a LLM to add labels to text to improve text classification accuracy.
Revised: (2022) use a LLM to add labels to text to improve text classification accuracy.

Original: In Yu et al.
Revised: In Yu et al.

Original: (2022b), the outputs of a
Revised: (2022b), the outputs of a

Original: GPT-2 model are used to train an encoder on top of a vision model to generate multimodal image representations for a variety of tasks.
Revised: GPT-2 model are used to train an encoder on top of a vision model to generate multimodal image representations for a variety of tasks.

Original: Su et al.
Revised: Su et al.

Original: (2022) utilize a language model to perform image captioning by iteritively generating candidate image captions with a LLM and then using feedback from an open vocabulary model to align it to a given image.
Revised: (2022) utilize a language model to perform image captioning by iteritively generating candidate image captions with a LLM and then using feedback from an open vocabulary model to align it to a given image.

Original: Similarly, Yang et al.
Revised: Similarly, Yang et al.

Original: (2022) use GPTalong with text descriptions of images for the Visual Question Answering (VQA) task.
Revised: (2022) use GPTalong with text descriptions of images for the Visual Question Answering (VQA) task.

Original: However, unlike CuPL these prior works are either purely language tasks (common sense reasoning, text classification) or multimodal with some language component (image captioning, VQA).
Revised: However, unlike CuPL these prior works are either purely language tasks (common sense reasoning, text classification) or multimodal with some language component (image captioning, VQA).

Original: In our work, we demonstrate how LLM generated text can be used to improve purely visual image classification tasks across a number of benchmarks.
Revised: In our work, we demonstrate how LLM generated text can be used to improve purely visual image classification tasks across a number of benchmarks.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 4. P ROMPT E NGINEERING
Revised: 4. P ROMPT E NGINEERING

Original: Previous efforts have explored methods for obtaining successful natural language prompts.
Revised: Previous efforts have explored methods for obtaining successful natural language prompts.

Original: For both open vocabulary image classification models as well as LLMs, the format of prompts is known to highly affect accuracy (Schick & Sch ¨ utze, 2021;
Revised: For both open vocabulary image classification models as well as LLMs, the format of prompts is known to highly affect accuracy (Schick & Sch ¨ utze, 2021;

Original: Radford et al., 2021; Brown et al., 2020; Gao et al., 2020).
Revised: Radford et al., 2021; Brown et al., 2020; Gao et al., 2020).

Original: This has led to a large effort to find optimal prompt formats.
Revised: This has led to a large effort to find optimal prompt formats.

Original: Proposed methods include crowd-sourcing high performing prompts (Bach et al., 2022) as well as framing prompts to induce models to give explanations as well as answers (Wei et al., 2022; Kojima et al., 2022; Nye et al., 2021).
Revised: Proposed methods include crowd-sourcing high performing prompts (Bach et al., 2022) as well as framing prompts to induce models to give explanations as well as answers (Wei et al., 2022; Kojima et al., 2022; Nye et al., 2021).

Original: Additional works have proposed learning prompts via gradient based methods (Zhang et al., 2021;
Revised: Additional works have proposed learning prompts via gradient based methods (Zhang et al., 2021;

Original: Qin & Eisner, 2021; Li & Liang, 2021; Lester et al., 2021; Shin et al., 2020), retrieval from a database (Rubin et al., 2022), or reformatting/rephrasing existing prompts (Jiang et al., 2020;
Revised: Qin & Eisner, 2021; Li & Liang, 2021; Lester et al., 2021; Shin et al., 2020), retrieval from a database (Rubin et al., 2022), or reformatting/rephrasing existing prompts (Jiang et al., 2020;

Original: Rubin et al., 2022).
Revised: Rubin et al., 2022).

Original: Most relevant to this work are a number of methods for designing optimal prompts for zero-shot image classification with open vocabulary models.
Revised: Most relevant to this work are a number of methods for designing optimal prompts for zero-shot image classification with open vocabulary models.

Original: These methods learn prompts formats which yield high accuracy for image classification using either supervised (Zhou et al., 2022; Rao et al., 2022) or unsupervised (Huang et al., 2022) methods.
Revised: These methods learn prompts formats which yield high accuracy for image classification using either supervised (Zhou et al., 2022; Rao et al., 2022) or unsupervised (Huang et al., 2022) methods.

Original: However, unlike these prior works this work requires no additional training or labeled data.
Revised: However, unlike these prior works this work requires no additional training or labeled data.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We demonstrate that leveraging knowledge from an LLM can immediately improve zero-shot accuracy on a variety of image classification tasks, with much less hand-engineering efforts to craft natural language prompts.
Revised: We demonstrate that leveraging knowledge from an LLM can immediately improve zero-shot accuracy on a variety of image classification tasks, with much less hand-engineering efforts to craft natural language prompts.

Original: Furthermore, prompts can be customized to the desired categories, rather than a general template that applies to all existing image categories.
Revised: Furthermore, prompts can be customized to the desired categories, rather than a general template that applies to all existing image categories.

Original: Finally, using prompts generated by LLMs lowers the barrier of prior knowledge about the dataset, which is often required when crafting prompt templates.
Revised: Finally, using prompts generated by LLMs lowers the barrier of prior knowledge about the dataset, which is often required when crafting prompt templates.

Original: Querying an LLM for prompt construction is simple, straightforward and as our results suggested, immediately beneficial.
Revised: Querying an LLM for prompt construction is simple, straightforward and as our results suggested, immediately beneficial.

Original: The hypothesis that a joint force of LLMs and open vocabulary models would improve zero-shot image classification is thoroughly tested in this work.
Revised: The hypothesis that a joint force of LLMs and open vocabulary models would improve zero-shot image classification is thoroughly tested in this work.

Original: We hope these findings serve as a useful tool towards understanding and improving zero-shot image classification, and more generally, the consolidation of model capacities and modalities through natural language.
Revised: We hope these findings serve as a useful tool towards understanding and improving zero-shot image classification, and more generally, the consolidation of model capacities and modalities through natural language.

Original:  R EPRODUCIBILITY
Revised:  R EPRODUCIBILITY

Original: We have a number of measures to ensure the reproducibility of this work.
Revised: We have a number of measures to ensure the reproducibility of this work.

Original: First, in the supplementary material we include the code to generate image-prompts for ImageNet and evaluate the accuracy of these prompts.
Revised: First, in the supplementary material we include the code to generate image-prompts for ImageNet and evaluate the accuracy of these prompts.

Original: In Section 2, we note all hyperparameters used for the LLM.
Revised: In Section 2, we note all hyperparameters used for the LLM.

Original: Additionally, in the appendix we include all LLM-prompts used to generate image-prompts for each of the 15 datasets.
Revised: Additionally, in the appendix we include all LLM-prompts used to generate image-prompts for each of the 15 datasets.

Original: In the supplementary material, we include all generated image-prompts for all dataset, for both
Revised: In the supplementary material, we include all generated image-prompts for all dataset, for both

Original: CuPL (base) and CuPL (full).
Revised: CuPL (base) and CuPL (full).

Original: O VERVIEW
Revised: O VERVIEW

Original:  A C U PL (F ULL P ROMPTS ) VS S TANDARD P ROMPTS
Revised: A. CuPL (Full Prompts) vs Standard Prompts B. CuPL Base Prompts C. Evaluation Metric D. Open-Source LLM E. Single Sentence Baseline F. Robustness G. CuPL Improvement Analysis H. Error Analysis I. Image-Prompt Distribution J. Tempurature Analysis K. Example Generated image-prompts A C U PL (F ULL P ROMPTS ) VS S TANDARD P ROMPTS

Original: We detail the hand-written prompt templates used for CuPL (full prompts) versus standard CLIP
Revised: We detail the hand-written prompt templates used for CuPL (full prompts) versus standard CLIP

Original: Radford et al.
Revised: Radford et al.

Original: (2021) prompt templates.
Revised: (2021) prompt templates.

Original: For CuPL, hand-written prompt templates are needed for the LLM-prompts, while for the standard method hand-written prompt templates are needed for the image-prompts.
Revised: For CuPL, hand-written prompt templates are needed for the LLM-prompts, while for the standard method hand-written prompt templates are needed for the image-prompts.

Original: Note that many of the hand-written templates for the standard method encode information about the datasets.
Revised: Note that many of the hand-written templates for the standard method encode information about the datasets.

Original: For example, ”a toy {} ” demonstrates knowledge that objects are sometimes represented as a toy version of an object rather than as the literal object.
Revised: For example, ”a toy {} ” demonstrates knowledge that objects are sometimes represented as a toy version of an object rather than as the literal object.

Original: CuPL prompts remain much more general (e.g. ”Describe what a {} looks like”).
Revised: CuPL prompts remain much more general (e.g. ”Describe what a {} looks like”).

Original: Caltech
Revised: Caltech

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe what a(n) {} looks like: a photo of a {} .
Revised: Describe what a(n) {} looks like: a photo of a {} .

Original: a photo of the {} .
Revised: a photo of the {} .

Original: Describe a(n) {} : a painting of a {} .
Revised: Describe a(n) {} : a painting of a {} .

Original: a painting of the {} .
Revised: a painting of the {} .

Original: What are the identifying characteristics of a(n) {} ?
Revised: What are the identifying characteristics of a(n) {} ?

Original: a plastic {} . the plastic {} .
Revised: a plastic {} . the plastic {} .

Original: a sculpture of a {} . a sculpture of the {} .
Revised: a sculpture of a {} . a sculpture of the {} .

Original: a sketch of a {} . a sketch of the {} .
Revised: a sketch of a {} . a sketch of the {} .

Original: a tattoo of a {} . a tattoo of the {} .
Revised: a tattoo of a {} . a tattoo of the {} .

Original: a toy {} . the toy {} .
Revised: a toy {} . the toy {} .

Original: a rendition of a {} . a rendition of the {} .
Revised: a rendition of a {} . a rendition of the {} .

Original: a embroidered {} . the embroidered {} .
Revised: a embroidered {} . the embroidered {} .

Original: a cartoon {} . the cartoon {} .
Revised: a cartoon {} . the cartoon {} .

Original: a {} in a video game. the {} in a video game.
Revised: a {} in a video game. the {} in a video game.

Original: a plushie {} . the plushie {} .
Revised: a plushie {} . the plushie {} .

Original: a origami {} . the origami {} .
Revised: a origami {} . the origami {} .

Original: art of a {} . 
Revised: art of a {} . art of the {} . graffiti of a {} . graffiti of the {} .

Original: [Table]
Revised: a drawing of a {} . a drawing of the {} . a doodle of a {} . a doodle of the {} .

Original: Food 
Revised: Food

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe what {} looks like a photo of {} , a type of food.
Revised: Describe what {} looks like a photo of {} , a type of food.

Original: Visually describe {}
Revised: Visually describe {}

Original: How can you tell that the food in this photo is {} ?
Revised: How can you tell that the food in this photo is {} ?

Original: Stanford Cars
Revised: Stanford Cars

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: How can you identify a(n) {} ?
Revised: How can you identify a(n) {} ?

Original: a photo of a {} .
Revised: a photo of a {} .

Original: Description of a(n) {} , a type of car.
Revised: Description of a(n) {} , a type of car.

Original: a photo of the {} .
Revised: a photo of the {} .

Original: A caption of a photo of a(n) {} : a photo of my {} .
Revised: A caption of a photo of a(n) {} : a photo of my {} .

Original: What are the primary characteristics of a(n) {} ?
Revised: What are the primary characteristics of a(n) {} ?

Original: i love my {} !
Revised: i love my {} !

Original: Description of the exterior of a(n) {} a photo of my dirty {} .
Revised: Description of the exterior of a(n) {} a photo of my dirty {} .

Original: What are the identifying characteristics of a(n) {} , a type of car?
Revised: What are the identifying characteristics of a(n) {} , a type of car?

Original: a photo of my clean {} .
Revised: a photo of my clean {} .

Original: Describe an image from the internet of a(n) {} a photo of my new {} .
Revised: Describe an image from the internet of a(n) {} a photo of my new {} .

Original: What does a(n) {} look like?
Revised: What does a(n) {} look like?

Original: a photo of my old {} .
Revised: a photo of my old {} .

Original: Describe what a(n) {} , a type of car, looks like
Revised: Describe what a(n) {} , a type of car, looks like

Original: Oxford Pets
Revised: Oxford Pets

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe what a pet {} looks like a photo of a , a type of pet.
Revised: Describe what a pet {} looks like a photo of a , a type of pet.

Original: Visually describe a(n) ‘ {} ’, a type of pet.
Revised: Visually describe a(n) ‘ {} ’, a type of pet.

Original: ImageNet
Revised: ImageNet

Original: CuPL 
Revised: CuPL hand-written Standard

Original: [Table]
Revised: [Table]

Original: -written
Revised: -written

Original: Standard hand-written

Original: Describe what a(n) {} looks like a bad photo of a {} .
Revised: Describe what a(n) {} looks like a bad photo of a {} .

Original: the origami {} .
Revised: the origami {} .

Original: How can you identify a(n) {} ?
Revised: How can you identify a(n) {} ?

Original: a photo of many {} .
Revised: a photo of many {} .

Original: the {} in a video game.
Revised: the {} in a video game.

Original: What does a(n) look like?
Revised: What does a(n) look like?

Original: a sculpture of a {} .
Revised: a sculpture of a {} .

Original: a sketch of a {} .
Revised: a sketch of a {} .

Original: Describe an image from the internet of a(n) {} a photo of the hard to see {} .
Revised: Describe an image from the internet of a(n) {} a photo of the hard to see {} .

Original: a doodle of the {} .
Revised: a doodle of the {} .

Original: A caption of an image of a(n) {} : a low resolution photo of the {} .
Revised: A caption of an image of a(n) {} : a low resolution photo of the {} .

Original: a origami {} .
Revised: a origami {} .

Original: a rendering of a {} .a low resolution photo of a {} . graffiti of a {} . the toy {} .
Revised: a rendering of a {} .

Original: a bad photo of the {} . a rendition of the {} . a cropped photo ofthe {} . aphoto of the clean {} .
Revised: a low resolution photo of a {} .

Original: a tattoo of a {} .
Revised: graffiti of a {} .

Original: a photo of a large {

Original: [Table]
Revised: [Table]

Original: FGVC Aircraft
Revised: FGVC Aircraft

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe a(n) {} aircraft a photo of a {} , a type of aircraft.
Revised: Describe a(n) {} aircraft a photo of a {} , a type of aircraft.

Original: Describe the {} aircraft a photo of the {} , a type of aircraft.
Revised: Describe the {} aircraft a photo of the {} , a type of aircraft.

Original: DTD
Revised: DTD

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: What does “ {} ” material look like?
Revised: What does “ {} ” material look like?

Original: a photo of a {} texture.
Revised: a photo of a {} texture.

Original: What does a “ {} ” surface look like?
Revised: What does a “ {} ” surface look like?

Original: a photo of a {} pattern.
Revised: a photo of a {} pattern.

Original: What does a “ {} ” texture look like?
Revised: What does a “ {} ” texture look like?

Original: a photo of a {} thing.
Revised: a photo of a {} thing.

Original: What does a “ {} ” object look like?
Revised: What does a “ {} ” object look like?

Original: a photo of a {} object.
Revised: a photo of a {} object.

Original: What does a “ {} ” thing look like?
Revised: What does a “ {} ” thing look like?

Original: a photo of the {} texture.
Revised: a photo of the {} texture.

Original: What does a “ {} ” pattern look like?
Revised: What does a “ {} ” pattern look like?

Original: a photo of the {} pattern.
Revised: a photo of the {} pattern.

Original: a photo of the {} thing. a photo of the {} object.
Revised: a photo of the {} thing. a photo of the {} object.

Original: SUN
Revised: SUN

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe what a(n) {} looks like a photo of a {} .
Revised: Describe what a(n) {} looks like a photo of a {} .

Original: How can you identify a(n) {} ?
Revised: How can you identify a(n) {} ?

Original: a photo of the {} .
Revised: a photo of the {} .

Original: Describe a photo of a(n) {}
Revised: Describe a photo of a(n) {}

Original: KineticsCuPL hand-written
Revised: KineticsCuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe the action ” {} ” a photo of a person {} .
Revised: Describe the action ” {} ” a photo of a person {} .

Original: What does a person {} look like?
Revised: What does a person {} look like?

Original: a photo of a person using {} .
Revised: a photo of a person using {} .

Original: What does the act of {} look like?
Revised: What does the act of {} look like?

Original: a photo of a person doing {} .
Revised: a photo of a person doing {} .

Original: Describe ” {} ” a photo of a person during {} .
Revised: Describe ” {} ” a photo of a person during {} .

Original: a photo of a person performing {} .
Revised: a photo of a person performing {} .

Original: a photo of a person practicing {} .
Revised: a photo of a person practicing {} .

Original: a video of {} .
Revised: a video of {} .

Original: a video of a person {} .
Revised: a video of a person {} .

Original: a video of a person using {} . a video of a person doing {} .
Revised: a video of a person using {} . a video of a person doing {} .

Original: a video of a person during {} .
Revised: a video of a person during {} .

Original: a video of a person performing {} . a video of a person practicing {} .
Revised: a video of a person performing {} . a video of a person practicing {} .

Original: a example of {} .
Revised: a example of {} .

Original: a example of a person {} .
Revised: a example of a person {} .

Original: a example of a person using {} . a example of a person doing {} .
Revised: a example of a person using {} . a example of a person doing {} .

Original: a example of a person during {} .
Revised: a example of a person during {

Original: a example of a person performing {} . a example of a person practicing {} .
Revised: a example of a person performing {} . a example of a person practicing {} .

Original: a demonstration of {} .
Revised: a demonstration of {} .

Original: a demonstration of a person {} .
Revised: a demonstration of a person {} .

Original: a demonstration of a person using {} . a demonstration of a person doing {} 
Revised: a demonstration of a person using {} . a demonstration of a person doing {} .

Original: a demonstration of a person during {} .
Revised: a demonstration of a person during {} .

Original: a demonstration of a person performing {} . a demonstration of a person practicing {} .
Revised: a demonstration of a person performing {} . a demonstration of a person practicing {} .

Original: [Table]
Revised: [Table]

Original: CuPL
Revised: CuPL

Original: [Table]
Revised: [Table]

Original: Standard
Revised: Standard

Original: [Table]
Revised: [Table]

Original: -written
Revised: -written

Original: Describe the action of {} a photo of a person {} .
Revised: Describe the action of {} a photo of a person {} .

Original: What does the act of {} look like?
Revised: What does the act of {} look like?

Original: a video of a person {} .
Revised: a video of a person {} .

Original: What does a person doing {} look like?
Revised: What does a person doing {} look like?

Original: a example of a person {} .
Revised: a example of a person {} .

Original: Describe “ {} “ a demonstration of a person {} .
Revised: Describe “ {} “ a demonstration of a person {} .

Original: Describe the action “ {} “ a photo of the person {} .
Revised: Describe the action “ {} “ a photo of the person {} .

Original: a video of the person {} .
Revised: a video of the person {} .

Original: a example of the person {} .
Revised: a example of the person {} .

Original: a demonstration of the person {} .
Revised: a demonstration of the person {} .

Original: a photo of a person using {} . a video of a person using {} .
Revised: a photo of a person using {} . a video of a person using {} .

Original: a example of a person using {} . a demonstration of a person using {} .
Revised: a example of a person using {} . a demonstration of a person using {} .

Original: a photo of the person using {} . a video of the person using {} .
Revised: a photo of the person using {} . a video of the person using {} .

Original: a example of the person using {} . a demonstration of the person using {} .
Revised: a example of the person using {} . a demonstration of the person using {} .

Original: a photo of a person doing {} . a video of a person doing {} .
Revised: a photo of a person doing {} . a video of a person doing {} .

Original: a example of a person doing {} . a demonstration of a person doing {} .
Revised: a example of a person doing {} . a demonstration of a person doing {} .

Original: a photo of the person doing {} . a video of the person doing {} .
Revised: a photo of the person doing {} . a video of the person doing {} .

Original: a example of the person doing {} .
Revised: a example of the person doing {} .

Original: a demonstration of the person doing {} .
Revised: a demonstration of the person doing {} .

Original: a photo of a person during {} . a video of a person during {} .
Revised: a photo of a person during {} . a video of a person during {} .

Original: a example of a person during {} .
Revised: a example of a person during {} .

Original: a demonstration of a person during {} .
Revised: a demonstration of a person during {} .

Original: a photo of the person during {} . a video of the person during {} .
Revised: a photo of the person during {} . a video of the person during {} .

Original: a example of the person during {} .
Revised: a example of the person during {} .

Original: a demonstration of the person during {} .
Revised: a demonstration of the person during {} .

Original: a photo of a person performing {} . a video of a person performing {} .
Revised: a photo of a person performing {} . a video of a person performing {} .

Original: a example of a person performing {} . a demonstration of a person performing {} .
Revised: a example of a person performing {} . a demonstration of a person performing {} .

Original: a photo of the person performing {} . a video of the person performing {} .
Revised: a photo of the person performing {} . a video of the person performing {} .

Original: a example of the person performing {} .
Revised: a example of the person performing {} .

Original: a demonstration of the person performing {} .
Revised: a demonstration of the person performing {} .

Original: a photo of a person practicing {} . a video of a person practicing {} .
Revised: a photo of a person practicing {} . a video of a person practicing {} .

Original: a example of a person practicing {} . a demonstration of a person practicing {} .
Revised: a example of a person practicing {} . a demonstration of a person practicing {} .

Original: a photo of the person practicing {}
Revised: a photo of the person practicing {}

Original: a video of the person practicing {} .
Revised: a video of the person practicing {} .

Original: a example of the person practicing {} .
Revised: a example of the person practicing {} .

Original: a demonstration of the person practicing {} .
Revised: a demonstration of the person practicing {} .

Original: RESISC
Revised: RESISC

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe a satellite photo of a(n) {} satellite imagery of {} .
Revised: Describe a satellite photo of a(n) {} satellite imagery of {} .

Original: Describe a(n) {} as it would appear in an aerial image aerial imagery of {} .
Revised: Describe a(n) {} as it would appear in an aerial image aerial imagery of {} .

Original: How can you identify a(n) {} in an aerial photo?
Revised: How can you identify a(n) {} in an aerial photo?

Original: satellite photo of {} .
Revised: satellite photo of {} .

Original: Describe the satellite photo of a(n) {} aerial photo of {} . Describe an aerial photo of a(n) {} satellite view of {} .
Revised: Describe the satellite photo of a(n) {} aerial photo of {} . Describe an aerial photo of a(n) {} satellite view of {} .

Original: aerial view of {} .
Revised: aerial view of {} .

Original: satellite imagery of a {} .
Revised: satellite imagery of a {} .

Original: aerial imagery of a {} .
Revised: aerial imagery of a {} .

Original: satellite photo of a {} .
Revised: satellite photo of a {} .

Original: aerial photo of a {} .
Revised: aerial photo of a {} .

Original: satellite view of a {} .
Revised: satellite view of a {} .

Original: aerial view of a {} .
Revised: aerial view of a {} .

Original: satellite imagery of the {} .
Revised: satellite imagery of the {} .

Original: aerial imagery of the {} .
Revised: aerial imagery of the {} .

Original: satellite photo of the {} .
Revised: satellite photo of the {} .

Original: aerial photo of the {} .
Revised: aerial photo of the {} .

Original: satellite view of the {} .
Revised: satellite view of the {} .

Original: aerial view of the {} .
Revised: aerial view of the {} .

Original: Birdsnap
Revised: Birdsnap

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe what the bird {} looks like: a photo of a {} , a type of bird.
Revised: Describe what the bird {} looks like: a photo of a {} , a type of bird.

Original: Describe the bird {} :
Revised: Describe the bird {} :

Original: What are the identifying characteristics of the bird {} ?
Revised: What are the identifying characteristics of the bird {} ?

Original: Flowers
Revised: Flowers

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe how to identify a(n) {} , a type of flower a photo of a {} , a type of flower.’
Revised: Describe how to identify a(n) {} , a type of flower a photo of a {} , a type of flower.’

Original: What does a(n) {} flower look like?
Revised: What does a(n) {} flower look like?

Original: CIFARCuPL hand-written
Revised: CIFARCuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe what a(n) {} looks like a photo of a {} .
Revised: Describe what a(n) {} looks like a photo of a {} .

Original: Describe a(n) {} : a blurry photo of a {} .
Revised: Describe a(n) {} : a blurry photo of a {} .

Original: What are the identifying characteristics of a(n) {} ?
Revised: What are the identifying characteristics of a(n) {} ?

Original: a black and white photo of a {} .
Revised: a black and white photo of a {} .

Original: a low contrast photo of a {} . a high contrast photo of a {} .
Revised: a low contrast photo of a {} . a high contrast photo of a {} .

Original: a bad photo of a {} .
Revised: a bad photo of a {} .

Original: a good photo of a {} .
Revised: a good photo of a {} .

Original: a photo of a small {} .
Revised: a photo of a small {} .

Original: a photo of a big {} .
Revised: a photo of a big {} .

Original: a photo of the {} .
Revised: a photo of the {} .

Original: a blurry photo of the {} .
Revised: a blurry photo of the {} .

Original: a black and white photo of the {}
Revised: a black and white photo of the {}

Original: a low contrast photo of the {} . a high contrast photo of the {} .
Revised: a low contrast photo of the {} . a high contrast photo of the {} .

Original: a bad photo of the {} .
Revised: a bad photo of the {} .

Original: a good photo of the {} .
Revised: a good photo of the {} .

Original: a photo of the small {} .
Revised: a photo of the small {} .

Original: a photo of the big {} .
Revised: a photo of the big {} .

Original: CIFAR- 
Revised: CIFAR-

Original: CuPL hand-written
Revised: CuPL hand-written

Original: Standard hand-written
Revised: Standard hand-written

Original: Describe a photo of a(n) {} : a photo of a {} .
Revised: Describe a photo of a(n) {} : a photo of a {} .

Original: What are the identifying characteristics of a(n) {} ?
Revised: What are the identifying characteristics of a(n) {} ?

Original: a blurry photo of a {} .
Revised: a blurry photo of a {} .

Original: Describe what a(n) {} looks like: a black and white photo of a {} .
Revised: Describe what a(n) {} looks like: a black and white photo of a {} .

Original: Describe a(n) {} : a low contrast photo of a {} .
Revised: Describe a(n) {} : a low contrast photo of a {} .

Original: a high contrast photo of a {} .
Revised: a high contrast photo of a {} .

Original: a bad photo of a {} .
Revised: a bad photo of a {} .

Original: a good photo of a {} .
Revised: a good photo of a {} .

Original: a photo of a small {} .
Revised: a photo of a small {} .

Original: a photo of a big {} .
Revised: a photo of a big {} .

Original: a photo of the {} .
Revised: a photo of the {} .

Original: a blurry photo of the {} .
Revised: a blurry photo of the {} .

Original: a black and white photo of the {} .
Revised: a black and white photo of the {} .

Original: a low contrast photo of the {} . a high contrast photo of the {} .
Revised: a low contrast photo of the {} . a high contrast photo of the {} .

Original: a bad photo of the {} .
Revised: a bad photo of the {} .

Original: a good photo of the {} .
Revised: a good photo of the {} .

Original: a photo of the small {} .
Revised: a photo of the small {} .

Original: a photo of the big {} .
Revised: a photo of the big {} .

Original: B C U PL B ASE P ROMPTS
Revised: B C U PL B ASE P ROMPTS

Original: The three general sentences used in the base prompt setting are:
Revised: The three general sentences used in the base prompt setting are:

Original: Describe what a/the looks like:
Revised: Describe what a/the looks like:

Original: Describe a/the :
Revised: Describe a/the :

Original: What are the identifying characteristics of a/the ?
Revised: What are the identifying characteristics of a/the ?

Original: Here we specify the type filled in for each of the examined datasets, as well as the article used for that dataset (‘a(n)’ or ‘the’):
Revised: Here we specify the type filled in for each of the examined datasets, as well as the article used for that dataset (‘a(n)’ or ‘the’):

Original: Dataset
Revised: Dataset

Original: Base LLM-prompt type specification
Revised: Base LLM-prompt type specification

Original: ImageNet a(n) {}
Revised: ImageNet a(n) {}

Original: DTD the texture {}
Revised: DTD the texture {}

Original: StanfordCars the car {}
Revised: StanfordCars the car {}

Original: SUN a(n) {}
Revised: SUN a(n) {}

Original: Foodthe food {}
Revised: Foodthe food {}

Original: FGVC Aircraft the aircraft {}
Revised: FGVC Aircraft the aircraft {}

Original: Oxford Pets a pet {}
Revised: Oxford Pets a pet {}

Original: Caltech a(n) {}
Revised: Caltech a(n) {}

Original: CIFARa(n) {} CIFARa(n) {}
Revised: CIFARa(n) {} CIFARa(n) {}

Original: Flowersthe flower {}
Revised: Flowersthe flower {}

Original: Kineticsthe action of {}
Revised: Kineticsthe action of {}

Original: UCF the action of {}
Revised: UCF the action of {}

Original: RESISC a satellite photo of {}
Revised: RESISC a satellite photo of {}

Original: Birdsnap the bird {}
Revised: Birdsnap the bird {}

Original: C E VALUATION M ETRIC
Revised: C E VALUATION M ETRIC

Original: I m a g e N e t
Revised: I m a g e N e t

Original: D T D
Revised: D T D

Original: S t a n f o r d
Revised: S t a n f o r d

Original: C a r s
Revised: C a r s

Original: S UN
Revised: S UN

Original: F ood
Revised: F ood

Original: F GV
Revised: F GV

Original: C A i r
Revised: C A i r

Original: c r a f t
Revised: c r a f t

Original: O x f o r d P e t s
Revised: O x f o r d P e t s

Original: C a lt ec h
Revised: C a lt ec h

Original: F l o w e r s
Revised: F l o w e r s

Original: U C F
Revised: U C F

Original: K i n e ti c sR E S
Revised: K i n e ti c sR E S

Original: I
Revised: I

Original: S C
Revised: S C

Original: C
Revised: [Table] I [Table] [Table]

Original: I F
Revised: [Table] i [Table]

Original: A
Revised: [Table] While GPT-3 (Brown et al., 2020) demonstrates higher performance on a number of tasks compared to smaller open-source models, open-source models are sometime more accessible. We therefore show improvement using GPT-J-6B (Wang & Komatsuzaki, 2021), a small open-source model available on HuggingFace (Wolf et al., 2019).

Revised: We find that we are able to surpass human written prompts with prompts generated by this model as shown in

Revised: Table, though we still fall short of those generated by GPT-3.

Revised: Additionally, we employ a number of strategies to increase the accuracy of the lower quality GPT-J-6B generations.

Revised: First, we generate at a lower temperature (0.3) to prevent irrelevant or nonsensical generations, which we find occur more frequently in smaller models.

Revised: Additionally, we generate 5 times more Image-prompts per

Revised: LLM-prompt than we do when using GPT-3 (Brown et al., 2020).

Revised: We also add punctuation to the end of all LLM-prompts to encourage the LLM to begin new sentences.

Revised: Finally, we filter out any

Revised: Image-prompts which do not contain the name of the ImageNet category they are meant to describe, as well as remove a number of unicode characters from the generations (e.g. ‘u2019’).

Revised: We present these findings as a way to make CuPL a more accessible options until large high performance models become available to the public.

Revised: E S INGLE S ENTENCE B ASELINE

Revised: [Table]

Revised: One of the primary benefits of CuPL is that it decreases the amount of necessary hand-engineering.

Revised: However, this could also be done by decreasing the number of total hand-written templates used, which comes at a loss in performance.

Revised: We present this as a ‘low effort’ baseline, where we use only the hand constructed template of ‘a photo of a {} ’.

Revised: We compare this with two CuPL baselines.

Original: RC
Revised: The first is the baseline in which we also only construct one hand written template: ‘Describe what a {} looks like’. We then use this to generate 10 Image-prompts. The second baseline is a single CuPL generated sentence, generated with the prompt ‘Describe what a {} looks like’. For this experiment, we generate at a temperature of 0.3 as we find that higher tempuratures are only helpful when we are able to ensemble many diverse prompts, not when we are limited to one.

Revised: We find that CuPL outperforms a single hand-written template under both of these settings, as shown in Table 5.

Revised: F R OBUSTNESS

Revised: In addition to the previously mentioned benefits of open vocabulary models, one of the important advances made by CLIP (Radford et al., 2021) is an increased robustness on out-of-distribution data.

Revised: Fine-tuning has been shown to degrade performance on out-of-distribution tasks (Wortsman et al., 2022), however zero-shot CLIP is robust to these distribution shifts.

Revised: We show improvement on two common distribution shifts in Table 6, demonstrating that CuPL maintains the robustness of CLIP.

Original:  I F A 
Revised: [Table] G C U PL I MPROVEMENT A NALYSIS G. VISUAL SIMILARITY ANALYSIS In Figure 7, we provide initial analysis on the categories where the CuPL algorithm improves the most over standard hand-written prompts. We find that the improvement is not uniformly distributed, but rather some classes see a large improvement, while others see a decrease in per class accuracy.

Revised: Interestingly, there are often two similar categories where one sees a large increase in accuracy and the other sees a decrease.

Revised: For example, the ‘mushroom’ class has an approximately 40pp increase, while ‘agaric’ (a subclass of mushroom) is one of the classes with the largest drop in accuracy.

Revised: In order to better understand this phenomenon, we examine the change in accuracy between CuPL and the standard method of prompting in the image embedding space.

Revised: Thus we are able to visualize the close relationship between categories like ‘agaric’ and ‘mushroom’.

Revised: In order to be able to visualize the high dimensional CLIP image embedding in two dimensions, we utilize the t-distributed stochastic neighbor embedding algorithm (Van der Maaten & Hinton, 2008).

Revised: Figure 8 visualizes image features (reduced into two dimensions) in relation to CuPL improvement.

Revised: As was suggested by Figure 7, we see in Figure 8 that when there is a class that has a large increase in accuracy with CuPL prompts (‘mushroom’, ‘graduation cap’, ‘monitor’)

Revised: there is often a decrease in class accuracy for a visually related class.

Revised: This means that when choosing between two similar or co-occuring classes, CuPL has a different distribution of classification than the standard method (e.g. the standard method prefers ‘canoe’ over ‘paddle’ much more strongly than CuPL).

Revised: This suggests that the overall accuracy improvement of CuPL over the standard method may come (at least in part) from better distinguishing between two visually similar classes.

Revised: While it may be over-correcting from the mistakes of the standard method (as demonstrated by the drop in accuracy in one of the two similar classes), the CuPL predictions appear to be overall more accurate, as demonstrated by the overall higher accuracy.

Revised: G. C O - OCCURRENCES BETWEEN OBJECTS

Revised: Many of the frequently confused pairs in Figure 8 are objects that are likely to occur in an image (i.e.

Revised: ‘canoe’-‘paddle’ or ’graduation cap’-‘academic gown’ or ’monitor’-‘desk’).

Revised: One potential benefit of CuPL captions is that they are able to capture co-occurrences as well.

Revised: For example, one CuPL prompt for the ‘canoe’ class is A canoe is typically a narrow boat with pointed ends that is propelled with a paddle.

Revised: This caption contains the word ‘paddle’ which is frequently confused with ‘canoe’ and likely to be present in images, even where the correct label is ‘canoe’.

Revised: We therefore investigate the effectiveness of CuPL captions on images which contain more than one ImageNet object.

Revised: We attain this by using the ImageNet-ReaL dataset (Beyer et al., 2020) which relabels ImageNet images with all applicable labels, so an image with both a ‘canoe’ and a ‘paddle’ would have both

Original: RB i r d s  n a p
Revised: [Figure] N e t C u P L acc u r ac y vs B a s e li n e acc u r ac y labels. We then tag images as having multiple ImageNet objects or only one ImageNet object based on the ReaL dataset.

Revised: Finally, we compute ImageNet accuracy across each of these two sets (using standard ImageNet labels).

Revised: Results are given in Table 7.

Revised: ImageNet

Revised: (Beyer et al., 2020) to images which

Revised: [Table]

Revised: H E RROR A NALYSIS

Revised: In Figure 9, we present an error analysis of our model using two different metrics.

Revised: The first is an analysis between the model prediction and the correct class using the visual similarity of these labels.

Revised: To capture this, we first attain an average visual embedding of each class by taking the mean of each image in that class and then normalizing that mean.

Revised: Then for each class we rank how similar each of the other 999 classes are by the distance between these embeddings.

Revised: If the models makes an incorrect prediction, but it predicts the class with the closest embedding to the correct label, then we refer to this as an image offset of 1.

Revised: Additionally, we examine the prediction errors in terms of the linguistic similarity of the labels.

Revised: We do this with the WordNet (Miller, 1995) similarity of two labels.

Revised: For example, if the label of the prediction and the ground-truth label share the same parent in the WordNet tree, that is a WordNet offset of 2.

Original: [Table]
Revised: [Figure]

Revised: N u m b e r o f M o d e l E rr o r s

Revised: While slight, there is a difference in the errors made by CuPL compared to the errors made by the baseline as shown in Figure 9.

Revised: CuPL is more likely to have an error that has an image offset ofthan the baseline.

Revised: However, the baseline is more likely to have an error that has a WordNet offset of 1 than CuPL.

Revised: This implies that CuPL may be taking advantage of the visually descriptive language of the captions, as even when the model makes errors, they tend to favor categories that are visually similar to the ground

Revised: However, the baseline method does not have visual in its

Revised: -prompts which may lead to its errors aligning more linguistically with the ground-truth.

Revised: [Figure]

Revised: I I MAGE - PROMPT D ISTRIBUTION

Revised: When generating Image-prompts, we use an ensemble of prompts generated by different LLMprompts (e.g. ‘What does a {} look like?).

Revised: As we find that the diversity of Image-prompts is correlated with accuracy, it is valuable to understand how different LLM-prompts affect the diversity of Image-prompts.

Revised: To accomplish this, we visualize the text embedding of Image-prompts for a selection of classes using t-sne (Van der Maaten & Hinton, 2008) dimension reduction.

Revised: We then color Image-prompts that were generated by the same LLM-prompt.

Revised: As shown in Figure 10, we find that while there is a slight clustering of Image-prompts by LLM-prompts, there is also a large amount of overlap.

Revised: J T EMPERATURE A NALYSIS

Revised: We provide several further analyses of the effect of temperature in prompt generation.

Revised: First, we visualize the distribution of Image-prompts that have been generated with a variety of different temperatures.

Revised: We do this by selecting prompts for 3 different ImageNet classes at 3 different temperatures.

Revised: We then perform dimentionality reduction on the text embeddings of these prompts in order to visualize their distribution.

Revised: As shown in Figure 11, Image-prompts generated with a temperature of 0.1 are clustered in a few different locations.

Revised: Image-prompts generated with a temperature of 0. are more widely, and Image-prompts generated with a temperature of 0.9 have a similar, but even wider distribution.

Revised: Additionally in Section K, we give all generated prompts for the ImageNet class ‘Tench’ at three different temperatures.

Revised: At the lowest temperature, the generated Image-prompts are nearly identical when generated with the same LLM-prompts.

Revised: As the temperature increases, so does the difference in the generated prompts.

Revised: [Figure]

Original: D E XAMPLE G ENERATED IMAGE - PROMPTS
Revised: K E XAMPLE G ENERATED IMAGE - PROMPTS

Original: A selection of LLM-generated image-prompts for a subset of ImageNet categories.
Revised: A selection of LLM-generated image-prompts for a subset of ImageNet categories.

Original: We give allimage-prompts for the first ImageNetcategory of “Tench” and then 10 randomly selected prompts for a number of randomly selected ImageNet categories.
Revised: We give allimage-prompts for the first ImageNet

Original: D. A LL GENERATED IMAGE - PROMPTS FOR “T ENCH ” CATEGORY
Revised: K. A LL GENERATED IMAGE - PROMPTS FOR “T ENCH ” CATEGORY

Original: A tench is a freshwater fish of the carp family."
Revised: [Table] "A tench is a freshwater fish of the carp family.", "A tench is a freshwater fish that is typically brown or olive in color. ","Atench is a fresh water fish that can grow up to 2 feet in length.", "A tench is a freshwater fish of the family Cyprinidae.", "A tench is a freshwater fish of the carp family.", "A tench is a freshwater fish with a dark green back and light-colored sides.", [Table]

Original: "A tench is a freshwater fish that is typically  brownor olive in color."
Revised: is a freshwater fish that is typically greenish [Table] brown in color

Original: "Atench is a fresh water fish that can grow up to 2 feet in length."
Revised: [Table] a brassy sheen [Table]

Original: "A tench is a freshwater fish of the family Cyprinidae."
Revised: tench is a freshwater fish that typically has a dark green back, light [Table] sides [Table]

Original:  "A tench is a freshwater fish of the carp family."
Revised: a white belly [Table] "A tench is a freshwater fish that can grow up to two feet long.", "Tench are a freshwater fish found in Europe." , "A tench is a freshwater fish that is typically olive green in color with dark spots.", "A tench is a freshwater fish that is typically olive green in color with a brownish tint.", "A tench is a freshwater fish of the Cyprinidae family.", "Tench are a freshwater fish found in Europe."

Original:  "A tench is a freshwater fish with a dark green back and light-colored sides."
Revised: , "A tench is a freshwater fish that has a dark green back, light brown sides, and a white belly.", "A tench is a freshwater fish that is typically olive-green in color with a brownish dorsal fin." , "A tench is a freshwater fish of the cyprinid family.", "A tench is a freshwater fish of the carp family.", "A tench is a freshwater fish that is typically olive green in color with a brownish back.", "The tench is a freshwater fish of the carp family Cyprinidae. ", "A tench is a freshwater fish that is typically greenish-brown in color.", "A tench is a freshwater fish of the carp family.", "A tench is a freshwater fish of the carp family.", "Tench have olive green backs and flanks, with yellowish bellies." , "A tench is a freshwater fish of the carp family.", "A tench is a freshwater fish of the cyprinid family.", "A tench is a freshwater fish that can grow up to 30 inches long."

Original: "Tenchare a freshwater fish found in Europe."
Revised: , "A tench is a freshwater fish that is typically greenish-brown in color. ","Atench is a freshwater fish that can grow up to about two feet long.", "A tench is a freshwater fish with a brownish-green back and sides, and a yellowish-brown belly." , "A tench is a freshwater fish that can grow up to two feet long.", "A tench is a freshwater fish that looks similar to a carp.", "A tench is a freshwater fish that is typically greenish-brown in color.", "Atench is a freshwater fish that is part of the [Table]

Original:  "A tench is asmall freshwater fish in the carp family."
Revised: family.", " A [Table] is a freshwater fish of the cyprinid family.", "A tench is a freshwater fish that can grow up to two feet long."

Original: "A tench is  a heavyset freshwater fish with a mottled brown body and a small, flat head."
Revised: , "The [Table] is of a tench fish swimming in water." , "The image is of a tench fish swimming in a pond.", "

Revised: The

Original: "A tench  is  a freshwater fish that looks similar to a carp."
Revised: [Table] is of a tench fish swimming in a pond." , "The image is of a tench fish swimming in a pond.", " The

Original: "A tench is  a freshwater fish  in  the carp family."
Revised: [Table] is of a tench fish swimming in a pond.", " In the [Table]

Original: "A tench 
Revised: , a tench is swimming in a pond with lily pads." , "The [Table] is of a tench fish swimming in a pond."

Revised: , "The image is of a tench fish swimming in a pond.", "The image is of a tench fish swimming in a pond."

Revised: , "The image is of a tench fish swimming in

Original: [Table]
Revised: [Table]

Revised: pond.

Revised: [Table]

Revised: [Table]

Revised: A

Original: "The tench is a freshwater fish native to Europe.", "This image shows a large, dark green tench swimming in a pond."
Revised: [Table] fish, native to Europe, characterized by its greenish-brown color and spots. [Table] [Table]

Original: , "An image of a tench from the internet would likely show a dark green fish with a lighter underside.", "The image is of a tench fish.", "The image is of a tench fish on a white background.", "A tench is a freshwater fish of the Cyprinidae family.", "The image is of a tench swimming in a murky pond.", "In the image, a tench swims in a pond with lily pads."
Revised: [Table] [Table] [Table] [Table]

Revised: [Table]

Original: , " A tench in a river.", "A tench (Tinca tinca) is a freshwater fish in the carp family that is found throughout Europe."
Revised: [Table] [Table] [Table] [Table]

Original: , " Tench (Tinca tinca), a member of the carp family (Cyprinidae), native to Eurasia.", " A tench, a freshwater fish in the family Cyprinidae.", " The tench (Tinca tinca) is a freshwater fish of the cyprinid family found throughout Eurasia."
Revised: [Table] [Table] [Table] [Table]

Original: , " A tench in a Finnish lake.", "A tench (Tinca tinca) is a freshwater fish belonging to the carp family (Cyprinidae).
Revised: Temperature = 0. [Table] [Table] [Table]

Revised: up

Original: ", "A tench in a fishpond.", " The common tench is a freshwater fish of the cyprinid family found throughout Eurasia.", "Tench (Tinca tinca) in a pond."
Revised: [Table] a [Table] fish that is typically

Revised: [Table]

Revised: a

Revised: [Table]

Revised: a

Revised: [Table]

Revised: a

Original: D. S AMPLE GENERATED IMAGE - PROMPTS FOR RANDOMLY SELECTED I MAGE N ET CATEGORIES
Revised: [Table] [Table] [Table] IMAGE - PROMPTS FOR RANDOMLY SELECTED I MAGE N ET CATEGORIES

Original: "bubble":"Abubble is a sustained period of inflated asset prices.", "A bubble looks like a sphere of air."
Revised: "bubble [Table] :"Abubble is a [Table]

Revised: of

Revised: [Table]

Original: , "A bubble is often characterized by rapidly increasing prices in an asset or security, followed by a sharp decrease in prices.", "A bubble looks like a small, round, thin film of soap filled with air.", "A bubble looks like a round sphere of soap film.", " \"A bubble being blown in the park.", "A close-up of a soap bubble with a thin film of water in between two layers of
Revised: [Table] "A bubble looks like a sphere of air." [Table] "A bubble is often characterized by rapidly increasing prices in an asset or security, followed by a sharp decrease in prices."

Revised: [Table]

Original: air.", "A bubble is a spherical shape made up of a  thin film of soap water.", "A bubble is a circle of air surrounded by water.", "A bubble looks like a round, thin layer of soap surrounding a pocket of air."
Revised: A bubble looks like a small, round, thin film of soap filled with air. [Table] A bubble looks like a round sphere of soap film. [Table]

Revised: [Table]

Revised: A bubble being blown in the park.

Original: "kit fox": "A kit fox is a small species of fox, about the size of a domestic cat.
Revised: [Table] A [Table] [Table]

Revised: of a

Revised: [Table]

Revised: a

Revised: [Table]

Revised: of

Revised: [Table]

Revised: two

Revised: [Table]

Revised: [Table]

Revised: A

Revised: [Table]

Revised: a

Revised: [Table]

Revised: A

Revised: [Table]

Revised: a

Revised: [Table]

Revised: [Table]

Revised: A

Revised: [Table]

Revised: looks like a

Revised: [Table]

Revised: thin

Revised: [Table]

Revised: of

Revised: [Table]

Original:  ", "You can identify a kit fox by its small size, its big ears, and its long, bushy tail.", " A kit fox laying on the ground in a desert habitat.", "A kit fox is a small fox found in North America."
Revised: of air. [Table] "kit fox": "A kit fox is a small species of fox, about the size of a domestic cat. ", "You can identify a kit fox by its small size, its big ears

Revised: [Table]

Revised: and its long, bushy tail.", " A kit fox laying on the ground in a desert habitat."

Original: , "A kit fox is a small fox  with a sleek coat of fur.", "A kit fox is a small fox with large ears, a long, black-tipped tail, and pale fur.", "A kit fox has a reddish coat, with white patches on its chest and throat.
Revised: , "A kit fox is a small fox found in North America.", "A kit fox is a small fox with a sleek coat of fur.", "A kit fox is a small fox with large ears, a long, black-tipped tail [Table] and pale fur. ", "A kit fox has a reddish coat, with white patches on its chest and throat.

Original: ","Thiskit
Revised: ","Thiskit

Original: fox has a reddish coat and large ears.", "A kit fox looks like a small fox with a pointed nose, large ears, and a long, bushy tail.", "A kit fox is a small species of fox."
Revised: fox has a reddish coat and large ears.", "A kit fox looks like a small fox with a pointed nose, large ears, and a long, bushy tail.", "A kit fox is a small species of fox."

Original: "toy terrier": "A toy terrier looks like a very small version of a terrier.", "The image is of a toy terrier that is mostly white with brown spots.", "You can identify a toy terrier by looking for a compact, short-legged dog with a short muzzle.", " Cute little guy.", "Toy terriers are miniature versionsof terriers, such as the Jack
Revised: "toy terrier": "A toy terrier looks like a very small version of a terrier.", "The image is of a toy terrier that is mostly white with brown spots.", "You can identify a toy terrier by looking for a compact, short-legged dog with a short muzzle. ", " Cute little guy.", "Toy terriers are miniature versions

Original:  Russell Terrier.", "A toy terrier is a small, lightweight breed of dog.", "The image is of a small, brown toy terrier.", "A toy terrier typically has a long, narrow head with pointy ears, and a small, compact body."
Revised: of terriers, such as the Jack Russell Terrier. ", "A toy terrier is a small, lightweight breed of dog.", "The image is of a small, brown toy terrier.", "A toy terrier typically has a long, narrow head with pointy ears, and a small, compact body."

Original: , "A toy terrier is a small, short-legged dog with a long body, pointy nose, and large ears.", "A small, brown and white toy terrier is sitting on a beige couch, looking at the camera."
Revised: , "A toy terrier is a small, short-legged dog with a long body, pointy nose, and large ears.", "A small, brown and white toy terrier is sitting on a beige couch, looking at the camera."

Original: "mousetrap":"Intheimage, there is a mousetrap made of wood and metal.", "An image of a mousetrap from the internet would most likely show a traditional wooden mousetrap with a metal spring."
Revised: "mousetrap":"Intheimage, there is a mousetrap made of wood and metal.", "An image of a mousetrap from the internet would most likely show a traditional wooden mousetrap with a metal spring."

Original: , "A mousetrap is a device made to catch and kill mice.", "A mousetrap is a small device that is used to catch mice.", "The classic mousetrap consists of a wooden base with a metal spring mounted on one end.", "The mousetrap is a small wooden box with a metal spring inside.", "The mousetrap is a simple device that has been used for centuries to catch mice."
Revised: , "A mousetrap is a device made to catch and kill mice.", "A mousetrap is a small device that is used to catch mice.", "The classic mousetrap consists of a wooden base with a metal spring mounted on one end.", "The mousetrap is a small wooden box with a metal spring inside.", "The mousetrap is a simple device that has been used for centuries to catch mice."

Original: , "The classic mousetrap - simple, effective, and deadly.
Revised: , "The classic mousetrap - simple, effective, and deadly.

Original: ", "The most common way to identify a mousetrap is by its small size and rectangular shape.", "A mousetrap typically has a wire or wooden frame that is baited with food and springs open quickly to snap shut on the mouse when it attempts to steal the bait."
Revised: "The most common way to identify a mousetrap is by its small size and rectangular shape.", "A mousetrap typically has a wire or wooden frame that is baited with food and springs open quickly to snap shut on the mouse when it attempts to steal the bait."

Original: "dog sled": "A dog sled is a vehicle on runners, typically with a thin frame and a flat bottom, that is used to convey goods or passengers over snow or ice.", "Adog sled is a toboggan pulled by dogs, typically over snow."
Revised: "dog sled": "A dog sled is a vehicle on runners, typically with a thin frame and a flat bottom, that is used to convey goods or passengers over snow or ice.", "Adog sled is a toboggan pulled by dogs, typically over snow."

Original: , "A dog sled looks like a bed on runners that is pulled by dogs.", "A dog sled is typically a heavy frame on runners that is pulled by one or more dogs. ", "A dog sled is traditionally a sled pulled by dogs, used for transportation, racing, or other purposes.", " teamA dog sled team is a group of dogs that are harnessed together to pull a sled.", "A dog sled looks like a small, open vehicle that is pulled by one or more dogs.
Revised: , "A dog sled looks like a bed on runners that is pulled by dogs.", "A dog sled is typically a heavy frame on runners that is pulled by one or more dogs.", "A dog sled is traditionally a sled pulled by dogs, used for transportation, racing, or other purposes.", " teamA dog sled team is a group of dogs that are harnessed together to pull a sled.", "A dog sled looks like a small, open vehicle that is pulled by one or more dogs.

Original: ", "Dog sledding in winter.", " pulled by huskiesThe image is of a dog sled pulled by huskies."
Revised: ", "Dog sledding in winter.", " pulled by huskiesThe image is of a dog sled pulled by huskies."

Original: , "A dog sled looks like a large cart that is pulled by a team of dogs."
Revised: , "A dog sled looks like a large cart that is pulled by a team of dogs."

Original: "geyser":"Ageyser is a hot spring that periodically erupts, shooting a column of water and steam into the air.", "A geyser looks like a hole in the ground that sometimes spurts hot water and steam into the air.", "Geysers are hot springs that periodically spout water and steam into the air.
Revised: "geyser":"Ageyser is a hot spring that periodically erupts, shooting a column of water and steam into the air.", "A geyser looks like a hole in the ground that sometimes spurts hot water and steam into the air.", "Geysers are hot springs that periodically spout water and steam into the air.

Original: ","Theimage is of a geyser erupting.", "A geyser is a hot spring that periodically erupts, spraying water into the air.", "A geyser typically looks like a cone of rocks with a small hole at the top."
Revised: ","Theimage is of a geyser erupting.", "A geyser is a hot spring that periodically erupts, spraying water into the air.", "A geyser typically looks like a cone of rocks with a small hole at the top."

Original: , "Ageyser is a hot spring where water intermittently boils, sending a jet of hot water and steam into the air.", 
Revised: , "Ageyser is a hot spring where water intermittently boils, sending a jet of hot water and steam into the air.", "A geyser is a hot spring that periodically shoots a stream of hot water and steam into the air.

Original: [Table]
Revised: [Table]

Original: car."
Revised: go-

Original: , "A go-kart is a small, lightweight vehicle withfour wheels that is propelled by a small engine."
Revised: [Table] Ago-kart typically looks like a small car or buggy with a small engine in the back.",

Original: "black-and-white colobus": "The Black-and-White Colobus is a type of Old World monkey, found in Africa.
Revised: [Table]

Original: ","Ablack-and-white colobus has long black fur, and a white face with  a black triangle around the eyes."
Revised: A go-kart is a small vehicle with four wheels, a steering wheel [Table] and a gas pedal." [Table]

Original: , "Colobus Monkey in the TreesThis elegant colobus monkey is swinging through the trees in search of food.", " monkeyIn this image, a black-and-white colobus monkey is shown perched atop a tree branch." , "The black-and-white colobus monkey is one of the most beautiful and distinctive of all the colobus monkeys."
Revised: [Table]

Original: , "The black-and-white colobus monkey is a species of primate in the Colobidae family.", "Colobus monkeys are generally black with white patches on their face, back, and sides."
Revised: [Table] type [Table]

Original: , "The black-and-white colobus is a species of Old World monkey." , " monkeyThe image is of a black-and-white colobus monkey sitting on a tree branch.", " monkeyIn the image, the black-and-white colobus monkey is sitting in a tree." "sock":"Asock usually has a cuff at the top, and a heel at the bottom.", " A black sock with a white line running down the middle.", "A sock is typically a garment worn on the feet and made from asoft material, such as cotton.",
Revised: a

Original: [Table]
Revised: [Table]

Original: southern black widow spider is a small, black spider with a red hourglass-shaped mark on its underside.", "There’s an image on the internet of a southern black widow that’s really cool." , "Asouthern black widow can be identified by its black coloration with a red hourglass shape on its abdomen.", "A Southern black widow is a type of spider that is black with a red hourglass shape on its belly."
Revised: vessel

Original: "catamaran":"Acatamaran is a sailboat that has two hulls, or wide bodies, that are connected by beams.", "The easiest way to identify a catamaran is by its two hulls.", "A catamaran is a type of boat that has two parallel hulls.
Revised: [Table]

Original: ", "A catamaran is a multi-hulled vessel with two parallel hulls of equal size.
Revised: a multi [Table] hulled vessel with two parallel hulls of equal size [Table]

Original: ","Mydream boat!
Revised: of [Table]

Original: A sleek catamaran that can zip through the waves.", "A catamaran is a multi-hulled vessel with two parallel hulls of equal size."
Revised: white [Table] blue

Original: , "The image is of a white catamaran with blue trim.", "A catamaran is a type of boat that has two hulls, or platforms, that are parallel to each other.
Revised: [Table] is a type of boat that has two hulls [Table] or platforms

Original: ", "The image is of a yellow catamaran with white trim, sitting in calm water.
Revised: [Table]

Original: ","Acatamaran is a type of sailing vessel that consists of two parallel hulls of equal size."
Revised: that are parallel to each other

Original: [Table]
Revised: [Table]

Original: beach
Revised: of a

Original: [Table]
Revised: [Table]

Original: Blueskies, white sands
Revised: white trim

Original: [Table]
Revised: [Table]

Original: and clear turquoise waters make this beach a paradise
Revised: is a type of sailing vessel that consists of two parallel hulls of equal size

Original: [Table]
Revised: [Table]

Original: beachusually has sand and water
Revised: "Blueskies, white sands, and clear turquoise waters make this beach a paradise.

Original: [Table]
Revised: [Table]

Original:  identification can be accomplished through the identification of physical characteristics [Table] is a naturally occurring feature of the landscape [Table]
Revised: Abeachusually has sand and water.", "Beach identification can be accomplished through the identification of physical characteristics.

Original:  sun sets over the ocean, casting a beautiful orange hue in the sky [Table] is a large body of water with sand or small rocks at the shore [Table]
Revised: ", "A beach is a naturally occurring feature of the landscape.", "The sun sets over the ocean, casting a beautiful orange hue in the sky.", "Abeach is a large body of water with sand or small rocks at the shore.", "Intheimage, the beach is Brilliant white with crystal blue waters."

Original: the [Table] , the beach is Brilliant white with crystal [Table]
Revised: , "The beach looks like a long strip of land next to the ocean.

Original: waters [Table] looks like a long strip of land next to the ocean
Revised: ", "A beach typically looks like a large, flat expanse of sand with some rocks or other natural features nearby."

Original: [Table] typically looks like a [Table] expanse of
Revised: "rotary dial telephone": "A rotary dial is a device used to dial telephone numbers.", "Rotary dial telephone from the mid-20th century.", "A rotary dial telephone is a phone with a circular dial on the front face.

Original: [Table] or
Revised: ","Whenlooking

Original: [Table]
Revised: at a rotary telephone, you can tell it is a rotary phone by the Place the phone’s receiver on your ear and listen for a dial tone.", "A rotary dial phone is an older model phone that has a circular device with numbers on it that you rotate with your finger to dial a number.",