SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1. Carefully look at the reference image. First of all, try to coarsely describe the class of the given image (e.g. table, dog, toy, backpack, etc.). Then pay attention to details, try to figure out what distinguishes this specific object from another from its class: its shape, texture, coloring, additional elements or specific details, etc.
2. If the reference object is present only on one of the pictures select the picture where it is present. If the reference object is not present on any of the pictures select Equal. Else: compare two generated images by their resemblance to the reference. We suggest going from the most general traits to the more specific details:
3. Which of the generated images better captures the class of the original object? If both images equally reflect the coarse class of the object, proceed to step 4. Otherwise, please select a corresponding image.
4. Which of the generated images better reflects the shape of the object being learned? If you think that both images equally reflect the shape of the object, proceed to step 5. Otherwise, please select a corresponding image.
5. Which of the generated images better reflects the colouring of the object being learned? If you think that both images reflect the object's color equally, proceed to step 6. Otherwise, please select a corresponding image.
6. Are there any details that one generated image reflected better than another? If yes, please select a corresponding image. Otherwise, please select "Equally similar".
<REFERENCE IMAGE>
Generated image A:
<GENERATED IMAGE>
Generated image B:
Which of the generated images resembles the reference more?
(a) Instructions for Reconstruction survey.
How to answer the question: Each prompt (text description that was used to generate image) contains a word or phrase in brackets like <cat-statue> (this is a unique identifier of the object being learned by the network) and the description of a scene in which we want to see this object depicted. Descriptions may include specifications of different places, circumstances, change of reference object color, texture, shape, etc.
1. First, read the prompt replacing the object identifier with the word "something". For example if the prompt is "A <cat-statue> in front of the Eiffel Tower", read the prompt as "A something in front of the Eiffel Tower".
2. Now try to identify which image better matches this modified prompt. If one image is better than another, please vote for the corresponding image. Else: proceed to the next step.
3. Now put back the reference object identifier into the prompt and try to answer the following question: "Which image have a <reference-image-object> incorporated into the scene?"
4. To do so, first carefully study the reference image object: try to coarsely classify it, pay attention to its shape, coloring, texture, any patterns and details.
5. Now, for example our example prompt "A <cat-statue> in front of the Eiffel Tower" if one image depicts a random object in front of the Eiffel Tower and the other depicts something similar to the <cat-statue> (e.g. a cat toy or a cat statuette of the similar shape and/or coloring. More specific details are not crucial), the latter image should be chosen.
6. If one image incorporates learned object into the scene and another does not or does it in some details better than another, please select corresponding image. Else: please vote "equal".
Text description: <Text description>
Which of the generated images matches the text description more?
(b) Instructions for Customization survey.
Figure 15: Human annotation interface.
J Correlations of timesteps sampling intervals and metrics
To investigate the impact of varied timesteps, we divide the interval from 0 to 1000 into three equal parts (beginning, middle, and end) and evaluate the  $\mathcal{L}_{det}$  with timesteps sampled from each subinterval. Next, we run the training process over 1500 steps for 9 concepts, with sampling images every 50 steps and calculating four CLIP-based quality metrics: train image similarity, train text similarity, validation image similarity, and validation text similarity.
Table 4: Pearson correlations between losses and quality metrics for different timestep sampling intervals. All results are statistically significant (p-value  $< {0.01}$  )
Begin (0-333)	Middle (333-666)	End (666-1000)	Full (0-1000)
Train text score	0.27	0.33	0.30	0.36
Validation text score	0.48	0.51	0.37	0.50
Train image score	-0.63	-0.64	-0.56	-0.62
Validation image score	-0.58	-0.60	-0.53	-0.56
Inspired by previous research [15], we hypothesize that different timestep intervals might exhibit similarities with different metrics. However, as indicated in Table 4, our results do not reveal such behavior. The values within the rows demonstrate remarkable similarity to each other across all subintervals, even though there are different trends for particular metrics.