  Synthesising photo-realistic images from natural language is one of the
challenging problems in computer vision. Over the past decade, a number of
approaches have been proposed, of which the improved Stacked Generative
Adversarial Network (StackGAN-v2) has proven capable of generating high
resolution images that reflect the details specified in the input text
descriptions. In this paper, we aim to assess the robustness and
fault-tolerance capability of the StackGAN-v2 model by introducing variations
in the training data. However, due to the working principle of Generative
Adversarial Network (GAN), it is difficult to predict the output of the model
when the training data are modified. Hence, in this work, we adopt Metamorphic
Testing technique to evaluate the robustness of the model with a variety of
unexpected training dataset. As such, we first implement StackGAN-v2 algorithm
and test the pre-trained model provided by the original authors to establish a
ground truth for our experiments. We then identify a metamorphic relation, from
which test cases are generated. Further, metamorphic relations were derived
successively based on the observations of prior test results. Finally, we
synthesise the results from our experiment of all the metamorphic relations and
found that StackGAN-v2 algorithm is susceptible to input images with obtrusive
objects, even if it overlaps with the main object minimally, which was not
reported by the authors and users of StackGAN-v2 model. The proposed
metamorphic relations can be applied to other text-to-image synthesis models to
not only verify the robustness but also to help researchers understand and
interpret the results made by the machine learning models.
