DeepStyle (Part 2 ): The Fashion GAN | by Raymond Cheng | Towards Data Science

If you haven't read Part 1, please read it first since this article assumes you have read Part 1 already.
towardsdatascience.com
Now assuming you have read Part 1, let’s go on.
After gathering the Pinterest database, now we can perform inference on these images using our previously trained Faster R-CNN. But before we do that, we need to first add the functionality where we will crop the detected object and save the resulting image, since this functionality is not provided right out of the box. You can do this by going to the github repo of this project and download the vis.py . Then, navigate to detectron/utils and replace the existing vis.py with the downloaded version. The new vis.py is the same with the one already provided but with one major difference — crop the object detected and save it in a directory.
The added code predicts the category for the detected object, and if the category is “Full” meaning full-body clothing, then it will crop the image and save it in the directory specified. We only save full-body clothing because we want to be able to generate full-body clothing as well, not only simple shirts or skirts.
After tiny modifications, we are ready to run our model on the Pinterest dataset! We can run inference on our Faster R-CNN previously trained by:
python tools/infer.py \   --im [path/to/image.jpg] \   --rpn-pkl [path/to/rpn/model.pkl] \   --rpn-cfg configs/12_2017_baselines/e2e_faster_rcnn_R-50-FPN_1x.yaml \   --output-dir [path/to/output/dir]
The [path/to/image.jpg] is the directory where we store our Pinterest images, --rpn-pkl is where we previously saved our model .pkl file, --rpn-cfg is where we stored our configs file, and finally, --output-dir is where we want to save our predictions. However, this --output-dir is not that important as it will contain the uncropped images with the predictions. Where we want to look for is the directory we specified in vis.py because that’s where the cropped images will be saved.
After performing inference on the model, we should get the cropped images with the clothing centered, and the model, as well as the background, mostly removed. Even though there is still some noise, what we have is already good enough.
Now that we finally have our high-quality clothing images, we can start building the DCGAN model!
Note: The code is based on the official DCGAN tutorial from Pytorch where you access from here. The code won’t be explained in too detail, you can refer to the tutorial for more detailed explanations.
Let’s begin. First we have to import all the necessary libraries:
Next, we set up all the variables that we’ll need later on:
After setting up the variables, we now create the dataset and dataloader which we will feed into our model later. We resize the images, center crop them to the desired image size, and normalize them. Our image size is set to 64 because smaller sizes are normally more consistent.
We also plot some training images to visualize:
After that, we define the weights initialization on the generator and discriminator about the be built:
We build the generator:
And the discriminator!
Then we define the training process. We use BCELoss function because the job of the discriminator is identifying whether an image is real or fake. We setup Adam optimizers for both the generator and the discriminator. Then, we update both networks batch-by-batch:
This will start the training process. The output is:
The training process will take a while. After training, we can plot the generator and discriminator loss during training:
Then after all this work, this is the last step — to save the images to local hard disk:
And finally guys, after all this work! We get to see the generated results:
Now we can plot and compare between the real images from our dataset and generated images, side-by-side.
Not bad right? The fake images are not that far off from real images. In fact, some actually look quite high-fashion to my eyes 😁.
And that’s it guys! Hope you all enjoy my article and hope to see you again. Again, the entire code shown in this article as well as the paper I wrote can be accessed in my github repo! If you like my content, please follow me on Medium, I will post regularly about topics on deep learning!