# Dynamic Fashion Video Synthesis from Static Imagery
## 1 Introduction  
The rise of online clothing shopping has brought about challenges for both customers and businesses. Customers often have to guess whether a product will look good on them or fit properly, leading to a higher risk of returns and lower satisfaction for both parties [1]. Additionally, customers are unable to physically experiment with fashion products, such as feeling and flow when worn, which can result in a less satisfying online shopping experience compared to in-person shopping.  
Currently, there is active ongoing research on how deep learning can help fashion businesses prosper and allow customers to have a better shopping experience [2]. One area of focus is image-based virtual try-on, which uses a deep learning model to combine images of the customer and the desired clothing product to create a try-on image [3, 4]. This helps customers visualise how the clothing item will look on them, making it easier for them to make a purchase decision. Not only can deep learning approaches improve customer shopping experience, but they can make business processes more effective. There are deep learning models that can transform rough fashion design sketches into real products [5], allowing businesses to quickly develop and create new products. This speeds up the experimentation process and enables businesses to bring their initial ideas to life in a timely manner.  
While other diffusion models are either text-to-video or image-to-video using pre-recorded pose sequences, we have developed a new deep learning framework that can help businesses market their fashion products in a more engaging way. This model creates a shortvideo from a still image, showcasing how a piece of clothing moves and flows when worn by an actor. Using a latent diffusion model [6], as well as additional components such as a variational autoencoder (VAE) [7], contrastive language-image pre-training (CLIP) [8], a frame interpolator and attention [9], we are able to capture vital characteristics from the still image and generate a high-fidelity video. Our model stands out from others because it can generate natural and convincing movements with just a single image, unlike other diffusion models that rely on pre-recorded pose sequences or are limited to text data. In addition, we prioritise making diffusion models more efficient to speed up video synthesis and reduce computational demands. This makes it more affordable for businesses, as they do not need to invest in expensive AI accelerators, and reduces the inference time so that customers can see results quickly. Efficiency is often overlooked in the research community. The contribution from our work is the following:  
* Developed a generative model that produces spontaneous fashion videos with realistic movements from a still image. The model is a latent diffusion model, utilising a cross-attention mechanism to combine the noisy latent with the conditioning data.
* Utilised a pre-trained VAE decoder to process each frame from the denoised latent space to synthesise a complete fashion video.
* Demonstration of local and global conditioning enables the model to preserve the most detail from the conditioning image.  
We share our source code and provide pre-trained models on our GitHub repository located at [[https://github.com/1702609/FashionFlow](https://github.com/1702609/FashionFlow)]([https://github.com/1702609/FashionFlow](https://github.com/1702609/FashionFlow)) (accessed on 6 August 2024).