# Inv-ReVersion: Enhanced Relation Inversion Based on Text-to-Image Diffusion Models
## 1 Introduction  
In recent years, text-to-image diffusion models [1; 2; 3; 4] have made significant progress. These models can generate high-quality and diverse images from text prompts written in natural language. In practical applications, there is a growing demand for personalized and customized models from users, who expect the models to generate images reflecting specific concepts. In response to this demand, recent studies such as LoRA [5], Textual Inversion [6], DreamBooth [7], and Custom Diffusion [8] have greatly enhanced the adaptability and expressiveness of diffusion models in specific downstream tasks through fine tuning [9; 10].  
However, while these methods excel in capturing the appearance features of images, research on exploring high-level semantics such as the relations between entities in images is scarce. In fact, the relations between entities are one of the key elements that constitute the meaning of images. Capturing these relations is not only a challenging task but also poses higher requirements for the framework: first, the model needs to understand the interactions between entities in images, which involves deep learning of complex semantics; second, the presence of relations often means that there are multiple entities in the image, and when generating images containing multiple entities, it is necessary to prevent improper fusion of features between different entities [11]; finally, given that existing diffusion model fine-tuning methods mainly focus on generating concepts with specific appearances, there is a relative lack of research in capturing relations between entities in images. In this context, ReVersion [12] pioneeringly proposed this issue and attempted to fine-tune the model to generate images with specific relations using the inversion methods.  
Although ReVersion has achieved some success in generating images with specific relation concepts, it has limitations in Zero-Shot [13; 14; 15]; namely, it cannot effectivelygenerate uncommon entities and relations in the real world. To address this issue, we propose the Inv-ReVersion framework, which improves upon ReVersion and achieves better results (Figure 1). We designed a method of inverse relation representation in text prompts, which can effectively reduce feature fusion between entities. Based on the relation-steering loss in the ReVersion framework, we modified the structure of the contrastive loss to enrich the types of relations that can be learned. In addition, we combined digital image processing technology and designed a high-frequency suppressor from the perspective of frequency domain analysis, which helps the model focus less on image details and more on understanding the overall meaning of the images.  
We implemented our framework on Stable Diffusion 1.5 [16] and conducted experiments on a dataset containing 10 different relation concepts. The experimental results show that Inv-ReVersion can significantly reduce the feature fusion problem between entities and has good reconstruction ability for complex abstract relations such as behavioral relations. We studied the contributions of each component of our framework through ablation experiments and compared our framework with existing baselines.  
In summary, our main contributions are:  
* We propose the inverse relation text expansion method, which can reduce feature fusion between entities in generated images.
* We propose a part-of-speech weighted control loss function, enriching the relation categories