Culture- TRIP : Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement (2025)

Abstract: Text-to-Image models, including Stable Diffusion, have significantly improved in generating images that are highly semantically aligned with the given prompts. However, existing models may fail to produce appropriate images for the cultural concepts or objects that are not well known or underrepresented in western cultures, such as ‘hangari’ (Korean utensil). In this paper, we propose a novel approach, Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement (Culture-TRIP) , which refines the prompt in order to improve the alignment of the image with such culture nouns in text-to-image models. Our approach (1) retrieves cultural contexts and visual details related to the culture nouns in the prompt and (2) iteratively refines and evaluates the prompt based on a set of cultural criteria and large language models. The refinement process utilizes the information retrieved from Wikipedia and the Web. Our user survey, conducted with 66 participants from eight different countries demonstrates that our proposed approach enhances the alignment between the images and the prompts. In particular, C-TRIP demonstrates improved alignment between the generated images and underrepresented culture nouns. Resource can be found at https://shane3606.github.io/Culture-TRIP .

## introduction to date, many text-to-image models @xcite @xcite have demonstrated remarkable improvements. despite the outstanding performance in the text-to-image models, the models fail to align the images with the culture nouns in the prompts, such as 'ao dai'(a vietnamese clothing) or 'hangari' (a korean utensil). most of these issues stem from the large training datasets gathered by crawling the internet without paying attention to the details of the cultural elements @xcite . furthermore, internet access varies significantly across countries @xcite , leading to challenges in appropriately aligning culture nouns due to insufficient data, as shown in figure 1 @xcite . representation is important in ai applications. appropriate representation can positively affect viewers, while inappropriate ones can negatively affect them and can even be harmful @xcite . culture nouns are essential elements that often represent the identity and the uniqueness of the given culture. the misrepresentation of culture nouns by existing text-to-image models may cause dissatisfaction in the corresponding countries. moreover, the models may reinforce harmful stereotypes about particular cultures. in this paper, we introduce a new approach that generates more culturally aligned images for the given culture nouns, called culturally-aware text-to-image generation with iterative prompt refinement (c-trip). c-trip focuses on refining the prompt to ensure that the culture nouns are appropriately represented in the generated images, as shown in figure 1 (c). our goal is to generate culturally-aware images by appropriately aligning culture nouns with images from underrepresented countries. our research question is, how can we refine the prompt so that text-to-image models generate images that appropriately align with culture nouns? the culture capsules @xcite is an educational approach designed to help learners who have not directly experienced a culture gain the proper understanding. this approach explains cultural contexts and visual details, enabling learners to gain comprehensive understand unfamiliar cultures. through this process, learners can develop a deeper profound awareness of different cultures. inspired by the culture capsules, c-trip first retrieves cultural contexts and visual details related to the culture nouns in the prompt and then iteratively refines and evaluates the prompt against the criteria used in culture education. large language models (llms) are used for this iterative refinement and evaluation process, guiding the textto-image model in generating images for culture nouns. we conducted experiments across eight countries, refining a total of 10,000 prompts by using 50 prompt templates for each of the 25 culture nouns per country. to evaluate our results, we recruited participants who were a native of the corresponding country and had a high familiarity with the culture to rank images generated by stable diffusion 2 @xcite , with or without the proposed iterative prompt refinement. the 66 participants across eight countries evaluated the 990 generated images, and c-trip received ratings for cultural alignment that were 18.84% on average higher than the baseline's. in particular, our approach demonstrated more improvement for relatively the unrecognized/underrepresented culture nouns (uc nouns for short) than for the recognized/common culture nouns (rc nouns for short). our contributions are as follows: ## method inspired by the culture capsules @xcite , an educational method that introduces unfamiliar cultures through cultural contexts and visual details, our method refines the prompt for text-to-image models by incorporating cultural contexts and visual details relevant to the culture noun. in order to include only information essential for cultural expressions in the image, external information is retrieved, and an iterative refinement and evaluation process is conducted based on the criteria derived from both cultural contexts and visual details. our overall architecture is depicted in figure 2 . we first retrieve cultural information from wikipedia and web content (section 3.1), then second iteratively refine and evaluate the prompt based on scores (section 3.2) until the stop condition is satisfied. ## results qualitative comparison figure 3 compares the images generated from each c-trip configuration and the base prompt described in section 4.2. the refined prompt generated by c-trip provides cultural knowledge to stable diffusion 2, contributing to the generation of culturally-aware images. c-trip 5 , which includes the visual details criteria, demonstrated higher quality representation. that is, stable diffusion 2 can produce better images when the prompts include appropriate cultural contexts and visual details. with these enhancements, c-trip effectively improves the model's ability to generate culturally relevant images. ## ablation study for uc nouns the culture capsule approach is a method that teaches learners who have not experienced a particular culture. with this idea in mind, we aimed to apply a similar approach to words that stable diffusion 2 is not familiar with. in this section, we analyzed and compared the unrecognized/underrepresented culture nouns (uc nouns) and the common/recognized culture nouns (rc nouns). ## conclusion in this paper, we introduced c-trip (culturally-aware text-to-image generation with iterative prompt refinement), a novel approach that iteratively refines prompts to improve the alignment of culture nouns with images generated by existing text-to-image models without any fine-tuning. experiments across eight countries demonstrated that c-trip significantly improves the alignment of culture nouns in generated images, particularly for underrepresented uc nouns. user surveys and automatic evaluations consistently present c-trip's superior performance in cultural representation and the semantic consistency.