Prompt2Poster: Automatically Artistic Chinese Poster Creation from Prompt Only

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 11:03:26

Abstract: As a critical component in graphic design, artistic posters are widely applied in the advertising and entertainment industry, thus the automatic poster creation from user-provided prompts has become increasingly desired recently. Although existing Text2Image methods create impressive images aligned with given prompts, they fail to generate ideal artistic posters, especially posters with Chinese texts. To create desired artistic Chinese posters including an aligned background, reasonable layouts, and stylized graphical texts from given prompts only, we propose an automatic poster creation framework, named Prompt2Poster. Our framework first utilizes the capacity of the powerful Large Language Model (LLM) to extract user intention from provided prompts and generate the aligned background. For the harmonious layout and graphical text generation, we propose Controllable Layout Generator (CLG) and Graphical Text Generator (GTG) modules that both leverage sufficient multi-modal information, leading to accurate and pleasurable visual results. Comprehensive experiments demonstrate that our Prompt2Poster achieves superior performance especially on text quality and visual harmony than existing poster creation methods. Our codes will be released after the paper review. Our codes will be released after the paper review.

Main paper: 
 introduction: 
 Designing and integrating visually appealing backgrounds and Chinese graphical texts for artistic Chinese posters represent a critical task in visual information presentation. Traditional Chinese poster creation requires professional designers to invest a significant amount of time. In recent years, some methods [10, 22, 31, 32] have been proposed to partly automate the poster creation through multistep division. However, these methods involve manual operations in some steps, such as selecting existing backgrounds or element attributes, still leading to the necessity of expensive professional labor. Intuitively, obtaining a poster only from a user-provided prompt describing the desired artistic effect would align naturally with human interaction instincts. This significantly streamlines the design process, enhancing creation efficiency, and saving extensive manual labor. Thus, such a prompt-guided automatic poster creation framework has gradually caught greater attention for high interaction, efficiency, and economy.With the proposal and subsequent developments of Denoising Diffusion Probabilistic Model [3, 12, [25] [26] [27] [28] , Large Text-to-Image (T2I) Diffusion Models have demonstrated remarkable performance in generating aesthetically pleasing and style-diverse images by user prompts. Some fine-tuning or plus versions of these T2I Diffusion models [3, 12, [25] [26] [27] [28] , with linguistic and optionally graphical control through LoRA [14] and ControlNet [37] , have achieved state-of-the-art for prompt-guided poster generation.Despite performing excellently in prompt-guided poster generation, limited by the diffusion framework [3, 12, [25] [26] [27] [28] , these methods face a critical problem of data distribution mixing. For an artistic Chinese poster, the background and Chinese graphical texts 树下，雪花飘飘地落下， 周围的空气寒冷而清新。 图上写着"雪树"，"别 有一番素雅之美"。 (Beneath trees, snowflakes drift down, filling the air with a cold yet crisp freshness."Snow-Covered Tree" and "A Distinctive and Simple Beauty" are written on the image.) 夜晚，湖面倒映出摇曳的 光芒。远方的山影与湖面 相连。图上写着"闪湖" 和"山水间的一抹静美"。 (At night, the lake reflects shimmering lights. Distant mountains blend with the tranquil waters. "Glistening Lake" and "A Touch of Serene Beauty Between Mountains and Water" are written on the image.)Figure 2 : Artistic Chinese posters generated by our Prompt2Poster and other poster generation methods, including SD XL [25] , DALL•E 3 [1], Midjourney, GlyphControl [36] , and SD XL + ControlNet (Canny) [37] . The leftmost column represents the prompts, while the remaining six columns display the images generated by different methods. Visual results demonstrate that Prompt2Poster achieves more accurate and pleasurable artistic Chinese poster creation than the compared methods.are extremely different in terms of data distribution. Most of these prompt-guided poster generation methods try to obtain a poster containing the two different distributions from a single distribution, largely weakening their data fitting for the Chinese graphical texts. Thus, they fail to promise the visual correctness and style diversity of Chinese graphical texts on the posters, which are the keys to artistic Chinese poster creation.To guarantee the fitting of the two different data distributions, we propose a novel poster generation framework, named Prompt2-Poster, shown in Fig. 1 . According to Fig. 1 , although only fed into linguistic control from the user prompt, inner multi-modal information is fully utilized to catch the two different data distributions. Specifically, a Large Language Model (LLM) [20] first extracts various information from the user-provided prompt. Then, a pre-trained Large Text-to-Image (T2I) Diffusion Model with high fitting for the poster background data distribution, generates a high-quality background based on the extracted information. After that, a novel Controllable Layout Generator (CLG), is proposed to harmoniously decide the placement for Chinese graphical texts on the background, based on both the visual and linguistic information, coming from the LLM and T2I model, respectively. The output text layout from CLG contains the placement for Chinese graphical texts. This geometrical text layout, along with previous visual and linguistic information is further sent to a novel proposed Graphical Text Generator (GTG). GTG utilizes the three modal information and continuous spaces to generate correct and style-diverse Chinese graphical texts from the graphic data distribution. A high-quality artistic Chinese poster is conveniently finished after integrating stylized Chinese graphical texts with the background image based on the text layout. As shown in Fig. 2 , our Prompt2Poster has a much stronger ability to generate correct and diverse Chinese graphical texts on the poster with a specific prompt, for treating the two different distributions separately. Extensive experiments show the artistic superiority of our Prompt2Poster, CLG, and GTG for artistic Chinese poster generation.Our contribution can be summarized as follows:• We introduce Prompt2Poster, a unique framework designed specifically for the automatic creation of artistic Chinese posters. This system fully leverages multi-modal information to accommodate different data distributions. • A Controllable Text Layout Generator, named CLG, is proposed to produce text layout according to both the background image and Chinese texts, promising harmony between background and Chinese graphical texts. • We've developed an all-new Graphical Text Generator (GTG) that operates initially in the continuous graphic feature space, aside from predicting color in pixel space. This technique not only maintains text correctness but also enhances the artistic style diversity of Chinese graphical texts. • Through extensive experimentation, we have been able to demonstrate the effectiveness of our Prompt2Poster, along with its integral modules CLG and GTG, in the creation of artistic Chinese posters. The results highlight the capability of our system to reliably produce high-quality posters, reinforcing the value and potential of our approach. 
related work: 
 Prompt-Guided Poster Generation. With the rapid development of Large Text-to-Image (T2I) Diffusion Models [3, 12, [25] [26] [27] [28] , T2I-Diffusion-based poster systems have been widely applied to prompt-guided poster generation for their huge interaction, efficiency and economy. By adding linguistic and optional graphical control by LoRA [14] and ControlNet [37] , they return an attractive poster from the noise. However, generating visually correct and various Chinese graphical texts on posters remains a significant challenge for general T2I methods. Recent poster-specific generation methods including AutoPoster [22] , Desigen [33] , and Visual Layout Composer [29] are proposed, which achieve remarkable creation results for artistic posters. However, their methods most focus on component layout planning thus requiring visual elements as input necessarily, resulting in limitations when users desire to create posters from scratch. We highlight that our Prompt2Poster is a fully automatic and generative framework for artistic Chinese poster creation, which achieves correct and pleasing Chinese posters from given prompts only.Controllable Text Layout Generation. The core of the previous controllable text layout generation models [5, 10, 13, 31, 38] is to predict the placement of graphical texts which is visually harmonious and do not cover the semantic area of the background. Traditional methods [10, 31] take template or optimization strategies while deep-learning-based methods [5, 13, 38] utilize Condition-GAN [9] to achieve better results. However, only receiving the background image, all previous advanced learning-based methods [5, 13, 38] are not able to consider Chinese texts, leading to their complete incompatibility with our Prompt2Poster. Stylized Graphical Text Generation. Following most poster systems [10, 22, 31, 32] , in our Prompt2Poster, the stylized attributes of Chinese graphical texts contain the font and color. For font design, previous works primarily concentrate on selecting the most suitable font from a feasible discrete font space, by heuristic method [6] or network learning [22, 31] . The selection in discrete space limits the font variation of Chinese graphical texts. In the realm of color design, some works [15, 31] predict colors based on color rules [16, 30] , while the recent TextPainter [8] utilizes a color style encoder to extract color features and use them to reconstruct a visual graphical text. 
method: 
 In this section, we first introduce the overview of our automatic artistic Chinese poster creation framework Prompt2-Poster in Sec. 3.1. We then describe the design details of two proposed modules including Controllable Layout Generator (CLG) and Graphical Text Generator (GTG) in the following Sec. Given a user prompt 𝒚, Prompt2Poster aims to automatically create an artistic Chinese poster that satisfies all requirements described by the 𝒚. Concretely, Prompt2Poster firstly processes the prompt 𝒚 through a Large Language Model (LLM) [20] to extract the style and content information 𝒔 of the desired poster background, as well as the Chinese text, set 𝑻 :𝒔, 𝑻 = LLM(𝒚).(1)Here, LLM(•) can be various powerful models, such as GPT series [4, 23] and PaLM series [7, 19, 24] . 𝑻 is the text set containing 𝑀 lines of Chinese texts which are expected to appear on the poster graphically:EQUATIONwhere 𝒕 𝑖 is the 𝑖-th Chinese text line consisting of 𝑁 𝑖 Chinese characters defined by the user, and 𝒄 𝑖 𝑗 is the 𝑗-th character in the 𝑖-th text line.According to the extracted background information 𝒔, Prompt2-Poster leverages the Large Text-to-Image (T2I) Diffusion Model to create the poster background 𝑰 with high quality and diversity:EQUATION)We underline that T2I( EQUATIONwhere CLG(•) is our designed Controllable Text Layout Generator, which will be further introduced in Sec. 3.2.With 𝑰 , 𝑻 and 𝑳, we propose a Stylized Graphical Text Generator to create precise and diverse stylized Chinese graphical texts, different from the poster background in terms of data distribution. Each Chinese text line is graphically generated, separately:EQUATIONwhere GTG(•) is our Graphical Text Generator introduced in Sec. 3.3, and 𝒈 𝑖 , 𝒐 𝑖 are the font and color of the 𝑖-th Chinese graphical text line, respectively. Finally, according to the text layout 𝑳, Prompt2Poster integrates the background image 𝑰 and Chinese graphical texts together through geometrical pasting. In our Prompt2Poster, although starting from only the linguistic modal 𝒚, visual and geometrical information 𝑰 and 𝑳 also play an important role in the poster generation, which fully utilizes the inner multi-modal information. Through separately generating backgrounds and Chinese graphical texts with different data distributions, Prompt2Poster can obtain overall harmonious and visually appealing artistic Chinese posters with satisfying graphical texts.To guarantee harmony between the background and Chinese graphical texts, how to place the graphical texts on the background elegantly is extremely critical. Serving as a module in our Prompt2-Poster, our Controllable Layout Generator (CLG) determines the placement of Chinese graphical texts by predicting their center locations and sizes on the background image. Previous state-of-theart controllable text layout models [5, 13, 38] are all designed for generating elements only according to the background 𝑰 . To make the text layout generation controlled by both the background and Chinese texts, further acceptance of linguistic information in our CLG is necessary. This is achieved by adding trainable Number Encoder (NE), Rate Encoder (RE), and fixed Post-processing Cropping module (PC) based on an advanced layout model DS-GAN. Besides considering background 𝑰 , CLG promises the text layout 𝑳 completely matches the Chinese text set 𝑻 in terms of element number and text length [13] .The generation process of CLG is shown in Fig. 3 . A structurally similar discriminator is also designed for training, and more training details are introduced in the supplementary material. Specifically, for a text set 𝑻 containing 𝑀 lines of Chinese texts, NE receives an 𝑥 dimension one-hot layout number vector 𝑯 = (ℎ 1 , ℎ 2 , ..., ℎ 𝑥 ) and RE takes a same dimension rate vector 𝑹 = (𝑟 1 , 𝑟 2 , ..., 𝑟 𝑥 ). 𝑥 is a predefined large number that 𝑀 ≤ 𝑥. The value of ℎ 𝑖 (1 ≤ 𝑖 ≤ 𝑥) is 1 if 𝑖 = 𝑀 else 0, and 𝑟 𝑖 = 𝑁 𝑖 . Only the first 𝑀 elements in 𝑹 have corresponding rate values, and other elements are all zero. Through number control from NE, when there are 𝑀 lines of Chinese texts in 𝑻 , CLG is endowed with the ability to produce text layout 𝑳 with 𝑀 elements exactly. As each graphical character in the same text captures the same geometrical square space [11] , introduced in Sec. 3.3, the size rate of the 𝑖-th element 𝑤 𝑖 /ℎ 𝑖 in 𝑳 must be equal to 𝑖 𝑁 . With RE, the origin DS-GAN generator can directly generate an estimated text layout 𝑳 * , satisfying 𝑤 𝑖 /ℎ 𝑖 very close to 𝑖 𝑁 . Then, PC transforms the 𝑳 * to 𝑳 whose size rate of element completely matches the text length (𝑤 𝑖 /ℎ 𝑖 = 𝑁 𝑖 ), which can be expressed as:EQUATIONEQUATIONwhere 𝑥 * 𝑖 , 𝑦 * 𝑖 and 𝑤 * 𝑖 , ℎ * 𝑖 is the estimated center location and size of the 𝑖-th Chinese graphical text. Upon processing by PC, the text layout 𝑳 thoroughly takes into account both the background image and Chinese text lines. This approach ensures the overall aesthetics and harmony of the final poster. For an artistic Chinese poster, the correctness and style diversity of Chinese graphical texts are extremely crucial. The most critical weakness of previous stylized models is that they predict font in discrete space, leading to loss of font diversity [6, 22, 31] . Our proposed Graphical Text Generator (GTG) generates stylized Chinese graphical texts with diverse fonts and colors while guaranteeing their correctness. Thus, as the main improvement, we utilize the continuous feature space of Diff-Font [11] G 𝑓 to generate Chinese graphical texts character-wise, and each graphical Chinese character in the same text line captures the same geometrical square space. Specifically, G 𝑓 takes a font style encoder [34] E 𝑓 to create a continuous feature style space. Each style embedding in the space determines a specific font of the Chinese graphical character. For the same graphical character, any interpolation of two style embeddings brings a final font style in between, shown in Fig. 5 .Therefore, we can acquire Chinese graphical characters with unlimited font styles theoretically, by first predicting style embeddings 𝒆 and then getting the fonts with G 𝑓 based on 𝒆:EQUATION)This highly enriches the font diversity compared with directly selecting limited font attributes in discrete space [6, 22, 31] . Following previous works [15, 31] , the color prediction is directly in continuous pixel space. Noticing that G 𝑓 is a state-of-the-art font generation model, using which in our GTG naturally guarantees high correctness of generated Chinese graphical texts. As shown in Fig. 4 , the prediction of font attribute is achieved through modal alignment with E 𝑓 , and the prediction of color attribute is accomplished through training from scratch. Specifically, the generation of the 𝑖-th font 𝒈 𝑖 is determined by the Chinese text line 𝒕 𝑖 , background image 𝑰 , and layout 𝒍 𝑖 . Since the local area has a strong influence on the style, we further obtain the local background 𝑰 𝑖 based on 𝒍 𝑖 . Then, the pre-trained Text Encoder in Chinese CLIP [35] is used to encode 𝒕 𝑖 to 𝒇 𝑡 𝑖 , and Image Encoder in the same space is utilized to encode 𝑰 and 𝑰 𝑖 to 𝒇 𝐼 𝑖 and 𝒇 𝑙 𝑖 . Subsequently, we concatenate all the encoded features as the input feature 𝒇 𝒊 = concat(𝒇 𝑡 𝑖 , 𝒇 𝐼 𝑖 , 𝒇 𝑙 𝑖 ) for prediction. To predict the corresponding style embedding 𝒆 𝑖 and color value 𝒐 𝑖 from concatenated feature 𝒇 𝒊 . We propose a Text Attributes Predictor (TAP), which incorporates a Feature Fusion Module (FFM) consisting of multiple MLPs, a linear Font Prediction Head (H 𝑒 ) and a linear Color Prediction Head (H 𝑜 ) for predicting the style embedding 𝒆 𝑖 and color value 𝒐 𝑖 respectively:EQUATION)The font of the 𝑖-th Chinese graphical text 𝒈 𝑖 is then obtained through G 𝑓 character-wise, based on their shared style embedding.We underline that the trainable module in Prompt2Poster is the Controllable Layout Generator (CLG) and the Graphical Text Generator (GTG). Both generators are supervised separately under corresponding losses, and combined with other proposed components as Prompt2Poster framework after training for automatic artistic Chinese poster creation.The training of CLG is inherited from the basic DS-GAN [13] , with a composition loss consisting of an adversarial component, a Due to the significant difference in the scales of L 𝑓 𝑜𝑛𝑡 and L 𝑐𝑜𝑙𝑜𝑟 , we use uncertainty to balance multiple loss functions [17, 21] , thus GTG is optimized as: 11) where 𝛾 1 and 𝛾 2 are learnable parameters, while 𝜇 is the weight to scale the regularization term.EQUATION 
experiments: 
 In this section, we present both quantitative and qualitative results to demonstrate proposed Prompt2Poster achieves superior artistic Chinese poster creation performance. Furthermore, extensive ablations are provided to certify the effectiveness of designed components in our Controllable Layout Generator and Graphical Text Generator. The implementation details are omitted due to the length limitation and can be found in our Appendix.We compared our Prompt2Poster with other five advanced promptguided poster generation methods, including SD XL [25] , DALL•E 3 [1], Midjourney, GlyphControl [36] , and SD XL + ControlNet (Canny) [37] for artistic Chinese poster creation. Following previous works [10, 22, 31, 32] , we evaluate an artistic Chinese poster produced by the above methods from three aspects, including text quality, visual harmony, and overall attraction.They are mainly determined by the visual correctness and style diversity of Chinese graphical texts, the matching between the background and Chinese graphical texts, and the overall artistic attraction to humans, respectively.Since these artistic metrics are subjective, we conduct a user study to provide an intuitive quantification. Specifically, 41 people of various ages, gender, and occupation are invited to evaluate 30 groups' posters. Each group contains 6 posters which are generated by the above six methods from the same prompt. For the 6 posters in the same group, each participant compares and scores them in terms of the above three metrics respectively. It means that a poster will receive three different scores from a participant, corresponding to 暮色的草地中，萤火虫飞舞，星空 在湖面上闪烁。图中写着"夜幕降 临"，"自然的魔法"，"每个萤 火都带着秘密"。 (In the twilight meadow, fireflies dance and the starry sky twinkles on the lake. "Nightfall Descends," "Nature's Magic," and "Each Firefly Carries a Secret" are written on the image.) 机场里行人匆匆, 玻璃墙上灯光闪 烁。图上写着"出发"，"旅途独 特"，"每一步都载着梦想"。 (In the bustling airport, lights twinkle on the glass walls. "Departure," "Unique Journey," and "Each Step Carries a Dream" are written on the image.) the three metrics respectively. We set all the scores ranging from 0 to 10, and a higher score means the poster is better in the degree of a specific metric. Under the setting, we collect a total of 22140 ratings, bringing sufficient data support for the comparison. According to [10, 22, 31, 32] , for each metric, instead of directly calculating the average scores for different methods, we statistic the times that the methods receive the highest score. The results are visually displayed through three pie charts, shown in Fig. 7 . According to Fig. 7 , for each metric, our Prompt2Poster all has a much higher frequency of receiving the highest score compared with any other methods. Concretely, for the occurrence percentage of achieving the best text quality and the best overall attraction, we are both nearly twice as much as the second-best method. For the occurrence percentage of achieving the best visual harmony, we are even close to three times as much as the second-best method. These indicate the great power of our Prompt2Poster for artistic Chinese poster generation. More details about the design of the user study are listed in the supplementary materials. Unlike other kinds of poster creation, the correctness of Chinese graphical texts is the foundation of artistic Chinese poster creation. Thus, a further comparison between our Prompt2Poster and other methods in terms of the correctness of Chinese graphical texts is conducted. Given some user prompts, all the methods generate corresponding posters and we only focus on the correctness of Chinese graphical texts, shown in Fig. 8 .According to Fig. 8 , except for SD XL + ControlNet (Canny) and ours, the other four methods all fail to generate satisfying artistic Chinese posters in terms of correctness. Despite SD XL + ControlNet (Canny) having a better performance than the above four methods, it's still inferior to our Prompt2Poster which has a stronger correctness guarantee. These demonstrate that separately generating backgrounds and graphical texts from different distributions bring better fitting for the graphical data, enhancing the graphical correctness. As shown in Fig. 6 , Prompt2Poster can generate posters with Chinese texts in different row numbers, incorporating elegant Chinese graphical texts. This is realized by a multi-modal framework design, which fully utilizes inner multimodal information and completely exploits the potential of each module in the framework. by removing RE and NE. Following [38] , we utilize a self-regression strategy (SR) for SR-CLG to control the element number in the text Layout. PC is retained since it promises the text layout to finally match the length of the Chinese text. Training details for all these models are introduced in the supplementary materials.To evaluate the text layout generated by the three models, We randomly selected 1500 background images and corresponding Chinese text lines as the input for them to generate specific text layouts. Following [13, 38] , we compare the three models in terms of four numerical indexes, including Ove [18] , Ali [2] , Occ [13] , and Rea [38] . These indexes measure the alignment, occlusion, and visual harmony of the text layout. For each index, the lower the value is, the better the layout is. The results are shown in Table . 1.According to the Table . 1, taking self-regression strategy [38] without NE has a much higher Ove, which means the produced layout elements probably overlap together. This reflects the significance of the NE. Besides, the differences between Ali, Occ, and Rea among the three models are not obvious, indicating that adding modules for text control does not weaken other abilities. For a more intuitive visual comparison, we also run the original DS-GAN [13] by only providing the background images. Related results are shown in Fig. 9 . According to Fig. 9 , origin DS-GAN [13] generates text layouts with complete mistakes of number and size, leading to full incompatibility in our Prompt2Poster. Although promising the correct number of elements in 𝑻 , SR-CLG faces serious element overlap, resulting in unacceptable results. Lacking RE, CLG (w/o RE) cannot estimate the final text layout in advance. This leads to significant cropping by PC and reduces space utilization. However, with the addition of NE and RE, our CLG can produce a text layout that harmonizes with background and aligns texts accurately. Graphical Text Generator. Besides keeping the visual correctness, the main purpose of utilizing GTG in our Prompt2Poster is to bring a huge style diversity of Chinese graphical texts, especially fonts. To verify that using GTG brings Prompt2Poster strongly stylized ability, we remove this module and obtain another system, named Prompt2Poster (w/o GTG). Specifically, following [37] , we only exploit graphical templates to control the generation of Chinese graphical texts. The templates are created with the help of our CLG. More details are shown in the supplementary materials. We compared the style diversity of the Chinese graphical texts for Prompt2Poster (w/o GTG) and Prompt2Poster for each character, illustrated in Fig. 10 . As Fig. 10 shows, without GTG, the styles of those generated Chinese graphical characters are much more monotonous. Particularly, although on different backgrounds, they seem to be very similar in terms of font style. With GTG, in terms of the graphical text style, our Prompt2Poster produces more diverse results and the fonts are much more elegant and varied.In addition, it's worth mentioning that three modal inputs, including vision, geometry, and linguistics, all influence the output of our GTG. As shown in Fig. 11 , any changes to the background image, layout, and Chinese text will bring variations in the generated Chinese graphical text. This full utilization of multi-modal information largely enhances the style diversity and harmony of the generated Chinese graphical texts.  
conclusion: 
 In this work, we propose an automatic prompt-guided poster creation framework named Prompt2Poster for creating desired artistic Chinese posters including an aligned background, reasonable layouts, and stylized graphical texts. Prompt2Poster first extracts the user intention and generates the aligned background supported by powerful large models, and then creates visually pleasurable content with our carefully designed modules including Controllable Layout Generator and Graphical Text Generator, leading to harmonious layout and accurate graphical texts. Extensive experiments demonstrate the superiority of our Prompt2Poster over other prompt-only poster generation methods. We believe our fully automatic poster creation framework will inspire potential research and interesting applications in the graphic design area. 
 
 Appendix:  


Decision: This paper presents a new approach to automatic poster creation leveraging a Large Language Model and advanced graphics generation techniques. The strengths of this work lie in its novel pipeline and the impressive visual results showcased. However, several limitations are also raised by the reviewers. These include providing quantitative comparisons with SOTA methods, clarifying certain technical components, and ensuring the controllability and diversity of the generated outputs.

Reviews: review: paper_summary: paper_summary: The paper presents Prompt2Poster, an automatic framework for creating artistic Chinese posters from user-provided prompts. Utilizing a Large Language Model (LLM), it extracts user intent to generate an aligned background and then employs a Controllable Layout Generator (CLG) and Graphical Text Generator (GTG) to produce harmonious layouts and stylized text. The work's main contributions include proposing a novel approach to artistic Chinese poster creation that leverages multi-modal information to accommodate different data distributions. The paper demonstrates superior performance in text quality and visual harmony compared to existing methods.
strength_weakness: strength_weakness: - Validity of Claims: Prompt2Poster is underpinned by large language models and advanced graphics generation techniques, offering a method for automatic poster creation. It demonstrates innovation and effectiveness in artistic poster creation through carefully designed modules.
- Importance and Novelty of Contributions: The work introduces a new framework for generating artistic Chinese posters, which is a novel contribution to the existing literature. It enhances the level of automation in poster generation and improves the visual and artistic impact of the posters.
- Relevance to ACM MM Community: The work is highly relevant to the research areas of the ACM MM community as it involves the creation, processing, and understanding of multimedia content, which are core technical issues in the multimedia field.
limitations: limitations: - Controllability Concern: The article transforms human control into an automatically generative model, and maintaining controllability in doing so is a concern.
- Robustness Concerns: The visualization results are all pictures of the landscape, there are no posters of the objects or movie posters.
- Comparative Analysis: The lack of comparison with the methodology of the related work section makes one more skeptical of its effectiveness.
paper_summary: paper_summary: This paper presents a method for generating Chinese posters based on prompts. The process begins with a Language Model (LLM) that extracts background descriptions and text content. Following the background description, a Text-to-Image (T2I) model creates the background image. Then, taking into account the background image and the text content, the layout for the texts are estimated. Finally, a graphical text generator predicts the text's color and font, which are combined with the background image through a font generation model. The paper introduces two multimodal models at the layout generation and text attribute estimation stages, called CLG and GTG, respectively.
strength_weakness: strength_weakness: This work effectively leverages a variety of recent models and displays impressive results.
limitations: limitations: - Limited technical novelty. Predicting the font and color of text has been explored in the work referenced in the paper [22], and the concept of predicting the embedding of the font, rather than the set of fonts itself, has also been addressed in existing research (Modeling Fonts in Context: Font Prediction on Web Designs, Pacific Graphics 2018).
- Lack of proper baselines for the whole pipeline, layout generator, and text generator.  For example, GlyphControl and SD XL GlyphControl with ControlNet could be fine-tuned on Chinese datasets before making comparisons, which is a very intuitive and fundamental approach.
- Lack quantitative metrics to assess the text generator's performance in terms of diversity, textual accuracy, harmony among multiple lines of text, and other aspects. From Figure 6, it is observed that within the same poster, the fonts and colors between different lines of text is not very harmonious, especially when there are three lines of text, resulting in significant variance in font styles between lines.
- Lack more quantitative analysis of the layout generator. For example, what impact do the number encoder and rate encoder have on the layout generation, and is the final layout consistent with the input conditions?
paper_summary: paper_summary: This paper aims to generate artistic posters based on the text prompt. The main contribution of this paper is the proposed pipeline for the artistic posters generation. Within this pipeline, a Controllable Text Layout Generator is proposed to generate text layout based on the background image and Chinese text, and a Graphical Text Generator is proposed to generate text with various colored fonts. The authors claim their model achieves superior performance than other existing methods, especially in text quality and visual harmony. The authors also claim that they will release their code after the paper review.
strength_weakness: strength_weakness: 1. From the generated samples provided in this manuscript, the quality of the poster is promising. 
2. This method can generate text images with various colors and fonts.
3. Based on the provided samples, the visual quality has significantly improved compared with other state-of-the-art methods. 
4. This paper is well-prepared and well-written, making it easy to follow.
limitations: limitations: 1. I notice the comparison in this paper mainly focus on the visualization comparison. The authors need to provide a quantitative comparison with other SOTA methods. 
2. In this paper, the authors use SDXL, DALL E, Midjourney, GlyphControl, and SDXL+ControlNet for comparison, and only GlyphControl is designed for text generation. As the authors introduced in related works, there are some generative models focusing on text synthesis. I recommended the authors provide one other text generation model to verify their model. 
3. Generating Chinese characters is difficult, since the number of characters is huge and the structure is complex. How many Chinese characters are supported in this model? What about the generative quality of uncommonly used Chinese characters? 
4. What about the generation results for the characters with complex structures?
5. I notice the characters of the generated text only have a single color. Could this model generate text with the characters in different colors?  
6. I suggest authors add a section to discuss the limitations of their model.
paper_summary: paper_summary: Given that existing generative models either only output images or involve manual operations at certain steps for poster design, this paper focuses on poster generation based solely on a textual prompt as input. This approach is more convenient for users as it does not require them to provide a background image. The proposed model includes three modules: 1) A pre-trained Large Text-to-Image (T2I) Diffusion Model for background generation, where a large language model (LLM) is introduced to extract user intention from the provided prompts. 2) A Controllable Layout Generator (CLG) for layout generation conditioned on the generated background image. 3) A Graphical Text Generator (GTG) that generates text according to the prompt, layout, and background. The qualitative experimental results confirm the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1) The idea of generating a complete poster based on a prompt is novel and promising
2) This paper implements a comprehensive framework for poster generation by introducing diverse functions, which involves the whole poster design process.
3) The proposed evaluation metrics, including text quality, visual harmony, and overall attraction, are novel and reasonable. These metrics reflect a more subjective evaluation of the generated results.
limitations: limitations: 1) How does the LLM extract texts expected to appear on the poster and divide the whole sentence into several text lines considering the line break in the font layout. In other words, how does it decide the number of lines (M)  for a text set (T).
2) The introduction of the trainable Number Encoder (NE), Rate Encoder (RE), and fixed Post-processing Cropping module (PC) needs to be clarified through a real-world case.
3) The Graphical Text Generator is complex and needs to be explained through equations. Additionally, for the CLIP-based predictor, it should be clarified whether the predicted color set and font style are chosen from a predefined set. If not, an explanation of how the prediction works is necessary.
4) The paper lacks quantitative experiments that compare the proposed model with SOTA (State of the Art) methods, which includes text to image generation, text+image to layout and font generation.
5) The experimental setup for both the baselines and the proposed framework should be clarified, including the models and datasets adopted for each part of the model or framework.