Case Study of GAI for Generating Novel Images for Real-World Embroidery
KATE GLAZKO∗ , University of Washington, USA
ANIKA ARUGUNTA∗ , University of Washington, USA
JANELLE CHAN∗ , University of Washington, USA
NANCY JIMENEZ-GARCIA∗ , University of Washington, USA
TASHFIA SHARMIN∗ , University of Washington, USA
JENNIFER MANKOFF∗ , University of Washington, USA
In this paper, we present a case study exploring the potential use of Generative Artificial Intelligence (GAI) to address the real-world
need of making the design of embroiderable art patterns more accessible. Through an auto-ethnographic case study by a disabled-led
team, we examine the application of GAI as an assistive technology in generating embroidery patterns, addressing the complexity
involved in designing culturally-relevant patterns as well as those that meet specific needs regarding detail and color. We detail theiterative process of prompt engineering custom GPTs tailored for producing specific visual outputs, emphasizing the nuances ofachieving desirable results that align with real-world embroidery requirements. Our findings underscore the mixed outcomes of
employing GAI for producing embroiderable images, from facilitating creativity and inclusion to navigating the unpredictability ofAI-generated designs. Future work aims to refine GAI tools we explored for generating embroiderable images to make them more
performant and accessible, with the goal of fostering more inclusion in the domains of creativity and making.

1

INTRODUCTION AND BACKGROUND

The growing presence of ChatGPT, DALLE-3, Midjourney, and other Generative Artificial Intelligence (GAI) tools has
sparked discussions and research regarding their use in creativity, [11, 26], artwork [7, 10, 15], and making [6, 19, 21, 24].
Emerging research has even highlighted the potential of using GAI to increase accessibility in these domains, such as
easing the process of creating concept sketches for novel art or simplifying the process of learning about visual designs
detailed in the research of Glazko et. al [9]. Utilizing GAI to help make creative work more accessible to people withdisabilities could help address a critically un-met need. People with disabilities have historically been marginalized andleft out of opportunities to participate in creative work [12, 20], making/fabrication [13, 18, 22, 25], and even cultural
heritage due to inaccessibility [2, 5, 16, 17].
Embroidery, an art form with deep cultural value and relevance across global communities, is one example of acreative domain where inaccessibility impacts the participation of disabled makers. Prior work has demonstrated how
integrating embroidery with technology, such as using computionally-generated patterns, improves accessibility (e.g.,[3]). Likewise, research such as Zhang et al. has explored the potential of digital tools to increase participation in
culturally-relevant and traditional embroidery while facilitating learning in novices [27], while Ilieva et al. have calledout the potential for embroidery digitization to serve as a catalyst for sustainable practices in the fashion industry while
preserving digital heritage [14]. However, the process of creating novel designs, such as those suitable for machine
embroidery, can itself be inaccessible to people with disabilities affecting motor coordination, dexterity, or cognition [8].
State-of-the art tools for embroidery digitization such as Hatch [4] which simplify the digitization part of the process
do not simplify or make accessible the process of creating artwork suitable for embroidery. GAI has been proposed as
one method for simplifying image generation and making the creative process more accessible [8, 9].
Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s).

∗ Authors contributed equally to the paper

GenAICHI: CHI 2024 Workshop on Generative AI and HCI

1

Case Study of GAI for Generating Novel Images for Real-World Embroidery

Glazko et al.

In this work, we strive to understand how GAI can be used to foster both creativity and inclusion in the process ofcreating embroiderable artwork patterns. Designing artwork suitable for embroidery can be a complex process, withvariables such as levels of detail and color having noted impacts on the quality of a digitization [23]. This work puts
forward a case study of using GAI as an assistive technology to increase the accessibility of creating embroiderable
images, conducted by disabled-led team of researchers with varying access needs. We present an auto-ethnographic
case study where we use GAI as an access-enabling technology for embroidery and document our experiences. We
finish by detailing important questions for our future work. By investigating the capabilities and limitations of LLMs increative applications such as making computerized embroidery more accessible, we move closer to realizing a future
where AI can contribute meaningfully to creative endeavors, fostering a more inclusive and diverse cultural landscape.
2 CASE STUDIES OF GAI FOR PRODUCING EMBROIDERY IMAGES
2.1

MethodWe conducted an initial evaluation of DALL-E 2 for embroidery (Fall ’23) and a structured, multi-phase case study ofincrementally improving DALL-E 3 with few-shot learning [1] (Winter ’24). The initial evaluation was performed by
one author (A1) with disabilities, and the follow-up case study was conducted by a team of five researchers (A1-A5) withvarying disabled identities/access needs, and with a variety of intersectional identities (i.e. racial identities, genders,
LGBTQIA+ status). For both studies, DALL-E was used to generate images (PNGs) that were converted to vectors
through Adobe software. We note that using PNG to SVG conversion introduced complexity and impacted accessibility
in these early explorations, but GPT-4’s capability for producing SVG outputs was not performant (as shown in Fig. 1a).
In future prototypes of a LLM-based embroidery tool, we can automate this process through libraries without creating
any additional user interaction. Embrilliance and Inkscape were used interchangeably to convert image vectors toembroidery files (.JEF or .PES). Two computerized embroidery machines, a Brother SE1900 and Janome Skyline 9 wereused in our evaluations, with no notable differences in performance due hardware capabilities.
2.2

Initial Exploration

An initial exploration into using GAI for creating machine-embroiderable images was conducted in Fall 2023 by A1,
a disabled researcher whose cognitive and motor disabilities negatively impact their ability to physically or digitallydraw designs. A1 wanted to see if DALL-E 2 could be used to improve access for themselves and produce novel designs
for embroidery on-demand without the need to draw. A1 recorded a detailed log of their experiences using DALL-E 2 toproduce the images that could translate to real-world embroidered patterns. Issues encountered in generating images
through DALL-E 2 included: images with properties that resulted in failed embroidery runs: images with too manysmall details as shown in 1b, images with jagged lines, images that are cut off, images that are not centered, amonst
other issues. A1 had to constantly update their prompt to explicitly define properties needed to produce an image thatwas possible to embroider such as the one seen in Fig. 1c, a repetitive and time-consuming process.
2.3

GAI and Embroiderable Images Case Study

The initial exploration demonstrated the need for constant prompt engineering to produce satisfactory results from
DALL-E 2. The team wanted to see if there were ways to reduce user burden and embed a set of real-world image
properties into GAI by default. To do this, we (A2-A5) iteratively attempted to produce embroiderable images withspecific properties embedded in GPT-4 through the use of the no-code, custom GPTs tool (CU-GPTs). CU-GPTs improves
GenAICHI: CHI 2024 Workshop on Generative AI and HCI

2

Case Study of GAI for Generating Novel Images for Real-World Embroidery(a) Best example of generated SVG dolphin (not recognizable)(b) Generated PNG dolphin with no realworld prompt constraints (led to embroidery errors)

Glazko et al.

(c) Generated PNG duck with real-world
prompt instructions (i.e. image complexity, line width, position)

Fig. 1. DALL-E 2 Initial Exploration

performance for specific through modifying directive through written tasks and through the inclusion of training files
and images. A2-A5 each created a CU-GPTs and assessed its performance in producing specific visual outputs withreal-world properties over the course of two months. A1 interacted with and evaluated the CU-GPTs as an end-user
would with no prompt engineering, producing machine embroidery files from the generated SVGs and fabricating thedesigns. Few-shot learning approaches and optimizations were documented when they occurred. We each independently
wrote down our experiences with CU-GPTs in a shared document. The output CU-GPTs were evaluated qualitatively
on a weekly basis, and from reviewing our logs, we identified common themes.
2.3.1 Optimizing GAI for Producing Specific Image Outputs. Each member of our team produced a CU-GPT withunique physical properties and constraints. The types of embroiderable content the CU-GPTs were trained to produce
included images of: paisleys, lotuses, monstera leaves, and mountain scenery. The CU-GPTs were trained with a no-code,
few-shot learning approach and were iteratively improved through prompt engineering and adding training files. The
additional training files and iterations improved the performance of some of the CU-GPTs, which more consistently
began to produce images that obeyed specific, real-world properties. However, we noted that very explicit instructions,such as specifically banning CU-GPTs from images that had embroidery textures built-in (with texture resulting inoverly-complex SVGs), were necessary for improved performance.
Final Prompt: Generate an image of a simple, minimal-detail Monstera leaf silhouette with a clean
outline. Strictly avoid any appearance of embroidery. Ensure that the image is centered and that itis only the outline of the Monstera leaf with minimal fenestrations. The image should have a whitebackground and should not include small/intricate details. It is difficult for my embroidery machine toprint out small details, so please avoid them. (A3, Monstera CU-GPTs)
A5 likewise found specific, detailed instruction was needed in order to produce a clean, plain background for embroiderable images of mountain scenery. In one instance, she received an image she was very satisfied with, but CU-GPTs
rendered the image inside of a visual embroidery hoop, resulting in a pattern too complex to embroider as shown in Fig.
2c.
2.3.2 Not Fixed By Few-Shot: Instructions Ignored and Inaccuracies. A surprising frustration of using GAI was illustrated
by A4, who tried to use a CU-GPTs to produce images of paisleys, describing her initial experience with CU-GPTs "I
also liked how it was able to emulate cultural patterns inside the paisley" (A4). However, A4 noted how the paisleys
GenAICHI: CHI 2024 Workshop on Generative AI and HCI

3

Case Study of GAI for Generating Novel Images for Real-World Embroidery(a) Success: Machine embroidered generated lotus
image(b) Success: Generated
Monstera converted to an
Embrilliance embroidery
file

Glazko et al.

(c) Error: Generated
Mountain Scene withunwantedbackgroundresulting in 500+ SVGpaths(d) Error: Generated Paisley with excessive dots,
failed at SVG conversion
stage

Fig. 2. Custom GPTs Few-Shot Learning Case Study

produced by the CU-GPTs consistently produced intricate, complicated details that would be difficult to embroider,
specifically dots. A4 made multiple attempts to modify the CU-GPTs to exclude dots from the images through modifying
the directive and uploading sample file images without any dots featured. A4’s CU-GPTs was occasionally able toidentify that the resulting image had dots, and would issue an apology and re-generate the image. Most of the time, theresulting image would have no fewer dots. We found it surprising that despite providing sample images with paisleys
and no dots, CU-GPTs was not able to produce outputs without dots. A Google Image search yieled multiple examplesof paisleys without dots.
2.3.3 CU-GPTs (Mostly) Translated to the Real-World. A1 interacted with the CU-GPTs as they previously had with
out-of-the-box DALL-E 2 and GPT-4. They converted the generated images to SVGs and imported them to Embrilliance,
counting the number of paths, jumps, and difficult-to-embroider details. Overall, A1 described encountering fewer
issues with the CU-GPTs images when compared to the initial evaluations; encountering much fewer issues withimages being off-center, seeing fewer SVGs with excessive paths, and requiring fewer back-and-forth conversations
with GPT. A1 was more satisfied overall with the embroidery files due to the ease of creating successful designs suchas the Monstera shown in Fig. 2b, produced from the CU-GPTs images than in the initial exploration, describing thelotus they successfully embroidered as "beautiful", as shown in Fig. 2a, describing "It is easier to check if an image will
successfully embroider than trying to draw something like a lotus by hand with a stylus" (A1).
3 CONCLUSION, FUTURE WORK, AND OPEN QUESTIONS FOR WORKSHOPThis study highlights the benefits and challenges of using generative AI, specifically custom GPTs (CU-GPTs), in creating
images suitable for embroidery with real-world constraints. Our future work will focus on combining instructions
from the most consistently-performant CU-GPTs such as the one seen in Fig.2b and Fig. 2a to create an optimal
prompt instruction for a CU-GPTs capable of producing more universal imagery. We will quantitatively evaluate our
CU-GPTs and assess whether a more complex approach, such as using images generated from our CU-GPTs, to train
our own model specializing in images for embroidery is needed. Lastly, we will design and test an accessible, GAI-based
embroidery generation tool with disabled creators and makers.
Through our work, we hope to answer the question: "Do generative AI algorithms contribute needed serendipity to
the design process—or simply randomness—or worse, chaos?" We believe our work has demonstrated a mix of both–
GenAICHI: CHI 2024 Workshop on Generative AI and HCI

4

Case Study of GAI for Generating Novel Images for Real-World Embroidery

Glazko et al.

the potential to bring serendipity and joy of making to individuals who previously lacked access, coupled with theboundless frustrations of attempting to control and manipulate the performance of a black-box entity.
REFERENCES[1] Toufique Ahmed and Premkumar Devanbu. Few-shot training llms for project-specific code-summarization. In Proceedings of the 37th IEEE/ACM
International Conference on Automated Software Engineering, pages 1–5, 2022.
[2] Alberto Arenghi and Maria Agostiano. Cultural heritage and disability: Can ict be the ‘missing piece’to face cultural heritage accessibility problems?
In Smart Objects and Technologies for Social Good: Second International Conference, GOODTECHS 2016, Venice, Italy, November 30–December 1, 2016,
Proceedings 2, pages 70–77. Springer, 2017.
[3] Yidan Cao, Karen Anne Cochrane, and Lian Loke. Computationally augmenting traditional embroidery practices: an autobiographical design processwith first-person patient experience for amblyopia follow up treatment activity. In Designing Interactive Systems Conference, pages 1133–1147, 2022.
[4] Hatch Embroidery. Hatch embroidery. https://hatchembroidery.com. Accessed: 2024-05-06.
[5] Ashley Fisher. Out of sight, out of mind: accessibility for people with hidden disabilities in museums and heritage sites. PhD thesis, University of York,
2023.
[6] Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generativemodel of high quality 3d textured shapes learned from images. Advances In Neural Information Processing Systems, 35:31841–31854, 2022.
[7] David Garcia, Joseph Davis, Charles Rodriguez, Richard Miller, and Thomas Martinez. Blending art and technology: Creating a multimedia
experience with gpt-3, dall-e mini, and google tts.
[8] Kate S. Glazko and Jennifer Mankoff. Case study of gai for accessible design and prototyping. AVA: Accessibility, Vision, and Autonomy Meet CVPR
2023 Workshop, 2023.
[9] Kate S Glazko, Momona Yamagami, Aashaka Desai, Kelly Avery Mack, Venkatesh Potluri, Xuhai Xu, and Jennifer Mankoff. An autoethnographic
case study of generative artificial intelligence’s utility for accessibility. In Proceedings of the 25th International ACM SIGACCESS Conference on
Computers and Accessibility, pages 1–8, 2023.
[10] Jennifer Haase, Djordje Djurica, and Jan Mendling. The art of inspiring creativity: Exploring the unique impact of ai-generated images. 2023.
[11] Jun-Yan He, Zhi-Qi Cheng, Chenyang Li, Jingdong Sun, Wangmeng Xiang, Xianhui Lin, Xiaoyang Kang, Zengke Jin, Yusen Hu, Bin Luo, et al.
Wordart designer: User-driven artistic typography synthesis using large language models. arXiv preprint arXiv:2310.18332, 2023.
[12] Rhiann Holloway. Artists as defenders: Disability art as means to mobilise human rights, 2021.
[13] Amy Hurst and Shaun Kane. Making" making" accessible. In Proceedings of the 12th international conference on interaction design and children, pages
635–638, 2013.
[14] Julieta Ilieva, Vanya Stoykova, and Zlatin Zlatev. Application of digitized embroideries from the bulgarian folk costume for modern fashion
accessories.
[15] Hyung-Kwon Ko, Gwanmo Park, Hyeon Jeon, Jaemin Jo, Juho Kim, and Jinwook Seo. Large-scale text-to-image generation models for visual artists’creative works. In Proceedings of the 28th International Conference on Intelligent User Interfaces, pages 919–933, 2023.
[16] Antonio Laurìa et al. Accessibility to cultural heritage as a design challenge. Arenghi, A., Garofolo, I., Sormoen, O.(a cura di), Accessibility as a key
enabling knowlwdge for enhancement of cultural heritage. Milano: Franco Angeli, pages 90–104, 2016.
[17] Ann Leahy and Delia Ferri. Barriers and facilitators to cultural participation by people with disabilities: a narrative literature review. Scandinavian
Journal of Disability Research, 24(1):68–81, 2022.
[18] Ben Leduc-Mills, Jaymes Dec, and John Schimmel. Evaluating accessibility in fabrication tools for children. In Proceedings of the 12th international
conference on interaction design and children, pages 617–620, 2013.
[19] Vivian Liu, Jo Vermeulen, George Fitzmaurice, and Justin Matejka. 3dall-e: Integrating text-to-image ai in 3d design workflows. In Proceedings of the2023 ACM designing interactive systems conference, pages 1955–1977, 2023.
[20] Gloria Yuet Kwan Ma. An ecological approach to self-reflections on the inaccessibility of arts and cultural activities to wheelchair users. Disability& Society, pages 1–21, 2022.
[21] Liane Makatura, Michael Foshey, Bohan Wang, Felix HähnLein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine
Owens, Peter Yichen Chen, et al. How can large language models help humans in design and manufacturing? arXiv preprint arXiv:2307.14377, 2023.
[22] JooYoung Seo and Gabriela T Richard. Scaffolding all abilities into makerspaces: a design framework for universal, accessible and intersectionally
inclusive making and learning. Information and Learning Sciences, 122(11/12):795–815, 2021.
[23] SewandGrew. Tips for optimizing embroidery designs for digitizing. https://sewandgrew.com/blog/tips-for-optimizing-embroidery-designs-fordigitizing, Apr 2023. Accessed: 2024-02-27.
[24] Ayça Sönmez and Arzu Gönenç Sorguç. Computer-aided fabrication technologies as computational design mediators. eCAADe 39-Mass Customization
In Design, 1:465–474, 2021.
[25] Katherine Steele, Brianna Blaser, and Maya Cakmak. Accessible making: Designing makerspaces for accessibility. International Journal of Designsfor Learning, 9(1):114–121, 2018.
[26] Florent Vinchon, Valentin Gironnay, and Todd Lubart. The creative ai-land: Exploring new forms of creativity. 2023.
[27] Lekai Zhang, Ming Li, Lingyan Zhang, Xiaojian Liu, Zhichuan Tang, and Yingfan Wang. Mastersu: The sustainable development of su embroidery
based on digital technology. Sustainability, 14(12):7094, 2022.

GenAICHI: CHI 2024 Workshop on Generative AI and HCI

5