Collaborative Co-Design Practices for Supporting Synthetic Data Generation in Large Language Models: A Pilot Study (2025)

Abstract: Large language models (LLMs) are increasingly embedded in development pipelines and the daily workflows of AI practitioners. However, their effectiveness depends on access to high-quality datasets that are sufficiently large, diverse, and contextually relevant. Existing datasets often fall short of these requirements, prompting the use of synthetic data (SD) generation. A critical step in this process is the creation of human seed examples, which guide the generation of SD tailored to specific tasks. We propose a participatory methodology for seed example generation, involving multidisciplinary teams in structured workshops to co-create examples aligned with Responsible AI principles. In a pilot study with a Responsible AI team, we facilitated hands-on activities to produce seed examples and evaluated the resulting data across three dimensions: diversity, sensibility, and relevance. Our findings suggest that participatory approaches can enhance the representativeness and contextual fidelity of synthetic datasets. We provide a reproducible framework to support NLP practitioners in generating high-quality seed data for LLM development and deployment

## introduction in recent years, there has been a growing interest in the integration of artificial intelligence (ai), particularly in the natural language process field, into human-centered design, particularly in the context of human-ai collaboration-how humans and intelligent systems can work together to achieve shared goals and augment human capabilities @xcite @xcite . this shift has prompted a wave of research exploring the human role in ai pipelines @xcite @xcite @xcite , including how we "teach" machines through annotation, crowdsourcing, and interaction design @xcite @xcite . as ai systems become more embedded in everyday life, concerns about their alignment with human values and intentions-known as the ai alignment problem-have gained prominence @xcite @xcite @xcite . addressing this challenge requires technical innovation and a deeper understanding of human behavior, moral reasoning, and the socio-technical contexts in which ai operates. in particular, the development of value-aligned ai systems increasingly relies on synthetic data generation (sdg), where human-created "seed examples" serve as foundational templates for training models at scale @xcite @xcite . despite their critical role, the processes and practices surrounding seed example creation remain underexplored @xcite . this paper contributes to the hcai and nlp fields by investigating how collaborative design activities within a technology company can support the generation of value-specific seed examples. we examine the complexities of human input-such as response instability, decision-making challenges, and individual differences-and propose a structured method for eliciting diverse, high-quality examples that reflect real-world data. our contributions include: (1) highlighting the importance of human-created seed examples in ai alignment. (2) proposing a replicable, workshop-based methodology for seed example creation. (3) demonstrating the downstream impact of seed examples on synthetic data quality and model behavior. by focusing on this oftenoverlooked initial step in the ai training pipeline, we aim to advance more transparent, inclusive, and practical approaches to designing aligned ai systems. 129 2 background ## the project: mitigators this paper is part of a broader research initiative to address the mitigation problem by decoupling it from the original llm response generation, allowing for a post-hoc approach. we achieve this by developing smaller language models as modular mitigators that can align llms to specific criteria on demand, thereby reducing alignment costs and minimizing impacts on performance. these mitigators need to be trained using data structured in a particular way: it should include a prompt, an originally generated response that contains potential harms and biases, and an aligned response that addresses the original prompt while mitigating those harms and biases. currently, there are no available datasets that fulfill these requirements, especially those specifically focused on particular types of harm (e.g., social bias, profanity, etc.). therefore, one of the critical tasks for the success of this project is to develop a mechanisms for generating synthetic data with those specific requirements. ## generating human seed examples in a previous studies with ai practitioners showed that practitioners in charge of developing llms require additional support in the data generation process, underscoring opportunities for improved methodological transparency in synthetic data generation @xcite . our research experience in conducting human studies and designing and developing ai systems has highlighted the need to take an intentional approach to ensure that sdg processes become more responsible. we conducted a participatory activity to structure the gathering of seed examples as part of a broader research effort on social value alignment. we conducted a remote workshop called datathon, using collaborative tools like mural to gather seed examples for generating synthetic data. the gathered seed examples from the datathon would be included as in-context learning (icl) foot_0 for generating synthetic data. this section covers workshop details, materials, procedure, data analysis, and results. ## analyzing the human curated seed examples in this section, we describe and examine the seed examples generated by the participants during the datathon workshops. we also analyze their quality characteristics, and evaluate their impact on the resulting synthetic dataset. in our 'mitigators' alignment approach, these human-curated seed examples are used specifically as in-context learning (icl) examples. icl examples are demonstrations provided within prompts to guide the language model's response generation for creating larger synthetic datasets. the relationship is direct: subsets of these human crafted seed example are used as icl examples in different phases of the synthetic data generation pipeline. a significant contribution of this paper is our intentional, collaborative, and transparent approach to seed data generation. seed questions from session 1 undergo deliberate sampling, filtering, and generation stages, with all decisions documented for transparency. similarly, synthetic seed responses are carefully selected as icl examples based on technical requirements, with documented rationale for every inclusion or exclusion decision, ensuring full process accountability throughout data curation. ## lessons learned and discussion in this paper, we presented our effort to introduce and drive a human-oriented, participatory workshop for creating seed data (e.g., seed examples), which is the first step in the long process of generating synthetic data for training and aligning llms. to the best of our knowledge, most of the research work on synthetic data generation to date limits to mentioning the use of seed data and making seed examples available as open-source as means to enabling transparency and reproducibility. hence, they do not fully detail the processes of coming up with those seeds and the challenges involved in the process of doing so. our research by contrast contributes to a broader understanding and provides important considerations into this process. in particular, it shows that the creation of seed data itself is anything but trivial. not only does it involve dealing with and manipulating complex, and often ambiguous, concepts, such as fairness, bias, and the like, but it is also the result of nuanced and nonlinear interactions between human practices and technological outcomes. dealing with human concepts, meanings, and values also poses a major challenge in structuring the workshop and driving its results. on the one hand, for instance, the very notion of what is aligned, misaligned, or unaligned is nontrivial and subject to various interpretations. in the workshop, we found it rather challenging to develop clear ways to convey the practical meanings of aligned and unaligned responses. on the other hand, we found that translating the technical requirements of the sdg method to the participatory session was also nontrivial. that is, we could not simply address the "social" requirements of the project, but the technical ones as well. we often needed to "translate" between these two realities. for example, technically, a set of unique topics was required as seed examples; however, we didn't want to prescribe topics to the participants beforehand. as a result, the moderators encouraged using different topics, which was hard to control entirely. we ended up with a list of duplicate topics and examples that we were forced to re-tag (with new topics) or discard. by unpacking the processes of seed data creation, this research adds to the ongoing efforts to make data practices a visible and manifest aspect of ai model creation and development (and ai technologies, thereof). as stated by various authors (see section 2.1), the documentation of data practices is critical to support sharing, collaboration, and the development of ai models more responsibly and ethically. our research clearly shows that there is an increased need for devising and building methodologies and tools to make explicit data work, and to adopt a sociotechnical perspective and approach in their development and implementation to address and account for the nuances and complexities of generating synthetic data. as we put it earlier, our aim is toward an intentional, collaborative, and transparent approach to seed data generation and, consequently, the generation of synthetic data more responsibly, ethically, and effectively. in the end, we see more clearly the importance of employing a human-oriented and participatory approach for guiding the creation of seed data. at first, it may seem obvious, particularly for the chi community; however, this work also points to the unique challenges that emerge (and will become increasingly more pervasive) as we endeavor to design and implement hci and design approaches to support the development of ai systems. we will be asked to investigate and address the very question of machines and human value alignment, which requires on the one hand a deep understanding of the ways in which humans manifest social values and, on the other, great familiarity with the technologies being developed so that we can evaluate the potential impacts and risks of decisions that are made during these efforts. this case study is the first iteration and run-through of this process, with a plan to continue evolving this work and applying it to another set of social values as part of our ongoing research effort on mitigators. ## limitations while our participatory approach offers valuable insights and helps to foster inclusive model alignment, it is not without limitations. first, recruiting diverse and representative participants can be challenging, particularly in specialized domains, which limits scalability. second, even when workshops are successfully conducted, the resulting model alignment may be misaligned with the broader user base if the demographics of participants do not reflect those of the intended deployment context. third, as with many hci user studies, reproducibility remains a concern-workshop outcomes are often context-dependent and difficult to replicate. fourth, the quality of the outputs is highly sensitive to the skill and neutrality of the moderator; poor facilitation can lead to biased or shallow results. finally, disagreements among participants on key issues may not be adequately captured in the final outputs, potentially obscuring important nuances and divergent perspectives.