Inclusive Leadership in the Age of AI : A Dataset and Comparative Study of LLM s vs. Real-Life Leaders in Workplace Action Planning (2025)

Abstract: Generative Large Language Models have emerged as useful tools, reshaping professional workflows. However, their efficacy in inherently complex and human-centric tasks such as leadership and strategic planning remains underexplored. In this interdisciplinary study, we present a novel dataset and compare LLMs and human leaders in the context of workplace action planning, specifically focusing on translating the abstract idea of inclusion into actionable SMART goals. We developed the Leader Success Bot, a script-based chatbot co-designed with domain experts, to guide more than 250 real-life leaders in generating inclusive workplace action plans. We systematically prompted seven state-of-the-art chat-based LLMs to perform the same task using the socio-demographic data of real-life leaders and instructions co-developed with domain experts. Our publicly released dataset enables direct comparison between human and LLM-generated workplace action plans, offering insights into their respective strengths, biases, and limitations. Our findings highlight critical gaps and opportunities for LLMs in leadership applications, fostering interdisciplinary collaboration and NLP applications.

## introduction generative large language models (llms) are now widely used in both daily life and professional settings. at the workplace, they are extensively used for a variety of tasks, including seeking advice, exploring new topics, and managing communication workflows @xcite . with the rising importance of llm usage in professional work settings, researchers have begun to investigate how llms can support complex, high-level functions such as human resource management, idea generation, and workplace communication @xcite . an important area where llms are being used in the workplace is in assisting individuals and teams in formulating clear, goal-oriented action plans, as it is a cornerstone of effective leadership. research shows concrete, trackable, and specific goals help people stay focused, assess progress, and adapt their behavior to reach targets @xcite . however, to evaluate the effectiveness of llm-generated action plans and compare them meaningfully with those crafted by humans, we first need a benchmark rooted in real-world leadership behavior. thus, we present a novel dataset comprising action plans authored by over 250 real-life leaders. we systematically compare these human-written plans with those generated by leading llms (gpt-4o mini, gemini-2.0-flash, command-a-03-2025, mistral-large, llama-3.3-70b, deepseek-r1, and qwen-plus), focusing on their ability to generate smart (specific, measurable, actionable, relevant, and time-bound) action plans to foster inclusion in teams. inclusion refers to deliberate efforts that ensure individuals from diverse backgrounds feel respected, valued, and empowered to participate fully in organizational activities. unlike diversity, which emphasizes representation, inclusion engages everyone in decision-making and team processes @xcite . researchers have shown that teams make better decisions, generate more creative ideas, and solve complex problems more effectively when individuals feel safe to contribute, express their views, and participate in organizational efforts @xcite @xcite @xcite @xcite . neglecting inclusion reduces employee engagement and long-term growth @xcite . the impact of inclusion reaches beyond internal operations. it shapes how organizations influence broader social systems and public discourse @xcite . in workplaces, inclusion operates at both interpersonal and organizational levels @xcite , and language plays a crucial part in delivering the right message across team members. as it is challenging to transform an abstract concept such as inclusion into actionable results, we collaborated with two domain experts to develop a step-wise training process to inform and educate real-life leaders on creating inclusion action plans. at the same time, researchers have called out the need to integrate technology as a 'jolt' to modify interactions and dynamics at workplaces @xcite . consequently, chatbots are commonly used as a tool in various sectors @xcite . additionally, @xcite has highlighted the need to curate inclusive data based on community-centered frameworks. a focus on inclusive leadership is critical as it is a foundational leadership style that directly shapes how leaders give feedback, resolve conflict, and motivate diverse teams @xcite . accordingly, we build our interdisciplinary project by using the four dimensions of inclusion, that is: (1) recognizing employees' sense of individuality (uniqueness); (2) strengthening belongingness (feeling like an esteemed member of the team); (3) showing appreciation, and (4) supporting organizational efforts, as conceptualized by @xcite . we use their proposed four-dimensional framework of inclusive leadership, from the domain of organizational behavior, to create a rich dataset with our domain experts and then compare it against seven state-ofthe-art llms in the nlp domain @xcite . it is important to note that though many evaluation datasets exist and have been used to study and evaluate domain adaptation of llms @xcite , to our knowledge, our inclusive dataset is novel for the leadership and organizational behavior domain. in order to bridge the gap between theoretical llm capabilities and applied leadership contexts, offering insights into their real-world usability rather than artificial testing scenarios, we utilize a script-based chatbot to avoid inducing ai bias in leader training and llm-prompting parts @xcite . our contributions are threefold: ## related work interactive chatbots are utilized in various contexts to facilitate external and internal communication, conduct employee training, offer customer services, generate ideas, foster well-being, help with recruitment @xcite @xcite and facilitate teamwork @xcite . while some researchers have explored language style @xcite and identified appropriate register for chatbot design @xcite , few chatbots address the topic of diversity, equity, and inclusion (for an exception see @xcite ). to ensure bias is curtailed in the design of language models, it is important to first identify it. researchers have analyzed leadership-related gender biases perpetuated by llms @xcite . however, much of the analysis has been limited to a few handpicked llms @xcite . to address this, we examine a set of seven state-of-the-art llms in our study. further, @xcite highlighted the trade-off between task efficiency and biased analysis when using llms in a domain-specific manner. prior research on domain-adaptation of llms @xcite @xcite has focused mostly on the domains of news, science, medicine, and government, to our knowledge. we aim to extend this work to the organizational behavior and the hr domain by comparing the real-world usability of llms and actual leaders in workplace action planning. ## evaluation of action plans generated by llms and real-life leaders we conduct a comprehensive linguistic analysis of action plans produced by llms @xcite @xcite @xcite @xcite and real-life leaders, foot_1 comparing structural variations, readability, sentence-level similarity, sentiment, and emotional patterns to systematically evaluate their similarities and differences across multiple dimensions of language use. for all these methods, except for human evaluation, we used the standard python libraries and packages. overall, in comparison to the real-life leaders, gpt-4o mini has a well-balanced pos usage, with more nouns and adjectives, indicating instructional action plans @xcite . the action plans generated by gpt-4o mini have fewer verbs compared to real-life leaders, making it less action-oriented. deepseek-r1 has the lowest verb usage among all sources in generating action plans, suggesting less dynamic text. gemini-2.0-flash and llama-3.3-70b show extremes. while gemini-2.0-flash is the most verbose (average response length: 412 words), llama-3.3-70b generated the longest sentences with an average of 41 words per sentence and high noun usage, indicating likely dense, complex action plans. notably, even though deepseek-r1 has a longer response length (263) than real-life leaders and most llms, it is the most concise in sentence length (14 words/sentence) and has a fairly balanced pos distribution, suggesting fragmentation or bullet-like structure in the generated action plans. real-life leaders, on average, use the most number of pronouns (0.122) and verbs (0.159), indicating a personal tone @xcite and action-oriented plans. llms such as gemini-2.0-flash (0.071), gpt-4o mini (0.070), qwen-plus (0.069), and command-a-03-2025 (0.070) use more adjectives in their action plans than real-life leaders (0.051) and other llms, indicating more descriptive action plans. in essence, real-life leaders are concise, actionoriented, and people-focused, indicating that they are better tuned to real-world action-planning and team engagement. llms vary dramatically in sentence length and words per sentence; therefore, they need to be adapted for the domain-specific audience. overuse of nouns and underuse of verbs and pronouns across all llms make the action plans abstract or impersonal @xcite . ## recommendations llms are widely adopted tools in workplaces. in this interdisciplinary study, spanning nlp and organizational leadership, we draw on our findings and dataset to offer actionable insights and targeted recommendations for future research at the intersection of these domains. future research should consider the following: we also provide a use-case-specific summary of our recommendations in figure 6 . ## conclusion our work advances research on llms in highstakes professional settings such as leadership. we collected a diverse dataset from over 250 real-life leaders, providing a valuable resource for future studies. our comparative analysis shows that current llms often diverge from authentic leadership communication in readability, intensity, and authority. by grounding fine-tuning in real-world communication patterns, emphasizing readability, simplification, and domain-specific language, future models can become more inclusive and effective in workplace applications. our dataset enables researchers and developers to adapt llm behavior to leadership contexts and communication goals. we envision it as a springboard for responsible, context-aware llm deployment in domains such as organizational leadership and human resource management.