Is this Chatbot Trying to Sell Something? Towards Oversight of Chatbot Sales Tactics (2025)

Abstract: This research investigates the detection of covert sales tactics in human-chatbot interactions with a focus on the classification of solicited and unsolicited product recommendations. A custom dataset of 630 conversations was generated using a Large Language Model (LLM) to simulate chatbot-user interactions in various contexts, such as when interacting with users from different age groups, recommending different types of products and using different types of sales tactics. We then employ various approaches, including BiLSTM-based classification with sentence and word-level embeddings, as well as zero-shot, few-shot and CoT classification on large state-of-the-art LLMs. Our results show that few-shot GPT4 (86.44%) is the most accurate model on our dataset, followed by our compact SBERT+BiLSTM model (78.63%) - despite its small size.Our work demonstrates the feasibility of implementing oversight algorithms for monitoring chatbot conversations for undesired practices and that such monitoring could potentially be implemented locally on-device to mitigate privacy concerns. This research thus lays the groundwork for the development of auditing and oversight methods for virtual assistants such as chatbots, allowing consumer protection agencies to monitor the ethical use of conversational AI.

## introduction the rapid growth of chatbots and virtual assistants has been significantly driven by the success of ai systems like chatgpt since 2022, impacting emarketing @xcite . these systems serve dual roles as supportive companions providing emotional assistance, such as snapchat's my ai, and as strategic tools for marketers to engage consumers @xcite . however, this dual functionality introduces ethical issues involving privacy concerns, trust erosion and potential manipulative practices that may prioritise commercial objectives over user wellbeing @xcite @xcite . the european union's ai act further underscores the need for regulation of ai systems capable of influencing human decisionmaking @xcite . existing nlp research has extensively explored conversational systems, focusing particularly on their persuasive capabilities @xcite @xcite . despite this attention, relatively little research has specifically addressed automatic identification of unsolicited or hidden sales recommendations in chatbot conversations. given the increasing use of chatbots in customer service roles and rising demands for ethical ai oversight, addressing this gap is increasingly important @xcite . furthermore, detecting different types of product recommendations is important for improving the transparency and accountability of chatbot interactions. unsolicited recommendations, in particular, can indicate covert sales tactics that may undermine user trust and raise regulatory concerns. this paper addresses this gap by introducing a new dataset and exploring initial detection methods for hidden sales tactics in chatbot conversations. it aims to provide a foundation for future work on detecting such hidden strategies in conversational ai. our research investigates one main research question: how well can nlp models detect solicited and unsolicited product recommendations by chatbots? our findings reveal that a compact model combining sbert embeddings with bil-stm achieves strong accuracy (78.63%), highlighting its practical suitability for local deployment and ethical oversight of chatbot interactions. ## related work as conversational ai becomes more common in commercial applications, concerns have emerged around its potential to subtly manipulate users @xcite . these concerns are particularly relevant in marketing contexts, where virtual assistants are often designed to guide users towards product choices. research has shown that ai systems may be incentivised to influence user behaviour for commercial benefit @xcite , sometimes in ways that are not transparent @xcite @xcite . such covert influence can undermine autonomy by bypassing rational decision-making processes. several recent studies have highlighted the ethical risks of persuasive and emotionally engaging ai. for instance, anthropomorphic design and rapport-building strategies @xcite @xcite can increase user trust but also make users more vulnerable to manipulation. framing techniques and social cues embedded in dialogue @xcite have been shown to shape decisionmaking, blurring the line between persuasion and manipulation. to better understand these tactics, datasets such as mentalmanip @xcite and detection techniques like intent-aware prompting @xcite have been introduced. however, most work focuses on manipulation in general or in social contexts, leaving a gap in detecting manipulative behaviours specifically related to covert product recommendations. our work addresses this gap by introducing a new problem: the controlled detection of covert sales tactics in chatbot-user interactions. instead of collecting real-world conversations, we simulate a variety of controlled scenarios involving different types of product recommendations. we then evaluate a range of nlp models to examine their effectiveness in identifying unsolicited or hidden promotional content. this approach allows us to assess the feasibility of implementing oversight mechanisms for conversational ai in a rigorous and privacy-conscious manner. ## methodology this section presents the methodology employed in generating a dataset of 630 simulated human-ai conversations in diverse contexts and involving diverse products, users, and sales tactics. to achieve this diversity, we used two different approaches to conversation-generation: one which starts from product diversification and another from tactic diversification. ## experiments we investigated multiple approaches for the classification of chatbot selling (solicited, unsolicited, nosales): implementing both a traditional architecture in the form of a bilstm-based model with various word embedding combinations, as well as zero-shot prompting on larger state-of-the-art llms. ## conclusion in this paper we demonstrated the potential of both llms and small locally runnable models (approx. 400mb in the case of sbert-bilstm) for chatbot sales tactic monitoring and oversight. for this purpose we developed a dataset of simulated humanchatbot conversations in which chatbots make solicited, unsolicited, and no-product recommendations. we also showed that sbert-bilstm outperforms larger zero-shot-llm methods, and displays promising preliminary performance (78.63% accuracy) close to few-shot gpt4. our dataset and results lay the groundwork for the development of oversight methods for virtual assistants such as chatbots, allowing consumer protection agencies to monitor the ethical use of conversational ai.