A gri LLM :Harnessing Transformers for Framer Queries (2024)

Abstract: Agriculture, vital for global sustenance, necessitates innovative solutions due to a lack of organized domain experts, particularly in developing countries where many farmers are impoverished and cannot afford expert consulting. Initiatives like Farmers Helpline play a crucial role in such countries, yet challenges such as high operational costs persist. Automating query resolution can alleviate the burden on traditional call centers, providing farmers with immediate and contextually relevant information.The integration of Agriculture and Artificial Intelligence (AI) offers a transformative opportunity to empower farmers and bridge information gaps.Language models like transformers, the rising stars of AI, possess remarkable language understanding capabilities, making them ideal for addressing information gaps in agriculture.This work explores and demonstrates the transformative potential of Large Language Models (LLMs) in automating query resolution for agricultural farmers, leveraging their expertise in deciphering natural language and understanding context. Using a subset of a vast dataset of real-world farmer queries collected in India, our study focuses on approximately 4 million queries from the state of Tamil Nadu, spanning various sectors, seasonal crops, and query types.

## introduction agriculture serves as a pivotal foundation for global economies, engaging a considerable workforce and making noteworthy contributions to the gross domestic product (gdp) in numerous countries @xcite . this is especially true for developing countries, where agriculture often forms the backbone of the economy, providing employment for a large portion of the population and significantly driving economic growth and stability @xcite . while playing a crucial role, the sector encounters numerous challenges, especially concerning the distribution of knowledge and accessibility to expert guidance. these factors are essential for improving agricultural productivity and sustainability. overcoming these challenges through better information dissemination and access to expert resources is critical for enhancing agricultural outcomes and ensuring long-term sustainability. by addressing these issues, developing countries can boost their agricultural productivity, foster economic development, and reduce poverty @xcite . in recent years, the rapid digitization of the agricultural sector, facilitated by advancements in information and communication technologies (ict), has opened new avenues for addressing these challenges. initiatives like the kisan call center (kcc) in india have marked significant steps toward bridging the information gap faced by farmers @xcite . nevertheless, limitations like network congestion, restricted operational hours, and the proficiency of call center agents emphasize the requirement for scalable and efficient solutions. to boost efficiency and effectiveness in the agricultural sector, there is an urgent demand for advanced solutions capable of addressing the various challenges robustly faced by farmers. conventional methods like manual surveys and on-site visits by experts, while beneficial, are frequently time-consuming and expensive @xcite . the irregular resource allocation makes it challenging to conduct these methods regularly, often taking several days or weeks. this underscores the need for an automated solution to promptly and efficiently address farmer queries. historically, automated query answering relied on keyword matching and predefined rules, leading to robotic and inaccurate responses. however, the recent emergence of language models and transformers signals a transformative phase. recent progress in large language models (llms) and autoregressive llms such as chat-gpt, gemini, mistral, and llama has demonstrated considerable promise. through extensive pretraining, these models can produce high-quality, contextually relevant summaries that are both informative and accessible. however, they come with a substantial computational cost and often require fine-tuning for specific tasks, demanding significant computational resources and energy @xcite . sequence-to-sequence models present a promising solution, effectively addressing these computational challenges. their encoder-decoder structure enables them to efficiently manage input and output sequences, making them well-suited for tasks with small input-output sizes. compared to llms, they are more computationally efficient, requiring fewer resources for fine-tuning and inference while maintaining high performance and suiting some tasks better than causal models @xcite . these models also excel in scenarios where a specific tone, particularly one understandable to users in regions like developing countries, needs to be modeled. this is particularly relevant in regions where users are accustomed to a particular tone of english and generally prefer shorter, more keywordhighlighted conversations. this can be challenging for autoregressive llms when fine-tuned, as they are pre-trained on a large corpus of english text primarily contributed by people from america and europe. this can lead to model forgetting infor-mation learned during pre-training, hallucinations, and subpar performance on downstream tasks. significant progress has been made in developing conversational ai interfaces tailored for various sectors, resulting in diverse solutions that meet the specific needs of stakeholders @xcite . however, there has been limited exploration in the context of agricultural use cases. in particular, there has been no investigation into using large language models (llms) for natural language generation (nlg) to address query answering in the agricultural sector. this work demonstrates the nlg capabilities of llms in automating query resolution within this domain. leveraging their proficiency in interpreting natural language and understanding context, our study aims to present a reliable solution to the challenges faced by farmers in their daily lives. this research is pioneering in its application of llms for sequence-to-sequence generation in the agricultural context. unlike previous efforts that did not fully utilize the advanced capabilities of llms, our approach harnesses these models for natural language generation using a subset of an extensive dataset of real-world farmer queries collected in india. to enhance the quality and clarity of the answers, ensuring they are understandable to the minority of youths with basic education involved in agriculture, we integrated a pre-trained foundation llm @xcite . this model enriches the structure and corrects the grammar of the answers generated by our base model. ## results and analysis we performed experiments with various state-ofthe-art and baseline model architectures as described in section 3.3, and the results of fine-tuning are shown in to assess the consistency and robustness of the llm models for the query generation task across various distributions, we aimed to utilize metadata. this involved a detailed analysis of the models based on three key attributes: seasons (rabi, kharif, and jayad, which are local names for indian agricultural seasons), query types (such as plant protection, nutrient management, fertilizer usage guidelines for specific conditions, government schemes, and regional cultural practices), and crop sectors (e.g., horticulture crops like fruits and vegetables, and other agricultural crops). rounded results. the model exhibits consistently strong performance across different sectors and seasons, though some variability is observed among query types. notably, queries related to fertilizer usage and nutrient management yielded the best outcomes. we saw consistent performance trends across different attributes in all the models we developed, though with increased standard variation. thus, we can conclude that its capabilities are robust and consistent across almost all diverse scenarios. ## conclusion and future works in this work, we harness the power of language models for sequence-to-sequence text generation to aid in addressing farmer queries over the dataset, which consists of authentic farmer query conversations with domain experts from the kcc dataset. our approach involved rigorous data cleaning and preprocessing to eliminate noise from the dataset, marking one of the initial applications of llms for natural language generation in this domain. the fine-tuned models demonstrate robust performance and effective generalization across various attributes, resulting in high-quality text generation. by enhancing tone and grammar with pre-trained large language models (llms), the rephrased output can be tailored to a more general and wider educated audience rather than just the local population. this approach has the potential to contribute to societal well-being by alleviating the workload on conventional call centers and domain experts, providing farmers with immediate and contextually relevant information. this, in turn, establishes the groundwork for a more inclusive and responsive agricultural ecosystem, ensuring universal access to vital knowledge for all farmers and fostering a sustainable and pros-perous future for global agriculture. in future research endeavors, we aim to enhance our proposed methodology by integrating metadata collected alongside farmer inquiries to improve the training using large language models (llms), enhancing their effectiveness. furthermore, we plan to expand our work to diverse regions across india present in the kcc dataset, enabling a more comprehensive resolution of farmers' concerns and collaborating with regional languages relevant to offer more localized assistance.