## 1 Introduction & Background  
The rapid advancement of Artificial Intelligence (AI) technologies has created unique opportunities for enhancing public service delivery, business operations, and governance. Large Language Models (LLMs) have emerged as powerful tools capable of transforming a wide range of applications through their advanced data processing and natural language understanding capabilities. This paper explores innovative approaches that leverage LLMs to improve efficiency, accuracy, and accessibility across various domains, with a focus on government services, business applications, and other processes. The integration of LLMs into e-government services represents a paradigm shift in public administration, leveraging advanced data processing and automation to significantly enhance service delivery. These models hold substantial potential for optimizing operations, increasing efficiency, and improving citizen interactions through more intelligent and responsive systems [1]. The lack of utilization of AI-based systems in e-government can lead to significant losses and missed opportunities for governments. In terms of competitiveness and innovation, governments that fail to adopt AI-based systems risk falling behind their more technologically advanced counterparts, both domestically and globally. Governments also miss opportunities for cost savings by relying on manual processes that require more human resources, leading to higher operational costs. The absence of automation and optimization prevents them from realizing potential cost savings [2]. Furthermore, without AI-powered data analysis, governments miss out on valuable insights that could inform policy decisions and service improvements. This results in missed opportunities to identify patterns, trends, and areas for optimization within public service delivery [3]. AI-based systems can help tackle challenges in public sector activities, including education, health services, sustainability, public transportation, security services and supervision, waste management, management of electricity and water resources, and the integration of various public services [4]. Embracing the digital transformative potential of AI-powered technologies can enable governments to deliver better public services, make more informed decisions and ultimately, better serve their citizens [5]. The research landscape of LLMs has been marked by significant advancements that have expanded their applicability and efficiency. A notable contribution in this space is the development of an efficient AI model tailored for small- and medium-sized enterprises (SMEs), utilizing the BLOOM approach for multilingual adaptability in English and Vietnamese. By implementing Low-Rank Adaptation (LoRA) and DeepSpeed, this study reduces computational costs and training time, making advanced AI accessible to SMEs with limited resources. The model's domain-specific customization and robust performance validation in smart city applications further demonstrate its versatility and cost-efficiency [6; 7]. In the orbit of AI governance, a conceptual framework has been proposed to address the complexities of regulating AI and autonomous systems. This layered approach divides governance into social, legal, ethical, and technical layers, ensuring a comprehensive and adaptable system. The framework emphasizes transparency, accountability, and inclusivity, providing a structured method to navigate the challenges posed by AI technologies. This study advances current methodologies by integrating modularity and layered governance, offering a scalable and flexible model applicable across various contexts [8]. The integration of LLMs with traditional methods to enhance intelligent Q&A systems in the government sector represents another significant advancement. The Large Language Model Help (LLMH) framework leverages the contextual understanding capabilities of LLMs and combines them with traditional retrieval methods to address challenges such as intent recognition and transfer in government-related queries.