OpenRaiser/Intern-Atlas huggingface dataset:
An optimised cuckoo-based discrete symbiotic organisms search strategy for tasks scheduling in cloud computing environment
Venue: arXiv cs.DC 2023
Year: 2023

Currently, the cloud computing paradigm is experiencing rapid growth as there is a shift from other distributed computing methods and traditional IT infrastructure towards it. Consequently, optimised task scheduling techniques have become crucial in managing the expanding cloud computing environment. In cloud computing, numerous tasks need to be scheduled on a limited number of diverse virtual machines to minimise the imbalance between the local and global search space; and optimise system utilisation. Task scheduling is a challenging problem known as NP-complete, which means that there is no exact solution, and we can only achieve near-optimal results, particularly when using large-scale tasks in the context of cloud computing. This paper proposes an optimised strategy, Cuckoo-based Discrete Symbiotic Organisms Search (C-DSOS) that incorporated with Levy-Flight for optimal task scheduling in the cloud computing environment to minimise degree of imbalance. The strategy is based on the Standard Symbiotic Organism Search (SOS), which is a nature-inspired metaheuristic optimisation algorithm designed for numerical optimisation problems. SOS simulates the symbiotic relationships observed in ecosystems, such as mutualism, commensalism, and parasitism. To evaluate the proposed technique, the CloudSim toolkit simulator was used to conduct experiments. The results demonstrated that C-DSOS outperforms the Simulated Annealing Symbiotic Organism Search (SASOS) algorithm, which is a benchmarked algorithm commonly used in task scheduling problems. C-DSOS exhibits a favourable convergence rate, especially when using larger search spaces, making it suitable for task scheduling problems in the cloud. For the analysis, a t-test was employed, reveals that C-DSOS is statistically significant compared to the benchmarked SASOS algorithm, particularly for scenarios involving a large search space.
Evaluating Multi-Global Server Architecture for Federated Learning
Venue: arXiv cs.LG 2023
Year: 2023

Federated learning (FL) with a single global server framework is currently a popular approach for training machine learning models on decentralized environment, such as mobile devices and edge devices. However, the centralized server architecture poses a risk as any challenge on the central/global server would result in the failure of the entire system. To minimize this risk, we propose a novel federated learning framework that leverages the deployment of multiple global servers. We posit that implementing multiple global servers in federated learning can enhance efficiency by capitalizing on local collaborations and aggregating knowledge, and the error tolerance in regard to communication failure in the single server framework would be handled. We therefore propose a novel framework that leverages the deployment of multiple global servers. We conducted a series of experiments using a dataset containing the event history of electric vehicle (EV) charging at numerous stations. We deployed a federated learning setup with multiple global servers and client servers, where each client-server strategically represented a different region and a global server was responsible for aggregating local updates from those devices. Our preliminary results of the global models demonstrate that the difference in performance attributed to multiple servers is less than 1%. While the hypothesis of enhanced model efficiency was not as expected, the rule for handling communication challenges added to the algorithm could resolve the error tolerance issue. Future research can focus on identifying specific uses for the deployment of multiple global servers.
How to Optimize the Environmental Impact of Transformed NoSQL Schemas through a Multidimensional Cost Model?
Venue: arXiv cs.DB 2023
Year: 2023

The complexity of database systems has increased significantly along with the continuous growth of data, resulting in NoSQL systems and forcing Information Systems (IS) architects to constantly adapt their data models (i.e., the data structure of information stored in the database) and carefully choose the best option(s) for storing and managing data. In this context, we propose %in this paper an automatic global approach for leading data models' transformation process. This approach starts with the generation of all possible solutions. It then relies on a cost model that helps to compare these generated data models in a logical level to finally choose the best one for the given use case. This cost model integrates both data model and queries cost. It also takes into consideration the environmental impact of a data model as well as its financial and its time costs. This work presents for the first time a multidimensional cost model encompassing time, environmental and financial constraints, which compares data models leading to the choice of the optimal one for a given use case. In addition, a simulation for data model's transformation and cost computation has been developed based on our approach.
The Importance of Collective Privacy in Digital Sexual and Reproductive Health
Venue: arXiv cs.CR 2023
Year: 2023

There is an abundance of digital sexual and reproductive health technologies that presents a concern regarding their potential sensitive data breaches. We analyzed 15 Internet of Things (IoT) devices with sexual and reproductive tracking services and found this ever-extending collection of data implicates many beyond the individual including partner, child, and family. Results suggest that digital sexual and reproductive health data privacy is both an individual and collective endeavor.
Learning to Skip for Language Modeling
Venue: arXiv cs.CL 2023
Year: 2023

Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning. However, most language models allocate the same amount of parameters or computation to each token, disregarding the complexity or importance of the input data. We argue that in language model pretraining, a variable amount of computation should be assigned to different tokens, and this can be efficiently achieved via a simple routing mechanism. Different from conventional early stopping techniques where tokens can early exit at only early layers, we propose a more general method that dynamically skips the execution of a layer (or module) for any input token with a binary router. In our extensive evaluation across 24 NLP tasks, we demonstrate that the proposed method can significantly improve the 1-shot performance compared to other competitive baselines only at mild extra cost for inference.
Optimizing and Fine-tuning Large Language Model for Urban Renewal
Venue: arXiv cs.CL 2023
Year: 2023

This study aims to innovatively explore adaptive applications of large language models (LLM) in urban renewal. It also aims to improve its performance and text generation quality for knowledge question-answering (QA) tasks. Based on the ChatGLM, we automatically generate QA datasets using urban renewal scientific literature corpora in a self-instruct manner and then conduct joint fine-tuning training on the model using the Prefix and LoRA fine-tuning methods to create an LLM for urban renewal. By guiding the LLM to automatically generate QA data based on prompt words and given text, it is possible to quickly obtain datasets in the urban renewal field and provide data support for the fine-tuning training of LLMs. The experimental results show that the joint fine-tuning training method proposed in this study can significantly improve the performance of LLM on the QA tasks. Compared with LoRA fine-tuning, the method improves the Bleu and Rouge metrics on the test by about 5%; compared with the model before fine-tuning, the method improves the Bleu and Rouge metrics by about 15%-20%. This study demonstrates the effectiveness and superiority of the joint fine-tuning method using Prefix and LoRA for ChatGLM in the urban renewal knowledge QA tasks. It provides a new approach for fine-tuning LLMs on urban renewal-related tasks.
A Comparative and Experimental Study on Automatic Question Answering Systems and its Robustness against Word Jumbling
Venue: arXiv cs.CL 2023
Year: 2023

Question answer generation using Natural Language Processing models is ubiquitous in the world around us. It is used in many use cases such as the building of chat bots, suggestive prompts in google search and also as a way of navigating information in banking mobile applications etc. It is highly relevant because a frequently asked questions (FAQ) list can only have a finite amount of questions but a model which can perform question answer generation could be able to answer completely new questions that are within the scope of the data. This helps us to be able to answer new questions accurately as long as it is a relevant question. In commercial applications, it can be used to increase customer satisfaction and ease of usage. However a lot of data is generated by humans so it is susceptible to human error and this can adversely affect the model's performance and we are investigating this through our work
Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval
Venue: arXiv cs.CL 2023
Year: 2023

Neural 'dense' retrieval models are state of the art for many datasets, however these models often exhibit limited domain transfer ability. Existing approaches to adaptation are unwieldy, such as requiring explicit supervision, complex model architectures, or massive external models. We present $\texttt{ABEL}$, a simple but effective unsupervised method to enhance passage retrieval in zero-shot settings. Our technique follows a straightforward loop: a dense retriever learns from supervision signals provided by a reranker, and subsequently, the reranker is updated based on feedback from the improved retriever. By iterating this loop, the two components mutually enhance one another's performance. Experimental results demonstrate that our unsupervised $\texttt{ABEL}$ model outperforms both leading supervised and unsupervised retrievers on the BEIR benchmark. Meanwhile, it exhibits strong adaptation abilities to tasks and domains that were unseen during training. By either fine-tuning $\texttt{ABEL}$ on labelled data or integrating it with existing supervised dense retrievers, we achieve state-of-the-art results.\footnote{Source code is available at \url{https://github.com/Fantabulous-J/BootSwitch}.}
Only Positive Cases: 5-fold High-order Attention Interaction Model for Skin Segmentation Derived Classification
Venue: arXiv cs.CV 2023
Year: 2023

Computer-aided diagnosis of skin diseases is an important tool. However, the interpretability of computer-aided diagnosis is currently poor. Dermatologists and patients cannot intuitively understand the learning and prediction process of neural networks, which will lead to a decrease in the credibility of computer-aided diagnosis. In addition, traditional methods need to be trained using negative samples in order to predict the presence or absence of a lesion, but medical data is often in short supply. In this paper, we propose a multiple high-order attention interaction model (MHA-UNet) for use in a highly explainable skin lesion segmentation task. MHA-UNet is able to obtain the presence or absence of a lesion by explainable reasoning without the need for training on negative samples. Specifically, we propose a high-order attention interaction mechanism that introduces squeeze attention to a higher level for feature attention. In addition, a multiple high-order attention interaction (MHAblock) module is proposed by combining the different features of different orders. For classifying the presence or absence of lesions, we conducted classification experiments on several publicly available datasets in the absence of negative samples, based on explainable reasoning about the interaction of 5 attention orders of MHAblock. The highest positive detection rate obtained from the experiments was 81.0% and the highest negative detection rate was 83.5%. For segmentation experiments, comparison experiments of the proposed method with 13 medical segmentation models and external validation experiments with 8 state-of-the-art models in three public datasets and our clinical dataset demonstrate the state-of-the-art performance of our model. The code is available from https://github.com/wurenkai/MHA-UNet.
Toward a real-time TCP SYN Flood DDoS mitigation using Adaptive Neuro-Fuzzy classifier and SDN Assistance in Fog Computing
Venue: arXiv cs.CR 2023
Year: 2023

The growth of the Internet of Things (IoT) has recently impacted our daily lives in many ways. As a result, a massive volume of data is generated and needs to be processed in a short period of time. Therefore, the combination of computing models such as cloud computing is necessary. The main disadvantage of the cloud platform is its high latency due to the centralized mainframe. Fortunately, a distributed paradigm known as fog computing has emerged to overcome this problem, offering cloud services with low latency and high-access bandwidth to support many IoT application scenarios. However, Attacks against fog servers can take many forms, such as Distributed Denial of Service (DDoS) attacks that severely affect the reliability and availability of fog services. To address these challenges, we propose mitigation of Fog computing-based SYN Flood DDoS attacks using an Adaptive Neuro-Fuzzy Inference System (ANFIS) and Software Defined Networking (SDN) Assistance (FASA). The simulation results show that FASA system outperforms other algorithms in terms of accuracy, precision, recall, and F1-score. This shows how crucial our system is for detecting and mitigating TCP SYN floods DDoS attacks.
MoDS: Model-oriented Data Selection for Instruction Tuning
Venue: arXiv cs.CL 2023
Year: 2023

Instruction tuning has become the de facto method to equip large language models (LLMs) with the ability of following user instructions. Usually, hundreds of thousands or millions of instruction-following pairs are employed to fine-tune the foundation LLMs. Recently, some studies show that a small number of high-quality instruction data is enough. However, how to select appropriate instruction data for a given LLM is still an open problem. To address this problem, in this paper we present a model-oriented data selection (MoDS) approach, which selects instruction data based on a new criteria considering three aspects: quality, coverage and necessity. First, our approach utilizes a quality evaluation model to filter out the high-quality subset from the original instruction dataset, and then designs an algorithm to further select from the high-quality subset a seed instruction dataset with good coverage. The seed dataset is applied to fine-tune the foundation LLM to obtain an initial instruction-following LLM. Finally, we develop a necessity evaluation model to find out the instruction data which are performed badly in the initial instruction-following LLM and consider them necessary instructions to further improve the LLMs. In this way, we can get a small high-quality, broad-coverage and high-necessity subset from the original instruction datasets. Experimental results show that, the model fine-tuned with 4,000 instruction pairs selected by our approach could perform better than the model fine-tuned with the full original dataset which includes 214k instruction data.
The Battleship Approach to the Low Resource Entity Matching Problem
Venue: arXiv cs.DB 2023
Year: 2023

Entity matching, a core data integration problem, is the task of deciding whether two data tuples refer to the same real-world entity. Recent advances in deep learning methods, using pre-trained language models, were proposed for resolving entity matching. Although demonstrating unprecedented results, these solutions suffer from a major drawback as they require large amounts of labeled data for training, and, as such, are inadequate to be applied to low resource entity matching problems. To overcome the challenge of obtaining sufficient labeled data we offer a new active learning approach, focusing on a selection mechanism that exploits unique properties of entity matching. We argue that a distributed representation of a tuple pair indicates its informativeness when considered among other pairs. This is used consequently in our approach that iteratively utilizes space-aware considerations. Bringing it all together, we treat the low resource entity matching problem as a Battleship game, hunting indicative samples, focusing on positive ones, through awareness of the latent space along with careful planning of next sampling iterations. An extensive experimental analysis shows that the proposed algorithm outperforms state-of-the-art active learning solutions to low resource entity matching, and although using less samples, can be as successful as state-of-the-art fully trained known algorithms.
Two Approaches to the Identity of Processes in BFO
Venue: arXiv cs.IR 2023
Year: 2023

This paper aims to explore processes and their identity with a focus on the upper ontology Basic Formal Ontology (BFO). We begin with a classification based on two basic classes of changes of independent continuants: changes with respect to a single specifically dependent continuant thereof or with respect to the spatial region that its parts occupy. We accordingly distinguish two kinds of simple processes: specifically dependent continuant changes and spatial changes. Next, we investigate a compositional approach to the identity of processes: the identity of any process is determined by the identity of the simple processes that compose them. Then, we consider a causal approach to the identity of processes with recourse to a dispositional view of processes according to which any process is a realization of some disposition. We also examine assumptions on which these two approaches to the identity of processes are based.
Justifiable Artificial Intelligence: Engineering Large Language Models for Legal Applications
Venue: arXiv cs.CL 2023
Year: 2023

In this work, I discuss how Large Language Models can be applied in the legal domain, circumventing their current drawbacks. Despite their large success and acceptance, their lack of explainability hinders legal experts to trust in their output, and this happens rightfully so. However, in this paper, I argue in favor of a new view, Justifiable Artificial Intelligence, instead of focusing on Explainable Artificial Intelligence. I discuss in this paper how gaining evidence for and against a Large Language Model's output may make their generated texts more trustworthy - or hold them accountable for misinformation.
Analytical Queries: A Comprehensive Survey
Venue: arXiv cs.DB 2023
Year: 2023

Modern hardware heterogeneity brings efficiency and performance opportunities for analytical query processing. In the presence of continuous data volume and complexity growth, bridging the gap between recent hardware advancements and the data processing tools ecosystem is paramount for improving the speed of ETL and model development. In this paper, we present a comprehensive overview of existing analytical query processing approaches as well as the use and design of systems that use heterogeneous hardware for the task. We then analyze state-of-the-art solutions and identify missing pieces. The last two chapters discuss the identified problems and present our view on how the ecosystem should evolve.
SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Models
Venue: arXiv cs.RO 2023
Year: 2023

Realistic scene-level multi-agent motion simulations are crucial for developing and evaluating self-driving algorithms. However, most existing works focus on generating trajectories for a certain single agent type, and typically ignore the consistency of generated trajectories. In this paper, we propose a novel framework based on diffusion models, called SceneDM, to generate joint and consistent future motions of all the agents, including vehicles, bicycles, pedestrians, etc., in a scene. To enhance the consistency of the generated trajectories, we resort to a new Transformer-based network to effectively handle agent-agent interactions in the inverse process of motion diffusion. In consideration of the smoothness of agent trajectories, we further design a simple yet effective consistent diffusion approach, to improve the model in exploiting short-term temporal dependencies. Furthermore, a scene-level scoring function is attached to evaluate the safety and road-adherence of the generated agent's motions and help filter out unrealistic simulations. Finally, SceneDM achieves state-of-the-art results on the Waymo Sim Agents Benchmark. Project webpage is available at https://alperen-hub.github.io/SceneDM.
FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene Syntax
Venue: arXiv cs.CV 2023
Year: 2023

Text-to-video (T2V) generation is a rapidly growing research area that aims to translate the scenes, objects, and actions within complex video text into a sequence of coherent visual frames. We present FlowZero, a novel framework that combines Large Language Models (LLMs) with image diffusion models to generate temporally-coherent videos. FlowZero uses LLMs to understand complex spatio-temporal dynamics from text, where LLMs can generate a comprehensive dynamic scene syntax (DSS) containing scene descriptions, object layouts, and background motion patterns. These elements in DSS are then used to guide the image diffusion model for video generation with smooth object motions and frame-to-frame coherence. Moreover, FlowZero incorporates an iterative self-refinement process, enhancing the alignment between the spatio-temporal layouts and the textual prompts for the videos. To enhance global coherence, we propose enriching the initial noise of each frame with motion dynamics to control the background movement and camera motion adaptively. By using spatio-temporal syntaxes to guide the diffusion process, FlowZero achieves improvement in zero-shot video synthesis, generating coherent videos with vivid motion.
Utilizing Explainability Techniques for Reinforcement Learning Model Assurance
Venue: arXiv cs.LG 2023
Year: 2023

Explainable Reinforcement Learning (XRL) can provide transparency into the decision-making process of a Deep Reinforcement Learning (DRL) model and increase user trust and adoption in real-world use cases. By utilizing XRL techniques, researchers can identify potential vulnerabilities within a trained DRL model prior to deployment, therefore limiting the potential for mission failure or mistakes by the system. This paper introduces the ARLIN (Assured RL Model Interrogation) Toolkit, an open-source Python library that identifies potential vulnerabilities and critical points within trained DRL models through detailed, human-interpretable explainability outputs. To illustrate ARLIN's effectiveness, we provide explainability visualizations and vulnerability analysis for a publicly available DRL model. The open-source code repository is available for download at https://github.com/mitre/arlin.