OpenRaiser/Intern-Atlas huggingface dataset:
Logarithm-transform aided Gaussian Sampling for Few-Shot Learning
Venue: arXiv cs.CV 2023
Year: 2023

Few-shot image classification has recently witnessed the rise of representation learning being utilised for models to adapt to new classes using only a few training examples. Therefore, the properties of the representations, such as their underlying probability distributions, assume vital importance. Representations sampled from Gaussian distributions have been used in recent works, [19] to train classifiers for few-shot classification. These methods rely on transforming the distributions of experimental data to approximate Gaussian distributions for their functioning. In this paper, I propose a novel Gaussian transform, that outperforms existing methods on transforming experimental data into Gaussian-like distributions. I then utilise this novel transformation for few-shot image classification and show significant gains in performance, while sampling lesser data.
Epistemic Logic Programs: a study of some properties
Venue: arXiv cs.AI 2023
Year: 2023

Epistemic Logic Programs (ELPs), extend Answer Set Programming (ASP) with epistemic operators. The semantics of such programs is provided in terms of world views, which are sets of belief sets, i.e., syntactically, sets of sets of atoms. Different semantic approaches propose different characterizations of world views. Recent work has introduced semantic properties that should be met by any semantics for ELPs, like the Epistemic Splitting Property, that, if satisfied, allows to modularly compute world views in a bottom-up fashion, analogously to ``traditional'' ASP. We analyze the possibility of changing the perspective, shifting from a bottom-up to a top-down approach to splitting. We propose a basic top-down approach, which we prove to be equivalent to the bottom-up one. We then propose an extended approach, where our new definition: (i) is provably applicable to many of the existing semantics; (ii) operates similarly to ``traditional'' ASP; (iii) provably coincides under any semantics with the bottom-up notion of splitting at least on the class of Epistemically Stratified Programs (which are, intuitively, those where the use of epistemic operators is stratified); (iv) better adheres to common ASP programming methodology.
Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey
Venue: arXiv cs.LG 2023
Year: 2023

Differential Privacy has become a widely popular method for data protection in machine learning, especially since it allows formulating strict mathematical privacy guarantees. This survey provides an overview of the state-of-the-art of differentially private centralized deep learning, thorough analyses of recent advances and open problems, as well as a discussion of potential future developments in the field. Based on a systematic literature review, the following topics are addressed: auditing and evaluation methods for private models, improvements of privacy-utility trade-offs, protection against a broad range of threats and attacks, differentially private generative models, and emerging application domains.
Efficient Supervised Training of Audio Transformers for Music Representation Learning
Venue: arXiv cs.SD 2023
Year: 2023

In this work, we address music representation learning using convolution-free transformers. We build on top of existing spectrogram-based audio transformers such as AST and train our models on a supervised task using patchout training similar to PaSST. In contrast to previous works, we study how specific design decisions affect downstream music tagging tasks instead of focusing on the training task. We assess the impact of initializing the models with different pre-trained weights, using various input audio segment lengths, using learned representations from different blocks and tokens of the transformer for downstream tasks, and applying patchout at inference to speed up feature extraction. We find that 1) initializing the model from ImageNet or AudioSet weights and using longer input segments are beneficial both for the training and downstream tasks, 2) the best representations for the considered downstream tasks are located in the middle blocks of the transformer, and 3) using patchout at inference allows faster processing than our convolutional baselines while maintaining superior performance. The resulting models, MAEST, are publicly available and obtain the best performance among open models in music tagging tasks.
Hadoop-Oriented SVM-LRU (H-SVM-LRU): An Intelligent Cache Replacement Algorithm to Improve MapReduce Performance
Venue: arXiv cs.DC 2023
Year: 2023

Modern applications can generate a large amount of data from different sources with high velocity, a combination that is difficult to store and process via traditional tools. Hadoop is one framework that is used for the parallel processing of a large amount of data in a distributed environment, however, various challenges can lead to poor performance. Two particular issues that can limit performance are the high access time for I/O operations and the recomputation of intermediate data. The combination of these two issues can result in resource wastage. In recent years, there have been attempts to overcome these problems by using caching mechanisms. Due to cache space limitations, it is crucial to use this space efficiently and avoid cache pollution (the cache contains data that is not used in the future). We propose Hadoop-oriented SVM-LRU (HSVM- LRU) to improve Hadoop performance. For this purpose, we use an intelligent cache replacement algorithm, SVM-LRU, that combines the well-known LRU mechanism with a machine learning algorithm, SVM, to classify cached data into two groups based on their future usage. Experimental results show a significant decrease in execution time as a result of an increased cache hit ratio, leading to a positive impact on Hadoop performance.
Further results on the Hunters and Rabbit game through monotonicity
Venue: arXiv cs.DM 2023
Year: 2023

Hunters and Rabbit game is played on a graph $G$ where the Hunter player shoots at $k$ vertices in every round while the Rabbit player occupies an unknown vertex and, if not shot, must move to a neighbouring vertex after each round. The Rabbit player wins if it can ensure that its position is never shot. The Hunter player wins otherwise. The hunter number $h(G)$ of a graph $G$ is the minimum integer $k$ such that the Hunter player has a winning strategy (i.e., allowing him to win whatever be the strategy of the Rabbit player). This game has been studied in several graph classes, in particular in bipartite graphs (grids, trees, hypercubes...), but the computational complexity of computing $h(G)$ remains open in general graphs and even in trees. To progress further, we propose a notion of monotonicity for the Hunters and Rabbit game imposing that, roughly, a vertex that has already been shot ``must not host the rabbit anymore''. This allows us to obtain new results in various graph classes. Let the monotone hunter number be denoted by $mh(G)$. We show that $pw(G) \leq mh(G) \leq pw(G)+1$ for any graph $G$ with pathwidth $pw(G)$, implying that computing $mh(G)$, or even approximating $mh(G)$ up to an additive constant, is NP-hard. Then, we show that $mh(G)$ can be computed in polynomial time in split graphs, interval graphs, cographs and trees. These results go through structural characterisations which allow us to relate the monotone hunter number with the pathwidth in some of these graph classes. In all cases, this allows us to specify the hunter number or to show that there may be an arbitrary gap between $h$ and $mh$, i.e., that monotonicity does not help. In particular, we show that, for every $k\geq 3$, there exists a tree $T$ with $h(T)=2$ and $mh(T)=k$. We conclude by proving that computing $h$ (resp., $mh$) is FPT parameterised by the minimum size of a vertex cover.
Augment to Interpret: Unsupervised and Inherently Interpretable Graph Embeddings
Venue: arXiv cs.LG 2023
Year: 2023

Unsupervised learning allows us to leverage unlabelled data, which has become abundantly available, and to create embeddings that are usable on a variety of downstream tasks. However, the typical lack of interpretability of unsupervised representation learning has become a limiting factor with regard to recent transparent-AI regulations. In this paper, we study graph representation learning and we show that data augmentation that preserves semantics can be learned and used to produce interpretations. Our framework, which we named INGENIOUS, creates inherently interpretable embeddings and eliminates the need for costly additional post-hoc analysis. We also introduce additional metrics addressing the lack of formalism and metrics in the understudied area of unsupervised-representation learning interpretability. Our results are supported by an experimental study applied to both graph-level and node-level tasks and show that interpretable embeddings provide state-of-the-art performance on subsequent downstream tasks.
Language Models as a Service: Overview of a New Paradigm and its Challenges
Venue: arXiv cs.AI 2023
Year: 2023

Some of the most powerful language models currently are proprietary systems, accessible only via (typically restrictive) web or software programming interfaces. This is the Language-Models-as-a-Service (LMaaS) paradigm. In contrast with scenarios where full model access is available, as in the case of open-source models, such closed-off language models present specific challenges for evaluating, benchmarking, and testing them. This paper has two goals: on the one hand, we delineate how the aforementioned challenges act as impediments to the accessibility, replicability, reliability, and trustworthiness of LMaaS. We systematically examine the issues that arise from a lack of information about language models for each of these four aspects. We conduct a detailed analysis of existing solutions and put forth a number of considered recommendations, and highlight the directions for future advancements. On the other hand, it serves as a comprehensive resource for existing knowledge on current, major LMaaS, offering a synthesized overview of the licences and capabilities their interfaces offer.
Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation
Venue: arXiv cs.CL 2023
Year: 2023

Zero-shot translation (ZST), which is generally based on a multilingual neural machine translation model, aims to translate between unseen language pairs in training data. The common practice to guide the zero-shot language mapping during inference is to deliberately insert the source and target language IDs, e.g., <EN> for English and <DE> for German. Recent studies have shown that language IDs sometimes fail to navigate the ZST task, making them suffer from the off-target problem (non-target language words exist in the generated translation) and, therefore, difficult to apply the current multilingual translation model to a broad range of zero-shot language scenarios. To understand when and why the navigation capabilities of language IDs are weakened, we compare two extreme decoder input cases in the ZST directions: Off-Target (OFF) and On-Target (ON) cases. By contrastively visualizing the contextual word representations (CWRs) of these cases with teacher forcing, we show that 1) the CWRs of different languages are effectively distributed in separate regions when the sentence and ID are matched (ON setting), and 2) if the sentence and ID are unmatched (OFF setting), the CWRs of different languages are chaotically distributed. Our analyses suggest that although they work well in ideal ON settings, language IDs become fragile and lose their navigation ability when faced with off-target tokens, which commonly exist during inference but are rare in training scenarios. In response, we employ unlikelihood tuning on the negative (OFF) samples to minimize their probability such that the language IDs can discriminate between the on- and off-target tokens during training. Experiments spanning 40 ZST directions show that our method reduces the off-target ratio by -48.0% on average, leading to a +9.1 BLEU improvement with only an extra +0.3% tuning cost.
Perceiving Extrinsic Contacts from Touch Improves Learning Insertion Policies
Venue: arXiv cs.RO 2023
Year: 2023

Robotic manipulation tasks such as object insertion typically involve interactions between object and environment, namely extrinsic contacts. Prior work on Neural Contact Fields (NCF) use intrinsic tactile sensing between gripper and object to estimate extrinsic contacts in simulation. However, its effectiveness and utility in real-world tasks remains unknown. In this work, we improve NCF to enable sim-to-real transfer and use it to train policies for mug-in-cupholder and bowl-in-dishrack insertion tasks. We find our model NCF-v2, is capable of estimating extrinsic contacts in the real-world. Furthermore, our insertion policy with NCF-v2 outperforms policies without it, achieving 33% higher success and 1.36x faster execution on mug-in-cupholder, and 13% higher success and 1.27x faster execution on bowl-in-dishrack.
Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learning
Venue: arXiv cs.RO 2023
Year: 2023

Quadruped robots require robust and general locomotion skills to exploit their mobility potential in complex and challenging environments. In this work, we present the first implementation of a robust end-to-end learning-based controller on the Solo12 quadruped. Our method is based on deep reinforcement learning of joint impedance references. The resulting control policies follow a commanded velocity reference while being efficient in its energy consumption, robust and easy to deploy. We detail the learning procedure and method for transfer on the real robot. In our experiments, we show that the Solo12 robot is a suitable open-source platform for research combining learning and control because of the easiness in transferring and deploying learned controllers.
Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning
Venue: arXiv cs.NE 2023
Year: 2023

A normative approach called Similarity Matching was recently introduced for deriving and understanding the algorithmic basis of neural computation focused on unsupervised problems. It involves deriving algorithms from computational objectives and evaluating their compatibility with anatomical and physiological observations. In particular, it introduces neural architectures by considering dual alternatives instead of primal formulations of popular models such as PCA. However, its connection to the Representer theorem remains unexplored. In this work, we propose to use teachings from this approach to explore supervised learning algorithms and clarify the notion of Hebbian learning. We examine regularized supervised learning and elucidate the emergence of neural architecture and additive versus multiplicative update rules. In this work, we focus not on developing new algorithms but on showing that the Representer theorem offers the perfect lens to study biologically plausible learning algorithms. We argue that many past and current advancements in the field rely on some form of dual formulation to introduce biological plausibility. In short, as long as a dual formulation exists, it is possible to derive biologically plausible algorithms. Our work sheds light on the pivotal role of the Representer theorem in advancing our comprehension of neural computation.
Prediction and Interpretation of Vehicle Trajectories in the Graph Spectral Domain
Venue: arXiv cs.AI 2023
Year: 2023

This work provides a comprehensive analysis and interpretation of the graph spectral representation of traffic scenarios. Based on a spatio-temporal vehicle interaction graph, an observed traffic scenario can be transformed into the graph spectral domain by means of the multidimensional Graph Fourier Transformation. Since these spectral scenario representations have shown to successfully incorporate the complex and interactive nature of traffic scenarios, the beneficial feature representation is employed for the purpose of predicting vehicle trajectories. This work introduces GFTNNv2, a deep learning network predicting vehicle trajectories in the graph spectral domain. Evaluation of the GFTNNv2 on the publicly available datasets highD and NGSIM shows a performance gain of up to 25% in comparison to state-of-the-art prediction approaches.
On Security Strategies for Addressing Potential Vulnerabilities in 6G Technologies Deployable in Healthcare
Venue: arXiv cs.CR 2023
Year: 2023

Researchers are now focusing on 6G as a new network technology that will bring significant gains over the previous generations while many sectors are still implementing the 5G network in their business processes and operations. Meanwhile, key technological fields that will be influenced by 6G networks have been identified. These include distributed artificial intelligence, intelligent radio, real-time intelligent edge computing, and 3D intercoms. Additionally, each area and potential application of 6G is supported by relevant emerging technologies. Nevertheless, these 6G technology and applications have significant security vulnerabilities that must be addressed before the complete adoption of 6G networks. The healthcare is one of the sectors that are benefiting from the great features introduced in the 5G networks that enhance digital communications and data protection; that notwithstanding, there are still security flaws in 5G technologies that can be transferred to the 6G networks if not properly addressed. This paper highlights the key areas of 6G networks that would provide grand support for the development of healthcare systems. It also identifies certain vulnerabilities in the previous cellular networks that are transferable to 6G networks, and suggests security strategies including zero trust initiatives that could be implemented to address the security concerns.
GPT-Lab: Next Generation Of Optimal Chemistry Discovery By GPT Driven Robotic Lab
Venue: arXiv cs.AI 2023
Year: 2023

The integration of robots in chemical experiments has enhanced experimental efficiency, but lacking the human intelligence to comprehend literature, they seldom provide assistance in experimental design. Therefore, achieving full-process autonomy from experiment design to validation in self-driven laboratories (SDL) remains a challenge. The introduction of Generative Pre-trained Transformers (GPT), particularly GPT-4, into robotic experimentation offers a solution. We introduce GPT-Lab, a paradigm that employs GPT models to give robots human-like intelligence. With our robotic experimentation platform, GPT-Lab mines literature for materials and methods and validates findings through high-throughput synthesis. As a demonstration, GPT-Lab analyzed 500 articles, identified 18 potential reagents, and successfully produced an accurate humidity colorimetric sensor with a root mean square error (RMSE) of 2.68%. This showcases the rapid materials discovery and validation potential of our system.
Cognizance of Post-COVID-19 Multi-Organ Dysfunction through Machine Learning Analysis
Venue: arXiv cs.LG 2023
Year: 2023

In the year 2022, a total of 466 patients from various cities across Iraq were included in this study. This research paper focuses on the application of machine learning techniques to analyse and predict multi-organ dysfunction in individuals experiencing Post-COVID-19 Syndrome, commonly known as Long COVID. Post-COVID-19 Syndrome presents a wide array of persistent symptoms affecting various organ systems, posing a significant challenge to healthcare. Leveraging the power of artificial intelligence, this study aims to enhance early detection and management of this complex condition. The paper outlines the importance of data collection and preprocessing, feature selection and engineering, model development and validation, and ethical considerations in conducting research in this field. By improving our understanding of Post-COVID-19 Syndrome through machine learning, healthcare providers can identify at-risk individuals and offer timely interventions, potentially improving patient outcomes and quality of life. Further research is essential to refine models, validate their clinical utility, and explore treatment options for Long COVID. Keywords: Post-COVID-19 Syndrome, Machine Learning, Multi-Organ Dysfunction, Healthcare, Artificial Intelligence.
Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
Venue: arXiv cs.CL 2023
Year: 2023

Popular prompt strategies like Chain-of-Thought Prompting can dramatically improve the reasoning abilities of Large Language Models (LLMs) in various domains. However, such hand-crafted prompt-strategies are often sub-optimal. In this paper, we present Promptbreeder, a general-purpose self-referential self-improvement mechanism that evolves and adapts prompts for a given domain. Driven by an LLM, Promptbreeder mutates a population of task-prompts, and subsequently evaluates them for fitness on a training set. Crucially, the mutation of these task-prompts is governed by mutation-prompts that the LLM generates and improves throughout evolution in a self-referential way. That is, Promptbreeder is not just improving task-prompts, but it is also improving the mutationprompts that improve these task-prompts. Promptbreeder outperforms state-of-the-art prompt strategies such as Chain-of-Thought and Plan-and-Solve Prompting on commonly used arithmetic and commonsense reasoning benchmarks. Furthermore, Promptbreeder is able to evolve intricate task-prompts for the challenging problem of hate speech classification.
Applications of Federated Learning in IoT for Hyper Personalisation
Venue: arXiv cs.LG 2023
Year: 2023

Billions of IoT devices are being deployed, taking advantage of faster internet, and the opportunity to access more endpoints. Vast quantities of data are being generated constantly by these devices but are not effectively being utilised. Using FL training machine learning models over these multiple clients without having to bring it to a central server. We explore how to use such a model to implement ultra levels of personalization unlike before
Stochastic Implicit Neural Signed Distance Functions for Safe Motion Planning under Sensing Uncertainty
Venue: arXiv cs.RO 2023
Year: 2023

Motion planning under sensing uncertainty is critical for robots in unstructured environments to guarantee safety for both the robot and any nearby humans. Most work on planning under uncertainty does not scale to high-dimensional robots such as manipulators, assumes simplified geometry of the robot or environment, or requires per-object knowledge of noise. Instead, we propose a method that directly models sensor-specific aleatoric uncertainty to find safe motions for high-dimensional systems in complex environments, without exact knowledge of environment geometry. We combine a novel implicit neural model of stochastic signed distance functions with a hierarchical optimization-based motion planner to plan low-risk motions without sacrificing path quality. Our method also explicitly bounds the risk of the path, offering trustworthiness. We empirically validate that our method produces safe motions and accurate risk bounds and is safer than baseline approaches.