OpenRaiser/Intern-Atlas huggingface dataset:
Formalization of the principles of brain Programming (Brain Principles Programming)
Venue: arXiv cs.AI 2022
Year: 2022

In the monograph "Strong artificial intelligence. On the Approaches to Superintelligence" contains an overview of general artificial intelligence (AGI). As an anthropomorphic research area, it includes Brain Principles Programming (BPP) -- the formalization of universal mechanisms (principles) of the brain work with information, which are implemented at all levels of the organization of nervous tissue. This monograph contains a formalization of these principles in terms of category theory. However, this formalization is not enough to develop algorithms for working with information. In this paper, for the description and modeling of BPP, it is proposed to apply mathematical models and algorithms developed earlier, which modeling cognitive functions and base on well-known physiological, psychological and other natural science theories. The paper uses mathematical models and algorithms of the following theories: P.K.Anokhin Theory of Functional Brain Systems, Eleanor Rosch prototypical categorization theory, Bob Rehder theory of causal models and "natural" classification. As a result, a formalization of BPP is obtained and computer experiments demonstrating the operation of algorithms are presented.
Perturbative methods for mostly monotonic probabilistic satisfiability problems
Venue: arXiv cs.DM 2022
Year: 2022

The probabilistic satisfiability of a logical expression is a fundamental concept known as the partition function in statistical physics and field theory, an evaluation of a related graph's Tutte polynomial in mathematics, and the Moore-Shannon network reliability of that graph in engineering. It is the crucial element for decision-making under uncertainty. Not surprisingly, it is provably hard to compute exactly or even to approximate. Many of these applications are concerned only with a subset of problems for which the solutions are monotonic functions. Here we extend the weak- and strong-coupling methods of statistical physics to heterogeneous satisfiability problems and introduce a novel approach to constructing lower and upper bounds on the approximation error for monotonic problems. These bounds combine information from both perturbative analyses to produce bounds that are tight in the sense that they are saturated by some problem instance that is compatible with all the information contained in either approximation.
Multi-Robot Synergistic Localization in Dynamic Environments
Venue: arXiv cs.RO 2022
Year: 2022

A mobile robot's precise location information is critical for navigation and task processing, especially for a multi-robot system (MRS) to collaborate and collect valuable data from the field. However, a robot in situations where it does not have access to GPS signals, such as in an environmentally controlled, indoor, or underground environment, finds it difficult to locate using its sensor alone. As a result, robots sharing their local information to improve their localization estimates benefit the entire MRS team. There have been several attempts to model-based multi-robot localization using Radio Signal Strength Indicator (RSSI) as a source to calculate bearing information. We also utilize the RSSI for wireless networks generated through the communication of multiple robots in a system and aim to localize agents with high accuracy and efficiency in a dynamic environment for shared information fusion to refine the localization estimation. This estimator structure reduces one source of measurement correlation while appropriately incorporating others. This paper proposes a decentralized Multi-robot Synergistic Localization System (MRSL) for a dense and dynamic environment. Robots update their position estimation whenever new information receives from their neighbors. When the system senses the presence of other robots in the region, it exchanges position estimates and merges the received data to improve its localization accuracy. Our approach uses Bayesian rule-based integration, which has shown to be computationally efficient and applicable to asynchronous robotics communication. We have performed extensive simulation experiments with a varying number of robots to analyze the algorithm. MRSL's localization accuracy with RSSI outperformed other algorithms from the literature, showing a significant promise for future development.
Compromised account detection using authorship verification: a novel approach
Venue: arXiv cs.CR 2022
Year: 2022

Compromising legitimate accounts is a way of disseminating malicious content to a large user base in Online Social Networks (OSNs). Since the accounts cause lots of damages to the user and consequently to other users on OSNs, early detection is very important. This paper proposes a novel approach based on authorship verification to identify compromised twitter accounts. As the approach only uses the features extracted from the last user's post, it helps to early detection to control the damage. As a result, the malicious message without a user profile can be detected with satisfying accuracy. Experiments were constructed using a real-world dataset of compromised accounts on Twitter. The result showed that the model is suitable for detection due to achieving an accuracy of 89%.
Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping
Venue: arXiv cs.LG 2022
Year: 2022

In machine learning, an agent needs to estimate uncertainty to efficiently explore and adapt and to make effective decisions. A common approach to uncertainty estimation maintains an ensemble of models. In recent years, several approaches have been proposed for training ensembles, and conflicting views prevail with regards to the importance of various ingredients of these approaches. In this paper, we aim to address the benefits of two ingredients -- prior functions and bootstrapping -- which have come into question. We show that prior functions can significantly improve an ensemble agent's joint predictions across inputs and that bootstrapping affords additional benefits if the signal-to-noise ratio varies across inputs. Our claims are justified by both theoretical and experimental results.
Scalable Online Disease Diagnosis via Multi-Model-Fused Actor-Critic Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

For those seeking healthcare advice online, AI based dialogue agents capable of interacting with patients to perform automatic disease diagnosis are a viable option. This application necessitates efficient inquiry of relevant disease symptoms in order to make accurate diagnosis recommendations. This can be formulated as a problem of sequential feature (symptom) selection and classification for which reinforcement learning (RL) approaches have been proposed as a natural solution. They perform well when the feature space is small, that is, the number of symptoms and diagnosable disease categories is limited, but they frequently fail in assignments with a large number of features. To address this challenge, we propose a Multi-Model-Fused Actor-Critic (MMF-AC) RL framework that consists of a generative actor network and a diagnostic critic network. The actor incorporates a Variational AutoEncoder (VAE) to model the uncertainty induced by partial observations of features, thereby facilitating in making appropriate inquiries. In the critic network, a supervised diagnosis model for disease predictions is involved to precisely estimate the state-value function. Furthermore, inspired by the medical concept of differential diagnosis, we combine the generative and diagnosis models to create a novel reward shaping mechanism to address the sparse reward problem in large search spaces. We conduct extensive experiments on both synthetic and real-world datasets for empirical evaluations. The results demonstrate that our approach outperforms state-of-the-art methods in terms of diagnostic accuracy and interaction efficiency while also being more effectively scalable to large search spaces. Besides, our method is adaptable to both categorical and continuous features, making it ideal for online applications.
Progress Report: A Deep Learning Guided Exploration of Affine Unimodular Loop Transformations
Venue: arXiv cs.PL 2022
Year: 2022

In this paper, we present a work in progress about a deep learning based approach for automatic code optimization in polyhedral compilers. The proposed technique explores combinations of affine and non-affine loop transformations to find the sequence of transformations that minimizes the execution time of a given program. This exploration is guided by a deep learning based cost model that evaluates the speedup that each sequence of transformations would yield. Preliminary results show that the proposed techniques achieve a 2.35x geometric mean speedup over state of the art polyhedral compilers (Pluto).
Wavelet Regularization Benefits Adversarial Training
Venue: arXiv cs.CV 2022
Year: 2022

Adversarial training methods are state-of-the-art (SOTA) empirical defense methods against adversarial examples. Many regularization methods have been proven to be effective with the combination of adversarial training. Nevertheless, such regularization methods are implemented in the time domain. Since adversarial vulnerability can be regarded as a high-frequency phenomenon, it is essential to regulate the adversarially-trained neural network models in the frequency domain. Faced with these challenges, we make a theoretical analysis on the regularization property of wavelets which can enhance adversarial training. We propose a wavelet regularization method based on the Haar wavelet decomposition which is named Wavelet Average Pooling. This wavelet regularization module is integrated into the wide residual neural network so that a new WideWaveletResNet model is formed. On the datasets of CIFAR-10 and CIFAR-100, our proposed Adversarial Wavelet Training method realizes considerable robustness under different types of attacks. It verifies the assumption that our wavelet regularization method can enhance adversarial robustness especially in the deep wide neural networks. The visualization experiments of the Frequency Principle (F-Principle) and interpretability are implemented to show the effectiveness of our method. A detailed comparison based on different wavelet base functions is presented. The code is available at the repository: \url{https://github.com/momo1986/AdversarialWaveletTraining}.
Attestation Mechanisms for Trusted Execution Environments Demystified
Venue: arXiv cs.CR 2022
Year: 2022

Attestation is a fundamental building block to establish trust over software systems. When used in conjunction with trusted execution environments, it guarantees the genuineness of the code executed against powerful attackers and threats, paving the way for adoption in several sensitive application domains. This paper reviews remote attestation principles and explains how the modern and industrially well-established trusted execution environments Intel SGX, Arm TrustZone and AMD SEV, as well as emerging RISC-V solutions, leverage these mechanisms.
Scalable Joint Learning of Wireless Multiple-Access Policies and their Signaling
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we apply an multi-agent reinforcement learning (MARL) framework allowing the base station (BS) and the user equipments (UEs) to jointly learn a channel access policy and its signaling in a wireless multiple access scenario. In this framework, the BS and UEs are reinforcement learning (RL) agents that need to cooperate in order to deliver data. The comparison with a contention-free and a contention-based baselines shows that our framework achieves a superior performance in terms of goodput even in high traffic situations while maintaining a low collision rate. The scalability of the proposed method is studied, since it is a major problem in MARL and this paper provides the first results in order to address it.
The Open corpus of the Veps and Karelian languages: overview and applications
Venue: arXiv cs.CL 2022
Year: 2022

A growing priority in the study of Baltic-Finnic languages of the Republic of Karelia has been the methods and tools of corpus linguistics. Since 2016, linguists, mathematicians, and programmers at the Karelian Research Centre have been working with the Open Corpus of the Veps and Karelian Languages (VepKar), which is an extension of the Veps Corpus created in 2009. The VepKar corpus comprises texts in Karelian and Veps, multifunctional dictionaries linked to them, and software with an advanced system of search using various criteria of the texts (language, genre, etc.) and numerous linguistic categories (lexical and grammatical search in texts was implemented thanks to the generator of word forms that we created earlier). A corpus of 3000 texts was compiled, texts were uploaded and marked up, the system for classifying texts into languages, dialects, types and genres was introduced, and the word-form generator was created. Future plans include developing a speech module for working with audio recordings and a syntactic tagging module using morphological analysis outputs. Owing to continuous functional advancements in the corpus manager and ongoing VepKar enrichment with new material and text markup, users can handle a wide range of scientific and applied tasks. In creating the universal national VepKar corpus, its developers and managers strive to preserve and exhibit as fully as possible the state of the Veps and Karelian languages in the 19th-21st centuries.
Joint Modelling of Quantum and Classical Noise over Unity Quantum Channel
Venue: arXiv cs.IT 2022
Year: 2022

For a continuous-input-continuous-output arbitrarily distributed quantum channel carrying classical information, the channel capacity can be computed in terms of the distribution of the channel envelope, received signal strength over a quantum propagation field and the noise spectral density. If the channel envelope is considered to be unity with unit received signal strength, the factor controlling the capacity is the noise. Quantum channel carrying classical information will suffer from the combination of classical and quantum noise. Assuming additive Gaussian-distributed classical noise and Poisson-distributed quantum noise, we formulate a hybrid noise model by deriving a joint Gaussian- Poisson distribution in this letter. For the transmitted signal, we consider the mean of signal sample space instead of considering a particular distribution and study how the maximum mutual information varies over such mean value. Capacity is estimated by maximizing the mutual information over unity channel envelope.
Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning
Venue: arXiv cs.CL 2022
Year: 2022

Much literature has shown that prompt-based learning is an efficient method to make use of the large pre-trained language model. Recent works also exhibit the possibility of steering a chatbot's output by plugging in an appropriate prompt. Gradient-based methods are often used to perturb the prompts. However, some language models are not even available to the public. In this work, we first explored the combination of prompting and reinforcement learning (RL) to steer models' generation without accessing any of the models' parameters. Second, to reduce the training effort and enhance the generalizability to the unseen task, we apply multi-task learning to make the model learn to generalize to new tasks better. The experiment results show that our proposed method can successfully control several state-of-the-art (SOTA) dialogue models without accessing their parameters. Furthermore, the model demonstrates the strong ability to quickly adapt to an unseen task in fewer steps than the baseline model.
Accelerating Score-based Generative Models for High-Resolution Image Synthesis
Venue: arXiv cs.CV 2022
Year: 2022

Score-based generative models (SGMs) have recently emerged as a promising class of generative models. The key idea is to produce high-quality images by recurrently adding Gaussian noises and gradients to a Gaussian sample until converging to the target distribution, a.k.a. the diffusion sampling. To ensure stability of convergence in sampling and generation quality, however, this sequential sampling process has to take a small step size and many sampling iterations (e.g., 2000). Several acceleration methods have been proposed with focus on low-resolution generation. In this work, we consider the acceleration of high-resolution generation with SGMs, a more challenging yet more important problem. We prove theoretically that this slow convergence drawback is primarily due to the ignorance of the target distribution. Further, we introduce a novel Target Distribution Aware Sampling (TDAS) method by leveraging the structural priors in space and frequency domains. Extensive experiments on CIFAR-10, CelebA, LSUN, and FFHQ datasets validate that TDAS can consistently accelerate state-of-the-art SGMs, particularly on more challenging high resolution (1024x1024) image generation tasks by up to 18.4x, whilst largely maintaining the synthesis quality. With fewer sampling iterations, TDAS can still generate good quality images. In contrast, the existing methods degrade drastically or even fails completely
Towards Continuous Systematic Literature Review in Software Engineering
Venue: arXiv cs.SE 2022
Year: 2022

Context: New scientific evidence continuously arises with advances in Software Engineering (SE) research. Conventionally, Systematic Literature Reviews (SLRs) are not updated or updated intermittently, leaving gaps between updates, during which time the SLR may be missing crucial new evidence. Goal: We propose and evaluate a concept and process called Continuous Systematic Literature Review (CSLR) in SE. Method: To elaborate on the CSLR concept and process, we performed a synthesis of evidence by conducting a meta-ethnography, addressing knowledge from varied research areas. Furthermore, we conducted a case study to evaluate the CSLR process. Results: We describe the resulting CSLR process in BPMN format. The case study results provide indications on the importance and feasibility of applying CSLR in practice to continuously update SLR evidence in SE. Conclusion: The CSLR concept and process provide a feasible and systematic way to continuously incorporate new evidence into SLRs, supporting trustworthy and up-to-date evidence for SLRs in SE.
Automating Ambiguity: Challenges and Pitfalls of Artificial Intelligence
Venue: arXiv cs.CY 2022
Year: 2022

Machine learning (ML) and artificial intelligence (AI) tools increasingly permeate every possible social, political, and economic sphere; sorting, taxonomizing and predicting complex human behaviour and social phenomena. However, from fallacious and naive groundings regarding complex adaptive systems to datasets underlying models, these systems are beset by problems, challenges, and limitations. They remain opaque and unreliable, and fail to consider societal and structural oppressive systems, disproportionately negatively impacting those at the margins of society while benefiting the most powerful. The various challenges, problems and pitfalls of these systems are a hot topic of research in various areas, such as critical data/algorithm studies, science and technology studies (STS), embodied and enactive cognitive science, complexity science, Afro-feminism, and the broadly construed emerging field of Fairness, Accountability, and Transparency (FAccT). Yet, these fields of enquiry often proceed in silos. This thesis weaves together seemingly disparate fields of enquiry to examine core scientific and ethical challenges, pitfalls, and problems of AI. In this thesis I, a) review the historical and cultural ecology from which AI research emerges, b) examine the shaky scientific grounds of machine prediction of complex behaviour illustrating how predicting complex behaviour with precision is impossible in principle, c) audit large scale datasets behind current AI demonstrating how they embed societal historical and structural injustices, d) study the seemingly neutral values of ML research and put forward 67 prominent values underlying ML research, e) examine some of the insidious and worrying applications of computer vision research, and f) put forward a framework for approaching challenges, failures and problems surrounding ML systems as well as alternative ways forward.
Neo-GNNs: Neighborhood Overlap-aware Graph Neural Networks for Link Prediction
Venue: arXiv cs.LG 2022
Year: 2022

Graph Neural Networks (GNNs) have been widely applied to various fields for learning over graph-structured data. They have shown significant improvements over traditional heuristic methods in various tasks such as node classification and graph classification. However, since GNNs heavily rely on smoothed node features rather than graph structure, they often show poor performance than simple heuristic methods in link prediction where the structural information, e.g., overlapped neighborhoods, degrees, and shortest paths, is crucial. To address this limitation, we propose Neighborhood Overlap-aware Graph Neural Networks (Neo-GNNs) that learn useful structural features from an adjacency matrix and estimate overlapped neighborhoods for link prediction. Our Neo-GNNs generalize neighborhood overlap-based heuristic methods and handle overlapped multi-hop neighborhoods. Our extensive experiments on Open Graph Benchmark datasets (OGB) demonstrate that Neo-GNNs consistently achieve state-of-the-art performance in link prediction. Our code is publicly available at https://github.com/seongjunyun/Neo_GNNs.
Exploiting In-Slot Micro-Synchronism for S-ALOHA
Venue: arXiv cs.IT 2022
Year: 2022

Proliferation of the urban Internet-of-Things (IoTs) for smart cities has fuelled massive amounts of data over wireless cellular networks. Random access (RA) system of wireless cellular networks, e.g., 5G New Radio (NR), based on S-ALOHA system should cope with ever-growing IoT traffic. This work proposes S-ALOHA system with time offsets (TOs), where one slot consists of K TOs and one packet transmission time. The length of the overall TOs is a fraction of a packet transmission time. In the system users (re)transmit to the boundary of a TO randomly selected. This enables the base station (BS) to inform the users of who transmits the first and the last packets in the slot with collision so that the two users can retransmit successfully in the following two slots respectively. Our throughput analysis compared to simulations shows that adopting even with three and four TOs surpasses the throughput limit of S-ALOHA system without TOs. Additionally, we propose two Bayesian-optimized backoff algorithms for S-ALOHA system with TOs, with which users can apply throughput-optimal (re)transmission probability or uniform backoff window even in unsaturated traffic scenarios. Numerical results demonstrate that the proposed backoff algorithms can achieve the throughput close to an ideal system and drastically reduce the access delay compared to S-ALOHA system.