OpenRaiser/Intern-Atlas huggingface dataset:
Smart Scheduling based on Deep Reinforcement Learning for Cellular Networks
Venue: arXiv cs.LG 2021
Year: 2021

To improve the system performance towards the Shannon limit, advanced radio resource management mechanisms play a fundamental role. In particular, scheduling should receive much attention, because it allocates radio resources among different users in terms of their channel conditions and QoS requirements. The difficulties of scheduling algorithms are the tradeoffs need to be made among multiple objectives, such as throughput, fairness and packet drop rate. We propose a smart scheduling scheme based on deep reinforcement learning (DRL). We not only verify the performance gain achieved, but also provide implementation-friend designs, i.e., a scalable neural network design for the agent and a virtual environment training framework. With the scalable neural network design, the DRL agent can easily handle the cases when the number of active users is time-varying without the need to redesign and retrain the DRL agent. Training the DRL agent in a virtual environment offline first and using it as the initial version in the practical usage helps to prevent the system from suffering from performance and robustness degradation due to the time-consuming training. Through both simulations and field tests, we show that the DRL-based smart scheduling outperforms the conventional scheduling method and can be adopted in practical systems.
QoS-Constrained Federated Learning Empowered by Intelligent Reflecting Surface
Venue: arXiv cs.IT 2021
Year: 2021

This paper investigates the model aggregation process in an over-the-air federated learning (AirFL) system, where an intelligent reflecting surface (IRS) is deployed to assist the transmission from users to the base station (BS). With the purpose of overcoming the absence of the security examination against malicious individuals, successive interference cancellation (SIC) is adopted as a basis to support analyzing statistic characteristics of model parameters from devices. The objective of this paper is to minimize the mean-square-error by jointly optimizing the receive beamforming vector at the BS, transmit power allocation at users, and phase shift matrix of the IRS, subject to the transmit power constraint for devices, unit-modulus constraint for reflecting elements, SIC decoding order constraint and quality-of-service constraint. To address this complicated problem, alternating optimization is employed to decompose it into three subproblems, where the optimal receive beamforming vector is obtained by solving the first subproblem with the Lagrange dual method. Then, the convex relaxation method is applied to the transmit power allocation subproblem to find a suboptimal solution. Eventually, the phase shift matrix subproblem is addressed by invoking the semidefinite relaxation. Simulation results validate the availability of IRS and the effectiveness of the proposed scheme in improving federated learning performance.
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel
Venue: arXiv cs.LG 2021
Year: 2021

The Neural Tangent Kernel (NTK) has recently attracted intense study, as it describes the evolution of an over-parameterized Neural Network (NN) trained by gradient descent. However, it is now well-known that gradient descent is not always a good optimizer for NNs, which can partially explain the unsatisfactory practical performance of the NTK regression estimator. In this paper, we introduce the Weighted Neural Tangent Kernel (WNTK), a generalized and improved tool, which can capture an over-parameterized NN's training dynamics under different optimizers. Theoretically, in the infinite-width limit, we prove: i) the stability of the WNTK at initialization and during training, and ii) the equivalence between the WNTK regression estimator and the corresponding NN estimator with different learning rates on different parameters. With the proposed weight update algorithm, both empirical and analytical WNTKs outperform the corresponding NTKs in numerical experiments.
Blockchain-based Digital Twins: Research Trends, Issues, and Future Challenges
Venue: arXiv cs.CR 2021
Year: 2021

Industrial processes rely on sensory data for decision-making processes, risk assessment, and performance evaluation. Extracting actionable insights from the collected data calls for an infrastructure that can ensure the dissemination of trustworthy data. For the physical data to be trustworthy, it needs to be cross-validated through multiple sensor sources with overlapping fields of view. Cross-validated data can then be stored on the blockchain, to maintain its integrity and trustworthiness. Once trustworthy data is recorded on the blockchain, product lifecycle events can be fed into data-driven systems for process monitoring, diagnostics, and optimized control. In this regard, Digital Twins (DTs) can be leveraged to draw intelligent conclusions from data by identifying the faults and recommending precautionary measures ahead of critical events. Empowering DTs with blockchain in industrial use-cases targets key challenges of disparate data repositories, untrustworthy data dissemination, and the need for predictive maintenance. In this survey, while highlighting the key benefits of using blockchain-based DTs, we present a comprehensive review of the state-of-the-art research results for blockchain-based DTs. Based on the current research trends, we discuss a trustworthy blockchain-based DTs framework. We highlight the role of Artificial Intelligence (AI) in blockchain-based DTs. Furthermore, we discuss current and future research and deployment challenges of blockchain-supported DTs that require further investigation.
Adversarially Optimized Mixup for Robust Classification
Venue: arXiv cs.LG 2021
Year: 2021

Mixup is a procedure for data augmentation that trains networks to make smoothly interpolated predictions between datapoints. Adversarial training is a strong form of data augmentation that optimizes for worst-case predictions in a compact space around each data-point, resulting in neural networks that make much more robust predictions. In this paper, we bring these ideas together by adversarially probing the space between datapoints, using projected gradient descent (PGD). The fundamental approach in this work is to leverage backpropagation through the mixup interpolation during training to optimize for places where the network makes unsmooth and incongruous predictions. Additionally, we also explore several modifications and nuances, like optimization of the mixup ratio and geometrical label assignment, and discuss their impact on enhancing network robustness. Through these ideas, we have been able to train networks that robustly generalize better; experiments on CIFAR-10 and CIFAR-100 demonstrate consistent improvements in accuracy against strong adversaries, including the recent strong ensemble attack AutoAttack. Our source code would be released for reproducibility.
Project-Level Encoding for Neural Source Code Summarization of Subroutines
Venue: arXiv cs.SE 2021
Year: 2021

Source code summarization of a subroutine is the task of writing a short, natural language description of that subroutine. The description usually serves in documentation aimed at programmers, where even brief phrase (e.g. "compresses data to a zip file") can help readers rapidly comprehend what a subroutine does without resorting to reading the code itself. Techniques based on neural networks (and encoder-decoder model designs in particular) have established themselves as the state-of-the-art. Yet a problem widely recognized with these models is that they assume the information needed to create a summary is present within the code being summarized itself - an assumption which is at odds with program comprehension literature. Thus a current research frontier lies in the question of encoding source code context into neural models of summarization. In this paper, we present a project-level encoder to improve models of code summarization. By project-level, we mean that we create a vectorized representation of selected code files in a software project, and use that representation to augment the encoder of state-of-the-art neural code summarization techniques. We demonstrate how our encoder improves several existing models, and provide guidelines for maximizing improvement while controlling time and resource costs in model size.
A Succinct Multivariate Lazy Multivariate Tower AD for Weil Algebra Computation
Venue: arXiv cs.SC 2021
Year: 2021

We propose a functional implementation of \emph{Multivariate Tower Automatic Differentiation}. Our implementation is intended to be used in implementing $C^\infty$-structure computation of an arbitrary Weil algebra, which we discussed in the previous work.
Formal Verification for Node-Based Visual Scripts Using Symbolic Model Checking
Venue: arXiv cs.SE 2021
Year: 2021

Visual script languages with a node-based interface have commonly been used in the video game industry. We examined the bug database obtained in the development of FINAL FANTASY XV (FFXV), and noticed that several types of bugs were caused by simple mis-descriptions of visual scripts and could therefore be mechanically detected. We propose a method for the automatic verification of visual scripts in order to improve productivity of video game development. Our method can automatically detect those bugs by using symbolic model checking. We show a translation algorithm which can automatically convert a visual script to an input model for NuSMV that is an implementation of symbolic model checking. For a preliminary evaluation, we applied our method to visual scripts used in the production for FFXV. The evaluation results demonstrate that our method can detect bugs of scripts and works well in a reasonable time.
D3p -- A Python Package for Differentially-Private Probabilistic Programming
Venue: arXiv cs.LG 2021
Year: 2021

We present d3p, a software package designed to help fielding runtime efficient widely-applicable Bayesian inference under differential privacy guarantees. d3p achieves general applicability to a wide range of probabilistic modelling problems by implementing the differentially private variational inference algorithm, allowing users to fit any parametric probabilistic model with a differentiable density function. d3p adopts the probabilistic programming paradigm as a powerful way for the user to flexibly define such models. We demonstrate the use of our software on a hierarchical logistic regression example, showing the expressiveness of the modelling approach as well as the ease of running the parameter inference. We also perform an empirical evaluation of the runtime of the private inference on a complex model and find a $\sim$10 fold speed-up compared to an implementation using TensorFlow Privacy.
Multi-Programming-Language Commits in OSS: An Empirical Study on Apache Projects
Venue: arXiv cs.SE 2021
Year: 2021

Modern software systems, such as Spark, are usually written in multiple programming languages (PLs). Besides benefiting from code reuse, such systems can also take advantages of specific PLs to implement certain features, to meet various quality needs, and to improve development efficiency. In this context, a change to such systems may need to modify source files written in different PLs. We define a multi-programming-language commit (MPLC) in a version control system (e.g., Git) as a commit that involves modified source files written in two or more PLs. To our knowledge, the phenomenon of MPLCs in software development has not been explored yet. In light of the potential impact of MPLCs on development difficulty and software quality, we performed an empirical study to understand the state of MPLCs, their change complexity, as well as their impact on open time of issues and bug proneness of source files in real-life software projects. By exploring the MPLCs in 20 non-trivial Apache projects with 205,994 commits, we obtained the following findings: (1) 9% of the commits from all the projects are MPLCs, and the proportion of MPLCs in 80% of the projects goes to a relatively stable level; (2) more than 90% of the MPLCs from all the projects involve source files written in two PLs; (3) the change complexity of MPLCs is significantly higher than that of non-MPLCs in all projects; (4) issues fixed in MPLCs take significantly longer to be resolved than issues fixed in non-MPLCs in 80% of the projects; and (5) source files that have been modified in MPLCs tend to be more bug-prone than source files that have never been modified in MPLCs. These findings provide practitioners with useful insights on the architecture design and quality management of software systems written in multiple PLs.
Recognizing LTLf/PLTLf Goals in Fully Observable Non-Deterministic Domain Models
Venue: arXiv cs.AI 2021
Year: 2021

Goal Recognition is the task of discerning the correct intended goal that an agent aims to achieve, given a set of possible goals, a domain model, and a sequence of observations as a sample of the plan being executed in the environment. Existing approaches assume that the possible goals are formalized as a conjunction in deterministic settings. In this paper, we develop a novel approach that is capable of recognizing temporally extended goals in Fully Observable Non-Deterministic (FOND) planning domain models, focusing on goals on finite traces expressed in Linear Temporal Logic (LTLf) and (Pure) Past Linear Temporal Logic (PLTLf). We empirically evaluate our goal recognition approach using different LTLf and PLTLf goals over six common FOND planning domain models, and show that our approach is accurate to recognize temporally extended goals at several levels of observability.
Autoregressive Belief Propagation for Decoding Block Codes
Venue: arXiv cs.IT 2021
Year: 2021

We revisit recent methods that employ graph neural networks for decoding error correcting codes and employ messages that are computed in an autoregressive manner. The outgoing messages of the variable nodes are conditioned not only on the incoming messages, but also on an estimation of the SNR and on the inferred codeword and on two downstream computations: (i) an extended vector of parity check outcomes, (ii) the mismatch between the inferred codeword and the re-encoding of the information bits of this codeword. Unlike most learned methods in the field, our method violates the symmetry conditions that enable the other methods to train exclusively with the zero-word. Despite not having the luxury of training on a single word, and the inability to train on more than a small fraction of the relevant sample space, we demonstrate effective training. The new method obtains a bit error rate that outperforms the latest methods by a sizable margin.
Detection of fake news on CoViD-19 on Web Search Engines
Venue: arXiv cs.LG 2021
Year: 2021

In early January 2020, after China reported the first cases of the new coronavirus (SARS-CoV-2) in the city of Wuhan, unreliable and not fully accurate information has started spreading faster than the virus itself. Alongside this pandemic, people have experienced a parallel infodemic, i.e., an overabundance of information, some of which misleading or even harmful, that has widely spread around the globe. Although Social Media are increasingly being used as information source, Web Search Engines, like Google or Yahoo!, still represent a powerful and trustworthy resource for finding information on the Web. This is due to their capability to capture the largest amount of information, helping users quickly identify the most relevant, useful, although not always the most reliable, results for their search queries. This study aims to detect potential misleading and fake contents by capturing and analysing textual information, which flow through Search Engines. By using a real-world dataset associated with recent CoViD-19 pandemic, we first apply re-sampling techniques for class imbalance, then we use existing Machine Learning algorithms for classification of not reliable news. By extracting lexical and host-based features of associated Uniform Resource Locators (URLs) for news articles, we show that the proposed methods, so common in phishing and malicious URLs detection, can improve the efficiency and performance of classifiers. Based on these findings, we suggest that the use of both textual and URLs features can improve the effectiveness of fake news detection methods.
Regenerativity of Viterbi process for pairwise Markov models
Venue: arXiv cs.IT 2021
Year: 2021

For hidden Markov models one of the most popular estimates of the hidden chain is the Viterbi path -- the path maximising the posterior probability. We consider a more general setting, called the pairwise Markov model (PMM), where the joint process consisting of finite-state hidden process and observation process is assumed to be a Markov chain. It has been recently proven that under some conditions the Viterbi path of the PMM can almost surely be extended to infinity, thereby defining the infinite Viterbi decoding of the observation sequence, called the Viterbi process. This was done by constructing a block of observations, called a barrier, which ensures that the Viterbi path goes trough a given state whenever this block occurs in the observation sequence. In this paper we prove that the joint process consisting of Viterbi process and PMM is regenerative. The proof involves a delicate construction of regeneration times which coincide with the occurrences of barriers. As one possible application of our theory, some results on the asymptotics of the Viterbi training algorithm are derived.
Almost (Weighted) Proportional Allocations for Indivisible Chores
Venue: arXiv cs.GT 2021
Year: 2021

In this paper, we study how to fairly allocate m indivisible chores to n (asymmetric) agents. We consider (weighted) proportionality up to any item (PROPX) and show that a (weighted) PROPX allocation always exists and can be computed efficiently. For chores, we argue that PROPX might be a more reliable relaxation for proportionality by the facts that any PROPX allocation ensures 2-approximation of maximin share (MMS) fairness [Budish, 2011] for symmetric agents and of anyprice share (APS) fairness [Babaioff et al, 2021] for asymmetric agents. APS allocations for chores have not been studied before the current work, and our result implies a 2-approximation algorithm. Another by-product result is that an EFX and a weighted EF1 allocation for indivisible chores exist if all agents have the same ordinal preference, which might be of independent interest. We then consider the partial information setting and design algorithms that only use agents' ordinal preferences to compute approximately PROPX allocations. Our algorithm achieves a 2-approximation for both symmetric and asymmetric agents, and the approximation ratio is optimal. Finally, we study the price of fairness (PoF), i.e., the loss in social welfare by enforcing allocations to be (weighted) PROPX. We prove that the tight ratio for PoF is Theta(n) for symmetric agents and unbounded for asymmetric agents.
Rate-Diverse Gaussian Multiple Access: Efficient Encoder and Decoder Designs
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we develop a pair of rate-diverse encoder and decoder for a two-user Gaussian multiple access channel (GMAC). The proposed scheme enables the users to transmit with the same codeword length but different coding rates under diverse user channel conditions. First, we propose the row-combining (RC) method and row-extending (RE) method to design practical low-density parity-check (LDPC) channel codes for rate-diverse GMAC. Second, we develop an iterative rate-diverse joint user messages decoding (RDJD) algorithm for GMAC, where all user messages are decoded with a single parity-check matrix. In contrast to the conventional network-coded multiple access (NCMA) and compute-forward multiple access (CFMA) schemes that first recover a linear combination of the transmitted codewords and then decode both user messages, this work can decode both the user messages simultaneously. Extrinsic information transfer (EXIT) chart analysis and simulation results indicate that RDJD can achieve gains up to 1.0 dB over NCMA and CFMA in the two-user GMAC. In particular, we show that there exists an optimal rate allocation for the two users to achieve the best decoding performance given the channel conditions and sum rate.
Mea culpa: How developers fix their own simple bugs differently from other developers
Venue: arXiv cs.SE 2021
Year: 2021

In this work, we study how the authorship of code affects bug-fixing commits using the SStuBs dataset, a collection of single-statement bug fix changes in popular Java Maven projects. More specifically, we study the differences in characteristics between simple bug fixes by the original author -- that is, the developer who submitted the bug-inducing commit -- and by different developers (i.e., non-authors). Our study shows that nearly half (i.e., 44.3%) of simple bugs are fixed by a different developer. We found that bug fixes by the original author and by different developers differed qualitatively and quantitatively. We observed that bug-fixing time by authors is much shorter than that of other developers. We also found that bug-fixing commits by authors tended to be larger in size and scope, and address multiple issues, whereas bug-fixing commits by other developers tended to be smaller and more focused on the bug itself. Future research can further study the different patterns in bug-fixing and create more tailored tools based on the developer's needs.
Evaluating Post-Training Compression in GANs using Locality-Sensitive Hashing
Venue: arXiv cs.LG 2021
Year: 2021

The analysis of the compression effects in generative adversarial networks (GANs) after training, i.e. without any fine-tuning, remains an unstudied, albeit important, topic with the increasing trend of their computation and memory requirements. While existing works discuss the difficulty of compressing GANs during training, requiring novel methods designed with the instability of GANs training in mind, we show that existing compression methods (namely clipping and quantization) may be directly applied to compress GANs post-training, without any additional changes. High compression levels may distort the generated set, likely leading to an increase of outliers that may negatively affect the overall assessment of existing k-nearest neighbor (KNN) based metrics. We propose two new precision and recall metrics based on locality-sensitive hashing (LSH), which, on top of increasing the outlier robustness, decrease the complexity of assessing an evaluation sample against $n$ reference samples from $O(n)$ to $O(\log(n))$, if using LSH and KNN, and to $O(1)$, if only applying LSH. We show that low-bit compression of several pre-trained GANs on multiple datasets induces a trade-off between precision and recall, retaining sample quality while sacrificing sample diversity.