OpenRaiser/Intern-Atlas huggingface dataset:
Mediumband Wireless Communication
Venue: arXiv cs.IT 2022
Year: 2022

The fundamental phenomenon widely known as multipath is unavoidable in wireless communication, and affects almost every element of modern wireless communication systems. The impact of multipath on the received signal depends on whether the delay spread (i.e., spread of time delays associated with different multipath components) is large or small relative to the signalling period of the wireless communication system. In narrowband systems, the delay spread is about one tenth (or less) of the signalling period. The delay spread and the signalling period of broadband systems are in the same order of magnitude. In between these two extremes, there appears to exist an important, yet overlooked, class of systems whose delay spread is neither small nor large enough for them to fall into these two basic classes. In this paper, the effect of multipath on this class of systems denoted henceforth as mediumband is studied, and its channel is characterized in compact form in order to enable future research into this class of wireless communication systems.
Generalization Through the Lens of Learning Dynamics
Venue: arXiv cs.LG 2022
Year: 2022

A machine learning (ML) system must learn not only to match the output of a target function on a training set, but also to generalize to novel situations in order to yield accurate predictions at deployment. In most practical applications, the user cannot exhaustively enumerate every possible input to the model; strong generalization performance is therefore crucial to the development of ML systems which are performant and reliable enough to be deployed in the real world. While generalization is well-understood theoretically in a number of hypothesis classes, the impressive generalization performance of deep neural networks has stymied theoreticians. In deep reinforcement learning (RL), our understanding of generalization is further complicated by the conflict between generalization and stability in widely-used RL algorithms. This thesis will provide insight into generalization by studying the learning dynamics of deep neural networks in both supervised and reinforcement learning tasks.
Untargeted Attack against Federated Recommendation Systems via Poisonous Item Embeddings and the Defense
Venue: arXiv cs.IR 2022
Year: 2022

Federated recommendation (FedRec) can train personalized recommenders without collecting user data, but the decentralized nature makes it susceptible to poisoning attacks. Most previous studies focus on the targeted attack to promote certain items, while the untargeted attack that aims to degrade the overall performance of the FedRec system remains less explored. In fact, untargeted attacks can disrupt the user experience and bring severe financial loss to the service provider. However, existing untargeted attack methods are either inapplicable or ineffective against FedRec systems. In this paper, we delve into the untargeted attack and its defense for FedRec systems. (i) We propose ClusterAttack, a novel untargeted attack method. It uploads poisonous gradients that converge the item embeddings into several dense clusters, which make the recommender generate similar scores for these items in the same cluster and perturb the ranking order. (ii) We propose a uniformity-based defense mechanism (UNION) to protect FedRec systems from such attacks. We design a contrastive learning task that regularizes the item embeddings toward a uniform distribution. Then the server filters out these malicious gradients by estimating the uniformity of updated item embeddings. Experiments on two public datasets show that ClusterAttack can effectively degrade the performance of FedRec systems while circumventing many defense methods, and UNION can improve the resistance of the system against various untargeted attacks, including our ClusterAttack.
Is Research Funding Always Beneficial? A Cross-Disciplinary Analysis of UK Research 2014-20
Venue: arXiv cs.DL 2022
Year: 2022

The search for and management of external funding now occupies much valuable researcher time. Whilst funding is essential for some types of research and beneficial for others, it may also constrain academic choice and creativity. Thus, it is important to assess whether it is ever detrimental or unnecessary. Here we investigate whether funded research tends to be higher quality in all fields and for all major research funders. Based on peer review quality scores for 113,877 articles from all fields in the UK's Research Excellence Framework (REF) 2021, we estimate that there are substantial disciplinary differences in the proportion of funded journal articles, from Theology and Religious Studies (16%+) to Biological Sciences (91%+). The results suggest that funded research is likely to be higher quality overall, for all the largest research funders, and for all fields, even after factoring out research team size. There are differences between funders in the average quality of the research they support, however. Funding seems particularly beneficial in health-related fields. The results do not show cause and effect and do not take into account the amount of funding received but are consistent with funding either improving research quality or being won by high quality researchers or projects. In summary, there are no broad fields of research in which funding is irrelevant, so no fields can afford to ignore it. The results also show that citations are not effective proxies for research quality in the arts and humanities and most social sciences for evaluating research funding.
Beyond circular-arc graphs -- recognizing lollipop graphs and medusa graphs
Venue: arXiv cs.DS 2022
Year: 2022

In 1992 Biró, Hujter and Tuza introduced, for every fixed connected graph $H$, the class of $H$-graphs, defined as the intersection graphs of connected subgraphs of some subdivision of $H$. Recently, quite a lot of research has been devoted to understanding the tractability border for various computational problems, such as recognition or isomorphism testing, in classes of $H$-graphs for different graphs $H$. In this work we undertake this research topic, focusing on the recognition problem. Chaplick, Töpfer, Voborn\'ık, and Zeman showed, for every fixed tree $T$, a polynomial-time algorithm recognizing $T$-graphs. Tucker showed a polynomial time algorithm recognizing $K_3$-graphs (circular-arc graphs). On the other hand, Chaplick at al. showed that recognition of $H$-graphs is $NP$-hard if $H$ contains two different cycles sharing an edge. The main two results of this work narrow the gap between the $NP$-hard and $P$ cases of $H$-graphs recognition. First, we show that recognition of $H$-graphs is $NP$-hard when $H$ contains two different cycles. On the other hand, we show a polynomial-time algorithm recognizing $L$-graphs, where $L$ is a graph containing a cycle and an edge attached to it ($L$-graphs are called lollipop graphs). Our work leaves open the recognition problems of $M$-graphs for every unicyclic graph $M$ different from a cycle and a lollipop. Other results of this work, which shed some light on the cases that remain open, are as follows. Firstly, the recognition of $M$-graphs, where $M$ is a fixed unicyclic graph, admits a polynomial time algorithm if we restrict the input to graphs containing particular holes (hence recognition of $M$-graphs is probably most difficult for chordal graphs). Secondly, the recognition of medusa graphs, which are defined as the union of $M$-graphs, where $M$ runs over all unicyclic graphs, is $NP$-complete.
End-to-End Speech Translation of Arabic to English Broadcast News
Venue: arXiv cs.CL 2022
Year: 2022

Speech translation (ST) is the task of directly translating acoustic speech signals in a source language into text in a foreign language. ST task has been addressed, for a long time, using a pipeline approach with two modules : first an Automatic Speech Recognition (ASR) in the source language followed by a text-to-text Machine translation (MT). In the past few years, we have seen a paradigm shift towards the end-to-end approaches using sequence-to-sequence deep neural network models. This paper presents our efforts towards the development of the first Broadcast News end-to-end Arabic to English speech translation system. Starting from independent ASR and MT LDC releases, we were able to identify about 92 hours of Arabic audio recordings for which the manual transcription was also translated into English at the segment level. These data was used to train and compare pipeline and end-to-end speech translation systems under multiple scenarios including transfer learning and data augmentation techniques.
Low-rank Tensor Assisted K-space Generative Model for Parallel Imaging Reconstruction
Venue: arXiv cs.CV 2022
Year: 2022

Although recent deep learning methods, especially generative models, have shown good performance in fast magnetic resonance imaging, there is still much room for improvement in high-dimensional generation. Considering that internal dimensions in score-based generative models have a critical impact on estimating the gradient of the data distribution, we present a new idea, low-rank tensor assisted k-space generative model (LR-KGM), for parallel imaging reconstruction. This means that we transform original prior information into high-dimensional prior information for learning. More specifically, the multi-channel data is constructed into a large Hankel matrix and the matrix is subsequently folded into tensor for prior learning. In the testing phase, the low-rank rotation strategy is utilized to impose low-rank constraints on tensor output of the generative network. Furthermore, we alternately use traditional generative iterations and low-rank high-dimensional tensor iterations for reconstruction. Experimental comparisons with the state-of-the-arts demonstrated that the proposed LR-KGM method achieved better performance.
Technical Debt Management in OSS Projects: An Empirical Study on GitHub
Venue: arXiv cs.SE 2022
Year: 2022

Technical debt (TD) refers to delayed tasks and immature artifacts that may bring short-term benefits but incur extra costs of change during maintenance and evolution in the long term. TD has been extensively studied in the past decade, and numerous open source software (OSS) projects were used to explore specific aspects of TD and validate various approaches for TD management (TDM). However, there still lacks a comprehensive understanding on the practice of TDM in OSS development, which penetrates the OSS community's perception of the TD concept and how TD is managed in OSS development. To this end, we conducted an empirical study on the whole GitHub to explore the adoption and execution of TDM based on issues in OSS projects. We collected 35,278 issues labeled as TD (TD issues) distributed over 3,598 repositories in total from the issue tracking system of GitHub between 2009 and 2020. The findings are that: (1) the OSS community is embracing the TD concept; (2) the analysis of TD instances shows that TD may affect both internal and external quality of software systems; (3) only one TD issue was identified in 31.1% of the repositories and all TD issues were identified by only one developer in 69.0% of the repositories; (4) TDM was ignored in 27.3% of the repositories after TD issues were identified; and (5) among the repositories with TD labels, 32.9% have abandoned TDM while only 8.2% adopt TDM as a consistent practice. These findings provide valuable insights for practitioners in TDM and promising research directions for further investigation.
Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks
Venue: arXiv cs.RO 2022
Year: 2022

In order to avoid conventional controlling methods which created obstacles due to the complexity of systems and intense demand on data density, developing modern and more efficient control methods are required. In this way, reinforcement learning off-policy and model-free algorithms help to avoid working with complex models. In terms of speed and accuracy, they become prominent methods because the algorithms use their past experience to learn the optimal policies. In this study, three reinforcement learning algorithms; DDPG, TD3 and SAC have been used to train Fetch robotic manipulator for four different tasks in MuJoCo simulation environment. All of these algorithms are off-policy and able to achieve their desired target by optimizing both policy and value functions. In the current study, the efficiency and the speed of these three algorithms are analyzed in a controlled environment.
Towards a Learner-Centered Explainable AI: Lessons from the learning sciences
Venue: arXiv cs.HC 2022
Year: 2022

In this short paper, we argue for a refocusing of XAI around human learning goals. Drawing upon approaches and theories from the learning sciences, we propose a framework for the learner-centered design and evaluation of XAI systems. We illustrate our framework through an ongoing case study in the context of AI-augmented social work.
Random Feature Models for Learning Interacting Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

Particle dynamics and multi-agent systems provide accurate dynamical models for studying and forecasting the behavior of complex interacting systems. They often take the form of a high-dimensional system of differential equations parameterized by an interaction kernel that models the underlying attractive or repulsive forces between agents. We consider the problem of constructing a data-based approximation of the interacting forces directly from noisy observations of the paths of the agents in time. The learned interaction kernels are then used to predict the agents behavior over a longer time interval. The approximation developed in this work uses a randomized feature algorithm and a sparse randomized feature approach. Sparsity-promoting regression provides a mechanism for pruning the randomly generated features which was observed to be beneficial when one has limited data, in particular, leading to less overfitting than other approaches. In addition, imposing sparsity reduces the kernel evaluation cost which significantly lowers the simulation cost for forecasting the multi-agent systems. Our method is applied to various examples, including first-order systems with homogeneous and heterogeneous interactions, second order homogeneous systems, and a new sheep swarming system.
ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning enables cooperative training among massively distributed clients by sharing their learned local model parameters. However, with increasing model size, deploying federated learning requires a large communication bandwidth, which limits its deployment in wireless networks. To address this bottleneck, we introduce a residual-based federated learning framework (ResFed), where residuals rather than model parameters are transmitted in communication networks for training. In particular, we integrate two pairs of shared predictors for the model prediction in both server-to-client and client-to-server communication. By employing a common prediction rule, both locally and globally updated models are always fully recoverable in clients and the server. We highlight that the residuals only indicate the quasi-update of a model in a single inter-round, and hence contain more dense information and have a lower entropy than the model, comparing to model weights and gradients. Based on this property, we further conduct lossy compression of the residuals by sparsification and quantization and encode them for efficient communication. The experimental evaluation shows that our ResFed needs remarkably less communication costs and achieves better accuracy by leveraging less sensitive residuals, compared to standard federated learning. For instance, to train a 4.08 MB CNN model on CIFAR-10 with 10 clients under non-independent and identically distributed (Non-IID) setting, our approach achieves a compression ratio over 700X in each communication round with minimum impact on the accuracy. To reach an accuracy of 70%, it saves around 99% of the total communication volume from 587.61 Mb to 6.79 Mb in up-streaming and to 4.61 Mb in down-streaming on average for all clients.
Error-aware Quantization through Noise Tempering
Venue: arXiv cs.LG 2022
Year: 2022

Quantization has become a predominant approach for model compression, enabling deployment of large models trained on GPUs onto smaller form-factor devices for inference. Quantization-aware training (QAT) optimizes model parameters with respect to the end task while simulating quantization error, leading to better performance than post-training quantization. Approximation of gradients through the non-differentiable quantization operator is typically achieved using the straight-through estimator (STE) or additive noise. However, STE-based methods suffer from instability due to biased gradients, whereas existing noise-based methods cannot reduce the resulting variance. In this work, we incorporate exponentially decaying quantization-error-aware noise together with a learnable scale of task loss gradient to approximate the effect of a quantization operator. We show this method combines gradient scale and quantization noise in a better optimized way, providing finer-grained estimation of gradients at each weight and activation layer's quantizer bin size. Our controlled noise also contains an implicit curvature term that could encourage flatter minima, which we show is indeed the case in our experiments. Experiments training ResNet architectures on the CIFAR-10, CIFAR-100 and ImageNet benchmarks show that our method obtains state-of-the-art top-1 classification accuracy for uniform (non mixed-precision) quantization, out-performing previous methods by 0.5-1.2% absolute.
Wireless Channel Delay Spread Performance Evaluation of a Building Layout
Venue: arXiv cs.NI 2022
Year: 2022

Evaluating a building's wireless performance during the building design process is a new paradigm for wireless communications and building design. This paper proposes the earliest building wireless performance (BWP) evaluation theory focusing on the channel delay spread (DS). The novel contributions of this paper lie in the following aspects. 1) We define a new metric called DS gain, which is the first metric for evaluating the channel DS performance of a building under design. 2) We propose an analytical model to compute the metric quickly and accurately. 3) The proposed scheme is validated via Monte-Carlo simulations under typical indoor scenarios. Numerical results reveal that building design has a clear impact on the root mean square (RMS) DS. In the future, architects need to design a building taking its DS gain into account carefully. Otherwise, indoor networks in it will suffer from severe signal inter-symbol interference (ISI) due to an over large DS gain.
On an Interpretation of ResNets via Solution Constructions
Venue: arXiv cs.LG 2022
Year: 2022

This paper first constructs a typical solution of ResNets for multi-category classifications by the principle of gate-network controls and deep-layer classifications, from which a general interpretation of the ResNet architecture is given and the performance mechanism is explained. We then use more solutions to further demonstrate the generality of that interpretation. The universal-approximation capability of ResNets is proved.
Privacy-Preserving Record Linkage
Venue: arXiv cs.DB 2022
Year: 2022

Given several databases containing person-specific data held by different organizations, Privacy-Preserving Record Linkage (PPRL) aims to identify and link records that correspond to the same entity/individual across different databases based on the matching of personal identifying attributes, such as name and address, without revealing the actual values in these attributes due to privacy concerns. This reference work entry defines the PPRL problem, reviews the literature and key findings, and discusses applications and research challenges.
MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations
Venue: arXiv cs.LG 2022
Year: 2022

Poor sample efficiency continues to be the primary challenge for deployment of deep Reinforcement Learning (RL) algorithms for real-world applications, and in particular for visuo-motor control. Model-based RL has the potential to be highly sample efficient by concurrently learning a world model and using synthetic rollouts for planning and policy improvement. However, in practice, sample-efficient learning with model-based RL is bottlenecked by the exploration challenge. In this work, we find that leveraging just a handful of demonstrations can dramatically improve the sample-efficiency of model-based RL. Simply appending demonstrations to the interaction dataset, however, does not suffice. We identify key ingredients for leveraging demonstrations in model learning -- policy pretraining, targeted exploration, and oversampling of demonstration data -- which forms the three phases of our model-based RL framework. We empirically study three complex visuo-motor control domains and find that our method is 150%-250% more successful in completing sparse reward tasks compared to prior approaches in the low data regime (100K interaction steps, 5 demonstrations). Code and videos are available at: https://nicklashansen.github.io/modemrl
Instrumental Variables in Causal Inference and Machine Learning: A Survey
Venue: arXiv cs.LG 2022
Year: 2022

Causal inference is the process of using assumptions, study designs, and estimation strategies to draw conclusions about the causal relationships between variables based on data. This allows researchers to better understand the underlying mechanisms at work in complex systems and make more informed decisions. In many settings, we may not fully observe all the confounders that affect both the treatment and outcome variables, complicating the estimation of causal effects. To address this problem, a growing literature in both causal inference and machine learning proposes to use Instrumental Variables (IV). This paper serves as the first effort to systematically and comprehensively introduce and discuss the IV methods and their applications in both causal inference and machine learning. First, we provide the formal definition of IVs and discuss the identification problem of IV regression methods under different assumptions. Second, we categorize the existing work on IV methods into three streams according to the focus on the proposed methods, including two-stage least squares with IVs, control function with IVs, and evaluation of IVs. For each stream, we present both the classical causal inference methods, and recent developments in the machine learning literature. Then, we introduce a variety of applications of IV methods in real-world scenarios and provide a summary of the available datasets and algorithms. Finally, we summarize the literature, discuss the open problems and suggest promising future research directions for IV methods and their applications. We also develop a toolkit of IVs methods reviewed in this survey at https://github.com/causal-machine-learning-lab/mliv.