OpenRaiser/Intern-Atlas huggingface dataset:
Hybrid Beamforming and Combining for Millimeter Wave Full Duplex Massive MIMO Interference Channel
Venue: arXiv cs.IT 2022
Year: 2022

Full Duplex (FD) communication can revolutionize wireless communications as it avoids using independent channels for bi-directional communications. This work generalizes the point-to-point FD communication in millimeter wave (mmWave) band consisting of K-pairs of massive MIMO FD nodes operating simultaneously. We present a novel joint hybrid beamforming (HYBF) and combining scheme for weighted sum-rate (WSR) maximization to enable the coexistence of massive MIMO FD links cost-efficiently. The proposed algorithm relies on alternative optimization based on the minorization-maximization method. Moreover, we present a novel SI and massive MIMO interference channel aware power allocation scheme to include the optimal power control. Simulation results show significant performance improvement compared to a traditional bidirectional fully digital half-duplex (HD) system.
DeepTrack: Lightweight Deep Learning for Vehicle Path Prediction in Highways
Venue: arXiv cs.LG 2022
Year: 2022

Vehicle trajectory prediction is essential for enabling safety-critical intelligent transportation systems (ITS) applications used in management and operations. While there have been some promising advances in the field, there is a need for modern deep learning algorithms that allow real-time trajectory prediction on embedded IoT devices. This article presents DeepTrack, a novel deep learning algorithm customized for real-time vehicle trajectory prediction and monitoring applications in arterial management, freeway management, traffic incident management, and work zone management for high-speed incoming traffic. In contrast to previous methods, the vehicle dynamics are encoded using Temporal Convolutional Networks (TCNs) to provide more robust time prediction with less computation. DeepTrack also uses depthwise convolution, which reduces the complexity of models compared to existing approaches in terms of model size and operations. Overall, our experimental results demonstrate that DeepTrack achieves comparable accuracy to state-of-the-art trajectory prediction models but with smaller model sizes and lower computational complexity, making it more suitable for real-world deployment.
Interference-Aware Accurate Signal Recovery in sub-1 GHz UHF Band Reuse-1 Cellular OFDMA Downlinks
Venue: arXiv cs.IT 2022
Year: 2022

Reuse-1 systems operating in the sub-1 GHz UHF band are limited by substantial co-channel interference (CCI). In such orthogonal frequency division multiple access (OFDMA) cellular systems, the inter-sector or inter-tower interference (ITI) makes accurate signal recovery quite challenging as sub-1 GHz bands only support single-input single-output (SISO) links. Interference-aware receiver algorithms are essential to mitigate the ITI in such low-frequency bands. Such algorithms enable ubiquitous mobile broadband access over the entire homeland, say with >95% geographical coverage with quality of service guarantees. One element of the interference-aware signal recovery is the least-squares-based joint channel estimation scheme that uses non-orthogonal pilot subcarriers. This estimator is then compared with a variant that uses orthogonal pilot subcarriers to bring out the advantage of this joint estimator. It is shown that the proposed joint estimator requires fewer pilots to be well-determined when compared to its under-determined orthogonal counterpart. Moreover, it is easy to implement and does not require any knowledge of channel statistics. This work also derives a compensation factor needed for the interference-aware detector in the presence of inter-carrier interference (ICI) originating from multiple transmitters. Simulation results show that the proposed joint channel estimator outperforms traditional estimators at moderate to high frequency selectivity. The proposed compensation factor to the joint detector is found to be essential for recovering the transmitted signal in the absence of phase-tracking pilots.
Multiple Classifiers Based Maximum Classifier Discrepancy for Unsupervised Domain Adaptation
Venue: arXiv cs.CV 2022
Year: 2022

Adversarial training based on the maximum classifier discrepancy between two classifier structures has achieved great success in unsupervised domain adaptation tasks for image classification. The approach adopts the structure of two classifiers, though simple and intuitive, the learned classification boundary may not well represent the data property in the new domain. In this paper, we propose to extend the structure to multiple classifiers to further boost its performance. To this end, we develop a very straightforward approach to adding more classifiers. We employ the principle that the classifiers are different from each other to construct a discrepancy loss function for multiple classifiers. The proposed construction method of loss function makes it possible to add any number of classifiers to the original framework. The proposed approach is validated through extensive experimental evaluations. We demonstrate that, on average, adopting the structure of three classifiers normally yields the best performance as a trade-off between accuracy and efficiency. With minimum extra computational costs, the proposed approach can significantly improve the performance of the original algorithm. The source code of the proposed approach can be downloaded from \url{https://github.com/rucv/MMCD\_DA}.
Tree-Algorithms with Multi-Packet Reception and Successive Interference Cancellation
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we perform a thorough analysis of tree-algorithms with multi-packet reception (MPR) and successive interference cancellation (SIC), showing a number of novel results. We first derive the basic performance parameters, which are the expected length of the collision resolution interval and the normalized throughput, conditioned on the number of contending users. We then study their asymptotic behaviour, identifying an oscillatory component that amplifies with the increase in MPR. In the next step, we derive the throughput for the gated and windowed access, assuming Poisson arrivals. We show that for windowed access, the bound on maximum stable normalized throughput increases with the increase in MPR. his implies that investing in advanced physical capabilities, i.e., MPR and SIC pays off from the perspective of the medium access control algorithm.
Communication-Efficient Federated Learning via Predictive Coding
Venue: arXiv cs.DC 2022
Year: 2022

Federated learning can enable remote workers to collaboratively train a shared machine learning model while allowing training data to be kept locally. In the use case of wireless mobile devices, the communication overhead is a critical bottleneck due to limited power and bandwidth. Prior work has utilized various data compression tools such as quantization and sparsification to reduce the overhead. In this paper, we propose a predictive coding based compression scheme for federated learning. The scheme has shared prediction functions among all devices and allows each worker to transmit a compressed residual vector derived from the reference. In each communication round, we select the predictor and quantizer based on the rate-distortion cost, and further reduce the redundancy with entropy coding. Extensive simulations reveal that the communication cost can be reduced up to 99% with even better learning performance when compared with other baseline methods.
"Robot Steganography"?: Opportunities and Challenges
Venue: arXiv cs.RO 2022
Year: 2022

Robots are being designed to communicate with people in various public and domestic venues in a helpful, discreet way. Here, we use a speculative approach to shine light on a new concept of robot steganography (RS), that a robot could seek to help vulnerable populations by discreetly warning of potential threats. We first identify some potentially useful scenarios for RS related to safety and security -- concerns that are estimated to cost the world trillions of dollars each year -- with a focus on two kinds of robots, an autonomous vehicle (AV) and a socially assistive humanoid robot (SAR). Next, we propose that existing, powerful, computer-based steganography (CS) approaches can be adopted with little effort in new contexts (SARs), while also pointing out potential benefits of human-like steganography (HS): although less efficient and robust than CS, HS represents a currently-unused form of RS that could also be used to avoid requiring computers or detection by more technically advanced adversaries. This analysis also introduces some unique challenges of RS that arise from message generation, indirect perception, and effects of perspective. For this, we explore some related theoretical and practical concerns for selecting carrier signals and generating messages, also making available some code and a video demo. Finally, we report on checking the current feasibility of the RS concept via a simplified user study, confirming that messages can be hidden in a robot's behaviors. The immediate implication is that RS could help to improve people's lives and mitigate some costly problems -- suggesting the usefulness of further discussion, ideation, and consideration by designers.
Your fairness may vary: Pretrained language model fairness in toxic text classification
Venue: arXiv cs.CL 2022
Year: 2022

The popularity of pretrained language models in natural language processing systems calls for a careful evaluation of such models in down-stream tasks, which have a higher potential for societal impact. The evaluation of such systems usually focuses on accuracy measures. Our findings in this paper call for attention to be paid to fairness measures as well. Through the analysis of more than a dozen pretrained language models of varying sizes on two toxic text classification tasks (English), we demonstrate that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics. Specifically, we observe that fairness can vary even more than accuracy with increasing training data size and different random initializations. At the same time, we find that little of the fairness variation is explained by model size, despite claims in the literature. To improve model fairness without retraining, we show that two post-processing methods developed for structured, tabular data can be successfully applied to a range of pretrained language models. Warning: This paper contains samples of offensive text.
A Gaze Data-based Comparative Study to Build a Trustworthy Human-AI Collaboration in Crash Anticipation
Venue: arXiv cs.HC 2022
Year: 2022

Vehicles with a safety function for anticipating crashes in advance can enhance drivers' ability to avoid crashes. As dashboard cameras have become a low-cost sensor device accessible to almost every vehicle, deep neural networks for crash anticipation from a dashboard camera are receiving growing interest. However, drivers' trust in the Artificial Intelligence (AI)-enabled safety function is built on the validation of its safety enhancement toward zero deaths. This paper is motivated to establish a method that uses gaze data and corresponding measures to evaluate human drivers' ability to anticipate crashes. A laboratory experiment is designed and performed, wherein a screen-based eye tracker collects the gaze data of six volunteers while watching 100 driving videos that include both normal and crash scenarios. Statistical analyses of the experimental data show that, on average, drivers can anticipate a crash up to 2.61 seconds before it occurs in this pilot study. The chance that drivers have successfully anticipated crashes before they occur is 92.8%. A state-of-the-art AI model can anticipate crashes 1.02 seconds earlier than drivers on average. The study finds that crash-involving traffic agents in the driving videos can vary drivers' instant attention level, average attention level, and spatial attention distribution. This finding supports the development of a spatial-temporal attention mechanism for AI models to strengthen their ability to anticipate crashes. Results from the comparison also suggest the development of collaborative intelligence that keeps human-in-the-loop of AI models to further enhance the reliability of AI-enabled safety functions.
Localized Shape Modelling with Global Coherence: An Inverse Spectral Approach
Venue: arXiv cs.CV 2022
Year: 2022

Many natural shapes have most of their characterizing features concentrated over a few regions in space. For example, humans and animals have distinctive head shapes, while inorganic objects like chairs and airplanes are made of well-localized functional parts with specific geometric features. Often, these features are strongly correlated -- a modification of facial traits in a quadruped should induce changes to the body structure. However, in shape modelling applications, these types of edits are among the hardest ones; they require high precision, but also a global awareness of the entire shape. Even in the deep learning era, obtaining manipulable representations that satisfy such requirements is an open problem posing significant constraints. In this work, we address this problem by defining a data-driven model upon a family of linear operators (variants of the mesh Laplacian), whose spectra capture global and local geometric properties of the shape at hand. Modifications to these spectra are translated to semantically valid deformations of the corresponding surface. By explicitly decoupling the global from the local surface features, our pipeline allows to perform local edits while simultaneously maintaining a global stylistic coherence. We empirically demonstrate how our learning-based model generalizes to shape representations not seen at training time, and we systematically analyze different choices of local operators over diverse shape categories.
Relational E-Matching
Venue: arXiv cs.DB 2022
Year: 2022

We present a new approach to e-matching based on relational join; in particular, we apply recent database query execution techniques to guarantee worst-case optimal run time. Compared to the conventional backtracking approach that always searches the e-graph "top down", our new relational e-matching approach can better exploit pattern structure by searching the e-graph according to an optimized query plan. We also establish the first data complexity result for e-matching, bounding run time as a function of the e-graph size and output size. We prototyped and evaluated our technique in the state-of-the-art egg e-graph framework. Compared to a conventional baseline, relational e-matching is simpler to implement and orders of magnitude faster in practice.
Deep Stable neural networks: large-width asymptotics and convergence rates
Venue: arXiv cs.LG 2022
Year: 2022

In modern deep learning, there is a recent and growing literature on the interplay between large-width asymptotic properties of deep Gaussian neural networks (NNs), i.e. deep NNs with Gaussian-distributed weights, and Gaussian stochastic processes (SPs). Such an interplay has proved to be critical in Bayesian inference under Gaussian SP priors, kernel regression for infinitely wide deep NNs trained via gradient descent, and information propagation within infinitely wide NNs. Motivated by empirical analyses that show the potential of replacing Gaussian distributions with Stable distributions for the NN's weights, in this paper we present a rigorous analysis of the large-width asymptotic behaviour of (fully connected) feed-forward deep Stable NNs, i.e. deep NNs with Stable-distributed weights. We show that as the width goes to infinity jointly over the NN's layers, i.e. the ``joint growth" setting, a rescaled deep Stable NN converges weakly to a Stable SP whose distribution is characterized recursively through the NN's layers. Because of the non-triangular structure of the NN, this is a non-standard asymptotic problem, to which we propose an inductive approach of independent interest. Then, we establish sup-norm convergence rates of the rescaled deep Stable NN to the Stable SP, under the ``joint growth" and a ``sequential growth" of the width over the NN's layers. Such a result provides the difference between the ``joint growth" and the ``sequential growth" settings, showing that the former leads to a slower rate than the latter, depending on the depth of the layer and the number of inputs of the NN. Our work extends some recent results on infinitely wide limits for deep Gaussian NNs to the more general deep Stable NNs, providing the first result on convergence rates in the ``joint growth" setting.
EENLP: Cross-lingual Eastern European NLP Index
Venue: arXiv cs.CL 2022
Year: 2022

Motivated by the sparsity of NLP resources for Eastern European languages, we present a broad index of existing Eastern European language resources (90+ datasets and 45+ models) published as a github repository open for updates from the community. Furthermore, to support the evaluation of commonsense reasoning tasks, we provide hand-crafted cross-lingual datasets for five different semantic tasks (namely news categorization, paraphrase detection, Natural Language Inference (NLI) task, tweet sentiment detection, and news sentiment detection) for some of the Eastern European languages. We perform several experiments with the existing multilingual models on these datasets to define the performance baselines and compare them to the existing results for other languages.
Beyond No Regret: Instance-Dependent PAC Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

The theory of reinforcement learning has focused on two fundamental problems: achieving low regret, and identifying $ε$-optimal policies. While a simple reduction allows one to apply a low-regret algorithm to obtain an $ε$-optimal policy and achieve the worst-case optimal rate, it is unknown whether low-regret algorithms can obtain the instance-optimal rate for policy identification. We show this is not possible -- there exists a fundamental tradeoff between achieving low regret and identifying an $ε$-optimal policy at the instance-optimal rate. Motivated by our negative finding, we propose a new measure of instance-dependent sample complexity for PAC tabular reinforcement learning which explicitly accounts for the attainable state visitation distributions in the underlying MDP. We then propose and analyze a novel, planning-based algorithm which attains this sample complexity -- yielding a complexity which scales with the suboptimality gaps and the "reachability" of a state. We show our algorithm is nearly minimax optimal, and on several examples that our instance-dependent sample complexity offers significant improvements over worst-case bounds.
GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

In this paper we propose \texttt{GIFAIR-FL}: a framework that imposes \textbf{G}roup and \textbf{I}ndividual \textbf{FAIR}ness to \textbf{F}ederated \textbf{L}earning settings. By adding a regularization term, our algorithm penalizes the spread in the loss of client groups to drive the optimizer to fair solutions. Our framework \texttt{GIFAIR-FL} can accommodate both global and personalized settings. Theoretically, we show convergence in non-convex and strongly convex settings. Our convergence guarantees hold for both $i.i.d.$ and non-$i.i.d.$ data. To demonstrate the empirical performance of our algorithm, we apply our method to image classification and text prediction tasks. Compared to existing algorithms, our method shows improved fairness results while retaining superior or similar prediction accuracy.
Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approach
Venue: arXiv cs.CV 2022
Year: 2022

We are concerned with a worst-case scenario in model generalization, in the sense that a model aims to perform well on many unseen domains while there is only one single domain available for training. We propose Meta-Learning based Adversarial Domain Augmentation to solve this Out-of-Domain generalization problem. The key idea is to leverage adversarial training to create "fictitious" yet "challenging" populations, from which a model can learn to generalize with theoretical guarantees. To facilitate fast and desirable domain augmentation, we cast the model training in a meta-learning scheme and use a Wasserstein Auto-Encoder to relax the widely used worst-case constraint. We further improve our method by integrating uncertainty quantification for efficient domain generalization. Extensive experiments on multiple benchmark datasets indicate its superior performance in tackling single domain generalization.
Domain-Centered Support for Layout, Tasks, and Specification for Control Flow Graph Visualization
Venue: arXiv cs.HC 2022
Year: 2022

Computing professionals in areas like compilers, performance analysis, and security often analyze and manipulate control flow graphs (CFGs) in their work. CFGs are directed networks that describe possible orderings of instructions in the execution of a program. Visualizing a CFG is a common activity in developing or debugging computational approaches that use them. However, general graph drawing layouts, including the hierarchical ones frequently applied to CFGs, do not capture CFG-specific structures or tasks and thus the resulting drawing may not match the needs of their audience, especially for more complicated programs. While several algorithms offer flexibility in specifying the layout, they often require expertise with graph drawing layouts and primitives that these potential users do not have. To bring domain-specific CFG drawing to this audience, we develop CFGConf, a library designed to match the abstraction level of CFG experts. CFGConf provides a JSON interface that produces drawings that can stand-alone or be integrated into multi-view visualization systems. We developed CFGConf through an interactive design process with experts while incorporating lessons learned from previous CFG visualization systems, a survey of CFG drawing conventions in computing systems conferences, and existing design principles for notations. We evaluate CFGConf in terms of expressiveness, usability, and notational efficiency through a user study and illustrative examples. CFG experts were able to use the library to produce the domain-aware layouts and appreciated the task-aware nature of the specification.
Differentiable Moving Horizon Estimation for Robust Flight Control
Venue: arXiv cs.RO 2022
Year: 2022

Estimating and reacting to external disturbances is of fundamental importance for robust control of quadrotors. Existing estimators typically require significant tuning or training with a large amount of data, including the ground truth, to achieve satisfactory performance. This paper proposes a data-efficient differentiable moving horizon estimation (DMHE) algorithm that can automatically tune the MHE parameters online and also adapt to different scenarios. We achieve this by deriving the analytical gradient of the estimated trajectory from MHE with respect to the tuning parameters, enabling end-to-end learning for auto-tuning. Most interestingly, we show that the gradient can be calculated efficiently from a Kalman filter in a recursive form. Moreover, we develop a model-based policy gradient algorithm to learn the parameters directly from the trajectory tracking errors without the need for the ground truth. The proposed DMHE can be further embedded as a layer with other neural networks for joint optimization. Finally, we demonstrate the effectiveness of the proposed method via both simulation and experiments on quadrotors, where challenging scenarios such as sudden payload change and flying in downwash are examined.