OpenRaiser/Intern-Atlas huggingface dataset:
User Scheduling and Power Allocation for Precoded Multi-Beam High Throughput Satellite Systems with Individual Quality of Service Constraints
Venue: arXiv cs.IT 2021
Year: 2021

For extensive coverage areas, multi-beam high throughput satellite (MB-HTS) communication is a promising technology that plays a crucial role in delivering broadband services to many users with diverse Quality of Service (QoS) requirements. This paper focuses on MB-HTS systems where all beams reuse the same spectrum. In particular, we propose a novel user scheduling and power allocation design capable of providing guarantees in terms of the individual QoS requirements while maximizing the system throughput under a limited power budget. Precoding is employed in the forward link to mitigate mutual interference at the users in multiple-access scenarios over different coherence time intervals. The combinatorial optimization structure from user scheduling requires an extremely high cost to obtain the global optimum even when a reduced number of users fit into a time slot. Therefore, we propose a heuristic algorithm yielding good trade-off between performance and computational complexity, applicable to a static operation framework of geostationary (GEO) satellite networks. Although the power allocation optimization is signomial programming, non-convex on a standard form, the solution can be lower bounded by the global optimum of a geometric program with a hidden convex structure. A local solution to the joint user scheduling and power allocation problem is consequently obtained by a successive optimization approach. Numerical results demonstrate the effectiveness of our algorithms on large-scale systems by providing better QoS satisfaction combined with outstanding overall system throughput.
A Regularized Wasserstein Framework for Graph Kernels
Venue: arXiv cs.LG 2021
Year: 2021

We propose a learning framework for graph kernels, which is theoretically grounded on regularizing optimal transport. This framework provides a novel optimal transport distance metric, namely Regularized Wasserstein (RW) discrepancy, which can preserve both features and structure of graphs via Wasserstein distances on features and their local variations, local barycenters and global connectivity. Two strongly convex regularization terms are introduced to improve the learning ability. One is to relax an optimal alignment between graphs to be a cluster-to-cluster mapping between their locally connected vertices, thereby preserving the local clustering structure of graphs. The other is to take into account node degree distributions in order to better preserve the global structure of graphs. We also design an efficient algorithm to enable a fast approximation for solving the optimization problem. Theoretically, our framework is robust and can guarantee the convergence and numerical stability in optimization. We have empirically validated our method using 12 datasets against 16 state-of-the-art baselines. The experimental results show that our method consistently outperforms all state-of-the-art methods on all benchmark databases for both graphs with discrete attributes and graphs with continuous attributes.
Simplicial Convolutional Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Graphs can model networked data by representing them as nodes and their pairwise relationships as edges. Recently, signal processing and neural networks have been extended to process and learn from data on graphs, with achievements in tasks like graph signal reconstruction, graph or node classifications, and link prediction. However, these methods are only suitable for data defined on the nodes of a graph. In this paper, we propose a simplicial convolutional neural network (SCNN) architecture to learn from data defined on simplices, e.g., nodes, edges, triangles, etc. We study the SCNN permutation and orientation equivariance, complexity, and spectral analysis. Finally, we test the SCNN performance for imputing citations on a coauthorship complex.
A multi-order smoothed particle hydrodynamics method for cardiac electromechanics with the Purkinje network
Venue: arXiv cs.CE 2021
Year: 2021

In previous work, Zhang et al. (2021) \cite{zhang2021integrative} developed an integrated smoothed particle hydrodynamics (SPH) method to simulate the principle aspects of cardiac function, including electrophysiology, passive and active mechanical response of the myocardium. As the inclusion of the Purkinje network in electrocardiology is recognized as fundamental to accurately describing the electrical activation in the right and left ventricles, in this paper, we present a multi-order SPH method to handle the electrical propagation through the Purkinje system and in the myocardium with monodomain/monodomain coupling strategy. We first propose an efficient algorithm for network generation on arbitrarily complex surface by exploiting level-set geometry representation and cell-linked list neighbor search algorithm. Then, a reduced-order SPH method is developed to solve the one-dimensional monodomain equation to characterize the fast electrical activation through the Purkinje network. Finally, a multi-order coupling paradigm is introduced to capture the coupled nature of potential propagation arising from the interaction between the network and the myocardium. A set of numerical examples are studied to assess the computational performance, accuracy and versatility of the proposed methods. In particular, numerical study performed in realistic left ventricle demonstrates that the present method features all the physiological issues that characterize a heartbeat simulation, including the initiation of the signal in the Purkinje network and the systolic and diastolic phases. As expected, the results underlie the importance of using physiologically realistic Purkinje network for modeling cardiac functions.
Characterizing Learning Dynamics of Deep Neural Networks via Complex Networks
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we interpret Deep Neural Networks with Complex Network Theory. Complex Network Theory (CNT) represents Deep Neural Networks (DNNs) as directed weighted graphs to study them as dynamical systems. We efficiently adapt CNT measures to examine the evolution of the learning process of DNNs with different initializations and architectures: we introduce metrics for nodes/neurons and layers, namely Nodes Strength and Layers Fluctuation. Our framework distills trends in the learning dynamics and separates low from high accurate networks. We characterize populations of neural networks (ensemble analysis) and single instances (individual analysis). We tackle standard problems of image recognition, for which we show that specific learning dynamics are indistinguishable when analysed through the solely Link-Weights analysis. Further, Nodes Strength and Layers Fluctuations make unprecedented behaviours emerge: accurate networks, when compared to under-trained models, show substantially divergent distributions with the greater extremity of deviations. On top of this study, we provide an efficient implementation of the CNT metrics for both Convolutional and Fully Connected Networks, to fasten the research in this direction.
On The Transferability of Deep-Q Networks
Venue: arXiv cs.LG 2021
Year: 2021

Transfer Learning (TL) is an efficient machine learning paradigm that allows overcoming some of the hurdles that characterize the successful training of deep neural networks, ranging from long training times to the needs of large datasets. While exploiting TL is a well established and successful training practice in Supervised Learning (SL), its applicability in Deep Reinforcement Learning (DRL) is rarer. In this paper, we study the level of transferability of three different variants of Deep-Q Networks on popular DRL benchmarks as well as on a set of novel, carefully designed control tasks. Our results show that transferring neural networks in a DRL context can be particularly challenging and is a process which in most cases results in negative transfer. In the attempt of understanding why Deep-Q Networks transfer so poorly, we gain novel insights into the training dynamics that characterizes this family of algorithms.
How good does a Defect Predictor need to be to guide Search-Based Software Testing?
Venue: arXiv cs.SE 2021
Year: 2021

Defect predictors, static bug detectors and humans inspecting the code can locate the parts of the program that are buggy before they are discovered through testing. Automated test generators such as search-based software testing (SBST) techniques can use this information to direct their search for test cases to likely buggy code, thus speeding up the process of detecting existing bugs. However, often the predictions given by these tools or humans are imprecise, which can misguide the SBST technique and may deteriorate its performance. In this paper, we study the impact of imprecision in defect prediction on the bug detection effectiveness of SBST. Our study finds that the recall of the defect predictor, i.e., the probability of correctly identifying buggy code, has a significant impact on bug detection effectiveness of SBST with a large effect size. On the other hand, the effect of precision, a measure for false alarms, is not of meaningful practical significance as indicated by a very small effect size. In particular, the SBST technique finds 7.5 less bugs on average (out of 420 bugs) for every 5% decrements of the recall. In the context of combining defect prediction and SBST, our recommendation for practice is to increase the recall of defect predictors at the expense of precision, while maintaining a precision of at least 75%. To account for the imprecision of defect predictors, in particular low recall values, SBST techniques should be designed to search for test cases that also cover the predicted non-buggy parts of the program, while prioritising the parts that have been predicted as buggy.
The Information Geometry of Unsupervised Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

How can a reinforcement learning (RL) agent prepare to solve downstream tasks if those tasks are not known a priori? One approach is unsupervised skill discovery, a class of algorithms that learn a set of policies without access to a reward function. Such algorithms bear a close resemblance to representation learning algorithms (e.g., contrastive learning) in supervised learning, in that both are pretraining algorithms that maximize some approximation to a mutual information objective. While prior work has shown that the set of skills learned by such methods can accelerate downstream RL tasks, prior work offers little analysis into whether these skill learning algorithms are optimal, or even what notion of optimality would be appropriate to apply to them. In this work, we show that unsupervised skill discovery algorithms based on mutual information maximization do not learn skills that are optimal for every possible reward function. However, we show that the distribution over skills provides an optimal initialization minimizing regret against adversarially-chosen reward functions, assuming a certain type of adaptation procedure. Our analysis also provides a geometric perspective on these skill learning methods.
ParaDiS: Parallelly Distributable Slimmable Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

When several limited power devices are available, one of the most efficient ways to make profit of these resources, while reducing the processing latency and communication load, is to run in parallel several neural sub-networks and to fuse the result at the end of processing. However, such a combination of sub-networks must be trained specifically for each particular configuration of devices (characterized by number of devices and their capacities) which may vary over different model deployments and even within the same deployment. In this work we introduce parallelly distributable slimmable (ParaDiS) neural networks that are splittable in parallel among various device configurations without retraining. While inspired by slimmable networks allowing instant adaptation to resources on just one device, ParaDiS networks consist of several multi-device distributable configurations or switches that strongly share the parameters between them. We evaluate ParaDiS framework on MobileNet v1 and ResNet-50 architectures on ImageNet classification task and WDSR architecture for image super-resolution task. We show that ParaDiS switches achieve similar or better accuracy than the individual models, i.e., distributed models of the same structure trained individually. Moreover, we show that, as compared to universally slimmable networks that are not distributable, the accuracy of distributable ParaDiS switches either does not drop at all or drops by a maximum of 1 % only in the worst cases. Finally, once distributed over several devices, ParaDiS outperforms greatly slimmable models.
Blind Coherent Preamble Detection via Neural Networks
Venue: arXiv cs.IT 2021
Year: 2021

In wireless communications systems, the user equipment (UE) transmits a random access preamble sequence to the base station (BS) to be detected and synchronized. In standardized cellular communications systems Zadoff-Chu sequences has been proposed due to their constant amplitude zero autocorrelation (CAZAC) properties. The conventional approach is to use matched filters to detect the sequence. Sequences arrived from different antennas and time instances are summed up to reduce the noise variance. Since the knowledge of the channel is unknown at this stage, a coherent combining scheme would be very difficult to implement. In this work, we leverage the system design knowledge and propose a neural network (NN) sequence detector and timing advanced estimator. We do not replace the whole process of preamble detection by a NN. Instead, we propose to use NN only for \textit{blind} coherent combining of the signals in the detector to compensate for the channel effect, thus maximize the signal to noise ratio. We have further reduced the problem's complexity using Kronecker approximation model for channel covariance matrices, thereby, reducing the size of required NN. The analysis on timing advanced estimation and sequences detection has been performed and compared with the matched filter baseline.
Cluster Analysis on Jester Dataset: A Review
Venue: arXiv cs.LG 2021
Year: 2021

Unsupervised Machine Learning Paradigms are often the only methodology to rely on, given a Pattern Recognition Task with no target label or annotations being present. In such scenarios, data preparation is a crucial step to be performed so that the Unsupervised Paradigms work with as much perfection as possible. But, when there is no sufficient or missing data being present in each and every instance of a dataset, data preparation becomes a challenge itself. One such case-study is the Jester Dataset that has missing values which are basically ratings given by Joke-Readers to a specified set of 100 jokes. In order to perform a Cluster Analysis on such a dataset, the data preparation step should involve filling the missing ratings with appropriate values followed by cluster analysis using an Unsupervised ML Paradigm. In this study, the most recent and probably the only work that involves Cluster Analysis on the Jester Dataset of Jokes is reviewed and validated with corrections and future scope.
On the Application of Uplink/Downlink Decoupled Access in Heterogeneous Mobile Edge Computing
Venue: arXiv cs.IT 2021
Year: 2021

Mobile edge computing (MEC) is a key player in low latency 5G networks with the task to resolve the conflict between computationally-intensive mobile applications and resource-limited mobile devices (MDs). As such, there has been intense interest in this topic, especially in multi-user single-server and homogeneous multi-server scenarios. However, the research in the heterogeneous multi-server scenario is limited, where the servers are located at small base-stations (SBSs), macro base-stations (MBSs), or the cloud with different computing and communication capabilities. On the other hand, computational-tasks offloading is limited by the type of MD-BS association with almost all previous works focusing on offloading the MD's computational tasks to the MEC servers/cloudlets at its serving BS. However, in multi-BS association, or downlink/uplink decoupled (DUDe) scenarios, an MD can be served by multiple BSs and hence has multiple offloading choices. Motivated by this, we proposed a joint BS association and subchannel allocation algorithm based on a student-project allocation (SPA) matching approach to minimize the network sum-latency, which break the constraint that one MD must connect to the same BS in the UL and DL, and jointly consider the communication and computational disparity of SBS and MBS cloudlets in heterogeneous MEC networks. Moreover, an optimal power allocation scheme is proposed to optimize the system performance subject to the predefined quality of service constraints. Our results show that the proposed scheme is superior to benchmark techniques in enabling effective use of the computational and communication resources in heterogeneous MEC networks.
Real-time Optimal Landing Control of the MIT Mini Cheetah
Venue: arXiv cs.RO 2021
Year: 2021

Quadrupedal landing is a complex process involving large impacts, elaborate contact transitions, and is a crucial recovery behavior observed in many biological animals. This work presents a real-time, optimal landing controller that is free of pre-specified contact schedules. The controller determines optimal touchdown postures and reaction force profiles and is able to recover from a variety of falling configurations. The quadrupedal platform used, the MIT Mini Cheetah, recovered safely from drops of up to 8 m in simulation, as well as from a range of orientations and planar velocities. The controller is also tested on hardware, successfully recovering from drops of up to 2 m.
Shallow Features Guide Unsupervised Domain Adaptation for Semantic Segmentation at Class Boundaries
Venue: arXiv cs.CV 2021
Year: 2021

Although deep neural networks have achieved remarkable results for the task of semantic segmentation, they usually fail to generalize towards new domains, especially when performing synthetic-to-real adaptation. Such domain shift is particularly noticeable along class boundaries, invalidating one of the main goals of semantic segmentation that consists in obtaining sharp segmentation masks. In this work, we specifically address this core problem in the context of Unsupervised Domain Adaptation and present a novel low-level adaptation strategy that allows us to obtain sharp predictions. Moreover, inspired by recent self-training techniques, we introduce an effective data augmentation that alleviates the noise typically present at semantic boundaries when employing pseudo-labels for self-training. Our contributions can be easily integrated into other popular adaptation frameworks, and extensive experiments show that they effectively improve performance along class boundaries.
Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection
Venue: arXiv cs.CV 2021
Year: 2021

In industrial manufacturing processes, errors frequently occur at unpredictable times and in unknown manifestations. We tackle the problem of automatic defect detection without requiring any image samples of defective parts. Recent works model the distribution of defect-free image data, using either strong statistical priors or overly simplified data representations. In contrast, our approach handles fine-grained representations incorporating the global and local image context while flexibly estimating the density. To this end, we propose a novel fully convolutional cross-scale normalizing flow (CS-Flow) that jointly processes multiple feature maps of different scales. Using normalizing flows to assign meaningful likelihoods to input samples allows for efficient defect detection on image-level. Moreover, due to the preserved spatial arrangement the latent space of the normalizing flow is interpretable which enables to localize defective regions in the image. Our work sets a new state-of-the-art in image-level defect detection on the benchmark datasets Magnetic Tile Defects and MVTec AD showing a 100% AUROC on 4 out of 15 classes.
Distribution Preserving Multiple Hypotheses Prediction for Uncertainty Modeling
Venue: arXiv cs.LG 2021
Year: 2021

Many supervised machine learning tasks, such as future state prediction in dynamical systems, require precise modeling of a forecast's uncertainty. The Multiple Hypotheses Prediction (MHP) approach addresses this problem by providing several hypotheses that represent possible outcomes. Unfortunately, with the common $l_2$ loss function, these hypotheses do not preserve the data distribution's characteristics. We propose an alternative loss for distribution preserving MHP and review relevant theorems supporting our claims. Furthermore, we empirically show that our approach yields more representative hypotheses on a synthetic and a real-world motion prediction data set. The outputs of the proposed method can directly be used in sampling-based Monte-Carlo methods.
Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings
Venue: arXiv cs.CL 2021
Year: 2021

Recent studies have proposed different methods to improve multilingual word representations in contextualized settings including techniques that align between source and target embedding spaces. For contextualized embeddings, alignment becomes more complex as we additionally take context into consideration. In this work, we propose using Optimal Transport (OT) as an alignment objective during fine-tuning to further improve multilingual contextualized representations for downstream cross-lingual transfer. This approach does not require word-alignment pairs prior to fine-tuning that may lead to sub-optimal matching and instead learns the word alignments within context in an unsupervised manner. It also allows different types of mappings due to soft matching between source and target sentences. We benchmark our proposed method on two tasks (XNLI and XQuAD) and achieve improvements over baselines as well as competitive results compared to similar recent works.
Predicting the Popularity of Games on Steam
Venue: arXiv cs.LG 2021
Year: 2021

The video game industry has seen rapid growth over the last decade. Thousands of video games are released and played by millions of people every year, creating a large community of players. Steam is a leading gaming platform and social networking site, which allows its users to purchase and store games. A by-product of Steam is a large database of information about games, players, and gaming behavior. In this paper, we take recent video games released on Steam and aim to discover the relation between game popularity and a game's features that can be acquired through Steam. We approach this task by predicting the popularity of Steam games in the early stages after their release and we use a Bayesian approach to understand the influence of a game's price, size, supported languages, release date, and genres on its player count. We implement several models and discover that a genre-based hierarchical approach achieves the best performance. We further analyze the model and interpret its coefficients, which indicate that games released at the beginning of the month and games of certain genres correlate with game popularity.