OpenRaiser/Intern-Atlas huggingface dataset:
A semi-supervised autoencoder framework for joint generation and classification of breathing
Venue: arXiv cs.LG 2021
Year: 2021

One of the main problems with biomedical signals is the limited amount of patient-specific data and the significant amount of time needed to record the sufficient number of samples needed for diagnostic and treatment purposes. In this study, we present a framework to simultaneously generate and classify biomedical time series based on a modified Adversarial Autoencoder (AAE) algorithm and one-dimensional convolutions. Our work is based on breathing time series, with specific motivation to capture breathing motion during radiotherapy lung cancer treatments. First, we explore the potential in using the Variational Autoencoder (VAE) and AAE algorithms to model breathing from individual patients. We extend the AAE algorithm to allow joint semi-supervised classification and generation of different types of signals. To simplify the modeling task, we introduce a pre-processing and post-processing compressing algorithm that transforms the multi-dimensional time series into vectors containing time and position values, which are transformed back into time series through an additional neural network. By incorporating few labeled samples during training, our model outperforms other purely discriminative networks in classifying breathing baseline shift irregularities from a dataset completely different from the training set. To our knowledge, the presented framework is the first approach that unifies generation and classification within a single model for this type of biomedical data, enabling both computer aided diagnosis and augmentation of labeled samples within a single framework.
Interpretable Data-Driven Demand Modelling for On-Demand Transit Services
Venue: arXiv cs.CY 2021
Year: 2021

In recent years, with the advancements in information and communication technology, different emerging on-demand shared mobility services have been introduced as innovative solutions in the low-density areas, including on-demand transit (ODT), mobility on-demand (MOD) transit, and crowdsourced mobility services. However, due to their infancy, there is a strong need to understand and model the demand for these services. In this study, we developed trip production and distribution models for ODT services at Dissemination areas (DA) level using four machine learning algorithms: Random Forest (RF), Bagging, Artificial Neural Network (ANN) and Deep Neural Network (DNN). The data used in the modelling process were acquired from Belleville's ODT operational data and 2016 census data. Bayesian optimalization approach was used to find the optimal architecture of the adopted algorithms. Moreover, post-hoc model was employed to interpret the predictions and examine the importance of the explanatory variables. The results showed that the land-use type was the most important variable in the trip production model. On the other hand, the demographic characteristics of the trip destination were the most important variables in the trip distribution model. Moreover, the results revealed that higher trip distribution levels are expected between dissemination areas with commercial/industrial land-use type and dissemination areas with high-density residential land-use. Our findings suggest that the performance of ODT services can be further enhanced by (a) locating idle vehicles in the neighbourhoods with commercial/industrial land-use and (b) using the spatio-temporal demand models obtained in this work to continuously update the operating fleet size.
Constrained Online Learning to Mitigate Distortion Effects in Pulse-Agile Cognitive Radar
Venue: arXiv cs.IT 2021
Year: 2021

Pulse-agile radar systems have demonstrated favorable performance in dynamic electromagnetic scenarios. However, the use of non-identical waveforms within a radar's coherent processing interval may lead to harmful distortion effects when pulse-Doppler processing is used. This paper presents an online learning framework to optimize detection performance while mitigating harmful sidelobe levels. The radar waveform selection process is formulated as a linear contextual bandit problem, within which waveform adaptations which exceed a tolerable level of expected distortion are eliminated. The constrained online learning approach is effective and computationally feasible, evidenced by simulations in a radar-communication coexistence scenario and in the presence of intentional adaptive jamming. This approach is applied to both stochastic and adversarial contextual bandit learning models and the detection performance in dynamic scenarios is evaluated.
Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones
Venue: arXiv cs.LG 2021
Year: 2021

Safety remains a central obstacle preventing widespread use of RL in the real world: learning new tasks in uncertain environments requires extensive exploration, but safety requires limiting exploration. We propose Recovery RL, an algorithm which navigates this tradeoff by (1) leveraging offline data to learn about constraint violating zones before policy learning and (2) separating the goals of improving task performance and constraint satisfaction across two policies: a task policy that only optimizes the task reward and a recovery policy that guides the agent to safety when constraint violation is likely. We evaluate Recovery RL on 6 simulation domains, including two contact-rich manipulation tasks and an image-based navigation task, and an image-based obstacle avoidance task on a physical robot. We compare Recovery RL to 5 prior safe RL methods which jointly optimize for task performance and safety via constrained optimization or reward shaping and find that Recovery RL outperforms the next best prior method across all domains. Results suggest that Recovery RL trades off constraint violations and task successes 2 - 20 times more efficiently in simulation domains and 3 times more efficiently in physical experiments. See https://tinyurl.com/rl-recovery for videos and supplementary material.
Active Sampling Count Sketch (ASCS) for Online Sparse Estimation of a Trillion Scale Covariance Matrix
Venue: arXiv cs.DS 2021
Year: 2021

Estimating and storing the covariance (or correlation) matrix of high-dimensional data is computationally challenging because both memory and computational requirements scale quadratically with the dimension. Fortunately, high-dimensional covariance matrices as observed in text, click-through, meta-genomics datasets, etc are often sparse. In this paper, we consider the problem of efficient sparse estimation of covariance matrices with possibly trillions of entries. The size of the datasets we target requires the algorithm to be online, as more than one pass over the data is prohibitive. In this paper, we propose Active Sampling Count Sketch (ASCS), an online and one-pass sketching algorithm, that recovers the large entries of the covariance matrix accurately. Count Sketch (CS), and other sub-linear compressed sensing algorithms, offer a natural solution to the problem in theory. However, vanilla CS does not work well in practice due to a low signal-to-noise ratio (SNR). At the heart of our approach is a novel active sampling strategy that increases the SNR of classical CS. We demonstrate the practicality of our algorithm with synthetic data and real-world high dimensional datasets. ASCS significantly improves over vanilla CS, demonstrating the merit of our active sampling strategy.
A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Highly skewed long-tail item distribution is very common in recommendation systems. It significantly hurts model performance on tail items. To improve tail-item recommendation, we conduct research to transfer knowledge from head items to tail items, leveraging the rich user feedback in head items and the semantic connections between head and tail items. Specifically, we propose a novel dual transfer learning framework that jointly learns the knowledge transfer from both model-level and item-level: 1. The model-level knowledge transfer builds a generic meta-mapping of model parameters from few-shot to many-shot model. It captures the implicit data augmentation on the model-level to improve the representation learning of tail items. 2. The item-level transfer connects head and tail items through item-level features, to ensure a smooth transfer of meta-mapping from head items to tail items. The two types of transfers are incorporated to ensure the learned knowledge from head items can be well applied for tail item representation learning in the long-tail distribution settings. Through extensive experiments on two benchmark datasets, results show that our proposed dual transfer learning framework significantly outperforms other state-of-the-art methods for tail item recommendation in hit ratio and NDCG. It is also very encouraging that our framework further improves head items and overall performance on top of the gains on tail items.
CliniQG4QA: Generating Diverse Questions for Domain Adaptation of Clinical Question Answering
Venue: arXiv cs.CL 2021
Year: 2021

Clinical question answering (QA) aims to automatically answer questions from medical professionals based on clinical texts. Studies show that neural QA models trained on one corpus may not generalize well to new clinical texts from a different institute or a different patient group, where large-scale QA pairs are not readily available for model retraining. To address this challenge, we propose a simple yet effective framework, CliniQG4QA, which leverages question generation (QG) to synthesize QA pairs on new clinical contexts and boosts QA models without requiring manual annotations. In order to generate diverse types of questions that are essential for training QA models, we further introduce a seq2seq-based question phrase prediction (QPP) module that can be used together with most existing QG models to diversify the generation. Our comprehensive experiment results show that the QA corpus generated by our framework can improve QA models on the new contexts (up to 8% absolute gain in terms of Exact Match), and that the QPP module plays a crucial role in achieving the gain.
Finding Efficient Domination for $S_{1,1,5}$-Free Bipartite Graphs in Polynomial Time
Venue: arXiv cs.DM 2021
Year: 2021

A vertex set $D$ in a finite undirected graph $G$ is an {\em efficient dominating set} (e.d.s.\ for short) of $G$ if every vertex of $G$ is dominated by exactly one vertex of $D$. The \emph{Efficient Domination} (ED) problem, which asks for the existence of an e.d.s.\ in $G$, is \NP-complete for various $H$-free bipartite graphs, e.g., Lu and Tang showed that ED is \NP-complete for chordal bipartite graphs and for planar bipartite graphs; actually, ED is \NP-complete even for planar bipartite graphs with vertex degree at most 3 and girth at least $g$ for every fixed $g$. Thus, ED is \NP-complete for $K_{1,4}$-free bipartite graphs and for $C_4$-free bipartite graphs. In this paper, we show that ED can be solved in polynomial time for $S_{1,1,5}$-free bipartite graphs.
On the Impact of Communities on Semi-supervised Classification Using Graph Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Graph Neural Networks (GNNs) are effective in many applications. Still, there is a limited understanding of the effect of common graph structures on the learning process of GNNs. In this work, we systematically study the impact of community structure on the performance of GNNs in semi-supervised node classification on graphs. Following an ablation study on six datasets, we measure the performance of GNNs on the original graphs, and the change in performance in the presence and the absence of community structure. Our results suggest that communities typically have a major impact on the learning process and classification performance. For example, in cases where the majority of nodes from one community share a single classification label, breaking up community structure results in a significant performance drop. On the other hand, for cases where labels show low correlation with communities, we find that the graph structure is rather irrelevant to the learning process, and a feature-only baseline becomes hard to beat. With our work, we provide deeper insights in the abilities and limitations of GNNs, including a set of general guidelines for model selection based on the graph structure.
Knowledge-Based Construction of Confusion Matrices for Multi-Label Classification Algorithms using Semantic Similarity Measures
Venue: arXiv cs.LG 2021
Year: 2021

So far, multi-label classification algorithms have been evaluated using statistical methods that do not consider the semantics of the considered classes and that fully depend on abstract computations such as Bayesian Reasoning. Currently, there are several attempts to develop ontology-based methods for a better assessment of supervised classification algorithms. In this research paper, we define a novel approach that aligns expected labels with predicted labels in multi-label classification using ontology-driven feature-based semantic similarity measures and we use it to develop a method for creating precise confusion matrices for a more effective evaluation of multi-label classification algorithms.
The Journal Coverage of Web of Science, Scopus and Dimensions: A Comparative Analysis
Venue: arXiv cs.DL 2021
Year: 2021

Traditionally, Web of Science and Scopus have been the two most widely used databases for bibliometric analyses. However, during the last few years some new scholarly databases, such as Dimensions, have come up. Several previous studies have compared different databases, either through a direct comparison of article coverage or by comparing the citations across the databases. This article attempts to compare the journal coverage of the three databases: Web of Science, Scopus and Dimensions. The most recent master journal lists of the three databases have been used for the purpose of identifying the overlapping and unique journals covered in the databases. The results indicate that the databases have significantly different journal coverage, with the Web of Science being most selective and Dimensions being the most exhaustive. About 99.11% and 96.61% of the journals indexed in Web of Science are also indexed in Scopus and Dimensions, respectively. Scopus has 96.42% of its indexed journals also covered by Dimensions. Dimensions database has the most exhaustive coverage, with 82.22% more journals covered as compared to Web of Science and 48.17% more journals covered as compared to Scopus. We also analysed the research outputs for 20 highly productive countries for the 2010-2019 period, as indexed in the three databases, and identified database-induced variations in research output volume, rank and global share of different countries. In addition to variations in overall coverage of research output from different countries, the three databases appear to have differential coverage of different disciplines.
Optimal Sample Complexity of Subgradient Descent for Amplitude Flow via Non-Lipschitz Matrix Concentration
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of recovering a real-valued $n$-dimensional signal from $m$ phaseless, linear measurements and analyze the amplitude-based non-smooth least squares objective. We establish local convergence of subgradient descent with optimal sample complexity based on the uniform concentration of a random, discontinuous matrix-valued operator arising from the objective's gradient dynamics. While common techniques to establish uniform concentration of random functions exploit Lipschitz continuity, we prove that the discontinuous matrix-valued operator satisfies a uniform matrix concentration inequality when the measurement vectors are Gaussian as soon as $m = Ω(n)$ with high probability. We then show that satisfaction of this inequality is sufficient for subgradient descent with proper initialization to converge linearly to the true solution up to the global sign ambiguity. As a consequence, this guarantees local convergence for Gaussian measurements at optimal sample complexity. The concentration methods in the present work have previously been used to establish recovery guarantees for a variety of inverse problems under generative neural network priors. This paper demonstrates the applicability of these techniques to more traditional inverse problems and serves as a pedagogical introduction to those results.
Linear Classifiers that Encourage Constructive Adaptation
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning systems are often used in settings where individuals adapt their features to obtain a desired outcome. In such settings, strategic behavior leads to a sharp loss in model performance in deployment. In this work, we aim to address this problem by learning classifiers that encourage decision subjects to change their features in a way that leads to improvement in both predicted \emph{and} true outcome. We frame the dynamics of prediction and adaptation as a two-stage game, and characterize optimal strategies for the model designer and its decision subjects. In benchmarks on simulated and real-world datasets, we find that classifiers trained using our method maintain the accuracy of existing approaches while inducing higher levels of improvement and less manipulation.
A multiscale model of terrain dynamics for real-time earthmoving simulation
Venue: arXiv cs.CE 2021
Year: 2021

A multiscale model for real-time simulation of terrain dynamics is explored. To represent the dynamics on different scales the model combines the description of soil as a continuous solid, as distinct particles and as rigid multibodies. The models are dynamically coupled to each other and to the earthmoving equipment. Agitated soil is represented by a hybrid of contacting particles and continuum solid, with the moving equipment and resting soil as geometric boundaries. Each zone of active soil is aggregated into distinct bodies, with the proper mass, momentum and frictional-cohesive properties, which constrain the equipment's multibody dynamics. The particle model parameters are pre-calibrated to the bulk mechanical parameters for a wide range of different soils. The result is a computationally efficient model for earthmoving operations that resolve the motion of the soil, using a fast iterative solver, and provide realistic forces and dynamic for the equipment, using a direct solver for high numerical precision. Numerical simulations of excavation and bulldozing operations are performed to validate the model and measure the computational performance. Reference data is produced using coupled discrete element and multibody dynamics simulations at relatively high resolution. The digging resistance and soil displacements with the real-time multiscale model agree with the reference model up to 10-25%, and run more than three orders of magnitude faster.
On the Distribution of SINR for Widely Linear MMSE MIMO Systems with Rectilinear or Quasi-Rectilinear Signals
Venue: arXiv cs.IT 2021
Year: 2021

Although the widely linear least mean square error (WLMMSE) receiver has been an appealing option for multiple-input-multiple-output (MIMO) wireless systems, a statistical understanding on its pose-detection signal-to-interference-plus-noise ratio (SINR) in detail is still missing. To this end, we consider a WLMMSE MIMO transmission system with rectilinear or quasi-rectilinear (QR) signals over the uncorrelated Rayleigh fading channel and investigate the statistical properties of its SINR for an arbitrary antenna configuration with $N_t$ transmit antennas and $N_r$ receive ones. We first derive an analytic probability density function (PDF) of the SINR in terms of the confluent hypergeometric function of the second kind, for WLMMSE MIMO systems with an arbitrary $N_r$ and $N_t=2, 3$. For a more general case in practice, i.e., $N_t>3$, we resort to the moment generating function to obtain an approximate but closed form PDF under some mild conditions, which, as expected, is more Gaussian-like as $2N_r-N_t$ increases. The so-derived PDFs are able to provide key insights into the WLMMSE MIMO receiver in terms of the outage probability, the symbol error rate, and the diversity gain, all presented in closed form. In particular, its diversity gain and the gain improvement over the conventional LMMSE one are explicitly quantified as $N_r-(N_t-1)/2$ and $(N_t-1)/2$, respectively. Finally, Monte Carlo simulations support the analysis.
Technical Report: Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments
Venue: arXiv cs.RO 2021
Year: 2021

Complex manipulation tasks, such as rearrangement planning of numerous objects, are combinatorially hard problems. Existing algorithms either do not scale well or assume a great deal of prior knowledge about the environment, and few offer any rigorous guarantees. In this paper, we propose a novel hybrid control architecture for achieving such tasks with mobile manipulators. On the discrete side, we enrich a temporal logic specification with mobile manipulation primitives such as moving to a point, and grasping or moving an object. Such specifications are translated to an automaton representation, which orchestrates the physical grounding of the task to mobility or manipulation controllers. The grounding from the discrete to the continuous reactive controller is online and can respond to the discovery of unknown obstacles or decide to push out of the way movable objects that prohibit task accomplishment. Despite the problem complexity, we prove that, under specific conditions, our architecture enjoys provable completeness on the discrete side, provable termination on the continuous side, and avoids all obstacles in the environment. Simulations illustrate the efficiency of our architecture that can handle tasks of increased complexity while also responding to unknown obstacles or unanticipated adverse configurations.
10 Years Later: Cloud Computing is Closing the Performance Gap
Venue: arXiv cs.DC 2021
Year: 2021

Can cloud computing infrastructures provide HPC-competitive performance for scientific applications broadly? Despite prolific related literature, this question remains open. Answers are crucial for designing future systems and democratizing high-performance computing. We present a multi-level approach to investigate the performance gap between HPC and cloud computing, isolating different variables that contribute to this gap. Our experiments are divided into (i) hardware and system microbenchmarks and (ii) user application proxies. The results show that today's high-end cloud computing can deliver HPC-competitive performance not only for computationally intensive applications but also for memory- and communication-intensive applications - at least at modest scales - thanks to the high-speed memory systems and interconnects and dedicated batch scheduling now available on some cloud platforms.