OpenRaiser/Intern-Atlas huggingface dataset:
A Topological Approach for Motion Track Discrimination
Venue: arXiv cs.CV 2021
Year: 2021

Detecting small targets at range is difficult because there is not enough spatial information present in an image sub-region containing the target to use correlation-based methods to differentiate it from dynamic confusers present in the scene. Moreover, this lack of spatial information also disqualifies the use of most state-of-the-art deep learning image-based classifiers. Here, we use characteristics of target tracks extracted from video sequences as data from which to derive distinguishing topological features that help robustly differentiate targets of interest from confusers. In particular, we calculate persistent homology from time-delayed embeddings of dynamic statistics calculated from motion tracks extracted from a wide field-of-view video stream. In short, we use topological methods to extract features related to target motion dynamics that are useful for classification and disambiguation and show that small targets can be detected at range with high probability.
Massive MIMO under Double Scattering Channels: Power Minimization and Congestion Controls
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers a massive MIMO system under the double scattering channels. We derive a closed-form expression of the uplink ergodic spectral efficiency (SE) by exploiting the maximum-ratio combining technique with imperfect channel state information. We then formulate and solve a total uplink data power optimization problem that aims at simultaneously satisfying the required SEs from all the users with limited power resources. We further propose algorithms to cope with the congestion issue appearing when at least one user is served by lower SE than requested. Numerical results illustrate the effectiveness of our proposed power optimization. More importantly, our proposed congestion-handling algorithms can guarantee the required SEs to many users under congestion, even when the SE requirement is high.
SQLRepair: Identifying and Repairing Mistakes in Student-Authored SQL Queries
Venue: arXiv cs.SE 2021
Year: 2021

Computer science educators seek to understand the types of mistakes that students make when learning a new (programming) language so that they can help students avoid those mistakes in the future. While educators know what mistakes students regularly make in languages such as C and Python, students struggle with SQL and regularly make mistakes when working with it. We present an analysis of mistakes that students made when first working with SQL, classify the types of errors introduced, and provide suggestions on how to avoid them going forward. In addition, we present an automated tool, SQLRepair, that is capable of repairing errors introduced by undergraduate programmers when writing SQL queries. Our results show that students find repairs produced by our tool comparable in understandability to queries written by themselves or by other students, suggesting that SQL repair tools may be useful in an educational context. We also provide to the community a benchmark of SQL queries written by the students in our study that we used for evaluation of SQLRepair.
Self-supervised learning for fast and scalable time series hyper-parameter tuning
Venue: arXiv cs.LG 2021
Year: 2021

Hyper-parameters of time series models play an important role in time series analysis. Slight differences in hyper-parameters might lead to very different forecast results for a given model, and therefore, selecting good hyper-parameter values is indispensable. Most of the existing generic hyper-parameter tuning methods, such as Grid Search, Random Search, Bayesian Optimal Search, are based on one key component - search, and thus they are computationally expensive and cannot be applied to fast and scalable time-series hyper-parameter tuning (HPT). We propose a self-supervised learning framework for HPT (SSL-HPT), which uses time series features as inputs and produces optimal hyper-parameters. SSL-HPT algorithm is 6-20x faster at getting hyper-parameters compared to other search based algorithms while producing comparable accurate forecasting results in various applications.
Predicting Customer Lifetime Values -- ecommerce use case
Venue: arXiv cs.LG 2021
Year: 2021

Predicting customer future purchases and lifetime value is a key metrics for managing marketing campaigns and optimizing marketing spend. This task is specifically challenging when the relationships between the customer and the firm are of a noncontractual nature and therefore the future purchases need to be predicted based mostly on historical purchases. This work compares two approaches to predict customer future purchases, first using a 'buy-till-you-die' statistical model to predict customer behavior and later using a neural network on the same dataset and comparing the results. This comparison will lead to both quantitative and qualitative analysis of those two methods as well as recommendation on how to proceed in different cases and opportunities for future research.
On the Performance of the Primary and Secondary Links in a 3-D Underlay Cognitive Molecular Communication
Venue: arXiv cs.IT 2021
Year: 2021

Molecular communication often involves coexisting links where certain links may have priority over others. In this work, we consider a system in three-dimensional (3-D) space with two coexisting communication links, each between a point transmitter and fully-absorbing spherical receiver (FAR), where the one link (termed primary) has priority over the second link (termed secondary). The system implements the underlay cognitive-communication strategy for the co-existence of both links, which use the same type of molecules for information transfer. Mutual influence of FARs existing in the same communication medium results in competition for capturing the information-carrying molecules. In this work, first, we derive an approximate hitting probability equation for a diffusion-limited molecular communication system with two spherical FARs of different sizes considering the effect of molecular degradation. The derived equation is then used for the performance analysis of primary and secondary links in a cognitive molecular communication scenario. We show that the simple transmit control strategy at the secondary transmitter can improve the performance of the overall system. We study the influence of molecular degradation and decision threshold on the system performance. We also show that the systems parameters need to be carefully set to improve the performance of the system.
Causal Inference for Time series Analysis: Problems, Methods and Evaluation
Venue: arXiv cs.LG 2021
Year: 2021

Time series data is a collection of chronological observations which is generated by several domains such as medical and financial fields. Over the years, different tasks such as classification, forecasting, and clustering have been proposed to analyze this type of data. Time series data has been also used to study the effect of interventions over time. Moreover, in many fields of science, learning the causal structure of dynamic systems and time series data is considered an interesting task which plays an important role in scientific discoveries. Estimating the effect of an intervention and identifying the causal relations from the data can be performed via causal inference. Existing surveys on time series discuss traditional tasks such as classification and forecasting or explain the details of the approaches proposed to solve a specific task. In this paper, we focus on two causal inference tasks, i.e., treatment effect estimation and causal discovery for time series data, and provide a comprehensive review of the approaches in each task. Furthermore, we curate a list of commonly used evaluation metrics and datasets for each task and provide in-depth insight. These metrics and datasets can serve as benchmarks for research in the field.
Predicting Clinical Deterioration in Hospitals
Venue: arXiv cs.LG 2021
Year: 2021

Responding rapidly to a patient who is demonstrating signs of imminent clinical deterioration is a basic tenet of patient care. This gave rise to a patient safety intervention philosophy known as a Rapid Response System (RRS), whereby a patient who meets a pre-determined set of criteria for imminent clinical deterioration is immediately assessed and treated, with the goal of mitigating the deterioration and preventing intensive care unit (ICU) transfer, cardiac arrest, or death. While RRSs have been widely adopted, multiple systematic reviews have failed to find evidence of their effectiveness. Typically, RRS criteria are simple, expert (consensus) defined rules that identify significant physiologic abnormalities or are based on clinical observation. If one can find a pattern in the patient's data earlier than the onset of the physiologic derangement manifest in the current criteria, intervention strategies might be more effective. In this paper, we apply machine learning to electronic medical records (EMR) to infer if patients are at risk for clinical deterioration. Our models are more sensitive and offer greater advance prediction time compared with existing rule-based methods that are currently utilized in hospitals. Our results warrant further testing in the field; if successful, hospitals can integrate our approach into their existing IT systems and use the alerts generated by the model to prevent ICU transfer, cardiac arrest, or death, or to reduce the ICU length of stay.
CASA-Based Speaker Identification Using Cascaded GMM-CNN Classifier in Noisy and Emotional Talking Conditions
Venue: arXiv cs.SD 2021
Year: 2021

This work aims at intensifying text-independent speaker identification performance in real application situations such as noisy and emotional talking conditions. This is achieved by incorporating two different modules: a Computational Auditory Scene Analysis CASA based pre-processing module for noise reduction and cascaded Gaussian Mixture Model Convolutional Neural Network GMM-CNN classifier for speaker identification followed by emotion recognition. This research proposes and evaluates a novel algorithm to improve the accuracy of speaker identification in emotional and highly-noise susceptible conditions. Experiments demonstrate that the proposed model yields promising results in comparison with other classifiers when Speech Under Simulated and Actual Stress SUSAS database, Emirati Speech Database ESD, the Ryerson Audio-Visual Database of Emotional Speech and Song RAVDESS database and the Fluent Speech Commands database are used in a noisy environment.
Zero-one laws for provability logic: Axiomatizing validity in almost all models and almost all frames
Venue: arXiv cs.LO 2021
Year: 2021

It has been shown in the late 1960s that each formula of first-order logic without constants and function symbols obeys a zero-one law: As the number of elements of finite models increases, every formula holds either in almost all or in almost no models of that size. Therefore, many properties of models, such as having an even number of elements, cannot be expressed in the language of first-order logic. For modal logics, limit behavior for models and frames may differ. Halpern and Kapron proved zero-one laws for classes of models corresponding to the modal logics K, T, S4, and S5. In this paper, we prove zero-one laws for provability logic with respect to both model and frame validity. Moreover, we axiomatize validity in almost all relevant finite models and in almost all relevant finite frames, leading to two different axiom systems. In the proofs, we use a combinatorial result by Kleitman and Rothschild about the structure of almost all finite partial orders. On the way, we also show that a previous result by Halpern and Kapron about the axiomatization of almost sure frame validity for S4 is not correct. Finally, we consider the complexity of deciding whether a given formula is almost surely valid in the relevant finite models and frames.
The Benefit of the Doubt: Uncertainty Aware Sensing for Edge Computing Platforms
Venue: arXiv cs.LG 2021
Year: 2021

Neural networks (NNs) lack measures of "reliability" estimation that would enable reasoning over their predictions. Despite the vital importance, especially in areas of human well-being and health, state-of-the-art uncertainty estimation techniques are computationally expensive when applied to resource-constrained devices. We propose an efficient framework for predictive uncertainty estimation in NNs deployed on embedded edge systems with no need for fine-tuning or re-training strategies. To meet the energy and latency requirements of these embedded platforms the framework is built from the ground up to provide predictive uncertainty based only on one forward pass and a negligible amount of additional matrix multiplications with theoretically proven correctness. Our aim is to enable already trained deep learning models to generate uncertainty estimates on resource-limited devices at inference time focusing on classification tasks. This framework is founded on theoretical developments casting dropout training as approximate inference in Bayesian NNs. Our layerwise distribution approximation to the convolution layer cascades through the network, providing uncertainty estimates in one single run which ensures minimal overhead, especially compared with uncertainty techniques that require multiple forwards passes and an equal linear rise in energy and latency requirements making them unsuitable in practice. We demonstrate that it yields better performance and flexibility over previous work based on multilayer perceptrons to obtain uncertainty estimates. Our evaluation with mobile applications datasets shows that our approach not only obtains robust and accurate uncertainty estimations but also outperforms state-of-the-art methods in terms of systems performance, reducing energy consumption (up to 28x), keeping the memory overhead at a minimum while still improving accuracy (up to 16%).
EventScore: An Automated Real-time Early Warning Score for Clinical Events
Venue: arXiv cs.LG 2021
Year: 2021

Early prediction of patients at risk of clinical deterioration can help physicians intervene and alter their clinical course towards better outcomes. In addition to the accuracy requirement, early warning systems must make the predictions early enough to give physicians enough time to intervene. Interpretability is also one of the challenges when building such systems since being able to justify the reasoning behind model decisions is desirable in clinical practice. In this work, we built an interpretable model for the early prediction of various adverse clinical events indicative of clinical deterioration. The model is evaluated on two datasets and four clinical events. The first dataset is collected in a predominantly COVID-19 positive population at Stony Brook Hospital. The second dataset is the MIMIC III dataset. The model was trained to provide early warning scores for ventilation, ICU transfer, and mortality prediction tasks on the Stony Brook Hospital dataset and to predict mortality and the need for vasopressors on the MIMIC III dataset. Our model first separates each feature into multiple ranges and then uses logistic regression with lasso penalization to select the subset of ranges for each feature. The model training is completely automated and doesn't require expert knowledge like other early warning scores. We compare our model to the Modified Early Warning Score (MEWS) and quick SOFA (qSOFA), commonly used in hospitals. We show that our model outperforms these models in the area under the receiver operating characteristic curve (AUROC) while having a similar or better median detection time on all clinical events, even when using fewer features. Unlike MEWS and qSOFA, our model can be entirely automated without requiring any manually recorded features. We also show that discretization improves model performance by comparing our model to a baseline logistic regression model.
Neural BRDF Representation and Importance Sampling
Venue: arXiv cs.GR 2021
Year: 2021

Controlled capture of real-world material appearance yields tabulated sets of highly realistic reflectance data. In practice, however, its high memory footprint requires compressing into a representation that can be used efficiently in rendering while remaining faithful to the original. Previous works in appearance encoding often prioritised one of these requirements at the expense of the other, by either applying high-fidelity array compression strategies not suited for efficient queries during rendering, or by fitting a compact analytic model that lacks expressiveness. We present a compact neural network-based representation of BRDF data that combines high-accuracy reconstruction with efficient practical rendering via built-in interpolation of reflectance. We encode BRDFs as lightweight networks, and propose a training scheme with adaptive angular sampling, critical for the accurate reconstruction of specular highlights. Additionally, we propose a novel approach to make our representation amenable to importance sampling: rather than inverting the trained networks, we learn to encode them in a more compact embedding that can be mapped to parameters of an analytic BRDF for which importance sampling is known. We evaluate encoding results on isotropic and anisotropic BRDFs from multiple real-world datasets, and importance sampling performance for isotropic BRDFs mapped to two different analytic models.
Polynomial Approximations of Conditional Expectations in Scalar Gaussian Channels
Venue: arXiv cs.IT 2021
Year: 2021

We consider a channel $Y=X+N$ where $X$ is a random variable satisfying $\mathbb{E}[|X|]<\infty$ and $N$ is an independent standard normal random variable. We show that the minimum mean-square error estimator of $X$ from $Y,$ which is given by the conditional expectation $\mathbb{E}[X \mid Y],$ is a polynomial in $Y$ if and only if it is linear or constant; these two cases correspond to $X$ being Gaussian or a constant, respectively. We also prove that the higher-order derivatives of $y \mapsto \mathbb{E}[X \mid Y=y]$ are expressible as multivariate polynomials in the functions $y \mapsto \mathbb{E}\left[ \left( X - \mathbb{E}[X \mid Y] \right)^k \mid Y = y \right]$ for $k\in \mathbb{N}.$ These expressions yield bounds on the $2$-norm of the derivatives of the conditional expectation. These bounds imply that, if $X$ has a compactly-supported density that is even and decreasing on the positive half-line, then the error in approximating the conditional expectation $\mathbb{E}[X \mid Y]$ by polynomials in $Y$ of degree at most $n$ decays faster than any polynomial in $n.$
HyperPocket: Generative Point Cloud Completion
Venue: arXiv cs.CV 2021
Year: 2021

Scanning real-life scenes with modern registration devices typically give incomplete point cloud representations, mostly due to the limitations of the scanning process and 3D occlusions. Therefore, completing such partial representations remains a fundamental challenge of many computer vision applications. Most of the existing approaches aim to solve this problem by learning to reconstruct individual 3D objects in a synthetic setup of an uncluttered environment, which is far from a real-life scenario. In this work, we reformulate the problem of point cloud completion into an object hallucination task. Thus, we introduce a novel autoencoder-based architecture called HyperPocket that disentangles latent representations and, as a result, enables the generation of multiple variants of the completed 3D point clouds. We split point cloud processing into two disjoint data streams and leverage a hypernetwork paradigm to fill the spaces, dubbed pockets, that are left by the missing object parts. As a result, the generated point clouds are not only smooth but also plausible and geometrically consistent with the scene. Our method offers competitive performances to the other state-of-the-art models, and it enables a~plethora of novel applications.
An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning
Venue: arXiv cs.CL 2021
Year: 2021

We present a joint model for entity-level relation extraction from documents. In contrast to other approaches - which focus on local intra-sentence mention pairs and thus require annotations on mention level - our model operates on entity level. To do so, a multi-task approach is followed that builds upon coreference resolution and gathers relevant signals via multi-instance learning with multi-level representations combining global entity and local mention information. We achieve state-of-the-art relation extraction results on the DocRED dataset and report the first entity-level end-to-end relation extraction results for future reference. Finally, our experimental results suggest that a joint approach is on par with task-specific learning, though more efficient due to shared parameters and training steps.
Edge Deletion to Restrict the Size of an Epidemic
Venue: arXiv cs.DS 2021
Year: 2021

Given a graph $G=(V,E)$, a set $\mathcal{F}$ of forbidden subgraphs, we study $\mathcal{F}$-Free Edge Deletion, where the goal is to remove minimum number of edges such that the resulting graph does not contain any $F\in \mathcal{F}$ as a subgraph. For the parameter treewidth, the question of whether the problem is FPT has remained open. Here we give a negative answer by showing that the problem is W[1]-hard when parameterized by the treewidth, which rules out FPT algorithms under common assumption. Thus we give a solution to the conjecture posted by Jessica Enright and Kitty Meeks in [Algorithmica 80 (2018) 1857-1889]. We also prove that the $\mathcal{F}$-Free Edge Deletion problem is W[2]-hard when parameterized by the solution size $k$, feedback vertex set number or pathwidth of the input graph. A special case of particular interest is the situation in which $\mathcal{F}$ is the set $\mathcal{T}_{h+1}$ of all trees on $h+1$ vertices, so that we delete edges in order to obtain a graph in which every component contains at most $h$ vertices. This is desirable from the point of view of restricting the spread of disease in transmission network. We prove that the $\mathcal{T}_{h+1}$-Free Edge Deletion problem is fixed-parameter tractable (FPT) when parameterized by the vertex cover number. We also prove that it admits a kernel with $O(hk)$ vertices and $O(h^2k)$ edges, when parameterized by combined parameters $h$ and the solution size $k$.
SWAGAN: A Style-based Wavelet-driven Generative Model
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, considerable progress has been made in the visual quality of Generative Adversarial Networks (GANs). Even so, these networks still suffer from degradation in quality for high-frequency content, stemming from a spectrally biased architecture, and similarly unfavorable loss functions. To address this issue, we present a novel general-purpose Style and WAvelet based GAN (SWAGAN) that implements progressive generation in the frequency domain. SWAGAN incorporates wavelets throughout its generator and discriminator architectures, enforcing a frequency-aware latent representation at every step of the way. This approach yields enhancements in the visual quality of the generated images, and considerably increases computational performance. We demonstrate the advantage of our method by integrating it into the SyleGAN2 framework, and verifying that content generation in the wavelet domain leads to higher quality images with more realistic high-frequency content. Furthermore, we verify that our model's latent space retains the qualities that allow StyleGAN to serve as a basis for a multitude of editing tasks, and show that our frequency-aware approach also induces improved downstream visual quality.