OpenRaiser/Intern-Atlas huggingface dataset:
Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Data
Venue: arXiv cs.CV 2021
Year: 2021

Clinical decision-making in oncology involves multimodal data such as radiology scans, molecular profiling, histopathology slides, and clinical factors. Despite the importance of these modalities individually, no deep learning framework to date has combined them all to predict patient prognosis. Here, we predict the overall survival (OS) of glioma patients from diverse multimodal data with a Deep Orthogonal Fusion (DOF) model. The model learns to combine information from multiparametric MRI exams, biopsy-based modalities (such as H&E slide images and/or DNA sequencing), and clinical variables into a comprehensive multimodal risk score. Prognostic embeddings from each modality are learned and combined via attention-gated tensor fusion. To maximize the information gleaned from each modality, we introduce a multimodal orthogonalization (MMO) loss term that increases model performance by incentivizing constituent embeddings to be more complementary. DOF predicts OS in glioma patients with a median C-index of 0.788 +/- 0.067, significantly outperforming (p=0.023) the best performing unimodal model with a median C-index of 0.718 +/- 0.064. The prognostic model significantly stratifies glioma patients by OS within clinical subsets, adding further granularity to prognostic clinical grading and molecular subtyping.
Online Euclidean Spanners
Venue: arXiv cs.CG 2021
Year: 2021

In this paper, we study the online Euclidean spanners problem for points in $\mathbb{R}^d$. Suppose we are given a sequence of $n$ points $(s_1,s_2,\ldots, s_n)$ in $\mathbb{R}^d$, where point $s_i$ is presented in step~$i$ for $i=1,\ldots, n$. The objective of an online algorithm is to maintain a geometric $t$-spanner on $S_i=\{s_1,\ldots, s_i\}$ for each step~$i$. First, we establish a lower bound of $Ω(\varepsilon^{-1}\log n / \log \varepsilon^{-1})$ for the competitive ratio of any online $(1+\varepsilon)$-spanner algorithm, for a sequence of $n$ points in 1-dimension. We show that this bound is tight, and there is an online algorithm that can maintain a $(1+\varepsilon)$-spanner with competitive ratio $O(\varepsilon^{-1}\log n / \log \varepsilon^{-1})$. Next, we design online algorithms for sequences of points in $\mathbb{R}^d$, for any constant $d\ge 2$, under the $L_2$ norm. We show that previously known incremental algorithms achieve a competitive ratio $O(\varepsilon^{-(d+1)}\log n)$. However, if the algorithm is allowed to use additional points (Steiner points), then it is possible to substantially improve the competitive ratio in terms of $\varepsilon$. We describe an online Steiner $(1+\varepsilon)$-spanner algorithm with competitive ratio $O(\varepsilon^{(1-d)/2} \log n)$. As a counterpart, we show that the dependence on $n$ cannot be eliminated in dimensions $d \ge 2$. In particular, we prove that any online spanner algorithm for a sequence of $n$ points in $\mathbb{R}^d$ under the $L_2$ norm has competitive ratio $Ω(f(n))$, where $\lim_{n\rightarrow \infty}f(n)=\infty$. Finally, we provide improved lower bounds under the $L_1$ norm: $Ω(\varepsilon^{-2}/\log \varepsilon^{-1})$ in the plane and $Ω(\varepsilon^{-d})$ in $\mathbb{R}^d$ for $d\geq 3$.
Interactive decoding of words from visual speech recognition models
Venue: arXiv cs.CL 2021
Year: 2021

This work describes an interactive decoding method to improve the performance of visual speech recognition systems using user input to compensate for the inherent ambiguity of the task. Unlike most phoneme-to-word decoding pipelines, which produce phonemes and feed these through a finite state transducer, our method instead expands words in lockstep, facilitating the insertion of interaction points at each word position. Interaction points enable us to solicit input during decoding, allowing users to interactively direct the decoding process. We simulate the behavior of user input using an oracle to give an automated evaluation, and show promise for the use of this method for text input.
Blind Image Super-Resolution via Contrastive Representation Learning
Venue: arXiv cs.CV 2021
Year: 2021

Image super-resolution (SR) research has witnessed impressive progress thanks to the advance of convolutional neural networks (CNNs) in recent years. However, most existing SR methods are non-blind and assume that degradation has a single fixed and known distribution (e.g., bicubic) which struggle while handling degradation in real-world data that usually follows a multi-modal, spatially variant, and unknown distribution. The recent blind SR studies address this issue via degradation estimation, but they do not generalize well to multi-source degradation and cannot handle spatially variant degradation. We design CRL-SR, a contrastive representation learning network that focuses on blind SR of images with multi-modal and spatially variant distributions. CRL-SR addresses the blind SR challenges from two perspectives. The first is contrastive decoupling encoding which introduces contrastive learning to extract resolution-invariant embedding and discard resolution-variant embedding under the guidance of a bidirectional contrastive loss. The second is contrastive feature refinement which generates lost or corrupted high-frequency details under the guidance of a conditional contrastive loss. Extensive experiments on synthetic datasets and real images show that the proposed CRL-SR can handle multi-modal and spatially variant degradation effectively under blind settings and it also outperforms state-of-the-art SR methods qualitatively and quantitatively.
Formation of coalition structures as a non-cooperative game
Venue: arXiv cs.GT 2021
Year: 2021

We study coalition structure formation with intra and inter-coalition externalities in the introduced family of nested non-cooperative simultaneous finite games. A non-cooperative game embeds a coalition structure formation mechanism, and has two outcomes: an allocation of players over coalitions and a payoff for every player. Coalition structures of a game are described by Young diagrams. They serve to enumerate coalition structures and allocations of players over them. For every coalition structure a player has a set of finite strategies. A player chooses a coalition structure and a strategy. A (social) mechanism eliminates conflicts in individual choices and produces final coalition structures. Every final coalition structure is a non-cooperative game. Mixed equilibrium always exists and consists of a mixed strategy profile, payoffs and equilibrium coalition structures. We use a maximum coalition size to parametrize the family of the games. The non-cooperative game of Nash is a partial case of the model. The result is different from the Shapley value, a strong Nash, coalition-proof equilibria, core solutions, and other equilibrium concepts. We supply few non-cooperative coalition structure stability criteria.
Essence of Factual Knowledge
Venue: arXiv cs.DB 2021
Year: 2021

Knowledge bases are collections of domain-specific and commonsense facts. Recently, the sizes of KBs are rocketing due to automatic extraction for knowledge and facts. For example, the number of facts in WikiData is up to 974 million! According to our observation, current KBs, especially domain KBs, show strong relevance in relations according to some topics. These patterns can be used to conclude and infer for part of facts in the KBs. Therefore, the original KBs can be minimzed by extracting patterns and essential facts.
TMUML: A Singular TM Model with UML Use Cases and Classes
Venue: arXiv cs.SE 2021
Year: 2021

In the systems and software modeling field, a conceptual model involves modeling with concepts to support development and design. An example of a conceptual model is a description developed using the Unified Modeling Language (UML). UML uses a model multiplicity formulation approach, wherein a number of models are used to represent alternative views. By contrast, a model singularity approach uses only a single integrated model. Each of these styles of modeling has its strengths and weaknesses. This paper introduces a partial solution to the issue of multiplicity vs. singularity in modeling by adopting UML use cases and class models into the conceptual thinging machine (TM) model. To apply use cases, we adopt the observation that a use-case diagram is a description that shows the internal structure of the part of the system represented by the use case in addition to being useful to people outside of the system. Additionally, the UML class diagram is recast in TM representation. Accordingly, we develop a TMUML model that embraces the TM specification of the UML class diagram and the internal structure extracted from the UML use case. TMUML modeling introduces some of the advantages that have made UML a popular modeling language to TM modeling. At the same time, this approach supplies UML with partial model singularity. The paper details experimentation with TMUML using examples from the literature. Our results indicate that mixing UML with other models could be a viable approach.
Formal Semantics of a Classical-Quantum Language
Venue: arXiv cs.LO 2021
Year: 2021

We investigate the formal semantics of a simple imperative language that has both classical and quantum constructs. More specifically, we provide an operational semantics, a denotational semantics and two Hoare-style proof systems: an abstract one and a concrete one. The two proof systems are satisfaction-based, as inspired by the program logics of Barthe et al for probabilistic programs. The abstract proof system turns out to be sound and relatively complete, while the concrete one is sound only.
Few-shot Learning for Unsupervised Feature Selection
Venue: arXiv cs.LG 2021
Year: 2021

We propose a few-shot learning method for unsupervised feature selection, which is a task to select a subset of relevant features in unlabeled data. Existing methods usually require many instances for feature selection. However, sufficient instances are often unavailable in practice. The proposed method can select a subset of relevant features in a target task given a few unlabeled target instances by training with unlabeled instances in multiple source tasks. Our model consists of a feature selector and decoder. The feature selector outputs a subset of relevant features taking a few unlabeled instances as input such that the decoder can reconstruct the original features of unseen instances from the selected ones. The feature selector uses the Concrete random variables to select features via gradient descent. To encode task-specific properties from a few unlabeled instances to the model, the Concrete random variables and decoder are modeled using permutation-invariant neural networks that take a few unlabeled instances as input. Our model is trained by minimizing the expected test reconstruction error given a few unlabeled instances that is calculated with datasets in source tasks. We experimentally demonstrate that the proposed method outperforms existing feature selection methods.
A Levy Flight based Narrow Passage Sampling Method for Probabilistic Roadmap Planners
Venue: arXiv cs.RO 2021
Year: 2021

Sampling based probabilistic roadmap planners (PRM) have been successful in motion planning of robots with higher degrees of freedom, but may fail to capture the connectivity of the configuration space in scenarios with a critical narrow passage. In this paper, we show a novel technique based on Levy Flights to generate key samples in the narrow regions of configuration space, which, when combined with a PRM, improves the completeness of the planner. The technique substantially improves sample quality at the expense of a minimal additional computation, when compared with pure random walk based methods, however, still outperforms state of the art random bridge building method, in terms of number of collision calls, computational overhead and sample quality. The method is robust to the changes in the parameters related to the structure of the narrow passage, thus giving an additional generality. A number of 2D & 3D motion planning simulations are presented which shows the effectiveness of the method.
Privacy in Distributed Computations based on Real Number Secret Sharing
Venue: arXiv cs.CR 2021
Year: 2021

Privacy preservation in distributed computations is an important subject as digitization and new technologies enable collection and storage of vast amounts of data, including private data belonging to individuals. To this end, there is a need for a privacy preserving computation framework that minimises the leak of private information during computations while being efficient enough for practical usage. This paper presents a step towards such a framework with the proposal of a real number secret sharing scheme that works directly on real numbers without the need for conversion to integers which is the case in related schemes. The scheme offers computations like addition, multiplication, and division to be performed directly on secret shared data (the cipher text version of the data). Simulations show that the scheme is much more efficient in terms of accuracy than its counterpart version based on integers and finite field arithmetic. The drawback with the proposed scheme is that it is not perfectly secure. However, we provide a privacy analysis of the scheme, where we show that the leaked information can be upper bounded and asymptotically goes to zero. To demonstrate the scheme, we use it to perform Kalman filtering directly on secret shared data.
A Collective Anomaly Detection Method Over Bitcoin Network
Venue: arXiv cs.CR 2021
Year: 2021

The popularity and amazing attractiveness of cryptocurrencies, and especially Bitcoin, absorb countless enthusiasts daily. Although Blockchain technology prevents fraudulent behavior, it cannot detect fraud on its own. There are always unimaginable ways to commit fraud, and the need to use anomaly detection methods to identify abnormal and fraudulent behaviors has become a necessity. The main purpose of this study is to present a new method for detecting anomalies in Bitcoin with more appropriate efficiency. For this purpose, in this study, the diagnosis of the collective anomaly was used, and instead of diagnosing the anomaly of individual addresses and wallets, the anomaly of users was examined, and the anomaly was more visible among users who had multiple wallets. In addition to using the collective anomaly detection method in this study, the Trimmed_Kmeans algorithm was used for clustering and the proposed method succeeded in identifying 14 users who had committed theft, fraud, and hack with 26 addresses in 9 cases. Compared to previous works, which detected a maximum of 7 addresses in 5 cases of fraud, the proposed method has performed well. Therefore, the proposed method, by presenting a new approach, in addition to reducing the processing power to extract features, succeeded in detecting abnormal users and also was able to find more transactions and addresses committed a scam.
Misinformation Detection on YouTube Using Video Captions
Venue: arXiv cs.LG 2021
Year: 2021

Millions of people use platforms such as YouTube, Facebook, Twitter, and other mass media. Due to the accessibility of these platforms, they are often used to establish a narrative, conduct propaganda, and disseminate misinformation. This work proposes an approach that uses state-of-the-art NLP techniques to extract features from video captions (subtitles). To evaluate our approach, we utilize a publicly accessible and labeled dataset for classifying videos as misinformation or not. The motivation behind exploring video captions stems from our analysis of videos metadata. Attributes such as the number of views, likes, dislikes, and comments are ineffective as videos are hard to differentiate using this information. Using caption dataset, the proposed models can classify videos among three classes (Misinformation, Debunking Misinformation, and Neutral) with 0.85 to 0.90 F1-score. To emphasize the relevance of the misinformation class, we re-formulate our classification problem as a two-class classification - Misinformation vs. others (Debunking Misinformation and Neutral). In our experiments, the proposed models can classify videos with 0.92 to 0.95 F1-score and 0.78 to 0.90 AUC ROC.
Ultrasound Video Transformers for Cardiac Ejection Fraction Estimation
Venue: arXiv cs.CV 2021
Year: 2021

Cardiac ultrasound imaging is used to diagnose various heart diseases. Common analysis pipelines involve manual processing of the video frames by expert clinicians. This suffers from intra- and inter-observer variability. We propose a novel approach to ultrasound video analysis using a transformer architecture based on a Residual Auto-Encoder Network and a BERT model adapted for token classification. This enables videos of any length to be processed. We apply our model to the task of End-Systolic (ES) and End-Diastolic (ED) frame detection and the automated computation of the left ventricular ejection fraction. We achieve an average frame distance of 3.36 frames for the ES and 7.17 frames for the ED on videos of arbitrary length. Our end-to-end learnable approach can estimate the ejection fraction with a MAE of 5.95 and $R^2$ of 0.52 in 0.15s per video, showing that segmentation is not the only way to predict ejection fraction. Code and models are available at https://github.com/HReynaud/UVT.
DeformRS: Certifying Input Deformations with Randomized Smoothing
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks are vulnerable to input deformations in the form of vector fields of pixel displacements and to other parameterized geometric deformations e.g. translations, rotations, etc. Current input deformation certification methods either 1. do not scale to deep networks on large input datasets, or 2. can only certify a specific class of deformations, e.g. only rotations. We reformulate certification in randomized smoothing setting for both general vector field and parameterized deformations and propose DeformRS-VF and DeformRS-Par, respectively. Our new formulation scales to large networks on large input datasets. For instance, DeformRS-Par certifies rich deformations, covering translations, rotations, scaling, affine deformations, and other visually aligned deformations such as ones parameterized by Discrete-Cosine-Transform basis. Extensive experiments on MNIST, CIFAR10, and ImageNet show competitive performance of DeformRS-Par achieving a certified accuracy of $39\%$ against perturbed rotations in the set $[-10\degree,10\degree]$ on ImageNet.
Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning
Venue: arXiv cs.NI 2021
Year: 2021

This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users. To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users and enhance their immersive visual experiences, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited. Owing to user movement and time-varying wireless channels, the wireless VR experience enhancement problem is formulated as a sequence-dependent and mixed-integer problem with a goal of maximizing users' feeling of presence (FoP) in the virtual world, subject to power consumption constraints on access points (APs) and users' head-mounted displays (HMDs). The problem, however, is hard to be directly solved due to the lack of users' accurate tracking information and the sequence-dependent and mixed-integer characteristics. To overcome this challenge, we develop a parallel echo state network (ESN) learning method to predict users' tracking information by training fresh and historical tracking samples separately collected by APs. With the learnt results, we propose a deep reinforcement learning (DRL) based optimization algorithm to solve the formulated problem. In this algorithm, we implement deep neural networks (DNNs) as a scalable solution to produce integer decision variables and solving a continuous power control problem to criticize the integer decision variables. Finally, the performance of the proposed algorithm is compared with various benchmark algorithms, and the impact of different design parameters is also discussed. Simulation results demonstrate that the proposed algorithm is more 4.14% energy-efficient than the benchmark algorithms.
WiCluster: Passive Indoor 2D/3D Positioning using WiFi without Precise Labels
Venue: arXiv cs.NI 2021
Year: 2021

We introduce WiCluster, a new machine learning (ML) approach for passive indoor positioning using radio frequency (RF) channel state information (CSI). WiCluster can predict both a zone-level position and a precise 2D or 3D position, without using any precise position labels during training. Prior CSI-based indoor positioning work has relied on non-parametric approaches using digital signal-processing (DSP) and, more recently, parametric approaches (e.g., fully supervised ML methods). However these do not handle the complexity of real-world environments well and do not meet requirements for large-scale commercial deployments: the accuracy of DSP-based method deteriorates significantly in non-line-of-sight conditions, while supervised ML methods need large amounts of hard-to-acquire centimeter accuracy position labels. In contrast, WiCluster is precise, requires weaker label-information that can be easily collected, and works well in non-line-of-sight conditions. Our first contribution is a novel dimensionality reduction method for charting. It combines a triplet-loss with a multi-scale clustering-loss to map the high-dimensional CSI representation to a 2D/3D latent space. Our second contribution is two weakly supervised losses that map this latent space into a Cartesian map, resulting in meter-accuracy position results. These losses only require simple to acquire priors: a sketch of the floorplan, approximate access-point locations and a few CSI packets that are labelled with the corresponding zone in the floorplan. Thirdly, we report results and a robustness study for 2D positioning in two single-floor office buildings and 3D positioning in a two-story home.
4C: A Computation, Communication, and Control Co-Design Framework for CAVs
Venue: arXiv cs.DC 2021
Year: 2021

Connected and autonomous vehicles (CAVs) are promising due to their potential safety and efficiency benefits and have attracted massive investment and interest from government agencies, industry, and academia. With more computing and communication resources are available, both vehicles and edge servers are equipped with a set of camera-based vision sensors, also known as Visual IoT (V-IoT) techniques, for sensing and perception. Tremendous efforts have been made for achieving programmable communication, computation, and control. However, they are conducted mainly in the silo mode, limiting the responsiveness and efficiency of handling challenging scenarios in the real world. To improve the end-to-end performance, we envision that future CAVs require the co-design of communication, computation, and control. This paper presents our vision of the end-to-end design principle for CAVs, called 4C, which extends the V-IoT system by providing a unified communication, computation, and control co-design framework. With programmable communications, fine-grained heterogeneous computation, and efficient vehicle controls in 4C, CAVs can handle critical scenarios and achieve energy-efficient autonomous driving. Finally, we present several challenges to achieving the vision of the 4C framework.