OpenRaiser/Intern-Atlas huggingface dataset:
Unsupervised Learning of Local Discriminative Representation for Medical Images
Venue: arXiv cs.CV 2021
Year: 2021

Local discriminative representation is needed in many medical image analysis tasks such as identifying sub-types of lesion or segmenting detailed components of anatomical structures. However, the commonly applied supervised representation learning methods require a large amount of annotated data, and unsupervised discriminative representation learning distinguishes different images by learning a global feature, both of which are not suitable for localized medical image analysis tasks. In order to avoid the limitations of these two methods, we introduce local discrimination into unsupervised representation learning in this work. The model contains two branches: one is an embedding branch which learns an embedding function to disperse dissimilar pixels over a low-dimensional hypersphere; and the other is a clustering branch which learns a clustering function to classify similar pixels into the same cluster. These two branches are trained simultaneously in a mutually beneficial pattern, and the learnt local discriminative representations are able to well measure the similarity of local image regions. These representations can be transferred to enhance various downstream tasks. Meanwhile, they can also be applied to cluster anatomical structures from unlabeled medical images under the guidance of topological priors from simulation or other structures with similar topological characteristics. The effectiveness and usefulness of the proposed method are demonstrated by enhancing various downstream tasks and clustering anatomical structures in retinal images and chest X-ray images.
Age-optimal Scheduling over Hybrid Channels
Venue: arXiv cs.IT 2021
Year: 2021

We consider the problem of minimizing the age of information when a source can transmit status updates over two heterogeneous channels. Our work is motivated by recent developments in 5G mmWave technology, where transmissions may occur over an unreliable but fast (e.g., mmWave) channel or a slow reliable (e.g., sub-6GHz) channel. The unreliable channel is modeled as a time-correlated Gilbert-Elliot channel at a high rate when the channel is in the 'ON' state. The reliable channel provides a deterministic but lower data rate. The scheduling strategy determines the channel to be used for transmission in each time slot, aiming to minimize the time-average age of information (AoI). The optimal scheduling problem is formulated as a Markov Decision Process (MDP), which is challenging to solve because super-modularity does not hold in a part of the state space. We address this challenge and show that a multi-dimensional threshold-type scheduling policy is optimal for minimizing the age. By exploiting the structure of the MDP and analyzing the discrete-time Markov chains (DTMCs) of the threshold-type policy, we devise a low-complexity bisection algorithm to compute the optimal thresholds. We compare different scheduling policies using numerical simulations.
Causality-Aware Neighborhood Methods for Recommender Systems
Venue: arXiv cs.IR 2021
Year: 2021

The business objectives of recommenders, such as increasing sales, are aligned with the causal effect of recommendations. Previous recommenders targeting for the causal effect employ the inverse propensity scoring (IPS) in causal inference. However, IPS is prone to suffer from high variance. The matching estimator is another representative method in causal inference field. It does not use propensity and hence free from the above variance problem. In this work, we unify traditional neighborhood recommendation methods with the matching estimator, and develop robust ranking methods for the causal effect of recommendations. Our experiments demonstrate that the proposed methods outperform various baselines in ranking metrics for the causal effect. The results suggest that the proposed methods can achieve more sales and user engagement than previous recommenders.
From Weakly Supervised Learning to Biquality Learning: an Introduction
Venue: arXiv cs.LG 2021
Year: 2021

The field of Weakly Supervised Learning (WSL) has recently seen a surge of popularity, with numerous papers addressing different types of "supervision deficiencies". In WSL use cases, a variety of situations exists where the collected "information" is imperfect. The paradigm of WSL attempts to list and cover these problems with associated solutions. In this paper, we review the research progress on WSL with the aim to make it as a brief introduction to this field. We present the three axis of WSL cube and an overview of most of all the elements of their facets. We propose three measurable quantities that acts as coordinates in the previously defined cube namely: Quality, Adaptability and Quantity of information. Thus we suggest that Biquality Learning framework can be defined as a plan of the WSL cube and propose to re-discover previously unrelated patches in WSL literature as a unified Biquality Learning literature.
$\mathbb{X}$Resolution Correspondence Networks
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we aim at establishing accurate dense correspondences between a pair of images with overlapping field of view under challenging illumination variation, viewpoint changes, and style differences. Through an extensive ablation study of the state-of-the-art correspondence networks, we surprisingly discovered that the widely adopted 4D correlation tensor and its related learning and processing modules could be de-parameterised and removed from training with merely a minor impact over the final matching accuracy. Disabling these computational expensive modules dramatically speeds up the training procedure and allows to use 4 times bigger batch size, which in turn compensates for the accuracy drop. Together with a multi-GPU inference stage, our method facilitates the systematic investigation of the relationship between matching accuracy and up-sampling resolution of the native testing images from 1280 to 4K. This leads to discovery of the existence of an optimal resolution $\mathbb{X}$ that produces accurate matching performance surpassing the state-of-the-art methods particularly over the lower error band on public benchmarks for the proposed network.
Addressing Feature Suppression in Unsupervised Visual Representations
Venue: arXiv cs.LG 2021
Year: 2021

Contrastive learning is one of the fastest growing research areas in machine learning due to its ability to learn useful representations without labeled data. However, contrastive learning is susceptible to feature suppression, i.e., it may discard important information relevant to the task of interest, and learn irrelevant features. Past work has addressed this limitation via handcrafted data augmentations that eliminate irrelevant information. This approach however does not work across all datasets and tasks. Further, data augmentations fail in addressing feature suppression in multi-attribute classification when one attribute can suppress features relevant to other attributes. In this paper, we analyze the objective function of contrastive learning and formally prove that it is vulnerable to feature suppression. We then present predictive contrastive learning (PCL), a framework for learning unsupervised representations that are robust to feature suppression. The key idea is to force the learned representation to predict the input, and hence prevent it from discarding important information. Extensive experiments verify that PCL is robust to feature suppression and outperforms state-of-the-art contrastive learning methods on a variety of datasets and tasks.
Deep embedded clustering of coral reef bioacoustics
Venue: arXiv cs.LG 2021
Year: 2021

Deep clustering was applied to unlabeled, automatically detected signals in a coral reef soundscape to distinguish fish pulse calls from segments of whale song. Deep embedded clustering (DEC) learned latent features and formed classification clusters using fixed-length power spectrograms of the signals. Handpicked spectral and temporal features were also extracted and clustered with Gaussian mixture models (GMM) and conventional clustering. DEC, GMM, and conventional clustering were tested on simulated datasets of fish pulse calls (fish) and whale song units (whale) with randomized bandwidth, duration, and SNR. Both GMM and DEC achieved high accuracy and identified clusters with fish, whale, and overlapping fish and whale signals. Conventional clustering methods had low accuracy in scenarios with unequal-sized clusters or overlapping signals. Fish and whale signals recorded near Hawaii in February-March 2020 were clustered with DEC, GMM, and conventional clustering. DEC features demonstrated the highest accuracy of 77.5% on a small, manually labeled dataset for classifying signals into fish and whale clusters.
Data Leverage: A Framework for Empowering the Public in its Relationship with Technology Companies
Venue: arXiv cs.CY 2021
Year: 2021

Many powerful computing technologies rely on implicit and explicit data contributions from the public. This dependency suggests a potential source of leverage for the public in its relationship with technology companies: by reducing, stopping, redirecting, or otherwise manipulating data contributions, the public can reduce the effectiveness of many lucrative technologies. In this paper, we synthesize emerging research that seeks to better understand and help people action this \textit{data leverage}. Drawing on prior work in areas including machine learning, human-computer interaction, and fairness and accountability in computing, we present a framework for understanding data leverage that highlights new opportunities to change technology company behavior related to privacy, economic inequality, content moderation and other areas of societal concern. Our framework also points towards ways that policymakers can bolster data leverage as a means of changing the balance of power between the public and tech companies.
The Parameterized Suffix Tray
Venue: arXiv cs.DS 2021
Year: 2021

Let $Σ$ and $Π$ be disjoint alphabets, respectively called the static alphabet and the parameterized alphabet. Two strings $x$ and $y$ over $Σ\cup Π$ of equal length are said to parameterized match (p-match) if there exists a renaming bijection $f$ on $Σ$ and $Π$ which is identity on $Σ$ and maps the characters of $x$ to those of $y$ so that the two strings become identical. The indexing version of the problem of finding p-matching occurrences of a given pattern in the text is a well-studied topic in string matching. In this paper, we present a state-of-the-art indexing structure for p-matching called the parameterized suffix tray of an input text $T$, denoted by $\mathsf{PSTray}(T)$. We show that $\mathsf{PSTray}(T)$ occupies $O(n)$ space and supports pattern matching queries in $O(m + \log (σ+π) + \mathit{occ})$ time, where $n$ is the length of $T$, $m$ is the length of a query pattern $P$, $π$ is the number of distinct symbols of $|Π|$ in $T$, $σ$ is the number of distinct symbols of $|Σ|$ in $T$ and $\mathit{occ}$ is the number of p-matching occurrences of $P$ in $T$. We also present how to build $\mathsf{PSTray}(T)$ in $O(n)$ time from the parameterized suffix tree of $T$.
Weakly-supervised Semantic Segmentation in Cityscape via Hyperspectral Image
Venue: arXiv cs.CV 2021
Year: 2021

High-resolution hyperspectral images (HSIs) contain the response of each pixel in different spectral bands, which can be used to effectively distinguish various objects in complex scenes. While HSI cameras have become low cost, algorithms based on it have not been well exploited. In this paper, we focus on a novel topic, weakly-supervised semantic segmentation in cityscape via HSIs. It is based on the idea that high-resolution HSIs in city scenes contain rich spectral information, which can be easily associated to semantics without manual labeling. Therefore, it enables low cost, highly reliable semantic segmentation in complex scenes. Specifically, in this paper, we theoretically analyze the HSIs and introduce a weakly-supervised HSI semantic segmentation framework, which utilizes spectral information to improve the coarse labels to a finer degree. The experimental results show that our method can obtain highly competitive labels and even have higher edge fineness than artificial fine labels in some classes. At the same time, the results also show that the refined labels can effectively improve the effect of semantic segmentation. The combination of HSIs and semantic segmentation proves that HSIs have great potential in high-level visual tasks.
Achieving State Machine Replication without Honest Players
Venue: arXiv cs.GT 2021
Year: 2021

Existing standards for player characterisation in tokenised state machine replication protocols depend on honest players who will always follow the protocol, regardless of possible token increases for deviating. Given the ever-increasing market capitalisation of these tokenised protocols, honesty is becoming more expensive and more unrealistic. As such, this out-dated player characterisation must be removed to provide true guarantees of safety and liveness in a major stride towards universal trust in state machine replication protocols and a new scale of adoption. As all current state machine replication protocols are built on these legacy standards, it is imperative that a new player model is identified and utilised to reflect the true nature of players in tokenised protocols, now and into the future. To this effect, we propose the ByRa player model for state machine replication protocols. In the ByRa model, players either attempt to maximise their tokenised rewards, or behave adversarially. This merges the fields of game theory and distributed systems, an intersection in which tokenised state machine replication protocols exist, but on which little formalisation has been carried out. In the ByRa model, we identify the properties of strong incentive compatibility in expectation and fairness that all protocols must satisfy in order to achieve state machine replication. We then provide Tenderstake, a protocol which provably satisfies these properties, and by doing so, achieves state machine replication in the ByRa model.
Scaling Laws for Gaussian Random Many-Access Channels
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers a Gaussian multiple-access channel with random user activity where the total number of users $\ell_n$ and the average number of active users $k_n$ may grow with the blocklength $n$. For this channel, it studies the maximum number of bits that can be transmitted reliably per unit-energy as a function of $\ell_n$ and $k_n$. When all users are active with probability one, i.e., $\ell_n = k_n$, it is demonstrated that if $k_n$ is of an order strictly below $n/\log n$, then each user can achieve the single-user capacity per unit-energy $(\log e)/N_0$ (where $N_0/ 2$ is the noise power) by using an orthogonal-access scheme. In contrast, if $k_n$ is of an order strictly above $n/\log n$, then the capacity per unit-energy is zero. Consequently, there is a sharp transition between orders of growth where interference-free communication is feasible and orders of growth where reliable communication at a positive rate per unit-energy is infeasible. It is further demonstrated that orthogonal-access schemes in combination with orthogonal codebooks, which achieve the capacity per unit-energy when the number of users is bounded, can be strictly suboptimal. When the user activity is random, i.e., when $\ell_n$ and $k_n$ are different, it is demonstrated that if $k_n\log \ell_n$ is sublinear in $n$, then each user can achieve the single-user capacity per unit-energy $(\log e)/N_0$. Conversely, if $k_n\log \ell_n$ is superlinear in $n$, then the capacity per unit-energy is zero. Consequently, there is again a sharp transition between orders of growth where interference-free communication is feasible and orders of growth where reliable communication at a positive rate is infeasible that depends on the asymptotic behaviours of both $\ell_n$ and $k_n$. It is further demonstrated that orthogonal-access schemes, which are optimal when $\ell_n = k_n$, can be strictly suboptimal.
A closed form scale bound for the $(ε, δ)$-differentially private Gaussian Mechanism valid for all privacy regimes
Venue: arXiv cs.CR 2021
Year: 2021

The standard closed form lower bound on $σ$ for providing $(ε, δ)$-differential privacy by adding zero mean Gaussian noise with variance $σ^2$ is $σ> Δ\sqrt {2}(ε^{-1}) \sqrt {\log \left( 5/4δ^{-1} \right)}$ for $ε\in (0,1)$. We present a similar closed form bound $σ\geq Δ(ε\sqrt{2})^{-1} \left(\sqrt{az+ε} + s\sqrt{az}\right)$ for $z=-\log(4δ(1-δ))$ and $(a,s)=(1,1)$ if $δ\leq 1/2$ and $(a,s)=(π/4,-1)$ otherwise. Our bound is valid for all $ε> 0$ and is always lower (better). We also present a sufficient condition for $(ε, δ)$-differential privacy when adding noise distributed according to even and log-concave densities supported everywhere.
SpaceML: Distributed Open-source Research with Citizen Scientists for the Advancement of Space Technology for NASA
Venue: arXiv cs.CV 2021
Year: 2021

Traditionally, academic labs conduct open-ended research with the primary focus on discoveries with long-term value, rather than direct products that can be deployed in the real world. On the other hand, research in the industry is driven by its expected commercial return on investment, and hence focuses on a real world product with short-term timelines. In both cases, opportunity is selective, often available to researchers with advanced educational backgrounds. Research often happens behind closed doors and may be kept confidential until either its publication or product release, exacerbating the problem of AI reproducibility and slowing down future research by others in the field. As many research organizations tend to exclusively focus on specific areas, opportunities for interdisciplinary research reduce. Undertaking long-term bold research in unexplored fields with non-commercial yet great public value is hard due to factors including the high upfront risk, budgetary constraints, and a lack of availability of data and experts in niche fields. Only a few companies or well-funded research labs can afford to do such long-term research. With research organizations focused on an exploding array of fields and resources spread thin, opportunities for the maturation of interdisciplinary research reduce. Apart from these exigencies, there is also a need to engage citizen scientists through open-source contributors to play an active part in the research dialogue. We present a short case study of SpaceML, an extension of the Frontier Development Lab, an AI accelerator for NASA. SpaceML distributes open-source research and invites volunteer citizen scientists to partake in development and deployment of high social value products at the intersection of space and AI.
Configuring Test Generators using Bug Reports: A Case Study of GCC Compiler and Csmith
Venue: arXiv cs.SE 2021
Year: 2021

The correctness of compilers is instrumental in the safety and reliability of other software systems, as bugs in compilers can produce executables that do not reflect the intent of programmers. Such errors are difficult to identify and debug. Random test program generators are commonly used in testing compilers, and they have been effective in uncovering bugs. However, the problem of guiding these test generators to produce test programs that are more likely to find bugs remains challenging. In this paper, we use the code snippets in the bug reports to guide the test generation. The main idea of this work is to extract insights from the bug reports about the language features that are more prone to inadequate implementation and using the insights to guide the test generators. We use the GCC C compiler to evaluate the effectiveness of this approach. In particular, we first cluster the test programs in the GCC bugs reports based on their features. We then use the centroids of the clusters to compute configurations for Csmith, a popular test generator for C compilers. We evaluated this approach on eight versions of GCC and found that our approach provides higher coverage and triggers more miscompilation failures than the state-of-the-art test generation techniques for GCC.
GrADyS: Exploring movement awareness for efficient routing in Ground-and-Air Dynamic Sensor Networks
Venue: arXiv cs.NI 2021
Year: 2021

Several situations exist where a geographic region of some size needs to be scanned or monitored through many sensors. Still, it is either absolutely impossible or prohibitively expensive to deploy and maintain wireless communication infrastructure for the distributed sensors. Either because the region is hidden behind walls, not easily accessible, hard to get through, or infected with some lethal bacteria or virus transmitter. In this case, the best is to scatter (disposable) sensors in the region and let them transmit the collected sensor data by wireless means to an overflying UAV/drone. Which then physically hauls the collected data from the monitored area to a central base station that functions as a gateway to the Internet. The project GrADyS aims to research two sets of problems regarding such data collection. The former aims to coordinate several autonomous UAVs in a distributed manner to collect the generated data while relying only on ad-hoc communication. The latter aims to develop routing protocols to mesh networks Bluetooth Mesh's Low Power Nodes. Both research lines already present preliminary results that are presented in this paper.
Image-based Intraluminal Contact Force Monitoring in Robotic Vascular Navigation
Venue: arXiv cs.RO 2021
Year: 2021

Embolization, stroke, ischaemic lesion, and perforation remain significant concerns in endovascular interventions. Intravascular sensing of tool interaction with the arteries is advantageous to minimize such complications and enhance navigation safety. Intraluminal information is currently limited due to the lack of intravascular contact sensing technologies. We present monitoring of the intraluminal tool interaction with the arterial wall using an image-based estimation approach within vascular robotic navigation. The proposed image-based method employs continuous finite element simulation of the tool using imaging data to estimate multi-point forces along tool-vessel wall interaction. We implemented imaging algorithms to detect and track contacts, and compute pose measurements. The model is constructed based on the nonlinear beam element and flexural rigidity profile over the tool length. During remote cannulation of aortic arteries, intraluminal monitoring achieved tracking local contact forces, building a contour map of force on the arterial wall and estimating tool structural stress. Results suggest that high risk intraluminal forces may happen even with low insertion force. The presented online monitoring system delivers insight into the intraluminal behavior of endovascular tools and is well suited for intraoperative visual guidance for the clinician, robotic control of vascular procedures and research on interventional device design.