OpenRaiser/Intern-Atlas huggingface dataset:
TransMed: Transformers Advance Multi-modal Medical Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

Over the past decade, convolutional neural networks (CNN) have shown very competitive performance in medical image analysis tasks, such as disease classification, tumor segmentation, and lesion detection. CNN has great advantages in extracting local features of images. However, due to the locality of convolution operation, it can not deal with long-range relationships well. Recently, transformers have been applied to computer vision and achieved remarkable success in large-scale datasets. Compared with natural images, multi-modal medical images have explicit and important long-range dependencies, and effective multi-modal fusion strategies can greatly improve the performance of deep models. This prompts us to study transformer-based structures and apply them to multi-modal medical images. Existing transformer-based network architectures require large-scale datasets to achieve better performance. However, medical imaging datasets are relatively small, which makes it difficult to apply pure transformers to medical image analysis. Therefore, we propose TransMed for multi-modal medical image classification. TransMed combines the advantages of CNN and transformer to efficiently extract low-level features of images and establish long-range dependencies between modalities. We evaluated our model for the challenging problem of preoperative diagnosis of parotid gland tumors, and the experimental results show the advantages of our proposed method. We argue that the combination of CNN and transformer has tremendous potential in a large number of medical image analysis tasks. To our best knowledge, this is the first work to apply transformers to medical image classification.
Delaunay triangulations of generalized Bolza surfaces
Venue: arXiv cs.CG 2021
Year: 2021

The Bolza surface can be seen as the quotient of the hyperbolic plane, represented by the Poincaré disk model, under the action of the group generated by the hyperbolic isometries identifying opposite sides of a regular octagon centered at the origin. We consider generalized Bolza surfaces $\mathbb{M}_g$, where the octagon is replaced by a regular $4g$-gon, leading to a genus $g$ surface. We propose an extension of Bowyer's algorithm to these surfaces. In particular, we compute the value of the systole of $\mathbb{M}_g$. We also propose algorithms computing small sets of points on $\mathbb{M}_g$ that are used to initialize Bowyer's algorithm.
Multi-Pretext Attention Network for Few-shot Learning with Self-supervision
Venue: arXiv cs.CV 2021
Year: 2021

Few-shot learning is an interesting and challenging study, which enables machines to learn from few samples like humans. Existing studies rarely exploit auxiliary information from large amount of unlabeled data. Self-supervised learning is emerged as an efficient method to utilize unlabeled data. Existing self-supervised learning methods always rely on the combination of geometric transformations for the single sample by augmentation, while seriously neglect the endogenous correlation information among different samples that is the same important for the task. In this work, we propose a Graph-driven Clustering (GC), a novel augmentation-free method for self-supervised learning, which does not rely on any auxiliary sample and utilizes the endogenous correlation information among input samples. Besides, we propose Multi-pretext Attention Network (MAN), which exploits a specific attention mechanism to combine the traditional augmentation-relied methods and our GC, adaptively learning their optimized weights to improve the performance and enabling the feature extractor to obtain more universal representations. We evaluate our MAN extensively on miniImageNet and tieredImageNet datasets and the results demonstrate that the proposed method outperforms the state-of-the-art (SOTA) relevant methods.
Quality-Aware Network for Human Parsing
Venue: arXiv cs.CV 2021
Year: 2021

How to estimate the quality of the network output is an important issue, and currently there is no effective solution in the field of human parsing. In order to solve this problem, this work proposes a statistical method based on the output probability map to calculate the pixel quality information, which is called pixel score. In addition, the Quality-Aware Module (QAM) is proposed to fuse the different quality information, the purpose of which is to estimate the quality of human parsing results. We combine QAM with a concise and effective network design to propose Quality-Aware Network (QANet) for human parsing. Benefiting from the superiority of QAM and QANet, we achieve the best performance on three multiple and one single human parsing benchmarks, including CIHP, MHP-v2, Pascal-Person-Part and LIP. Without increasing the training and inference time, QAM improves the AP$^\text{r}$ criterion by more than 10 points in the multiple human parsing task. QAM can be extended to other tasks with good quality estimation, e.g. instance segmentation. Specifically, QAM improves Mask R-CNN by ~1% mAP on COCO and LVISv1.0 datasets. Based on the proposed QAM and QANet, our overall system wins 1st place in CVPR2019 COCO DensePose Challenge, and 1st place in Track 1 & 2 of CVPR2020 LIP Challenge. Code and models are available at https://github.com/soeaver/QANet.
Deep Superpixel Cut for Unsupervised Image Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Image segmentation, one of the most critical vision tasks, has been studied for many years. Most of the early algorithms are unsupervised methods, which use hand-crafted features to divide the image into many regions. Recently, owing to the great success of deep learning technology, CNNs based methods show superior performance in image segmentation. However, these methods rely on a large number of human annotations, which are expensive to collect. In this paper, we propose a deep unsupervised method for image segmentation, which contains the following two stages. First, a Superpixelwise Autoencoder (SuperAE) is designed to learn the deep embedding and reconstruct a smoothed image, then the smoothed image is passed to generate superpixels. Second, we present a novel clustering algorithm called Deep Superpixel Cut (DSC), which measures the deep similarity between superpixels and formulates image segmentation as a soft partitioning problem. Via backpropagation, DSC adaptively partitions the superpixels into perceptual regions. Experimental results on the BSDS500 dataset demonstrate the effectiveness of the proposed method.
Subtrajectory Clustering: Finding Set Covers for Set Systems of Subcurves
Venue: arXiv cs.CG 2021
Year: 2021

We study subtrajectory clustering under the Fréchet distance. Given one or more trajectories, the task is to split the trajectories into several parts, such that the parts have a good clustering structure. We approach this problem via a new set cover formulation, which we think provides a natural formalization of the problem as it is studied in many applications. Given a polygonal curve $P$ with $n$ vertices in fixed dimension, integers $k$, $\ell \geq 1$, and a real value $Δ> 0$, the goal is to find $k$ center curves of complexity at most $\ell$ such that every point on $P$ is covered by a subtrajectory that has small Fréchet distance to one of the $k$ center curves ($\leq Δ$). In many application scenarios, one is interested in finding clusters of small complexity, which is controlled by the parameter $\ell$. Our main result is a bicriterial approximation algorithm: if there exists a solution for given parameters $k$, $\ell$, and $Δ$, then our algorithm finds a set of $k'$ center curves of complexity at most $\ell$ with covering radius $Δ'$ with $k' \in O( k \ell^2 \log (k \ell))$, and $Δ'\leq 19 Δ$. Moreover, within these approximation bounds, we can minimize $k$ while keeping the other parameters fixed. If $\ell$ is a constant independent of $n$, then, the approximation factor for the number of clusters $k$ is $O(\log k)$ and the approximation factor for the radius $Δ$ is constant. In this case, the algorithm has expected running time in $ \tilde{O}\left( k m^2 + mn\right)$ and uses space in $O(n+m)$, where $m=\lceil\frac{L}Δ\rceil$ and $L$ is the total arclength of the curve $P$.
Exploring Blockchain for The Coordination of Distributed Energy Resources
Venue: arXiv cs.DC 2021
Year: 2021

The fast growth of distributed energy resources (DERs), such as distributed renewables (e.g., rooftop PV panels), energy storage systems, electric vehicles, and controllable appliances, drives the power system toward a decentralized system with bidirectional power flow. The coordination of DERs through an aggregator, such as a utility, system operator, or a third-party coordinator, emerges as a promising paradigm. However, it is not well understood how to enable trust between the aggregator and DERs to integrate DERs efficiently. In this paper, we develop a trustable and distributed coordination system for DERs using blockchain technology. We model various DERs and formulate a cost minimization problem for DERs to optimize their energy trading, scheduling, and demand response. We use the alternating direction method of multipliers (ADMM) to solve the problem in a distributed fashion. To implement the distributed algorithm in a trustable way, we design a smart contract to update multipliers and communicate with DERs in a blockchain network. We validate our design by experiments using real-world data, and the simulation results demonstrate the effectiveness of our algorithm.
On spatial variation in the detectability and density of social media user protest supporters
Venue: arXiv cs.SI 2021
Year: 2021

Although much has been published regarding street protests on social media, few works have attempted to characterize social media users' spatial behavior in such events. The research reported here uses spatial capture-recapture methods to determine the influence of the built environment, physical proximity to protest location, and collective posting rhythm on variations in users' spatial detectability and density during a protest in Mexico City. The best-obtained model, together with explaining the spatial density of users, shows that there is high variability in the detectability of social media user protest supporters and that the collective posting rhythm and the day of observation are significant explanatory factors. The implication is that studies of collective spatial behavior would benefit by focussing on users' activity centres and their urban environment, rather than their physical proximity to the protest location, the latter being unable to adequately explain spatial variations in users' detectability and density during the protest event.
Knowledge-based Extraction of Cause-Effect Relations from Biomedical Text
Venue: arXiv cs.CL 2021
Year: 2021

We propose a knowledge-based approach for extraction of Cause-Effect (CE) relations from biomedical text. Our approach is a combination of an unsupervised machine learning technique to discover causal triggers and a set of high-precision linguistic rules to identify cause/effect arguments of these causal triggers. We evaluate our approach using a corpus of 58,761 Leukaemia-related PubMed abstracts consisting of 568,528 sentences. We could extract 152,655 CE triplets from this corpus where each triplet consists of a cause phrase, an effect phrase and a causal trigger. As compared to the existing knowledge base - SemMedDB (Kilicoglu et al., 2012), the number of extractions are almost twice. Moreover, the proposed approach outperformed the existing technique SemRep (Rindflesch and Fiszman, 2003) on a dataset of 500 sentences.
Symmetry meets AI
Venue: arXiv cs.LG 2021
Year: 2021

We explore whether Neural Networks (NNs) can {\it discover} the presence of symmetries as they learn to perform a task. For this, we train hundreds of NNs on a {\it decoy task} based on well-controlled Physics templates, where no information on symmetry is provided. We use the output from the last hidden layer of all these NNs, projected to fewer dimensions, as the input for a symmetry classification task, and show that information on symmetry had indeed been identified by the original NN without guidance. As an interdisciplinary application of this procedure, we identify the presence and level of symmetry in artistic paintings from different styles such as those of Picasso, Pollock and Van Gogh.
Stay on Topic, Please: Aligning User Comments to the Content of a News Article
Venue: arXiv cs.IR 2021
Year: 2021

Social scientists have shown that up to 50% if the content posted to a news article have no relation to its journalistic content. In this study we propose a classification algorithm to categorize user comments posted to a new article base don their alignment to its content. The alignment seek to match user comments to an article based on similarity off content, entities in discussion, and topic. We proposed a BERTAC, BAERT-based approach that learn jointly article-comment embeddings and infers the relevance class of comments. We introduce an ordinal classification loss that penalizes the difference between the predicted and true label. We conduct a thorough study to show influence of the proposed loss on the learning process. The results on five representative news outlets show that our approach can learn the comment class with up to 36% average accuracy improvement compering to the baselines, and up to 25% compering to the BA-BC model. BA-BC is out approach that consists of two models aimed to capture dis-jointly the formal language of news articles and the informal language of comments. We also conduct a user study to evaluate human labeling performance to understand the difficulty of the classification task. The user agreement on comment-article alignment is "moderate" per Krippendorff's alpha score, which suggests that the classification task is difficult.
Full-Resilient Memory-Optimum Multi-Party Non-Interactive Key Exchange
Venue: arXiv cs.CR 2021
Year: 2021

Multi-Party Non-Interactive Key Exchange (MP-NIKE) is a fundamental cryptographic primitive in which users register into a key generation centre and receive a public/private key pair each. After that, any subset of these users can compute a shared key without any interaction. Nowadays, IoT devices suffer from a high number and large size of messages exchanged in the Key Management Protocol (KMP). To overcome this, an MP-NIKE scheme can eliminate the airtime and latency of messages transferred between IoT devices. MP-NIKE schemes can be realized by using multilinear maps. There are several attempts for constructing multilinear maps based on indistinguishable obfuscation, lattices and the Chinese Remainder Theorem (CRT). Nevertheless, these schemes are inefficient in terms of computation cost and memory overhead. Besides, several attacks have been recently reported against CRT-based and lattice-based multilinear maps. There is only one modular exponentiation-based MP-NIKE scheme in the literature which has been claimed to be both secure and efficient. In this article, we present an attack on this scheme based on the Euclidean algorithm, in which two colluding users can obtain the shared key of any arbitrary subgroup of users. We also propose an efficient and secure MP-NIKE scheme. We show how our proposal is secure in the random oracle model assuming the hardness of the root extraction modulo a composite number.
Towards Learning an Unbiased Classifier from Biased Data via Conditional Adversarial Debiasing
Venue: arXiv cs.CV 2021
Year: 2021

Bias in classifiers is a severe issue of modern deep learning methods, especially for their application in safety- and security-critical areas. Often, the bias of a classifier is a direct consequence of a bias in the training dataset, frequently caused by the co-occurrence of relevant features and irrelevant ones. To mitigate this issue, we require learning algorithms that prevent the propagation of bias from the dataset into the classifier. We present a novel adversarial debiasing method, which addresses a feature that is spuriously connected to the labels of training images but statistically independent of the labels for test images. Thus, the automatic identification of relevant features during training is perturbed by irrelevant features. This is the case in a wide range of bias-related problems for many computer vision tasks, such as automatic skin cancer detection or driver assistance. We argue by a mathematical proof that our approach is superior to existing techniques for the abovementioned bias. Our experiments show that our approach performs better than state-of-the-art techniques on a well-known benchmark dataset with real-world images of cats and dogs.
Why flatness does and does not correlate with generalization for deep neural networks
Venue: arXiv cs.LG 2021
Year: 2021

The intuition that local flatness of the loss landscape is correlated with better generalization for deep neural networks (DNNs) has been explored for decades, spawning many different flatness measures. Recently, this link with generalization has been called into question by a demonstration that many measures of flatness are vulnerable to parameter re-scaling which arbitrarily changes their value without changing neural network outputs. Here we show that, in addition, some popular variants of SGD such as Adam and Entropy-SGD, can also break the flatness-generalization correlation. As an alternative to flatness measures, we use a function based picture and propose using the log of Bayesian prior upon initialization, $\log P(f)$, as a predictor of the generalization when a DNN converges on function $f$ after training to zero error. The prior is directly proportional to the Bayesian posterior for functions that give zero error on a test set. For the case of image classification, we show that $\log P(f)$ is a significantly more robust predictor of generalization than flatness measures are. Whilst local flatness measures fail under parameter re-scaling, the prior/posterior, which is global quantity, remains invariant under re-scaling. Moreover, the correlation with generalization as a function of data complexity remains good for different variants of SGD.
CreativeGAN: Editing Generative Adversarial Networks for Creative Design Synthesis
Venue: arXiv cs.LG 2021
Year: 2021

Modern machine learning techniques, such as deep neural networks, are transforming many disciplines ranging from image recognition to language understanding, by uncovering patterns in big data and making accurate predictions. They have also shown promising results for synthesizing new designs, which is crucial for creating products and enabling innovation. Generative models, including generative adversarial networks (GANs), have proven to be effective for design synthesis with applications ranging from product design to metamaterial design. These automated computational design methods can support human designers, who typically create designs by a time-consuming process of iteratively exploring ideas using experience and heuristics. However, there are still challenges remaining in automatically synthesizing `creative' designs. GAN models, however, are not capable of generating unique designs, a key to innovation and a major gap in AI-based design automation applications. This paper proposes an automated method, named CreativeGAN, for generating novel designs. It does so by identifying components that make a design unique and modifying a GAN model such that it becomes more likely to generate designs with identified unique components. The method combines state-of-art novelty detection, segmentation, novelty localization, rewriting, and generative models for creative design synthesis. Using a dataset of bicycle designs, we demonstrate that the method can create new bicycle designs with unique frames and handles, and generalize rare novelties to a broad set of designs. Our automated method requires no human intervention and demonstrates a way to rethink creative design synthesis and exploration.
Grasp Stability Analysis with Passive Reactions
Venue: arXiv cs.RO 2021
Year: 2021

[...] We argue that the traditional grasp modeling theory assumes a complexity that most robotic hands do not possess and is therefore of limited applicability to the robotic hands commonly used today. We discuss these limitations of the existing grasp modeling literature and set out to develop our own tools for the analysis of passive effects in robotic grasping. We show that problems of this kind are difficult to solve due to the non-convexity of the Maximum Dissipation Principle (MDP), which is part of the Coulomb friction law. We show that for planar grasps the MDP can be decomposed into a number of piecewise convex problems, which can be solved for efficiently. [...] Thus, we present the first polynomial runtime algorithm for the determination of passive stability of planar grasps. For the spacial case we [...] describe a convex relaxation of the Coulomb friction law and a successive hierarchical tightening approach that allows us to find solutions to the exact problem including the maximum dissipation principle. [...] The generality of our grasp model allows us to solve a wide variety of problems such as the computation of optimal actuator commands. This makes our framework a valuable tool for practical manipulation applications. Our work is relevant beyond robotic manipulation as it applies to the stability of any assembly of rigid bodies with frictional contacts, unilateral constraints and externally applied wrenches. Finally, we argue that with the advent of data-driven methods as well as the emergence of a new generation of highly sensorized hands there are opportunities for the application of the traditional grasp modeling theory to fields such as robotic in-hand manipulation through model-free reinforcement learning. We present a method that applies traditional grasp models to maintain quasi-static stability throughout a nominally model-free reinforcement learning task. [...]
CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
Venue: arXiv cs.CL 2021
Year: 2021

Many specialized domains remain untouched by deep learning, as large labeled datasets require expensive expert annotators. We address this bottleneck within the legal domain by introducing the Contract Understanding Atticus Dataset (CUAD), a new dataset for legal contract review. CUAD was created with dozens of legal experts from The Atticus Project and consists of over 13,000 annotations. The task is to highlight salient portions of a contract that are important for a human to review. We find that Transformer models have nascent performance, but that this performance is strongly influenced by model design and training dataset size. Despite these promising results, there is still substantial room for improvement. As one of the only large, specialized NLP benchmarks annotated by experts, CUAD can serve as a challenging research benchmark for the broader NLP community.
What is Multimodality?
Venue: arXiv cs.AI 2021
Year: 2021

The last years have shown rapid developments in the field of multimodal machine learning, combining e.g., vision, text or speech. In this position paper we explain how the field uses outdated definitions of multimodality that prove unfit for the machine learning era. We propose a new task-relative definition of (multi)modality in the context of multimodal machine learning that focuses on representations and information that are relevant for a given machine learning task. With our new definition of multimodality we aim to provide a missing foundation for multimodal research, an important component of language grounding and a crucial milestone towards NLU.