OpenRaiser/Intern-Atlas huggingface dataset:
Perceiving and Modeling Density is All You Need for Image Dehazing
Venue: arXiv cs.CV 2021
Year: 2021

In the real world, the degradation of images taken under haze can be quite complex, where the spatial distribution of haze is varied from image to image. Recent methods adopt deep neural networks to recover clean scenes from hazy images directly. However, due to the paradox caused by the variation of real captured haze and the fixed degradation parameters of the current networks, the generalization ability of recent dehazing methods on real-world hazy images is not ideal.To address the problem of modeling real-world haze degradation, we propose to solve this problem by perceiving and modeling density for uneven haze distribution. We propose a novel Separable Hybrid Attention (SHA) module to encode haze density by capturing features in the orthogonal directions to achieve this goal. Moreover, a density map is proposed to model the uneven distribution of the haze explicitly. The density map generates positional encoding in a semi-supervised way. Such a haze density perceiving and modeling capture the unevenly distributed degeneration at the feature level effectively. Through a suitable combination of SHA and density map, we design a novel dehazing network architecture, which achieves a good complexity-performance trade-off. The extensive experiments on two large-scale datasets demonstrate that our method surpasses all state-of-the-art approaches by a large margin both quantitatively and qualitatively, boosting the best published PSNR metric from 28.53 dB to 33.49 dB on the Haze4k test dataset and from 37.17 dB to 38.41 dB on the SOTS indoor test dataset.
Interactive segmentation using U-Net with weight map and dynamic user interactions
Venue: arXiv cs.CV 2021
Year: 2021

Interactive segmentation has recently attracted attention for specialized tasks where expert input is required to further enhance the segmentation performance. In this work, we propose a novel interactive segmentation framework, where user clicks are dynamically adapted in size based on the current segmentation mask. The clicked regions form a weight map and are fed to a deep neural network as a novel weighted loss function. To evaluate our loss function, an interactive U-Net (IU-Net) model which applies both foreground and background user clicks as the main method of interaction is employed. We train and validate on the BCV dataset, while testing on spleen and colon cancer CT images from the MSD dataset to improve the overall segmentation accuracy in comparison to the standard U-Net using our weighted loss function. Applying dynamic user click sizes increases the overall accuracy by 5.60% and 10.39% respectively by utilizing only a single user interaction.
Detecting Cross-Language Plagiarism using Open Knowledge Graphs
Venue: arXiv cs.CL 2021
Year: 2021

Identifying cross-language plagiarism is challenging, especially for distant language pairs and sense-for-sense translations. We introduce the new multilingual retrieval model Cross-Language Ontology-Based Similarity Analysis (CL-OSA) for this task. CL-OSA represents documents as entity vectors obtained from the open knowledge graph Wikidata. Opposed to other methods, CL-OSA does not require computationally expensive machine translation, nor pre-training using comparable or parallel corpora. It reliably disambiguates homonyms and scales to allow its application to Web-scale document collections. We show that CL-OSA outperforms state-of-the-art methods for retrieving candidate documents from five large, topically diverse test corpora that include distant language pairs like Japanese-English. For identifying cross-language plagiarism at the character level, CL-OSA primarily improves the detection of sense-for-sense translations. For these challenging cases, CL-OSA's performance in terms of the well-established PlagDet score exceeds that of the best competitor by more than factor two. The code and data of our study are openly available.
Wiggling Weights to Improve the Robustness of Classifiers
Venue: arXiv cs.CV 2021
Year: 2021

Robustness against unwanted perturbations is an important aspect of deploying neural network classifiers in the real world. Common natural perturbations include noise, saturation, occlusion, viewpoint changes, and blur deformations. All of them can be modelled by the newly proposed transform-augmented convolutional networks. While many approaches for robustness train the network by providing augmented data to the network, we aim to integrate perturbations in the network architecture to achieve improved and more general robustness. To demonstrate that wiggling the weights consistently improves classification, we choose a standard network and modify it to a transform-augmented network. On perturbed CIFAR-10 images, the modified network delivers a better performance than the original network. For the much smaller STL-10 dataset, in addition to delivering better general robustness, wiggling even improves the classification of unperturbed, clean images substantially. We conclude that wiggled transform-augmented networks acquire good robustness even for perturbations not seen during training.
TransMix: Attend to Mix for Vision Transformers
Venue: arXiv cs.CV 2021
Year: 2021

Mixup-based augmentation has been found to be effective for generalizing models during training, especially for Vision Transformers (ViTs) since they can easily overfit. However, previous mixup-based methods have an underlying prior knowledge that the linearly interpolated ratio of targets should be kept the same as the ratio proposed in input interpolation. This may lead to a strange phenomenon that sometimes there is no valid object in the mixed image due to the random process in augmentation but there is still response in the label space. To bridge such gap between the input and label spaces, we propose TransMix, which mixes labels based on the attention maps of Vision Transformers. The confidence of the label will be larger if the corresponding input image is weighted higher by the attention map. TransMix is embarrassingly simple and can be implemented in just a few lines of code without introducing any extra parameters and FLOPs to ViT-based models. Experimental results show that our method can consistently improve various ViT-based models at scales on ImageNet classification. After pre-trained with TransMix on ImageNet, the ViT-based models also demonstrate better transferability to semantic segmentation, object detection and instance segmentation. TransMix also exhibits to be more robust when evaluating on 4 different benchmarks. Code will be made publicly available at https://github.com/Beckschen/TransMix.
Method for representing an exponent in a fifth-dimensional hypercomplex number systems using a hypercomplex computing software
Venue: arXiv cs.SE 2021
Year: 2021

The structure of method for constructing a representation of an exponential function in hypercomplex number systems(HNS) by the method of solving an associated system of linear differential equations is considered. Brief information about the hypercomplex computing software (HCS) is given. With the use of HCS, the necessary cumbersome operations on symbolic expressions were performed when constructing the representation of the exponent in the fifth-dimensional HNS. Fragments of programs in the environment of HCS and results of symbolic calculations are resulted
Learning To Estimate Regions Of Attraction Of Autonomous Dynamical Systems Using Physics-Informed Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

When learning to perform motor tasks in a simulated environment, neural networks must be allowed to explore their action space to discover new potentially viable solutions. However, in an online learning scenario with physical hardware, this exploration must be constrained by relevant safety considerations in order to avoid damage to the agent's hardware and environment. We aim to address this problem by training a neural network, which we will refer to as a "safety network", to estimate the region of attraction (ROA) of a controlled autonomous dynamical system. This safety network can thereby be used to quantify the relative safety of proposed control actions and prevent the selection of damaging actions. Here we present our development of the safety network by training an artificial neural network (ANN) to represent the ROA of several autonomous dynamical system benchmark problems. The training of this network is predicated upon both Lyapunov theory and neural solutions to partial differential equations (PDEs). By learning to approximate the viscosity solution to a specially chosen PDE that contains the dynamics of the system of interest, the safety network learns to approximate a particular function, similar to a Lyapunov function, whose zero level set is boundary of the ROA. We train our safety network to solve these PDEs in a semi-supervised manner following a modified version of the Physics Informed Neural Network (PINN) approach, utilizing a loss function that penalizes disagreement with the PDE's initial and boundary conditions, as well as non-zero residual and variational terms. In future work we intend to apply this technique to reinforcement learning agents during motor learning tasks.
SmartSlice: Dynamic, self-optimization of applications QoS requests to 5G networks
Venue: arXiv cs.NI 2021
Year: 2021

Applications can tailor a network slice by specifying a variety of QoS attributes related to application-specific performance, function or operation. However, some QoS attributes like guaranteed bandwidth required by the application do vary over time. For example, network bandwidth needs of video streams from surveillance cameras can vary a lot depending on the environmental conditions and the content in the video streams. In this paper, we propose a novel, dynamic QoS attribute prediction technique that assists any application to make optimal resource reservation requests at all times. Standard forecasting using traditional cost functions like MAE, MSE, RMSE, MDA, etc. don't work well because they do not take into account the direction (whether the forecasting of resources is more or less than needed), magnitude (by how much the forecast deviates, and in which direction), or frequency (how many times the forecast deviates from actual needs, and in which direction). The direction, magnitude and frequency have a direct impact on the application's accuracy of insights, and the operational costs. We propose a new, parameterized cost function that takes into account all three of them, and guides the design of a new prediction technique. To the best of our knowledge, this is the first work that considers time-varying application requirements and dynamically adjusts slice QoS requests to 5G networks in order to ensure a balance between application's accuracy and operational costs. In a real-world deployment of a surveillance video analytics application over 17 cameras, we show that our technique outperforms other traditional forecasting methods, and it saves 34% of network bandwidth (over a ~24 hour period) when compared to a static, one-time reservation.
Sketch-based Creativity Support Tools using Deep Learning
Venue: arXiv cs.HC 2021
Year: 2021

Sketching is a natural and effective visual communication medium commonly used in creative processes. Recent developments in deep-learning models drastically improved machines' ability in understanding and generating visual content. An exciting area of development explores deep-learning approaches used to model human sketches, opening opportunities for creative applications. This chapter describes three fundamental steps in developing deep-learning-driven creativity support tools that consumes and generates sketches: 1) a data collection effort that generated a new paired dataset between sketches and mobile user interfaces; 2) a sketch-based user interface retrieval system adapted from state-of-the-art computer vision techniques; and, 3) a conversational sketching system that supports the novel interaction of a natural-language-based sketch/critique authoring process. In this chapter, we survey relevant prior work in both the deep-learning and human-computer-interaction communities, document the data collection process and the systems' architectures in detail, present qualitative and quantitative results, and paint the landscape of several future research directions in this exciting area.
Esophageal virtual disease landscape using mechanics-informed machine learning
Venue: arXiv cs.LG 2021
Year: 2021

The pathogenesis of esophageal disorders is related to the esophageal wall mechanics. Therefore, to understand the underlying fundamental mechanisms behind various esophageal disorders, it is crucial to map the esophageal wall mechanics-based parameters onto physiological and pathophysiological conditions corresponding to altered bolus transit and supraphysiologic IBP. In this work, we present a hybrid framework that combines fluid mechanics and machine learning to identify the underlying physics of the various esophageal disorders and maps them onto a parameter space which we call the virtual disease landscape (VDL). A one-dimensional inverse model processes the output from an esophageal diagnostic device called endoscopic functional lumen imaging probe (EndoFLIP) to estimate the mechanical "health" of the esophagus by predicting a set of mechanics-based parameters such as esophageal wall stiffness, muscle contraction pattern and active relaxation of esophageal walls. The mechanics-based parameters were then used to train a neural network that consists of a variational autoencoder (VAE) that generates a latent space and a side network that predicts mechanical work metrics for estimating esophagogastric junction motility. The latent vectors along with a set of discrete mechanics-based parameters define the VDL and form clusters corresponding to the various esophageal disorders. The VDL not only distinguishes different disorders but can also be used to predict disease progression in time. Finally, we also demonstrate the clinical applicability of this framework for estimating the effectiveness of a treatment and track patient condition after a treatment.
Sum Capacity Loss Quantification With Optimal and Sub-Optimal Precoding in Heterogeneous Multiuser Channels
Venue: arXiv cs.IT 2021
Year: 2021

We analytically approximate the expected sum capacity loss between the optimal downlink precoding technique of dirty paper coding (DPC), and the sub-optimal technique of zero-forcing precoding, for multiuser channels. We also consider the most general case of multi-stream transmission to multiple users, where we evaluate the expected sum capacity loss between DPC and block diagonalization precoding. Unlike previously, assuming heterogeneous Ricean fading, we utilize the well known affine approximation to predict the expected sum capacity difference between both precoder types (optimal and sub-optimal) over a wide range of system and propagation parameters. Furthermore, for single-stream transmission, we consider the problem of weighted sum capacity maximization, where a similar quantification of the sum capacity difference between the two precoder types is presented. In doing so, we disclose that power allocation to different users proportional to their individual weights asymptotically maximizes the weighted sum capacity. Numerical simulations are presented to demonstrate the tightness of the developed expressions relative to their simulated counterparts.
Autoencoder-based Semantic Communication Systems with Relay Channels
Venue: arXiv cs.IT 2021
Year: 2021

In this letter, we propose a semantic communication scheme for wireless relay channels based on Autoencoder, named AESC, which encodes and decodes sentences from the semantic dimension. The Autoencoder module provides anti-noise performance for the system. Meanwhile, a novel semantic forward (SF) mode is designed for the relay node to forward the semantic information at the semantic level, especially for the scenarios that there is no common knowledge shared between the source and destination nodes. Numerical results show that the AESC achieves better stability performance than the traditional communication schemes, and the proposed SF mode provides a significant performance gain compared to the traditional forward protocols.
Does BERT look at sentiment lexicon?
Venue: arXiv cs.CL 2021
Year: 2021

The main approaches to sentiment analysis are rule-based methods and ma-chine learning, in particular, deep neural network models with the Trans-former architecture, including BERT. The performance of neural network models in the tasks of sentiment analysis is superior to the performance of rule-based methods. The reasons for this situation remain unclear due to the poor interpretability of deep neural network models. One of the main keys to understanding the fundamental differences between the two approaches is the analysis of how sentiment lexicon is taken into account in neural network models. To this end, we study the attention weights matrices of the Russian-language RuBERT model. We fine-tune RuBERT on sentiment text corpora and compare the distributions of attention weights for sentiment and neutral lexicons. It turns out that, on average, 3/4 of the heads of various model var-iants statistically pay more attention to the sentiment lexicon compared to the neutral one.
Uncertainty-aware Low-Rank Q-Matrix Estimation for Deep Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Value estimation is one key problem in Reinforcement Learning. Albeit many successes have been achieved by Deep Reinforcement Learning (DRL) in different fields, the underlying structure and learning dynamics of value function, especially with complex function approximation, are not fully understood. In this paper, we report that decreasing rank of $Q$-matrix widely exists during learning process across a series of continuous control tasks for different popular algorithms. We hypothesize that the low-rank phenomenon indicates the common learning dynamics of $Q$-matrix from stochastic high dimensional space to smooth low dimensional space. Moreover, we reveal a positive correlation between value matrix rank and value estimation uncertainty. Inspired by above evidence, we propose a novel Uncertainty-Aware Low-rank Q-matrix Estimation (UA-LQE) algorithm as a general framework to facilitate the learning of value function. Through quantifying the uncertainty of state-action value estimation, we selectively erase the entries of highly uncertain values in state-action value matrix and conduct low-rank matrix reconstruction for them to recover their values. Such a reconstruction exploits the underlying structure of value matrix to improve the value approximation, thus leading to a more efficient learning process of value function. In the experiments, we evaluate the efficacy of UA-LQE in several representative OpenAI MuJoCo continuous control tasks.
IRONWAN: Increasing Reliability of Overlapping Networks in LoRaWAN
Venue: arXiv cs.NI 2021
Year: 2021

LoRaWAN deployments follow an ad-hoc deployment model that has organically led to overlapping communication networks, sharing the wireless spectrum, and completely unaware of each other. LoRaWAN uses ALOHA-style communication where it is almost impossible to schedule transmission between networks belonging to different owners properly. The inability to schedule overlapping networks will cause inter-network interference, which will increase node-to-gateway message losses and gateway-to-node acknowledgement failures. This problem is likely to get worse as the number of LoRaWAN networks increase. In response to this problem, we propose IRONWAN, a wireless overlay network that shares communication resources without modifications to underlying protocols. It utilises the broadcast nature of radio communication and enables gateway-to-gateway communication to facilitate the search for failed messages and transmit failed acknowledgements already received and cached in overlapping network's gateways. IRONWAN uses two novel algorithms, a Real-time Message Inter-arrival Predictor, to highlight when a server has not received an expected uplink message. The Interference Predictor ensures that extra gateway-to-gateway communication does not negatively impact communication bandwidth. We evaluate IRONWAN on a 1000-node simulator with up to ten gateways and a 10-node testbed with 2-gateways. Results show that IRONWAN can achieve up to 12\% higher packet delivery ratio (PDR) and total messages received per node while increasing the minimum PDR by up to 28\%. These improvements save up to 50\% node's energy. Finally, we demonstrate that IRONWAN has comparable performance to an optimal solution (wired, centralised) but with 2-32 times lower communication costs. IRONWAN also has up to 14\% better PDR when compared to FLIP, a wired-distributed gateway-to-gateway protocol in certain scenarios.
Neural Image Beauty Predictor Based on Bradley-Terry Model
Venue: arXiv cs.CV 2021
Year: 2021

Image beauty assessment is an important subject of computer vision. Therefore, building a model to mimic the image beauty assessment becomes an important task. To better imitate the behaviours of the human visual system (HVS), a complete survey about images of different categories should be implemented. This work focuses on image beauty assessment. In this study, the pairwise evaluation method was used, which is based on the Bradley-Terry model. We believe that this method is more accurate than other image rating methods within an image group. Additionally, Convolution neural network (CNN), which is fit for image quality assessment, is used in this work. The first part of this study is a survey about the image beauty comparison of different images. The Bradley-Terry model is used for the calculated scores, which are the target of CNN model. The second part of this work focuses on the results of the image beauty prediction, including landscape images, architecture images and portrait images. The models are pretrained by the AVA dataset to improve the performance later. Then, the CNN model is trained with the surveyed images and corresponding scores. Furthermore, this work compares the results of four CNN base networks, i.e., Alex net, VGG net, Squeeze net and LSiM net, as discussed in literature. In the end, the model is evaluated by the accuracy in pairs, correlation coefficient and relative error calculated by survey results. Satisfactory results are achieved by our proposed methods with about 70 percent accuracy in pairs. Our work sheds more light on the novel image beauty assessment method. While more studies should be conducted, this method is a promising step.
Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems via Fine-Grained Contrastive Learning
Venue: arXiv cs.CL 2021
Year: 2021

Retrieve-based dialogue response selection aims to find a proper response from a candidate set given a multi-turn context. Pre-trained language models (PLMs) based methods have yielded significant improvements on this task. The sequence representation plays a key role in the learning of matching degree between the dialogue context and the response. However, we observe that different context-response pairs sharing the same context always have a greater similarity in the sequence representations calculated by PLMs, which makes it hard to distinguish positive responses from negative ones. Motivated by this, we propose a novel \textbf{F}ine-\textbf{G}rained \textbf{C}ontrastive (FGC) learning method for the response selection task based on PLMs. This FGC learning strategy helps PLMs to generate more distinguishable matching representations of each dialogue at fine grains, and further make better predictions on choosing positive responses. Empirical studies on two benchmark datasets demonstrate that the proposed FGC learning method can generally and significantly improve the model performance of existing PLM-based matching models.