OpenRaiser/Intern-Atlas huggingface dataset:
Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions
Venue: arXiv cs.LG 2021
Year: 2021

This paper studies the theoretical underpinnings of machine learning of ergodic Itô diffusions. The objective is to understand the convergence properties of the invariant statistics when the underlying system of stochastic differential equations (SDEs) is empirically estimated with a supervised regression framework. Using the perturbation theory of ergodic Markov chains and the linear response theory, we deduce a linear dependence of the errors of one-point and two-point invariant statistics on the error in the learning of the drift and diffusion coefficients. More importantly, our study shows that the usual $L^2$-norm characterization of the learning generalization error is insufficient for achieving this linear dependence result. We find that sufficient conditions for such a linear dependence result are through learning algorithms that produce a uniformly Lipschitz and consistent estimator in the hypothesis space that retains certain characteristics of the drift coefficients, such as the usual linear growth condition that guarantees the existence of solutions of the underlying SDEs. We examine these conditions on two well-understood learning algorithms: the kernel-based spectral regression method and the shallow random neural networks with the ReLU activation function.
Probabilistic Sufficient Explanations
Venue: arXiv cs.LG 2021
Year: 2021

Understanding the behavior of learned classifiers is an important task, and various black-box explanations, logical reasoning approaches, and model-specific methods have been proposed. In this paper, we introduce probabilistic sufficient explanations, which formulate explaining an instance of classification as choosing the "simplest" subset of features such that only observing those features is "sufficient" to explain the classification. That is, sufficient to give us strong probabilistic guarantees that the model will behave similarly when all features are observed under the data distribution. In addition, we leverage tractable probabilistic reasoning tools such as probabilistic circuits and expected predictions to design a scalable algorithm for finding the desired explanations while keeping the guarantees intact. Our experiments demonstrate the effectiveness of our algorithm in finding sufficient explanations, and showcase its advantages compared to Anchors and logical explanations.
Visual representation of negation: Real world data analysis on comic image design
Venue: arXiv cs.CV 2021
Year: 2021

There has been a widely held view that visual representations (e.g., photographs and illustrations) do not depict negation, for example, one that can be expressed by a sentence "the train is not coming". This view is empirically challenged by analyzing the real-world visual representations of comic (manga) illustrations. In the experiment using image captioning tasks, we gave people comic illustrations and asked them to explain what they could read from them. The collected data showed that some comic illustrations could depict negation without any aid of sequences (multiple panels) or conventional devices (special symbols). This type of comic illustrations was subjected to further experiments, classifying images into those containing negation and those not containing negation. While this image classification was easy for humans, it was difficult for data-driven machines, i.e., deep learning models (CNN), to achieve the same high performance. Given the findings, we argue that some comic illustrations evoke background knowledge and thus can depict negation with purely visual elements.
Certification of Iterative Predictions in Bayesian Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of computing reach-avoid probabilities for iterative predictions made with Bayesian neural network (BNN) models. Specifically, we leverage bound propagation techniques and backward recursion to compute lower bounds for the probability that trajectories of the BNN model reach a given set of states while avoiding a set of unsafe states. We use the lower bounds in the context of control and reinforcement learning to provide safety certification for given control policies, as well as to synthesize control policies that improve the certification bounds. On a set of benchmarks, we demonstrate that our framework can be employed to certify policies over BNNs predictions for problems of more than $10$ dimensions, and to effectively synthesize policies that significantly increase the lower bound on the satisfaction probability.
Global Context for improving recognition of Online Handwritten Mathematical Expressions
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a temporal classification method for all three subtasks of symbol segmentation, symbol recognition and relation classification in online handwritten mathematical expressions (HMEs). The classification model is trained by multiple paths of symbols and spatial relations derived from the Symbol Relation Tree (SRT) representation of HMEs. The method benefits from global context of a deep bidirectional Long Short-term Memory network, which learns the temporal classification directly from online handwriting by the Connectionist Temporal Classification loss. To recognize an online HME, a symbol-level parse tree with Context-Free Grammar is constructed, where symbols and spatial relations are obtained from the temporal classification results. We show the effectiveness of the proposed method on the two latest CROHME datasets.
Have you tried Neural Topic Models? Comparative Analysis of Neural and Non-Neural Topic Models with Application to COVID-19 Twitter Data
Venue: arXiv cs.CL 2021
Year: 2021

Topic models are widely used in studying social phenomena. We conduct a comparative study examining state-of-the-art neural versus non-neural topic models, performing a rigorous quantitative and qualitative assessment on a dataset of tweets about the COVID-19 pandemic. Our results show that not only do neural topic models outperform their classical counterparts on standard evaluation metrics, but they also produce more coherent topics, which are of great benefit when studying complex social problems. We also propose a novel regularization term for neural topic models, which is designed to address the well-documented problem of mode collapse, and demonstrate its effectiveness.
Experiencing Utopia. A Positive Approach to Design Fiction
Venue: arXiv cs.HC 2021
Year: 2021

Design Fiction is known for its provocative and often dystopian speculations about the future. In this paper, we present an alternative approach that focuses primarily on the positive. We propose to imagine, enact, and evaluate utopia with participants. By doing so, we react to four main critiques concerning Design Fiction: (1) its negativity, (2) its contextlessness, (3) its elitist authorship, and (4) its missing evaluation methods.
Pyramid Fusion Dark Channel Prior for Single Image Dehazing
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose the pyramid fusion dark channel prior (PF-DCP) for single image dehazing. Based on the well-known Dark Channel Prior (DCP), we introduce an easy yet effective approach PF-DCP by employing the DCP algorithm at a pyramid of multi-scale images to alleviate the problem of patch size selection. In this case, we obtain the final transmission map by fusing transmission maps at each level to recover a high-quality haze-free image. Experiments on RESIDE SOTS show that PF-DCP not only outperforms the traditional prior-based methods with a large margin but also achieves comparable or even better results of state-of-art deep learning approaches. Furthermore, the visual quality is also greatly improved with much fewer color distortions and halo artifacts.
Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model
Venue: arXiv cs.CV 2021
Year: 2021

As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively, particularly for heterogeneous data sources. Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e., Houston2013 -- hyperspectral and multispectral data, Berlin -- hyperspectral and synthetic aperture radar (SAR) data, Augsburg -- hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification. Extensive experiments conducted on the three datasets demonstrate the superiority and advancement of our S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines. Furthermore, the baseline codes and datasets used in this paper will be made available freely at https://github.com/danfenghong/ISPRS_S2FL.
An Analysis of Twitter Users From The Perspective of Their Behavior, Language, Region and Development Indices -- A Study of 80 Million Tweets
Venue: arXiv cs.SI 2021
Year: 2021

The need for a comprehensive study to explore various aspects of online social media has been instigated by many researchers. This paper gives an insight into the social platform, Twitter. In this present work, we have illustrated stepwise procedure for crawling the data and discuss the key issues related to extracting associated features that can be useful in Twitter-related research while crawling these data from Application Programming Interfaces (APIs). Further, the data that comprises of over 86 million tweets have been analysed from various perspective including the most used languages, most frequent words, most frequent users, countries with most and least tweets and re-tweets, etc. The analysis reveals that the users' data associated with Twitter has a high affinity for researches in the various domain that includes politics, social science, economics, and linguistics, etc. In addition, the relation between Twitter users of a country and its human development index has been identified. It is observed that countries with very high human development indices have a relatively higher number of tweets compared to low human development indices countries. It is envisaged that the present study shall open many doors of researches in information processing and data science.
The Effectiveness of Kinematic Constraints on The Accuracy of Trajectory Profile of Human Walking Using PSPB Technique
Venue: arXiv cs.RO 2021
Year: 2021

Many methods have been developed in trajectory planning in order to achieve smooth and accurate motion with considering the constraints of kinematics constraints such as angular position, velocity, acceleration, and jerk. The problem of using the combination of n-order polynomials is that there is no ideally match between the segments of trajectory path at the via point in terms of the number of kinematic constraints. It leads to generate undesirable trajectory path at the via point that connects between two segments of the trajectory path. In this paper, we aim to investigate the effect of increasing to higher order polynomial blends on the accuracy of the via points with considering different kinematics constraints. Based on that, the methodology that was used in this paper is based on the polynomial segment with the higher polynomial blend (PSPB). Three techniques implemented which are 4-3-4 PSPB, 5-4-5 PSPB, and 6-5-6 PSPB. Each technique implemented based on applying different kinematic constraints. The three techniques validated using a modeling design in SemiMechanics. According to the methodology, the result analyzed and discussed in terms of angular position, angular velocity, angular acceleration, and angular jerk based on Root Mean Square Error (RMSE) and Average Difference Error (ADE). The result shows that RMSE of angular position for 4-3-4 PSPB-1, 4-3-4 PSPB-2, 5-4-5 PSPB-1, 5-4-5 PSPB-2, 6-5-6 PSPB-1, and 6-5-6 PSPB-2 are 0.4574, 0.0172, 10.9089, 0.1242, 0.6153, and 0.3128 degrees respectively. At the same time, the ADE are 0.0455, 0.0017, 1.0855, 0.0124, 0.0612, and 0.0311 degrees respectively. Thus, the error is increased obviously when there is no ideal match at the via point in terms of a number of kinematic constraints.
Measuring the impact of spammers on e-mail and Twitter networks
Venue: arXiv cs.SI 2021
Year: 2021

This paper investigates the research question if senders of large amounts of irrelevant or unsolicited information - commonly called "spammers" - distort the network structure of social networks. Two large social networks are analyzed, the first extracted from the Twitter discourse about a big telecommunication company, and the second obtained from three years of email communication of 200 managers working for a large multinational company. This work compares network robustness and the stability of centrality and interaction metrics, as well as the use of language, after removing spammers and the most and least connected nodes. The results show that spammers do not significantly alter the structure of the information-carrying network, for most of the social indicators. The authors additionally investigate the correlation between e-mail subject line and content by tracking language sentiment, emotionality, and complexity, addressing the cases where collecting email bodies is not permitted for privacy reasons. The findings extend the research about robustness and stability of social networks metrics, after the application of graph simplification strategies. The results have practical implication for network analysts and for those company managers who rely on network analytics (applied to company emails and social media data) to support their decision-making processes.
Joint Triplet Autoencoder for Histopathological Colon Cancer Nuclei Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning has shown a great improvement in the performance of visual tasks. Image retrieval is the task of extracting the visually similar images from a database for a query image. The feature matching is performed to rank the images. Various hand-designed features have been derived in past to represent the images. Nowadays, the power of deep learning is being utilized for automatic feature learning from data in the field of biomedical image analysis. Autoencoder and Siamese networks are two deep learning models to learn the latent space (i.e., features or embedding). Autoencoder works based on the reconstruction of the image from latent space. Siamese network utilizes the triplets to learn the intra-class similarity and inter-class dissimilarity. Moreover, Autoencoder is unsupervised, whereas Siamese network is supervised. We propose a Joint Triplet Autoencoder Network (JTANet) by facilitating the triplet learning in autoencoder framework. A joint supervised learning for Siamese network and unsupervised learning for Autoencoder is performed. Moreover, the Encoder network of Autoencoder is shared with Siamese network and referred as the Siamcoder network. The features are extracted by using the trained Siamcoder network for retrieval purpose. The experiments are performed over Histopathological Routine Colon Cancer dataset. We have observed the promising performance using the proposed JTANet model against the Autoencoder and Siamese models for colon cancer nuclei retrieval in histopathological images.
A Fair and Scalable Power Control Scheme in Multi-cell Massive MIMO
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies the transmit power optimization in a multi-cell massive multiple-input multiple-output (MIMO) system. To overcome the scalability issue of network-wide max-min fairness (NW-MMF), we propose a novel power control (PC) scheme. This scheme maximizes the geometric mean (GM) of the per-cell max-min spectral efficiency (SE). To solve this new optimization problem, we prove that it can be rewritten in a convex form and then solved using standard tools. To provide a fair comparison with the available utility functions in the literature, we solve the network-wide proportional fairness (NW-PF) PC as well. The NW-PF focuses on maximizing the sum SE, thereby ignoring fairness, but gives some extra attention to the weakest users. The simulation results highlight the benefits of our model which is balancing between NW-PF and NW-MMF.
Contention-Aware GPU Partitioning and Task-to-Partition Allocation for Real-Time Workloads
Venue: arXiv cs.DC 2021
Year: 2021

In order to satisfy timing constraints, modern real-time applications require massively parallel accelerators such as General Purpose Graphic Processing Units (GPGPUs). Generation after generation, the number of computing clusters made available in novel GPU architectures is steadily increasing, hence, investigating suitable scheduling approaches is now mandatory. Such scheduling approaches are related to mapping different and concurrent compute kernels within the GPU computing clusters, hence grouping GPU computing clusters into schedulable partitions. In this paper we propose novel techniques to define GPU partitions; this allows us to define suitable task-to-partition allocation mechanisms in which tasks are GPU compute kernels featuring different timing requirements. Such mechanisms will take into account the interference that GPU kernels experience when running in overlapping time windows. Hence, an effective and simple way to quantify the magnitude of such interference is also presented. We demonstrate the efficiency of the proposed approaches against the classical techniques that considered the GPU as a single, non-partitionable resource.
The Two-Dimensional Swept Rule Applied on Heterogeneous Architectures
Venue: arXiv cs.DC 2021
Year: 2021

The partial differential equations describing compressible fluid flows can be notoriously difficult to resolve on a pragmatic scale and often require the use of high performance computing systems and/or accelerators. However, these systems face scaling issues such as latency, the fixed cost of communicating information between devices in the system. The swept rule is a technique designed to minimize these costs by obtaining a solution to unsteady equations at as many possible spatial locations and times prior to communicating. In this study, we implemented and tested the swept rule for solving two-dimensional problems on heterogeneous computing systems across two distinct systems. Our solver showed a speedup range of 0.22-2.71 for the heat diffusion equation and 0.52-1.46 for the compressible Euler equations. We can conclude from this study that the swept rule offers both potential for speedups and slowdowns and that care should be taken when designing such a solver to maximize benefits. These results can help make decisions to maximize these benefits and inform designs.
Interactive Query Formulation using Spider Queries
Venue: arXiv cs.DB 2021
Year: 2021

Effective information disclosure in the context of databases with a large conceptual schema is known to be a non-trivial problem. In particular the formulation of ad-hoc queries is a major problem in such contexts. Existing approaches for tackling this problem include graphical query interfaces, query by navigation, query by construction, and point to point queries. In this article we propose the spider query mechanism as a final corner stone for an easy to use computer supported query formulation mechanism for InfoAssisant. The basic idea behind a spider query is to build a (partial) query of all information considered to be relevant with respect to a given object type. The result of this process is always a tree that fans out over existing conceptual schema (a spider). We also provide a brief discussion on the integration of the spider quer mechanism with the existing query by navigation, query by construction, and point to point query
Hierarchical Consistency Regularized Mean Teacher for Semi-supervised 3D Left Atrium Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning has achieved promising segmentation performance on 3D left atrium MR images. However, annotations for segmentation tasks are expensive, costly and difficult to obtain. In this paper, we introduce a novel hierarchical consistency regularized mean teacher framework for 3D left atrium segmentation. In each iteration, the student model is optimized by multi-scale deep supervision and hierarchical consistency regularization, concurrently. Extensive experiments have shown that our method achieves competitive performance as compared with full annotation, outperforming other state-of-the-art semi-supervised segmentation methods.
NVCache: A Plug-and-Play NVMM-based I/O Booster for Legacy Systems
Venue: arXiv cs.DC 2021
Year: 2021

This paper introduces NVCache, an approach that uses a non-volatile main memory (NVMM) as a write cache to improve the write performance of legacy applications. We compare NVCache against file systems tailored for NVMM (Ext4-DAX and NOVA) and with I/O-heavy applications (SQLite, RocksDB). Our evaluation shows that NVCache reaches the performance level of the existing state-of-the-art systems for NVMM, but without their limitations: NVCache does not limit the size of the stored data to the size of the NVMM, and works transparently with unmodified legacy applications, providing additional persistence guarantees even when their source code is not available.
LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond
Venue: arXiv cs.CV 2021
Year: 2021

Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. Last few years have witnessed impressive progress propelled by deep learning methods. However, one critical challenge faced by existing methods is to strike a sweet spot of deep model complexity and resulting SISR quality. This paper addresses this pain point by proposing a linearly-assembled pixel-adaptive regression network (LAPAR), which casts the direct LR to HR mapping learning into a linear coefficient regression task over a dictionary of multiple predefined filter bases. Such a parametric representation renders our model highly lightweight and easy to optimize while achieving state-of-the-art results on SISR benchmarks. Moreover, based on the same idea, LAPAR is extended to tackle other restoration tasks, e.g., image denoising and JPEG image deblocking, and again, yields strong performance. The code is available at https://github.com/dvlab-research/Simple-SR.