OpenRaiser/Intern-Atlas huggingface dataset:
Deep Learning Models in Detection of Dietary Supplement Adverse Event Signals from Twitter
Venue: arXiv cs.CL 2021
Year: 2021

Objective: The objective of this study is to develop a deep learning pipeline to detect signals on dietary supplement-related adverse events (DS AEs) from Twitter. Material and Methods: We obtained 247,807 tweets ranging from 2012 to 2018 that mentioned both DS and AE. We annotated biomedical entities and relations on 2,000 randomly selected tweets. For the concept extraction task, we compared the performance of traditional word embeddings with SVM, CRF and LSTM-CRF classifiers to BERT models. For the relation extraction task, we compared GloVe vectors with CNN classifiers to BERT models. We chose the best performing models in each task to assemble an end-to-end deep learning pipeline to detect DS AE signals and compared the results to the known DS AEs from a DS knowledge base (i.e., iDISK). Results: In both tasks, the BERT-based models outperformed traditional word embeddings. The best performing concept extraction model is the BioBERT model that can identify supplement, symptom, and body organ entities with F1-scores of 0.8646, 0.8497, and 0.7104, respectively. The best performing relation extraction model is the BERT model that can identify purpose and AE relations with F1-scores of 0.8335 and 0.7538, respectively. The end-to-end pipeline was able to extract DS indication and DS AEs with an F1-score of 0.7459 and 0,7414, respectively. Comparing to the iDISK, we could find both known and novel DS-AEs. Conclusion: We have demonstrated the feasibility of detecting DS AE signals from Twitter with a BioBERT-based deep learning pipeline.
A Survey of Race, Racism, and Anti-Racism in NLP
Venue: arXiv cs.CL 2021
Year: 2021

Despite inextricable ties between race and language, little work has considered race in NLP research and development. In this work, we survey 79 papers from the ACL anthology that mention race. These papers reveal various types of race-related bias in all stages of NLP model development, highlighting the need for proactive consideration of how NLP systems can uphold racial hierarchies. However, persistent gaps in research on race and NLP remain: race has been siloed as a niche topic and remains ignored in many NLP tasks; most work operationalizes race as a fixed single-dimensional variable with a ground-truth label, which risks reinforcing differences produced by historical racism; and the voices of historically marginalized people are nearly absent in NLP literature. By identifying where and how NLP literature has and has not considered race, especially in comparison to related fields, our work calls for inclusion and racial justice in NLP research practices.
ConvDySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention and Convolutional Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Learning node representations on temporal graphs is a fundamental step to learn real-word dynamic graphs efficiently. Real-world graphs have the nature of continuously evolving over time, such as changing edges weights, removing and adding nodes and appearing and disappearing of edges, while previous graph representation learning methods focused generally on static graphs. We present ConvDySAT as an enhancement of DySAT, one of the state-of-the-art dynamic methods, by augmenting convolution neural networks with the self-attention mechanism, the employed method in DySAT to express the structural and temporal evolution. We conducted single-step link prediction on a communication network and rating network, Experimental results show significant performance gains for ConvDySAT over various state-of-the-art methods.
A Competitive Analysis of Online Multi-Agent Path Finding
Venue: arXiv cs.AI 2021
Year: 2021

We study online Multi-Agent Path Finding (MAPF), where new agents are constantly revealed over time and all agents must find collision-free paths to their given goal locations. We generalize existing complexity results of (offline) MAPF to online MAPF. We classify online MAPF algorithms into different categories based on (1) controllability (the set of agents that they can plan paths for at each time) and (2) rationality (the quality of paths they plan) and study the relationships between them. We perform a competitive analysis for each category of online MAPF algorithms with respect to commonly-used objective functions. We show that a naive algorithm that routes newly-revealed agents one at a time in sequence achieves a competitive ratio that is asymptotically bounded from both below and above by the number of agents with respect to flowtime and makespan. We then show a counter-intuitive result that, if rerouting of previously-revealed agents is not allowed, any rational online MAPF algorithms, including ones that plan optimal paths for all newly-revealed agents, have the same asymptotic competitive ratio as the naive algorithm, even on 2D 4-neighbor grids. We also derive constant lower bounds on the competitive ratio of any rational online MAPF algorithms that allow rerouting. The results thus provide theoretical insights into the effectiveness of using MAPF algorithms in an online setting for the first time.
KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers
Venue: arXiv cs.CL 2021
Year: 2021

The goal of database question answering is to enable natural language querying of real-life relational databases in diverse application domains. Recently, large-scale datasets such as Spider and WikiSQL facilitated novel modeling techniques for text-to-SQL parsing, improving zero-shot generalization to unseen databases. In this work, we examine the challenges that still prevent these techniques from practical deployment. First, we present KaggleDBQA, a new cross-domain evaluation dataset of real Web databases, with domain-specific data types, original formatting, and unrestricted questions. Second, we re-examine the choice of evaluation tasks for text-to-SQL parsers as applied in real-life settings. Finally, we augment our in-domain evaluation task with database documentation, a naturally occurring source of implicit domain knowledge. We show that KaggleDBQA presents a challenge to state-of-the-art zero-shot parsers but a more realistic evaluation setting and creative use of associated database documentation boosts their accuracy by over 13.2%, doubling their performance.
No Finite Model Property for Logics of Quantified Announcements
Venue: arXiv cs.LO 2021
Year: 2021

Quantification over public announcements shifts the perspective from reasoning strictly about the results of a particular announcement to reasoning about the existence of an announcement that achieves some certain epistemic goal. Depending on the type of the quantification, we get different formalisms, the most known of which are arbitrary public announcement logic (APAL), group announcement logic (GAL), and coalition announcement logic (CAL). It has been an open question whether the logics have the finite model property, and in the paper we answer the question negatively. We also discuss how this result is connected to other open questions in the field.
Knowledge from Probability
Venue: arXiv cs.LO 2021
Year: 2021

We give a probabilistic analysis of inductive knowledge and belief and explore its predictions concerning knowledge about the future, about laws of nature, and about the values of inexactly measured quantities. The analysis combines a theory of knowledge and belief formulated in terms of relations of comparative normality with a probabilistic reduction of those relations. It predicts that only highly probable propositions are believed, and that many widely held principles of belief-revision fail.
Do Language Models Perform Generalizable Commonsense Inference?
Venue: arXiv cs.CL 2021
Year: 2021

Inspired by evidence that pretrained language models (LMs) encode commonsense knowledge, recent work has applied LMs to automatically populate commonsense knowledge graphs (CKGs). However, there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. This paper analyzes the ability of LMs to perform generalizable commonsense inference, in terms of knowledge capacity, transferability, and induction. Our experiments with these three aspects show that: (1) LMs can adapt to different schemas defined by multiple CKGs but fail to reuse the knowledge to generalize to new relations. (2) Adapted LMs generalize well to unseen subjects, but less so on novel objects. Future work should investigate how to improve the transferability and induction of commonsense mining from LMs.
Creating A New Color Space utilizing PSO and FCM to Perform Skin Detection by using Neural Network and ANFIS
Venue: arXiv cs.CV 2021
Year: 2021

Skin color detection is an essential required step in various applications related to computer vision. These applications will include face detection, finding pornographic images in movies and photos, finding ethnicity, age, diagnosis, and so on. Therefore, proposing a proper skin detection method can provide solution to several problems. In this study, first a new color space is created using FCM and PSO algorithms. Then, skin classification has been performed in the new color space utilizing linear and nonlinear modes. Additionally, it has been done in RGB and LAB color spaces by using ANFIS and neural network. Skin detection in RBG color space has been performed using Mahalanobis distance and Euclidean distance algorithms. In comparison, this method has 18.38% higher accuracy than the most accurate method on the same database. Additionally, this method has achieved 90.05% in equal error rate (1-EER) in testing COMPAQ dataset and 92.93% accuracy in testing Pratheepan dataset, which compared to the previous method on COMPAQ database, 1-EER has increased by %0.87.
Continuous-Depth Neural Models for Dynamic Graph Prediction
Venue: arXiv cs.LG 2021
Year: 2021

We introduce the framework of continuous-depth graph neural networks (GNNs). Neural graph differential equations (Neural GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with static GNN models and is extended to dynamic and stochastic settings through hybrid dynamical system theory. Here, Neural GDEs improve performance by exploiting the underlying dynamics geometry, further introducing the ability to accommodate irregularly sampled data. Results prove the effectiveness of the proposed models across applications, such as traffic forecasting or prediction in genetic regulatory networks.
A Comparison for Patch-level Classification of Deep Learning Methods on Transparent Environmental Microorganism Images: from Convolutional Neural Networks to Visual Transformers
Venue: arXiv cs.CV 2021
Year: 2021

Nowadays, analysis of Transparent Environmental Microorganism Images (T-EM images) in the field of computer vision has gradually become a new and interesting spot. This paper compares different deep learning classification performance for the problem that T-EM images are challenging to analyze. We crop the T-EM images into 8 * 8 and 224 * 224 pixel patches in the same proportion and then divide the two different pixel patches into foreground and background according to ground truth. We also use four convolutional neural networks and a novel ViT network model to compare the foreground and background classification experiments. We conclude that ViT performs the worst in classifying 8 * 8 pixel patches, but it outperforms most convolutional neural networks in classifying 224 * 224 pixel patches.
Efficient recursive least squares solver for rank-deficient matrices
Venue: arXiv cs.MS 2021
Year: 2021

Updating a linear least squares solution can be critical for near real-time signalprocessing applications. The Greville algorithm proposes a simple formula for updating the pseudoinverse of a matrix A $\in$ R nxm with rank r. In this paper, we explicitly derive a similar formula by maintaining a general rank factorization, which we call rank-Greville. Based on this formula, we implemented a recursive least squares algorithm exploiting the rank-deficiency of A, achieving the update of the minimum-norm least-squares solution in O(mr) operations and, therefore, solving the linear least-squares problem from scratch in O(nmr) operations. We empirically confirmed that this algorithm displays a better asymptotic time complexity than LAPACK solvers for rank-deficient matrices. The numerical stability of rank-Greville was found to be comparable to Cholesky-based solvers. Nonetheless, our implementation supports exact numerical representations of rationals, due to its remarkable algebraic simplicity.
MMD-MIX: Value Function Factorisation with Maximum Mean Discrepancy for Cooperative Multi-Agent Reinforcement Learning
Venue: arXiv cs.MA 2021
Year: 2021

In the real world, many tasks require multiple agents to cooperate with each other under the condition of local observations. To solve such problems, many multi-agent reinforcement learning methods based on Centralized Training with Decentralized Execution have been proposed. One representative class of work is value decomposition, which decomposes the global joint Q-value $Q_\text{jt}$ into individual Q-values $Q_a$ to guide individuals' behaviors, e.g. VDN (Value-Decomposition Networks) and QMIX. However, these baselines often ignore the randomness in the situation. We propose MMD-MIX, a method that combines distributional reinforcement learning and value decomposition to alleviate the above weaknesses. Besides, to improve data sampling efficiency, we were inspired by REM (Random Ensemble Mixture) which is a robust RL algorithm to explicitly introduce randomness into the MMD-MIX. The experiments demonstrate that MMD-MIX outperforms prior baselines in the StarCraft Multi-Agent Challenge (SMAC) environment.
HAP-Reserved Communications in Space-Air-Ground Integrated Networks
Venue: arXiv cs.NI 2021
Year: 2021

Terrestrial communication networks have experienced significant development in recent years by providing emerging services for ground users. However, one critical challenge raised is to provide full coverage (especially in dense high-rise urban environments) for ground users due to scarce network resources and limited coverage. To meet this challenge, we propose a high altitude platform (HAP)-reserved ground-air-space (GAS) transmission scheme, which combines with the ground-to-space (G2S) transmission scheme to strengthen the terrestrial communication and save the transmission power. To integrate the two transmission schemes, we propose a transmission control strategy. Wherein, the ground user decides its transmission scheme, i.e., switches between the GAS link transmission and the G2S link transmission with a probability. We then maximize the overall throughput and derive the optimal probability that a ground user adopts the GAS transmission scheme. Numerical results demonstrate the superiority of the proposed transmission control strategy.
Proof-of-Vax: Studying User Preferences and Perception of Covid Vaccination Certificates
Venue: arXiv cs.HC 2021
Year: 2021

Digital tools play an important role in fighting the current global COVID-19 pandemic. We conducted a representative online study in Germany on a sample of 599 participants to evaluate the user perception of vaccination certificates. We investigated five different variants of vaccination certificates, based on deployed and planned designs in a between-group design, including paper-based and app-based variants. Our main results show that the willingness to use and adopt vaccination certificates is generally high. Overall, paper-based vaccination certificates were favored over app-based solutions. The willingness to use digital apps decreased significantly by a higher disposition to privacy, and increased by higher worries about the pandemic and acceptance of the coronavirus vaccination. Vaccination certificates resemble an interesting use case for studying privacy perceptions for health related data. We hope that our work will be able to educate the currently ongoing design of vaccination certificates, will give us deeper insights into privacy of health-related data and apps, and prepare us for future potential applications of vaccination certificates and health apps in general.
Gradient-based Label Binning in Multi-label Classification
Venue: arXiv cs.LG 2021
Year: 2021

In multi-label classification, where a single example may be associated with several class labels at the same time, the ability to model dependencies between labels is considered crucial to effectively optimize non-decomposable evaluation measures, such as the Subset 0/1 loss. The gradient boosting framework provides a well-studied foundation for learning models that are specifically tailored to such a loss function and recent research attests the ability to achieve high predictive accuracy in the multi-label setting. The utilization of second-order derivatives, as used by many recent boosting approaches, helps to guide the minimization of non-decomposable losses, due to the information about pairs of labels it incorporates into the optimization process. On the downside, this comes with high computational costs, even if the number of labels is small. In this work, we address the computational bottleneck of such approach -- the need to solve a system of linear equations -- by integrating a novel approximation technique into the boosting procedure. Based on the derivatives computed during training, we dynamically group the labels into a predefined number of bins to impose an upper bound on the dimensionality of the linear system. Our experiments, using an existing rule-based algorithm, suggest that this may boost the speed of training, without any significant loss in predictive performance.
The Hitchhiker's Guide to Prior-Shift Adaptation
Venue: arXiv cs.CV 2021
Year: 2021

In many computer vision classification tasks, class priors at test time often differ from priors on the training set. In the case of such prior shift, classifiers must be adapted correspondingly to maintain close to optimal performance. This paper analyzes methods for adaptation of probabilistic classifiers to new priors and for estimating new priors on an unlabeled test set. We propose a novel method to address a known issue of prior estimation methods based on confusion matrices, where inconsistent estimates of decision probabilities and confusion matrices lead to negative values in the estimated priors. Experiments on fine-grained image classification datasets provide insight into the best practice of prior shift estimation and classifier adaptation, and show that the proposed method achieves state-of-the-art results in prior adaptation. Applying the best practice to two tasks with naturally imbalanced priors, learning from web-crawled images and plant species classification, increased the recognition accuracy by 1.1% and 3.4% respectively.
Categorising Fine-to-Coarse Grained Misinformation: An Empirical Study of COVID-19 Infodemic
Venue: arXiv cs.SI 2021
Year: 2021

The spreading COVID-19 misinformation over social media already draws the attention of many researchers. According to Google Scholar, about 26000 COVID-19 related misinformation studies have been published to date. Most of these studies focusing on 1) detect and/or 2) analysing the characteristics of COVID-19 related misinformation. However, the study of the social behaviours related to misinformation is often neglected. In this paper, we introduce a fine-grained annotated misinformation tweets dataset including social behaviours annotation (e.g. comment or question to the misinformation). The dataset not only allows social behaviours analysis but also suitable for both evidence-based or non-evidence-based misinformation classification task. In addition, we introduce leave claim out validation in our experiments and demonstrate the misinformation classification performance could be significantly different when applying to real-world unseen misinformation.
Digital Twins, Internet of Things and Mobile Medicine: a Review of Current Platforms to Support Smart Healthcare
Venue: arXiv cs.CY 2021
Year: 2021

As the population grows, the need for a quality level of medical services grows correspondingly, so does the demand for information technology in medicine. The concept of "Smart Healthcare" offers many approaches aimed at solving the acute problems faced by modern healthcare. In this paper, we review the main problems of modern healthcare, analyze existing approaches and technologies in the areas of digital twins, the Internet of Things and mobile medicine, determine their effectiveness in solving the set problems, consider the technologies that are used to monitor and treat patients and propose the concept of the Smart Healthcare platform.
Fully Dynamic Algorithms for Minimum Weight Cycle and Related Problems
Venue: arXiv cs.DS 2021
Year: 2021

We consider the directed minimum weight cycle problem in the fully dynamic setting. To the best of our knowledge, so far no fully dynamic algorithms have been designed specifically for the minimum weight cycle problem in general digraphs. One can achieve $\tilde{O}(n^2)$ amortized update time by simply invoking the fully dynamic APSP algorithm of Demetrescu and Italiano [J. ACM'04]. This bound, however, yields no improvement over the trivial recompute-from-scratch algorithm for sparse graphs. Our first contribution is a very simple deterministic $(1+ε)$-approximate algorithm supporting vertex updates (i.e., changing all edges incident to a specified vertex) in conditionally near-optimal $\tilde{O}(m\log{(W)}/ε)$ amortized time for digraphs with real edge weights in $[1,W]$. Using known techniques, the algorithm can be implemented on planar graphs and also gives some new sublinear fully dynamic algorithms maintaining approximate cuts and flows in planar digraphs. Additionally, we show a Monte Carlo randomized exact fully dynamic minimum weight cycle algorithm with $\tilde{O}(mn^{2/3})$ worst-case update that works for real edge weights. To this end, we generalize the exact fully dynamic APSP data structure of Abraham et al. [SODA'17] to solve the ``multiple-pairs shortest paths problem'', where one is interested in computing distances for some $k$ (instead of all $n^2$) fixed source-target pairs after each update. We show that in such a scenario, $\tilde{O}((m+k)n^{2/3})$ worst-case update time is possible.