OpenRaiser/Intern-Atlas huggingface dataset:
An Effective System for Multi-format Information Extraction
Venue: arXiv cs.CL 2021
Year: 2021

The multi-format information extraction task in the 2021 Language and Intelligence Challenge is designed to comprehensively evaluate information extraction from different dimensions. It consists of an multiple slots relation extraction subtask and two event extraction subtasks that extract events from both sentence-level and document-level. Here we describe our system for this multi-format information extraction competition task. Specifically, for the relation extraction subtask, we convert it to a traditional triple extraction task and design a voting based method that makes full use of existing models. For the sentence-level event extraction subtask, we convert it to a NER task and use a pointer labeling based method for extraction. Furthermore, considering the annotated trigger information may be helpful for event extraction, we design an auxiliary trigger recognition model and use the multi-task learning mechanism to integrate the trigger features into the event extraction model. For the document-level event extraction subtask, we design an Encoder-Decoder based method and propose a Transformer-alike decoder. Finally,our system ranks No.4 on the test set leader-board of this multi-format information extraction task, and its F1 scores for the subtasks of relation extraction, event extractions of sentence-level and document-level are 79.887%, 85.179%, and 70.828% respectively. The codes of our model are available at {https://github.com/neukg/MultiIE}.
Supply of engineering techniques and software design patterns in psychoanalysis and psychometrics sciences
Venue: arXiv cs.SE 2021
Year: 2021

The purpose of this study is to introduce software technologies and models and artificial intelligence algorithms to improve the weaknesses of CBT (Cognitive Behavior Therapy) method in psychotherapy. The presentation method for this purpose is the implementation of psychometric experiments in which the hidden human variables are inferred from the answers of tests. In this report, we describe the various models of Item Response Theory and measure the hidden components of ability and complementary parameters of the reality of the individual's situation. Psychometrics, selecting the appropriate model and estimating its parameters have been introduced and implemented using R language developed libraries. Due to the high flexibility of the Multi variant Rasch mixture Model, machine learning has been applied to this method of data modeling. BIC and CML were used to determine the number of hidden classes of the model and its parameters respectively, to obtain Measurement Invariance. The sensitivity of items to hidden attributes varies between groups (DIF), so methods for detecting it are introduced. This simulation is done based on the Verbal Aggression Dataset. We also analyze and compile a reference model based on this certificate based on the discovered patterns of software engineering. Other achievements of this study are related to providing a solution to explain the reengineering problems of the mind, by preparing an identity card for the clients by an ontology. Finally, applying the developed knowledge in the form of system thinking and recommended patterns in software engineering during the treatment process is pointed out.
State Complexity of Projection on Languages Recognized by Permutation Automata and Commuting Letters
Venue: arXiv cs.FL 2021
Year: 2021

The projected language of a general deterministic automaton with $n$ states is recognizable by a deterministic automaton with $2^{n-1} + 2^{n-m} - 1$ states, where $m$ denotes the number of states incident to unobservable non-loop transitions, and this bound is best possible. Here, we derive the tight bound $2^{n - \lceil \frac{m}{2} \rceil} - 1$ for permutation automata. For a state-partition automaton with $n$ states (also called automata with the observer property) the projected language is recognizable with $n$ states. Up to now, these, and finite languages projected onto unary languages, were the only classes of automata known to possess this property. We show that this is also true for commutative automata and we find commutative automata that are not state-partition automata.
One Time Pad and the Short Key Dream
Venue: arXiv cs.CR 2021
Year: 2021

This is a survey on the One Time Pad (OTP) and its derivatives, from its origins to modern times. OTP, if used correctly, is (the only) cryptographic code that no computing power, present or future, can break. Naturally, the discussion shifts to the creation of long random sequences, starting from short ones, which can be easily shared. We could call it the Short Key Dream. Many problems inevitably arise, which affect many fields of computer science, mathematics and knowledge in general. This work presents a vast bibliography that includes fundamental classical works and current papers on randomness, pseudorandom number generators, compressibility, unpredictability and more.
On the performance of GPU accelerated q-LSKUM based meshfree solvers in Fortran, C++, Python, and Julia
Venue: arXiv cs.PL 2021
Year: 2021

This report presents a comprehensive analysis of the performance of GPU accelerated meshfree CFD solvers for two-dimensional compressible flows in Fortran, C++, Python, and Julia. The programming model CUDA is used to develop the GPU codes. The meshfree solver is based on the least squares kinetic upwind method with entropy variables (q-LSKUM). To assess the computational efficiency of the GPU solvers and to compare their relative performance, benchmark calculations are performed on seven levels of point distribution. To analyse the difference in their run-times, the computationally intensive kernel is profiled. Various performance metrics are investigated from the profiled data to determine the cause of observed variation in run-times. To address some of the performance related issues, various optimisation strategies are employed. The optimised GPU codes are compared with the naive codes, and conclusions are drawn from their performance.
Exploring Transferable and Robust Adversarial Perturbation Generation from the Perspective of Network Hierarchy
Venue: arXiv cs.CV 2021
Year: 2021

The transferability and robustness of adversarial examples are two practical yet important properties for black-box adversarial attacks. In this paper, we explore effective mechanisms to boost both of them from the perspective of network hierarchy, where a typical network can be hierarchically divided into output stage, intermediate stage and input stage. Since over-specialization of source model, we can hardly improve the transferability and robustness of the adversarial perturbations in the output stage. Therefore, we focus on the intermediate and input stages in this paper and propose a transferable and robust adversarial perturbation generation (TRAP) method. Specifically, we propose the dynamically guided mechanism to continuously calculate accurate directional guidances for perturbation generation in the intermediate stage. In the input stage, instead of the single-form transformation augmentations adopted in the existing methods, we leverage multiform affine transformation augmentations to further enrich the input diversity and boost the robustness and transferability of the adversarial perturbations. Extensive experiments demonstrate that our TRAP achieves impressive transferability and high robustness against certain interferences.
WiseR: An end-to-end structure learning and deployment framework for causal graphical models
Venue: arXiv cs.LG 2021
Year: 2021

Structure learning offers an expressive, versatile and explainable approach to causal and mechanistic modeling of complex biological data. We present wiseR, an open source application for learning, evaluating and deploying robust causal graphical models using graph neural networks and Bayesian networks. We demonstrate the utility of this application through application on for biomarker discovery in a COVID-19 clinical dataset.
Toward the Understanding of Deep Text Matching Models for Information Retrieval
Venue: arXiv cs.IR 2021
Year: 2021

Semantic text matching is a critical problem in information retrieval. Recently, deep learning techniques have been widely used in this area and obtained significant performance improvements. However, most models are black boxes and it is hard to understand what happened in the matching process, due to the poor interpretability of deep learning. This paper aims at tackling this problem. The key idea is to test whether existing deep text matching methods satisfy some fundamental heuristics in information retrieval. Specifically, four heuristics are used in our study, i.e., term frequency constraint, term discrimination constraint, length normalization constraints, and TF-length constraint. Since deep matching models usually contain many parameters, it is difficult to conduct a theoretical study for these complicated functions. In this paper, We propose an empirical testing method. Specifically, We first construct some queries and documents to make them satisfy the assumption in a constraint, and then test to which extend a deep text matching model trained on the original dataset satisfies the corresponding constraint. Besides, a famous attribution based interpretation method, namely integrated gradient, is adopted to conduct detailed analysis and guide for feasible improvement. Experimental results on LETOR 4.0 and MS Marco show that all the investigated deep text matching methods, both representation and interaction based methods, satisfy the above constraints with high probabilities in statistics. We further extend these constraints to the semantic settings, which are shown to be better satisfied for all the deep text matching models. These empirical findings give clear understandings on why deep text matching models usually perform well in information retrieval. We believe the proposed evaluation methodology will be useful for testing future deep text matching models.
Real-time Human-Centric Segmentation for Complex Video Scenes
Venue: arXiv cs.CV 2021
Year: 2021

Most existing video tasks related to "human" focus on the segmentation of salient humans, ignoring the unspecified others in the video. Few studies have focused on segmenting and tracking all humans in a complex video, including pedestrians and humans of other states (e.g., seated, riding, or occluded). In this paper, we propose a novel framework, abbreviated as HVISNet, that segments and tracks all presented people in given videos based on a one-stage detector. To better evaluate complex scenes, we offer a new benchmark called HVIS (Human Video Instance Segmentation), which comprises 1447 human instance masks in 805 high-resolution videos in diverse scenes. Extensive experiments show that our proposed HVISNet outperforms the state-of-the-art methods in terms of accuracy at a real-time inference speed (30 FPS), especially on complex video scenes. We also notice that using the center of the bounding box to distinguish different individuals severely deteriorates the segmentation accuracy, especially in heavily occluded conditions. This common phenomenon is referred to as the ambiguous positive samples problem. To alleviate this problem, we propose a mechanism named Inner Center Sampling to improve the accuracy of instance segmentation. Such a plug-and-play inner center sampling mechanism can be incorporated in any instance segmentation models based on a one-stage detector to improve the performance. In particular, it gains 4.1 mAP improvement on the state-of-the-art method in the case of occluded humans. Code and data are available at https://github.com/IIGROUP/HVISNet.
Multihop RIS-Assisted FSO-RF System Over Double Generalized Gamma Fading
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable intelligent surface (RIS) is a promising technology to avoid signal blockage by creating virtual line-of-sight (LOS) connectivity for free-space optical (FSO) and radio frequency (RF) wireless systems. This paper considers a mixed FSO-RF system by employing multiple RISs in both the links for multihop transmissions to extend the communication range. We develop probability density function (PDF) and cumulative density function (CDF) of the signal-to-noise ratio (SNR) for the cascaded channels by considering double generalized gamma (dGG) turbulence with pointing errors for the FSO link and the dGG distribution to model the signal fading for the RF. We derive exact closed-form expressions of the outage probability, average bit-error-rate (BER), and ergodic capacity using the decode-and-forward (DF) relaying for the mixed system. We also present asymptotic analysis on the performance in the high SNR regime depicting the impact of channel parameters on the diversity order of the system. We use computer simulations to demonstrate the effect of system and channel parameters on the RIS-aided multihop transmissions.
Weakly Supervised Classification Using Group-Level Labels
Venue: arXiv cs.LG 2021
Year: 2021

In many applications, finding adequate labeled data to train predictive models is a major challenge. In this work, we propose methods to use group-level binary labels as weak supervision to train instance-level binary classification models. Aggregate labels are common in several domains where annotating on a group-level might be cheaper or might be the only way to provide annotated data without infringing on privacy. We model group-level labels as Class Conditional Noisy (CCN) labels for individual instances and use the noisy labels to regularize predictions of the model trained on the strongly-labeled instances. Our experiments on real-world application of land cover mapping shows the utility of the proposed method in leveraging group-level labels, both in the presence and absence of class imbalance.
Private Delivery Networks -- Extended Abstract
Venue: arXiv cs.CY 2021
Year: 2021

The past decade has seen tremendous shifts in how people live, work, and buy goods, with an increased reliance on e-commerce and deliveries. Purchase histories generated through e-commerce can be highly personal, revealing identifying information about individuals and households. Constructing profiles from these data allows for the targeting of individuals and communities through practices such as targeted marketing and information campaigns. Furthermore, when purchase profiles are connected with delivery addresses, these data can measure the demographics of a local community and allow for individualized targeting to reach beyond the digital realm to the physical one. Events that accelerated shifts towards e-commerce, such as an infectious disease epidemic, have also widened equity gaps. This work is about alternative e-commerce delivery network models that address both rising privacy and wealth inequality concerns. This includes strategies that mask and add noise to purchase histories, and allow people to "buy privacy" through charitable contributions.
Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards
Venue: arXiv cs.DB 2021
Year: 2021

Developing documentation guidelines and easy-to-use templates for datasets and models is a challenging task, especially given the variety of backgrounds, skills, and incentives of the people involved in the building of natural language processing (NLP) tools. Nevertheless, the adoption of standard documentation practices across the field of NLP promotes more accessible and detailed descriptions of NLP datasets and models, while supporting researchers and developers in reflecting on their work. To help with the standardization of documentation, we present two case studies of efforts that aim to develop reusable documentation templates -- the HuggingFace data card, a general purpose card for datasets in NLP, and the GEM benchmark data and model cards with a focus on natural language generation. We describe our process for developing these templates, including the identification of relevant stakeholder groups, the definition of a set of guiding principles, the use of existing templates as our foundation, and iterative revisions based on feedback.
ANOMALYMAXQ:Anomaly-Structured Maximization to Query in Attributed Network
Venue: arXiv cs.DS 2021
Year: 2021

The detection of anomaly subgraphs naturally appears in various real-life tasks, yet label noise seriously interferes with the result. As a motivation for our work, we focus on inaccurate supervision and use prior knowledge to reduce effects of noise, like query graphs. Anomalies in attributed networks exhibit structured-properties, e.g., anomaly in money laundering with "ring structure" property. It is the main challenge to fast and approximate query anomaly in attributed networks. We propose a novel search method: 1) decomposing a query graph into stars; 2) sorting attributed vertices; and 3) assembling anomaly stars under the root vertex sequence into near query. We present ANOMALYMAXQ and perform on 68,411 company network (Tianyancha dataset),7.72m patent networks (Company patents) and so on. Extensive experiments show that our method has high robustness and fast response time. When running the patent dataset,the average running time to query the graph once is about 252 seconds.
RRLFSOR: An Efficient Self-Supervised Learning Strategy of Graph Convolutional Networks
Venue: arXiv cs.LG 2021
Year: 2021

Graph Convolutional Networks (GCNs) are widely used in many applications yet still need large amounts of labelled data for training. Besides, the adjacency matrix of GCNs is stable, which makes the data processing strategy cannot efficiently adjust the quantity of training data from the built graph structures.To further improve the performance and the self-learning ability of GCNs,in this paper, we propose an efficient self-supervised learning strategy of GCNs,named randomly removed links with a fixed step at one region (RRLFSOR).RRLFSOR can be regarded as a new data augmenter to improve over-smoothing.RRLFSOR is examined on two efficient and representative GCN models with three public citation network datasets-Cora,PubMed,and Citeseer.Experiments on transductive link prediction tasks show that our strategy outperforms the baseline models consistently by up to 21.34% in terms of accuracy on three benchmark datasets.
SPMoE: Generate Multiple Pattern-Aware Outputs with Sparse Pattern Mixture of Experts
Venue: arXiv cs.CL 2021
Year: 2021

Many generation tasks follow a one-to-many mapping relationship: each input could be associated with multiple outputs. Existing methods like Conditional Variational AutoEncoder(CVAE) employ a latent variable to model this one-to-many relationship. However, this high-dimensional and dense latent variable lacks explainability and usually leads to poor and uncontrollable generations. In this paper, we innovatively introduce the linguistic concept of pattern to decompose the one-to-many mapping into multiple one-to-one mappings and further propose a model named Sparse Pattern Mixture of Experts(SPMoE). Each one-to-one mapping is associated with a conditional generation pattern and is modeled with an expert in SPMoE. To ensure each language pattern can be exclusively handled with an expert model for better explainability and diversity, a sparse mechanism is employed to coordinate all the expert models in SPMoE. We assess the performance of our SPMoE on the paraphrase generation task and the experiment results prove that SPMoE can achieve a good balance in terms of quality, pattern-level diversity, and corpus-level diversity.
O-HAS: Optical Hardware Accelerator Search for Boosting Both Acceleration Performance and Development Speed
Venue: arXiv cs.LG 2021
Year: 2021

The recent breakthroughs and prohibitive complexities of Deep Neural Networks (DNNs) have excited extensive interest in domain-specific DNN accelerators, among which optical DNN accelerators are particularly promising thanks to their unprecedented potential of achieving superior performance-per-watt. However, the development of optical DNN accelerators is much slower than that of electrical DNN accelerators. One key challenge is that while many techniques have been developed to facilitate the development of electrical DNN accelerators, techniques that support or expedite optical DNN accelerator design remain much less explored, limiting both the achievable performance and the innovation development of optical DNN accelerators. To this end, we develop the first-of-its-kind framework dubbed O-HAS, which for the first time demonstrates automated Optical Hardware Accelerator Search for boosting both the acceleration efficiency and development speed of optical DNN accelerators. Specifically, our O-HAS consists of two integrated enablers: (1) an O-Cost Predictor, which can accurately yet efficiently predict an optical accelerator's energy and latency based on the DNN model parameters and the optical accelerator design; and (2) an O-Search Engine, which can automatically explore the large design space of optical DNN accelerators and identify the optimal accelerators (i.e., the micro-architectures and algorithm-to-accelerator mapping methods) in order to maximize the target acceleration efficiency. Extensive experiments and ablation studies consistently validate the effectiveness of both our O-Cost Predictor and O-Search Engine as well as the excellent efficiency of O-HAS generated optical accelerators.
A heuristic for listing almost-clique minimal separators of a graph
Venue: arXiv cs.DS 2021
Year: 2021

Bodlaender and Koster (Discrete Mathematics 2006) introduced the notion of almost-clique separators in the context of computing the treewidth $\tw(G)$ of a given graph $G$. A separator $S \subseteq V(G)$ of $G$ is an \emph{almost-clique separator} if $S \setminus \{v\}$ is a clique of $G$ for some $v \in S$. $S$ is a \emph{minimal separator} if $S$ has at least two full components, where a full component of $S$ is a connected component $C$ of $G \setminus S$ such that $N_G(C) = S$. They observed that if $S$ is an almost-clique minimal separator of $G$ then $\tw(G \cup K(S)) = \tw(G)$, where $K(S)$ is the complete graph on vertex set $S$: in words, filling an almost-clique minimal separator into a clique does not increase the treewidth. Based on this observation, they proposed a preprocessing method for treewidth computation, a fundamental step of which is to find a preferably maximal set of pairwise non-crossing almost-clique minimal separators of a graph. In this paper, we present a heuristic for this step, which is based on the following empirical observation. For graph $G$ and a minimal triangulation $H$ of $G$, let $\QQ(H, G)$ denote the set of all almost-clique minimal separators of $G$ that are minimal separators of $H$. Note that since the minimal separators of $H$ are pairwise non-crossing, so are those in $\QQ(H, G)$. We observe from experiments that $\QQ(H, G)$ is remarkably close to maximal, especially when the minimal triangulation $H$ is computed by an algorithm aiming for small treewidth. This observation leads to an efficient implementation of the preprocessing method proposed by Bodlaender and Koster. Experiments on instances from PACE 2017 and other sources show that this implementation is extremely fast and effective for graphs of practical interest.