OpenRaiser/Intern-Atlas huggingface dataset:
Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks
Venue: arXiv cs.CL 2021
Year: 2021

In zero-shot cross-lingual transfer, a supervised NLP task trained on a corpus in one language is directly applicable to another language without any additional training. A source of cross-lingual transfer can be as straightforward as lexical overlap between languages (e.g., use of the same scripts, shared subwords) that naturally forces text embeddings to occupy a similar representation space. Recently introduced cross-lingual language model (XLM) pretraining brings out neural parameter sharing in Transformer-style networks as the most important factor for the transfer. In this paper, we aim to validate the hypothetically strong cross-lingual transfer properties induced by XLM pretraining. Particularly, we take XLM-RoBERTa (XLMR) in our experiments that extend semantic textual similarity (STS), SQuAD and KorQuAD for machine reading comprehension, sentiment analysis, and alignment of sentence embeddings under various cross-lingual settings. Our results indicate that the presence of cross-lingual transfer is most pronounced in STS, sentiment analysis the next, and MRC the last. That is, the complexity of a downstream task softens the degree of crosslingual transfer. All of our results are empirically observed and measured, and we make our code and data publicly available.
Software Effort Estimation Accuracy Prediction of Machine Learning Techniques: A Systematic Performance Evaluation
Venue: arXiv cs.SE 2021
Year: 2021

Software effort estimation accuracy is a key factor in effective planning, controlling and to deliver a successful software project within budget and schedule. The overestimation and underestimation both are the key challenges for future software development, henceforth there is a continuous need for accuracy in software effort estimation (SEE). The researchers and practitioners are striving to identify which machine learning estimation technique gives more accurate results based on evaluation measures, datasets and the other relevant attributes. The authors of related research are generally not aware of previously published results of machine learning effort estimation techniques. The main aim of this study is to assist the researchers to know which machine learning technique yields the promising effort estimation accuracy prediction in the software development. In this paper, the performance of the machine learning ensemble technique is investigated with the solo technique based on two most commonly used accuracy evaluation metrics. We used the systematic literature review methodology proposed by Kitchenham and Charters. This includes searching for the most relevant papers, applying quality assessment criteria, extracting data and drawing results. We have evaluated a state-of-the-art accuracy performance of 28 selected studies (14 ensemble, 14 solo) using Mean Magnitude of Relative Error (MMRE) and PRED (25) as a set of reliable accuracy metrics for performance evaluation of accuracy among two techniques to report the research questions stated in this study. We found that machine learning techniques are the most frequently implemented in the construction of ensemble effort estimation (EEE) techniques. The results of this study revealed that the EEE techniques usually yield a promising estimation accuracy than the solo techniques.
Channel Matching: An Adaptive Technique to Increase the Accuracy of Soft Decisions
Venue: arXiv cs.IT 2021
Year: 2021

Nonlinear interference is modeled by a time-varying conditionally Gaussian channel. It is shown that approximating this channel with a time-invariant channel imposes considerable loss in the performance of channel decoding. An adaptive method to maintain decoding performance is described.
Few-Shot Semantic Parsing for New Predicates
Venue: arXiv cs.CL 2021
Year: 2021

In this work, we investigate the problems of semantic parsing in a few-shot learning setting. In this setting, we are provided with utterance-logical form pairs per new predicate. The state-of-the-art neural semantic parsers achieve less than 25% accuracy on benchmark datasets when k= 1. To tackle this problem, we proposed to i) apply a designated meta-learning method to train the model; ii) regularize attention scores with alignment statistics; iii) apply a smoothing technique in pre-training. As a result, our method consistently outperforms all the baselines in both one and two-shot settings.
Model-free Data-Driven simulation of inelastic materials using structured data sets, tangent space information and transition rules
Venue: arXiv cs.CE 2021
Year: 2021

Model-free data-driven computational mechanics replaces phenomenological constitutive functions by numerical simulations based on data sets of representative samples in stress-strain space. The distance of strain and stress pairs from the data set is minimized, subject to equilibrium and compatibility constraints. Although this method operates well for non-linear elastic problems, there are challenges dealing with history-dependent materials, since one and the same point in stress-strain space might correspond to different material behaviour.In recent literature, this issue has been treated by including local histories into the data set. However, there is still the necessity to include models for the evolution of specific internal variables. Thus, a mixed formulation of classical and data-driven modeling is obtained. In the presented approach, the data set is augmented with directions in the tangent space of points in stress-strain space. Moreover, the data set is divided into subsets corresponding to different material behaviour. Based on this classification, transition rules map the modeling points to the various subsets. The approach will be applied to non-linear elasticity and elasto-plasticity with isotropic hardening.
Automatic Detection of Causality in Requirement Artifacts: the CiRA Approach
Venue: arXiv cs.SE 2021
Year: 2021

System behavior is often expressed by causal relations in requirements (e.g., If event 1, then event 2). Automatically extracting this embedded causal knowledge supports not only reasoning about requirements dependencies, but also various automated engineering tasks such as seamless derivation of test cases. However, causality extraction from natural language is still an open research challenge as existing approaches fail to extract causality with reasonable performance. We understand causality extraction from requirements as a two-step problem: First, we need to detect if requirements have causal properties or not. Second, we need to understand and extract their causal relations. At present, though, we lack knowledge about the form and complexity of causality in requirements, which is necessary to develop a suitable approach addressing these two problems. We conduct an exploratory case study with 14,983 sentences from 53 requirements documents originating from 18 different domains and shed light on the form and complexity of causality in requirements. Based on our findings, we develop a tool-supported approach for causality detection (CiRA). This constitutes a first step towards causality extraction from NL requirements. We report on a case study and the resulting tool-supported approach for causality detection in requirements. Our case study corroborates, among other things, that causality is, in fact, a widely used linguistic pattern to describe system behavior, as about a third of the analyzed sentences are causal. We further demonstrate that our tool CiRA achieves a macro-F1 score of 82 % on real word data and that it outperforms related approaches with an average gain of 11.06 % in macro-Recall and 11.43 % in macro-Precision. Finally, we disclose our open data sets as well as our tool to foster the discourse on the automatic detection of causality in the RE community.
Semi-synthesis: A fast way to produce effective datasets for stereo matching
Venue: arXiv cs.CV 2021
Year: 2021

Stereo matching is an important problem in computer vision which has drawn tremendous research attention for decades. Recent years, data-driven methods with convolutional neural networks (CNNs) are continuously pushing stereo matching to new heights. However, data-driven methods require large amount of training data, which is not an easy task for real stereo data due to the annotation difficulties of per-pixel ground-truth disparity. Though synthetic dataset is proposed to fill the gaps of large data demand, the fine-tuning on real dataset is still needed due to the domain variances between synthetic data and real data. In this paper, we found that in synthetic datasets, close-to-real-scene texture rendering is a key factor to boost up stereo matching performance, while close-to-real-scene 3D modeling is less important. We then propose semi-synthetic, an effective and fast way to synthesize large amount of data with close-to-real-scene texture to minimize the gap between synthetic data and real data. Extensive experiments demonstrate that models trained with our proposed semi-synthetic datasets achieve significantly better performance than with general synthetic datasets, especially on real data benchmarks with limited training data. With further fine-tuning on the real dataset, we also achieve SOTA performance on Middlebury and competitive results on KITTI and ETH3D datasets.
Intelligent Routing to Enhance Energy Consumption in Wireless Sensor Network: A survey
Venue: arXiv cs.NI 2021
Year: 2021

Nowadays, the network and the Internet applications have gained a substantial importance in the digital world, due to the great impact which it provides for health and community services. Among the most important services that have been provided are smart devices and vital factor measurement devices for patients, whether in a hospital or outside the hospital. Furthermore, sensors that collect medical data or measurements of temperature and humidity, in various critical environments. The proper types of network that may be used in such difficult environment are Wireless Sensor Networks, that used to sense and process data. Additionally, the Wireless Sensor Networks have been used in the environment of Internet of Things and smart cities in the general services and health fields. All these reasons have made researchers focus on Wireless Sensor Networks and addressing the challenges that face them. The most important challenge facing this type of network is energy consumption and increase battery life. This paper discusses the methodologies used in energy conservation in Wireless Sensor Networks, such as data reduction technology, shortest path selection and artificial intelligence algorithms used in smart routing and energy saving. Besides, we have introduced comparisons between the standard algorithms which are suggested by the researchers, to make a clear picture of the energy consumption problem and propose some effective solutions in Wireless Sensor Networks field.
Propagation and reconstruction of re-entry uncertainties using continuity equation and simplicial interpolation
Venue: arXiv cs.CE 2021
Year: 2021

This work proposes a continuum-based approach for the propagation of uncertainties in the initial conditions and parameters for the analysis and prediction of spacecraft re-entries. Using the continuity equation together with the re-entry dynamics, the joint probability distribution of the uncertainties is propagated in time for specific sampled points. At each time instant, the joint probability distribution function is then reconstructed from the scattered data using a gradient-enhanced linear interpolation based on a simplicial representation of the state space. Uncertainties in the initial conditions at re-entry and in the ballistic coefficient for three representative test cases are considered: a three-state and a six-state steep Earth re-entry and a six-state unguided lifting entry at Mars. The paper shows the comparison of the proposed method with Monte Carlo based techniques in terms of quality of the obtained marginal distributions and runtime as a function of the number of samples used.
Spark NLP: Natural Language Understanding at Scale
Venue: arXiv cs.CL 2021
Year: 2021

Spark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant and accurate NLP annotations for machine learning pipelines that can scale easily in a distributed environment. Spark NLP comes with 1100 pre trained pipelines and models in more than 192 languages. It supports nearly all the NLP tasks and modules that can be used seamlessly in a cluster. Downloaded more than 2.7 million times and experiencing nine times growth since January 2020, Spark NLP is used by 54% of healthcare organizations as the worlds most widely used NLP library in the enterprise.
White Paper: Challenges and Considerations for the Creation of a Large Labelled Repository of Online Videos with Questionable Content
Venue: arXiv cs.CV 2021
Year: 2021

This white paper presents a summary of the discussions regarding critical considerations to develop an extensive repository of online videos annotated with labels indicating questionable content. The main discussion points include: 1) the type of appropriate labels that will result in a valuable repository for the larger AI community; 2) how to design the collection and annotation process, as well as the distribution of the corpus to maximize its potential impact; and, 3) what actions we can take to reduce risk of trauma to annotators.
Exploring Rulial Space: The Case of Turing Machines
Venue: arXiv cs.DM 2021
Year: 2021

As an example of the concept of rulial space, we explore the case of simple Turing machines. We construct the rulial multiway graph which represents the behavior of all possible Turing machines with a certain class of rules. This graph (which is a Cayley graph of a "Turing machine group") gives a map of the space of non-deterministic Turing machines. We investigate the subgraph formed by deterministic machines, and explore the relationship to the P vs. NP problem. We also consider the implications of features of rulial space for physics, including estimating the maximum speed \r{ho} in rulial space, relations between rulial black holes and computational reducibility, and interpretations of hypercomputation.
A Blockchain-based Trust System for Decentralised Applications: When trustless needs trust
Venue: arXiv cs.CR 2021
Year: 2021

Blockchain technology has been envisaged to commence an era of decentralised applications and services (DApps) without the need for a trusted intermediary. Such DApps open a marketplace in which services are delivered to end-users by contributors which are then incentivised by cryptocurrencies in an automated, peer-to-peer, and trustless fashion. However, blockchain, consolidated by smart contracts, only ensures on-chain data security, autonomy and integrity of the business logic execution defined in smart contracts. It cannot guarantee the quality of service of DApps, which entirely depends on the services' performance. Thus, there is a critical need for a trust system to reduce the risk of dealing with fraudulent counterparts in a blockchain network. These reasons motivate us to develop a fully decentralised trust framework deployed on top of a blockchain platform, operating along with DApps in the marketplace to demoralise deceptive entities while encouraging trustworthy ones. The trust system works as an underlying decentralised service providing a feedback mechanism for end-users and maintaining trust relationships among them in the ecosystem accordingly. We believe this research fortifies the DApps ecosystem by introducing an universal trust middleware for DApps as well as shedding light on the implementation of a decentralised trust system.
RewardRating: A Mechanism Design Approach to Improve Rating Systems
Venue: arXiv cs.GT 2021
Year: 2021

Nowadays, rating systems play a crucial role in the attraction of customers for different services. However, as it is difficult to detect a fake rating, attackers can potentially impact the rating's aggregated score unfairly. This malicious behavior can negatively affect users and businesses. To overcome this problem, we take a mechanism-design approach to increase the cost of fake ratings while providing incentives for honest ratings. Our proposed mechanism \textit{RewardRating} is inspired by the stock market model in which users can invest in their ratings for services and receive a reward based on future ratings. First, we formally model the problem and discuss budget-balanced and incentive-compatibility specifications. Then, we suggest a profit-sharing scheme to cover the rating system's requirements. Finally, we analyze the performance of our proposed mechanism.
RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content
Venue: arXiv cs.CV 2021
Year: 2021

Blind or no-reference video quality assessment of user-generated content (UGC) has become a trending, challenging, heretofore unsolved problem. Accurate and efficient video quality predictors suitable for this content are thus in great demand to achieve more intelligent analysis and processing of UGC videos. Previous studies have shown that natural scene statistics and deep learning features are both sufficient to capture spatial distortions, which contribute to a significant aspect of UGC video quality issues. However, these models are either incapable or inefficient for predicting the quality of complex and diverse UGC videos in practical applications. Here we introduce an effective and efficient video quality model for UGC content, which we dub the Rapid and Accurate Video Quality Evaluator (RAPIQUE), which we show performs comparably to state-of-the-art (SOTA) models but with orders-of-magnitude faster runtime. RAPIQUE combines and leverages the advantages of both quality-aware scene statistics features and semantics-aware deep convolutional features, allowing us to design the first general and efficient spatial and temporal (space-time) bandpass statistics model for video quality modeling. Our experimental results on recent large-scale UGC video quality databases show that RAPIQUE delivers top performances on all the datasets at a considerably lower computational expense. We hope this work promotes and inspires further efforts towards practical modeling of video quality problems for potential real-time and low-latency applications. To promote public usage, an implementation of RAPIQUE has been made freely available online: \url{https://github.com/vztu/RAPIQUE}.
Supervised Momentum Contrastive Learning for Few-Shot Classification
Venue: arXiv cs.CV 2021
Year: 2021

Few-shot learning aims to transfer information from one task to enable generalization on novel tasks given a few examples. This information is present both in the domain and the class labels. In this work we investigate the complementary roles of these two sources of information by combining instance-discriminative contrastive learning and supervised learning in a single framework called Supervised Momentum Contrastive learning (SUPMOCO). Our approach avoids a problem observed in supervised learning where information in images not relevant to the task is discarded, which hampers their generalization to novel tasks. We show that (self-supervised) contrastive learning and supervised learning are mutually beneficial, leading to a new state-of-the-art on the META-DATASET - a recently introduced benchmark for few-shot learning. Our method is based on a simple modification of MOCO and scales better than prior work on combining supervised and self-supervised learning. This allows us to easily combine data from multiple domains leading to further improvements.
The Effect of Class Definitions on the Transferability of Adversarial Attacks Against Forensic CNNs
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, convolutional neural networks (CNNs) have been widely used by researchers to perform forensic tasks such as image tampering detection. At the same time, adversarial attacks have been developed that are capable of fooling CNN-based classifiers. Understanding the transferability of adversarial attacks, i.e. an attacks ability to attack a different CNN than the one it was trained against, has important implications for designing CNNs that are resistant to attacks. While attacks on object recognition CNNs are believed to be transferrable, recent work by Barni et al. has shown that attacks on forensic CNNs have difficulty transferring to other CNN architectures or CNNs trained using different datasets. In this paper, we demonstrate that adversarial attacks on forensic CNNs are even less transferrable than previously thought even between virtually identical CNN architectures! We show that several common adversarial attacks against CNNs trained to identify image manipulation fail to transfer to CNNs whose only difference is in the class definitions (i.e. the same CNN architectures trained using the same data). We note that all formulations of class definitions contain the unaltered class. This has important implications for the future design of forensic CNNs that are robust to adversarial and anti-forensic attacks.
Automatic Comic Generation with Stylistic Multi-page Layouts and Emotion-driven Text Balloon Generation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a fully automatic system for generating comic books from videos without any human intervention. Given an input video along with its subtitles, our approach first extracts informative keyframes by analyzing the subtitles, and stylizes keyframes into comic-style images. Then, we propose a novel automatic multi-page layout framework, which can allocate the images across multiple pages and synthesize visually interesting layouts based on the rich semantics of the images (e.g., importance and inter-image relation). Finally, as opposed to using the same type of balloon as in previous works, we propose an emotion-aware balloon generation method to create different types of word balloons by analyzing the emotion of subtitles and audios. Our method is able to vary balloon shapes and word sizes in balloons in response to different emotions, leading to more enriched reading experience. Once the balloons are generated, they are placed adjacent to their corresponding speakers via speaker detection. Our results show that our method, without requiring any user inputs, can generate high-quality comic pages with visually rich layouts and balloons. Our user studies also demonstrate that users prefer our generated results over those by state-of-the-art comic generation systems.
Self-stabilizing Algorithm for Maximal Distance-2 Independent Set
Venue: arXiv cs.DC 2021
Year: 2021

In graph theory, an independent set is a subset of nodes where there are no two adjacent nodes. The independent set is maximal if no node outside the independent set can join it. In network applications, maximal independent sets can be used as cluster heads in ad hoc and wireless sensor networks. In order to deal with any failure in networks, self-stabilizing algorithms have been proposed in the literature to calculate the maximal independent set under different hypotheses. In this paper, we propose a self-stabilizing algorithm to compute a maximal independent set where nodes of the independent set are far from each other at least with distance 3. We prove the correctness and the convergence of the proposed algorithm. Simulation tests show the ability of our algorithm to find a reduced number of nodes in large scale networks which allows strong control of networks