OpenRaiser/Intern-Atlas huggingface dataset:
Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Information
Venue: arXiv cs.CL 2021
Year: 2021

Context-aware neural machine translation (NMT) incorporates contextual information of surrounding texts, that can improve the translation quality of document-level machine translation. Many existing works on context-aware NMT have focused on developing new model architectures for incorporating additional contexts and have shown some promising results. However, most existing works rely on cross-entropy loss, resulting in limited use of contextual information. In this paper, we propose CorefCL, a novel data augmentation and contrastive learning scheme based on coreference between the source and contextual sentences. By corrupting automatically detected coreference mentions in the contextual sentence, CorefCL can train the model to be sensitive to coreference inconsistency. We experimented with our method on common context-aware NMT models and two document-level translation tasks. In the experiments, our method consistently improved BLEU of compared models on English-German and English-Korean tasks. We also show that our method significantly improves coreference resolution in the English-German contrastive test suite.
Fine-Grained Few Shot Learning with Foreground Object Transformation
Venue: arXiv cs.CV 2021
Year: 2021

Traditional fine-grained image classification generally requires abundant labeled samples to deal with the low inter-class variance but high intra-class variance problem. However, in many scenarios we may have limited samples for some novel sub-categories, leading to the fine-grained few shot learning (FG-FSL) setting. To address this challenging task, we propose a novel method named foreground object transformation (FOT), which is composed of a foreground object extractor and a posture transformation generator. The former aims to remove image background, which tends to increase the difficulty of fine-grained image classification as it amplifies the intra-class variance while reduces inter-class variance. The latter transforms the posture of the foreground object to generate additional samples for the novel sub-category. As a data augmentation method, FOT can be conveniently applied to any existing few shot learning algorithm and greatly improve its performance on FG-FSL tasks. In particular, in combination with FOT, simple fine-tuning baseline methods can be competitive with the state-of-the-art methods both in inductive setting and transductive setting. Moreover, FOT can further boost the performances of latest excellent methods and bring them up to the new state-of-the-art. In addition, we also show the effectiveness of FOT on general FSL tasks.
On the Age of Information of a Queuing System with Heterogeneous Servers
Venue: arXiv cs.IT 2021
Year: 2021

An optimal control problem with heterogeneous servers to minimize the average age of information (AoI) is considered. Each server maintains a separate queue, and each packet arriving to the system is randomly routed to one of the servers. Assuming Poisson arrivals and exponentially distributed service times, we first derive an exact expression of the average AoI for two heterogeneous servers. Next, to solve for the optimal average AoI, a close approximation is derived, called the approximate AoI, this is shown to be useful for multi-server systems as well. We show that for the optimal approximate AoI, server utilization (ratio of arrival rate and service rate) for each server should be same as the optimal server utilization with a single server queue. For two identical servers, it is shown that the average AoI is approximately 5/8 times the average AoI of a single server. Furthermore, the average AoI is shown to decrease considerably with the addition of more servers to the system.
Perturbation CheckLists for Evaluating NLG Evaluation Metrics
Venue: arXiv cs.CL 2021
Year: 2021

Natural Language Generation (NLG) evaluation is a multifaceted task requiring assessment of multiple desirable criteria, e.g., fluency, coherency, coverage, relevance, adequacy, overall quality, etc. Across existing datasets for 6 NLG tasks, we observe that the human evaluation scores on these multiple criteria are often not correlated. For example, there is a very low correlation between human scores on fluency and data coverage for the task of structured data to text generation. This suggests that the current recipe of proposing new automatic evaluation metrics for NLG by showing that they correlate well with scores assigned by humans for a single criteria (overall quality) alone is inadequate. Indeed, our extensive study involving 25 automatic evaluation metrics across 6 different tasks and 18 different evaluation criteria shows that there is no single metric which correlates well with human scores on all desirable criteria, for most NLG tasks. Given this situation, we propose CheckLists for better design and evaluation of automatic metrics. We design templates which target a specific criteria (e.g., coverage) and perturb the output such that the quality gets affected only along this specific criteria (e.g., the coverage drops). We show that existing evaluation metrics are not robust against even such simple perturbations and disagree with scores assigned by humans to the perturbed output. The proposed templates thus allow for a fine-grained assessment of automatic evaluation metrics exposing their limitations and will facilitate better design, analysis and evaluation of such metrics.
Deep Joint Source-Channel Coding for Multi-Task Network
Venue: arXiv cs.CV 2021
Year: 2021

Multi-task learning (MTL) is an efficient way to improve the performance of related tasks by sharing knowledge. However, most existing MTL networks run on a single end and are not suitable for collaborative intelligence (CI) scenarios. In this work, we propose an MTL network with a deep joint source-channel coding (JSCC) framework, which allows operating under CI scenarios. We first propose a feature fusion based MTL network (FFMNet) for joint object detection and semantic segmentation. Compared with other MTL networks, FFMNet gets higher performance with fewer parameters. Then FFMNet is split into two parts, which run on a mobile device and an edge server respectively. The feature generated by the mobile device is transmitted through the wireless channel to the edge server. To reduce the transmission overhead of the intermediate feature, a deep JSCC network is designed. By combining two networks together, the whole model achieves 512x compression for the intermediate feature and a performance loss within 2% on both tasks. At last, by training with noise, the FFMNet with JSCC is robust to various channel conditions and outperforms the separate source and channel coding scheme.
DisCERN:Discovering Counterfactual Explanations using Relevance Features from Neighbourhoods
Venue: arXiv cs.LG 2021
Year: 2021

Counterfactual explanations focus on "actionable knowledge" to help end-users understand how a machine learning outcome could be changed to a more desirable outcome. For this purpose a counterfactual explainer needs to discover input dependencies that relate to outcome changes. Identifying the minimum subset of feature changes needed to action an output change in the decision is an interesting challenge for counterfactual explainers. The DisCERN algorithm introduced in this paper is a case-based counter-factual explainer. Here counterfactuals are formed by replacing feature values from a nearest unlike neighbour (NUN) until an actionable change is observed. We show how widely adopted feature relevance-based explainers (i.e. LIME, SHAP), can inform DisCERN to identify the minimum subset of "actionable features". We demonstrate our DisCERN algorithm on five datasets in a comparative study with the widely used optimisation-based counterfactual approach DiCE. Our results demonstrate that DisCERN is an effective strategy to minimise actionable changes necessary to create good counterfactual explanations.
Cyber-Security in the Emerging World of Smart Everything
Venue: arXiv cs.CY 2021
Year: 2021

The fourth industrial revolution (4IR) is a revolution many authors believe have come to stay. It is a revolution that has been fast blurring the line between physical, digital and biological technologies. These disruptive technologies largely rely on high-speed internet connectivity, Cloud technologies, Augmented Reality, Additive Manufacturing, Data science and Artificial Intelligence. Most developed economies have embraced the it while the developing economies are struggling to adopt 4IR because they lack the requisite skills, knowledge and technology. Thus, this study investigates Nigeria as one of the developing economies to understand her readiness for 4IR and the level of preparedness to mitigate the sophisticated cyber-attacks that comes with it. The investigation adopted quantitative research approach and developed an online questionnaire that was shared amongst the population of interest that includes academic, industry experts and relevant stakeholders. The questionnaire returned 116 valid responses which were analysed with descriptive statistical tools in SPSS. Results suggest that 60 of the respondents opined that Nigerian government at are not showing enough evidence to demonstrate her preparedness to leverage these promised potentials by developing 4IR relevant laws, strong institutional frameworks and policies. They lack significant development capacity to mitigate risks associated with digital ecosystem and cyber ecosystem that are ushered in by the 4IR. In the universities, 52 of the courses offered at the undergraduate and 42 at the post-graduate levels are relevant in the development of skills required in the revolution. The study recommends that the government at all levels make adequate efforts in developing the countrys intangible assets. In all, this paper posits that successful implementation of these could equip Nigeria to embrace the 4IR in all its aspects.
On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference
Venue: arXiv cs.LG 2021
Year: 2021

Previous work on planning as active inference addresses finite horizon problems and solutions valid for online planning. We propose solving the general Stochastic Shortest-Path Markov Decision Process (SSP MDP) as probabilistic inference. Furthermore, we discuss online and offline methods for planning under uncertainty. In an SSP MDP, the horizon is indefinite and unknown a priori. SSP MDPs generalize finite and infinite horizon MDPs and are widely used in the artificial intelligence community. Additionally, we highlight some of the differences between solving an MDP using dynamic programming approaches widely used in the artificial intelligence community and approaches used in the active inference community.
Dependability-Aware Routing and Scheduling for Time-Sensitive Networking
Venue: arXiv cs.NI 2021
Year: 2021

Time-Sensitive Networking (TSN) extends IEEE 802.1 Ethernet for safety-critical and real-time applications in several areas, e.g., automotive, aerospace or industrial automation. However, many of these systems also have stringent security requirements, and security attacks may impair safety. Given a TSN-based distributed architecture, a set of applications with tasks and messages, as well as a set of security and redundancy requirements, we are interested to synthesize a system configuration such that the real-time, safety and security requirements are upheld. We use the Timed Efficient Stream Loss-Tolerant Authentication (TESLA) low-resource multicast authentication protocol to guarantee the security requirements, and redundant disjunct message routes to tolerate link failures. We consider that tasks are dispatched using a static cyclic schedule table and that the messages use the time-sensitive traffic class in TSN, which relies on schedule tables (called Gate Control Lists, GCLs) in the network switches. A configuration consists of the schedule tables for tasks as well as the disjoint routes and GCLs for messages. We propose a Constraint Programming-based formulation which can be used to find an optimal solution with respect to our cost function. Additionally, we propose a Simulated Annealing based metaheuristic, which can find good solution for large test cases. We evaluate both approaches on several test cases.
Learning-Based UE Classification in Millimeter-Wave Cellular Systems With Mobility
Venue: arXiv cs.IT 2021
Year: 2021

Millimeter-wave cellular communication requires beamforming procedures that enable alignment of the transmitter and receiver beams as the user equipment (UE) moves. For efficient beam tracking it is advantageous to classify users according to their traffic and mobility patterns. Research to date has demonstrated efficient ways of machine learning based UE classification. Although different machine learning approaches have shown success, most of them are based on physical layer attributes of the received signal. This, however, imposes additional complexity and requires access to those lower layer signals. In this paper, we show that traditional supervised and even unsupervised machine learning methods can successfully be applied on higher layer channel measurement reports in order to perform UE classification, thereby reducing the complexity of the classification process.
An open-source implementation of a phase-field model for brittle fracture using Gridap in Julia
Venue: arXiv cs.CE 2021
Year: 2021

This article proposes an open-source implementation of a phase-field model for brittle fracture using a recently developed finite element toolbox, Gridap in Julia. The present work exploits the advantages of both the phase-field model and Gridap toolbox for simulating fracture in brittle materials. On one hand, the use of the phase-field model, which is a continuum approach and uses a diffuse representation of sharp cracks, enables the proposed implementation to overcome such well-known drawbacks of the discrete approach for predicting complex crack paths as the need for re-meshing, enrichment of finite element shape functions and an explicit tracking of the crack surfaces. On the other hand, the use of Gridap makes the proposed implementation very compact and user-friendly that requires low memory usage, and provides a high degree of flexibility to the users in defining weak forms of partial differential equations. A test on a notched beam under symmetric three-point bending and a set of tests on a notched beam with three holes under asymmetric three-point bending is considered to demonstrate how the proposed Gridap based phase-field Julia code can be used to simulate fracture in brittle materials.
Visualization for Villainy
Venue: arXiv cs.HC 2021
Year: 2021

Visualization has long been seen as a dependable and trustworthy tool for carrying out analysis and communication tasks -- a view reinforced by the growing interest in applying it to socially positive ends. However, despite the benign light in which visualization is usually perceived, it carries the potential to do harm to people, places, concepts, and things. In this paper, we capitalize on this negative potential to serve an underrepresented (but technologically engaged) group: villains. To achieve these ends, we introduce a design space for this type of graphical violence, which allows us to unify prior work on deceptive visualization with novel data-driven dastardly deeds, such as emotional spear phishing and unsafe data physicalization. By charting this vile charting landscape, we open new doors to collaboration with terrifying domain experts, and hopefully, make the world just a bit worse.
Predictable universally unique identification of sequential events on complex objects
Venue: arXiv cs.DS 2021
Year: 2021

Universal identifiers and hashing have been widely adopted in computer science from distributed financial transactions to data science. This is a consequence of their capability to avoid many shortcomings of relative identifiers, such as limited scope and the need for central management. However, the current identifiers in use are isolated entities which cannot provide much information about the relationship between objects. As an example, if one has both the identifiers of an object and its transformed version, no information about how they are related can be obtained without resorting to some kind of authority or additionally appended information. Moreover, given an input object and an arbitrarily long sequence of costly steps, one cannot currently predict the identifier of the outcome without actually processing the entire sequence. The capability of predicting the resulting identifier and avoiding redundant calculations is highly desirable in an efficient unmanaged system. In this paper, we propose a new kind of unique identifier that is calculated from the list of events that can produce an object, instead of directly identifying it by content. This provides an identification scheme regardless of the object's current existence, thus allowing to inexpensively check for its content in a database and retrieve it when it has already been calculated before. These identifiers are proposed in the context of abstract algebra, where objects are represented by elements that can be operated according to useful properties, such as associativity, order sensitivity when desired, and reversibility, while simplicity of implementation and strong guarantees are given by well known group theory results.
Augmented Abstractive Summarization With Document-LevelSemantic Graph
Venue: arXiv cs.CL 2021
Year: 2021

Previous abstractive methods apply sequence-to-sequence structures to generate summary without a module to assist the system to detect vital mentions and relationships within a document. To address this problem, we utilize semantic graph to boost the generation performance. Firstly, we extract important entities from each document and then establish a graph inspired by the idea of distant supervision \citep{mintz-etal-2009-distant}. Then, we combine a Bi-LSTM with a graph encoder to obtain the representation of each graph node. A novel neural decoder is presented to leverage the information of such entity graphs. Automatic and human evaluations show the effectiveness of our technique.
Joint prediction of truecasing and punctuation for conversational speech in low-resource scenarios
Venue: arXiv cs.CL 2021
Year: 2021

Capitalization and punctuation are important cues for comprehending written texts and conversational transcripts. Yet, many ASR systems do not produce punctuated and case-formatted speech transcripts. We propose to use a multi-task system that can exploit the relations between casing and punctuation to improve their prediction performance. Whereas text data for predicting punctuation and truecasing is seemingly abundant, we argue that written text resources are inadequate as training data for conversational models. We quantify the mismatch between written and conversational text domains by comparing the joint distributions of punctuation and word cases, and by testing our model cross-domain. Further, we show that by training the model in the written text domain and then transfer learning to conversations, we can achieve reasonable performance with less data.
NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternatives
Venue: arXiv cs.CL 2021
Year: 2021

Recent years have seen an increasing need for gender-neutral and inclusive language. Within the field of NLP, there are various mono- and bilingual use cases where gender inclusive language is appropriate, if not preferred due to ambiguity or uncertainty in terms of the gender of referents. In this work, we present a rule-based and a neural approach to gender-neutral rewriting for English along with manually curated synthetic data (WinoBias+) and natural data (OpenSubtitles and Reddit) benchmarks. A detailed manual and automatic evaluation highlights how our NeuTral Rewriter, trained on data generated by the rule-based approach, obtains word error rates (WER) below 0.18% on synthetic, in-domain and out-domain test sets.
Machine Learning for Online Algorithm Selection under Censored Feedback
Venue: arXiv cs.LG 2021
Year: 2021

In online algorithm selection (OAS), instances of an algorithmic problem class are presented to an agent one after another, and the agent has to quickly select a presumably best algorithm from a fixed set of candidate algorithms. For decision problems such as satisfiability (SAT), quality typically refers to the algorithm's runtime. As the latter is known to exhibit a heavy-tail distribution, an algorithm is normally stopped when exceeding a predefined upper time limit. As a consequence, machine learning methods used to optimize an algorithm selection strategy in a data-driven manner need to deal with right-censored samples, a problem that has received little attention in the literature so far. In this work, we revisit multi-armed bandit algorithms for OAS and discuss their capability of dealing with the problem. Moreover, we adapt them towards runtime-oriented losses, allowing for partially censored data while keeping a space- and time-complexity independent of the time horizon. In an extensive experimental evaluation on an adapted version of the ASlib benchmark, we demonstrate that theoretically well-founded methods based on Thompson sampling perform specifically strong and improve in comparison to existing methods.
Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration
Venue: arXiv cs.CL 2021
Year: 2021

Phrase representations derived from BERT often do not exhibit complex phrasal compositionality, as the model relies instead on lexical similarity to determine semantic relatedness. In this paper, we propose a contrastive fine-tuning objective that enables BERT to produce more powerful phrase embeddings. Our approach (Phrase-BERT) relies on a dataset of diverse phrasal paraphrases, which is automatically generated using a paraphrase generation model, as well as a large-scale dataset of phrases in context mined from the Books3 corpus. Phrase-BERT outperforms baselines across a variety of phrase-level similarity tasks, while also demonstrating increased lexical diversity between nearest neighbors in the vector space. Finally, as a case study, we show that Phrase-BERT embeddings can be easily integrated with a simple autoencoder to build a phrase-based neural topic model that interprets topics as mixtures of words and phrases by performing a nearest neighbor search in the embedding space. Crowdsourced evaluations demonstrate that this phrase-based topic model produces more coherent and meaningful topics than baseline word and phrase-level topic models, further validating the utility of Phrase-BERT.
The correlation coefficient between citation metrics and winning a Nobel or Abel Prize
Venue: arXiv cs.DL 2021
Year: 2021

Computing such correlation coefficient would be straightforward had we had available the rankings given by the prize committee to all scientists in the pool. In reality we only have citation rankings for all scientists. This means, however, that we have the ordinal rankings of the prize winners with regard to citation metrics. I use maximum likelihood method to infer the most probable correlation coefficient to produce the observed pattern of ordinal ranks of the prize winners. I get the correlation coefficients of 0.47 and 0.59 between the composite citation indicator and getting Abel Prize and Fields Medal, respectively. The correlation coefficient between getting a Nobel Prize and the Q-factor is 0.65. These coefficients are of the same magnitude as the correlation coefficient between Elo ratings of the chess players and their popularity measured as numbers of webpages mentioning the players.