OpenRaiser/Intern-Atlas huggingface dataset:
Contrastive Clustering: Toward Unsupervised Bias Reduction for Emotion and Sentiment Classification
Venue: arXiv cs.CL 2021
Year: 2021

Background: When neural network emotion and sentiment classifiers are used in public health informatics studies, biases present in the classifiers could produce inadvertently misleading results. Objective: This study assesses the impact of bias on COVID-19 topics, and demonstrates an automatic algorithm for reducing bias when applied to COVID-19 social media texts. This could help public health informatics studies produce more timely results during crises, with a reduced risk of misleading results. Methods: Emotion and sentiment classifiers were applied to COVID-19 data before and after debiasing the classifiers using unsupervised contrastive clustering. Contrastive clustering approximates the degree to which tokens exhibit a causal versus correlational relationship with emotion or sentiment, by contrasting the tokens' relative salience to topics versus emotions or sentiments. Results: Contrastive clustering distinguishes correlation from causation for tokens with an F1 score of 0.753. Masking bias prone tokens from the classifier input decreases the classifier's overall F1 score by 0.02 (anger) and 0.033 (negative sentiment), but improves the F1 score for sentences annotated as bias prone by 0.155 (anger) and 0.103 (negative sentiment). Averaging across topics, debiasing reduces anger estimates by 14.4% and negative sentiment estimates by 8.0%. Conclusions: Contrastive clustering reduces algorithmic bias in emotion and sentiment classification for social media text pertaining to the COVID-19 pandemic. Public health informatics studies should account for bias, due to its prevalence across a range of topics. Further research is needed to improve bias reduction techniques and to explore the adverse impact of bias on public health informatics analyses.
On a Partition LP Relaxation for Min-Cost 2-Node Connected Spanning Subgraphs
Venue: arXiv cs.DS 2021
Year: 2021

Our motivation is to improve on the best approximation guarantee known for the problem of finding a minimum-cost 2-node connected spanning subgraph of a given undirected graph with nonnegative edge costs. We present an LP (Linear Programming) relaxation based on partition constraints. The special case where the input contains a spanning tree of zero cost is called 2NC-TAP. We present a greedy algorithm for 2NC-TAP, and we analyze it via dual-fitting for our partition LP relaxation. Keywords: 2-node connected graphs, approximation algorithms, connectivity augmentation, greedy algorithm, network design, partition relaxation
Public Policymaking for International Agricultural Trade using Association Rules and Ensemble Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

International economics has a long history of improving our understanding of factors causing trade, and the consequences of free flow of goods and services across countries. The recent shocks to the free trade regime, especially trade disputes among major economies, as well as black swan events, such as trade wars and pandemics, raise the need for improved predictions to inform policy decisions. AI methods are allowing economists to solve such prediction problems in new ways. In this manuscript, we present novel methods that predict and associate food and agricultural commodities traded internationally. Association Rules (AR) analysis has been deployed successfully for economic scenarios at the consumer or store level, such as for market basket analysis. In our work however, we present analysis of imports and exports associations and their effects on commodity trade flows. Moreover, Ensemble Machine Learning methods are developed to provide improved agricultural trade predictions, outlier events' implications, and quantitative pointers to policy makers.
Lifelong Vehicle Trajectory Prediction Framework Based on Generative Replay
Venue: arXiv cs.RO 2021
Year: 2021

Accurate trajectory prediction of vehicles is essential for reliable autonomous driving. To maintain consistent performance as a vehicle driving around different cities, it is crucial to adapt to changing traffic circumstances and achieve lifelong trajectory prediction model. To realize it, catastrophic forgetting is a main problem to be addressed. In this paper, a divergence measurement method based on conditional Kullback-Leibler divergence is proposed first to evaluate spatiotemporal dependency difference among varied driving circumstances. Then based on generative replay, a novel lifelong vehicle trajectory prediction framework is developed. The framework consists of a conditional generation model and a vehicle trajectory prediction model. The conditional generation model is a generative adversarial network conditioned on position configuration of vehicles. After learning and merging trajectory distribution of vehicles across different cities, the generation model replays trajectories with prior samplings as inputs, which alleviates catastrophic forgetting. The vehicle trajectory prediction model is trained by the replayed trajectories and achieves consistent prediction performance on visited cities. A lifelong experiment setup is established on four open datasets including five tasks. Spatiotemporal dependency divergence is calculated for different tasks. Even though these divergence, the proposed framework exhibits lifelong learning ability and achieves consistent performance on all tasks.
Automatic Analysis of Linguistic Features in Journal Articles of Different Academic Impacts with Feature Engineering Techniques
Venue: arXiv cs.CL 2021
Year: 2021

English research articles (RAs) are an essential genre in academia, so the attempts to employ NLP to assist the development of academic writing ability have received considerable attention in the last two decades. However, there has been no study employing feature engineering techniques to investigate the linguistic features of RAs of different academic impacts (i.e., the papers of high/moderate citation times published in the journals of high/moderate impact factors). This study attempts to extract micro-level linguistic features in high- and moderate-impact journal RAs, using feature engineering methods. We extracted 25 highly relevant features from the Corpus of English Journal Articles through feature selection methods. All papers in the corpus deal with COVID-19 medical empirical studies. The selected features were then validated of the classification performance in terms of consistency and accuracy through supervised machine learning methods. Results showed that 24 linguistic features such as the overlapping of content words between adjacent sentences, the use of third-person pronouns, auxiliary verbs, tense, emotional words provide consistent and accurate predictions for journal articles with different academic impacts. Lastly, the random forest model is shown to be the best model to fit the relationship between these 24 features and journal articles with high and moderate impacts. These findings can be used to inform academic writing courses and lay the foundation for developing automatic evaluation systems for L2 graduate students.
Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Video instance segmentation aims to detect, segment, and track objects in a video. Current approaches extend image-level segmentation algorithms to the temporal domain. However, this results in temporally inconsistent masks. In this work, we identify the mask quality due to temporal stability as a performance bottleneck. Motivated by this, we propose a video instance segmentation method that alleviates the problem due to missing detections. Since this cannot be solved simply using spatial information, we leverage temporal context using inter-frame attentions. This allows our network to refocus on missing objects using box predictions from the neighbouring frame, thereby overcoming missing detections. Our method significantly outperforms previous state-of-the-art algorithms using the Mask R-CNN backbone, by achieving 36.0% mAP on the YouTube-VIS benchmark. Additionally, our method is completely online and requires no future frames. Our code is publicly available at https://github.com/anirudh-chakravarthy/ObjProp.
A Probabilistic Hard Attention Model For Sequentially Observed Scenes
Venue: arXiv cs.CV 2021
Year: 2021

A visual hard attention model actively selects and observes a sequence of subregions in an image to make a prediction. The majority of hard attention models determine the attention-worthy regions by first analyzing a complete image. However, it may be the case that the entire image is not available initially but instead sensed gradually through a series of partial observations. In this paper, we design an efficient hard attention model for classifying such sequentially observed scenes. The presented model never observes an image completely. To select informative regions under partial observability, the model uses Bayesian Optimal Experiment Design. First, it synthesizes the features of the unobserved regions based on the already observed regions. Then, it uses the predicted features to estimate the expected information gain (EIG) attained, should various regions be attended. Finally, the model attends to the actual content on the location where the EIG mentioned above is maximum. The model uses a) a recurrent feature aggregator to maintain a recurrent state, b) a linear classifier to predict the class label, c) a Partial variational autoencoder to predict the features of unobserved regions. We use normalizing flows in Partial VAE to handle multi-modality in the feature-synthesis problem. We train our model using a differentiable objective and test it on five datasets. Our model gains 2-10% higher accuracy than the baseline models when both have seen only a couple of glimpses.
Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data
Venue: arXiv cs.CL 2021
Year: 2021

Recent advancements in end-to-end speech synthesis have made it possible to generate highly natural speech. However, training these models typically requires a large amount of high-fidelity speech data, and for unseen texts, the prosody of synthesized speech is relatively unnatural. To address these issues, we propose to combine a fine-tuned BERT-based front-end with a pre-trained FastSpeech2-based acoustic model to improve prosody modeling. The pre-trained BERT is fine-tuned on the polyphone disambiguation task, the joint Chinese word segmentation (CWS) and part-of-speech (POS) tagging task, and the prosody structure prediction (PSP) task in a multi-task learning framework. FastSpeech 2 is pre-trained on large-scale external data that are noisy but easier to obtain. Experimental results show that both the fine-tuned BERT model and the pre-trained FastSpeech 2 can improve prosody, especially for those structurally complex sentences.
Measuring Uncertainty in Translation Quality Evaluation (TQE)
Venue: arXiv cs.CL 2021
Year: 2021

From both human translators (HT) and machine translation (MT) researchers' point of view, translation quality evaluation (TQE) is an essential task. Translation service providers (TSPs) have to deliver large volumes of translations which meet customer specifications with harsh constraints of required quality level in tight time-frames and costs. MT researchers strive to make their models better, which also requires reliable quality evaluation. While automatic machine translation evaluation (MTE) metrics and quality estimation (QE) tools are widely available and easy to access, existing automated tools are not good enough, and human assessment from professional translators (HAP) are often chosen as the golden standard \cite{han-etal-2021-TQA}. Human evaluations, however, are often accused of having low reliability and agreement. Is this caused by subjectivity or statistics is at play? How to avoid the entire text to be checked and be more efficient with TQE from cost and efficiency perspectives, and what is the optimal sample size of the translated text, so as to reliably estimate the translation quality of the entire material? This work carries out such motivated research to correctly estimate the confidence intervals \cite{Brown_etal2001Interval} depending on the sample size of the translated text, e.g. the amount of words or sentences, that needs to be processed on TQE workflow step for confident and reliable evaluation of overall translation quality. The methodology we applied for this work is from Bernoulli Statistical Distribution Modelling (BSDM) and Monte Carlo Sampling Analysis (MCSA).
Parsing Data Formats of the Inputs and Outputs of Geographic Models with Code Analysis
Venue: arXiv cs.SE 2021
Year: 2021

Model web services provide an approach for implementing and facilitating the sharing of geographic models. The description and acquisition of inputs and outputs (IO) of geographic models is a key issue in constructing and using model web services. These approaches for describing and acquiring the data formats of the IO of geographic models can be classified into two categories, i.e., intermediate-data-format-based and native-data-format-based. Nonetheless, these two categories mainly consider the description of the IO of geographical models but relatively pay little attention to the acquisition. To address this issue, this paper proposes an approach for automatically parsing data formats of the IO utilizing the relationship between the IO and source codes. This proposed approach can utilize such a strict and coupling relationship and the expression form of the data formats in the source codes to retrospectively derive the IO data format and automatically generate data format documentation. The feasibility of the proposed approach has been verified via a geographical model coded in the FORTRAN language, which shows that it significantly improves the efficiency of writing data format specifications and promotes sharing geographic models as model web services.
A Note on the Gains from Trade of the Random-Offerer Mechanism
Venue: arXiv cs.GT 2021
Year: 2021

We study the classic bilateral trade setting. Myerson and Satterthwaite show that there is no Bayesian incentive compatible and budget-balanced mechanism that obtains the gains from trade of the first-best mechanism. Consider the random-offerer mechanism: with probability $\frac{1}{2}$ run the \emph{seller-offering} mechanism, in which the seller offers the buyer a take-it-or-leave-it price that maximizes the expected profit of the seller, and with probability $\frac{1}{2}$ run the \emph{buyer-offering} mechanism. Very recently, Deng, Mao, Sivan, and Wang showed that the gains from trade of the random-offerer mechanism is at least a constant factor of $\frac 1 {8.23}\approx 0.121$ of the gains from trade of the first best mechanism. Perhaps a natural conjecture is that the gains-from-trade of the random-offerer mechanism, which is known to be at least half of the gains-from-trade of the second-best mechanism, is also at least half of the gains-from-trade of the first-best mechanism. However, in this note we exhibit distributions such as the gains-from trade of the random-offerer mechanism is smaller than a $0.495$-fraction of the gains-from-trade of the first-best mechanism.
Learnable Locality-Sensitive Hashing for Video Anomaly Detection
Venue: arXiv cs.CV 2021
Year: 2021

Video anomaly detection (VAD) mainly refers to identifying anomalous events that have not occurred in the training set where only normal samples are available. Existing works usually formulate VAD as a reconstruction or prediction problem. However, the adaptability and scalability of these methods are limited. In this paper, we propose a novel distance-based VAD method to take advantage of all the available normal data efficiently and flexibly. In our method, the smaller the distance between a testing sample and normal samples, the higher the probability that the testing sample is normal. Specifically, we propose to use locality-sensitive hashing (LSH) to map samples whose similarity exceeds a certain threshold into the same bucket in advance. In this manner, the complexity of near neighbor search is cut down significantly. To make the samples that are semantically similar get closer and samples not similar get further apart, we propose a novel learnable version of LSH that embeds LSH into a neural network and optimizes the hash functions with contrastive learning strategy. The proposed method is robust to data imbalance and can handle the large intra-class variations in normal data flexibly. Besides, it has a good ability of scalability. Extensive experiments demonstrate the superiority of our method, which achieves new state-of-the-art results on VAD benchmarks.
Reproducible and Adaptable Log Data Generation for Sound Cybersecurity Experiments
Venue: arXiv cs.CR 2021
Year: 2021

Artifacts such as log data and network traffic are fundamental for cybersecurity research, e.g., in the area of intrusion detection. Yet, most research is based on artifacts that are not available to others or cannot be adapted to own purposes, thus making it difficult to reproduce and build on existing work. In this paper, we identify the challenges of artifact generation with the goal of conducting sound experiments that are valid, controlled, and reproducible. We argue that testbeds for artifact generation have to be designed specifically with reproducibility and adaptability in mind. To achieve this goal, we present SOCBED, our proof-of-concept implementation and the first testbed with a focus on generating realistic log data for cybersecurity experiments in a reproducible and adaptable manner. SOCBED enables researchers to reproduce testbed instances on commodity computers, adapt them according to own requirements, and verify their correct functionality. We evaluate SOCBED with an exemplary, practical experiment on detecting a multi-step intrusion of an enterprise network and show that the resulting experiment is indeed valid, controlled, and reproducible. Both SOCBED and the log dataset underlying our evaluation are freely available.
Near Optimal VNF Placement in Edge-Enabled 6G Networks
Venue: arXiv cs.NI 2021
Year: 2021

Softwarization and virtualization are key concepts for emerging industries that require ultra-low latency. This is only possible if computing resources, traditionally centralized at the core of communication networks, are moved closer to the user, to the network edge. However, the realization of Edge Computing (EC) in the sixth generation (6G) of mobile networks requires efficient resource allocation mechanisms for the placement of the Virtual Network Functions (VNFs). Machine learning (ML) methods, and more specifically, Reinforcement Learning (RL), are a promising approach to solve this problem. The main contributions of this work are twofold: first, we obtain the theoretical performance bound for VNF placement in EC-enabled6G networks by formulating the problem mathematically as a finite Markov Decision Process (MDP) and solving it using a dynamic programming method called Policy Iteration (PI). Second, we develop a practical solution to the problem using RL, where the problem is treated with Q-Learning that considers both computational and communication resources when placing VNFs in the network. The simulation results under different settings of the system parameters show that the performance of the Q-Learning approach is close to the optimal PI algorithm (without having its restrictive assumptions on service statistics). This is particularly interesting when the EC resources are scarce and efficient management of these resources is required.
Functional Broadcast Repair of Multiple Partial Failures in Wireless Distributed Storage Systems
Venue: arXiv cs.DC 2021
Year: 2021

We consider a distributed storage system with $n$ nodes, where a user can recover the stored file from any $k$ nodes, and study the problem of repairing $r$ partially failed nodes. We consider \textit{broadcast repair}, that is, $d$ surviving nodes transmit broadcast messages on an error-free wireless channel to the $r$ nodes being repaired, which are then used, together with the surviving data in the local memories of the failed nodes, to recover the lost content. First, we derive the trade-off between the storage capacity and the repair bandwidth for partial repair of multiple failed nodes, based on the cut-set bound for information flow graphs. It is shown that utilizing the broadcast nature of the wireless medium and the surviving contents at the partially failed nodes reduces the repair bandwidth per node significantly. Then, we list a set of invariant conditions that are sufficient for a functional repair code to be feasible. We further propose a scheme for functional repair of multiple failed nodes that satisfies the invariant conditions with high probability, and its extension to the repair of partial failures. The performance of the proposed scheme meets the cut-set bound on all the points on the trade-off curve for all admissible parameters when $k$ is divisible by $r$, while employing linear subpacketization, which is an important practical consideration in the design of distributed storage codes. Unlike random linear codes, which are conventionally used for functional repair of failed nodes, the proposed repair scheme has lower overhead, lower input-output cost, and lower computational complexity during repair.
IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages
Venue: arXiv cs.CL 2021
Year: 2021

Sentiment analysis of social media posts and comments for various marketing and emotional purposes is gaining recognition. With the increasing presence of code-mixed content in various native languages, there is a need for ardent research to produce promising results. This research paper bestows a tiny contribution to this research in the form of sentiment analysis of code-mixed social media comments in the popular Dravidian languages Kannada, Tamil and Malayalam. It describes the work for the shared task conducted by Dravidian-CodeMix at FIRE 2021 by employing pre-trained models like ULMFiT and multilingual BERT fine-tuned on the code-mixed dataset, transliteration (TRAI) of the same, English translations (TRAA) of the TRAI data and the combination of all the three. The results are recorded in this research paper where the best models stood 4th, 5th and 10th ranks in the Tamil, Kannada and Malayalam tasks respectively.
Birkhoff's Completeness Theorem for Multi-Sorted Algebras Formalized in Agda
Venue: arXiv cs.LO 2021
Year: 2021

This document provides a formal proof of Birkhoff's completeness theorem for multi-sorted algebras which states that any equational entailment valid in all models is also provable in the equational theory. More precisely, if a certain equation is valid in all models that validate a fixed set of equations, then this equation is derivable from that set using the proof rules for a congruence. The proof has been formalized in Agda version 2.6.2 with the Agda Standard Library version 1.7 and this document reproduces the commented Agda code.