OpenRaiser/Intern-Atlas huggingface dataset:
Analyzing the Use of Character-Level Translation with Sparse and Noisy Datasets
Venue: arXiv cs.CL 2021
Year: 2021

This paper provides an analysis of character-level machine translation models used in pivot-based translation when applied to sparse and noisy datasets, such as crowdsourced movie subtitles. In our experiments, we find that such character-level models cut the number of untranslated words by over 40% and are especially competitive (improvements of 2-3 BLEU points) in the case of limited training data. We explore the impact of character alignment, phrase table filtering, bitext size and the choice of pivot language on translation quality. We further compare cascaded translation models to the use of synthetic training data via multiple pivots, and we find that the latter works significantly better. Finally, we demonstrate that neither word-nor character-BLEU correlate perfectly with human judgments, due to BLEU's sensitivity to length.
Exposing Paid Opinion Manipulation Trolls
Venue: arXiv cs.LG 2021
Year: 2021

Recently, Web forums have been invaded by opinion manipulation trolls. Some trolls try to influence the other users driven by their own convictions, while in other cases they can be organized and paid, e.g., by a political party or a PR agency that gives them specific instructions what to write. Finding paid trolls automatically using machine learning is a hard task, as there is no enough training data to train a classifier; yet some test data is possible to obtain, as these trolls are sometimes caught and widely exposed. In this paper, we solve the training data problem by assuming that a user who is called a troll by several different people is likely to be such, and one who has never been called a troll is unlikely to be such. We compare the profiles of (i) paid trolls vs. (ii)"mentioned" trolls vs. (iii) non-trolls, and we further show that a classifier trained to distinguish (ii) from (iii) does quite well also at telling apart (i) from (iii).
The connections among Hamming metric, $b$-symbol metric, and $r$-th generalized Hamming metric
Venue: arXiv cs.CR 2021
Year: 2021

The $r$-th generalized Hamming metric and the $b$-symbol metric are two different generalizations of Hamming metric. The former is used on the wire-tap channel of Type II, and the latter is motivated by the limitations of the reading process in high-density data storage systems and applied to a read channel that outputs overlapping symbols. In this paper, we study the connections among the three metrics (that is, Hamming metric, $b$-symbol metric, and $r$-th generalized Hamming metric) mentioned above and give a conjecture about the $b$-symbol Griesmer Bound for cyclic codes. %Furthermore, we explore the combinatorial function of the size of the $b$-symbol weight set of a code $C$.
Identifying and Mitigating Gender Bias in Hyperbolic Word Embeddings
Venue: arXiv cs.CL 2021
Year: 2021

Euclidean word embedding models such as GloVe and Word2Vec have been shown to reflect human-like gender biases. In this paper, we extend the study of gender bias to the recently popularized hyperbolic word embeddings. We propose gyrocosine bias, a novel measure for quantifying gender bias in hyperbolic word representations and observe a significant presence of gender bias. To address this problem, we propose Poincaré Gender Debias (PGD), a novel debiasing procedure for hyperbolic word representations. Experiments on a suit of evaluation tests show that PGD effectively reduces bias while adding a minimal semantic offset.
Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health
Venue: arXiv cs.CL 2021
Year: 2021

Many statistical models have high accuracy on test benchmarks, but are not explainable, struggle in low-resource scenarios, cannot be reused for multiple tasks, and cannot easily integrate domain expertise. These factors limit their use, particularly in settings such as mental health, where it is difficult to annotate datasets and model outputs have significant impact. We introduce a micromodel architecture to address these challenges. Our approach allows researchers to build interpretable representations that embed domain knowledge and provide explanations throughout the model's decision process. We demonstrate the idea on multiple mental health tasks: depression classification, PTSD classification, and suicidal risk assessment. Our systems consistently produce strong results, even in low-resource scenarios, and are more interpretable than alternative methods.
Fighting the Fog: Evaluating the Clarity of Privacy Disclosures in the Age of CCPA
Venue: arXiv cs.CY 2021
Year: 2021

Vagueness and ambiguity in privacy policies threaten the ability of consumers to make informed choices about how businesses collect, use, and share their personal information. The California Consumer Privacy Act (CCPA) of 2018 was intended to provide Californian consumers with more control by mandating that businesses (1) clearly disclose their data practices and (2) provide choices for consumers to opt out of specific data practices. In this work, we explore to what extent CCPA's disclosure requirements, as implemented in actual privacy policies, can help consumers to answer questions about the data practices of businesses. First, we analyzed 95 privacy policies from popular websites; our findings showed that there is considerable variance in how businesses interpret CCPA's definitions. Then, our user survey of 364 Californian consumers showed that this variance affects the ability of users to understand the data practices of businesses. Our results suggest that CCPA's mandates for privacy disclosures, as currently implemented, have not yet yielded the level of clarity they were designed to deliver, due to both vagueness and ambiguity in CCPA itself as well as potential non-compliance by businesses in their privacy policies.
An Automated Data Engineering Pipeline for Anomaly Detection of IoT Sensor Data
Venue: arXiv cs.LG 2021
Year: 2021

The rapid development in the field of System of Chip (SoC) technology, Internet of Things (IoT), cloud computing, and artificial intelligence has brought more possibilities of improving and solving the current problems. With data analytics and the use of machine learning/deep learning, it is made possible to learn the underlying patterns and make decisions based on what was learned from massive data generated from IoT sensors. When combined with cloud computing, the whole pipeline can be automated, and free of manual controls and operations. In this paper, an implementation of an automated data engineering pipeline for anomaly detection of IoT sensor data is studied and proposed. The process involves the use of IoT sensors, Raspberry Pis, Amazon Web Services (AWS) and multiple machine learning techniques with the intent to identify anomalous cases for the smart home security system.
Not Color Blind: AI Predicts Racial Identity from Black and White Retinal Vessel Segmentations
Venue: arXiv cs.CV 2021
Year: 2021

Background: Artificial intelligence (AI) may demonstrate racial bias when skin or choroidal pigmentation is present in medical images. Recent studies have shown that convolutional neural networks (CNNs) can predict race from images that were not previously thought to contain race-specific features. We evaluate whether grayscale retinal vessel maps (RVMs) of patients screened for retinopathy of prematurity (ROP) contain race-specific features. Methods: 4095 retinal fundus images (RFIs) were collected from 245 Black and White infants. A U-Net generated RVMs from RFIs, which were subsequently thresholded, binarized, or skeletonized. To determine whether RVM differences between Black and White eyes were physiological, CNNs were trained to predict race from color RFIs, raw RVMs, and thresholded, binarized, or skeletonized RVMs. Area under the precision-recall curve (AUC-PR) was evaluated. Findings: CNNs predicted race from RFIs near perfectly (image-level AUC-PR: 0.999, subject-level AUC-PR: 1.000). Raw RVMs were almost as informative as color RFIs (image-level AUC-PR: 0.938, subject-level AUC-PR: 0.995). Ultimately, CNNs were able to detect whether RFIs or RVMs were from Black or White babies, regardless of whether images contained color, vessel segmentation brightness differences were nullified, or vessel segmentation widths were normalized. Interpretation: AI can detect race from grayscale RVMs that were not thought to contain racial information. Two potential explanations for these findings are that: retinal vessels physiologically differ between Black and White babies or the U-Net segments the retinal vasculature differently for various fundus pigmentations. Either way, the implications remain the same: AI algorithms have potential to demonstrate racial bias in practice, even when preliminary attempts to remove such information from the underlying images appear to be successful.
Reinforcement Learning for Quantitative Trading
Venue: arXiv cs.LG 2021
Year: 2021

Quantitative trading (QT), which refers to the usage of mathematical models and data-driven techniques in analyzing the financial market, has been a popular topic in both academia and financial industry since 1970s. In the last decade, reinforcement learning (RL) has garnered significant interest in many domains such as robotics and video games, owing to its outstanding ability on solving complex sequential decision making problems. RL's impact is pervasive, recently demonstrating its ability to conquer many challenging QT tasks. It is a flourishing research direction to explore RL techniques' potential on QT tasks. This paper aims at providing a comprehensive survey of research efforts on RL-based methods for QT tasks. More concretely, we devise a taxonomy of RL-based QT models, along with a comprehensive summary of the state of the art. Finally, we discuss current challenges and propose future research directions in this exciting field.
Domain Generalization for Vision-based Driving Trajectory Generation
Venue: arXiv cs.CV 2021
Year: 2021

One of the challenges in vision-based driving trajectory generation is dealing with out-of-distribution scenarios. In this paper, we propose a domain generalization method for vision-based driving trajectory generation for autonomous vehicles in urban environments, which can be seen as a solution to extend the Invariant Risk Minimization (IRM) method in complex problems. We leverage an adversarial learning approach to train a trajectory generator as the decoder. Based on the pre-trained decoder, we infer the latent variables corresponding to the trajectories, and pre-train the encoder by regressing the inferred latent variable. Finally, we fix the decoder but fine-tune the encoder with the final trajectory loss. We compare our proposed method with the state-of-the-art trajectory generation method and some recent domain generalization methods on both datasets and simulation, demonstrating that our method has better generalization ability.
Separating Circuits : Switching Lemmas and Random Restrictions
Venue: arXiv cs.CC 2021
Year: 2021

This was submitted as a final project for CS254B, taught by Li Yang Tan and Tom Knowles. The field of Circuit Complexity utilises careful analysis of Boolean Circuit Functions in order to extract meaningful information about a range of complexity classes. In particular, the complexity class $P / \text{Poly}$ has played a central role in much of the historical attempts to tackle the problem of whether solution and verification are equivalent i.e. the central $P$ versus $NP$ problem. Whilst circuits can potentially be easier to analyse than Turing Machines due to their non-uniform nature of computation (program size is allowed to depend on the input size), it is notoriously hard to establish lower bounds for them. In this report, we will touch upon several results published by Hastad, Sipser and Razborov that will highlight a dynamic interplay between circuit complexity and many of the central ideas of modern-day complexity theory, and in particular the central importance of Hastad's Switching Lemma.
One-shot Key Information Extraction from Document with Deep Partial Graph Matching
Venue: arXiv cs.CV 2021
Year: 2021

Automating the Key Information Extraction (KIE) from documents improves efficiency, productivity, and security in many industrial scenarios such as rapid indexing and archiving. Many existing supervised learning methods for the KIE task need to feed a large number of labeled samples and learn separate models for different types of documents. However, collecting and labeling a large dataset is time-consuming and is not a user-friendly requirement for many cloud platforms. To overcome these challenges, we propose a deep end-to-end trainable network for one-shot KIE using partial graph matching. Contrary to previous methods that the learning of similarity and solving are optimized separately, our method enables the learning of the two processes in an end-to-end framework. Existing one-shot KIE methods are either template or simple attention-based learning approach that struggle to handle texts that are shifted beyond their desired positions caused by printers, as illustrated in Fig.1. To solve this problem, we add one-to-(at most)-one constraint such that we will find the globally optimized solution even if some texts are drifted. Further, we design a multimodal context ensemble block to boost the performance through fusing features of spatial, textual, and aspect representations. To promote research of KIE, we collected and annotated a one-shot document KIE dataset named DKIE with diverse types of images. The DKIE dataset consists of 2.5K document images captured by mobile phones in natural scenes, and it is the largest available one-shot KIE dataset up to now. The results of experiments on DKIE show that our method achieved state-of-the-art performance compared with recent one-shot and supervised learning approaches. The dataset and proposed one-shot KIE model will be released soo
ParaLiNGAM: Parallel Causal Structure Learning for Linear non-Gaussian Acyclic Models
Venue: arXiv cs.DC 2021
Year: 2021

One of the key objectives in many fields in machine learning is to discover causal relationships among a set of variables from observational data. In linear non-Gaussian acyclic models (LiNGAM), it can be shown that the true underlying causal structure can be identified uniquely from merely observational data. DirectLiNGAM algorithm is a well-known solution to learn the true causal structure in high dimensional setting. DirectLiNGAM algorithm executes in a sequence of iterations and it performs a set of comparisons between pairs of variables in each iteration. Unfortunately, the runtime of this algorithm grows significantly as the number of variables increases. In this paper, we propose a parallel algorithm, called ParaLiNGAM, to learn casual structures based on DirectLiNGAM algorithm. We propose a threshold mechanism that can reduce the number of comparisons remarkably compared with the sequential solution. Moreover, in order to further reduce runtime, we employ a messaging mechanism between workers and derive some mathematical formulations to simplify the execution of comparisons. We also present an implementation of ParaLiNGAM on GPU, considering hardware constraints. Experimental results on synthetic and real data show that the implementation of proposed algorithm on GPU can outperform DirectLiNGAM by a factor up to 4600 X.
Second Order WinoBias (SoWinoBias) Test Set for Latent Gender Bias Detection in Coreference Resolution
Venue: arXiv cs.CL 2021
Year: 2021

We observe an instance of gender-induced bias in a downstream application, despite the absence of explicit gender words in the test cases. We provide a test set, SoWinoBias, for the purpose of measuring such latent gender bias in coreference resolution systems. We evaluate the performance of current debiasing methods on the SoWinoBias test set, especially in reference to the method's design and altered embedding space properties. See https://github.com/hillarydawkins/SoWinoBias.
Overage and Staleness Metrics for Status Update Systems
Venue: arXiv cs.IT 2021
Year: 2021

Status update systems consist of sensors that take measurements of a physical parameter and transmit them to a remote receiver. Age of Information (AoI) has been studied extensively as a metric for the freshness of information in such systems with and without an enforced hard or soft deadline. In this paper, we propose three metrics for status update systems to measure the ability of different queuing systems to meet a threshold requirement for the AoI. The {\em overage probability} is defined as the probability that the age of the most recent update packet held by the receiver is larger than the threshold. The {\em stale update probability} is the probability that an update is stale, i.e., its age has exceeded the deadline, when it is delivered to the receiver. Finally, the {\em average overage} is defined as the time average of the overage (i.e., age beyond the threshold), and is a measure of the average ``staleness'' of the update packets held by the receiver. We investigate these metrics in three typical status update queuing systems -- M/G/1/1, M/G/1/$2^*$, and M/M/1. Numerical results show the performances for these metrics under different parameter settings and different service distributions. The differences between the average overage and average AoI are also shown. Our results demonstrate that a lower bound exists for the stale update probability when the buffer size is limited. Further, we observe that the overage probability decreases and the stale update probability increases as the update arrival rate increases.
Temporal Clustering with External Memory Network for Disease Progression Modeling
Venue: arXiv cs.LG 2021
Year: 2021

Disease progression modeling (DPM) involves using mathematical frameworks to quantitatively measure the severity of how certain disease progresses. DPM is useful in many ways such as predicting health state, categorizing disease stages, and assessing patients disease trajectory etc. Recently, with wider availability of electronic health records (EHR) and the broad application of data-driven machine learning method, DPM has attracted much attention yet remains two major challenges: (i) Due to the existence of irregularity, heterogeneity and long-term dependency in EHRs, most existing DPM methods might not be able to provide comprehensive patient representations. (ii) Lots of records in EHRs might be irrelevant to the target disease. Most existing models learn to automatically focus on the relevant information instead of explicitly capture the target-relevant events, which might make the learned model suboptimal. To address these two issues, we propose Temporal Clustering with External Memory Network (TC-EMNet) for DPM that groups patients with similar trajectories to form disease clusters/stages. TC-EMNet uses a variational autoencoder (VAE) to capture internal complexity from the input data and utilizes an external memory work to capture long term distance information, both of which are helpful for producing comprehensive patient states. Last but not least, k-means algorithm is adopted to cluster the extracted comprehensive patient states to capture disease progression. Experiments on two real-world datasets show that our model demonstrates competitive clustering performance against state-of-the-art methods and is able to identify clinically meaningful clusters. The visualization of the extracted patient states shows that the proposed model can generate better patient states than the baselines.
Customs Fraud Detection in the Presence of Concept Drift
Venue: arXiv cs.AI 2021
Year: 2021

Capturing the changing trade pattern is critical in customs fraud detection. As new goods are imported and novel frauds arise, a drift-aware fraud detection system is needed to detect both known frauds and unknown frauds within a limited budget. The current paper proposes ADAPT, an adaptive selection method that controls the balance between exploitation and exploration strategies used for customs fraud detection. ADAPT makes use of the model performance trends and the amount of concept drift to determine the best exploration ratio at every time. Experiments on data from four countries over several years show that each country requires a different amount of exploration for maintaining its fraud detection system. We find the system with ADAPT can gradually adapt to the dataset and find the appropriate amount of exploration ratio with high performance.
Semantic Communications With AI Tasks
Venue: arXiv cs.CV 2021
Year: 2021

A radical paradigm shift of wireless networks from ``connected things'' to ``connected intelligence'' undergoes, which coincides with the Shanno and Weaver's envisions: Communications will transform from the technical level to the semantic level. This article proposes a semantic communication method with artificial intelligence tasks (SC-AIT). First, the architecture of SC-AIT is elaborated. Then, based on the proposed architecture, we implement SC-AIT for a image classifications task. A prototype of SC-AIT is also established for surface defect detection, is conducted. Experimental results show that SC-AIT has much lower bandwidth requirements, and can achieve more than $40\%$ classification accuracy gains compared with the communications at the technical level. Future trends and key challenges for semantic communications are also identified.
Reflexivity in Issues of Scale and Representation in a Digital Humanities Project
Venue: arXiv cs.CL 2021
Year: 2021

In this paper, we explore issues that we have encountered in developing a pipeline that combines natural language processing with data analysis and visualization techniques. The characteristics of the corpus - being comprised of diaries of a single person spanning several decades - present both conceptual challenges in terms of issues of representation, and affordances as a source for historical research. We consider these issues in a team context with a particular focus on the generation and interpretation of visualizations.
Reconfiguring Data Infrastructure Ecosystem in Africa: A Primer Toward Digital Sovereignty
Venue: arXiv cs.CY 2021
Year: 2021

The growth of the Internet and its associated technologies; including digital services have tremendously impacted our society. However, scholars have noted a trend in data flow and collection; and have alleged mass surveillance and digital supremacy. To this end therefore, nations of the world such as Russia, China, Germany, Canada, France and Brazil among others have taken steps toward changing the narrative. The question now is, should Africans join these giants in this school of thought on digital sovereignty or fold their hands to remain on the other side of the divide? This question among others are the main reasons that provoked the thoughts of putting this paper together. This is with a view to demystifying the strategies to reconfigure data infrastructure in the context of Africa. It also highlights the benefits of digital technologies and its propensity to foster all round development in the continent as it relates to economic face-lift, employment creation, national security, among others. There is therefore a need for African nations to design appropriate blueprint to ensure security of her digital infrastructure and the flow of data within her cyber space. In addition, a roadmap in the immediate, short- and long-term in accordance with the framework of African developmental goals should be put in place to guide the implementation.