OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging Multiple Online Sources for Accurate Income Verification
Venue: arXiv cs.IR 2021
Year: 2021

Income verification is the problem of validating a person's stated income given basic identity information such as name, location, job title and employer. It is widely used in the context of mortgage lending, rental applications and other financial risk models. However, the current processes surrounding verification involve significant human effort and document gathering which can be both time-consuming and expensive. In this paper, we propose a novel model for verifying an individual's income given very limited identity information typically available in loan applications. Our model is a combination of a deep neural network and hand-engineered features. The hand engineered features are based upon matching the input information against income records extracted automatically from various publicly available online sources (e.g. payscale.com, H-1B filings, government employee salaries). We conduct experiments on two data sets, one simulated from H-1B records and the other from a real-world data set of peer-to-peer (P2P) loan applications obtained from one of the world's largest P2P lending platform. Our results show a significant reduction in error of 3-6% relative to several strong baselines. We also perform ablation studies to demonstrate that a combined model is indeed necessary to achieve state-of-the-art performance on this task.
Coded Faster-than-Nyquist Signaling for Short Packet Communications
Venue: arXiv cs.IT 2021
Year: 2021

Ultra-reliable low-latency communication (URLLC) requires short packets of data transmission. It is known that when the packet length becomes short, the achievable rate is subject to a penalty when compared to the channel capacity. In this paper, we propose to use faster-than-Nyquist (FTN) signaling to compensate for the achievable rate loss of short packet communications. We investigate the performance of a combination of a low complexity detector of FTN signaling used with nonbinary low-density parity-check (NB-LDPC) codes that is suitable for low-latency and short block length requirements of URLLC systems. Our investigation shows that such combination of low-complexity FTN signaling detection and NB-LDPC codes outperforms the use of close-to-optimal FTN signaling detectors with LDPC codes in terms of error rate performance and also has a considerably lower computational complexity.
Low-Power Multi-Camera Object Re-Identification using Hierarchical Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Low-power computer vision on embedded devices has many applications. This paper describes a low-power technique for the object re-identification (reID) problem: matching a query image against a gallery of previously seen images. State-of-the-art techniques rely on large, computationally-intensive Deep Neural Networks (DNNs). We propose a novel hierarchical DNN architecture that uses attribute labels in the training dataset to perform efficient object reID. At each node in the hierarchy, a small DNN identifies a different attribute of the query image. The small DNN at each leaf node is specialized to re-identify a subset of the gallery: only the images with the attributes identified along the path from the root to a leaf. Thus, a query image is re-identified accurately after processing with a few small DNNs. We compare our method with state-of-the-art object reID techniques. With a 4% loss in accuracy, our approach realizes significant resource savings: 74% less memory, 72% fewer operations, and 67% lower query latency, yielding 65% less energy consumption.
Multi-Pair Text Style Transfer on Unbalanced Data
Venue: arXiv cs.CL 2021
Year: 2021

Text-style transfer aims to convert text given in one domain into another by paraphrasing the sentence or substituting the keywords without altering the content. By necessity, state-of-the-art methods have evolved to accommodate nonparallel training data, as it is frequently the case there are multiple data sources of unequal size, with a mixture of labeled and unlabeled sentences. Moreover, the inherent style defined within each source might be distinct. A generic bidirectional (e.g., formal $\Leftrightarrow$ informal) style transfer regardless of different groups may not generalize well to different applications. In this work, we developed a task adaptive meta-learning framework that can simultaneously perform a multi-pair text-style transfer using a single model. The proposed method can adaptively balance the difference of meta-knowledge across multiple tasks. Results show that our method leads to better quantitative performance as well as coherent style variations. Common challenges of unbalanced data and mismatched domains are handled well by this method.
Discrepancies in Epidemiological Modeling of Aggregated Heterogeneous Data
Venue: arXiv cs.AI 2021
Year: 2021

Within epidemiological modeling, the majority of analyses assume a single epidemic process for generating ground-truth data. However, this assumed data generation process can be unrealistic, since data sources for epidemics are often aggregated across geographic regions and communities. As a result, state-of-the-art models for estimating epidemiological parameters, e.g.~transmission rates, can be inappropriate when faced with complex systems. Our work empirically demonstrates some limitations of applying epidemiological models to aggregated datasets. We generate three complex outbreak scenarios by combining incidence curves from multiple epidemics that are independently simulated via SEIR models with different sets of parameters. Using these scenarios, we assess the robustness of a state-of-the-art Bayesian inference method that estimates the epidemic trajectory from viral load surveillance data. We evaluate two data-generating models within this Bayesian inference framework: a simple exponential growth model and a highly flexible Gaussian process prior model. Our results show that both models generate accurate transmission rate estimates for the combined incidence curve at the cost of generating biased estimates for each underlying epidemic, reflecting highly heterogeneous underlying population dynamics. The exponential growth model, while interpretable, is unable to capture the complexity of the underlying epidemics. With sufficient surveillance data, the Gaussian process prior model captures the shape of complex trajectories, but is imprecise for periods of low data coverage. Thus, our results highlight the potential pitfalls of neglecting complexity and heterogeneity in the data generation process, which can mask underlying location- and population-specific epidemic dynamics.
Hole Detection and Healing in Hybrid Sensor Networks
Venue: arXiv cs.CG 2021
Year: 2021

Although monitoring and covering are fundamental goals of a wireless sensor network (WSN), the accidental death of sensors or the running out of their energy would result in holes in the WSN. Such holes have the potential to disrupt the primary functions of WSNs. This paper investigates the hole detection and healing problems in hybrid WSNs with non-identical sensor sensing ranges. In particular, we aim to propose centralized algorithms for detecting holes in a given region and maximizing the area covered by a WSN in the presence of environmental obstacles. To precisely identify the boundary of the holes, we use an additively weighted Voronoi diagram and a polynomial-time algorithm.Furthermore, since this problem is known to be computationally difficult, we propose a centralized greedy 1/2-approximation algorithm to maximize the area covered by sensors. Finally, we implement the algorithms and run simulations to show that our approximation algorithm efficiently covers the holes by moving the mobile sensors.
Two-Faced Humans on Twitter and Facebook: Harvesting Social Multimedia for Human Personality Profiling
Venue: arXiv cs.SI 2021
Year: 2021

Human personality traits are the key drivers behind our decision-making, influencing our life path on a daily basis. Inference of personality traits, such as Myers-Briggs Personality Type, as well as an understanding of dependencies between personality traits and users' behavior on various social media platforms is of crucial importance to modern research and industry applications. The emergence of diverse and cross-purpose social media avenues makes it possible to perform user personality profiling automatically and efficiently based on data represented across multiple data modalities. However, the research efforts on personality profiling from multi-source multi-modal social media data are relatively sparse, and the level of impact of different social network data on machine learning performance has yet to be comprehensively evaluated. Furthermore, there is not such dataset in the research community to benchmark. This study is one of the first attempts towards bridging such an important research gap. Specifically, in this work, we infer the Myers-Briggs Personality Type indicators, by applying a novel multi-view fusion framework, called "PERS" and comparing the performance results not just across data modalities but also with respect to different social network data sources. Our experimental results demonstrate the PERS's ability to learn from multi-view data for personality profiling by efficiently leveraging on the significantly different data arriving from diverse social multimedia sources. We have also found that the selection of a machine learning approach is of crucial importance when choosing social network data sources and that people tend to reveal multiple facets of their personality in different social media avenues. Our released social multimedia dataset facilitates future research on this direction.
On predicting research grants productivity
Venue: arXiv cs.DL 2021
Year: 2021

Understanding the reasons associated with successful proposals is of paramount importance to improve evaluation processes. In this context, we analyzed whether bibliometric features are able to predict the success of research grants. We extracted features aiming at characterizing the academic history of Brazilian researchers, including research topics, affiliations, number of publications and visibility. The extracted features were then used to predict grants productivity via machine learning in three major research areas, namely Medicine, Dentistry and Veterinary Medicine. We found that research subject and publication history play a role in predicting productivity. In addition, institution-based features turned out to be relevant when combined with other features. While the best results outperformed text-based attributes, the evaluated features were not highly discriminative. Our findings indicate that predicting grants success, at least with the considered set of bibliometric features, is not a trivial task.
CPM-2: Large-scale Cost-effective Pre-trained Language Models
Venue: arXiv cs.CL 2021
Year: 2021

In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-world scenarios. We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference. (1) We introduce knowledge inheritance to accelerate the pre-training process by exploiting existing PLMs instead of training models from scratch. (2) We explore the best practice of prompt tuning with large-scale PLMs. Compared with conventional fine-tuning, prompt tuning significantly reduces the number of task-specific parameters. (3) We implement a new inference toolkit, namely InfMoE, for using large-scale PLMs with limited computational resources. Based on our cost-effective pipeline, we pre-train two models: an encoder-decoder bilingual model with 11 billion parameters (CPM-2) and its corresponding MoE version with 198 billion parameters. In our experiments, we compare CPM-2 with mT5 on downstream tasks. Experimental results show that CPM-2 has excellent general language intelligence. Moreover, we validate the efficiency of InfMoE when conducting inference of large-scale models having tens of billions of parameters on a single GPU. All source code and model parameters are available at https://github.com/TsinghuaAI/CPM.
Is Shapley Value fair? Improving Client Selection for Mavericks in Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

Shapley Value is commonly adopted to measure and incentivize client participation in federated learning. In this paper, we show -- theoretically and through simulations -- that Shapley Value underestimates the contribution of a common type of client: the Maverick. Mavericks are clients that differ both in data distribution and data quantity and can be the sole owners of certain types of data. Selecting the right clients at the right moment is important for federated learning to reduce convergence times and improve accuracy. We propose FedEMD, an adaptive client selection strategy based on the Wasserstein distance between the local and global data distributions. As FedEMD adapts the selection probability such that Mavericks are preferably selected when the model benefits from improvement on rare classes, it consistently ensures the fast convergence in the presence of different types of Mavericks. Compared to existing strategies, including Shapley Value-based ones, FedEMD improves the convergence of neural network classifiers by at least 26.9% for FedAvg aggregation compared with the state of the art.
On Stein Variational Neural Network Ensembles
Venue: arXiv cs.LG 2021
Year: 2021

Ensembles of deep neural networks have achieved great success recently, but they do not offer a proper Bayesian justification. Moreover, while they allow for averaging of predictions over several hypotheses, they do not provide any guarantees for their diversity, leading to redundant solutions in function space. In contrast, particle-based inference methods, such as Stein variational gradient descent (SVGD), offer a Bayesian framework, but rely on the choice of a kernel to measure the similarity between ensemble members. In this work, we study different SVGD methods operating in the weight space, function space, and in a hybrid setting. We compare the SVGD approaches to other ensembling-based methods in terms of their theoretical properties and assess their empirical performance on synthetic and real-world tasks. We find that SVGD using functional and hybrid kernels can overcome the limitations of deep ensembles. It improves on functional diversity and uncertainty estimation and approaches the true Bayesian posterior more closely. Moreover, we show that using stochastic SVGD updates, as opposed to the standard deterministic ones, can further improve the performance.
On Simple Mechanisms for Dependent Items
Venue: arXiv cs.GT 2021
Year: 2021

We study the problem of selling $n$ heterogeneous items to a single buyer, whose values for different items are dependent. Under arbitrary dependence, Hart and Nisan show that no simple mechanism can achieve a non-negligible fraction of the optimal revenue even with only two items. We consider the setting where the buyer's type is drawn from a correlated distribution that can be captured by a Markov Random Field, one of the most prominent frameworks for modeling high-dimensional distributions with structure. If the buyer's valuation is additive or unit-demand, we extend the result to all MRFs and show that max(SRev,BRev) can achieve an $Ω\left(\frac{1}{e^{O(Δ)}}\right)$-fraction of the optimal revenue, where $Δ$ is a parameter of the MRF that is determined by how much the value of an item can be influenced by the values of the other items. We further show that the exponential dependence on $Δ$ is unavoidable for our approach and a polynomial dependence on $Δ$ is unavoidable for any approach. When the buyer has a XOS valuation, we show that max(Srev,Brev) achieves at least an $Ω\left(\frac{1}{e^{O(Δ)}+\frac{1}{\sqrt{nγ}}}\right)$-fraction of the optimal revenue, where $γ$ is the spectral gap of the Glauber dynamics of the MRF. Note that in the special case of independently distributed items, $Δ=0$ and $\frac{1}{nγ}\leq 1$, and our results recover the known constant factor approximations for a XOS buyer. We further extend our parametric approximation to several other well-studied dependency measures such as the Dobrushin coefficient and the inverse temperature. Our results are based on the Duality-Framework by Cai et al. and a new concentration inequality for XOS functions over dependent random variables.
Trainable Class Prototypes for Few-Shot Learning
Venue: arXiv cs.CV 2021
Year: 2021

Metric learning is a widely used method for few shot learning in which the quality of prototypes plays a key role in the algorithm. In this paper we propose the trainable prototypes for distance measure instead of the artificial ones within the meta-training and task-training framework. Also to avoid the disadvantages that the episodic meta-training brought, we adopt non-episodic meta-training based on self-supervised learning. Overall we solve the few-shot tasks in two phases: meta-training a transferable feature extractor via self-supervised learning and training the prototypes for metric classification. In addition, the simple attention mechanism is used in both meta-training and task-training. Our method achieves state-of-the-art performance in a variety of established few-shot tasks on the standard few-shot visual classification dataset, with about 20% increase compared to the available unsupervised few-shot learning methods.
Robust Pooling through the Data Mode
Venue: arXiv cs.CV 2021
Year: 2021

The task of learning from point cloud data is always challenging due to the often occurrence of noise and outliers in the data. Such data inaccuracies can significantly influence the performance of state-of-the-art deep learning networks and their ability to classify or segment objects. While there are some robust deep learning approaches, they are computationally too expensive for real-time applications. This paper proposes a deep learning solution that includes a novel robust pooling layer which greatly enhances network robustness and performs significantly faster than state-of-the-art approaches. The proposed pooling layer looks for data a mode/cluster using two methods, RANSAC, and histogram, as clusters are indicative of models. We tested the pooling layer into frameworks such as Point-based and graph-based neural networks, and the tests showed enhanced robustness as compared to robust state-of-the-art methods.
Fostering Student Engagement in a Mobile Formative Assessment System for High-School Economics
Venue: arXiv cs.CY 2021
Year: 2021

In a mobile learning environment, students can learn via mobile devices without being limited by time and space. Therefore, it is vital to develop tools to assist students to learn and assess their knowledge in such environments. This paper presents a tool/application for formative self-assessment. The tool supports the selection of questions based on user-defined criteria concerning (1) the difficulty level; (2) the associated concepts; and (3) the purposes of the test taker. The main purpose of the presented tool is to better support the learning aims of the participants and to increase their engagement in the learning process. The focus of this study is to evaluate the tool using quizzes in Microeconomics to realize its potential in this specific domain. Teachers and students were involved in the experiments conducted. The experiments demonstrated that the presented tool is usable; it motivates the students and improves their understanding
Detection Of Primary User Emulation Attack (PUEA) In Cognitive Radio Networks Using One-Class Classification
Venue: arXiv cs.NI 2021
Year: 2021

Opportunistic usage of spectrum owned by licensed (or primary) users is the cornerstone on which the Cognitive Radio technology is built. Unlicensed (or secondary) users that thus use the spectrum rely opportunistically on spectrum sensing to determine the presence of primary user signal. In such a context, an attacker may mimic the behavior of a primary user (PU) to deceive the secondary users (SUs) into believing that a PU signal is present whereas it is not. Such an attack is known as the Primary User Emulation Attack (PUEA). A malicious user may launch a PUEA with the intention of grabbing the vacant bands for its own transmission. Another reason may be to simply disrupt the functioning of the Cognitive Radio Network (CRN). This work investigates the use of one-class classification for detecting PUEA in an infrastructure-based CRN. We opine that sensing data collected at the fusion center mainly for Collaborative Spectrum Sensing (CSS) can be exploited to characterize a PU signal. The PU signal features thus learned can aid in distinguishing a PU signal from a PU signal emulation. In particular, we investigate the use of one-class classification techniques, viz., Isolation Forest, Support Vector Machines (SVM), Minimum Covariance Determinant(MCD) and Local Outlier Factor(LOF) for detection of PUEA attacks. Simulation results support the validity of using one-class classification for detection of PUEA.
Friendly Training: Neural Networks Can Adapt Data To Make Learning Easier
Venue: arXiv cs.LG 2021
Year: 2021

In the last decade, motivated by the success of Deep Learning, the scientific community proposed several approaches to make the learning procedure of Neural Networks more effective. When focussing on the way in which the training data are provided to the learning machine, we can distinguish between the classic random selection of stochastic gradient-based optimization and more involved techniques that devise curricula to organize data, and progressively increase the complexity of the training set. In this paper, we propose a novel training procedure named Friendly Training that, differently from the aforementioned approaches, involves altering the training examples in order to help the model to better fulfil its learning criterion. The model is allowed to simplify those examples that are too hard to be classified at a certain stage of the training procedure. The data transformation is controlled by a developmental plan that progressively reduces its impact during training, until it completely vanishes. In a sense, this is the opposite of what is commonly done in order to increase robustness against adversarial examples, i.e., Adversarial Training. Experiments on multiple datasets are provided, showing that Friendly Training yields improvements with respect to informed data sub-selection routines and random selection, especially in deep convolutional architectures. Results suggest that adapting the input data is a feasible way to stabilize learning and improve the generalization skills of the network.