OpenRaiser/Intern-Atlas huggingface dataset:
Mechanism Design with Moral Bidders
Venue: arXiv cs.GT 2021
Year: 2021

A rapidly growing literature on lying in behavioral economics and psychology shows that individuals often do not lie even when lying maximizes their utility. In this work, we attempt to incorporate these findings into the theory of mechanism design. We consider players that have a preference for truth-telling and will only lie if their benefit from lying is sufficiently larger than the loss of the others. To accommodate such players, we introduce $α$-moral mechanisms, in which the gain of a player from misreporting his true value, comparing to truth-telling, is at most $α$ times the loss that the others incur due to misreporting. We develop a theory of moral mechanisms in the canonical setting of single-item auctions. We identify similarities and disparities to the standard theory of truthful mechanisms. In particular, we show that the allocation function does not uniquely determine the payments and is unlikely to admit a simple characterization. In contrast, recall that monotonicity characterizes the allocation function of truthful mechanisms. Our main technical effort is invested in determining whether the auctioneer can exploit the preference for truth-telling of the players to extract more revenue comparing to truthful mechanisms. We show that the auctioneer can extract more revenue when the values of the players are correlated, even when there are only two players. However, we show that truthful mechanisms are revenue-maximizing even among moral ones when the values of the players are independently drawn from certain identical distributions. As a by product we get an alternative proof to Myerson's characterization in the settings that we consider. We flesh out this approach by providing an alternative proof to Myerson's characterization that does not involve moral mechanisms whenever the values are independently drawn from regular distributions.
MCS-HMS: A Multi-Cluster Selection Strategy for the Human Mental Search Algorithm
Venue: arXiv cs.NE 2021
Year: 2021

Population-based metaheuristic algorithms have received significant attention in global optimisation. Human Mental Search (HMS) is a relatively recent population-based metaheuristic that has been shown to work well in comparison to other algorithms. However, HMS is time-consuming and suffers from relatively poor exploration. Having clustered the candidate solutions, HMS selects a winner cluster with the best mean objective function. This is not necessarily the best criterion to choose the winner group and limits the exploration ability of the algorithm. In this paper, we propose an improvement to the HMS algorithm in which the best bids from multiple clusters are used to benefit from enhanced exploration. We also use a one-step k-means algorithm in the clustering phase to improve the speed of the algorithm. Our experimental results show that MCS-HMS outperforms HMS as well as other population-based metaheuristic algorithms
Improving Sum-Rate of Cell-Free Massive MIMO with Expanded Compute-and-Forward
Venue: arXiv cs.IT 2021
Year: 2021

Cell-free massive multiple-input multiple-output (MIMO) employs a large number of distributed access points (APs) to serve a small number of user equipments (UEs) via the same time/frequency resource. Due to the strong macro diversity gain, cell-free massive MIMO can considerably improve the achievable sum-rate compared to conventional cellular massive MIMO. However, the performance of cell-free massive MIMO is upper limited by inter-user interference (IUI) when employing simple maximum ratio combining (MRC) at receivers. To harness IUI, the expanded compute-and-forward (ECF) framework is adopted. In particular, we propose power control algorithms for the parallel computation and successive computation in the ECF framework, respectively, to exploit the performance gain and then improve the system performance. Furthermore, we propose an AP selection scheme and the application of different decoding orders for the successive computation. Finally, numerical results demonstrate that ECF frameworks outperform the conventional CF and MRC frameworks in terms of achievable sum-rate.
End-to-end Learning for Fair Ranking Systems
Venue: arXiv cs.LG 2021
Year: 2021

The learning-to-rank problem aims at ranking items to maximize exposure of those most relevant to a user query. A desirable property of such ranking systems is to guarantee some notion of fairness among specified item groups. While fairness has recently been considered in the context of learning-to-rank systems, current methods cannot provide guarantees on the fairness of the proposed ranking policies. This paper addresses this gap and introduces Smart Predict and Optimize for Fair Ranking (SPOFR), an integrated optimization and learning framework for fairness-constrained learning to rank. The end-to-end SPOFR framework includes a constrained optimization sub-model and produces ranking policies that are guaranteed to satisfy fairness constraints while allowing for fine control of the fairness-utility tradeoff. SPOFR is shown to significantly improve current state-of-the-art fair learning-to-rank systems with respect to established performance metrics.
COVID Induced Digital Inequality for Senior Citizens
Venue: arXiv cs.HC 2021
Year: 2021

The global pandemic of COVID-19 has fundamentally changed how people interact, especially with the introduction of technology-based measures that aim at curbing the spread of the virus. As the country that currently implements one of the tightest technology-based COVID prevention policy, China has protected its citizen with a prolonged peaceful time of zero case as well as a fast reaction to potential upsurging of the disease. However, such mobile-based technology does come with sacrifices, especially for senior citizens who find themselves difficult to adapt to modern technologies. In this study, we demonstrated the fact that most senior citizens find it difficult to use the health code apps called ''JKM'', to which they responded by cutting down on travel and reducing local commuting to locations where the verification of JKM is needed. Such compromise has physical and mental consequences and leads to inequalities in infrastructure, social isolation and self-sufficiency. As we illustrated in the paper, such decrease in life quality of senior citizens can be greatly reduced if improvements on the user interactions of the JKM can be implemented. To the best of our knowledge, we are the first systemic study of digital inequality due to mobile-based COVID prevention technologies for senior citizens in China. As similar technologies become widely adopted around the world, we wish to shed light on how widened digital inequality increasingly affects the life quality of senior citizens in the pandemic era.
More Romanian word embeddings from the RETEROM project
Venue: arXiv cs.CL 2021
Year: 2021

Automatically learned vector representations of words, also known as "word embeddings", are becoming a basic building block for more and more natural language processing algorithms. There are different ways and tools for constructing word embeddings. Most of the approaches rely on raw texts, the construction items being the word occurrences and/or letter n-grams. More elaborated research is using additional linguistic features extracted after text preprocessing. Morphology is clearly served by vector representations constructed from raw texts and letter n-grams. Syntax and semantics studies may profit more from the vector representations constructed with additional features such as lemma, part-of-speech, syntactic or semantic dependants associated with each word. One of the key objectives of the ReTeRom project is the development of advanced technologies for Romanian natural language processing, including morphological, syntactic and semantic analysis of text. As such, we plan to develop an open-access large library of ready-to-use word embeddings sets, each set being characterized by different parameters: used features (wordforms, letter n-grams, lemmas, POSes etc.), vector lengths, window/context size and frequency thresholds. To this end, the previously created sets of word embeddings (based on word occurrences) on the CoRoLa corpus (Păiş and Tufiş, 2018) are and will be further augmented with new representations learned from the same corpus by using specific features such as lemmas and parts of speech. Furthermore, in order to better understand and explore the vectors, graphical representations will be available by customized interfaces.
FCOSR: A Simple Anchor-free Rotated Detector for Aerial Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Existing anchor-base oriented object detection methods have achieved amazing results, but these methods require some manual preset boxes, which introduces additional hyperparameters and calculations. The existing anchor-free methods usually have complex architectures and are not easy to deploy. Our goal is to propose an algorithm which is simple and easy-to-deploy for aerial image detection. In this paper, we present a one-stage anchor-free rotated object detector (FCOSR) based on FCOS, which can be deployed on most platforms. The FCOSR has a simple architecture consisting of only convolution layers. Our work focuses on the label assignment strategy for the training phase. We use ellipse center sampling method to define a suitable sampling region for oriented bounding box (OBB). The fuzzy sample assignment strategy provides reasonable labels for overlapping objects. To solve the insufficient sampling problem, a multi-level sampling module is designed. These strategies allocate more appropriate labels to training samples. Our algorithm achieves 79.25, 75.41, and 90.15 mAP on DOTA1.0, DOTA1.5, and HRSC2016 datasets, respectively. FCOSR demonstrates superior performance to other methods in single-scale evaluation. We convert a lightweight FCOSR model to TensorRT format, which achieves 73.93 mAP on DOTA1.0 at a speed of 10.68 FPS on Jetson Xavier NX with single scale. The code is available at: https://github.com/lzh420202/FCOSR
Automatic Detection of Depression from Stratified Samples of Audio Data
Venue: arXiv cs.SD 2021
Year: 2021

Depression is a common mental disorder which has been affecting millions of people around the world and becoming more severe with the arrival of COVID-19. Nevertheless proper diagnosis is not accessible in many regions due to a severe shortage of psychiatrists. This scarcity is worsened in low-income countries which have a psychiatrist to population ratio 210 times lower than that of countries with better economies. This study aimed to explore applications of deep learning in diagnosing depression from voice samples. We collected data from the DAIC-WOZ database which contained 189 vocal recordings from 154 individuals. Voice samples from a patient with a PHQ-8 score equal or higher than 10 were deemed as depressed and those with a PHQ-8 score lower than 10 were considered healthy. We applied mel-spectrogram to extract relevant features from the audio. Three types of encoders were tested i.e. 1D CNN, 1D CNN-LSTM, and 1D CNN-GRU. After tuning hyperparameters systematically, we found that 1D CNN-GRU encoder with a kernel size of 5 and 15 seconds of recording data appeared to have the best performance with F1 score of 0.75, precision of 0.64, and recall of 0.92.
Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations
Venue: arXiv cs.SE 2021
Year: 2021

Software clone detection identifies similar code snippets. It has been an active research topic that attracts extensive attention over the last two decades. In recent years, machine learning (ML) based detectors, especially deep learning-based ones, have demonstrated impressive capability on clone detection. It seems that this longstanding problem has already been tamed owing to the advances in ML techniques. In this work, we would like to challenge the robustness of the recent ML-based clone detectors through code semantic-preserving transformations. We first utilize fifteen simple code transformation operators combined with commonly-used heuristics (i.e., Random Search, Genetic Algorithm, and Markov Chain Monte Carlo) to perform equivalent program transformation. Furthermore, we propose a deep reinforcement learning-based sequence generation (DRLSG) strategy to effectively guide the search process of generating clones that could escape from the detection. We then evaluate the ML-based detectors with the pairs of original and generated clones. We realize our method in a framework named CloneGen. CloneGen In evaluation, we challenge the two state-of-the-art ML-based detectors and four traditional detectors with the code clones after semantic-preserving transformations via the aid of CloneGen. Surprisingly, our experiments show that, despite the notable successes achieved by existing clone detectors, the ML models inside these detectors still cannot distinguish numerous clones produced by the code transformations in CloneGen. In addition, adversarial training of ML-based clone detectors using clones generated by CloneGen can improve their robustness and accuracy. CloneGen Meanwhile, compared with the commonly-used heuristics, the DRLSG strategy has shown the best effectiveness in generating code clones to decrease the detection accuracy of the ML-based detectors.
HoughCL: Finding Better Positive Pairs in Dense Self-supervised Learning
Venue: arXiv cs.CV 2021
Year: 2021

Recently, self-supervised methods show remarkable achievements in image-level representation learning. Nevertheless, their image-level self-supervisions lead the learned representation to sub-optimal for dense prediction tasks, such as object detection, instance segmentation, etc. To tackle this issue, several recent self-supervised learning methods have extended image-level single embedding to pixel-level dense embeddings. Unlike image-level representation learning, due to the spatial deformation of augmentation, it is difficult to sample pixel-level positive pairs. Previous studies have sampled pixel-level positive pairs using the winner-takes-all among similarity or thresholding warped distance between dense embeddings. However, these naive methods can be struggled by background clutter and outliers problems. In this paper, we introduce Hough Contrastive Learning (HoughCL), a Hough space based method that enforces geometric consistency between two dense features. HoughCL achieves robustness against background clutter and outliers. Furthermore, compared to baseline, our dense positive pairing method has no additional learnable parameters and has a small extra computation cost. Compared to previous works, our method shows better or comparable performance on dense prediction fine-tuning tasks.
WEM: A Node Importance Algorithm in Weighted Networks
Venue: arXiv cs.SI 2021
Year: 2021

In view of the node importance in weighted networks, weighted expected method (WEM), was proposed in this paper, which take an advantages of uncertain graph algorithm. First, a weight processing method is proposed based on the relationship between the weight of edges and the intensity of contact between nodes, and the calculation method of the contribution of the weight of edges to the node importance is defined, even if there are two converse situations in the reality. Then, because of the use of dynamic programming method, which reduces the complexity of computational time to a linear level, WEM will be more suitable for the calculation in large weighted networks. Owning to its features of calculating, WEM can, to the greatest extent, ensure a precise order of node essential scores for each node. The node connectivity experiment and SIR simulation experiment show that the WEM has higher accuracy and relatively low time complexity.
Jointly Dynamic Topic Model for Recognition of Lead-lag Relationship in Two Text Corpora
Venue: arXiv cs.CL 2021
Year: 2021

Topic evolution modeling has received significant attentions in recent decades. Although various topic evolution models have been proposed, most studies focus on the single document corpus. However in practice, we can easily access data from multiple sources and also observe relationships between them. Then it is of great interest to recognize the relationship between multiple text corpora and further utilize this relationship to improve topic modeling. In this work, we focus on a special type of relationship between two text corpora, which we define as the "lead-lag relationship". This relationship characterizes the phenomenon that one text corpus would influence the topics to be discussed in the other text corpus in the future. To discover the lead-lag relationship, we propose a jointly dynamic topic model and also develop an embedding extension to address the modeling problem of large-scale text corpus. With the recognized lead-lag relationship, the similarities of the two text corpora can be figured out and the quality of topic learning in both corpora can be improved. We numerically investigate the performance of the jointly dynamic topic modeling approach using synthetic data. Finally, we apply the proposed model on two text corpora consisting of statistical papers and the graduation theses. Results show the proposed model can well recognize the lead-lag relationship between the two corpora, and the specific and shared topic patterns in the two corpora are also discovered.
Surprise Minimization Revision Operators
Venue: arXiv cs.AI 2021
Year: 2021

Prominent approaches to belief revision prescribe the adoption of a new belief that is as close as possible to the prior belief, in a process that, even in the standard case, can be described as attempting to minimize surprise. Here we extend the existing model by proposing a measure of surprise, dubbed relative surprise, in which surprise is computed with respect not just to the prior belief, but also to the broader context provided by the new information, using a measure derived from familiar distance notions between truth-value assignments. We characterize the surprise minimization revision operator thus defined using a set of intuitive rationality postulates in the AGM mould, along the way obtaining representation results for other existing revision operators in the literature, such as the Dalal operator and a recently introduced distance-based min-max operator.
Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation
Venue: arXiv cs.CV 2021
Year: 2021

As the adoption of deep learning techniques in industrial applications grows with increasing speed and scale, successful deployment of deep learning models often hinges on the availability, volume, and quality of annotated data. In this paper, we tackle the problems of efficient data labeling and annotation verification under the human-in-the-loop setting. We showcase that the latest advancements in the field of self-supervised visual representation learning can lead to tools and methods that benefit the curation and engineering of natural image datasets, reducing annotation cost and increasing annotation quality. We propose a unifying framework by leveraging self-supervised semi-supervised learning and use it to construct workflows for data labeling and annotation verification tasks. We demonstrate the effectiveness of our workflows over existing methodologies. On active learning task, our method achieves 97.0% Top-1 Accuracy on CIFAR10 with 0.1% annotated data, and 83.9% Top-1 Accuracy on CIFAR100 with 10% annotated data. When learning with 50% of wrong labels, our method achieves 97.4% Top-1 Accuracy on CIFAR10 and 85.5% Top-1 Accuracy on CIFAR100.
Adaptive Transfer Learning: a simple but effective transfer learning
Venue: arXiv cs.LG 2021
Year: 2021

Transfer learning (TL) leverages previously obtained knowledge to learn new tasks efficiently and has been used to train deep learning (DL) models with limited amount of data. When TL is applied to DL, pretrained (teacher) models are fine-tuned to build domain specific (student) models. This fine-tuning relies on the fact that DL model can be decomposed to classifiers and feature extractors, and a line of studies showed that the same feature extractors can be used to train classifiers on multiple tasks. Furthermore, recent studies proposed multiple algorithms that can fine-tune teacher models' feature extractors to train student models more efficiently. We note that regardless of the fine-tuning of feature extractors, the classifiers of student models are trained with final outputs of feature extractors (i.e., the outputs of penultimate layers). However, a recent study suggested that feature maps in ResNets across layers could be functionally equivalent, raising the possibility that feature maps inside the feature extractors can also be used to train student models' classifiers. Inspired by this study, we tested if feature maps in the hidden layers of the teacher models can be used to improve the student models' accuracy (i.e., TL's efficiency). Specifically, we developed 'adaptive transfer learning (ATL)', which can choose an optimal set of feature maps for TL, and tested it in the few-shot learning setting. Our empirical evaluations suggest that ATL can help DL models learn more efficiently, especially when available examples are limited.
A Dynamic Programming Algorithm to Compute Joint Distribution of Order Statistics on Graphs
Venue: arXiv cs.DS 2021
Year: 2021

Order statistics play a fundamental role in statistical procedures such as risk estimation, outlier detection, and multiple hypothesis testing as well as in the analyses of mechanism design, queues, load balancing, and various other logistical processes involving ranks. In some of these cases, it may be desirable to compute the \textit{exact} values from the joint distribution of $d$ order statistics. While this problem is already computationally difficult even in the case of $n$ independent random variables, the random variables often have no such independence guarantees. Existing methods obtain the cumulative distribution indirectly by first computing and then aggregating over the marginal distributions. In this paper, we provide a more direct, efficient algorithm to compute cumulative joint order statistic distributions of dependent random variables that improves an existing dynamic programming solution via dimensionality reduction techniques. Our solution guarantees a $O(\frac{d^{d-1}}{n})$ and $O(d^{d})$ factor of improvement in both time and space complexity respectively over previous methods.
Gradient Temporal Difference with Momentum: Stability and Convergence
Venue: arXiv cs.LG 2021
Year: 2021

Gradient temporal difference (Gradient TD) algorithms are a popular class of stochastic approximation (SA) algorithms used for policy evaluation in reinforcement learning. Here, we consider Gradient TD algorithms with an additional heavy ball momentum term and provide choice of step size and momentum parameter that ensures almost sure convergence of these algorithms asymptotically. In doing so, we decompose the heavy ball Gradient TD iterates into three separate iterates with different step sizes. We first analyze these iterates under one-timescale SA setting using results from current literature. However, the one-timescale case is restrictive and a more general analysis can be provided by looking at a three-timescale decomposition of the iterates. In the process, we provide the first conditions for stability and convergence of general three-timescale SA. We then prove that the heavy ball Gradient TD algorithm is convergent using our three-timescale SA analysis. Finally, we evaluate these algorithms on standard RL problems and report improvement in performance over the vanilla algorithms.