OpenRaiser/Intern-Atlas huggingface dataset:
Intrinsically Sparse Long Short-Term Memory Networks
Venue: arXiv cs.NE 2019
Year: 2019

Long Short-Term Memory (LSTM) has achieved state-of-the-art performances on a wide range of tasks. Its outstanding performance is guaranteed by the long-term memory ability which matches the sequential data perfectly and the gating structure controlling the information flow. However, LSTMs are prone to be memory-bandwidth limited in realistic applications and need an unbearable period of training and inference time as the model size is ever-increasing. To tackle this problem, various efficient model compression methods have been proposed. Most of them need a big and expensive pre-trained model which is a nightmare for resource-limited devices where the memory budget is strictly limited. To remedy this situation, in this paper, we incorporate the Sparse Evolutionary Training (SET) procedure into LSTM, proposing a novel model dubbed SET-LSTM. Rather than starting with a fully-connected architecture, SET-LSTM has a sparse topology and dramatically fewer parameters in both phases, training and inference. Considering the specific architecture of LSTMs, we replace the LSTM cells and embedding layers with sparse structures and further on, use an evolutionary strategy to adapt the sparse connectivity to the data. Additionally, we find that SET-LSTM can provide many different good combinations of sparse connectivity to substitute the overparameterized optimization problem of dense neural networks. Evaluated on four sentiment analysis classification datasets, the results demonstrate that our proposed model is able to achieve usually better performance than its fully connected counterpart while having less than 4\% of its parameters.
What Do Multiwinner Voting Rules Do? An Experiment Over the Two-Dimensional Euclidean Domain
Venue: arXiv cs.GT 2019
Year: 2019

We visualize aggregate outputs of popular multiwinner voting rules--SNTV, STV, Bloc, k-Borda, Monroe, Chamberlin--Courant, and HarmonicBorda--for elections generated according to the two-dimensional Euclidean model. We consider three applications of multiwinner voting, namely, parliamentary elections, portfolio/movie selection, and shortlisting, and use our results to understand which of our rules seem to be best suited for each application. In particular, we show that STV (one of the few nontrivial rules used in real high-stake elections) exhibits excellent performance, whereas the Bloc rule (also often used in practice) performs poorly.
Capacity of Single-Server Single-Message Private Information Retrieval with Private Coded Side Information
Venue: arXiv cs.IT 2019
Year: 2019

We study the problem of single-server single-message Private Information Retrieval with Private Coded Side Information (PIR-PCSI). In this problem, there is a server that stores a database, and a user who knows a random linear combination of a random subset of messages in the database. The number of messages contributing to the user's side information is known to the server a priori, whereas their indices and coefficients are unknown to the server a priori. The user wants to retrieve a message from the server (with minimum download cost), while protecting the identities of both the demand and side information messages. Depending on whether the demand is part of the coded side information or not, we consider two different models for the problem. For the model in which the demand does not contribute to the side information, we prove a lower bound on the minimum download cost for all (linear and non-linear) PIR protocols; and for the other model wherein the demand is one of the messages contributing to the side information, we prove a lower bound for all scalar-linear PIR protocols. In addition, we propose novel PIR protocols that achieve these lower bounds.
Soft labeling by Distilling Anatomical knowledge for Improved MS Lesion Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

This paper explores the use of a soft ground-truth mask ("soft mask'') to train a Fully Convolutional Neural Network (FCNN) for segmentation of Multiple Sclerosis (MS) lesions. Detection and segmentation of MS lesions is a complex task largely due to the extreme unbalanced data, with very small number of lesion pixels that can be used for training. Utilizing the anatomical knowledge that the lesion surrounding pixels may also include some lesion level information, we suggest to increase the data set of the lesion class with neighboring pixel data - with a reduced confidence weight. A soft mask is constructed by morphological dilation of the binary segmentation mask provided by a given expert, where expert-marked voxels receive label 1 and voxels of the dilated region are assigned a soft label. In the methodology proposed, the FCNN is trained using the soft mask. On the ISBI 2015 challenge dataset, this is shown to provide a better precision-recall tradeoff and to achieve a higher average Dice similarity coefficient. We also show that by using this soft mask scheme we can improve the network segmentation performance when compared to a second independent expert.
On the mapping of Points of Interest through StreetView imagery and paid crowdsourcing
Venue: arXiv cs.CY 2019
Year: 2019

The use of volunteers has emerged as low-cost alternative to generate accurate geographical information, an approach known as Volunteered Geographic Information (VGI). However, VGI is limited by the number and availability of volunteers in the area to be mapped, hindering scalability for large areas and making difficult to map within a time-frame. Fortunately, the availability of street-view imagery enables the virtual exploration of urban environments, making possible the recruitment of contributors not necessarily located in the area to be mapped. In this paper, we describe the design, implementation, and evaluation of the Virtual City Explorer (VCE), a system to collect the coordinates of Points of Interest within a bounded area on top of a street view service with the use of paid crowdworkers. Our evaluation suggests that paid crowdworkers are effective for finding PoIs, and cover almost all the area. With respect to completeness, our approach does not find all PoIs found by experts or VGI communities, but is able to find PoIs that were not found by them, suggesting complementarity. We also studied the impact of making PoIs already discovered by a certain number of workers \emph{taboo} for incoming workers, finding that it encourages more exploration from workers , increase the number of detected PoIs , and reduce costs.
The CATS Hackathon: Creating and Refining Test Items for Cybersecurity Concept Inventories
Venue: arXiv cs.CR 2019
Year: 2019

For two days in February 2018, 17 cybersecurity educators and professionals from government and industry met in a "hackathon" to refine existing draft multiple-choice test items, and to create new ones, for a Cybersecurity Concept Inventory (CCI) and Cybersecurity Curriculum Assessment (CCA) being developed as part of the Cybersecurity Assessment Tools (CATS) Project. We report on the results of the CATS Hackathon, discussing the methods we used to develop test items, highlighting the evolution of a sample test item through this process, and offering suggestions to others who may wish to organize similar hackathons. Each test item embodies a scenario, question stem, and five answer choices. During the Hackathon, participants organized into teams to (1) Generate new scenarios and question stems, (2) Extend CCI items into CCA items, and generate new answer choices for new scenarios and stems, and (3) Review and refine draft CCA test items. The CATS Project provides rigorous evidence-based instruments for assessing and evaluating educational practices; these instruments can help identify pedagogies and content that are effective in teaching cybersecurity. The CCI measures how well students understand basic concepts in cybersecurity---especially adversarial thinking---after a first course in the field. The CCA measures how well students understand core concepts after completing a full cybersecurity curriculum.
Reward Shaping via Meta-Learning
Venue: arXiv cs.LG 2019
Year: 2019

Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL). However, designing shaping functions usually requires much expert knowledge and hand-engineering, and the difficulties are further exacerbated given multiple similar tasks to solve. In this paper, we consider reward shaping on a distribution of tasks, and propose a general meta-learning framework to automatically learn the efficient reward shaping on newly sampled tasks, assuming only shared state space but not necessarily action space. We first derive the theoretically optimal reward shaping in terms of credit assignment in model-free RL. We then propose a value-based meta-learning algorithm to extract an effective prior over the optimal reward shaping. The prior can be applied directly to new tasks, or provably adapted to the task-posterior while solving the task within few gradient updates. We demonstrate the effectiveness of our shaping through significantly improved learning efficiency and interpretable visualizations across various settings, including notably a successful transfer from DQN to DDPG.
Concrete Autoencoders for Differentiable Feature Selection and Reconstruction
Venue: arXiv cs.LG 2019
Year: 2019

We introduce the concrete autoencoder, an end-to-end differentiable method for global feature selection, which efficiently identifies a subset of the most informative features and simultaneously learns a neural network to reconstruct the input data from the selected features. Our method is unsupervised, and is based on using a concrete selector layer as the encoder and using a standard neural network as the decoder. During the training phase, the temperature of the concrete selector layer is gradually decreased, which encourages a user-specified number of discrete features to be learned. During test time, the selected features can be used with the decoder network to reconstruct the remaining input features. We evaluate concrete autoencoders on a variety of datasets, where they significantly outperform state-of-the-art methods for feature selection and data reconstruction. In particular, on a large-scale gene expression dataset, the concrete autoencoder selects a small subset of genes whose expression levels can be use to impute the expression levels of the remaining genes. In doing so, it improves on the current widely-used expert-curated L1000 landmark genes, potentially reducing measurement costs by 20%. The concrete autoencoder can be implemented by adding just a few lines of code to a standard autoencoder.
End-to-End Performance Analysis of Underwater Optical Wireless Relaying and Routing Techniques Under Location Uncertainty
Venue: arXiv cs.NI 2019
Year: 2019

On the contrary of low speed and high delay acoustic systems, underwater optical wireless communication (UOWC) can deliver a high speed and low latency service at the expense of short communication ranges. Therefore, multihop communication is of utmost importance to improve degree of connectivity and overall performance of underwater optical wireless networks (UOWNs). In this regard, this paper investigates relaying and routing techniques and provides their end-to-end (E2E) performance analysis under the location uncertainty. To achieve robust and reliable links, we first consider adaptive beamwidths and derive the divergence angles under the absence and presence of a pointing-acquisitioning-and-tracking (PAT) mechanism. Thereafter, important E2E performance metrics (e.g., data rate, bit error rate, transmission power, amplifier gain, etc.) are obtained for two potential relaying techniques; decode & forward (DF) and optical amplify & forward (AF). We develop centralized routing schemes for both relaying techniques to optimize E2E rate, bit error rate, and power consumption. Alternatively, a distributed routing protocol, namely Light Path Routing (LiPaR), is proposed by leveraging the range-beamwidth tradeoff of UOWCs. LiPaR is especially shown to be favorable when there is no PAT mechanism and available network information. In order to show the benefits of multihop communications, extensive simulations are conducted to compare different routing and relaying schemes under different network parameters and underwater environments.
NeuralSampler: Euclidean Point Cloud Auto-Encoder and Sampler
Venue: arXiv cs.CV 2019
Year: 2019

Most algorithms that rely on deep learning-based approaches to generate 3D point sets can only produce clouds containing fixed number of points. Furthermore, they typically require large networks parameterized by many weights, which makes them hard to train. In this paper, we propose an auto-encoder architecture that can both encode and decode clouds of arbitrary size and demonstrate its effectiveness at upsampling sparse point clouds. Interestingly, we can do so using less than half as many parameters as state-of-the-art architectures while still delivering better performance. We will make our code base fully available.
Degraded Historical Documents Images Binarization Using a Combination of Enhanced Techniques
Venue: arXiv cs.CV 2019
Year: 2019

Document image binarization is the initial step and a crucial in many document analysis and recognition scheme. In fact, it is still a relevant research subject and a fundamental challenge due to its importance and influence. This paper provides an original multi-phases system that hybridizes various efficient image thresholding methods in order to get the best binarization output. First, to improve contrast in particularly defective images, the application of CLAHE algorithm is suggested and justified. We then use a cooperative technique to segment image into two separated classes. At the end, a special transformation is applied for the purpose of removing scattered noise and of correcting characters forms. Experimentations demonstrate the precision and the robustness of our framework applied on historical degraded documents images within three benchmarks compared to other noted methods.
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
Venue: arXiv cs.IR 2019
Year: 2019

We present a large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives. We analyze the potential allocation harms that can result from semantic representation bias. To do so, we study the impact on occupation classification of including explicit gender indicators---such as first names and pronouns---in different semantic representations of online biographies. Additionally, we quantify the bias that remains when these indicators are "scrubbed," and describe proxy behavior that occurs in the absence of explicit gender indicators. As we demonstrate, differences in true positive rates between genders are correlated with existing gender imbalances in occupations, which may compound these imbalances.
An End-to-End Solution for Effectively Demoting Watermarked Images in Image Search
Venue: arXiv cs.CV 2019
Year: 2019

We propose an end-to-end solution, from watermark feature generation to metric design, for effectively demoting watermarked images surfed by a real world image search engine. We use a few fundamental techniques to obtain effective watermark features of images in the image search index, and utilize the signals in a commercial search engine to improve the image search quality. We collect a diverse and large set (about 1M) of images with human labels indicating whether the image contains visible watermark. We train a few deep convolutional neural networks to extract watermark information from the raw images. The deep CNN classifiers we trained can achieve high accuracy on the watermark test data set. We also analyze the images based on their domains to get watermark information from a domain-based watermark classifier. We design a new novel hybrid metric which includes the relevance, image attractiveness and watermark information all together. We demonstrate that using these watermark signals together with the new metric in image search ranker can significantly demote the watermarked images during the online image ranking.
End-to-End Discriminative Deep Network for Liver Lesion Classification
Venue: arXiv cs.CV 2019
Year: 2019

Colorectal liver metastasis is one of most aggressive liver malignancies. While the definition of lesion type based on CT images determines the diagnosis and therapeutic strategy, the discrimination between cancerous and non-cancerous lesions are critical and requires highly skilled expertise, experience and time. In the present work we introduce an end-to-end deep learning approach to assist in the discrimination between liver metastases from colorectal cancer and benign cysts in abdominal CT images of the liver. Our approach incorporates the efficient feature extraction of InceptionV3 combined with residual connections and pre-trained weights from ImageNet. The architecture also includes fully connected classification layers to generate a probabilistic output of lesion type. We use an in-house clinical biobank with 230 liver lesions originating from 63 patients. With an accuracy of 0.96 and a F1-score of 0.92, the results obtained with the proposed approach surpasses state of the art methods. Our work provides the basis for incorporating machine learning tools in specialized radiology software to assist physicians in the early detection and treatment of liver lesions.
Stochastic Linear Bandits with Hidden Low Rank Structure
Venue: arXiv cs.LG 2019
Year: 2019

High-dimensional representations often have a lower dimensional underlying structure. This is particularly the case in many decision making settings. For example, when the representation of actions is generated from a deep neural network, it is reasonable to expect a low-rank structure whereas conventional structures like sparsity are not valid anymore. Subspace recovery methods, such as Principle Component Analysis (PCA) can find the underlying low-rank structures in the feature space and reduce the complexity of the learning tasks. In this work, we propose Projected Stochastic Linear Bandit (PSLB), an algorithm for high dimensional stochastic linear bandits (SLB) when the representation of actions has an underlying low-dimensional subspace structure. PSLB deploys PCA based projection to iteratively find the low rank structure in SLBs. We show that deploying projection methods assures dimensionality reduction and results in a tighter regret upper bound that is in terms of the dimensionality of the subspace and its properties, rather than the dimensionality of the ambient space. We modify the image classification task into the SLB setting and empirically show that, when a pre-trained DNN provides the high dimensional feature representations, deploying PSLB results in significant reduction of regret and faster convergence to an accurate model compared to state-of-art algorithm.
Neural Related Work Summarization with a Joint Context-driven Attention Mechanism
Venue: arXiv cs.CL 2019
Year: 2019

Conventional solutions to automatic related work summarization rely heavily on human-engineered features. In this paper, we develop a neural data-driven summarizer by leveraging the seq2seq paradigm, in which a joint context-driven attention mechanism is proposed to measure the contextual relevance within full texts and a heterogeneous bibliography graph simultaneously. Our motivation is to maintain the topic coherency between a related work section and its target document, where both the textual and graphic contexts play a big role in characterizing the relationship among scientific publications accurately. Experimental results on a large dataset show that our approach achieves a considerable improvement over a typical seq2seq summarizer and five classical summarization baselines.
User Donations in a Crowdsourced Video System
Venue: arXiv cs.SI 2019
Year: 2019

Crowdsourced video systems like YouTube and Twitch.tv have been a major internet phenomenon and are nowadays entertaining over a billion users. In addition to video sharing and viewing, over the years they have developed new features to boost the community engagement and some managed to attract users to donate, to the community as well as to other users. User donation directly reflects and influences user engagement in the community, and has a great impact on the success of such systems. Nevertheless, user donations in crowdsourced video systems remain trade secrets for most companies and to date are still unexplored. In this work, we attempt to fill this gap, and we obtain and provide a publicly available dataset on user donations in one crowdsourced video system named BiliBili. Based on information on nearly 40 thousand donators, we examine the dynamics of user donations and their social relationships, we quantitively reveal the factors that potentially impact user donation, and we adopt machine-learned classifiers and network representation learning models to timely and accurately predict the destinations of the majority and the individual donations.
Techniques for System Information Broadcast in Cell-Free Massive MIMO
Venue: arXiv cs.IT 2019
Year: 2019

We consider transmission of system information in a cell-free massive MIMO system, when the transmitting access points do not have any channel state information and the receiving terminal has to estimate the channel based on downlink pilots. We analyze the system performance in terms of outage rate and coverage probability, and use space-time block codes to increase performance. We propose a heuristic method for pilot/data power optimization that can be applied without any channel state information at the access points. We also analyze the problem of grouping the access points, which is needed when the single-antenna access points jointly transmit a space-time block code.
Secrecy Throughput Maximization for Full-Duplex Wireless Powered IoT Networks under Fairness Constraints
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we study the secrecy throughput of a full-duplex wireless powered communication network (WPCN) for internet of things (IoT). The WPCN consists of a full-duplex multi-antenna base station (BS) and a number of sensor nodes. The BS transmits energy all the time, and each node harvests energy prior to its transmission time slot. The nodes sequentially transmit their confidential information to the BS, and the other nodes are considered as potential eavesdroppers. We first formulate the sum secrecy throughput optimization problem of all the nodes. The optimization variables are the duration of the time slots and the BS beamforming vectors in different time slots. The problem is shown to be non-convex. To tackle the problem, we propose a suboptimal two stage approach, referred to as sum secrecy throughput maximization (SSTM). In the first stage, the BS focuses its beamforming to blind the potential eavesdroppers (other nodes) during information transmission time slots. Then, the optimal beamforming vector in the initial non-information transmission time slot and the optimal time slots are derived. We then consider fairness among the nodes and propose max-min fair (MMF) and proportional fair (PLF) algorithms. The MMF algorithm maximizes the minimum secrecy throughput of the nodes, while the PLF tries to achieve a good trade-off between the sum secrecy throughput and fairness among the nodes. Through numerical simulations, we first demonstrate the superior performance of the SSTM to uniform time slotting and beamforming in different settings. Then, we show the effectiveness of the proposed fair algorithms.