OpenRaiser/Intern-Atlas huggingface dataset:
Unconventional Bio-Inspired Model for Design of Logic Gates
Venue: arXiv cs.ET 2020
Year: 2020

During the last years, a well studied biological substrate, namely Physarum polycephalum, has been proven efficient on finding appropriate and efficient solutions in hard to solve complex mathematical problems. The plasmodium of P. polycephalum is a single-cell that serves as a prosperous bio-computational example. Consequently, it has been successfully utilized in the past to solve a variety of path problems in graphs and combinatorial problems. In this work, this interesting behaviour is mimicked by a robust unconventional computational model, drawing inspiration from the notion of Cellular and Learning Automata. Namely, we employ principles of Cellular Automata (CAs) enriched with learning capabilities to develop a robust computational model, able of modelling appropriately the aforementioned biological substrate and, thus, capturing its computational capabilities. CAs are very efficient in modelling biological systems and solving scientific problems, owing to their ability of incarnating essential properties of a system where global behaviour arises as an effect of simple components, interacting locally. The resulting computational tool, after combining CAs with learning capabilities, should be appropriate for modelling the behaviour of living organisms. Thus, the inherent abilities and computational characteristics of the proposed bio-inspired model are stressed towards the experimental verification of Physarum's ability to model Logic Gates, while trying to find minimal paths in properly configured mazes with food sources. The presented simulation results for various Logic Gates are found in good agreement, both qualitatively and quantitatively, with the corresponding experimental results, proving the efficacy of this unconventional bio-inspired model and providing useful insights for its enhanced usage in various computing applications.
Fairness through Experimentation: Inequality in A/B testing as an approach to responsible design
Venue: arXiv cs.SI 2020
Year: 2020

As technology continues to advance, there is increasing concern about individuals being left behind. Many businesses are striving to adopt responsible design practices and avoid any unintended consequences of their products and services, ranging from privacy vulnerabilities to algorithmic bias. We propose a novel approach to fairness and inclusiveness based on experimentation. We use experimentation because we want to assess not only the intrinsic properties of products and algorithms but also their impact on people. We do this by introducing an inequality approach to A/B testing, leveraging the Atkinson index from the economics literature. We show how to perform causal inference over this inequality measure. We also introduce the concept of site-wide inequality impact, which captures the inclusiveness impact of targeting specific subpopulations for experiments, and show how to conduct statistical inference on this impact. We provide real examples from LinkedIn, as well as an open-source, highly scalable implementation of the computation of the Atkinson index and its variance in Spark/Scala. We also provide over a year's worth of learnings -- gathered by deploying our method at scale and analyzing thousands of experiments -- on which areas and which kinds of product innovations seem to inherently foster fairness through inclusiveness.
Cooperative NOMA for Downlink Asymmetric Interference Cancellation
Venue: arXiv cs.IT 2020
Year: 2020

This letter advances the non-orthogonal multiple access (NOMA) technique for cellular downlink co-channel interference mitigation, via exploiting the (limited) cooperation among base stations (BSs). Specifically, we consider a simplified but practically relevant scenario of two co-channel cells with asymmetric interference, i.e., only the user in one cell receives the strong interference from the BS in the other cell. To mitigate such interference, we propose a new cooperative NOMA scheme, where the interfered user's serving BS sends a superposed signal comprising both the desired message and the co-channel user's message (shared by the interfering BS). The co-channel user's signal is aimed to add constructively with the interfering BS's signal at the interfered user's receiver so that the combined interference with enhanced power can be effectively decoded and cancelled. This thus leads to a new problem on how to optimally allocate the transmit power for the two superposed messages. We provide the closed-form solution to this problem and investigate the conditions under which the performance of the proposed scheme is superior over the existing schemes.
AoI-Delay Tradeoff in Mobile Edge Caching with Freshness-Aware Content Refreshing
Venue: arXiv cs.NI 2020
Year: 2020

Mobile edge caching can effectively reduce service delay but may introduce information staleness, calling for timely content refreshing. However, content refreshing consumes additional transmission resources and may degrade the delay performance of mobile systems. In this work, we propose a freshness-aware refreshing scheme to balance the service delay and content freshness measured by Age of Information (AoI). Specifically, the cached content items will be refreshed to the up-to-date version upon user requests if the AoI exceeds a certain threshold (named as refreshing window). The average AoI and service delay are derived in closed forms approximately, which reveals an AoI-delay tradeoff relationship with respect to the refreshing window. In addition, the refreshing window is optimized to minimize the average delay while meeting the AoI requirements, and the results indicate to set a smaller refreshing window for the popular content items. Extensive simulations are conducted on the OMNeT++ platform to validate the analytical results. The results indicate that the proposed scheme can restrain frequent refreshing as the request arrival rate increases, whereby the average delay can be reduced by around 80% while maintaining the AoI below one second in heavily-loaded scenarios.
Understanding patient complaint characteristics using contextual clinical BERT embeddings
Venue: arXiv cs.CL 2020
Year: 2020

In clinical conversational applications, extracted entities tend to capture the main subject of a patient's complaint, namely symptoms or diseases. However, they mostly fail to recognize the characterizations of a complaint such as the time, the onset, and the severity. For example, if the input is "I have a headache and it is extreme", state-of-the-art models only recognize the main symptom entity - headache, but ignore the severity factor of "extreme", that characterizes headache. In this paper, we design a two-stage approach to detect the characterizations of entities like symptoms presented by general users in contexts where they would describe their symptoms to a clinician. We use Word2Vec and BERT to encode clinical text given by the patients. We transform the output and re-frame the task as multi-label classification problem. Finally, we combine the processed encodings with the Linear Discriminant Analysis (LDA) algorithm to classify the characterizations of the main entity. Experimental results demonstrate that our method achieves 40-50% improvement on the accuracy over the state-of-the-art models.
Approximating Voting Rules from Truncated Ballots
Venue: arXiv cs.GT 2020
Year: 2020

Classical voting rules assume that ballots are complete preference orders over candidates. However, when the number of candidates is large enough, it is too costly to ask the voters to rank all candidates. We suggest to fix a rank k, to ask all voters to specify their best k candidates, and then to consider "top-k approximations" of rules, which take only into account the top-k candidates of each ballot. We consider two measures of the quality of the approximation: the probability of selecting the same winner as the original rule, and the score ratio. We do a worst-case study (for the latter measure only), and for both measures, an average-case study and a study from real data sets.
Traffic Modelling and Prediction via Symbolic Regression on Road Sensor Data
Venue: arXiv cs.NE 2020
Year: 2020

The continuous expansion of the urban traffic sensing infrastructure has led to a surge in the volume of widely available road related data. Consequently, increasing effort is being dedicated to the creation of intelligent transportation systems, where decisions on issues ranging from city-wide road maintenance planning to improving the commuting experience are informed by computational models of urban traffic instead of being left entirely to humans. The automation of traffic management has received substantial attention from the research community, however, most approaches target highways, produce predictions valid for a limited time window or require expensive retraining of available models in order to accurately forecast traffic at a new location. In this article, we propose a novel and accurate traffic flow prediction method based on symbolic regression enhanced with a lag operator. Our approach produces robust models suitable for the intricacies of urban roads, much more difficult to predict than highways. Additionally, there is no need to retrain the model for a period of up to 9 weeks. Furthermore, the proposed method generates models that are transferable to other segments of the road network, similar to, yet geographically distinct from the ones they were initially trained on. We demonstrate the achievement of these claims by conducting extensive experiments on data collected from the Darmstadt urban infrastructure.
Optimizing Non-Orthogonal Multiple Access in Random Access Networks
Venue: arXiv cs.NI 2020
Year: 2020

Non-orthogonal multiple access (NOMA) has been considered as a promising solution for improving the spectrum efficiency of next-generation wireless networks. In this paper, the performance of a p-persistent slotted ALOHA system in support of NOMA transmissions is investigated. Specifically, wireless users can choose to use high or low power for data transmissions with certain probabilities. To achieve the maximum network throughput, an analytical framework is developed to analyze the successful transmission probability of NOMA and long term average throughput of users involved in the non-orthogonal transmissions. The feasible region of the maximum number of concurrent users using high and low power to ensure successful NOMA transmissions are quantified. Based on the analysis, an algorithm is proposed to find the optimal transmission probabilities for users to choose high and low power to achieve the maximum system throughput. In addition, the impact of power settings on the network performance is further investigated. Simulations are conducted to validate the analysis.
Combining Parametric Land Surface Models with Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

A hybrid machine learning and process-based-modeling (PBM) approach is proposed and evaluated at a handful of AmeriFlux sites to simulate the top-layer soil moisture state. The Hybrid-PBM (HPBM) employed here uses the Noah land-surface model integrated with Gaussian Processes. It is designed to correct the model only in climatological situations similar to the training data else it reverts to the PBM. In this way, our approach avoids bad predictions in scenarios where similar training data is not available and incorporates our physical understanding of the system. Here we assume an autoregressive model and obtain out-of-sample results with upwards of a 3-fold reduction in the RMSE using a one-year leave-one-out cross-validation at each of the selected sites. A path is outlined for using hybrid modeling to build global land-surface models with the potential to significantly outperform the current state-of-the-art.
Cleaning Denial Constraint Violations through Relaxation
Venue: arXiv cs.DB 2020
Year: 2020

Data cleaning is a time-consuming process that depends on the data analysis that users perform. Existing solutions treat data cleaning as a separate offline process that takes place before analysis begins. Applying data cleaning before analysis assumes a priori knowledge of the inconsistencies and the query workload, thereby requiring effort on understanding and cleaning the data that is unnecessary for the analysis. We propose an approach that performs probabilistic repair of denial constraint violations on-demand, driven by the exploratory analysis that users perform. We introduce Daisy, a system that seamlessly integrates data cleaning into the analysis by relaxing query results. Daisy executes analytical query-workloads over dirty data by weaving cleaning operators into the query plan. Our evaluation shows that Daisy adapts to the workload and outperforms traditional offline cleaning on both synthetic and real-world workloads.
Transformer on a Diet
Venue: arXiv cs.CL 2020
Year: 2020

Transformer has been widely used thanks to its ability to capture sequence information in an efficient way. However, recent developments, such as BERT and GPT-2, deliver only heavy architectures with a focus on effectiveness. In this paper, we explore three carefully-designed light Transformer architectures to figure out whether the Transformer with less computations could produce competitive results. Experimental results on language model benchmark datasets hint that such trade-off is promising, and the light Transformer reduces 70% parameters at best, while obtains competitive perplexity compared to standard Transformer. The source code is publicly available.
Simultaneous Energy Harvesting and Information Transmission in a MIMO Full-Duplex System: A Machine Learning-Based Design
Venue: arXiv cs.IT 2020
Year: 2020

We propose a multiple-input multiple-output (MIMO)-based full-duplex (FD) scheme that enables wireless devices to simultaneously transmit information and harvest energy using the same time-frequency resources. In this scheme, for a MIMO point-to-point set up, the energy transmitting device simultaneously receives information from the energy harvesting device. Furthermore, the self-interference (SI) at the energy harvesting device caused by the FD mode of operation is utilized as a desired power signal to be harvested by the device. For implementation-friendly antenna selection and MIMO precoding at both the devices, we propose two methods: (i) a sub-optimal method based on relaxation, and (ii) a hybrid deep reinforcement learning (DRL)-based method, specifically, a deep deterministic policy gradient (DDPG)-deep double Q-network (DDQN) method. Finally, we study the performance of the proposed system under the two implementation methods and compare it with that of the conventional time switching-based simultaneous wireless information and power transfer (SWIPT) method. Findings show that the proposed system gives a significant improvement in spectral efficiency compared to the time switching-based SWIPT. In particular, the DRL-based method provides the highest spectral efficiency. Furthermore, numerical results show that, for the considered system set up, the number of antennas in each device should exceed three to mitigate self-interference to an acceptable level.
Simulation Pipeline for Traffic Evacuation in Urban Areas and Emergency Traffic Management Policy Improvements through Case Studies
Venue: arXiv cs.AI 2020
Year: 2020

Traffic evacuation plays a critical role in saving lives in devastating disasters such as hurricanes, wildfires, floods, earthquakes, etc. An ability to evaluate evacuation plans in advance for these rare events, including identifying traffic flow bottlenecks, improving traffic management policies, and understanding the robustness of the traffic management policy are critical for emergency management. Given the rareness of such events and the corresponding lack of real data, traffic simulation provides a flexible and versatile approach for such scenarios, and furthermore allows dynamic interaction with the simulated evacuation. In this paper, we build a traffic simulation pipeline to explore the above problems, covering many aspects of evacuation, including map creation, demand generation, vehicle behavior, bottleneck identification, traffic management policy improvement, and results analysis. We apply the pipeline to two case studies in California. The first is Paradise, which was destroyed by a large wildfire in 2018 and experienced catastrophic traffic jams during the evacuation. The second is Mill Valley, which has high risk of wildfire and potential traffic issues since the city is situated in a narrow valley.
Sthread: In-Vivo Model Checking of Multithreaded Programs
Venue: arXiv cs.PL 2020
Year: 2020

This work strives to make formal verification of POSIX multithreaded programs easily accessible to general programmers. Sthread operates directly on multithreaded C/C++ programs, without the need for an intermediate formal model. Sthread is in-vivo in that it provides a drop-in replacement for the pthread library, and operates directly on the compiled target executable and application libraries. There is no compiler-generated intermediate representation. The system calls in the application remain unaltered. Optionally, the programmer can add a small amount of additional native C code to include assertions based on the user's algorithm, declarations of shared memory regions, and progress/liveness conditions. The work has two important motivations: (i) It can be used to verify correctness of a concurrent algorithm being implemented with multithreading; and (ii) it can also be used pedagogically to provide immediate feedback to students learning either to employ POSIX threads system calls or to implement multithreaded algorithms. This work represents the first example of in-vivo model checking operating directly on the standard multithreaded executable and its libraries, without the aid of a compiler-generated intermediate representation. Sthread leverages the open-source SimGrid libraries, and will eventually be integrated into SimGrid. Sthread employs a non-preemptive model in which thread context switches occur only at multithreaded system calls (e.g., mutex, semaphore) or before accesses to shared memory regions. The emphasis is on finding "algorithmic bugs" (bugs in an original algorithm, implemented as POSIX threads and shared memory regions. This work is in contrast to Context-Bounded Analysis (CBA), which assumes a preemptive model for threads, and emphasizes implementation bugs such as buffer overruns and write-after-free for memory allocation. In particular, the Sthread in-vivo approach has strong future potential for pedagogy, by providing immediate feedback to students who are first learning the correct use of Pthreads system calls in implementation of concurrent algorithms based on multithreading.
Semantic Relatedness and Taxonomic Word Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

This paper connects a series of papers dealing with taxonomic word embeddings. It begins by noting that there are different types of semantic relatedness and that different lexical representations encode different forms of relatedness. A particularly important distinction within semantic relatedness is that of thematic versus taxonomic relatedness. Next, we present a number of experiments that analyse taxonomic embeddings that have been trained on a synthetic corpus that has been generated via a random walk over a taxonomy. These experiments demonstrate how the properties of the synthetic corpus, such as the percentage of rare words, are affected by the shape of the knowledge graph the corpus is generated from. Finally, we explore the interactions between the relative sizes of natural and synthetic corpora on the performance of embeddings when taxonomic and thematic embeddings are combined.
Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network
Venue: arXiv cs.CV 2020
Year: 2020

Effective understanding of the environment and accurate trajectory prediction of surrounding dynamic obstacles are indispensable for intelligent mobile systems (like autonomous vehicles and social robots) to achieve safe and high-quality planning when they navigate in highly interactive and crowded scenarios. Due to the existence of frequent interactions and uncertainty in the scene evolution, it is desired for the prediction system to enable relational reasoning on different entities and provide a distribution of future trajectories for each agent. In this paper, we propose a generic generative neural system (called Social-WaGDAT) for multi-agent trajectory prediction, which makes a step forward to explicit interaction modeling by incorporating relational inductive biases with a dynamic graph representation and leverages both trajectory and scene context information. We also employ an efficient kinematic constraint layer applied to vehicle trajectory prediction which not only ensures physical feasibility but also enhances model performance. The proposed system is evaluated on three public benchmark datasets for trajectory prediction, where the agents cover pedestrians, cyclists and on-road vehicles. The experimental results demonstrate that our model achieves better performance than various baseline approaches in terms of prediction accuracy.
Why Do Line Drawings Work? A Realism Hypothesis
Venue: arXiv cs.CV 2020
Year: 2020

Why is it that we can recognize object identity and 3D shape from line drawings, even though they do not exist in the natural world? This paper hypothesizes that the human visual system perceives line drawings as if they were approximately realistic images. Moreover, the techniques of line drawing are chosen to accurately convey shape to a human observer. Several implications and variants of this hypothesis are explored.
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
Venue: arXiv cs.CL 2020
Year: 2020

Fine-tuning pretrained contextual word embedding models to supervised downstream tasks has become commonplace in natural language processing. This process, however, is often brittle: even with the same hyperparameter values, distinct random seeds can lead to substantially different results. To better understand this phenomenon, we experiment with four datasets from the GLUE benchmark, fine-tuning BERT hundreds of times on each while varying only the random seeds. We find substantial performance increases compared to previously reported results, and we quantify how the performance of the best-found model varies as a function of the number of fine-tuning trials. Further, we examine two factors influenced by the choice of random seed: weight initialization and training data order. We find that both contribute comparably to the variance of out-of-sample performance, and that some weight initializations perform well across all tasks explored. On small datasets, we observe that many fine-tuning trials diverge part of the way through training, and we offer best practices for practitioners to stop training less promising runs early. We publicly release all of our experimental data, including training and validation scores for 2,100 trials, to encourage further analysis of training dynamics during fine-tuning.