OpenRaiser/Intern-Atlas huggingface dataset:
AuctionWhisk: Using an Auction-Inspired Approach for Function Placement in Serverless Fog Platforms
Venue: arXiv cs.DC 2021
Year: 2021

The Function-as-a-Service (FaaS) paradigm has a lot of potential as a computing model for fog environments comprising both cloud and edge nodes, as compute requests can be scheduled across the entire fog continuum in a fine-grained manner. When the request rate exceeds capacity limits at the resource-constrained edge, some functions need to be offloaded towards the cloud. In this paper, we present an auction-inspired approach in which application developers bid on resources while fog nodes decide locally which functions to execute and which to offload in order to maximize revenue. Unlike many current approaches to function placement in the fog, our approach can work in an online and decentralized manner. We also present our proof-of-concept prototype AuctionWhisk that illustrates how such an approach can be implemented in a real FaaS platform. Through a number of simulation runs and system experiments, we show that revenue for overloaded nodes can be maximized without dropping function requests.
StackGAN: Facial Image Generation Optimizations
Venue: arXiv cs.CV 2021
Year: 2021

Current state-of-the-art photorealistic generators are computationally expensive, involve unstable training processes, and have real and synthetic distributions that are dissimilar in higher-dimensional spaces. To solve these issues, we propose a variant of the StackGAN architecture. The new architecture incorporates conditional generators to construct an image in many stages. In our model, we generate grayscale facial images in two different stages: noise to edges (stage one) and edges to grayscale (stage two). Our model is trained with the CelebA facial image dataset and achieved a Fréchet Inception Distance (FID) score of 73 for edge images and a score of 59 for grayscale images generated using the synthetic edge images. Although our model achieved subpar results in relation to state-of-the-art models, dropout layers could reduce the overfitting in our conditional mapping. Additionally, since most images can be broken down into important features, improvements to our model can generalize to other datasets. Therefore, our model can potentially serve as a superior alternative to traditional means of generating photorealistic images.
Public sentiment analysis and topic modeling regarding COVID-19 vaccines on the Reddit social media platform: A call to action for strengthening vaccine confidence
Venue: arXiv cs.IR 2021
Year: 2021

The COVID-19 pandemic fueled one of the most rapid vaccine developments in history. However, misinformation spread through online social media often leads to negative vaccine sentiment and hesitancy. To investigate COVID-19 vaccine-related discussion in social media, we conducted a sentiment analysis and Latent Dirichlet Allocation topic modeling on textual data collected from 13 Reddit communities focusing on the COVID-19 vaccine from Dec 1, 2020, to May 15, 2021. Data were aggregated and analyzed by month to detect changes in any sentiment and latent topics. ty analysis suggested these communities expressed more positive sentiment than negative regarding the vaccine-related discussions and has remained static over time. Topic modeling revealed community members mainly focused on side effects rather than outlandish conspiracy theories. Covid-19 vaccine-related content from 13 subreddits show that the sentiments expressed in these communities are overall more positive than negative and have not meaningfully changed since December 2020. Keywords indicating vaccine hesitancy were detected throughout the LDA topic modeling. Public sentiment and topic modeling analysis regarding vaccines could facilitate the implementation of appropriate messaging, digital interventions, and new policies to promote vaccine confidence.
E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling
Venue: arXiv cs.IR 2021
Year: 2021

Promotions and discounts are essential components of modern e-commerce platforms, where they are often used to incentivize customers towards purchase completion. Promotions also affect revenue and may incur a monetary loss that is often limited by a dedicated promotional budget. We study the Online Constrained Multiple-Choice Promotions Personalization Problem, where the optimization goal is to select for each customer which promotion to present in order to maximize purchase completions, while also complying with global budget limitations. Our work formalizes the problem as an Online Multiple Choice Knapsack Problem and extends the existent literature by addressing cases with negative weights and values. We provide a real-time adaptive method that guarantees budget constraints compliance and achieves above 99.7% of the optimal promotional impact on various datasets. Our method is evaluated on a large-scale experimental study at one of the leading online travel platforms in the world.
Learning Highly Recursive Input Grammars
Venue: arXiv cs.SE 2021
Year: 2021

This paper presents Arvada, an algorithm for learning context-free grammars from a set of positive examples and a Boolean-valued oracle. Arvada learns a context-free grammar by building parse trees from the positive examples. Starting from initially flat trees, Arvada builds structure to these trees with a key operation: it bubbles sequences of sibling nodes in the trees into a new node, adding a layer of indirection to the tree. Bubbling operations enable recursive generalization in the learned grammar. We evaluate Arvada against GLADE and find it achieves on average increases of 4.98x in recall and 3.13x in F1 score, while incurring only a 1.27x slowdown and requiring only 0.87x as many calls to the oracle. Arvada has a particularly marked improvement over GLADE on grammars with highly recursive structure, like those of programming languages.
RoboRun: A Robot Runtime to Exploit Spatial Heterogeneity
Venue: arXiv cs.RO 2021
Year: 2021

The limited onboard energy of autonomous mobile robots poses a tremendous challenge for practical deployment. Hence, efficient computing solutions are imperative. A crucial shortcoming of state-of-the-art computing solutions is that they ignore the robot's operating environment heterogeneity and make static, worst-case assumptions. As this heterogeneity impacts the system's computing payload, an optimal system must dynamically capture these changes in the environment and adjust its computational resources accordingly. This paper introduces RoboRun, a mobile-robot runtime that dynamically exploits the compute-environment synergy to improve performance and energy. We implement RoboRun in the Robot Operating System (ROS) and evaluate it on autonomous drones. We compare RoboRun against a state-of-the-art static design and show 4.5X and 4X improvements in mission time and energy, respectively, as well as a 36% reduction in CPU utilization.
Noisy Labels for Weakly Supervised Gamma Hadron Classification
Venue: arXiv cs.LG 2021
Year: 2021

Gamma hadron classification, a central machine learning task in gamma ray astronomy, is conventionally tackled with supervised learning. However, the supervised approach requires annotated training data to be produced in sophisticated and costly simulations. We propose to instead solve gamma hadron classification with a noisy label approach that only uses unlabeled data recorded by the real telescope. To this end, we employ the significance of detection as a learning criterion which addresses this form of weak supervision. We show that models which are based on the significance of detection deliver state-of-the-art results, despite being exclusively trained with noisy labels; put differently, our models do not require the costly simulated ground-truth labels that astronomers otherwise employ for classifier training. Our weakly supervised models exhibit competitive performances also on imbalanced data sets that stem from a variety of other application domains. In contrast to existing work on class-conditional label noise, we assume that only one of the class-wise noise rates is known.
Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithms
Venue: arXiv cs.LG 2021
Year: 2021

Backpropagation is the default algorithm for training deep neural networks due to its simplicity, efficiency and high convergence rate. However, its requirements make it impossible to be implemented in a human brain. In recent years, more biologically plausible learning methods have been proposed. Some of these methods can match backpropagation accuracy, and simultaneously provide other extra benefits such as faster training on specialized hardware (e.g., ASICs) or higher robustness against adversarial attacks. While the interest in the field is growing, there is a necessity for open-source libraries and toolkits to foster research and benchmark algorithms. In this paper, we present BioTorch, a software framework to create, train, and benchmark biologically motivated neural networks. In addition, we investigate the performance of several feedback alignment methods proposed in the literature, thereby unveiling the importance of the forward and backward weight initialization and optimizer choice. Finally, we provide a novel robustness study of these methods against state-of-the-art white and black-box adversarial attacks.
Modularity and Heavy-Tailed Degree Distributions
Venue: arXiv cs.SI 2021
Year: 2021

Identifying clusters of vertices in graphs continues to be an important problem, and modularity continues to be used as a tool for solving the problem. Modularity, which measures the quality of a division of the vertices into clusters, explicitly treats vertices of different degrees differently, imposing a larger penalty when high-degree vertices are put in the same cluster. We claim that this unequal treatment negatively impacts the performance of clustering algorithms based on modularity for graphs with heavy-tailed degree distributions. We used the Greedy Modularity hill-climb to find clusters in graphs with power-law degree distributions and observed that it performed poorly clustering low-degree vertices. We propose a simple variant of modularity that we call flat modularity. We found that, using the same algorithm with the modified score instead, we improved the performance of the clustering algorithm on low-degree vertices and the overall performance as well. We believe that the small change -- from modularity to flat modularity -- could improve the output in the many real-world processes that rely on modularity to measure the quality of a division into clusters with only a small change, which is really a simplification, to the implementation.
Want To Reduce Labeling Cost? GPT-3 Can Help
Venue: arXiv cs.CL 2021
Year: 2021

Data annotation is a time-consuming and labor-intensive process for many NLP tasks. Although there exist various methods to produce pseudo data labels, they are often task-specific and require a decent amount of labeled data to start with. Recently, the immense language model GPT-3 with 175 billion parameters has achieved tremendous improvement across many few-shot learning tasks. In this paper, we explore ways to leverage GPT-3 as a low-cost data labeler to train other models. We find that, to make the downstream model achieve the same performance on a variety of NLU and NLG tasks, it costs 50% to 96% less to use labels from GPT-3 than using labels from humans. Furthermore, we propose a novel framework of combining pseudo labels from GPT-3 with human labels, which leads to even better performance with limited labeling budget. These results present a cost-effective data labeling methodology that is generalizable to many practical applications.
DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptions
Venue: arXiv cs.LG 2021
Year: 2021

Estimation of causal effects involves crucial assumptions about the data-generating process, such as directionality of effect, presence of instrumental variables or mediators, and whether all relevant confounders are observed. Violation of any of these assumptions leads to significant error in the effect estimate. However, unlike cross-validation for predictive models, there is no global validator method for a causal estimate. As a result, expressing different causal assumptions formally and validating them (to the extent possible) becomes critical for any analysis. We present DoWhy, a framework that allows explicit declaration of assumptions through a causal graph and provides multiple validation tests to check a subset of these assumptions. Our experience with DoWhy highlights a number of open questions for future research: developing new ways beyond causal graphs to express assumptions, the role of causal discovery in learning relevant parts of the graph, and developing validation tests that can better detect errors, both for average and conditional treatment effects. DoWhy is available at https://github.com/microsoft/dowhy.
The UW Virtual Brain Project: An immersive approach to teaching functional neuroanatomy
Venue: arXiv cs.HC 2021
Year: 2021

Learning functional neuroanatomy requires forming mental representations of 3D structure, but forming such representations from 2D textbook diagrams can be challenging. We address this challenge in the UW Virtual Brain Project by developing 3D narrated diagrams, which are interactive, guided tours through 3D models of perceptual systems. Lessons can be experienced in virtual realty (VR) or on a personal computer monitor (PC). We predicted participants would learn from lessons presented on both VR and PC devices (comparing pre-test/post-test scores), but that VR would be more effective for achieving both content-based learning outcomes (i.e test performance) and experience-based learning outcomes (i.e., reported enjoyment and ease of use). All participants received lessons about the visual system and auditory system, one in VR and one on a PC(order counterbalanced). We assessed content learning using a drawing/labeling task on paper (2D drawing) in Experiment 1 and a Looking Glass autostereoscopic display (3D drawing) in Experiment 2. In both experiments, we found that the UW Virtual Brain Project lessons were effective for teaching functional neuroanatomy, with no difference between devices. However, participants reported VR was more enjoyable and easier to use. We also evaluated the VR lessons in our Classroom Implementation during an undergraduate course on perception. Students reported that the VR lessons helped them make progress on course learning outcomes, especially for learning system pathways. They suggested lessons could be improved byadding more examples and providing more time to explore in VR.
CWmin Estimation and Collision Identification in Wi-Fi Systems
Venue: arXiv cs.NI 2021
Year: 2021

Wi-Fi networks are susceptible to aggressive behavior caused by selfish or malicious devices that reduce their minimum contention window size (CWmin) to below the standard CWmin. In this paper, we propose a scheme called Minimum Contention Window Estimation (CWE) to detect aggressive stations with low CWmin's, where the AP estimates the CWmin value of all stations transmitting uplink by monitoring their backoff values over a period of time and keeping track of the idle time each station spends during backoff. To correctly estimate each backoff value, we present a cross-correlation-based technique that uses the frequency offset between the AP and each station to identify stations involved in uplink collisions. The AP constructs empirical distributions for the monitored backoff values and compares them with a set of nominal PMF's, created via Markov analysis of the DCF protocol to estimate CWmin of various stations. After detecting the aggressive stations, the AP can choose to stop serving those stations. Simulation results show that the accuracy of our collision detection technique is 96%, 94%, and 88% when there are 3, 6, and 9 stations in the WLAN, respectively. For the former WLAN settings, the estimation accuracy of CWE scheme is 100%, 98.81%, and 96.3%, respectively.
Rapidly and accurately estimating brain strain and strain rate across head impact types with transfer learning and data fusion
Venue: arXiv cs.LG 2021
Year: 2021

Brain strain and strain rate are effective in predicting traumatic brain injury (TBI) caused by head impacts. However, state-of-the-art finite element modeling (FEM) demands considerable computational time in the computation, limiting its application in real-time TBI risk monitoring. To accelerate, machine learning head models (MLHMs) were developed, and the model accuracy was found to decrease when the training/test datasets were from different head impacts types. However, the size of dataset for specific impact types may not be enough for model training. To address the computational cost of FEM, the limited strain rate prediction, and the generalizability of MLHMs to on-field datasets, we propose data fusion and transfer learning to develop a series of MLHMs to predict the maximum principal strain (MPS) and maximum principal strain rate (MPSR). We trained and tested the MLHMs on 13,623 head impacts from simulations, American football, mixed martial arts, car crash, and compared against the models trained on only simulations or only on-field impacts. The MLHMs developed with transfer learning are significantly more accurate in estimating MPS and MPSR than other models, with a mean absolute error (MAE) smaller than 0.03 in predicting MPS and smaller than 7 (1/s) in predicting MPSR on all impact datasets. The MLHMs can be applied to various head impact types for rapidly and accurately calculating brain strain and strain rate. Besides the clinical applications in real-time brain strain and strain rate monitoring, this model helps researchers estimate the brain strain and strain rate caused by head impacts more efficiently than FEM.
Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Channel pruning is a promising technique to compress the parameters of deep convolutional neural networks(DCNN) and to speed up the inference. This paper aims to address the long-standing inefficiency of channel pruning. Most channel pruning methods recover the prediction accuracy by re-training the pruned model from the remaining parameters or random initialization. This re-training process is heavily dependent on the sufficiency of computational resources, training data, and human interference(tuning the training strategy). In this paper, a highly efficient pruning method is proposed to significantly reduce the cost of pruning DCNN. The main contributions of our method include: 1) pruning compensation, a fast and data-efficient substitute of re-training to minimize the post-pruning reconstruction loss of features, 2) compensation-aware pruning(CaP), a novel pruning algorithm to remove redundant or less-weighted channels by minimizing the loss of information, and 3) binary structural search with step constraint to minimize human interference. On benchmarks including CIFAR-10/100 and ImageNet, our method shows competitive pruning performance among the state-of-the-art retraining-based pruning methods and, more importantly, reduces the processing time by 95% and data usage by 90%.
The five Is: Key principles for interpretable and safe conversational AI
Venue: arXiv cs.CL 2021
Year: 2021

In this position paper, we present five key principles, namely interpretability, inherent capability to explain, independent data, interactive learning, and inquisitiveness, for the development of conversational AI that, unlike the currently popular black box approaches, is transparent and accountable. At present, there is a growing concern with the use of black box statistical language models: While displaying impressive average performance, such systems are also prone to occasional spectacular failures, for which there is no clear remedy. In an effort to initiate a discussion on possible alternatives, we outline and exemplify how our five principles enable the development of conversational AI systems that are transparent and thus safer for use. We also present some of the challenges inherent in the implementation of those principles.
Robust Symbol-Level Precoding and Passive Beamforming for IRS-Aided Communications
Venue: arXiv cs.IT 2021
Year: 2021

This paper investigates a joint beamforming design in a multiuser multiple-input single-output (MISO) communication network aided with an intelligent reflecting surface (IRS) panel. The symbol-level precoding (SLP) is adopted to enhance the system performance by exploiting the multiuser interference (MUI) with consideration of bounded channel uncertainty. The joint beamforming design is formulated into a nonconvex worst-case robust programming to minimize the transmit power subject to single-to-noise ratio (SNR) requirements. To address the challenges due to the constant modulus and the coupling of the beamformers, we first study the single-user case. Specifically, we propose and compare two algorithms based on the semidefinite relaxation (SDR) and alternating optimization (AO) methods, respectively. It turns out that the AO-based algorithm has much lower computational complexity but with almost the same power to the SDR-based algorithm. Then, we apply the AO technique to the multiuser case and thereby develop an algorithm based on the proximal gradient descent (PGD) method. The algorithm can be generalized to the case of finite-resolution IRS and the scenario with direct links from the transmitter to the users. Numerical results show that the SLP can significantly improve the system performance. Meanwhile, 3-bit phase shifters can achieve near-optimal power performance.
Synbit: Synthesizing Bidirectional Programs using Unidirectional Sketches
Venue: arXiv cs.PL 2021
Year: 2021

We propose a technique for synthesizing bidirectional programs from the corresponding unidirectional code plus a few input/output examples. The core ideas are: (1) constructing a sketch using the given unidirectional program as a specification, and (2) filling the sketch in a modular fashion by exploiting the properties of bidirectional programs. These ideas are enabled by our choice of programming language, HOBiT, which is specifically designed to maintain the unidirectional program structure in bidirectional programming, and keep the parts that control bidirectional behavior modular. To evaluate our approach, we implemented it in a tool called Synbit and used it to generate bidirectional programs for intricate microbenchmarks, as well as for a few larger, more realistic problems. We also compared Synbit to a state-of-the-art unidirectional synthesis tool on the task of synthesizing backward computations.
Identifying Ransomware Actors in the Bitcoin Network
Venue: arXiv cs.CR 2021
Year: 2021

Due to the pseudo-anonymity of the Bitcoin network, users can hide behind their bitcoin addresses that can be generated in unlimited quantity, on the fly, without any formal links between them. Thus, it is being used for payment transfer by the actors involved in ransomware and other illegal activities. The other activity we consider is related to gambling since gambling is often used for transferring illegal funds. The question addressed here is that given temporally limited graphs of Bitcoin transactions, to what extent can one identify common patterns associated with these fraudulent activities and apply them to find other ransomware actors. The problem is rather complex, given that thousands of addresses can belong to the same actor without any obvious links between them and any common pattern of behavior. The main contribution of this paper is to introduce and apply new algorithms for local clustering and supervised graph machine learning for identifying malicious actors. We show that very local subgraphs of the known such actors are sufficient to differentiate between ransomware, random and gambling actors with 85% prediction accuracy on the test data set.