OpenRaiser/Intern-Atlas huggingface dataset:
Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications
Venue: arXiv cs.PF 2020
Year: 2020

Applying deep learning to object detection provides the capability to accurately detect and classify complex objects in the real world. However, currently, few mobile applications use deep learning because such technology is computation-intensive and energy-consuming. This paper, to the best of our knowledge, presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection, either locally on the smartphone or remotely on an edge server. In order to accurately measure the energy consumption on the smartphone and obtain the breakdown of energy consumed by each phase of the object detection processing pipeline, we propose a new measurement strategy. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection. Furthermore, several insights and research opportunities are proposed based on our experimental results. These findings from our experimental study will guide the design of energy-efficient processing pipeline of CNN-based object detection.
Fast and Accurate Anomaly Detection in Dynamic Graphs with a Two-Pronged Approach
Venue: arXiv cs.SI 2020
Year: 2020

Given a dynamic graph stream, how can we detect the sudden appearance of anomalous patterns, such as link spam, follower boosting, or denial of service attacks? Additionally, can we categorize the types of anomalies that occur in practice, and theoretically analyze the anomalous signs arising from each type? In this work, we propose AnomRank, an online algorithm for anomaly detection in dynamic graphs. AnomRank uses a two-pronged approach defining two novel metrics for anomalousness. Each metric tracks the derivatives of its own version of a 'node score' (or node importance) function. This allows us to detect sudden changes in the importance of any node. We show theoretically and experimentally that the two-pronged approach successfully detects two common types of anomalies: sudden weight changes along an edge, and sudden structural changes to the graph. AnomRank is (a) Fast and Accurate: up to 49.5x faster or 35% more accurate than state-of-the-art methods, (b) Scalable: linear in the number of edges in the input graph, processing millions of edges within 2 seconds on a stock laptop/desktop, and (c) Theoretically Sound: providing theoretical guarantees of the two-pronged approach.
An Autonomous Driving Framework for Long-term Decision-making and Short-term Trajectory Planning on Frenet Space
Venue: arXiv cs.RO 2020
Year: 2020

In this paper, we present a hierarchical framework for decision-making and planning on highway driving tasks. We utilized intelligent driving models (IDM and MOBIL) to generate long-term decisions based on the traffic situation flowing around the ego. The decisions both maximize ego performance while respecting other vehicles' objectives. Short-term trajectory optimization is performed on the Frenet space to make the calculations invariant to the road's three-dimensional curvatures. A novel obstacle avoidance approach is introduced on the Frenet frame for the moving obstacles. The optimization explores the driving corridors to generate spatiotemporal polynomial trajectories to navigate through the traffic safely and obey the BP commands. The framework also introduces a heuristic supervisor that identifies unexpected situations and recalculates each module in case of a potential emergency. Experiments in CARLA simulation have shown the potential and the scalability of the framework in implementing various driving styles that match human behavior.
Learning Causal Bayesian Networks from Text
Venue: arXiv cs.CL 2020
Year: 2020

Causal relationships form the basis for reasoning and decision-making in Artificial Intelligence systems. To exploit the large volume of textual data available today, the automatic discovery of causal relationships from text has emerged as a significant challenge in recent years. Existing approaches in this realm are limited to the extraction of low-level relations among individual events. To overcome the limitations of the existing approaches, in this paper, we propose a method for automatic inference of causal relationships from human written language at conceptual level. To this end, we leverage the characteristics of hierarchy of concepts and linguistic variables created from text, and represent the extracted causal relationships in the form of a Causal Bayesian Network. Our experiments demonstrate superiority of our approach over the existing approaches in inferring complex causal reasoning from the text.
Joint Channel Estimation and Signal Recovery in RIS-Assisted Multi-User MISO Communications
Venue: arXiv cs.IT 2020
Year: 2020

Reconfigurable Intelligent Surfaces (RISs) have been recently considered as an energy-efficient solution for future wireless networks. Their dynamic and low-power configuration enables coverage extension, massive connectivity, and low-latency communications. Channel estimation and signal recovery in RISbased systems are among the most critical technical challenges, due to the large number of unknown variables referring to the RIS unit elements and the transmitted signals. In this paper, we focus on the downlink of a RIS-assisted multi-user Multiple Input Single Output (MISO) communication system and present a joint channel estimation and signal recovery scheme based on the PARAllel FACtor (PARAFAC) decomposition. This decomposition unfolds the cascaded channel model and facilitates signal recovery using the Bilinear Generalized Approximate Message Passing (BiG-AMP) algorithm. The proposed method includes an alternating least squares algorithm to iteratively estimate the equivalent matrix, which consists of the transmitted signals and the channels between the base station and RIS, as well as the channels between the RIS and the multiple users. Our selective simulation results show that the proposed scheme outperforms a benchmark scheme that uses genie-aided information knowledge. We also provide insights on the impact of different RIS parameter settings on the proposed scheme.
Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
Venue: arXiv cs.AI 2020
Year: 2020

Remarkable success of modern image-based AI methods and the resulting interest in their applications in critical decision-making processes has led to a surge in efforts to make such intelligent systems transparent and explainable. The need for explainable AI does not stem only from ethical and moral grounds but also from stricter legislation around the world mandating clear and justifiable explanations of any decision taken or assisted by AI. Especially in the medical context where Computer-Aided Diagnosis can have a direct influence on the treatment and well-being of patients, transparency is of utmost importance for safe transition from lab research to real world clinical practice. This paper provides a comprehensive overview of current state-of-the-art in explaining and interpreting Deep Learning based algorithms in applications of medical research and diagnosis of diseases. We discuss early achievements in development of explainable AI for validation of known disease criteria, exploration of new potential biomarkers, as well as methods for the subsequent correction of AI models. Various explanation methods like visual, textual, post-hoc, ante-hoc, local and global have been thoroughly and critically analyzed. Subsequently, we also highlight some of the remaining challenges that stand in the way of practical applications of AI as a clinical decision support tool and provide recommendations for the direction of future research.
Anticipatory Ethics and the Role of Uncertainty
Venue: arXiv cs.CY 2020
Year: 2020

Making conjectures about future consequences of a technology is an exercise in trying to reduce various forms of uncertainty. Both to produce and reason about these conjectures requires understanding their potential limitations. In other words, we need systematic ways of considering uncertainty associated with given conjectures for downstream consequences. In this work, we frame the task of considering future consequences as an anticipatory ethics problem, where the goal is to develop scenarios that reflect plausible outcomes and their ethical implications following a technology's introduction into society. In order to shed light on how various forms of uncertainty might inform how we reason about a resulting scenario, we provide a characterization of the types of uncertainty that arise in a potential scenario-building process.
t-EVA: Time-Efficient t-SNE Video Annotation
Venue: arXiv cs.CV 2020
Year: 2020

Video understanding has received more attention in the past few years due to the availability of several large-scale video datasets. However, annotating large-scale video datasets are cost-intensive. In this work, we propose a time-efficient video annotation method using spatio-temporal feature similarity and t-SNE dimensionality reduction to speed up the annotation process massively. Placing the same actions from different videos near each other in the two-dimensional space based on feature similarity helps the annotator to group-label video clips. We evaluate our method on two subsets of the ActivityNet (v1.3) and a subset of the Sports-1M dataset. We show that t-EVA can outperform other video annotation tools while maintaining test accuracy on video classification.
Molecular representation learning with language models and domain-relevant auxiliary tasks
Venue: arXiv cs.LG 2020
Year: 2020

We apply a Transformer architecture, specifically BERT, to learn flexible and high quality molecular representations for drug discovery problems. We study the impact of using different combinations of self-supervised tasks for pre-training, and present our results for the established Virtual Screening and QSAR benchmarks. We show that: i) The selection of appropriate self-supervised task(s) for pre-training has a significant impact on performance in subsequent downstream tasks such as Virtual Screening. ii) Using auxiliary tasks with more domain relevance for Chemistry, such as learning to predict calculated molecular properties, increases the fidelity of our learnt representations. iii) Finally, we show that molecular representations learnt by our model `MolBert' improve upon the current state of the art on the benchmark datasets.
Two Stage Transformer Model for COVID-19 Fake News Detection and Fact Checking
Venue: arXiv cs.CL 2020
Year: 2020

The rapid advancement of technology in online communication via social media platforms has led to a prolific rise in the spread of misinformation and fake news. Fake news is especially rampant in the current COVID-19 pandemic, leading to people believing in false and potentially harmful claims and stories. Detecting fake news quickly can alleviate the spread of panic, chaos and potential health hazards. We developed a two stage automated pipeline for COVID-19 fake news detection using state of the art machine learning models for natural language processing. The first model leverages a novel fact checking algorithm that retrieves the most relevant facts concerning user claims about particular COVID-19 claims. The second model verifies the level of truth in the claim by computing the textual entailment between the claim and the true facts retrieved from a manually curated COVID-19 dataset. The dataset is based on a publicly available knowledge source consisting of more than 5000 COVID-19 false claims and verified explanations, a subset of which was internally annotated and cross-validated to train and evaluate our models. We evaluate a series of models based on classical text-based features to more contextual Transformer based models and observe that a model pipeline based on BERT and ALBERT for the two stages respectively yields the best results.
CYPUR-NN: Crop Yield Prediction Using Regression and Neural Networks
Venue: arXiv cs.CV 2020
Year: 2020

Our recent study using historic data of paddy yield and associated conditions include humidity, luminescence, and temperature. By incorporating regression models and neural networks (NN), one can produce highly satisfactory forecasting of paddy yield. Simulations indicate that our model can predict paddy yield with high accuracy while concurrently detecting diseases that may exist and are oblivious to the human eye. Crop Yield Prediction Using Regression and Neural Networks (CYPUR-NN) is developed here as a system that will facilitate agriculturists and farmers to predict yield from a picture or by entering values via a web interface. CYPUR-NN has been tested on stock images and the experimental results are promising.
Depth-Aware Action Recognition: Pose-Motion Encoding through Temporal Heatmaps
Venue: arXiv cs.CV 2020
Year: 2020

Most state-of-the-art methods for action recognition rely only on 2D spatial features encoding appearance, motion or pose. However, 2D data lacks the depth information, which is crucial for recognizing fine-grained actions. In this paper, we propose a depth-aware volumetric descriptor that encodes pose and motion information in a unified representation for action classification in-the-wild. Our framework is robust to many challenges inherent to action recognition, e.g. variation in viewpoint, scene, clothing and body shape. The key component of our method is the Depth-Aware Pose Motion representation (DA-PoTion), a new video descriptor that encodes the 3D movement of semantic keypoints of the human body. Given a video, we produce human joint heatmaps for each frame using a state-of-the-art 3D human pose regressor and we give each of them a unique color code according to the relative time in the clip. Then, we aggregate such 3D time-encoded heatmaps for all human joints to obtain a fixed-size descriptor (DA-PoTion), which is suitable for classifying actions using a shallow 3D convolutional neural network (CNN). The DA-PoTion alone defines a new state-of-the-art on the Penn Action Dataset. Moreover, we leverage the intrinsic complementarity of our pose motion descriptor with appearance based approaches by combining it with Inflated 3D ConvNet (I3D) to define a new state-of-the-art on the JHMDB Dataset.
Generative Layout Modeling using Constraint Graphs
Venue: arXiv cs.CV 2020
Year: 2020

We propose a new generative model for layout generation. We generate layouts in three steps. First, we generate the layout elements as nodes in a layout graph. Second, we compute constraints between layout elements as edges in the layout graph. Third, we solve for the final layout using constrained optimization. For the first two steps, we build on recent transformer architectures. The layout optimization implements the constraints efficiently. We show three practical contributions compared to the state of the art: our work requires no user input, produces higher quality layouts, and enables many novel capabilities for conditional layout generation.
Creating and Maintaining Filipino and Japanese Students Social Capital with Facebook
Venue: arXiv cs.CY 2020
Year: 2020

This study investigated perceptions and patterns of Facebook use among Filipino and Japanese undergraduate students and the relationship of these factors to creating and maintaining students social capital, international posture, and willingness to communicate. The survey of undergraduate students was conducted online and 483 valid responses were obtained. Data revealed the characteristic uses of FB by Filipino and Japanese undergraduate students. An interrelation model among six factors, International Posture, WTC, Perception, Bridging, Bonding, and Utilization for Filipino students showed the importance of utilization or FB usage for bridging social capital and bonding social capital. For Japanese students, bonding social capital mediated between utilization or FB usage and bridging. Bridging social capital was established only through bonding social capital. Thus, unless Japanese students are close enough to their FB friends, they do not construct new relationships on FB that will influence Filipino students in the process of virtual internationalization in the future.
Episodic Self-Imitation Learning with Hindsight
Venue: arXiv cs.AI 2020
Year: 2020

Episodic self-imitation learning, a novel self-imitation algorithm with a trajectory selection module and an adaptive loss function, is proposed to speed up reinforcement learning. Compared to the original self-imitation learning algorithm, which samples good state-action pairs from the experience replay buffer, our agent leverages entire episodes with hindsight to aid self-imitation learning. A selection module is introduced to filter uninformative samples from each episode of the update. The proposed method overcomes the limitations of the standard self-imitation learning algorithm, a transitions-based method which performs poorly in handling continuous control environments with sparse rewards. From the experiments, episodic self-imitation learning is shown to perform better than baseline on-policy algorithms, achieving comparable performance to state-of-the-art off-policy algorithms in several simulated robot control tasks. The trajectory selection module is shown to prevent the agent learning undesirable hindsight experiences. With the capability of solving sparse reward problems in continuous control settings, episodic self-imitation learning has the potential to be applied to real-world problems that have continuous action spaces, such as robot guidance and manipulation.
Towards real-time population estimates: introducing Twitter daily estimates of residents and non-residents at the county level
Venue: arXiv cs.SI 2020
Year: 2020

The study of migrations and mobility has historically been severely limited by the absence of reliable data or the temporal sparsity of the available data. Using geospatial digital trace data, the study of population movements can be much more precisely and dynamically measured. Our research seeks to develop a near real-time (one-day lag) Twitter census that gives a more temporally granular picture of local and non-local population at the county level. Leveraging geotagged tweets to determine the home location of all active Twitter users, we contribute to the field of digital and computational demography by obtaining accurate daily Twitter population stocks (residents and non-residents). Internal validation reveals over 80% of accuracy when compared with users self-reported home location. External validation results suggest these stocks correlate with available statistics of residents/non-residents at the county level and can accurately reflect regular (seasonal tourism) and non-regular events such as the Great American Solar Eclipse of 2017. The findings demonstrate that Twitter holds potential to introduce the dynamic component often lacking in population estimates.
Interactive Machine Learning of Musical Gesture
Venue: arXiv cs.LG 2020
Year: 2020

This chapter presents an overview of Interactive Machine Learning (IML) techniques applied to the analysis and design of musical gestures. We go through the main challenges and needs related to capturing, analysing, and applying IML techniques to human bodily gestures with the purpose of performing with sound synthesis systems. We discuss how different algorithms may be used to accomplish different tasks, including interacting with complex synthesis techniques and exploring interaction possibilities by means of Reinforcement Learning (RL) in an interaction paradigm we developed called Assisted Interactive Machine Learning (AIML). We conclude the chapter with a description of how some of these techniques were employed by the authors for the development of four musical pieces, thus outlining the implications that IML have for musical practice.
PowerNet: Transferable Dynamic IR Drop Estimation via Maximum Convolutional Neural Network
Venue: arXiv cs.LG 2020
Year: 2020

IR drop is a fundamental constraint required by almost all chip designs. However, its evaluation usually takes a long time that hinders mitigation techniques for fixing its violations. In this work, we develop a fast dynamic IR drop estimation technique, named PowerNet, based on a convolutional neural network (CNN). It can handle both vector-based and vectorless IR analyses. Moreover, the proposed CNN model is general and transferable to different designs. This is in contrast to most existing machine learning (ML) approaches, where a model is applicable only to a specific design. Experimental results show that PowerNet outperforms the latest ML method by 9% in accuracy for the challenging case of vectorless IR drop and achieves a 30 times speedup compared to an accurate IR drop commercial tool. Further, a mitigation tool guided by PowerNet reduces IR drop hotspots by 26% and 31% on two industrial designs, respectively, with very limited modification on their power grids.
Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
Venue: arXiv cs.LG 2020
Year: 2020

Deep neural networks (DNNs) are known to be prone to adversarial attacks, for which many remedies are proposed. While adversarial training (AT) is regarded as the most robust defense, it suffers from poor performance both on clean examples and under other types of attacks, e.g. attacks with larger perturbations. Meanwhile, regularizers that encourage uncertain outputs, such as entropy maximization (EntM) and label smoothing (LS) can maintain accuracy on clean examples and improve performance under weak attacks, yet their ability to defend against strong attacks is still in doubt. In this paper, we revisit uncertainty promotion regularizers, including EntM and LS, in the field of adversarial learning. We show that EntM and LS alone provide robustness only under small perturbations. Contrarily, we show that uncertainty promotion regularizers complement AT in a principled manner, consistently improving performance on both clean examples and under various attacks, especially attacks with large perturbations. We further analyze how uncertainty promotion regularizers enhance the performance of AT from the perspective of Jacobian matrices $\nabla_X f(X;θ)$, and find out that EntM effectively shrinks the norm of Jacobian matrices and hence promotes robustness.