OpenRaiser/Intern-Atlas huggingface dataset:
Transfer Learning with Graph Neural Networks for Short-Term Highway Traffic Forecasting
Venue: arXiv cs.LG 2020
Year: 2020

Highway traffic modeling and forecasting approaches are critical for intelligent transportation systems. Recently, deep-learning-based traffic forecasting methods have emerged as state of the art for a wide range of traffic forecasting tasks. However, these methods require a large amount of training data, which needs to be collected over a significant period of time. This can present a number of challenges for the development and deployment of data-driven learning methods for highway networks that suffer from lack of historical data. A promising approach to address this issue is transfer learning, where a model trained on one part of the highway network can be adapted for a different part of the highway network. We focus on diffusion convolutional recurrent neural network (DCRNN), a state-of-the-art graph neural network for highway network forecasting. It models the complex spatial and temporal dynamics of the highway network using a graph-based diffusion convolution operation within a recurrent neural network. DCRNN cannot perform transfer learning, however, because it learns location-specific traffic patterns, which cannot be used for unseen regions of the network. To that end, we develop a new transfer learning approach for DCRNN, where a single model trained on data-rich regions of the highway network can be used to forecast traffic on unseen regions of the highway network. We evaluate the ability of our approach to forecast the traffic on the entire California highway network with one year of time series data. We show that TL-DCRNN can learn from several regions of the California highway network and forecast the traffic on the unseen regions of the network with high accuracy. Moreover, we demonstrate that TL-DCRNN can learn from San Francisco region traffic data and can forecast traffic on the Los Angeles region and vice versa.
One-vs-Rest Network-based Deep Probability Model for Open Set Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Unknown examples that are unseen during training often appear in real-world computer vision tasks, and an intelligent self-learning system should be able to differentiate between known and unknown examples. Open set recognition, which addresses this problem, has been studied for approximately a decade. However, conventional open set recognition methods based on deep neural networks (DNNs) lack a foundation for post recognition score analysis. In this paper, we propose a DNN structure in which multiple one-vs-rest sigmoid networks follow a convolutional neural network feature extractor. A one-vs-rest network, which is composed of rectified linear unit activation functions for the hidden layers and a single sigmoid target class output node, can maximize the ability to learn information from nonmatch examples. Furthermore, the network yields a sophisticated nonlinear features-to-output mapping that is explainable in the feature space. By introducing extreme value theory-based calibration techniques, the nonlinear and explainable mapping provides a well-grounded class membership probability models. Our experiments show that one-vs-rest networks can provide more informative hidden representations for unknown examples than the commonly used SoftMax layer. In addition, the proposed probability model outperformed the state-of-the art methods in open set classification scenarios.
Knowledge-guided Deep Reinforcement Learning for Interactive Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

Interactive recommendation aims to learn from dynamic interactions between items and users to achieve responsiveness and accuracy. Reinforcement learning is inherently advantageous for coping with dynamic environments and thus has attracted increasing attention in interactive recommendation research. Inspired by knowledge-aware recommendation, we proposed Knowledge-Guided deep Reinforcement learning (KGRL) to harness the advantages of both reinforcement learning and knowledge graphs for interactive recommendation. This model is implemented upon the actor-critic network framework. It maintains a local knowledge network to guide decision-making and employs the attention mechanism to capture long-term semantics between items. We have conducted comprehensive experiments in a simulated online environment with six public real-world datasets and demonstrated the superiority of our model over several state-of-the-art methods.
Beamspace Precoding and Beam Selection for Wideband Millimeter-Wave MIMO Relying on Lens Antenna Arrays
Venue: arXiv cs.IT 2020
Year: 2020

Millimeter-wave (mmWave) multiple-input multiple-out (MIMO) systems relying on lens antenna arrays are capable of achieving a high antenna-gain at a considerably reduced number of radio frequency (RF) chains via beam selection. However, the traditional beam selection network suffers from significant performance loss in wideband systems due to the effect of beam squint. In this paper, we propose a phase shifter-aided beam selection network, which enables a single RF chain to support multiple focused-energy beams, for mitigating the beam squint in wideband mmWave MIMO systems. Based on this architecture, we additionally design an efficient transmit precoder (TPC) for maximizing the achievable sum-rate, which is composed of beam selection and beamspace precoding. Specifically, we decouple the design problems of beamspace precoding and beam selection by exploiting the fact that the beam selection matrix has a limited number of candidates. For the beamspace precoding design, we propose a successive interference cancellation (SIC)-based method, which decomposes the associated optimization problem into a series of subproblems and solves them successively. For the beam selection design, we propose an energy-max beam selection method for avoiding the high complexity of exhaustive search, and derive the number of required beams for striking an attractive trade-off between the hardware cost and system performance. Our simulation results show that the proposed beamspace precoding and beam selection methods achieve both a higher sum-rate and a higher energy efficiency than its conventional counterparts.
Adaptive Neuron-wise Discriminant Criterion and Adaptive Center Loss at Hidden Layer for Deep Convolutional Neural Network
Venue: arXiv cs.CV 2020
Year: 2020

A deep convolutional neural network (CNN) has been widely used in image classification and gives better classification accuracy than the other techniques. The softmax cross-entropy loss function is often used for classification tasks. There are some works to introduce the additional terms in the objective function for training to make the features of the output layer more discriminative. The neuron-wise discriminant criterion makes the input feature of each neuron in the output layer discriminative by introducing the discriminant criterion to each of the features. Similarly, the center loss was introduced to the features before the softmax activation function for face recognition to make the deep features discriminative. The ReLU function is often used for the network as an active function in the hidden layers of the CNN. However, it is observed that the deep features trained by using the ReLU function are not discriminative enough and show elongated shapes. In this paper, we propose to use the neuron-wise discriminant criterion at the output layer and the center-loss at the hidden layer. Also, we introduce the online computation of the means of each class with the exponential forgetting. We named them adaptive neuron-wise discriminant criterion and adaptive center loss, respectively. The effectiveness of the integration of the adaptive neuron-wise discriminant criterion and the adaptive center loss is shown by the experiments with MNSIT, FashionMNIST, CIFAR10, CIFAR100, and STL10. Source code is at https://github.com/i13abe/Adaptive-discriminant-and-center
Show Us the Way: Learning to Manage Dialog from Demonstrations
Venue: arXiv cs.CL 2020
Year: 2020

We present our submission to the End-to-End Multi-Domain Dialog Challenge Track of the Eighth Dialog System Technology Challenge. Our proposed dialog system adopts a pipeline architecture, with distinct components for Natural Language Understanding, Dialog State Tracking, Dialog Management and Natural Language Generation. At the core of our system is a reinforcement learning algorithm which uses Deep Q-learning from Demonstrations to learn a dialog policy with the help of expert examples. We find that demonstrations are essential to training an accurate dialog policy where both state and action spaces are large. Evaluation of our Dialog Management component shows that our approach is effective - beating supervised and reinforcement learning baselines.
Whence the Expected Free Energy?
Venue: arXiv cs.AI 2020
Year: 2020

The Expected Free Energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the Variational Free Energy (VFE) remain unclear. In this paper, we investigate the origins of the EFE in detail and show that it is not simply "the free energy in the future". We present a functional that we argue is the natural extension of the VFE, but which actively discourages exploratory behaviour, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the Free-Energy of the Expected Future (FEEF), which possesses both the epistemic component of the EFE as well as an intuitive mathematical grounding as the divergence between predicted and desired futures.
GEVO: GPU Code Optimization using Evolutionary Computation
Venue: arXiv cs.NE 2020
Year: 2020

GPUs are a key enabler of the revolution in machine learning and high performance computing, functioning as de facto co-processors to accelerate large-scale computation. As the programming stack and tool support have matured, GPUs have also become accessible to programmers, who may lack detailed knowledge of the underlying architecture and fail to fully leverage the GPU's computation power. GEVO (Gpu optimization using EVOlutionary computation) is a tool for automatically discovering optimization opportunities and tuning the performance of GPU kernels in the LLVM representation. GEVO uses population-based search to find edits to GPU code compiled to LLVM-IR and improves performance on desired criteria while retaining required functionality. We demonstrate that GEVO improves the execution time of the GPU programs in the Rodinia benchmark suite and the machine learning models, SVM and ResNet18, on NVIDIA Tesla P100. For the Rodinia benchmarks, GEVO improves GPU kernel runtime performance by an average of 49.48% and by as much as 412% over the fully compiler-optimized baseline. If kernel output accuracy is relaxed to tolerate up to 1% error, GEVO can find kernel variants that outperform the baseline version by an average of 51.08%. For the machine learning workloads, GEVO achieves kernel performance improvement for SVM on the MNIST handwriting recognition (3.24X) and the a9a income prediction (2.93X) datasets with no loss of model accuracy. GEVO achieves 1.79X kernel performance improvement on image classification using ResNet18/CIFAR-10, with less than 1% model accuracy reduction.
A Data-Driven Frequency Scaling Approach for Deadline-aware Energy Efficient Scheduling on Graphics Processing Units (GPUs)
Venue: arXiv cs.DC 2020
Year: 2020

Modern computing paradigms, such as cloud computing, are increasingly adopting GPUs to boost their computing capabilities primarily due to the heterogeneous nature of AI/ML/deep learning workloads. However, the energy consumption of GPUs is a critical problem. Dynamic Voltage Frequency Scaling (DVFS) is a widely used technique to reduce the dynamic power of GPUs. Yet, configuring the optimal clock frequency for essential performance requirements is a non-trivial task due to the complex nonlinear relationship between the application's runtime performance characteristics, energy, and execution time. It becomes more challenging when different applications behave distinctively with similar clock settings. Simple analytical solutions and standard GPU frequency scaling heuristics fail to capture these intricacies and scale the frequencies appropriately. In this regard, we propose a data-driven frequency scaling technique by predicting the power and execution time of a given application over different clock settings. We collect the data from application profiling and train the models to predict the outcome accurately. The proposed solution is generic and can be easily extended to different kinds of workloads and GPU architectures. Furthermore, using this frequency scaling by prediction models, we present a deadline-aware application scheduling algorithm to reduce energy consumption while simultaneously meeting their deadlines. We conduct real extensive experiments on NVIDIA GPUs using several benchmark applications. The experiment results have shown that our prediction models have high accuracy with the average RMSE values of 0.38 and 0.05 for energy and time prediction, respectively. Also, the scheduling algorithm consumes 15.07% less energy as compared to the baseline policies.
Natural Language Processing with Deep Learning for Medical Adverse Event Detection from Free-Text Medical Narratives: A Case Study of Detecting Total Hip Replacement Dislocation
Venue: arXiv cs.CL 2020
Year: 2020

Accurate and timely detection of medical adverse events (AEs) from free-text medical narratives is challenging. Natural language processing (NLP) with deep learning has already shown great potential for analyzing free-text data, but its application for medical AE detection has been limited. In this study we proposed deep learning based NLP (DL-NLP) models for efficient and accurate hip dislocation AE detection following total hip replacement from standard (radiology notes) and non-standard (follow-up telephone notes) free-text medical narratives. We benchmarked these proposed models with a wide variety of traditional machine learning based NLP (ML-NLP) models, and also assessed the accuracy of International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) codes in capturing these hip dislocation AEs in a multi-center orthopaedic registry. All DL-NLP models out-performed all of the ML-NLP models, with a convolutional neural network (CNN) model achieving the best overall performance (Kappa = 0.97 for radiology notes, and Kappa = 1.00 for follow-up telephone notes). On the other hand, the ICD/CPT codes of the patients who sustained a hip dislocation AE were only 75.24% accurate, showing the potential of the proposed model to be used in largescale orthopaedic registries for accurate and efficient hip dislocation AE detection to improve the quality of care and patient outcome.
An integrated light management system with real-time light measurement and human perception
Venue: arXiv cs.CV 2020
Year: 2020

Illumination is important for well-being, productivity and safety across several environments, including offices, retail shops and industrial warehouses. Current techniques for setting up lighting require extensive and expert support and need to be repeated if the scene changes. Here we propose the first fully-automated light management system (LMS) which measures lighting in real-time, leveraging an RGBD sensor and a radiosity-based light propagation model. Thanks to the integration of light distribution and perception curves into the radiosity, we outperform a commercial software (Relux) on a newly introduced dataset. Furthermore, our proposed LMS is the first to estimate both the presence and the attention of the people in the environment, as well as their light perception. Our new LMS adapts therefore lighting to the scene and human activity and it is capable of saving up to 66%, as we experimentally quantify,without compromising the lighting quality.
On homotopy continuation based singularity distance computations for 3-RPR manipulators
Venue: arXiv cs.CG 2020
Year: 2020

It is known that parallel manipulators suffer from singular configurations. Evaluating the distance between a given configuration to the closest singular one is of interest for industrial applications (e.g.\ singularity-free path planning). For parallel manipulators of Stewart-Gough type, geometric meaningful distance measures are known, which are used for the computation of the singularity distance as the global minimizer of an optimization problem. In the case of hexapods and linear pentapods the critical points of the corresponding polynomial Lagrange function cannot be found by the Grobner basis method due to the degree and number of unknowns. But this polynomial system of equations can be solved by software tools of numerical algebraic geometry relying on homotopy continuation. To gain experiences for the treatment of the mentioned spatial manipulators, this paper attempts to find minimal multi-homogeneous Bezout numbers for the homotopy continuation based singularity distance computation with respect to various algebraic motion representations of planar Euclidean/equiform kinematics. The homogenous and non-homogenous representations under study are compared and discussed based on the 3-RPR manipulator.
DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications
Venue: arXiv cs.AI 2020
Year: 2020

One of the limitations of deep learning models with sparse features today stems from the predefined nature of their input, which requires a dictionary be defined prior to the training. With this paper we propose both a theory and a working system design which remove this limitation, and show that the resulting models are able to perform better and efficiently run at a much larger scale. Specifically, we achieve this by decoupling a model's content from its form to tackle architecture evolution and memory growth separately. To efficiently handle model growth, we propose a new neuron model, called DynamicCell, drawing inspiration from from the free energy principle [15] to introduce the concept of reaction to discharge non-digestive energy, which also subsumes gradient descent based approaches as its special cases. We implement DynamicCell by introducing a new server into TensorFlow to take over most of the work involving model growth. Consequently, it enables any existing deep learning models to efficiently handle arbitrary number of distinct sparse features (e.g., search queries), and grow incessantly without redefining the model. Most notably, one of our models, which has been reliably running in production for over a year, is capable of suggesting high quality keywords for advertisers of Google Smart Campaigns and achieved significant accuracy gains based on a challenging metric -- evidence that data-driven, self-evolving systems can potentially exceed the performance of traditional rule-based approaches.
From Horseback Riding to Changing the World: UX Competence as a Journey
Venue: arXiv cs.HC 2020
Year: 2020

In this paper, we explore the notion of competence in UX based on the perspective of practitioners. As a result of this exploration, we observed four domains through which we conceptualize a plan of sources of competence that describes the ways a UX practitioner develop competence. Based on this plane, we present the idea of competence as a journey. A journey whose furthest stage implies an urge towards transforming society and UX practice.
Multi-Modal Face Anti-Spoofing Based on Central Difference Networks
Venue: arXiv cs.CV 2020
Year: 2020

Face anti-spoofing (FAS) plays a vital role in securing face recognition systems from presentation attacks. Existing multi-modal FAS methods rely on stacked vanilla convolutions, which is weak in describing detailed intrinsic information from modalities and easily being ineffective when the domain shifts (e.g., cross attack and cross ethnicity). In this paper, we extend the central difference convolutional networks (CDCN) \cite{yu2020searching} to a multi-modal version, intending to capture intrinsic spoofing patterns among three modalities (RGB, depth and infrared). Meanwhile, we also give an elaborate study about single-modal based CDCN. Our approach won the first place in "Track Multi-Modal" as well as the second place in "Track Single-Modal (RGB)" of ChaLearn Face Anti-spoofing Attack Detection Challenge@CVPR2020 \cite{liu2020cross}. Our final submission obtains 1.02$\pm$0.59\% and 4.84$\pm$1.79\% ACER in "Track Multi-Modal" and "Track Single-Modal (RGB)", respectively. The codes are available at{https://github.com/ZitongYu/CDCN}.
Projection-Cost-Preserving Sketches: Proof Strategies and Constructions
Venue: arXiv cs.DS 2020
Year: 2020

In this note we illustrate how common matrix approximation methods, such as random projection and random sampling, yield projection-cost-preserving sketches, as introduced in [FSS13, CEM+15]. A projection-cost-preserving sketch is a matrix approximation which, for a given parameter $k$, approximately preserves the distance of the target matrix to all $k$-dimensional subspaces. Such sketches have applications to scalable algorithms for linear algebra, data science, and machine learning. Our goal is to simplify the presentation of proof techniques introduced in [CEM+15] and [CMM17] so that they can serve as a guide for future work. We also refer the reader to [CYD19], which gives a similar simplified exposition of the proof covered in Section 2.
Structuring Communities for Sharing Human Digital Memories in a Social P2P Network
Venue: arXiv cs.SI 2020
Year: 2020

A community is sub-network inside P2P networks that partition the network into groups of similar peers to improve performance by reducing network traffic and high search query success rate. Large communities are common in online social networks than traditional file-sharing P2P networks because many people capture huge amounts of data through their lives. This increases the number of hosts bearing similar data in the network and hence increases the size of communities. This article presents a Memory Thread-based Communities for our entity-based social P2P network that partition the network into groups of peers sharing data belonging to an entity - person, place, object or interest, having its own digital memory or be a part another memory. These connected peers having further similarities by organizing the network using linear orderings. A Memory-Thread is the collection of digital memories having a common reference key and organized according to some form of correlation. The simulation results show an increase in network performance for the proposed scheme along with a decrease in network overhead and higher query success rate compared to other similar schemes. The network maintains its performance even while the network traffic and size increase.