pwc-archive/papers-with-abstracts huggingface dataset:
EB-DEVS: A Formal Framework for Modeling and Simulation of Emergent Behavior in Dynamic Complex Systems
Authors: Daniel J. Foguelman, Philipp Henning, Adelinde Uhrmacher, Rodrigo Castro
Date: 2020-10-10 00:00:00

Emergent behavior is a key feature defining a system under study as a complex system. Simulation has been recognized as the only way to deal with the study of the emergency of properties (at a macroscopic level) among groups of system components (at a microscopic level), for the manifestations of emergent structures cannot be deduced from analysing components in isolation. A systems-oriented generalisation must consider the presence of feedback loops (micro components react to macro properties), interaction among components of different classes (modular composition) and layered interaction of subsystems operating at different spatio-temporal scales (hierarchical organisation). In this work we introduce Emergent Behavior-DEVS (EB-DEVS) a Modeling and Simulation (M&S) formalism that permits reasoning about complex systems where emergent behavior is placed at the forefront of the analysis activity. EB-DEVS builds on the DEVS formalism, adding upward/downward communication channels to well-established capabilities for modular and hierarchical M&S of heterogeneous multi-formalism systems. EB-DEVS takes a minimalist stance on expressiveness, introducing a small set of extensions on Classic DEVS that can cope with emergent behavior, and making both formalisms interoperable (the modeler decides which subsystems deserve to be expressed via micro-macro dynamics). We present three case studies: flocks of birds with learning, population epidemics with vaccination and sub-cellular dynamics with homeostasis, through which we showcase how EB-DEVS performs by placing emergent properties at the center of the M&S process.
Cross-Stack Workload Characterization of Deep Recommendation Systems
Authors: Samuel Hsia, Udit Gupta, Mark Wilkening, Carole-Jean Wu, Gu-Yeon Wei, David Brooks
Date: 2020-10-10 00:00:00
Tasks: CPU, Recommendation Systems

Deep learning based recommendation systems form the backbone of most personalized cloud services. Though the computer architecture community has recently started to take notice of deep recommendation inference, the resulting solutions have taken wildly different approaches - ranging from near memory processing to at-scale optimizations. To better design future hardware systems for deep recommendation inference, we must first systematically examine and characterize the underlying systems-level impact of design decisions across the different levels of the execution stack. In this paper, we characterize eight industry-representative deep recommendation models at three different levels of the execution stack: algorithms and software, systems platforms, and hardware microarchitectures. Through this cross-stack characterization, we first show that system deployment choices (i.e., CPUs or GPUs, batch size granularity) can give us up to 15x speedup. To better understand the bottlenecks for further optimization, we look at both software operator usage breakdown and CPU frontend and backend microarchitectural inefficiencies. Finally, we model the correlation between key algorithmic model architecture features and hardware bottlenecks, revealing the absence of a single dominant algorithmic component behind each hardware bottleneck.
Boosted EfficientNet: Detection of Lymph Node Metastases in Breast Cancer Using Convolutional Neural Network
Authors: Jun Wang, Qianying Liu, Haotian Xie, Zhaogang Yang, Hefeng Zhou
Date: 2020-10-10 00:00:00
Tasks: Data Augmentation, Image Cropping, whole slide images

In recent years, advances in the development of whole-slide images have laid a foundation for the utilization of digital images in pathology. With the assistance of computer images analysis that automatically identifies tissue or cell types, they have greatly improved the histopathologic interpretation and diagnosis accuracy. In this paper, the Convolutional Neutral Network (CNN) has been adapted to predict and classify lymph node metastasis in breast cancer. Unlike traditional image cropping methods that are only suitable for large resolution images, we propose a novel data augmentation method named Random Center Cropping (RCC) to facilitate small resolution images. RCC enriches the datasets while retaining the image resolution and the center area of images. In addition, we reduce the downsampling scale of the network to further facilitate small resolution images better. Moreover, Attention and Feature Fusion (FF) mechanisms are employed to improve the semantic information of images. Experiments demonstrate that our methods boost performances of basic CNN architectures. And the best-performed method achieves an accuracy of 97.96% and an AUC of 99.68% on RPCam datasets, respectively.
Vehicle predictive trajectory patterns from isochronous data
Authors: D. Damian
Date: 2020-10-10 00:00:00

Measuring and analyzing sensor data is the basic technique in vehicle dynamics development and with the advancement of embedded and data acquisition systems it is possible to analyze large data sets. In this paper a detailed method is presented for assessing and mapping isochronous trajectory patterns in Graz (Austria) by using data fusion from video, ArduinoUno and the compass sensor HDMM01. The predictive isochronous trajectory patterns are derived from the data values for a predefined time horizon. Both extreme driving behavior and hazardous road geometries can be identified. It is possible to provide instant road sensor data which can be used to compare the data from a trajectory path as well as for different time instances. Results of this study show that the trajectory patterns are successful in predicting the likely evolution of a current trajectory pattern and can provide assessment on future driving situations. The obtained data from this study can be useful as reference in future city planning for energy saving driving pathways as well as vehicle design and engineering improvements based on quantitative and relevant dynamic measurements.
Historical Credibility for Movie Reviews and Its Application to Weakly Supervised Classification
Authors: Min-Seon Kim, Bo-Young Lim, Han-Sub Shin, Hyuk-Yoon Kwon
Date: 2020-10-10 00:00:00
Tasks: Classification, Weakly Supervised Classification

In this study, we deal with the problem of judging the credibility of movie reviews. The problem is challenging because even experts cannot clearly and efficiently judge the credibility of a movie review and the number of movie reviews is very large. To tackle this problem, we propose historical credibility that judges the credibility of reviews based on the historical ratings and textual reviews written by each reviewer. For this, we present three kinds of criteria that can clearly classify the reviews into trusted or distrusted ones. We validate the effectiveness of the proposed historical credibility through extensive analysis. Specifically, we show that characteristics between the trusted or distrusted reviews are quite distinguishable in terms of three viewpoints: 1) distribution, 2) statistics, and 3) correlation. Then, we apply historical credibility to a weakly supervised model to classify a given review as a trusted or distrusted one. First, we show that it is significantly efficient because the entire data set is annotated according to the predefined criteria. Indeed, it can annotate 6,400 movie reviews only in 0.093 seconds, which occupy only 0.55%~1.88% of the total learning time when we use LSTM and SVM as the learning model. Second, we show that the historical credibility-based classification model clearly outperforms the textual review-based classification model. Specifically, the classification accuracy of the former outperforms that of the latter by up to 11.7%~13.4%. In addition, we clearly confirm that our classification model shows higher accuracy as the data size increases.
Reinforcement Learning on Computational Resource Allocation of Cloud-based Wireless Networks
Authors: Beiran Chen, Yi Zhang, George Iosifidis, Mingming Liu
Date: 2020-10-10 00:00:00
Tasks: CPU, Management, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Wireless networks used for Internet of Things (IoT) are expected to largely involve cloud-based computing and processing. Softwarised and centralised signal processing and network switching in the cloud enables flexible network control and management. In a cloud environment, dynamic computational resource allocation is essential to save energy while maintaining the performance of the processes. The stochastic features of the Central Processing Unit (CPU) load variation as well as the possible complex parallelisation situations of the cloud processes makes the dynamic resource allocation an interesting research challenge. This paper models this dynamic computational resource allocation problem into a Markov Decision Process (MDP) and designs a model-based reinforcement-learning agent to optimise the dynamic resource allocation of the CPU usage. Value iteration method is used for the reinforcement-learning agent to pick up the optimal policy during the MDP. To evaluate our performance we analyse two types of processes that can be used in the cloud-based IoT networks with different levels of parallelisation capabilities, i.e., Software-Defined Radio (SDR) and Software-Defined Networking (SDN). The results show that our agent rapidly converges to the optimal policy, stably performs in different parameter settings, outperforms or at least equally performs compared to a baseline algorithm in energy savings for different scenarios.
An Encoder-Decoder CNN for Hair Removal in Dermoscopic Images
Authors: Lidia Talavera-Martínez, Pedro Bibiloni, Manuel González-Hidalgo
Date: 2020-10-10 00:00:00
Tasks: Decoder

The process of removing occluding hair has a relevant role in the early and accurate diagnosis of skin cancer. It consists of detecting hairs and restore the texture below them, which is sporadically occluded. In this work, we present a model based on convolutional neural networks for hair removal in dermoscopic images. During the network's training, we use a combined loss function to improve the restoration ability of the proposed model. In order to train the CNN and to quantitatively validate their performance, we simulate the presence of skin hair in hairless images extracted from publicly known datasets such as the PH2, dermquest, dermis, EDRA2002, and the ISIC Data Archive. As far as we know, there is no other hair removal method based on deep learning. Thus, we compare our results with six state-of-the-art algorithms based on traditional computer vision techniques by means of similarity measures that compare the reference hairless image and the one with hair simulated. Finally, a statistical test is used to compare the methods. Both qualitative and quantitative results demonstrate the effectiveness of our network.
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor
Authors: Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
Date: 2020-10-10 00:00:00
Proceeding: ACL 2021 5
Tasks: Dependency Parsing, Knowledge Distillation, Structured Prediction

Knowledge distillation is a critical technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). The objective function of knowledge distillation is typically the cross-entropy between the teacher and the student's output distributions. However, for structured prediction problems, the output space is exponential in size; therefore, the cross-entropy objective becomes intractable to compute and optimize directly. In this paper, we derive a factorized form of the knowledge distillation objective for structured prediction, which is tractable for many typical choices of the teacher and student models. In particular, we show the tractability and empirical effectiveness of structural knowledge distillation between sequence labeling and dependency parsing models under four different scenarios: 1) the teacher and student share the same factorization form of the output structure scoring function; 2) the student factorization produces more fine-grained substructures than the teacher factorization; 3) the teacher factorization produces more fine-grained substructures than the student factorization; 4) the factorization forms from the teacher and the student are incompatible.
Category-Learning with Context-Augmented Autoencoder
Authors: Denis Kuzminykh, Laida Kushnareva, Timofey Grigoryev, Alexander Zatolokin
Date: 2020-10-10 00:00:00
Tasks: Data Augmentation

Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. Biological neural networks are known to solve this problem quite well in unsupervised manner, yet unsupervised artificial neural networks either struggle to do it or require fine tuning for each task individually. We associate this with the fact that a biological brain learns in the context of the relationships between observations, while an artificial network does not. We also notice that, though a naive data augmentation technique can be very useful for supervised learning problems, autoencoders typically fail to generalize transformations from data augmentations. Thus, we believe that providing additional knowledge about relationships between data samples will improve model's capability of finding useful inner data representation. More formally, we consider a dataset not as a manifold, but as a category, where the examples are objects. Two these objects are connected by a morphism, if they actually represent different transformations of the same entity. Following this formalism, we propose a novel method of using data augmentations when training autoencoders. We train a Variational Autoencoder in such a way, that it makes transformation outcome predictable by auxiliary network in terms of the hidden representation. We believe that the classification accuracy of a linear classifier on the learned representation is a good metric to measure its interpretability. In our experiments, present approach outperforms $\beta$-VAE and is comparable with Gaussian-mixture VAE.
Compressing Transformer-Based Semantic Parsing Models using Compositional Code Embeddings
Authors: Prafull Prakash, Saurabh Kumar Shashidhar, Wenlong Zhao, Subendhu Rongali, Haidar Khan, Michael Kayser
Date: 2020-10-10 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Semantic Parsing

The current state-of-the-art task-oriented semantic parsing models use BERT or RoBERTa as pretrained encoders; these models have huge memory footprints. This poses a challenge to their deployment for voice assistants such as Amazon Alexa and Google Assistant on edge devices with limited memory budgets. We propose to learn compositional code embeddings to greatly reduce the sizes of BERT-base and RoBERTa-base. We also apply the technique to DistilBERT, ALBERT-base, and ALBERT-large, three already compressed BERT variants which attain similar state-of-the-art performances on semantic parsing with much smaller model sizes. We observe 95.15% ~ 98.46% embedding compression rates and 20.47% ~ 34.22% encoder compression rates, while preserving greater than 97.5% semantic parsing performances. We provide the recipe for training and analyze the trade-off between code embedding sizes and downstream performances.
Beyond Language: Learning Commonsense from Images for Reasoning
Authors: Wanqing Cui, Yanyan Lan, Liang Pang, Jiafeng Guo, Xueqi Cheng
Date: 2020-10-10 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Language Modeling, Language Modelling, Layout Generation, Question Answering

This paper proposes a novel approach to learn commonsense from images, instead of limited raw texts or costly constructed knowledge bases, for the commonsense reasoning problem in NLP. Our motivation comes from the fact that an image is worth a thousand words, where richer scene information could be leveraged to help distill the commonsense knowledge, which is often hidden in languages. Our approach, namely Loire, consists of two stages. In the first stage, a bi-modal sequence-to-sequence approach is utilized to conduct the scene layout generation task, based on a text representation model ViBERT. In this way, the required visual scene knowledge, such as spatial relations, will be encoded in ViBERT by the supervised learning process with some bi-modal data like COCO. Then ViBERT is concatenated with a pre-trained language model to perform the downstream commonsense reasoning tasks. Experimental results on two commonsense reasoning problems, i.e. commonsense question answering and pronoun resolution, demonstrate that Loire outperforms traditional language-based methods. We also give some case studies to show what knowledge is learned from images and explain how the generated scene layout helps the commonsense reasoning process.
A Recursive Markov Boundary-Based Approach to Causal Structure Learning
Authors: Ehsan Mokhtarian, Sina Akbari, AmirEmad Ghassami, Negar Kiyavash
Date: 2020-10-10 00:00:00

Constraint-based methods are one of the main approaches for causal structure learning that are particularly valued as they are asymptotically guaranteed to find a structure that is Markov equivalent to the causal graph of the system. On the other hand, they may require an exponentially large number of conditional independence (CI) tests in the number of variables of the system. In this paper, we propose a novel recursive constraint-based method for causal structure learning that significantly reduces the required number of CI tests compared to the existing literature. The idea of the proposed approach is to use Markov boundary information to identify a specific variable that can be removed from the set of variables without affecting the statistical dependencies among the other variables. Having identified such a variable, we discover its neighborhood, remove that variable from the set of variables, and recursively learn the causal structure over the remaining variables. We further provide a lower bound on the number of CI tests required by any constraint-based method. Comparing this lower bound to our achievable bound demonstrates the efficiency of the proposed approach. Our experimental results show that the proposed algorithm outperforms state-of-the-art both on synthetic and real-world structures.
The emergence of Explainability of Intelligent Systems: Delivering Explainable and Personalised Recommendations for Energy Efficiency
Authors: Christos Sardianos, Iraklis Varlamis, Christos Chronis, George Dimitrakopoulos, Abdullah Alsalemi, Yassine Himeur, Faycal Bensaali, Abbes Amira
Date: 2020-10-10 00:00:00
Tasks: Decision Making, Recommendation Systems

The recent advances in artificial intelligence namely in machine learning and deep learning, have boosted the performance of intelligent systems in several ways. This gave rise to human expectations, but also created the need for a deeper understanding of how intelligent systems think and decide. The concept of explainability appeared, in the extent of explaining the internal system mechanics in human terms. Recommendation systems are intelligent systems that support human decision making, and as such, they have to be explainable in order to increase user trust and improve the acceptance of recommendations. In this work, we focus on a context-aware recommendation system for energy efficiency and develop a mechanism for explainable and persuasive recommendations, which are personalized to user preferences and habits. The persuasive facts either emphasize on the economical saving prospects (Econ) or on a positive ecological impact (Eco) and explanations provide the reason for recommending an energy saving action. Based on a study conducted using a Telegram bot, different scenarios have been validated with actual data and human feedback. Current results show a total increase of 19\% on the recommendation acceptance ratio when both economical and ecological persuasive facts are employed. This revolutionary approach on recommendation systems, demonstrates how intelligent recommendations can effectively encourage energy saving behavior.
Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns
Authors: Lei Zhou, Liang Ding, Koichi Takeda
Date: 2020-10-10 00:00:00
Proceeding: WMT (EMNLP) 2020 11
Tasks: Sentence, Translation

This paper describes our submission of the WMT 2020 Shared Task on Sentence Level Direct Assessment, Quality Estimation (QE). In this study, we empirically reveal the \textit{mismatching issue} when directly adopting BERTScore to QE. Specifically, there exist lots of mismatching errors between the source sentence and translated candidate sentence with token pairwise similarity. In response to this issue, we propose to expose explicit cross-lingual patterns, \textit{e.g.} word alignments and generation score, to our proposed zero-shot models. Experiments show that our proposed QE model with explicit cross-lingual patterns could alleviate the mismatching issue, thereby improving the performance. Encouragingly, our zero-shot QE method could achieve comparable performance with supervised QE method, and even outperforms the supervised counterpart on 2 out of 6 directions. We expect our work could shed light on the zero-shot QE model improvement.
FIND: Human-in-the-Loop Debugging Deep Text Classifiers
Authors: Piyawat Lertvittayakumjorn, Lucia Specia, Francesca Toni
Date: 2020-10-10 00:00:00
Proceeding: EMNLP 2020 11

Since obtaining a perfect training dataset (i.e., a dataset which is considerably large, unbiased, and well-representative of unseen cases) is hardly possible, many real-world text classifiers are trained on the available, yet imperfect, datasets. These classifiers are thus likely to have undesirable properties. For instance, they may have biases against some sub-populations or may not work effectively in the wild due to overfitting. In this paper, we propose FIND -- a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. Experiments show that by using FIND, humans can improve CNN text classifiers which were trained under different types of imperfect datasets (including datasets with biases and datasets with dissimilar train-test distributions).
An Empirical Investigation of Beam-Aware Training in Supertagging
Authors: Renato Negrinho, Matthew R. Gormley, Geoffrey J. Gordon
Date: 2020-10-10 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Structured Prediction

Structured prediction is often approached by training a locally normalized model with maximum likelihood and decoding approximately with beam search. This approach leads to mismatches as, during training, the model is not exposed to its mistakes and does not use beam search. Beam-aware training aims to address these problems, but unfortunately, it is not yet widely used due to a lack of understanding about how it impacts performance, when it is most useful, and whether it is stable. Recently, Negrinho et al. (2018) proposed a meta-algorithm that captures beam-aware training algorithms and suggests new ones, but unfortunately did not provide empirical results. In this paper, we begin an empirical investigation: we train the supertagging model of Vaswani et al. (2016) and a simpler model with instantiations of the meta-algorithm. We explore the influence of various design choices and make recommendations for choosing them. We observe that beam-aware training improves performance for both models, with large improvements for the simpler model which must effectively manage uncertainty during decoding. Our results suggest that a model must be learned with search to maximize its effectiveness.