pwc-archive/papers-with-abstracts huggingface dataset:
Persistent And Scalable JADE: A Cloud based InMemory Multi-agent Framework
Authors: Nauman Khalid, Ghalib Ahmed Tahir, Peter Bloodsworth
Date: 2020-09-14 00:00:00

Multi-agent systems are often limited in terms of persistenceand scalability. This issue is more prevalent for applications inwhich agent states changes frequently. This makes the existingmethods less usable as they increase the agent's complexityand are less scalable. This research study has presented anovel in-memory agent persistence framework. Two prototypeshave been implemented, one using the proposed solution andthe other using an established agent persistency environment.Experimental results confirm that the proposed framework ismore scalable than existing approaches whilst providing asimilar level of persistency. These findings will help futurereal-time multiagent systems to become scalable and persistentin a dynamic cloud environment.
Completely Self-Supervised Crowd Counting via Distribution Matching
Authors: Deepak Babu Sam, Abhinav Agarwalla, Jimmy Joseph, Vishwanath A. Sindagi, R. Venkatesh Babu, Vishal M. Patel
Date: 2020-09-14 00:00:00
Tasks: Crowd Counting, Density Estimation

Dense crowd counting is a challenging task that demands millions of head annotations for training models. Though existing self-supervised approaches could learn good representations, they require some labeled data to map these features to the end task of density estimation. We mitigate this issue with the proposed paradigm of complete self-supervision, which does not need even a single labeled image. The only input required to train, apart from a large set of unlabeled crowd images, is the approximate upper limit of the crowd count for the given dataset. Our method dwells on the idea that natural crowds follow a power law distribution, which could be leveraged to yield error signals for backpropagation. A density regressor is first pretrained with self-supervision and then the distribution of predictions is matched to the prior by optimizing Sinkhorn distance between the two. Experiments show that this results in effective learning of crowd features and delivers significant counting performance. Furthermore, we establish the superiority of our method in less data setting as well. The code and models for our approach is available at https://github.com/val-iisc/css-ccnn.
Synbols: Probing Learning Algorithms with Synthetic Datasets
Authors: Alexandre Lacoste, Pau Rodríguez, Frédéric Branchaud-Charron, Parmida Atighehchian, Massimo Caccia, Issam Laradji, Alexandre Drouin, Matt Craddock, Laurent Charlin, David Vázquez
Date: 2020-09-14 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Active Learning, Few-Shot Learning, Object Counting, Out-of-Distribution Generalization, Representation Learning

Progress in the field of machine learning has been fueled by the introduction of benchmark datasets pushing the limits of existing algorithms. Enabling the design of datasets to test specific properties and failure modes of learning algorithms is thus a problem of high interest, as it has a direct impact on innovation in the field. In this sense, we introduce Synbols -- Synthetic Symbols -- a tool for rapidly generating new datasets with a rich composition of latent features rendered in low resolution images. Synbols leverages the large amount of symbols available in the Unicode standard and the wide range of artistic font provided by the open font community. Our tool's high-level interface provides a language for rapidly generating new distributions on the latent features, including various types of textures and occlusions. To showcase the versatility of Synbols, we use it to dissect the limitations and flaws in standard learning algorithms in various learning setups including supervised learning, active learning, out of distribution generalization, unsupervised representation learning, and object counting.
Adaptive Convolution Kernel for Artificial Neural Networks
Authors: F. Boray Tek, İlker Çam, Deniz Karlı
Date: 2020-09-14 00:00:00
Tasks: image-classification, Image Classification

Many deep neural networks are built by using stacked convolutional layers of fixed and single size (often 3$\times$3) kernels. This paper describes a method for training the size of convolutional kernels to provide varying size kernels in a single layer. The method utilizes a differentiable, and therefore backpropagation-trainable Gaussian envelope which can grow or shrink in a base grid. Our experiments compared the proposed adaptive layers to ordinary convolution layers in a simple two-layer network, a deeper residual network, and a U-Net architecture. The results in the popular image classification datasets such as MNIST, MNIST-CLUTTERED, CIFAR-10, Fashion, and ``Faces in the Wild'' showed that the adaptive kernels can provide statistically significant improvements on ordinary convolution kernels. A segmentation experiment in the Oxford-Pets dataset demonstrated that replacing a single ordinary convolution layer in a U-shaped network with a single 7$\times$7 adaptive layer can improve its learning performance and ability to generalize.
P-DIFF: Learning Classifier with Noisy Labels based on Probability Difference Distributions
Authors: Wei Hu, QiHao Zhao, Yangyu Huang, Fan Zhang
Date: 2020-09-14 00:00:00

Learning deep neural network (DNN) classifier with noisy labels is a challenging task because the DNN can easily over-fit on these noisy labels due to its high capability. In this paper, we present a very simple but effective training paradigm called P-DIFF, which can train DNN classifiers but obviously alleviate the adverse impact of noisy labels. Our proposed probability difference distribution implicitly reflects the probability of a training sample to be clean, then this probability is employed to re-weight the corresponding sample during the training process. P-DIFF can also achieve good performance even without prior knowledge on the noise rate of training samples. Experiments on benchmark datasets also demonstrate that P-DIFF is superior to the state-of-the-art sample selection methods.
4Seasons: A Cross-Season Dataset for Multi-Weather SLAM in Autonomous Driving
Authors: Patrick Wenzel, Rui Wang, Nan Yang, Qing Cheng, Qadeer Khan, Lukas von Stumberg, Niclas Zeller, Daniel Cremers
Date: 2020-09-14 00:00:00
Tasks: Autonomous Driving, Visual Odometry

We present a novel dataset covering seasonal and challenging perceptual conditions for autonomous driving. Among others, it enables research on visual odometry, global place recognition, and map-based re-localization tracking. The data was collected in different scenarios and under a wide variety of weather conditions and illuminations, including day and night. This resulted in more than 350 km of recordings in nine different environments ranging from multi-level parking garage over urban (including tunnels) to countryside and highway. We provide globally consistent reference poses with up-to centimeter accuracy obtained from the fusion of direct stereo visual-inertial odometry with RTK-GNSS. The full dataset is available at https://www.4seasons-dataset.com.
PRAFlow_RVC: Pyramid Recurrent All-Pairs Field Transforms for Optical Flow Estimation in Robust Vision Challenge 2020
Authors: Zhexiong Wan, Yuxin Mao, Yuchao Dai
Date: 2020-09-14 00:00:00
Tasks: All, Optical Flow Estimation

Optical flow estimation is an important computer vision task, which aims at estimating the dense correspondences between two frames. RAFT (Recurrent All Pairs Field Transforms) currently represents the state-of-the-art in optical flow estimation. It has excellent generalization ability and has obtained outstanding results across several benchmarks. To further improve the robustness and achieve accurate optical flow estimation, we present PRAFlow (Pyramid Recurrent All-Pairs Flow), which builds upon the pyramid network structure. Due to computational limitation, our proposed network structure only uses two pyramid layers. At each layer, the RAFT unit is used to estimate the optical flow at the current resolution. Our model was trained on several simulate and real-image datasets, submitted to multiple leaderboards using the same model and parameters, and won the 2nd place in the optical flow task of ECCV 2020 workshop: Robust Vision Challenge.
Data fusion strategies for energy efficiency in buildings: Overview, challenges and novel orientations
Authors: Yassine Himeur, Abdullah Alsalemi, Ayman Al-Kababji, Faycal Bensaali, Abbes Amira
Date: 2020-09-14 00:00:00

Recently, tremendous interest has been devoted to develop data fusion strategies for energy efficiency in buildings, where various kinds of information can be processed. However, applying the appropriate data fusion strategy to design an efficient energy efficiency system is not straightforward; it requires a priori knowledge of existing fusion strategies, their applications and their properties. To this regard, seeking to provide the energy research community with a better understanding of data fusion strategies in building energy saving systems, their principles, advantages, and potential applications, this paper proposes an extensive survey of existing data fusion mechanisms deployed to reduce excessive consumption and promote sustainability. We investigate their conceptualizations, advantages, challenges and drawbacks, as well as performing a taxonomy of existing data fusion strategies and other contributing factors. Following, a comprehensive comparison of the state-of-the-art data fusion based energy efficiency frameworks is conducted using various parameters, including data fusion level, data fusion techniques, behavioral change influencer, behavioral change incentive, recorded data, platform architecture, IoT technology and application scenario. Moreover, a novel method for electrical appliance identification is proposed based on the fusion of 2D local texture descriptors, where 1D power signals are transformed into 2D space and treated as images. The empirical evaluation, conducted on three real datasets, shows promising performance, in which up to 99.68% accuracy and 99.52% F1 score have been attained. In addition, various open research challenges and future orientations to improve data fusion based energy efficiency ecosystems are explored.
Short-Term Forecasting COVID-19 Cases In Turkey Using Long Short-Term Memory Network
Authors: Selahattin Serdar Helli, Çağkan Demirci, Onur Çoban, Andaç Hamamci
Date: 2020-09-14 00:00:00

COVID-19 has been one of the most severe diseases, causing a harsh pandemic all over the world, since December 2019. The aim of this study is to evaluate the value of Long Short-Term Memory (LSTM) Networks in forecasting the total number of COVID-19 cases in Turkey. The COVID-19 data for 30 days, between March 24 and April 23, 2020, are used to estimate the next fifteen days. The mean absolute error of the LSTM Network for 15 days estimation is 1,69$\pm$1.35%. Whereas, for the same data, the error of the Box-Jenkins method is 3.24$\pm$1.56%, Prophet method is 6.88$\pm$4.96% and Holt-Winters Additive method with Damped Trend is 0.47$\pm$0.28%. Additionally, when the number of deaths data is also provided with the number of total cases to the input of LSTM Network, the mean error reduces to 0.99$\pm$0.51%. Consequently, addition of the number of deaths data to the input, results a lower error in forecasting, compared to using only the number of total cases as the input. However, Holt-Winters Additive method with Damped Trend gives superior results to LSTM Networks in forecasting the total number of COVID-19 cases.
Reservoir Memory Machines as Neural Computers
Authors: Benjamin Paaßen, Alexander Schulz, Terrence C. Stewart, Barbara Hammer
Date: 2020-09-14 00:00:00

Differentiable neural computers extend artificial neural networks with an explicit memory without interference, thus enabling the model to perform classic computation tasks such as graph traversal. However, such models are difficult to train, requiring long training times and large datasets. In this work, we achieve some of the computational capabilities of differentiable neural computers with a model that can be trained very efficiently, namely an echo state network with an explicit memory without interference. This extension enables echo state networks to recognize all regular languages, including those that contractive echo state networks provably can not recognize. Further, we demonstrate experimentally that our model performs comparably to its fully-trained deep version on several typical benchmark tasks for differentiable neural computers.
Adaptive Generation Model: A New Ensemble Method
Authors: Jiacheng Ruan, Jiahao Li
Date: 2020-09-14 00:00:00
Tasks: BIG-bench Machine Learning, Ensemble Learning, model

As a common method in Machine Learning, Ensemble Method is used to train multiple models from a data set and obtain better results through certain combination strategies. Stacking method, as representatives of Ensemble Learning methods, is often used in Machine Learning Competitions such as Kaggle. This paper proposes a variant of Stacking Model based on the idea of gcForest, namely Adaptive Generation Model (AGM). It means that the adaptive generation is performed not only in the horizontal direction to expand the width of each layer model, but also in the vertical direction to expand the depth of the model. For base models of AGM, they all come from preset basic Machine Learning Models. In addition, a feature augmentation method is added between layers to further improve the overall accuracy of the model. Finally, through comparative experiments on 7 data sets, the results show that the accuracy of AGM are better than its previous models.
UPB at SemEval-2020 Task 6: Pretrained Language Models for Definition Extraction
Authors: Andrei-Marius Avram, Dumitru-Clementin Cercel, Costin-Gabriel Chiru
Date: 2020-09-11 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Definition Extraction, General Classification, Language Modeling, Language Modelling, Relation Classification

This work presents our contribution in the context of the 6th task of SemEval-2020: Extracting Definitions from Free Text in Textbooks (DeftEval). This competition consists of three subtasks with different levels of granularity: (1) classification of sentences as definitional or non-definitional,(2) labeling of definitional sentences, and (3) relation classification. We use various pretrained language models (i.e., BERT, XLNet, RoBERTa, SciBERT, and ALBERT) to solve each of the three subtasks of the competition. Specifically, for each language model variant, we experiment by both freezing its weights and fine-tuning them. We also explore a multi-task architecture that was trained to jointly predict the outputs for the second and the third subtasks. Our best performing model evaluated on the DeftEval dataset obtains the 32nd place for the first subtask and the 37th place for the second subtask. The code is available for further research at: https://github.com/avramandrei/DeftEval.
Solving Arithmetic Word Problems by Scoring Equations with Recursive Neural Networks
Authors: Klim Zaporojets, Giannis Bekoulis, Johannes Deleu, Thomas Demeester, Chris Develder
Date: 2020-09-11 00:00:00

Solving arithmetic word problems is a cornerstone task in assessing language understanding and reasoning capabilities in NLP systems. Recent works use automatic extraction and ranking of candidate solution equations providing the answer to arithmetic word problems. In this work, we explore novel approaches to score such candidate solution equations using tree-structured recursive neural network (Tree-RNN) configurations. The advantage of this Tree-RNN approach over using more established sequential representations, is that it can naturally capture the structure of the equations. Our proposed method consists of transforming the mathematical expression of the equation into an expression tree. Further, we encode this tree into a Tree-RNN by using different Tree-LSTM architectures. Experimental results show that our proposed method (i) improves overall performance with more than 3% accuracy points compared to previous state-of-the-art, and with over 15% points on a subset of problems that require more complex reasoning, and (ii) outperforms sequential LSTMs by 4% accuracy points on such more complex problems.
Coreference Resolution System for Indonesian Text with Mention Pair Method and Singleton Exclusion using Convolutional Neural Network
Authors: Turfa Auliarachman, Ayu Purwarianti
Date: 2020-09-11 00:00:00
Tasks: coreference-resolution, Coreference Resolution, Word Embeddings

Neural network has shown promising performance on coreference resolution systems that uses mention pair method. With deep neural network, it can learn hidden and deep relations between two mentions. However, there is no work on coreference resolution for Indonesian text that uses this learning technique. The state-of-the-art system for Indonesian text only states the use of lexical and syntactic features can improve the existing coreference resolution system. In this paper, we propose a new coreference resolution system for Indonesian text with mention pair method that uses deep neural network to learn the relations of the two mentions. In addition to lexical and syntactic features, in order to learn the representation of the mentions words and context, we use word embeddings and feed them to Convolutional Neural Network (CNN). Furthermore, we do singleton exclusion using singleton classifier component to prevent singleton mentions entering any entity clusters at the end. Achieving 67.37% without singleton exclusion, 63.27% with trained singleton classifier, and 75.95% with gold singleton classifier on CoNLL average F1 score, our proposed system outperforms the state-of-the-art system.
Investigating Bi-LSTM and CRF with POS Tag Embedding for Indonesian Named Entity Tagger
Authors: Devin Hoesen, Ayu Purwarianti
Date: 2020-09-11 00:00:00
Tasks: POS, TAG

Researches on Indonesian named entity (NE) tagger have been conducted since years ago. However, most did not use deep learning and instead employed traditional machine learning algorithms such as association rule, support vector machine, random forest, na\"ive bayes, etc. In those researches, word lists as gazetteers or clue words were provided to enhance the accuracy. Here, we attempt to employ deep learning in our Indonesian NE tagger. We use long short-term memory (LSTM) as the topology since it is the state-of-the-art of NE tagger. By using LSTM, we do not need a word list in order to enhance the accuracy. Basically, there are two main things that we investigate. The first is the output layer of the network: Softmax vs conditional random field (CRF). The second is the usage of part of speech (POS) tag embedding input layer. Using 8400 sentences as the training data and 97 sentences as the evaluation data, we find that using POS tag embedding as additional input improves the performance of our Indonesian NE tagger. As for the comparison between Softmax and CRF, we find that both architectures have a weakness in classifying an NE tag.
Relation Detection for Indonesian Language using Deep Neural Network -- Support Vector Machine
Authors: Ramos Janoah Hasudungan, Ayu Purwarianti
Date: 2020-09-12 00:00:00
Tasks: POS, Position, Relation, TAG

Relation Detection is a task to determine whether two entities are related or not. In this paper, we employ neural network to do relation detection between two named entities for Indonesian Language. We used feature such as word embedding, position embedding, POS-Tag embedding, and character embedding. For the model, we divide the model into two parts: Front-part classifier (Convolutional layer or LSTM layer) and Back-part classifier (Dense layer or SVM). We did grid search method of neural network hyper parameter and SVM. We used 6000 Indonesian sentences for training process and 1,125 for testing. The best result is 0.8083 on F1-Score using Convolutional Layer as front-part and SVM as back-part.
Improving Indonesian Text Classification Using Multilingual Language Model
Authors: Ilham Firdausi Putra, Ayu Purwarianti
Date: 2020-09-12 00:00:00
Tasks: Classification, General Classification, Language Modeling, Language Modelling, model, Sentiment Analysis, text-classification, Text Classification

Compared to English, the amount of labeled data for Indonesian text classification tasks is very small. Recently developed multilingual language models have shown its ability to create multilingual representations effectively. This paper investigates the effect of combining English and Indonesian data on building Indonesian text classification (e.g., sentiment analysis and hate speech) using multilingual language models. Using the feature-based approach, we observe its performance on various data sizes and total added English data. The experiment showed that the addition of English data, especially if the amount of Indonesian data is small, improves performance. Using the fine-tuning approach, we further showed its effectiveness in utilizing the English language to build Indonesian text classification models.
Improving Bi-LSTM Performance for Indonesian Sentiment Analysis Using Paragraph Vector
Authors: Ayu Purwarianti, Ida Ayu Putu Ari Crisdayanti
Date: 2020-09-12 00:00:00
Tasks: Sentiment Analysis, Sentiment Classification

Bidirectional Long Short-Term Memory Network (Bi-LSTM) has shown promising performance in sentiment classification task. It processes inputs as sequence of information. Due to this behavior, sentiment predictions by Bi-LSTM were influenced by words sequence and the first or last phrases of the texts tend to have stronger features than other phrases. Meanwhile, in the problem scope of Indonesian sentiment analysis, phrases that express the sentiment of a document might not appear in the first or last part of the document that can lead to incorrect sentiment classification. To this end, we propose the using of an existing document representation method called paragraph vector as additional input features for Bi-LSTM. This vector provides information context of the document for each sequence processing. The paragraph vector is simply concatenated to each word vector of the document. This representation also helps to differentiate ambiguous Indonesian words. Bi-LSTM and paragraph vector were previously used as separate methods. Combining the two methods has shown a significant performance improvement of Indonesian sentiment analysis model. Several case studies on testing data showed that the proposed method can handle the sentiment phrases position problem encountered by Bi-LSTM.