pwc-archive/papers-with-abstracts huggingface dataset:
Learning local regularization for variational image restoration
Authors: Jean Prost, Antoine Houdard, Andrés Almansa, Nicolas Papadakis
Date: 2021-02-11 00:00:00
Tasks: Deblurring, Denoising, Image Restoration

In this work, we propose a framework to learn a local regularization model for solving general image restoration problems. This regularizer is defined with a fully convolutional neural network that sees the image through a receptive field corresponding to small image patches. The regularizer is then learned as a critic between unpaired distributions of clean and degraded patches using a Wasserstein generative adversarial networks based energy. This yields a regularization function that can be incorporated in any image restoration problem. The efficiency of the framework is finally shown on denoising and deblurring applications.
Transfer Learning Approach for Arabic Offensive Language Detection System -- BERT-Based Model
Authors: Fatemah Husain, Ozlem Uzuner
Date: 2021-02-09 00:00:00
Tasks: Transfer Learning

Developing a system to detect online offensive language is very important to the health and the security of online users. Studies have shown that cyberhate, online harassment and other misuses of technology are on the rise, particularly during the global Coronavirus pandemic in 2020. According to the latest report by the Anti-Defamation League (ADL), 35% of online users reported online harassment related to their identity-based characteristics, which is a 3% increase over 2019. Applying advanced techniques from the Natural Language Processing (NLP) field to support the development of an online hate-free community is a critical task for social justice. Transfer learning enhances the performance of the classifier by allowing the transfer of knowledge from one domain or one dataset to others that have not been seen before, thus, supporting the classifier to be more generalizable. In our study, we apply the principles of transfer learning cross multiple Arabic offensive language datasets to compare the effects on system performance. This study aims at investigating the effects of fine-tuning and training Bidirectional Encoder Representations from Transformers (BERT) model on multiple Arabic offensive language datasets individually and testing it using other datasets individually. Our experiment starts with a comparison among multiple BERT models to guide the selection of the main model that is used for our study. The study also investigates the effects of concatenating all datasets to be used for fine-tuning and training BERT model. Our results demonstrate the limited effects of transfer learning on the performance of the classifiers, particularly for highly dialectic comments.
Differentiable Generative Phonology
Authors: Shijie Wu, Edoardo Maria Ponti, Ryan Cotterell
Date: 2021-02-10 00:00:00

The goal of generative phonology, as formulated by Chomsky and Halle (1968), is to specify a formal system that explains the set of attested phonological strings in a language. Traditionally, a collection of rules (or constraints, in the case of optimality theory) and underlying forms (UF) are posited to work in tandem to generate phonological strings. However, the degree of abstraction of UFs with respect to their concrete realizations is contentious. As the main contribution of our work, we implement the phonological generative system as a neural model differentiable end-to-end, rather than as a set of rules or constraints. Contrary to traditional phonology, in our model, UFs are continuous vectors in $\mathbb{R}^d$, rather than discrete strings. As a consequence, UFs are discovered automatically rather than posited by linguists, and the model can scale to the size of a realistic vocabulary. Moreover, we compare several modes of the generative process, contemplating: i) the presence or absence of an underlying representation in between morphemes and surface forms (SFs); and ii) the conditional dependence or independence of UFs with respect to SFs. We evaluate the ability of each mode to predict attested phonological strings on 2 datasets covering 5 and 28 languages, respectively. The results corroborate two tenets of generative phonology, viz. the necessity for UFs and their independence from SFs. In general, our neural model of generative phonology learns both UFs and SFs automatically and on a large-scale.
Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation
Authors: Renjie Zheng, Junkun Chen, Mingbo Ma, Liang Huang
Date: 2021-02-10 00:00:00
Tasks: Language Modeling, Language Modelling, Machine Translation, Representation Learning, speech-recognition, Speech Recognition, Translation

Recently, representation learning for text and speech has successfully improved many language related tasks. However, all existing methods suffer from two limitations: (a) they only learn from one input modality, while a unified representation for both speech and text is needed by tasks such as end-to-end speech translation, and as a result,(b) they can not exploit various large-scale text and speech data and their performance is limited by the scarcity of parallel speech translation data.To address these problems, we propose a Fused Acoustic and Text Masked Language Model (FAT-MLM) which jointly learns a unified representation for both acoustic and text input from various types of corpora including parallel data for speech recognition and machine translation, and even pure speech and text data. Within this cross-modal representation learning framework, we further present an end-to-end model for Fused Acoustic and Text Speech Translation (FAT-ST). Experiments on three translation directions show that by fine-tuning from FAT-MLM, our proposed speech translation models substantially improve translation quality by up to +5.9 BLEU.
Toward Improving Coherence and Diversity of Slogan Generation
Authors: Yiping Jin, Akshay Bhatia, Dittaya Wanvarie, Phu T. V. Le
Date: 2021-02-11 00:00:00
Tasks: Diversity, Marketing, POS, TAG

Previous work in slogan generation focused on utilising slogan skeletons mined from existing slogans. While some generated slogans can be catchy, they are often not coherent with the company's focus or style across their marketing communications because the skeletons are mined from other companies' slogans. We propose a sequence-to-sequence (seq2seq) transformer model to generate slogans from a brief company description. A naive seq2seq model fine-tuned for slogan generation is prone to introducing false information. We use company name delexicalisation and entity masking to alleviate this problem and improve the generated slogans' quality and truthfulness. Furthermore, we apply conditional training based on the first words' POS tag to generate syntactically diverse slogans. Our best model achieved a ROUGE-1/-2/-L F1 score of 35.58/18.47/33.32. Besides, automatic and human evaluations indicate that our method generates significantly more factual, diverse and catchy slogans than strong LSTM and transformer seq2seq baselines.
Text Compression-aided Transformer Encoding
Authors: Zuchao Li, Zhuosheng Zhang, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita
Date: 2021-02-11 00:00:00
Tasks: Text Compression

Text encoding is one of the most important steps in Natural Language Processing (NLP). It has been done well by the self-attention mechanism in the current state-of-the-art Transformer encoder, which has brought about significant improvements in the performance of many NLP tasks. Though the Transformer encoder may effectively capture general information in its resulting representations, the backbone information, meaning the gist of the input text, is not specifically focused on. In this paper, we propose explicit and implicit text compression approaches to enhance the Transformer encoding and evaluate models using this approach on several typical downstream tasks that rely on the encoding heavily. Our explicit text compression approaches use dedicated models to compress text, while our implicit text compression approach simply adds an additional module to the main model to handle text compression. We propose three ways of integration, namely backbone source-side fusion, target-side fusion, and both-side fusion, to integrate the backbone information into Transformer-based models for various downstream tasks. Our evaluation on benchmark datasets shows that the proposed explicit and implicit text compression approaches improve results in comparison to strong baselines. We therefore conclude, when comparing the encodings to the baseline models, text compression helps the encoders to learn better language representations.
An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning
Authors: Markus Eberts, Adrian Ulges
Date: 2021-02-11 00:00:00
Proceeding: EACL 2021 2
Tasks: coreference-resolution, Coreference Resolution, Document-level Relation Extraction, Joint Entity and Relation Extraction, Named Entity Recognition (NER), Nested Named Entity Recognition, Relation, Relation Extraction, Sentence

We present a joint model for entity-level relation extraction from documents. In contrast to other approaches - which focus on local intra-sentence mention pairs and thus require annotations on mention level - our model operates on entity level. To do so, a multi-task approach is followed that builds upon coreference resolution and gathers relevant signals via multi-instance learning with multi-level representations combining global entity and local mention information. We achieve state-of-the-art relation extraction results on the DocRED dataset and report the first entity-level end-to-end relation extraction results for future reference. Finally, our experimental results suggest that a joint approach is on par with task-specific learning, though more efficient due to shared parameters and training steps.
Language Independent Emotion Quantification using Non linear Modelling of Speech
Authors: Uddalok Sarkar, Sayan Nag, Chirayata Bhattacharya, Shankha Sanyal, Archi Banerjee, Ranjan Sengupta, Dipak Ghosh
Date: 2021-02-11 00:00:00
Tasks: Clustering

At present emotion extraction from speech is a very important issue due to its diverse applications. Hence, it becomes absolutely necessary to obtain models that take into consideration the speaking styles of a person, vocal tract information, timbral qualities and other congenital information regarding his voice. Our speech production system is a nonlinear system like most other real world systems. Hence the need arises for modelling our speech information using nonlinear techniques. In this work we have modelled our articulation system using nonlinear multifractal analysis. The multifractal spectral width and scaling exponents reveals essentially the complexity associated with the speech signals taken. The multifractal spectrums are well distinguishable the in low fluctuation region in case of different emotions. The source characteristics have been quantified with the help of different non-linear models like Multi-Fractal Detrended Fluctuation Analysis, Wavelet Transform Modulus Maxima. The Results obtained from this study gives a very good result in emotion clustering.
A Fractal Approach to Characterize Emotions in Audio and Visual Domain: A Study on Cross-Modal Interaction
Authors: Sayan Nag, Uddalok Sarkar, Shankha Sanyal, Archi Banerjee, Souparno Roy, Samir Karmakar, Ranjan Sengupta, Dipak Ghosh
Date: 2021-02-11 00:00:00

It is already known that both auditory and visual stimulus is able to convey emotions in human mind to different extent. The strength or intensity of the emotional arousal vary depending on the type of stimulus chosen. In this study, we try to investigate the emotional arousal in a cross-modal scenario involving both auditory and visual stimulus while studying their source characteristics. A robust fractal analytic technique called Detrended Fluctuation Analysis (DFA) and its 2D analogue has been used to characterize three (3) standardized audio and video signals quantifying their scaling exponent corresponding to positive and negative valence. It was found that there is significant difference in scaling exponents corresponding to the two different modalities. Detrended Cross Correlation Analysis (DCCA) has also been applied to decipher degree of cross-correlation among the individual audio and visual stimulus. This is the first of its kind study which proposes a novel algorithm with which emotional arousal can be classified in cross-modal scenario using only the source audio and visual signals while also attempting a correlation between them.
A Comparative Study on Efficiencies of Variants of Convolutional Neural Networks based on Image Classification Task
Authors: Ayush Sharma
Date: 2020-10-15 00:00:00
Tasks: Classification, General Classification, Image Augmentation, image-classification, Image Classification

Deep neural networks have shown their high performance on image classification tasks but meanwhile more training difficulties. Due to its complexity and vanishing gradient, it usually takes a long time and a lot of computational resources to train deeper neural networks. Deep Residual networks (ResNets), however, can make the training process easier and faster. And at the same time, it achieves better accuracy compared to their equivalent neural networks. Deep Residual Networks have been proven to be a very successful model on image classification. Deep neural networks demonstrate to have a high performance on image classification tasks while being more difficult to train. We built two very different networks from scratch based on the idea of Densely Connected Convolution Networks. The architecture of the networks is designed based on the image resolution of this specific dataset and by calculating the Receptive Field of the convolution layers. We also used some non-conventional techniques related to image augmentation and Early stopping to improve the accuracy of our models. The networks are trained under high constraints and low computation resources
Time-series forecasting of Bitcoin prices using high-dimensional features: a machine learning approach
Authors: Mohammed Mudassir, Shada Bennbaia, Devrim Unal & Mohammad Hammoudeh
Date: 2020-07-04 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Time Series, Time Series Analysis, Time Series Forecasting

Bitcoin is a decentralized cryptocurrency, which is a type of digital asset that provides the basis for peer-to-peer financial transactions based on blockchain technology. One of the main problems with decentralized cryptocurrencies is price volatility, which indicates the need for studying the underlying price model. Moreover, Bitcoin prices exhibit non-stationary behavior, where the statistical distribution of data changes over time. This paper demonstrates high-performance machine learning-based classification and regression models for predicting Bitcoin price movements and prices in short and medium terms. In previous works, machine learning-based classification has been studied for an only one-day time frame, while this work goes beyond that by using machine learning-based models for one, seven, thirty and ninety days. The developed models are feasible and have high performance, with the classification models scoring up to 65% accuracy for next-day forecast and scoring from 62 to 64% accuracy for seventh–ninetieth-day forecast. For daily price forecast, the error percentage is as low as 1.44%, while it varies from 2.88 to 4.10% for horizons of seven to ninety days. These results indicate that the presented models outperform the existing models in the literature.
Inter-communication between Programmable Logic Controllers using IoT technologies: A Modbus RTU/MQTT Approach
Date: 2021-02-11 00:00:00

Internet of Things (IoT) can be widely used in various applications such as manufacturing industry,achieving high operational efficiency and increased productivity.The exploitation of IoT paradigm made more feasible the use of distributed control systems (DCS) where more than one PLCs implement an industrial application.In case of having more than one PLC,it is obligatory to ensure their inter-communication.In this paper,a solution for PLCs' communication using Message Queuing Telemetry Transport (MQTT) IoT-protocol is presented.An experimental industrial plant was used,controlled by two PLCs.For the process' implementation,two Siemens gateways Simatic IoT2020 were integrated and communicated with the PLCs through Modbus Remote Terminal Unit (RTU) industrial protocol.Finally,data exchange between the two gateways was established using MQTT protocol.
Millimeter Wave MIMO based Depth Maps for Wireless Virtual and Augmented Reality
Authors: Abdelrahman Taha, Qi Qu, Sam Alex, Ping Wang, William L. Abbott, Ahmed Alkhateeb
Date: 2021-02-11 00:00:00

Augmented and virtual reality systems (AR/VR) are rapidly becoming key components of the wireless landscape. For immersive AR/VR experience, these devices should be able to construct accurate depth perception of the surrounding environment. Current AR/VR devices rely heavily on using RGB-D depth cameras to achieve this goal. The performance of these depth cameras, however, has clear limitations in several scenarios, such as the cases with shiny objects, dark surfaces, and abrupt color transition among other limitations. In this paper, we propose a novel solution for AR/VR depth map construction using mmWave MIMO communication transceivers. This is motivated by the deployment of advanced mmWave communication systems in future AR/VR devices for meeting the high data rate demands and by the interesting propagation characteristics of mmWave signals. Accounting for the constraints on these systems, we develop a comprehensive framework for constructing accurate and high-resolution depth maps using mmWave systems. In this framework, we developed new sensing beamforming codebook approaches that are specific for the depth map construction objective. Using these codebooks, and leveraging tools from successive interference cancellation, we develop a joint beam processing approach that can construct high-resolution depth maps using practical mmWave antenna arrays. Extensive simulation results highlight the potential of the proposed solution in building accurate depth maps. Further, these simulations show the promising gains of mmWave based depth perception compared to RGB-based approaches in several important use cases.
Structural identifiability analysis of PDEs: A case study in continuous age-structured epidemic models
Authors: Marissa Renardy, Denise Kirschner, Marisa Eisenberg
Date: 2021-02-11 00:00:00

Computational and mathematical models rely heavily on estimated parameter values for model development. Identifiability analysis determines how well the parameters of a model can be estimated from experimental data. Identifiability analysis is crucial for interpreting and determining confidence in model parameter values and to provide biologically relevant predictions. Structural identifiability analysis, in which one assumes data to be noiseless and arbitrarily fine-grained, has been extensively studied in the context of ordinary differential equation (ODE) models, but has not yet been widely explored for age-structured partial differential equation (PDE) models. These models present additional difficulties due to increased number of variables and partial derivatives as well as the presence of boundary conditions. In this work, we establish a pipeline for structural identifiability analysis of age-structured PDE models using a differential algebra framework and derive identifiability results for specific age-structured models. We use epidemic models to demonstrate this framework because of their wide-spread use in many different diseases and for the corresponding parallel work previously done for ODEs. In our application of the identifiability analysis pipeline, we focus on a Susceptible-Exposed-Infected model for which we compare identifiability results for a PDE and corresponding ODE system and explore effects of age-dependent parameters on identifiability. We also show how practical identifiability analysis can be applied in this example.
Research Methods of Assessing Global Value Chains
Authors: Sourish Dutta
Date: 2021-02-08 00:00:00

The global production (as a system of creating values) is eventually forming a vast web of value chains that explains the transitional structures of global trade and development of the world economy. It is truly a new wave of globalisation, and we can term it as the global value chains (GVCs), creating the nexus among firms, workers and consumers around the globe. The emergence of this new scenario is asking how an economy's businesses, producers and employees are connecting to the global economy and capturing the gains out of it regarding different dimensions of economic development. Indeed, this GVC approach is very crucial for understanding the organisation of the global industries (including firms) through analysing the statics and dynamics of different economic players involved in this complex global production network. Its widespread notion deals with various global issues (including regional value chains also) from the top down to the bottom up, founding a scope for policy analysis.
A Geometric Nonlinear Stochastic Filter for Simultaneous Localization and Mapping
Authors: Hashim A. Hashim
Date: 2021-02-11 00:00:00
Tasks: Position, Simultaneous Localization and Mapping

Simultaneous Localization and Mapping (SLAM) is one of the key robotics tasks as it tackles simultaneous mapping of the unknown environment defined by multiple landmark positions and localization of the unknown pose (i.e., attitude and position) of the robot in three-dimensional (3D) space. The true SLAM problem is modeled on the Lie group of $\mathbb{SLAM}_{n}\left(3\right)$, and its true dynamics rely on angular and translational velocities. This paper proposes a novel geometric nonlinear stochastic estimator algorithm for SLAM on $\mathbb{SLAM}_{n}\left(3\right)$ that precisely mimics the nonlinear motion dynamics of the true SLAM problem. Unlike existing solutions, the proposed stochastic filter takes into account unknown constant bias and noise attached to the velocity measurements. The proposed nonlinear stochastic estimator on manifold is guaranteed to produce good results provided with the measurements of angular velocities, translational velocities, landmarks, and inertial measurement unit (IMU). Simulation and experimental results reflect the ability of the proposed filter to successfully estimate the six-degrees-of-freedom (6 DoF) robot's pose and landmark positions. Keywords: Simultaneous Localization and Mapping, nonlinear stochastic observer for SLAM, stochastic differential equations, pose estimator, position, attitude, Brownian motion process, inertial measurement unit, landmarks, features, SDE, SO(3), SE(3), SLAM.
WAMS-Based Model-Free Wide-Area Damping Control by Voltage Source Converters
Date: 2021-02-10 00:00:00

In this paper, a novel model-free wide-area damping control (WADC) method is proposed, which can achieve full decoupling of modes and damp multiple critical inter-area oscillations simultaneously using grid-connected voltage source converters (VSCs). The proposed method is purely measurement based and requires no knowledge of the network topology and the dynamic model parameters. Hence, the designed controller using VSCs can update the control signals online as the system operating condition varies. Numerical studies in the modified IEEE 68-bus system with grid-connected VSCs show that the proposed method can estimate the system dynamic model accurately and can damp inter-area oscillations effectively under different working conditions and network topologies.