pwc-archive/papers-with-abstracts huggingface dataset:
Simultaneous Neural Machine Translation with Constituent Label Prediction
Authors: Yasumasa Kano, Katsuhito Sudoh, Satoshi Nakamura
Date: 2021-10-26 00:00:00
Proceeding: WMT (EMNLP) 2021 11
Tasks: Machine Translation, Prediction, Translation

Simultaneous translation is a task in which translation begins before the speaker has finished speaking, so it is important to decide when to start the translation process. However, deciding whether to read more input words or start to translate is difficult for language pairs with different word orders such as English and Japanese. Motivated by the concept of pre-reordering, we propose a couple of simple decision rules using the label of the next constituent predicted by incremental constituent label prediction. In experiments on English-to-Japanese simultaneous translation, the proposed method outperformed baselines in the quality-latency trade-off.
Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretable
Authors: Ruiliu Fu, Han Wang, Xuejun Zhang, Jun Zhou, Yonghong Yan
Date: 2021-10-26 00:00:00
Proceeding: Findings (EMNLP) 2021 11
Tasks: Relation

Multi-hop QA requires the machine to answer complex questions through finding multiple clues and reasoning, and provide explanatory evidence to demonstrate the machine reasoning process. We propose Relation Extractor-Reader and Comparator (RERC), a three-stage framework based on complex question decomposition, which is the first work that the RERC model has been proposed and applied in solving the multi-hop QA challenges. The Relation Extractor decomposes the complex question, and then the Reader answers the sub-questions in turn, and finally the Comparator performs numerical comparison and summarizes all to get the final answer, where the entire process itself constitutes a complete reasoning evidence path. In the 2WikiMultiHopQA dataset, our RERC model has achieved the most advanced performance, with a winning joint F1 score of 53.58 on the leaderboard. All indicators of our RERC are close to human performance, with only 1.95 behind the human level in F1 score of support fact. At the same time, the evidence path provided by our RERC framework has excellent readability and faithfulness.
AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain
Authors: Jimin Hong, Taehee Kim, Hyesu Lim, Jaegul Choo
Date: 2021-10-26 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Language Modeling, Language Modelling, Transfer Learning

During the fine-tuning phase of transfer learning, the pretrained vocabulary remains unchanged, while model parameters are updated. The vocabulary generated based on the pretrained data is suboptimal for downstream data when domain discrepancy exists. We propose to consider the vocabulary as an optimizable parameter, allowing us to update the vocabulary by expanding it with domain-specific vocabulary based on a tokenization statistic. Furthermore, we preserve the embeddings of the added words from overfitting to downstream data by utilizing knowledge learned from a pretrained language model with a regularization term. Our method achieved consistent performance improvements on diverse domains (i.e., biomedical, computer science, news, and reviews).
Task-Specific Dependency-based Word Embedding Methods
Authors: Chengwei Wei, Bin Wang, C. -C. Jay Kuo
Date: 2021-10-26 00:00:00
Tasks: text-classification, Text Classification

Two task-specific dependency-based word embedding methods are proposed for text classification in this work. In contrast with universal word embedding methods that work for generic tasks, we design task-specific word embedding methods to offer better performance in a specific task. Our methods follow the PPMI matrix factorization framework and derive word contexts from the dependency parse tree. The first one, called the dependency-based word embedding (DWE), chooses keywords and neighbor words of a target word in the dependency parse tree as contexts to build the word-context matrix. The second method, named class-enhanced dependency-based word embedding (CEDWE), learns from word-context as well as word-class co-occurrence statistics. DWE and CEDWE are evaluated on popular text classification datasets to demonstrate their effectiveness. It is shown by experimental results they outperform several state-of-the-art word embedding methods.
Exposure of occupations to technologies of the fourth industrial revolution
Authors: Benjamin Meindl, Morgan R. Frank, Joana Mendonça
Date: 2021-10-25 00:00:00

The fourth industrial revolution (4IR) is likely to have a substantial impact on the economy. Companies need to build up capabilities to implement new technologies, and automation may make some occupations obsolete. However, where, when, and how the change will happen remain to be determined. Robust empirical indicators of technological progress linked to occupations can help to illuminate this change. With this aim, we provide such an indicator based on patent data. Using natural language processing, we calculate patent exposure scores for more than 900 occupations, which represent the technological progress related to them. To provide a lens on the impact of the 4IR, we differentiate between traditional and 4IR patent exposure. Our method differs from previous approaches in that it both accounts for the diversity of task-level patent exposures within an occupation and reflects work activities more accurately. We find that exposure to 4IR patents differs from traditional patent exposure. Manual tasks, and accordingly occupations such as construction and production, are exposed mainly to traditional (non-4IR) patents but have low exposure to 4IR patents. The analysis suggests that 4IR technologies may have a negative impact on job growth; this impact appears 10 to 20 years after patent filing. Further, we compared the 4IR exposure to other automation and AI exposure scores. Whereas many measures refer to theoretical automation potential, our patent-based indicator reflects actual technology diffusion. Our work not only allows analyses of the impact of 4IR technologies as a whole, but also provides exposure scores for more than 300 technology fields, such as AI and smart office technologies. Finally, the work provides a general mapping of patents to tasks and occupations, which enables future researchers to construct individual exposure measures.
Improving the Diversity of Unsupervised Paraphrasing with Embedding Outputs
Authors: Monisha Jegadeesan, Sachin Kumar, John Wieting, Yulia Tsvetkov
Date: 2021-10-25 00:00:00
Proceeding: EMNLP (MRL) 2021 11
Tasks: Diversity, Paraphrase Generation, Word Embeddings

We present a novel technique for zero-shot paraphrase generation. The key contribution is an end-to-end multilingual paraphrasing model that is trained using translated parallel corpora to generate paraphrases into "meaning spaces" -- replacing the final softmax layer with word embeddings. This architectural modification, plus a training procedure that incorporates an autoencoding objective, enables effective parameter sharing across languages for more fluent monolingual rewriting, and facilitates fluency and diversity in generation. Our continuous-output paraphrase generation models outperform zero-shot paraphrasing baselines when evaluated on two languages using a battery of computational metrics as well as in human assessment.
Findings from Experiments of On-line Joint Reinforcement Learning of Semantic Parser and Dialogue Manager with real Users
Authors: Matthieu Riou, Bassam Jabaian, Stéphane Huet, Fabrice Lefèvre
Date: 2021-10-25 00:00:00
Tasks: Dialogue Management, Management, Semantic Parsing, speech-recognition, Speech Recognition

Design of dialogue systems has witnessed many advances lately, yet acquiring huge set of data remains an hindrance to their fast development for a new task or language. Besides, training interactive systems with batch data is not satisfactory. On-line learning is pursued in this paper as a convenient way to alleviate these difficulties. After the system modules are initiated, a single process handles data collection, annotation and use in training algorithms. A new challenge is to control the cost of the on-line learning borne by the user. Our work focuses on learning the semantic parsing and dialogue management modules (speech recognition and synthesis offer ready-for-use solutions). In this context we investigate several variants of simultaneous learning which are tested in user trials. In our experiments, with varying merits, they can all achieve good performance with only a few hundreds of training dialogues and overstep a handcrafted system. The analysis of these experiments gives us some insights, discussed in the paper, into the difficulty for the system's trainers to establish a coherent and constant behavioural strategy to enable a fast and good-quality training phase.
Research on the Inverse Kinematics Prediction of a Soft Biomimetic Actuator via BP Neural Network
Authors: Huichen Ma, Junjie Zhou, Jian Zhang, Lingyu Zhang
Date: 2021-10-26 00:00:00
Tasks: Motion Planning, Position

In this work, we address the inverse kinetics problem of motion planning of soft biomimetic actuators driven by three chambers. Soft biomimetic actuators have been applied in many applications owing to their intrinsic softness. Although a mathematical model can be derived to describe the inverse dynamics of this actuator, it is still not accurate to capture the nonlinearity and uncertainty of the material and the system. Besides, such a complex model is time-consuming, so it is not easy to apply in the real-time control unit. Therefore, developing a model-free approach in this area could be a new idea. To overcome these intrinsic problems, we propose a back-propagation (BP) neural network learning the inverse kinetics of the soft biomimetic actuator moving in three-dimensional space. After training with sample data, the BP neural network model can represent the relation between the manipulator tip position and the pressure applied to the chambers. The proposed algorithm is more precise than the analytical model. The results show that a desired terminal position can be achieved with a degree of accuracy of 2.46% relative average error with respect to the total actuator length.
Optimal non-pharmaceutical intervention policy for Covid-19 epidemic via neuroevolution algorithm
Authors: Arash Saeidpour, Pejman Rohani
Date: 2021-10-23 00:00:00

National responses to the Covid-19 pandemic varied markedly across countries, from business-as-usual to complete shutdowns. Policies aimed at disrupting the viral transmission cycle and preventing the healthcare system from being overwhelmed, simultaneously exact an economic toll. We developed a intervention policy model that comprised the relative human, economic and healthcare costs of non-pharmaceutical epidemic intervention and arrived at the optimal strategy using the neuroevolution algorithm. The proposed model finds the minimum required reduction in contact rates to maintain the burden on the healthcare system below the maximum capacity. We find that such a policy renders a sharp increase in the control strength at the early stages of the epidemic, followed by a steady increase in the subsequent ten weeks as the epidemic approaches its peak, and finally control strength is gradually decreased as the population moves towards herd immunity. We have also shown how such a model can provide an efficient adaptive intervention policy at different stages of the epidemic without having access to the entire history of its progression in the population. This work emphasizes the importance of imposing intervention measures early and provides insights into adaptive intervention policies to minimize the economic impacts of the epidemic without putting an extra burden on the healthcare system.
Model-Free Prediction of Chaotic Systems Using High Efficient Next-generation Reservoir Computing
Authors: Zhuo Liu, Leisheng Jin
Date: 2021-10-19 00:00:00

To predict the future evolution of dynamical systems purely from observations of the past data is of great potential application. In this work, a new formulated paradigm of reservoir computing is proposed for achieving model-free predication for both low-dimensional and very large spatiotemporal chaotic systems. Compared with traditional reservoir computing models, it is more efficient in terms of predication length, training data set required and computational expense. By taking the Lorenz and Kuramoto-Sivashinsky equations as two classical examples of dynamical systems, numerical simulations are conducted, and the results show our model excels at predication tasks than the latest reservoir computing methods.
Dendritic Self-Organizing Maps for Continual Learning
Authors: Kosmas Pinitas, Spyridon Chavlis, Panayiota Poirazi
Date: 2021-10-18 00:00:00
Tasks: Continual Learning, Split-CIFAR-10, Split-MNIST

Current deep learning architectures show remarkable performance when trained in large-scale, controlled datasets. However, the predictive ability of these architectures significantly decreases when learning new classes incrementally. This is due to their inclination to forget the knowledge acquired from previously seen data, a phenomenon termed catastrophic-forgetting. On the other hand, Self-Organizing Maps (SOMs) can model the input space utilizing constrained k-means and thus maintain past knowledge. Here, we propose a novel algorithm inspired by biological neurons, termed Dendritic-Self-Organizing Map (DendSOM). DendSOM consists of a single layer of SOMs, which extract patterns from specific regions of the input space accompanied by a set of hit matrices, one per SOM, which estimate the association between units and labels. The best-matching unit of an input pattern is selected using the maximum cosine similarity rule, while the point-wise mutual information is employed for class inference. DendSOM performs unsupervised feature extraction as it does not use labels for targeted updating of the weights. It outperforms classical SOMs and several state-of-the-art continual learning algorithms on benchmark datasets, such as the Split-MNIST and Split-CIFAR-10. We propose that the incorporation of neuronal properties in SOMs may help remedy catastrophic forgetting.
Resolving Anomalies in the Behaviour of a Modularity Inducing Problem Domain with Distributional Fitness Evaluation
Authors: Zhenyue Qin, Tom Gedeon, Bob McKay
Date: 2021-10-15 00:00:00

Discrete gene regulatory networks (GRNs) play a vital role in the study of robustness and modularity. A common method of evaluating the robustness of GRNs is to measure their ability to regulate a set of perturbed gene activation patterns back to their unperturbed forms. Usually, perturbations are obtained by collecting random samples produced by a predefined distribution of gene activation patterns. This sampling method introduces stochasticity, in turn inducing dynamicity. This dynamicity is imposed on top of an already complex fitness landscape. So where sampling is used, it is important to understand which effects arise from the structure of the fitness landscape, and which arise from the dynamicity imposed on it. Stochasticity of the fitness function also causes difficulties in reproducibility and in post-experimental analyses. We develop a deterministic distributional fitness evaluation by considering the complete distribution of gene activity patterns, so as to avoid stochasticity in fitness assessment. This fitness evaluation facilitates repeatability. Its determinism permits us to ascertain theoretical bounds on the fitness, and thus to identify whether the algorithm has reached a global optimum. It enables us to differentiate the effects of the problem domain from those of the noisy fitness evaluation, and thus to resolve two remaining anomalies in the behaviour of the problem domain of~\citet{espinosa2010specialization}. We also reveal some properties of solution GRNs that lead them to be robust and modular, leading to a deeper understanding of the nature of the problem domain. We conclude by discussing potential directions toward simulating and understanding the emergence of modularity in larger, more complex domains, which is key both to generating more useful modular solutions, and to understanding the ubiquity of modularity in biological systems.
Using Traceless Genetic Programming for Solving Multiobjective Optimization Problems
Authors: Mihai Oltean, Crina Grosan
Date: 2021-10-07 00:00:00
Tasks: Multiobjective Optimization

Traceless Genetic Programming (TGP) is a Genetic Programming (GP) variant that is used in cases where the focus is rather the output of the program than the program itself. The main difference between TGP and other GP techniques is that TGP does not explicitly store the evolved computer programs. Two genetic operators are used in conjunction with TGP: crossover and insertion. In this paper, we shall focus on how to apply TGP for solving multi-objective optimization problems which are quite unusual for GP. Each TGP individual stores the output of a computer program (tree) representing a point in the search space. Numerical experiments show that TGP is able to solve very fast and very well the considered test problems.
Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal
Authors: Gianluca Demartini, Kevin Roitero, Stefano Mizzaro
Date: 2021-10-26 00:00:00
Tasks: Fairness, Management

Due to the widespread use of data-powered systems in our everyday lives, the notions of bias and fairness gained significant attention among researchers and practitioners, in both industry and academia. Such issues typically emerge from the data, which comes with varying levels of quality, used to train systems. With the commercialization and employment of such systems that are sometimes delegated to make life-changing decisions, a significant effort is being made towards the identification and removal of possible sources of bias that may surface to the final end-user. In this position paper, we instead argue that bias is not something that should necessarily be removed in all cases, and the attention and effort should shift from bias removal to the identification, measurement, indexing, surfacing, and adjustment of bias, which we name bias management. We argue that if correctly managed, bias can be a resource that can be made transparent to the the users and empower them to make informed choices about their experience with the system.
A Pipeline for Graph-Based Monitoring of the Changes in the Information Space of Russian Social Media during the Lockdown
Authors: V. Danilova, S. Popova, V. Karpova
Date: 2021-10-24 00:00:00
Tasks: Time Series, Time Series Analysis

With the COVID-19 outbreak and the subsequent lockdown, social media became a vital communication tool. The sudden outburst of online activity influenced information spread and consumption patterns. It increases the relevance of studying the dynamics of social networks and developing data processing pipelines that allow a comprehensive analysis of social media data in the temporal dimension. This paper scopes the weekly dynamics of the information space represented by Russian social media (Twitter and Livejournal) during a critical period (massive COVID-19 outbreak and first governmental measures). The approach is twofold: a) build the time series of topic similarity indicators by identifying COVID-related topics in each week and measuring user contribution to the topic space, and b) cluster user activity and display user-topic relationships on graphs in a dashboard application. The paper describes the development of the pipeline, explains the choices made and provides a case study of the adaptation to virus control measures. The results confirm that social processes and behaviour in response to pandemic-triggered changes can be successfully traced in social media. Moreover, the adaptation trends revealed by psychological and sociological studies are reflected in our data and can be explored using the proposed method.
Dynamic Local Geometry Capture in 3D PointCloud Classification
Authors: Shivanand Venkanna Sheshappanavar, Chandra Kambhamettu
Date: 2021-10-19 00:00:00
Proceeding: IEEE 4th International Conference on Multimedia Information Processing and Retrieval (MIPR) 2021 10
Tasks: 3D Classification, 3D Point Cloud Classification, Classification

With the advent of PointNet, the popularity of deep neural networks has increased in point cloud analysis. PointNet's successor, PointNet++, partitions the input point cloud and recursively applies PointNet to capture local geometry. PointNet++ model uses ball querying for local geometry capture in its set abstraction layers. Several models based on single-scale grouping of PointNet++ continue to use ball querying with a fixed-radius ball. Due to its uniform scale in all directions, a ball lacks orientation and is ineffective in capturing complex local neighborhoods. Few recent models replace a fixed-sized ball with a fixed-sized ellipsoid or a fixed-sized cuboid to capture local neighborhoods. However, these methods are not still fully effective in capturing varying geometry proportions from different local neighborhoods on the object surface. We propose a novel technique of dynamically oriented and scaled ellipsoid based on unique local information to capture the local geometry better. We also propose ReducedPointNet++, a single set abstraction-based single scale grouping model. Our model, along with dynamically oriented and scaled ellipsoid querying, achieves 92.1% classification accuracy on the ModelNet40 dataset. We achieve state-of-the-art 3D classification results on all six variants of the real-world ScanObjectNN dataset with an accuracy of 82.0% on the most challenging variant.
Progressive Boundary Refinement Network for Temporal Action Detection
Authors: Qinying Liu, Zilei Wang
Date: 2020-04-03 00:00:00
Proceeding: AAAI 2020 4
Tasks: Action Detection

Temporal action detection is a challenging task due to vagueness
of action boundaries. To tackle this issue, we propose an
end-to-end progressive boundary refinement network (PBRNet)
in this paper. PBRNet belongs to the family of one-stage
detectors and is equipped with three cascaded detection modules
for localizing action boundary more and more precisely.
Specifically, PBRNet mainly consists of coarse pyramidal detection,
refined pyramidal detection, and fine-grained detection.
The first two modules build two feature pyramids to perform
the anchor-based detection, and the third one explores
the frame-level features to refine the boundaries of each action
instance. In the fined-grained detection module, three
frame-level classification branches are proposed to augment
the frame-level features and update the confidence scores of
action instances. Evidently, PBRNet integrates the anchorbased
and frame-level methods. We experimentally evaluate
the proposed PBRNet and comprehensively investigate
the effect of the main components. The results show PBRNet
achieves the state-of-the-art detection performances on
two popular benchmarks: THUMOS’14 and ActivityNet, and
meanwhile possesses a high inference speed.