pwc-archive/papers-with-abstracts huggingface dataset:
Early completion based on multiple dosages to accelerate maximum tolerated dose-finding
Authors: Masahiro Kojima
Date: 2021-09-22 00:00:00

Background: Phase I trials desire to identify the maximum tolerated dose (MTD) early and proceed quickly to an expansion cohort or phase II trial for efficacy. We propose an early completion method based on multiple dosages to accelerate the identification of MTD for model-assisted designs. Methods: The early completion performs based on each dose-assignment probability for multiple dosages. The formulas of probabilities are simple to calculate. We evaluated the early completion for an actual trial. In addition, a simulation study performed the percentage of correct MTD selection and early completion. Results: In the actual trial, the early completion could perform, and the MTD estimation phase could be shortened. In the simulation study, the change in the percentage of correct MTD selection from non-early completion version was 2.2% at the maximum, indicating that there was almost no decrease. In addition, depending on the toxicity rate, the percentage of correct MTD selection improved compared to the non-early completion version. The early completion performed from 18.1% to 90.8%. Conclusion: We propose the early completion method with not only little reduction in accuracy but also improving the percentage of correct MTD selection. We conclude the early completion can perform unproblematically for model-assisted design.
Uncertainty, volatility and the persistence norms of financial time series
Authors: Simon Rudkin, Wanling Qiu, Pawel Dlotko
Date: 2021-09-30 00:00:00
Tasks: regression, Time Series, Time Series Analysis, Topological Data Analysis

Norms of Persistent Homology introduced in topological data analysis are seen as indicators of system instability, analogous to the changing predictability that is captured in financial market uncertainty indexes. This paper demonstrates norms from the financial markets are significant in explaining financial uncertainty, whilst macroeconomic uncertainty is only explainable by market volatility. Meanwhile, volatility is insignificant in the determination of norms when uncertainty enters the regression. Persistence norms therefore have potential as a further tool in asset pricing, and also as a means of capturing signals from financial time series beyond volatility.
Tree-Constrained Graph Neural Networks For Argument Mining
Authors: Federico Ruggeri, Marco Lippi, Paolo Torroni
Date: 2021-09-02 00:00:00
Tasks: Argument Mining, Sentence, Sentence Classification

We propose a novel architecture for Graph Neural Networks that is inspired by the idea behind Tree Kernels of measuring similarity between trees by taking into account their common substructures, named fragments. By imposing a series of regularization constraints to the learning problem, we exploit a pooling mechanism that incorporates such notion of fragments within the node soft assignment function that produces the embeddings. We present an extensive experimental evaluation on a collection of sentence classification tasks conducted on several argument mining corpora, showing that the proposed approach performs well with respect to state-of-the-art techniques.
Combining Transformers with Natural Language Explanations
Authors: Federico Ruggeri, Marco Lippi, Paolo Torroni
Date: 2021-09-02 00:00:00
Tasks: Argument Mining

Many NLP applications require models to be interpretable. However, many successful neural architectures, including transformers, still lack effective interpretation methods. A possible solution could rely on building explanations from domain knowledge, which is often available as plain, natural language text. We thus propose an extension to transformer models that makes use of external memories to store natural language explanations and use them to explain classification outputs. We conduct an experimental evaluation on two domains, legal text analysis and argument mining, to show that our approach can produce relevant explanations while retaining or even improving classification performance.
A Web Scale Entity Extraction System
Authors: Xuanting Cai, Quanbin Ma, Pan Li, Jianyu Liu, Qi Zeng, Zhengkan Yang, Pushkar Tripathi
Date: 2021-08-27 00:00:00
Proceeding: Findings (EMNLP) 2021 11

Understanding the semantic meaning of content on the web through the lens of entities and concepts has many practical advantages. However, when building large-scale entity extraction systems, practitioners are facing unique challenges involving finding the best ways to leverage the scale and variety of data available on internet platforms. We present learnings from our efforts in building an entity extraction system for multiple document types at large scale using multi-modal Transformers. We empirically demonstrate the effectiveness of multi-lingual, multi-task and cross-document type learning. We also discuss the label collection schemes that help to minimize the amount of noise in the collected data.
HAT4RD: Hierarchical Adversarial Training for Rumor Detection on Social Media
Authors: Shiwen Ni, Jiawen Li, Hung-Yu Kao
Date: 2021-08-29 00:00:00

With the development of social media, social communication has changed. While this facilitates people's communication and access to information, it also provides an ideal platform for spreading rumors. In normal or critical situations, rumors will affect people's judgment and even endanger social security. However, natural language is high-dimensional and sparse, and the same rumor may be expressed in hundreds of ways on social media. As such, the robustness and generalization of the current rumor detection model are put into question. We proposed a novel \textbf{h}ierarchical \textbf{a}dversarial \textbf{t}raining method for \textbf{r}umor \textbf{d}etection (HAT4RD) on social media. Specifically, HAT4RD is based on gradient ascent by adding adversarial perturbations to the embedding layers of post-level and event-level modules to deceive the detector. At the same time, the detector uses stochastic gradient descent to minimize the adversarial risk to learn a more robust model. In this way, the post-level and event-level sample spaces are enhanced, and we have verified the robustness of our model under a variety of adversarial attacks. Moreover, visual experiments indicate that the proposed model drifts into an area with a flat loss landscape, leading to better generalization. We evaluate our proposed method on three public rumors datasets from two commonly used social platforms (Twitter and Weibo). Experiment results demonstrate that our model achieves better results than state-of-the-art methods.
Zero-shot Natural Language Video Localization
Authors: Jinwoo Nam, Daechul Ahn, Dongyeop Kang, Seong Jong Ha, Jonghyun Choi
Date: 2021-08-29 00:00:00
Proceeding: ICCV 2021 10
Tasks: Image Captioning

Understanding videos to localize moments with natural language often requires large expensive annotated video regions paired with language queries. To eliminate the annotation costs, we make a first attempt to train a natural language video localization model in zero-shot manner. Inspired by unsupervised image captioning setup, we merely require random text corpora, unlabeled video collections, and an off-the-shelf object detector to train a model. With the unpaired data, we propose to generate pseudo-supervision of candidate temporal regions and corresponding query sentences, and develop a simple NLVL model to train with the pseudo-supervision. Our empirical validations show that the proposed pseudo-supervised method outperforms several baseline approaches and a number of methods using stronger supervision on Charades-STA and ActivityNet-Captions.
External knowledge transfer deployment inside a simple double agent Viterbi algorithm
Authors: Zied Baklouti
Date: 2021-09-06 00:00:00
Tasks: Transfer Learning

We consider in this paper deploying external knowledge transfer inside a simple double agent Viterbi algorithm which is an algorithm firstly introduced by the author in his preprint "Hidden Markov Based Mathematical Model dedicated to Extract Ingredients from Recipe Text". The key challenge of this work lies in discovering the reason why our old model does have bad performances when it is confronted with estimating ingredient state for unknown words and see if deploying external knowledge transfer directly on calculating state matrix could be the solution instead of deploying it only on back propagating step.
Attention based Sequence to Sequence Learning for Machine Translation of Low Resourced Indic Languages -- A case of Sanskrit to Hindi
Authors: Vishvajit Bakarola, Jitendra Nasriwala
Date: 2021-09-07 00:00:00
Tasks: Machine Translation, NMT, Translation

Deep Learning techniques are powerful in mimicking humans in a particular set of problems. They have achieved a remarkable performance in complex learning tasks. Deep learning inspired Neural Machine Translation (NMT) is a proficient technique that outperforms traditional machine translation. Performing machine-aided translation on Indic languages has always been a challenging task considering their rich and diverse grammar. The neural machine translation has shown quality results compared to the traditional machine translation approaches. The fully automatic machine translation becomes problematic when it comes to low-resourced languages, especially with Sanskrit. This paper presents attention mechanism based neural machine translation by selectively focusing on a particular part of language sentences during translation. The work shows the construction of Sanskrit to Hindi bilingual parallel corpus with nearly 10K samples and having 178,000 tokens. The neural translation model equipped with an attention mechanism has been trained on Sanskrit to Hindi parallel corpus. The approach has shown the significance of attention mechanisms to overcome long-term dependencies, primarily associated with low resources Indic languages. The paper shows the attention plots on testing data to demonstrate the alignment between source and translated words. For the evaluation of the translated sentences, manual score based human evaluation and automatic evaluation metric based techniques have been adopted. The attention mechanism based neural translation has achieved 88% accuracy in human evaluation and a BLEU score of 0.92 on Sanskrit to Hindi translation.
Optimization Networks for Integrated Machine Learning
Authors: Michael Kommenda, Johannes Karder, Andreas Beham, Bogdan Burlacu, Gabriel Kronberger, Stefan Wagner, Michael Affenzeller
Date: 2021-09-01 00:00:00
Tasks: BIG-bench Machine Learning, Combinatorial Optimization, feature selection, Model Selection

Optimization networks are a new methodology for holistically solving interrelated problems that have been developed with combinatorial optimization problems in mind. In this contribution we revisit the core principles of optimization networks and demonstrate their suitability for solving machine learning problems. We use feature selection in combination with linear model creation as a benchmark application and compare the results of optimization networks to ordinary least squares with optional elastic net regularization. Based on this example we justify the advantages of optimization networks by adapting the network to solve other machine learning problems. Finally, optimization analysis is presented, where optimal input values of a system have to be found to achieve desired output values. Optimization analysis can be divided into three subproblems: model creation to describe the system, model selection to choose the most appropriate one and parameter optimization to obtain the input values. Therefore, optimization networks are an obvious choice for handling optimization analysis tasks.
MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation
Authors: Alexandros Karargyris, Renato Umeton, Micah J. Sheller, Alejandro Aristizabal, Johnu George, Srini Bala, Daniel J. Beutel, Victor Bittorf, Akshay Chaudhari, Alexander Chowdhury, Cody Coleman, Bala Desinghu, Gregory Diamos, Debo Dutta, Diane Feddema, Grigori Fursin, Junyi Guo, Xinyuan Huang, David Kanter, Satyananda Kashyap, Nicholas Lane, Indranil Mallick, Pietro Mascagni, Virendra Mehta, Vivek Natarajan, Nikola Nikolov, Nicolas Padoy, Gennady Pekhimenko, Vijay Janapa Reddi, G Anthony Reina, Pablo Ribalta, Jacob Rosenthal, Abhishek Singh, Jayaraman J. Thiagarajan, Anna Wuest, Maria Xenochristou, Daguang Xu, Poonam Yadav, Michael Rosenthal, Massimo Loda, Jason M. Johnson, Peter Mattson
Date: 2021-09-29 00:00:00
Tasks: Benchmarking, Philosophy

Medical AI has tremendous potential to advance healthcare by supporting the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving provider and patient experience. We argue that unlocking this potential requires a systematic way to measure the performance of medical AI models on large-scale heterogeneous data. To meet this need, we are building MedPerf, an open framework for benchmarking machine learning in the medical domain. MedPerf will enable federated evaluation in which models are securely distributed to different facilities for evaluation, thereby empowering healthcare organizations to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status, and our roadmap. We call for researchers and organizations to join us in creating the MedPerf open benchmarking platform.
DESTRESS: Computation-Optimal and Communication-Efficient Decentralized Nonconvex Finite-Sum Optimization
Authors: Boyue Li, Zhize Li, Yuejie Chi
Date: 2021-10-04 00:00:00
Tasks: Federated Learning

Emerging applications in multi-agent environments such as internet-of-things, networked sensing, autonomous systems and federated learning, call for decentralized algorithms for finite-sum optimizations that are resource-efficient in terms of both computation and communication. In this paper, we consider the prototypical setting where the agents work collaboratively to minimize the sum of local loss functions by only communicating with their neighbors over a predetermined network topology. We develop a new algorithm, called DEcentralized STochastic REcurSive gradient methodS (DESTRESS) for nonconvex finite-sum optimization, which matches the optimal incremental first-order oracle (IFO) complexity of centralized algorithms for finding first-order stationary points, while maintaining communication efficiency. Detailed theoretical and numerical comparisons corroborate that the resource efficiencies of DESTRESS improve upon prior decentralized algorithms over a wide range of parameter regimes. DESTRESS leverages several key algorithm design ideas including randomly activated stochastic recursive gradient updates with mini-batches for local computation, gradient tracking with extra mixing (i.e., multiple gossiping rounds) for per-iteration communication, together with careful choices of hyper-parameters and new analysis frameworks to provably achieve a desirable computation-communication trade-off.
Predicting Destinations by a Deep Learning based Approach
Authors: Jiajie Xu, Jing Zhao, Rui Zhou, Chengfei Liu
Date: 2019-08-05 00:00:00
Proceeding: IEEE Transactions on Knowledge and Data Engineering 2019 8
Tasks: Deep Learning

Destination prediction is known as an important problem for many location based services (LBSs). Existing solutions
generally apply probabilistic models or neural network models to predict destinations over a subtrajectory, and adopt the standard attention mechanism to improve the prediction accuracy. However, the standard attention mechanism uses fixed feature representations, and has a limited ability to represent distinct features of locations. Besides, existing methods rarely take the impact of spatial and temporal characteristics of the trajectory into account. Their accuracies in fine-granularity prediction are always not satisfactory due to the data sparsity problem. Thus, in this paper, a carefully designed deep learning model called LATL model is presented. It not only adopts an adaptive attention network to model the distinct features of locations, but also implements time gates and distance gates into the Long Short-Term Memory (LSTM) network to capture the spatial-temporal relation between consecutive locations. Furthermore, to better understand the mobility patterns in different spatial granularities, and explore the fusion of multi-granularity learning capability, a hierarchical model that utilizes tailored combination of different neural networks under multiple spatial granularities is further proposed. Extensive empirical studies verify that the newly proposed models perform effectively and settle the problem nicely.
Low-cost brain computer interface for everyday use
Authors: Ildar Rakhmatulin
Date: 2021-09-29 00:00:00
Proceeding: Experimental Brain Research 2021 9
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG)

With the growth in electroencephalography (EEG) based applications the demand for affordable consumer solutions is increasing. Here we describe a compact, low-cost EEG device suitable for daily use. The data are transferred from the device to a personal server using the TCP-IP protocol, allowing for wireless operation and a decent range of motion for the user. The device is compact, having a circular shape with a radius of only 25 mm, which would allow for comfortable daily use during both daytime and nighttime. Our solution is also very cost effective, approximately $350 for 24 electrodes. The built-in noise suppression capability improves the accuracy of recordings with a peak input noise below 0.35 μV. Here, we provide the results of the tests for the developed device. On our GitHub page, we provide detailed specification of the steps involved in building this EEG device which should be helpful to readers designing similar devices for their needs https://github.com/Ildaron/ironbci.
CENN: Conservative energy method based on neural networks with subdomains for solving variational problems involving heterogeneous and complex geometries
Authors: Yizheng Wang, Jia Sun, Wei Li, Zaiyuan Lu, Yinghua Liu
Date: 2021-09-25 00:00:00

We propose a conservative energy method based on neural networks with subdomains for solving variational problems (CENN), where the admissible function satisfying the essential boundary condition without boundary penalty is constructed by the radial basis function (RBF), particular solution neural network, and general neural network. The loss term is the potential energy, optimized based on the principle of minimum potential energy. The loss term at the interfaces has the lower order derivative compared to the strong form PINN with subdomains. The advantage of the proposed method is higher efficiency, more accurate, and less hyperparameters than the strong form PINN with subdomains. Another advantage of the proposed method is that it can apply to complex geometries based on the special construction of the admissible function. To analyze its performance, the proposed method CENN is used to model representative PDEs, the examples include strong discontinuity, singularity, complex boundary, non-linear, and heterogeneous problems. Furthermore, it outperforms other methods when dealing with heterogeneous problems.
SurvTRACE: Transformers for Survival Analysis with Competing Events
Authors: Zifeng Wang, Jimeng Sun
Date: 2021-10-02 00:00:00
Tasks: Multi-Task Learning, Selection bias, Survival Analysis

In medicine, survival analysis studies the time duration to events of interest such as mortality. One major challenge is how to deal with multiple competing events (e.g., multiple disease diagnoses). In this work, we propose a transformer-based model that does not make the assumption for the underlying survival distribution and is capable of handling competing events, namely SurvTRACE. We account for the implicit \emph{confounders} in the observational setting in multi-events scenarios, which causes selection bias as the predicted survival probability is influenced by irrelevant factors. To sufficiently utilize the survival data to train transformers from scratch, multiple auxiliary tasks are designed for multi-task learning. The model hence learns a strong shared representation from all these tasks and in turn serves for better survival analysis. We further demonstrate how to inspect the covariate relevance and importance through interpretable attention mechanisms of SurvTRACE, which suffices to great potential in enhancing clinical trial design and new treatment development. Experiments on METABRIC, SUPPORT, and SEER data with 470k patients validate the all-around superiority of our method.
Perhaps PTLMs Should Go to School -- A Task to Assess Open Book and Closed Book QA
Authors: Manuel R. Ciosici, Joe Cecil, Alex Hedges, Dong-Ho Lee, Marjorie Freedman, Ralph Weischedel
Date: 2021-10-04 00:00:00
Tasks: Question Answering

Our goal is to deliver a new task and leaderboard to stimulate research on question answering and pre-trained language models (PTLMs) to understand a significant instructional document, e.g., an introductory college textbook or a manual. PTLMs have shown great success in many question-answering tasks, given significant supervised training, but much less so in zero-shot settings. We propose a new task that includes two college-level introductory texts in the social sciences (American Government 2e) and humanities (U.S. History), hundreds of true/false statements based on review questions written by the textbook authors, validation/development tests based on the first eight chapters of the textbooks, blind tests based on the remaining textbook chapters, and baseline results given state-of-the-art PTLMs. Since the questions are balanced, random performance should be ~50%. T5, fine-tuned with BoolQ achieves the same performance, suggesting that the textbook's content is not pre-represented in the PTLM. Taking the exam closed book, but having read the textbook (i.e., adding the textbook to T5's pre-training), yields at best minor improvement (56%), suggesting that the PTLM may not have "understood" the textbook (or perhaps misunderstood the questions). Performance is better (~60%) when the exam is taken open-book (i.e., allowing the machine to automatically retrieve a paragraph and use it to answer the question).