pwc-archive/papers-with-abstracts huggingface dataset:
Modelling Periodic Measurement Data Having a Piecewise Polynomial Trend Using the Method of Variable Projection
Authors: Johannes Handler, Dimitar Ninevski, Paul O'Leary
Date: 2021-09-15 00:00:00
Tasks: Prediction Intervals

This paper presents a new method for modelling periodic signals having an aperiodic trend, using the method of variable projection. It is a major extension to the IEEE-standard 1057 by permitting the background to be time varying; additionally, any number of harmonics of the periodic portion can be modelled. This paper focuses on using B-Splines to implement a piecewise polynomial model for the aperiodic portion of the signal. A thorough algebraic derivation of the method is presented, as well as a comparison to using global polynomial approximation. It is proven that B-Splines work better for modelling a more complicated aperiodic portion when compared to higher order polynomials. Furthermore, the piecewise polynomial model is capable of modelling the local signal variations produced by the interaction of a control system with a process in industrial applications. The method of variable projection reduces the problem to a one-dimensional nonlinear optimization, combined with a linear least-squares computation. An added benefit of using the method of variable projection is the possibility to calculate the covariances of the linear coefficients of the model, enabling the calculation of confidence and prediction intervals. The method is tested on both real measurement data acquired in industrial processes, as well as synthetic data. The method shows promising results for the precise characterization of periodic signals embedded in highly complex aperiodic backgrounds. Finally, snippets of the m-code are provided, together with a toolbox for B-Splines, which permit the implementation of the complete computation.
A novel hotspot of gelsolin instability and aggregation propensity triggers a new mechanism of amyloidosis
Authors: Michela Bollati, Luisa Diomede, Toni Giorgino, Carmina Natale, Elisa Fagnani, Irene Boniardi, Alberto Barbiroli, Rebecca Alemani, Marten Beeg, Marco Gobbi, Ana Fakin, Eloise Mastrangelo, Mario Milani, Gianluca Presciuttini, Edi Gabellieri, Patrizia Cioni, Matteo de Rosa
Date: 2021-09-15 00:00:00

The multidomain protein gelsolin (GSN) is composed of six homologous modules, sequentially named G1 to G6. Single point substitutions in this protein are responsible for AGel amyloidosis, a hereditary disease characterized by progressive corneal lattice dystrophy, cutis laxa, and polyneuropathy. Several different amyloidogenic variants of GSN have been identified over the years, but only the most common D187N/Y mutants, in G2, have been thoroughly characterized, and the underlying functional mechanistic link between mutation, altered protein structure, susceptibility to aberrant furin cleavage and aggregative potential resolved. Little is known about the recently identified mutations A551P, E553K and M517R hosted at the interface between G4 and G5, whose aggregation process likely follows an alternative pathway. We demonstrate that these three substitutions impair temperature and pressure stability of GSN but do not increase its susceptibility to furin cleavage, the first event of the canonical aggregation pathway. The variants are also characterized by a higher tendency to aggregate in the unproteolysed forms and show a higher proteotoxicity in a C. elegans-based assay. Structural studies point to a destabilization of the interface between G4 and G5 due to three different structural determinants: beta-strand breaking, steric hindrance and/or charge repulsion, all implying the impairment of interdomain contacts. All available evidence suggests that the rearrangement of the protein global architecture triggers a furin-independent aggregation of the protein, supporting the existence of a non-canonical pathway of gelsolin amyloidosis pathogenesis.
Mapping Input Noise to Escape Noise in Integrate-and-fire neurons: A Level-Crossing Approach
Authors: Tilo Schwalger
Date: 2021-09-15 00:00:00
Tasks: Gaussian Processes

Noise in spiking neurons is commonly modeled by a noisy input current or by generating output spikes stochastically with a voltage-dependent hazard rate ("escape noise"). While input noise lends itself to modeling biophysical noise processes, the phenomenological escape noise is mathematically more tractable. Using the level-crossing theory for differentiable Gaussian processes, we derive an approximate mapping between colored input noise and escape noise in leaky integrate-and-fire neurons. This mapping requires the first-passage-time (FPT) density of an overdamped Brownian particle driven by colored noise with respect to an arbitrarily moving boundary. Starting from the Wiener-Rice series for the FPT density, we apply the second-order decoupling approximation of Stratonovich to the case of moving boundaries and derive a simplified hazard-rate representation that is local in time and numerically efficient. This simplification requires the calculation of the non-stationary auto-correlation function of the level-crossing process: For exponentially correlated input noise (Ornstein-Uhlenbeck process), we obtain an exact formula for the zero-lag auto-correlation as a function of noise parameters, mean membrane potential and its speed, as well as an exponential approximation of the full auto-correlation function. The theory well predicts the FPT and interspike interval densities as well as the population activities obtained from simulations with time-dependent stimulus or boundary. The agreement with simulations is strongly enhanced compared to a first-order decoupling approximation that neglects correlations between level crossings. The second-order approximation also improves upon a previously proposed theory in the subthreshold regime. Depending on a simplicity-accuracy trade-off, all considered approximations represent useful mappings from colored input noise to escape noise.
Critical weaknesses in shielding strategies for COVID-19
Authors: Cameron A. Smith, Christian A. Yates, Ben Ashby
Date: 2021-09-15 00:00:00

The COVID-19 pandemic, caused by the coronavirus SARS-CoV-2, has led to a wide range of non-pharmaceutical interventions being implemented around the world to curb transmission. However, the economic and social costs of some of these measures, especially lockdowns, has been high. An alternative and widely discussed public health strategy for the COVID-19 pandemic would have been to 'shield' those most vulnerable to COVID-19 (minimising their contacts with others), while allowing infection to spread among lower risk individuals with the aim of reaching herd immunity. Here we retrospectively explore the effectiveness of this strategy using a stochastic SEIR framework, showing that even under the unrealistic assumption of perfect shielding, hospitals would have been rapidly overwhelmed with many avoidable deaths among lower risk individuals. Crucially, even a small (20%) reduction in the effectiveness of shielding would have likely led to a large increase (>150%) in the number of deaths compared to perfect shielding. Our findings demonstrate that shielding the vulnerable while allowing infections to spread among the wider population would not have been a viable public health strategy for COVID-19 and is unlikely to be effective for future pandemics.
DisUnknown: Distilling Unknown Factors for Disentanglement Learning
Authors: Sitao Xiang, Yuming Gu, Pengda Xiang, Menglei Chai, Hao Li, Yajie Zhao, Mingming He
Date: 2021-09-16 00:00:00
Proceeding: ICCV 2021 10
Tasks: Disentanglement

Disentangling data into interpretable and independent factors is critical for controllable generation tasks. With the availability of labeled data, supervision can help enforce the separation of specific factors as expected. However, it is often expensive or even impossible to label every single factor to achieve fully-supervised disentanglement. In this paper, we adopt a general setting where all factors that are hard to label or identify are encapsulated as a single unknown factor. Under this setting, we propose a flexible weakly-supervised multi-factor disentanglement framework DisUnknown, which Distills Unknown factors for enabling multi-conditional generation regarding both labeled and unknown factors. Specifically, a two-stage training approach is adopted to first disentangle the unknown factor with an effective and robust training method, and then train the final generator with the proper disentanglement of all labeled factors utilizing the unknown distillation. To demonstrate the generalization capacity and scalability of our method, we evaluate it on multiple benchmark datasets qualitatively and quantitatively and further apply it to various real-world applications on complicated datasets.
M2RNet: Multi-modal and Multi-scale Refined Network for RGB-D Salient Object Detection
Authors: Xian Fang, Jinchao Zhu, Ruixun Zhang, Xiuli Shao, Hongpeng Wang
Date: 2021-09-16 00:00:00
Tasks: object-detection, Object Detection, RGB-D Salient Object Detection, Salient Object Detection

Salient object detection is a fundamental topic in computer vision. Previous methods based on RGB-D often suffer from the incompatibility of multi-modal feature fusion and the insufficiency of multi-scale feature aggregation. To tackle these two dilemmas, we propose a novel multi-modal and multi-scale refined network (M2RNet). Three essential components are presented in this network. The nested dual attention module (NDAM) explicitly exploits the combined features of RGB and depth flows. The adjacent interactive aggregation module (AIAM) gradually integrates the neighbor features of high, middle and low levels. The joint hybrid optimization loss (JHOL) makes the predictions have a prominent outline. Extensive experiments demonstrate that our method outperforms other state-of-the-art approaches.
Evolutionary Self-Replication as a Mechanism for Producing Artificial Intelligence
Authors: Samuel Schmidgall, Joseph Hays
Date: 2021-09-16 00:00:00
Tasks: Atari Games

Can reproduction alone in the context of survival produce intelligence in our machines? In this work, self-replication is explored as a mechanism for the emergence of intelligent behavior in modern learning environments. By focusing purely on survival, while undergoing natural selection, evolved organisms are shown to produce meaningful, complex, and intelligent behavior, demonstrating creative solutions to challenging problems without any notion of reward or objectives. Atari and robotic learning environments are re-defined in terms of natural selection, and the behavior which emerged in self-replicating organisms during these experiments is described in detail.
Neural Architecture Search in operational context: a remote sensing case-study
Authors: Anthony Cazasnoves, Pierre-Antoine Ganaye, Kévin Sanchis, Tugdual Ceillier
Date: 2021-09-15 00:00:00
Tasks: Autonomous Driving, Image Segmentation, Neural Architecture Search, Segmentation, Semantic Segmentation

Deep learning has become in recent years a cornerstone tool fueling key innovations in the industry, such as autonomous driving. To attain good performances, the neural network architecture used for a given application must be chosen with care. These architectures are often handcrafted and therefore prone to human biases and sub-optimal selection. Neural Architecture Search (NAS) is a framework introduced to mitigate such risks by jointly optimizing the network architectures and its weights. Albeit its novelty, it was applied on complex tasks with significant results - e.g. semantic image segmentation. In this technical paper, we aim to evaluate its ability to tackle a challenging operational task: semantic segmentation of objects of interest in satellite imagery. Designing a NAS framework is not trivial and has strong dependencies to hardware constraints. We therefore motivate our NAS approach selection and provide corresponding implementation details. We also present novel ideas to carry out other such use-case studies.
Cross-Register Projection for Headline Part of Speech Tagging
Authors: Adrian Benton, Hanyang Li, Igor Malioutov
Date: 2021-09-15 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Open Information Extraction, Part-Of-Speech Tagging, POS, POS Tagging, Sentence, Sentence Compression

Part of speech (POS) tagging is a familiar NLP task. State of the art taggers routinely achieve token-level accuracies of over 97% on news body text, evidence that the problem is well understood. However, the register of English news headlines, "headlinese", is very different from the register of long-form text, causing POS tagging models to underperform on headlines. In this work, we automatically annotate news headlines with POS tags by projecting predicted tags from corresponding sentences in news bodies. We train a multi-domain POS tagger on both long-form and headline text and show that joint training on both registers improves over training on just one or naively concatenating training sets. We evaluate on a newly-annotated corpus of over 5,248 English news headlines from the Google sentence compression corpus, and show that our model yields a 23% relative error reduction per token and 19% per headline. In addition, we demonstrate that better headline POS tags can improve the performance of a syntax-based open information extraction system. We make POSH, the POS-tagged Headline corpus, available to encourage research in improved NLP models for news headlines.
Comparing Euclidean and Hyperbolic Embeddings on the WordNet Nouns Hypernymy Graph
Authors: Sameer Bansal, Adrian Benton
Date: 2021-09-15 00:00:00
Proceeding: EMNLP (insights) 2021 11

Nickel and Kiela (2017) present a new method for embedding tree nodes in the Poincare ball, and suggest that these hyperbolic embeddings are far more effective than Euclidean embeddings at embedding nodes in large, hierarchically structured graphs like the WordNet nouns hypernymy tree. This is especially true in low dimensions (Nickel and Kiela, 2017, Table 1). In this work, we seek to reproduce their experiments on embedding and reconstructing the WordNet nouns hypernymy graph. Counter to what they report, we find that Euclidean embeddings are able to represent this tree at least as well as Poincare embeddings, when allowed at least 50 dimensions. We note that this does not diminish the significance of their work given the impressive performance of hyperbolic embeddings in very low-dimensional settings. However, given the wide influence of their work, our aim here is to present an updated and more accurate comparison between the Euclidean and hyperbolic embeddings.
Making Heads and Tails of Models with Marginal Calibration for Sparse Tagsets
Authors: Michael Kranzlein, Nelson F. Liu, Nathan Schneider
Date: 2021-09-15 00:00:00
Proceeding: Findings (EMNLP) 2021 11
Tasks: TAG

For interpreting the behavior of a probabilistic model, it is useful to measure a model's calibration--the extent to which it produces reliable confidence scores. We address the open problem of calibration for tagging models with sparse tagsets, and recommend strategies to measure and reduce calibration error (CE) in such models. We show that several post-hoc recalibration techniques all reduce calibration error across the marginal distribution for two existing sequence taggers. Moreover, we propose tag frequency grouping (TFG) as a way to measure calibration error in different frequency bands. Further, recalibrating each group separately promotes a more equitable reduction of calibration error across the tag frequency spectrum.
Dialogue State Tracking with a Language Model using Schema-Driven Prompting
Authors: Chia-Hsuan Lee, Hao Cheng, Mari Ostendorf
Date: 2021-09-15 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Dialogue State Tracking, Language Modeling, Language Modelling, Multi-domain Dialogue State Tracking

Task-oriented conversational systems often use dialogue state tracking to represent the user's intentions, which involves filling in values of pre-defined slots. Many approaches have been proposed, often using task-specific architectures with special-purpose classifiers. Recently, good results have been obtained using more general architectures based on pretrained language models. Here, we introduce a new variation of the language modeling approach that uses schema-driven prompting to provide task-aware history encoding that is used for both categorical and non-categorical slots. We further improve performance by augmenting the prompting with schema descriptions, a naturally occurring source of in-domain knowledge. Our purely generative system achieves state-of-the-art performance on MultiWOZ 2.2 and achieves competitive performance on two other benchmarks: MultiWOZ 2.1 and M2M. The data and code will be available at https://github.com/chiahsuan156/DST-as-Prompting.
Tied & Reduced RNN-T Decoder
Authors: Rami Botros, Tara N. Sainath, Robert David, Emmanuel Guzman, Wei Li, Yanzhang He
Date: 2021-09-15 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, Prediction

Previous works on the Recurrent Neural Network-Transducer (RNN-T) models have shown that, under some conditions, it is possible to simplify its prediction network with little or no loss in recognition accuracy (arXiv:2003.07705 [eess.AS], [2], arXiv:2012.06749 [cs.CL]). This is done by limiting the context size of previous labels and/or using a simpler architecture for its layers instead of LSTMs. The benefits of such changes include reduction in model size, faster inference and power savings, which are all useful for on-device applications. In this work, we study ways to make the RNN-T decoder (prediction network + joint network) smaller and faster without degradation in recognition performance. Our prediction network performs a simple weighted averaging of the input embeddings, and shares its embedding matrix weights with the joint network's output layer (a.k.a. weight tying, commonly used in language modeling arXiv:1611.01462 [cs.LG]). This simple design, when used in conjunction with additional Edit-based Minimum Bayes Risk (EMBR) training, reduces the RNN-T Decoder from 23M parameters to just 2M, without affecting word-error rate (WER).
Text as Causal Mediators: Research Design for Causal Estimates of Differential Treatment of Social Groups via Language Aspects
Authors: Katherine A. Keith, Douglas Rice, Brendan O'Connor
Date: 2021-09-15 00:00:00
Proceeding: EMNLP (CINLP) 2021 11
Tasks: Decision Making

Using observed language to understand interpersonal interactions is important in high-stakes decision making. We propose a causal research design for observational (non-experimental) data to estimate the natural direct and indirect effects of social group signals (e.g. race or gender) on speakers' responses with separate aspects of language as causal mediators. We illustrate the promises and challenges of this framework via a theoretical case study of the effect of an advocate's gender on interruptions from justices during U.S. Supreme Court oral arguments. We also discuss challenges conceptualizing and operationalizing causal variables such as gender and language that comprise of many components, and we articulate technical open challenges such as temporal dependence between language mediators in conversational settings.
"It doesn't look good for a date": Transforming Critiques into Preferences for Conversational Recommendation Systems
Authors: Victor S. Bursztyn, Jennifer Healey, Nedim Lipka, Eunyee Koh, Doug Downey, Larry Birnbaum
Date: 2021-09-15 00:00:00
Tasks: Common Sense Reasoning, Conversational Recommendation, Language Modeling, Language Modelling, Recommendation Systems

Conversations aimed at determining good recommendations are iterative in nature. People often express their preferences in terms of a critique of the current recommendation (e.g., "It doesn't look good for a date"), requiring some degree of common sense for a preference to be inferred. In this work, we present a method for transforming a user critique into a positive preference (e.g., "I prefer more romantic") in order to retrieve reviews pertaining to potentially better recommendations (e.g., "Perfect for a romantic dinner"). We leverage a large neural language model (LM) in a few-shot setting to perform critique-to-preference transformation, and we test two methods for retrieving recommendations: one that matches embeddings, and another that fine-tunes an LM for the task. We instantiate this approach in the restaurant domain and evaluate it using a new dataset of restaurant critiques. In an ablation study, we show that utilizing critique-to-preference transformation improves recommendations, and that there are at least three general cases that explain this improved performance.
CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning
Authors: Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca J. Passonneau, Rui Zhang
Date: 2021-09-15 00:00:00
Proceeding: ACL 2022 5
Tasks: Contrastive Learning, Few-shot NER, Named Entity Recognition, Named Entity Recognition (NER)

Named Entity Recognition (NER) in Few-Shot setting is imperative for entity tagging in low resource domains. Existing approaches only learn class-specific semantic features and intermediate representations from source domains. This affects generalizability to unseen target domains, resulting in suboptimal performances. To this end, we present CONTaiNER, a novel contrastive learning technique that optimizes the inter-token distribution distance for Few-Shot NER. Instead of optimizing class-specific attributes, CONTaiNER optimizes a generalized objective of differentiating between token categories based on their Gaussian-distributed embeddings. This effectively alleviates overfitting issues originating from training domains. Our experiments in several traditional test domains (OntoNotes, CoNLL'03, WNUT '17, GUM) and a new large scale Few-Shot NER dataset (Few-NERD) demonstrate that on average, CONTaiNER outperforms previous methods by 3%-13% absolute F1 points while showing consistent performance trends, even in challenging scenarios where previous approaches could not achieve appreciable performance.
On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation
Authors: Dan Iter, David Grangier
Date: 2021-09-15 00:00:00
Tasks: Domain Adaptation, Domain Generalization, Language Modeling, Language Modelling, Machine Translation, Translation

Domain adaptation of neural networks commonly relies on three training phases: pretraining, selected data training and then fine tuning. Data selection improves target domain generalization by training further on pretraining data identified by relying on a small sample of target domain data. This work examines the benefit of data selection for language modeling and machine translation. Our experiments assess the complementarity of selection with fine tuning and result in practical recommendations: (i) selected data must be similar to the fine-tuning domain but not so much as to erode the complementary effect of fine-tuning; (ii) there is a trade-off between selecting little data for fast but limited progress or much data for slow but long lasting progress; (iii) data selection can be applied early during pretraining, with performance gains comparable to long pretraining session; (iv) data selection from domain classifiers is often more effective than the popular contrastive data selection method.