pwc-archive/papers-with-abstracts huggingface dataset:
SciFive: a text-to-text transformer model for biomedical literature
Authors: Long N. Phan, James T. Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, Grégoire Altan-Bonnet
Date: 2021-05-28 00:00:00
Tasks: Document Classification, Drug–drug Interaction Extraction, Named Entity Recognition (NER), Natural Language Inference, Question Answering, Relation, Relation Extraction, Text Generation

In this report, we introduce SciFive, a domain-specific T5 model that has been pre-trained on large biomedical corpora. Our model outperforms the current SOTA methods (i.e. BERT, BioBERT, Base T5) on tasks in named entity relation, relation extraction, natural language inference, and question-answering. We show that text-generation methods have significant potential in a broad array of biomedical NLP tasks, particularly those requiring longer, more complex outputs. Our results support the exploration of more difficult text generation tasks and the development of new methods in this area
Detection of marine floating plastic using Sentinel-2 imagery and machine learning models
Authors: Srikanta Sannigrahi, Bidroha Basu, Arunima Sarkar Basu, Francesco Pilla
Date: 2021-05-27 00:00:00
Tasks: BIG-bench Machine Learning

The increasing level of marine plastic pollution poses severe threats to the marine ecosystem and biodiversity. The present study attempted to explore the full functionality of open Sentinel satellite data and ML models for detecting and classifying floating plastic debris in Mytilene (Greece), Limassol (Cyprus), Calabria (Italy), and Beirut (Lebanon). Two ML models, i.e. Support Vector Machine (SVM) and Random Forest (RF) were utilized to carry out the classification analysis. In-situ plastic location data was collected from the control experiment conducted in Mytilene, Greece and Limassol, Cyprus, and the same was considered for training the models. Both remote sensing bands and spectral indices were used for developing the ML models. A spectral signature profile for plastic was created for discriminating the floating plastic from other marine debris. A newly developed index, kernel Normalized Difference Vegetation Index (kNDVI), was incorporated into the modelling to examine its contribution to model performances. Both SVM and RF were performed well in five models and test case combinations. Among the two ML models, the highest performance was measured for the RF. The inclusion of kNDVI was found effective and increased the model performances, reflected by high balanced accuracy measured for model 2 (~80% to ~98 % for SVM and ~87% to ~97 % for RF). Using the best-performed model, an automated floating plastic detection system was developed and tested in Calabria and Beirut. For both sites, the trained model had detected the floating plastic with ~99% accuracy. Among the six predictors, the FDI was found the most important variable for detecting marine floating plastic. These findings collectively suggest that high-resolution remote sensing imagery and the automated ML models can be an effective alternative for the cost-effective detection of marine floating plastic.
Distinctive Image Features from Scale-Invariant Keypoints
Authors: David Lowe
Date: 2004-01-05 00:00:00
Tasks: Image Registration, Object, Object Recognition, Patch Matching

This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Beyond Outlier Detection: Outlier Interpretation by Attention-Guided Triplet Deviation Network
Authors: Hongzuo Xu, Yijie Wang, Songlei Jian, Zhenyu Huang, Ning Liu, Yongjun Wang, Fei Li
Date: 2021-04-19 00:00:00
Tasks: Anomaly Detection, Outlier Detection, Outlier Interpretation, Triplet

Outlier detection is an important task in many domains and is intensively studied in the past decade. Further, how to explain outliers, i.e., outlier interpretation, is more significant, which can provide valuable insights for analysts to better understand, solve, and prevent these detected outliers. However, only limited studies consider this problem. Most of the existing methods are based on the score-and-search manner. They select a feature subspace as interpretation per queried outlier by estimating outlying scores of the outlier in searched subspaces. Due to the tremendous searching space, they have to utilize pruning strategies and set a maximum subspace length, resulting in suboptimal interpretation results. Accordingly, this paper proposes a novel Attention-guided Triplet deviation network for Outlier interpretatioN (ATON). Instead of searching a subspace, ATON directly learns an embedding space and learns how to attach attention to each embedding dimension (i.e., capturing the contribution of each dimension to the outlierness of the queried outlier). Specifically, ATON consists of a feature embedding module and a customized self-attention learning module, which are optimized by a triplet deviation-based loss function. We obtain an optimal attention-guided embedding space with expanded high-level information and rich semantics, and thus outlying behaviors of the queried outlier can be better unfolded. ATON finally distills a subspace of original features from the embedding module and the attention coefficient. With the good generality, ATON can be employed as an additional step of any black-box outlier detector. A comprehensive suite of experiments is conducted to evaluate the effectiveness and efficiency of ATON. The proposed ATON significantly outperforms state-of-the-art competitors on 12 real-world datasets and obtains good scalability w.r.t. both data dimensionality and data size. This is the first work to release the ground-truth outlier interpretation annotations of real-world datasets.
A DEEP ADVERSARIAL LEARNING METHODOLOGY FOR DESIGNING MICROSTRUCTURAL MATERIAL SYSTEMS
Authors: Xiaolin Li, Zijiang Yang, L. Catherine Brinson, Alok Choudhary, Ankit Agrawal, Wei Chen
Date: 2018-08-26 00:00:00
Tasks: Bayesian Optimization, Dimensionality Reduction

In Computational Materials Design (CMD), it is well recognized that identifying key microstructure characteristics is crucial for determining material design variables. However, existing
microstructure characterization and reconstruction (MCR) techniques have limitations to be applied for materials design. Some
MCR approaches are not applicable for material microstructural design because no parameters are available to serve as
design variables, while others introduce significant information
loss in either microstructure representation and/or dimensionality reduction. In this work, we present a deep adversarial learning methodology that overcomes the limitations of existing MCR
techniques. In the proposed methodology, generative adversarial
networks (GAN) are trained to learn the mapping between latent
variables and microstructures. Thereafter, the low-dimensional
latent variables serve as design variables, and a Bayesian optimization framework is applied to obtain microstructures with desired material property. Due to the special design of the network
architecture, the proposed methodology is able to identify the latent (design) variables with desired dimensionality, as well as
capturing complex material microstructural characteristics. The
validity of the proposed methodology is tested numerically on a
synthetic microstructure dataset and its effectiveness for materials design is evaluated through a case study of optimizing optical
performance for energy absorption. Additional features, such
as scalability and transferability, are also demonstrated in this
work. In essence, the proposed methodology provides an end-toend solution for microstructural design, in which GAN reduces
information loss and preserves more microstructural characteristics, and the GP-Hedge optimization improves the efficiency of
design exploration
Large-scale nonlinear Granger causality for inferring directed dependence from short multivariate time-series data
Authors: Axel Wismüller, Adora M. DSouza, M. Ali Vosoughi & Anas Abidin
Date: 2021-04-09 00:00:00
Tasks: Causal Discovery, Causal Inference, Community Detection, Learning Network Representations, Time Series, Time Series Analysis

A key challenge to gaining insight into complex systems is inferring nonlinear causal directional relations from observational time-series data. Specifically, estimating causal relationships between interacting components in large systems with only short recordings over few temporal observations remains an important, yet unresolved problem. Here, we introduce large-scale nonlinear Granger causality (lsNGC) which facilitates conditional Granger causality between two multivariate time series conditioned on a large number of confounding time series with a small number of observations. By modeling interactions with nonlinear state-space transformations from limited observational data, lsNGC identifies casual relations with no explicit a priori assumptions on functional interdependence between component time series in a computationally efficient manner. Additionally, our method provides a mathematical formulation revealing statistical significance of inferred causal relations. We extensively study the ability of lsNGC in inferring directed relations from two-node to thirty-four node chaotic time-series systems. Our results suggest that lsNGC captures meaningful interactions from limited observational data, where it performs favorably when compared to traditionally used methods. Finally, we demonstrate the applicability of lsNGC to estimating causality in large, real-world systems by inferring directional nonlinear, causal relationships among a large number of relatively short time series acquired from functional Magnetic Resonance Imaging (fMRI) data of the human brain.
Rethinking Graph Transformers with Spectral Attention
Authors: Devin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau, Prudencio Tossou
Date: 2021-06-07 00:00:00
Proceeding: NeurIPS 2021 12

In recent years, the Transformer architecture has proven to be very successful in sequence processing, but its application to other data structures, such as graphs, has remained limited due to the difficulty of properly defining positions. Here, we present the $\textit{Spectral Attention Network}$ (SAN), which uses a learned positional encoding (LPE) that can take advantage of the full Laplacian spectrum to learn the position of each node in a given graph. This LPE is then added to the node features of the graph and passed to a fully-connected Transformer. By leveraging the full spectrum of the Laplacian, our model is theoretically powerful in distinguishing graphs, and can better detect similar sub-structures from their resonance. Further, by fully connecting the graph, the Transformer does not suffer from over-squashing, an information bottleneck of most GNNs, and enables better modeling of physical phenomenons such as heat transfer and electric interaction. When tested empirically on a set of 4 standard datasets, our model performs on par or better than state-of-the-art GNNs, and outperforms any attention-based model by a wide margin, becoming the first fully-connected architecture to perform well on graph benchmarks.
Noise Conditional Flow Model for Learning the Super-Resolution Space
Authors: Younggeun Kim, Donghee Son
Date: 2021-06-06 00:00:00
Tasks: Diversity, Super-Resolution

Fundamentally, super-resolution is ill-posed problem because a low-resolution image can be obtained from many high-resolution images. Recent studies for super-resolution cannot create diverse super-resolution images. Although SRFlow tried to account for ill-posed nature of the super-resolution by predicting multiple high-resolution images given a low-resolution image, there is room to improve the diversity and visual quality. In this paper, we propose Noise Conditional flow model for Super-Resolution, NCSR, which increases the visual quality and diversity of images through noise conditional layer. To learn more diverse data distribution, we add noise to training data. However, low-quality images are resulted from adding noise. We propose the noise conditional layer to overcome this phenomenon. The noise conditional layer makes our model generate more diverse images with higher visual quality than other works. Furthermore, we show that this layer can overcome data distribution mismatch, a problem that arises in normalizing flow models. With these benefits, NCSR outperforms baseline in diversity and visual quality and achieves better visual quality than traditional GAN-based models. We also get outperformed scores at NTIRE 2021 challenge.
RobustNav: Towards Benchmarking Robustness in Embodied Navigation
Authors: Prithvijit Chattopadhyay, Judy Hoffman, Roozbeh Mottaghi, Aniruddha Kembhavi
Date: 2021-06-08 00:00:00
Proceeding: ICCV 2021 10
Tasks: Benchmarking, Data Augmentation, Visual Navigation

As an attempt towards assessing the robustness of embodied navigation agents, we propose RobustNav, a framework to quantify the performance of embodied navigation agents when exposed to a wide variety of visual - affecting RGB inputs - and dynamics - affecting transition dynamics - corruptions. Most recent efforts in visual navigation have typically focused on generalizing to novel target environments with similar appearance and dynamics characteristics. With RobustNav, we find that some standard embodied navigation agents significantly underperform (or fail) in the presence of visual or dynamics corruptions. We systematically analyze the kind of idiosyncrasies that emerge in the behavior of such agents when operating under corruptions. Finally, for visual corruptions in RobustNav, we show that while standard techniques to improve robustness such as data-augmentation and self-supervised adaptation offer some zero-shot resistance and improvements in navigation performance, there is still a long way to go in terms of recovering lost performance relative to clean "non-corrupt" settings, warranting more research in this direction. Our code is available at https://github.com/allenai/robustnav
Variational AutoEncoder for Reference based Image Super-Resolution
Authors: Zhi-Song Liu, Wan-Chi Siu, Li-Wen Wang
Date: 2021-06-08 00:00:00
Tasks: Image Super-Resolution, SSIM, Super-Resolution

In this paper, we propose a novel reference based image super-resolution approach via Variational AutoEncoder (RefVAE). Existing state-of-the-art methods mainly focus on single image super-resolution which cannot perform well on large upsampling factors, e.g., 8$\times$. We propose a reference based image super-resolution, for which any arbitrary image can act as a reference for super-resolution. Even using random map or low-resolution image itself, the proposed RefVAE can transfer the knowledge from the reference to the super-resolved images. Depending upon different references, the proposed method can generate different versions of super-resolved images from a hidden super-resolution space. Besides using different datasets for some standard evaluations with PSNR and SSIM, we also took part in the NTIRE2021 SR Space challenge and have provided results of the randomness evaluation of our approach. Compared to other state-of-the-art methods, our approach achieves higher diverse scores.
Self-supervised Graph-level Representation Learning with Local and Global Structure
Authors: Minghao Xu, Hang Wang, Bingbing Ni, Hongyu Guo, Jian Tang
Date: 2021-06-08 00:00:00
Tasks: Graph Representation Learning, Representation Learning

This paper studies unsupervised/self-supervised whole-graph representation learning, which is critical in many tasks such as molecule properties prediction in drug and material discovery. Existing methods mainly focus on preserving the local similarity structure between different graph instances but fail to discover the global semantic structure of the entire data set. In this paper, we propose a unified framework called Local-instance and Global-semantic Learning (GraphLoG) for self-supervised whole-graph representation learning. Specifically, besides preserving the local similarities, GraphLoG introduces the hierarchical prototypes to capture the global semantic clusters. An efficient online expectation-maximization (EM) algorithm is further developed for learning the model. We evaluate GraphLoG by pre-training it on massive unlabeled graphs followed by fine-tuning on downstream tasks. Extensive experiments on both chemical and biological benchmark data sets demonstrate the effectiveness of the proposed approach.
Hybrid Method Based on NARX models and Machine Learning for Pattern Recognition
Authors: P. H. O. Silva, A. S. Cerqueira, E. G. Nepomuceno
Date: 2021-06-08 00:00:00
Tasks: BIG-bench Machine Learning

This work presents a novel technique that integrates the methodologies of machine learning and system identification to solve multiclass problems. Such an approach allows to extract and select sets of representative features with reduced dimensionality, as well as predicts categorical outputs. The efficiency of the method was tested by running case studies investigated in machine learning, obtaining better absolute results when compared with classical classification algorithms.
DIPS-Plus: The Enhanced Database of Interacting Protein Structures for Interface Prediction
Authors: Alex Morehead, Chen Chen, Ada Sedova, Jianlin Cheng
Date: 2021-06-06 00:00:00
Tasks: Drug Discovery, Protein Interface Prediction

How and where proteins interface with one another can ultimately impact the proteins' functions along with a range of other biological processes. As such, precise computational methods for protein interface prediction (PIP) come highly sought after as they could yield significant advances in drug discovery and design as well as protein function analysis. However, the traditional benchmark dataset for this task, Docking Benchmark 5 (DB5), contains only a modest 230 complexes for training, validating, and testing different machine learning algorithms. In this work, we expand on a dataset recently introduced for this task, the Database of Interacting Protein Structures (DIPS), to present DIPS-Plus, an enhanced, feature-rich dataset of 42,112 complexes for geometric deep learning of protein interfaces. The previous version of DIPS contains only the Cartesian coordinates and types of the atoms comprising a given protein complex, whereas DIPS-Plus now includes a plethora of new residue-level features including protrusion indices, half-sphere amino acid compositions, and new profile hidden Markov model (HMM)-based sequence features for each amino acid, giving researchers a large, well-curated feature bank for training protein interface prediction methods. We demonstrate through rigorous benchmarks that training an existing state-of-the-art (SOTA) model for PIP on DIPS-Plus yields SOTA results, surpassing the performance of all other models trained on residue-level and atom-level encodings of protein complexes to date.
FEAR: A Simple Lightweight Method to Rank Architectures
Authors: Debadeepta Dey, Shital Shah, Sebastien Bubeck
Date: 2021-06-07 00:00:00
Tasks: Neural Architecture Search

The fundamental problem in Neural Architecture Search (NAS) is to efficiently find high-performing architectures from a given search space. We propose a simple but powerful method which we call FEAR, for ranking architectures in any search space. FEAR leverages the viewpoint that neural networks are powerful non-linear feature extractors. First, we train different architectures in the search space to the same training or validation error. Then, we compare the usefulness of the features extracted by each architecture. We do so with a quick training keeping most of the architecture frozen. This gives fast estimates of the relative performance. We validate FEAR on Natsbench topology search space on three different datasets against competing baselines and show strong ranking correlation especially compared to recently proposed zero-cost methods. FEAR particularly excels at ranking high-performance architectures in the search space. When used in the inner loop of discrete search algorithms like random search, FEAR can cut down the search time by approximately 2.4X without losing accuracy. We additionally empirically study very recently proposed zero-cost measures for ranking and find that they breakdown in ranking performance as training proceeds and also that data-agnostic ranking scores which ignore the dataset do not generalize across dissimilar datasets.
BERT Learns to Teach: Knowledge Distillation with Meta Learning
Authors: Wangchunshu Zhou, Canwen Xu, Julian McAuley
Date: 2021-06-08 00:00:00
Proceeding: ACL 2022 5
Tasks: Knowledge Distillation, Meta-Learning

We present Knowledge Distillation with Meta Learning (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is fixed during training. We show the teacher network can learn to better transfer knowledge to the student network (i.e., learning to teach) with the feedback from the performance of the distilled student network in a meta learning framework. Moreover, we introduce a pilot update mechanism to improve the alignment between the inner-learner and meta-learner in meta learning algorithms that focus on an improved inner-learner. Experiments on various benchmarks show that MetaDistil can yield significant improvements compared with traditional KD algorithms and is less sensitive to the choice of different student capacity and hyperparameters, facilitating the use of KD on different tasks and models.
Simulated Adversarial Testing of Face Recognition Models
Authors: Nataniel Ruiz, Adam Kortylewski, Weichao Qiu, Cihang Xie, Sarah Adel Bargal, Alan Yuille, Stan Sclaroff
Date: 2021-06-08 00:00:00
Proceeding: CVPR 2022 1
Tasks: BIG-bench Machine Learning, Face Recognition

Most machine learning models are validated and tested on fixed datasets. This can give an incomplete picture of the capabilities and weaknesses of the model. Such weaknesses can be revealed at test time in the real world. The risks involved in such failures can be loss of profits, loss of time or even loss of life in certain critical applications. In order to alleviate this issue, simulators can be controlled in a fine-grained manner using interpretable parameters to explore the semantic image manifold. In this work, we propose a framework for learning how to test machine learning algorithms using simulators in an adversarial manner in order to find weaknesses in the model before deploying it in critical scenarios. We apply this method in a face recognition setup. We show that certain weaknesses of models trained on real data can be discovered using simulated samples. Using our proposed method, we can find adversarial synthetic faces that fool contemporary face recognition models. This demonstrates the fact that these models have weaknesses that are not measured by commonly used validation datasets. We hypothesize that this type of adversarial examples are not isolated, but usually lie in connected spaces in the latent space of the simulator. We present a method to find these adversarial regions as opposed to the typical adversarial points found in the adversarial example literature.