pwc-archive/papers-with-abstracts huggingface dataset:
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
Authors: Dongxu Li, Junnan Li, Hongdong Li, Juan Carlos Niebles, Steven C. H. Hoi
Date: 2021-12-17 00:00:00
Proceeding: CVPR 2022 1
Tasks: cross-modal alignment, Entity Alignment, Retrieval, Video Retrieval, Visual Question Answering (VQA), Zero-Shot Video Retrieval

Video-and-language pre-training has shown promising improvements on various downstream tasks. Most previous methods capture cross-modal interactions with a transformer-based multimodal encoder, not fully addressing the misalignment between unimodal video and text features. Besides, learning fine-grained visual-language alignment usually requires off-the-shelf object detectors to provide object information, which is bottlenecked by the detector's limited vocabulary and expensive computation cost. We propose Align and Prompt: an efficient and effective video-and-language pre-training framework with better cross-modal alignment. First, we introduce a video-text contrastive (VTC) loss to align unimodal video-text features at the instance level, which eases the modeling of cross-modal interactions. Then, we propose a new visually-grounded pre-training task, prompting entity modeling (PEM), which aims to learn fine-grained region-entity alignment. To achieve this, we first introduce an entity prompter module, which is trained with VTC to produce the similarity between a video crop and text prompts instantiated with entity names. The PEM task then asks the model to predict the entity pseudo-labels (i.e~normalized similarity scores) for randomly-selected video crops. The resulting pre-trained model achieves state-of-the-art performance on both text-video retrieval and videoQA, outperforming prior work by a substantial margin. Our code and pre-trained models are available at https://github.com/salesforce/ALPRO.
FedNI: Federated Graph Learning with Network Inpainting for Population-Based Disease Prediction
Authors: Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, Xiaoxiao Li
Date: 2021-12-19 00:00:00
Tasks: Disease Prediction, Federated Learning, Generative Adversarial Network, Graph Learning

Graph Convolutional Neural Networks (GCNs) are widely used for graph analysis. Specifically, in medical applications, GCNs can be used for disease prediction on a population graph, where graph nodes represent individuals and edges represent individual similarities. However, GCNs rely on a vast amount of data, which is challenging to collect for a single medical institution. In addition, a critical challenge that most medical institutions continue to face is addressing disease prediction in isolation with incomplete data information. To address these issues, Federated Learning (FL) allows isolated local institutions to collaboratively train a global model without data sharing. In this work, we propose a framework, FedNI, to leverage network inpainting and inter-institutional data via FL. Specifically, we first federatively train missing node and edge predictor using a graph generative adversarial network (GAN) to complete the missing information of local networks. Then we train a global GCN node classifier across institutions using a federated graph learning platform. The novel design enables us to build more accurate machine learning models by leveraging federated learning and also graph learning approaches. We demonstrate that our federated model outperforms local and baseline FL methods with significant margins on two public neuroimaging datasets.
Discovering State Variables Hidden in Experimental Data
Authors: Boyuan Chen, Kuang Huang, Sunand Raghupathi, Ishaan Chandratreya, Qiang Du, Hod Lipson
Date: 2021-12-20 00:00:00
Tasks: Symbolic Regression

All physical laws are described as relationships between state variables that give a complete and non-redundant description of the relevant system dynamics. However, despite the prevalence of computing power and AI, the process of identifying the hidden state variables themselves has resisted automation. Most data-driven methods for modeling physical phenomena still assume that observed data streams already correspond to relevant state variables. A key challenge is to identify the possible sets of state variables from scratch, given only high-dimensional observational data. Here we propose a new principle for determining how many state variables an observed system is likely to have, and what these variables might be, directly from video streams. We demonstrate the effectiveness of this approach using video recordings of a variety of physical dynamical systems, ranging from elastic double pendulums to fire flames. Without any prior knowledge of the underlying physics, our algorithm discovers the intrinsic dimension of the observed dynamics and identifies candidate sets of state variables. We suggest that this approach could help catalyze the understanding, prediction and control of increasingly complex systems. Project website is at: https://www.cs.columbia.edu/~bchen/neural-state-variables
Distributed switched model-based predictive control for distributed large-scale systems with switched topology
Authors: Morteza Alinia Ahandani, Hamed Kharrati, Farzad Hashemzadeh, Mahdi Baradarannia
Date: 2021-12-19 00:00:00

Distributed switched large-scale systems are composed by dynamically coupled subsystems, in which interactions among subsystems vary over time according to an exogenous input signal named switching signal. In this paper, we present a distributed robust switched model-based predictive control (DSwMPC) to control such systems that are subject to local state and input constraints. The proposed method guarantees stabilizing the origin of the whole closed-loop system and ensures the constraints satisfaction in the presence of an unknown switching signal. In the distributed model-based predictive control (DMPC) used in this work, by considering the interactions among subsystems as an additive disturbance, the effect of the switch is reflected on the dynamic equation, local, and consistency constraint sets of the nominal subsystems. In the DSwMPC, to compensate the effect of switching signal which creates a time-varying network topology, a robust tube-based switched model-based predictive control (RSwMPC) with switched robust control invariant (switchRCI) set as the target set robust to unknown mode switching is used as local controller. The scheme performance is assessed using three typical examples. In the first example, the switching times are unknown in prior, but the next neighborhood sets are assumed to be known in prior. In the second and third cases, both of them are supposed to be unknown in prior. The simulation results show that the input and state constraints are satisfied by the proposed DSwMPC at all times. They also validate that the closed-loop system converges to the origin. Also, a comparison of the DSwMPC with a centralized SwMPC (CSwMPC) and a decentralized SwMPC (DeSwMPC) shows that the DSwMPC outperforms the DecSwMPC and also the shapes of response curves under the CSwMPC are very similar to those obtained by the DSwMPC.
RvS: What is Essential for Offline RL via Supervised Learning?
Authors: Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, Sergey Levine
Date: 2021-12-20 00:00:00
Tasks: Offline RL

Recent work has shown that supervised learning alone, without temporal difference (TD) learning, can be remarkably effective for offline RL. When does this hold true, and which algorithmic components are necessary? Through extensive experiments, we boil supervised learning for offline RL down to its essential elements. In every environment suite we consider, simply maximizing likelihood with a two-layer feedforward MLP is competitive with state-of-the-art results of substantially more complex methods based on TD learning or sequence modeling with Transformers. Carefully choosing model capacity (e.g., via regularization or architecture) and choosing which information to condition on (e.g., goals or rewards) are critical for performance. These insights serve as a field guide for practitioners doing Reinforcement Learning via Supervised Learning (which we coin "RvS learning"). They also probe the limits of existing RvS methods, which are comparatively weak on random data, and suggest a number of open problems.
DeepUME: Learning the Universal Manifold Embedding for Robust Point Cloud Registration
Authors: Natalie Lang, Joseph M. Francos
Date: 2021-12-18 00:00:00
Tasks: Point Cloud Registration

Registration of point clouds related by rigid transformations is one of the fundamental problems in computer vision. However, a solution to the practical scenario of aligning sparsely and differently sampled observations in the presence of noise is still lacking. We approach registration in this scenario with a fusion of the closed-form Universal Mani-fold Embedding (UME) method and a deep neural network. The two are combined into a single unified framework, named DeepUME, trained end-to-end and in an unsupervised manner. To successfully provide a global solution in the presence of large transformations, we employ an SO(3)-invariant coordinate system to learn both a joint-resampling strategy of the point clouds and SO(3)-invariant features. These features are then utilized by the geometric UME method for transformation estimation. The parameters of DeepUME are optimized using a metric designed to overcome an ambiguity problem emerging in the registration of symmetric shapes, when noisy scenarios are considered. We show that our hybrid method outperforms state-of-the-art registration methods in various scenarios, and generalizes well to unseen data sets. Our code is publicly available.
A-ESRGAN: Training Real-World Blind Super-Resolution with Attention U-Net Discriminators
Authors: Zihao Wei, Yidong Huang, Yuang Chen, Chenhao Zheng, Jinnan Gao
Date: 2021-12-19 00:00:00
Tasks: Blind Super-Resolution, Generative Adversarial Network, Image Super-Resolution, Super-Resolution

Blind image super-resolution(SR) is a long-standing task in CV that aims to restore low-resolution images suffering from unknown and complex distortions. Recent work has largely focused on adopting more complicated degradation models to emulate real-world degradations. The resulting models have made breakthroughs in perceptual loss and yield perceptually convincing results. However, the limitation brought by current generative adversarial network structures is still significant: treating pixels equally leads to the ignorance of the image's structural features, and results in performance drawbacks such as twisted lines and background over-sharpening or blurring. In this paper, we present A-ESRGAN, a GAN model for blind SR tasks featuring an attention U-Net based, multi-scale discriminator that can be seamlessly integrated with other generators. To our knowledge, this is the first work to introduce attention U-Net structure as the discriminator of GAN to solve blind SR problems. And the paper also gives an interpretation for the mechanism behind multi-scale attention U-Net that brings performance breakthrough to the model. Through comparison experiments with prior works, our model presents state-of-the-art level performance on the non-reference natural image quality evaluator metric. And our ablation studies have shown that with our discriminator, the RRDB based generator can leverage the structural features of an image in multiple scales, and consequently yields more perceptually realistic high-resolution images compared to prior works.
Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control
Authors: Liang Zhang, Qiang Wu, Jun Shen, Linyuan Lü, Bo Du, Jianqing Wu
Date: 2021-12-19 00:00:00
Tasks: Reinforcement Learning (RL), Traffic Signal Control

Many studies confirmed that a proper traffic state representation is more important than complex algorithms for the classical traffic signal control (TSC) problem. In this paper, we (1) present a novel, flexible and efficient method, namely advanced max pressure (Advanced-MP), taking both running and queuing vehicles into consideration to decide whether to change current signal phase; (2) inventively design the traffic movement representation with the efficient pressure and effective running vehicles from Advanced-MP, namely advanced traffic state (ATS); and (3) develop a reinforcement learning (RL) based algorithm template, called Advanced-XLight, by combining ATS with the latest RL approaches, and generate two RL algorithms, namely "Advanced-MPLight" and "Advanced-CoLight" from Advanced-XLight. Comprehensive experiments on multiple real-world datasets show that: (1) the Advanced-MP outperforms baseline methods, and it is also efficient and reliable for deployment; and (2) Advanced-MPLight and Advanced-CoLight can achieve the state-of-the-art.
Efficient Large Scale Language Modeling with Mixtures of Experts
Authors: Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, Giri Anantharaman, Xian Li, Shuohui Chen, Halil Akin, Mandeep Baines, Louis Martin, Xing Zhou, Punit Singh Koura, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Mona Diab, Zornitsa Kozareva, Ves Stoyanov
Date: 2021-12-20 00:00:00
Tasks: Language Modeling, Language Modelling, Mixture-of-Experts

Mixture of Experts layers (MoEs) enable efficient scaling of language models through conditional computation. This paper presents a detailed empirical study of how autoregressive MoE language models scale in comparison with dense models in a wide range of settings: in- and out-of-domain language modeling, zero- and few-shot priming, and full-shot fine-tuning. With the exception of fine-tuning, we find MoEs to be substantially more compute efficient. At more modest training budgets, MoEs can match the performance of dense models using $\sim$4 times less compute. This gap narrows at scale, but our largest MoE model (1.1T parameters) consistently outperforms a compute-equivalent dense model (6.7B parameters). Overall, this performance gap varies greatly across tasks and domains, suggesting that MoE and dense models generalize differently in ways that are worthy of future study. We make our code and models publicly available for research use.
Skip-connected 3D DenseNet for volumetric infant brain MRI segmentation
Authors: Toan Duc Bui, Jitae Shin, Taesup Moon
Date: 2019-09-01 00:00:00
Proceeding: Biomedical Signal Processing and Control 2019 9
Tasks: Infant Brain Mri Segmentation, MRI segmentation, Segmentation

Automatic 6-month infant brain tissue segmentation of magnetic resonance imaging (MRI) is still less accurate owing to the low intensity contrast among tissues. To tackle the problem, we introduce an accurate segmentation method for volumetric infant brain MRI built upon a densely connected network that achieves state-of-the-art accuracy. Specifically, we carefully design a fully convolutional densely connected network with skip connections such that the information from different levels of dense blocks can be directly combined to achieve highly accurate segmentation results. The proposed network, called 3D-SkipDenseSeg, exploits the advantage of the recently DenseNet for classification task and extends this to segment the 6-month infant brain tissue segmentation of magnetic resonance imaging (MRI). Experimental results demonstrate a competitive performance with regard to both segmentation accuracy and parameter efficiency of the proposed method over the existing methods; namely, the proposed 3D-SkipDenseSeg achieved the best dice similarity coefficient (DSC) of 90.37 ± 1.38% (WM), 92.27 ± 0.81% (GM), and 95.79 ± 0.54% (CSF) among the 21 participating teams in the 6-month infant brain dataset (iSeg-2017) and required only 10–30% of the parameters compared to similar deep learning-based methods.
Artificial Intelligence and Dimensionality Reduction: Tools for approaching future communications
Authors: Alejandro Ramírez-Arroyo, Luz García, Antonio Alex-Amor, Juan F. Valenzuela-Valdés
Date: 2021-12-20 00:00:00
Tasks: Clustering, Dimensionality Reduction

This article presents a novel application of the t-distributed Stochastic Neighbor Embedding (t-SNE) clustering algorithm to the telecommunication field. t-SNE is a dimensionality reduction (DR) algorithm that allows the visualization of large dataset into a 2D plot. We present the applicability of this algorithm in a communication channel dataset formed by several scenarios (anechoic, reverberation, indoor and outdoor), and by using six channel features. Applying this artificial intelligence (AI) technique, we are able to separate different environments into several clusters allowing a clear visualization of the scenarios. Throughout the article, it is proved that t-SNE has the ability to cluster into several subclasses, obtaining internal classifications within the scenarios themselves. t-SNE comparison with different dimensionality reduction techniques (PCA, Isomap) is also provided throughout the paper. Furthermore, post-processing techniques are used to modify communication scenarios, recreating a real communication scenario from measurements acquired in an anechoic chamber. The dimensionality reduction and classification by using t-SNE and Variational AutoEncoders (VAE) show good performance distinguishing between the recreation and the real communication scenario. The combination of these two techniques opens up the possibility for new scenario recreations for future mobile communications. This work shows the potential of AI as a powerful tool for clustering, classification and generation of new 5G propagation scenarios.
Few-shot Learning with Multilingual Language Models
Authors: Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
Date: 2021-12-20 00:00:00
Tasks: Cross-Lingual Transfer, Few-Shot Learning, Hate Speech Detection, Machine Translation, Natural Language Inference, Translation, Zero-Shot Learning

Large-scale generative language models such as GPT-3 are competitive few-shot learners. While these models are known to be able to jointly represent many different languages, their training data is dominated by English, potentially limiting their cross-lingual generalization. In this work, we train multilingual generative language models on a corpus covering a diverse set of languages, and study their few- and zero-shot learning capabilities in a wide range of tasks. Our largest model with 7.5 billion parameters sets new state of the art in few-shot learning in more than 20 representative languages, outperforming GPT-3 of comparable size in multilingual commonsense reasoning (with +7.4% absolute accuracy improvement in 0-shot settings and +9.4% in 4-shot settings) and natural language inference (+5.4% in each of 0-shot and 4-shot settings). On the FLORES-101 machine translation benchmark, our model outperforms GPT-3 on 171 out of 182 directions with 32 training examples, while surpassing the official supervised baseline in 45 directions. We conduct an in-depth analysis of different multilingual prompting approaches, showing in particular that strong few-shot learning performance across languages can be achieved via cross-lingual transfer through both templates and demonstration examples. Finally, we evaluate our models in social value tasks such as hate speech detection in five languages and find it has limitations similar to comparable sized GPT-3 models.
NetKet 3: Machine Learning Toolbox for Many-Body Quantum Systems
Authors: Filippo Vicentini, Damian Hofmann, Attila Szabó, Dian Wu, Christopher Roth, Clemens Giuliani, Gabriel Pescia, Jannes Nys, Vladimir Vargas-Calderon, Nikita Astrakhantsev, Giuseppe Carleo
Date: 2021-12-20 00:00:00
Tasks: BIG-bench Machine Learning, Chunking, GPU, Quantum State Tomography, Variational Monte Carlo

We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural-network quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network ans\"atze in pure Python code using the concise notation of machine-learning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.
Proving Theorems using Incremental Learning and Hindsight Experience Replay
Authors: Eser Aygün, Laurent Orseau, Ankit Anand, Xavier Glorot, Vlad Firoiu, Lei M. Zhang, Doina Precup, Shibl Mourad
Date: 2021-12-20 00:00:00
Tasks: Automated Theorem Proving, Incremental Learning

Traditional automated theorem provers for first-order logic depend on speed-optimized search and many handcrafted heuristics that are designed to work best over a wide range of domains. Machine learning approaches in literature either depend on these traditional provers to bootstrap themselves or fall short on reaching comparable performance. In this paper, we propose a general incremental learning algorithm for training domain specific provers for first-order logic without equality, based only on a basic given-clause algorithm, but using a learned clause-scoring function. Clauses are represented as graphs and presented to transformer networks with spectral features. To address the sparsity and the initial lack of training data as well as the lack of a natural curriculum, we adapt hindsight experience replay to theorem proving, so as to be able to learn even when no proof can be found. We show that provers trained this way can match and sometimes surpass state-of-the-art traditional provers on the TPTP dataset in terms of both quantity and quality of the proofs.
A Deep Learning Based Workflow for Detection of Lung Nodules With Chest Radiograph
Authors: Yang Tai, Yu-Wen Fang, Fang-Yi Su, Jung-Hsien Chiang
Date: 2021-12-19 00:00:00
Tasks: Segmentation, Sensitivity, Specificity

PURPOSE: This study aimed to develop a deep learning-based tool to detect and localize lung nodules with chest radiographs(CXRs). We expected it to enhance the efficiency of interpreting CXRs and reduce the possibilities of delayed diagnosis of lung cancer. MATERIALS AND METHODS: We collected CXRs from NCKUH database and VBD, an open-source medical image dataset, as our training and validation data. A number of CXRs from the Ministry of Health and Welfare(MOHW) database served as our test data. We built a segmentation model to identify lung areas from CXRs, and sliced them into 16 patches. Physicians labeled the CXRs by clicking the patches. These labeled patches were then used to train and fine-tune a deep neural network(DNN) model, classifying the patches as positive or negative. Finally, we test the DNN model with the lung patches of CXRs from MOHW. RESULTS: Our segmentation model identified the lung regions well from the whole CXR. The Intersection over Union(IoU) between the ground truth and the segmentation result was 0.9228. In addition, our DNN model achieved a sensitivity of 0.81, specificity of 0.82, and AUROC of 0.869 in 98 of 125 cases. For the other 27 difficult cases, the sensitivity was 0.54, specificity 0.494, and AUROC 0.682. Overall, we obtained a sensitivity of 0.78, specificity of 0.79, and AUROC 0.837. CONCLUSIONS: Our two-step workflow is comparable to state-of-the-art algorithms in the sensitivity and specificity of localizing lung nodules from CXRs. Notably, our workflow provides an efficient way for specialists to label the data, which is valuable for relevant researches because of the relative rarity of labeled medical image data.