pwc-archive/papers-with-abstracts huggingface dataset:
Development of AI-cloud based high-sensitivity wireless smart sensor for port structure monitoring
Authors: Junsik Shin, Junyoung Park, JongWoong Park
Date: 2022-09-24 00:00:00
Tasks: object-detection, Object Detection, Sensitivity

Regular structural monitoring of port structure is crucial to cope with rapid degeneration owing to its exposure to saline and collisional environment. However, most of the inspections are being done visually by human in irregular-basis. To overcome the complication, lots of research related to vibration-based monitoring system with sensor has been devised. Nonetheless, it was difficult to measure ambient vibration due to port's diminutive amplitude and specify the exact timing of berthing, which is the major excitation source. This study developed a novel cloud-AI based wireless sensor system with high-sensitivity accelerometer M-A352, which has 0.2uG/sqrt(Hz) noise density, 0.003mg of ultra-low noise feature, and 1000Hz of sampling frequency. The sensor is triggered based on either predefined schedule or long rangefinder. After that, the detection of ship is done by AI object detection technique called Faster R-CNN with backbone network of ResNet for the convolution part. Coordinate and size of the detected anchor box is further processed to certify the berthing ship. Collected data are automatically sent to the cloud server through LTE CAT 1 modem within 10Mbps. The system was installed in the actual port field in Korea for few days as a preliminary investigation of proposed system. Additionally, acceleration, slope, and temperature data are analyzed to suggest the possibility of vibration-based port condition assessment.
Deep learning based sferics recognition for AMT data processing in the dead band
Authors: Enhua Jiang, Rujun Chen, Xinming Wu, Jianxin Liu, Debin Zhu, Weiqiang Liu
Date: 2022-09-22 00:00:00
Tasks: Time Series, Time Series Analysis

In the audio magnetotellurics (AMT) sounding data processing, the absence of sferic signals in some time ranges typically results in a lack of energy in the AMT dead band, which may cause unreliable resistivity estimate. We propose a deep convolutional neural network (CNN) to automatically recognize sferic signals from redundantly recorded data in a long time range and use them to compensate for the resistivity estimation. We train the CNN by using field time series data with different signal to noise rations that were acquired from different regions in mainland China. To solve the potential overfitting problem due to the limited number of sferic labels, we propose a training strategy that randomly generates training samples (with random data augmentations) while optimizing the CNN model parameters. We stop the training process and data generation until the training loss converges. In addition, we use a weighted binary cross-entropy loss function to solve the sample imbalance problem to better optimize the network, use multiple reasonable metrics to evaluate network performance, and carry out ablation experiments to optimally choose the model hyperparameters. Extensive field data applications show that our trained CNN can robustly recognize sferic signals from noisy time series for subsequent impedance estimation. The subsequent processing results show that our method can significantly improve S/N and effectively solve the problem of lack of energy in dead band. Compared to the traditional processing method without sferic compensation, our method can generate a smoother and more reasonable apparent resistivity-phase curves and depolarized phase tensor, correct the estimation error of sudden drop of high-frequency apparent resistivity and abnormal behavior of phase reversal, and finally better restore the real shallow subsurface resistivity structure.
Resource competition in Three-gene-motif & Emergence of Feed-forward response: A Spatiotemporal Study
Authors: Priya Chakraborty, Ushasi Roy, Sayantari Ghosh
Date: 2022-09-28 00:00:00

Feed-forward dynamics, which is well-known to have several important implications in nonlinear dynamical systems, frequently occurs in gene expression motifs, and has been well explored experimentally and mathematically. However, dependency of the components of a genetic circuit upon its host, due to the requirement for resources like ribosome, ATP, transcription factors, tRNA, etc., and related effects are of utmost importance, which is commonly ignored in mathematical models. In a resource-limited environment, two apparently unconnected genes can compete for resources for their respective expression and may exhibit indirect regulatory connection; an emergent response thus arises in the system completely because of resource competition. In this work, we have shown how the responses of the feed-forward loop (FFL), a well-studied regulatory genetic motif, can be recreated considering the resource competition in a three-gene pathway. Exploring the genetic system with temporal as well as spatiotemporal stability analysis, interesting transient and steady-state responses have been observed. The genetic motifs explored in this paper show many of the characteristic features of the conventional FFL structure, like response delay and pulse generation. Most interestingly, in a two-dimensional cellular arrangement, characteristic pattern formation under a concentration gradient of input signals have also been observed. This study pinpoints a larger area of research and exploration in synthetic and cellular systems, which will reveal novel controlling ideas and unique behavioral changes in the system for its context dependencies.
Publication Bias in Asset Pricing Research
Authors: Andrew Y. Chen, Tom Zimmermann
Date: 2022-09-27 00:00:00

Researchers are more likely to share notable findings. As a result, published findings tend to overstate the magnitude of real-world phenomena. This bias is a natural concern for asset pricing research, which has found hundreds of return predictors and little consensus on their origins. Empirical evidence on publication bias comes from large scale meta-studies. Meta-studies of cross-sectional return predictability have settled on four stylized facts that demonstrate publication bias is not a dominant factor: (1) almost all findings can be replicated, (2) predictability persists out-of-sample, (3) empirical $t$-statistics are much larger than 2.0, and (4) predictors are weakly correlated. Each of these facts has been demonstrated in at least three meta-studies. Empirical Bayes statistics turn these facts into publication bias corrections. Estimates from three meta-studies find that the average correction (shrinkage) accounts for only 10 to 15 percent of in-sample mean returns and that the risk of inference going in the wrong direction (the false discovery rate) is less than 10%. Meta-studies also find that $t$-statistic hurdles exceed 3.0 in multiple testing algorithms and that returns are 30 to 50 percent weaker in alternative portfolio tests. These facts are easily misinterpreted as evidence of publication bias effects. We clarify these misinterpretations and others, including the conflating of ``mostly false findings'' with ``many insignificant findings,'' ``data snooping'' with ``liquidity effects,'' and ``failed replications'' with ``insignificant ad-hoc trading strategies.'' Meta-studies outside of the cross-sectional literature are rare. The four facts from cross-sectional meta-studies provide a framework for future research. We illustrate with a preliminary re-examination of equity premium predictability.
Knowledge-Aware Bayesian Deep Topic Model
Authors: Dongsheng Wang, Yishi Xu, Miaoge Li, Zhibin Duan, Chaojie Wang, Bo Chen, Mingyuan Zhou
Date: 2022-09-20 00:00:00
Tasks: model, Topic Models

We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Although embedded topic models (ETMs) and its variants have gained promising performance in text analysis, they mainly focus on mining word co-occurrence patterns, ignoring potentially easy-to-obtain prior topic hierarchies that could help enhance topic coherence. While several knowledge-based topic models have recently been proposed, they are either only applicable to shallow hierarchies or sensitive to the quality of the provided prior knowledge. To this end, we develop a novel deep ETM that jointly models the documents and the given prior knowledge by embedding the words and topics into the same space. Guided by the provided knowledge, the proposed model tends to discover topic hierarchies that are organized into interpretable taxonomies. Besides, with a technique for adapting a given graph, our extended version allows the provided prior topic structure to be finetuned to match the target corpus. Extensive experiments show that our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
Real-time cryo-EM data pre-processing with Warp
Authors: Dimitry Tegunov, Patrick Cramer
Date: 2018-06-14 00:00:00
Proceeding: bioRxiv 2018 6
Tasks: Benchmarking, Image Reconstruction

The acquisition of cryo-electron microscopy (cryo-EM) data from biological specimens is currently largely uncoupled from subsequent data evaluation, correction and processing. Therefore, the acquisition strategy is difficult to optimize during data collection, often leading to suboptimal microscope usage and disappointing results. Here we provide Warp, a software for real-time evaluation, correction, and processing of cryo-EM data during their acquisition. Warp evaluates and monitors key parameters for each recorded micrograph or tomographic tilt series in real time. Warp also rapidly corrects micrographs for global and local motion, and estimates the local defocus with the use of novel algorithms. The software further includes a deep learning-based particle picking algorithm that rivals human accuracy to make the pre-processing pipeline truly automated. The output from Warp can be directly fed into established tools for particle classification and 3D image reconstruction. In a benchmarking study we show that Warp automatically processed a published cryo-EM data set for influenza virus hemagglutinin, leading to an improvement of the nominal resolution from 3.9 Å to 3.2 Å. Warp is easy to install, computationally inexpensive, and has an intuitive and streamlined user interface.
Discrete Predictor-Corrector Diffusion Models for Image Synthesis
Authors: Anonymous
Date: 2022-09-29 00:00:00
Proceeding: ICLR Anonymous Submission 2022 9
Tasks: Conditional Image Generation, Image Generation

We introduce Discrete Predictor-Corrector diffusion models (DPC), extending predictor-corrector samplers in Gaussian diffusion models to the discrete case. Predictor-corrector samplers are a class of samplers for diffusion models, which improve on ancestral samplers by correcting the sampling distribution of intermediate diffusion states using MCMC methods. In DPC, the Langevin corrector, which does not have a direct counterpart in discrete space, is replaced with a discrete MCMC transition defined by a learned corrector kernel. The corrector kernel is trained to make the correction steps achieve asymptotic convergence, in distribution, to the correct marginal of the intermediate diffusion states. Equipped with DPC, we revisit recent transformer-based  non-autoregressive generative models through the lens of discrete diffusion, and find that DPC can alleviate the compounding decoding error due to the parallel sampling of visual tokens. Our experiments show that DPC improves upon existing discrete latent space models for class-conditional image generation on ImageNet, and outperforms continuous diffusion models and GANs, according to standard metrics and user preference studies.
Dilated Neighborhood Attention Transformer
Authors: Ali Hassani, Humphrey Shi
Date: 2022-09-29 00:00:00
Tasks: Image Classification, Instance Segmentation, Object Detection, Panoptic Segmentation, Segmentation, Semantic Segmentation

Transformers are quickly becoming one of the most heavily applied deep learning architectures across modalities, domains, and tasks. In vision, on top of ongoing efforts into plain transformers, hierarchical transformers have also gained significant attention, thanks to their performance and easy integration into existing frameworks. These models typically employ localized attention mechanisms, such as the sliding-window Neighborhood Attention (NA) or Swin Transformer's Shifted Window Self Attention. While effective at reducing self attention's quadratic complexity, local attention weakens two of the most desirable properties of self attention: long range inter-dependency modeling, and global receptive field. In this paper, we introduce Dilated Neighborhood Attention (DiNA), a natural, flexible and efficient extension to NA that can capture more global context and expand receptive fields exponentially at no additional cost. NA's local attention and DiNA's sparse global attention complement each other, and therefore we introduce Dilated Neighborhood Attention Transformer (DiNAT), a new hierarchical vision transformer built upon both. DiNAT variants enjoy significant improvements over strong baselines such as NAT, Swin, and ConvNeXt. Our large model is faster and ahead of its Swin counterpart by 1.6% box AP in COCO object detection, 1.4% mask AP in COCO instance segmentation, and 1.4% mIoU in ADE20K semantic segmentation. Paired with new frameworks, our large variant is the new state of the art panoptic segmentation model on COCO (58.5 PQ) and ADE20K (49.4 PQ), and instance segmentation model on Cityscapes (45.1 AP) and ADE20K (35.4 AP) (no extra data). It also matches the state of the art specialized semantic segmentation models on ADE20K (58.1 mIoU), and ranks second on Cityscapes (84.5 mIoU) (no extra data).
A case study of spatiotemporal forecasting techniques for weather forecasting
Authors: Shakir Showkat Sofi, Ivan Oseledets
Date: 2022-09-29 00:00:00
Tasks: Time Series, Time Series Analysis, Weather Forecasting

The majority of real-world processes are spatiotemporal, and the data generated by them exhibits both spatial and temporal evolution. Weather is one of the most essential processes in this domain, and weather forecasting has become a crucial part of our daily routine. Weather data analysis is considered the most complex and challenging task. Although numerical weather prediction models are currently state-of-the-art, they are resource-intensive and time-consuming. Numerous studies have proposed time series-based models as a viable alternative to numerical forecasts. Recent research in the area of time series analysis indicates significant advancements, particularly regarding the use of state-space-based models (white box) and, more recently, the integration of machine learning and deep neural network-based models (black box). The most famous examples of such models are RNNs and transformers. These models have demonstrated remarkable results in the field of time-series analysis and have demonstrated effectiveness in modelling temporal correlations. It is crucial to capture both temporal and spatial correlations for a spatiotemporal process, as the values at nearby locations and time affect the values of a spatiotemporal process at a specific point. This self-contained paper explores various regional data-driven weather forecasting methods, i.e., forecasting over multiple latitude-longitude points (matrix-shaped spatial grid) to capture spatiotemporal correlations. The results showed that spatiotemporal prediction models reduced computational costs while improving accuracy. In particular, the proposed tensor train dynamic mode decomposition-based forecasting model has comparable accuracy to the state-of-the-art models without the need for training. We provide convincing numerical experiments to show that the proposed approach is practical.
Understanding Collapse in Non-Contrastive Siamese Representation Learning
Authors: Alexander C. Li, Alexei A. Efros, Deepak Pathak
Date: 2022-09-29 00:00:00
Tasks: Continual Learning, Contrastive Learning, Representation Learning

Contrastive methods have led a recent surge in the performance of self-supervised representation learning (SSL). Recent methods like BYOL or SimSiam purportedly distill these contrastive methods down to their essence, removing bells and whistles, including the negative examples, that do not contribute to downstream performance. These "non-contrastive" methods work surprisingly well without using negatives even though the global minimum lies at trivial collapse. We empirically analyze these non-contrastive methods and find that SimSiam is extraordinarily sensitive to dataset and model size. In particular, SimSiam representations undergo partial dimensional collapse if the model is too small relative to the dataset size. We propose a metric to measure the degree of this collapse and show that it can be used to forecast the downstream task performance without any fine-tuning or labels. We further analyze architectural design choices and their effect on the downstream performance. Finally, we demonstrate that shifting to a continual learning setting acts as a regularizer and prevents collapse, and a hybrid between continual and multi-epoch training can improve linear probe accuracy by as many as 18 percentage points using ResNet-18 on ImageNet. Our project page is at https://alexanderli.com/noncontrastive-ssl/.
Compositional Semantic Parsing with Large Language Models
Authors: Andrew Drozdov, Nathanael Schärli, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, Denny Zhou
Date: 2022-09-29 00:00:00
Tasks: Semantic Parsing

Humans can reason compositionally when presented with new tasks. Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artificial compositional generalization tasks such as SCAN. In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and refine these prompting techniques to address them. Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse. This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches. Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.
REST: REtrieve & Self-Train for generative action recognition
Authors: Adrian Bulat, Enrique Sanchez, Brais Martinez, Georgios Tzimiropoulos
Date: 2022-09-29 00:00:00
Tasks: Action Recognition, Caption Generation, Contrastive Learning, Retrieval, Temporal Action Localization, Video Recognition, Zero-Shot Action Recognition

This work is on training a generative action/video recognition model whose output is a free-form action-specific caption describing the video (rather than an action class label). A generative approach has practical advantages like producing more fine-grained and human-readable output, and being naturally open-world. To this end, we propose to adapt a pre-trained generative Vision & Language (V&L) Foundation Model for video/action recognition. While recently there have been a few attempts to adapt V&L models trained with contrastive learning (e.g. CLIP) for video/action, to the best of our knowledge, we propose the very first method that sets outs to accomplish this goal for a generative model. We firstly show that direct fine-tuning of a generative model to produce action classes suffers from severe overfitting. To alleviate this, we introduce REST, a training framework consisting of two key components: an unsupervised method for adapting the generative model to action/video by means of pseudo-caption generation and Self-training, i.e. without using any action-specific labels; (b) a Retrieval approach based on CLIP for discovering a diverse set of pseudo-captions for each video to train the model. Importantly, we show that both components are necessary to obtain high accuracy. We evaluate REST on the problem of zero-shot action recognition where we show that our approach is very competitive when compared to contrastive learning-based methods. Code will be made available.
Optimistic MLE -- A Generic Model-based Algorithm for Partially Observable Sequential Decision Making
Authors: Qinghua Liu, Praneeth Netrapalli, Csaba Szepesvári, Chi Jin
Date: 2022-09-29 00:00:00
Tasks: Decision Making, Model-based Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

This paper introduces a simple efficient learning algorithms for general sequential decision making. The algorithm combines Optimism for exploration with Maximum Likelihood Estimation for model estimation, which is thus named OMLE. We prove that OMLE learns the near-optimal policies of an enormously rich class of sequential decision making problems in a polynomial number of samples. This rich class includes not only a majority of known tractable model-based Reinforcement Learning (RL) problems (such as tabular MDPs, factored MDPs, low witness rank problems, tabular weakly-revealing/observable POMDPs and multi-step decodable POMDPs), but also many new challenging RL problems especially in the partially observable setting that were not previously known to be tractable. Notably, the new problems addressed by this paper include (1) observable POMDPs with continuous observation and function approximation, where we achieve the first sample complexity that is completely independent of the size of observation space; (2) well-conditioned low-rank sequential decision making problems (also known as Predictive State Representations (PSRs)), which include and generalize all known tractable POMDP examples under a more intrinsic representation; (3) general sequential decision making problems under SAIL condition, which unifies our existing understandings of model-based RL in both fully observable and partially observable settings. SAIL condition is identified by this paper, which can be viewed as a natural generalization of Bellman/witness rank to address partial observability. This paper also presents a reward-free variant of OMLE algorithm, which learns approximate dynamic models that enable the computation of near-optimal policies for all reward functions simultaneously.