OpenRaiser/Intern-Atlas huggingface dataset:
Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Images
Venue: arXiv cs.LG 2022
Year: 2022

Recent methods for reinforcement learning from images use auxiliary tasks to learn image features that are used by the agent's policy or Q-function. In particular, methods based on contrastive learning that induce linearity of the latent dynamics or invariance to data augmentation have been shown to greatly improve the sample efficiency of the reinforcement learning algorithm and the generalizability of the learned embedding. We further argue, that explicitly improving Markovianity of the learned embedding is desirable and propose a self-supervised representation learning method which integrates contrastive learning with dynamic models to synergistically combine these three objectives: (1) We maximize the InfoNCE bound on the mutual information between the state- and action-embedding and the embedding of the next state to induce a linearly predictive embedding without explicitly learning a linear transition model, (2) we further improve Markovianity of the learned embedding by explicitly learning a non-linear transition model using regression, and (3) we maximize the mutual information between the two nonlinear predictions of the next embeddings based on the current action and two independent augmentations of the current state, which naturally induces transformation invariance not only for the state embedding, but also for the nonlinear transition model. Experimental evaluation on the Deepmind control suite shows that our proposed method achieves higher sample efficiency and better generalization than state-of-art methods based on contrastive learning or reconstruction.
Local Constraint-Based Causal Discovery under Selection Bias
Venue: arXiv cs.LG 2022
Year: 2022

We consider the problem of discovering causal relations from independence constraints selection bias in addition to confounding is present. While the seminal FCI algorithm is sound and complete in this setup, no criterion for the causal interpretation of its output under selection bias is presently known. We focus instead on local patterns of independence relations, where we find no sound method for only three variable that can include background knowledge. Y-Structure patterns are shown to be sound in predicting causal relations from data under selection bias, where cycles may be present. We introduce a finite-sample scoring rule for Y-Structures that is shown to successfully predict causal relations in simulation experiments that include selection mechanisms. On real-world microarray data, we show that a Y-Structure variant performs well across different datasets, potentially circumventing spurious correlations due to selection bias.
Identification in Tree-shaped Linear Structural Causal Models
Venue: arXiv cs.AI 2022
Year: 2022

Linear structural equation models represent direct causal effects as directed edges and confounding factors as bidirected edges. An open problem is to identify the causal parameters from correlations between the nodes. We investigate models, whose directed component forms a tree, and show that there, besides classical instrumental variables, missing cycles of bidirected edges can be used to identify the model. They can yield systems of quadratic equations that we explicitly solve to obtain one or two solutions for the causal parameters of adjacent directed edges. We show how multiple missing cycles can be combined to obtain a unique solution. This results in an algorithm that can identify instances that previously required approaches based on Gröbner bases, which have doubly-exponential time complexity in the number of structural parameters.
Comparison of LSTM autoencoder based deep learning enabled Bayesian inference using two time series reconstruction approaches
Venue: arXiv cs.LG 2022
Year: 2022

In this work, we use a combination of Bayesian inference, Markov chain Monte Carlo and deep learning in the form of LSTM autoencoders to build and test a framework to provide robust estimates of injection rate from ground surface data in coupled flow and geomechanics problems. We use LSTM autoencoders to reconstruct the displacement time series for grid points on the top surface of a faulting due to water injection problem. We then deploy this LSTM autoencoder based model instead of the high fidelity model in the Bayesian inference framework to estimate injection rate from displacement input.
Second-order Symmetric Non-negative Latent Factor Analysis
Venue: arXiv cs.LG 2022
Year: 2022

Precise representation of large-scale undirected network is the basis for understanding relations within a massive entity set. The undirected network representation task can be efficiently addressed by a symmetry non-negative latent factor (SNLF) model, whose objective is clearly non-convex. However, existing SNLF models commonly adopt a first-order optimizer that cannot well handle the non-convex objective, thereby resulting in inaccurate representation results. On the other hand, higher-order learning algorithms are expected to make a breakthrough, but their computation efficiency are greatly limited due to the direct manipulation of the Hessian matrix, which can be huge in undirected network representation tasks. Aiming at addressing this issue, this study proposes to incorporate an efficient second-order method into SNLF, thereby establishing a second-order symmetric non-negative latent factor analysis model for undirected network with two-fold ideas: a) incorporating a mapping strategy into SNLF model to form an unconstrained model, and b) training the unconstrained model with a specially designed second order method to acquire a proper second-order step efficiently. Empirical studies indicate that proposed model outperforms state-of-the-art models in representation accuracy with affordable computational burden.
Portability for GPU-accelerated molecular docking applications for cloud and HPC: can portable compiler directives provide performance across all platforms?
Venue: arXiv cs.DC 2022
Year: 2022

High-throughput structure-based screening of drug-like molecules has become a common tool in biomedical research. Recently, acceleration with graphics processing units (GPUs) has provided a large performance boost for molecular docking programs. Both cloud and high-performance computing (HPC) resources have been used for large screens with molecular docking programs; while NVIDIA GPUs have dominated cloud and HPC resources, new vendors such as AMD and Intel are now entering the field, creating the problem of software portability across different GPUs. Ideally, software productivity could be maximized with portable programming models that are able to maintain high performance across architectures. While in many cases compiler directives have been used as an easy way to offload parallel regions of a CPU-based program to a GPU accelerator, they may also be an attractive programming model for providing portability across different GPU vendors, in which case the porting process may proceed in the reverse direction: from low-level, architecture-specific code to higher-level directive-based abstractions. MiniMDock is a new mini-application (miniapp) designed to capture the essential computational kernels found in molecular docking calculations, such as are used in pharmaceutical drug discovery efforts, in order to test different solutions for porting across GPU architectures. Here we extend MiniMDock to GPU offloading with OpenMP directives, and compare to performance of kernels using CUDA, and HIP on both NVIDIA and AMD GPUs, as well as across different compilers, exploring performance bottlenecks. We document this reverse-porting process, from highly optimized device code to a higher-level version using directives, compare code structure, and describe barriers that were overcome in this effort.
DetFlowTrack: 3D Multi-object Tracking based on Simultaneous Optimization of Object Detection and Scene Flow Estimation
Venue: arXiv cs.CV 2022
Year: 2022

3D Multi-Object Tracking (MOT) is an important part of the unmanned vehicle perception module. Most methods optimize object detection and data association independently. These methods make the network structure complicated and limit the improvement of MOT accuracy. we proposed a 3D MOT framework based on simultaneous optimization of object detection and scene flow estimation. In the framework, a detection-guidance scene flow module is proposed to relieve the problem of incorrect inter-frame assocation. For more accurate scene flow label especially in the case of motion with rotation, a box-transformation-based scene flow ground truth calculation method is proposed. Experimental results on the KITTI MOT dataset show competitive results over the state-of-the-arts and the robustness under extreme motion with rotation.
Optimal Keywords Grouping in Sponsored Search Advertising under Uncertain Environments
Venue: arXiv cs.AI 2022
Year: 2022

In sponsored search advertising, advertisers need to make a series of keyword decisions. Among them, how to group these keywords to form several adgroups within a campaign is a challenging task, due to the highly uncertain environment of search advertising. This paper proposes a stochastic programming model for keywords grouping, taking click-through rate and conversion rate as random variables, with consideration of budget constraints and advertisers' risk-tolerance. A branch-and-bound algorithm is developed to solve our model. Furthermore, we conduct computational experiments to evaluate the effectiveness of our model and solution, with two real-world datasets collected from reports and logs of search advertising campaigns. Experimental results illustrated that our keywords grouping approach outperforms five baselines, and it can approximately approach the optimum in a steady way. This research generates several interesting findings that illuminate critical managerial insights for advertisers in sponsored search advertising. First, keywords grouping does matter for advertisers, especially in the situation with a large number of keywords. Second, in keyword grouping decisions, the marginal profit does not necessarily show the marginal diminishing phenomenon as the budget increases. Such that, it's a worthy try for advertisers to increase their budget in keywords grouping decisions, in order to obtain additional profit. Third, the optimal keywords grouping solution is a result of multifaceted trade-off among various advertising factors. In particular, assigning more keywords into adgroups or having more budget won't certainly lead to higher profits. This suggests a warning for advertisers that it's not wise to take the number of keywords as the criterion for keywords grouping decisions.
Time-to-Label: Temporal Consistency for Self-Supervised Monocular 3D Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Monocular 3D object detection continues to attract attention due to the cost benefits and wider availability of RGB cameras. Despite the recent advances and the ability to acquire data at scale, annotation cost and complexity still limit the size of 3D object detection datasets in the supervised settings. Self-supervised methods, on the other hand, aim at training deep networks relying on pretext tasks or various consistency constraints. Moreover, other 3D perception tasks (such as depth estimation) have shown the benefits of temporal priors as a self-supervision signal. In this work, we argue that the temporal consistency on the level of object poses, provides an important supervision signal given the strong prior on physical motion. Specifically, we propose a self-supervised loss which uses this consistency, in addition to render-and-compare losses, to refine noisy pose predictions and derive high-quality pseudo labels. To assess the effectiveness of the proposed method, we finetune a synthetically trained monocular 3D object detection model using the pseudo-labels that we generated on real data. Evaluation on the standard KITTI3D benchmark demonstrates that our method reaches competitive performance compared to other monocular self-supervised and supervised methods.
OPAL: Occlusion Pattern Aware Loss for Unsupervised Light Field Disparity Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Light field disparity estimation is an essential task in computer vision with various applications. Although supervised learning-based methods have achieved both higher accuracy and efficiency than traditional optimization-based methods, the dependency on ground-truth disparity for training limits the overall generalization performance not to say for real-world scenarios where the ground-truth disparity is hard to capture. In this paper, we argue that unsupervised methods can achieve comparable accuracy, but, more importantly, much higher generalization capacity and efficiency than supervised methods. Specifically, we present the Occlusion Pattern Aware Loss, named OPAL, which successfully extracts and encodes the general occlusion patterns inherent in the light field for loss calculation. OPAL enables: i) accurate and robust estimation by effectively handling occlusions without using any ground-truth information for training and ii) much efficient performance by significantly reducing the network parameters required for accurate inference. Besides, a transformer-based network and a refinement module are proposed for achieving even more accurate results. Extensive experiments demonstrate our method not only significantly improves the accuracy compared with the SOTA unsupervised methods, but also possesses strong generalization capacity, even for real-world data, compared with supervised methods. Our code will be made publicly available.
Undoing Seamlessness: Exploring Seams for Critical Visualization
Venue: arXiv cs.HC 2022
Year: 2022

While seamful design has been part of discourses and work within HCI contexts for some time, it has not yet been fully explored in data visualization design. At the same time, critics of visualization have been arguing that the representation of data as contextual, contingent, relational, partial, heterogeneous, and situated is currently lacking in visualization. Seamful visualization promises a fresh perspective on visualization design as we seek to find more expressive encodings and novel approaches to representing data that acknowledge their wider qualities and limitations. By consulting seams in other realms and exploring existing seams and seamfulness in visualization, this paper offers a foundation for conceptualizing seamful visualization, points towards the value of seams and seamfulness in critical visualization, and proposes principles for engaging with seamful visualization in practice and research.
Detecting GAN-generated Images by Orthogonal Training of Multiple CNNs
Venue: arXiv cs.CV 2022
Year: 2022

In the last few years, we have witnessed the rise of a series of deep learning methods to generate synthetic images that look extremely realistic. These techniques prove useful in the movie industry and for artistic purposes. However, they also prove dangerous if used to spread fake news or to generate fake online accounts. For this reason, detecting if an image is an actual photograph or has been synthetically generated is becoming an urgent necessity. This paper proposes a detector of synthetic images based on an ensemble of Convolutional Neural Networks (CNNs). We consider the problem of detecting images generated with techniques not available at training time. This is a common scenario, given that new image generators are published more and more frequently. To solve this issue, we leverage two main ideas: (i) CNNs should provide orthogonal results to better contribute to the ensemble; (ii) original images are better defined than synthetic ones, thus they should be better trusted at testing time. Experiments show that pursuing these two ideas improves the detector accuracy on NVIDIA's newly generated StyleGAN3 images, never used in training.
Feature Transformation for Cross-domain Few-shot Remote Sensing Scene Classification
Venue: arXiv cs.CV 2022
Year: 2022

Effectively classifying remote sensing scenes is still a challenge due to the increasing spatial resolution of remote imaging and large variances between remote sensing images. Existing research has greatly improved the performance of remote sensing scene classification (RSSC). However, these methods are not applicable to cross-domain few-shot problems where target domain is with very limited training samples available and has a different data distribution from source domain. To improve the model's applicability, we propose the feature-wise transformation module (FTM) in this paper. FTM transfers the feature distribution learned on source domain to that of target domain by a very simple affine operation with negligible additional parameters. Moreover, FTM can be effectively learned on target domain in the case of few training data available and is agnostic to specific network structures. Experiments on RSSC and land-cover mapping tasks verified its capability to handle cross-domain few-shot problems. By comparison with directly finetuning, FTM achieves better performance and possesses better transferability and fine-grained discriminability. \textit{Code will be publicly available.}
Speech watermarking: an approach for the forensic analysis of digital telephonic recordings
Venue: arXiv cs.CR 2022
Year: 2022

In this article, the authors discuss the problem of forensic authentication of digital audio recordings. Although forensic audio has been addressed in several articles, the existing approaches are focused on analog magnetic recordings, which are less prevalent because of the large amount of digital recorders available on the market (optical, solid state, hard disks, etc.). An approach based on digital signal processing that consists of spread spectrum techniques for speech watermarking is presented. This approach presents the advantage that the authentication is based on the signal itself rather than the recording format. Thus, it is valid for usual recording devices in police-controlled telephone intercepts. In addition, our proposal allows for the introduction of relevant information such as the recording date and time and all the relevant data (this is not always possible with classical systems). Our experimental results reveal that the speech watermarking procedure does not interfere in a significant way with the posterior forensic speaker identification.
Postcertificates for Revocation Transparency
Venue: arXiv cs.CR 2022
Year: 2022

The modern Internet is highly dependent on trust communicated via certificates. However, in some cases, certificates become untrusted, and it is necessary to revoke them. In practice, the problem of secure revocation is still open. Furthermore, the existing procedures do not leave a transparent and immutable revocation history. We propose and evaluate a new revocation transparency protocol that introduces postcertificates and utilizes the existing Certificate Transparency (CT) logs. The protocol is practical, has a low deployment cost, provides an immutable history of revocations, enables delegation, and helps to detect revocation-related misbehavior by certificate authorities (CAs). With this protocol, a holder of a postcertificate can bypass the issuing CA and autonomously initiate the revocation process via submission of the postcertificate to a CT log. The CAs are required to monitor CT logs and proceed with the revocation upon detection of a postcertificate. Revocation status delivery is performed independently and with an arbitrary status protocol. Postcertificates can increase the accountability of the CAs and empower the certificate owners by giving them additional control over the status of the certificates. We evaluate the protocol, measure log and monitor performance, and conclude that it is possible to provide revocation transparency using existing CT logs.
A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions
Venue: arXiv cs.CV 2022
Year: 2022

Recent developments in video analysis of sports and computer vision techniques have achieved significant improvements to enable a variety of critical operations. To provide enhanced information, such as detailed complex analysis in sports like soccer, basketball, cricket, badminton, etc., studies have focused mainly on computer vision techniques employed to carry out different tasks. This paper presents a comprehensive review of sports video analysis for various applications high-level analysis such as detection and classification of players, tracking player or ball in sports and predicting the trajectories of player or ball, recognizing the teams strategies, classifying various events in sports. The paper further discusses published works in a variety of application-specific tasks related to sports and the present researchers views regarding them. Since there is a wide research scope in sports for deploying computer vision techniques in various sports, some of the publicly available datasets related to a particular sport have been provided. This work reviews a detailed discussion on some of the artificial intelligence(AI)applications in sports vision, GPU-based work stations, and embedded platforms. Finally, this review identifies the research directions, probable challenges, and future trends in the area of visual recognition in sports.
Semi-parametric Makeup Transfer via Semantic-aware Correspondence
Venue: arXiv cs.CV 2022
Year: 2022

The large discrepancy between the source non-makeup image and the reference makeup image is one of the key challenges in makeup transfer. Conventional approaches for makeup transfer either learn disentangled representation or perform pixel-wise correspondence in a parametric way between two images. We argue that non-parametric techniques have a high potential for addressing the pose, expression, and occlusion discrepancies. To this end, this paper proposes a \textbf{S}emi-\textbf{p}arametric \textbf{M}akeup \textbf{T}ransfer (SpMT) method, which combines the reciprocal strengths of non-parametric and parametric mechanisms. The non-parametric component is a novel \textbf{S}emantic-\textbf{a}ware \textbf{C}orrespondence (SaC) module that explicitly reconstructs content representation with makeup representation under the strong constraint of component semantics. The reconstructed representation is desired to preserve the spatial and identity information of the source image while "wearing" the makeup of the reference image. The output image is synthesized via a parametric decoder that draws on the reconstructed representation. Extensive experiments demonstrate the superiority of our method in terms of visual quality, robustness, and flexibility. Code and pre-trained model are available at \url{https://github.com/AnonymScholar/SpMT.