Reviewerly/exHarmony huggingface dataset:
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

When it comes to the formation of real-looking images using some complex models, Generative Adversarial Networks do not disappoint. The complex models involved are often the types with infeasible maximum likelihoods. Be that as it may, there is not yet any proof for the convergence of GANs training. This paper proposes a TTUR (a two-time scale update rule) for training the Generative Adversarial Networks with a descent of stochastic gradient based on haphazard loss functions. The two time-scale update rule has separate learning rates for the generator and the discriminator. With the aid of the stochastic approximation theory, this paper demonstrates that the TTUR reaches a point of convergence under the influence of mild assumption to a kind of remote and stationary state known as Nash equilibrium. This unification or meeting point principle also applies to the widespread Adam optimization. This is a form or replacement optimization algorithm designed into stochastic gradient descent and used for tutoring the deep learning models in the system. For the Adam optimization theory, this paper evinces that it is in line with the dynamics of a weighty ball in a frictional state. Thus, we prove that it favours flat minina in the objective perspective of things. To carry out an evaluation of how GANs perform during the image creation process, this paper presents what we have termed the 'Fréchet Inception Distance", also known as FID—a concept known to dwell on the resemblance between the images created and the real ones in a way that is more improved compared to the Inception Score. Experimentally, the TTUR helps in the bettering of DCGANs and Improved Wasserstein GANs (WGAN-GP). This makes it perform better than the traditional CelebA GAN training, LSUN Bedrooms, CIFAR-10, SVHN and the One Billion Word Benchmark.
A survey of Bayesian Network structure learning

Abstract Bayesian Networks (BNs) have become increasingly popular over the last few decades as a tool for reasoning under uncertainty in fields as diverse as medicine, biology, epidemiology, economics and the social sciences. This is especially true in real-world areas where we seek to answer complex questions based on hypothetical evidence to determine actions for intervention. However, determining the graphical structure of a BN remains a major challenge, especially when modelling a problem under causal assumptions. Solutions to this problem include the automated discovery of BN graphs from data, constructing them based on expert knowledge, or a combination of the two. This paper provides a comprehensive review of combinatoric algorithms proposed for learning BN structure from data, describing 74 algorithms including prototypical, well-established and state-of-the-art approaches. The basic approach of each algorithm is described in consistent terms, and the similarities and differences between them highlighted. Methods of evaluating algorithms and their comparative performance are discussed including the consistency of claims made in the literature. Approaches for dealing with data noise in real-world datasets and incorporating expert knowledge into the learning process are also covered.
Learning Modal-Invariant Angular Metric by Cyclic Projection Network for VIS-NIR Person Re-Identification

Person re-identification across visible and near-infrared cameras (VIS-NIR Re-ID) has widespread applications. The challenge of this task lies in heterogeneous image matching. Existing methods attempt to learn discriminative features via complex feature extraction strategies. Nevertheless, the distributions of visible and near-infrared features are disparate caused by modal gap, which significantly affects feature metric and makes the performance of the existing models poor. To address this problem, we propose a novel approach from the perspective of metric learning. We conduct metric learning on a well-designed angular space. Geometrically, features are mapped from the original space to the hypersphere manifold, which eliminates the variations of feature norm and concentrates on the angle between the feature and the target category. Specifically, we propose a cyclic projection network (CPN) that transforms features into an angle-related space while identity information is preserved. Furthermore, we proposed three kinds of loss functions, AICAL, LAL and DAL, in angular space for angular metric learning. Multiple experiments on two existing public datasets, SYSU-MM01 and RegDB, show that performance of our method greatly exceeds the SOTA performance.
Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review

Since whole-slide imaging has been commercially available for over 2 decades, digital pathology has become a constantly expanding aspect of the pathology profession that will continue to significantly impact how pathologists conduct their craft. While some aspects, such as whole-slide imaging for archiving, consulting, and teaching, have gained broader acceptance, other facets such as quantitative tissue image analysis and artificial intelligence-based assessments are still met with some reservations. While most vendors in this space have focused on diagnostic applications, that is, viewing one or few slides at a time, some are developing solutions tailored more specifically to the various aspects of veterinary pathology including updated diagnostic, discovery, and research applications. This has especially advanced the use of digital pathology in toxicologic pathology and drug development, for primary reads as well as peer reviews. It is crucial that pathologists gain a deeper understanding of digital pathology and tissue image analysis technology and their applications in order to fully use these tools in a way that enhances and improves the pathologist's assessment as well as work environment. This review focuses on an updated introduction to the basics of digital pathology and image analysis and introduces emerging topics around artificial intelligence and machine learning.
Neural Image Compression via Attentional Multi-scale Back Projection and Frequency Decomposition

In recent years, neural image compression emerges as a rapidly developing topic in computer vision, where the state-of-the-art approaches now exhibit superior compression performance than their conventional counterparts. Despite the great progress, current methods still have limitations in preserving fine spatial details for optimal reconstruction, especially at low compression rates. We make three contributions in tackling this issue. First, we develop a novel back projection method with attentional and multi-scale feature fusion for augmented representation power. Our back projection method recalibrates the current estimation by establishing feedback connections between high-level and low-level attributes in an attentional and discriminative manner. Second, we propose to decompose the input image and separately process the distinct frequency components, whose derived latents are recombined using a novel dual attention module, so that details inside regions of interest could be explicitly manipulated. Third, we propose a novel training scheme for reducing the latent rounding residual. Experimental results show that, when measured in PSNR, our model reduces BD-rate by 9.88% and 10.32% over the state-of-the-art method, and 4.12% and 4.32% over the latest coding standard Versatile Video Coding (VVC) on the Kodak and CLIC2020 Professional Validation dataset, respectively. Our approach also produces more visually pleasant images when optimized for MS-SSIM. The significant improvement upon existing methods shows the effectiveness of our method in preserving and remedying spatial information for enhanced compression quality.
VSR: A Unified Framework for Document Layout Analysis Combining Vision, Semantics and Relations

Document layout analysis is crucial for understanding document structures. On this task, vision and semantics of documents, and relations between layout components contribute to the understanding process. Though many works have been proposed to exploit the above information, they show unsatisfactory results. NLP-based methods model layout analysis as a sequence labeling task and show insufficient capabilities in layout modeling. CV-based methods model layout analysis as a detection or segmentation task, but bear limitations of inefficient modality fusion and lack of relation modeling between layout components. To address the above limitations, we propose a unified framework VSR for document layout analysis, combining vision, semantics and relations. VSR supports both NLP-based and CV-based methods. Specifically, we first introduce vision through document image and semantics through text embedding maps. Then, modality-specific visual and semantic features are extracted using a two-stream network, which are adaptively fused to make full use of complementary information. Finally, given component candidates, a relation module based on graph neural network is incorported to model relations between components and output final results. On three popular benchmarks, VSR outperforms previous models by large margins. Code will be released soon.
Attribute Sentiment Scoring with Online Text Reviews: Accounting for Language Structure and Missing Attributes

The authors address two significant challenges in using online text reviews to obtain fine-grained, attribute-level sentiment ratings. First, in contrast to methods that rely on word frequency, they develop a deep learning convolutional–long short-term memory hybrid model to account for language structure. The convolutional layer accounts for spatial structure (adjacent word groups or phrases), and long short-term memory accounts for the sequential structure of language (sentiment distributed and modified across nonadjacent phrases). Second, they address the problem of missing attributes in text when constructing attribute sentiment scores, as reviewers write about only a subset of attributes and remain silent on others. They develop a model-based imputation strategy using a structural model of heterogeneous rating behavior. Using Yelp restaurant review data, they show superior attribute sentiment scoring accuracy with their model. They identify three reviewer segments with different motivations: status seeking, altruism/want voice, and need to vent/praise. Surprisingly, attribute mentions in reviews are driven by the need to inform and vent/praise rather than by attribute importance. The heterogeneous model-based imputation performs better than other common imputations and, importantly, leads to managerially significant corrections in restaurant attribute ratings. More broadly, the results suggest that social science research should pay more attention to reducing measurement error in variables constructed from text.
Variational Feature Disentangling for Fine-Grained Few-Shot Classification

Data augmentation is an intuitive step towards solving the problem of few-shot classification. However, ensuring both discriminability and diversity in the augmented samples is challenging. To address this, we propose a feature disentanglement framework that allows us to augment features with randomly sampled intra-class variations while preserving their class-discriminative features. Specifically, we disentangle a feature representation into two components: one represents the intra-class variance and the other encodes the class-discriminative information. We assume that the intra-class variance induced by variations in poses, backgrounds, or illumination conditions is shared across all classes and can be modelled via a common distribution. Then we sample features repeatedly from the learned intra-class variability distribution and add them to the class-discriminative features to get the augmented features. Such a data augmentation scheme ensures that the augmented features inherit crucial class-discriminative features while exhibiting large intra-class variance. Our method significantly outperforms the state-of-the-art methods on multiple challenging fine-grained few-shot image classification benchmarks. Code is available at: https://github.com/cvlab-stonybrook/vfd-iccv21
ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignment

The recent progress of CNN has dramatically improved face alignment performance. However, few works have paid attention to the error-bias with respect to error distribution of facial landmarks. In this paper, we investigate the error-bias issue in face alignment, where the distributions of landmark errors tend to spread along the tangent line to landmark curves. This error-bias is not trivial since it is closely connected to the ambiguous landmark labeling task. Inspired by this observation, we seek a way to leverage the error-bias property for better convergence of CNN model. To this end, we propose anisotropic direction loss (ADL) and anisotropic attention module (AAM) for coordinate and heatmap regression, respectively. ADL imposes strong binding force in normal direction for each landmark point on facial boundaries. On the other hand, AAM is an attention module which can get anisotropic attention mask focusing on the region of point and its local edge connected by adjacent points, it has a stronger response in tangent than in normal, which means relaxed constraints in the tangent. These two methods work in a complementary manner to learn both facial structures and texture details. Finally, we integrate them into an optimized end-to-end training pipeline named ADNet. Our ADNet achieves state-of-the-art results on 300W, WFLW and COFW datasets, which demonstrates the effectiveness and robustness.
Towards the next generation of the LinkedGeoData project using virtual knowledge graphs

With the advancement of Semantic Technologies, large geospatial data sources have been increasingly published as Linked data on the Web. The LinkedGeoData project is one of the most prominent such projects to create a large knowledge graph from OpenStreetMap (OSM) with global coverage and interlinking of other data sources. In this paper, we report on the ongoing effort of exposing the relational database in LinkedGeoData as a SPARQL endpoint using Virtual Knowledge Graph (VKG) technology. Specifically, we present two realizations of VKGs, using the two systems Sparqlify and Ontop. In order to improve compliance with the OGC GeoSPARQL standard, we have implemented GeoSPARQL support in Ontop v4. Moreover, we have evaluated the VKG-powered LinkedGeoData in the test areas of Italy and Germany. Our experiments demonstrate that such system supports complex GeoSPARQL queries, which confirms that query answering in the VKG approach is efficient.
Prototypical Matching and Open Set Rejection for Zero-Shot Semantic Segmentation

The DCNN methods in addressing semantic segmentation demand vast amount of pixel-wise annotated training samples. In this work, we present zero-shot semantic segmentation, which aims to identify not only the seen classes contained in training but also the novel classes that have never been seen. We adopt a stringent inductive setting in which only the instances of seen classes are accessible during training. We propose an open-aware prototypical matching approach to accomplish the segmentation. The prototypical way extracts the visual representations by a set of prototypes, making it convenient and flexible to add new unseen classes. A prototype projection is trained to map the semantic representations towards prototypes based on seen instances, and will generate prototypes for unseen classes. Moreover, an open-set rejection is utilized to detect objects that do not belong to any seen classes, which greatly reduces the misclassification of unseen objects into seen classes due to the lack of seen training instances. We apply the framework on two segmentation datasets, Pascal VOC 2012 and Pascal Context, and achieve impressively state-of-the-art performance.
Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction

Abstract In this work, we propose a 3D fully convolutional architecture for video saliency prediction that employs hierarchical supervision on intermediate maps (referred to as conspicuity maps ) generated using features extracted at different abstraction levels. We provide the base hierarchical learning mechanism with two techniques for domain adaptation and domain-specific learning . For the former, we encourage the model to unsupervisedly learn hierarchical general features using gradient reversal at multiple scales, to enhance generalization capabilities on datasets for which no annotations are provided during training. As for domain specialization, we employ domain-specific operations (namely, priors, smoothing and batch normalization) by specializing the learned features on individual datasets in order to maximize performance. The results of our experiments show that the proposed model yields state-of-the-art accuracy on supervised saliency prediction. When the base hierarchical model is empowered with domain-specific modules, performance improves, outperforming state-of-the-art models on three out of five metrics on the DHF1K benchmark and reaching the second-best results on the other two. When, instead, we test it in an unsupervised domain adaptation setting, by enabling hierarchical gradient reversal layers, we obtain performance comparable to supervised state-of-the-art. Source code, trained models and example outputs are publicly available at https://github.com/perceivelab/hd2s .
Cross-Modal Attention for MRI and Ultrasound Volume Registration

Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registration. Our proposed cross-modal attention block effectively maps each of the features in one volume to all features in the corresponding volume. Our experimental results demonstrate that a CNN network designed with the cross-modal attention block embedded outperforms an advanced CNN network 10 times of its size. We also incorporated visualization techniques to improve the interpretability of our network. The source code of our work is available at https://github.com/DIAL-RPI/Attention-Reg.
PnP-DETR: Towards Efficient Visual Analysis with Transformers

Recently, DETR [3] pioneered the solution of vision tasks with transformers, it directly translates the image feature map into the object detection result. Though effective, translating the full feature map can be costly due to redundant computation on some area like the background. In this work, we encapsulate the idea of reducing spatial redundancy into a novel poll and pool (PnP) sampling module, with which we build an end-to-end PnP-DETR architecture that adaptively allocates its computation spatially to be more efficient. Concretely, the PnP module abstracts the image feature map into fine foreground object feature vectors and a small number of coarse background contextual feature vectors. The transformer models information interaction within the fine-coarse feature space and translates the features into the detection result. Moreover, the PnP-augmented model can instantly achieve various desired trade-offs between performance and computation with a single model by varying the sampled feature length, without requiring to train multiple models as existing methods. Thus it offers greater flexibility for deployment in diverse scenarios with varying computation constraint. We further validate the generalizability of the PnP module on panoptic segmentation and the recent transformer-based image recognition model ViT [7] and show consistent efficiency gain. We believe our method makes a step for efficient visual analysis with transformers, wherein spatial redundancy is commonly observed. Code and models will be available.
A Survey on Digital Certificates Approaches for the COVID-19 Pandemic

Digital COVID-19 certificates serve as reliable proof that an individual was vaccinated, tested negative, or healed from COVID-19, facilitating health, occupational, educational, and travel activities during the pandemic.This paper contributes the first to our knowledge state-of-the-art and holistic review of this ecosystem, attempting to answer the following questions: 1) is there a harmonization among academia, organizations, and governments in terms of the certificate deployment technology?; 2) what is the proliferation of such schemes worldwide and how similar are they?; 3) are smartphone applications that accompany such schemes privacy-preserving from an end-user's perspective?To respond to these questions, a four-tier approach is followed: (a) we scrutinize the so far academic works suggesting some type of digital certificate, highlighting common characteristics and weaknesses; (b) we constructively report on the different initiatives proposed by organizations or alliances; (c) we briefly review 54 country initiatives around the globe; and (d) we analyze both statically and dynamically all official Android smartphone applications offered for such certificates to reveal possible hiccups affecting the security or privacy of the end-user.From a bird's eye view, the great majority of the proposed or developed schemes follow either the blockchain model or the asymmetric cryptosystem, the spread of schemes especially in Europe and partly in Asia is high, some degree of distinctiveness among the relevant schemes developed by countries does exist, and there are substantial variations regarding the privacy level of the applications between Europe on the one hand and Asia and America on the other.
The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation

We introduce CenterGroup, an attention-based framework to estimate human poses from a set of identity-agnostic keypoints and person center predictions in an image. Our approach uses a transformer to obtain context-aware embeddings for all detected keypoints and centers and then applies multi-head attention to directly group joints into their corresponding person centers. While most bottom-up methods rely on non-learnable clustering at inference, CenterGroup uses a fully differentiable attention mechanism that we train end-to-end together with our keypoint detector. As a result, our method obtains state-of-the-art performance with up to 2.5x faster inference time than competing bottom-up approaches. Our code is available at https://github.com/dvl-tum/center-group
Optimized U-Net for Brain Tumor Segmentation

We propose an optimized U-Net architecture for a brain tumor segmentation task in the BraTS21 challenge. To find the optimal model architecture and the learning schedule, we have run an extensive ablation study to test: deep supervision loss, Focal loss, decoder attention, drop block, and residual connections. Additionally, we have searched for the optimal depth of the U-Net encoder, number of convolutional channels and post-processing strategy. Our method won the validation phase and took third place in the test phase. We have open-sourced the code to reproduce our BraTS21 submission at the NVIDIA Deep Learning Examples GitHub Repository ( https://github.com/NVIDIA/DeepLearningExamples/blob/master/PyTorch/Segmentation/nnUNet/notebooks/BraTS21.ipynb ).