Reviewerly/exHarmony huggingface dataset:
Cryptocurrency: the future money here now…

Purpose This paper aims to look at cryptocurrencies – digital monies – as a dynamic and decentralized means of exchange for payments of goods and services. Design/methodology/approach Literature review and online search in wikis and blogs are used to find information on cryptocurrencies, their uses and technology components, to empower and improve the economy and society. Findings Cryptocurrency is viewed vis-à-vis its impact on the economy and society, showing how economies will be boosted and driven toward empowerment of the poverty-stricken and the society will be powerfully grown in eliminating wretchedness and squalor. Originality/value This study analyzes cryptocurrency and its benefits and identifies its uses in the library and information sector as well as generally as a current and future decentralized monetary system.
Entanglement enhancement from coherent feedback-controlled and cascaded nondegenerate optical parametric amplifiers

The entanglement enhancement of the entangled states from the nondegenerate optical parametric amplifier (NOPA) is an important issue in quantum information science and technology. The coherent feedback-controlled NOPA (CFC-NOPA) and cascaded NOPA (C-NOPA) are two promising schemes on entanglement enhancement. In this study, the features of the quantum entanglement of the entangled states produced by the two schemes is totally studied and compared based on feasible physical parameters of realistic systems. We demonstrate that the CFC-NOPA shows better superiority on entanglement enhancement and better robustness versus the losses of the system for low frequency range (below several hundred kHz). For the high frequency range (several MHz), the situation is inverse, and the C-NOPA is more advantageous. Our results can provide useful references for the design of high-quality entanglement resources.
Open-Set Domain Adaptation Under Few Source-Domain Labeled Samples

Recently, the notion of closed-set few-shot domain adaptation (FSDA) has been introduced where limited supervision is present in the source domain. However, FSDA overlooks the fact that the unlabeled target domain may contain new classes unseen in the source domain. To this end, we introduce the novel problem definition of few-shot open-set DA (FosDA) where the source domain contains few labeled samples together with a large pool of unlabeled data, and the target domain consists of test samples from the known as well as new categories. We propose an end-to-end model called FosDANet to tackle such a scenario which operates on two principles: to generate confident pseudo-labels for the unlabeled source samples and to classwise align the source and target domains for the known classes while rejecting the unknown-class data. A combination of a self-supervised loss and a novel triplet-based relation learning module is devised to aid in confident pseudo-labeling, and a dual adversarial learning scheme is proposed for domain alignment. Extensive experiments were performed on five datasets: Office-31, Office-Home, Adoptiape, and two new datasets we designed: Mini-domainNet and a remote sensing benchmark called NPU-RSDA. FosDANet is found to consistently outperform the relevant literature.
Multi-Scale Exposure Fusion via Content Adaptive Edge-Preserving Smoothing Pyramids

Multi-scale exposure fusion (MEF) is an efficient way to fuse differently exposed low dynamic range (LDR) images of a high dynamic range (HDR) scene into an information enriched LDR image. In this paper, a new MEF algorithm is proposed to merge the differently exposed LDR images by introducing novel content adaptive edge-preserving smoothing (CAS) pyramids for the weight maps of all the LDR images. With the proposed CAS pyramids, details in the brightest and darkest regions of the HDR scene are preserved better than existing MEF algorithms on top of the Gaussian pyramids and edge-preserving smoothing pyramids. Comparisons experimentally demonstrate the effectiveness of the proposed algorithm to nine state-of-the-art MEF algorithms from both subjective and objective points of view regardless the image sizes.
Cell Selection-based Data Reduction Pipeline for Whole Slide Image Analysis of Acute Myeloid Leukemia

Computer-aided analyses of cells in Whole Slide Images (WSIs) have become an important topic in digital pathology. Despite the recent success of deep learning in biomedical research, these methods are still difficult to apply to multi-gigabyte WSIs. To overcome this difficulty, a variety of patch-based solutions have been introduced, which however all suffer from certain limitations compared to manual examinations and often fail to meet the specificities of cytological inspections. Here we introduce an alternative scheme which incorporates clinical expertise in the selection process to automatically identify the clinically relevant areas. By using a bone marrow smear dataset containing 22-gigapixel images of 153 patients, we introduce a novel pipeline combining unsupervised and supervised methodologies to gradually select the most appropriate single-cell regions, which are subsequently used in multiple medically crucial Acute Myeloid Leukemia (AML) predictions. Our approach is capable of dealing with a variety of common WSI challenges, massively limits the manual annotation effort, reduces the data by a factor of up to 99.9% and achieves super-human performance on the final cytological prediction tasks.
DAtRNet: Disentangling Fashion Attribute Embedding for Substitute Item Retrieval

Interactive substitute recommendation for fashion products improves the online retail customer experience. Traditional fashion search platforms incorporate product metadata between the query products and the products to be retrieved. In this paper, we propose DAtRNet, an attribute representation network to disentangle the features in the query product. It is used to recommend attribute-aware substitute items based on the conditional similarity of the retrieved products. The proposed architecture relies on attribute-level similarity providing a fine-grained recommendation. In addition, a concurrent axial attention mechanism is proposed that generates global information embedding and adaptively re-calibrates the soft attention masks. Overall, the end-to-end framework enables the system to disentangle the attribute features and independently deals with them to enhance its flexibility towards one or multiple attributes. The proposed method outperforms the state-of- the-art by a significant margin.
What’s in a Caption? Dataset-Specific Linguistic Diversity and Its Effect on Visual Description Models and Metrics

While there have been significant gains in the field of automated video description, the generalization performance of automated description models to novel domains remains a major barrier to using these systems in the real world. Most visual description methods are known to capture and exploit patterns in the training data leading to evaluation metric increases, but what are those patterns? In this work, we examine several popular visual description datasets, and capture, analyze, and understand the dataset-specific linguistic patterns that models exploit but do not generalize to new domains. At the token level, sample level, and dataset level, we find that caption diversity is a major driving factor behind the generation of generic and uninformative captions. We further show that state-of-the-art models even outperform held-out ground truth captions on modern metrics, and that this effect is an artifact of linguistic diversity in datasets. Understanding this linguistic diversity is key to building strong captioning models, we recommend several methods and approaches for maintaining diversity in the collection of new data, and dealing with the consequences of limited diversity when using current models and metrics.
PAND: Precise Action Recognition on Naturalistic Driving

Temporal action localization for untrimmed videos is a difficult problem in computer vision. It is challenge to infer the start and end of activity instances on small-scale datasets covering multi-view information accurately. In this paper, we propose an effective activity temporal localization and classification method to localize the temporal boundaries and predict the class label of activities for naturalistic driving. Our approach includes (i) a distraction behavior recognition and localization method in naturalistic driving videos on small-scale data sets, (ii) a strategy that uses multi-branch network to make full use of information from different channels, (iii)a post-processing method for selecting and correcting temporal range to ensure that our system finds accurate boundaries. In addition, the frame-level object detection information is also utilized. Extensive experiments prove the effectiveness of our method and we rank the 6th on the Test-A2 of the 6th AI City Challenge track 3.
Few-Shot Supervised Prototype Alignment for Pedestrian Detection on Fisheye Images

Complete and pre-trained models are readily available for download for object detection and can perform well on datasets containing everyday images. Domain adaptation is used to transfer models to more specific datasets with characteristics not present in pre-training. We propose the novel adaptation setting of pedestrian detection in fisheye images, where target samples are scarce but annotated. Our setting provides interesting new challenges for adaptation due to global perspective changes and geometric distortions not found in existing adaptation tasks. To this end, we introduce loss coupling for unsupervised adversarial adaptation and boost prototype-based adaptation with ground-truth information. We additionally propose a novel supervised adaptation head for features in the bounding box regressor. Our method leads to more stable adversarial training and outperforms supervised and unsupervised baselines. Our method requires half the amount of training samples for small datasets to achieve the same performance as supervised fine-tuning.
Adaptive Bitrate Quantization Scheme Without Codebook for Learned Image Compression

We propose a generic approach to quantization without codebook in learned image compression called one-hot max (OHM, Ω) quantization. It reorganizes the feature space resulting in an additional dimension, along which vector quantization yields one-hot vectors by comparing activations. Furthermore, we show how to integrate Ω quantization into a compression system with bitrate adaptation, i.e., full control over bitrate during inference. We perform experiments on both MNIST and Kodak and report on rate-distortion trade-offs comparing with the integer rounding reference. For low bitrates (< 0.4 bpp), our proposed quantizer yields better performance while exhibiting also other advantageous training and inference properties. Code is available at https://github.com/ifnspaml/OHMQ.
HSI-Guided Intrinsic Image Decomposition for Outdoor Scenes

Intrinisic image decomposition (IID) aims to recover the reflectance and shading components from images and is the prerequisite to many downstream computer vision applications, such as image editing and image relighting. Due to the inherent difficulty in acquiring ground truth reflectance and shading, existing datasets are either synthetic indoor scenes or objects using graphics rendering (e.g., CGIntrinsics and ShapeNet etc.) or real photos with very sparse manual annotation (e.g., IIW and SAW etc.). Accompanied with the complex nature of outdoor scenes, most IID methods focus on the decomposition of indoor environment. There is still a long way to go before we can handle IID of outdoor scenes. In this paper, we take the attempt to perform intrinsic image decomposition for outdoor scenes when RGB image is not the only thing we can get from the enviroment. With the observation of prior work where nir-infrared (NIR) images are transparent to a range of colourants/dyes, we propose to extend it to more spectra by collecting hyperspectral imaging (HSI) data which are well aligned with RGB images and to perform IID with both of them. We also apply existing mainstream IID methods for comparison to examine current progress and challenges at the road towards IID outdoors. We still make some improvements and find problems when performing IID for outdoor scenes, even though we do not handle it perfectly. The data we collect will be made publicly available for further potential investigation.
Doubling down: sparse grounding with an additional, almost-matching caption for detection-oriented multimodal pretraining

A common paradigm in deep learning applications for computer vision is self-supervised pretraining followed by supervised fine-tuning on a target task. In the self-supervision step, a model is trained in a supervised fashion, but the source of supervision needs to be implicitly defined by the data. Image-caption alignment is often used as such a source of implicit supervision in multimodal pretraining, and grounding (i.e., matching word tokens with visual tokens) is one way to exploit it. We introduce a strategy to take advantage of an underexplored structure in image-caption datasets: the relationship between captions matched with different images but mentioning the same objects. Given an image-caption pair, we find an additional caption that mentions one of the objects the first caption mentions, and we impose a sparse grounding between the image and the second caption so that only a few word tokens are grounded in the image. Our goal is to learn a better feature representation for the objects mentioned by both captions, encouraging grounding between the additional caption and the image to focus on the common objects only. We report superior grounding performance when comparing our approach with a previously-published pretraining strategy, and we show the benefit of our proposed double-caption grounding on two downstream detection tasks: supervised detection and open-vocabulary detection.
TinyOps: ImageNet Scale Deep Learning on Microcontrollers

Deep Learning on microcontroller (MCU) based IoT devices is extremely challenging due to memory constraints. Prior approaches focus on using internal memory or external memories exclusively which limit either accuracy or latency. We find that a hybrid method using internal and external MCU memories outperforms both approaches in accuracy and latency. We develop TinyOps, an inference engine which accelerates inference latency of models in slow external memory, using a partitioning and overlaying scheme via the available Direct Memory Access (DMA) peripheral to combine the advantages of external memory (size) and internal memory (speed). Experimental results show that architectures deployed with TinyOps significantly outperform models designed for internal memory with up to 6% higher accuracy and importantly, 1.3-2.2x faster inference latency to set the state-of-the-art in TinyML ImageNet classification. Our work shows that the TinyOps space is more efficient compared to the internal or external memory design spaces and should be explored further for TinyML applications.
Network Amplification with Efficient MACs Allocation

Recent studies on deep convolutional neural networks present a simple paradigm of architecture design, i.e., models with more MACs typically achieve better accuracies, such as EfficientNet and RegNet. These works try to enlarge the network architecture with one unified rule by sampling and statistical methods. However, the rule is not prospective to the design of large networks because it is obtained from the experience of researchers on small network architectures. In this paper, we propose to enlarge the capacity of CNN models by fine-grained MACs allocation for the width, depth and resolution on the stage level. In particular, starting from a base small model, we gradually add extra channels, layers or resolution by using a dynamic programming manner. With step-by-step modifying the computations on different stages, the enlarged network will be equipped with optimal allocation and utilization of MACs. On Efficient-Net, our method consistently outperforms the performance of the original scaling method. In particular, the proposed method is used to enlarge models sourced by GhostNet, we achieve state-of-the-art 80.9% and 84.3% ImageNet top-1 accuracies under the setting of 600M and 4.4B MACs, respectively.
Disentangled Loss for Low-Bit Quantization-Aware Training

Quantization-Aware Training (QAT) has recently showed a lot of potential for low-bit settings in the context of image classification. Approaches based on QAT are using the Cross Entropy Loss function which is the reference loss function in this domain. We investigate quantization-aware training with disentangled loss functions. We qualify a loss to disentangle as it encourages the network output space to be easily discriminated with linear functions. We introduce a new method, Disentangled Loss Quantization Aware Training, as our tool to empirically demonstrate that the quantization procedure benefits from those loss functions. Results show that the proposed method substantially reduces the loss in top-1 accuracy for low-bit quantization on CIFAR10, CIFAR100 and ImageNet. Our best result brings the top-1 Accuracy of a Resnet-18 from 63.1% to 64.0% with binary weights and 2-bit activations when trained on ImageNet.
Asymmetric Information Distillation Network for Lightweight Super Resolution

The purpose of this paper is to design a lightweight network to achieve image super resolution performance equivalent to SRResNet. We design an asymmetric information distillation block (AIDB) with distillation information multiplexing and asymmetric information extraction capabilities to better achieve this goal. Distillation information multiplexing refers to the repeated processing of distilled information to supplement the ability of key information extraction. Asymmetric information enhancement block (AIEB) refers to identify different features in the image by the horizontal and vertical feature extraction. AIEB greatly reduces the number of parameters, and distillation information multiplexing works as a supplement to the lost high dimensional information. A large number of experiments show that our asymmetric information distillation network (AIDN) achieves a better balance of performance and complexity than SOTA model. Moreover, Our proposed AIDN ranked second in the model complexity track of NTIRE2022 efficient super resolution challenge. Compared with the first place in this track, we achieves higher PSNR performance on testset with a slight disadvantage in the number of parameters. The code is available at https://github.com/zzksdu/AIDN.
Self-Supervised Variable Rate Image Compression using Visual Attention

The recent success of self-supervised learning relies on its ability to learn the representations from self-defined pseudo-labels that are applied to several downstream tasks. Motivated by this ability, we present a deep image compression technique, which learns the lossy reconstruction of raw images from the self-supervised learned representation of SimCLR ResNet-50 architecture. Our framework uses a feature pyramid to achieve the variable rate compression of the image using a self-attention map for the optimal allocation of bits. The paper provides an overview to observe the effects of contrastive self-supervised representations and the self-attention map on the distortion and perceptual quality of the reconstructed image. The experiments are performed on a different class of images to show that the proposed method outperforms the other variable rate deep compression models without compromising the perceptual quality of the images.
Slimmable Video Codec

Neural video compression has emerged as a novel paradigm combining trainable multilayer neural net-works and machine learning, achieving competitive rate-distortion (RD) performances, but still remaining impractical due to heavy neural architectures, with large memory and computational demands. In addition, models are usually optimized for a single RD tradeoff. Recent slimmable image codecs can dynamically adjust their model capacity to gracefully reduce the memory and computation requirements, without harming RD performance. In this paper we propose a slimmable video codec (SlimVC), by integrating a slimmable temporal entropy model in a slimmable autoencoder. Despite a significantly more complex architecture, we show that slimming remains a powerful mechanism to control rate, memory footprint, computational cost and latency, all being important requirements for practical video compression.
Temporal Driver Action Localization using Action Classification Methods

Driver distraction recognition is an essential computer vision task that can play a key role in increasing traffic safety and reducing traffic accidents. In this paper, we propose a temporal driver action localization (TDAL) framework for classifying driver distraction actions, as well as identifying the start and end time of a given driver action. The TDAL framework consists of three stages: preprocessing, 'which takes untrimmed video as input and generates multiple clips; action classification, which classifies the clips; and finally, the classifier output is sent to the temporal action localization to generate the start and end times of the distracted actions. The proposed framework achieves an F1 score of27.06% on Track 3A2 dataset of NVIDIA AI City 2022 Challenge. The findings show that the TDAL framework contributes to fine-grained driver distraction recognition and paves the way for the development of smart and safe transportation. Code will be available soon.
Multi-encoder Network for Parameter Reduction of a Kernel-based Interpolation Architecture

Video frame interpolation involves the synthesis of new frames from existing ones. Convolutional neural networks (CNNs) have been at the forefront of the recent advances in this field. One popular CNN-based approach involves the application of generated kernels to the input frames to obtain an interpolated frame. Despite all the benefits interpolation methods offer, many of these networks require a lot of parameters, with more parameters meaning a heavier computational burden. Reducing the size of the model typically impacts performance negatively. This paper presents a method for parameter reduction for a popular flow-less kernel-based network (Adaptive Collaboration of Flows). Through our technique of removing the layers that require the most parameters and replacing them with smaller encoders, we reduce the number of parameters of the network and even achieve better performance compared to the original method. This is achieved by deploying rotation to force each individual encoder to learn different features from the input images. Ablations are conducted to justify design choices and an evaluation on how our method performs on full-length videos is presented.
Area Under the ROC Curve Maximization for Metric Learning

Most popular metric learning losses have no direct relation with the evaluation metrics that are subsequently applied to evaluate their performance. We hypothesize that training a metric learning model by maximizing the area under the ROC curve (which is a typical performance measure of recognition systems) can induce an implicit ranking suitable for retrieval problems. This hypothesis is supported by previous work that proved that a curve dominates in ROC space if and only if it dominates in Precision-Recall space. To test this hypothesis, we design and maximize an approximated, derivable relaxation of the area under the ROC curve. The proposed AUC loss achieves state-of-the-art results on two large scale retrieval benchmark datasets (Stanford Online Products and DeepFashion In-Shop). Moreover, the AUC loss achieves comparable performance to more complex, domain specific, state-of-the-art methods for vehicle re-identification.