Reviewerly/exHarmony huggingface dataset:
Learning Compatible Embeddings

Achieving backward compatibility when rolling out new models can highly reduce costs or even bypass feature re-encoding of existing gallery images for in-production visual retrieval systems. Previous related works usually leverage losses used in knowledge distillation which can cause performance degradations or not guarantee compatibility. To address these issues, we propose a general framework called Learning Compatible Embeddings (LCE) which is applicable for both cross model compatibility and compatible training in direct/forward/backward manners. Our compatibility is achieved by aligning class centers between models directly or via a transformation, and restricting more compact intra-class distributions for the new model. Experiments are conducted in extensive scenarios such as changes of training dataset, loss functions, network architectures as well as feature dimensions, and demonstrate that LCE efficiently enables model compatibility with marginal sacrifices of accuracies. The code will be available at https://github.com/IrvingMeng/LCE.
XtremeAugment: Getting More From Your Data Through Combination of Image Collection and Image Augmentation

Deep convolutional neural networks are highly efficient for computer vision tasks using plenty of training data.However, there remains a problem of small training datasets.For addressing this problem the training pipeline which handles rare object types and an overall lack of training data to build wellperforming models that provide stable predictions is required.This article reports on the comprehensive framework XtremeAugment which provides an easy, reliable, and scalable way to collect image datasets and to efficiently label and augment collected data.The presented framework consists of two augmentation techniques that can be used independently and complement each other when applied together.These are Hardware Dataset Augmentation (HDA) and Object-Based Augmentation (OBA).HDA allows the users to collect more data and spend less time on manual data labeling.OBA significantly increases the training data variability and remains the distribution of the augmented images being close to the original dataset.We assess the proposed approach for the apple spoil segmentation scenario.Our results demonstrate a substantial increase in the model accuracy reaching 0.91 F1-score and outperforming the baseline model for up to 0.62 F1-score for a few-shot learning case in the wild data.The highest benefit of applying XtremeAugment is achieved for the cases where we collect images in the controlled indoor environment, but have to use the model in the wild.
A novel sky image-based solar irradiance nowcasting model with convolutional block attention mechanism

Global horizontal irradiance (GHI) is a crucial factor impacting photovoltaic (PV) production, and is required for accurate real-time photovoltaic power forecasting. And it is a new effective solution to obtain the real-time GHI by the sky images because the GHI is mainly affected by the cloud cover and cloud motion. Therefore, the research proposes a unique artificial intelligence approach for real-time GHI forecasting (‘nowcasting’) based on sky images, which can significantly enhance GHI forecasting accuracy on cloudy sky days. First, a new nowcasting model with convolutional block attention module (CBAM) is proposed, which is based on Visual Geometry Group (VGG) networks. Then, taking the local cloud cover (LCC) as a numerical feature, we coupled it with the cloud feature in the sky image to improve the forecasting performance of the GHI model. Finally, to verify the effectiveness and advantages of the proposed method, when compared to state-of-the-art methods, such as Sun’s model, Jiang’s model, and others, the proposed method outperforms them in GHI forecasting as demonstrated by the 11.67% nRMSE, 7.97% nMAE, 27.69% MAPE, and 0.91 CORR results on the ASI-16 dataset.
Camera, LiDAR, and Radar Sensor Fusion Based on Bayesian Neural Network (CLR-BNN)

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Perception</i> in automated vehicles (AV) is the main factor in achieving safe driving. In this perception task, multi-object detection (MOD) in diverse driving situations is the main challenge. Our recent survey [Ravindran <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> (2021)] shows the limitations of deep neural networks (DNN) in predicting the uncertainties of object detection in MOD. This research proposed a camera, LiDAR and RADAR sensor fusion Bayesian neural network (CLR-BNN) to improve detection accuracy and reduce uncertainties in diverse driving situations using these three primary sensing devices. The experiments were performed using the nuScence dataset with incorporation of various noises. The CLR-BNN performed better than its deterministic sensor fusion model (CLR-DNN) in terms of mAP. The CLR-BNN also showed improvement in categorical and bounding box location uncertainty using sensor fusion in diverse driving conditions. The uncertainty predictions of the CLR-BNN were validated using the calibration curve and other performance metrics.
Re-Aging GAN: Toward Personalized Face Age Transformation

Face age transformation aims to synthesize past or future face images by reflecting the age factor on given faces. Ideally, this task should synthesize natural-looking faces across various age groups while maintaining identity. However, most of the existing work has focused on only one of these or is difficult to train while unnatural artifacts still appear. In this work, we propose Re-Aging GAN (RAGAN), a novel single framework considering all the critical factors in age transformation. Our framework achieves state-of-the-art personalized face age transformation by compelling the input identity to perform the self-guidance of the generation process. Specifically, RAGAN can learn the personalized age features by using high-order interactions between given identity and target age. Learned personalized age features are identity information that is recalibrated according to the target age. Hence, such features encompass identity and target age information that provides important clues on how an input identity should be at a certain age. Experimental result shows the lowest FID and KID scores and the highest age recognition accuracy compared to previous methods. The proposed method also demonstrates the visual superiority with fewer artifacts, identity preservation, and natural transformation across various age groups.
QUEST: systematically approximating Quantum circuits for higher output fidelity

We present QUEST, a procedure to systematically generate approximations for quantum circuits to reduce their CNOT gate count. Our approach employs circuit partitioning for scalability with procedures to 1) reduce circuit length using approximate synthesis, 2) improve fidelity by running circuits that represent key samples in the approximation space, and 3) reason about approximation upper bound. Our evaluation results indicate that our approach of "dissimilar" approximations provides close fidelity to the original circuit. Overall, the results indicate that QUEST can reduce CNOT gate count by 30-80% on ideal systems and decrease the impact of noise on existing and near-future quantum systems.
Unifying Nonlocal Blocks for Neural Networks

The nonlocal-based blocks are designed for capturing long-range spatial-temporal dependencies in computer vision tasks. Although having shown excellent performance, they still lack the mechanism to encode the rich, structured information among elements in an image or video. In this paper, to theoretically analyze the property of these nonlocal-based blocks, we provide a new perspective to interpret them, where we view them as a set of graph filters generated on a fully-connected graph. Specifically, when choosing the Chebyshev graph filter, a unified formulation can be derived for explaining and analyzing the existing nonlocal-based blocks (e.g., nonlocal block, nonlocal stage, double attention block). Furthermore, by concerning the property of spectral, we propose an efficient and robust spectral nonlocal block, which can be more robust and flexible to catch long-range dependencies when inserted into deep neural networks than the existing nonlocal blocks. Experimental results demonstrate the clear-cut improvements and practical applicabilities of our method on image classification, action recognition, semantic segmentation, and person re-identification tasks. Code are available at https://github.com/zh460045050/SNL_ICCV2021.
MS2Net: Multi-Scale and Multi-Stage Feature Fusion for Blurred Image Super-Resolution

At present, most mainstream algorithms for single image super-resolution (SISR) assume the image degradation process as an ideal degradation process (e.g. bicubic downscaling), which violates the actual degeneration conditions. In real-world image capturing, objects often move in a dynamic environment, and camera shake also often occurs, which results in serious blurs. Our work focuses on the task of image super-resolution with heavy motion blur, for which we adopt a network with two branches: one branch for image deblurring and the other one for super-resolution. Since the features obtained by the deblurring are rich in details, we apply their features as supplementary information to the super-resolution branch. Based on the adopted dual-branch framework, our major technical novelties lie in two novel modules: Multi-Scale Feature Fusion (MSFF1) module which fuses features of different scale from the deblurring branch to get local and global information, and Multi-Stage Feature Fusion (MSFF2) module which further filters useful information with attention. We evaluate the proposed method under various blur scenarios on the benchmark datasets, demonstrating competitive performance against existing methods.
Hierarchical feature disentangling network for universal domain adaptation

Universal Domain Adaptation (UniDA) aims to address a more practical problem compared with traditional Close-Set Domain Adaptation (CSDA). Besides the domain gap in traditional CSDA, the common and private label sets across domains are unknown in UniDA leading to an additional category gap. Without considering the category gap for domain adversarial training to extract domain-relevant features, existing methods may suffer from the feature misalignment problem and result in negative transfer. This paper proposes a Hierarchical Feature Disentangling Network (HFDN) to disentangle domain-relevant features into domain-specific and category-shift features for latent variables caused by domain gap and category gap, respectively. Domain-specific features are trained to distinguish the source domain from the target one by discovering domain-specific attributes (e.g. illumination, style), and adversarially aligned to bridge the domain gap for knowledge transfer. Category-shift features are extracted to distinguish domains by identifying private classes across domains, so that they can be leveraged to assign larger weights for samples from the common label set. Experiments show that the proposed HFDN surpasses state-of-the-art CSDA, partial DA, open-set DA and UniDA models.
Training Social Skills in Virtual Reality Machine Learning as a Process of Co-Creation

The training of social skills in interactive Virtual Reality scenes has become an issue of rising interest in research and development. This is also the case of the project Virtual Skills Lab that deals with the question how the potential of such a training can be applied to the context of organizations. In a virtual office scene, the interaction between the player and the computer-generated agent is based on speech recognition and conversational Artificial Intelligence (AI). While studies and experiments have shown that humans are able to treat agents in Virtual Reality as if they were real persons, it is still impossible to establish truly spontaneous forms of interaction by combining speech recognition, conversational AI and Virtual Reality. Hence, existing applications for social skills training in Virtual Reality focus on the possibility to literally take over the other’s perspective or the ability to overcome social anxieties. Yet, social interaction in the workplace is always embedded in a given organizational culture and therefore characterized by behavioral patterns. In this sense, a Virtual Reality training tool for social interaction in the workplace can be enriched by design elements that enable machine learning in terms of pattern recognition. The paper outlines how the project team has dealt with the existing limitations of conversational AI in the process of working out the storyline and what design elements can be introduced in the Virtual Reality interaction in order to enable the machinic recognition of patterns of organizational culture.
A study of Security Threats and Attacks in Cloud Computing

In recent decades, cloud computing (CC) has attracted great consideration due to steadily growing needs. Organizations moving towards cloud data storage solutions have various advantages. These contain efficient IT management, efficient access from anyplace in the globe with steady internet access and the cost efficiency of CC scheme. More investigation is needed for connected security and privacy problems in the cloud. The previously published studies offered a possible answer to these difficulties to researchers from academic circles, industry and standards areas. This survey presents narrative reviews of problems and requirements for cloud safety, identified threats and known vulnerabilities. This paper is actually designed to examine the many components of CC, as well as current security and privacy challenges. This report provides an extensive review of security risks associated to various cloud computing variables. In addition, this paper proposes a novel classification of emerging security solutions in this field. This article focuses on and explores the details of the security challenges for cloud companies like the supplier of cloud services, the data owner and the cloud user confront.
Bias Loss for Mobile Neural Networks

Compact convolutional neural networks (CNNs) have witnessed exceptional improvements in performance in recent years. However, they still fail to provide the same predictive power as CNNs with a large number of parameters. The diverse and even abundant features captured by the layers is an important characteristic of these successful CNNs. However, differences in this characteristic between large CNNs and their compact counterparts have rarely been investigated. In compact CNNs, due to the limited number of parameters, abundant features are unlikely to be obtained, and feature diversity becomes an essential characteristic. Diverse features present in the activation maps derived from a data point during model inference may indicate the presence of a set of unique descriptors necessary to distinguish between objects of different classes. In contrast, data points with low feature diversity may not provide a sufficient amount of unique descriptors to make a valid prediction; we refer to them as random predictions. Random predictions can negatively impact the optimization process and harm the final performance. This paper proposes addressing the problem raised by random predictions by reshaping the standard cross-entropy to make it biased toward data points with a limited number of unique descriptive features. Our novel Bias Loss focuses the training on a set of valuable data points and prevents the vast number of samples with poor learning features from misleading the optimization process. Furthermore, to show the importance of diversity, we present a family of SkipblockNet models whose architectures are brought to boost the number of unique descriptors in the last layers. Experiments conducted on benchmark datasets demonstrate the superiority of the proposed loss function over the cross-entropy loss. Moreover, our SkipblockNet-M can achieve 1% higher classification accuracy than MobileNetV3 Large with similar computational cost on the ImageNet ILSVRC-2012 classification dataset. The code is available on the link - https://github.com/lusinlu/biasloss_skipblocknet.
Contrast and Classify: Training Robust VQA Models

Recent Visual Question Answering (VQA) models have shown impressive performance on the VQA benchmark but remain sensitive to small linguistic variations in input questions. Existing approaches address this by augmenting the dataset with question paraphrases from visual question generation models or adversarial perturbations. These approaches use the combined data to learn an answer classifier by minimizing the standard cross-entropy loss. To more effectively leverage augmented data, we build on the recent success in contrastive learning. We propose a novel training paradigm (ConClaT) that optimizes both cross-entropy and contrastive losses. The contrastive loss encourages representations to be robust to linguistic variations in questions while the cross-entropy loss preserves the discriminative power of representations for answer prediction.We find that optimizing both losses – either alternately or jointly – is key to effective training. On the VQA-Rephrasings [44] benchmark, which measures the VQA model’s answer consistency across human paraphrases of a question, ConClaT improves Consensus Score by 1.63% over an improved baseline. In addition, on the standard VQA 2.0 benchmark, we improve the VQA accuracy by 0.78% overall. We also show that ConClaT is agnostic to the type of data-augmentation strategy used.
Planar Surface Reconstruction from Sparse Views

The paper studies planar surface reconstruction of indoor scenes from two views with unknown camera poses. While prior approaches have successfully created object-centric reconstructions of many scenes, they fail to exploit other structures, such as planes, which are typically the dominant components of indoor scenes. In this paper, we re-construct planar surfaces from multiple views, while jointly estimating camera pose. Our experiments demonstrate that our method is able to advance the state of the art of re-construction from sparse views, on challenging scenes from Matterport3D.
Deep 3D Mask Volume for View Synthesis of Dynamic Scenes

Image view synthesis has seen great success in reconstructing photorealistic visuals, thanks to deep learning and various novel representations. The next key step in immersive virtual experiences is view synthesis of dynamic scenes. However, several challenges exist due to the lack of high-quality training datasets, and the additional time dimension for videos of dynamic scenes. To address this issue, we introduce a multi-view video dataset, captured with a custom 10-camera rig in 120FPS. The dataset contains 96 high-quality scenes showing various visual effects and human interactions in outdoor scenes. We develop a new algorithm, Deep 3D Mask Volume, which enables temporally-stable view extrapolation from binocular videos of dynamic scenes, captured by static cameras. Our algorithm addresses the temporal inconsistency of disocclusions by identifying the error-prone areas with a 3D mask volume, and replaces them with static background observed throughout the video. Our method enables manipulation in 3D space as opposed to simple 2D masks, We demonstrate better temporal stability than frame-by-frame static view synthesis methods, or those that use 2D masks. The resulting view synthesis videos show minimal flickering artifacts and allow for larger translational movements.
Scaling-up Disentanglement for Image Translation

Image translation methods typically aim to manipulate a set of labeled attributes (given as supervision at training time e.g. domain label) while leaving the unlabeled attributes intact. Current methods achieve either: (i) disentanglement, which exhibits low visual fidelity and can only be satisfied where the attributes are perfectly uncorrelated. (ii) visually-plausible translations, which are clearly not disentangled. In this work, we propose OverLORD, a single framework for disentangling labeled and unlabeled attributes as well as synthesizing high-fidelity images, which is composed of two stages; (i) Disentanglement: Learning disentangled representations with latent optimization. Differently from previous approaches, we do not rely on adversarial training or any architectural biases. (ii) Synthesis: Training feed-forward encoders for inferring the learned attributes and tuning the generator in an adversarial manner to increase the perceptual quality. When the labeled and unlabeled attributes are correlated, we model an additional representation that accounts for the correlated attributes and improves disentanglement. We highlight that our flexible framework covers multiple settings as disentangling labeled attributes, pose and appearance, localized concepts, and shape and texture. We present significantly better disentanglement with higher translation quality and greater output diversity than state-of-the-art methods.
Augmented Reality Application in Classroom: An Immersive Taxonomy

Innovating learning technologies and modernizing the education system can improve education. A blend of AR and VR technology improve student learning, motivation, and it also facilitates teamwork and group cooperation. Knowledge gaps and terminologies are mixed in this article on existing taxonomies of augmented reality and virtual reality and present analysis of existing taxonomies. Confusion is likely to arise among educators, researchers, and developers. A methodology for creating a taxonomy of applications related to augmented reality and virtual reality is presented. The taxonomy emphasizes the interaction between the technology that provides the augmented reality and the user's perception of the content. Because in education augmented reality is a relatively new topic and little has also been studied in STEM education, Analyses of augmented reality technologies are performed, which are applicable to teaching course material in the natural and mathematics sciences. In addition, it can accommodate a wide range devices and approaches developed over time to serve a number of purposes. This article illustrates how augmented reality can be used in the mathematical geometry lesson.
TPPSUPPLY : A traceable and privacy-preserving blockchain system architecture for the supply chain

Traceability and auditability are key structures in supply chain management and construction. However, trust is the most important aspect of customers in these systems. Also, relying on third parties to trade in centralized systems is indispensable. Although current exist frameworks for these solutions in the supply chain, these have work poor traceability and lack of real-time information, and especially lack of privacy. In this article, we propose a framework for supply chain traceability that preserves privacy from third parties by using smart contracts. In the proposed framework, digital signature and verification are provided by using existing cryptographic techniques in off-chain and on-chain smart contract integration, and it is demonstrated the applicability of architecture by testing. The architecture also provides both anonymity and traceability depending on the user’s request. Finally, thanks to traceability and auditability in the proposed system, customers and other parties can view with a single product ID and also verify with digital signature the claims of the actors in the system.