Reviewerly/exHarmony huggingface dataset:
Investigating the relationships between class probabilities and users’ appropriate trust in computer vision classifications of ambiguous images

The large-scale adoption of systems that automate classifications using Machine Learning (ML) algorithms raises pressing challenges as they support or make decisions with profound consequences for human beings. It is important to understand how users’ trust is affected by ML models’ suggestions, even when those models are wrong. Many research efforts have focused on the user’s ability to interpret what a model has learned. In this paper, we seek to understand another aspect of ML interpretability: whether and how the presence of classification probabilities and their different distributions are related to users’ trust in model outcomes, especially in ambiguous instances. To this end, we conducted two online surveys in which we asked participants to evaluate their agreement with image classifications of pictures of animals made by an ML model. In the first, we analyze their trust before and after presenting them the model classification probabilities. In the second, we investigate the relationships between class probability distributions and users’ trust in the model. We found that, in some cases, the additional information is correlated with undue trust in the model’s classifications. However, in others, they are associated with inappropriate skepticism.
Weakly Supervised Object Detection with Symmetry Context

Recently, weakly supervised object detection (WSOD) with image-level annotation has attracted great attention in the field of computer vision. The problem is often formulated as multiple instance learning in the existing studies, which are often trapped by discriminative object parts and fail to localize the object boundary precisely. In this work, we alleviate this problem by exploiting contextual information that may potentially increase object localization accuracy. Specifically, we propose novel context proposal mining strategies and a Symmetry Context Module to leverage surrounding contextual information of precomputed region proposals. Both naive and Gaussian-based context proposal mining methods are adopted to yield informative context proposals symmetrically surrounding region proposals. Then mined context proposals are fed into our Symmetry Context Module to encourage the model to select proposals that contain the whole object, rather than the most discriminative object parts. Experimental results show that the mean Average Precision (mAP) of the proposed method achieves 52.4% on the PASCAL VOC 2007 dataset, outperforming the state-of-the-art methods and demonstrating its effectiveness for weakly supervised object detection.
Correction Algorithm of Tennis Dynamic Image Serving Path Based on Symmetric Algorithm

The existing target tracking algorithm of the tennis dynamic image serving path cannot correct the serving path in time according to the physical characteristics of the target, resulting in poor correction accuracy and stability. To this end, this paper proposes correction algorithm of tennis dynamic image serving path based on symmetric algorithm. Dynamic images are obtained using the visual acquisition model. On this basis, the contour of the moving target is obtained by the symmetric algorithm, and the complete moving target is obtained by the background difference method. The physical properties of moving objects are analyzed through a tennis serve visual model. The path correction model based on physical features is used to construct the relevant model so as to realize the path correction of the moving target. The experimental results show that the highest accuracy of the algorithm for tennis net and bottom line is 92.88% and 93.10%, respectively, and the average image correction accuracy is 95%. The tracking accuracy of service paths in complex backgrounds is 95%. These data show that the proposed algorithm has high correction accuracy and stability.
Dynamic Dual-Threshold Virtual Machine Merging Method Based on Three-Way Decision

Cloud computing, an emerging computing paradigm, has been widely considered due to its high scalability and availability. An essential stage of cloud computing is the cloud virtual machine migration technology. Nevertheless, the current trigger timing of virtual machine migration in cloud data centers is inaccurate, resulting in insufficient virtual machine consolidation. Furthermore, the high and low workload fluctuations are also a potential symmetrical problem worthy of attention. This paper proposes a virtual machine energy-saving merging method based on a three-way decision (ESMM-3WD). Firstly, we need to calculate the load fluctuation of the physical machine and divide the load fluctuation into three parts. Furthermore, the corresponding mathematical model predicts the load according to the different classification categories. Then, the predicted load value is used to dynamically adjust the threshold to improve the virtual machine merge probability. Finally, the simulation experiment is carried out on the cloud computing simulation platform cloudsim plus. The experimental results show that the virtual machine energy-saving merging method based on the three-way decision proposed in this paper can better reduce the number of migrations, increase the number of physical machines shut down, better improve the probability of virtual machine merger, and achieve the purpose of reducing the energy consumption of the data center.
CoPrGAN: Image-to-Image Translation via Content Preservation

Image-to-image translation is an interesting and challenging application. At present, it has developed from single-domain to multi-domain and many other aspects. But losing content structure is still an inevitable problem, which is manifested in the details of the objects or the global features of the images. We propose a new framework named CoPrGAN to alleviate the lack of content. CoPrGAN focuses on the expressive ability of content and style in different dimensions. In this way, the model uses multiple dynamic paths between the content encoder and the decoder to transform domains while preserving content structure. The content structure we are concerned with is not just posture and location, but also birthmarks, hair color, environment, etc. Experiments are arranged in animal face change that focus on local details and seasonal change that focus on global information. Both the comparative experiments with state-of-the-art and the ablation experiments demonstrate the superiority of CoPrGAN.
Unsupervised Multi-view Multi-person 3D Pose Estimation Using Reprojection Error

This work addresses multi-view multi-person 3D pose estimation in synchronized and calibrated camera views. Recent approaches estimate neural network weights in a supervised way; they rely on ground truth annotated datasets to compute the loss function and optimize the weights in the network. However, manually labeling ground truth datasets is labor-intensive, expensive, and prone to errors. Consequently, it is preferable not to rely heavily on labeled datasets. This work proposes an unsupervised approach to estimating 3D human poses requiring only an off-the-shelf 2D pose estimation method and the intrinsic and extrinsic camera parameters. Our approach uses reprojection error as a loss function instead of comparing the predicted 3D pose with the ground truth. First, we estimate the 3D pose of each person using the plane sweep stereo approach, in which the depth of each 2D joint related to each person is estimated in a selected target view. The estimated 3D pose is then projected onto each of the other views using camera parameters. Finally, the 2D reprojection error in the image plane is computed by comparing it with the estimated 2D pose corresponding to the same person. The 2D poses that correspond to the same person are identified using virtual depth planes, where each 3D pose is projected onto the reference view and compared to find the nearest 2D pose. Our proposed method learns to estimate 3D pose in an end-to-end unsupervised manner and does not require any manual parameter tuning, yet we achieved results close to state-of-the-art supervised methods on a public dataset. Our method achieves only 5.8% points below the fully supervised state-of-the-art method and only 5.1% points below the best geometric approach in the Campus dataset.
Jacobian Ensembles Improve Robustness Trade-Offs to Adversarial Attacks

Deep neural networks have become an integral part of our software infrastructure and are being deployed in many widely-used and safety-critical applications. However, their integration into many systems also brings with it the vulnerability to test time attacks in the form of Universal Adversarial Perturbations (UAPs). UAPs are a class of perturbations that when applied to any input causes model misclassification. Although there is an ongoing effort to defend models against these adversarial attacks, it is often difficult to reconcile the trade-offs in model accuracy and robustness to adversarial attacks. Jacobian regularization has been shown to improve the robustness of models against UAPs, whilst model ensembles have been widely adopted to improve both predictive performance and model robustness. In this work, we propose a novel approach, Jacobian Ensembles – a combination of Jacobian regularization and model ensembles to significantly increase the robustness against UAPs whilst maintaining or improving model accuracy. Our results show that Jacobian Ensembles achieves previously unseen levels of accuracy and robustness, greatly improving over previous methods that tend to skew towards only either accuracy or robustness.
Utilize Spatial Prior in Ground Truth: Spatial-Enhanced Loss for Semantic Segmentation

Most supervised semantic segmentation methods to date choose cross-entropy loss (CE) as the default choice. Standard CE treats all pixels in the image indiscriminately, which lacks consideration of context differences between pixels, leading to the model being overwhelmed by numerous homogeneous pixels in large-scale objects. It ignores an essential spatial prior that can be deduced from Ground Truth-the segmentation edges, which can be practical to distinguish the excessive homogeneous pixels. Therefore, we propose a novel loss function termed Spatial-enhanced Loss (SL), in which the image is spatially separated into the edge region and the body region with the assistance of the edge derived from Ground Truth. Experiments evidence that SL has impressive superiority over Focal Loss, standard cross-entropy loss, class-balanced cross-entropy loss and Dice Loss. We achieve substantial improvements on multiple models without using any tricks, up to 1.60% mIoU.
DensEMANN + Sparsification: Experiments for Further Shrinking Already Small Automatically Generated DenseNet

This paper presents a few experiments that we carried out using DensEMANN (an algorithm that we are developing for automatically generating small and efficient DenseNet neural networks) and various algorithms for pruning or sparsifying neural networks at different granularity levels. The pruning algorithms that we used are based on the Lottery Ticket algorithm by Frankle and Carbin (2019), and on the Dense-Sparse-Dense (DSD) training algorithm by Han et al. (2017). Our experiments show that the pruning method based on DSD training is very efficient for reducing the parameter count of both human-designed and DensEMANN-generated neural networks while making them recover their original accuracy, and that this is especially true when sparsification is performed at the granularity level of individual convolution weights (by means of a mask that zeroes them out). Further research is nevertheless necessary to find out if (and how) this method can become an alternative to DensEMANN, or work in tandem with it, for actually shrinking already small and efficient neural networks.
Improving Stylized Image Captioning with Better Use of Transformer

Stylized Image Captioning aims to generate more human-like captions that can preserve the semantics of the image and have various style characteristics. Most existing methods adopt a framework consisting of CNN for encoding and RNN for decoding and develop the decoder to get better results. Compared with the attention in RNN which only models inter-modal interactions (i.e., object-to-word), attention mechanisms in Transformer can’t only characterize inter-modal interactions but also intra-modal interactions (i.e., word-to-word and object-to-object). However, even Transformer Encoder is adopted to help capture internal relationships of image regions, it’s still hard for decoders to stylize captions. To this end, we attempt to reconstruct transformer to achieve the goals of stylization and content preservation in two ways. Moreover, to better preserve image content, we adopt a Multi-Task Learning framework that generates factual and stylized captions of the same image. In this way, our models can generate stylized captions having style and also maintaining strong relevance with images. Experiments on two benchmark datasets indicate that our models outperform the state-of-the-art systems on average.
Hierarchical Dynamics in Deep Echo State Networks

Reservoir computing (RC) is a popular approach to the efficient design of recurrent neural networks (RNNs), where the dynamical part of the model is initialized and left untrained. Deep echo state networks (ESNs) combined the deep learning approach with RC, by structuring the reservoir in multiple layers, thus offering the striking advantage of encoding the input sequence on different time-scales. A key factor for the effectiveness of ESNs is the echo state property (ESP), which ensures the asymptotic stability of the reservoir dynamics. In this paper, we perform an in-depth theoretical analysis of asymptotic dynamics in Deep ESNs with different contractivity hierarchies, offering a more accurate sufficient condition of the ESP. We investigate how different hierarchies of contractivity affect memory capacity and predictive performance in regression tasks, concluding that structuring reservoir layers in decreasing contractivity is the best design choice. The results of this paper can potentially be applied also to the design of fully-trained RNNs.
Liquid State Machine on Loihi: Memory Metric for Performance Prediction

Liquid State Machine (LSM) is a spiking variant of recurrent neural networks with promising results for speech, video and other temporal datasets classification. LSM employ a network of fixed and randomly connected neurons, called a reservoir. Parameter selection for building the best performing reservoir is a difficult task given the vast parameter space. A memory metric extracted from a state-space approximation of the LSM has been proposed in the past and empirically shown to be best-in-class for performance prediction. However, the working principle of this memory metric has not been studied. We first show equivalence of LSM simulated on MATLAB to those run on Intel’s neuromorphic chip Loihi. This enables us to perform in-depth statistical analysis of the memory metric on Loihi: effect of weight scaling and effect of time averaging window. Analysis of state space matrices generated with a reasonably sized averaging window reveal that the diagonal elements are sufficient to capture network dynamics. This strengthens the relevance of the first order decay constant based memory metric which correlates well with the classification performance.
Unsupervised Domain Adaptation Using Temporal Association for Segmentation and Its Application to C. elegans Time-Lapse Images

With the widespread use of live-cell imaging technologies for observing the dynamics of live cells under a microscope, there is a need for semantic image segmentation to quantitatively detect structural changes of the targeting objects in the various resulting time-lapse images. Although supervised learning of convolutional neural networks such as U-Net is one way to achieve segmentation, pixel-level labeling for images is a time-consuming task and requires domain knowledge. Unsupervised domain adaptation (UDA) methods have been proposed to transfer the learning knowledge gained from labeled datasets (i.e., the source domain) to unlabeled datasets (i.e., the target domain). However, to date, there has been no reported application of a UDA method to consider the temporal association of features with the aim of using semantic segmentation for time-lapse microscopy images. This paper proposes a new UDA method that considers the class-specific feature distribution in both the source and target domains as well as the temporal association of features. We also present manually created annotations of true nuclear regions for two kinds of open time-lapse image datasets of Caenorhabditis elegans early embryos. Using the annotated datasets, we demonstrate that the accuracy of the proposed method is more than 7% higher than that of other UDA methods with static images and videos as inputs. The code and annotations are available at https://github.com/tohsato-lab/T-MCD .
Context Reasoning Attention Network: Generating Plausible Distractors for Multi-choice Questions

Distractor generation, which aims to generate the wrong option part of multi-choice questions, has been proposed to assist educators to test the examinees’ reading comprehension and reasoning ability. Recently, some Seq2Seq-based models have been proposed to solve the task of automatic distractor generation. However, they did not make full use of context information to generate distractors. In order to overcome this shortcoming, we propose a context reasoning attention network for distractor generation. Experimental results show that our model outperforms state-of-the-art baselines and improves the distractive ability of the generated distractors in terms of automatic evaluation and human evaluation.
Construction of a Prediction Model for Distance Education Quality Assessment Based on Convolutional Neural Network

This paper introduces the principles and operation steps of convolution and pooling of convolutional neural networks in detail. In view of the shortcomings of fixed sampling points and single receptive field in traditional convolution and pooling forms, deformable convolution and deformable pooling are introduced to enhance the network's ability to adapt to image details and large displacement problems. The concepts of warp, loop optimization, and network stack are introduced. In order to improve the optimization performance of the algorithm, three subnetwork structures and stack models are designed, and various methods are used to improve the prediction accuracy of distance education quality assessment. In order to improve the accuracy and timeliness of education quality assessment, this paper proposes a distance education quality assessment model based on mining algorithms. The prediction index is selected by the improved BP neural network. It is required to establish the input layer node as the input vector based on the number of data sources since the input layer is used for data input. The neural network is trained with a quarter of the mining data, and the mining algorithm is further trained with network error trials. A fuzzy relationship matrix is created based on the assessment of teaching quality's hierarchical structure. This leads to the conclusion of the fuzzy thorough evaluation of the effectiveness of distant learning. Experiments show that the proposed model has an average accuracy of 96%, the average teaching quality modeling time is 25.44 ms, and the evaluation speed is fast.
BPGG: Bidirectional Prototype Generation and Guidance Network for Few-Shot Anomaly Localization

Few-shot anomaly localization task is pixel-level detection of unseen images with only a tiny amount of anomaly training samples. Bound by reality, most conventional training data are defect-free, and models are difficult to accommodate various anomaly types. To this end, we propose a bidirectional prototype generation and guidance network (BPGG), which implements non-parametric metric learning with the help of prototypes. We first trade the position of the support set and query set to construct the adaptive reverse branch. The bidirectional branch structure forces the support set and query set to align with each other and build a consistent metric space. For leveraging the benefits of regular data, we also insert the normal images into the support set and balance the proportion of normal and defective samples. Our experimental study on the MVTec anomaly detection dataset demonstrates that our proposed algorithm outperforms current few-shot SOTA methods, comparable to other unsupervised and self-supervised algorithms. Besides, our BPGG Network is general to detect various types of real-world defects and perform stable detection. The rational utilization of data and innovative architecture in our study provide a novel breakthrough for the task of anomaly location.
ASTra: A Novel Algorithm-Level Approach to Imbalanced Classification

We propose a novel output layer activation function, which we name ASTra (Asymmetric Sigmoid Transfer function), which makes the classification of minority examples, in scenarios of high imbalance, more tractable. We combine this with a loss function that helps to effectively target minority misclassification. These two methods can be used together or separately, with their combination recommended for the most severely imbalanced cases. The proposed approach is tested on datasets with IRs from 588.24 to 4000 and very few minority examples (in some datasets, as few as five). Results using neural networks with from two to 12 hidden units are demonstrated to be comparable to, or better than, equivalent results obtained in a recent study that deployed a wide range of complex, hybrid data-level ensemble classifiers.
Semantic Diversity Image Translation Based on Deep Feature Difference and Attention Mechanism

To improve the semantic diversity and visual authenticity of image translation, in this paper we propose Generative Adversarial Networks based on deep feature difference and attention mechanism. In our model, we employ a pre-trained image classification network to extract different level features of the generated images and then calculate the perceptual similarity based on these features. Such perceptual similarities are introduced into the adversarial learning to control the differences in high-level semantic features. Furthermore, in order to ensure the quality of the translated image, the residual network structure with the attention mechanism is utilized in our model. According to the importance of each feature channel, the different attention is strengthened on the useful features. In addition, we not only employ the discriminator on the generated images to discriminate whether the images are real or fake, but also make a discrimination on the reconstructed images. The main contributions of this paper are as follows: (1) Previous studies on the diversity of image translation mainly focus on the low-level features of the image, such as hair color and skin color. Our method focuses on semantic differences through deep features, with the aim of improving the semantic diversity of images in the styles. (2) Our model adopts the channel-attention mechanism and reconstructed-image discriminator, which not only highlight the feature details of the generated image, but also ensure the translated images as real as possible. The results on Celeba-HQ dataset verify the superiority of our model on both visual quality and semantic diversity.