Reviewerly/exHarmony huggingface dataset:
Sex differences in emoji use, familiarity, and valence

Emojis (particularly smiley emojis, ☺) are increasingly used in computer-mediated communication as well as in applied domains within marketing, healthcare, and psychology. The emotional negativity bias in the facial emotion processing literature posits that women are more sensitive to negative facial emotion than are men. Given the similarity in neural processing between human faces and smiley emojis, women may likewise view negative smiley emojis as more negative than do men. Moreover, the familiarity of the emoji and the participants' overall emoji use may increase the positivity of the emoji. To investigate these potential influences of sex, familiarity, and emoji use on the valence of smiley emojis, we assessed the familiarity and the perceived valence for 70 iOS facial emojis in a large sample (N = 299; 163 women) of United States college students (Mage = 19.66, SDage = 2.72). Results indicated higher emoji usage and familiarity ratings for women than for men. In assessing valence we found higher overall positive ratings for men than for women. Consistent with the emotional negativity bias, this sex difference was limited to the negative smiley emojis with no sex difference in valence for the positive emojis. The obtained sex differences in smiley emojis’ use, familiarity, and valence are an important consideration in the selection of such stimuli in future studies.
A Multi-Stream Graph Convolutional Networks-Hidden Conditional Random Field Model for Skeleton-Based Action Recognition

Recently, Graph Convolutional Network(GCN) methods for skeleton-based action recognition have achieved great success due to their ability to preserve structural information of the skeleton. However, these methods abandon the structural information in the classification stage by employing traditional fully-connected layers and softmax classifier, leading to sub-optimal performance. In this work, a novel Graph Convolutional Networks-Hidden conditional Random Field (GCN-HCRF) model is proposed to solve this problem. The proposed method combines GCN with HCRF to retain the human skeleton structure information even during the classification stage. Our model is trained end-to-end by utilizing the message passing from the belief propagation algorithm on the human structure graph. To further capture spatial and temporal information, we propose a multi-stream framework which takes the relative coordinate of the joints and bone direction as two static feature streams, and the temporal displacements between two consecutive frames as the dynamic feature stream. Experimental results on three challenging benchmarks (NTU RGB+D, N-UCLA, SYSU) show the superior performance of the proposed model over state-of-the-art models.
Adaptive iterative attack towards explainable adversarial robustness

Image classifiers based on deep neural networks show severe vulnerability when facing adversarial examples crafted on purpose. Designing more effective and efficient adversarial attacks is attracting considerable interest due to its potential contribution to interpretability of deep learning and validation of neural networks’ robustness. However, current iterative attacks use a fixed step size for each noise-adding step, making further investigation into the effect of variable step size on model robustness ripe for exploration. We prove that if the upper bound of noise added to the original image is fixed, the attack effect can be improved if the step size is positively correlated with the gradient obtained at each step by querying the target model. In this paper, we propose Ada-FGSM (Adaptive FGSM), a new iterative attack that adaptively allocates step size of noises according to gradient information at each step. Improvement of success rate and accuracy decrease measured on ImageNet with multiple models emphasizes the validity of our method. We analyze the process of iterative attack by visualizing their trajectory and gradient contour, and further explain the vulnerability of deep neural networks to variable step size adversarial examples.
Learning motion representation for real-time spatio-temporal action localization

The current deep learning based spatio-temporal action localization methods that using motion information (predominated is optical flow) obtain the state-of-the-art performance. However, since the optical flow is pre-computed, leading to these methods face two problems – the computational efficiency is low and the whole network is not end-to-end trainable. We propose a novel spatio-temporal action localization approach with an integrated optical flow sub-network to address these two issues. Specifically, our designed flow subnet can estimate optical flow efficiently and accurately by using multiple consecutive RGB frames rather than two adjacent frames in a deep network, simultaneously, action localization is implemented in the same network interactive with flow computation end-to-end. To faster the speed, we exploit a neural network based feature fusion method in a pyramid hierarchical manner. It fuses spatial and temporal features at different granularities via combination function (i.e. concatenation) and point-wise convolution to obtain multiscale spatio-temporal action features. Experimental results on three publicly available datasets, e.g. UCF101-24, JHMDB and AVA show that with both RGB appearance and optical flow cues, the proposed method gets the state-of-the-art performance in both efficiency and accuracy. Noticeably, it gets a significant improvement on efficiency. Compared to the currently most efficient method, it is 1.9 times faster in the running speed and 1.3% video-mAP more accurate on the UCF101-24. Our proposed method reaches real-time computation for the first time (up to 38 FPS).
Revenue and Energy Cost-Optimized Biobjective Task Scheduling for Green Cloud Data Centers

The significant growth in the number and types of tasks of heterogeneous applications in green cloud data centers (GCDCs) dramatically increases their providers' revenue from users as well as energy consumption. It is a big challenge to maximize such revenue, while minimizing energy cost in a market where prices of electricity, availability of renewable power generation, and behind-the-meter renewable generation contract models differ among the geographical sites of the GCDCs. A multiobjective optimization method that investigates such spatial differences in the GCDCs is for the first time proposed to trade off such two objectives by cost-effectively executing all tasks while meeting their delay constraints. In each time slot, a constrained biobjective optimization problem is formulated and solved by an improved multiobjective evolutionary algorithm based on decomposition. Realistic data-based simulations prove that the proposed method achieves a larger total profit in faster convergence speed than the two state-of-the-art algorithms. Note to Practitioners-This article considers the tradeoff between profit maximization and energy cost minimization for the green cloud data center (GCDC) providers while meeting the delay constraints of all tasks. Current task-scheduling methods fail to take the advantage of spatial variations in many factors, e.g., prices of electricity and availability of renewable power generation at geographically distributed GCDC locations. As a result, they fail to execute all tasks of heterogeneous applications within their delay bounds in a high-revenue and low-energy-cost manner. In this article, a multiobjective optimization method that addresses the disadvantages of the existing methods is proposed. It is realized by a proposed intelligent optimization algorithm. Simulations demonstrate that in comparison with the two state-of-the-art scheduling algorithms, the proposed one increases the profit and reduces the convergence time. It can be readily implemented and integrated into actual industrial GCDCs.
ZoomCount: A Zooming Mechanism for Crowd Counting in Static Images

This paper proposes a novel approach for crowd counting in low to high density scenarios in static images. Current approaches cannot handle huge crowd diversity well and thus perform poorly in extreme cases, where the crowd density in different regions of an image is either too low or too high, leading to crowd underestimation or overestimation. The proposed solution is based on the observation that detecting and handling such extreme cases in a specialized way leads to better crowd estimation. Additionally, existing methods find it hard to differentiate between the actual crowd and the cluttered background regions, resulting in further count overestimation. To address these issues, we propose a simple yet effective modular approach, where an input image is first subdivided into fixed-size patches and then fed to a four-way classification module labeling each image patch as low, medium, high-dense or no-crowd. This module also provides a count for each label, which is then analyzed via a specifically devised novel decision module to decide whether the image belongs to any of the two extreme cases (very low or very high density) or a normal case. Images, specified as high- or low-density extreme or a normal case, pass through dedicated zooming or normal patch-making blocks respectively before routing to the regressor in the form of fixed-size patches for crowd estimate. Extensive experimental evaluations demonstrate that the proposed approach outperforms the state-of-the-art methods on four benchmarks under most of the evaluation criteria.
DeOccNet: Learning to See Through Foreground Occlusions in Light Fields

Background objects occluded in some views of a light field (LF) camera can be seen by other views. Consequently, occluded surfaces are possible to be reconstructed from LF images. In this paper, we handle the LF de-occlusion (LF-DeOcc) problem using a deep encoder-decoder network (namely, DeOccNet). In our method, sub-aperture images (SAIs) are first given to the encoder to incorporate both spatial and angular information. The encoded representations are then used by the decoder to render an occlusion-free center-view SAI. To the best of our knowledge, DeOccNet is the first deep learning-based LF-DeOcc method. To handle the insufficiency oftraining data, we propose an LF synthesis approach to embed selected occlusion masks into existing LF images. Besides, several synthetic and real-world LFs are developed for performance evaluation. Experimental results show that, after training on the generated data, our DeOccNet can effectively remove foreground occlusions and achieves superior performance as compared to other state-of-the-art methods. Source codes are available at: https://github.com/YingqianWang/DeOccNet.
Targeted Mismatch Adversarial Attack: Query With a Flower to Retrieve the Tower

Access to online visual search engines implies sharing of private user content - the query images. We introduce the concept of targeted mismatch attack for deep learning based retrieval systems to generate an adversarial image to conceal the query image. The generated image looks nothing like the user intended query, but leads to identical or very similar retrieval results. Transferring attacks to fully unseen networks is challenging. We show successful attacks to partially unknown systems, by designing various loss functions for the adversarial image construction. These include loss functions, for example, for unknown global pooling operation or unknown input resolution by the retrieval system. We evaluate the attacks on standard retrieval benchmarks and compare the results retrieved with the original and adversarial image.
SiamFPN: A Deep Learning Method for Accurate and Real-Time Maritime Ship Tracking

Visual object tracking plays an essential role in various maritime applications. However, most of the existing tracking methods belong to generative models, which only focus on the features of the object and require the target has significant visual saliency for accurate tracking. While the visual saliency is available in most of the common tracking conditions, these methods may fail when facing challenging situations. In this paper, a deep learning based tracking method is proposed to track maritime ships, namely, SiamFPN. In SiamFPN, a modified Siamese Network is combined with multi-RPNs to build a tracking pipeline. Concretely, A ResNet-50 with an FPN structure is used as the CNN of the detection subnetwork of Siamese, and a template subnetwork is parallel to the detection. In order to strengthen the discriminative ability, three RPNs are deployed to process the output of Siamese Network. Moreover, a historical impacts based proposal selection method is developed for selecting correct target areas. Finally, a dataset is collected for training and testing SiamFPN and validating our excellent performance over the other four recent SOTA trackers. Based on the experimental results, we achieved 74 % on average accuracy with real-time speed.
Improving breast mass classification by shared data with domain transformation using a generative adversarial network

Training of a convolutional neural network (CNN) generally requires a large dataset. However, it is not easy to collect a large medical image dataset. The purpose of this study is to investigate the utility of synthetic images in training CNNs and to demonstrate the applicability of unrelated images by domain transformation. Mammograms showing 202 benign and 212 malignant masses were used for evaluation. To create synthetic data, a cycle generative adversarial network was trained with 599 lung nodules in computed tomography (CT) and 1430 breast masses on digitized mammograms (DDSM). A CNN was trained for classification between benign and malignant masses. The classification performance was compared between the networks trained with the original data, augmented data, synthetic data, DDSM images, and natural images (ImageNet dataset). The results were evaluated in terms of the classification accuracy and the area under the receiver operating characteristic curves (AUC). The classification accuracy improved from 65.7% to 67.1% with data augmentation. The use of an ImageNet pretrained model was useful (79.2%). Performance was slightly improved when synthetic images or the DDSM images only were used for pretraining (67.6 and 72.5%, respectively). When the ImageNet pretrained model was trained with the synthetic images, the classification performance slightly improved (81.4%), although the difference in AUCs was not statistically significant. The use of the synthetic images had an effect similar to the DDSM images. The results of the proposed study indicated that the synthetic data generated from unrelated lesions by domain transformation could be used to increase the training samples.
Expanding Modes of Reflection in Design Futuring

Design futuring approaches, such as speculative design, design fiction and others, seek to (re)envision futures and explore alternatives. As design futuring becomes established in HCI design research, there is an opportunity to expand and develop these approaches. To that end, by reflecting on our own research and examining related work, we contribute five modes of reflection. These modes concern formgiving, temporality, researcher positionality, real-world engagement, and knowledge production. We illustrate the value of each mode through careful analysis of selected design exemplars and provide questions to interrogate the practice of design futuring. Each reflective mode offers productive resources for design practitioners and researchers to articulate their work, generate new directions for their work, and analyze their own and others' work.
Deterministic chaos game: A new fractal based pseudo-random number generator and its cryptographic application

In this paper, a digital image encryption algorithm is proposed based on the generalized model of the chaos game. The chaos game is a well-known fractal, which acts as a pseudo-random number generator (PRNG) in the proposed encryption algorithm. The foundation of the chaos game is based on basic points and its distance ratio that determine the basis of how they distribute random values in 2D or 3D space. These basic points are entered by the user interface and are the result of an encrypted image with a fractal structure. The use of the bifurcation diagram and Lyapunov exponent analysis showed that the proposed chaos game has the dynamical behavior, and fully chaotic characteristic, and can be used as a secure PRNG in cryptography systems. In the proposed method, the region of interest is determined by a number of Bases, and the fractal mechanism of chaos game for the encryption process is performed on the image. This process is very sensitive to any changes in keys and refers to confusion. The evaluation results of security and performance analysis on standard images confirm the efficiency of the proposed method and demonstrate that the proposed method is robust against attacks.
A New Fusion Estimation Method for Multi-Rate Multi-Sensor Systems With Missing Measurements

A new fusion strategy is introduced in this article to estimate state for multi-rate multi-sensor systems with missing measurements. N sensors, which possess various sampling rates, render the measurements. Missing measurements with a certain probability pattern are also investigated. For these types of systems, Multi-rate Kalman filters are designed to estimate a target position at various sampling rates. Next, Ordered Weighted Averaging (OWA) operator is utilized to integrate multi-rate Kalman filters and improve the estimation quality. A new fusion strategy based on a real covariance matrix is introduced for updating the weighting factors, and proof of convergence is granted. Simulation studies on a tracking system verify the superior performance of the proposed fusion strategy in comparison with the Kalman filter, the multi-rate Kalman filters, and also the previous fusion methodology.
Illumination-Invariant Flotation Froth Color Measuring via Wasserstein Distance-Based CycleGAN With Structure-Preserving Constraint

Froth color can be referred to as a direct and instant indicator to the key flotation production index, for example, concentrate grade. However, it is intractable to measure the froth color robustly due to the adverse interference of time-varying and uncontrollable multisource illuminations in the flotation process monitoring. In this article, we proposed an illumination-invariant froth color measuring method by solving a structure-preserved image-to-image color translation task via an introduced Wasserstein distance-based structure-preserving CycleGAN, called WDSPCGAN. WDSPCGAN is comprised of two generative adversarial networks (GANs), which have their own discriminators but share two generators, using an improved U-net-like full convolution network to conduct the spatial structure-preserved color translation. By an adversarial game training of the two GANs, WDSPCGAN can map the color domain of froth images under any illumination to that of the referencing illumination, while maintaining the structure and texture invariance. The proposed method is validated on two public benchmark color constancy datasets and applied to an industrial bauxite flotation process. The experimental results show that WDSPCGAN can achieve illumination-invariant color features of froth images under various unknown lighting conditions while keeping their structures and textures unchanged. In addition, WDSPCGAN can be updated online to ensure its adaptability to any operational conditions. Hence, it has the potential for being popularized to the online monitoring of the flotation concentrate grade.
Friend or Faux: Graph-Based Early Detection of Fake Accounts on Social Networks

In this paper, we study the problem of early detection of fake user accounts on social networks based solely on their network connectivity with other users. Removing such accounts is a core task for maintaining the integrity of social networks, and early detection helps to reduce the harm that such accounts inflict. However, new fake accounts are notoriously difficult to detect via graph-based algorithms, as their small number of connections are unlikely to reflect a significant structural difference from those of new real accounts. We present the SybilEdge algorithm, which determines whether a new user is a fake account (‘sybil’) by aggregating over (I) her choices of friend request targets and (II) these targets’ respective responses. SybilEdge performs this aggregation giving more weight to a user’s choices of targets to the extent that these targets are preferred by other fakes versus real users, and also to the extent that these targets respond differently to fakes versus real users. We show that SybilEdge rapidly detects new fake users at scale on the Facebook network and outperforms state-of-the-art algorithms. We also show that SybilEdge is robust to label noise in the training data, to different prevalences of fake accounts in the network, and to several different ways fakes can select targets for their friend requests. To our knowledge, this is the first time a graph-based algorithm has been shown to achieve high performance (AUC > 0.9) on new users who have only sent a small number of friend requests.
Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases

With the rapid development of knowledge bases (KBs), link prediction task, which completes KBs with missing facts, has been broadly studied in especially binary relational KBs (a.k.a knowledge graph) with powerful tensor decomposition related methods. However, the ubiquitous n-ary relational KBs with higher-arity relational facts are paid less attention, in which existing translation based and neural network based approaches have weak expressiveness and high complexity in modeling various relations. Tensor decomposition has not been considered for n-ary relational KBs, while directly extending tensor decomposition related methods of binary relational KBs to the n-ary case does not yield satisfactory results due to exponential model complexity and their strong assumptions on binary relations. To generalize tensor decomposition for n-ary relational KBs, in this work, we propose GETD, a generalized model based on Tucker decomposition and Tensor Ring decomposition. The existing negative sampling technique is also generalized to the n-ary case for GETD. In addition, we theoretically prove that GETD is fully expressive to completely represent any KBs. Extensive evaluations on two representative n-ary relational KB datasets demonstrate the superior performance of GETD, significantly improving the state-of-the-art methods by over 15%. Moreover, GETD further obtains the state-of-the-art results on the benchmark binary relational KB datasets.
Adaptive Deep Cascade Broad Learning System and Its Application in Image Denoising

This article proposes a novel regularization deep cascade broad learning system (DCBLS) architecture, which includes one cascaded feature mapping nodes layer and one cascaded enhancement nodes layer. Then, the transformation feature representation is easily obtained by incorporating the enhancement nodes and the feature mapping nodes. Once such a representation is established, a final output layer is constructed by implementing a simple convex optimization model. Furthermore, a parallelization framework on the new method is designed to make it compatible with large-scale data. Simultaneously, an adaptive regularization parameter criterion is adopted under some conditions. Moreover, the stability and error estimate of this method are discussed and proved mathematically. The proposed method could extract sufficient available information from the raw data compared with the standard broad learning system and could achieve compellent successes in image denoising. The experiments results on benchmark datasets, including natural images as well as hyperspectral images, verify the effectiveness and superiority of the proposed method in comparison with the state-of-the-art approaches for image denoising.