Reviewerly/exHarmony huggingface dataset:
Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis

Camera captured human pose is an outcome of several sources of variation. Performance of supervised 3D pose estimation approaches comes at the cost of dispensing with variations, such as shape and appearance, that may be useful for solving other related tasks. As a result, the learned model not only inculcates task-bias but also dataset-bias because of its strong reliance on the annotated samples, which also holds true for weakly-supervised models. Acknowledging this, we propose a self-supervised learning framework to disentangle such variations from unlabeled video frames. We leverage the prior knowledge on human skeleton and poses in the form of a single part-based 2D puppet model, human pose articulation constraints, and a set of unpaired 3D poses. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, not only facilitates discovery of interpretable pose disentanglement, but also allows us to operate on videos with diverse camera movements. Qualitative results on unseen in-the-wild datasets establish our superior generalization across multiple tasks beyond the primary tasks of 3D pose estimation and part segmentation. Furthermore, we demonstrate state-of-the-art weakly-supervised 3D pose estimation performance on both Human3.6M and MPI-INF-3DHP datasets.
Semantic segmentation using stride spatial pyramid pooling and dual attention decoder

Semantic segmentation is an end-to-end task that requires both semantic and spatial accuracy. It is important for deep learning-based segmentation methods to effectively utilize the high-level feature map whose semantic information is abundant and the low-level feature map whose spatial information is accurate. However, existing segmentation networks typically cannot take full advantage of these two kinds of feature maps, leading to inferior performance. This paper attempts to overcome this challenge by introducing two novel structures. On the one hand, we propose a structure called stride spatial pyramid pooling (SSPP) to capture multiscale semantic information from the high-level feature map. Compared with existing pyramid pooling methods based on the atrous convolution, the SSPP structure is able to gather more information from the high-level feature map with faster inference speed, which improves the utilization efficiency of the high-level feature map significantly. On the other hand, we propose a dual attention decoder consisting of a channel attention branch and a spatial attention branch to make full use of the high- and low-level feature maps simultaneously. The dual attention decoder can result in a more “semantic” low-level feature map and a high-level feature map with more accurate spatial information, which bridges the gap between these two kinds of feature maps and benefits their fusion. We evaluate the proposed model on several publicly available semantic image segmentation benchmarks including PASCAL VOC 2012, Cityscapes and COCO-Stuff. The qualitative and quantitative results demonstrate that our method can achieve the state-of-the-art performance.
Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentences

In this paper, we consider a novel task, Spatio-Temporal Video Grounding for Multi-Form Sentences (STVG). Given an untrimmed video and a declarative/interrogative sentence depicting an object, STVG aims to localize the spatio-temporal tube of the queried object. STVG has two challenging settings: (1) We need to localize spatio-temporal object tubes from untrimmed videos, where the object may only exist in a very small segment of the video; (2) We deal with multi-form sentences, including the declarative sentences with explicit objects and interrogative sentences with unknown objects. Existing methods cannot tackle the STVG task due to the ineffective tube pre-generation and the lack of object relationship modeling. Thus, we then propose a novel Spatio-Temporal Graph Reasoning Network (STGRN) for this task. First, we build a spatio-temporal region graph to capture the region relationships with temporal object dynamics, which involves the implicit and explicit spatial subgraphs in each frame and the temporal dynamic subgraph across frames. We then incorporate textual clues into the graph and develop the multi-step cross-modal graph reasoning. Next, we introduce a spatio-temporal localizer with a dynamic selection method to directly retrieve the spatio-temporal tubes without tube pre-generation. Moreover, we contribute a large-scale video grounding dataset VidSTG based on video relation dataset VidOR. The extensive experiments demonstrate the effectiveness of our method.
Consensus Affinity Graph Learning for Multiple Kernel Clustering

Significant attention to multiple kernel graph-based clustering (MKGC) has emerged in recent years, primarily due to the superiority of multiple kernel learning (MKL) and the outstanding performance of graph-based clustering. However, many existing MKGC methods design a fat model that poses challenges for computational cost and clustering performance, as they learn both an affinity graph and an extra consensus kernel cumbersomely. To tackle this challenging problem, this article proposes a new MKGC method to learn a consensus affinity graph directly. By using the self-expressiveness graph learning and an adaptive local structure learning term, the local manifold structure of the data in kernel space is preserved for learning multiple candidate affinity graphs from a kernel pool first. After that, these candidate affinity graphs are synthesized to learn a consensus affinity graph via a thin autoweighted fusion model, in which a self-tuned Laplacian rank constraint and a top- k neighbors sparse strategy are introduced to improve the quality of the consensus affinity graph for accurate clustering purposes. The experimental results on ten benchmark datasets and two synthetic datasets show that the proposed method consistently and significantly outperforms the state-of-the-art methods.
Real-Time Moving Object Detection in High-Resolution Video Sensing

This paper addresses real-time moving object detection with high accuracy in high-resolution video frames. A previously developed framework for moving object detection is modified to enable real-time processing of high-resolution images. First, a computationally efficient method is employed, which detects moving regions on a resized image while maintaining moving regions on the original image with mapping coordinates. Second, a light backbone deep neural network in place of a more complex one is utilized. Third, the focal loss function is employed to alleviate the imbalance between positive and negative samples. The results of the extensive experimentations conducted indicate that the modified framework developed in this paper achieves a processing rate of 21 frames per second with 86.15% accuracy on the dataset SimitMovingDataset, which contains high-resolution images of the size 1920 × 1080.
Adolescent Identity Search Algorithm (AISA): A novel metaheuristic approach for solving optimization problems

This paper proposes a novel population-based metaheuristic optimization algorithm, called Adolescent Identity Search Algorithm (AISA), which is inspired by the process of identity development/search of adolescents. AISA simulates the identity formation behavior of adolescents in the peer group. This behavior is modeled mathematically to solve optimization problems. The proposed algorithm is evaluated on thirty-nine well-known unimodal, multimodal, fixed-dimensional multimodal, composite and CEC 2019 benchmark functions to test exploration, exploitation, local optima avoidance, and convergence properties. The results are verified by an extensive comparative study with thirteen state-of-art metaheuristic algorithms. Furthermore, AISA has been used to solve IIR system identification and inverse kinematics problem of a seven Degrees Of Freedom (7-DOF) robot manipulator considered as the real-life engineering applications. The overall optimization results demonstrate that AISA possesses a strong and robust capability to produce superior performance over other competitor metaheuristic algorithms in solving various complex numerical optimization problems.
Magnetic resonance-based synthetic computed tomography images generated using generative adversarial networks for nasopharyngeal carcinoma radiotherapy treatment planning

Background and purposeTo investigate the feasibility of synthesizing computed tomography (CT) images from magnetic resonance (MR) images using generative adversarial networks (GANs) for nasopharyngeal carcinoma (NPC) intensity-modulated radiotherapy (IMRT) planning.Materials and methodsConventional T1-weighted MR images and CT images were acquired from 173 NPC patients. The MR and CT images of 28 patients were randomly chosen as the independent tested set. The remaining images were used to build a conditional GAN (cGAN) and a cycle-consistency GAN (cycleGAN). A U-net was used as the generator in cGAN, whereas a residual-Unet was used as the generator in cycleGAN. The cGAN was trained using the deformable registered MR-CT image pairs, whereas the cycleGAN was trained using the unregistered MR and CT images. The generated synthetic CT (SCT) images from cGAN and cycleGAN were compared with the true CT images with respect to their Hounsfield Unit (HU) discrepancy and dosimetric accuracy for NPC IMRT plans.ResultsThe mean absolute errors within the body were 69.67 ± 9.27 HU and 100.62 ± 7.39 HU for the cGAN and cycleGAN, respectively. The 2%/2-mm γ passing rates were (98.68 ± 0.94)% and (98.52 ± 1.13)% for the cGAN and cycleGAN, respectively. Meanwhile, the absolute dose discrepancies within the regions of interest were (0.49 ± 0.24)% and (0.62 ± 0.36)%, respectively.ConclusionBoth cGAN and cycleGAN could swiftly generate accurate SCT volume images from MR images, with high dosimetric accuracy for NPC IMRT planning. cGAN was preferable if high-quality MR-CT image pairs were available.
More time for science: Using Testable to create and share behavioral experiments faster, recruit better participants, and engage students in hands-on research

A major pain for researchers in all fields is that they have less and less time for actual science activities: reading, thinking, coming up with new theories and hypotheses, testing, analyzing data, writing. In psychology, three of the most time-consuming nonactual science activities are: learning how to program an experiment, recruiting participants, and preparing teaching materials. Testable (www.testable.org) provides a suite of academic tools to speed things up considerably. The Testable software allows the development of most psychology experiments in minutes, using a natural language form and a spreadsheet. Furthermore, any experiment can be easily converted into a social experiment in Testable Arena, with multiple participants interacting and viewing each other's responses. Experiments can then be published to Testable Library, a public repository for demonstration and sharing purposes. Participants can be recruited from Testable Minds, the subject pool with the most advanced participants verification system. Testable Minds employs multiple checks (such as face authentication) to ensure participants have accurate demographics (age, sex, location), are human, unique, and reliable. Finally, the Testable Class module can be used to teach psychology through experiments. It features over 50 ready-made classic psychology experiments, fully customizable, which instructors can add to their classes, together with their own experiments. These experiments can then be made available to students to do, import, modify, and use to collect data as part of their class. These Testable tools, backed up by a strong team of academic advisors and thousands of users, can save psychology researchers and other behavioral scientists valuable time for science.
Secure Image Encryption Algorithm Based on Hyperchaos and Dynamic DNA Coding

In this paper, we construct a five dimensional continuous hyperchaotic system and propose an image encryption scheme based on the hyperchaotic system, which adopts DNA dynamic coding mechanism and classical scrambling diffusion encryption structure. In the diffusion stage, two rounds of diffusion are adopted and the rules of DNA encoding (DNA decoding) are dynamically changed according to the pixel value of the plaintext image, that is, the rules of DNA encoding (DNA decoding) used to encrypt different images are different, which makes the algorithm can resist chosen-plaintext attack. The encryption (decryption) key is only the initial value of the chaotic system, which overcomes the difficulty of key management in the “one time pad” encryption system. The experimental results and security analysis show that the algorithm has some advantages of large key space, no obvious statistical characteristics of ciphertext, sensitivity to plaintext and key and able to resist differential attacks and chosen plaintext attack. It has good application prospects.
Consumer decisions with artificially intelligent voice assistants

Abstract Consumers are widely adopting Artificially Intelligent Voice Assistants (AIVAs). AIVAs now handle many different everyday tasks and are also increasingly assisting consumers with purchasing decisions, making AIVAs a rich topic for marketing researchers. We develop a series of propositions regarding how consumer decision-making processes may change when moved from traditional online purchase environments to AI-powered voice-based dialogs, in the hopes of encouraging further academic thinking and research in this rapidly developing, high impact area of consumer-firm interaction. We also provide suggestions for marketing managers and policymakers on points to pay attention to when they respond to the proposed effects of AIVAs on consumer decisions.
Human–computer interaction-based Decision Support System with Applications in Data Mining

Human–computer interaction (HCI) plays a vital role in modern intelligent systems, such as brain–machine integration, human action recognition, telemedicine, and somatosensory game. A decision support system is a combination of the information system and decision-making technology. Visual human–computer interaction decision-making is a key technique in the decision support system. This paper proposes a new visual decision-making system applicable for industrial applications, i.e. data mining topics. To assist the performance of the proposed visual decision-making system, it is designed for data mining technique applications. Furthermore, the architecture of the decision support system is analyzed based on practical data mining case study. The comprehensive experiment shows that the proposed method is effective and robust in comparison to other methods.
Enhancing artificial bee colony algorithm with multi-elite guidance

Artificial bee colony (ABC) algorithm is a relatively new paradigm of swarm intelligence based optimization technique, which has attracted a lot of attention for its simple structure and good performance. For some complex optimization problems, however, the performance of ABC is challenged due to its solution search equation that has strong explorative ability but poor exploitative ability. To solve this defect, in this work, we propose an improved ABC algorithm by using multi-elite guidance, which has the benefits of utilizing valuable information from elite individuals to guide search while without losing population diversity. First, we construct an elite group by selecting some elite individuals, and then introduce two improved solution search equations into the employed bee phase and onlooker bee phase based on the elite group, respectively. Last, we develop a modified neighborhood search operator by utilizing the elite group as well, which aims to achieve a better tradeoff between explorative and exploitative abilities. To verify our approach, 50 well-known test functions and one real-world optimization problem are used in the experiments, including 22 scalable basic test functions and 28 complex CEC2013 test functions. Seven different well-established ABC variants are involved in the comparison and the results show that our approach can achieve better or at least comparable performance on most of the test functions.
Cross-modal knowledge reasoning for knowledge-based visual question answering

Knowledge-based Visual Question Answering (KVQA) requires external knowledge beyond the visible content to answer questions about an image. This ability is challenging but indispensable to achieve general VQA. One limitation of existing KVQA solutions is that they jointly embed all kinds of information without fine-grained selection, which introduces unexpected noises for reasoning the correct answer. How to capture the question-oriented and information-complementary evidence remains a key challenge to solve the problem. Inspired by the human cognition theory, in this paper, we depict an image by multiple knowledge graphs from the visual, semantic and factual views. Thereinto, the visual graph and semantic graph are regarded as image-conditioned instantiation of the factual graph. On top of these new representations, we re-formulate Knowledge-based Visual Question Answering as a recurrent reasoning process for obtaining complementary evidence from multimodal information. To this end, we decompose the model into a series of memory-based reasoning steps, each performed by a G raph-based R ead, U pdate, and C ontrol ( GRUC ) module that conducts parallel reasoning over both visual and semantic information. By stacking the modules multiple times, our model performs transitive reasoning and obtains question-oriented concept representations under the constrain of different modalities. Finally, we perform graph neural networks to infer the global-optimal answer by jointly considering all the concepts. We achieve a new state-of-the-art performance on three popular benchmark datasets, including FVQA, Visual7W-KB and OK-VQA, and demonstrate the effectiveness and interpretability of our model with extensive experiments.
A Deep Learning Model Based on Multi-Objective Particle Swarm Optimization for Scene Classification in Unmanned Aerial Vehicles

Recently, the increase in inexpensive and compact unmanned aerial vehicles (UAVs) and light-weight imaging sensors has led to an interest in using them in various remote sensing applications. The processes of collecting, calibrating, registering, and processing data from miniature UAVs and interpreting the data semantically are time-consuming. In UAV aerial imagery, learning effective image representations is central to the scene classification process. Earlier approaches to the scene classification process depended on feature coding methods with low-level hand-engineered features or unsupervised feature learning. These methods could produce mid-level image features with restricted representational abilities, which generally yielded mediocre results. The development of convolutional neural networks (CNNs) has made image classification more efficient. Due to the limited resources in UAVs, it is hard to fine-tune the hyperparameters and the trade-offs between classifier results and computation complexity. This paper introduces a new multi-objective optimization model for evolving state-of-the-art deep CNNs for scene classification, which generates the non-dominant solutions in an automated way at the Pareto front. We use a set of two benchmark datasets to test the performance of the scene classification model and make a detailed comparative study. The proposed method attains a very low computational time of 80 sec and maximum accuracy of 97.88% compared to all other methods. The proposed method is found to be appropriate for the effective scene classification of images captured by UAVs.
Towards Cognitive Recommender Systems

Intelligence is the ability to learn from experience and use domain experts’ knowledge to adapt to new situations. In this context, an intelligent Recommender System should be able to learn from domain experts’ knowledge and experience, as it is vital to know the domain that the items will be recommended. Traditionally, Recommender Systems have been recognized as playlist generators for video/music services (e.g., Netflix and Spotify), e-commerce product recommenders (e.g., Amazon and eBay), or social content recommenders (e.g., Facebook and Twitter). However, Recommender Systems in modern enterprises are highly data-/knowledge-driven and may rely on users’ cognitive aspects such as personality, behavior, and attitude. In this paper, we survey and summarize previously published studies on Recommender Systems to help readers understand our method’s contributions to the field in this context. We discuss the current limitations of the state of the art approaches in Recommender Systems and the need for our new approach: A vision and a general framework for a new type of data-driven, knowledge-driven, and cognition-driven Recommender Systems, namely, Cognitive Recommender Systems. Cognitive Recommender Systems will be the new type of intelligent Recommender Systems that understand the user’s preferences, detect changes in user preferences over time, predict user’s unknown favorites, and explore adaptive mechanisms to enable intelligent actions within the compound and changing environments. We present a motivating scenario in banking and argue that existing Recommender Systems: (i) do not use domain experts’ knowledge to adapt to new situations; (ii) may not be able to predict the ratings or preferences a customer would give to a product (e.g., loan, deposit, or trust service); and (iii) do not support data capture and analytics around customers’ cognitive activities and use it to provide intelligent and time-aware recommendations.
Generative Feature Replay For Class-Incremental Learning

Humans are capable of learning new tasks without forgetting previous ones, while neural networks fail due to catastrophic forgetting between new and previously-learned tasks. We consider a class-incremental setting which means that the task-ID is unknown at inference time. The imbalance between old and new classes typically results in a bias of the network towards the newest ones. This imbalance problem can either be addressed by storing exemplars from previous tasks, or by using image replay methods. However, the latter can only be applied to toy datasets since image generation for complex datasets is a hard problem.We propose a solution to the imbalance problem based on generative feature replay which does not require any exemplars. To do this, we split the network into two parts: a feature extractor and a classifier. To prevent forgetting, we combine generative feature replay in the classifier with feature distillation in the feature extractor. Through feature generation, our method reduces the complexity of generative replay and prevents the imbalance problem. Our approach is computationally efficient and scalable to large datasets. Experiments confirm that our approach achieves state-of-the-art results on CIFAR-100 and ImageNet, while requiring only a fraction of the storage needed for exemplar-based continual learning. Code available at https://github.com/xialeiliu/GFR-IL.