Reviewerly/exHarmony huggingface dataset:
Unsupervised Robust Disentangling of Latent Characteristics for Image Synthesis

Deep generative models come with the promise to learn an explainable representation for visual objects that allows image sampling, synthesis, and selective modification. The main challenge is to learn to properly model the independent latent characteristics of an object, especially its appearance and pose. We present a novel approach that learns disentangled representations of these characteristics and explains them individually. Training requires only pairs of images depicting the same object appearance, but no pose annotations. We propose an additional classifier that estimates the minimal amount of regularization required to enforce disentanglement. Thus both representations together can completely explain an image while being independent of each other. Previous methods based on adversarial approaches fail to enforce this independence, while methods based on variational approaches lead to uninformative representations. In experiments on diverse object categories, the approach successfully recombines pose and appearance to reconstruct and retarget novel synthesized images. We achieve significant improvements over state-of-the-art methods which utilize the same level of supervision, and reach performances comparable to those of pose-supervised approaches. However, we can handle the vast body of articulated object classes for which no pose models/annotations are available.
TianGong-ST

Web search session data is precious for a wide range of Information Retrieval (IR) tasks, such as session search, query suggestion, click through rate (CTR) prediction and so on. Numerous studies have shown the great potential of considering context information for search system optimization. The well-known TREC Session Tracks have enhanced the development in this domain to a great extent. However, they are mainly collected via user studies or crowdsourcing experiments and normally contain only tens to thousands sessions, which are deficient for the investigation with more sophisticated models. To tackle this obstacle, we present a new dataset that contains 147,155 refined web search sessions with both click-based and human-annotated relevance labels. The sessions are sampled from a huge search log thus can reflect real search scenarios. The proposed dataset can support a wide range of session-level or task-based IR studies. As an example, we test several interactive search models with both the PSCM and human relevance labels provided by this dataset and report the performance as a reference for future studies of session search.
Jointly Learning of Visual and Auditory: A New Approach for RS Image and Audio Cross-Modal Retrieval

Remote sensing (RS) images are widely used in civilian and military fields. With the highly increasing image data, it has become a challenging issue to achieve fast and efficient RS image retrieval. However, the existing image retrieval methods, text-based or content-based, are still limited in the applications; for example, text input is inefficient, and the sample image for query is often unavailable. It is known that speech is a natural and convenient way of communication. Therefore, a novel speech-image cross-modal retrieval approach, named deep visual-audio network (DVAN), is presented in this article, which can establish the direct relationship between image and speech from paired image-audio data. The model mainly has three parts: 1) Image feature extraction, which is used to extract effective features of RS images; 2) audio feature learning, which is used to recognizing key information from raw data, and AudioNet, as part of DVAN, is proposed to obtain more distinguishing features; 3) multimodal embedding, which is used to learn the direct correlations of two modalities. Experimental results on RS image audio dataset demonstrate that the proposed method is effective and speech-image retrieval is feasible, and it provides a new way for faster and more convenient RS image retrieval.
Discriminative globality and locality preserving graph embedding for dimensionality reduction

Graph embedding in dimensionality reduction has attracted much attention in the high-dimensional data analysis. Graph construction in graph embedding plays an important role in the quality of dimensionality reduction. However, the discrimination information and the geometrical distributions of data samples are not fully exploited for discovering the essential geometrical and discriminant structures of data and strengthening the pattern discrimination in graph constructions of graph embedding. To overcome the limitations of graph constructions, in this article we propose a novel graph-based dimensionality reduction method entitled discriminative globality and locality preserving graph embedding (DGLPGE) by designing the informative globality and locality preserving graph constructions. In the constructed graphs, bidirectional weights of edges are newly defined by considering both the geometrical distributions of each point of edges and the class discrimination. Using the adjacent weights of graphs, we characterize the intra-class globality preserving scatter, the inter-class globality preserving scatter and the locality preserving scatter to formulate the objective function of DGLPGE in order to optimize the projection of dimensionality reduction. Extensive experiments demonstrate that the proposed DGLPGE often outperforms the state-of-the-art dimensionality reduction methods.
Reinforcement Neural Fuzzy Surrogate-Assisted Multiobjective Evolutionary Fuzzy Systems With Robot Learning Control Application

This paper proposes a new reinforcement neural fuzzy surrogate (RNFS)-assisted multiobjective evolutionary optimization (RNFS-MEO) algorithm to boost the learning efficiency of data-driven fuzzy controllers (FCs). The RNFS-MEO is applied to evolve a population of FCs in a multiobjective robot wall-following control problem in order to reduce the number of time-consuming control trials and the implementation time of learning. In the RNFS-MEO, the RNFS is incorporated into a typical multiobjective continuous ant colony optimization algorithm to improve its learning efficiency. The RNFS estimates the accumulated multiobjective function values of the FCs in a colony without applying them to control a process, which helps reduce the number of control trials. The RNFS is trained online through structure and parameter learning based on the reinforcement signals from controlling a process. Considering the influence of the current control signals on the future states of a controlled process, the temporal difference technique is used in the RNFS training so that it estimates not only the current but also the future objective function values. The colony of FCs in the RNFS-MEO is repeatedly evolved based on the RNFS estimated values or the objective function values from real evaluations until a colony of successful FCs is found. The RNFS-MEO-based FC learning approach is applied to a robot wall-following control problem. Simulations and experiments on the robot control application are performed to verify the effectiveness and efficiency of the RNFS-MEO.
Quantum Markovian master equations: Resonance theory shows validity for all time scales

Quantum systems coupled to environments exhibit intricate dynamics. The master equation gives a Markov approximation of the dynamics, allowing for analytic and numerical treatments. It is ubiquitous in theoretical and applied quantum sciences. The accuracy of the master equation approximation was so far proven for small values of the system–environment interaction coupling strength λ, under the additional constraint that time t must not exceed an upper bound, λ2t≤ constant. Here, we show that the Markov approximation is valid for fixed small coupling strength and for all times. We also construct a new approximate Markovian dynamics – a completely positive, trace preserving semigroup – which is asymptotically in time exact, to all orders in the coupling.
Deep Gabor convolution network for person re-identification

Person re-identification is an import problem in computer vision fields and more and more deep neural network models have been developed for representation learning in this task due to their good performance. However, compared with hand-crafted feature representations, deep learned features cannot not be interpreted easily. To meet this demand, motivated by the Gabor filters’ good interpretability and the deep neural network models’ reliable learning ability, we propose a new convolution module for deep neural networks based on Gabor function (Gabor convolution). Compared with classical convolution module, every parameter in the proposed Gabor convolution kernel has a specific meaning while classical one has not. The Gabor convolution module has a good texture representation ability and is effective when it is embedded in the low layers of a network. Besides, in order to make the proposed Gabor module meaningful, a new loss function designed for this module is proposed as a regularizer of total loss function. By embedding the Gabor convolution module to the Resnet-50 network, we show that it has a good representation learning ability for person re-identification. And experiments on three widely used person re-identification datasets show favorable results compared with the state-of-the-arts.
An end-to-end deep learning system for medieval writer identification

This paper presents an end-to-end system to identify writers in medieval manuscripts. The proposed system consists in a three-step model for detection and classification of lines in the manuscript and page writer identification. The first two steps are based on deep neural networks trained with transfer learning techniques and specialized to solve the task in hand. The third stage is a weighted majority vote row-decision combiner that assigns to each page a writer. The main goal of this paper is to study the applicability of deep learning in this context when a relatively small training dataset is available. We tested our system with several state-of-the-art deep architectures on a digitized manuscript known as the Avila Bible, using only 9.6% of the total pages for training. Our approach proves to be very effective in identifying page writers, reaching a peak of 96.48% of accuracy and 96.56% of F1 score.
Spatiotemporal Feature Residual Propagation for Action Prediction

Recognizing actions from limited preliminary video observations has seen considerable recent progress. Typically, however, such progress has been had without explicitly modeling fine-grained motion evolution as a potentially valuable information source. In this study, we address this task by investigating how action patterns evolve over time in a spatial feature space. There are three key components to our system. First, we work with intermediate-layer ConvNet features, which allow for abstraction from raw data, while retaining spatial layout, which is sacrificed in approaches that rely on vectorized global representations. Second, instead of propagating features per se, we propagate their residuals across time, which allows for a compact representation that reduces redundancy while retaining essential information about evolution over time. Third, we employ a Kalman filter to combat error build-up and unify across prediction start times. Extensive experimental results on the JHMDB21, UCF101 and BIT datasets show that our approach leads to a new state-of-the-art in action prediction.
Spatio-Temporal Semantic Segmentation for Drone Detection

The democratization of drones over the past decade has opened wide cracks in airspace security. Research in drone detection and neutralization for critical infrastructures is a very active area with a number of open issues, such as robust detection of drones based on opto-electronic imaging. Indeed, drones at a certain distance only represent a few pixel points on an image, even on a high resolution camera, and can be easily mistaken for birds or any other flying objects in the airspace. In this context, we propose a spatio-temporal semantic segmentation approach based on convolutional neural networks. We handle the problem of detecting very small targets by using a U-Net architecture to identify areas of interest within the larger image. Then, we use a classification network, ResNet, to determine whether those areas contain a drone or not. To further help the localization and classification process, we provide spatiotemporal input patches to our networks. Drones are mostly moving targets, and birds do not follow the same kinds of trajectories; therefore, this additional feature significantly increases overall performance. This work was carried out in the context of the 2019 Drone-vs-Bird detection Challenge. The evaluation is conducted on the provided dataset under several configurations.
Localization of radiance transformation for image dehazing in wavelet domain

Hazy image observation is a serious phenomenon occurred by random scattering of atmospheric elements. This reduces the scene prominence in terms of color contrast and makes the vision system poor. To overcome this issue, dehazing of digital images is dealt with much attention in the field of computer vision. Dense scattering in hazy image results in loss of original spectral information of the object. It is required to remove the unwanted artifacts of weather during image acquisition process of the given object to improve the visual insight of an acquired image. It is found that most of the existing methods fail to retrieve the sufficient information from hazy images. To address such problem, a robust single image dehazing method is introduced to recover the haze-free image. This is performed by calculating the atmospheric light of the given hazy images by decomposing and retaining the high frequency sub bands using wavelet domain. The dense haze elimination is performed on approximated low frequency sub band of the given hazy image. On reconstruction process, well-refined dehaze image is obtained. Moreover, transmission maps are also produced as a byproduct of this method.
3DACN: 3D Augmented convolutional network for time series data

Time series data and non-time series data are increasing in the credit system of financial market, so that an effective and intelligent data mining model plays a critical role to analyze hybrid time series data. In addition, traditional mining models sometimes fail to converge because of imbalanced data problem. Therefore, we propose a 3D Augmented Convolutional Network (3DACN) to extract time series information and solve the serious imbalanced data problem. By using the augmented algorithm on time series data, hybrid time series data are enlarged to generate more examples on the minority classes. 3DACN ensures the latent variables with an Expectation-Maximization(EM) algorithm to improve F1 score (F1) and Area Under Curve (AUC). Experimental results show that in the benchmark of Bank database, it can gain F1 score by 81.1% and the AUC by 88.2% respectively; while in the benchmark of Credit Risk database, the 3DACN can reach high performance on F1 score by 88.1% and the AUC by 88.4%.
DADA-2000: Can Driving Accident be Predicted by Driver Attentionƒ Analyzed by A Benchmark

Driver attention prediction is currently becoming the focus in safe driving research community, such as the DR(eye)VE project and newly emerged Berkeley DeepDrive Attention (BDD-A) database in critical situations. In safe driving, an essential task is to predict the incoming accidents as early as possible. BDD-A was aware of this problem and collected the driver attention in laboratory because of the rarity of such scenes. Nevertheless, BDD-A focuses the critical situations which do not encounter actual accidents, and just faces the driver attention prediction task, without a close step for accident prediction. In contrast to this, we explore the view of drivers' eyes for capturing multiple kinds of accidents, and construct a more diverse and larger video benchmark than ever before with the driver attention and the driving accident annotation simultaneously (named as DADA-2000), which has 2000 video clips owning about 658, 476 frames on 54 kinds of accidents. These clips are crowd-sourced and captured in various occasions (highway, urban, rural, and tunnel), weather (sunny, rainy and snowy) and light conditions (daytime and nighttime). For the driver attention representation, we collect the maps of fixations, saccade scan path and focusing time. The accidents are annotated by their categories, the accident window in clips and spatial locations of the crash-objects. Based on the analysis, we obtain a quantitative and positive answer for the question in this paper.
University EAP Students' Perceptions of Using a Prototype Virtual Reality Learning Environment to Learn Writing Structure

This study investigates English language learner (ELL) perceptions of using a prototype virtual reality learning environment (VRLE) designed for teaching and learning writing structure. A mixed-methods approach was used, incorporating pre- and post-participation questionnaires, as well as semi-structured interviews. Participants consisted of 10 ELLs enrolled in first year English for Academic Purposes (EAP) courses at a university in Mainland, China. Results indicate that while the majority of students enjoyed using the VRLE, they maintained varying attitudes regarding the usefulness of the VRLE. Additionally, results from a correlation analysis suggest that learners' attitudes towards the material or content being learned (EAP writing) significantly correlates with learners' attitudes towards using the VRLE for learning writing structure.
MobileFAN: Transferring deep hidden representation for face alignment

Facial landmark detection is a crucial prerequisite for many face analysis applications. Deep learning-based methods currently dominate the approach of addressing the facial landmark detection. However, such works generally introduce a large number of parameters, resulting in high memory cost. In this paper, we aim for a lightweight as well as effective solution to facial landmark detection. To this end, we propose an effective lightweight model, namely Mobile Face Alignment Network (MobileFAN), using a simple backbone MobileNetV2 as the encoder and three deconvolutional layers as the decoder. The proposed MobileFAN, with only 8% of the model size and lower computational cost, achieves superior or equivalent performance compared with state-of-the-art models. Moreover, by transferring the geometric structural information of a face graph from a large complex model to our proposed MobileFAN through feature-aligned distillation and feature-similarity distillation, the performance of MobileFAN is further improved in effectiveness and efficiency for face alignment. Extensive experiment results on three challenging facial landmark estimation benchmarks including COFW, 300W and WFLW show the superiority of our proposed MobileFAN against state-of-the-art methods.
A 2 $\times$ 30k-Spin Multi-Chip Scalable CMOS Annealing Processor Based on a Processing-in-Memory Approach for Solving Large-Scale Combinatorial Optimization Problems

The world's first 2 x 30k-spin multi-chip CMOS annealing processor (AP)-based on the processing-in-memory approach for solving large-scale combinatorial optimization problem-was developed. To expand the bit width of coefficients and enhance the scalability of the AP, it has three key features: an expandable and high-accuracy spin operator for local communication, a highly integrated spin circuit using direct access to SRAM, and a low-latency inter-chip interface that does not affect the runtime or results of the annealing process. The AP is fabricated on the basis of 40-nm CMOS technology. It was experimentally demonstrated that the spin-flip ratio of the processor agrees well with theoretical values based on the Gibbs distribution over a wide temperature range. As a result, under two-chip operation with 2 x 30k spins, the AP achieves an annealing time of 22 μs, which is 455 times and 2.6 x 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> times faster than those achieved by our previous CMOS-AP and a conventional CPU, respectively. Moreover, its energy efficiency is 1.75 x 105 times higher than that of a conventional CPU-based algorithm.
Intelligent software engineering in the context of agile software development: A systematic literature review

CONTEXT: Intelligent Software Engineering (ISE) refers to the application of intelligent techniques to software engineering. We define an "intelligent technique" as a technique that explores data (from digital artifacts or domain experts) for knowledge discovery, reasoning, learning, planning, natural language processing, perception or supporting decision-making. OBJECTIVE: The purpose of this study is to synthesize and analyze the state of the art of the field of applying intelligent techniques to Agile Software Development (ASD). Furthermore, we assess its maturity and identify adoption risks. METHOD: Using a systematic literature review, we identified 104 primary studies, resulting in 93 unique studies. RESULTS: We identified that there is a positive trend in the number of studies applying intelligent techniques to ASD. Also, we determined that reasoning under uncertainty (mainly, Bayesian network), search-based solutions, and machine learning are the most popular intelligent techniques in the context of ASD. In terms of purposes, the most popular ones are effort estimation, requirements prioritization, resource allocation, requirements selection, and requirements management. Furthermore, we discovered that the primary goal of applying intelligent techniques is to support decision making. As a consequence, the adoption risks in terms of the safety of the current solutions are low. Finally, we highlight the trend of using explainable intelligent techniques. CONCLUSION: Overall, although the topic area is up-and-coming, for many areas of application, it is still in its infancy. So, this means that there is a need for more empirical studies, and there are a plethora of new opportunities for researchers.
Positioning of blockchain mechanisms in IOT-powered smart home systems: A gateway-based approach

This paper reviews a number of existing smart home Internet of Thing (IoT) automation architectures, providing a credible appraisal of that environment, and, furthermore assesses the applicability of blockchain mechanisms to this use case. Specifically, the paper affords the conclusion that in most instances of home automation, a potential topological place to start the blockchain is in the home gateway that supports the processing and computing required to aggregate or in-network-preprocess the data flows from the various appliances; however, the computing and storage requirements may limit this deployment paradigm to a subset of cases. Alternatively, the place to start the blockchain could be in the fog-to-core gateway point, but in this case this process does not fully guarantee end-to-end integrity. The actual Proof of Work (PoW) functionality would likely be in the core network and not in the relatively low computing-power home gateway. Another alternative envisions private local blockchains to be instituted in the home area network, but still not necessarily for all appliance-generated transactions due to the large volume of such status, event, and usage data.