Reviewerly/exHarmony huggingface dataset:
Optical flow-free generative and adversarial network: generative and adversarial network-based video super-resolution method by optical flow-free motion estimation and compensation

Except for recovering the image detail texture information, the main difference between video super-resolution (VSR) and single-image super-resolution (SR) is that VSR focuses on alleviating the deficiency of temporal coherence between video frames. Motion estimation and motion compensation is the common technique used to strengthen the temporal correlation between frames. Most motion estimation methods are based on optical flow. The optical flow method has three basic assumptions: the movement scale is small; the luminance channel is constant; and every pixel in the same image has the same moving trend. In some scenes with complex motion, the accuracy of the underlying optical flow estimator is limited, which leads to artifacts in the video reconstruction. In recent years, generative adversarial network (GAN) has been widely used for VSR reconstruction, which can acquire more realistic texture details for single frame reconstruction. Based on the above reasons, we explore a GAN-based VSR method by optical flow-free motion estimation and compensation [optical flow-free generative and adversarial network (COFGAN)], which completes motion estimation by producing temporal dimension. COFGAN develops better motion estimation result and improves the performance of VSR without optical flow. To verify the motion estimation effect in complex scenes, long-term sequence real dynamic scene dataset realistic and dynamic scenes is applied for training and testing. We compare the performance of proposed COFGAN with earlier works such as video enhancement with task-oriented flow (TOFlow), frame-recurrent video super-resolution (FRVSR), learning temporal coherence via self-supervision for GAN-based video generation (TecoGAN), and so on. Our method achieves significant performance in the temporal coherence metrics performance of learning perceptual image patch similarity (tLP) (0.47) and performance of optical flow estimation (tOF) (7.07) with ×4 up-scaling factor. Compared to the best performance method TecoGAN in previous work, the proposed method promotes 29% of tLP and 26% of tOF. Moreover, COFGAN reaches the best accuracy on the commonly used video sequence datasets Vid4 and ToS3.
FALNet: flow-based attention lightweight network for human pose estimation

Deep neural networks achieve significant progress in human pose estimation (HPE), but there are still many difficulties in practical applications. One of the important reasons is that the current advanced models are very complex leading to huge computation cost. We propose a flow-based attention lightweight network (FALNet) for HPE to solve this problem. We first design a cheap lightweight bottleneck to reduce the model size and complexity with two components: depthwise convolution and cheap unit. Then we propose a flow-based fusion attention block to generate and aggregate multi-scale features of the same layer effectively. We demonstrate the effectiveness of our methods on two benchmark datasets: the COCO dataset and the MPII dataset. Our network FALNet-50 only has 2.2 M parameters and 0.66G floating-point operations (FLOPs), achieving comparable or even better accuracy with smaller complexity. Moreover, we show the speed advantage of our network during inference. Specifically, our FALNet-50 achieves 70.4 in average precision score on COCO val2017 with 16 FPS inference speed on a smartphone.
Semi-Supervised Crowd Counting via Multiple Representation Learning

There has been a growing interest in counting crowds through computer vision and machine learning techniques in recent years. Despite that significant progress has been made, most existing methods heavily rely on fully-supervised learning and require a lot of labeled data. To alleviate the reliance, we focus on the semi-supervised learning paradigm. Usually, crowd counting is converted to a density estimation problem. The model is trained to predict a density map and obtains the total count by accumulating densities over all the locations. In particular, we find that there could be multiple density map representations for a given image in a way that they differ in probability distribution forms but reach a consensus on their total counts. Therefore, we propose multiple representation learning to train several models. Each model focuses on a specific density representation and utilizes the count consistency between models to supervise unlabeled data. To bypass the explicit density regression problem, which makes a strong parametric assumption on the underlying density distribution, we propose an implicit density representation method based on the kernel mean embedding. Extensive experiments demonstrate that our approach outperforms state-of-the-art semi-supervised methods significantly.
Deep Adversarial Reinforcement Learning Method to Generate Control Policies Robust Against Worst-Case Value Predictions

Over the last decade, methods for autonomous control by artificial intelligence have been extensively developed based on deep reinforcement learning (DRL) technologies. However, despite these advances, robustness to noise in observation data remains as an issue in autonomous control policies implemented using DRL in practical applications. In this study, we present a general robust adversarial learning technology applicable to DRL. During these adversarial learning processes, policies are trained to output consistent control actions through regularization learning, even for adversarial input examples. Importantly, these adversarial examples are produced to lead the current policy to predict the worst action at each state. Although a naive implementation of regularization learning may cause DRL model to learn a biased objective function, our methods were found to minimize bias. When implemented as a modification of a deep Q-network for discrete-action problems in Atari 2600 games and of a deep deterministic policy gradient for continuous-action tasks in Pybullet, our new adversarial learning frameworks showed significantly enhanced robustness against adversarial and random noise added to the input compared to several recently proposed methods.
Approaching Collateral Optimization for NISQ and Quantum-Inspired Computing (May 2023)

Collateral optimization refers to the systematic allocation of financial assets to satisfy obligations or secure transactions, while simultaneously minimizing costs and optimizing the usage of available resources. This involves assessing number of characteristics, such as cost of funding and quality of the underlying assets to ascertain the optimal collateral quantity to be posted to cover exposure arising from a given transaction or a set of transactions. One of the common objectives is to minimise the cost of collateral required to mitigate the risk associated with a particular transaction or a portfolio of transactions while ensuring sufficient protection for the involved parties. Often, this results in a large-scale combinatorial optimization problem. In this study, we initially present a Mixed Integer Linear Programming (MILP) formulation for the collateral optimization problem, followed by a Quadratic Unconstrained Binary optimization (QUBO) formulation in order to pave the way towards approaching the problem in a hybrid-quantum and NISQ-ready way. We conduct local computational small-scale tests using various Software Development Kits (SDKs) and discuss the behavior of our formulations as well as the potential for performance enhancements. We find that while the QUBO based approaches fail to find the global optima in the small scale experiments, they are reasonably close suggesting their potential for large instances. We further survey the recent literature that proposes alternative ways to attack combinatorial optimization problems suitable for collateral optimization.
A Centralized Task Allocation Algorithm for a Multi-Robot Inspection Mission With Sensing Specifications

Recently, considerable attention has focused on enhancing the security and safety of industries with high-risk level activities in order to protect the equipment and environment. In particular, chemical processes and nuclear power generation may have a deep impact on their surroundings. In the case of major events, such as chemical spills, oil rig explosions, or nuclear accidents, collecting accurate and rapidly-evolving data becomes a challenging task. So, coordinating a fleet of autonomous mobile robots is a very promising way to deal with unpredicted events and also prevent malicious actions. This paper addresses the problem of assigning optimally a set of tasks to a set of mobile robots equipped with different sensors to minimize a global objective function. The robots perform sensing tasks in order to monitor the area and to facilitate firefighters and inspectors work if a disaster occurs by providing the necessary measures. For this purpose, a centralized Genetic Algorithm (GA) is proposed to determine the task each robot will perform and the order of execution. The proposed approach is tested through a simulation scenario of a grid map environment that represents an industrial area of the city of Le Havre, France. Moreover, a comparative study is conducted with the Hybrid Filtered Beam Search (HFBS) approach and the Mixed-Integer Linear Programming (MILP) solver Cplex. The results demonstrate that the GA approach offers a favorable balance between optimality and execution time.
Hybrid Teaching–Learning-Based Optimization for Workflow Scheduling in Cloud Environment

At present, workflow scheduling in cloud computing environment is still a challenging optimization topic due to its NP-complete characteristics. In order to obtain better scheduling results, researchers are constantly coming up with new methods. In this study, we offer a hybrid metaheuristic for solving workflow scheduling in cloud to minimize the makespan of the workflow considering the heterogeneity of virtual resources. This hybrid approach combines the excellent optimization properties of Heterogeneous Earliest Finish Time (HEFT), Teaching–Learning-Based Optimization (TLBO), Opposition-Based Learning (OBL), and genetic manipulations, which is named Hybrid TLBO (HTLBO). Firstly, a HEFT-based method is proposed to produce the high-quality diverse initial population. Secondly, a Mixed OBL (MOBL) model is designed, in which the boundary search information and the population historical search information are systematically taken into account. Finally, an enhanced learner stage using genetic operations are added to effectively help the algorithm to jump out of the local optima. Rigorous experiments over various scientific workflows are conducted to validate HTLBO’s performance. The obtained results are compared to HEFT and some state-of-the-art hybrid metaheuristics in terms of average makespan, running time and non-parametric statistics. A significant improvement in schedule quality demonstrates that HTLBO can increase population diversity and achieve a good balance between scheduling effectiveness and efficiency.
Single image deraining via multi-scale feature-based deep convolutional neural network

It is challenging to remove rain-streaks from a single image because the rain-streaks are spatially varying. Although data-driven rain removal methods have reported promising performance recently, there are still some defects, such as data dependency and insufficient interpretation. A single image deraining algorithm that combines model-based and data-driven approaches is introduced. First, high-frequency information extracted by an improved weighted guided image filter (iWGIF) is used to learn the rain-streaks to avoid interference from other information through the input image. Then, the input and the learned rain streaks are transferred adaptively from the image domain to the feature domain to learn useful features for high quality image deraining. Finally, networks with multi-scale attention mechanisms are used to restore high-quality images from the latent features. Experiments show that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both qualitative and quantitative measures.
The fastest generation of multipartite entanglement with natural interactions

Abstract Natural interactions among multiple quantum objects are fundamentally composed of two-body terms only. In contradistinction, single global unitaries that generate highly entangled states typically arise from Hamiltonians that couple multiple individual subsystems simultaneously. Here, we study the time to produce strongly nonclassical multipartite correlations with a single unitary generated by the natural interactions. We restrict the symmetry of two-body interactions to match the symmetry of the target states and focus on the fastest generation of multipartite entangled Greenberger–Horne–Zeilinger, W, Dicke and absolutely maximally entangled states for up to seven qubits. These results are obtained by constraining the energy in the system and accordingly can be seen as state-dependent quantum speed limits for symmetry-adjusted natural interactions. They give rise to a counter-intuitive effect where the creation of particular entangled states with an increasing number of particles does not require more time. The methods used rely on extensive numerical simulations and analytical estimations.
Multimodal Machine Learning for Sign Language Prediction

Numerous applications, including translation tools, interpreting services, video remote interpreting, human-computer interaction, online hand tracking of human communication in desktop settings, real-time multi-person recognition systems, games, virtual reality settings, robot controls, and natural language communications, benefit from sign language recognition advantages. Multimodal data contains information from different sources such as video, sensors, electrocardiograms (ECGs), while emotions refer to the non-verbal cues that accompany language use, such as facial expressions and body posture. Integrating these additional sources of information helps to better understand the user’s intent, which improves the performance of the sign language recognition model. To build such a model, a set of multimodal data and emotions must be collected. This data set should be differentiated and cover different individual/isolated signs, emotions and body gestures. The model is designed to integrate multimodal data and emotions, which would involve combining different machine and deep learning algorithms adapted to different types of data. In addition, the model will need to be trained to recognize the different emotions that accompany sign language. Once the model is trained, it can be tested on the test dataset to assess its performance and also plan for a test on real data (with signing people). In this paper we propose a study to use the multi-modal machine learning for sign recognition language.
Irregular Step of Changing for Neural Network Data Sets Improves the Accuracy of Resistive Sensors Calculation

A linear multiport is considered as a model of multiwire communication lines with resistive sensors of physical quantities or as a model of the sensors themselves. The calculation of sensor resistance from measured input currents using a neural network as an approximation problem is investigated. To demonstrate such a problem, using the parameters of multiports with one and two sensor loads, the corresponding number of input currents for a particular set of loads is calculated in a given range of changes in their values. The input and target vectors are composed in this way. The dimension of the input vector is equal to the number of input currents. Numerical experiments were carried out in the MATLAB Deep Learning package for the feed-forward network. The trained model is further tested on the control data set to ensure the given computation accuracy conditionally for “all possible” load values. The data sets generation is carried out with both the traditionally constant and an irregular or variable step of change in values. For the irregular step, in the divided data into training, test and validation sets, an internal pattern is excluded and the network shows a greater ability to generalize. The same control data set shows a reduction in relative error for a series of numerical experiments. The repeatability of training results with preset value of relative error is introduced, as special index for quantitative assessment of training quality when comparing training results of neural networks with generated training data. The obtained results provide the basis for the study of both chains with a large number of loads, as well as other approximation and regression problems.
Generalizability of Self-Supervised Training Models for Digital Pathology: A Multicountry Comparison in Colorectal Cancer

In this multicountry study, we aim to explore the effectiveness of self-supervised learning (SSL) in colorectal cancer (CRC)-related predictive tasks using large amount of unlabeled digital pathology imaging data.We adopted SimSiam to conduct self-supervised pretraining on two large whole-slide image CRC data sets from the United States and Australia. The SSL pretrained encoder is then used in several predictive tasks, including supervised predictive tasks (tissue classification, microsatellite instability v microsatellite stability classification), and weakly supervised predictive tasks (polyp type classification and adenoma grading, and 5-year survival prediction). Performance on the tasks was compared between models using SSL pretraining and those using ImageNet pretraining, and performance for one-country pretraining was compared with two-country pretraining.We demonstrate that SSL pretraining outperforms ImageNet pretraining in predictive tasks, that is, SSL pretraining outperforms the ImageNet pretraining by 3.01% of F1 score on average over supervised predictive tasks and 1.53% of AUC on average over weakly supervised predictive tasks. Furthermore, two-country SSL pretraining has shown more stable performance than single-country pretraining, that is, two-country pretraining outperforms at least one of the single-country pretrainings by 1.93% of F1 on average over supervised predictive tasks and 1.36% of AUC on average over weakly-supervised predictive tasks.We find that using unlabeled image data for SSL pretraining in CRC related tasks is more effective than using ImageNet pretraining. Furthermore, SSL pretraining using data from multiple countries achieve more stable performance and better generalization than single-country pretraining.
DMNet: Dynamic Memory Network for RGB-D Salient Object Detection

RGB-D salient object detection aims to identify the most salient object using the color and depth information of images. Currently, most of the existing salient object detection models use the classical U-Net architecture, which uses up-sampling and short link decoding to exploit saliency cues after generating multi-level features by successive convolution and pooling operations. However, the multi-level encoding and progressive up-sampling decoding used by these models may lose some of the high-level semantic information during feature extraction and information fusion and some of the detailed information during image recovery, which in turn affects the quality of generated saliency maps. To solve these problems, we propose a dynamic memory network (DMNet) consisting of an interactive enhancement encoder and a dynamic memory decoder, which achieves high-precision detection of salient objects with a more complete and superior encoding and decoding strategy. Specifically, in the interactive enhancement encoder, we propose a feature interactive fusion module (FIFM), which can enhance intermediate-scale RGB features by fusing RGB features of three adjacent scales. Moreover, we design a depth-guided dense fusion module (DDFM) to fuse the features from RGB and depth modalities. In the dynamic memory decoder, we design a full-dimensional dynamic convolutional expansion module (FDCEM) to enhance the information representation capability of features at different scales. Furthermore, we also design a gated decoding module (GDM) to decode features of different scales and eliminate non-salient information. Extensive experimental results on STERE, SIP, NLPR, NJU2K, SSD, and DES datasets demonstrate that our model outperforms most of the state-of-the-art RGB-D SOD methods. In addition, the experimental results on three RGB-T datasets, VT821, VT1000, and VT5000 also show that our model can be effectively used for RGB-T SOD.
Prospectivity modelling of critical mineral deposits using a generative adversarial network with oversampling and positive-unlabelled bagging

The demand for critical minerals is rapidly increasing worldwide, yet future global supply remains uncertain due to the difficulty in discovering new deposits using traditional methods. To increase the success rate of exploration projects for these vital resources, the use of artificial intelligence is continuously increasing for big and complex data analysis. This study proposes a new machine learning-based framework that tackles common problems associated with exploring critical mineral deposits, such as the shortage of known mineral occurrences, challenges in selecting negative samples in barren regions, and unbalanced training data. Our framework combines an improved generative adversarial network with positive and unlabelled learning to enhance efficiency. To test the performance of the framework, we create prospectivity maps of mafic-ultramafic intrusion-hosted mineralisation for cobalt, chromium, and nickel in the Gawler Craton, South Australia. The models are trained on a carefully selected set of independent features based on a conceptual model derived from open-access exploration data, resulting in high and stable performance. The prospectivity maps show a strong spatial correlation between high probabilities and known mineral occurrences and predict potential greenfield regions for future exploration. Our models demonstrate a significantly higher accuracy compared to a conventional approach using a standard random forest classifier and reveal that geophysical features play a crucial role in mapping prospective regions of critical minerals. Overall, our framework has the potential to significantly enhance critical mineral exploration by providing a more accurate and efficient approach to identifying prospective regions for future mining operations.
DBT-PDP: Provable data possession with outsourced data batch transfer based on blockchain

In the scenario of large-scale data ownership transactions, existing data integrity auditing schemes are faced with security risks from malicious third-party auditors and are inefficient in both calculation and communication, which greatly affects their practicability. This paper proposes a data integrity audit scheme based on blockchain where data ownership can be traded in batches. A data tag structure which supports data ownership batch transaction is adopted in our scheme. The update process of data tag does not involve the unique information of each data, so that any user can complete ownership transactions of multiple data in a single transaction through a single transaction auxiliary information. At the same time, smart contract is introduced into our scheme to perform data integrity audit belongs to third-party auditors, therefore our scheme can free from potential security risks of malicious third-party auditors. Safety analysis shows that our scheme is proved to be safe under the stochastic prediction model and k-CEIDH hypothesis. Compared with similar schemes, the experiment shows that communication overhead and computing time of data ownership transaction in our scheme is lower. Meanwhile, the communication overhead and computing time of our scheme is similar to that of similar schemes in data integrity audit.