Reviewerly/exHarmony huggingface dataset:
MS<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si1.svg"><mml:msup><mml:mrow /><mml:mn>2</mml:mn></mml:msup></mml:math>GAH: Multi-label semantic supervised graph attention hashing for robust cross-modal retrieval

Due to the strong nonlinear representation capabilities of deep neural networks and the low storage and high efficiency characteristics of hash learning, deep cross-modal hashing has been propelled to the forefront of academics. How to preferably bridge semantic relevance to further bridge the semantic modality gap is the vital bottleneck to improve model performance. Confronting samples with rich semantics, how to comprehensively explore the hidden correlations and establish more precise modality relationships is the primary issue to be solved. In this work, we propose a novel deep hashing method called Multi-Label Semantic Supervised Graph Attention Hashing (MS2GAH), which is an end-to-end framework that integrates graph attention networks (GATs). It constructs graph features through the adjacency of nodes and assigns different weights to adjacent edges to enhance the robustness of the model. Simultaneously, multi-label annotations are utilized to bridge the semantic relevance between modalities in a more fine-grained manner. To make preferable use of rich semantic information, an end-to-end label encoder is designed to mine high-level semantics from multi-label annotations to guide the feature extraction process of specific-modality networks, thereby further narrowing the modality gap. Finally, extensive experiments have been conducted on four datasets, and the results show that MS2GAH is superior to other baselines and one step forward.
Bayesian modeling of human–AI complementarity

SignificanceWith the increase in artificial intelligence in real-world applications, there is interest in building hybrid systems that take both human and machine predictions into account. Previous work has shown the benefits of separately combining the predictions of diverse machine classifiers or groups of people. Using a Bayesian modeling framework, we extend these results by systematically investigating the factors that influence the performance of hybrid combinations of human and machine classifiers while taking into account the unique ways human and algorithmic confidence is expressed.
Enhancing IoT anomaly detection performance for federated learning

Federated Learning (FL) with mobile computing and the Internet of Things (IoT) is an effective cooperative learning approach. However, several technical challenges still need to be addressed. For instance, dividing the training process among several devices may impact the performance of Machine Learning (ML) algorithms, often significantly degrading prediction accuracy compared to centralized learning. One of the primary reasons for such performance degradation is that each device can access only a small fraction of data (that it generates), which limits the efficacy of the local ML model constructed on that device. The performance degradation could be exacerbated when the participating devices produce different classes of events, which is known as the class balance problem. Moreover, if the participating devices are of different types, each device may never observe the same types of events, which leads to the device heterogeneity problem. In this study, we investigate how data augmentation can be applied to address these challenges and improving detection performance in an anomaly detection task using IoT datasets. Our extensive experimental results with three publicly accessible IoT datasets show the performance improvement of up to 22.9% with the approach of data augmentation, compared to the baseline (without relying on data augmentation). In particular, stratified random sampling and uniform random sampling show the best improvement in detection performance with only a modest increase in computation time, whereas the data augmentation scheme using Generative Adversarial Networks is the most time-consuming with limited performance benefits.
An Improved Teaching-Learning-Based Optimization Algorithm with Reinforcement Learning Strategy for Solving Optimization Problems

This paper presents an improved teaching-learning-based optimization (TLBO) algorithm for solving optimization problems, called RLTLBO. First, a new learning mode considering the effect of the teacher is presented. Second, the Q-Learning method in reinforcement learning (RL) is introduced to build a switching mechanism between two different learning modes in the learner phase. Finally, ROBL is adopted after both the teacher and learner phases to improve the local optima avoidance ability of RLTLBO. These two strategies effectively enhance the convergence speed and accuracy of the proposed algorithm. RLTLBO is analyzed on 23 standard benchmark functions and eight CEC2017 test functions to verify the optimization performance. The results reveal that proposed algorithm provides effective and efficient performance in solving benchmark test functions. Moreover, RLTLBO is also applied to solve eight industrial engineering design problems. Compared with the basic TLBO and seven state-of-the-art algorithms, the results illustrate that RLTLBO has superior performance and promising prospects for dealing with real-world optimization problems. The source codes of the RLTLBO are publicly available at https://github.com/WangShuang92/RLTLBO.
Weakly supervised segmentation with cross-modality equivariant constraints

Weakly supervised learning has emerged as an appealing alternative to alleviate the need for large labeled datasets in semantic segmentation. Most current approaches exploit class activation maps (CAMs), which can be generated from image-level annotations. Nevertheless, resulting maps have been demonstrated to be highly discriminant, failing to serve as optimal proxy pixel-level labels. We present a novel learning strategy that leverages self-supervision in a multi-modal image scenario to significantly enhance original CAMs. In particular, the proposed method is based on two observations. First, the learning of fully-supervised segmentation networks implicitly imposes equivariance by means of data augmentation, whereas this implicit constraint disappears on CAMs generated with image tags. And second, the commonalities between image modalities can be employed as an efficient self-supervisory signal, correcting the inconsistency shown by CAMs obtained across multiple modalities. To effectively train our model, we integrate a novel loss function that includes a within-modality and a cross-modality equivariant term to explicitly impose these constraints during training. In addition, we add a KL-divergence on the class prediction distributions to facilitate the information exchange between modalities which, combined with the equivariant regularizers further improves the performance of our model. Exhaustive experiments on the popular multi-modal BraTS and prostate DECATHLON segmentation challenge datasets demonstrate that our approach outperforms relevant recent literature under the same learning conditions.
PKRU-safe

After more than twenty-five years of research, memory safety violations remain one of the major causes of security vulnerabilities in real-world software. Memory-safe languages, like Rust, have demonstrated that compiler technology can assist developers in writing efficient low-level code without the risk of memory corruption. However, many memory-safe languages still have to interface with unsafe code to some extent, which opens up the possibility for attackers to exploit memory-corruption vulnerabilities in the unsafe part of the system and subvert the safety guarantees provided by the memory-safe language.
Mobile robot path planning using fuzzy enhanced improved Multi-Objective particle swarm optimization (FIMOPSO)

This paper introduces a method for car-like mobile robot path planning (CRPP). The robot works in both dynamic and static situations. The aim of this method is to explore the best safe path with minimum path length, minimum motor torque, minimum travel time, minimum robot acceleration and maximum obstacle avoidance. Kinodynamic and non-holonomic constraints related with car-like robot are considered. Fuzzy enhanced Improved Multi-objective Particle Swarm Optimization (FIMOPSO) algorithm is proposed to solve the CRPP problem. Fuzzy inference system is used for obstacle avoidance. In the proposed FIMOPSO, five improvements are made. Proposed technique is compared with Multi-objective Strength Pareto Evolutionary Algorithm 2 (MOSPEA2) technique. Experiments on a custom-made car-like robot are ensuring the quality of proposed technique. This research works show that proposed FIMOPSO is another alternative technique to CRPP problems. Paths dictated by FIMOPSO are safe, collision free, feasible, and possible and can be practically implemented. Fuzzy inference system works well for safe robot travel. FIMOPSO simulation paths are acceptable. Since, the deviation between experiment and simulation is less than 2%.
Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video

Learning to capture human motion is essential to 3D human pose and shape estimation from monocular video. However, the existing methods mainly rely on recurrent or convolutional operation to model such temporal information, which limits the ability to capture non-local context relations of human motion. To address this problem, we propose a motion pose and shape network (MPS-Net) to effectively capture humans in motion to estimate accurate and temporally coherent 3D human pose and shape from a video. Specifically, we first propose a motion continuity attention (MoCA) module that leverages visual cues observed from human motion to adaptively recalibrate the range that needs attention in the sequence to better capture the motion continuity dependencies. Then, we develop a hierarchical attentive feature integration (HAFI) module to effectively combine adjacent past and future feature represen-tations to strengthen temporal correlation and refine the feature representation of the current frame. By coupling the MoCA and HAFI modules, the proposed MPS-Net excels in estimating 3D human pose and shape in the video. Though conceptually simple, our MPS-Net not only outperforms the state-of-the-art methods on the 3DPW, MPI-INF-3DHP, and Human3.6M benchmark datasets, but also uses fewer network parameters. The video demos can be found at https://mps-net.github.io/MPS-Net/.
ConfusionVis: Comparative evaluation and selection of multi-class classifiers based on confusion matrices

In machine learning, the presumably best model is selected from a variety of model candidates generated by testing different model types, hyperparameters, or feature subsets. The advent of deep learning has made model selection even more challenging due to the huge parameter search space. Relying on a single metric to select the best model does not consider class imbalances or the different costs of misclassifications. We argue that incorporating human knowledge to interactively analyse the per-class errors and class confusions over all model candidates enables a more efficient training process and yields better models for given applications. This paper proposes the model-agnostic approach ConfusionVis which allows to comparatively evaluate and select multi-class classifiers based on their confusion matrices. This contributes to making the models’ results understandable, while treating the models as black boxes. Therefore, we propose a novel method to measure and visualise distances between confusion matrices and an interactive query interface to incorporate all composition levels of class errors. The approach is evaluated in a user study and the applicability is shown by a case study where marine biologists investigate the conservation efforts of baleen whales by classifying whale species in acoustic recordings. ConfusionVis is available online: https://www.ml-and-vis.org/confusionvis.
Adversarial attack and defense technologies in natural language processing: A survey

Recently, the adversarial attack and defense technology has made remarkable achievements and has been widely applied in the computer vision field, promoting its rapid development in other fields, primarily the natural language processing domain. However, discrete semantic texts bring additional restrictions and challenges to successfully implementing adversarial attacks and defenses. This survey systematically summarizes the current progress of adversarial techniques in the natural language processing field. We first briefly introduce the textual adversarial example’s particularity, vectorization, and evaluation metrics. More importantly, we categorize textual adversarial attacks according to the combination of semantic granularity and example generation strategy. Next, we present commonly used datasets and adversarial attack applications in diverse natural language processing tasks. Besides, we classify defense strategies as passive and active methods considering both input data and victim models. Finally, we present several challenging issues and future research directions in this domain.
BSM-ether: Bribery selfish mining in blockchain-based healthcare systems

Blockchain-enabled distributed networks present a new infrastructure for building reliable and privacy-preserving healthcare utilities. Among them, Ethereum networks have gained specialized attention for their high usability and security. In this paper, we raise the awareness of Ethereum's vulnerability due to selfish mining, in which ill-disposed miners are feasible to receive more rewards than honest ones. To demonstrate this, we compose a new bribery selfish mining scheme, the BSM-Ether, targeted to Ethereum. The BSM-Ether attack can be easily conducted and has higher rewards than other existing malicious attacks. Besides, we present a formal and rigorous analysis of multiple kinds of rewards from the BSM-Ether attack. Simulation experiments show the high effectiveness of BSM-Ether – the attacker can get a higher revenue with few efforts. To tackle this variability, we present some implementation level proposals and suggestions for building healthcare systems on top of Ethereum to minimize the negative effect of the security deficiency of underlying systems.
A Multitasking Flexible Sensor via Reservoir Computing

Natural disasters are reported globally, and one source of severe damage to cities is flooding caused by locally heavy rain. Sharing of local weather information can save lives. However, it is difficult to collect local weather information in real-time because such data collection requires bulky, expensive sensors. For local, real-time monitoring of heavy rain and wind, a sensor system should be simple and low-cost so that it can be attached to a variety of surfaces, including roofs, vehicles, and umbrellas. To develop simple, low-cost multitasking sensors located on nonplanar surfaces, a flexible rain sensor to monitor waterdrop volume and wind velocity is devised. To monitor both simultaneously, a laser-induced graphene-based superhydrophobic conductive film is introduced. Using the superhydrophobic surface, water dynamics are measured when waterdrops collide with the sensor surface, and obtained time-series data are processed using "reservoir computing" to extract the volume and velocity from a single sensor as multitasking electronics. As a proof-of-concept, it is shown that the sensor measures continuous, long-term volume and wind-change dynamics. The results demonstrate feasibility of multitasking electronics with reservoir computing to reduce sensor integration complexity with low power consumption for both sensor and signal processing.
Deep Learning Approaches for Video Compression: A Bibliometric Analysis

Every data and kind of data need a physical drive to store it. There has been an explosion in the volume of images, video, and other similar data types circulated over the internet. Users using the internet expect intelligible data, even under the pressure of multiple resource constraints such as bandwidth bottleneck and noisy channels. Therefore, data compression is becoming a fundamental problem in wider engineering communities. There has been some related work on data compression using neural networks. Various machine learning approaches are currently applied in data compression techniques and tested to obtain better lossy and lossless compression results. A very efficient and variety of research is already available for image compression. However, this is not the case for video compression. Because of the explosion of big data and the excess use of cameras in various places globally, around 82% of the data generated involve videos. Proposed approaches have used Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), and Generative Adversarial Networks (GANs), and various variants of Autoencoders (AEs) are used in their approaches. All newly proposed methods aim to increase performance (reducing bitrate up to 50% at the same data quality and complexity). This paper presents a bibliometric analysis and literature survey of all Deep Learning (DL) methods used in video compression in recent years. Scopus and Web of Science are well-known research databases. The results retrieved from them are used for this analytical study. Two types of analysis are performed on the extracted documents. They include quantitative and qualitative results. In quantitative analysis, records are analyzed based on their citations, keywords, source of publication, and country of publication. The qualitative analysis provides information on DL-based approaches for video compression, as well as the advantages, disadvantages, and challenges of using them.
Dancing with service robots: The impacts of employee-robot collaboration on hotel employees’ job crafting

The increasing use of service robots, which serve as close working “partners” for hotel employees, accelerates the need to understand employees’ reactions in employee-robot collaboration. The purpose of this study is to investigate the impacts of employees’ perceptions (i.e., perceived risk, perceived playfulness, performance expectancy, and effort expectancy) on job crafting toward employee-robot collaboration in hospitality. Drawing on the Sense-Think-Act framework, this study uses a structural equation model to analyze the multiple paths. By using paper- and pencil- surveys at three-time points, data were collected from 407 frontline employees working in 16 hotel chains in China and analyzed via AMOS and SPSS. The findings suggest that perceived risk and perceived playfulness are the key drivers of employees’ effort expectancy and performance expectancy, and perceived risk has the strongest positive impact on effort expectancy. Furthermore, performance expectancy and effort expectancy have a staged impact on job crafting. The findings are valuable to researchers and managers for future research and implementation of service robots in the hotel industry.
A blockchain-based framework for on-site construction environmental monitoring: Proof of concept

On-site construction environmental monitoring (OCEM) can help to govern the behaviors of construction stakeholders in terms of pollutant emissions and environmental protection. Current OCEM systems, nevertheless, were built utilizing a centralized architecture, such as the government's local database, which leads to information imbalance and disputes among multiple stakeholders. When there are no trustworthy data records to justify construction companies' compliance, for example, arguments or even conflicts may emerge between neighboring residents and the contractors. Blockchain has the potential to be a reliable platform for environmental monitoring due to its decentralization, immutability, transparency, and autonomous enforcement of agreements. However, there has been limited investigation into the blockchain's possibilities in OCEM. Aiming to bridge the knowledge gap, this paper proposed a blockchain-enabled framework for OCEM. To begin, an overall conceptual framework was created, in which construction pollutant data was collected via sensors and then uploaded to the blockchain network. Then, based on the consensus algorithm, the on-chained information flow was introduced. Smart contracts were programmed to automatically monitor the level of construction pollutants and evaluate environmental performance. Finally, a case study was carried out to validate the theoretical viability of the proposed framework in a laboratory environment, in which a blockchain prototype based on the Hyperledger Fabric architecture was presented. The results indicate that blockchain can assist OCEM by supplying trustworthy environmental data and allowing for continuous monitoring. This research investigated the blockchain's potential in OCEM and created a prototype system that can be fine-tuned for use in real-world scenarios.
NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results

In this paper, we summarize the 1st NTIRE challenge on stereo image super-resolution (restoration of rich details in a pair of low-resolution stereo images) with a focus on new solutions and results. This challenge has 1 track aiming at the stereo image super-resolution problem under a standard bicubic degradation. In total, 238 participants were successfully registered, and 21 teams competed in the final testing phase. Among those participants, 20 teams successfully submitted results with PSNR (RGB) scores better than the baseline. This challenge establishes a new benchmark for stereo image SR.
Quantum service-oriented computing: current landscape and challenges

Abstract The development that quantum computing technologies are achieving is beginning to attract the interest of companies that could potentially be users of quantum software. Thus, it is perfectly feasible that during the next few years hybrid systems will start to appear integrating both the classical software systems of companies and new quantum ones providing solutions to problems that still remain unmanageable today. A natural way to support such integration is Service-Oriented Computing. While conceptually the invocation of a quantum software service is similar to that of a classical one, technically there are many differences and technological limitations, which refer to platform independence, decoupling, scalability, etc. To highlight these differences and the difficulties to develop quality quantum services, this paper takes a well-known problem to which a quantum solution can be provided, integer factorization, making use of the Amazon Braket quantum service platform. The exercise of trying to provide the factorization as a quantum service is carried out following the best practices, design patterns and standards existing in the implementation of classical services. This case study is used to highlight the rough edges and limitations that arise in the integration of classical-quantum hybrid systems using service-oriented computing. The conclusion of the study allows us to point out directions in which to focus research efforts in order to achieve effective quantum service-oriented computing.