Reviewerly/exHarmony huggingface dataset:
Crossmodal bipolar attention for multimodal classification on social media

Multimodal classification of social media is used to classify data from different modalities into different categories, which is essential for understanding user behavior on the web. In this paper, we focus on classifying image-text pairs, specifically user-generated content on social media. Recently, the transformer network, a kind of self-attention network, has been widely studied in the disciplines of visual computing and language processing. In the attention mechanism, positive correlation is considered. However, multimedia content posted on social media is diverse. Images and text are not always consistent, and contrary information is also helpful for representation. Therefore, it is equally important to detect conflicts based on negative or inverse attention. Inspired by the attention mechanism, we propose a novel model, namely Crossmodal Bipolar Attention Network (CBAN). Different from existing positive dot-product and additive attention mechanisms, we propose a bipolar attention mechanism, which fuses visual and textual information through their direct and inverse semantic relationships to classify multimodal data. We conducted experiments on multiple multimodal classification data sets, for performing sentiment analysis, sarcasm detection, crisis categorization and hate-speech detection. Experimental results show that our proposed CBAN consistently outperforms state-of-the-art methods in all classification tasks.
Relay knowledge distillation for efficiently boosting the performance of shallow networks

To reduce the computational consumption and memory footprint of powerful deep neural networks for applications on edge devices, many model compression methods have been proposed. Among them, knowledge distillation, a topic that has been widely studied, aims at training a lightweight and powerful student network under the guidance of a well-trained but cumbersome teacher network. Most existing knowledge distillation methods focus on how to better define knowledge and the student is forced to mimic the representation space of the teacher, ignoring leveraging the powerful teacher architecture to improve the student intermediate representations for approaching the proficient performance of the teacher. Since the teacher naturally has more powerful feature extraction ability and is always available at the training process of the student, we introduce Relay Knowledge Distillation (ReKD) for efficiently boosting the student performance. Concretely, ReKD transfers the student intermediate representations to the teacher, making the student trained with the teacher in an interactive manner. By directly utilizing the teacher powerful feature extraction ability to improve the student intermediate representations, the knowledge of the teacher is implicitly distilled to the student. Extensive experiments on diverse image classification datasets demonstrate the proposed ReKD can significantly improve the student performance, even making it possible for the student to outperform the teacher.
M2YOLOF: Based on effective receptive fields and multiple-in-single-out encoder for object detection

Object detection under one-level feature is a difficult task, which requires that different scale object representations can be extracted on one feature map, as well as the balance between quality and quantity of positive samples play a key role in model training. YOLOF with real-time detection speed solves the partial problems about object scale and sample quantity balance. To further improve performance especially in smaller objects, we propose a new object detector called M2YOLOF. The main ingredients are a multi-in-single-out encoder that joints attention to strengthen the local feature and global representation of each multi-scale object, and a dynamic sample selection policy that using effective receptive fields to rationalize the quantity of positive samples. M2YOLOF strengthen the contextual details of feature map and balances the rationality of training samples. Extensive experiments on COCO benchmark prove the effectiveness of our method, with an image size of [1333,800], using ResNet50 as backbone, running at 29 FPS on 2080Ti and achieving 39.2 AP. It is 1.7 AP higher than YOLOF but GFLOPs of our method only increases by <9%.
Image Color Rendering Based on Hinge-Cross-Entropy GAN in Internet of Medical Things

Computer-aided diagnosis based on image color rendering promotes medical image analysis and doctor-patient communication by highlighting important information of medical diagnosis.To overcome the limitations of the color rendering method based on deep learning, such as poor model stability, poor rendering quality, fuzzy boundaries and crossed color boundaries, we propose a novel hinge-cross-entropy generative adversarial network (HCEGAN).The self-attention mechanism was added and improved to focus on the important information of the image.And the hinge-cross-entropy loss function was used to stabilize the training process of GAN models.In this study, we implement the HCEGAN model for image color rendering based on DIV2 K and COCO datasets, and evaluate the results using SSIM and PSNR.The experimental results show that the proposed HCEGAN automatically re-renders images, significantly improves the quality of color rendering and greatly improves the stability of prior GAN models.
Hadamard matrix-guided multi-modal hashing for multi-modal retrieval

Multi-modal hashing can encode heterogeneous multi-modal data into compact binary codes, which has been extensively studied to solve large-scale multi-modal retrieval. However, since pioneer methods do not exploit fully the potential discriminative information in category labels and the rich complementary information between multi-modal data, their retrieval performance is limited. To address this problem, we propose a novel multi-modal hashing method that performs subspace learning and target feature learning in an overall framework. On the one hand, the proposed method captures the complementary information between multi-modal data by adaptive projection learning. To enhance the feature representation ability, the multi-modal spaces are reconstructed via the collective matrix factorization. On the other hand, the target binary codes that are predefined by the Hadamard matrix are softened into the learnable target features, which can promote the inter-class separability and preserve the intra-class difference. The extensive experiment results conducted on three public datasets show that the proposed method outperforms state-of-the-art methods.
Reducing data dimension boosts neural network-based stage-specific malaria detection

Although malaria has been known for more than 4 thousand years1, it still imposes a global burden with approx. 240 million annual cases2. Improvement in diagnostic techniques is a prerequisite for its global elimination. Despite its main limitations, being time-consuming and subjective, light microscopy on Giemsa-stained blood smears is still the gold-standard diagnostic method used worldwide. Autonomous computer assisted recognition of malaria infected red blood cells (RBCs) using neural networks (NNs) has the potential to overcome these deficiencies, if a fast, high-accuracy detection can be achieved using low computational power and limited sets of microscopy images for training the NN. Here, we report on a novel NN-based scheme that is capable of the high-speed classification of RBCs into four categories-healthy ones and three classes of infected ones according to the parasite age-with an accuracy as high as 98%. Importantly, we observe that a smart reduction of data dimension, using characteristic one-dimensional cross-sections of the RBC images, not only speeds up the classification but also significantly improves its performance with respect to the usual two-dimensional NN schemes. Via comparative studies on RBC images recorded by two additional techniques, fluorescence and atomic force microscopy, we demonstrate that our method is universally applicable for different types of microscopy images. This robustness against imaging platform-specific features is crucial for diagnostic applications. Our approach for the reduction of data dimension could be straightforwardly generalised for the classification of different parasites, cells and other types of objects.
Analysing Eye Gaze Patterns during Confusion and Errors in Human–Agent Collaborations

As human–agent collaborations become more prevalent, it is increasingly important for an agent to be able to adapt to their collaborator and explain their own behavior. In order to do so, they need to be able to identify critical states during the interaction that call for proactive clarifications or behavioral adaptations. In this paper, we explore whether the agent could infer such states from the human’s eye gaze for which we compare gaze patterns across different situations in a collaborative task. Our findings show that the human’s gaze patterns significantly differ between times at which the user is confused about the task, times at which the agent makes an error, and times of normal workflow. During errors the amount of gaze towards the agent increases, while during confusion the amount towards the environment increases. We conclude that these signals could tell the agent what and when to explain.
Genetic Algorithm-Based Task Scheduling in Cloud Computing Using MapReduce Framework

Task scheduling is an essential component of any distributed system because it routes tasks to appropriate resources for execution, such as grids, clouds, and peer-to-peer networks. Common scheduling algorithms include downsides, such as high temporal complexity, non-simultaneous processing of input tasks, and longer program execution times. Exploration-based scheduling algorithms prioritize tasks using a variety of methods, resulting in long execution times on heterogeneous distributed computing systems. As a result, task prioritization becomes a bottleneck in such systems. It is appropriate to prioritize tasks with the shortest execution time using faster algorithms. The genetic algorithm (GA) is one of the evolutionary approaches used to solve complex problems quickly. This paper proposes a parallel GA with a MapReduce architecture for scheduling jobs on cloud computing with various priority queues. The fundamental aim of this study is to employ a MapReduce architecture to minimize the total execution time of the task scheduling process in the cloud computing environment. The proposed method accomplishes task scheduling in two stages: first, the GA was used in conjunction with heuristic techniques to assign tasks to processors, and then the GA was used in conjunction with the MapReduce framework to assign jobs to processors. In our experiments, we consider heterogeneous resources that differ in their ability to execute various tasks, as well as running a job on different resources with varying execution durations. The results show that the proposed method outperforms other algorithms such as particle swarm optimization, whale optimization algorithm, moth-flame optimization, and intelligent water drops.
A User-Centered Medical Data Sharing Scheme for Privacy-Preserving Machine Learning

With the rapid development and application of artificial intelligence technology, medical data play an increasingly important role in the medical field. However, there are privacy protection and data ownership issues in the process of data sharing, which brings difficulties to machine learning and data mining. On the one hand, for fear that they may risk being held accountable by users or even breaking the law due to these issues, healthcare providers are reluctant to share medical data. On the other hand, users are also reluctant to share medical data due to the possibility of privacy disclosure in the data sharing process. To improve the security and privacy of shared medical data, we propose a user-centered medical data sharing scheme for privacy-preserving machine learning. Our solution combines blockchain and a trusted execution environment to ensure that adversaries cannot steal the ownership and control of user data during sharing. A blockchain-based noninteractive key sharing scheme is proposed that allows only the users and the TEE to decrypt the shared data. At the same time, we design an auditing mechanism to facilitate users to audit the sharing process. The security analysis shows that the scheme ensures the privacy and security of user data during storage and sharing. We have completed simulation experiments to demonstrate the effectiveness and efficiency of our scheme.
Ore fluid characteristics and depositional processes of the hydrothermal gold mineralisation at Damang, Southwestern Ghana

Gold mineralisation at Damang in the southwestern part of Ghana occurs in two types of geological settings: (1) as paleoplacer in the Banket Series conglomerates of the Tarkwaian system and (2) in hydrothermal sheeted quartz vein system that crosscuts all the Tarkwaian lithologies including the palaeoplacer units. This paper focused on the second type because not much research has be conducted on the fluid inclusion and stable isotope characteristics of the Damang deposit to reveal the nature of the hydrothermal ore-forming associated the quartz veins and gold deposition. Microthermometry measurements of fluid inclusions in mineralised quartz veins indicated two sets of temperature-salinity variations: a relatively low temperature (142–292 °C) and moderate salinity aqueous fluids that range from 2.4 to 8.9 wt% NaCl equivalent, and a high temperature (210–393.6 °C) with low salinity aqueous-carbonic fluids with the salinity ranging from 1.0 to 2.6 wt% NaCl equivalent. These fluids are inferred to have unmixed from a homogeneous fluid with an average salinity of 3.85 ± 2.42 wt % NaCl equivalent. Laser Raman spectroscopic data on the fluid inclusions confirmed an aqueous-carbonic (H2O–CO2–NaCl) composition of the fluids with minor amounts of H2S, CH4 and SO2 as volatile species. The temperature of gold precipitation based on overlapped temperatures of chlorite, fluid inclusions and stable isotope geothermometers, indicated a mean temperature of 324.7 ± 20 °C and corresponding pressures from 1.1 to 3.5 kbar. The calculated oxygen isotopic composition of the ore fluids, δ18Ofluid using the estimated mineralisation temperature indicated average values of 9.3 ± 1.2‰. This isotopic range and the low salinity of the fluids (<10 wt % NaCl equiv.) are suggestive of metamorphic origin, which may be attributed to devolatisation reactions in the underlying Birimian greenstones. This is consistent with orogenic lode gold mineralisation in many Precambrian greenstone belts worldwide. The close association of gold with sulphides is consistent with gold transport as bisulphide complexes. A combination of factors including sulphidation and phase separation, concomitant with temperature decrease are inferred to be the principal factors that controlled the hydrothermal gold precipitation at Damang.
The Impact of an In-Home Co-Located Robotic Coach in Helping People Make Fewer Exercise Mistakes

Regular exercise provides many mental and physical health benefits. However, when exercises are done incorrectly, it can lead to injuries. Because the COVID-19 pandemic made it challenging to exercise in communal spaces, the growth of virtual fitness programs was accelerated, putting people at risk of sustaining exercise-related injuries as they received little to no feedback on their exercising techniques. Co-located robots could be one potential enhancement to virtual training programs as they can cause higher learning gains, more compliance, and more enjoyment than non-co-located robots. In this study, we compare the effects of a physically present robot by having a person exercise either with a robot (robot condition) or a video of a robot displayed on a tablet (tablet condition). Participants (N=25) had an exercise system in their homes for two weeks. Participants who exercised with the co-located robot made fewer mistakes than those who exercised with the video-displayed robot. Furthermore, participants in the robot condition reported a higher fitness increase and more motivation to exercise than participants in the tablet condition.
NATURAL-SIMULATION MODEL OF PHOTOVOLTAIC STATION GENERATION IN PROCESS OF ELECTRICITY BALANCING IN ELECTRICAL POWER SYSTEM

The paper analyzes the methods of stabilizing generation schedules of photovoltaic stations (PV) in electric power systems (EPS) in the process of balancing electricity. Since PV is characterized by the instability of electricity production due to dependence on weather conditions, an automatic system for forecasting their generation schedules (ASFG) for the next day has been created to increase the energy efficiency of PV. The process of automating the prediction of the power stations as part of the balancing group of the power stations and the algorithm for adjusting the prediction of power plant generation are considered. The criterion for managing the forecasting process is the minimization of the difference between the values ​ of forecasted and actual generation for the same period of time. Checking the performance and tuning of the ASFG PV in order to evaluate its functioning and the effectiveness of its application in the task of balancing the states of the EPS is possible only by means of simulation. It is shown that based on the nature of the process of forecasting the generation of PV using ASFG, it is advisable to use simulation modeling. Since the actual value of generation is constantly monitored during balancing using ASFG, it is possible to use these values ​​during simulation and proceed to real-time simulation. In this case, modeling is considered as an experimental method of research, according to which it is not the object itself that is subjected to perturbations and research, but the software-implemented computer model of the object. The real-life simulation model of the operation of the PV makes it possible to more fully consider the various modes of their operation in the process of balancing the modes of the EPS as part of the balancing group and to more reasonably choose decisions regarding the participation of the PV in generation, taking into account weather conditions and the limitations of the system operators of transmission and distribution of electricity.
Does information communication promote financial development? Empirical evidence from China

Sound information communication is an important guarantee of financial development. In this study, the impact of information communication on financial development is analyzed using panel data on Chinese cities. The instrumental variable method is adopted to deal with endogeneity between information communication and financial development, and a quasi-natural experiment is conducted to further alleviate endogeneity. Finally, we consider the heterogeneity of the impact of information communicated on financial development. The results show that information communication facilities, information communication levels, and information communication penetration rates can promote urban financial development. Financial development improves significantly in the pilot cities under the implementation of “Broadband China” strategy. And the impact of information communicated on financial development is more obvious in cities in the Northeast Economic Circle, the Pan-Pearl River Delta Economic Circle, the Southwest Economic Circle in China, and cities with a low level of foreign investment.
Attention-Guided Collaborative Counting

Existing crowd counting designs usually exploit multi-branch structures to address the scale diversity problem. However, branches in these structures work in a competitive rather than collaborative way. In this paper, we focus on promoting collaboration between branches. Specifically, we propose an attention-guided collaborative counting module (AGCCM) comprising an attention-guided module (AGM) and a collaborative counting module (CCM). The CCM promotes collaboration among branches by recombining each branch's output into an independent count and joint counts with other branches. The AGM capturing the global attention map through a transformer structure with a pair of foreground-background related loss functions can distinguish the advantages of different branches. The loss functions do not require additional labels and crowd division. In addition, we design two kinds of bidirectional transformers (Bi-Transformers) to decouple the global attention to row attention and column attention. The proposed Bi-Transformers are able to reduce the computational complexity and handle images in any resolution without cropping the image into small patches. Extensive experiments on several public datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art crowd counting methods.
Adaptive Federated Learning With Non-IID Data

Abstract With the widespread use of Internet of things(IoT) devices, it generates an enormous volume of data, and it is a challenge to mine the IoT data value while ensuring security and privacy. Federated learning is a decentralized approach for training data located on edge devices, such as mobile phones and IoT devices, while keeping privacy, efficiency, and security. However, the Non-IID (non-independent and identically distributed) data, always greatly impacts the performance of the global model. In this paper, we propose a FedDynamic algorithm to solve the statistical challenge of federated learning caused by Non-IID. As Non-IID data can lead to significant differences in model parameters between edge devices, we set different weights for different devices during model aggregation to get a high-performance global model. We analyze and exact key indices (local model accuracy, local data quality, and model difference between local models and the global model), which can reflect the quality of the model, and calculate the aggregation weight for edge devices based on the key indices. Furthermore, we dynamically adjust aggregation weight based on accuracy’s variety to solve weight staleness during the training process. Experiments on the MNIST, FMNIST, EMNIST, CINIC-10 and CIFAR-10 datasets show that the FedDynamic algorithm has better accuracy and convergence performance, compared to the FedAvg, FedProx and Scaffold algorithms.
ACR-MLM: a privacy-preserving framework for anonymous and confidential rewarding in blockchain-based multi-level marketing

Network marketing is a trading technique that provides companies with the opportunity to increase sales. With the increasing number of Internet-based purchases, several threats are increasingly observed in this field, such as use privacy violations, company owner (CO) fraud, the changing of sold products' information, and the scalability of selling networks. This study presents the concept of a blockchain-based market called ACR-MLM that functions based on the multi-level marketing (MLM) model, through which registered users receive anonymous and confidential rewards for their own and their subgroups' sales. Applying a public blockchain as the ACR-MLM framework's infrastructure solves existing problems in MLM-based markets, such as CO fraud (against the government or its users), user privacy violations (obtaining their real names or subgroup users), and scalability (when vast numbers of users have been registered). To provide confidentiality and scalability to the ACR-MLM framework, hierarchical identity-based encryption (HIBE) was applied with a functional encryption (FE) scheme. Finally, the security of ACR-MLM is analyzed using the random oracle (RO) model and then evaluated.