Reviewerly/exHarmony huggingface dataset:
ArtCap: A Dataset for Image Captioning of Fine Art Paintings

The image captioning of fine art paintings aims at generating content descriptions for the paintings. Due to the complexity of modeling both image and language, this task usually needs sufficient training data. However, different from photographic image captioning, there are few satisfactory datasets for painting captioning. In this article, we introduce a painting captioning dataset (named the ArtCap dataset), which contains 3606 paintings and five descriptions for each painting. We present the carefully designed construction pipeline of our dataset and further evaluate our dataset from two aspects of annotation quality and application effectiveness, respectively. For the annotation quality, we compare the global characteristics, annotation content, and annotation consistency of our dataset with other painting descriptions datasets. For application effectiveness, we employ our dataset and other painting descriptions datasets to train image captioning models and analyze the captioning performances. The results demonstrate the promising annotation quality and application effectiveness of our dataset.
An Emotion Recognition Method Based on Eye Movement and Audiovisual Features in MOOC Learning Environment

In recent years, more and more people have begun to use massive online open course (MOOC) platforms for distance learning.However, due to the space-time isolation between teachers and students, the negative emotional state of students in MOOC learning cannot be identified timely.Therefore, students cannot receive immediate feedback about their emotional states.In order to identify and classify learners' emotions in video learning scenarios, we propose a multimodal emotion recognition method based on eye movement signals, audio signals, and video images.In this method, two novel features are proposed: feature of coordinate difference of eyemovement (FCDE) and pixel change rate sequence (PCRS).FCDE is extracted by combining eye movement coordinate trajectory and video optical flow trajectory, which can represent the learner's attention degree.PCRS is extracted from the video image, which can represent the speed of image switching.A feature extraction network based on convolutional neural network (CNN) (FE-CNN) is designed to extract the deep features of the three modals.The extracted deep features are inputted into the emotion classification CNN (EC-CNN) to classify the emotions, including interest, happiness, confusion, and boredom.In single modal identification, the recognition accuracies corresponding to the three modals are 64.32%,74.67%, and 71.88%.The three modals are fused by feature-level fusion, decision-level fusion, and modellevel fusion methods, and the evaluation experiment results show that the method of decision-level fusion achieved the highest score of 81.90% of emotion recognition.Finally, the effectiveness of FCDE, FE-CNN, and EC-CNN modules is verified by ablation experiments.
Assessment of Wealth Distribution in Blockchain Online Social Media

Online social networks (OSNs) revolutionized how people interact with each other, and nowadays, thanks to blockchain technology, new solutions are being considered, giving birth to blockchain online social media (BOSMs). BOSMs use the blockchain to redistribute with their users the wealth generated by the platform through a rewarding system, assigning better rewards to socially impactful users. Thus, these new systems are characterized by highly intertwined economical and social aspects and constitute a new scenario in the world of social networks. Many scenarios, economic and social alike, show a phenomenon known as “the rich-get-richer”, which states that the richest actors of a system tend to become richer over time. To the best of our knowledge, in the scenario of BOSMs, where users can acquire cryptocurrency through their social actions, this type of phenomenon was not yet studied. In this article, we propose a methodological framework composed of three hypotheses that can help study the rich-get-richer phenomenon through a set of measures and indices. In addition, we apply the proposed framework to the Steem case study, showing how unevenly wealth is distributed on its blockchain and comparing our results to other scenarios.
MTBullyGNN: A Graph Neural Network-Based Multitask Framework for Cyberbullying Detection

Cyberbullying is a malady of social media, and its automatic detection is critically important considering its virulence, velocity of spreading, and the scale of the havoc it can wreak. However, the problem is challenging due to its disguised behavior, noise in the content, and, in recent times, introduction of code-mixing. In this work, we propose MTBullyGNN a novel graph neural network (GNN)-based multitask (MT) framework that solves sentiment-aided cyberbullying detection (CD) from code-mixed language. The GNN helps detect unlabelled or noisy label nodes (sentences) accurately by aggregating information from similarly labeled nodes. To connect nodes, we apply cosine similarity between sentences and create a single text graph for a benchmark code-mixed cyberbullying corpus, BullySent. Experimental results illustrate that MTBullyGNN outperforms the state-of-the-art (SOTA) methods for both the single (CD) and MT (CD and sentiment) settings by up to 4.46% and 4.92% in classification accuracy, respectively. Furthermore, another benchmark Hindi–English code-mixed single-task dataset has also been considered to illustrate the robustness of our proposed model. The code will be made publicly available in the camera-ready version.
Visual Analysis of Money Laundering in Cryptocurrency Exchange

Blockchain-based cryptocurrencies, such as Bitcoin (BTC) and Ethereum (ETH), are newly emerging financial assets. Cryptocurrency exchanges are marketplaces for cryptocurrency circulation while becoming a new venue for money laundering. In this work, we cooperate with a cryptocurrency exchange to investigate new solutions for anti-money laundering in cryptocurrency exchanges. First, we learn the domain knowledge of cryptocurrency transactions and summarize data analytical requirements of transaction supervisors in their daily work of anti-money laundering. Then, we propose a visual analysis approach to support their daily work. The approach consists of a new algorithm that automatically detects suspicious money laundering accounts and a multiviewed user interface that visualizes the algorithm results and relevant transaction data. An abacus-inspired visualization is designed in the interface to depict transaction patterns contained in numerous cryptocurrency transactions, which can help supervisors find money laundering clues and deduce the trading tactic adopted by launderers. Finally, an algorithm performance experiment, a case study, and a field study are conducted with real-world data to demonstrate the effectiveness of our solution.
Blockchain Oriented Effective Charity Process During Pandemics and Emergencies

During any emergency, a donation is considered a moral responsibility all over the globe. The lack of transparency and oversight in charity donations hurts people’s enthusiasm to donate. Donors are distrustful about how their funds are utilized. The use of blockchain technology (BCT) will provide a solution to make the donation procedure more viable. It is a distributed technology that offers a secure and transparent environment by avoiding the involvement of third parties between contributors and charities. This article proposed a blockchain-based donation mechanism for the convenience of charity organizations, donors, and beneficiaries during disasters, pandemics such as Covid-19, and other emergencies. All transactions can be traced in blockchain, giving donors visibility into where and how their funds are utilized. This article contributes to improving donations’ openness to strengthen public interest in donations and encourage BCT in charity. Ethereum blockchain is used to implement the proposed framework and provides a convenient donation platform. Smart contracts are used to make donations, which build trust between contributors, beneficiaries, and charity organizations. The blockchain-based donation method saves time, lowers donation costs, and eliminates the chances of dubious campaign funds. This study will contribute to improving emergency recovery efforts.
Zooming Into Video Conferencing Privacy

The COVID-19 pandemic outbreak, with its related social distancing and shelter-in-place measures, has dramatically affected ways in which people communicate with each other, forcing people to find new ways to collaborate, study, celebrate special occasions, and meet with family and friends. One of the most popular solutions that have emerged is the use of video conferencing applications to replace face-to-face meetings with virtual meetings. This resulted in unprecedented growth in the number of video conferencing users. In this study, we explored privacy issues that may be at risk by attending virtual meetings. We extracted private information from collage images of meeting participants that are publicly posted on the Web. We used image processing, text recognition tools, as well as social network analysis to explore our web crawling curated dataset of over 15,700 collage images, and over 142,000 face images of meeting participants. We demonstrate that video conference users are facing prevalent security and privacy threats. Our results indicate that it is relatively easy to collect thousands of publicly available images of video conference meetings and extract personal information about the participants, including their face images, age, gender, usernames, and sometimes even full names. This type of extracted data can vastly and easily jeopardize people's security and privacy both in the online and real-world, affecting not only adults but also more vulnerable segments of society, such as young children and older adults. Finally, we show that cross-referencing facial image data with social network data may put participants at additional privacy risks they may not be aware of and that it is possible to identify users that appear in several video conference meetings, thus providing a potential to maliciously aggregate different sources of information about a target individual.
Hadoop-based secure storage solution for big data in cloud computing environment

In order to address the problems of the single encryption algorithm, such as low encryption efficiency and unreliable metadata for static data storage of big data platforms in the cloud computing environment, we propose a Hadoop based big data secure storage scheme. Firstly, in order to disperse the NameNode service from a single server to multiple servers, we combine HDFS federation and HDFS high-availability mechanisms, and use the Zookeeper distributed coordination mechanism to coordinate each node to achieve dual-channel storage. Then, we improve the ECC encryption algorithm for the encryption of ordinary data, and adopt a homomorphic encryption algorithm to encrypt data that needs to be calculated. To accelerate the encryption, we adopt the dual-thread encryption mode. Finally, the HDFS control module is designed to combine the encryption algorithm with the storage model. Experimental results show that the proposed solution solves the problem of a single point of failure of metadata, performs well in terms of metadata reliability, and can realize the fault tolerance of the server. The improved encryption algorithm integrates the dual-channel storage mode, and the encryption storage efficiency improves by 27.6% on average.
A Novel DAO-Based Parallel Enterprise Management Framework in Web3 Era

This article proposes a novel parallel management mode based on decentralized autonomous organizations (DAOs) for enterprises by utilizing the artificial systems, computational experiments, parallel execution (ACP) approach, parallel intelligence theory, and blockchain technologies, to realize the distributed management of an enterprise. The artificial enterprise DAO (EnDAO) corresponding to the actual enterprise is constructed, and they constitute a parallel system via virtual–real interaction and parallel execution. Through the non-fungible token (NFT)-based incentive mechanism, metaverse-based virtual learning and training, as well as DAO-based distributed management and decision-making, the management and control of the actual enterprise as well as its employees can be carried out. By virtue of the virtual–real interactions of three types of employees, as well as the virtual–real feedback of three closed loops in the parallel systems, DAO-based parallel management for enterprises can realize descriptive intelligence, predictive intelligence, and prescriptive intelligence. On this basis, this article takes the recruitment-oriented key performance indicator (KPI) management of a startup technology enterprise as the case to introduce the operation processes and illustrate the superiorities of the proposed DAO-based enterprise parallel management mode.
Transformer and GAN-Based Super-Resolution Reconstruction Network for Medical Images

Super-resolution reconstruction in medical imaging has become more demanding due to the necessity of obtaining high-quality images with minimal radiation dose, such as in low-field magnetic resonance imaging (MRI). However, image super-resolution reconstruction remains a difficult task because of the complexity and high textual requirements for diagnosis purpose. In this paper, we offer a deep learning based strategy for reconstructing medical images from low resolutions utilizing Transformer and generative adversarial networks (T-GANs). The integrated system can extract more precise texture information and focus more on important locations through global image matching after successfully inserting Transformer into the generative adversarial network for picture reconstruction. Furthermore, we weighted the combination of content loss, adversarial loss, and adversarial feature loss as the final multi-task loss function during the training of our proposed model T-GAN. In comparison to established measures like peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM), our suggested T-GAN achieves optimal performance and recovers more texture features in super-resolution reconstruction of MRI scanned images of the knees and belly.
Hybrid ultra-short-term PV power forecasting system for deterministic forecasting and uncertainty analysis

The rapid development of the photovoltaic industry provides a new source of power for the continued operation of the over-consumed energy world. While providing new opportunities for global energy systems, it also poses challenges for power grids. Therefore, it is a priority to fully grasp the characteristics of photovoltaic changes and accurately forecast and analyze them. To enrich the existing research, a novel hybrid prediction system considering meteorological factors is constructed. First, a feature selection module is introduced to select features and assign weights to exogenous meteorological factors, which breaks through the limitations of single-data dimension prediction. Second, shallow and deep learning models are flexibly applied and multi-objective intelligent optimization strategies are introduced to construct deterministic combinatorial prediction models. The module can effectively increase the diversity of prediction models while fully weighing the accuracy and stability of prediction to meet the needs of different information users. Finally, an interval prediction model is constructed to further enrich the PV power prediction system from the perspective of uncertainty analysis. The empirical study is carried out with 5-minute interval data at three sites, and the results show that the hybrid system obtains superior out-of-sample forecasting performance with technical feasibility and general applicability.
Dense Text Retrieval Based on Pretrained Language Models: A Survey

Text retrieval is a long-standing research topic on information seeking, where a system is required to return relevant information resources to user’s queries in natural language. From heuristic-based retrieval methods to learning-based ranking functions, the underlying retrieval models have been continually evolved with the ever-lasting technical innovation. To design effective retrieval models, a key point lies in how to learn text representations and model the relevance matching. The recent success of pretrained language models (PLM) sheds light on developing more capable text-retrieval approaches by leveraging the excellent modeling capacity of PLMs. With powerful PLMs, we can effectively learn the semantic representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling. Such a retrieval approach is called dense retrieval , since it employs dense vectors to represent the texts. Considering the rapid progress on dense retrieval, this survey systematically reviews the recent progress on PLM-based dense retrieval. Different from previous surveys on dense retrieval, we take a new perspective to organize the related studies by four major aspects, including architecture, training, indexing and integration, and thoroughly summarize the mainstream techniques for each aspect. We extensively collect the recent advances on this topic, and include 300+ reference papers. To support our survey, we create a website for providing useful resources, and release a code repository for dense retrieval. This survey aims to provide a comprehensive, practical reference focused on the major progress for dense text retrieval.
Fusion of transfer learning models with LSTM for detection of breast cancer using ultrasound images

Breast Cancer (BC) is one of the top reasons for fatality in women worldwide. As a result, timely identification is critical for successful therapy and excellent survival rates. Transfer Learning (TL) approaches have recently shown promise in aiding in the early recognition of BC. In this work, three TL models, MobileNetV2, ResNet50, and VGG16, were combined with LSTM to extract the features from Ultrasound Images (USIs). Furthermore, the Synthetic Minority Over-sampling Technique (SMOTE) with Tomek (SMOTETomek) was employed to balance the extracted features. The proposed method with VGG16 achieved an F1 score of 99.0 %, Matthews Correlation Coefficient (MCC) and Kappa Coefficient of 98.9 % with an Area Under Curve (AUC) of 1.0. The K-fold method was applied for cross-validation and achieved an average F1 score of 96 %. Moreover, the Gradient-weighted Class Activation Mapping (Grad-CAM) method was applied for visualization, and the Local Interpretable Model-agnostic Explanations (LIME) method was applied for interpretability. The Normal Approximation Interval (NAI) and bootstrapping methods were used to calculate Confidence Intervals (CIs). The proposed method achieved a Lower CI (LCI), Upper CI (UCI), and Mean CI (MCI) of 96.50 %, 99.75 %, and 98.13 %, respectively, with the NAI, while 95 % LCI of 93.81 %, an UCI of 96.00 %, and a bootstrap mean of 94.90 % with the bootstrap method. Furthermore, the performance of the six state-of-the-art (SOTA) TL models, such as Xception, NASNetMobile, InceptionResNetV2, MobileNetV2, ResNet50, and VGG16, were compared with the proposed method.
A Novel Tensor Decomposition-Based Efficient Detector for Low-Altitude Aerial Objects With Knowledge Distillation Scheme

Unmanned aerial vehicles (UAVs) have gained significant attention in practical applications, especially the low-altitude aerial (LAA) object detection imposes stringent requirements on recognition accuracy and computational resources. In this paper, the LAA images-oriented tensor decomposition and knowledge distillation-based network (TDKD-Net) is proposed, where the TT-format TD (tensor decomposition) and equal-weighted response-based KD (knowledge distillation) methods are designed to minimize redundant parameters while ensuring comparable performance. Moreover, some robust network structures are developed, including the small object detection head and the dual-domain attention mechanism, which enable the model to leverage the learned knowledge from small-scale targets and selectively focus on salient features. Considering the imbalance of bounding box regression samples and the inaccuracy of regression geometric factors, the focal and efficient IoU (intersection of union) loss with optimal transport assignment (F-EIoU-OTA) mechanism is proposed to improve the detection accuracy. The proposed TDKD-Net is comprehensively evaluated through extensive experiments, and the results have demonstrated the effectiveness and superiority of the developed methods in comparison to other advanced detection algorithms, which also present high generalization and strong robustness. As a resource-efficient precise network, the complex detection of small and occluded LAA objects is also well addressed by TDKD-Net, which provides useful insights on handling imbalanced issues and realizing domain adaptation.
Spherical autoregressive models, with application to distributional and compositional time series

We introduce a new class of autoregressive models for spherical time series. The dimension of the spheres on which the observations of the time series are situated may be finite-dimensional or infinite-dimensional, where in the latter case we consider the Hilbert sphere. Spherical time series arise in various settings. We focus here on distributional and compositional time series. Applying a square root transformation to the densities of the observations of a distributional time series maps the distributional observations to the Hilbert sphere, equipped with the Fisher–Rao metric. Likewise, applying a square root transformation to the components of the observations of a compositional time series maps the compositional observations to a finite-dimensional sphere, equipped with the geodesic metric on spheres. The challenge in modeling such time series lies in the intrinsic non-linearity of spheres and Hilbert spheres, where conventional arithmetic operations such as addition or scalar multiplication are no longer available. To address this difficulty, we consider rotation operators to map observations on the sphere. Specifically, we introduce a class of skew-symmetric operators such that the associated exponential operators are rotation operators that for each given pair of points on the sphere map the first point of the pair to the second point of the pair. We exploit the fact that the space of skew-symmetric operators is Hilbertian to develop autoregressive modeling of geometric differences that correspond to rotations of spherical and distributional time series. Differences expressed in terms of rotations can be taken between the Fréchet mean and the observations or between consecutive observations of the time series. We derive theoretical properties of the ensuing autoregressive models and showcase these approaches with several motivating data. These include a time series of yearly observations of bivariate distributions of the minimum/maximum temperatures for a period of 120 days during each summer for the years 1990-2018 at Los Angeles (LAX) and John F. Kennedy (JFK) international airports. A second data application concerns a compositional time series with annual observations of compositions of energy sources for power generation in the U.S..