Reviewerly/exHarmony huggingface dataset:
Image captioning based on scene graphs: A survey

Although recent developments in deep learning have brought several tasks closer to human performance, there is still a significant gap between human and machine performance in certain image captioning tasks. Image captioning is the process of creating a textual description of an image. Image captioning focuses on recognizing the main regions of an image, their attributes, and their relationships. It aims to generate textual descriptions that are syntactically and semantically correct. For simple image descriptions, deep learning-based techniques perform well in terms of intricacies and constraints. However, it is challenging to construct sentences when faced with complicated scenes with many entities and relationships, such as how to concurrently solve diversity, anchoring, and controllability—a seemingly simple ability for humans. Scene graphs can significantly alleviate this problem by fully mining spatial and semantic information. However, despite these promising findings, they are fragmented and do not form a systematic comparative overview. We provide a comprehensive overview of the available scene-graph-based image captioning methods in this survey. The foundations of these techniques are discussed to examine their performance, strengths, and constraints. Furthermore, we discuss the comparisons of the state-of-the-art methods, datasets, and commonly utilized evaluation measures. Finally, we conclude the survey with an in-depth discussion of the present and future research challenges. This study will assist readers in comprehending how scene graphs can be applied to image captioning.
Atmospheric Scattering Model Induced Statistical Characteristics Estimation for Underwater Image Restoration

Underwater images often suffer from color deviation and low contrast due to selective absorption and light scattering, whose degradation is generally described by an Atmospheric Scattering Model (ASM). However, it is challenging to design hand-craft priors to estimate the transmission map and global light within ASM. To avoid the estimation on these two variables, in this paper, we establish a statistical characteristics relationship between underwater and recovered images based on ASM. With this relationship, a novel lightweight model is proposed for efficient Underwater Image Restoration (UIR). Within our proposed model, the UIR problem is disentangled into global restoration and local compensation, for which two modules are developed. Extensive experimental results demonstrate that our proposed method can effectively improve color deviation and low contrast while preserving details, and outperform state-of-the-art methods.
NLP Methods’ Information Extraction for Textual Data: An Analytical Study

Information Extraction (IE) is the process of automatically extracting pertinent information from unstructured or semi-structured data, and it typically involves the analysis of human language text through natural language processing (NLP). Rules-based methods (RBM), Supervised-learning-based methods, and Unsupervised-Learning-based methods are the three basic methods used by the IE system. This work aims to explore, analyze the various approaches, and illustrate the difficulties encountered while using textual data in different forms, domains, and sizes of datasets from a preexisting information extraction using various categories of IE methods. This study presents an analytical study of various approaches to different information extraction methods used to analysis of textual data.
Propounding First Artificial Intelligence Approach for Predicting Robbery Behavior Potential in an Indoor Security Camera

Crime prediction in video-surveillance systems is required to prevent incident and protect assets.In this sense, our article proposes first artificial intelligence approach for Robbery Behavior Potential (RBP) prediction and detection in an indoor camera.Our method is based on three detection modules including head cover, crowd and loitering detection modules for timely actions and preventing robbery.The two first modules are implemented by retraining YOLOV5 model with our gathered dataset which is annotated manually.In addition, we innovate a novel definition for loitering detection module which is based on DeepSORT algorithm.A fuzzy inference machine renders an expert knowledge as rules and then makes final decision about predicted robbery potential.This is laborious due to: different manner of robber, different angle of surveillance camera and low resolution of video images.We accomplished our experiment on real world video surveillance images and reaching the F1-score of 0.537.Hence, to make an experimental comparison with the other related works, we define threshold value for RBP to evaluate video images as a robbery detection problem.Under this assumption, the experimental results show that the proposed method performs significantly better in detecting the robbery as compared to the robbery detection methods by distinctly report with F1-score of 0.607.We strongly believe that the application of the proposed method could cause reduction of robbery detriment in a control center of surveillance cameras by predicting and preventing incident of robbery.On the other hand, situational awareness of human operator enhances and more cameras can be managed.
Can the Metaverse and Its Associated Digital Tools and Technologies Provide an Opportunity for Destinations to Address the Vulnerability of Overtourism?

Overtourism has become a critical problem in many popular destinations around the world, leading to negative impacts on the environment, local communities, and the quality of the visitor experience. This article explores the potential of the metaverse, a virtual world that combines elements of augmented reality and virtual reality, and other new digital technologies for addressing the challenges of overtourism. Drawing on a systematic literature review of recent studies, the currently available technologies are listed, and their potential implications for the tourism industry and local communities are portrayed. It was found that while there is some evidence to suggest that the metaverse and its associated digital technologies such as the extended realities/XR (virtual reality/VR, augmented reality/AR, mixed reality/MR), can help mitigate overtourism, further research is necessary to investigate their effectiveness in reducing overtourism and their potential unintended consequences. This article makes an original contribution to overtourism research and adds to existing knowledge by providing a practical list of currently available technologies that could help DMOs and tourism professionals mitigate the negative effects of overtourism. The article concludes with suggestions for future research and practice in the field, highlighting the importance of continued exploration and innovation in this area.
Strategy for implementing blockchain technology in accounting: Perspectives of stakeholders in a developing nation

Abstract This study applies the Unified Theory of Acceptance and Use of Technology model to explore the factors that potential users may prioritize when intending to use blockchain technology in accounting. Primary data was collected through a structured questionnaire from 217 respondents. Both factor analysis and regression models were applied to achieve the study's purposes. The findings indicate that respondents consider privacy, usefulness, cost, and facilitation issues when intending to use blockchain technology in accounting to enhance data reliability and reduce the workload of auditors. Bangladesh, a developing country, has immense opportunities to ensure a transparent society by applying blockchain technology in accounting, with cooperation from stakeholders such as firms, the government, regulatory authorities, and users of accounting information. It is expected that the findings of this study will help the concerned authorities to formulate appropriate strategies for the implementation of blockchain technology in the field of accounting in Bangladesh.
Container Allocation in Cloud Environment Using Multi-Agent Deep Reinforcement Learning

Nowadays, many computation tasks are carried out using cloud computing services and virtualization technology. The intensive resource requirements of virtual machines have led to the adoption of a lighter solution based on containers. Containers isolate packaged applications and their dependencies, and they can also operate as part of distributed applications. Containers can be distributed over a cluster of computers with available resources, such as the CPU, memory, and communication bandwidth. Any container distribution mechanism should consider resource availability and their impact on overall performance. This work suggests a new approach to assigning containers to servers in the cloud, while meeting computing and communication resource requirements and minimizing the overall task completion time. We introduce a multi-agent environment using a deep reinforcement learning-based decision mechanism. The high action space complexity is tackled by decentralizing the allocation decisions among multiple agents. Considering the interactions among the agents, we introduce a new cooperative mechanism for a state and reward design, resulting in efficient container assignments. The performances of both long short term memory (LSTM) and memory augmented-based agents are examined, for solving the challenging container assignment problem. Experimental results demonstrated an improvement of up to 28% in the execution runtime compared to existing bin-packing heuristics and the common Kubernetes industrial tool.
Experimenting in the cloud: The digital divide's impact on innovation

This paper builds on a recent National Science Foundation working paper that identified a strong association between cloud use and various types of innovation but did not consider whether 1) cloud adoption is a reliable indicator of the innovation orientation of a firm, or 2) cloud adoption enables various types of innovation. We estimate propensity score matching and endogenous treatment effect models to control for innovation orientation, producing evidence to test the second explanation. Findings support an enabling effect of the cloud on innovation providing concrete evidence of the adverse impact of the digital divide.
Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.
Explainable AI and Causal Understanding: Counterfactual Approaches Considered

Abstract The counterfactual approach to explainable AI (XAI) seeks to provide understanding of AI systems through the provision of counterfactual explanations. In a recent systematic review, Chou et al. (Inform Fus 81:59–83, 2022) argue that the counterfactual approach does not clearly provide causal understanding . They diagnose the problem in terms of the underlying framework within which the counterfactual approach has been developed. To date, the counterfactual approach has not been developed in concert with the approach for specifying causes developed by Pearl (Causality: Models, reasoning, and inference. Cambridge University Press, 2000) and Woodward (Making things happen: A theory of causal explanation. Oxford University Press, 2003). In this paper, I build on Chou et al.’s work by applying the Pearl-Woodward approach. I argue that the standard counterfactual approach to XAI is capable of delivering causal understanding, but that there are limitations on its capacity to do so. I suggest a way to overcome these limitations.
Ontology and Software Tools for the Formalization of the Visualisation of Cultural Heritage Knowledge Graphs

Over the last decade, several projects have been developed to digitise and semanticise cultural heritage data. They have been developed to preserve and maintain this heritage, but also to make it accessible to all types of users and to other sectors such as tourism and education. These developments combine the use of knowledge graphs and interactive visualisation tools with web technologies. Although remarkably interesting projects have been developed, the data visualisation tools in these projects tend to focus on the project context. Consequently, it is difficult to reuse the results of these projects. In addition, there are characteristics of cultural heritage information, such as uncertainty and spatial and temporal granularity, that have not been considered, and how to deal with them has not been described. The same is true for other aspects, such as the relationships between different objects. Considering these problems, this study presents a model that formalises how to visualise this information. The design of an ontology that implements this model, based on other works such as VUMO or VISO, is described. Furthermore, the design and development of a software framework that allows the visualisation of this information through a web application are presented. The evaluation of the application of this framework in projects such as SILKNOW or Arxiu Valencià del Disseny is outlined.
Point-of-interest Recommendation using Deep Semantic Model

Under the current paradigm, POI (Point-of-interest) recommendation tasks are mainly focused on representation learning. Therefore, the quality of the trajectory embeddings plays a key role in prediction. Existing methods mainly focus on learning the contextual information of check-in sequences, however, the contextual information is not the only medium for establishing state transitions between check-ins. Semantic information has been proven a powerful medium that can be learned with semantic data and injected into the source sequential embeddings for better prediction in other recommendation tasks. Unfortunately, for POI recommendation tasks, most of the key elements in the trajectory are discrete and there is no explicit semantic information. Therefore we argue that the deep and rich semantic information hidden in trajectories has not been fully exploited currently and how learning the deep semantic information from discrete trajectory data to improve the quality of the trajectory embedding is the key to further improving recommendation performance. We propose DSMR, a deep semantic recommender model for the next POI recommendation task to mitigate the issue. Specifically, we use prompt engineering to carry out continuous semantic modeling of discrete trajectory data and use the pre-trained language model to extract its implicit deep semantic information to establish causal transfer constraints between check-ins through the medium of semantics. Meanwhile, we propose a new position encoding function, temporal interval encoding, to avoid the neglect of temporal information of the check-ins sequence under the self-attention mechanism. Extensive experiments on two real-world datasets demonstrate the superior performance of our model to state-of-the-art approaches.
Attention-based multimodal sentiment analysis and emotion recognition using deep neural networks

There has been a growing interest in multimodal sentiment analysis and emotion recognition in recent years due to its wide range of practical applications. Multiple modalities allow for the integration of complementary information, improving the accuracy and precision of sentiment and emotion recognition tasks. However, working with multiple modalities presents several challenges, including handling data source heterogeneity, fusing information, aligning and synchronizing modalities, and designing effective feature extraction techniques that capture discriminative information from each modality. This paper introduces a novel framework called "Attention-based Multimodal Sentiment Analysis and Emotion Recognition (AMSAER)" to address these challenges. This framework leverages intra-modality discriminative features and inter-modality correlations in visual, audio, and textual modalities. It incorporates an attention mechanism to facilitate sentiment and emotion classification based on visual, textual, and acoustic inputs by emphasizing relevant aspects of the task. The proposed approach employs separate models for each modality to automatically extract discriminative semantic words, image regions, and audio features. A deep hierarchical model is then developed, incorporating intermediate fusion to learn hierarchical correlations between the modalities at bimodal and trimodal levels. Finally, the framework combines four distinct models through decision-level fusion to enable multimodal sentiment analysis and emotion recognition. The effectiveness of the proposed framework is demonstrated through extensive experiments conducted on the publicly available Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset. The results confirm a notable performance improvement compared to state-of-the-art methods, attaining 85% and 93% accuracy for sentiment analysis and emotion classification, respectively. Additionally, when considering class-wise accuracy, the results indicate that the "angry" emotion and "positive" sentiment are classified more effectively than the other emotions and sentiments, achieving 96.80% and 93.14% accuracy, respectively.
A resource scheduling method for cloud data centers based on thermal management

Abstract With the rapid growth of cloud computing services, the high energy consumption of cloud data centers has become a critical concern of the cloud computing society. While virtual machine (VM) consolidation is often used to reduce energy consumption, excessive VM consolidation may lead to local hot spots and increase the risk of equipment failure. One possible solution to this problem is to utilize thermal-aware scheduling, but existing approaches have trouble realizing the balance between SLA and energy consumption. This paper proposes a novel method to manage cloud data center resources based on thermal management (TM-VMC), which optimizes total energy consumption and proactively prevents hot spots from a global perspective. Its VM consolidation process includes four phases where the VMs scheduler uses an improved ant colony algorithm (UACO) to find appropriate target hosts for VMs based on server temperature and utilization status obtained in real-time. Experimental results show that the TM-VMC approach can proactively avoid data center hot spots and significantly reduce energy consumption while maintaining low Service Level Agreement (SLA) violation rates compared to existing mainstream VM consolidation algorithms with workloads from real-world data centers.
Gender prediction with descriptive textual data using a Machine Learning approach

Social media are well-established means of online communication, generating vast amounts of data. In this paper, we focus on Twitter and investigate behavioural differences between male and female users on social media. Using Natural Language Processing and Machine Learning approaches, we propose a user gender identification method that considers both the tweets and the Twitter profile description of a user. For experimentation and evaluation, we enriched and used an existing Twitter User Gender Classification dataset, which is freely available on Kaggle. We considered a variety of methods and components, such as the Bag of Words model, pre-trained word embeddings (GLOVE, BERT, GPT2 and Word2Vec) and machine learners, e.g., Naïve Bayes, Support Vector Machines and Random Forests. Evaluation results have shown that including the Twitter profile description of a user significantly improves gender classification accuracy, by 10% approximately. Stanford’s GLOVE embedding model, pre-trained on 2 billion tweets, 27 billion tokens and a vocabulary size of 1.2 million words, achieved the highest gender prediction accuracy, considering both the tweets and the profile description of a user. Statistical significance has been assessed using McNemar’s two-tailed test.
Multi-feature fused collaborative attention network for sequential recommendation with semantic-enriched contrastive learning

Recently, graph neural network (GNN) has been widely used in sequential recommendation because of its powerful ability to capture high-order collaborative relations, greatly promoting recommendation performance. However, some existing GNN-based methods fail to make full use of multiple relevant features of nodes and ignore the impact of semantic association between nodes on extracting user preferences. To this end, we propose a multi-feature fused collaborative attention network MASR, which sufficiently learns the temporal and positional features of nodes, and innovatively measures the importance of these two features for analyzing the nodes' dynamic patterns. In addition, we incorporate semantic-enriched contrastive learning into collaborative filtering to enhance the semantic association between nodes and reduce the noise from the structural neighborhood, which has a positive effect on the sequential recommendation. Compared with the baseline models, the performance of MASR on MovieLens, CDs and Beauty datasets is improved by 2.0%, 2.1% and 1.7% respectively, proving its effectiveness in the sequential recommendation.