Reviewerly/exHarmony huggingface dataset:
Visual Prompt Tuning

The current modus operandi in adapting pre-trained models involves updating all the backbone parameters, i.e., full fine-tuning. This paper introduces Visual Prompt Tuning (VPT) as an efficient and effective alternative to full fine-tuning for large-scale Transformer models in vision. Taking inspiration from recent advances in efficiently tuning large language models, VPT introduces only a small amount (less than 1% of model parameters) of trainable parameters in the input space while keeping the model backbone frozen. Via extensive experiments on a wide variety of downstream recognition tasks, we show that VPT achieves significant performance gains compared to other parameter efficient tuning protocols. Most importantly, VPT even outperforms full fine-tuning in many cases across model capacities and training data scales, while reducing per-task storage cost. Code is available at github.com/kmnp/vpt .
Stochastic Gradient Push for Distributed Deep Learning

Distributed data-parallel algorithms aim to accelerate the training of deep neural networks by parallelizing the computation of large mini-batch gradient updates across multiple nodes. Approaches that synchronize nodes using exact distributed averaging (e.g., via AllReduce) are sensitive to stragglers and communication delays. The PushSum gossip algorithm is robust to these issues, but only performs approximate distributed averaging. This paper studies Stochastic Gradient Push (SGP), which combines PushSum with stochastic gradient updates. We prove that SGP converges to a stationary point of smooth, non-convex objectives at the same sub-linear rate as SGD, and that all nodes achieve consensus. We empirically validate the performance of SGP on image classification (ResNet-50, ImageNet) and machine translation (Transformer, WMT'16 En-De) workloads. Our code will be made publicly available.
Enhancing Attention-Based LSTM With Position Context for Aspect-Level Sentiment Classification

Aspect-level sentiment classification is an interesting but challenging research problem, namely, the prediction of the sentiment polarity toward a specific aspect term of an opinionated sentence. Previous attention-based recurrent neural networks have been proposed to address this problem because attention mechanism is capable of finding out those words contributing more to the prediction than others and have shown great promise. However, the major drawback of these attention-based approaches is that the explicit position context is ignored. Drawing inspirations from the manner modeling the position context in information retrieval and question answering, we hypothesize that we should pay much more attention to the context words neighboring to the aspect than those far away, especially when one review sentence is a long sequence or contains multiple aspect terms. Based on this conjecture, in this paper, we put forward a new attentive LSTM model, dubbed PosATT-LSTM, which not only takes into account the importance of each context word but also incorporates the position-aware vectors, which represents the explicit position context between the aspect and its context words. We conduct substantial experiments on the SemEval 2014 datasets, and the encouraging results indicate the efficacy of our proposed approach.
Emotion Recognition From Body Movement

Automatic emotion recognition from the analysis of body movement has tremendous potential to revolutionize virtual reality, robotics, behavior modeling, and biometric identity recognition domains. A computer system capable of recognizing human emotion from the body can also significantly change the way we interact with the computers. One of the significant challenges is to identify emotion-specific features from a vast number of descriptors of human body movements. In this paper, we introduce a novel two-layer feature selection framework for emotion classification from a comprehensive list of body movement features. We used the feature selection framework to accurately recognize five basic emotions: happiness, sadness, fear, anger, and neutral. In the first layer, a unique combination of Analysis of Variance (ANOVA) and Multivariate Analysis of Variance (MANOVA) was utilized to eliminate irrelevant features. In the second layer, a binary chromosome-based genetic algorithm was proposed to select a feature subset from the relevant list of features that maximizes the emotion recognition rate. Score and rank-level fusion were applied to further improve the accuracy of the system. The proposed system was validated on proprietary and public datasets, containing 30 subjects. Different action scenarios, such as walking and sitting actions, as well as an action-independent case, were considered. Based on the experimental results, the proposed emotion recognition system achieved a very high emotion recognition rate outperforming all of the state-of-the-art methods. The proposed system achieved recognition accuracy of 90.0% during walking, 96.0% during sitting, and 86.66% in an action-independent scenario, demonstrating high accuracy and robustness of the developed method.
3D touchless multiorder reflection structural color sensing display

A 3D touchless display is created where a human finger position is visualized in structural colors of a soft photonic crystal.
Analysis Survey on Deepfake detection and Recognition with Convolutional Neural Networks

Deep Learning (DL) is the most efficient technique to handle a wide range of challenging problems such as data analytics, diagnosing diseases, detecting anomalies, etc. The development of DL has raised some privacy, justice, and national security issues. Deepfake is a DL-based application that has been very popular in recent years and is one of the reasons for these problems. Deepfake technology can create fake images and videos that are difficult for humans to recognize as real or not. Therefore, it needs to be proposed some automated methods for devices to detect and evaluate threats. In another word, digital and visual media must maintain their integrity. A set of rules used for Deepfake and some methods to detect the content created by Deepfake have been proposed in the literature. This paper summarizes what we have in the critical discussion about the problems, opportunities, and prospects of Deepfake technology. We aim for this work to be an alternative guide to getting knowledge of Deepfake detection methods. First, we cover Deepfake history and Deepfake techniques. Then, we present how a better and more robust Deepfake detection method can be designed to deal with fake content.
Co-evolutionary competitive swarm optimizer with three-phase for large-scale complex optimization problem

Practical optimization problems often involve a large number of variables, and solving them in a reasonable amount of time becomes a challenge. Competitive swarm optimizer (CSO) is an efficient variant of particle swarm optimization (PSO) algorithm and has been applied extensively to deal with a variety of practical large-scale optimization problems. In this article, a novel co-evolutionary method with three-phase, namely TPCSO, is developed by incorporating a novel multi-phase cooperative evolutionary technique to enhance the convergence and the search ability of CSO. In the modified CSO, the population is evenly decomposed into two sub-populations, then the update strategy of each sub-population is adjusted by the requirements of the diversity and convergence during the evolution process. In the first phase, the diversity is paid more attention in order to explore more regions. And in the second phase, the promising area in two sub-populations are exploited by introducing excellent particles of two sub-populations. The third phase focuses on the convergence by learning from the global best solution. Finally, the performance of TPCSO is evaluated and proved by large-scale benchmark functions selected from CEC’2010 and CEC’2013. The experimental and statistical results show that TPCSO can effectively solve these large-scale problems and fast obtain the optimal results with higher accuracy by comparing with several algorithms.
Improved YOLOv3 model with feature map cropping for multi-scale road object detection

Abstract Road object detection is an essential and imperative step for driving intelligent vehicles. Generally, road objects, such as vehicles and pedestrians, present the characteristic of multi-scale and uncertain distribution which puts a high demand on the detection algorithm. Therefore, this paper proposes a YOLOv3 (You Only Look Once v3)-based method aimed at enhancing the capability of cross-scale detection and focusing on the valuable area. The proposed method fills an urgent need for multi-scale detection, and its individual components will be useful in road object detection. The K-means-GIoU algorithm is designed to generate a priori boxes whose shapes are close to real boxes. This greatly reduces the complexity of training, paving the way for fast convergence. Then, a detection branch is added to detect small targets, and a feature map cropping module is introduced into the newly added detection branch to remove the areas with high probability of background targets and easy-to-detect targets, and the cropped areas of the feature map are filled with a value of 0. Further, a channel attention module and spatial attention module are added to strengthen the network’s attention to major regions. The experiment results on the KITTI dataset show that the proposed method maintains a fast detection speed and increases the mAP (mean average precision) value by as much as 2.86 <?CDATA $\%$?> <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mi mathvariant="normal">%</mml:mi> </mml:math> compared with YOLOv3-ultralytics, and especially improves the detection performance for small-scale objects.
MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification

We introduce MedMNIST v2, a large-scale MNIST-like dataset collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D. All images are pre-processed into a small size of 28 × 28 (2D) or 28 × 28 × 28 (3D) with the corresponding classification labels so that no background knowledge is required for users. Covering primary data modalities in biomedical images, MedMNIST v2 is designed to perform classification on lightweight 2D and 3D images with various dataset scales (from 100 to 100,000) and diverse tasks (binary/multi-class, ordinal regression, and multi-label). The resulting dataset, consisting of 708,069 2D images and 9,998 3D images in total, could support numerous research/educational purposes in biomedical image analysis, computer vision, and machine learning. We benchmark several baseline methods on MedMNIST v2, including 2D/3D neural networks and open-source/commercial AutoML tools. The data and code are publicly available at https://medmnist.com/ .
Title-Guided Encoding for Keyphrase Generation

Keyphrase generation (KG) aims to generate a set of keyphrases given a document, which is a fundamental task in natural language processing (NLP). Most previous methods solve this problem in an extractive manner, while recently, several attempts are made under the generative setting using deep neural networks. However, the state-of-the-art generative methods simply treat the document title and the document main body equally, ignoring the leading role of the title to the overall document. To solve this problem, we introduce a new model called Title-Guided Network (TG-Net) for automatic keyphrase generation task based on the encoderdecoder architecture with two new features: (i) the title is additionally employed as a query-like input, and (ii) a titleguided encoder gathers the relevant information from the title to each word in the document. Experiments on a range of KG datasets demonstrate that our model outperforms the state-of-the-art models with a large margin, especially for documents with either very low or very high title length ratios.
How to ensure the confidentiality of electronic medical records on the cloud: A technical perspective

From a technical perspective, for electronic medical records (EMR), this paper proposes an effective confidential management solution on the cloud, whose basic idea is to deploy a trusted local server between the untrusted cloud and each trusted client of a medical information management system, responsible for running an EMR cloud hierarchical storage model and an EMR cloud segmentation query model. (1) The EMR cloud hierarchical storage model is responsible for storing light EMR data items (such as patient basic information) on the local server, while encrypting heavy EMR data items (such as patient medical images) and storing them on the cloud, to ensure the confidentiality of electronic medical records on the cloud. (2) The EMR cloud segmentation query model performs EMR related query operations through the collaborative interaction between the local server and the cloud server, to ensure the accuracy and efficiency of each EMR query statement. Finally, both theoretical analysis and experimental evaluation demonstrate the effectiveness of the proposed solution for confidentiality management of electronic medical records on the cloud, i.e., which can ensure the confidentiality of electronic medical records on the untrusted cloud, without compromising the availability of an existing medical information management system.
Understanding the social sustainability of the Metaverse by integrating UTAUT2 and big five personality traits: A hybrid SEM-ANN approach

The Metaverse is an immersive virtual universe where users interact with each other using an avatar. The Metaverse is promised to offer numerous opportunities for many sectors. While the Metaverse promotes the social interaction between users, there is a scarcity of knowledge on what affects its social sustainability. Hence, this research develops a model by integrating the UTAUT2 constructs and big five personality traits to understand the social sustainability of the Metaverse. The model is tested by employing a hybrid covariance-based structural equation modeling (CB-SEM) and artificial neural network (ANN) approach based on collecting data from 446 Metaverse users. The CB-SEM results showed that performance expectancy, social influence, hedonic motivation, price value, habit, agreeableness, neuroticism, and openness significantly impact the social sustainability of the Metaverse, while no significant effect is reported regarding effort expectancy, facilitating conditions, conscientiousness, and extraversion. All these factors explained 80% of the variance in social sustainability. The ANN results showed that habit is the most important factor in predicting social sustainability. Drawing on these findings, the study offers several theoretical contributions and sheds light on several practical implications for developers, designers, and decision-makers promoting the use of the Metaverse.
Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions

Sentiment analysis (SA) has gained much traction In the field of artificial intelligence (AI) and natural language processing (NLP). There is growing demand to automate analysis of user sentiment towards products or services. Opinions are increasingly being shared online in the form of videos rather than text alone. This has led to SA using multiple modalities, termed Multimodal Sentiment Analysis (MSA), becoming an important research area. MSA utilises latest advancements in machine learning and deep learning at various stages including for multimodal feature extraction and fusion and sentiment polarity detection, with aims to minimize error rate and improve performance. This survey paper examines primary taxonomy and newly released multimodal fusion architectures. Recent developments in MSA architectures are divided into ten categories, namely early fusion, late fusion, hybrid fusion, model-level fusion, tensor fusion, hierarchical fusion, bi-modal fusion, attention-based fusion, quantum-based fusion and word-level fusion. A comparison of several architectural evolutions in terms of MSA fusion categories and their relative strengths and limitations are presented. Finally, a number of interdisciplinary applications and future research directions are proposed.
A comprehensive review on ensemble deep learning: Opportunities and challenges

In machine learning, two approaches outperform traditional algorithms: ensemble learning and deep learning. The former refers to methods that integrate multiple base models in the same framework to obtain a stronger model that outperforms them. The success of an ensemble method depends on several factors, including how the baseline models are trained and how they are combined. In the literature, there are common approaches to building an ensemble model successfully applied in several domains. On the other hand, deep learning-based models have improved the predictive accuracy of machine learning across a wide range of domains. Despite the diversity of deep learning architectures and their ability to deal with complex problems and the ability to extract features automatically, the main challenge in deep learning is that it requires a lot of expertise and experience to tune the optimal hyper-parameters, which makes it a tedious and time-consuming task. Numerous recent research efforts have been made to approach ensemble learning to deep learning to overcome this challenge. Most of these efforts focus on simple ensemble methods that have some limitations. Hence, this review paper provides comprehensive reviews of the various strategies for ensemble learning, especially in the case of deep learning. Also, it explains in detail the various features or factors that influence the success of ensemble methods. In addition, it presents and accurately categorized several research efforts that used ensemble learning in a wide range of domains.
New Frontiers: An Update on Computer-Aided Diagnosis for Breast Imaging in the Age of Artificial Intelligence

New Frontiers: An Update on Computer-Aided Diagnosis for Breast Imaging in the Age of Artificial IntelligenceYiming Gao1, Krzysztof J. Geras2, Alana A. Lewin1 and Linda Moy1,3Audio Available | Share
Chinese Image Captioning via Fuzzy Attention-based DenseNet-BiLSTM

Chinese image description generation tasks usually have some challenges, such as single-feature extraction, lack of global information, and lack of detailed description of the image content. To address these limitations, we propose a fuzzy attention-based DenseNet-BiLSTM Chinese image captioning method in this article. In the proposed method, we first improve the densely connected network to extract features of the image at different scales and to enhance the model’s ability to capture the weak features. At the same time, a bidirectional LSTM is used as the decoder to enhance the use of context information. The introduction of an improved fuzzy attention mechanism effectively improves the problem of correspondence between image features and contextual information. We conduct experiments on the AI Challenger dataset to evaluate the performance of the model. The results show that compared with other models, our proposed model achieves higher scores in objective quantitative evaluation indicators, including BLEU <?TeX $@1$?> , BLEU <?TeX $@4$?> , METEOR, ROUGEl, and CIDEr. The generated description sentence can accurately express the image content.
Traceability vs. sustainability in supply chains: The implications of blockchain

This research proposes a game theory model in a supply chain (SC) involving one manufacturer and one retailer. The SC works in a global market in which consumers are located worldwide and subject to traceability issues that can create distrust of the product quality. This issue can be resolved by implementing blockchain technology which provides benefits in terms of high traceability along with low transaction costs. However, blockchain negatively impacts the environment because of their high energy consumption. Therefore, in this study, we capture the trade-offs between traceability and sustainability for blockchain adoption by characterizing a game theory model. Our findings show that high levels of distrust pushes firms to avoid the implementation of blockchain. In such circumstances, blockchain is not sufficient to make consumers recognize the product quality and trust the firms’ practices. In contrast, low levels of distrust can make blockchain an economically suitable technology conditioned to minimal environmental damages; otherwise, firms need to carefully evaluate the trade-offs between distrust and sustainability. Since the adoption of blockchain leads to an increase in prices and decrease of distrust, two factors determine whether to pursue this technology or not: low consumer sensitivity to price and high sensitivity to quality. In this study, we develop three specific cases where we model: 1) the direct impact of blockchain on distrust, 2) a stochastic distrust term, and 3) a Stackelberg game. Each case confirms our results and strengthens the robustness of our findings.
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Deep neural networks have been well-known for their superb handling of various machine learning and artificial intelligence tasks. However, due to their over-parameterized black-box nature, it is often difficult to understand the prediction results of deep models. In recent years, many interpretation tools have been proposed to explain or reveal how deep models make decisions. In this paper, we review this line of research and try to make a comprehensive survey. Specifically, we first introduce and clarify two basic concepts—interpretations and interpretability—that people usually get confused about. To address the research efforts in interpretations, we elaborate the designs of a number of interpretation algorithms, from different perspectives, by proposing a new taxonomy. Then, to understand the interpretation results, we also survey the performance metrics for evaluating interpretation algorithms. Further, we summarize the current works in evaluating models’ interpretability using “trustworthy” interpretation algorithms. Finally, we review and discuss the connections between deep models’ interpretations and other factors, such as adversarial robustness and learning from interpretations, and we introduce several open-source libraries for interpretation algorithms and evaluation approaches.