Reviewerly/exHarmony huggingface dataset:
A Survey on Long-Tailed Visual Recognition

The heavy reliance on data is one of the major reasons that currently limit the development of deep learning. Data quality directly dominates the effect of deep learning models, and the long-tailed distribution is one of the factors affecting data quality. The long-tailed phenomenon is prevalent due to the prevalence of power law in nature. In this case, the performance of deep learning models is often dominated by the head classes while the learning of the tail classes is severely underdeveloped. In order to learn adequately for all classes, many researchers have studied and preliminarily addressed the long-tailed problem. In this survey, we focus on the problems caused by long-tailed data distribution, sort out the representative long-tailed visual recognition datasets and summarize some mainstream long-tailed studies. Specifically, we summarize these studies into ten categories from the perspective of representation learning, and outline the highlights and limitations of each category. Besides, we have studied four quantitative metrics for evaluating the imbalance, and suggest using the Gini coefficient to evaluate the long-tailedness of a dataset. Based on the Gini coefficient, we quantitatively study 20 widely-used and large-scale visual datasets proposed in the last decade, and find that the long-tailed phenomenon is widespread and has not been fully studied. Finally, we provide several future directions for the development of long-tailed learning to provide more ideas for readers.
A Complete Process of Text Classification System Using State-of-the-Art NLP Models

With the rapid advancement of information technology, online information has been exponentially growing day by day, especially in the form of text documents such as news events, company reports, reviews on products, stocks-related reports, medical reports, tweets, and so on. Due to this, online monitoring and text mining has become a prominent task. During the past decade, significant efforts have been made on mining text documents using machine and deep learning models such as supervised, semisupervised, and unsupervised. Our area of the discussion covers state-of-the-art learning models for text mining or solving various challenging NLP (natural language processing) problems using the classification of texts. This paper summarizes several machine learning and deep learning algorithms used in text classification with their advantages and shortcomings. This paper would also help the readers understand various subtasks, along with old and recent literature, required during the process of text classification. We believe that readers would be able to find scope for further improvements in the area of text classification or to propose new techniques of text classification applicable in any domain of their interest.
Deepsign: Sign Language Detection and Recognition Using Deep Learning

The predominant means of communication is speech; however, there are persons whose speaking or hearing abilities are impaired. Communication presents a significant barrier for persons with such disabilities. The use of deep learning methods can help to reduce communication barriers. This paper proposes a deep learning-based model that detects and recognizes the words from a person’s gestures. Deep learning models, namely, LSTM and GRU (feedback-based learning models), are used to recognize signs from isolated Indian Sign Language (ISL) video frames. The four different sequential combinations of LSTM and GRU (as there are two layers of LSTM and two layers of GRU) were used with our own dataset, IISL2020. The proposed model, consisting of a single layer of LSTM followed by GRU, achieves around 97% accuracy over 11 different signs. This method may help persons who are unaware of sign language to communicate with persons whose speech or hearing is impaired.
Comprehending and Ordering Semantics for Image Captioning

Comprehending the rich semantics in an image and ordering them in linguistic order are essential to compose a visually-grounded and linguistically coherent description for image captioning. Modern techniques commonly capitalize on a pre-trained object detector/classifier to mine the semantics in an image, while leaving the inherent linguistic ordering of semantics under-exploited. In this paper, we propose a new recipe of Transformer-style structure, namely Comprehending and Ordering Semantics Networks (COS-Net), that novelly unifies an enriched semantic comprehending and a learnable semantic ordering processes into a single architecture. Technically, we initially utilize a cross-modal retrieval model to search the relevant sentences of each image, and all words in the searched sentences are taken as primary semantic cues. Next, a novel semantic comprehender is devised to filter out the irrelevant semantic words in primary semantic cues, and mean-while infer the missing relevant semantic words visually grounded in the image. After that, we feed all the screened and enriched semantic words into a semantic ranker, which learns to allocate all semantic words in linguistic order as humans. Such sequence of ordered semantic words are further integrated with visual tokens of images to trigger sentence generation. Empirical evidences show that COS-Net clearly surpasses the state-of-the-art approaches on COCO and achieves to-date the best CIDEr score of 141.1% on Karpathy test split. Source code is available at https://github.com/YehLi/xmodaler/tree/master/configs/image_caption/cosnet.
Research on Art Teaching Practice Supported by Virtual Reality (VR) Technology in the Primary Schools

Nowadays, teaching and learning methods are constantly changing with the development and popularization of information technology. Many teaching activities are exploring the integration of virtual technology. However, the specific effects of VR are challenging to verify. In this paper, “teaching in VR environment” and “traditional teaching” were designed to carry out a series of teaching comparison practices between two groups of a primary school. By analyzing the experimental data of the experimental group and the control group, the research found that it is easier to enter mental flow in virtual reality, and the introduction of virtual reality technology is positively correlated with learning engagement. What is more, compared with traditional teaching and learning methods, virtual reality technology and related software can help individuals give full play to their creativity.
LOWESS smoothing and Random Forest based GRU model: A short-term photovoltaic power generation forecasting method

Accurate prediction of photovoltaic power generation is vital to guarantee smooth operation of power stations and ensure users’ electricity consumption. As a good forecasting tool, Gated Recurrent Unit method has been widely used in different forecasting areas. However, the existing studies ignore the impact of data fluctuations on prediction accuracy, to fill the gaps and enhance prediction accuracy, several different data smoothing techniques are introduced and compared to reduce fluctuations, Random Forest method is used for feature selection, and RepeatVector layer extended by attribute dimensions and TimeDistributed layer with full connectivity are utilized to optimize the Gated Recurrent Unit model. A real-world case from the photovoltaic power plant in Xuhui District, Shanghai, China, is adopted to evaluate the performance of proposed method. The comparing results with Recurrent Neural Networks and Long Short-Term Memory, and the actual data as well, show that the proposed prediction method can effectively improve the prediction accuracy of photovoltaic power generation. We also use the daily and monthly data of The Desert Knowledge Australia Solar Centre in Australia to investigate whether the proposed method is suitable for short-term or medium and long-term prediction. The results indicate that our method is more appropriate for short-term prediction. • Introducing and comparing different data smoothing technologies to reduce the data fluctuation. • Using Random Forest to extract the characteristics of natural factors. • Adding RepeatVector layer and TimeDistributed layer into the GRU to improve prediction accuracy. • Utilizing the dataset from Shanghai, China and three prediction models to verify the accuracy. • Utilizing dataset from Alice Springs to verify that our method is more suitable for short-term forecasting.
Reputation-Aware Supplier Assessment for Blockchain-Enabled Supply Chain in Industry 4.0

In Industry 4.0, the development of intelligent supply chains is the key to improving enterprise efficiency and customer satisfaction. At present, the strategy with blockchain to optimize the supply chain ensures the security of data sharing and interaction between different suppliers. However, the quality of service (QoS) loss in time efficiency and supplier cooperation is still a severe problem in these strategies. Therefore, it is essential to conduct reputable supplier selection to reduce QoS loss under the premise of blockchain security. Meanwhile, choosing a reputable supplier as the primary peer in the blockchain consensus process can also effectively reduce transaction latency. To solve this problem, a reputation-aware supplier assessment system, named SAS, is proposed. Specifically, Canopy and K-medoids are utilized to classify suppliers preliminarily. Then backpropagation neural network is deployed to evaluate supplier reputation. The results show that SAS is effective in supplier reputation evaluation through extensive experiments.
Channel-Equalization-HAR: A Light-weight Convolutional Neural Network for Wearable Sensor Based Human Activity Recognition

Recently, human activity recognition (HAR) that uses wearable sensors has become a research hotspot because its wide applications in real-world scenarios. Essentially, HAR can be treated as multi-channel time series classification problem, where different channels may come from heterogeneous sensor modalities. Deep learning, especially convolutional neural networks (CNNs) have made breakthroughs in ubiquitous HAR scenario. Various normalization methods enable layers of networks to learn more independently by normalizing hybrid sensor features. However, normalization tends to produce a channel collapse phenomenon, where many channels generates tiny values. Most channels are inhibited and contribute very little to output. As a result, the network has to rely on only a few valid channels, which inevitably impair the generality ability. In this paper, we provide an alternative called Channel Equalization to reactivate these inhibited channels by performing whitening or decorrelation operation, which compels all channels to contribute more or less to feature representation. Extensive experiments are conducted on several public HAR benchmarks, which indicate that the proposed method significantly surpasses recent SOTA at negligible computational overhead. To our knowledge, the Channel Equalization is for the first time to be applied in multimodal HAR scenario. Finally, the actual operation is evaluated on an embedded platform.
LeVit-UNet: Make Faster Encoders with Transformer for Biomedical Image Segmentation

Biomedical image segmentation plays an essential role in developing computer-assisted diagnosis and treatment systems, yet it still faces numerous challenges. In the past few years, Convolutional Neural Networks (CNNs) have been successfully applied to the task of biomedical image segmentation. Regrettably, due to the locality of convolution operations, these CNN-based architectures have their limitations in learning long-range spatial relations and global context in images, which might be crucial to the success of biomedical image segmentation. Meanwhile, the vision Transformer (ViT) architectures own the remarkable ability to extract long-range semantic features with the shortcoming of their computation complexity. To make biomedical image segmentation more efficient and accurate, we present LeViT-UNet, which integrates a LeViT Transformer module into the UNet architecture, with the goal of exploring the effectiveness of fusion between local and global features together. With the help of Transformer blocks, LeViT-UNet aims to achieve a better trade-off between efficiency and accuracy by effectively reusing the spatial information of feature maps. Our experiments on several challenging segmentation benchmarks indicate that the proposed LeViT-UNet achieved competitive performance compared with various state-of-the-art methods in terms of efficiency and accuracy, suggesting that LeViT can be a faster feature encoder for biomedical images segmentation. LeViT-UNet-384, for instance, achieves Dice similarity coefficient (DSC) of 78.53% and 90.32% with a segmentation speed of 85 frames per second (FPS) in the Synapse and ACDC datasets, respectively. Therefore, the proposed architecture could be beneficial for prospective clinic trials conducted by the radiologists. Our source codes are publicly available at https://github.com/apple1986/LeViT_UNet .
Distributed Motion Planning for Safe Autonomous Vehicle Overtaking via Artificial Potential Field

Autonomous driving of multi-lane vehicle platoons have attracted significant attention in recent years due to their potential to enhance the traffic-carrying capacity of the roads and produce better safety for drivers and passengers. This paper proposes a distributed motion planning algorithm to ensure safe overtaking of autonomous vehicles in a dynamic environment using the Artificial Potential Field method. Unlike the conventional overtaking techniques, autonomous driving strategies can be used to implement safe overtaking via formation control of unmanned vehicles in a complex vehicle platoon in the presence of human-operated vehicles. Firstly, we formulate the overtaking problem of a group of autonomous vehicles into a multi-target tracking problem, where the targets are dynamic. To model a multi-vehicle system consisting of both autonomous and human-operated vehicles, we introduce the notion of velocity difference potential field and acceleration difference potential field. We then analyze the stability of the multi-lane vehicle platoon and propose an optimization-based algorithm for solving the overtaking problem by placing a dynamic target in the traditional artificial potential field. A simulation case study has been performed to verify the feasibility and effectiveness of the proposed distributed motion control strategy for safe overtaking in a multi-lane vehicle platoon.
A narrative review of immersive virtual reality’s ergonomics and risks at the workplace: cybersickness, visual fatigue, muscular fatigue, acute stress, and mental overload

Abstract This narrative review synthesizes and introduces 386 previous works about virtual reality-induced symptoms and effects by focusing on cybersickness, visual fatigue, muscle fatigue, acute stress, and mental overload. Usually, these VRISE are treated independently in the literature, although virtual reality is increasingly considered an option to replace PCs at the workplace, which encourages us to consider them all at once. We emphasize the context of office-like tasks in VR, gathering 57 articles meeting our inclusion/exclusion criteria. Cybersickness symptoms, influenced by fifty factors, could prevent workers from using VR. It is studied but requires more research to reach a theoretical consensus. VR can lead to more visual fatigue than other screen uses, influenced by fifteen factors, mainly due to vergence-accommodation conflicts. This side effect requires more testing and clarification on how it differs from cybersickness. VR can provoke muscle fatigue and musculoskeletal discomfort, influenced by fifteen factors, depending on tasks and interactions. VR could lead to acute stress due to technostress, task difficulty, time pressure, and public speaking. VR also potentially leads to mental overload, mainly due to task load, time pressure, and intrinsically due interaction and interface of the virtual environment. We propose a research agenda to tackle VR ergonomics and risks issues at the workplace.
Binary Approaches of Quantum-Based Avian Navigation Optimizer to Select Effective Features from High-Dimensional Medical Data

Many metaheuristic approaches have been developed to select effective features from different medical datasets in a feasible time. However, most of them cannot scale well to large medical datasets, where they fail to maximize the classification accuracy and simultaneously minimize the number of selected features. Therefore, this paper is devoted to developing an efficient binary version of the quantum-based avian navigation optimizer algorithm (QANA) named BQANA, utilizing the scalability of the QANA to effectively select the optimal feature subset from high-dimensional medical datasets using two different approaches. In the first approach, several binary versions of the QANA are developed using S-shaped, V-shaped, U-shaped, Z-shaped, and quadratic transfer functions to map the continuous solutions of the canonical QANA to binary ones. In the second approach, the QANA is mapped to binary space by converting each variable to 0 or 1 using a threshold. To evaluate the proposed algorithm, first, all binary versions of the QANA are assessed on different medical datasets with varied feature sizes, including Pima, HeartEW, Lymphography, SPECT Heart, PenglungEW, Parkinson, Colon, SRBCT, Leukemia, and Prostate tumor. The results show that the BQANA developed by the second approach is superior to other binary versions of the QANA to find the optimal feature subset from the medical datasets. Then, the BQANA was compared with nine well-known binary metaheuristic algorithms, and the results were statistically assessed using the Friedman test. The experimental and statistical results demonstrate that the proposed BQANA has merit for feature selection from medical datasets.
ROLAND: Graph Learning Framework for Dynamic Graphs

Graph Neural Networks (GNNs) have been successfully applied to many real-world static graphs. However, the success of static graphs has not fully translated to dynamic graphs due to the limitations in model design, evaluation settings, and training strategies. Concretely, existing dynamic GNNs do not incorporate state-of-the-art designs from static GNNs, which limits their performance. Current evaluation settings for dynamic GNNs do not fully reflect the evolving nature of dynamic graphs. Finally, commonly used training methods for dynamic GNNs are not scalable. Here we propose ROLAND, an effective graph representation learning framework for real-world dynamic graphs. At its core, the ROLAND framework can help researchers easily repurpose any static GNN to dynamic graphs. Our insight is to view the node embeddings at different GNN layers as hierarchical node states and then recurrently update them over time. We then introduce a live-update evaluation setting for dynamic graphs that mimics real-world use cases, where GNNs are making predictions and being updated on a rolling basis. Finally, we propose a scalable and efficient training approach for dynamic GNNs via incremental training and meta-learning. We conduct experiments over eight different dynamic graph datasets on future link prediction tasks. Models built using the ROLAND framework achieve on average 62.7% relative mean reciprocal rank (MRR) improvement over state-of-the-art baselines under the standard evaluation settings on three datasets. We find state-of-the-art baselines experience out-of-memory errors for larger datasets, while ROLAND can easily scale to dynamic graphs with 56 million edges. After re-implementing these baselines using the ROLAND training strategy, ROLAND models still achieve on average 15.5% relative MRR improvement over the baselines.
MST-GAT: A multimodal spatial–temporal graph attention network for time series anomaly detection

Multimodal time series (MTS) anomaly detection is crucial for maintaining the safety and stability of working devices (e.g., water treatment system and spacecraft), whose data are characterized by multivariate time series with diverse modalities. Although recent deep learning methods show great potential in anomaly detection, they do not explicitly capture spatial–temporal relationships between univariate time series of different modalities, resulting in more false negatives and false positives. In this paper, we propose a multimodal spatial–temporal graph attention network (MST-GAT) to tackle this problem. MST-GAT first employs a multimodal graph attention network (M-GAT) and a temporal convolution network to capture the spatial–temporal correlation in multimodal time series. Specifically, M-GAT uses a multi-head attention module and two relational attention modules (i.e., intra- and inter-modal attention) to model modal correlations explicitly. Furthermore, MST-GAT optimizes the reconstruction and prediction modules simultaneously. Experimental results on four multimodal benchmarks demonstrate that MST-GAT outperforms the state-of-the-art baselines. Further analysis indicates that MST-GAT strengthens the interpretability of detected anomalies by locating the most anomalous univariate time series.
MetaChain: A Novel Blockchain-based Framework for Metaverse Applications

Metaverse has recently attracted paramount attention due to its potential for future Internet. However, to fully realize such potential, Metaverse applications have to overcome various challenges such as massive resource demands, interoperability among applications, and security and privacy concerns. In this paper, we propose MetaChain, a novel blockchain-based framework to address emerging challenges for the development of Metaverse applications. In particular, by utilizing the smart contract mechanism, MetaChain can effectively manage and automate complex interactions among the Metaverse Service Provider (MSP) and the Metaverse users (MUs). In addition, to allow the MSP to efficiently allocate its resources for Metaverse applications and MUs’ demands, we design a novel sharding scheme to improve the underlying blockchain’s scalability. Moreover, to leverage MUs’ resources as well as to attract more MUs to support Metaverse operations, we develop an incentive mechanism using the Stackelberg game theory that rewards MUs’ contributions to the Metaverse. Through numerical experiments, we clearly show the impacts of the MUs’ behaviors and how the incentive mechanism can attract more MUs and resources to the Metaverse.