Reviewerly/exHarmony huggingface dataset:
Multi-Stream Concept Drift Self-Adaptation Using Graph Neural Network

Concept drift is the phenomenon where the data distribution in a data stream changes over time. It is a ubiquitous problem in the real-world, for example, a traffic accident would cause a jam in a certain period, leading to a distribution change in traffic speed. Most research in the concept drift field focuses on single data stream, however, few of them consider multi-stream environments which are more in line with the application needs. To fill this gap, we propose a multi-stream prediction setting and a multi-stream concept drift self-adaptation framework using graph neural network, named SAGN. In SAGN, we reconsider the learning procedure of GNN-based predictors from an aspect of concept drift adaptation for multi-stream. By this design, the prediction task is converted into online streaming data tasks in sub-graphs. Each sub-graph corresponds to an adaptation target and will be updated over time. In this way, locally we can overcome drift in each sub-graph by a designed adaptation technique, and globally the correlation between different data streams is well-preserved as a graph structure. Therefore, whether drift occurs or not, in one or several streams, SAGN can provide consistently accurate prediction results. We comprehensively tested SAGN on both synthetic and real-world, drift and non-drift data in the multi-step prediction task. The experiment results show that SAGN is able to achieve state-of-the-art performance in most cases.
Study on Deep Learning Models for Human Pose Estimation and its Real Time Application

In computer vision, human pose estimation details the posture of the person’s body structure that can be Kinematic, Planer, and Volumetric in an image or video. However, pose detection is often critical to be driven by distinct human actions. Thus, this survey report analysis the recent progression of the bottom-up and top-down human pose evaluation models. This survey report focuses on 2D and 3D skeleton-based human pose detection from the captured Red Green Blue(RGB) images. We have condensed the performance of the recent pose recognition, tracking, and detection techniques that utilize pose estimation from colour images as captured and then exhibit room for much more refinement in this domain. In this paper, scrutinize the study of human pose estimation models like 2d and 3d HPE for identify human movements such as running, dancing, sport so on and recent computer vision-based advances. This study has included various methods for detecting in two and three dimensions. This paper summarises the deep learning models for HPE, dataset, and challenges.
Automated Diagnosis of Breast Cancer using Combined Features and Random Forest Classifier

In recent years, several computer-aided diagnosis (CAD) models have been proposed for the early detection of breast cancer to minimize the mortality rate among women. However, the CAD model needs better accuracy with fewer features to improvise the model’s efficiency. In this work, we propose an effective automated CAD model for mammogram image classification using the least number of features. The model presented here employs a two-dimensional discrete wavelet transform for feature extraction from mammogram images. The combined technique called PCA and LDA utilize feature reduction and select significant features. The combined reduced feature matrices are then fed into a random forest classifier, which is used to make the classification between normal and abnormal mammography images. Conventional datasets including INbreast, MIAS, and DDSM were used to test and evaluate the proposed CAD model with minimal features.
Credit Investigation and Comprehensive Risk Management System based Big Data Analytics in Commercial Banking

The banking industry has experienced significant transformations in terms of how effectively they function and provide services over the past few decades. The banking services infrastructure will be challenged by an expanding worldwide population. While serving a sizable segment of clients, it improves the number of consumers, online transactions, and produces enormous amounts of data. Today, banks in the US and other nations use Big Data Analytics (BDA) to handle this scenario daily. It looks for different trends in their databases in order to help their organizations make more money. Banks are changing from a straightforward approach to managing credit risk to a comprehensive risk management methodology. Banking dangers originate from numerous systems and channels. Big data technology offers an insightful and effective method for managing data, making it appropriate for use in risk management applications that call for complicated data analysis and increased data. The big data architecture of a banking credit investigation and integrated risk management system is described in this analysis. Comparisons and analyses unambiguously show that the described system performs better. Hence, this model shows that efficiency and security has improved.
Randomly Wired Graph Neural Network for Chinese NER

Recurrent Neural Networks (RNN), tracking the features of characters and words sequentially, have achieved great success in Chinese Named Entity Recognition (NER). However, such chain-like models are inadequate to capture hierarchical and nested entities due to their poor structures. Besides, existing graph-based models on NER are limited to using graphs for modeling text sequences, which aims at obtaining an expected text-to-graph structure to segment the entities’ boundaries by hand design. Thus, we propose a Randomly Wired Graph Neural Network (RWGNN) in terms of neural network architecture, which forms a novel context encoder and automatically generate multi-directional wired pattern in graph networks by using random graph algorithms. We also incorporate lexical information with reorganized and original word representations to enhance the global dependencies. Our experimental results show that RWGNN has very promising results consistently on five different domain datasets. Furthermore, ablation experiment, complexity analysis, case study, study on sentence length, and inference speed of RWGNN also be conducted to evaluate the rationality of our model.
ConvPose: A modern pure ConvNet for human pose estimation

Transformer-based networks almost thoroughly outperformed those based on convolutional neural network (ConvNet) and predominate in the field of pose estimation. To get off the hook and resuscitate ConvNets, we propose ConvPose, which is a pure ConvNet that does not utilize conventional improvement strategies like attention mechanisms and lightweight approaches, but instead pioneeringly modernizes network structures. The modernization process includes: deepening the stem cell and transition layers, using a separate pointwise convolution layer, adopting a batch normalization (BN) layer after resizing the feature maps, employing large-kernel depthwise separable convolutions and designing re-parameterized-style structures, constructing two consecutive modules that contain a mixer and an inverted bottleneck, etc. All of these designs are similar to the corresponding Transformer architectures, which means translating Transformer-specific components into convolutional variations and incorporating them into a ConvNet. A modern ConvNet not only maintains the simplicity of convolutional, but also takes advantage of Transformers. The experiments show that ConvPose-BL achieves a 76.0 Average Precision (AP) score on the COCO val2017 dataset. ConvPose performs on par or better than the existing representative networks those based on Transformer and ConvNet, and represents slight superiority in terms of speed.
A novel hand-eye semi-automatic calibration process for laser profilometers using machine learning

Hand-eye calibration is a vital step in the laser profilometer integrated measurement system. However, the widely applied standard sphere calibration process requires extensive laborious human work and expertise. To standardize and streamline the hand-eye calibration process for Industry 4.0, a semi-automatic calibration process was proposed, containing a calibration postures generation model and an intelligent circle detection method based on machine learning. With the proposed process, the measurement system can collect required data with minimal human participation and automatically processes them, which can significantly improve hand-eye calibration accuracy and efficiency. Additionally, an exhaustive study of the influence of the number of postures on the calibration accuracy of the industrial robot and automated fibre placement (AFP) machine demonstrates the proposed model’s effectiveness. Moreover, the current study’s results can also be excellent tools for further hand-eye calibration research. Our code and dataset are made publicly at: https://github.com/tangyipeng100/hand_eye_cali_circle_segmentation.
A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition

We propose a quantum kernel learning (QKL) framework to address the inherent data sparsity issues often encountered in training large-scare acoustic models in low-resource scenarios. We project acoustic features based on classical-to-quantum feature encoding. Different from existing quantum convolution techniques, we utilize QKL with features in the quantum space to design kernel-based classifiers. Experimental results on challenging spoken command recognition tasks for a few low-resource languages, such as Arabic, Georgian, Chuvash, and Lithuanian, show that the proposed QKL-based hybrid approach attains good improvements over existing classical and quantum solutions.
Freevc: Towards High-Quality Text-Free One-Shot Voice Conversion

Voice conversion (VC) can be achieved by first extracting source content information and target speaker information, and then reconstructing waveform with these information. However, current approaches normally either extract dirty content information with speaker information leaked in, or demand a large amount of annotated data for training. Besides, the quality of reconstructed waveform can be degraded by the mismatch between conversion model and vocoder. In this paper, we adopt the end-to-end framework of VITS for high-quality waveform reconstruction, and propose strategies for clean content information extraction without text annotation. We disentangle content information by imposing an information bottleneck to WavLM features, and propose the spectrogram-resize based data augmentation to improve the purity of extracted content information. Experimental results show that the proposed method outperforms the latest VC models trained with annotated data and has greater robustness.
Euro: Espnet Unsupervised ASR Open-Source Toolkit

This paper describes the ESPnet Unsupervised ASR Open-source Toolkit (EURO), an end-to-end open-source toolkit for unsupervised automatic speech recognition (UASR). EURO adopts the state-of-the-art UASR learning method introduced by the Wav2vec-U, originally implemented at FAIRSEQ, which leverages self-supervised speech representations and adversarial training. In addition to wav2vec2, EURO extends the functionality and promotes reproducibility for UASR tasks by integrating S3PRL and k2, resulting in flexible frontends from 27 self-supervised models and various graph-based decoding strategies. EURO is implemented in ESPnet and follows its unified pipeline to provide UASR recipes with a complete setup. This improves the pipeline’s efficiency and allows EURO to be easily applied to existing datasets in ESPnet. Extensive experiments on three mainstream self-supervised models demonstrate the toolkit’s effectiveness and achieve state-of-the-art UASR performance on TIMIT and LibriSpeech datasets. EURO will be publicly available at https://github.com/espnet/espnet, aiming to promote this exciting and emerging research area based on UASR through open-source activity.
DMFormer: Closing the gap Between CNN and Vision Transformers

Vision transformers have shown excellent performance in computer vision tasks. As the computation cost of their self-attention mechanism is expensive, recent works tried to replace the self-attention mechanism in vision transformers with convolutional operations, which is more efficient with built-in inductive bias. However, these efforts either ignore multi-level features or lack dynamic prosperity, leading to sub-optimal performance. In this paper, we propose a Dynamic Multi-level Attention mechanism (DMA), which captures different patterns of input images by multiple kernel sizes and enables input-adaptive weights with a gating mechanism. Based on DMA, we present an efficient backbone network named DMFormer. DMFormer adopts the overall architecture of vision transformers, while replacing the self-attention mechanism with our proposed DMA. Extensive experimental results on ImageNet-1K and ADE20K datasets demonstrated that DMFormer achieves state-of-the-art performance, which outperforms similar-sized vision transformers(ViTs) and convolutional neural networks (CNNs).
A Large-Scale Pretrained Deep Model for Phishing URL Detection

Phishing attacks have always been a security issue that has attracted great attention in the cyber security community. Recently, the famous pre-trained models is being used as an anti-phishing solution. However, existing studies either simply transfer models pre-trained on text to phishing detection task, or pre-train models using only extremely small phishing samples. In this paper, we propose PhishBERT, a veritable pretrained deep transformer network model for phishing URL detection. Using a tailor pre-training objective, PhishBERT obtained a general understanding of various URLs by being pretrained on a corpus of more than 3 billion unlabeled URL data. It is then transferred to the detection task of benign and malicious URL data, with supervised fine-tuning using adversarial methods. Extensive and rigorous benchmark studies verify that PhishBERT is significantly superior to the current state-of-the-art methods in terms of efficiency, robustness and accuracy on the task of phishing website detection.
Fast Single-Person 2D Human Pose Estimation Using Multi-Task Convolutional Neural Networks

This paper presents a novel neural module for enhancing existing fast and lightweight 2D human pose estimation CNNs, in order to increase their accuracy. A baseline stem CNN is augmented by a collateral module, which is tasked to encode global spatial and semantic information and provide it to the stem network during inference. The latter one outputs the final 2D human pose estimations. Since global information encoding is an inherent subtask of 2D human pose estimation, this particular setup allows the stem network to better focus on the local details of the input image and on precisely localizing each body joint, thus increasing overall 2D human pose estimation accuracy. Furthermore, the collateral module is designed to be lightweight, adding negligible runtime computational cost, so that the unified architecture retains the fast execution property of the stem network. Evaluation of the proposed method on public 2D human pose estimation datasets shows that it increases the accuracy of different baseline stem CNNs, while outperforming all competing fast 2D human pose estimation methods.
Multimodal Knowledge Distillation for Arbitrary-Oriented Object Detection in Aerial Images

Recently, many arbitrary-oriented object detection (AOOD) methods have been proposed and applied to remote sensing and other fields. For aerial platforms, lightweight structure and multimodal adaptations of convolutional neural network (CNN) models are urgently needed. Due to the limited model size, the performance of existing lightweight AOOD methods is low, especially in multimodal tasks. In this paper, a multimodal knowledge distillation (MKD) method is proposed for AOOD in aerial images. In MKD, a multimodal dynamic label assignment strategy is designed to select the optimal positive samples dynamically to adapt to different modalities and environments. Different multimodal localization and feature distillation modules are designed to make multimodal knowledge to be complementary and effectively learned by the lightweight model. Experiments on the public dataset demonstrated the effectiveness and advancement of MKD.
BECTRA: Transducer-Based End-To-End ASR with Bert-Enhanced Encoder

We present BERT-CTC-Transducer (BECTRA), a novel end-to-end automatic speech recognition (E2E-ASR) model formulated by the transducer with a BERT-enhanced encoder. Integrating a large-scale pre-trained language model (LM) into E2E-ASR has been actively studied, aiming to utilize versatile linguistic knowledge for generating accurate text. One crucial factor that makes this integration challenging lies in the vocabulary mismatch; the vocabulary constructed for a pre-trained LM is generally too large for E2E-ASR training and is likely to have a mismatch against a target ASR domain. To overcome such an issue, we propose BECTRA, an extended version of our previous BERT-CTC, that realizes BERT-based E2E-ASR using a vocabulary of interest. BECTRA is a transducer-based model, which adopts BERT-CTC for its encoder and trains an ASR-specific decoder using a vocabulary suitable for a target task. With the combination of the transducer and BERT-CTC, we also propose a novel inference algorithm for taking advantage of both autoregressive and non-autoregressive decoding. Experimental results on several ASR tasks, varying in amounts of data, speaking styles, and languages, demonstrate that BECTRA outperforms BERT-CTC by effectively dealing with the vocabulary mismatch while exploiting BERT knowledge.
Gated Contextual Adapters For Selective Contextual Biasing In Neural Transducers

Neural contextual biasing for end-to-end neural ASR transducers has shown significant improvements in the recognition of named entities, such as contact names or device names. However, it comes with the cost of increased compute, as the biasing layers (which are usually based on cross-attention) add complexity to the neural transducers. In this paper, we propose gated contextual biasing models that can estimate at runtime when contextual biasing is needed and can toggle it on or off. That way, contextual biasing does not run on every audio frame, but only on the frames where it can be helpful for correct ASR recognition. We show that our gated contextual biasing models can maintain all the performance improvements of contextual biasing while offering significant compute-cost saving, as the contextual biasing needs to be executed for fewer than 15% of the audio frames.
I3D: Transformer Architectures with Input-Dependent Dynamic Depth for Speech Recognition

Transformer-based end-to-end speech recognition has achieved great success. However, the large footprint and computational overhead make it difficult to deploy these models in some real-world applications. Model compression techniques can reduce the model size and speed up inference, but the compressed model has a fixed architecture which might be suboptimal. We propose a novel Transformer encoder with Input-Dependent Dynamic Depth (I3D) to achieve strong performance-efficiency trade-offs. With a similar number of layers at inference time, I3D-based models outperform the vanilla Transformer and the static pruned model via iterative layer pruning. We also present interesting analysis on the gate probabilities and the input-dependency, which helps us better understand deep encoders.
Prediction of Bitcoin Price through LSTM, ARIMA, XGBoost, Prophet and Sentiment Analysis on Dynamic Streaming Data

The most popular cryptocurrency in the world is Bitcoin, which enables users to perform secure online transactions. When carrying out quick transactions, including cash transactions, this aids in keeping your money secret. Most of Consumers have been interested in the Bitcoin ecosystem in recent years. Predicting the bitcoin price accurately is a difficult task due to its high volatility. In this paper, we used deep learning and machine learning algorithms namely Long Short-Term Memory, Autoregressive Integrated Moving Average , XGBoost, Prophet and Sentiment analysis were performed on bitcoin data.The algorithms were trained on live streaming finanacial data, and results are compared based on predicted metrics like Root mean Square Error,Mean Absolute Error and R2. The results show that Sentiment analysis combined with LSTM provide better performance in bitcoin price prediction of all other algorithms.
Machine Learning-Based Exercise Posture Recognition System Using MediaPipe Pose Estimation Framework

Daily exercise is essential for good health, but incorrect posture during exercise can lead to pain and injury, especially for the elderly. Hiring a personal trainer can be expensive, and not everyone has access to one. This paper proposes a posture classifier system that uses machine learning to recognize various exercise postures. Instead of having a personal trainer, the goal of this research is to create an automated model that can properly assess exercise posture. From video datasource recorded by a fitness specialist through a simple web camera, we extract body skeleton sequences using the MediaPipe pose estimation framework and evaluate the performance of different machine learning models in detecting each posture class in each type of exercise using precision, recall, and accuracy metrics. The system achieves an average accuracy score of 100% on our test data on three types of exercises, demonstrating its potential as an affordable and accessible solution for monitoring and correcting body postures during exercise.
PFT-SSR: Parallax Fusion Transformer for Stereo Image Super-Resolution

Stereo image super-resolution aims to boost the performance of image super-resolution by exploiting the supplementary information provided by binocular systems. Although previous methods have achieved promising results, they did not fully utilize the information of cross-view and intra-view. To further unleash the potential of binocular images, in this letter, we propose a novel Transformer-based parallax fusion module called Parallax Fusion Transformer (PFT). PFT employs a Cross-view Fusion Transformer (CVFT) to utilize cross-view information and an Intra-view Refinement Transformer (IVRT) for intra-view feature refinement. Meanwhile, we adopted the Swin Transformer as the backbone for feature extraction and SR reconstruction to form a pure Transformer architecture called PFT-SSR. Extensive experiments and ablation studies show that PFT-SSR achieves competitive results and outperforms most SOTA methods. Source code is available at https://github.com/MIVRC/PFT-PyTorch.
Learning on Graphs under Label Noise

Node classification on graphs is a significant task with a wide range of applications, including social analysis and anomaly detection. Even though graph neural networks (GNNs) have produced promising results on this task, current techniques often presume that label information of nodes is accurate, which may not be the case in real-world applications. To tackle this issue, we investigate the problem of learning on graphs with label noise and develop a novel approach dubbed Consistent Graph Neural Network (CGNN) to solve it. Specifically, we employ graph contrastive learning as a regularization term, which promotes two views of augmented nodes to have consistent representations. Since this regularization term cannot utilize label information, it can enhance the robustness of node representations to label noise. Moreover, to detect noisy labels on the graph, we present a sample selection technique based on the homophily assumption, which identifies noisy nodes by measuring the consistency between the labels with their neighbors. Finally, we purify these confident noisy labels to permit efficient semantic graph learning. Extensive experiments on three well-known benchmark datasets demonstrate the superiority of our CGNN over competing approaches.