Reviewerly/exHarmony huggingface dataset:
Enhancing Whisper Model for Pronunciation Assessment with Multi-Adapters

Automatic pronunciation assessment is an important part of computer-aided pronunciation training. Due to the scarcity of non-native pronunciation assessment datasets and the fact that traditional speech assessment usually uses Goodness of Pronunciation(GOP) features, this may not provide enough information for word or sentence-level assessment. This paper aims to improve the performance of automatic pronunciation assessment from two aspects. First, to alleviate the problem of insufficient training data for pronunciation assessment, we use the weakly supervised learning model Whisper to build a pronunciation assessment model. With the Whisper encoder, Pearson correlation coefficient(PCC) performance is significantly improved compared to traditional acoustic features. Second, we propose a multi-adapters method that uses a multi-task loss to fine-tune the adapter while simultaneously learning phone, word, and sentence assessment tasks to boost sentence-level assessment task performance. In addition, through the experimental comparison of different scale models in the Whisper, the experimental results on the open-source dataset speechocean762 show that our proposed method achieves the best results in the medium.en model.
Holo-QoI: A Human Factor-Based Dataset and Prediction Framework for Assessing Quality of Interaction in Augmented Reality

With the revolution of Augmented Reality (AR) Head Mounted Display (HMD), AR technology enables the provision of immersive virtual content using various interaction methods. To bridge the real and virtual world, hand-based interactions have been utilized for their simplicity and effectiveness in interacting with AR applications. Thus, assessing the Quality of Interaction (QoI) is vital for enhancing the experience of AR applications, but it has been hindered by the absence of a comprehensive database that includes human factors. In this paper, we construct an interaction dataset, Holo-QoI, which includes 49 sets of human factor metadata with associated human subjective scores. To analyze the contribution of various factors to QoI, we process HMD metadata into human factor-based features (e.g. head, gaze, and hand motion). We then design a QoI prediction framework based on a Transformer architecture to model the relationship between human factor-based features and deep features, which is used to predict the QoI score of an interaction. Through rigorous experiments, we demonstrate that our proposed model achieves an 86% accuracy using the constructed dataset.
Transformer-based Image Compression with Variable Image Quality Objectives

This paper presents a Transformer-based image compression system that allows for a variable image quality objective according to the user’s preference. Optimizing a learned codec for different quality objectives leads to reconstructed images with varying visual characteristics. Our method provides the user with the flexibility to choose a trade-off between two image quality objectives using a single, shared model. Motivated by the success of prompt-tuning techniques, we introduce prompt tokens to condition our Transformer-based autoencoder. These prompt tokens are generated adaptively based on the user’s preference and input image through learning a prompt generation network. Extensive experiments on commonly used quality metrics demonstrate the effectiveness of our method in adapting the encoding and/or decoding processes to a variable quality objective. While offering the additional flexibility, our proposed method performs comparably to the single-objective methods in terms of rate-distortion performance.
Person Re-Identification for Multi-Camera, Multi-Object Tracking on Robotic Platforms

Person re-identification (ReID) is defined as the problem of matching identities of interest across disjoint camera views, or in the same camera in different occasions. It provides significant value in the context of public security, as the ability to run such an application across a dynamic multi-camera system will increase efficiency in identifying and tracking any persons of interest who may pose a threat to the population. Existing multi-object trackers are only able to perform tracking with one camera, as the same identity appearing in another camera will not be re-identified. This paper aims to serve as a basis for developing a deep learning-based extension of a single-camera multi-object tracker to a multi-camera use case on a mobile robotic platform.
Deepfake-speech Detection with Pathological Features and Multilayer Perceptron Neural Network

Deepfake speech, a misuse of speech technology, is of great concern since it seems natural and is difficult to detect. Although many methods using various speech features have been proposed, deepfake-speech detection accuracy must be improved, especially in real-world scenarios. Therefore, this paper presents a method for detecting deepfake speech on the basis of pathological features used by pathologists for assessing voice quality. The six-pathological features, including jitter, shimmer, harmonics-to-noise ratio, cepstral-harmonics-to-noise ratio, normalized noise energy, and glottal-to-noise excitation ratio, are fed to a multilayer perceptron neural network. We evaluated the proposed method using the Audio Deep Synthesis Detection Challenge dataset. The results indicate that the proposed model can be used for detecting deepfake speech. The proposed method’s accuracy, precision, recall, and F1-score were over 98% on the development set, and it outperformed the baseline method on the adaptation set.
Mecanum-Wheeled Robot Control Based on Deep Reinforcement Learning

The use of mecanum wheels for mobile robots has led to more flexibility in terms of the robots' navigation ability. This usage, however, leads to a more complex control system which mainly relies on a complete knowledge of the robot's kinematic model. This research aims to show that reinforcement learning can help develop control systems for robots with mecanum wheels. Since reinforcement learning does not require the robot's model explicitly to generate the control action, it can deal with the uncertainty and nonlinearity of the robot model by learning the action directly from interaction with the environment. In the paper, we proposed a novel reward function specifically designed for mecanum-wheeled robot control. The reward function is used to train the reinforcement learning algorithm to produce control signal for mecanum-wheeled robot. The results demonstrate the promising capability of reinforcement learning in controlling mecanum-wheeled robot towards a target location while maintaining its orientation.
Enhancing Autonomous Robot Navigation Based on Deep Reinforcement Learning: Comparative Analysis of Reward Functions in Diverse Environments

Autonomous robot navigation in complex environments presents a significant challenge due to efficient decision-making for reaching goals and avoiding obstacles. This paper addresses this issue through the use of deep reinforcement learning techniques and a comprehensive analysis of reward functions and their impact on autonomous navigation. The study emphasizes the importance of selecting the most effective reward functions to achieve maximum robot performance in a variety of scenarios. Moreover, we propose a new reward mechanism that enables the robot to avoid collisions when objects move faster than the robot, resulting in the robot halting its motion to allow the object to pass before resuming its course. The effectiveness of these reward functions is validated through simulations, providing valuable insights into the robustness of robot navigation. Further details and simulations can be found in the following link: https://youtu.be/pPQDc25vj1U
Enhancing Real-Time Semantic Segmentation with Textual Knowledge of Pre-Trained Vision-Language Model: A Lightweight Approach

In this paper, we present a lightweight method for real-time semantic segmentation models by leveraging the power of pre-trained vision-language models. Our approach incorporates the CLIP text encoder, which provides rich semantic embeddings for text labels, and effectively distills its rich textual knowledge to the segmentation model. The proposed framework integrates the image and text embeddings, enabling visual and textual information alignment. Besides, we introduce learnable prompt embeddings to capture class-specific information and enhance the semantic understanding of the model. To ensure efficient learning, we devise a two-stage training procedure that allows the segmentation backbone to learn from fixed text embeddings in the first stage and optimize the prompt embeddings in the second stage. Extensive experiments and ablation studies demonstrate the effectiveness of our method in significantly improving the performance of the real-time semantic segmentation model.
Analysis of Technique Characteristics Based on YOLOv5 for Real-Time Small Object Detection

Small object detection using deep learning is required in various fields such as military and medical as well as security/surveillance using surveillance and reconnaissance drones. We apply various techniques to YOLOv5 to build a real-time small object detection model and compare and analyze the results. We compare the contribution of 1) sufficient feature maps fusion through a heavier neck than the existing model, and 2) replacing up-sampling with Transposed Convolution to efficiently increase the feature map size. Each technique led to improved performance in detecting small objects than the existing models, and we will continue research to build a model with better performance by combining these techniques.
Corpuscular Volume Estimation Using a Small Amount of Blood via Microscope Images

Blood is an analysis serves as a valuable diagnostic tool that can easily and simply diagnose health, and in particular, red blood cells (RBC) or hematocrit (Hct) and mean corpuscular volume (MCV) can diagnose various diseases. Electrochemical methods have been proposed to find out RBC, Hct, and MCV, but it is difficult to use them for infants or the elderly because they require a large amount of blood. As an alternative, method like blood smear has been suggested to make a diagnosis with a small amount of blood, but it takes a long time to dry the sample. In this study, we present a methodology to make the simple and quick blood analysis. This method uses a microscope image of a small amount of blood, without drying the blood sample. From the image, in order to detect all the blood cells, we adopt a loop algorithm consist of concave point detection, ellipse fitting, and blob detection. To assess the validity and accuracy of the proposed method, an investigation is conducted to examine its correlation with the Sysmex XN-1000, a widely utilized medical reference standard; we show that the correlation coefficients are higher than 0.88, showing more correlation than the method in previous studies by comparing the estimation results. By developing this method, it will be possible to establish a simple quick blood analysis method using microscope images, decreasing a time consumed for the health checkup, saving more lives.
Exploring the i3DVAE-LSTM Framework for Generating Exceptionally Rare Anomaly Signals

In the era of data-driven approaches, ensuring data quality is crucial for developing effective machine learning and deep learning models. While data augmentation is commonly used to increase the sample size, it does not guarantee data quality. Data generation goes beyond augmentation by incorporating additional steps to ensure high-quality output samples. This technique is particularly valuable for anomaly classification tasks with limited training samples. A recent study introduced a 3DVAE-LSTM (3-Dimensional Variational Autoencoders-Long Short-Term Memory) approach for generating extremely rare case signals. Although this framework synthesized samples for training deep learning models, it faced challenges with long sequential data. To address this, the authors proposed an improved version called i3DVAE-LSTM (Improvement of 3-Dimensional Variational Autoencoders-Long Short-Term Memory) and presented the evaluation of the i3DVAE-LSTM framework. The proposed framework adopts a divide-and-conquer technique, splitting long sequence data into smaller fragments to enhance the quality of generated samples, which are then concatenated together. Experimental results demonstrated that classification models trained with data generated by i3DVAE-LSTM outperformed baselines in all aspects.
UDE-Based Non-Fragile Control Design for Interval Type-2 Fuzzy Systems

This paper investigates the tracking control problem for interval-type 2 fuzzy systems that incorporate unknown bounded uncertainty and unknown external disturbances. The control design incorporates the utilization of an uncertainty and disturbance estimator (UDE) to enhance robustness against these factors. Additionally, a non-fragile strategy is employed to effectively handle the impact of gain perturbation. Moreover, interval type-2 fuzzy sets are employed to represent the model uncertainties in the addressed system. The UDE algorithm is utilized based on the assumption that a continuous signal can be approximated and estimated using a filter with the appropriate bandwidth. Subsequently, the tracking control problem of the considered interval type-2 fuzzy system is equivalently transformed into the stability problem of an error system. Furthermore, conditions are derived to guarantee closed-loop system stability and achieve asymptotic disturbance rejection and reference tracking. Finally, simulation results are presented to demonstrate the advantages of the developed theoretical results.
Zone-of-Interest Strategy for the Creation of High-Resolution Adversarial Images Against Convolutional Neural Networks

Trained convolutional neural networks (CNNs) are among the leading tools used for the automatic classification of images. They are nevertheless exposed to attacks: Given an input clean image classified by a CNN in a category, carefully designed adversarial images may lead CNNs to erroneous classifications, although humans would still classify “correctly” the constructed adversarial images in the same category as the input image. Currently most attacks are performed in the image input size domain of the considered CNN, which is usually small. However, due to privacy concerns with personal images on social media, there is a demand for generating large adversarial images that preserve the visual information of original images with the highest possible quality, while preventing automatic tracking and personal identification. Creating large-size adversarial images is difficult due to speed, adversity, and visual quality challenges, in particular if a requirement on adversarial images is the inability for humans to notice any difference between them and the original clean images. This paper describes the zone-of-interest generic strategy that aims at increasing drastically the efficiency of any type of attack (white-box or black-box, untargeted or targeted) and any specific attack (FGSM, PGD, BIM, SimBA, AdvGAN, EA-based attacks, etc.) on CNNs. Instead of exploring the full image size, the strategy identifies zones on which to focus the attacks. Although applying to any image size, the strategy is especially valuable for large high-resolution images. This strategy can be combined with other generic approaches, like the noise blowing-up method, to further improve attacks' performances.
Tilted Circumnavigation of Multiple Drones Around Multiple Targets

This paper proposes a new approach for coordinating a group of drones to follow a moving circular path while maintaining a specified distance between them. Unlike existing methods, which track an ellipse around multiple targets, the proposed approach defines a tilted circle in the air, whose projection becomes the enclosing ellipse on the ground. This allows drones to track the circle at a constant velocity while adjusting their altitude, avoiding collisions with each other. A vector-field guidance law followed by an integral sliding-mode control is designed to ensure that drones meet three conditions under bounded disturbances: reaching the plane of the tilted circle, turning around its center with a time-varying radius, and avoiding collisions. Numerical simulations demonstrate the effectiveness of the proposed strategy in real-life scenarios.
Influence of Human Skills on Scrum Teams: A Case Study of Junior Team Software Development

Agile software development methodologies, including Scrum, have become popular for their flexibility and collaborative approach. However, the success of Scrum teams heavily depends on the dynamics among team members. This study explores the challenges related to Agile software development, specifically focusing on the influence of human skills on the Scrum team. While Scrum encompasses various well-defined aspects, it does not explicitly address personality traits. Consequently, this study seeks to explore how teams' performance is influenced by the human skills possessed by their members during the implementation of Agile methodologies. This study investigated eight teams of software engineering students engaged in software development projects by observing impacted team processes in each iteration. The results show that soft skills and J/P significantly influence team processes. However, their impact on the team's product varies across cycles. Additionally, soft skills interact with the Feeling (F)/ Thinking (T) and Judging (J)/Perceiving (P), affecting team processes and products.
Vocal Tract Length Perturbation-based Pseudo-Speaker Augmentation for Speaker Embedding Learning

Data augmentation is essential for constructing reliable automatic speaker verification (ASV) systems. It is well known that data augmentation increases the number of utterances by adding noise and is effective in most methods. However, the number of speakers in the training data also plays an important role in enhancing ASV system performance. The robustness of speaker embedding networks, which are used in ASV systems, relies on the number of speakers present in the training data. To address this, we propose a method called pseudo-speaker augmentation, which utilizes a technique called vocal tract length (VTL) warping. By changing a parameter, the VTL warping technique alters speaker characteristics, allowing us to easily increase the number of speakers. Since the speaker embedding network aims to classify speakers, having a larger number of speakers enhances its robustness. In our experiments, the pseudo-speaker augmentation method improved the performance of the speaker embedding-based ASV system, achieving an equal error rate of 4.058% on the JTubeSpeech database.
LSCR: Latent Space Coordination Relation for Anomaly Prediction

Anomaly time series prediction is a crucial yet challenging task in real-world systems. Existing techniques often require a substantial amount of data to achieve satisfactory performance, posing a significant challenge as anomaly data is scarce and difficult to obtain. Despite attempts to address this issue using traditional machine learning techniques, their effectiveness remains limited, resulting in performance degradation or costly trade-offs. Therefore, in this paper, we propose a novel approach called Latent Space Coordination Relation for Anomaly Prediction to overcome these challenges. Our framework leverages the power of the Variational Autoencoder (VAE) and learns the coordination relations of points in the latent space to detect anomalies. By exploiting the latent space, our method enables effective learning and prediction of anomalies. Additionally, the decoder of the VAE aids in restoring the data, further improving the accuracy of anomaly detection. Experimental results demonstrate that our approach outperforms baseline models when training data is limited. The predicted anomalous signals exhibit lower error rates, highlighting the efficacy of our method. This improvement is attributed to the utilization of the latent space for learning and assisting in anomaly prediction, along with the signal restoration capabilities of the decoder.
Reconstruction-Based Anomaly Detection with Noise-Induced Normal Sample Augmentation

Anomaly detection has been an actively researched field, particularly in the context of industrial automation. In recent years, significant efforts have been made towards reconstruction -based approaches for anomaly detection. This method involves using a model trained on normal data patterns to generate reconstructions of given data, and then evaluating the difference between this input and the model's reconstruction to detect anomalies. However, in practice, models often struggle to control the generalization boundary or encounter ambiguity, resulting in inadequate separation between normal and abnormal instances or performance degradation due to overfitting. To address these challenges, this paper proposes a process that applies noise to normal images and reconstructs them into normal images. The proposed method learns patterns of normal images through a reconstruction -based network, allowing for the detection of anomalies by comparing input images with their reconstructed counterparts. Additionally, various normalization techniques are explored to improve the performance of the network. The effectiveness of our approach is demonstrated through superior performance on the MVTec AD benchmark, providing opportunities for early detection and intervention in potential anomaly situations in real-world applications.
Cross-Domain Person Re-Identification Using Value Distribution Alignment

Person re-identification is a technique used to identify individuals across different camera views, with applications in security, surveillance, and public safety. However, it is a challenging task due to factors like varying lighting conditions, camera viewpoints, and the appearance changes of individuals. Consequently, performance degradation often occurs in cross-domain person re-identification when training and testing data come from different datasets. To tackle this issue, we propose a value alignment module that mitigates the impact of brightness differences between domains. The proposed module is simple but effective such that it adjusts the input data at the front end of the model by aligning the brightness distribution of image data with a specific distribution. The performance of the proposed module is evaluated on the cross-domain person re-identification setting using different datasets in training and testing, specifically the Market1501 and CUHK03 datasets. Experimental results demonstrate that applying the proposed module significantly improves the generalization performance of cross-domain person re-identification.
Deformable Aligned Fusion for Video Super Resolution

The key to the video super-resolution algorithm is to capture the information of adjacent frames to supplement the reconstruction of the current frame. It is necessary to apply the alignment operation to compensate the displacements of objects in different frames. We propose an innovative deformable aligned fusion block (DAFB) to exploit the complementary information from adjacent frames more efficiently. We also model the offsets of deformable alignments as probabilistic distributions to augment the robustness of a DAFB. Our proposed model outperforms state-of-the-art CNN-based models on benchmark testing datasets, including Vid4 and UDM10.