Reviewerly/exHarmony huggingface dataset:
New Problems in Active Sampling for Mobile Robotic Online Learning

AI models deployed in real-world tasks (e.g., surveillance, implicit mapping, health care) typically need to be online trained for better modelling of the changing real-world environments and various online training methods (e.g., domain adaptation, few shot learning) are proposed for refining the AI models based on training input incrementally sampled from the real world. However, in the whole loop of AI model online training, there is a section rarely discussed: how to sample training input from the real world. In this paper, we show from the perspective of online training of AI models deployed on edge devices (e.g., robots) that several problems in sampling of training input on the device are affecting the time and energy consumption for the online training process to reach high performance. Notably, the online training relies on training input consecutively sampled from the real world and the consecutive samples from nearby states (e.g., position and orientation of a camera) are too similar and would limit the training accuracy gain per training iteration; on the other hand, while we can choose to sample more about the inaccurate samples to better final training accuracy, it is costly to obtain the accuracy statistics of samples via traditional ways such as validating, especially for AI models deployed on edge devices. These findings aim to raise research effort for practical online training of AI models, so that they can achieve resiliently and sustainably high performance in real-world tasks.
Deep Learning for Regional Subsidence Crisis Prediction in Smart Grid Infrastructure

Power infrastructure and its connectivity are central to building a smart grid. The infrastructure regional subsidence caused by environmental factors or geological hazards may devastate grid systems. Therefore, it is critical to forecasting the infrastructure regional subsidence in smart grids. In this study, we used an InSAR time series subsidence dataset based on satellite remote sensing images to train, test and compare four deep learning-based Transformer series prediction models using transfer learning to achieve subsidence crisis monitoring and prediction in smart grid infrastructure areas, considering the influence of environmental factors on infrastructure regional subsidence. Meanwhile, an GIS for subsidence crisis prediction in smart grid infrastructure areas was developed based on the Autoformer model. It helps the power industry maintain the smart grid more efficiently and accurately while also saving a lot of money and labor.
AI-Based Approach for Behind-the-Meter Solar Photovoltaic Power Forecasting

There are hundreds of thousands of behind-the-meter (BTM) solar photovoltaic (PV) installations across the USA which generate a substantial amount of electricity. These installations can provide an untapped reservoir of dispatchable resources that can be counted on to provide reliable grid services. The BTM load prediction and management can play a pivotal role for effective and reliable operation of the grid system. For these reasons, this research focuses on developing an artificial intelligence (AI)-based approach to estimate the PV capacity based on predicted temperature and solar irradiance. The AI algorithm explored in this research is a subtractive clustering-based adaptive neuro-fuzzy inference system (ANFīS) model. Simulation results show the efficacy of the ANFIS method in predicting the BTM PV power. Also, the performance of the ANFIS system is better than that of the linear regression, regression trees, and regression ensembles models.
Optimized Paillier Homomorphic Encryption in Federated Learning for Speech Emotion Recognition

Context: Federated Learning is an approach to distributed machine learning that enables collaborative model training on end devices. FL enhances privacy as devices only share local model parameters instead of raw data with a central server. However, the central server or eavesdroppers could extract sensitive information from these shared parameters. This issue is crucial in applications like speech emotion recognition (SER) that deal with personal voice data. To address this, we propose Optimized Paillier Homomorphic Encryption (OPHE) for SER applications in FL. Paillier homomorphic encryption enables computations on ciphertext, preserving privacy but with high computation and communication overhead. The proposed OPHE method can reduce this overhead by combing Paillier homomorphic encryption with pruning. So, we employ OPHE in one of the use cases of a large research project (DAIS) funded by the European Commission using a public SER dataset.
Synthetic Aerial Dataset for UAV Detection via Text-to-Image Diffusion Models

In this work, we present an approach to generate a synthetic aerial dataset for efficient Unmanned Aerial Vehicle (UAV) detection. We propose controlling the output of a text-to-image diffusion model by applying additional input conditions. Specifically, we train a diffusion model that enables conditional inputs, i.e., binary masks that specify all tractable parameters, including quantity, scale, pose, color, background, etc. Diverse photorealistic images with corresponding ground truth bounding boxes are generated automatically in an end-to-end manner. Without any interference, the dataset can be scaled to a large magnitude to facilitate the training process of UAV detection. Experimental results of YOLOv7 trained on the synthetic dataset demonstrate an extensive precision increment on unseen datasets of real images.
Context-aware, Composable Anomaly Detection in Large-scale Mobile Networks

In a large-scale mobile network, due to the diversity of data characteristics, detection purposes of operation teams, and analytics and machine learning algorithm abilities, building big data anomaly detection pipelines without considering different analytics and team situations may not yield expected quality of analytics, including detection relevancy, performance and quality.This is especially for analytics subjects, such as mobile network zones, of which characteristics are dynamic and contextual.Moreover, due to the lack of labeled data and the high cost of creating labeled data, building anomaly detection analytics models based on (supervised) deep learning or advanced models is even more challenging from various aspects of effort, cost and deployment.In this paper, we present a novel framework that enables anomaly detection through context-aware, composable components to provide efficient detection pipelines suitable for lightweight, resource constrained and geographical operation teams.First, we identify and categorize different types of analytics feature contexts and evaluate existing algorithms suitable for these contexts, mapping anomaly detection algorithms, patterns and configurations for data pre-processing and unsupervised detection tasks in individual analytics functionality.These context-specific pipelines detect anomalies and their relevancy for dynamic analytics subjects such as mobile network zones.Then we develop dynamic configuration and combination techniques for such pipelines to produce highly relevant, multi-context detection of anomalies.Our framework provides flexibility and configurations for team contexts to carry out the anomaly detection in the team's operations.We will demonstrate our work through real data gathered for a large-scale mobile network covering multiple types of sites with different geographical zones and equipment.We especially focus on district zones and userdefined zones as analytics subjects that must be managed by teams in our experiments.
CNN-based perceptual hashing scheme for image groups suitable for security systems

Perceptual hashing, which generates a message digest showing how humans perceive similarity in images, is suitable for ensuring the equivalence between a modified/edited image and the original. Conventional perceptual hashing is mainly utilized for similarity-based image retrieval and is not appropriate for image identification, which is required in security systems such as digital rights management. We previously developed a construction method for perceptual hashing in security systems that utilizes a convolutional neural network (CNN). In practical applications, multiple different images are published in various media forms (e.g., articles or books), so generating an identical message digest for each of these images simultaneously makes content management easier. Therefore, in this work we extend our earlier CNN-based perceptual hashing scheme so that it can generate an identical message digest for images in a group. This approach reduces the computational cost for fine-tuning CNN compared to generating a perceptual hash for each image in a group individually.
Scenario-based Customer Service Training System with Honorific Exercise

Customer service plays an important role in increasing customer satisfaction in service industries. In this paper, we propose a scenario-based customer service training system with an honorific exercise. The training system in this paper uses voice conversation with a customer agent. With this system, users receive training on how to use honorifics in speech, and a diagnostics report is automatically generated. This paper introduces the honorific training system we have developed and describes the results of experiments.
Illustration Drawing Interface with Image Retrieval and Adjustable Grid Guidance

Due to the proliferation of digital drawing devices and the growing popularity of digital illustrations, it has become an emerging issue to support amateur users in the digital illustration practice. While existing drawing support systems guide observation in illustration practice, there are few works that support the observation process itself. To address this issue, we propose a novel illustration drawing interface for practice support that incorporates adjustable grid guidance for tracing and image retrieval of reference illustrations based on the user’s rough sketches. The proposed system takes a rough user-drawn sketched input and presents similar illustrations according to the image features in the collected dataset. The user can select the preferred illustration and reproduce it using the drawing interface and can adjust the fineness of the grid guidance. We conducted a user study to verify the effectiveness of the proposed system in supporting illustration drawing practices.
Investigating Individuals’ Preferences in Determining the Functions of Smartphone Apps for Fighting Pandemics: Best-Worst Scaling Survey Study

Smartphone apps have been beneficial in controlling and preventing the COVID-19 pandemic. However, there is a gap in research surrounding the importance of smartphone app functions from a user's perspective. Although the insights and opinions of different stakeholders, such as policymakers and medical professionals, can influence the success of a public health policy, any strategy will face difficulty in achieving the expected effect if it is not based on a method that users can accept.This study aimed to assess the importance of a hypothetical smartphone app's functions for managing health during a pandemic based on the perspective of user preferences.A cross-sectional and web-based survey using the best-worst scaling (BWS) method was used to investigate the general population's preferences for important smartphone app functions. Participants were recruited from a professional surveying company's web-based surveying panel. The attributes of the BWS questionnaire were developed based on a robust process, including literature review, interviews, and expert discussion. A balanced incomplete block design was used to construct the choice task to ensure the effectiveness of the research design. Count analysis, conditional logit model analysis, and mixed logit analysis were used to estimate preference heterogeneity among respondents.The responses of 2153 participants were eligible for analysis. Nearly 55% (1192/2153) were female, and the mean age was 31.4 years. Most participants (1765/2153, 81.9%) had completed tertiary or higher education, and approximately 70% (1523/2153) were urban residents. The 3 most vital functions according to their selection were "surveillance and monitoring of infected cases," "quick self-screening," and "early detection of infected cases." The mixed logit regression model identified significant heterogeneity in preferences among respondents, and stratified analysis showed that some heterogeneities varied in respondents by demographics and COVID-19-related characteristics. Participants who preferred to use the app were more likely to assign a high weight to the preventive functions than those who did not prefer to use it. Conversely, participants who showed lower willingness to use the app tended to indicate a higher preference for supportive functions than those who preferred to use it.This study ranks the importance of smartphone app features that provide health care services during a pandemic based on the general population's preferences in China. It provides empirical evidence for decision-makers to develop eHealth policies and strategies that address future public health crises from a person-centered care perspective. Continued use of apps and smart investment in digital health can help improve health outcomes and reduce the burden of disease on individuals and communities.
Dataset Placement and Data Loading Optimizations for Cloud-Native Deep Learning Workloads

The primary challenge facing cloud-based deep learning systems is the need for efficient orchestration of large-scale datasets with diverse data formats and provisioning of high-performance data loading capabilities. To that end, we present DLCache, a cloud-native dataset management and runtime-aware data-loading solution for deep learning training jobs. DLCache supports the low-latency and high-throughput I/O requirements of DL training jobs using cloud buckets as persistent data storage and a dedicated computation cluster for training. DLCache comprises four layers: a control plane, a metadata plane, an operator plane, and a multi-tier storage plane, which are seamlessly integrated with the Kubernetes ecosystem thereby providing ease of deployment, scalability, and self-healing. For efficient memory utilization, DLCache is designed with an on-the-fly and best-effort caching mechanism that can auto-scale the cache according to runtime configurations, resource constraints, and training speeds. DLCache considers both frequency and freshness of data access as well as data preparation costs in making effective cache eviction decisions that result in reduced completion time for deep learning workloads. Results of evaluating DLCache on the Imagenet-ILSVRC and LibriSpeech datasets under various runtime configurations and simulated GPU computation time experiments showed up to a 147.49% and 156.67% improvement in data loading throughput, respectively, compared to the popular PyTorch framework.
Shedding Light on Darkness: Enhancing Object Detection Robustness with Synthetic Perturbations for Real-world Challenges

Robustness against distribution shifts is crucial for object detection models in real-world applications. In this study, we evaluate the performance of four state-of-the-art models against natural perturbations, retrain them with synthetic perturbations using the AugLy augmentation package, and assess their improved performance against natural perturbations. Our empirical ablation study focuses on the brightness perturbation modality using the COCO 2017 and ExDARK datasets. Our findings suggest that synthetic perturbations can effectively improve model robustness against real-world distribution shifts, providing valuable insights for deploying robust object detection models in real-world scenarios.
SVM at Edge: Low Cost Caching Prediction for Connected Edge Intelligence in Federated Machine Learning

This research work targets a problem in edge intelligent devices where caching becomes challenging task. We propose a Federated Learning (FL) technique that uses local model as Support Vector Machines (SVM) and global model as a Neural Network. Each participant in one FL run would train a SVM on its local data and use it to make predictions. Further, local aggregated predictions and learned SVM boundary used to train a global NN model. SVM is a simple and fast algorithm that can work well on small, local datasets, while Neural Networks are capable of modeling more complex relationships in data and can achieve higher accuracy on large, global datasets. The combination of these two models balances the trade-off between local accuracy and global accuracy, while preserving privacy by only sharing predictions. Results of this approach demonstrate its potential to provide a more accurate representation of data across multiple participants in a federated learning setup to predict cache and increase cache hit.
Skeleton-based action recognition with channel enhanced local graph window transformer

Recently, graph convolutional networks (GCNs) have achieved excellent performances in skeleton-based recognition. However, most of GCN-based methods are adopted with fixed skeleton graphs, which cannot adaptively model dependencies of nodes. To solve these problems, we propose a channel enhanced local graph window transformer (CE-GWT) to adaptively model spatial dependencies of skeleton. First, local neighborhood graphs are aligned as ordered vectors by average pooling. Attention based on aligned local graph window is proposed to adaptively explore the latent features of nodes. Second, we propose a channel enhanced module to enhance the modeling capacity of local self-attention on the channel dimension. The value matrices of different nodes are computed respectively with unshared parameters. Finally, we use these new concepts to build a series of model variants, and determine the best design choices for such architectures. Extensive experiments on three large-scale datasets demonstrate that our model could achieves comparable performance with the state-of-the-art works.
Multi-scale Attention Enhancement for Arbitrary Style Transfer via Contrast Learning

Arbitrary style transfer is to transfer any artistic style to the content image while preserving the content structure as much as possible. Although there is currently a lot of work to improve the effect of image migration, there are still problems, such as the content structure being destroyed or the image quality not being guaranteed. In this paper, we combine multi-scale attention with contrastive learning methods to propose an efficient model called AttCST. The critical part is to fuse the current features extracted by the pre-trained deep convolutional network with the previous shallow features. Make full use of the previous shallow features to supplement the texture information lost by the in-depth features. A contrastive learning model is then used to distinguish the same style from others, thereby enhancing the network’s learning ability. At the same time, we add a local loss to constrain the transfer module to improve the quality of generated images. We conduct extensive qualitative and quantitative analyses, the results of which demonstrate the effectiveness of our approach.
A Vision-Based Counter System for Children's Dribble

In this paper, a vision-based counter system for children's dribble based on Google MediaPipe's human posture recognition algorithm and YOLOv5 object recognition algorithm is introduced. Firstly, the hands’ coordinates of the child is located using MediaPipe, and the coordinates of the ball is found using YOLOv5; by the relative positions of the hands and the ball in the multi-frame images of the video, an action sequence of ball holding, touching and ball bouncing is recognized and used to count the number of dribble. The experimental results show that the system can count a single child's dribble accurately.
Improving Optical Flow Estimation Accuracy by Correlation Volumes Attention

Among the optical flow estimation algorithms, the algorithm based on the dense correlation volumes performs best. The correlation volume represents the matching relationship between pixels. Usually, we compute the correlation volumes of several pixels in the next frame corresponding to a certain pixel in the previous frame, but only the part of correlation volumes contains key information. We propose the Correlation Volumes Attention Network (CVA-Net), which assigns different weights according to the importance of correlation volumes, so that the optical flow update module perceives useful information from the input. Our network pretrains on popular datasets FlyingChairs and FlyingThings. On Sintel Clean and Sintel Final datasets, we obtain a new state-of-the-art performance, improving the average end-point error by 11.2% (1.27) and 3.3% (2.62). And on KITTI-15 datasets, we also obtain a new state-of-the-art performance and improves by 0.49 on average end-point error and 1.13 on Fl-all.
Pseudo 3D-Attention for Real Image Denoising

Attention mechanism has shown great potential in image noise reduction tasks. We can choose different attention mechanisms according to different noise reduction tasks to enhance the ability of network to extract corresponding information. The problem of blurring and smoothing of the denoised image in the real image denoising task is considered. This paper recommends an attention depth fusion mechanism to solve this problem. It is different from the previous use of serial or parallel channels and spatial attention mechanisms to output features directly. After parallel connection, we use the shuffle operation to achieve cross channel communication and accelerate feature fusion of the two attention mechanisms. Then use a simple MLP module to perform cross direction channel attention calculation on the acquired spatial and channel attention features. And completing the deep fusion and feature interaction of spatial attention mechanisms and channel attention mechanisms. Final we use the Cross-attention to further enhance ability of the model to extract global feature and forward long semantic information.We name this module Pseudo 3D Attention. Finally, we conduct evaluations on real image denoising benchmarks including SIDD, DND, CC15, PolyU. We proposed method achieve competitive results. In particular, PSNR of 39.71dB and SSIM of 0.961 in SIDD is achieved without extra an train set.
Research on enterprise text classification methods of BiLSTM and CNN based on BERT

The traditional enterprise text data classification method ignores the context of the text. Each word is independent from each other and cannot represent semantic information. The text description and classification effect is poor, and the feature engineering needs human intervention, so the generalization ability is not strong. In view of the low efficiency and accuracy of enterprise text data classification, this paper proposes a bidirectional encoder representation based on Transformer (BERT) The enterprise text classification model BBLC-ATT based on convolutional neural networks (CNN) and bi-directional long short-term memory (BiLSTM) neural networks and attention mechanism (Attention). The model uses BERT training word vector and combines the features of CNN and BiLSTM to capture local potential features and context information. Secondly, the feature vectors extracted from the hybrid network layer are input into the self-attention layer to extract the syntactic and semantic features between words in the enterprise text sentences. Finally, this paper compares BBLC-ATT model with traditional deep learning model in terms of accuracy, accuracy, recall and F1 value. The experimental results show that the BBLC-ATT model is superior to other models in all evaluation indicators, and the accuracy rate is increased by 3.28% - 15.86%.