Reviewerly/exHarmony huggingface dataset:
Exploring Augmented Reality Notification Placement while Communicating with Virtual Avatar

Augmented reality head-mounted displays (AR HMD) enable visual notifications directly into the user's field of view (FoV) to perform dual-task. However, research on the efficient presentation of notifications when communicating with a virtual avatar in augmented reality (AR) situation has been insufficient. For efficient AR notification presentations, it is necessary to understand where to display notifications without interfering with virtual avatar communication. We investigated two coordinate systems (world-fixed and display-fixed) and eight positions for AR notifications when conversing with the virtual avatar. Results showed that the top-left position increased response time and task load. Our study contributes as a cornerstone to designing AR notifications in communication with virtual avatars.
Open-Ended Visual Question Answering Model For Remote Sensing Images

In this paper, we present an open-ended visual question answering (VQA) model for remote sensing images, where the answers can be given in the form of short sentences, unlike closed-ended VQA. This model uses a vision and natural language transformers for embedding the image and its related question. The feature representations obtained at the output are concatenated and fed to a light transformer decoder for generating the answer in an autoregressive way. The complete architecture is trained in an end-to-end manner via the backpropagation algorithm. In the experiments, we evaluate the model on a manually labeled open-ended VQA dataset termed TextRS composed of 6245 image-question pairs.
Adversarial Training for Channel State Information Estimation in LTE Multi-antenna Systems

Deep neural networks can be utilised for channel state information (CSI) estimation in wireless communications. We aim to decrease the bit error rate of such networks without increasing their complexity, since the wireless environment requires solutions with high performance while constraining implementation cost. For this reason, we investigate the use of adversarial training, which has been successfully applied to image super-resolution tasks that share similarities with CSI estimation tasks. CSI estimators are usually trained in a Single-In Single-Out (SISO) configuration to estimate the channel between two specific antennas and then applied to multi-antenna configurations. We show that the performance of neural networks in the SISO training environment is not necessarily indicative of their performance in multi-antenna systems. The analysis shows that adversarial training does not provide advantages in the SISO environment, however, adversarially trained models can outperform non-adversarially trained models when applying antenna diversity to Long-Term Evolution systems. The use of a feature extractor network is also investigated in this study and is found to have the potential to enhance the performance of Multiple-In Multiple-Out antenna configurations at higher SNRs. This study emphasises the importance of testing neural networks in the context of use while also showing possible advantages of adversarial training in multi-antenna systems without necessarily increasing network complexity.
An Opt-in Framework for Privacy Protection in Audio-Based Applications

Installing audio-based applications exposes users to the risk of the data processor extracting additional information beyond the task the user permitted. To solve these privacy concerns, we propose to integrate an on-edge data obfuscation between the audio sensor and the recognition algorithm. We introduce a novel privacy loss metric and use adversarial learning to train an obfuscator. Contrary to existing work, our technique does not require users to specify which sensitive attributes they want to protect (opt-out) but instead only provide permission for specific tasks (opt-in). Moreover, we do not require retraining of recognition algorithms, making the obfuscated data compatible with existing methods. We experimentally validate our approach on four voice datasets and show that we can protect several attributes of the speaker, including gender, identity, and emotional state with a minimal recognition accuracy degradation.
Analysis and Satisfaction Evaluation of Online Learning Based on Artificial Intelligence

Innovative technology represented by artificial intelligence drives the change of educational concept and practice, the transformation of learning environment and teaching methods to intelligence, and online learning enters the era of learner sovereignty. In this paper, rough set algorithm is used to build an online learning quality evaluation index system, and online learning quality and satisfaction are evaluated and analyzed based on artificial intelligence. The results show that the accuracy of rough set algorithm is the highest, and the recall rate of rough set algorithm is the highest in different data sets, showing an overall upward trend, the highest recall rate is 93.58%. The weight percentages of the first-level indicators are curriculum environment experience (15%), of curriculum content experience (38%), of curriculum activity experience (26%), curriculum interaction experience (6%) and learning effect experience(15%). The corresponding evaluation scores are reflected accordingly, which can objectively describe the online quality evaluation.
A Neural Network Approach for Text Classification Using Low Dimensional Joint Embeddings of Words and Knowledge

The continuous expansion of textual data collection and dissemination in electronic means has made text classification a crucial task to help exploit, in a variety of applications, massive amounts of digital texts available nowadays. Knowledge Graphs (KGs) or their embeddings can provide additional semantics to improve text classification. However, most proposals from the literature rely solely on words found in the texts to classify them. A few text classification approaches employ knowledge embeddings besides word embeddings, but which are produced separately and not integrated into the same vector space. Different from previous proposals, this work applies an existing solution for generating text and knowledge embeddings in an integrated way to feed neural classifiers. Experiments using these joint embeddings with 50 dimensions yielded results comparable to those of state-of-the-art approaches on the AG News dataset and slightly superior to the BBC news dataset.
Multi-Scale Local Feature Fusion Network for Facial Expression Recognition

To solve the problem that facial expression recognition (FER) system in actual application scenariosis always interfered by complex background which lead to low accuracy, we designed a multi-scale local feature fusion network (MSLFnet) to improve the performance of FER in actual application scenarios. Middle-level facial features map are extracted from the backbone, and the middle-level local feature is generated by a patch-level local attention module, the network can obtain richer facial expressions. Experiments is carried out on the FER datasets RAF-DB and FER+ to verify the efficacy of the network. Experimental results show that the accuracy of the proposed network on RAF-DB and FER+ is 2.5% and 1% higher than original ResNet-18, proving the effectiveness of MSLFnet.
Analysis of Autoencoders with Vapnik-Chervonenkis Dimension

In statistical learning, the Vapnik-Chervonenkis(VC)-dimension has been widely used to analyze single-layer neural networks such as Perceptron and Support Vector Machine while utilizing it for multilayer networks has rarely been explored. This motivates us to introduce the VC-dimension method to autoencoder, one of important multilayer networks. The paper proposes several theoretical observations of analyzing the relationship among network architectures, activation functions, and the learning capacity and effectiveness of autoencoders. We also provide a theoretical VC-limitation result to quantify the boundary of hidden neurons in an Autoencoder.
A Comparative Study of Federated Versus Centralized Learning for Time Series Prediction in IoT

IoT devices have become increasingly popular in the last few years due to the potential of their sensors to feed data and provide insights into various applications in many fields. Today, such sensors are used in health care, environmental forecasting, and finance systems, to name a few. Predictive algorithms can leverage the temporal data provided by IoT sensors to enrich real-time applications to, for example, predict CO2 and temperature levels in a given region and provide public alerts. In this context, to find out the best solution for predicting time series generated from data collected by IoT devices, this work evaluates two machine learning approaches: federated learning and centralized learning. Federated learning implies training the algorithms in a distributed way across devices, while centralized learning takes data from devices into a server and focuses training on it. We performed experiments using Long Short Term Memory (LSTM) to predict the time series with federated and centralized strategies. The results show that the Federated Learning model predicts five time-steps of a time series with, on average, 78% less mean squared error and intakes 86% less communication load in the network than a Centralized solution.
Fast Analysis of Evolving Software Systems

In this paper, we present an algorithm that efficiently updates results of dataflow analysis in response to incremental changes. Our incremental algorithm work in two phases: it compute summaries for selected procedures in phase 1 by traversing the call graph in bottom-up order and, in phase 2, it updates the dataflow values for selected procedures by traversing call graph in top-down order, thus making the analysis faster. The selection of procedures is done by comparing summaries across the version. We have implemented this algorithm in our proprietary static analysis tool, used by many clientele over the years, for automated defect detection. An evaluation of our algorithm on a core banking application shows that on an average it takes 90 % lesser time in comparison to an exhaustive analysis, demonstrating practical benefit of our algorithm on a real-world evolving software system.
RotatGAT: Learning Knowledge Graph Embedding with Translation Assumptions and Graph Attention Networks

Knowledge Graph Embedding (KGE) is to learn continuous vectors of entities and relations in the Knowledge Graph (KG). Inspired by the R-GCN model, we propose a novel embedding learning model named RotatGAT, which combines the RotatE model and the GAT model. The goal is to overcome the shortcomings of R-GCN, that has a relatively high computing complexity and cannot distinguish the importance of neighbors. We introduce the RotatE model into RotatGAT to represent the embeddings of heterogeneous entities and relations in KG. Considering RotatE cannot use the structure information to learn entities' embeddings, we introduce the GAT model to learn the importance of neighbors of an entity and aggregate the feature information of neighbors for graph embedding learning. The link prediction experiments show the overall performance of RotatGAT on four benchmark datasets outperforms existing state-of-the-art models.
Motion Accumulation and Selection Network for Video Understanding

Numerous existing studies have demonstrated the significance of motion information for video understanding. In the literature, most of the previous methods rely heavily on the temporal differences of features extracted by convolutional networks (CNN) to represent the motion. However, this type of motion representation approach may have two poten-tial drawbacks: 1) The difference operation may cause the in-completeness of the extracted moving target contour; 2) Treating all the extracted motion features equally may lead to situations where some motion features will not contribute to the classification or even produce negative incentives. For these two challenges, we propose a motion accumulation and selection network (MAS-Net) based on a novel embedded 2D CNN. According to the experimental results on typical video datasets, it is shown that MAS-Net has achieved the state-of-the-arts on both Something-Something V1&V2, while the computational load is kept at a relatively low level.
Video description method based on sentence semantics and length loss calculation

The KL divergence loss function commonly used in existing models can represent a single mapping relationship of length distribution by learning specific visual content features, but it cannot accurately predict the length distribution of sentences in a one-to-many mapping relationship, resulting in the description of semantic information. incomplete. Second, the loss is calculated using a character-level negative log-likelihood function, ignoring sentence-level semantics, resulting in inaccurate semantic information of the generated descriptions. In this paper, a new length loss function is designed to adaptively adjust the error penalty by measuring the distance between the prediction and the reference length, so that the model can learn the optimal description length distribution in highly similar visual content. Secondly, through the semantic hidden state calculation of the decoding layer, a description generation loss function based on sentence semantics is designed, and the optimal sentence semantic description is iteratively obtained by comparing the semantics of the prediction and the reference description at the sentence level. The experimental results show that the model is tested on the MSVD and MSR-VTT datasets, and various indicators are significantly improved, which are better than the existing models.
Noisy Label Learning in Deep Learning

Currently, the construction of a large-scale manual annotation databases is still a prerequisite for the success of DNN. Although there is no shortage of data, there is a lack of clean label data in many fields, because it takes a lot of time and huge labor costs to build such a database. As many studies have shown that noisy label will seriously affect the stability and performance of the DNN. Learning from noisy labels has become more and more important, and many methods have been proposed by scholars. The purpose of this paper is to systematically summarize the different ideas for solving the noisy label learning problem, analyze the problems with existing methods, and try to analyze how to solve these problems. First, we will describe the problem of learning with label noise from the perspective of supervised learning. And then we will summarize the existing methods from the perspective of dataset usage. Subsequently, we will analyze the problems with the data and existing methods. Finally we will give some possible solution ideas.
PFedSim: An Efficient Federated Control Method for Clustered Training

To further promote the implementation and application of artificial intelligence technology under data silos, federated ecology has defined a set of standardized and systematic solutions from upstream data to downstream applications from a systematic perspective. As the core of the federated ecology, federated control can transform the abstract tasks of federated management into concrete execution rules. Clustered training is a common cooperation rule for the personalized federated training task of multi-party cooperation. However, it is difficult to generalize in large-scale cooperation scenarios because of increased data transmission pressure and server storage burden. Therefore, this paper designs a personalized federated control method based on similarity measures(pFedSim) so that the server can accurately distribute the cluster model to clients, thereby reducing the transmission cost. Compared with the existing clustered training method IFCA, pFedSim transfers the selection rights of cluster models from the client to the server. Model selection preference, accurate distribution of cluster models. Experiments show that in the personalized training task with s clusters, the method can obtain the same model level with 1/s data transmission. It is better than the current optimal clustering training method Macro-Acc increased by 2.25%.
Fine-grained relational learning for few-shot knowledge graph completion

Few-shot knowledge graph completion (FKGC) task aims to infer missing entities or relations by using few-shot support instances in the knowledge graph. Existing FKGC methods focus on the learning of few-shot relation representations, which are obtained by aggregating the neighbor information of each entity. However, most of these models take the entity's neighbor relations and entities as the same hierarchy and do not make fine-grained distinctions, resulting in entity embeddings with low expressiveness, which may further decrease the quality of learned few-shot relation embeddings. Moreover, many of those models directly use the concatenation of the entity embeddings as the relation representations, and neglect the valuable interaction between relations. In this paper, we propose a fine-grained relational learning framework IDEAL for few-shot knowledge graph completion task. Specifically, we first propose a unique hierarchical attention encoder to aggregate the neighbor information of each entity from two levels, i.e., the entity-relation level and the relation-entity level. Then a relation recoding validator is proposed to formulate the interaction between different relations. Instead of obtaining the few-shot relation representations by using the entity embeddings, the relation recoding validator module aggregates the neighbor relations of each entity to encode the few-shot relation, which can reduce the over-dependence on specific entities in the few-shot relation encoding phase. The relation recoding module is also extended with respect to the excellent performance of the transformer in modeling sequence information. We then introduce a transformer encoder to extract underlying and valuable sequence information between relations. Extensive experiments are conducted on two datasets, i.e., NELL and Wiki. The experimental results demonstrate that our model outperforms state-of-the-art FKGC methods. Besides, we devise the ablation study to demonstrate the effectiveness of each key component. The case study also shows the interpretability of our model intuitively.
Supervised Learning of Procedures from Tutorial Videos

Online educational platforms and MOOCs (Massive Open Online Courses) have made it so that learning can happen from anywhere across the globe. While this is extremely beneficial, videos are not accessible by everyone, due to time constraints and lower bandwidths. Textual step-by-step instructions will serve as a good alternative, being less time-consuming to follow than videos, and also requiring lesser bandwidth. In this work, the authors aim to create a system that extracts and presents a step-by-step tutorial from a tutorial video, which makes it much easier to follow as per the user’s convenience. This can be mainly accomplished by using Text Recognition for academic videos and Action Recognition for exercise videos. Text Recognition is accomplished using the Pytesseract package from Python which performs OCR. Action Recognition is performed with the help of a pre-trained OpenPose model. Additional information is extracted from both exercise and academic videos with the help of speech recognition. The information extracted from the videos are then segmented into procedural instructions which is easy to comprehend. For exercise videos, a frame-wise accuracy of 79.27% is obtained. Additionally, for academic videos, an accuracy of 78.47% is obtained.
Cricket Highlight Generation: Automatic Generation Framework Comprising Score Extraction and Action Recognition

Generation of highlights is the method of pulling the most intriguing clips from a sports video. It is an important activity for sports broadcasters, especially for the sport of cricket. This research proposes a simple yet efficient approach for the automated generation of Cricket highlights from the entirety of a match video by key event detection. The proposed approach comprises of a score extraction technique with the help of the EAST model and OCR (Optical Character Recognition) techniques which helps detect the keyframes, followed by an action recognition pipeline using CNN and LSTM applied on those detected keyframes. The Observed outcomes exemplify that the precision of the proposed method for automated cricket highlights generation is productive and valuable.
Classification-based Multi-task Learning for Efficient Pose Estimation Network

Human pose estimation is an interesting and underlying topic in various fields such as action recognition and human-computer interaction. Although many methods have been developed recently, they are still far from perfect in accuracy and speed at a time. In this paper, we propose a Classification-based Pose Estimation Network with Multi-task Learning (CPENML) based on the low-resolution feature map to improve accuracy and inference time simultaneously. The proposed CPENML consists of two ideas. Firstly, novel proposed keypoint and offset estimation tasks based on classification achieve better performance than regression. Secondly, the proposed Multi-Scale Network (MSN) makes robust feature maps and balances the keypoint and offset tasks to maximize performance. To prove the effectiveness of the proposed method, we conduct ablation studies on the COCO dataset for proposed ideas. Compared to benchmarks, we demonstrate the superiority of our proposed method on COCO dataset in terms of inference time and accuracy.
Design of Intelligent Household Separated-Waste Containers Based on Deep Learning

The separation of domestic waste is of great significance to the establishment of a good living environment and the maximum utilization of resources in cities. Besides, separation and recycling from the family has gradually attracted the attention of relevant government departments and researchers because the family is the source of domestic waste. Thus, in this paper, in view of the problems of the lack of knowledge of domestic waste separation and the lack of source separation, the overall design scheme of intelligent household separated-waste containers was formulated by using the technical characteristics of machine vision and the mechanical structure of household waste containers. An intelligent waste container system with image acquisition module, image processing module, waste separation module and control module was built, and the hardware was optimized and analyzed, and the design system was studied in practice.
Requirements Metamodeling for Self-Adaptive Embedded Systems

Embedded systems (ES) ubiquity has increased in the last two decades; it is seldom to find any electronic device that is not controlled by ES. Additionally, there are a large number of ES which need to modify their behavior at run time in response to changing environmental conditions or in the cases where the requirements themselves need to be changed. Up to now, few researchers are interested in the high-level design process of the self-adaptive embedded systems (SAES) specifically in the field of requirement engineering (RE). While there exit some metamodels on RE for SAES, to the best of the authors' knowledge, there is no comprehensive metamodel that can be used as a reference for the development of RE for SAES. For this reason, the objectives of this paper are twofold, first, to review the literature state of the art and practice of requirements engineering for self-adaptive embedded systems and secondly to propose a complete metamodel (MM4SAES) that defines all concepts and relationships that must be taken into account in the development of SAES