Reviewerly/exHarmony huggingface dataset:
VR+HD: Video Semantic Reconstruction From Spatio-Temporal Scene Graphs

With the development of computer science and deep learning networks, AI generation technology is becoming increasingly mature. Video has become one of the most important information carriers in our daily life because of their large amount of data and information. However, because of their large amount of information and complex semantics, video generation models, especially High Definition (HD) video, have been a difficult problem in the field of deep learning. Video semantic representation and semantic reconstruction are difficult tasks. Therefore, because video content is changeable and information is highly correlated, we propose a HD video generation model from a spatio-temporal scene graph: the spatio-temporal scene graph to video (StSg2vid) model. First we enter the spatio-temporal scene graph sequence as the semantic representation model of the information in each frame of the video. The scene graph used to describe the semantic information of each frame contains the motion progress of the object in the video at that moment, which is equivalent to a clock. A spatio-temporal scene graph transmits the relationship information between objects through the graph convolutional neural network and predicts the scene layout of the moment. Lastly, the image generation model predicts the frame image of the current moment. The frame at each moment depends on the scene layout at the current moment and the frame and scene layout at the previous moment. We introduced the flow net, wrapping prediction model and the SPADE network to generate images of each frame forecast. We used the Action genome dataset. Compared with the current state-of-the-art algorithms, the videos generated by our model achieve better results in both quantitative indicators and user evaluations. In addition, we also generalized the StSg2vid model into VR videos of indoor scenes, preliminarily explored the generation method of virtual reality (VR) videos, and achieved good results.
TAKDSR: Teacher Assistant Knowledge Distillation Framework for Graphics Image Super-Resolution

This paper presents a framework for effectively applying knowledge distillation (KD) to super-resolution (SR) tasks for computer graphics (CG) images. Specifically, we propose TAKDSR, a KD framework for SR using a teacher assistant (TA) network. Recently, the performance of SR models has improved dramatically thanks to the development of deep learning. SR models have evolved into a form that requires a considerable amount of computation and parameters while adopting a complex neural network structure to improve performance. However, it is difficult to utilize conventional high-performance SR models for real-time up-scaling in CG applications requiring high resolution and high frame rate. To solve this, we employ an approach that applies KD to a lightweight SR model. At this time, if the high-resolution (HR) image is used as input for the teacher to show superior performance to the student, a large performance difference occurs between the two due to the excessive performance of the teacher. As a result, the teacher’s knowledge has a significantly hard and complex nature, and when transferred to the student, the effect of KD can be rather weakened. Therefore, we adopt a TA network to facilitate the propagation of knowledge between teacher and student. At the same time, the distribution of compact features (CF), which are the decoder input of the teacher, is discretized so that it is compatible with the input distribution of the student, enabling effective KD. Experimental results demonstrate the proposed TAKDSR significantly improves the performance of a given SR model on CG image datasets.
Longitudinal Study of Usability and User Experience in Technical and Professional Communication Research

<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Introduction:</b> While usability/user experience (UX) has a long and intertwined history with technical and professional communication (TPC), it is unknown how usability/UX is reflected within TPC research and how that reflection has shifted over time. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Literature review:</b> Prior studies on the role of usability/UX in TPC have found that usability/UX appears infrequently in TPC research and curriculum requirements. However, usability/UX remains a routinely referenced core identity of TPC. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Research questions:</b> 1. To what degree is usability/UX studied in TPC scholarly journals? 2. When TPC researchers study usability/UX, what are they studying? <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methodology:</b> A database of TPC-based usability/UX articles was collected through a defined search method. Articles were coded for primary or secondary emphasis on usability/UX, contribution to TPC, object of analysis, method of data collection, and major takeaway. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</b> Less than 8% of the total publications in the field are tied to usability/UX, though the percentage has increased in the most recent timeframe (2020–2022). Publications are shifting from research that expands usability/UX knowledge to research that uses usability/UX to explain TPC phenomena. In addition, the object of analysis has shifted to process-centric analysis, design thinking has become an increasing component of TPC usability/UX research, and over a quarter of the research on usability/UX provided did not provide enough methodological description to enable replicability. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</b> Although usability/UX has been consistently published in the TPC research journals, the amount of research suggests that usability/UX is not core to TPC's field identity.
Two Novel Noise-Suppression Projection Neural Networks With Fixed-Time Convergence for Variational Inequalities and Applications

This article proposes two novel projection neural networks (PNNs) with fixed-time ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf{FIX_t}$</tex-math> </inline-formula> ) convergence to deal with variational inequality problems (VIPs). The remarkable features of the proposed PNNs are <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf{FIX_t}$</tex-math> </inline-formula> convergence and more accurate upper bounds for arbitrary initial conditions. The robustness of the proposed PNNs under bounded noises is further studied. In addition, the proposed PNNs are applied to deal with absolute value equations (AVEs), noncooperative games, and sparse signal reconstruction problems (SSRPs). The upper bounds of the settling time for the proposed PNNs are tighter than the bounds in the existing neural networks. The effectiveness and advantages of the proposed PNNs are confirmed by numerical examples.
Learned Wavelet Video Coding Using Motion Compensated Temporal Filtering

This paper presents an end-to-end trainable wavelet video coder based on motion-compensated temporal filtering. Thereby, it introduces a different coding scheme for learned video compression, which is dominated by residual and conditional coding approaches. By performing discrete wavelet transforms in temporal, horizontal, and vertical dimensions, an explainable framework with spatial and temporal scalability is obtained. This paper investigates a novel trainable motion-compensated temporal filtering module implemented using the lifting scheme. It demonstrates how multiple temporal decomposition levels can be considered during training. Furthermore, larger temporal displacements owing to the coding order are addressed and an extension adapting to different motion strengths during inference is introduced. The experimental analysis compares the proposed approach to learning-based coders and traditional hybrid video coding. Especially at high rates, the approach exhibits promising rate-distortion performance. The proposed method achieves average Bjontegaard Delta savings of up to 21% over HEVC, and outperforms state-of-the-art learned video coders.
Evaluating the Use of Mixed Reality in CSI Training Through the Integration of the Task-Technology Fit and Technology Acceptance Model

Despite the emerging literature on adopting Mixed Reality headsets in crime scene investigation ¬, it is still debatable on how to employ these headsets and its application for training purposes in higher education and police academies.Hence, this research presents a novel hybrid theoretical framework that combines the Task-technology Fit and Technology Acceptance Model variables and the most prominent features of MR headsets-immersion, interactivity and mobility.The main objective is to explore young investigators' behavioural intention to adopt MR headsets and their applications for investigation training practices.To validate the developed model, a questionnaire survey was the primary method used to collect data from 160 police academy students using the partial least squares-structural equation modelling technique.The empirical results revealed that task technology fit has a positive impact on the perceived usefulness of MR headset applications and no significant positive impact on the perceived ease of use applications of MR devices.On the contrary, individual technology fit has a positive impact on the perceived ease of use and no significant positive effects were found regarding the perceived usefulness of investigation training purposes.Furthermore, the results indicated that the mobility of MR wearable devices positively influences the perceived ease of use and the perceived usefulness for crime scene practices.The study also addresses the theoretical contributions and practical implications of these outcomes.
Masked Autoencoders in Computer Vision: A Comprehensive Survey

Masked autoencoders (MAE) is a deep learning method based on Transformer. Originally used for images, it has now been extended to video, audio, and some other temporal prediction tasks. In the field of computer vision, MAE performs well in classification, prediction, and target detection tasks. In terms of specific application, MAE has made many achievements in medical treatment, geography, 3D point cloud and machine troubleshooting. Since its introduction at the end of 2021, there have been more than 300 related preprints, and MAE has been significantly performed in tier one computer vision conferences during 2022 and 2023. In view of the current popularity of MAE and its future development prospects, we conduct a relatively comprehensive survey of MAE mainly covering officially published articles so far. We comb through and classify the improvements in MAE, demonstrating relatively representative applications in computer vision. Finally, as a summary, we discuss the possible future research directions and development areas based on the characteristics of MAE, hoping our work could be a reference for the future work of MAE.
Federated Learning and Mel-Spectrograms for Physical Violence Detection in Audio

Domestic violence has increased globally as the COVID-19 pandemic combines with economic and social stresses. Some works have used traditional feature extractors to identify features from sound signals to detect physical violence. However, these extractors have not performed well at recognizing physical violence in audio. Besides, the use of Machine Learning is limited by the trade-off between collecting more data while keeping users privacy. Federated Learning (FL) is a technique that allows the creation of client-server networks, in which anonymized training result can be uploaded to a central model, responsible for aggregating and keeping the model up to date, and then distribute the updated model to the client nodes. In this paper, we proposed a FL approach to the violence detection problem in audio signals. The framework was evaluated on a newly proposed synthetic dataset, in which audio signals are represented as mel-spectrograms images, augmented with violence extracts. Thereby, it treats it as a problem of image classification using pre-trained Convolutional Neural Networks (CNN). Inception v3, MobileNet v2, ResNet152 v2 and VGG-16 architectures were evaluated, with the MobileNet architecture presenting the best performance, in terms of accuracy (71.9%), with a loss of 3.6% when compared to the non-FL setting.
Disambiguation of Universal Dependencies Part-of-Speech Tags of Closed Class Words in Portuguese

This paper explores methods to disambiguate Part-of-Speech (PoS) tags for closed class words in Brazilian Portuguese corpora annotated according to the Universal Dependencies annotation model. We evaluate disambiguation methods of different paradigms, namely a Markov-based method, a widely adopted parsing tool, and a BERT-based language modeling method. We compare their performances with two baselines, and observe a significant increase of more than 10% over the baselines for all proposed methods. We also show that while the BERT-based model outperforms the others reaching for the best case a 98% accuracy predicting the correct PoS tag, the use of the three methods as an Ensemble method offers more stable result according to the smaller variance for the numerical results we performed.
Leveraging Sign Language Processing with Formal SignWriting and Deep Learning Architectures

Advances in sign language processing have not adequately kept pace with the tremendous progress that has been made in oral language processing. This fact serves as motivation for conducting research on the potential utilization of deep learning models within the domain of sign language processing. In this paper, we present a method that utilizes deep learning to build a latent and generalizable representation space for signs, leveraging Formal SignWriting notation and the concept of sentence-based representation to effectively address sign language tasks, such as sign classification. Extensive experiments demonstrate the potential of this method, achieving an average accuracy of $$81\%$$ on a subset of 70 signs with only 889 training data and $$69\%$$ on a subset of 338 signs with 3, 871 training data.
Binary Flying Squirrel Optimizer for Feature Selection

Bio-inspired optimization algorithms aim to address the most diverse problems without the need for derivatives, and they are independent of the shape of the search space. The Flying Squirrel Optimizer belongs to the family of bio-inspired algorithms and simulates the movement of flying squirrels from tree to tree in search of food. This paper proposes a binary version of the flying squirrel optimizer for feature selection problems. To elucidate the performance of the proposed algorithm, we employed six other well-known bio-inspired algorithms for comparison purposes in sixteen benchmark datasets widely known in the literature. Furthermore, we employ the binary flying squirrel optimizer in selecting gas concentrations to identify faults in power transformers. The results expressed that Binary Flying Squirrell Optimizer can either find compact feature sets or improve classification effectiveness, corroborating its robustness.
Efficient Density-Based Models for Multiple Machine Learning Solutions over Large Datasets

Unsupervised and semi-supervised machine learning is very advantageous in data-intensive applications. Density-based hierarchical clustering obtains a detailed description of the structures of clusters and outliers in a dataset through density functions. The resulting hierarchy of these algorithms can be derived from a minimal spanning tree whose edges quantify the maximum density required for the connected data to characterize clusters, given a minimum number of objects, MinPts, in a given neighborhood. CORE-SG is a powerful spanning graph capable of deriving multiple hierarchical solutions with different densities with computational performance far superior to its predecessors. However, density-based algorithms use pairwise similarity calculations, which leads such algorithms to an asymptotic complexity of $$O(n^2)$$ for n objects in the dataset, impractical in scenarios with large amounts of data. This article enables hierarchical machine learning models based on density by reducing the computational cost with the help of Data Bubbles, focusing on clustering and outlier detection. It presents a study of the impact of data summarization on the quality of unsupervised models with multiple densities and the gain in computational performance. We provide scalability for several machine learning methods based on these models to handle large volumes of data without a significant loss in the resulting quality, enabling potential new applications like density-based data stream clustering.
Police Report Similarity Search: A Case Study

Several crimes occur daily, and the initial investigation begins with a police report. In cities with high crime rates, it is impractical to expect the police to read and analyze every crime narrative. Some police reports may involve multiple victims or the same crime may be reported more than once. Additionally, police reports may exhibit similarities due to a shared modus operandi. This study addresses the challenge of providing a police report and searching for the most similar report in the database. A similar police report can be either another report with overlapping words or one that shares a similar modus operandi. One potential solution is to represent each police report as a feature vector and compare these vectors using a similarity function. Different methods can be employed to represent the narrative, including embedding vectors and count-based approaches such as TF-IDF. This research explores the use of pre-trained embedding representations at both the word and sentence levels, such as Universal Sentence Encoder, Word2Vec, RoBERTa, Doc2Vec, among others. We determine the most effective representation for capturing semantic and lexical similarities between police reports by comparing different embedding models. Furthermore, we compare the effectiveness of available pre-trained embedding models with a model trained specifically on a corpus of police reports. Another contribution of this work is the development of trained embedding models specifically tailored for the domain of police reports.
Community Detection for Multi-label Classification

Exploring label correlations is one of the main challenges in multi-label classification. The literature shows that prediction performances can be improved when classifiers learn these correlations. On the other hand, some works also argue that the multi-label classification methods cannot explore label correlations. The traditional multi-label local approach uses only information from individual labels, which makes it impractical to find relationships between them. In contrast, the multi-label global approach uses information from all labels simultaneously and may miss more specific relationships that are relevant. To overcome these limitations and verify if improving the prediction performances of multi-label classifiers is possible, we propose using Community Detection Methods to model label correlations and partition the label space into partitions between the local and global ones. These partitions, here named hybrid partitions, are formed of disjoint clusters of correlated labels, which are then used to build multi-label datasets and train multi-label classifiers. Since our proposal can generate several hybrid partitions, we validate all of them and choose the one that is considered the best. We compared our hybrid partitions with the local and global approaches and an approach that generates random partitions. Although our proposal improved the predictive performance of the used classifier in some datasets compared with other partitions, it also showed that, in general, independent of the approach used, the classifier still has difficulties learning several labels and predicting them correctly.
A Tool for Measuring Energy Consumption in Data Stream Mining

Energy consumption reduction is an increasing trend in machine learning given its relevance in socio-ecological importance. Consequently, it is important to quantify how real-time learning algorithms tailored for data streams and edge computing behave in terms of accuracy, processing time, memory usage, and energy consumption. In this work, we bring forward a tool for measuring energy consumption in the Massive Online Analysis (MOA). First, we analyze the energy consumption rates obtained by our tool against a gold-standard hardware solution, thus showing the robustness of our approach. Next, we experimentally analyze classification algorithms under different validation protocols and concept drift and highlight how such classifiers behave under such conditions. Results show that our tools enable the identification of different classifiers' energy consumption. In particular, it allows a better understanding of how energy consumption rates vary in drifting and non-drifting scenarios. Finally, given the insights obtained during experimentation on existing classifiers, we make our tool publicly available to the scientific community so that energy consumption is also accounted for in developing and comparing data stream mining algorithms.
Bayes and Laplace Versus the World: A New Label Attack Approach in Federated Environments Based on Bayesian Neural Networks

Federated Learning (FL) is a decentralized machine learning approach developed to ensure that training data remains on personal devices, preserving data privacy. However, the distributed nature of FL environments makes defense against malicious attacks a challenging task. This work proposes a new attack approach to poisoning labels using Bayesian neural networks in federated environments. The hypothesis is that a label poisoning attack model trained with the marginal likelihood loss can generate a less complex poisoned model, making it difficult to detect attacks. We present experimental results demonstrating the proposed approach's effectiveness in generating poisoned models in federated environments. Additionally, we analyze the performance of various defense mechanisms against different attack proposals, evaluating accuracy, precision, recall, and F1-score. The results show that our proposed attack mechanism is harder to defend when we adopt existing defense mechanisms against label poisoning attacks in FL, showing a difference of 18.48% for accuracy compared to the approach without malicious clients.
A Clustering Validation Index Based on Semantic Description

In clustering problems where the objective is not based on specifically spatial proximity, but rather on feature patterns and the semantic description, traditional internal cluster validation indices might not be appropriate. This article proposes a novel validity index to suggest the most appropriate number of clusters based on a semantic description of categorical databases. To assess our index, we also propose a synthetic data generator specifically designed for this type of application. We tested data sets with different configurations to assess the performance of the proposed index compared to well-known indices in the literature. Thus, we demonstrate that the index has great potential for discovering the number of clusters for the type of application studied and the data generator is able to produce relevant data sets for the internal validation process.