Reviewerly/exHarmony huggingface dataset:
Robust Secret Data Hiding for Transformer-based Neural Machine Translation

Hiding secret information in text is a research area of significant importance and a great challenge. In recent years, there have been huge developments and exciting advances in generation-based text information hiding techniques. Current generative text information hiding methods mainly establish correspondence between token and secret bits based on probability distributions given by language models. However, the semantic control of such methods is weak, and their robustness is not discussed. In this paper, we investigate an end-to-end generation-based text information hiding scheme. The proposed method uses a sequence-to-sequence model with adversarial training as a machine translation model. It converts the secret information into an embedding vector to be added to each position of the hidden state representation of the source language text, which in turn allows the model to automatically learn to produce translation results with the embedded secret information without using fixed rules. The semantics of the text with embedded secret messages obtained by translation can be controlled by the meaning of the source language text. Our experiments show that the proposed method can embed the secret message into the translation results with little loss of the translation quality and is robust to active attacks such as word deletion or synonym substitution.
Design and Implementation of a Three-Dimensional CAD Graphics Support Platform for Pumps Based on Open CASCADE

In the pump industry, designers commonly utilize mainstream three-dimensional computer-aided design (CAD) software (Unigraphics NX 12.0 and SolidWorks 2023). However, these CAD packages are generic and not optimized for the specific requirements of the pump industry. This leads to a lack of flexibility and increased complexity in their usage, as well as higher computational demands, resulting in elevated learning and operational costs. Additionally, there are concerns about potential information leaks and software restrictions. In this paper, we studied the organization architecture of commercial three-dimensional CAD software, and compared and analyzed the geometric kernels and rendering engines of mainstream three-dimensional software. Using the Open CASCADE geometric kernel and OpenSceneGraph rendering engine, together with the Visual Studio 2021 development environment and Qt interface library, we developed an autonomous copyright three-dimensional CAD graphics support platform for pumps. Based on the three-dimensional platform, we tested the commonly used graphics elements and basic algorithms required for pump modeling, and successfully designed and modeled the impeller and volute casing of a centrifugal pump. Through computational simulations and experimental verifications, we demonstrated that the accuracy and precision of the pump model designed on this platform meets the design requirements, indicating that this platform has practical pump design and modeling capabilities. Compared to commercial three-dimensional CAD software, this platform exhibits superior flexibility and interactivity in three-dimensional modeling that is specifically tailored for pump products. Consequently, it fully satisfies the needs for three-dimensional parameterized modeling and visualization of pumps.
How Object Information Improves Skeleton-based Human Action Recognition in Assembly Tasks

As the use of collaborative robots (cobots) in industrial manufacturing continues to grow, human action recognition for effective human-robot collaboration becomes increasingly important. This ability is crucial for cobots to act autonomously and assist in assembly tasks. Recently, skeleton-based approaches are often used as they tend to generalize better to different people and environments. However, when processing skeletons alone, information about the objects a human interacts with is lost. Therefore, we present a novel approach of integrating object information into skeleton-based action recognition. We enhance two state-of-the-art methods by treating object centers as further skeleton joints. Our experiments on the assembly dataset IKEA ASM show that our approach improves the performance of these state-of-the-art methods to a large extent when combining skeleton joints with objects predicted by a state-of-the-art instance segmentation model. Our research sheds light on the benefits of combining skeleton joints with object information for human action recognition in assembly tasks. We analyze the effect of the object detector on the combination for action classification and discuss the important factors that must be taken into account.
Attention map feature fusion network for Zero-Shot Sketch-based Image Retrieval

Zero-shot sketch-based image retrieval (ZS-SBIR) is a great and important computer vision problem. The image category in the test phase is a new category that was not visible in the training stage. Because sketches are extremely abstract, the commonly used backbone networks (such as VGG-16 and ResNet-50) cannot handle both sketches and photos. To solve this problem, we propose a novel and effective feature embedding model called Attention Map Feature Fusion (AMFF). The AMFF model combines the excellent feature extraction capability of the ResNet-50 network with the excellent representation capability of the attention network. By processing the residuals of the Res Net-50 network, the attention map is finally obtained without intro-ducing external semantic knowledge. Most previous approaches treat the ZS-SBIR problem as a classification problem, which ignores the huge domain gap between sketches and photos. This paper proposes an effective method to optimize the entire network, called domain-aware triplets (DAT). Domain feature discrimination and semantic feature embedding can be learned through DAT. In this paper, we also use the classification loss function, which is used to stabilize the training process to avoid getting trapped in a local optimum. Our code and related datasets are publicly available at https://github.com/haizhu12/ammln.git.
Just a Matter of Scale? Reevaluating Scale Equivariance in Convolutional Neural Networks

The widespread success of convolutional neural networks may largely be attributed to their intrinsic property of translation equivariance. However, convolutions are not equivariant to variations in scale and fail to generalize to objects of different sizes. Despite recent advances in this field, it remains unclear how well current methods generalize to unobserved scales on real-world data and to what extent scale equivariance plays a role. To address this, we propose the novel Scaled and Translated Image Recognition (STIR) benchmark based on four different domains. Additionally, we introduce a new family of models that applies many re-scaled kernels with shared weights in parallel and then selects the most appropriate one. Our experimental results on STIR show that both the existing and proposed approaches can improve generalization across scales compared to standard convolutions. We also demonstrate that our family of models is able to generalize well towards larger scales and improve scale equivariance. Moreover, due to their unique design we can validate that kernel selection is consistent with input scale. Even so, none of the evaluated models maintain their performance for large differences in scale, demonstrating that a general understanding of how scale equivariance can improve generalization and robustness is still lacking.
SwinVI:3D Swin Transformer Model with U-net for Video Inpainting

The goal of video inpainting is to fill in the local missingness of a given video as realistic as possible, it remains a challenging task, even with powerful deep learning methods. In recent years, Transformer has been introduced to video inpainting, and remarkable improvement has been achieved. However, it still suffers from the problems of generating blurry texture and requiring high computational cost. To address the two problems, we propose a new 3D Swin Transformer model (SwinVI) with U-net to improve the quality of video inpainting efficiently. We modify the vanilla Swin Transformer by extending the standard self-attention mechanism to a 3D self-attention mechanism, which enables the modified model to process spatio-temporal information simultaneously. SwinVI consists of U-net implemented by 3D Patch Merge and CNN-equipped upsampling module, which provides an end-to-end learning framework. This structural design empowers Swin VI to fully focus on background textures and moving objects to learn robust and more representative token vectors. Accordingly, to significantly improve the quality of video inpainting efficiently. We experimentally compare Swin VI with multiple methods on two challenging benchmarks. Experimental results demonstrate that the proposed Swin VI outperforms the state-of-the-art methods in RMSE, SSIM, and PSNR.
Leaky-Integrate-and-Fire Neuron-Like Long-Short-Term-Memory Units as Model System in Computational Biology

Biological neural networks encode information very efficiently, and dynamically react to sensory input on very small time scales. In contrast to most contemporary machine learning approaches which rely on rate neurons with continuous output, biological neural networks are based on spiking neurons with quasi-binary discrete output. In Artificial Intelligence (AI) Research, time series data are efficiently encoded in Long-Short- Term-Memory (LSTM) networks. Despite their strength in encoding time series data and making predictions, LSTM units are assumed to be biologically implausible. Nevertheless, recent studies show that LSTM unit networks indeed behave similar to biological neural networks. In this study, we show that a particular choice of parameters for the weights and gates in peephole LSTM units causes these units to show similar dynamic behaviour as biologically plausible leaky integrate-and-fire (LIF) neurons, which represent a simple biologically inspired spiking neuron model. We analyzed the spiking characteristics of the restricted peephole LSTM units and characterize the parameter space, in which these units show certain spiking characteristics. We conclude that tackling complex cognitive tasks with biologically plausible and explainable artificial neural networks is an important step to make progress in both fields, neuroscience and artificial intelligence.
CCH-YOLOX: Improved YOLOX for Challenging Vehicle Detection from UAV Images

In this paper, we focus on vehicle detection algorithms from UAV images in intelligent transportation applications. We propose an improved YOLOX model called CCHYOLOX for the problems of dense distribution and drastic scale changes of vehicle targets from the UAV viewpoint. Firstly, we construct a novel plug-and-play module, called Correlation Extraction and Feature Fusion (CEFF), to process the multi-layer adaptive fusion of pyramidal features. It enables the adaptive fusion of features in adjacent layers by adding spatial-awareness and extracting global channels correlation of adjacent layers features to mitigate the effect of target size diversity. Then, we design a special cascade strategy, including a feature alignment module named DConv, for single-stage and anchor-free detectors by considering the feature offset problem to produce more accurate detection bounding boxes. The cascade strategy allows the model to improve performance with little increase in computational cost. Finally, a high-resolution branch is designed for the small-target detection task, which greatly improves the detection accuracy of the model. Experiments on the challenging Visdrone-vehicle and Drone Vehicle datasets show that the proposed method effectively tackle the detection accuracy decrease caused by the above problems. CCH-YOLOX achieves 43.2% mAP and 57.3% mAP on the above two datasets, respectively, which is about 3%-4% higher than the strong baseline model (YOLOX) and exceeds many current popular models. The code is available at https://github.com/lz06787/CCH-YOLOX
Relaxed Attention for Transformer Models

The powerful modeling capabilities of all-attention-based transformer architectures often cause overfitting and-for natural language processing tasks-lead to an implicitly learned internal language model in the autoregressive transformer decoder complicating the integration of external language models. In this paper, we explore relaxed attention, a simple and easy-to-implement smoothing of the attention weights, yielding a two-fold improvement to the general transformer architecture: First, relaxed attention provides regularization when applied to the self-attention layers in the encoder. Second, we show that it naturally supports the integration of an external language model as it suppresses the implicitly learned internal language model by relaxing the cross attention in the decoder. We demonstrate the benefit of relaxed attention across several tasks from different applications with clear improvement in combination with recent benchmark approaches using various transformer model variants and sizes. Specifically, we exceed the former state-of-the-art performance of 26.90% word error rate on the largest public lip-reading LRS3 benchmark with a word error rate of 26.31%, as well as we achieve a top-performing BLEU score of 37.67 on the IWSLT14 (DE → EN) machine translation task without external language models and virtually no additional model parameters.
Online Trajectory Anomaly Detection Based on Intention Orientation

Trajectory anomaly detection has become increasingly important in many fields. However, most existing methods do not support online detection, and have limited performance in multi-source/multi-destination environments. To address these challenges, we propose an unsupervised method named intention orientation based trajectory anomaly detection (IO-TAD) which detects anomalies by inferring their underlying intentions in an online manner. IO-TAD is also robust in multi-source/multi-destination environments. To achieve this, we leverage Inverse Reinforcement Learning and recover the reward functions as a representation of the agents' intentions. Then we use the inferred value functions to quantify the agents' intention orientation pattern. The intuition behind our method is that different normal agents take different routes, or have different intentions, but the ways they achieve their intentions are similar. In other words, the values of normal trajectories tend to be monotonically increasing, while anomalous behaviour is non-monotonic. Our experiments show IO-TAD can achieve good online anomaly detection performance with up to 18.1% improvement in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$F_{1}$</tex> score over the comparable state-of-the-art methods.
Neural Network-based Fast Liver Ultrasound Image Segmentation

Ultrasound is quite popular among the clinicians’ fraternity, because the machine is cheap, easy to use, and mobile. However, the image is not that easy to study properly, if the examiner does not have adequate expertise. Image segmentation, being the first step to ease the study, is considered crucial. In this paper, we have presented a neural network-based image segmentation that is based on Pyramid Scene Parsing. Additionally, we have studied the importance of removing noise before the actual segmentation. We have tested the method on the data obtained from Hamad Medical Corporation and found Dice coefficient of 0.913 ± 0.024 while delivering a real-time performance of 37 frames per second.
Easier Notation – a Proposal for a Gloss-Based Scripting Language for Sign Language Generation Based on Lexical Data

We introduce EASIER Notation, a gloss-based scripting language to describe sign language content to be signed by an avatar and describe the functionality a lexical database for a sign language needs to provide in order to fully support the notation approach. In addition, we present the prototype of a text editor supporting EASIER Notation for human post-editing of machine translation output as well as pre-scribing signed utterances from scratch.
Investigation Of The Quality Of Pseudo-Labels For The Self-Supervised Speaker Verification Task

Optimizing a speaker embedding network in a discriminative fashion using clustering algorithm-driven pseudo-labels is one of the most widely used self-supervised speaker verification system training schemes. Although this kind self-supervised supervised training scheme showed impressive performance, recent studies have shown that label noise can significantly impact the performance. In this contribution, we explore various clustering algorithms to generate speaker pseudo-labels and conduct a fine-grained analysis on the relationship between the quality of the pseudo-labels and the speaker verification performance. Through experimental results, we also shed light on several previously unexplored and overlooked aspects of the pseudo-labels that can have an impact on the speaker verification performance. Furthermore, we observe that the performance of a self-supervised speaker verification system relies heavily on multiple qualitative aspects of the clustering algorithm used to generate the pseudo-labels. Additionally, we show that severe reduction in speaker verification performance can occur from overfitting to the noisy pseudo-labels and that the mixup data augmentation strategy can mitigate the memorization effects of label noise.
DualSlide: Global-to-Local Sketching Interface for Slide Content and Layout Design

Online learning and academic conferences have become pervasive and essential for education and professional development, especially since the onset of pandemics. Academic presentations usually require well-designed slides that are easily understood. Sketches that visually represent design intentions and are readily accessible to the average users. To assist non-expert users in creating visually appealing academic slides, we propose DualSlide, a global and local two-stage sketching interface system that provides image retrieval and user guidance. At the global stage, DualSlide provides a heat map canvas to display the distribution of all slide layouts in a dataset, allowing users to explore the reference slides efficiently. At the local stage of the system, detailed references and guidance for designing slide content, such as diagrams and fonts, can be provided. We further propose a sketch-matching algorithm to compare the user’s input sketch and similar diagrams. All user guidance can be adapted in real-time editing, and users can design slides with a high degree of freedom. We conducted a user study to verify the effectiveness and usability of the proposed DualSlide system confirming that DualSlide provides high retrieval accuracy and satisfactory design results with a good user experience.
Feature Decoupling in Self-supervised Representation Learning for Open Set Recognition

Assuming unknown classes could be present during classification, the open set recognition (OSR) task aims to classify an instance into a known class or reject it as unknown. In this paper, we use a two-stage training strategy for OSR problems. In the first stage, we introduce a self-supervised feature decoupling method that finds the content features of the input samples from the known classes. Specifically, our feature decoupling approach learns a representation that can be split into content features and transformation features. In the second stage, we fine-tune the content features with the class labels. The fine-tuned content features are then used for the OSR problems. To measure representation quality, we introduce intra-inter ratio (IIR). Our experimental results indicate that our proposed self-supervised approach outperforms others in image and malware OSR problems. Also, our analyses indicate that IIR is correlated with and can explain OSR performance.
Deep Reinforcement Learning with Importance Weighted A3C for QoE enhancement in Video Delivery Services

Adaptive bitrate (ABR) algorithms are used to adapt the video bitrate based on the network conditions to improve the overall video quality of experience (QoE). Recently, reinforcement learning (RL) and asynchronous advantage actor-critic (A3C) methods have been used to generate adaptive bit rate algorithms and they have been shown to improve the overall QoE as compared to fixed rule ABR algorithms. However, a common issue in the A3C methods is the lag between behaviour policy and target policy. As a result, the behaviour and the target policies are no longer synchronized which results in suboptimal updates. In this work, we present ALISA: An Actor-Learner Architecture with Importance Sampling for efficient learning in ABR algorithms. ALISA incorporates importance sampling weights to give more weightage to relevant experience to address the lag issues with the existing A3C methods. We present the design and implementation of ALISA, and compare its performance to state-of-the-art video rate adaptation algorithms including vanilla A3C implemented in the Pensieve framework and other fixed-rule schedulers like BB, BOLA, and RB. Our results show that ALISA improves average QoE by up to 25%-48% higher average QoE than Pensieve, and even more when compared to fixed-rule schedulers.
Commit Classification Into Software Maintenance Activities: A Systematic Literature Review

Commits represent an essential part of software development practices, serving as the means for collaboration and management of software changes made to a software project’s codebase. Changes are necessary for the survival of software, for instance, to fix faults, address security vulnerabilities, meet new functional requirements, and improve software performance. To aid software research and practice, several research endeavors attempted to automate the process of identifying the nature of the maintenance tasks performed in a commit based on the data associated with the commit. This paper presents a systematic literature review of supervised-learning-based models for commit classification into maintenance activities. Through the study selection process, 19 primary studies were identified, published between 2008 and 2023 in various journals and conference proceedings. For the independent variables for classification, features extracted from commit messages are prevalent, followed by features extracted from source code and commit metadata. In the majority of existing studies, multi-class classification approaches are used, whereas multi-label approaches are considered rarely. The most commonly used classification algorithms include Decision Tree, Random Forest, Naive Bayes, and Neural Network. Several datasets exist, consisting mainly of commits from different open-source projects based on the Java programming language. Some research gaps and open challenges were identified that can guide future research efforts.