Reviewerly/exHarmony huggingface dataset:
Research on Teaching Reform of Digital Signal Processing Course Based on Python

Digital signal processing (DSP), as a significant and rapidly developing discipline, has become the backbone course of most electronics, computer, communication, and other related majors. However, the computational complexity is large and cumbersome, making it difficult for students to verify firsthand, resulting in a lack of visualized results and a difficult understanding of theory. Therefore, this article attempts to reform DSP course teaching based on Python. In order to improve teaching quality and stimulate students’ interest in learning, we will explain the specific usage of Python and its extension function library in DSP from the perspectives of classroom teaching and experimental aspects. This article has been simulated and tested to obtain the use of Python to calculate and draw graphics, including FFT graphs, Time-domain waveforms and spectra, as well as the design of window functions for low-pass filters.
CPG-LS: Causal Perception Guided Linguistic Steganography

The current lexical substitution-based linguistic steganography primarily determines the substitutions by their linguistic suitability, overlooking the disparity in their capability against steganalysis. This oversight leads to the potential security risk by performing poor substitutions. To address this issue, this letter proposes a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</i> ausal <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</i> erception <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</i> uided <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> inguistic <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</i> teganography(CPG-LS) via elaborate and secure lexical substitutions. CPG-LS constructs a causal perception network by making full use of a trained CNN discriminator to assess the security of each original word in the cover text and its substitutable candidates for controlling the embedding of secret message. Particularly, the causal score of each word is comprehensively measured from two perspectives by the causal perception network, one is word saliency, and the other is anti-steganalysis capability. Finally, the selection of original words is explicitly guided by their causal scores for securely embedding secret message. As the causal score of a word decreases, the perturbation caused by modifying this word becomes smaller, leading to stronger anti-steganalysis capability and lower semantic distortion in the obtained stegotexts. The experimental results demonstrate that CPG-LS achieves better text quality and anti-steganalysis capability than existing similar methods.
Two-Stage Deep Learning Solution for Continuous Arabic Sign Language Recognition Using Word Count Prediction and Motion Images

Recognition of continuous sign language is challenging as the number of words is a sentence and their boundaries are unknown during the recognition stage. This work proposes a two-stage solution in which the number of words in a sign language sentence is predicted in the first stage. The sentence is then temporally segmented accordingly and each segment is represented in a single image using a novel solution that entails summation of frame differences using motion estimation and compensation. This results in a single image representation per sign language word referred to as a motion image. CNN transfer learning is used to convert each of these motion images into a feature vector which is used for either model generation or sign language recognition. As such, two deep learning models are generated; one for predicting the number of words per sentence and the other for recognizing the meaning of the sign language sentences. The proposed solution of predicting the number of words per sentence and thereafter segmenting the sentence into equal segments worked well. This is because each motion image can contain traces of previous or successive words. This byproduct of the proposed solution is advantageous as it puts words into context, thus justifying the excellent sign language recognition rates reported. It is shown that bidirectional LSTM layers result in the most accurate models for both stages. In the experimental results section we use an existing dataset that contains 40 sentences generated from 80 sign language words. The experiments revealed that the proposed solution resulted in a word and sentence recognition rates of 97.3% and 92.6% respectively. The percentage increase over the best results reported in the literature for the same dataset are 1.8% and 9.1% for both word and sentences recognitions respectively.
When, Where, and Which?: Navigating the Intersection of Computer Vision and Generative AI for Strategic Business Integration

In today’s rapidly evolving digital landscape, Artificial Intelligence (AI) exerts a profound influence on our daily lives, from predictive text in emails to the ever-present virtual assistants like Alexa and Siri. This scholarly article embarks on a comprehensive exploration of the expansive world of Artificial Intelligence, with a keen focus on the domains of generative AI and computer vision. Our objective is to provide businesses with a nuanced and in-depth understanding of these critical AI subfields. By doing so, we empower organizations to make informed and strategic decisions regarding the adoption of generative AI and computer vision technologies. Our ultimate goal is to equip businesses with the knowledge and insights necessary to harness the potential of these AI domains effectively, driving innovation and bolstering their competitive edge in an increasingly technology-driven world.
A Novel MAE-Based Self-Supervised Anomaly Detection and Localization Method

Despite significant advancements in self-supervised anomaly detection, multi-class anomaly detection tasks still pose substantial challenges. Most existing methods require individual network training for each category of objects. This paper presents a novel end-to-end approach for multi-class anomaly detection: self-supervised Mask-pretrained Anomaly Localization Autoencoder (MALA). Firstly, the masked autoencoder (MAE) and Pseudo Label Prediction Module (PLPM) are utilized to recover and perceive normal image patterns. Subsequently, the encoder weights are frozen for further end-to-end network training to predict anomalous maps directly. Token Balance Module(TBM) facilitates anomalous perception and improves anomaly segmentation. By utilizing the Visual Transformer and employing image inpainting as a proxy task, remarkable generalization results are achieved. The proposed method demonstrates its applicability across diverse styles of industrial products. Experiments are conducted on MVTech AD, VisA, KolektorSDD2, and MT datasets, achieving state-of-the-art results in multi-task anomaly detection and segmentation tasks. Specifically, we obtain image AUROC of 98.% and pixel AUROC of 97.1% on MVTech AD dataset, pixel AUROC of 97.1% on VisA dataset, and pixel AUROC of 98.7% on KolektorSDD2 dataset.
An Approach to Generate a Caption for an Image Collection Using Scene Graph Generation

Summarization is a challenging task that aims to generate a summary by grasping common information of a given set of information. Text summarization is a popular task of determining the topic or generating a textual summary of documents. In contrast, image summarization aims to find a representative summary of a collection of images. However, current methods are still restricted to generating a visual scene graph, tags, and noun phrases, but cannot generate a fitting textual description of an image collection. Thus, we introduce a novel framework for generating a summarized caption of an image collection. Since scene graph generation shows advancement in describing objects and their relationships on a single image, we use it in the proposed method to generate a scene graph for each image in an image collection. Then, we find common objects and their relationships from all scene graphs and represent them as a summarized scene graph. For this, we merge all scene graphs and select part of it by estimating the most common objects and relationships. Finally, the summarized scene graph is input into a captioning model. In addition, we introduce a technique to generalize specific words in the final caption into common concept words incorporating external knowledge. To evaluate the proposed method, we construct a dataset for this task by extending the annotation of the MS-COCO dataset using an image retrieval method. The evaluation of the proposed method on this dataset showed promising performance compared to text summarization-based methods.
A Viewpoint on Human Factors in Software Supply Chain Security: A Research Agenda

While securing dependencies and build systems is necessary, recent attacks have shown that developers are a commonly successfully attacked link in the chain. Therefore, a comprehensive approach that considers the human factor is crucial for effective software supply chain security.
Detecting Phishing Websites using Deep Learning

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
Building a Full-text Search Index Using "Transformer" Neural Network

The use of deep neural networks in information retrieval significantly improves its effectiveness, but negatively affects the performance of the process. To deal with this, we propose a new ranking model that uses the deep neural network of the "Transformer" architecture (in particular, BERT) for efficient information retrieval. In accordance with the proposed approach, contextualized vector representations are extracted from documents during indexing, after which these representations are clustered for each independent token. The resulting clusters reflect different meanings of the words and are indirectly used as inverted index keys. The values represent the documents in which these contextualized word meanings occur, along with the distances from each document to the contextualized embedding. Thus, after the indexing process, we obtain an index containing pre-calculated distances between the contextualized meanings of dictionary elements and documents. This approach helps us avoid the performance overhead of calculating distances online. At the search stage, the query is transformed into a set of contextualized vectors representing each query token, which allows us to use these vectors to retrieve most semantically close neighbor-tokens and use them to extract relevant documents from the index. This way of searching for contextualized embeddings consumes less memory and is more performant due to the use of an inverted index.
Analysis of the Performance Impact in Industrial Control Process of Wastewater Systems when using Distributed Ledger Technology Sensor Authentication

In recent years, the integration of advanced technologies in industrial control processes has gained significant attention, particularly in the domain of wastewater systems. One emerging technology with promising potential is Distributed Ledger Technology (DLT), which offers secure and transparent data management through blockchain-based solutions. This paper presents an in-depth analysis of the performance impact that arises when incorporating DLT-based sensor authentication in industrial control processes of wastewater systems. The study aims to evaluate the benefits and challenges associated with this integration, providing insights into the effectiveness and efficiency of DLT-based sensor authentication in ensuring data integrity and enhancing the overall control process performance.
Backward Curriculum Reinforcement Learning

Current reinforcement learning algorithms train an agent using forward-generated trajectories, which provide little guidance so that the agent can explore as much as possible. While realizing the value of reinforcement learning results from sufficient exploration, this approach leads to a trade-off in losing sample efficiency, an essential factor impacting algorithm performance. Previous tasks use reward-shaping techniques and network structure modification to increase sample efficiency. However, these methods require many steps to implement. In this work, we propose novel backward curriculum reinforcement learning that begins training the agent using the backward trajectory of the episode instead of the original forward trajectory. This approach provides the agent with a strong reward signal, enabling more sample-efficient learning. Moreover, our method only requires a minor change in the algorithm of reversing the order of the trajectory before agent training, allowing a straightforward application to any state-of-the-art algorithm.
Open-Ended Multi-Modal Relational Reasoning for Video Question Answering

In this paper, we introduce a robotic agent specifically designed to analyze external environments and address participants’ questions. The primary focus of this agent is to assist individuals using language-based interactions within video-based scenes. Our proposed method integrates video recognition technology and natural language processing models within the robotic agent. We investigate the crucial factors affecting human-robot interactions by examining pertinent issues arising between participants and robot agents. Methodologically, our experimental findings reveal a positive relationship between trust and interaction efficiency. Furthermore, our model demonstrates a 2% to 3% performance enhancement in comparison to other benchmark methods.
Effect of Augmented Reality User Interface on Task Performance, Cognitive Load, and Situational Awareness in Human-Robot Collaboration

Augmented Reality (AR) enables the transmission of intent using the physical area in which humans and robots interact as a shared canvas. Studies exploring AR for human-robot collaboration have reported mixed findings on the relationship between cognitive workload and task performance. In this study, we developed an AR user interface (UI) that guides the user to perform a pick-and-place task while collaborating with a robot. A repeated measures mixed-methods study with sixteen participants demonstrated that AR UI significantly impacted task performance, where users traveled longer distances at slower speeds to pick-and-place objects than the control group. Additionally, UI significantly impacted the cognitive load, where participants reported higher NASA-TLX scores while using AR UI. Finally, users reported significantly lower situational awareness and low usability scores while using AR UI. Our findings suggest that the AR UI negatively impacts human-robot collaboration, calling for further investigation.
Real-Time Multimodal Turn-taking Prediction to Enhance Cooperative Dialogue during Human-Agent Interaction

Predicting when it is an artificial agent’s turn to speak/act during human-agent interaction (HAI) poses a significant challenge due to the necessity of real-time processing, context sensitivity, capturing complex human behavior, effectively integrating multiple modalities, and addressing class imbalance. In this paper, we present a novel deep learning network-based approach for predicting turn-taking events in HAI that leverages information from multiple modalities, including text, audio, vision, and context data. Our study demonstrates that incorporating additional modalities, including in-game context data, enables a more comprehensive understanding of interaction dynamics leading to enhanced prediction accuracy for the artificial agent. The efficiency of the model also permits potential real-time applications. We evaluated our proposed model on an imbalanced dataset of both successful and failed turn-taking attempts during an HAI cooperative gameplay scenario, comprising over 125,000 instances, and employed a focal loss function to address class imbalance. Our model outperformed baseline models, such as Early Fusion LSTM (EF-LSTM), Late Fusion LSTM (LF-LSTM), and the state-of-the-art Multimodal Transformer (Mult). Additionally, we conducted an ablation study to investigate the contributions of individual modality components within our model, revealing the significant role of speech content cues. In conclusion, our proposed approach demonstrates considerable potential in predicting turn-taking events within HAI, providing a foundation for future research with physical robots during human-robot interaction (HRI).
Graph-Based Collision Avoidance Algorithm Among Swarm Agents

More than one homogenous or heterogenous type unmanned vehicle can work in a coordinated manner and perform large-scale swarm tasks (firefighting, search and rescue, mapping, and military operations, etc.) efficiently in a shorter time by sharing tasks. Collision of these vehicles is among the most significant problems encountered during their work. The crash of the vehicles causes the vehicles to be out of duty and, accordingly, to the mission’s failure. In this study, Quadrotor-type UAVs used as agents can go to any target point by receiving location, speed, and compass information with GPS and IMU sensors. In the application, the agents’ locations were kept and updated in a list in pairs, similar to the traversing process in the Optimized Bubble Sort Algorithm. The projections of the velocity vectors on the agents’ axis (local) on a single coordinate plane are taken to determine the collision situation between these two agents that are traveling instantaneously. These global velocity vectors, whose projections are taken, are re-projected to the edge formed by these two agents in the graph and then subtracted from each other. Suppose the size of the vector is greater than the distance between two agents obtained by any GPS distance algorithm (Pythagoras, Haversine, etc.). In this case, collision is detected, and the separation process is activated. Separation is the process of advancing one agent in the opposite direction of the other until a minimum safe distance is achieved, where one agent entered by the user will not affect the other. Once the separation is complete, the agent moves to the final destination point.
Hands-Free Interface using Breath for Robot-Assisted Operation

In this paper, we propose a novel hands-free interface for robot-assisted operation. Surgeons use both hands during surgery, making difficult for them to operate the display. However, by using a modality other than the hands to operate the display, the surgeon can check the necessary information for the operation in real time. Therefore, we developed a system that recognizes breath from the temperature change on the mask surface. Then, we compared the proposed system with a directly recognized breath method. Consequently, the proposed method had the same performance as directly recognized breath method and minimized the risk of contamination. We believe that the application of our developed system will contribute to the improvement of safety and efficiency in the medical field.
Intuitive Arm-Pointing based Home-Appliance Control from Multiple Camera Views

The purpose of this paper is to construct and evaluate a system to operate home appliances by pointing. In Human Machine Interface (HMI) design, a natural operating method is important. Pointing is a universal gesture for selecting an object. Arm-pointing to an appliance and selecting it to perform a simple operation is a very intuitive and easy-to-use method of operation. Many studies prepare data with locations of appliances and their sizes. In this paper, we a camera-based system where the user can simply point at an appliance to select and operate it is proposed. The user’s pointing direction and appliance locations are estimated automatically from image frames. This eliminates the need for any preparation beforehand and the appliances can be moved during operation. The proposed method was implemented and experimentally evaluated. It was found that the average recognition rates were about 87% and 57% when a humidifier and a TV were operated.
Recognizing diver hand gestures for human to robot communication underwater

The underwater environment provides a range of interesting applications for human-robot teams. A critical issue for such teams is the development of an appropriate communication mechanism between humans and robots operating at depth. Humans operating at depth have developed an applied gesture-based communication language that can be leveraged to enable this communication, but it would be expensive and perhaps impractical to develop a hand-labelled dataset of these gestures to support a machine learning-based approach to the task. To avoid the cost of hand labelling such a large dataset, here we automate the process of collecting a labelled dataset through the use of a simple model trained on a hand-labelled dataset that only identifies salient objects (divers, their heads and hands), and then use a weakly supervised learning process to label a complex set of diver gestures. The result of this process is a system that can recognize a large number of diver hand gestures. Performance of the resulting system is compared against a hand-labelled set of diver gestures.
“The wallpaper is ugly”: Indoor Localization using Vision and Language

We study the task of locating a user in a mapped indoor environment using natural language queries and images from the environment. Building on recent pretrained vision-language models, we learn a similarity score between text descriptions and images of locations in the environment. This score allows us to identify locations that best match the language query, estimating the user’s location. Our approach is capable of localizing on environments, text, and images that were not seen during training. One model, finetuned CLIP, outperformed humans in our evaluation.
Predicting the Impressions of Interaction with a Robot from Physical Actions Using AICO-Corpus Annotations

In many cases of human-human communication, humans interact with others while assuming their emotions and impressions based on not only verbal information but also non-verbal information. Similarly, during the human-robot interaction, predicting the impressions that a person has of the robot is important for the robot to change the behavior and realize good interaction. In this work, we try to use gaze and gesture annotation data in human-robot interaction from AICO-Corpus and show LSTM approach has the potential for the prediction about impressions of interaction with a robot. We also analyzed the types of nonverbal information that influence the impressions towards the robot in English and Japanese respectively.
Learning Clear Class Separation for Open-set 3D Detector in Autonomous Vehicle via Selective Forgetting

A trustworthy 3D detector is essential in the perception system of autonomous vehicles, ensuring accurate detection of their surroundings. However, autonomous vehicles have to operate in ever-changing real-world driving scenes, where unknown objects that do not belong to the training set are commonly encountered. Confusion about known and unknown objects could result in severe and dangerous consequences for road safety. To address this problem, we improve the reliability of autonomous driving systems by formulating open-set 3D object detection task. An Open-set 3D Detector (Open3Det) is proposed to reject unknown instances while maintaining performance on known categories. Distinct from 2D objects, clear space separation exists between each 3D instance. Motivated by this, we propose selective forgetting, a novel method capable of filtering out misleading predictions. Given a close-set teacher model, knowledge distillation is introduced to build a open-set student model. The student model preserves its predictions for known objects, whereas predictions of backgrounds and unknown instances are discarded to minimize misleading results. Extensive experiments and visualizations reveal the efficacy of the proposed method.