Reviewerly/exHarmony huggingface dataset:
Ship Detection with Optical Image Based on Attention and Loss Improved YOLO

Object detection is a critical research topic in computer vision. As a subtask of object detection, Ship detection has important research significance. Most ship detection research is based on SAR (Synthetic Aperture Radar) images in the existing research. However, compared with SAR images, optical images have more image feature information, which can assist the algorithm to better learn ship features. In addition, the research of optical image ship detection has more important commercial value. This paper conducts optical image ship detection experiments, applies the YOLO algorithm to the ship detection task, introduces the attention mechanism to transform the residual block in the DarkNet-53 network, and achieves more excellent performance. At the same time, this paper optimizes the recently proposed CIoU loss function which is better than the l <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</inf> −norm loss function and presents the AIoU loss function on this basis. By increasing the area penalty term, the AIoU loss improves performance while converging speed compared with CIoU loss. It is applied to ship detection in the optical image and compared with GIoU, DIoU, and CIoU loss functions, and it has achieved better results than them in the bounding box regression of ship detection.
Fast 3D face reconstruction from a single image combining attention mechanism and graph convolutional network

Abstract In recent years, researchers have made significant contributions to 3D face reconstruction with the rapid development of deep learning. However, learning-based methods often suffer from time and memory consumption. Simply removing network layers hardly solves the problem. In this study, we propose a solution that achieves fast and robust 3D face reconstruction from a single image without the need for accurate 3D data for training. In terms of increasing speed, we use a lightweight network as a facial feature extractor. As a result, our method reduces the reliance on graphics processing units, allowing fast inference on central processing units alone. To maintain robustness, we combine an attention mechanism and a graph convolutional network in parameter regression to concentrate on facial details. We experiment with different combinations of three loss functions to obtain the best results. In comparative experiments, we evaluate the performance of the proposed method and state-of-the-art methods on 3D face reconstruction and sparse face alignment, respectively. Experiments on a variety of datasets validate the effectiveness of our method.
MIntRec: A New Dataset for Multimodal Intent Recognition

Multimodal intent recognition is a significant task for understanding human language in real-world multimodal scenes. Most existing intent recognition methods have limitations in leveraging the multimodal information due to the restrictions of the benchmark datasets with only text information. This paper introduces a novel dataset for multimodal intent recognition (MIntRec) to address this issue. It formulates coarse-grained and fine-grained intent taxonomies based on the data collected from the TV series Superstore. The dataset consists of 2,224 high-quality samples with text, video, and audio modalities and has multimodal annotations among twenty intent categories. Furthermore, we provide annotated bounding boxes of speakers in each video segment and achieve an automatic process for speaker annotation. MIntRec is helpful for researchers to mine relationships between different modalities to enhance the capability of intent recognition. We extract features from each modality and model cross-modal interactions by adapting three powerful multimodal fusion methods to build baselines. Extensive experiments show that employing the non-verbal modalities achieves substantial improvements compared with the text-only modality, demonstrating the effectiveness of using multimodal information for intent recognition. The gap between the best-performing methods and humans indicates the challenge and importance of this task for the community. The full dataset and codes are available for use at https://github.com/thuiar/MIntRec.
Visual Knowledge Graph for Human Action Reasoning in Videos

Action recognition has been traditionally treated as a high-level video classification problem. However, such a manner lacks the detailed and semantic understanding of body movement, which is the critical knowledge to explain and infer complex human actions. To fill this gap, we propose to summarize a novel visual knowledge graph from over 15M detailed human annotations, for describing action as the distinct composition of body parts, part movements and interactive objects in videos. Based on it, we design a generic multi-modal Action Knowledge Understanding (AKU) framework, which can progressively infer human actions from body part movements in the videos, with assistance of visual-driven semantic knowledge mining. Finally, we validate AKU on the recent Kinetics-TPS benchmark, which contains body part parsing annotations for detailed understanding of human action in videos. The results show that, our AKU significantly boosts various video backbones with explainable action knowledge in both supervised and few shot settings, and outperforms the recent knowledge-based action recognition framework, e.g., our AKU achieves 83.9% accuracy on Kinetics-TPS while PaStaNet achieves 63.8% accuracy under the same backbone. The codes and models will be released at https://github.com/mayuelala/AKU.
Language interpretation in travel guidance platform: Text mining and sentiment analysis of TripAdvisor reviews

The opinions and feelings expressed by tourists in their reviews intuitively represent tourists' evaluation of travel destinations with distinct tones and strong emotions. Both consumers and product/service providers need help understanding and navigating the resulting information spaces, which are vast and dynamic. Traditional sentiment analysis is mostly based on statistics, which can analyze the sentiment of a large number of texts. However, it is difficult to classify the overall sentiment of a text, and the context-independent nature limits their representative power in a rich context, hurting performance in Natural Language Processing (NLP) tasks. This work proposes an aspect-based sentiment analysis model by extracting aspect-category and corresponding sentiment polarity from tourists' reviews, based on the Bidirectional Encoder Representation from Transformers (BERT) model. First, we design a text enhancement strategy which utilizes iterative translation across multiple languages, to generate a dataset of 4,000 reviews by extending a dataset of 2,000 online reviews on 1,000 tourist attractions. Then, the enhanced dataset is reorganized into 10 classifications by the Term Frequency-Inverse Document Frequency (TF-IDF) method. Finally, the aspect-based sentiment analysis is performed on the enhanced dataset, and the obtained sentiment polarity classification and prediction of the tourism review data make the expectations and appeals in tourists' language available. The experimental study generates generic and personalized recommendations for users based on the emotions in the language and helps merchants achieve more effective service and product upgrades.
Application of attention-DnCNN for ESPI fringe patterns denoising

Fringe patterns' denoising in electronic speckle pattern interferometry (ESPI) is an important step in phase extraction. In this study, we propose a new denoising method for ESPI fringe patterns based on a convolutional neural network (CNN). The proposed model which combines the attention mechanism and CNN is defined as attention-denoising CNN. In this model, owing to the attention mechanism, more attention will be paid to fringe information, and better filtering results will be achieved. The experimental results show that our proposed method can obtain excellent results, especially with high and large variation density ESPI fringe patterns.
Predicting the data structure prior to extreme events from passive observables using echo state network

Extreme events are defined as events that largely deviate from the nominal state of the system as observed in a time series. Due to the rarity and uncertainty of their occurrence, predicting extreme events has been challenging. In real life, some variables (passive variables) often encode significant information about the occurrence of extreme events manifested in another variable (active variable). For example, observables such as temperature, pressure, etc., act as passive variables in case of extreme precipitation events. These passive variables do not show any large excursion from the nominal condition yet carry the fingerprint of the extreme events. In this study, we propose a reservoir computation-based framework that can predict the preceding structure or pattern in the time evolution of the active variable that leads to an extreme event using information from the passive variable. An appropriate threshold height of events is a prerequisite for detecting extreme events and improving the skill of their prediction. We demonstrate that the magnitude of extreme events and the appearance of a coherent pattern before the arrival of the extreme event in a time series affect the prediction skill. Quantitatively, we confirm this using a metric describing the mean phase difference between the input time signals, which decreases when the magnitude of the extreme event is relatively higher, thereby increasing the predictability skill.
Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey

Abstract Reinforcement learning takes sequential decision-making approaches by learning the policy through trial and error based on interaction with the environment. Combining deep learning and reinforcement learning can empower the agent to learn the interactions and the distribution of rewards from state-action pairs to achieve effective and efficient solutions in more complex and dynamic environments. Deep reinforcement learning (DRL) has demonstrated astonishing performance in surpassing the human-level performance in the game domain and many other simulated environments. This paper introduces the basics of reinforcement learning and reviews various categories of DRL algorithms and DRL models developed for medical image analysis and radiation treatment planning optimization. We will also discuss the current challenges of DRL and approaches proposed to make DRL more generalizable and robust in a real-world environment. DRL algorithms, by fostering the designs of the reward function, agents interactions and environment models, can resolve the challenges from scarce and heterogeneous annotated medical image data, which has been a major obstacle to implementing deep learning models in the clinic. DRL is an active research area with enormous potential to improve deep learning applications in medical imaging and radiation therapy planning.
Defining the Semiclassical Limit of the Quantum Rabi Hamiltonian

The crossover from quantum to semiclassical behavior in the seminal Rabi model of light-matter interaction still, surprisingly, lacks a complete and rigorous understanding. A formalism for deriving the semiclassical model directly from the quantum Hamiltonian is developed here. Working in a displaced Fock-state basis $|\ensuremath{\alpha},n⟩$, the semiclassical limit is obtained by taking $|\ensuremath{\alpha}|\ensuremath{\rightarrow}\ensuremath{\infty}$ and the coupling to zero. This resolves the discrepancy between coherent-state dynamics and semiclassical Rabi oscillations in both standard and ultrastrong coupling and driving regimes. Furthermore, it provides a framework for studying the quantum-to-semiclassical transition, with potential applications in quantum technologies.
In-sensor neural network for high energy efficiency analog-to-information conversion

This work presents an on-chip analog-to-information conversion technique that utilizes analog hyper-dimensional computing based on reservoir-computing paradigm to process electrocardiograph (ECG) signals locally in-sensor and reduce radio frequency transmission by more than three orders-of-magnitude. Instead of transmitting the naturally sparse ECG signal or extracted features, the on-chip analog-to-information converter analyzes the ECG signal through a nonlinear reservoir kernel followed by an artificial neural network, and transmits the prediction results. The proposed technique is demonstrated for detection of sepsis onset and achieves state-of-the-art accuracy and energy efficiency while reducing sensor power by [Formula: see text] with test-chips prototyped in 65 nm CMOS.
RFPose-GAN: Data Augmentation for RFID based 3D Human Pose Tracking

In the age of Artificial Intelligence of Things (AIoT), human pose tracking has attracted increasing interest in many fields. To address the limitations of conventional vision based pose tracking techniques, Radio Frequency (RF) based pose monitoring has been proposed in recent years. However, most of the existing RF-based approaches depend on a vision-aided multi-model learning model, which requires extensive labeled data for supervised training. Collecting such large amounts of training data is time-consuming and costly. In this paper, we address this issue by proposing a Generative Adversarial Network (GAN) based data augmentation method, termed RFPose-GAN, to generate synthesized datasets to assist the training of multi-model neural networks. Our experimental results demonstrate the efficacy of the proposed data augmentation approach on improving the performance of 3D human pose tracking when there is only a limited amount of training data.
Detection and Classification of Drones Through Acoustic Features Using a Spike-Based Reservoir Computer for Low Power Applications

Over the past few decades, the use of drones for applications such as defense, reconnaissance, agriculture, law enforcement, and others has dramatically increased. Although drones are useful for these applications, they can also be utilized to perform malicious activities, thus compromising the safety and integrity of physical infrastructures. To address this issue, various techniques have been developed to detect and identify drones, including radar, visual analysis, and radio-frequency signal processing. Furthermore, deep learning algorithms based on the recognition of acoustic drone features have been proposed to automate the detection process and overcome the current limitations in modern drone detection systems. This paper presents an auditory drone detection and identification system based on the sparse, event-driven communication nature of Spiking Neural Networks (SNNs). We investigate the use of a spiking reservoir computing model, known as a Liquid State Machine (LSM), that offers a computationally light alternative to the deep learning approaches of previous works. The LSM based auditory drone detection and identification system is demonstrated on a publicly available acoustic drone dataset, achieving an accuracy of 97.13% and 93.25% on the detection and identification tasks, respectively. To the best of our knowledge, this work presents the first spike-based implementation of an auditory drone recognition system. Moreover, this paper highlights the potential for low size, weight, and power neuromorphic hardware deployment for drone applications that may be limited to energy-constrained environments.
Loss functions in the prediction of extreme events and chaotic dynamics using machine learning approach

In this paper we investigate the problem of prediction of chaotic dynamics and extreme events using an ensemble of deep neural networks. We test our framework on the artificial data generated using two systems containing high-amplitude outliers and extreme events: the Lienard system and two bursting Hindmarsh-Rose neurons with mutual chemical couplings. We study how the quality of the prediction depends on the type of loss function.
Forecasting on Covid-19 infection waves using a rough set filter driven moving average models

The pandemic outbreak of severe acute respiratory syndrome caused by the Coronavirus 2 disease in 2019, also known as SARS-COV-2 and COVID-19, has claimed over 5.6 million lives till now. The highly infectious nature of the Covid-19 virus has resulted into multiple massive upsurges in counts of new infections termed as 'waves.' These waves consist of numerous rising and falling counts of Covid-19 infection cases with changing dates that confuse analysts and researchers. Due to this confusion, the detection of emergence or drop of Covid waves is currently a subject of intensive research. Hence, we propose an algorithmic framework to forecast the upcoming details of Covid-19 infection waves for a region. The framework consists of a displaced double moving average ( δ DMA) algorithm for forecasting the start, rise, fall, and end of a Covid-19 wave. The forecast is generated by detection of potential dates with specific counts called 'markers.' This detection of markers is guided by decision rules generated through rough set theory. We also propose a novel 'corrected moving average' ( χ SMA) technique to forecast the upcoming count of new infections in a region. We implement our proposed framework on a database of Covid-19 infection specifics fetched from 12 countries, namely: Argentina, Colombia, New Zealand, Australia, Cuba, Jamaica, Belgium, Croatia, Libya, Kenya, Iran, and Myanmar. The database consists of day-wise time series of new and total infection counts from the date of first case till 31st January 2022 in each of the countries mentioned above. The δ DMA algorithm outperforms other baseline techniques in forecasting the rise and fall of Covid-19 waves with a forecast precision of 94.08%. The χ SMA algorithm also surpasses its counterparts in predicting the counts of new Covid-19 infections for the next day with the least mean absolute percentage error (MAPE) of 36.65%. Our proposed framework can be deployed to forecast the upcoming trends and counts of new Covid-19 infection cases under a minimum observation window of 7 days with high accuracy. With no perceptible impact of countermeasures on the pandemic until now, these forecasts will prove supportive to the administration and medical bodies in scaling and allotment of medical infrastructure and healthcare facilities.
On Blockchain: Design Principle, Building Blocks, Core Innovations, and Misconceptions

Blockchain has become one of the hottest research areas in recent years. The technology could potentially lead to a new generation of decentralized applications and decentralized autonomous organizations. Unfortunately, there is simply too much misinformation regarding blockchain. Most notably, blockchain has been used as a buzzword synonymous with data immutability and trust. In fact, this is far from the truth. In this article, we provide a concise description of exactly what blockchain technology is, including its design principle, building blocks, core innovations, and benefits. This is followed by an analysis of data immutability. We show that to create an insurmountable barrier against attacks on data immutability, decentralization and system scale are both necessary. Based on this analysis, we further dissect what benefits private and consortium blockchain could actually offer when decentralization is removed. We show that private and consortium blockchain cannot offer data immutability and trust as many works in the literature have claimed or implied. Instead, the centralized version of blockchain technology provides an elegant solution to achieving fault tolerance and atomic contract execution, which could make private and consortium blockchain useful for enterprises that would like to provide high availability to their customers and for their internal operations.
StrongBox

A wide range of Arm endpoints leverage integrated and discrete GPUs to accelerate computation such as image processing and numerical processing applications. However, in spite of these important use cases, Arm GPU security has yet to be scrutinized by the community. By exploiting vulnerabilities in the kernel, attackers can directly access sensitive data used during GPU computing, such as personally-identifiable image data in computer vision tasks. Existing work has used Trusted Execution Environments (TEEs) to address GPU security concerns on Intel-based platforms, while there are numerous architectural differences that lead to novel technical challenges in deploying TEEs for Arm GPUs. In addition, extant Arm-based GPU defenses are intended for secure machine learning, and lack generality. There is a need for generalizable and efficient Arm-based GPU security mechanisms.
VulCurator: a vulnerability-fixing commit detector

Open-source software (OSS) vulnerability management process is important nowadays, as the number of discovered OSS vulnerabilities is increasing over time. Monitoring vulnerability-fixing commits is a part of the standard process to prevent vulnerability exploitation. Manually detecting vulnerability-fixing commits is, however, time-consuming due to the possibly large number of commits to review. Recently, many techniques have been proposed to automatically detect vulnerability-fixing commits using machine learning. These solutions either: (1) did not use deep learning, or (2) use deep learning on only limited sources of information. This paper proposes VulCurator, a tool that leverages deep learning on richer sources of information, including commit messages, code changes and issue reports for vulnerability-fixing commit classification. Our experimental results show that VulCurator outperforms the state-of-the-art baselines up to 16.1% in terms of F1-score.
Means to valuable exploration: I. The blending of confirmation and exploration and how to resolve it

Data exploration has enormous potential to modify and create hypotheses, models, and theories. Harnessing the potential of transparent exploration replaces the common, flawed purpose of intransparent exploration: to produce results that appear to confirm a claim by hiding steps of an analysis. For transparent exploration to succeed, however, methodological guidance, elaboration and implementation in the publication system is required. We present some basic conceptions to stimulate further development. In this first of two parts, we describe the current blending of confirmatory and exploratory research and propose how to separate the two via severe testing. A claim is confirmed if it passes a test that probably would have failed if the claim was false. Such a severe test makes a risky prediction. It adheres to an evidential norm with a threshold, usually p &lt; α = .05, but other norms are possible, for example, with Bayesian approaches. To this end, adherence requires control against questionable research practices like p-hacking and HARKing. At present, preregistration seems to be the most feasible mode of control. Analyses that do not adhere to a norm or where this cannot be controlled should be considered as exploratory. We propose that exploration serves to modify or create new claims that are likely to pass severe testing with new data. Confirmation and exploration, if sound and transparent, benefit from one another. The second part will provide suggestions for planning and conducting exploration and for implementing more transparent exploratory research.