Reviewerly/exHarmony huggingface dataset:
Enabling PIM-based AES encryption for online video streaming

Encryption of streaming video is becoming critical to the success of commercial enterprise and to consumers alike. To meet copyright and privacy requirements, encrypting video data on-the-fly during transmission is necessary. In this work we employ Advanced Encryption Standard (AES) due to its security and flexibility. However, it is compute and memory-intensive. Increasing the number of concurrent streamers only exacerbates the problem. To ameliorate this situation, we propose a Processing-In Memory (PIM) architecture AESPIM to offload AES encryption computation to the memory side. Based on our experimental results, AES performance can be improved up to 42% with compared with the traditional CPU based implementation by significantly reducing data movement and increased memory bandwidth. We further study the data-level and user-level parallelism of the video streaming environment in our base design AESPIM architecture, and propose an advanced design based on user-level parallelism to achieve additional 34% performance improvement. Furthermore, we implement a QoS-aware scheduler to solve its potential workload imbalance bottleneck. Based on evaluation results, performance can be further optimized by up to 6% on average.
Boosting Human Decision-making with AI-Generated Decision Aids

Abstract Human decision-making is plagued by many systematic errors. Many of these errors can be avoided by providing decision aids that guide decision-makers to attend to the important information and integrate it according to a rational decision strategy. Designing such decision aids used to be a tedious manual process. Advances in cognitive science might make it possible to automate this process in the future. We recently introduced machine learning methods for discovering optimal strategies for human decision-making automatically and an automatic method for explaining those strategies to people. Decision aids constructed by this method were able to improve human decision-making. However, following the descriptions generated by this method is very tedious. We hypothesized that this problem can be overcome by conveying the automatically discovered decision strategy as a series of natural language instructions for how to reach a decision. Experiment 1 showed that people do indeed understand such procedural instructions more easily than the decision aids generated by our previous method. Encouraged by this finding, we developed an algorithm for translating the output of our previous method into procedural instructions. We applied the improved method to automatically generate decision aids for a naturalistic planning task (i.e., planning a road trip) and a naturalistic decision task (i.e., choosing a mortgage). Experiment 2 showed that these automatically generated decision aids significantly improved people’s performance in planning a road trip and choosing a mortgage. These findings suggest that AI-powered boosting might have potential for improving human decision-making in the real world.
Parallel temporal feature selection based on improved attention mechanism for dynamic gesture recognition

Abstract Dynamic gesture recognition has become a new type of interaction to meet the needs of daily interaction. It is the most natural, easy to operate, and intuitive, so it has a wide range of applications. The accuracy of gesture recognition depends on the ability to accurately learn the short-term and long-term spatiotemporal features of gestures. Our work is different from improving the performance of a single type of network with convnets-based models and recurrent neural network-based models or serial stacking of two heterogeneous networks, we proposed a fusion architecture that can simultaneously learn short-term and long-term spatiotemporal features of gestures, which combined convnets-based models and recurrent neural network-based models in parallel. At each stage of feature learning, the short-term and long-term spatiotemporal features of gestures are captured simultaneously, and the contribution of two heterogeneous networks to the classification results in spatial and channel axes that can be learned automatically by using the attention mechanism. The sequence and pooling operation of the channel attention module and spatial attention module are compared through experiments. And the proportion of short-term and long-term features of gestures on channel and spatial axes in each stage of feature learning is quantitatively analyzed, and the final model is determined according to the experimental results. The module can be used for end-to-end learning and the proposed method was validated on the EgoGesture, SKIG, and IsoGD datasets and got very competitive performance.
Novel Computer Tomography image enhancement deep neural networks for asphalt mixtures

This paper aims to solve two significant problems observed in Computer Tomography(CT) asphalt mixture images, which are 1) low-quality CT image, and 2) low efficiency, by intelligent word embedding methods. The first model is called Asphalt Mixture CT Image Enhancement Network (AMCTEN), which is featured by ignoring the semantic loss, while the second model, AMCTEN+, is feature by considering semantic loss without requiring semantic information. Two models can be integrated to realize end-to-end real-application image enhancement. Experimental results show that peak signal to noise ratio of AMCTEN and AMCTEN + in the test set increased by 79.0 % and 66.4 %, structural similarity increased by 15.1 % and 13.3 %, and mean square error decreased by 97.5 % and 95.3 %, respectively. Analysis of semantic feature spaces generated by the two models indicate that word embedding can construct an effective semantic feature space, fusing the semantic features with image features can improve the image quality, and the proposed model has superiority in multiple image enhancement tasks.
Deep learning methods for lesion detection on mammography images: a comparative analysis

Automatic lesion segmentation in mammography images assists in the diagnosis of breast cancer, which is the most common type of cancer especially among women. The robust segmentation of mammography images has been considered a backbreaking task due to: i) the low contrast of the lesion boundaries; ii) the extremely variable lesions' sizes and shapes; and iii) some extremely small lesions on the mammogram image. To overcome these drawbacks, Deep Learning methods have been implemented and have shown impressive results when applied to medical image segmentation. This work presents a benchmark for breast lesion segmentation in mammography images, where six state-of-the-art methods were evaluated on 1692 mammograms from a public dataset (CBIS-DDSM), and compared considering the following six metrics: i) Dice coefficient; ii) Jaccard index; iii) accuracy; iv) recall; v) specificity; and vi) precision. The base U-Net, UNETR, DynUNet, SegResNetVAE, RF-Net, MDA-Net architectures were trained with a combination of the cross-entropy and Dice loss functions. Although the networks presented Dice scores superior to 86%, two of them managed to distinguish themselves. In general, the results demonstrate the efficiency of the MDA-Net and DynUnet with Dice scores of 90.25% and 89.67%, and accuracy of 93.48% and 93.03%, respectively. Clinical Relevance--- The presented comparative study allowed to identify the current performance of deep learning strategies on the segmentation of breast lesions.
Dilated Convolution ResNet with Boosting Attention Modules and Combined Loss Functions for LDCT Image Denoising

With the increasing concern regarding the radiation exposure of patients undergoing computed tomography (CT) scans, researchers have been using deep learning techniques to improve the quality of denoised low-dose CT (LDCT) images. In this paper, a cascaded dilated residual network (ResNet) with integrated attention modules, specifically spatial- and channel- attention modules, is proposed. This experiment demonstrated how these attention modules improved the denoised CT image by testing a simple ResNet with and without the modules. Further, an investigation regarding the effectiveness of per-pixel loss, perceptual loss via VGG16-Net, and structural dissimilarity loss functions is also covered through an ablation experiment. By knowing how these loss functions affect the output denoised images, a combination of the these loss function is then proposed which aims to prevent edge over-smoothing, enhance textural details and finally, preserve structural details on the denoised images. Finally, a bench testing was also done by comparing the visual and quantitative results of the proposed model with the state-of-the-art models such as block matching 3D (BM3D), patch-GAN and dilated convolution with edge detection layer (DRL-E-MP) for accuracy.
Global Filter of Fusing Near-Infrared and Visible Images in Frequency Domain for Defogging

Exploiting complementary advantages of different reflection and scattering properties of near-infrared (NIR) images and visible (VIS) images, this letter first proposes a defogging model for single image input and then develops an extended model, a fusion model for NIR and VIS color images, to enhance the visibility of image objects in scattering environments. Our fusion model enhances the extracted details of NIR and VIS images by filtering with our defogging model in the frequency domain that takes into account the energy preservation of these two types of images in addition to the high resolution of the fused results. Finally, based on the initial fusion, we propose a color retention mapping method to keep the fusion results free of color distortion. Experimental results demonstrate that our proposed method not only achieves good defogging effect, but also can effectively combine the complementary NIR and VIS information in image color and visibility.
An Object Point Set Inductive Tracker for Multi-Object Tracking and Segmentation

Multi-object tracking and segmentation (MOTS) is a derivative task of multi-object tracking (MOT). The new setting encourages the learning of more discriminative high-quality embeddings. In this paper, we focus on the problem of exploring the relationship between the segmenter and the tracker, and propose an efficient Object Point set Inductive Tracker (OPITrack) based on it. First, we discover that after a single attention layer, the high-dimensional, key point embedding will show feature averaging. To alleviate this phenomenon, we propose an embedding generalization training strategy for sparse training and dense testing. This strategy allows the network to increase randomness in training and encourages the tracker to learn more discriminative features. In addition, to learn the desired embedding space, we propose a general Trip-hard sample augmentation loss. The loss uses patches that are not distinguishable by the segmenter to join the feature learning and force the embedding network to learn the difference between false positives and true positives. Our method was validated on two MOTS benchmark datasets and achieved promising results. In addition, our OPITrack can achieve better performance for the raw model while costing less video memory (VRAM) at training time.
Practical Covert Wireless Unidirectional Communication in IEEE 802.11 Environment

Covert communications, or covert channels, are commonly exploited to establish a data exfiltration channel from an insider on a trusted network to a malicious receiver outside the network without using normal communication of the network. It is because the malicious receiver is an unauthorized user of the communication network and so he cannot communicate with any entity in the network. In this study, we construct a new covert wireless unidirectional communication mechanism in an IEEE 802.11 environment. Our covert communication is based on a covert timing channel exploiting the beacon interval of a given commercial-like AP. Because the wireless covert channel we proposed can be implemented only with firmware modification to the WLAN MAC protocol, it is very suitable for application in a real public AP environment. In order to dramatically reduce the chance of covert signals being detected by others, a new and simple covert data encoding scheme, called ping-pong covert timing channel (PPCTC), is proposed, and we show that the covertness of the PPCTC is excellent compared to the previous timing-based covert channels. Although this wireless covert communication is unidirectional communication, since PPCTC has recovery characteristics against consecutive 2-bit errors, stable communication is guaranteed. Furthermore, a covert frame structure is presented for providing the confidentiality and integrity of the information transmitted via our covert channel. To the best of our knowledge, this is the first attempt.
High Per Parameter: A Large-Scale Study of Hyperparameter Tuning for Machine Learning Algorithms

Hyperparameters in machine learning (ML) have received a fair amount of attention, and hyperparameter tuning has come to be regarded as an important step in the ML pipeline. However, just how useful is said tuning? While smaller-scale experiments have been previously conducted, herein we carry out a large-scale investigation, specifically one involving 26 ML algorithms, 250 datasets (regression and both binary and multinomial classification), 6 score metrics, and 28,857,600 algorithm runs. Analyzing the results we conclude that for many ML algorithms, we should not expect considerable gains from hyperparameter tuning on average; however, there may be some datasets for which default hyperparameters perform poorly, especially for some algorithms. By defining a single hp_score value, which combines an algorithm’s accumulated statistics, we are able to rank the 26 ML algorithms from those expected to gain the most from hyperparameter tuning to those expected to gain the least. We believe such a study shall serve ML practitioners at large.
A Neuronavigation System Using a Mobile Augmented Reality Solution

Image-guided surgery has shown great utility in neurosurgery, especially in allowing for more accurate surgical planning and navigation. The current gold standard for image-guided neurosurgery is neuronavigation, which provides millimetric accuracy on such tasks. However, these approaches often require a complicated setup and have high cost, hindering their potential in low- and middle-income countries. The aim of this study was to develop and evaluate the performance of a mobile-based augmented reality neuronavigation solution under different conditions in a preclinical environment.The application was developed using the Swift programming language and was tested on a replica of a human scalp under variable lighting, with different numbers of registration points and target point position conditions. For each condition, reference points were input into the application, and the target points were computed for 10 iterations. The mean registration error and target error were used to assess the performance of the application.In the best-case scenario, the proposed solution had a mean target error of 2.6 ± 1.6 mm.Our approach provides a viable, low-cost, easy-to-use, portable method for locating points on the scalp surface with an accuracy of 2.6 ± 1.6 mm in the best-case scenario.
Discriminating seismic events using 1D and 2D CNNs: applications to volcanic and tectonic datasets

Abstract Detecting seismic events, discriminating between different event types, and picking P- and S-wave arrival times are fundamental but laborious tasks in seismology. In response to the ever-increasing volume of seismic observational data, machine learning (ML) methods have been applied to try to resolve these issues. Although it is straightforward to input standard (time-domain) seismic waveforms into ML models, many studies have used time–frequency-domain representations because the frequency components may be effective for discriminating events. However, detailed comparisons of the performances of these two methods are lacking. In this study, we compared the performances of 1D and 2D convolutional neural networks (CNNs) in discriminating events in datasets from two different tectonic settings: tectonic tremor and ordinary earthquakes observed at the Nankai trough, and eruption signals and other volcanic earthquakes at Sakurajima volcano. We found that the 1D and 2D CNNs performed similarly in these applications. Half of the misclassified events were misassigned the same labels in both CNNs, implying that the CNNs learned similar features inherent to the input signals and thus misclassified them similarly. Because the first convolutional layer of a 1D CNN applies a set of finite impulse response (FIR) filters to the input seismograms, these filters are thought to extract signals effective for discriminating events in the first step. Therefore, because our application was the discrimination of signals dominated by low- and high-frequency components, we tested which frequency components were effective for signal discriminations based on the filter responses alone. We found that the FIR filters comprised high-pass and low-pass filters with cut-off frequencies around 7–9 Hz, frequencies at which the magnitude relations of the input signal classes change. This difference in the power of high- and low-frequency components proved essential for correct signal classifications in our dataset. Graphical Abstract
An exploration of mutual information based on emotion–cause pair extraction

Emotion–cause pair extraction task (ECPE) aims to extract the emotions and causes from an unannotated text. The previous works are mostly limited to using deep networks to model the relation between the emotion clause and cause clause and lack exploration of the statistical dependence between them, such as the effects of emotion–cause causality on the mutual information of two clauses. In this paper, we preliminarily explore the difference between emotion–cause pairs and non emotion–cause pairs in their mutual information and further probe the relations among mutual information, emotion–cause pair, and their relative distance. Additionally, we find that mutual information can be used to measure the dependence strength of an emotion–cause causality on the context. Specifically, we formalize the ECPE as a probability problem and derive the joint distribution of the emotion clause and cause clause using the total probability formula. Based on the joint distribution, we estimate the mutual information (MI) between the emotion clause and cause clause, and further quantify the trend of mutual information in the training process. We conduct various experiments, and the experimental results show that our preliminary exploration is practical and effective. Meanwhile, we also prove our conjecture on the emotion–cause causality and mutual information.
Background knowledge in ontology matching: A survey

Ontology matching is an integral part for establishing semantic interoperability. One of the main challenges within the ontology matching operation is semantic heterogeneity, i.e. modeling differences between the two ontologies that are to be integrated. The semantics within most ontologies or schemas are, however, typically incomplete because they are designed within a certain context which is not explicitly modeled. Therefore, external background knowledge plays a major role in the task of (semi-) automated ontology and schema matching. In this survey, we introduce the reader to the general ontology matching problem. We review the background knowledge sources as well as the approaches applied to make use of external knowledge. Our survey covers all ontology matching systems that have been presented within the years 2004–2021 at a well-known ontology matching competition together with systematically selected publications in the research field. We present a classification system for external background knowledge, concept linking strategies, as well as for background knowledge exploitation approaches. We provide extensive examples and classify all ontology matching systems under review in a resource/strategy matrix obtained by coalescing the two classification systems. Lastly, we outline interesting and yet underexplored research directions of applying external knowledge within the ontology matching process.
Privacy-Preserving, Thermal Vision With Human in the Loop Fall Detection Alert System

To support the independent living of older adults in their own homes, it is essential to identify their abnormal behaviors before triggering an automated alert system. Existing normal vision sensing approaches to detect human falls in the activities of daily living (ADL) experienced acceptability issues due to outstanding privacy concerns when they are deployed in personal environments. Besides, false alerts (false-positive) fall detection has not been addressed thoroughly in systems that report abnormal human behaviors as emergency alerts to the information support. This article proposes a novel human-in-the-loop fall detection approach in the ADLs using a low-resolution thermal sensor array. The motivation for enabling a human interactive model, fall detection confirmation, is to influence resource efficiency by reducing false-positive alerts while keeping the false-negative fall predictions as low as possible. The proposed approach is based on the motion sequence classification of human movements using a recurrent neural network. The proposed approach is evaluated with comprehensive experiments using different learning techniques, users, and domestic environment conditions. This article shows a performance accuracy of 99.7% to detect human falls from various typical ADLs.
Uncertainty Modeling for Robust Domain Adaptation Under Noisy Environments

In this paper, we tackle the task of domain adaptation under noisy environments; this is a practical and challenging problem in which the source domain is corrupted with noise in its labels, its features, or both. Noise in the source domain leads to inaccurate visual representations and makes it harder to estimate and reduce the domain discrepancy between the source and target domains, resulting in severe performance degradation in the target domain. These challenges can be addressed with offline source sample selection following robust domain discrepancy reduction. To achieve reliable sample selection, we model the uncertainty in the predictions of a convolutional neural network (CNN) classifier and reweight the classification loss by this uncertainty. Such a reweighting mechanism reduces the contribution of noise, leading to improved noise robustness. We further propose UncertaintyRank, a novel regularizer, to encourage the uncertainty to be more sensitive to noisy labels, as label corruption brings more severe degradation. The uncertainty is also aggregated with the classification loss to eliminate the adverse effects of noisy representations while estimating the domain discrepancy. Extensive experiments validate the effectiveness of our method and verify that it performs favorably against existing state-of-the-art methods.
ENF Based Robust Media Time-Stamping

Electric Network Frequency (ENF) continuously fluctuates around a nominal value (50/60 Hz) due to a persistent imbalance between supplied and demanded power. In certain circumstances, ENF gets intrinsically embedded into audio and video recordings and can be extracted from these recordings. Consequently, ENF can be used in a number of media forensic applications, such as verifying the time of recording of the media. In this work, a robust media time-stamping approach is proposed for media whose ENF content is relatively contaminated. It essentially entails two procedures: first, detecting all useful, i.e., considerably accurate, samples of an estimated ENF signal, and then applying an adapted normalized cross-correlation process that is designed for exploiting just the selected ENF portions based on a binary mask of the identified accurate samples. Experimental results show that the proposed approach provides significantly increased performance.