Reviewerly/exHarmony huggingface dataset:
Smart Digital Forensic Readiness Model for Shadow IoT Devices

Internet of Things (IoT) is the network of physical objects for communication and data sharing. However, these devices can become shadow IoT devices when they connect to an existing network without the knowledge of the organization’s Information Technology team. More often than not, when shadow devices connect to a network, their inherent vulnerabilities are easily exploited by an adversary and all traces are removed after the attack or criminal activity. Hence, shadow connections pose a challenge for both security and forensic investigations. In this respect, a forensic readiness model for shadow device-inclusive networks is sorely needed for the purposes of forensic evidence gathering and preparedness, should a security or privacy breach occur. However, the hidden nature of shadow IoT devices does not facilitate the effective adoption of the most conventional digital and IoT forensic methods for capturing and preserving potential forensic evidence that might emanate from shadow devices in a network. Therefore, this paper aims to develop a conceptual model for smart digital forensic readiness of organizations with shadow IoT devices. This model will serve as a prototype for IoT device identification, IoT device monitoring, as well as digital potential evidence capturing and preservation for forensic readiness.
Data Twinning

In this work, we develop a method named Twinning, for partitioning a dataset into statistically similar twin sets. Twinning is based on SPlit, a recently proposed model-independent method for optimally splitting a dataset into training and testing sets. Twinning is orders of magnitude faster than the SPlit algorithm, which makes it applicable to Big Data problems such as data compression. Twinning can also be used for generating multiple splits of a given dataset to aid divide-and-conquer procedures and $k$-fold cross validation.
Semiquantum private comparison of size relationship based on &amp;lt;italic&amp;gt;d&amp;lt;/italic&amp;gt;-level single-particle states

In this paper, we propose a novel semiquantum private comparison (SQPC) protocol of size relationship based on d-level single-particle states. The designed protocol can compare the size relationship of different privacy messages from two classical users with the help of a semi-honest third party (TP), who is permitted to misbehave on her own but cannot be in collusion with anyone else. The correctness analysis shows that this protocol can gain correct comparison results. The security analysis turns out that this protocol can resist famous outside attacks and participant attacks. Moreover, this protocol can guarantee that TP does not know the accurate comparison results. Compared with the only existing SQPC protocol of size relationship (Quantum Inf. Process. 20:124 (2021)), this protocol takes advantage over it on the aspects of initial quantum resource, TP's measurement operations and TP's knowledge about the comparison results.
Multi-modal Video Retrieval in Virtual Reality with vitrivr-VR

In multimedia search, appropriate user interfaces (UIs) are essential to enable effective specification of the user’s information needs and the user-friendly presentation of search results. vitrivr-VR addresses these challenges and provides a novel Virtual Reality-based UI on top of the multimedia retrieval system vitrivr. In this paper we present the version of vitrivr-VR participating in the Video Browser Showdown (VBS) 2022. We describe our visual-text co-embedding feature and new query interfaces, namely text entry, pose queries and temporal queries.
Boost your email marketing campaign! Emojis as visual stimuli to influence customer engagement

Purpose In an effort to evaluate if and how emojis might boost customer engagement in email marketing, the current research aims to analyzes emojis' effects and investigates how certain features, such as the type of emoji or the way of emojis' presentation (repetition and position), moderate their effectiveness. Design/methodology/approach This research comprises four experimental studies. The first two analyze whether the presence and the type of emoji (facial vs non-facial) influence customer engagement. Then, two more studies analyze the influence of the presence and type of emoji, as well as repetitions (repetition vs no repetitions) and positions (left vs right). The data collection is through email marketing campaigns, with two types of brands. The analysis is a fractional factorial design in one- and two-way interactions to identify the influence of presentation features. Findings Emojis influence customer engagement and can trigger interactive behaviors and increase customer engagement, even in a channel that generally is less prone to interaction. The emojis increase the probability of opening the email, the number of times email gets opened and clicks to access additional content. Research limitations/implications The varied results suggest the need for more research to confirm the findings regarding presentation features. This research offers novel implications for interactive marketing and the theory of visual rhetoric because the research establishes how visual stimuli such as emojis can persuade and capture customer attention and increase customer engagement. Practical implications Practitioners can optimize email marketing by using emojis strategically to attract audience interest, provoke interactions, build relationships and generate word of mouth, leading to better consumer experiences, loyalty and brand profitability. Originality/value Most studies use semantic analysis to analyze emojis; the current study instead tests whether emojis can increase customer engagement in a less-interactive channel and then also considers the moderating role of several emoji-relevant variables. In addition, the authors apply the theory of visual rhetoric to explicate the original findings, which are gathered from a unique data set involving real marketing campaigns.
Detecting Deepfake Voice Using Explainable Deep Learning Techniques

Fake media, generated by methods such as deepfakes, have become indistinguishable from real media, but their detection has not improved at the same pace. Furthermore, the absence of interpretability on deepfake detection models makes their reliability questionable. In this paper, we present a human perception level of interpretability for deepfake audio detection. Based on their characteristics, we implement several explainable artificial intelligence (XAI) methods used for image classification on an audio-related task. In addition, by examining the human cognitive process of XAI on image classification, we suggest the use of a corresponding data format for providing interpretability. Using this novel concept, a fresh interpretation using attribution scores can be provided.
CrossData: Leveraging Text-Data Connections for Authoring Data Documents

Data documents play a central role in recording, presenting, and disseminating data. Despite the proliferation of applications and systems designed to support the analysis, visualization, and communication of data, writing data documents remains a laborious process, requiring a constant back-and-forth between data processing and writing tools. Interviews with eight professionals revealed that their workflows contained numerous tedious, repetitive, and error-prone operations. The key issue that we identified is the lack of persistent connection between text and data. Thus, we developed CrossData, a prototype that treats text-data connections as persistent, interactive, first-class objects. By automatically identifying, establishing, and leveraging text-data connections, CrossData enables rich interactions to assist in the authoring of data documents. An expert evaluation with eight users demonstrated the usefulness of CrossData, showing that it not only reduced the manual effort in writing data documents but also opened new possibilities to bridge the gap between data exploration and writing.
Automatic detection of cognitive impairment in elderly people using an entertainment chatbot with Natural Language Processing capabilities

Abstract Previous researchers have proposed intelligent systems for therapeutic monitoring of cognitive impairments. However, most existing practical approaches for this purpose are based on manual tests. This raises issues such as excessive caretaking effort and the white-coat effect. To avoid these issues, we present an intelligent conversational system for entertaining elderly people with news of their interest that monitors cognitive impairment transparently. Automatic chatbot dialogue stages allow assessing content description skills and detecting cognitive impairment with Machine Learning algorithms. We create these dialogue flows automatically from updated news items using Natural Language Generation techniques. The system also infers the gold standard of the answers to the questions, so it can assess cognitive capabilities automatically by comparing these answers with the user responses. It employs a similarity metric with values in [0, 1], in increasing level of similarity. To evaluate the performance and usability of our approach, we have conducted field tests with a test group of 30 elderly people in the earliest stages of dementia, under the supervision of gerontologists. In the experiments, we have analysed the effect of stress and concentration in these users. Those without cognitive impairment performed up to five times better. In particular, the similarity metric varied between 0.03, for stressed and unfocused participants, and 0.36, for relaxed and focused users. Finally, we developed a Machine Learning algorithm based on textual analysis features for automatic cognitive impairment detection, which attained accuracy, F-measure and recall levels above 80%. We have thus validated the automatic approach to detect cognitive impairment in elderly people based on entertainment content. The results suggest that the solution has strong potential for long-term user-friendly therapeutic monitoring of elderly people.
Pseudo Label Is Better Than Human Label

State-of-the-art automatic speech recognition (ASR) systems are trained with tens of thousands of hours of labeled speech data. Human transcription is expensive and time consuming. Factors such as the quality and consistency of the transcription can greatly affect the performance of the ASR models trained with these data. In this paper, we show that we can train a strong teacher model to produce high quality pseudo labels by utilizing recent self-supervised and semi-supervised learning techniques. Specifically, we use JUST (Joint Unsupervised/Supervised Training) and iterative noisy student teacher training to train a 600 million parameter bi-directional teacher model. This model achieved 4.0% word error rate (WER) on a voice search task, 11.1% relatively better than a baseline. We further show that by using this strong teacher model to generate high-quality pseudo labels for training, we can achieve 13.6% relative WER reduction (5.9% to 5.1%) for a streaming model compared to using human labels.
Image encryption using anti-synchronisation and Bogdanov transformation map

A new image encryption algorithm based on anti-synchronisation of Chen system and Bogdanov map is presented in this paper. To illustrate the performance and robustness of the new algorithm, some of security analyses against different cryptanalytic attacks are presented. The security analyses found that, the new algorithm can be applied to provide a suitable application through insecure networks (i.e., internet). As well as, the anti-synchronisation Chen system and Bogdanov transformation map not just limited to digital image encryption, but can be applied directly in several other applications such as real-time application (i.e., video encryption).
Opinion Mining of Readers’ Responses to Literary Prize Nominees on Twitter: A Case Study of Public Reaction to the Booker Prize (2018–2020)

The award of literary prizes such as the Booker Prize has a great impact on production and reception of literary works. In recent years, the Booker Prize awarding committee has faced challenges in selecting books that are more readable and popular. This study suggests that one feasible way to address this issue is to analyze readers’ response and public reaction to literary works in social media. In this regard, the present study aims to develop a data analytics technique that can analyze literary readers’ responses in Twitter and predict a potential prize winner. To achieve this focal goal, a Sentiment Analysis and Topic Modelling approach is designed to classify the public reactions to The Booker Prize nominees in Twitter. The data is extracted for three consecutive years of 2018, 2019 and 2020. In addition, the study utilizes Machine Learning to propose a prediction technique in selecting the best possible Booker Prize winner based on public opinion. The results reveal the main topics frequently appearing in the data as well as the positive and negative sentiments attached to them.
The Unboxing Experience: Exploration and Design of Initial Interactions Between Children and Social Robots

Social robots are increasingly introduced into children's lives as educational and social companions, yet little is known about how these products might best be introduced to their environments. The emergence of the "unboxing" phenomenon in media suggests that introduction is key to technology adoption where initial impressions are made. To better understand this phenomenon toward designing a positive unboxing experience in the context of social robots for children, we conducted three field studies with families of children aged 8 to 13: (1) an exploratory free-play activity ($n=12$); (2) a co-design session ($n=11$) that informed the development of a prototype box and a curated unboxing experience; and (3) a user study ($n=9$) that evaluated children's experiences. Our findings suggest the unboxing experience of social robots can be improved through the design of a creative aesthetic experience that engages the child socially to guide initial interactions and foster a positive child-robot relationship.
Introduction to Explainable AI

As Artificial Intelligence technologies are increasingly used to make important decisions and perform autonomous tasks, providing explanations to allow users and stakeholders to understand the AI has become a ubiquitous concern. Recently, a number of open-source toolkits are making the growing collection of Explainable AI (XAI) techniques accessible for researchers and practitioners to incorporate explanation features in AI systems. This course is open to anyone interested in implementing, designing and researching on the topic, aiming to provide an overview on the technical and design methods for XAI, as well as hands-on experience with an XAI toolkit.
Novel Side-Channel Attacks on Quasi-Cyclic Code-Based Cryptography

Chou suggested a constant-time implementation for quasi-cyclic moderatedensity parity-check (QC-MDPC) code-based cryptography to mitigate timing attacks at CHES 2016. This countermeasure was later found to become vulnerable to a differential power analysis (DPA) in private syndrome computation, as described by Rossi et al. at CHES 2017. The proposed DPA, however, still could not completely recover accurate secret indices, requiring further solving linear equations to obtain entire secret information. In this paper, we propose a multiple-trace attack which enables to completely recover accurate secret indices. We further propose a singletrace attack which can even work when using ephemeral keys or applying Rossi et al.’s DPA countermeasures. Our experiments show that the BIKE and LEDAcrypt may become vulnerable to our proposed attacks. The experiments are conducted using power consumption traces measured from ChipWhisperer-Lite XMEGA (8-bit processor) and ChipWhisperer UFO STM32F3 (32-bit processor) target boards.
Interpolative Distillation for Unifying Biased and Debiased Recommendation

Most recommender systems evaluate model performance offline through either: 1) normal biased test on factual interactions; or 2) debiased test with records from the randomized controlled trial. In fact, both tests only reflect part of the whole picture: factual interactions are collected from the recommendation policy, fitting them better implies benefiting the platform with higher click or conversion rate; in contrast, debiased test eliminates system-induced biases and thus is more reflective of user true preference. Nevertheless, we find that existing models exhibit trade-off on the two tests, and there lacks methods that perform well on both tests.
A Unified View on Semantic Information and Communication: A Probabilistic Logic Approach

This article aims to provide a unified and technical approach to semantic information, communication, and their interplay through the lens of probabilistic logic. To this end, on top of the existing technical communication (TC) layer, we additionally introduce a semantic communication (SC) layer that exchanges logically meaningful clauses in knowledge bases. To make these SC and TC layers interact, we propose various measures based on the entropy of a clause in a knowledge base. These measures allow us to delineate various technical issues on SC such as a message selection problem for improving the knowledge at a receiver. Extending this, we showcase selected examples in which SC and TC layers interact with each other while taking into account constraints on physical channels.
Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning

Neural discrete reasoning (NDR) has shown remarkable progress in combining deep models with discrete reasoning.However, we find that existing NDR solution suffers from large performance drop on hypothetical questions, e.g., "what the annualized rate of return would be if the revenue in 2020 was doubled".The key to hypothetical question answering (HQA) is counterfactual thinking, which is a natural ability of human reasoning but difficult for deep models.In this work, we devise a Learning to Imagine (L2I) module, which can be seamlessly incorporated into NDR models to perform the imagination of unseen counterfactual.In particular, we formulate counterfactual thinking into two steps: 1) identifying the fact to intervene, and 2) deriving the counterfactual from the fact and assumption, which are designed as neural networks.Based on TAT-QA, we construct a very challenging HQA dataset with 8,283 hypothetical questions.We apply the proposed L2I to TAGOP, the state-of-theart solution on TAT-QA, validating the rationality and effectiveness of our approach.
Visual Cluster Grounding for Image Captioning

Attention mechanisms have been extensively adopted in vision and language tasks such as image captioning. It encourages a captioning model to dynamically ground appropriate image regions when generating words or phrases, and it is critical to alleviate the problems of object hallucinations and language bias. However, current studies show that the grounding accuracy of existing captioners is still far from satisfactory. Recently, much effort is devoted to improving the grounding accuracy by linking the words to the full content of objects in images. However, due to the noisy grounding annotations and large variations of object appearance, such strict word-object alignment regularization may not be optimal for improving captioning performance. In this paper, to improve the performance of both grounding and captioning, we propose a novel grounding model which implicitly links the words to the evidence in the image. The proposed model encourages the captioner to dynamically focus on informative regions of the objects, which could be either discriminative parts or full object content. With slacked constraints, the proposed captioning model can capture correct linguistic characteristics and visual relevance, and then generate more grounded image captions. In addition, we propose a novel quantitative metric for evaluating the correctness of the soft attention mechanism by considering the overall contribution of all object proposals when generating certain words. The proposed grounding model can be seamlessly plugged into most attention-based architectures without introducing inference complexity. We conduct extensive experiments on Flickr30k [1] and MS COCO datasets [2], demonstrating that the proposed method consistently improves image captioning in both grounding and captioning. Besides, the proposed attention evaluation metric shows better consistency with the captioning performance.
Development of a novel and fast XRF instrument for large area heavy metal detection integrated with UAV

The disadvantages of the current chemical and instrumental analysis methods for soil heavy metal pollution are that they have a high detection cost, long cycle times, and may cause secondary pollution. The aims of this study were to improve the rapid detection of soil heavy metal pollution over large areas. This study combined aircraft technology, embedded development, computer software, electronic information, and other technical methods to create a novel solution to the problem, i.e., an integrated unmanned aerial vehicle (UAV) based soil heavy metal pollution rapid detection system (UAV-SHMPRDS) was built. The key technologies required for a rapid detection system were developed, including the development of a hardware system based on a UAV and an X-ray fluorescence spectrum (XRF) analyzer, the design and implementation of a control system software system, and the implementation of a data inversion processing algorithm. Finally, a prototype UAV-SHMPRDS was constructed. Testing showed that the system improved regionalized soil heavy metal pollution detection efficiency. This study provides new solutions for the current problems encountered in the actual rapid detection of soil heavy metal pollution.
An integrated framework for breast mass classification and diagnosis using stacked ensemble of residual neural networks

A computer-aided diagnosis (CAD) system requires automated stages of tumor detection, segmentation, and classification that are integrated sequentially into one framework to assist the radiologists with a final diagnosis decision. In this paper, we introduce the final step of breast mass classification and diagnosis using a stacked ensemble of residual neural network (ResNet) models (i.e. ResNet50V2, ResNet101V2, and ResNet152V2). The work presents the task of classifying the detected and segmented breast masses into malignant or benign, and diagnosing the Breast Imaging Reporting and Data System (BI-RADS) assessment category with a score from 2 to 6 and the shape as oval, round, lobulated, or irregular. The proposed methodology was evaluated on two publicly available datasets, the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and additionally on a private dataset. Comparative experiments were conducted on the individual models and an average ensemble of models with an XGBoost classifier. Qualitative and quantitative results show that the proposed model achieved better performance for (1) Pathology classification with an accuracy of 95.13%, 99.20%, and 95.88%; (2) BI-RADS category classification with an accuracy of 85.38%, 99%, and 96.08% respectively on CBIS-DDSM, INbreast, and the private dataset; and (3) shape classification with 90.02% on the CBIS-DDSM dataset. Our results demonstrate that our proposed integrated framework could benefit from all automated stages to outperform the latest deep learning methodologies.