Reviewerly/exHarmony huggingface dataset:
Sequential Vision to Language as Story: A Storytelling Dataset and Benchmarking

Storytelling is a remarkable human skill that plays a significant role in learning and experiencing everyday life. Developing narratives is central to human mental health development, simultaneously encapsulating broad details such as psychology, morality and common sense. Contemporary deep-learning algorithms require similar skills to be able to tell a story from a visual perspective. However, most algorithms function at a superficial or factual level, aligning descriptive text with images in a one-to-one manner without considering the temporal relation. Stories are more expressive in style, language and content, involving imaginary concepts not explicit in the images. An ideal deep learning system should learn and develop cohesive, meaningful, and causal stories. Unfortunately, most existing storytelling methods are trained and evaluated on a single dataset, i.e., the VIsual STorytelling (VIST) dataset. Multiple datasets are essential to test the generalization ability of algorithms. We bridge the gap and present a new dataset for expressive and coherent story creation. We present the Sequential Storytelling Image Dataset (SSID), consisting of open-source video frames accompanied by story-like annotations. We provide four annotations (stories) for each set of five images. The image sets are collected manually from publicly available videos in three domains: documentaries, lifestyle, and movies, and then annotated manually using Amazon Mechanical Turk. We perform a detailed analysis and benchmarking of the current VIST dataset and our new SSID dataset and show that both datasets exhibit high variance within their multiple ground truth stories corresponding to the same image set. Moreover, our dataset achieves lower mean average scores across all metrics, meaning that the ground truth stories of our dataset are more diverse. Finally, we train and evaluate existing state-of-the-art rhetorical storytelling methods on both datasets and show that our dataset is more challenging and requires sophisticated techniques to accurately detect a significant variety of events.
A Multilingual Spam Reviews Detection Based on Pre-Trained Word Embedding and Weighted Swarm Support Vector Machines

Online reviews are important information that customers seek when deciding to buy products or services. Also, organizations benefit from these reviews as essential feedback for their products or services. Such information required reliability, especially during the Covid-19 pandemic which showed a massive increase in online reviews due to quarantine and sitting at home. Not only the number of reviews was boosted but also the context and preferences during the pandemic. Therefore, spam reviewers reflect on these changes and improve their deception technique. Spam reviews usually consist of misleading, fake, or fraudulent reviews that tend to deceive customers for the purpose of making money or causing harm to other competitors. Hence, this work presents a Weighted Support Vector Machine (WSVM) and Harris Hawks Optimization (HHO) for spam review detection. The HHO works as an algorithm for optimizing hyperparameters and feature weighting. Three different language corpora have been used as datasets, namely English, Spanish, and Arabic in order to solve the multilingual problem in spam reviews. Moreover, pre-trained word embedding (BERT) has been applied alongside three-word representation methods (NGram-3, TFIDF, and One-hot encoding). Four experiments have been conducted, each focused on solving and demonstrating different aspects. In all experiments, the proposed approach showed excellent results compared with other state-of-the-art algorithms. In other words, the WSVM-HHO achieved an accuracy of 88.163%, 71.913%, 89.565%, and 84.270%, for English, Spanish, Arabic, and Multilingual datasets, respectively. Further, a deep analysis has been conducted to investigate the context of reviews before and after the COVID-19 situation. In addition, it has been generated to create a new dataset with statistical features and merge its previous textual features for improving detection performance.
Context Aware Crowd Tracking and Anomaly Detection via Deep Learning and Social Force Model

The world's expanding populace, the variety of human social factors, and the densely populated environment make humans feel uncertain.Individuals need a safety officer who generally deals with security viewpoints for this frailty.Currently, human monitoring techniques are time-consuming, work concentrated, and incapable.Therefore, autonomous surveillance frameworks are necessary for the modern day since they are able to address these problems.Nevertheless, hardships persist.The central concerns incorporate the detachment of the foreground from the scene and the understanding of the contextual structure of the environment for efficiently identifying unusual objects.In our work, we introduced a novel framework to tackle these difficulties by presenting a semantic segmentation technique for separating a foreground object.In our work, Super-pixels are generated using an improved watershed transform and then a conditional random field is implemented to obtain multi-object segmented frames by performing pixel-level labeling.Next, the Social Force model is introduced to extract the contextual structure of the environment via the fusion of a novel chosen particular histogram of an optical stream and inner force model.After using the computed social force, multi-people tracking is performed via three-dimensional template association using percentile rank and non-maximal suppression.Next, multi-object categorization is performed via deep learning Feature Pyramid Network.Finally, by considering the contextual structure of the environment, Jaccard similarity is utilized to make the decision for abnormality detection and identify the unusual objects from the scene.The invented framework is verified through rigorous investigations, and it obtained multipeople tracking efficiency of 92.2% and 89.1% over the UCSD and CUHK Avenue datasets.However, 95.2% and 93.7% abnormality detection efficiency is accomplished over UCSD and CUHK Avenue datasets, respectively.
RIO: Return Instruction Obfuscation for Bare-Metal IoT Devices

The proliferation of IoT comes with many challenges, among which security is one of the most serious issues.Since most IoT devices are designed based on bare-metal devices with low performance, low power, and small form factor, an attacker can easily perform firmware extraction and analysis.In addition, code reuse attacks, such as Return Oriented Programming (ROP) attacks, are possible based on the analyzed firmware information.Since most code reuse attacks are performed through a combination of gadgets that include return instructions, preventing an attacker from analyzing return instructions can be a fundamental solution for these attacks.To prevent code reuse attacks on bare-metal based IoT devices, this paper proposes Return Instruction Obfuscation (RIO).The proposed scheme encrypts all return instructions in the firmware and instruments the modules necessary to decrypt and execute the encrypted return instructions using a Low Level Virtual Machine (LLVM).Since all return instructions in the firmware are encrypted, the proposed scheme can prevent attackers from performing firmware analysis and gadget collection.The proposed scheme was implemented and evaluated on Nuvoton's NuMaker-PFM-M2351 development board with an ARM Cortex-M23 based SoC.
Domain-Specific Sentiment Analysis: An Optimized Deep Learning Approach for the Financial Markets

Although different studies are caried out by deep learning models for financial markets sentiment analysis, there is a lack of specific embedding method that regards the domain.Therefore, the goal of this study is to discover what type of embedding techniques along with different classification algorithms work better for the financial markets' sentiment analysis to present an optimized embedding method in the domain.In this paper we present a broad comparative study of multiple classification models trained for sentiment analysis and will improve their performance with an optimized embedding layer.We use a heterogeneous corpus of both formal (news headlines) and informal (tweets) text to increase the robustness and build the models with CBOW, GloVe, and BERT pre-trained embeddings as well as developing an optimized embedding layer to improve the results.The best results reported here are by our LSTM model with the fine-tuned embedding layer, which has an accuracy of 0.84 and a macro-average F1score of 0.8.Our results give evidence that the fine-tuned embedding is superior to utilising pretrained CBOW, GloVe, and BERT embeddings for financial markets sentiment analysis.We train SVM, MLP, CNN, generic RNN and LSTM models by a comprehensive approach in input data and algorithms.As a result, a sentiment analysis model is presented with a robust performance for different datasets in the domain.
Addressing Imbalance Problem for Multi Label Classification of Scholarly Articles

Scientific document classification is an important field of machine learning. Currently, scientific document category identification is done manually. There are already defined taxonomies available for categorizing scientific documents, such as the Association for Computing Machinery Computing Classification System (ACM CCS) and Bibsonomy. These taxonomies facilitate authors in the categories of their manuscripts. The incorporation of research work from a variety of domains in the assignment takes on the form of a Multi-Label Classification (MLC). Using MLC, it is possible to assign more than one class to a single document. To address the problem of MLC in its entirety, two distinct methods are used (Problem Transformation and Algorithm Adaptation). The MLC dataset is transformed into one or more single-label datasets through the application of the problem transformation technique. Whereas, a single classifier is modified during the algorithm adaptation process so that it can predict multiple labels. Currently, document classification is done using various techniques in the literature, but none of them paid much attention to the problem of imbalance in Multi-Label Datasets (MLD). However, many effective techniques for dealing with imbalance are available in the literature. The goal of this study is to find an effective technique for balancing datasets before multi-label classification to get better predictions for the classes with fewer instances. Six MLDs, nine transformation techniques and seven classifiers are evaluated in this research work. The proposed research will result in a more accurate recommendation of a research topic for a document. For imbalanced MLDs, LPROS is the best resampling technique using statistical tests. When compared to the other classifiers, the BRkNN classifier is better for MLC. This research will facilitate the classification of documents into their respective classes which can be used by various citation indexes.
Policy guidelines to facilitate collective action towards quantum-safety

As the development of quantum computers advances, actors relying on public key infrastructures (PKI) for secure information exchange are becoming aware of the disruptive implications. Currently, governments and businesses employ PKI for many core processes that may become insecure or unavailable when quantum computers break the cryptographic algorithms foundational to PKI. While standardization institutes are currently testing quantum safe cryptographic algorithms, there are no globally agreed-upon cryptographic solutions available. Actors looking to prepare for the implementation of quantum safe cryptographic algorithms lack methods that allow for collective planning and action across organizations, sectors, and nations. The goal of this policy paper is to elicit requirements for a serious game on QS PKI, and derive policy guidelines that actors can use to prepare and formulate governance arrangements. We followed a two-step approach, drawing on technology threat avoidance theory and collective action theory, followed by empirical grounding through a focus group. The results from the literature confirm that a serious game could be a suitable governance mechanism for QS PKI. The focus group results discussed 12 requirements and the requirement's relation to the theoretical background. From this, the findings section arrived at four policy guidelines derived from the requirements that can function as focus areas for further requirement development and as input for policy makers. The policy guidelines concluded are (1) prioritize increasing collective awareness through emphasizing social networks, (2) acknowledge the interdependencies in migrating towards QS PKI, (3) create an understanding of the technical standards in the field and their issuers, and (4) being highly realistic with both negative and positive scenarios to center the players’ understanding of real-world impact.
Fault Template Attack Based on Fault Probability

Template attack uses leaked information from various side channels and may break out the countermeasures embedded in the algorithm, which is considered a powerful side-channel attack method. However, the current template attack has the problems of a large amount of data required and poor portability. Aiming at the above problems, this paper proposes a fault template attack method based on fault probability. This method can recover the key with a small amount of data and without access to ciphertext. It applies to all cryptographic algorithms of the Substitution-Permutation Network (SPN) structure. In the template profiling stage, the search interval of points of interest is reduced by using the attack time window; the fault probability model of Hamming Weight is used to profile the highly discriminative template, and thus the key search space is reduced by 28 times. In the key recovery stage, the inverse operation is performed without traversing the key, significantly reducing the complexity of calculation. Finally, we carry out attacks based on the AES-128 algorithm of the Microcontroller Unit (MCU) and use 45 fault probability traces to profile 9 templates. The attacker can succeed in performing key recovery with only 10 attack traces obtained.
Towards an Organically Growing Hate Speech Dataset in Hate Speech Detection Systems in a Smart Mobility Application

The automatic detection of hate speech online poses several challenges. A top challenge is that hate speech changes its targets and its format periodically. While the lack of available training data is a general issue in many natural language processing applications, the forementioned challenge amplifies the problem especially when taking into consideration the challenge of producing well labelled datasets. Based on the concepts of quarantining hate speech and integrating a linguistics expert in a smart mobility service provided in an administrative district in Germany, this paper proposes an approach that targets improving the training dataset quantitively and qualitatively in a running smart mobility app, the SWIA app. This proactive approach provides a long-term solution for hate speech detection models that rely on labelled datasets for training. The paper also discusses technical and practical challenges unanswered by this approach.
IPIML: A Deep-Scan Earthquake Detection and Location Workflow Integrating Pair-Input Deep Learning Model and Migration Location Method

Optimized Deep Learning (DL)-based workflows can improve the efficiency and accuracy of earthquake detection and location processes. This paper introduces a six-step automated event detection, phase association, and earthquake location workflow, which integrates the state-of-the-art Pair-Input DL model and waveform Migration Location methods (IPIML). Applying IPIML on an 18-months dataset of Ghana Digital Seismic Network (GHSDN) recorded from 2012-2014, a catalog with 461 events is automatically obtained. Compared to other DL catalogs obtained using EQTransformer (EQT) and Siamese EQTransformer (S-EQT), the seismic event clusters in the IPIML catalog focus more on tectonically active regions or known seismogenic source areas and show a consistent depth distribution. The compiled catalog is 6.3 times larger than the reported catalog obtained by applying EQT with the default settings, indicating the importance of optimization and hyper-parameter tuning when applying DL models. As a result, a previously unknown seismogenic fault with a clear spatial trend has been identified using the new IPIML catalog, which provides more insights into the fault activities and seismic hazards in the region. The IPIML codes and data sets are available at the GitHub repository https://github.com/SigProSeismology/IPIML.git, contributing to the geoscience community.
ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning

Learning predictive models for unlabeled spatiotemporal data is challenging in part because visual dynamics can be highly entangled, especially in real scenes. In this paper, we refer to the multi-modal output distribution of predictive learning as spatiotemporal modes. We find an experimental phenomenon named spatiotemporal mode collapse (STMC) on most existing video prediction models, that is, features collapse into invalid representation subspaces due to the ambiguous understanding of mixed physical processes. We propose to quantify STMC and explore its solution for the first time in the context of unsupervised predictive learning. To this end, we present ModeRNN, a decoupling-aggregation framework that has a strong inductive bias of discovering the compositional structures of spatiotemporal modes between recurrent states. We first leverage a set of dynamic slots with independent parameters to extract individual building components of spatiotemporal modes. We then perform a weighted fusion of slot features to adaptively aggregate them into a unified hidden representation for recurrent updates. Through a series of experiments, we show high correlation between STMC and the fuzzy prediction results of future video frames. Besides, ModeRNN is shown to better mitigate STMC and achieve the state of the art on five video prediction datasets.
Fair prices for sustainability in agriculture and food. Requirements and design options for a data-based transparency system.

The incumbent minister of the German Federal Ministry of Food and Agriculture announced at the beginning of his tenure that farmers need more appreciation and that prices should more strongly express the ecological truth. He does not further tolerate an exploitative system that comes at the expense of people, animals, the environment, and the climate, and pleads for more appreciation and value creation for farmers. One approach to address this issue is the provision of authenticated, traceable, and irreversible data about the food origin and production process. This paper presents the methods, experiences and results of a project that examined in cooperation with relevant stakeholders the technical possibilities of a data space that provides production and process data to the customer. An integral part of this data space is a blockchain module that ensures the traceability and irreversibility of the data. The paper contributes to research on blockchain application in governmental systems that provide trustful information in a food data space and it provides insights in to the concerns of different stakeholders as well as their role in the overall governance of a blockchain supported data space.
A Taxonomy of Design Decisions for Central Bank Digital Currencies

In recent years, the amount of research on Central Bank Digital Currencies (CBDCs) by central banks has increased extensively. These can cover a broad range of applications, from the settlement of mass transactions to interbank payments. Depending on the design, different objectives can be pursued when introducing a CBDC, such as financial inclusion or enhancing payment efficiency. Although first attempts to organize the design aspects of CBDCs have been conducted, there seems to be no construct that comprehensively characterizes the essential design decisions. Considering both conceptual and empirical sources, we address this shortcoming by developing a taxonomy that represents and characterizes the fundamental design decisions to be made by central banks when creating a CBDC. The taxonomy enables its users to classify CBDCs and compare existing configurations. Moreover, it supports the derivation and discussion of implications as well as interactions between design decisions. The article thereby contributes to the existing literature by enhancing the understanding of the concept of CBDC, thus strengthening the conceptual foundation for future research. Furthermore, our findings can support the development of further frameworks addressing their design at a more profound level.
CFU-Net: A Coarse–Fine U-Net With Multilevel Attention for Medical Image Segmentation

The U-Net has achieved great successes in medical image segmentation. Most U-Nets follow the encoding-decoding-decision inference path, and propagate the features from encoding to decoding. However, the traditional approaches do not exploit the semantic differences among different organs and different image modalities, which are task-unaware and have limited generalization. To address these issues, this paper proposes a Coarse-Fine U-Net (CFU-Net) architecture with two embedded U-Nets, and designs a Multi-Level Attention Module (MLAM) to execute the multi-level information interaction. CFU-Net introduces an additional decoding path at lower level, which is formed as partly coupled two U-Nets with different depths, namely coarse U-Net and fine U-Net. Coarse U-Net obtains a coarse prediction which is then used to guide the decoding of fine U-Net. MLAM adjusts the features propagation in fine U-Net by exploiting the interactions of multi-level information, including decision information, contextual information, and long-range dependencies. In addition, CFU-Net is constructed using dynamic convolution to improve the adaptability of convolution. The performance of CFU-Net is evaluated on four different modalities datasets, including ISIC2018, BUSI, Kvasir-SEG, and LiTS. For the Dice/Intersection-over-Union (IoU) scores, CFU-Net obtains 0.82%/1.62%, 4.34%/6.89%, 5.23%/9.30%, and 5.11%/5.18% improvements over the state-of-the-art UNeXt on ISIC2018, BUSI, Kvasir-SEG, and LiTS datasets, respectively. Moreover, the superiority of CFU-Net on different modalities segmentation tasks can also demonstrate that our method has better generalization, which can be transferred into various disease diagnoses.
FarfetchFusion: Towards Fully Mobile Live 3D Telepresence Platform

We present FarfetchFusion, a fully mobile live 3D telepresence system. Enabling mobile live telepresence is a challenging problem as it requires i) realistic reconstruction of the user and ii) high responsiveness for immersive experience. We first thoroughly analyze the live 3D telepresence pipeline and identify three critical challenges: i) 3D data streaming latency and compression complexity, ii) computational complexity of volumetric fusion-based 3D reconstruction, and iii) inconsistent reconstruction quality due to sparsity of mobile 3D sensors. To tackle the challenges, we propose a disentangled fusion approach, which separates invariant regions and dynamically changing regions with our low-complexity spatio-temporal alignment technique, topology anchoring. We then design and implement an end-to-end system, which achieves realistic reconstruction quality comparable to existing server-based solutions while meeting the real-time performance requirements (<100 ms end-to-end latency, 30 fps throughput, <16 ms motion-to-photon latency) solely relying on mobile computation capability.