Reviewerly/exHarmony huggingface dataset:
Multimodal Summarization with Guidance of Multimodal Reference

Multimodal summarization with multimodal output (MSMO) is to generate a multimodal summary for a multimodal news report, which has been proven to effectively improve users' satisfaction. The existing MSMO methods are trained by the target of text modality, leading to the modality-bias problem that ignores the quality of model-selected image during training. To alleviate this problem, we propose a multimodal objective function with the guidance of multimodal reference to use the loss from the summary generation and the image selection. Due to the lack of multimodal reference data, we present two strategies, i.e., ROUGE-ranking and Order-ranking, to construct the multimodal reference by extending the text reference. Meanwhile, to better evaluate multimodal outputs, we propose a novel evaluation metric based on joint multimodal representation, projecting the model output and multimodal reference into a joint semantic space during evaluation. Experimental results have shown that our proposed model achieves the new state-of-the-art on both automatic and manual evaluation metrics. Besides, our proposed evaluation method can effectively improve the correlation with human judgments.
Capsule Routing via Variational Bayes

Capsule networks are a recently proposed type of neural network shown to outperform alternatives in challenging shape recognition tasks. In capsule networks, scalar neurons are replaced with capsule vectors or matrices, whose entries represent different properties of objects. The relationships between objects and their parts are learned via trainable viewpoint-invariant transformation matrices, and the presence of a given object is decided by the level of agreement among votes from its parts. This interaction occurs between capsule layers and is a process called routing-by-agreement. In this paper, we propose a new capsule routing algorithm derived from Variational Bayes for fitting a mixture of transforming gaussians, and show it is possible transform our capsule network into a Capsule-VAE. Our Bayesian approach addresses some of the inherent weaknesses of MLE based models such as the variance-collapse by modelling uncertainty over capsule pose parameters. We outperform the state-of-the-art on smallNORB using ≃50% fewer capsules than previously reported, achieve competitive performances on CIFAR-10, Fashion-MNIST, SVHN, and demonstrate significant improvement in MNIST to affNIST generalisation over previous works.1
Challenges and Opportunities with Causal Discovery Algorithms: Application to Alzheimer’s Pathophysiology

Abstract Causal Structure Discovery (CSD) is the problem of identifying causal relationships from large quantities of data through computational methods. With the limited ability of traditional association-based computational methods to discover causal relationships, CSD methodologies are gaining popularity. The goal of the study was to systematically examine whether (i) CSD methods can discover the known causal relationships from observational clinical data and (ii) to offer guidance to accurately discover known causal relationships. We used Alzheimer’s disease (AD), a complex progressive disease, as a model because the well-established evidence provides a “gold-standard” causal graph for evaluation. We evaluated two CSD methods, Fast Causal Inference (FCI) and Fast Greedy Equivalence Search (FGES) in their ability to discover this structure from data collected by the Alzheimer’s Disease Neuroimaging Initiative (ADNI). We used structural equation models (which is not designed for CSD) as control. We applied these methods under three scenarios defined by increasing amounts of background knowledge provided to the methods. The methods were evaluated by comparing the resulting causal relationships with the “gold standard” graph that was constructed from literature. Dedicated CSD methods managed to discover graphs that nearly coincided with the gold standard. For best results, CSD algorithms should be used with longitudinal data providing as much prior knowledge as possible.
Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory

We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem. The reformulations are governed by two user-defined parameters: a positive definite matrix defining a norm, and an arbitrary discrete or continuous distribution over random matrices. Our reformulation has several equivalent interpretations, allowing for researchers from various communities to leverage their domain-specific insights. In particular, our reformulation can be equivalently seen as a stochastic optimization problem, stochastic linear system, stochastic fixed point problem, and a probabilistic intersection problem. We prove sufficient, and necessary and sufficient, conditions for the reformulation to be exact. Further, we propose and analyze three stochastic algorithms for solving the reformulated problem---basic, parallel, and accelerated methods---with global linear convergence rates. The rates can be interpreted as condition numbers of a matrix which depends on the system matrix and on the reformulation parameters. This gives rise to a new phenomenon which we call stochastic preconditioning and which refers to the problem of finding parameters (matrix and distribution) leading to a sufficiently small condition number. Our basic method can be equivalently interpreted as stochastic gradient descent, stochastic Newton method, stochastic proximal point method, stochastic fixed point method, and stochastic projection method, with fixed stepsize (relaxation parameter), applied to the reformulations.
StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics

In machine learning (ML), ensemble methods such as bagging, boosting, and stacking are widely-established approaches that regularly achieve top-notch predictive performance. Stacking (also called "stacked generalization") is an ensemble method that combines heterogeneous base models, arranged in at least one layer, and then employs another metamodel to summarize the predictions of those models. Although it may be a highly-effective approach for increasing the predictive performance of ML, generating a stack of models from scratch can be a cumbersome trial-and-error process. This challenge stems from the enormous space of available solutions, with different sets of data instances and features that could be used for training, several algorithms to choose from, and instantiations of these algorithms using diverse parameters (i.e., models) that perform differently according to various metrics. In this work, we present a knowledge generation model, which supports ensemble learning with the use of visualization, and a visual analytics system for stacked generalization. Our system, StackGenVis, assists users in dynamically adapting performance metrics, managing data instances, selecting the most important features for a given data set, choosing a set of top-performant and diverse algorithms, and measuring the predictive performance. In consequence, our proposed tool helps users to decide between distinct models and to reduce the complexity of the resulting stack by removing overpromising and underperforming models. The applicability and effectiveness of StackGenVis are demonstrated with two use cases: a real-world healthcare data set and a collection of data related to sentiment/stance detection in texts. Finally, the tool has been evaluated through interviews with three ML experts.
Unsupervised Opinion Summarization with Noising and Denoising

The supervised training of high-capacity models on large datasets containing hundreds of thousands of document-summary pairs is critical to the recent success of deep learning techniques for abstractive summarization.Unfortunately, in most domains (other than news) such training data is not available and cannot be easily sourced.In this paper we enable the use of supervised learning for the setting where there are only documents available (e.g., product or business reviews) without ground truth summaries.We create a synthetic dataset from a corpus of user reviews by sampling a review, pretending it is a summary, and generating noisy versions thereof which we treat as pseudo-review input.We introduce several linguistically motivated noise generation functions and a summarization model which learns to denoise the input and generate the original review.At test time, the model accepts genuine reviews and generates a summary containing salient opinions, treating those that do not reach consensus as noise.Extensive automatic and human evaluation shows that our model brings substantial improvements over both abstractive and extractive baselines.
A Novel and Efficient 3D Multiple Images Encryption Scheme Based on Chaotic Systems and Swapping Operations

Single image encryption schemes are not efficient enough when a bunch of images is to be encrypted in some real-world setting. To overcome this problem, an efficient and secured multiple images encryption scheme is proposed in this study using two chaotic maps and simple row and column swapping operations in a 3D image space. The N input images are piled to make a 3D image. To confuse the given pixel data, two images are chosen randomly from this pile. The randomly chosen two rows from the two randomly chosen images are swapped with each other. In the same way, two randomly chosen columns are swapped with each other. The operation of randomly chosen two images, two rows, and two columns have been iterated an arbitrary number of times to throw the confusion effects in the pixels data. Intertwining Logistic Map (ILM) and Improved Piecewise Linear Chaotic Map (MPWLCM) have been used to get the four streams of random numbers. The three streams of the former map have been used to create the confusion effects, whereas the fourth stream of random numbers given by the latter map has been used for the diffusion effects. SHA-256 hash codes have been used to throw the plaintext sensitivity in the proposed cipher. Besides, a 256-bit user key has been employed to increase the key space. Both the simulation and the exhaustive security analyses carried out at the end vividly prove the security, resistance to the varied attacks, and the real-world applicability of the proposed cipher.
Memory-augmented Dense Predictive Coding for Video Representation Learning

The objective of this paper is self-supervised learning from video, in particular for representations for action recognition. We make the following contributions: (i) We propose a new architecture and learning framework Memory-augmented Dense Predictive Coding (MemDPC) for the task. It is trained with a predictive attention mechanism over the set of compressed memories, such that any future states can always be constructed by a convex combination of the condense representations, allowing to make multiple hypotheses efficiently. (ii) We investigate visual-only self-supervised video representation learning from RGB frames, or from unsupervised optical flow, or both. (iii) We thoroughly evaluate the quality of learnt representation on four different downstream tasks: action recognition, video retrieval, learning with scarce annotations, and unintentional action classification. In all cases, we demonstrate state-of-the-art or comparable performance over other approaches with orders of magnitude fewer training data.
PolarDet: a fast, more precise detector for rotated target in aerial images

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Humans can decipher adversarial images

How similar is the human mind to the sophisticated machine-learning systems that mirror its performance? Models of object categorization based on convolutional neural networks (CNNs) have achieved human-level benchmarks in assigning known labels to novel images. These advances promise to support transformative technologies such as autonomous vehicles and machine diagnosis; beyond this, they also serve as candidate models for the visual system itself -- not only in their output but perhaps even in their underlying mechanisms and principles. However, unlike human vision, CNNs can be "fooled" by adversarial examples -- carefully crafted images that appear as nonsense patterns to humans but are recognized as familiar objects by machines, or that appear as one object to humans and a different object to machines. This seemingly extreme divergence between human and machine classification challenges the promise of these new advances, both as applied image-recognition systems and also as models of the human mind. Surprisingly, however, little work has empirically investigated human classification of such adversarial stimuli: Does human and machine performance fundamentally diverge? Or could humans decipher such images and predict the machine's preferred labels? Here, we show that human and machine classification of adversarial stimuli are robustly related: In eight experiments on five prominent and diverse adversarial imagesets, human subjects reliably identified the machine's chosen label over relevant foils. This pattern persisted for images with strong antecedent identities, and even for images described as "totally unrecognizable to human eyes". We suggest that human intuition may be a more reliable guide to machine (mis)classification than has typically been imagined, and we explore the consequences of this result for minds and machines alike.
Double-encrypted watermarking algorithm based on cosine transform and fractional Fourier transform in invariant wavelet domain

The existing watermarking algorithms invariant wavelet domain are weak at resisting geometric attacks and have small embedding capacities. In this paper, a robust double-encrypted watermarking algorithm based on the fractional Fourier transform (FRFT) and discrete cosine transform (DCT) in invariant wavelet domain is proposed. A hybrid domain is obtained by applying the redistributed invariant wavelet transform (RIDWT) and cosine transform to the enlarged host image. The low-frequency and high-frequency regions of the domain have advantages against different attacks; thus, the two components are both selected for watermark embedding. To solve the false positive problem, the enlarged watermark is double-encrypted by the Arnold transform and the FRFT before a singular value decomposition is applied to it, which enhances the security of the algorithm due to the generation of more keys. Multiparameter particle swarm optimization (MP-PSO) is used to obtain the optimal embedding factors for achieving the balance between invisibility and robustness. The simulation results and comparative experiments show that the proposed algorithm exhibits high robustness under the premise of satisfying security, reliability and invisibility, especially for geometric attacks such as rotation, cropping and translation.
Teachers’ perspective on school development at German vocational schools during the Covid-19 pandemic

This article focuses on the challenges of teachers at vocational schools with regard to the process of digitalisation in school development during the Covid-19 pandemic. Besides the intrapersonal prerequisites for teaching with technology and the change in learning and teaching attitudes, preconditions concerning schools as part of a learning organisation are examined. A qualitative interview study including N = 18 participants was conducted to identify challenges for teachers, the role of school leadership, and cooperation between stakeholders. The main coping strategies proved to be a clear agenda by the school leadership in connection with reliable technological infrastructure and teachers' willingness to use digital teaching methods.
Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learning

Conversational recommender systems (CRS) enable the traditional recommender systems to explicitly acquire user preferences towards items and attributes through interactive conversations. Reinforcement learning (RL) is widely adopted to learn conversational recommendation policies to decide what attributes to ask, which items to recommend, and when to ask or recommend, at each conversation turn. However, existing methods mainly target at solving one or two of these three decision-making problems in CRS with separated conversation and recommendation components, which restrict the scalability and generality of CRS and fall short of preserving a stable training procedure. In the light of these challenges, we propose to formulate these three decision-making problems in CRS as a unified policy learning task. In order to systematically integrate conversation and recommendation components, we develop a dynamic weighted graph based RL method to learn a policy to select the action at each conversation turn, either asking an attribute or recommending items. Further, to deal with the sample efficiency issue, we propose two action selection strategies for reducing the candidate action space according to the preference and entropy information. Experimental results on two benchmark CRS datasets and a real-world E-Commerce application show that the proposed method not only significantly outperforms state-of-the-art methods but also enhances the scalability and stability of CRS.
Challenges in Automated Debiasing for Toxic Language Detection

Warning: this paper contains content that may be offensive or upsetting.Biased associations have been a challenge in the development of classifiers for detecting toxic language, hindering both fairness and accuracy.As potential solutions, we investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection.Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English).Our comprehensive experiments establish that existing methods are limited in their ability to prevent biased behavior in current toxicity detectors.We then propose an automatic, dialect-aware data correction method, as a proof-of-concept study.Despite the use of synthetic labels, this method reduces dialectal associations with toxicity.Overall, our findings show that debiasing a model trained on biased toxic language data is not as effective as simply relabeling the data to remove existing biases.
Cyber security threat modeling based on the MITRE Enterprise ATT&amp;CK Matrix

Abstract Enterprise systems are growing in complexity, and the adoption of cloud and mobile services has greatly increased the attack surface. To proactively address these security issues in enterprise systems, this paper proposes a threat modeling language for enterprise security based on the MITRE Enterprise ATT&amp;CK Matrix. It is designed using the Meta Attack Language framework and focuses on describing system assets, attack steps, defenses, and asset associations. The attack steps in the language represent adversary techniques as listed and described by MITRE. This entity-relationship model describes enterprise IT systems as a whole; by using available tools, the proposed language enables attack simulations on its system model instances. These simulations can be used to investigate security settings and architectural changes that might be implemented to secure the system more effectively. Our proposed language is tested with a number of unit and integration tests. This is visualized in the paper with two real cyber attacks modeled and simulated.
Machine learning of high dimensional data on a noisy quantum processor

Abstract Quantum kernel methods show promise for accelerating data analysis by efficiently learning relationships between input data points that have been encoded into an exponentially large Hilbert space. While this technique has been used successfully in small-scale experiments on synthetic datasets, the practical challenges of scaling to large circuits on noisy hardware have not been thoroughly addressed. Here, we present our findings from experimentally implementing a quantum kernel classifier on real high-dimensional data taken from the domain of cosmology using Google’s universal quantum processor, Sycamore. We construct a circuit ansatz that preserves kernel magnitudes that typically otherwise vanish due to an exponentially growing Hilbert space, and implement error mitigation specific to the task of computing quantum kernels on near-term hardware. Our experiment utilizes 17 qubits to classify uncompressed 67 dimensional data resulting in classification accuracy on a test set that is comparable to noiseless simulation.
An efficient way of text-based emotion analysis from social media using LRA-DNN

Text devices are effectively and heavily used for interactions these days. Emotion extraction from the text has derived huge importance and is upcoming area of research in Natural Language Processing. Recognition of emotions from text has high practical utilities for quality improvement like in Human-Computer Interaction, recommendation systems, online education, data mining and so on. However, there are the issues of irrelevant feature extraction during emotion extraction from text. It causes mis-prediction of emotion. To overcome such challenges, this paper proposes a Leaky Relu activated Deep Neural Network (LRA-DNN). The proposed model comes under four categories, such as pre-processing, feature extraction, ranking and classification. The collected data from the dataset are pre-processed for data cleansing, appropriate features are extracted from the pre-processed data, relevant ranks are assigned for each extracted feature in the ranking phase and finally, the data are classified and accurate output is obtained from the classification phase. Publically available datasets are used in this research to compare the results obtained by the proposed LRA-DNN with the previous state-of-art algorithms. The outcomes indicated that the proposed LRA-DNN obtains the highest accuracy, sensitivity, and specificity at the rate of 94.77%, 92.23%, and 95.91% respectively which is promising compared to the existing ANN, DNN and CNN methods. It also efficiently reduces the mis-prediction and misclassification error.
Malevolent Information Crawling Mechanism for Forming Structured Illegal Organisations in Hidden Networks

Terrorist groups like ISIS have made considerable use of the dark web (DW) to carry out their malicious objectives such as spreading propaganda, recruiting and radicalizing new recruits, and secretly raising finances. Al-Hayat Media Center, an ISIS media agency, released a link on their forum describing how to access their DW website. It also sent out the identical message over Telegram, which included links to a Tor service with a “.onion” address. This study develops an analytical framework for scraping and analyzing the DW on the internet. The authors successfully tested a web crawler to collect account information for thousands of merchants and their related marketplace listings using a case study marketplace. The paper explains how to scrape DW marketplaces in the most viable and effective way possible. The findings of the case study support the validity of the proposed analytical framework, which is useful for academics researching this emerging phenomena as well as investigators looking into illegal behavior on the DW.