Reviewerly/exHarmony huggingface dataset:
Novel quaternion polar complex exponential transform and its application in color image zero-watermarking

The robust watermarking method embeds the information in the image to protect the copyright. The embedding process will destroy the original image and cannot be effectively applied to the images used in the medicine and military with high integrity demand. A robust zero-watermarking algorithm for lossless copyright protection of color images without changing the original image was proposed in this paper. A novel quaternion polar complex exponential transform (NoQPCET) was firstly constructed with effectively improved performance; then the NoQPCET and chaotic system based robust color image zero-watermarking algorithm was suggested for lossless copyright protection of color images. In the proposed algorithm, the robustness against geometric attacks was enhanced due to the excellent geometric invariance of NoQPCET, and the security was also improved because of the initial value sensitivity of the chaotic system. Experimental results showed that this algorithm had very strong robustness against both geometric attacks and common image processing attacks, and was superior to other algorithms.
HGAMN: Heterogeneous Graph Attention Matching Network for Multilingual POI Retrieval at Baidu Maps

The increasing interest in international travel has raised the demand of retrieving point of interests (POIs) in multiple languages. This is even superior to find local venues such as restaurants and scenic spots in unfamiliar languages when traveling abroad. Multilingual POI retrieval, enabling users to find desired POIs in a demanded language using queries in numerous languages, has become an indispensable feature of today's global map applications such as Baidu Maps. This task is non-trivial because of two key challenges: (1) visiting sparsity and (2) multilingual query-POI matching. To this end, we propose a Heterogeneous Graph Attention Matching Network (HGAMN) to concurrently address both challenges. Specifically, we construct a heterogeneous graph that contains two types of nodes: POI node and query node using the search logs of Baidu Maps. First, to alleviate challenge #1, we construct edges between different POI nodes to link the low-frequency POIs with the high-frequency ones, which enables the transfer of knowledge from the latter to the former. Second, to mitigate challenge #2, we construct edges between POI and query nodes based on the co-occurrences between queries and POIs, where queries in different languages and formulations can be aggregated for individual POIs. Moreover, we develop an attention-based network to jointly learn node representations of the heterogeneous graph and further design a cross-attention module to fuse the representations of both types of nodes for query-POI relevance scoring. In this way, the relevance ranking between multilingual queries and POIs with different popularity can be better handled. Extensive experiments conducted on large-scale real-world datasets from Baidu Maps demonstrate the superiority and effectiveness of HGAMN. In addition, HGAMN has already been deployed in production at Baidu Maps, and it successfully keeps serving hundreds of millions of requests every day. Compared with the previously deployed model, HGAMN achieves significant performance improvement, which confirms that HGAMN is a practical and robust solution for large-scale real-world multilingual POI retrieval service.
On Semantic Similarity in Video Retrieval

Current video retrieval efforts all found their evaluation on an instance-based assumption, that only a single caption is relevant to a query video and vice versa. We demonstrate that this assumption results in performance comparisons often not indicative of models’ retrieval capabilities. We propose a move to semantic similarity video retrieval, where (i) multiple videos/captions can be deemed equally relevant, and their relative ranking does not affect a method’s reported performance and (ii) retrieved videos/captions are ranked by their similarity to a query. We propose several proxies to estimate semantic similarities in large-scale retrieval datasets, without additional annotations. Our analysis is performed on three commonly used video retrieval datasets (MSR-VTT, YouCook2 and EPIC-KITCHENS).
Multimodal Hate Speech Detection in Greek Social Media

Hateful and abusive speech presents a major challenge for all online social media platforms. Recent advances in Natural Language Processing and Natural Language Understanding allow for more accurate detection of hate speech in textual streams. This study presents a new multimodal approach to hate speech detection by combining Computer Vision and Natural Language processing models for abusive context detection. Our study focuses on Twitter messages and, more specifically, on hateful, xenophobic, and racist speech in Greek aimed at refugees and migrants. In our approach, we combine transfer learning and fine-tuning of Bidirectional Encoder Representations from Transformers (BERT) and Residual Neural Networks (Resnet). Our contribution includes the development of a new dataset for hate speech classification, consisting of tweet IDs, along with the code to obtain their visual appearance, as they would have been rendered in a web browser. We have also released a pre-trained Language Model trained on Greek tweets, which has been used in our experiments. We report a consistently high level of accuracy (accuracy score = 0.970, f1-score = 0.947 in our best model) in racist and xenophobic speech detection.
Inception Convolution with Efficient Dilation Search

As a variant of standard convolution, a dilated convolution can control effective receptive fields and handle large scale variance of objects without introducing additional computational costs. To fully explore the potential of dilated convolution, we proposed a new type of dilated convolution (referred to as inception convolution), where the convolution operations have independent dilation patterns among different axes, channels and layers. To develop a practical method for learning complex inception convolution based on the data, a simple but effective search algorithm, referred to as efficient dilation optimization (EDO), is developed. Based on statistical optimization, the EDO method operates in a low-cost manner and is extremely fast when it is applied on large scale datasets. Empirical results validate that our method achieves consistent performance gains for image recognition, object detection, instance segmentation, human detection, and human pose estimation. For instance, by simply replacing the 3 × 3 standard convolution in the ResNet-50 backbone with inception convolution, we significantly improve the AP of Faster R-CNN from 36.4% to 39.2% on MS COCO.
PSB2

For the past six years, researchers in genetic programming and other program synthesis disciplines have used the General Program Synthesis Benchmark Suite to benchmark many aspects of automatic program synthesis systems. These problems have been used to make notable progress toward the goal of general program synthesis: automatically creating the types of software that human programmers code. Many of the systems that have attempted the problems in the original benchmark suite have used it to demonstrate performance improvements granted through new techniques. Over time, the suite has gradually become outdated, hindering the accurate measurement of further improvements. The field needs a new set of more difficult benchmark problems to move beyond what was previously possible.
Text analysis using deep neural networks in digital humanities and information science

Abstract Combining computational technologies and humanities is an ongoing effort aimed at making resources such as texts, images, audio, video, and other artifacts digitally available, searchable, and analyzable. In recent years, deep neural networks (DNN) dominate the field of automatic text analysis and natural language processing (NLP), in some cases presenting a super‐human performance. DNNs are the state‐of‐the‐art machine learning algorithms solving many NLP tasks that are relevant for Digital Humanities (DH) research, such as spell checking, language detection, entity extraction, author detection, question answering, and other tasks. These supervised algorithms learn patterns from a large number of “right” and “wrong” examples and apply them to new examples. However, using DNNs for analyzing the text resources in DH research presents two main challenges: (un)availability of training data and a need for domain adaptation. This paper explores these challenges by analyzing multiple use‐cases of DH studies in recent literature and their possible solutions and lays out a practical decision model for DH experts for when and how to choose the appropriate deep learning approaches for their research. Moreover, in this paper, we aim to raise awareness of the benefits of utilizing deep learning models in the DH community.
Learning the Predictability of the Future

We introduce a framework for learning from unlabeled video what is predictable in the future. Instead of committing up front to features to predict, our approach learns from data which features are predictable. Based on the observation that hyperbolic geometry naturally and compactly encodes hierarchical structure, we propose a predictive model in hyperbolic space. When the model is most confident, it will predict at a concrete level of the hierarchy, but when the model is not confident, it learns to automatically select a higher level of abstraction. Experiments on two established datasets show the key role of hierarchical representations for action prediction. Although our representation is trained with unlabeled video, visualizations show that action hierarchies emerge in the representation.
OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets

We propose a novel framework for creating large-scale photorealistic datasets of indoor scenes, with ground truth geometry, material, lighting and semantics. Our goal is to make the dataset creation process widely accessible, transforming scans into photorealistic datasets with high-quality ground truth for appearance, layout, semantic labels, high quality spatially-varying BRDF and complex lighting, including direct, indirect and visibility components. This enables important applications in inverse rendering, scene understanding and robotics. We show that deep networks trained on the proposed dataset achieve competitive performance for shape, material and lighting estimation on real images, enabling photorealistic augmented reality applications, such as object insertion and material editing. We also show our semantic labels may be used for segmentation and multi-task learning. Finally, we demonstrate that our framework may also be integrated with physics engines, to create virtual robotics environments with unique ground truth such as friction coefficients and correspondence to real scenes. The dataset and all the tools to create such datasets will be made publicly available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Learning to Segment Rigid Motions from Two Frames

Appearance-based detectors achieve remarkable performance on common scenes, benefiting from high-capacity models and massive annotated data, but tend to fail for scenarios that lack training data. Geometric motion segmentation algorithms, however, generalize to novel scenes, but have yet to achieve comparable performance to appearance-based ones, due to noisy motion estimations and degenerate motion configurations. To combine the best of both worlds, we propose a modular network, whose architecture is motivated by a geometric analysis of what independent object motions can be recovered from an egomotion field. It takes two consecutive frames as input and predicts segmentation masks for the background and multiple rigidly moving objects, which are then parameterized by 3D rigid transforma tions. Our method achieves state-of-the-art performance for rigid motion segmentation on KITTI and Sintel. The inferred rigid motions lead to a significant improvement for depth and scene flow estimation.
COLIEE 2020: Methods for Legal Document Retrieval and Entailment

We present a summary of the 7th Competition on Legal Information Extraction and Entailment. The competition consists of four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and an unseen case (Task 2). The statute law component includes an information retrieval task (Task 3) and an entailment/question answering task (Task 4). Participation was open to any group based on any approach. Ten different teams participated in the case law competition tasks, most of them in more than one task. We received results from 9 teams for Task 1 (22 runs) and 8 teams for Task 2 (22 runs). On the statute law task, there were 14 different teams participating, most in more than one task. Eleven teams submitted a total of 28 runs for Task 3, and 13 teams submitted a total of 30 runs for Task 4. We summarize the approaches, our official evaluation, and analysis on our data and submission results.
A computational reproducibility study of PLOS ONE articles featuring longitudinal data analyses

Computational reproducibility is a corner stone for sound and credible research. Especially in complex statistical analyses-such as the analysis of longitudinal data-reproducing results is far from simple, especially if no source code is available. In this work we aimed to reproduce analyses of longitudinal data of 11 articles published in PLOS ONE. Inclusion criteria were the availability of data and author consent. We investigated the types of methods and software used and whether we were able to reproduce the data analysis using open source software. Most articles provided overview tables and simple visualisations. Generalised Estimating Equations (GEEs) were the most popular statistical models among the selected articles. Only one article used open source software and only one published part of the analysis code. Replication was difficult in most cases and required reverse engineering of results or contacting the authors. For three articles we were not able to reproduce the results, for another two only parts of them. For all but two articles we had to contact the authors to be able to reproduce the results. Our main learning is that reproducing papers is difficult if no code is supplied and leads to a high burden for those conducting the reproductions. Open data policies in journals are good, but to truly boost reproducibility we suggest adding open code policies.
Solving correlation clustering with QAOA and a Rydberg qudit system: a full-stack approach

We study the correlation clustering problem using the quantum approximate optimization algorithm (QAOA) and qudits, which constitute a natural platform for such non-binary problems. Specifically, we consider a neutral atom quantum computer and propose a full stack approach for correlation clustering, including Hamiltonian formulation of the algorithm, analysis of its performance, identification of a suitable level structure for ${}^{87}{\rm Sr}$ and specific gate design. We show the qudit implementation is superior to the qubit encoding as quantified by the gate count. For single layer QAOA, we also prove (conjecture) a lower bound of $0.6367$ ($0.6699$) for the approximation ratio on 3-regular graphs. Our numerical studies evaluate the algorithm's performance by considering complete and Erd\H{o}s-R\'enyi graphs of up to 7 vertices and clusters. We find that in all cases the QAOA surpasses the Swamy bound $0.7666$ for the approximation ratio for QAOA depths $p \geq 2$. Finally, by analysing the effect of errors when solving complete graphs we find that their inclusion severely limits the algorithm's performance.
An interpretable Neural Fuzzy Hammerstein-Wiener network for stock price prediction

An interpretable regression model is proposed in this paper for stock price prediction. Conventional offline neuro-fuzzy systems are only able to generate implications based on fuzzy rules induced during training, which requires the training data to be able to adequately represent all system behaviors. However, the distributions of test and training data could be significantly different, e.g., due to drastic data shifts. We address this problem through a novel approach that integrates a neuro-fuzzy system with the Hammerstein-Wiener model forming an indivisible five-layer network, where the implication of the neuro-fuzzy system is realized by the linear dynamic computation of the Hammerstein-Wiener model. The input and output nonlinearities of the Hammerstein-Wiener model are replaced by the nonlinear fuzzification and defuzzification processes of the fuzzy system so that the fuzzy linguistic rules, induced from the linear dynamic computation, can be used to interpret the inference processes. The effectiveness of the proposed model is evaluated on three financial stock datasets. Experimental results showed that the proposed Neural Fuzzy Hammerstein-Wiener (NFHW) outperforms other neuro-fuzzy systems and the conventional Hammerstein-Wiener model on these three datasets.
Tackling the Ill-Posedness of Super-Resolution through Adaptive Target Generation

By the one-to-many nature of the super-resolution (SR) problem, a single low-resolution (LR) image can be mapped to many high-resolution (HR) images. However, learning based SR algorithms are trained to map an LR image to the corresponding ground truth (GT) HR image in the training dataset. The training loss will increase and penalize the algorithm when the output does not exactly match the GT target, even when the outputs are mathematically valid candidates according to the SR framework. This becomes more problematic for the blind SR, as diverse unknown blur kernels exacerbate the ill-posedness of the problem. To this end, we propose a fundamentally different approach for the SR by introducing the concept of the adaptive target. The adaptive target is generated from the original GT target by a transformation to match the output of the SR network. The adaptive target provides an effective way for the SR algorithm to deal with the ill-posed nature of the SR, by providing the algorithm with the flexibility of accepting a variety of valid solutions. Experimental results show the effectiveness of our algorithm, especially for improving the perceptual quality of HR outputs.
Double color image encryption based on fractional order discrete improved Henon map and Rubik’s cube transform

A double color image encryption method based on DNA (deoxyribonucleic acid) computation and chaos is proposed. Differently from the conventional algorithms, double color images are encrypted at the same time so that we can save information of each other, which makes the encryption more safe and reliable. In addition, a new chaotic fractional order (FO) discrete improved Henon map (FODIHM) is proposed as a pseudo-random number generator. To ensure the plain-image sensitivity of the encryption algorithm, the initial value of FODIHM is calculated from the hash value of the color image (SHA-256) and from the three additional keys entered by the user. Furthermore, a Rubik’s cube transform scrambles the pixels in each color component of the two images. Then, each pixel in each color component of the two images is diffused by means of different DNA coding rules. Finally, the CAT transform, based on FO discrete Logistic map and the classic XOR, is used to further improve the security performance. The key space size of the proposed algorithm is of order 10135, which is about 30 orders of magnitude higher than those available in the literature. The information entropies are 7.9974 and 7.9973, which are very close to the ideal entropy value of 8. The values of the unified average changing intensity (NPCI) are 99.630 and 99.623, while the number of pixels change rate (UACR) are 33.473 and 33.553, which are also close to the ideal NPCR and UACR value of 99.6094 and 33.4635, respectively. The numerical results and security analysis prove that the algorithm has good resistance to several classic attacks.
Memory-guided Unsupervised Image-to-image Translation

We present a novel unsupervised framework for instance-level image-to-image translation. Although recent advances have been made by incorporating additional object annotations, existing methods often fail to handle images with multiple disparate objects. The main cause is that, during inference, they apply a global style to the whole image and do not consider the large style discrepancy between instance and background, or within instances. To address this problem, we propose a class-aware memory network that explicitly reasons about local style variations. A key-values memory structure, with a set of read/update operations, is introduced to record class-wise style variations and access them without requiring an object detector at the test time. The key stores a domain-agnostic content representation for allocating memory items, while the values encode domain-specific style representations. We also present a feature contrastive loss to boost the discriminative power of memory items. We show that by incorporating our memory, we can transfer class-aware and accurate style representations across domains. Experimental results demonstrate that our model outperforms recent instance-level methods and achieves state-of-the-art performance.
RRL-GAT: Graph Attention Network-Driven Multilabel Image Robust Representation Learning

Exploring the characterization laws of image data and improving the efficiency of image data characterization knowledge is essential to promote the development of the Internet of Things technology. Considering that images in the real world usually contain multiple objects, and the objects are closely dependent. For these reasons, it brings great challenges to the robust representation learning of multilabel images. In general, researchers model the relationship between objects based on a class activation map and use graph convolution to mine the dependencies between objects. However, graph structure data often contain noise, which means that the edges between nodes are sometimes not so reliable, and the relative importance of neighbors is also different. Based on this, our goal is to reduce noisy connections and false connections between objects, eliminate multilabel image representation bias, and learn robust representations. Therefore, we propose a robust representation learning method for multilabel images driven by graph attention network (RRL-GAT). Specifically, to reduce the accidental false connection of objects in the image, we propose the class attention graph convolution module (C-GAT) to mine the strong association structure between categories. Besides, for the dynamic correlation between objects in the image, we propose an adaptive graph attention convolution module (A-GAT) to capture the subtle dynamic dependencies in the image. The results on two authoritative data sets show that our method is significantly better than all current state-of-the-art methods. Besides, the visualization results show that RRL-GAT can capture the semantic relationship of a specific input image and has sufficient recognizability.