Reviewerly/exHarmony huggingface dataset:
Functorial Semantics as a Unifying Perspective on Logic Programming

Logic programming and its variations are widely used for formal reasoning in various areas of Computer Science, most notably Artificial Intelligence. In this paper we develop a systematic and unifying perspective for (ground) classical, probabilistic, weighted logic programs, based on categorical algebra. Our departure point is a formal distinction between the syntax and the semantics of programs, now regarded as separate categories. Then, we are able to characterise the various variants of logic program as different models for the same syntax category, i.e. structure-preserving functors in the spirit of Lawvere’s functorial semantics. As a first consequence of our approach, we showcase a series of semantic constructs for logic programming pictorially as certain string diagrams in the syntax category. Secondly, we describe the correspondence between probabilistic logic programs and Bayesian networks in terms of the associated models. Our analysis reveals that the correspondence can be phrased in purely syntactical terms, without resorting to the probabilistic domain of interpretation.
Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning

Instance discriminative self-supervised representation learning has been attracted attention thanks to its unsupervised nature and informative feature representation for downstream tasks. In practice, it commonly uses a larger number of negative samples than the number of supervised classes. However, there is an inconsistency in the existing analysis; theoretically, a large number of negative samples degrade classification performance on a downstream supervised task, while empirically, they improve the performance. We provide a novel framework to analyze this empirical result regarding negative samples using the coupon collector's problem. Our bound can implicitly incorporate the supervised loss of the downstream task in the self-supervised loss by increasing the number of negative samples. We confirm that our proposed analysis holds on real-world benchmark datasets.
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

Adapting large-scale pretrained language models to downstream tasks via fine-tuning is the standard method for achieving state-of-the-art performance on NLP benchmarks. However, fine-tuning all weights of models with millions or billions of parameters is sample-inefficient, unstable in low-resource settings, and wasteful as it requires storing a separate copy of the model for each task. Recent work has developed parameter-efficient fine-tuning methods, but these approaches either still require a relatively large number of parameters or underperform standard fine-tuning. In this work, we propose Compacter, a method for fine-tuning large-scale language models with a better trade-off between task performance and the number of trainable parameters than prior work. Compacter accomplishes this by building on top of ideas from adapters, low-rank optimization, and parameterized hypercomplex multiplication layers. Specifically, Compacter inserts task-specific weight matrices into a pretrained model's weights, which are computed efficiently as a sum of Kronecker products between shared ``slow'' weights and ``fast'' rank-one matrices defined per Compacter layer. By only training 0.047% of a pretrained model's parameters, Compacter performs on par with standard fine-tuning on GLUE and outperforms fine-tuning in low-resource settings. Our code is publicly available in this https URL
Korea-EU intellectual and cultural exchange

The exchange system for researchers is an important agreement between Korea and the European Union (EU), as they signed the agreement during the Korea-EU Summit in 2013. With 2020 marking the tenth anniversary of the strategic partnership between the EU and Korea, intellectual and cultural exchanges between the two regions are more vibrant than ever. This chapter aims to review the concepts behind knowledge diplomacy and cultural diplomacy as well as examining the meaning of cultural exchange. With the enactment of the Public Diplomacy Act in February 2016 and its effectuation in August of the same year, the Korea Foundation (KF) is officially Korea’s designated public diplomacy planning and promotion agency. In order to maintain and continue exchanges and cooperation with European countries, the KF has been conducting a next-generation leadership exchange program. In the case of Europe, the KF held bilateral forums with Germany, France, United Kingdom, Spain and Italy.
Graph Tree Memory Networks

We introduce Graph Tree Memory Networks that memorize and remember any data. This neural network has two memories. One consists of a queue-structured short-term memory to solve the class imbalance problem and long-term memory to store the distribution of objects, introducing the contents of storing and generating various datasets.
Coordinated constructional intensifiers: patterns, function and productivity

This investigation proposes a corpus-based description of a particular type of Italian phraseological construction, having a coordinated syntactic structure and playing the role of an intensifier. Through the analysis of data extracted from a corpus of Italian, the contribution aims at i) identifying and analysing different types of coordinated constructional intensifiers, ii) showing a formal representation of their syntactic-semantic schemas, iii) classifying them into different types on the basis of their functional, syntactic and semantic properties. The paper is structured as follows: the first section contains a brief description of the main types of intensifying strategies identified in the literature, deserving a special attention to Italian lexical strategies. The second section is devoted to the description of the general theoretical framework of this contribution, and to the analysis and definition of constructional intensifiers. The third section briefly describes the methods of data extraction and analysis. The fourth section is devoted to the analysis of Italian coordinated intensifying constructions, which are classified into completely and partially filled units, according to a set of parameters (i.e. lexical variation, syntactic cohesion, idiomaticity and lexicalisation). Then, the set of coordinated constructional intensifiers is evaluated against the presence of an abstract scheme, semantic and syntactic analysability, semantic predictability, productivity, and schematicity.
Grounded Graph Decoding improves Compositional Generalization in Question Answering

Question answering models struggle to generalize to novel compositions of training patterns, such to longer sequences or more complex test structures.Current end-to-end models learn a flat input embedding which can lose input syntax context.Prior approaches improve generalization by learning permutation invariant models, but these methods do not scale to more complex train-test splits.We propose Grounded Graph Decoding, a method to improve compositional generalization of language representations by grounding structured predictions with an attention mechanism.Grounding enables the model to retain syntax information from the input in thereby significantly improving generalization over complex inputs.By predicting a structured graph containing conjunctions of query clauses, we learn a group invariant representation without making assumptions on the target domain.Our model significantly outperforms state-ofthe-art baselines on the Compositional Freebase Questions (CFQ) dataset, a challenging benchmark for compositional generalization in question answering.Moreover, we effectively solve the MCD1 split with 98% accuracy.
Outcome-Driven Reinforcement Learning via Variational Inference

While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it. In this paper, we discuss a new perspective on reinforcement learning, recasting it as the problem of inferring actions that achieve desired outcomes, rather than a problem of maximizing rewards. To solve the resulting outcome-directed inference problem, we establish a novel variational inference formulation that allows us to derive a well-shaped reward function which can be learned directly from environment interactions. From the corresponding variational objective, we also derive a new probabilistic Bellman backup operator reminiscent of the standard Bellman backup operator and use it to develop an off-policy algorithm to solve goal-directed tasks. We empirically demonstrate that this method eliminates the need to design reward functions and leads to effective goal-directed behaviors.
Deep Kernels with Probabilistic Embeddings for Small-Data Learning

Gaussian Processes (GPs) are known to provide accurate predictions and uncertainty estimates even with small amounts of labeled data by capturing similarity between data points through their kernel function. However traditional GP kernels are not very effective at capturing similarity between high dimensional data points. Neural networks can be used to learn good representations that encode intricate structures in high dimensional data, and can be used as inputs to the GP kernel. However the huge data requirement of neural networks makes this approach ineffective in small data settings. To solves the conflicting problems of representation learning and data efficiency, we propose to learn deep kernels on probabilistic embeddings by using a probabilistic neural network. Our approach maps high-dimensional data to a probability distribution in a low dimensional subspace and then computes a kernel between these distributions to capture similarity. To enable end-to-end learning, we derive a functional gradient descent procedure for training the model. Experiments on a variety of datasets show that our approach outperforms the state-of-the-art in GP kernel learning in both supervised and semi-supervised settings. We also extend our approach to other small-data paradigms such as few-shot classification where it outperforms previous approaches on mini-Imagenet and CUB datasets.
Style transfer with target feature palette and attention coloring

Style transfer has attracted a lot of attentions, as it can change a given image into one with splendid artistic styles while preserving the image structure. However, conventional approaches easily lose image details and tend to produce unpleasant artifacts during style transfer. In this paper, to solve these problems, a novel artistic stylization method with target feature palettes is proposed, which can transfer key features accurately. Specifically, our method contains two modules, namely feature palette composition (FPC) and attention coloring (AC) modules. The FPC module captures representative features based on K-means clustering and produces a feature target palette. The following AC module calculates attention maps between content and style images, and transfers colors and patterns based on the attention map and the target palette. These modules enable the proposed stylization to focus on key features and generate plausibly transferred images. Thus, the contributions of the proposed method are to propose a novel deep learning-based style transfer method and present target feature palette and attention coloring modules, and provide in-depth analysis and insight on the proposed method via exhaustive ablation study. Qualitative and quantitative results show that our stylized images exhibit state-of-the-art performance, with strength in preserving core structures and details of the content image.
Compositional Generalization via Semantic Tagging

Although neural sequence-to-sequence models have been successfully applied to semantic parsing, they fail at compositional generalization, i.e., they are unable to systematically generalize to unseen compositions of seen components. Motivated by traditional semantic parsing where compositionality is explicitly accounted for by symbolic grammars, we propose a new decoding framework that preserves the expressivity and generality of sequence-to-sequence models while featuring lexicon-style alignments and disentangled information processing. Specifically, we decompose decoding into two phases where an input utterance is first tagged with semantic symbols representing the meaning of individual words, and then a sequence-to-sequence model is used to predict the final meaning representation conditioning on the utterance and the predicted tag sequence. Experimental results on three semantic parsing datasets show that the proposed approach consistently improves compositional generalization across model architectures, domains, and semantic formalisms.
Policy Optimization in Adversarial MDPs: Improved Exploration via Dilated Bonuses.

Policy optimization is a widely-used method in reinforcement learning. Due to its local-search nature, however, theoretical guarantees on global optimality often rely on extra assumptions on the Markov Decision Processes (MDPs) that bypass the challenge of global exploration. To eliminate the need of such assumptions, in this work, we develop a general solution that adds dilated bonuses to the policy update to facilitate global exploration. To showcase the power and generality of this technique, we apply it to several episodic MDP settings with adversarial losses and bandit feedback, improving and generalizing the state-of-the-art. Specifically, in the tabular case, we obtain $\widetilde{\mathcal{O}}(\sqrt{T})$ regret where $T$ is the number of episodes, improving the $\widetilde{\mathcal{O}}({T}^{2/3})$ regret bound by Shani et al. (2020). When the number of states is infinite, under the assumption that the state-action values are linear in some low-dimensional features, we obtain $\widetilde{\mathcal{O}}({T}^{2/3})$ regret with the help of a simulator, matching the result of Neu and Olkhovskaya (2020) while importantly removing the need of an exploratory policy that their algorithm requires. When a simulator is unavailable, we further consider a linear MDP setting and obtain $\widetilde{\mathcal{O}}({T}^{14/15})$ regret, which is the first result for linear MDPs with adversarial losses and bandit feedback.
Deep Learning-Based Engraving Segmentation of 3-D Inscriptions Extracted From the Rough Surface of Ancient Stelae

Ancient stelae are considered important historical sources. However, it is a challenge to recognize the inscriptions carved on stelae that have rough surfaces due to prolonged weathering. In this paper, we propose a deep learning-based method to extract engraved regions from the 3D scanned mesh of a stela. First, the uneven distribution of vertices in the mesh is transformed using a mesh subdivision method such that the vertices in the mesh are uniformly distributed. Then, surface features (depth, concave features, and local surface features) are extracted from the subdivided mesh. The depth represents the basic shape of the mesh and is obtained from the aligned mesh. The concave features effectively represent concave regions by using a Frangi filter, and the local surface features have the spin image technique applied to describe the fine shapes of neighboring vertices relative to a vertex. The mesh and the surface features are rasterized into feature images, and engraved regions are segmented from the feature images by using a FC-DenseNet. Our experiments confirm that the proposed method effectively extracts engraved regions of the inscriptions from the rough surface of a stela and it shows robustness to noisy and extremely abraded characters. The proposed method outperformed the second-best method, obtaining an F1 score, IoU, and SIRI of approximately 2.95%, 3.65%, and 7.53%, respectively.
Mono and Multi-Lingual Machine Translation Using Deep Attention-based Models

Machine translation (MT) is the automatic translation of natural language texts. The complexities and incompatibilities of natural languages make MT an arduous task facing several challenges, especially when it is to be compared to a human translation. Neural Machine Translation (NMT) has to make MT results closer to human expectations with the advent of deep-learning artificial intelligence. The newest deep learning approaches are based on Recurrent Neural Networks (RNN), transformers, complex convolutions, and employing encoder/decoder pairs. In this work, we propose a new attention-based encoder-decoder model with monolingual and multilingual for MT. The Training has been several models with single languages and one model with several languages on both of our long short-term memory (LSTM) architecture and Transformer. We show that the Transformer outperforms the LSTM within our specific neural machine translation task. These models are evaluated using IWSLT2016 datasets, which contain a training dataset for three languages, test2015 and test2016 dataset for testing. These experiments show a 93.9% accuracy, which we can estimate as a 5 BLEU point improvement over the previous studies. (metric used in MT).
TacticZero: Learning to Prove Theorems from Scratch with Deep Reinforcement Learning

We propose a novel approach to interactive theorem-proving (ITP) using deep reinforcement learning. The proposed framework is able to learn proof search strategies as well as tactic and arguments prediction in an end-to-end manner. We formulate the process of ITP as a Markov decision process (MDP) in which each state represents a set of potential derivation paths. This structure allows us to introduce a novel backtracking mechanism which enables the agent to efficiently discard (predicted) dead-end derivations and restart from promising alternatives. We implement the framework in the HOL4 theorem prover. Experimental results show that the framework outperforms existing automated theorem provers (i.e., hammers) available in HOL4 when evaluated on unseen problems. We further elaborate the role of key components of the framework using ablation studies.
Partition and Code: learning how to compress graphs

Can we use machine learning to compress graph data? The absence of ordering in graphs poses a significant challenge to conventional compression algorithms, limiting their attainable gains as well as their ability to discover relevant patterns. On the other hand, most graph compression approaches rely on domain-dependent handcrafted representations and cannot adapt to different underlying graph distributions. This work aims to establish the necessary principles a lossless graph compression method should follow to approach the entropy storage lower bound. Instead of making rigid assumptions about the graph distribution, we formulate the compressor as a probabilistic model that can be learned from data and generalise to unseen instances. Our Partition and Code framework entails three steps: first, a partitioning algorithm decomposes the graph into elementary structures, then these are mapped to the elements of a small dictionary on which we learn a probability distribution, and finally, an entropy encoder translates the representation into bits. All three steps are parametric and can be trained with gradient descent. We theoretically compare the compression quality of several graph encodings and prove, under mild conditions, a total ordering of their expected description lengths. Moreover, we show that, under the same conditions, PnC achieves compression gains w.r.t. the baselines that grow either linearly or quadratically with the number of vertices. Our algorithms are quantitatively evaluated on diverse real-world networks obtaining significant performance improvements with respect to different families of non-parametric and parametric graph compressors.
Virtual property: trusts of cryptocurrencies and other digital assets

Abstract The question whether digital assets such as cryptocurrencies are “property” as a matter of law has been the subject of academic debate almost since the inception of the bitcoin network in 2009. The question arises from the fact that such assets, and cryptocurrencies in particular, do not fit into traditional categories of property as understood by the common law, being neither “choses in possession” nor “choses in action”. However, following a series of decisions in the last couple of years, it now seems reasonably settled that this question will be answered in the affirmative, although no cases have so far reached the highest level of the courts. The consequences have yet to be worked out. One obvious consequence is that cryptocurrencies can be the subject of a trust. However, the unique features of digital assets give rise to a number of interesting problems, both theoretical and practical, on which there is little authoritative guidance.
Sequentially Utility Maximizing Path Planning Using a Distributed Pool Architecture

Abstract Operation of ground vehicles in remote unstructured terrains is a challenging task. While information on terrain parameters such as elevation, slope, and soil content and its use in tasks such as path planning and vehicle simulation is immensely useful, it requires detailed analyses that are computationally expensive and time-consuming. At the same time, higher-level decisions made without a full understanding of the terrain properties and vehicle capabilities are bound to be suboptimal, possibly even leading to mission failure. In this paper, we approach the problem from a decision-making standpoint to address both the aspects described above. The approach divides a given terrain into smaller cells and progressively finds the path to the goal. At each step, the operator’s utility between competing immediate terrain cells is compared so that the vehicle chooses the best next step considering detailed vehicle simulation. To account for the increased computational effort, a pool-based distributed computing architecture is employed to speed-up pathfinding and provide resilience towards failure of processors and communication links. The pool architecture also admits heterogeneous, geographically distributed processors and storage locations. Concurrently, the multiattribute utility formulation takes into account the tradeoff between different attributes and the uncertainty in the vehicle performance. The proposed method can be used for medium and long-range navigation in unstructured environments for efficient rough path planning or re-planning based on changing mission requirements and dynamically arriving terrain data. The method is demonstrated on data acquired from USGS sources.
Learning Hard Optimization Problems: A Data Generation Perspective

Optimization problems are ubiquitous in our societies and are present in almost every segment of the economy. Most of these optimization problems are NP-hard and computationally demanding, often requiring approximate solutions for large-scale instances. Machine learning frameworks that learn to approximate solutions to such hard optimization problems are a potentially promising avenue to address these difficulties, particularly when many closely related problem instances must be solved repeatedly. Supervised learning frameworks can train a model using the outputs of pre-solved instances. However, when the outputs are themselves approximations, when the optimization problem has symmetric solutions, and/or when the solver uses randomization, solutions to closely related instances may exhibit large differences and the learning task can become inherently more difficult. This paper demonstrates this critical challenge, connects the volatility of the training data to the ability of a model to approximate it, and proposes a method for producing (exact or approximate) solutions to optimization problems that are more amenable to supervised learning tasks. The effectiveness of the method is tested on hard non-linear nonconvex and discrete combinatorial problems.