Reviewerly/exHarmony huggingface dataset:
Link Prediction on N-ary Relational Facts: A Graph-based Approach

Link prediction on knowledge graphs (KGs) is a key research topic.Previous work mainly focused on binary relations, paying less attention to higher-arity relations although they are ubiquitous in real-world KGs.This paper considers link prediction upon n-ary relational facts and proposes a graph-based approach to this task.The key to our approach is to represent the nary structure of a fact as a small heterogeneous graph, and model this graph with edge-biased fully-connected attention.The fully-connected attention captures universal inter-vertex interactions, while with edge-aware attentive biases to particularly encode the graph structure and its heterogeneity.In this fashion, our approach fully models global and local dependencies in each n-ary fact, and hence can more effectively capture associations therein.Extensive evaluation verifies the effectiveness and superiority of our approach.It performs substantially and consistently better than current state-of-the-art across a variety of n-ary relational benchmarks.Our code is publicly available.
Evolving neural architecture using one shot model

Previous evolution based architecture search require high computational resources resulting in large search time. In this work, we propose a novel way of applying a simple genetic algorithm to the neural architecture search problem called EvNAS (Evolving Neural Architecture using One Shot Model) which reduces the search time significantly while still achieving better result than previous evolution based methods. The architectures are represented by architecture parameter of one shot model which results in the weight sharing among the given population of architectures and also weight inheritance from one generation to the next generation of architectures. We use the accuracy of partially trained architecture on validation data as a prediction of its fitness to reduce the search time. We also propose a decoding technique for the architecture parameter which is used to divert majority of the gradient information towards the given architecture and is also used for improving the fitness prediction of the given architecture from the one shot model during the search process. EvNAS searches for architecture on CIFAR-10 for 3.83 GPU day on a single GPU with top-1 test error 2.47%, which is then transferred to CIFAR-100 and ImageNet achieving top-1 error 16.37% and top-5 error 7.4% respectively.
PhotoChat: A Human-Human Dialogue Dataset With Photo Sharing Behavior For Joint Image-Text Modeling

We present a new human-human dialogue dataset -PhotoChat, the first dataset that casts light on the photo sharing behavior in online messaging.PhotoChat contains 12k dialogues, each of which is paired with a user photo that is shared during the conversation.Based on this dataset, we propose two tasks to facilitate research on image-text modeling: a photo-sharing intent prediction task that predicts whether one intends to share a photo in the next conversation turn, and a photo retrieval task that retrieves the most relevant photo according to the dialogue context.In addition, for both tasks, we provide baseline models using the state-of-the-art models and report their benchmark performances.The best image retrieval model achieves 10.4% recall@1 (out of 1000 candidates) and the best photo intent prediction model achieves 58.1% F1 score, indicating that the dataset presents interesting yet challenging real-world problems.We are releasing PhotoChat to facilitate future research work among the community.
Towards Visual Question Answering on Pathology Images

Pathology imaging is broadly used for identifying the causes and effects of diseases or injuries.Given a pathology image, being able to answer questions about the clinical findings contained in the image is very important for medical decision making.In this paper, we aim to develop a pathological visual question answering framework to analyze pathology images and answer medical questions related to these images.To build such a framework, we create PathVQA, a pathology VQA dataset with 32,795 questions asked from 4,998 pathology images.We also propose a three-level optimization framework which performs self-supervised pretraining and VQA finetuning end-to-end to learn powerful visual and textual representations jointly and automatically identifies and excludes noisy self-supervised examples from pretraining.We perform experiments on our created PathVQA dataset and the results demonstrate the effectiveness of our proposed methods.The datasets and code are available at https://github.com/UCSD-AI4H/PathVQA
DeepRobust: a Platform for Adversarial Attacks and Defenses

DeepRobust is a PyTorch platform for generating adversarial examples and building robust machine learning models for different data domains. Users can easily evaluate the attack performance against different defense methods with DeepRobust and get performance analyzing visualization. In this paper, we introduce the functions of DeepRobust with detailed instructions. We believe that DeepRobust is a useful tool to measure deep learning model robustness and to find the suitable countermeasures against adversarial attacks. The platform is kept updated and can be found at https://github.com/DSE-MSU/DeepRobust. More details of instruction can be found in the documentation at https://deeprobust.readthedocs.io/en/latest/.
POI recommendation method using LSTM-attention in LBSN considering privacy protection

Abstract Aiming at the problems of traditional point of interest (POI), such as sparse data, lack of negative feedback, and dynamic and periodic changes of user preferences, a POI recommendation method using deep learning in location-based social networks (LBSN) considering privacy protection is proposed. First, the idea of Embedding is used to quantify the user information, friend relationship, POI information, and so on, so as to obtain the internal relationship of the location. Then, based on the user's history and current POI check-in sequence set, the long- and short-term attention mechanism (LSA) is constructed, and the quantified information is used as the input of LSA to better capture the user's long-term and short-term preferences. Finally, the social network information and semantic information are fitted in different input layers, and the time and geographical location information of user's historical behavior are used to recommend the next POI for users. Gowalla and Brightkite datasets are used to demonstrate the proposed method. The results show that the performance of the proposed method is better than other comparison methods under different sparsity, location sequence length, and embedding length. When the number of iterations is 500, the recommended method tends to be stable, and the accuracy is 0.27. Moreover, the recommendation time of the proposed method is less than 130 ms, which is better than other comparative deep learning methods.
Calculating nonadiabatic couplings and Berry's phase by variational quantum eigensolvers

The variational quantum eigensolver (VQE) is an algorithm to find eigenenergies and eigenstates of systems in quantum chemistry and quantum many-body physics. The VQE is one of the most promising applications of near-term quantum devices to investigate such systems. Here we propose an extension of the VQE to calculate the nonadiabatic couplings of molecules in quantum chemical systems and Berry's phase in quantum many-body systems. Both quantities play an important role in understanding the properties of a system beyond the naive adiabatic picture, e.g., nonadiabatic dynamics and topological phase of matter. We provide quantum circuits and classical postprocessings to calculate the nonadiabatic couplings and Berry's phase. Specifically, we show that the evaluation of the nonadiabatic couplings reduces to that of expectation values of observables while that of Berry's phase also requires one additional Hadamard test. Furthermore, we simulate the photodissociation dynamics of a lithium fluoride molecule using the nonadiabatic couplings evaluated on a real quantum device. Our proposal widens the applicability of the VQE and the possibility of near-term quantum devices to study molecules and quantum many-body systems.1 MoreReceived 6 October 2020Accepted 20 May 2021DOI:https://doi.org/10.1103/PhysRevResearch.3.023244Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasChemical kinetics, dynamics & catalysisQuantum algorithmsQuantum computationTechniquesVariational approachInterdisciplinary PhysicsCondensed Matter & Materials PhysicsQuantum Information
Design of a Low-Overhead Random Number Generator Using CMOS-based Cascaded Chaotic Maps

We present a cascaded chaotic system as a hardware-efficient way of elevating the entropy in the chaotic behavior of CMOS-based chaotic maps. The chaotic performance of the proposed scheme is evaluated using the bifurcation plot, Lyapunov exponent, Kolmogorov entropy, and correlation coefficient. The improved entropy in the chaotic region benefits many security applications and is demonstrated experimentally in a new random number generator (RNG) design based on the proposed map. Unlike conventional mathematical chaotic map-based digital pseudo-random number generators (PRNG), the proposed design is not completely deterministic due to the high susceptibility of the core analog circuit to inevitable noise which renders this design closer to a true random number generator (TRNG). By leveraging the improved chaotic performance of the transistor-level cascaded map, significantly low area and power overhead are achieved in the RNG design and it passes three statistical tests namely, NIST, Diehard, and TestU01.
Zooid: a DSL for certified multiparty computation: from mechanised metatheory to certified multiparty processes

We design and implement Zooid, a domain specific language for certified multiparty communication, embedded in Coq and implemented atop our mechanisation framework of asynchronous multiparty session types (the first of its kind). Zooid provides a fully mechanised metatheory for the semantics of global and local types, and a fully verified end-point process language that faithfully reflects the type-level behaviours and thus inherits the global types properties such as deadlock freedom, protocol compliance, and liveness guarantees.
LeeBERT: Learned Early Exit for BERT with cross-level optimization

Pre-trained language models like BERT are performant in a wide range of natural language tasks.However, they are resource exhaustive and computationally expensive for industrial scenarios.Thus, early exits are adopted at each layer of BERT to perform adaptive computation by predicting easier samples with the first few layers to speed up the inference.In this work, to improve efficiency without performance drop, we propose a novel training scheme called Learned Early Exiting for BERT (LeeBERT).First, we ask each exit to learn from each other, rather than learning only from the last layer.Second, the weights of different loss terms are learned, thus balancing off different objectives.We formulate the optimization of LeeBERT as a bi-level optimization problem, and we propose a novel cross-level optimization (CLO) algorithm to improve the optimization results.Experiments on the GLUE benchmark show that our proposed methods improve the performance of the state-of-the-art (SOTA) early exiting methods for pre-trained models.
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search

Over the past half-decade, many methods have been considered for neural architecture search (NAS). Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very promising strategy for NAS when it is coupled with a neural predictor. Recent work has proposed different instantiations of this framework, for example, using Bayesian neural networks or graph convolutional networks as the predictive model within BO. However, the analyses in these papers often focus on the full-fledged NAS algorithm, so it is difficult to tell which individual components of the framework lead to the best performance. In this work, we give a thorough analysis of the BO + neural predictor framework by identifying five main components: the architecture encoding, neural predictor, uncertainty calibration method, acquisition function, and acquisition optimization strategy. We test several different methods for each component and also develop a novel path-based encoding scheme for neural architectures, which we show theoretically and empirically scales better than other encodings. Using all of our analyses, we develop a final algorithm called BANANAS, which achieves state-of-the-art performance on NAS search spaces. We adhere to the NAS research checklist (Lindauer and Hutter 2019) to facilitate best practices, and our code is available at this https URL.
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability

Fine-tuning large pre-trained models with taskspecific data has achieved great success in NLP.However, it has been demonstrated that the majority of information within the selfattention networks is redundant and not utilized effectively during the fine-tuning stage.This leads to inferior results when generalizing the obtained models to out-of-domain distributions.To this end, we propose a simple yet effective data augmentation technique, HiddenCut, to better regularize the model and encourage it to learn more generalizable features.Specifically, contiguous spans within the hidden space are dynamically and strategically dropped during training.Experiments show that our HiddenCut method outperforms the state-of-the-art augmentation methods on the GLUE benchmark, and consistently exhibits superior generalization performances on out-of-distribution and challenging counterexamples.We have publicly released our code at https://github.com/GT-SALT/HiddenCut.
Imagine, Reason and Write: Visual Storytelling with Graph Knowledge and Relational Reasoning

Visual storytelling is a task of creating a short story based on photo streams. Different from visual captions, stories contain not only factual descriptions, but also imaginary concepts that do not appear in the images. In this paper, we propose a novel imagine-reason-write generation framework (IRW) for visual storytelling, inspired by the logic of humans when they write the story. First, an imagine module is leveraged to learn the imaginative storyline explicitly, improving the coherence and reasonability of the generated story. Second, we employ a reason module to fully exploit the external knowledge (commonsense knowledge base) and task-specific knowledge (scene graph and event graph) with relational reasoning method based on the storyline. In this way, we can effectively capture the most informative commonsense and visual relationships among objects in images, which enhances the diversity and informativeness of the generated story. Finally, we integrate the imaginary concepts and relational knowledge to generate human-like story based on the original semantics of images. Extensive experiments on a benchmark dataset (i.e., VIST) demonstrate that the proposed IRW framework significantly outperforms the state-of-the-art methods across multiple evaluation metrics.
Evaluating the Quality of Internet-Based Education in Colleges Using the Regression Algorithm

An efficient and effective mechanism is required for quality education in schools, colleges, and universities. After the pandemic worldwide, every organization worked online from home instead of on the site. The educational institutes started online classes using various platforms. The quality of education using online classes is affected by numerous factors, including hardware, software, and the internet. Therefore, in this paper, the quality evaluation of internet education using a regression algorithm is proposed. The proposed system optimizes the hardware design and functions of the client end of the internet education of colleges and universities. It determines the time response data sequence of the grey prediction model and ensures that the proposed scheme’s prediction accuracy is high. By adopting the logistic regression, the rules for the overall evaluation of the education quality are formulated. The database of the rules for the overall evaluation of the education quality of the internet of colleges and universities is generated. Experimental results show that the response time of the education quality evaluation system is about 1.3 sec. The probability of failure is very low than the existing literature.
PENS: A Dataset and Generic Framework for Personalized News Headline Generation

In this paper, we formulate the personalized news headline generation problem whose goal is to output a user-specific title based on both a user's reading interests and a candidate news body to be exposed to her.To build up a benchmark for this problem, we publicize a large-scale dataset named PENS (PErsonalized News headlineS).The training set is collected from user impressions logs of Microsoft News, and the test set is manually created by hundreds of native speakers to enable a fair testbed for evaluating models in an offline mode.We propose a generic framework as a preparatory solution to our problem.At its heart, user preference is learned by leveraging the user behavioral data, and three kinds of user preference injections are proposed to personalize a text generator and establish personalized headlines.We investigate our dataset by implementing several state-of-the-art user modeling methods in our framework to demonstrate a benchmark score for the proposed dataset.
A graph neural network to model disruption in human-aware robot navigation

Abstract Autonomous navigation is a key skill for assistive and service robots. To be successful, robots have to minimise the disruption caused to humans while moving. This implies predicting how people will move and complying with social conventions. Avoiding disrupting personal spaces, people’s paths and interactions are examples of these social conventions. This paper leverages Graph Neural Networks to model robot disruption considering the movement of the humans and the robot so that the model built can be used by path planning algorithms. Along with the model, this paper presents an evolution of the dataset SocNav1 (Manso et al 2020) which considers the movement of the robot and the humans, and an updated scenario-to-graph transformation which is tested using different Graph Neural Network blocks. The model trained achieves close-to-human performance in the dataset. In addition to its accuracy, the main advantage of the approach is its scalability in terms of the number of social factors that can be considered in comparison with handcrafted models. The dataset and the model are available in a public repository ( https://github.com/gnns4hri/sngnnv2 ).
Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning

We present a mean-variance policy iteration (MVPI) framework for risk-averse control in a discounted infinite horizon MDP optimizing the variance of a per-step reward random variable. MVPI enjoys great flexibility in that any policy evaluation method and risk-neutral control method can be dropped in for risk-averse control off the shelf, in both on- and off-policy settings. This flexibility reduces the gap between risk-neutral control and risk-averse control and is achieved by working on a novel augmented MDP directly. We propose risk-averse TD3 as an example instantiating MVPI, which outperforms vanilla TD3 and many previous risk-averse control methods in challenging Mujoco robot simulation tasks under a risk-aware performance metric. This risk-averse TD3 is the first to introduce deterministic policies and off-policy learning into risk-averse reinforcement learning, both of which are key to the performance boost we show in Mujoco domains.
Learning to Segment Actions from Visual and Language Instructions via Differentiable Weak Sequence Alignment

We address the problem of unsupervised localization of task-relevant actions (key-steps) and feature learning in instructional videos using both visual and language instructions. Our key observation is that the sequences of visual and linguistic key-steps are weakly aligned: there is an ordered one-to-one correspondence between most visual and language key-steps, while some key-steps in one modality are absent in the other. To recover the two sequences, we develop an ordered prototype learning module, which extracts visual and linguistic prototypes representing key-steps. To find weak alignment and perform feature learning, we develop a differentiable weak sequence alignment (DWSA) method that finds ordered one-to-one matching between sequences while allowing some items in a sequence to stay unmatched. We develop an efficient forward and backward algorithm for computing the alignment and the loss derivative with respect to parameters of visual and language feature learning modules. By experiments on two instructional video datasets, we show that our method significantly improves the state of the art.
Encoding Syntactic Knowledge in Transformer Encoder for Intent Detection and Slot Filling

We propose a novel Transformer encoder-based architecture with syntactical knowledge encoded for intent detection and slot filling. Specifically, we encode syntactic knowledge into the Transformer encoder by jointly training it to predict syntactic parse ancestors and part-of-speech of each token via multi-task learning. Our model is based on self-attention and feed-forward layers and does not require external syntactic information to be available at inference time. Experiments show that on two benchmark datasets, our models with only two Transformer encoder layers achieve state-of-the-art results. Compared to the previously best performed model without pre-training, our models achieve absolute F1 score and accuracy improvement of 1.59 % and 0.85 % for slot filling and intent detection on the SNIPS dataset, respectively. Our models also achieve absolute F1 score and accuracy improvement of 0.1 % and 0.34 % for slot filling and intent detection on the ATIS dataset, respectively, over the previously best performed model. Furthermore, the visualization of the self-attention weights illustrates the benefits of incorporating syntactic information during training.