Reviewerly/exHarmony huggingface dataset:
Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling

Boundary information is critical for various Chinese language processing tasks, such as word segmentation, part-of-speech tagging, and named entity recognition.Previous studies usually resorted to the use of a high-quality external lexicon, where lexicon items can offer explicit boundary information.However, to ensure the quality of the lexicon, great human effort is always necessary, which has been generally ignored.In this work, we suggest unsupervised statistical boundary information instead, and propose an architecture to encode the information directly into pre-trained language models, resulting in Boundary-Aware BERT (BABERT).We apply BABERT for feature induction of Chinese sequence labeling tasks.Experimental results on ten benchmarks of Chinese sequence labeling demonstrate that BABERT can provide consistent improvements on all datasets.In addition, our method can complement previous supervised lexicon exploration, where further improvements can be achieved when integrated with external lexicon information.
MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text

While language Models store a massive amount of world knowledge implicitly in their parameters, even very large models often fail to encode information about rare entities and events, while incurring huge computational costs.Recently, retrieval-augmented models, such as REALM, RAG, and RETRO, have incorporated world knowledge into language generation by leveraging an external nonparametric index and have demonstrated impressive performance with constrained model sizes.However, these methods are restricted to retrieving only textual knowledge, neglecting the ubiquitous amount of knowledge in other modalities like images -much of which contains information not covered by any text.To address this limitation, we propose the first Multimodal Retrieval-Augmented Transformer (MuRAG), which accesses an external non-parametric multimodal memory to augment language generation.MuRAG is pretrained with a mixture of large-scale imagetext and text-only corpora using a joint contrastive and generative loss.We perform experiments on two different datasets that require retrieving and reasoning over both images and text to answer a given query: We-bQA, and MultimodalQA.Our results show that MuRAG achieves state-of-the-art accuracy, outperforming existing models by 10-20% absolute on both datasets and under both distractor and full-wiki settings.
DIGAT: Modeling News Recommendation with Dual-Graph Interaction

News recommendation (NR) is essential for online news services.Existing NR methods typically adopt a news-user representation learning framework, facing two potential limitations.First, in news encoder, single candidate news encoding suffers from an insufficient semantic information problem.Second, existing graphbased NR methods are promising but lack effective news-user feature interaction, rendering the graph-based recommendation suboptimal.To overcome these limitations, we propose dualinteractive graph attention networks (DIGAT) consisting of news-and user-graph channels.In the news-graph channel, we enrich the semantics of single candidate news by incorporating the semantically relevant news information with a semantic-augmented graph (SAG).In the user-graph channel, multi-level user interests are represented with a news-topic graph.Most notably, we design a dual-graph interaction process to perform effective feature interaction between the news and user graphs, which facilitates accurate news-user representation matching.Experiment results on the benchmark dataset MIND show that DIGAT outperforms existing news recommendation methods 1 .Further ablation studies and analyses validate the effectiveness of (i) semantic-augmented news graph modeling and (ii) dual-graph interaction.
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding

Prompt Tuning has been largely successful as a parameter-efficient method of conditioning large-scale pre-trained language models to perform downstream tasks.Thus far, soft prompt tuning learns a fixed set of task-specific continuous vectors, i.e., soft tokens that remain static across the task samples.A fixed prompt, however, may not generalize well to the diverse kinds of inputs the task comprises.In order to address this, we propose Vector-quantized Input-contextualized Prompts (VIP) 1 as an extension to the soft prompt tuning framework.VIP particularly focuses on two aspectscontextual prompts that learns input-specific contextualization of the soft prompt tokens through a small-scale sentence encoder and quantized prompts that maps the contextualized prompts to a set of learnable codebook vectors through a Vector quantization network.On various language understanding tasks like SuperGLUE, QA, Relation classification, NER and NLI, VIP outperforms the soft prompt tuning (PT) baseline by an average margin of 1.19%.Further, our generalization studies show that VIP learns more robust prompt representations, surpassing PT by a margin of 0.6% -5.3% on Out-of-domain QA and NLI tasks respectively, and by 0.75% on Multi-Task setup over 4 tasks spanning across 12 domains.
Extractive Summarization of Legal Decisions using Multi-task Learning and Maximal Marginal Relevance

Summarizing legal decisions requires the expertise of law practitioners, which is both timeand cost-intensive.This paper presents techniques for extractive summarization of legal decisions in a low-resource setting using limited expert annotated data.We test a set of models that locate relevant content using a sequential model and tackle redundancy by leveraging maximal marginal relevance to compose summaries.We also demonstrate an implicit approach to help train our proposed models generate more informative summaries.Our multi-task learning model variant leverages rhetorical role identification as an auxiliary task to further improve the summarizer.We perform extensive experiments on datasets containing legal decisions from the US Board of Veterans' Appeals and conduct quantitative and expert-ranked evaluations of our models.Our results show that the proposed approaches can achieve ROUGE scores vis-à-vis expert extracted summaries that match those achieved by inter-annotator comparison.
HiSMatch: Historical Structure Matching based Temporal Knowledge Graph Reasoning

A Temporal Knowledge Graph (TKG) is a sequence of KGs with respective timestamps, which adopts quadruples in the form of (subject, relation, object, timestamp) to describe dynamic facts.TKG reasoning has facilitated many real-world applications via answering such queries as (query entity, query relation, ?, future timestamp) about future.This is actually a matching task between a query and candidate entities based on their historical structures, which reflect behavioral trends of the entities at different timestamps.In addition, recent KGs provide background knowledge of all the entities, which is also helpful for the matching.Thus, in this paper, we propose the Historical Structure Matching (HiSMatch) model.It applies two structure encoders to capture the semantic information contained in the historical structures of the query and candidate entities.Besides, it adopts another encoder to integrate the background knowledge into the model.TKG reasoning experiments on six benchmark datasets demonstrate the significant improvement of the proposed HiSMatch model, with up to 5.6% performance improvement in MRR, compared to the state-of-the-art baselines.
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack

We present Twin Answer Sentences Attack (TASA), an adversarial attack method for question answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers.Despite phenomenal progress on general adversarial attacks, few works have investigated the vulnerability and attack specifically for QA models.In this work, we first explore the biases in the existing models and discover that they mainly rely on keyword matching between the question and context, and ignore the relevant contextual relations for answer prediction.Based on two biases above, TASA attacks the target model in two folds: (1) lowering the model's confidence on the gold answer with a perturbed answer sentence; (2) misguiding the model towards a wrong answer with a distracting answer sentence.Equipped with designed beam search and filtering methods, TASA can generate more effective attacks than existing textual attack methods while sustaining the quality of contexts, in extensive experiments on five QA datasets and human evaluations.
Commonsense Knowledge Salience Evaluation with a Benchmark Dataset in E-commerce

In e-commerce, the salience of commonsense knowledge (CSK) is beneficial for widespread applications such as product search and recommendation.For example, when users search for "running" in e-commerce, they would like to find products highly related to running, such as "running shoes" rather than "shoes".Nevertheless, many existing CSK collections rank statements solely by confidence scores, and there is no information about which ones are salient from a human perspective.In this work, we define the task of supervised salience evaluation, where given a CSK triple, the model is required to learn whether the triple is salient or not.In addition to formulating the new task, we also release a new Benchmark dataset of Salience Evaluation in E-commerce (BSEE) and hope to promote related research on commonsense knowledge salience evaluation.We conduct experiments in the dataset with several representative baseline models.The experimental results show that salience evaluation is a challenging task where models perform poorly on our evaluation set.We further propose a simple but effective approach, PMI-tuning, which shows promise for solving this novel problem 1 .
Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mapping

The many-to-many multilingual neural machine translation can translate between language pairs unseen during training, i.e., zeroshot translation.Improving zero-shot translation requires the model to learn universal representations and cross-mapping relationships to transfer the knowledge learned on the supervised directions to the zero-shot directions.In this work, we propose the state mover's distance based on the optimal theory to model the difference of the representations output by the encoder.Then, we bridge the gap between the semantic-equivalent representations of different languages at the token level by minimizing the proposed distance to learn universal representations.Besides, we propose an agreement-based training scheme, which can help the model make consistent predictions based on the semantic-equivalent sentences to learn universal cross-mapping relationships for all translation directions.The experimental results on diverse multilingual datasets show that our method can improve consistently compared with the baseline system and other contrast methods.The analysis proves that our method can better align the semantic space and improve the prediction consistency.
Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation

Non-autoregressive models achieve significant decoding speedup in neural machine translation but lack the ability to capture sequential dependency.Directed Acyclic Transformer (DA-Transformer) was recently proposed to model sequential dependency with a directed acyclic graph.Consequently, it has to apply a sequential decision process at inference time, which harms the global translation accuracy.In this paper, we present a Viterbi decoding framework for DA-Transformer, which guarantees to find the joint optimal solution for the translation and decoding path under any length constraint.Experimental results demonstrate that our approach consistently improves the performance of DA-Transformer while maintaining a similar decoding speedup.
Virtual Machine Load Balancing Using Improved ABC for Task Scheduling in Cloud Computing

Computing resources are now more accessible, powerful, and inexpensive than ever before thanks to the widespread adoption of the Internet and rapid advancements in processing and storage technology. Thanks to this development in expertise, a new kind of computing architecture known as "cloud computing" is now a practical reality. Load balancing task scheduling is an essential issue that has a direct impact on resource utilisation in cloud settings. One of the most important goals of scheduling is to distribute work among virtual machines in such a way that no individual machine is overburdened or underutilised. Considering load balancing scheduling is crucial because of the importance it has on the backend and frontend of the cloud research sector. Any time a cloud environment is able to effectively balance its load, resource utilisation improves. The challenge with load balancing in the cloud is that it is an NP-hard optimization delinquent. To solve this issue, this research suggests a method that combines a heuristic scheduling algorithm with an artificial bee colony; this method is referred to as enhanced Artificial Bee Colony for Task Scheduling (IABC-TS). For better cloud computing scheduling of virtual machines in both homogeneous and heterogeneous contexts, this approach is used. The goal of its implementation was to shorten makespan and distribute weight more evenly. The IABC-TS cloud computing system's scheduling performance was measured against that of competing swarm intelligence techniques. In our lab tests, we used CloudSim to simulate systems with various augmentation techniques to examine their makespan and load balancing ability. Based on the results of the experiments, it is clear that the suggested IABC-TS outperformed the baseline models.
The limitations of machine learning models for predicting scientific replicability

Errors in clinical decision-making are disturbingly common. Here, we show that structured information–sharing networks among clinicians significantly reduce diagnostic errors, and improve treatment recommendations, as compared to groups of ...Errors in clinical decision-making are disturbingly common. Recent studies have found that 10 to 15% of all clinical decisions regarding diagnoses and treatment are inaccurate. Here, we experimentally study the ability of structured information–sharing ...
A Semantic Approach for Extracting the Medical Association Rules

Healthcare data analytics has indispensable responsibility in advancing remedial decision-making and healthcare practices. In this paper, a novel semantic approach is introduced for extracting medical association rules, designed at discovering expressive interactions between medical things. Our methodology incorporates medical ontologies, and improved semantic measures to boost the accuracy and interpretability of the obtained rules. Our method delivers clinically pertinent information into patient perspectives on mental health, the accessibility of mental health resources, and the influence of physical health on mental well-being by mapping medical notions to ontological items and examining semantic relationships. Experimental validation on a case study of mental health dataset proves the dominance of our method over traditional association rule mining methods. The proposed semantic method presents valuable support to healthcare data analysis and decision-making processes.
A Hybrid Modified ABC-PSO Algorithm for Optimal Robotic Path Planner

Path planning is one of the most fundamental problems that must be dealt with before the mobile robot can navigate and explore autonomously in any environment. A good path-planning algorithm can save time and reduce mobile robot wear and capital investment. Path computing time and average path length are important factors over cost functions that reflect the algorithm's effectiveness, such as power consumption or average trip time. The Artificial Bee Colony (ABC) represents one of the most important global search algorithms. The main problem with ABC is that it suffers from a slow convergence rate due to lousy exploitation and tends to get trapped in the local minima. This paper proposes and evaluates a new robot path-planning algorithm named Modified Artificial Bee Colony (MABC). MABC algorithm design is based on modifying the ABC algorithm by cross-layer design between ABC and Particle Swarm Optimization (PSO) algorithms. The MABC is different from the original ABC algorithm in that it modifies the original one to use PSO's exploitation rather than its exploitation. On the other hand, the PSO algorithm has better exploitation but poor exploration characteristics. The evaluation and analysis were performed for several performance metrics and under different evaluation scenarios. It has been observed from the results that the MABC-PSO outperforms the original ABC with respect to average path length and convergence time which leads to improving the planning of the path.
Cross-Cultural Emotion Recognition With EEG and Eye Movement Signals Based on Multiple Stacked Broad Learning System

With increasing social globalization, interaction between people from different cultures has become more frequent. However, there are significant differences in the expression and comprehension of emotions across cultures. Therefore, developing computational models that can accurately identify emotions among different cultures has become a significant research problem. This study aims to investigate the similarities and differences in emotion cognition processes in different cultural groups by employing a fusion of electroencephalography (EEG) and eye movement (EM) signals. Specifically, an effective adaptive region selection method is proposed to investigate the most emotion-related activated brain regions in different groups. By selecting these commonly activated regions, we can eliminate redundant features and facilitate the development of portable acquisition devices. Subsequently, the multiple stacked broad learning system (MSBLS) is designed to explore the complementary information of EEG and EM features and the effective emotional information still contained in the residual value. The intracultural subject-dependent (ICSD), intracultural subject-independent, and cross-cultural subject-independent (CCSI) experiments have been conducted on the SEED-CHN, SEED-GER, and SEED-FRA datasets. Extensive experiments manifest that MSBLS achieves superior performance compared with current state-of-the-art methods. Moreover, we discover that some brain regions (the anterior frontal, temporal, and middle parieto-occipital lobes) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Gamma</i> frequency bands show greater activation during emotion cognition in diverse cultural groups.
Publicly Verifiable Zero-Knowledge and Post-Quantum Signatures from VOLE-in-the-Head

We present a new method for transforming zero-knowledge protocols in the designated verifier setting into public-coin protocols, which can be made non-interactive and publicly verifiable. Our transformation applies to a large class of ZK protocols based on oblivious transfer. In particular, we show that it can be applied to recent, fast protocols based on vector oblivious linear evaluation (VOLE), with a technique we call VOLE-in-the-head, upgrading these protocols to support public verifiability. Our resulting ZK protocols have linear proof size, and are simpler, smaller and faster than related approaches based on MPC-in-the-head. To build VOLE-in-the-head while supporting both binary circuits and large finite fields, we develop several new technical tools. One of these is a new proof of security for the SoftSpokenOT protocol (Crypto 2022), which generalizes it to produce certain types of VOLE correlations over large fields. Secondly, we present a new ZK protocol that is tailored to take advantage of this form of VOLE, which leads to a publicly verifiable VOLE-in-the-head protocol with only 2x more communication than the best, designated-verifier VOLE-based protocols. We analyze the soundness of our approach when made non-interactive using the Fiat-Shamir transform, using round-by-round soundness. As an application of the resulting NIZK, we present $$\textsf{FAEST}$$ , a post-quantum signature scheme based on AES. FAEST is the first AES-based signature scheme to be smaller than SPHINCS+, with signature sizes between 5.6 and 6.6kB at the 128-bit security level. Compared with the smallest version of SPHINCS+ (7.9kB), FAEST verification is slower, but the signing times are between 8x and 40x faster.
Evolutionary Verbalizer Search for Prompt-Based Few Shot Text Classification

Recent advances for few-shot text classification aim to wrap textual inputs with task-specific prompts to cloze questions. By processing them with a masked language model to predict the masked tokens and using a verbalizer that constructs the mapping between predicted words and target labels. This approach of using pre-trained language models is called prompt-based tuning, which could remarkably outperform conventional fine-tuning approach in the low-data scenario. As the core of prompt-based tuning, the verbalizer is usually handcrafted with human efforts or suboptimally searched by gradient descent. In this paper, we focus on automatically constructing the optimal verbalizer and propose a novel evolutionary verbalizer search (EVS) algorithm, to improve prompt-based tuning with the high-performance verbalizer. Specifically, inspired by evolutionary algorithm (EA), we utilize it to automatically evolve various verbalizers during the evolutionary procedure and select the best one after several iterations. Extensive few-shot experiments on five text classification datasets show the effectiveness of our method.
Revisiting Time-Space Tradeoffs for Function Inversion

We study the black-box function inversion problem, which is the problem of finding $$x \in [N]$$ such that $$f(x) = y$$ , given as input some challenge point y in the image of a function $$f : [N] \rightarrow [N]$$ , using T oracle queries to f and preprocessed advice $$\sigma \in \{0,1\}^S$$ depending on f. We prove a number of new results about this problem, as follows. All of the above results are most naturally described in a model with shared randomness (i.e., random coins shared between the preprocessing algorithm and the online algorithm). However, as an additional contribution, we show (using a technique from communication complexity due to Newman [IPL, 1991]) how to generically convert any algorithm that uses shared randomness into one that does not.
Multiple Data Augmentation Strategy for Enhancing the Performance of YOLOv7 Object Detection Algorithm

The object detection technique depends on various methods for duplicating the dataset without adding more images. Data augmentation is a popular method that assists deep neural networks in achieving better generalization performance and can be seen as a type of implicit regularization. This method is recommended in the case where the amount of high-quality data is limited, and gaining new examples is costly and time-consuming. In this paper, we trained YOLOv7 with a dataset that is part of the Open Images dataset that has 8,600 images with four classes (Car, Bus, Motorcycle, and Person). We used five different data augmentations techniques for duplicates and improvement of our dataset. The performance of the object detection algorithm was compared when using the proposed augmented dataset with a combination of two and three types of data augmentation with the result of the original data. The evaluation result for the augmented data gives a promising result for every object, and every kind of data augmentation gives a different improvement. The mAP@.5 of all classes was 76%, and F1-score was 74%. The proposed method increased the mAP@.5 value by +13% and F1-score by +10% for all objects.