Reviewerly/exHarmony huggingface dataset:
Constructing a Precise Fuzzy Feedforward Neural Network Using an Independent Fuzzification Approach

This study discusses how to fuzzify a feedforward neural network (FNN) to generate a fuzzy forecast that contains the actual value, while minimizing the average range of fuzzy forecasts. This topic has rarely been investigated in past studies, but is an essential step to constructing a precise fuzzy FNN (FFNN). Existing methods fuzzify all parameters at the same time, which re-sults in a nonlinear programming (NLP) problem that is not easy to solve. In contrast, in this study, the parameters of a FNN are fuzzified independently. In this way, the optimal values of fuzzy parameters can be derived theoretically. An illustrative example is used to illustrate the ap-plicability of the proposed methodology. According to the experimental results, fuzzifying the thresholds on hidden-layer nodes or the connection weights between input and hidden layers may not guarantee that all fuzzy forecasts contain the corresponding actual values. In contrast, fuzzi-fying the threshold on the output node and the connection weights between the hidden and out-put layers is more likely to achieve a 100% hit rate. The results lay a foundation for establishing a precise deep FFNN in the future.
MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks

We propose a novel way to incorporate expert knowledge into the training of deep neural networks. Many approaches encode domain constraints directly into the network architecture, requiring non-trivial or domain-specific engineering. In contrast, our approach, called MultiplexNet, represents domain knowledge as a quantifier-free logical formula in disjunctive normal form (DNF) which is easy to encode and to elicit from human experts. It introduces a latent Categorical variable that learns to choose which constraint term optimizes the error function of the network and it compiles the constraints directly into the output of existing learning algorithms. We demonstrate the efficacy of this approach empirically on several classical deep learning tasks, such as density estimation and classification in both supervised and unsupervised settings where prior knowledge about the domains was expressed as logical constraints. Our results show that the MultiplexNet approach learned to approximate unknown distributions well, often requiring fewer data samples than the alternative approaches. In some cases, MultiplexNet finds better solutions than the baselines; or solutions that could not be achieved with the alternative approaches. Our contribution is in encoding domain knowledge in a way that facilitates inference. We specifically focus on quantifier-free logical formulae that are specified over the output domain of a network. We show that this approach is both efficient and general; and critically, our approach guarantees 100% constraint satisfaction in a network's output.
A High-Density EEG Study Investigating VR Film Editing and Cognitive Event Segmentation Theory

This paper introduces a cognitive psychological experiment that was conducted to analyze how traditional film editing methods and the application of cognitive event segmentation theory perform in virtual reality (VR). Thirty volunteers were recruited and asked to watch a series of short VR videos designed in three dimensions: time, action (characters), and space. Electroencephalograms (EEG) were recorded simultaneously during their participation. Subjective results show that any of the editing methods used would lead to an increased load and reduced immersion. Furthermore, the cognition of event segmentation theory also plays an instructive role in VR editing, with differences mainly focusing on frontal, parietal, and central regions. On this basis, visual evoked potential (VEP) analysis was performed, and the standardized low-resolution brain electromagnetic tomography algorithm (sLORETA) traceability method was used to analyze the data. The results of the VEP analysis suggest that shearing usually elicits a late event-related potential component, while the sources of VEP are mainly the frontal and parietal lobes. The insights derived from this work can be used as guidance for VR content creation, allowing VR image editing to reveal greater richness and unique beauty.
A Simple and Effective Positional Encoding for Transformers

Transformer models are permutation equivariant.To supply the order and type information of the input tokens, position and segment embeddings are usually added to the input.Recent works proposed variations of positional encodings with relative position encodings achieving better performance.Our analysis shows that the gain actually comes from moving positional information to attention layer from the input.Motivated by this, we introduce Decoupled posItional attEntion for Transformers (DIET), a simple yet effective mechanism to encode position and segment information into the Transformer models.The proposed method has faster training and inference time, while achieving competitive performance on GLUE, XTREME and WMT benchmarks.We further generalize our method to long-range transformers and show performance gain.
Dynamical system analysis of a data-driven model constructed by reservoir computing

This study evaluates data-driven models from a dynamical system perspective, such as unstable fixed points, periodic orbits, chaotic saddle, Lyapunov exponents, manifold structures, and statistical values. We find that these dynamical characteristics can be reconstructed much more precisely by a data-driven model than by computing directly from training data. With this idea, we predict the laminar lasting time distribution of a particular macroscopic variable of chaotic fluid flow, which cannot be calculated from a direct numerical simulation of the Navier-Stokes equation because of its high computational cost.
Applying Agile Methodology in Mobile Software Engineering: Android Application Development and its Challenges

Applying Agile Methodology in Mobile Software Engineering: Android Application Development and its Challenges
Graph neural network initialisation of quantum approximate optimisation

Approximate combinatorial optimisation has emerged as one of the most promising application areas for quantum computers, particularly those in the near term. In this work, we focus on the quantum approximate optimisation algorithm (QAOA) for solving the MaxCut problem. Specifically, we address two problems in the QAOA, how to initialise the algorithm, and how to subsequently train the parameters to find an optimal solution. For the former, we propose graph neural networks (GNNs) as a warm-starting technique for QAOA. We demonstrate that merging GNNs with QAOA can outperform both approaches individually. Furthermore, we demonstrate how graph neural networks enables warm-start generalisation across not only graph instances, but also to increasing graph sizes, a feature not straightforwardly available to other warm-starting methods. For training the QAOA, we test several optimisers for the MaxCut problem up to 16 qubits and benchmark against vanilla gradient descent. These include quantum aware/agnostic and machine learning based/neural optimisers. Examples of the latter include reinforcement and meta-learning. With the incorporation of these initialisation and optimisation toolkits, we demonstrate how the optimisation problems can be solved using QAOA in an end-to-end differentiable pipeline.
Controlling hallucinations at word level in data-to-text generation

Abstract Data-to-Text Generation (DTG) is a subfield of Natural Language Generation aiming at transcribing structured data in natural language descriptions. The field has been recently boosted by the use of neural-based generators which exhibit on one side great syntactic skills without the need of hand-crafted pipelines; on the other side, the quality of the generated text reflects the quality of the training data, which in realistic settings only offer imperfectly aligned structure-text pairs. Consequently, state-of-art neural models include misleading statements –usually called hallucinations—in their outputs. The control of this phenomenon is today a major challenge for DTG, and is the problem addressed in the paper. Previous work deal with this issue at the instance level: using an alignment score for each table-reference pair. In contrast, we propose a finer-grained approach, arguing that hallucinations should rather be treated at the word level. Specifically, we propose a Multi-Branch Decoder which is able to leverage word-level labels to learn the relevant parts of each training instance. These labels are obtained following a simple and efficient scoring procedure based on co-occurrence analysis and dependency parsing. Extensive evaluations, via automated metrics and human judgment on the standard WikiBio benchmark, show the accuracy of our alignment labels and the effectiveness of the proposed Multi-Branch Decoder. Our model is able to reduce and control hallucinations, while keeping fluency and coherence in generated texts. Further experiments on a degraded version of ToTTo show that our model could be successfully used on very noisy settings.
Multi-Task and Multi-Modal Learning for RGB Dynamic Gesture Recognition

Gesture recognition is getting more and more popular due to various application possibilities in human-machine interaction. Existing multi-modal gesture recognition systems take multi-modal data as input to improve accuracy, but such methods require more modality sensors, which will greatly limit their application scenarios. Therefore we propose an end-to-end multi-task learning framework in training 2D convolutional neural networks. The framework can use the depth modality to improve accuracy during training and save costs by using only RGB modality during inference. Our framework is trained to learn a representation for multi-task learning: gesture segmentation and gesture recognition. Depth modality contains the prior information for the location of the gesture. Therefore it can be used as the supervision for gesture segmentation. A plug-and-play module named Multi-Scale-Decoder is designed to realize gesture segmentation, which contains two sub-decoder. It is used in the lower stage and higher stage respectively, and can help the network pay attention to key target areas, ignore irrelevant information, and extract more discriminant features. Additionally, the MSD module and depth modality are only used in the training stage to improve gesture recognition performance. Only RGB modality and network without MSD are required during inference. Experimental results on three public gesture recognition datasets show that our proposed method provides superior performance compared with existing gesture recognition frameworks. Moreover, using the proposed plug-and-play MSD in other 2D CNN-based frameworks also get an excellent accuracy improvement.
Experimental relativistic zero-knowledge proofs

Protecting secrets is a key challenge in our contemporary information-based era. In common situations, however, revealing secrets appears unavoidable; for instance, when identifying oneself in a bank to retrieve money. In turn, this may have highly undesirable consequences in the unlikely, yet not unrealistic, case where the bank's security gets compromised. This naturally raises the question of whether disclosing secrets is fundamentally necessary for identifying oneself, or more generally for proving a statement to be correct. Developments in computer science provide an elegant solution via the concept of zero-knowledge proofs: a prover can convince a verifier of the validity of a certain statement without facilitating the elaboration of a proof at all1. In this work, we report the experimental realization of such a zero-knowledge protocol involving two separated verifier–prover pairs2. Security is enforced via the physical principle of special relativity3, and no computational assumption (such as the existence of one-way functions) is required. Our implementation exclusively relies on off-the-shelf equipment and works at both short (60 m) and long distances (≥400 m) in about one second. This demonstrates the practical potential of multi-prover zero-knowledge protocols, promising for identification tasks and blockchain applications such as cryptocurrencies or smart contracts4. A zero-knowledge proof, which can be used to verify secret information, is reported here with security that is enforced by the laws of special relativity.
Designing a Block Cipher in Galois Extension Fields for IoT Security

This paper focuses on a block cipher adaptation of the Galois Extension Fields (GEF) combination technique for PRNGs and targets application in the Internet of Things (IoT) space, an area where the combination technique was concluded as a quality stream cipher. Electronic Codebook (ECB) and Cipher Feedback (CFB) variations of the cryptographic algorithm are discussed. Both modes offer computationally efficient, scalable cryptographic algorithms for use over a simple combination technique like XOR. The cryptographic algorithm relies on the use of quality PRNGs, but adds an additional layer of security while preserving maximal entropy and near-uniform distributions. The use of matrices with entries drawn from a Galois field extends this technique to block size chunks of plaintext, increasing diffusion, while only requiring linear operations that are quick to perform. The process of calculating the inverse differs only in using the modular inverse of the determinant, but this can be expedited by a look-up table. We validate this GEF block cipher with the NIST test suite. Additional statistical tests indicate the condensed plaintext results in a near-uniform distributed ciphertext across the entire field. The block cipher implemented on an MSP430 offers a faster, more power-efficient alternative to the Advanced Encryption Standard (AES) system. This cryptosystem is a secure, scalable option for IoT devices that must be mindful of time and power consumption.
An analytic layer-wise deep learning framework with applications to robotics

Deep learning (DL) has achieved great success in many applications, but it has been less well analyzed from the theoretical perspective. The unexplainable success of black-box DL models has raised questions among scientists and promoted the emergence of the field of explainable artificial intelligence (XAI). In robotics, it is particularly important to deploy DL algorithms in a predictable and stable manner as robots are active agents that need to interact safely with the physical world. This paper presents an analytic deep learning framework for fully connected neural networks, which can be applied for both regression problems and classification problems. Examples for regression and classification problems include online robot control and robot vision. We present two layer-wise learning algorithms such that the convergence of the learning systems can be analyzed. Firstly, an inverse layer-wise learning algorithm for multilayer networks with convergence analysis for each layer is presented to understand the problems of layer-wise deep learning. Secondly, a forward progressive learning algorithm where the deep networks are built progressively by using single hidden layer networks is developed to achieve better accuracy. It is shown that the progressive learning method can be used for fine-tuning of weights from convergence point of view. The effectiveness of the proposed framework is illustrated based on classical benchmark recognition tasks using the MNIST and CIFAR-10 datasets and the results show a good balance between performance and explainability. The proposed method is subsequently applied for online learning of robot kinematics and experimental results on kinematic control of UR5e robot with unknown model are presented.
Exploring Sentence Community for Document-Level Event Extraction

Document-level event extraction is critical to various natural language processing tasks for providing structured information.Existing approaches by sequential modeling neglect the complex logic structures for long texts.In this paper, we leverage the entity interactions and sentence interactions within long documents, and transform each document into an undirected unweighted graph by exploiting the relationship between sentences.We introduce the Sentence Community to represent each event as a subgraph.Furthermore, our framework SCDEE maintains the ability to extract multiple events by sentence community detection using graph attention networks and alleviate the role overlapping issue by predicting arguments in terms of roles.Experiments demonstrate that our framework achieves competitive results over state-of-the-art methods on the large-scale document-level event extraction dataset.
Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering

In this paper we propose a novel approach towards improving the efficiency of Question Answering (QA) systems by filtering out questions that will not be answered by them. This is based on an interesting new finding: the answer confidence scores of state-of-the-art QA systems can be approximated well by models solely using the input question text. This enables preemptive filtering of questions that are not answered by the system due to their answer confidence scores being lower than the system threshold. Specifically, we learn Transformer-based question models by distilling Transformer-based answering models. Our experiments on three popular QA datasets and one industrial QA benchmark demonstrate the ability of our question models to approximate the Precision/Recall curves of the target QA system well. These question models, when used as filters, can effectively trade off lower computation cost of QA systems for lower Recall, e.g., reducing computation by ~60%, while only losing ~3-4% of Recall.
Few-Shot Self-Rationalization with Natural Language Prompts

Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems.These models are, however, currently trained with a large amount of human-written free-text explanations for each task which hinders their broader usage.We propose to study a more realistic setting of self-rationalization using few training examples.We present FEB-a standardized collection of four existing English-language datasets and associated metrics.We identify the right prompting approach by extensively exploring natural language prompts on FEB.Then, by using this prompt and scaling the model size, we demonstrate that making progress on few-shot self-rationalization is possible.We show there is still ample room for improvement in this task: the average plausibility of generated explanations assessed by human annotators is at most 51% (with GPT-3), while plausibility of human explanations is 76%.We hope that FEB and our proposed approach will spur the community to take on the few-shot self-rationalization challenge.
Modeling Document-Level Context for Event Detection via Important Context Selection

Event Detection (ED) aims to recognize and classify trigger words of events in text.The recent progress has featured advanced transformer-based language models (e.g., BERT) as a critical component in stateof-the-art models for ED.However, the length limit for input texts is a barrier for such ED models as they cannot encode long-range document-level context that has been shown to be beneficial for ED.To address this issue, we propose a novel method to model documentlevel context with BERT for ED that dynamically selects relevant sentences in the document for the event prediction of the target sentence.The target sentence will be then augmented with the selected sentences and consumed entirely by BERT for improved representation learning for ED.To this end, the RE-INFORCE algorithm is employed to train the relevant sentence selection for ED.Several information types are then introduced to form the reward function for the training process, including ED performance, sentence similarity, and discourse relations.Our extensive experiments on multiple benchmark datasets reveal the effectiveness of the proposed model, leading to new state-of-the-art performance.
A Thorough Evaluation of Task-Specific Pretraining for Summarization

Task-agnostic pretraining objectives like masked language models or corrupted span prediction are applicable to a wide range of NLP downstream tasks (Raffel et al.,2019), but are outperformed by task-specific pretraining objectives like predicting extracted gap sentences on summarization (Zhang et al.,2020). We compare three summarization specific pretraining objectives with the task agnostic corrupted span prediction pretraining in controlled study. We also extend our study to a low resource and zero shot setup, to understand how many training examples are needed in order to ablate the task-specific pretraining without quality loss. Our results show that task-agnostic pretraining is sufficient for most cases which hopefully reduces the need for costly task-specific pretraining. We also report new state-of-the-art number for two summarization task using a T5 model with 11 billion parameters and an optimal beam search length penalty.
Success of digital adiabatic simulation with large Trotter step

The simulation of adiabatic evolution has deep connections with adiabatic quantum computation, the quantum approximate optimization algorithm, and adiabatic state preparation. Here we address the error analysis problem in quantum simulation of adiabatic process using Trotter formulas. We show that with additional conditions, the circuit depth can be linear in simulation time $T$. The improvement comes from the observation that the fidelity error here can't be estimated by the norm distance between evolution operators. This phenomenon is termed the robustness of discretization in digital adiabatic simulation. It can be explained in three steps, from analytical and numerical evidence: (1) The fidelity error should be estimated by applying adiabatic theorem on the effective Hamiltonian instead. (2) Because of the specialty of Riemann-Lebesgue lemma, most adiabatic process is naturally robust against discretization. (3) As the Trotter step gets larger, the spectral gap of effective Hamiltonian tends to close, which results in the failure of digital adiabatic simulation.
Comparative Analysis of Performance between Multimodal Implementation of Chatbot Based on News Classification Data Using Categories

In the modern era, the implementation of chatbot can be used in various fields of science. This research will focus on the application of sentence classification using the News Aggregator Dataset that is used to test the model against the categories determined to create the chatbot program. The results of the chatbot program trial by multimodal implementation applied four models (GRU, Bi-GRU, 1D CNN, 1D CNN Transpose) with six variations of parameters to produce the best results from the entire trial. The best test results from this research for the chatbot program using the 1D CNN Transpose model are the best models with detailed characteristics in this research, which produces an accuracy value of 0.9919. The test results on both types of chatbot are expected to produce sentence prediction results and precise and accurate detection results. The stages in making the program are explained in detail; therefore, it is hoped that program users can understand not only how to use the program by entering an input and receiving program output results that are explained in more detail in each sub-topic of this study.