Reviewerly/exHarmony huggingface dataset:
Cost Volume Pyramid Based Depth Inference for Multi-View Stereo

We propose a cost volume-based neural network for depth inference from multi-view images. We demonstrate that building a cost volume pyramid in a coarse-to-fine manner instead of constructing a cost volume at a fixed resolution leads to a compact, lightweight network and allows us inferring high resolution depth maps to achieve better reconstruction results. To this end, we first build a cost volume based on uniform sampling of fronto-parallel planes across the entire depth range at the coarsest resolution of an image. Then, given current depth estimate, we construct new cost volumes iteratively to perform depth map refinement. We show that working on cost volume pyramid can lead to a more compact, yet efficient network structure compared with existing works. We further show that the (residual) depth sampling can be fully determined by analytical geometric derivation, which serves as a principle for building compact cost volume pyramid. To demonstrate the effectiveness of our proposed framework, we extend our cost volume pyramid structure to handle the unsupervised depth inference scenario. Experimental results on benchmark datasets show that our model can perform 6x faster with similar performance as state-of-the-art methods for supervised scenario and demonstrates superior performance on unsupervised scenario. Code is available at https://github.com/JiayuYANG/CVP-MVSNet.
Differentiable Open-Ended Commonsense Reasoning

Current commonsense reasoning research focuses on developing models that use commonsense knowledge to answer multiple-choice questions.However, systems designed to answer multiple-choice questions may not be useful in applications that do not provide a small list of candidate answers to choose from.As a step towards making commonsense reasoning research more realistic and useful, we propose to study open-ended commonsense reasoning (OpenCSR) -the task of answering a commonsense question without any predefined choices -using as a resource only a knowledge corpus of commonsense facts written in natural language.OpenCSR is challenging due to a large decision space, and because many questions require implicit multi-hop reasoning.As an approach to OpenCSR, we propose DRFACT, an efficient Differentiable model for multi-hop Reasoning over knowledge Facts.To evaluate OpenCSR methods, we adapt three popular multiple-choice datasets, and collect multiple new answers to each test question via crowd-sourcing.Experiments show that DRFACT outperforms strong baseline methods by a large margin. 1
Refining Targeted Syntactic Evaluation of Language Models

Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb's conjugation.The method evaluates whether language models rate each grammatical sentence as more likely than its ungrammatical counterpart.We identify two distinct goals for TSE.First, evaluating the systematicity of a language model's syntactic knowledge: given a sentence, can it conjugate arbitrary verbs correctly?Second, evaluating a model's likely behavior: given a sentence, does the model concentrate its probability mass on correctly conjugated verbs, even if only on a subset of the possible verbs?We argue that current implementations of TSE do not directly capture either of these goals, and propose new metrics to capture each goal separately.Under our metrics, we find that TSE overestimates systematicity of language models, but that models score up to 40% better on verbs that they predict are likely in context.
A Predictive Text System for Medical Recommendations in Telemedicine: A Deep Learning Approach in the Arabic Context

We are currently witnessing an immense proliferation of natural language processing (NLP) applications. Natural language generation (NLG) has emerged from NLP and is now commonly utilized in various applications, including chatting applications. The objective of this paper is to propose a deep learning-based language generation model that simplifies the process of writing medical recommendations for doctors in an Arabic context, to improve service satisfaction and patient-doctor interactions. The developed language generation model is a predictive text system intended for next word prediction in a telemedicine service. Altibbi-a digital platform for telemedicine and teleconsultations services in the Middle East and the North Africa (MENA) region-was utilized as a case study for the textual prediction process. The proposed model was trained using data obtained from Altibbi databases related to medical recommendations, particularly gynecology, dermatology, psychiatric diseases, urology, and internist diseases. Variants of deep learning models were implemented and optimized for next word prediction, based on the unidirectional and bidirectional long short-term memory (LSTM and BiLSTM), the one-dimensional convolutional neural network (CONV1D), and a combination of LSTM and CONV1D (LSTM-CONV1D). The algorithms were trained using two versions of the datasets (i.e., 3-gram and 4-gram representations) and evaluated in terms of their training accuracy and loss, validation accuracy and loss, and testing accuracy per their matching scores. The proposed models' performances were comparable. CONV1D produced the most promising matching score.
A Software/Hardware Co-Design of Crystals-Dilithium Signature Scheme

As quantum computers become more affordable and commonplace, existing security systems that are based on classical cryptographic primitives, such as RSA and Elliptic Curve Cryptography ( ECC ), will no longer be secure. Hence, there has been interest in designing post-quantum cryptographic ( PQC ) schemes, such as those based on lattice-based cryptography ( LBC ). The potential of LBC schemes is evidenced by the number of such schemes passing the selection of NIST PQC Standardization Process Round-3. One such scheme is the Crystals-Dilithium signature scheme, which is based on the hard module-lattice problem. However, there is no efficient implementation of the Crystals-Dilithium signature scheme. Hence, in this article, we present a compact hardware architecture containing elaborate modular multiplication units using the Karatsuba algorithm along with smart generators of address sequence and twiddle factors for NTT, which can complete polynomial addition/multiplication with the parameter setting of Dilithium in a short clock period. Also, we propose a fast software/hardware co-design implementation on Field Programmable Gate Array ( FPGA ) for the Dilithium scheme with a tradeoff between speed and resource utilization. Our co-design implementation outperforms a pure C implementation on a Nios-II processor of the platform Altera DE2-115, in the sense that our implementation is 11.2 and 7.4 times faster for signature and verification, respectively. In addition, we also achieve approximately 51% and 31% speed improvement for signature and verification, in comparison to the pure C implementation on processor ARM Cortex-A9 of ZYNQ-7020 platform.
Entropy-Based Logic Explanations of Neural Networks

Explainable artificial intelligence has rapidly emerged since lawmakers have started requiring interpretable models for safety-critical domains. Concept-based neural networks have arisen as explainable-by-design methods as they leverage human-understandable symbols (i.e. concepts) to predict class memberships. However, most of these approaches focus on the identification of the most relevant concepts but do not provide concise, formal explanations of how such concepts are leveraged by the classifier to make predictions. In this paper, we propose a novel end-to-end differentiable approach enabling the extraction of logic explanations from neural networks using the formalism of First-Order Logic. The method relies on an entropy-based criterion which automatically identifies the most relevant concepts. We consider four different case studies to demonstrate that: (i) this entropy-based criterion enables the distillation of concise logic explanations in safety-critical domains from clinical data to computer vision; (ii) the proposed approach outperforms state-of-the-art white-box models in terms of classification accuracy.
Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation

A conventional approach to improving the performance of end-to-end speech translation (E2E-ST) models is to leverage the source transcription via pre-training and joint training with automatic speech recognition (ASR) and neural machine translation (NMT) tasks.However, since the input modalities are different, it is difficult to leverage source language text successfully.In this work, we focus on sequencelevel knowledge distillation (SeqKD) from external text-based NMT models.To leverage the full potential of the source language information, we propose backward SeqKD, SeqKD from a target-to-source backward NMT model.To this end, we train a bilingual E2E-ST model to predict paraphrased transcriptions as an auxiliary task with a single decoder.The paraphrases are generated from the translations in bitext via back-translation.We further propose bidirectional SeqKD in which SeqKD from both forward and backward NMT models is combined.Experimental evaluations on both autoregressive and non-autoregressive models show that SeqKD in each direction consistently improves the translation performance, and the effectiveness is complementary regardless of the model capacity.
Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization

Natural Language Inference (NLI) has garnered significant attention in recent years; however, the promise of applying NLI breakthroughs to other downstream NLP tasks has remained unfulfilled.In this work, we use the multiple-choice reading comprehension (MCRC) and checking factual correctness of textual summarization (CFCS) tasks to investigate potential reasons for this.Our findings show that: (1) the relatively shorter length of premises in traditional NLI datasets is the primary challenge prohibiting usage in downstream applications (which do better with longer contexts); (2) this challenge can be addressed by automatically converting resource-rich reading comprehension datasets into longer-premise NLI datasets; and (3) models trained on the converted, longer-premise datasets outperform those trained using shortpremise traditional NLI datasets on downstream tasks primarily due to the difference in premise lengths.
Understanding Hard Negatives in Noise Contrastive Estimation

The choice of negative examples is important in noise contrastive estimation.Recent works find that hard negatives-highest-scoring incorrect examples under the model-are effective in practice, but they are used without a formal justification.We develop analytical tools to understand the role of hard negatives.Specifically, we view the contrastive loss as a biased estimator of the gradient of the crossentropy loss, and show both theoretically and empirically that setting the negative distribution to be the model distribution results in bias reduction.We also derive a general form of the score function that unifies various architectures used in text retrieval.By combining hard negatives with appropriate score functions, we obtain strong results on the challenging task of zero-shot entity linking.
Temporal Reasoning on Implicit Events from Distant Supervision

We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit eventsevents that are not mentioned explicitly in natural language text but can be inferred from it.This introduces a new challenge in temporal reasoning research, where prior work has focused on explicitly mentioned events.Human readers can infer implicit events via commonsense reasoning, resulting in a more comprehensive understanding of the situation and, consequently, better reasoning about time.We find, however, that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events.To address this, we propose a neuro-symbolic temporal reasoning model, SYMTIME, which exploits distant supervision signals from largescale text and uses temporal rules to combine start times and durations to infer end times.SYMTIME outperforms strong baseline systems on TRACIE by 5%, and by 11% in a zero prior knowledge training setting.Our approach also generalizes to other temporal reasoning tasks, as evidenced by a gain of 1%-9% on MATRES, an explicit event benchmark.
Application of photovoltaic power generation in rail transit power supply system under the background of energy low carbon transformation

Low carbon economy, energy conservation and environmental protection is one of the important tasks of current and future economic and social development. The large-scale development and utilization of all kinds of clean energy has accelerated the speed of China's energy transformation. Rail transit system is a large power consumer. In recent years, the transportation system has been facing the triangle contradiction of new energy, cost and environmental protection. Connecting photovoltaic power generation to rail transit power supply system has many advantages: (1) it can reduce the operation cost of transportation system; (2) it can reduce the use of traditional thermal power; (3) it can reduce carbon emissions and protect the environment; (4) it can also promote the application of new energy. It makes a lot of sense. However, due to the randomness and uncertainty of photovoltaic power generation, the direct access of photovoltaic power generation to rail transit power supply system will bring a certain impact on rail transit power supply system. In this paper, the LSTM neural network is used to predict the load of photovoltaic power generation, which effectively ensures the accuracy of prediction, and then improves the stability of photovoltaic power generation when connected to the rail transit power supply system.
Herding behaviour in cryptocurrency and emerging financial markets

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Continual Learning for Text Classification with Information Disentanglement Based Regularization

Continual learning has become increasingly important as it enables NLP models to constantly learn and gain knowledge over time.Previous continual learning methods are mainly designed to preserve knowledge from previous tasks, without much emphasis on how to well generalize models to new tasks.In this work, we propose an information disentanglement based regularization method for continual learning on text classification.Our proposed method first disentangles text hidden spaces into representations that are generic to all tasks and representations specific to each individual task, and further regularizes these representations differently to better constrain the knowledge required to generalize.We also introduce two simple auxiliary tasks: next sentence prediction and task-id prediction, for learning better generic and specific representation spaces.Experiments conducted on large-scale benchmarks demonstrate the effectiveness of our method in continual text classification tasks with various sequences and lengths over state-of-the-art baselines.We have publicly released our code at https: //github.com/GT-SALT/IDBR.
Controls of a superconducting quantum parametron under a strong pump field

Abstract Pumped at approximately twice the natural frequency, a Josephson parametric oscillator called parametron or Kerr parametric oscillator shows self-oscillation. Quantum annealing and universal quantum computation using self-oscillating parametrons as qubits were proposed. However, controls of parametrons under the pump field are degraded by unwanted rapidly oscillating terms in the Hamiltonian, which we call non-resonant rapidly oscillating terms (NROTs) coming from the violation of the rotating wave approximation. Therefore, the pump field can be an intrinsic origin of the imperfection of controls of parametrons. Here, we theoretically study the influence of the NROTs on the accuracy of controls of a parametron: a cat-state creation and a single-qubit gate. It is shown that there is a trade-off relationship between the suppression of the nonadiabatic transitions and the validity of the rotating wave approximation in a conventional approach. We also show that the tailored time dependence of the detuning of the pump field can suppress both of the nonadiabatic transitions and the disturbance of the state of the parametron due to the NROTs.
Augmenting Knowledge-grounded Conversations with Sequential Knowledge Transition

Knowledge data are massive and widespread in the real-world, which can serve as good external sources to enrich conversations.However, in knowledge-grounded conversations, current models still lack the fine-grained control over knowledge selection and integration with dialogues, which finally leads to the knowledge-irrelevant response generation problems: 1) knowledge selection merely relies on the dialogue context, ignoring the inherent knowledge transitions along with conversation flows; 2) the models often over-fit during training, resulting with incoherent response by referring to unrelated tokens from specific knowledge content in the testing phase; 3) although response is generated upon the dialogue history and knowledge, the models often tend to overlook the selected knowledge, and hence generates knowledge-irrelevant response.To address these problems, we proposed to explicitly model the knowledge transition in sequential multi-turn conversations by abstracting knowledge into topic tags.Besides, to fully utilizing the selected knowledge in generative process, we propose pretraining a knowledge-aware response generator to pay more attention on the selected knowledge.In particular, a sequential knowledge transition model equipped with a pretrained knowledge-aware response generator (SKT-KG) formulates the high-level knowledge transition and fully utilizes the limited knowledge data.Experimental results on both structured and unstructured knowledgegrounded dialogue benchmarks indicate that our model achieves better performance over baseline models.
Review of Propulsion System Design Strategies for Unmanned Aerial Vehicles

The design of the propulsion system for Unmanned Aerial Vehicles (UAVs) demands an inclusive multidisciplinary approach from the earliest design phases, since every design choice strictly affects and is affected by the overall working conditions. This paper presents a review of the scientific literature focused on the design methods applied in defining and sizing the propulsion system of drones. The analysis, performed with a systematic approach, evaluated 123 papers according to two custom classification taxonomies, which investigated respectively the primary aim and specific content of the works. Finally, literature indications and hints were combined into an integrated framework for the functional design of the propulsion system of UAVs. The procedure aimed to support the designer in the preliminary selection of the propulsion candidates and the quick sizing of the supply system, during the first phases of the design process. According to the literature, design methods dramatically change depending on the expected applications and working conditions of UAVs, so that the detailed design of specific drone elements and propulsion components represents the focus of most of the papers in this field.
HiGAN: Handwriting Imitation Conditioned on Arbitrary-Length Texts and Disentangled Styles

Given limited handwriting scripts, humans can easily visualize (or imagine) what the handwritten words/texts would look like with other arbitrary textual contents. Moreover, a person also is able to imitate the handwriting styles of provided reference samples. Humans can do such hallucinations, perhaps because they can learn to disentangle the calligraphic styles and textual contents from given handwriting scripts. However, computers cannot study to do such flexible handwriting imitation with existing techniques. In this paper, we propose a novel handwriting imitation generative adversarial network (HiGAN) to mimic such hallucinations. Specifically, HiGAN can generate variable-length handwritten words/texts conditioned on arbitrary textual contents, which are unconstrained to any predefined corpus or out-of-vocabulary words. Moreover, HiGAN can flexibly control the handwriting styles of synthetic images by disentangling calligraphic styles from the reference samples. Experiments on handwriting benchmarks validate our superiority in terms of visual quality and scalability when comparing to the state-of-the-art methods for handwritten word/text synthesis. The code and pre-trained models can be found at https://github.com/ganji15/HiGAN.
What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks?

Crowdsourcing is widely used to create data for common natural language understanding tasks.Despite the importance of these datasets for measuring and refining model understanding of language, there has been little focus on the crowdsourcing methods used for collecting the datasets.In this paper, we compare the efficacy of interventions that have been proposed in prior work as ways of improving data quality.We use multiple-choice question answering as a testbed and run a randomized trial by assigning crowdworkers to write questions under one of four different data collection protocols.We find that asking workers to write explanations for their examples is an ineffective stand-alone strategy for boosting NLU example difficulty.However, we find that training crowdworkers, and then using an iterative process of collecting data, sending feedback, and qualifying workers based on expert judgments is an effective means of collecting challenging data.But using crowdsourced, instead of expert judgments, to qualify workers and send feedback does not prove to be effective.We observe that the data from the iterative protocol with expert assessments is more challenging by several measures.Notably, the humanmodel gap on the unanimous agreement portion of this data is, on average, twice as large as the gap for the baseline protocol data.* Equal contribution.† Work done while at New York University. 10We use pretrained models distributed with HuggingFace Transformers (Wolf et al., 2020).batch size of 8, learning rate of 1.0 × 10 -5 , and finetune the models using the Adam optimizer for 4 epochs on the RACE dataset.