Reviewerly/exHarmony huggingface dataset:
Single Pair Cross-Modality Super Resolution

Non-visual imaging sensors are widely used in the industry for different purposes. Those sensors are more expensive than visual (RGB) sensors, and usually produce images with lower resolution. To this end, Cross-Modality Super-Resolution methods were introduced, where an RGB image of a high-resolution assists in increasing the resolution of a low-resolution modality. However, fusing images from different modalities is not a trivial task, since each multi-modal pair varies greatly in its internal correlations. For this reason, traditional state-of-the-arts which are trained on external datasets often struggle with yielding an artifact-free result that is still loyal to the target modality characteristics.We present CMSR, a single-pair approach for Cross-Modality Super-Resolution. The network is internally trained on the two input images only, in a self-supervised manner, learns their internal statistics and correlations, and applies them to up-sample the target modality. CMSR contains an internal transformer which is trained on-the-fly together with the up-sampling process itself and without supervision, to allow dealing with pairs that are only weakly aligned. We show that CMSR produces state-of-the-art super resolved images, yet without introducing artifacts or irrelevant details that originate from the RGB image only.
Ad Headline Generation using Self-Critical Masked Language Model

Yashal Shakti Kanungo, Sumit Negi, Aruna Rajan. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers. 2021.
Towards Automating Medical Scribing : Clinic Visit Dialogue2Note Sentence Alignment and Snippet Summarization

Medical conversations from patient visits are routinely summarized into clinical notes for documentation of clinical care.The automatic creation of clinical note is particularly challenging given that it requires summarization over spoken language and multiple speaker turns; as well, clinical notes include highly technical semi-structured text.In this paper, we describe our corpus creation method and baseline systems for two NLP tasks, clinical dialogue2note sentence alignment and clinical dialogue2note snippet summarization.These two systems, as well as other models created from such a corpus, may be incorporated as parts of an overall end-to-end clinical note generation system.
Distance Learning Technologies and Innovations in Education for Sustainable Development

The article analyzes technologies for teachers’ professional development to work online in education for sustainable development. The authors emphasize that education for sustainable development (ESD) promotes developing the knowledge, skills, understanding, values and actions required to create a sustainable world, which ensures environmental protection and conservation, promotes social equity and encourages economic sustainability. ESD must engage a wide range of stakeholders from government, private sector, civil society, non-governmental organizations and the general public. The authors note that educational institutions should provide teachers with opportunities to improve their qualifications according to the extraordinary time and the need to have many processes online. The differences between traditional and distance learning systems are revealed. The authors suggest a structure of competencies (technical, professional and methodological) for a distance learning teacher. A promising model of distance learning technologies for teachers’ training is presented in the article.
The REPU CS’ Spanish–Quechua Submission to the AmericasNLP 2021 Shared Task on Open Machine Translation

We present the submission of REPUcs 1 to the AmericasNLP machine translation shared task for the low resource language pair Spanish-Quechua.Our neural machine translation system ranked first in Track two (development set not used for training) and third in Track one (training includes development data).Our contribution is focused on: (i) the collection of new parallel data from different web sources (poems, lyrics, lexicons, handbooks), and (ii) using large Spanish-English data for pre-training and then fine-tuning the Spanish-Quechua system.This paper describes the new parallel corpora and our approach in detail.
Photographic Reproduction and Enhancement Using HVS-Based Modified Histogram Equalization

Photographic reproduction and enhancement is challenging because it requires the preservation of all the visual information during the compression of the dynamic range of the input image. This paper presents a cascaded-architecture-type reproduction method that can simultaneously enhance local details and retain the naturalness of original global contrast. In the pre-processing stage, in addition to using a multiscale detail injection scheme to enhance the local details, the Stevens effect is considered for adapting different luminance levels and normally compressing the global feature. We propose a modified histogram equalization method in the reproduction stage, where individual histogram bin widths are first adjusted according to the property of overall image content. In addition, the human visual system (HVS) is considered so that a luminance-aware threshold can be used to control the maximum permissible width of each bin. Then, the global tone is modified by performing histogram equalization on the output modified histogram. Experimental results indicate that the proposed method can outperform the five state-of-the-art methods in terms of visual comparisons and several objective image quality evaluations.
Closed-Loop Region of Interest Enabling High Spatial and Temporal Resolutions in Object Detection and Tracking via Wireless Camera

The trade-off between spatial and temporal resolution remains a fundamental challenge in machine vision.A captured image often contains a significant amount of redundant information, and only a small region of interest (ROI) is necessary for object detection and tracking.In this paper, we first systematically characterize the effects of ROI on camera capturing, data transmission, and image processing.We then present the closed-loop ROI algorithm capable of high spatial and temporal resolution as well as wide scanning field of view (FOV) in single and multi-object detection and tracking via real-time wireless video streaming.With the feedback from real-time object tracking, the wireless camera is able to capture and transmit only the ROI which in turn enhances both the spatial and temporal resolution in object tracking.In addition, the proposed approach can still maintain a large FOV by processing regions outside of the ROI at lower spatial and temporal resolutions.When applied to a high spatial resolution wireless stream (5 MegaPixels), the closed-loop ROI algorithm improves the temporal resolution by up to 10× (from 2.4 FPS to 22.5 FPS).Specifically, camera processing is improved by up to 4.7×, data transmission is improved by up to 160×, and PC processing is improved by up to 2.5×.In a person tracking experiment, the closed-loop ROI algorithm enables a wide-angle camera to outperform both a normal wide-angle camera-which suffers from poor temporal resolution and motion blur-and a pan & tilt camera-which cannot automatically refresh tracking after the tracking is lost.
Deep Learning on Graphs for Natural Language Processing

Lingfei Wu, Yu Chen, Heng Ji, Yunyao Li. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials. 2021.
Text Editing by Command

A prevailing paradigm in neural text generation is one-shot generation, where text is produced in a single step.The one-shot setting is inadequate, however, when the constraints the user wishes to impose on the generated text are dynamic, especially when authoring longer documents.We address this limitation with an interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text.To this end, we propose a novel text editing task, and introduce WikiDocEdits, a dataset of singlesentence edits extracted from Wikipedia revision histories.We show that our Interactive Editor, a transformer-based model trained on this dataset, outperforms baselines and obtains positive results in both automatic and human evaluations.We present empirical and qualitative analyses of this model's performance.
Hie-BART: Document Summarization with Hierarchical BART

This paper proposes a new abstractive summarization model for documents, hierarchical BART (Hie-BART), which captures the hierarchical structures of documents (i.e., their sentence-word structures) in the BART model.Although the existing BART model has achieved state-of-the-art performance on document summarization tasks, it does not account for interactions between sentence-level and word-level information.In machine translation tasks, the performance of neural machine translation models can be improved with the incorporation of multi-granularity self-attention (MG-SA), which captures relationships between words and phrases.Inspired by previous work, the proposed Hie-BART model incorporates MG-SA into the encoder of the BART model for capturing sentence-word structures.Evaluations performed on the CNN/Daily Mail dataset show that the proposed Hie-BART model outperforms strong baselines and improves the performance of a non-hierarchical BART model (+0.23 ROUGE-L).
Fuzzy-Based Quality Adaptation Algorithm for Improving QoE from MPEG-DASH Video

Video clients employ HTTP-based adaptive bitrate (ABR) algorithms to optimize users’ quality of experience (QoE). ABR algorithms adopt video quality based on the network conditions during playback. The existing state-of-the-art ABR algorithms ignore the fact that video streaming services deploy segment durations differently in different services, and HTTP clients offer distinct buffer sizes. The existing ABR algorithms use fixed control laws and are designed with predefined client/server settings. As a result, adaptation algorithms fail to achieve optimal performance across a variety of video client settings and QoE objectives. We propose a buffer- and segment-aware fuzzy-based ABR algorithm that selects video rates for future video segments based on segment duration and the client’s buffer size in addition to throughput and playback buffer level. We demonstrate that the proposed algorithm guarantees high QoE across various video player settings and video content characteristics. The proposed algorithm efficiently utilizes bandwidth in order to download high-quality video segments and to guarantee high QoE. The results from our experiments reveal that the proposed adaptation algorithm outperforms state-of-the-art algorithms, providing improvements in average video rate, QoE, and bandwidth utilization, respectively, of 5% to 18%, about 13% to 30%, and up to 45%.
PIHKers at CMCL 2021 Shared Task: Cosine Similarity and Surprisal to Predict Human Reading Patterns.

Eye-tracking psycholinguistic studies have revealed that context-word semantic coherence and predictability influence language processing.In this paper we show our approach to predict eye-tracking features from the ZuCo dataset for the shared task of the Cognitive Modeling and Computational Linguistics (CMCL2021) workshop.Using both cosine similarity and surprisal within a regression model, we significantly improved the baseline Mean Absolute Error computed among five eye-tracking features.
Improving Autonomous Robotic Navigation Using Imitation Learning

Autonomous navigation to a specified waypoint is traditionally accomplished with a layered stack of global path planning and local motion planning modules that generate feasible and obstacle-free trajectories. While these modules can be modified to meet task-specific constraints and user preferences, current modification procedures require substantial effort on the part of an expert roboticist with a great deal of technical training. In this paper, we simplify this process by inserting a Machine Learning module between the global path planning and local motion planning modules of an off-the shelf navigation stack. This model can be trained with human demonstrations of the preferred navigation behavior, using a training procedure based on Behavioral Cloning, allowing for an intuitive modification of the navigation policy by non-technical users to suit task-specific constraints. We find that our approach can successfully adapt a robot’s navigation behavior to become more like that of a demonstrator. Moreover, for a fixed amount of demonstration data, we find that the proposed technique compares favorably to recent baselines with respect to both navigation success rate and trajectory similarity to the demonstrator.
Understanding dogwhistles politics

This paper aims to deepen our understanding of so-called covert dogwhistles. I discuss whether a covert dogwhistle is a specific sort of mechanism of manipulation or whether, on the contrary, it draws on other already familiar linguistic mechanisms such as implicatures or presuppositions. I put forward a series of arguments aimed at illustrating that implicatures and presuppositions, on the one hand, and covert dogwhistles, on the other, differ in their linguistic behaviour concerning plausible deniability, cancellability, calculability and mutual acceptance. I concluded this paper by outlining a simple theory for covert dogwhistles according to which they are attitude-foregrounders.
Anomaly Detection Using Signal Segmentation and One-Class Classification in Diffusion Process of Semiconductor Manufacturing

This paper proposes a new diagnostic method for sensor signals collected during semiconductor manufacturing. These signals provide important information for predicting the quality and yield of the finished product. Much of the data gathered during this process is time series data for fault detection and classification (FDC) in real time. This means that time series classification (TSC) must be performed during fabrication. With advances in semiconductor manufacturing, the distinction between normal and abnormal data has become increasingly significant as new challenges arise in their identification. One challenge is that an extremely high FDC performance is required, which directly impacts productivity and yield. However, general classification algorithms can have difficulty separating normal and abnormal data because of subtle differences. Another challenge is that the frequency of abnormal data is remarkably low. Hence, engineers can use only normal data to develop their models. This study presents a method that overcomes these problems and improves the FDC performance; it consists of two phases. Phase I has three steps: signal segmentation, feature extraction based on local outlier factors (LOF), and one-class classification (OCC) modeling using the isolation forest (iF) algorithm. Phase II, the test stage, consists of three steps: signal segmentation, feature extraction, and anomaly detection. The performance of the proposed method is superior to that of other baseline methods.
Supertagging-based Parsing with Linear Context-free Rewriting Systems

We present the first supertagging-based parser for linear context-free rewriting systems (LCFRS).It utilizes neural classifiers and outperforms previous LCFRS-based parsers in both accuracy and parsing speed by a wide margin.Our results keep up with the best (general) discontinuous parsers, particularly the scores for discontinuous constituents establish a new state of the art.The heart of our approach is an efficient lexicalization procedure which induces a lexical LCFRS from any discontinuous treebank.We describe a modification to usual chart-based LCFRS parsing that accounts for supertagging and introduce a procedure that transforms lexical LCFRS derivations into equivalent parse trees of the original treebank.Our approach is evaluated on the English Discontinuous Penn Treebank and the German treebanks Negra and Tiger.
Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction

Grammatical Error Correction (GEC) aims to correct writing errors and help language learners improve their writing skills.However, existing GEC models tend to produce spurious corrections or fail to detect lots of errors.The quality estimation model is necessary to ensure learners get accurate GEC results and avoid misleading from poorly corrected sentences.Well-trained GEC models can generate several high-quality hypotheses through decoding, such as beam search, which provide valuable GEC evidence and can be used to evaluate GEC quality.However, existing models neglect the possible GEC evidence from different hypotheses.This paper presents the Neural Verification Network (VERNet) for GEC quality estimation with multiple hypotheses.VERNet establishes interactions among hypotheses with a reasoning graph and conducts two kinds of attention mechanisms to propagate GEC evidence to verify the quality of generated hypotheses.Our experiments on four GEC datasets show that VERNet achieves state-of-the-art grammatical error detection performance, achieves the best quality estimation results, and significantly improves GEC performance by reranking hypotheses.
TuringAdvice: A Generative and Dynamic Evaluation of Language Use

We propose TuringAdvice, a new challenge task and dataset for language understanding models.Given a written situation that a real person is currently facing, a model must generate helpful advice in natural language.Our evaluation framework tests a fundamental aspect of human language understanding: our ability to use language to resolve open-ended situations by communicating with each other.Empirical results show that today's models struggle at TuringAdvice, even multibillion parameter models finetuned on 600k in-domain training examples.The best model, a finetuned T5, writes advice that is at least as helpful as human-written advice in only 14% of cases; a much larger non-finetunable GPT3 model does even worse at 4%.This low performance reveals language understanding errors that are hard to spot outside of a generative setting, showing much room for progress.
Adversarial Robustness Via Fisher-Rao Regularization

Adversarial robustness has become a topic of growing interest in machine learning since it was observed that neural networks tend to be brittle. We propose an information-geometric formulation of adversarial defense and introduce Fire, a new Fisher-Rao regularization for the categorical cross-entropy loss, which is based on the geodesic distance between the softmax outputs corresponding to natural and perturbed input features. Based on the information-geometric properties of the class of softmax distributions, we derive an explicit characterization of the Fisher-Rao Distance (FRD) for the binary and multiclass cases, and draw some interesting properties as well as connections with standard regularization metrics. Furthermore, we verify on a simple linear and Gaussian model, that all Pareto-optimal points in the accuracy-robustness region can be reached by Fire while other state-of-the-art methods fail. Empirically, we evaluate the performance of various classifiers trained with the proposed loss on standard datasets, showing up to a simultaneous 1% of improvement in terms of clean and robust performances while reducing the training time by 20% over the best-performing methods.
Dual Quaternion Embeddings for Link Prediction

The applications of knowledge graph have received much attention in the field of artificial intelligence. The quality of knowledge graphs is, however, often influenced by missing facts. To predict the missing facts, various solid transformation based models have been proposed by mapping knowledge graphs into low dimensional spaces. However, most of the existing transformation based approaches ignore that there are multiple relations between two entities, which is common in the real world. In order to address this challenge, we propose a novel approach called DualQuatE that maps entities and relations into a dual quaternion space. Specifically, entities are represented by pure quaternions and relations are modeled based on the combination of rotation and translation from head to tail entities. After that we utilize interactions of different translations and rotations to distinguish various relations between head and tail entities. Experimental results exhibit that the performance of DualQuatE is competitive compared to the existing state-of-the-art models.
Advice-Guided Reinforcement Learning in a non-Markovian Environment

We study a class of reinforcement learning tasks in which the agent receives its reward for complex, temporally-extended behaviors sparsely. For such tasks, the problem is how to augment the state-space so as to make the reward function Markovian in an efficient way. While some existing solutions assume that the reward function is explicitly provided to the learning algorithm (e.g., in the form of a reward machine), the others learn the reward function from the interactions with the environment, assuming no prior knowledge provided by the user. In this paper, we generalize both approaches and enable the user to give advice to the agent, representing the user’s best knowledge about the reward function, potentially fragmented, partial, or even incorrect. We formalize advice as a set of DFAs and present a reinforcement learning algorithm that takes advantage of such advice, with optimal con- vergence guarantee. The experiments show that using well- chosen advice can reduce the number of training steps needed for convergence to optimal policy, and can decrease the computation time to learn the reward function by up to two orders of magnitude.
Amodal Segmentation Based on Visible Region Segmentation and Shape Prior

Almost all existing amodal segmentation methods make the inferences of occluded regions by using features corresponding to the whole image. This is against the human's amodal perception, where human uses the visible part and the shape prior knowledge of the target to infer the occluded region. To mimic the behavior of human and solve the ambiguity in the learning, we propose a framework, it firstly estimates a coarse visible mask and a coarse amodal mask. Then based on the coarse prediction, our model infers the amodal mask by concentrating on the visible region and utilizing the shape prior in the memory. In this way, features corresponding to background and occlusion can be suppressed for amodal mask estimation. Consequently, the amodal mask would not be affected by what the occlusion is given the same visible regions. The leverage of shape prior makes the amodal mask estimation more robust and reasonable. Our proposed model is evaluated on three datasets. Experiments show that our proposed model outperforms existing state-of-the-art methods. The visualization of shape prior indicates that the category-specific feature in the codebook has certain interpretability. The code is available at https://github.com/YutingXiao/Amodal-Segmentation-Based-on-Visible-Region-Segmentation-and-Shape-Prior.