Reviewerly/exHarmony huggingface dataset:
Normalizing Mutual Information for Robust Adaptive Training for Translation

Despite the success of neural machine translation models, tensions between fluency of optimizing target language modeling and sourcefaithfulness remain as challenges.Previously, Conditional Bilingual Mutual Information (CBMI), a scoring metric for the importance of target sentences and tokens, was proposed to encourage fluent and faithful translations.The score is obtained by combining the probability from the translation model and the target language model, which is then used to assign different weights to losses from sentences and tokens.Meanwhile, we argue this metric is not properly normalized, for which we propose Normalized Pointwise Mutual Information (NPMI).NPMI utilizes an additional language model on source language to approximate the joint likelihood of source-target pair and the likelihood of the source, which is then used for normalizing the score.We showed that NPMI better captures the dependence between source-target and that NPMI-based token-level adaptive training brings improvements over baselines with empirical results from En-De, De-En, and En-Ro translation tasks.
Attentional Probe: Estimating a Module’s Functional Potential

In this paper, we seek to measure how much information a component in a neural network could extract from the representations fed into it.Our work stands in contrast to prior probing work, most of which investigates how much information a model's representations contain.This shift in perspective leads us to propose a new principle for probing, the architectural bottleneck principle: In order to estimate how much information a given component could extract, a probe should look exactly like the component.Relying on this principle, we estimate how much syntactic information is available to transformers through our attentional probe, a probe that exactly resembles a transformer's self-attention head.Experimentally, we find that, in three models (BERT, ALBERT, and RoBERTa), a sentence's syntax tree is mostly extractable by our probe, suggesting these models have access to syntactic information while composing their contextual representations.Whether this information is actually used by these models, however, remains an open question.https://github.com/rycolab/attentional-probe
A Hybrid Approach to Cross-lingual Product Review Summarization

We present a hybrid approach for product review summarization which consists of: (i) an unsupervised extractive step to extract the most important sentences out of all the reviews, and (ii) a supervised abstractive step to summarize the extracted sentences into a coherent short summary.This approach allows us to develop an efficient cross-lingual abstractive summarizer that can generate summaries in any language, given the extracted sentences out of thousands of reviews in a source language.In order to train and test the abstractive model, we create the Cross-lingual Amazon Reviews Summarization (CARS) dataset which provides English summaries for training, and English, French, Italian, Arabic, and Hindi summaries for testing based on selected English reviews.We show that the summaries generated by our model are as good as human written summaries in coherence, informativeness, non-redundancy, and fluency.
TransAdv: A Translation-based Adversarial Learning Framework for Zero-Resource Cross-Lingual Named Entity Recognition

Zero-Resource Cross-Lingual Named Entity Recognition aims at training an NER model of the target language using only labeled source language data and unlabeled target language data.Existing methods are mainly divided into three categories: model transfer based, data transfer based and knowledge transfer based.Each method has its own disadvantages, and combining more than one of them often leads to better performance.However, the performance of data transfer based methods is often limited by inevitable noise in the translation process.To handle the problem, we propose a framework named TransAdv to mitigate lexical and syntactic errors of word-by-word translated data, better utilizing the data by multi-level adversarial learning and multi-model knowledge distillation.Extensive experiments are conducted over 6 target languages with English as the source language, and the results show that TransAdv achieves competitive performance to the state-of-the-art models.
Generalization Differences between End-to-End and Neuro-Symbolic Vision-Language Reasoning Systems

For vision-and-language (VL) reasoning tasks, both fully connectionist, end-to-end methods and hybrid, neuro-symbolic methods have achieved high in-distribution performance.In which out-of-distribution settings does each paradigm excel?We investigate this question on both single-image and multi-image visual question-answering through four types of generalization tests: a novel segmentcombine test for multi-image queries, contrast set, compositional generalization, and crossbenchmark transfer.Vision-and-language endto-end (VLE2E) trained systems exhibit sizeable performance drops across all these tests.Neuro-symbolic (NS) methods suffer even more on cross-benchmark transfer from GQA to VQA, but they show smaller accuracy drops on the other generalization tests and their performance quickly improves by few-shot training.Overall, our results demonstrate the complementary benefits of these two paradigms, and emphasize the importance of using a diverse suite of generalization tests to fully characterize model robustness to distribution shift.Train A: There is at least 1 image with 2 bottles.Train B: Is the dark bottle on the table or not? Compositional GeneralizationContrast: There is less than 1 image with exactly 2 dark bottles on the table.
DualNER: A Dual-Teaching framework for Zero-shot Cross-lingual Named Entity Recognition

We present DualNER, a simple and effective framework to make full use of both annotated source language corpus and unlabeled target language text for zero-shot cross-lingual named entity recognition (NER).In particular, we combine two complementary learning paradigms of NER, i.e., sequence labeling and span prediction, into a unified multi-task framework.After obtaining a sufficient NER model trained on the source data, we further train it on the target data in a dual-teaching manner, in which the pseudo-labels for one task are constructed from the prediction of the other task.Moreover, based on the span prediction, an entityaware regularization is proposed to enhance the intrinsic cross-lingual alignment between the same entities in different languages.Experiments and analysis demonstrate the effectiveness of our DualNER.Code is available at https://github.com/lemon0830/dualNER.
Capturing Topic Framing via Masked Language Modeling

Differential framing of issues can lead to divergent world views on important issues.This is especially true in domains where the information presented can reach a large audience, such as traditional and social media.Scalable and reliable measurement of such differential framing is an important first step in addressing them.In this work, based on the intuition that framing affects the tone and word choices in written language, we propose a framework for modeling the differential framing of issues through masked token prediction via large-scale finetuned language models (LMs).Specifically, we explore three key factors for our framework: 1) prompt generation methods for the masked token prediction; 2) methods for normalizing the output of fine-tuned LMs; 3) robustness to the choice of pre-trained LMs used for finetuning.Through experiments on a dataset of articles from traditional media outlets covering five diverse and politically polarized topics, we show that our framework can capture differential framing of these topics with high reliability.
Mix-and-Match: Scalable Dialog Response Retrieval using Gaussian Mixture Embeddings

Embedding-based approaches for dialog response retrieval embed the context-response pairs as points in the embedding space.These approaches are scalable, but fail to account for the complex, many-to-many relationships that exist between context-response pairs.On the other end of the spectrum, there are approaches that feed the context-response pairs jointly through multiple layers of neural networks.These approaches can model the complex relationships between context-response pairs, but fail to scale when the set of responses is moderately large (>1000).In this paper, we propose a scalable model that can learn complex relationships between context-response pairs.Specifically, the model maps the contexts as well as responses to probability distributions over the embedding space.We train the models by optimizing the Kullback-Leibler divergence between the distributions induced by contextresponse pairs in the training data.We show that the resultant model achieves better performance as compared to other embedding-based approaches on publicly available conversation data.
Towards Explaining Subjective Ground of Individuals on Social Media

Large-scale language models have been reducing the gap between machines and humans in understanding the real world, yet understanding an individual's theory of mind and behavior from text is far from being resolved.This research proposes a neural model-Subjective Ground Attention-that learns subjective grounds of individuals and accounts for their judgments on situations of others posted on social media.Using simple attention modules as well as taking one's previous activities into consideration, we empirically show that our model provides human-readable explanations of an individual's subjective preference in judging social situations.We further qualitatively evaluate the explanations generated by the model and claim that our model learns an individual's subjective orientation towards abstract moral concepts.
Multilingual Multimodal Learning with Machine Translated Text

Most vision-and-language pretraining research focuses on English tasks.However, the creation of multilingual multimodal evaluation datasets (e.g.Multi30K, xGQA, XVNLI, and MaRVL) poses a new challenge in finding high-quality training data that is both multilingual and multimodal.In this paper, we investigate whether machine translating English multimodal data can be an effective proxy for the lack of readily available multilingual data.We call this framework TD-MML: Translated Data for Multilingual Multimodal Learning, and it can be applied to any multimodal dataset and model.We apply it to both pretraining and fine-tuning data with a state-of-the-art model.In order to prevent models from learning from low-quality translated text, we propose two metrics for automatically removing such translations from the resulting datasets.In experiments on five tasks across 20 languages in the IGLUE benchmark, we show that translated data can provide a useful signal for multilingual multimodal learning, both at pretraining and fine-tuning.
Seeded Hierarchical Clustering for Expert-Crafted Taxonomies

Practitioners from many disciplines (e.g., political science) use expert-crafted taxonomies to make sense of large, unlabeled corpora.In this work, we study Seeded Hierarchical Clustering (SHC): the task of automatically fitting unlabeled data to such taxonomies using only a small set of labeled examples.We propose HI-ERSEED, a novel weakly supervised algorithm for this task that uses only a small set of labeled seed examples.It is both data and computationally efficient.HIERSEED assigns documents to topics by weighing document density against topic hierarchical structure.It outperforms both unsupervised and supervised baselines for the SHC task on three real-world datasets.
Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parsers

Text-to-SQL parsers typically struggle with databases unseen during the train time.Adapting parsers to new databases is a challenging problem due to the lack of natural language queries in the new schemas.We present REFILL, a framework for synthesizing highquality and textually diverse parallel datasets for adapting a Text-to-SQL parser to a target schema.REFILL learns to retrieve-andedit text queries from the existing schemas and transfers them to the target schema.We show that retrieving diverse existing text, masking their schema-specific tokens, and refilling with tokens relevant to the target schema, leads to significantly more diverse text queries than achievable by standard SQL-to-Text generation methods.Through experiments spanning multiple databases, we demonstrate that fine-tuning parsers on datasets synthesized using REFILL consistently outperforms the prior data-augmentation methods.
Baked-in State Probing

Neural language models have been analyzed for their linguistic and extra-linguistic knowledge via probing.Of particular interest has been the following question: how much can a language model trained only on form learn about meaning?Recent work has demonstrated via probing classifiers that in the setting of simple procedural text, where by "meaning" we mean the underlying world state, language models have a non-trivial performance on world state tracking.However, our proposed evaluation based on model predictions shows differing results, suggesting that these models are either not capturing the world state or not using it.How do these results change if the model has access to the world state?We explore this alternate setting with access to the underlying world state only during training and investigate ways of "baking in" the state knowledge along with the primary task of language modeling.Our proposed approaches allow for state probing during inference simply via text prompts, avoiding any probing classifier machinery.In terms of performance, we show that baking in the state knowledge during training leads to significant improvements in state tracking performance and text generation quality. 1
Realistic Data Augmentation Framework for Enhancing Tabular Reasoning

Existing approaches to constructing training data for Natural Language Inference (NLI) tasks, such as for semi-structured table reasoning, are either via crowdsourcing or fully automatic methods.However, the former is expensive and time-consuming and thus limits scale, and the latter often produces naive examples that may lack complex reasoning.This paper develops a realistic semi-automated framework for data augmentation for tabular inference.Instead of manually generating a hypothesis for each table, our methodology generates hypothesis templates transferable to similar tables.In addition, our framework entails the creation of rational counterfactual tables based on human written logical constraints and premise paraphrasing.For our case study, we use the IN-FOTABS (Gupta et al., 2020), which is an entitycentric tabular inference dataset.We observed that our framework could generate human-like tabular inference examples, which could benefit training data augmentation, especially in the scenario with limited supervision.
A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss

For readability assessment, traditional methods mainly employ machine learning classifiers with hundreds of linguistic features.Although the deep learning model has become the prominent approach for almost all NLP tasks, it is less explored for readability assessment.In this paper, we propose a BERT-based model with feature projection and length-balanced loss (BERT-FP-LBL) for readability assessment.Specially, we present a new difficulty knowledge guided semi-supervised method to extract topic features to complement the traditional linguistic features.From the linguistic features, we employ projection filtering to extract orthogonal features to supplement BERT representations.Furthermore, we design a new length-balanced loss to handle the greatly varying length distribution of data.Our model achieves state-of-theart performances on two English benchmark datasets and one dataset of Chinese textbooks, and also achieves the near-perfect accuracy of 99% on one English dataset.Moreover, our proposed model obtains comparable results with human experts in consistency test.
Refinement Matters: Textual Description Needs to be Refined for Zero-shot Learning

Zero-Shot Learning (ZSL) has shown great promise at the intersection of vision and language, and generative methods for ZSL are predominant owing to their efficiency.Moreover, textual description or attribute plays a critical role in transferring knowledge from the seen to unseen classes in ZSL.Such generative approaches for ZSL are very costly to train and require the class description of the unseen classes during training.In this work, we propose a non-generative gating-based attribute refinement network for ZSL, which achieves similar accuracies to generative methods of ZSL, at a much lower computational cost.The refined attributes are mapped into the visual domain through an attribute embedder, and the whole network is guided by the circle loss and the well-known softmax cross-entropy loss to obtain a robust class embedding.We refer to our approach as Circle loss guided gatingbased Attribute-Refinement Network (CAR-Net).We perform extensive experiments on the five benchmark datasets over the various challenging scenarios viz., Generalized ZSL (GZSL), Continual GZSL (CGZSL), and conventional ZSL.We observe that the CARNet significantly outperforms recent non-generative ZSL methods and most generative ZSL methods in all three settings by a significant margin.Our extensive ablation study disentangles the performance of various components and justifies their importance 1 .
Exploring Euphemism Detection in Few-Shot and Zero-Shot Settings

This work builds upon the Euphemism Detection Shared Task proposed in the EMNLP 2022 FigLang Workshop, and extends it to fewshot and zero-shot settings.We demonstrate a few-shot and zero-shot formulation using the dataset from the shared task, and we conduct experiments in these settings using RoBERTa and GPT-3.Our results show that language models are able to classify euphemistic terms relatively well even on new terms unseen during training, indicating that it is able to capture higher-level concepts related to euphemisms.
Tinction of Black and White Images using a Novel Deep Learning Approach

Tinction is the process of converting grayscale photos into color images that are visually appealing and appear genuine. This process has been a challenge due to the involvement of human factors and the complexity of automatic conversion. To address this, an innovative method has been proposed that utilizes deep learning techniques, such as GAN and U-Net models, to learn the mapping between grayscale and color images. . In addition, the Fusion layer is performed in order to integrate the local information findings that are reliant on localised areas of the image in conjunction with the world wide-priors that are based on the full image for each class. This results in colorization results that are more aesthetically engaging. In the end, we validate our technique by doing an evaluation based on an information acquired, and we evaluate it in relation to the existing standard of practise, which leads to improvements.A CNN architecture is utilized in the proposed method to extract features from a grayscale image and produce a color image that closely resembles the original colored image. The model is trained on a vast dataset of both grayscale and colored images and is assessed using different metrics. The proposed method surpasses current leading methods regarding the accuracy of colorization and the visual quality of the resulting images. This technique has promising applications in image and video editing, as well as restoration.
Towards Improving the Performance of Pre-Trained Speech Models for Low-Resource Languages Through Lateral Inhibition

With the rise of bidirectional encoder representations from Transformer models in natural language processing, the speech community has adopted some of their development methodologies. Therefore, the Wav2Vec models were introduced to reduce the data required to obtain state-of-the-art results. This work leverages this knowledge and improves the performance of the pre-trained speech models by simply replacing the fine-tuning dense layer with a lateral inhibition layer inspired by the biological process. Our experiments on Romanian, a low-resource language, show an average improvement of 12.5% word error rate (WER) using the lateral inhibition layer. In addition, we obtain state-of-the-art results on both the Romanian Speech Corpus and the Robin Technical Acquisition Corpus with 1.78% WER and 29.64% WER, respectively.
IMPLEMENTASI TEKNIK FAILOVER RECURSIVE GATEWAY

The development of information technology and the internet in Indonesia every year shows very rapid progress in terms of reliable infrastructure, users, hardware, software and information systems. At Buana Perjuangan University, Karawang has two internet lanes designated for users of all academics, both LAN and wifi, and one more lane for servers. These conditions sometimes create problems when one of the internet lines is down which results in hampering service activities. Based on the background above, the authors propose the implementation of a Failover Recursive Gateway, with the aim that there will be no more cessation of administrative services when one of the internet lines is down.
Pedagogy and philanthropy by the call of the soul: Interview with O. V. Matvienko

A conversation between a postgraduate student of the Department of Pedagogy and Methodology of Foreign Language Teaching O. V. Yashchuk with prof,Doctor of Pedagogical Sciences, Corresponding Member of the National Academy of Pedagogical Sciences of Ukraine, Honoured Worker of Education of Ukraine O. V. Matvienko
MIDW‐Net: A multi‐tasking network architecture for radar intra‐pulse parameter description

Abstract The automatic modulation recognition (AMR) of radar signals has become a popular research topic in recent years. However, most algorithms focus on the type of signal modulation and lack further understanding of the signal. To address this gap, a network architecture for multi‐tasking intra‐pulse description words (MIDW‐Net) is proposed herein. In this framework, the denoising algorithm employs a convolutional denoising autoencoder, which is an effective method for suppressing noise interference and preserving signal information. The multiscale feature‐extraction capability of a feature pyramid network (FPN) is utilized to expand the perceptual domain without losing the high‐frequency features of the image. Finally, AMR and modulation parameter estimation are accomplished via multitask learning. Experiments performed on simulated radar signals using four intra‐pulse descriptors verified the effectiveness of the proposed algorithm.