Reviewerly/exHarmony huggingface dataset:
A Review of Winograd Schema Challenge Datasets and Approaches

The Winograd Schema Challenge is both a commonsense reasoning and natural language understanding challenge, introduced as an alternative to the Turing test. A Winograd schema is a pair of sentences differing in one or two words with a highly ambiguous pronoun, resolved differently in the two sentences, that appears to require commonsense knowledge to be resolved correctly. The examples were designed to be easily solvable by humans but difficult for machines, in principle requiring a deep understanding of the content of the text and the situation it describes. This paper reviews existing Winograd Schema Challenge benchmark datasets and approaches that have been published since its introduction.
The R Package stagedtrees for Structural Learning of Stratified Staged Trees

stagedtrees is an R package which includes several algorithms for learning the structure of staged trees and chain event graphs from data. Score-based and clustering-based algorithms are implemented, as well as various functionalities to plot the models and perform inference. The capabilities of stagedtrees are illustrated using mainly two datasets both included in the package or bundled in R.
FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning

We propose Falcon, an end-to-end 3-party protocol for efficient private training and inference of large machine learning models. Falcon presents four main advantages - (i) It is highly expressive with support for high capacity networks such as VGG16 (ii) it supports batch normalization which is important for training complex networks such as AlexNet (iii) Falcon guarantees security with abort against malicious adversaries, assuming an honest majority (iv) Lastly, Falcon presents new theoretical insights for protocol design that make it highly efficient and allow it to outperform existing secure deep learning solutions. Compared to prior art for private inference, we are about 8x faster than SecureNN (PETS'19) on average and comparable to ABY3 (CCS'18). We are about 16-200x more communication efficient than either of these. For private training, we are about 6x faster than SecureNN, 4.4x faster than ABY3 and about 2-60x more communication efficient. Our experiments in the WAN setting show that over large networks and datasets, compute operations dominate the overall latency of MPC, as opposed to the communication.
A fast and straightforward hand-eye calibration method using stereo camera

Robot hand-eye calibration is crucial for a robot to transform the target 3D position from the camera coordinate system into the robot coordinate system. The conventional method is to use a single-lens camera and calibration board and then solve the AX=XB problem. Recently with the popularization of stereo camera in robot industry, we design a fast and straightforward way to accomplish hand-eye calibration using the point cloud information from the stereo camera and orthogonal projection. We also investigate the number of pairs of data required for calculation in this procedure. We conclude that normally a minimal of 6 points are required for obtaining satisfying result and more pairs of data can provide better result. The procedure is demonstrated and tested on an electric robot arm, and further validated on a full ocean depth hydraulic robot arm. The procedure takes little time to set up and the formulas are concise. It can be applied easily to most robot arms for hand-eye calibration.
RESTRICTED MAXIMUM LIKELIHOOD ESTIMATION FOR MULTIVARIATE LINEAR MIXED MODEL IN ANALYZING PISA DATA FOR INDONESIAN STUDENTS

The Program for International Student Assessment (PISA), becomes one of the references or indicators used to assess the development of students' knowledge and skills in each member country of the Organization for Economic Cooperation and Development (OECD). The results of the PISA survey in 2018 placed Indonesia in the bottom 10, indicating that the implementation of the national education system has not been successful. This underlies the need for a more in-depth study of the factors that influence PISA data scores not only statistically qualitatively but also quantitatively which is still very rarely done. The data structure of the PISA survey results is complex, which involves multicollinearity, multivariate response variables, and random effects. Thus, it requires an appropriate statistical analysis method such as the multivariate mixed linear regression (MLMM) model. In this study, secondary data from the results of the 2018 PISA survey with Indonesian students as the smallest unit of observation were used as sample. School is used as an intercept random effect which is assumed to be normally distributed. Multicollinearity is overcome by selecting independent variables based on AIC and BIC values. Estimation of variance and random effect parameters was performed using the restricted maximum likelihood (REML) method. Based on the estimator of the variance of random effects for the response variables of mathematics, science, and reading literacy, it was obtained 1548.12, 1359.39, and 1082.48, respectively, which explains the significant effect of each school as a random effect on the three response variables.
Improving Neural Machine Translation with the Abstract Meaning Representation by Combining Graph and Sequence Transformers

Previous studies have shown that the Abstract Meaning Representation (AMR) can improve Neural Machine Translation (NMT).However, there has been little work investigating incorporating AMR graphs into Transformer models.In this work, we propose a novel encoder-decoder architecture which augments the Transformer model with a Heterogeneous Graph Transformer (Yao et al., 2020) which encodes source sentence AMR graphs.Experimental results demonstrate the proposed model outperforms the Transformer model and previous non-Transformer based models on two different language pairs in both the high resource setting and low resource setting.Our source code, training
CTM - A Model for Large-Scale Multi-View Tweet Topic Classification

Automatically associating social media posts with topics is an important prerequisite for effective search and recommendation on many social media platforms.However, topic classification of such posts is quite challenging because of (a) a large topic space (b) short text with weak topical cues and (c) multiple topic associations per post.In contrast to most prior work which only focuses on post classification into a small number of topics (10-20), we consider the task of large-scale topic classification in the context of Twitter where the topic space is 10 times larger with potentially multiple topic associations per Tweet.We address the challenges above by proposing a novel neural model, CTM that (a) supports a large topic space of 300 topics and (b) takes a holistic approach to tweet content modeling -leveraging multi-modal content, author context, and deeper semantic cues in the Tweet.Our method offers an effective way to classify Tweets into topics at scale by yielding superior performance to other approaches (a relative lift of 20% in median average precision score) and has been successfully deployed in production at Twitter.
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework

Vast efforts have been devoted to creating highperformance few-shot learners, i.e., large-scale pretrained language models (PLMs) that perform well with little downstream task training data.Training PLMs has incurred significant cost, but utilizing the few-shot learners is still challenging due to their enormous size.This work focuses on a crucial question: How to make effective use of these few-shot learners?We propose LMTurk, a novel approach that treats few-shot learners as crowdsourcing workers.The rationale is that crowdsourcing workers are in fact few-shot learners: They are shown a few illustrative examples to learn about a task and then start annotating.LMTurk employs few-shot learners built upon PLMs as workers.We show that the resulting annotations can be utilized to train models that solve the task well and are small enough to be deployable in practical scenarios.Active learning is integrated into LMTurk to reduce the amount of queries made to PLMs, minimizing the computational cost of running PLM inference passes.Altogether, LMTurk is an important step towards making effective use of current PLMs. 1
On Measuring Social Biases in Prompt-Based Multi-Task Learning

Large language models trained on a mixture of NLP tasks that are converted into a textto-text format using prompts, can generalize into novel forms of language and handle novel tasks.A large body of work within prompt engineering attempts to understand the effects of input forms and prompts in achieving superior performance.We consider an alternative measure and inquire whether the way in which an input is encoded affects social biases promoted in outputs.In this paper, we study T0, a large-scale multi-task text-to-text language model trained using prompt-based learning.We consider two different forms of semantically equivalent inputs: question-answer format and premise-hypothesis format.We use an existing bias benchmark for the former BBQ (Parrish et al., 2021) and create the first bias benchmark in natural language inference BBNLI with hand-written hypotheses while also converting each benchmark into the other form.The results on two benchmarks suggest that given two different formulations of essentially the same input, T0 conspicuously acts more biased in question answering form, which is seen during training, compared to premisehypothesis form which is unlike its training examples.
Reference-free Summarization Evaluation via Semantic Correlation and Compression Ratio

A document can be summarized in a number of ways.Reference-based evaluation of summarization has been criticized for its inflexibility.In this paper, we propose a new automatic reference-free evaluation metric that compares semantic distribution between source document and summary by pretrained language models and considers summary compression ratio.The experiments show that this metric is more consistent with human evaluation in terms of coherence, consistency, relevance, fluency.
Fast Bilingual Grapheme-To-Phoneme Conversion

Autoregressive transformer (ART)-based grapheme-to-phoneme (G2P) models have been proposed for bi/multilingual text-tospeech systems.Although they have achieved great success, they suffer from high inference latency in real-time industrial applications, especially processing long sentence.In this paper, we propose a fast and highperformance bilingual G2P model.For fast and exact decoding, we used a non-autoregressive structured transformer-based architecture and data augmentation for predicting output length.Our model achieved better performance than that of the previous autoregressive model and about 2700% faster inference speed.
Improving Few-Shot Relation Classification by Prototypical Representation Learning with Definition Text

Few-shot relation classification is difficult because the few instances available may not represent well the relation patterns.Some existing approaches explored extra information such as relation definition, in addition to the instances, to learn a better relation representation.However, the encoding of the extra information has been performed independently from the labeled instances.In this paper, we propose to learn a prototype encoder from relation definition text in a way that is useful for relation instance classification.To this end, we use a joint training approach to train both a prototype encoder from definition and an instance encoder.Extensive experiments on several datasets demonstrate the effectiveness and usefulness of our prototype encoder from definition text, enabling us to outperform state-ofthe-art approaches.
Interactive Symbol Grounding with Complex Referential Expressions

We present a procedure for learning to ground symbols from a sequence of stimuli consisting of an arbitrarily complex noun phrase (e.g."all but one green square above both red circles.")and its designation in the visual scene.Our distinctive approach combines: a) lazy fewshot learning to relate open-class words like green and above to their visual percepts; and b) symbolic reasoning with closed-class word categories like quantifiers and negation.We use this combination to estimate new training examples for grounding symbols that occur within a noun phrase but aren't designated by that noun phase (e.g, red in the above example), thereby potentially gaining data efficiency.We evaluate the approach in a visual reference resolution task, in which the learner starts out unaware of concepts that are part of the domain model and how they relate to visual percepts.
Adaptable Adapters

State-of-the-art pretrained NLP models contain a hundred million to trillion parameters.Adapters provide a parameter-efficient alternative for the full finetuning in which we can only finetune lightweight neural network layers on top of pretrained weights.Adapter layers are initialized randomly.However, existing work uses the same adapter architecture-i.e., the same adapter layer on top of each layer of the pretrained model-for every dataset, regardless of the properties of the dataset or the amount of available training data.In this work, we introduce adaptable adapters that contain (1) learning different activation functions for different layers and different input data, and (2) a learnable switch to select and only use the beneficial adapter layers.We show that adaptable adapters achieve on-par performances with the standard adapter architecture while using a considerably smaller number of adapter layers.In addition, we show that the selected adapter architecture by adaptable adapters transfers well across different data settings and similar tasks.We propose to use adaptable adapters for designing efficient and effective adapter architectures.The resulting adapters (a) contain about 50% of the learning parameters of the standard adapter and are therefore more efficient at training and inference, and require less storage space, and (b) achieve considerably higher performances in low-data settings. 1
Efficient Few-Shot Fine-Tuning for Opinion Summarization

Abstractive summarization models are typically pre-trained on large amounts of generic texts, then fine-tuned on tens or hundreds of thousands of annotated samples.However, in opinion summarization, large annotated datasets of reviews paired with reference summaries are not available and would be expensive to create.This calls for fine-tuning methods robust to overfitting on small datasets.In addition, generically pre-trained models are often not accustomed to the specifics of customer reviews and, after fine-tuning, yield summaries with disfluencies and semantic mistakes.To address these problems, we utilize an efficient few-shot method based on adapters which, as we show, can easily store in-domain knowledge.Instead of fine-tuning the entire model, we add adapters and pretrain them in a task-specific way on a large corpus of unannotated customer reviews, using held-out reviews as pseudo summaries.Then, fine-tune the adapters on the small available human-annotated dataset.We show that this self-supervised adapter pre-training improves summary quality over standard fine-tuning by 2.0 and 1.3 ROUGE-L points on the Amazon and Yelp datasets, respectively.Finally, for summary personalization, we condition on aspect keyword queries, automatically created from generic datasets.In the same vein, we pre-train the adapters in a query-based manner on customer reviews and then fine-tune them on annotated datasets.This results in better-organized summary content reflected in improved coherence and fewer redundancies.
Translating Diffusion, Wavelets, and Regularisation into Residual Networks

Convolutional neural networks (CNNs) often perform well, but their stability is poorly understood. To address this problem, we consider the simple prototypical problem of signal denoising, where classical approaches such as nonlinear diffusion, wavelet-based methods and regularisation offer provable stability guarantees. To transfer such guarantees to CNNs, we interpret numerical approximations of these classical methods as a specific residual network (ResNet) architecture. This leads to a dictionary which allows to translate diffusivities, shrinkage functions, and regularisers into activation functions, and enables a direct communication between the four research communities. On the CNN side, it does not only inspire new families of nonmonotone activation functions, but also introduces intrinsically stable architectures for an arbitrary number of layers.
Ask Me Anything in Your Native Language

Cross-lingual question answering is a thriving field in the modern world, helping people to search information on the web more efficiently.One of the important scenarios is to give an answer even there is no answer in the language a person asks a question with.We present a novel approach based on single encoder for query and passage for retrieval from multi-lingual collection, together with cross-lingual generative reader.It achieves a new state of the art in both retrieval and end-to-end tasks on the XOR TyDi dataset outperforming the previous results up to 10% on several languages.We find that our approach can be generalized to more than 20 languages in zero-shot approach and outperform all previous models by 12%.
Quantifying Adaptability in Pre-trained Language Models with 500 Tasks

When a neural language model (LM) is adapted to perform a new task, what aspects of the task predict the eventual performance of the model?In NLP, systematic features of LM generalization to individual examples are well characterized, but systematic aspects of LM adaptability to new tasks are not nearly as well understood.We present a large-scale empirical study of the features and limits of LM adaptability using a new benchmark, TASKBENCH500, built from 500 procedurally generated sequence modeling tasks.These tasks combine core aspects of language processing, including lexical semantics, sequence processing, memorization, logical reasoning, and world knowledge.Using TASKBENCH500, we evaluate three facets of adaptability, finding that: (1) adaptation procedures differ dramatically in their ability to memorize small datasets; (2) within a subset of task types, adaptation procedures exhibit compositional adaptability to complex tasks; and(3) failure to match training label distributions is explained by mismatches in the intrinsic difficulty of predicting individual labels.Our experiments show that adaptability to new tasks, like generalization to new examples, can be systematically described and understood, and we conclude with a discussion of additional aspects of adaptability that could be studied using the new benchmark. 1) Define tasks
NULL at SemEval-2022 Task 6: Intended Sarcasm Detection Using Stylistically Fused Contextualized Representation and Deep Learning

The intended sarcasm cannot be understood until the listener observes that the text's literal meaning violates truthfulness.Consequently, words and meanings play an essential role in specifying sarcasm.Enriched feature extraction techniques were proposed to capture both words and meanings in the contexts.Due to the overlapping features in sarcastic and nonsarcastic texts, a CNN model extracts local features from the combined class-dependent statistical embedding of sarcastic texts with contextualized embedding.Another component BiLSTM extracts long dependencies from combined non-sarcastic statistical and contextualized embeddings.This work combines a classifier that uses the combined high-level features of CNN and BiLSTM for sarcasm detection to produce the final predictions.The experimental analysis presented in this paper shows the effectiveness of the proposed method.
Challenges in Measuring Bias via Open-Ended Language Generation

Researchers have devised numerous ways to quantify social biases vested in pretrained language models. As some language models are capable of generating coherent completions given a set of textual prompts, several prompting datasets have been proposed to measure biases between social groups—posing language generation as a way of identifying biases. In this opinion paper, we analyze how specific choices of prompt sets, metrics, automatic tools and sampling strategies affect bias results. We find out that the practice of measuring biases through text completion is prone to yielding contradicting results under different experiment settings. We additionally provide recommendations for reporting biases in open-ended language generation for a more complete outlook of biases exhibited by a given language model. Code to reproduce the results is released under https://github.com/feyzaakyurek/bias-textgen.
Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions

The subjectivity of automatic hate speech detection makes it a complex task, reflected in different and incomplete definitions in NLP. We present hate speech criteria, developed with insights from a law and social science expert, that help researchers create more explicit definitions and annotation guidelines on five aspects: (1) target groups and (2) dominance, (3) perpetrator characteristics, (4) explicit presence of negative interactions, and the (5) type of consequences/effects. Definitions can be structured so that they cover a more broad or more narrow phenomenon and conscious choices can be made on specifying criteria or leaving them open. We argue that the goal and exact task developers have in mind should determine how the scope of hate speech is defined. We provide an overview of the properties of datasets from hatespeechdata.com that may help select the most suitable dataset for a specific scenario.
TechSSN at SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification using Deep Learning Models

Research is progressing in a fast manner in the field of offensive, hate speech, abusive and sarcastic data.Tackling hate speech against women is urgent and really needed to give respect to the lady of our life.This paper describes the system used for identifying misogynous content using images and text.The system developed by the team TECHSSN uses transformer models to detect the misogynous content from text and Convolutional Neural Network model for image data.Various models like BERT, ALBERT, XLNET and CNN are explored and the combination of ALBERT and CNN as an ensemble model provides better results than the rest.This system was developed for the task 5 of the competition, Se-mEval 2022.
Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback

Large language models (LMs), while powerful, are not immune to mistakes, but can be difficult to retrain.Our goal is for an LM to continue to improve after deployment, without retraining, using feedback from the user.Our approach pairs an LM with (i) a growing memory of cases where the user identified an output error and provided general feedback on how to correct it (ii) a corrector model, trained to translate this general feedback into specific edits to repair the model output.Given a new, unseen input, our model can then use feedback from similar, past cases to repair output errors that may occur.We instantiate our approach using an existing, fixed model for script generation, that takes a goal (e.g., "bake a cake") and generates a partially ordered sequence of actions to achieve that goal, sometimes containing errors.Our memory-enhanced system, FBNET, learns to apply user feedback to repair such errors (up to 30 points improvement), while making a start at avoiding similar past mistakes on new, unseen examples (up to 7 points improvement in a controlled setting).This is a first step towards strengthening deployed models, potentially broadening their utility.