Reviewerly/exHarmony huggingface dataset:
Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction

We investigate grounded language learning through real-world data, by modelling a teacher-learner dynamics through the natural interactions occurring between users and search engines; in particular, we explore the emergence of semantic generalization from unsupervised dense representations outside of synthetic environments.A grounding domain, a denotation function and a composition function are learned from user data only.We show how the resulting semantics for noun phrases exhibits compositional properties while being fully learnable without any explicit labelling.We benchmark our grounded semantics on compositionality and zero-shot inference tasks, and we show that it provides better results and better generalizations than SOTA non-grounded models, such as word2vec and BERT.
Adversarial Invariant Learning

Though machine learning algorithms are able to achieve pattern recognition from the correlation between data and labels, the presence of spurious features in the data decreases the robustness of these learned relationships with respect to varied testing environments. This is known as out-of-distribution (OoD) generalization problem. Recently, invariant risk minimization (IRM) attempts to tackle this issue by penalizing predictions based on the unstable spurious features in the data collected from different environments. However, similar to domain adaptation or domain generalization, a prevalent non-trivial limitation in these works is that the environment information is assigned by human specialists, i.e. a priori, or determined heuristically. However, an inappropriate group partitioning can dramatically deteriorate the OoD generalization and this process is expensive and time-consuming. To deal with this issue, we propose a novel theoretically principled min-max framework to iteratively construct a worst-case splitting, i.e. creating the most challenging environment splittings for the backbone learning paradigm (e.g. IRM) to learn the robust feature representation. We also design a differentiable training strategy to facilitate the feasible gradient- based computation. Numerical experiments show that our algorithmic framework has achieved superior and stable performance in various datasets, such as Colored MNIST and Punctuated Stanford sentiment treebank (SST). Furthermore, we also find our algorithm to be robust even to a strong data poisoning attack. To the best of our knowledge, this is one of the first to adopt differentiable environment splitting method to enable stable predictions across environments without environment index information, which achieves the state-of-the-art performance on datasets with strong spurious correlation, such as Colored MNIST.
Modeling Diagnostic Label Correlation for Automatic ICD Coding

Given the clinical notes written in electronic health records (EHRs), it is challenging to predict the diagnostic codes which is formulated as a multi-label classification task.The large set of labels, the hierarchical dependency, and the imbalanced data make this prediction task extremely hard.Most existing work built a binary prediction for each label independently, ignoring the dependencies between labels.To address this problem, we propose a two-stage framework to improve automatic ICD coding by capturing the label correlation.Specifically, we train a label set distribution estimator to rescore the probability of each label set candidate generated by a base predictor.This paper is the first attempt at learning the label set distribution as a reranking module for medical code prediction.In the experiments, our proposed framework is able to improve upon best-performing predictors on the benchmark MIMIC datasets. 1* Equal contribution. 1 The source code of this project is available at https://github.com/
On the information leakage quantification of camera fingerprint estimates

Abstract Camera fingerprints based on sensor PhotoResponse Non-Uniformity (PRNU) have gained broad popularity in forensic applications due to their ability to univocally identify the camera that captured a certain image. The fingerprint of a given sensor is extracted through some estimation method that requires a few images known to be taken with such sensor. In this paper, we show that the fingerprints extracted in this way leak a considerable amount of information from those images used in the estimation, thus constituting a potential threat to privacy. We propose to quantify the leakage via two measures: one based on the Mutual Information, and another based on the output of a membership inference test. Experiments with practical fingerprint estimators on a real-world image dataset confirm the validity of our measures and highlight the seriousness of the leakage and the importance of implementing techniques to mitigate it. Some of these techniques are presented and briefly discussed.
Enabling Language Representation with Knowledge Graph and Structured Semantic Information

Pre-trained language models have been widely recognized and applied. While common pre-training language representation models(PLMs) usually focus on grasping the co-occurrence of words or sentences in simple tasks, more and more researchers realize that external information, i.e., knowledge graph (KG) and clear structured semantics, can be vital in natural language understanding tasks. Therefore, using external information to enhance PLMs (such as BERT) has gradually become a popular direction. However, the existing improvement methods often only use a certain type of external information, and it is difficult to solve the problems of common PLMs that lack common sense and semantic incompleteness in one fell swoop. Suppose the model wants to integrate multiple external information. In that case, it not only requires the model to deal with the noise problem that external information may bring but also requires the model to ensure that different information can work together effectively. In this paper, we propose Sem-K-BERT, which integrates the information of KG and semantic role labeling(SRL) before and after the BERT encoding layer, and introduces a context-aware knowledge screening mechanism based on semantic correlation calculation and a text-semantic alignment mechanism to effectively integrate the two external information and reduce the impact of noise. Experiments and analysis on 8 different Chinese natural language processing tasks show that Sem-K-BERT has better performance than BERT and the BERT model that only incorporates KG. This indicates that the simultaneous use of knowledge graph and SRL information offers a promising solution to improve the performance of PLMs.
Extending the Kubernetes API

Kubernetes API is the brain of the cloud-native container management system; it makes you feel admiration, respect, and amazement at the same time. It is a complex API with multiple layers, various resources, and, fortunately, two extension points. This chapter will focus on extending the Kubernetes API by creating custom resources and API aggregation. At the end of this chapter, you will be able to create custom resources and controllers; namely, you will implement a Kubernetes operator. In addition, you will create and deploy extension API servers and use aggregated APIs in action.
Quantifying the Effects of COVID-19 on Restaurant Reviews

The COVID-19 pandemic has implications beyond physical health, affecting society and economies.Government efforts to slow down the spread of the virus have had a severe impact on many businesses, including restaurants.Mandatory policies such as restaurant closures, bans on social gatherings, and social distancing restrictions have affected restaurant operations as well as customer preferences (e.g., prompting a demand for stricter hygiene standards).As of now, however, it is not clear how and to what extent the pandemic has affected restaurant reviews, an analysis of which could potentially inform policies for addressing this ongoing situation.5 Hygiene: "masks," "gloves," "mask," "glove," "shield," "sanitize," "sanitizer," "sanitizing," "disinfect," "disinfecting," "face cover," "covering face," "face covers," "wipe," "wiping," and "wipes."Transmission: "cough," "spread," "infected," "cautious," "potential germs," "concerning," "worried," "covid test," "tested positive," and "asymptomatic."Social Distancing: "social distance," "social distancing," "six feet," "6 feet," "spaced out," "6ft," and "distanced."Racism: "racist," "xenophobia," "racism," "race," "xenophobic," "asian," and "asians."Sympathy and Support: "small business," "local business," "struggling," "support," "stress," "stressful," "suffer," "sympathy," and "stressed."Service: "takeout," "outdoor dining," "take out," "re-stocked," "restocked," "curbside pickup," "online order," "rude," and "service."Other: "covid," "pandemic," "quarantine," "covid19," "lockdown," "shutdown," and "cdc."
Multi-Feature Guided Low-Light Image Enhancement

Due to the characteristics of low signal-to-noise ratio and low contrast, low-light images will have problems such as color distortion, low visibility, and accompanying noise, which will cause the accuracy of the target detection problem to drop or even miss the detection target. However, recalibrating the dataset for this type of image will face problems such as increased cost or reduced model robustness. To solve this kind of problem, we propose a low-light image enhancement model based on deep learning. In this paper, the feature extraction is guided by the illumination map and noise map, and then the neural network is trained to predict the local affine model coefficients in the bilateral space. Through these methods, our network can effectively denoise and enhance images. We have conducted extensive experiments on the LOL datasets, and the results show that, compared with traditional image enhancement algorithms, the model is superior to traditional methods in image quality and speed.
Generating Cryptographic S-Boxes Using the Reinforcement Learning

Substitution boxes (S-boxes) are essential components of many cryptographic primitives. The Dijkstra algorithm, SAT solvers, and heuristic methods have been used to find bitsliced implementations of S-boxes. However, it is difficult to apply these methods for 8-bit S-boxes because of their size. Therefore, to implement these S-boxes so that the countermeasure of side-channel attack can be applied efficiently, using structures such as Feistel, Lai-Massey, and MISTY that can be bitsliced implemented with a small number of nonlinear operations has been widely used. Since S-boxes constructed with structures consist of small S-boxes and have specific designs, there are limitations to their cryptographic security and efficiency. In this paper, we propose a new method for generating S-boxes by stacking bitwise operations from the identity function, an approach that is different from existing methods. This method can be expressed in Markov decision process, and reinforcement learning is a suitable solver for Markov decision process. Our goal is to train this method to an agent through reinforcement learning to generate S-boxes to which the masking scheme, which is a countermeasure of side-channel attack, can be efficiently applied. In particular, our method provided various S-boxes superior or comparable to existing S-boxes. We produced 8-bit S-boxes with differential uniformity 16 (resp. 32) and linearity 128 (resp. 128), generated with nine (resp. eight) nonlinear operations, for the first time. To our best knowledge, this is the first study to construct cryptographic S-Box by incorporating reinforcement learning.
The Application of TBLT to English Reading Teaching in Junior High School

This paper demonstrates the restriction of traditional English reading teaching means, and further debates the feasibility and validity of TBLT in junior high school English reading teaching, which include the devise of reading teaching activities, the application of TBLT during teaching process in class and the evaluation of teachers and students after class. The intent of this paper is to dispose some drawbacks in English reading teaching in junior high school, in order to provide a feasible teaching method for English teachers and efficient learning methods for students.
SB_NITK at MEDIQA 2021: Leveraging Transfer Learning for Question Summarization in Medical Domain

Recent strides in the healthcare domain, have resulted in vast quantities of streaming data available for use for building intelligent knowledge-based applications.However, the challenges introduced to the huge volume, velocity of generation, variety and variability of this medical data have to be adequately addressed.In this paper, we describe the model and results for our submission at MEDIQA 2021 Question Summarization shared task.In order to improve the performance of summarization of consumer health questions, our method explores the use of transfer learning to utilize the knowledge of NLP transformers like BART, T5 and PEGASUS.The proposed models utilize the knowledge of pre-trained NLP transformers to achieve improved results when compared to conventional deep learning models such as LSTM, RNN etc.Our team SB_NITK ranked 12 th among the total 22 submissions in the official final rankings.Our BART based model achieved a ROUGE-2 F1 score of 0.139.
Event Representation with Sequential, Semi-Supervised Discrete Variables

Within the context of event modeling and understanding, we propose a new method for neural sequence modeling that takes partially-observed sequences of discrete, external knowledge into account.We construct a sequential neural variational autoencoder, which uses Gumbel-Softmax reparametrization within a carefully defined encoder, to allow for successful backpropagation during training.The core idea is to allow semisupervised external discrete knowledge to guide, but not restrict, the variational latent parameters during training.Our experiments indicate that our approach not only outperforms multiple baselines and the state-of-the-art in narrative script induction, but also converges more quickly.
Studi Korelasi Motivasi Belajar terhadap Hasil Belajar Matematika Selama Daring

The purpose of this study was to determine how much the relationship between learning motivation and learning outcomes online mathematics at SMP N 3 Wates. This research includes research with a quantitative approach and the type of research is ex post facto because it does not provide treatment to the research sample and the form of the relationship is a causal relationship. The population in this study were students of SMP N 3 Wates. The sampling technique was using cluster random sampling technique. The data collection techniques used were documentation and a learning motivation questionnaire. This study uses inferential statistical data analysis techniques, with several prerequisite tests. The results of this study indicate that the relationship between learning motivation and online mathematics learning outcomes at SMP N 3 Wates is in a low category.
UML Sequence Diagram: An Alternative Model

The UML sequence diagram is the second most common UML diagram that represents how objects interact and exchange messages over time.Sequence diagrams show how events or activities in a use case are mapped into operations of object classes in the class diagram.The general acceptance of sequence diagrams can be attributed to their relatively intuitive nature and ability to describe partial behaviors (as opposed to such diagrams as state charts).However, studies have shown that over 80% of graduating students were unable to create a software design or even a partial design, and many students had no idea how sequence diagrams were constrained by other models.Many students exhibited difficulties in identifying valid interacting objects and constructing messages with appropriate arguments.Additionally, according to authorities, even though many different semantics have been proposed for sequence diagrams (e.g., translations to state machines), there exists no suitable semantic basis refinement of required sequence diagram behavior because direct style semantics do not precisely capture required sequence diagram behaviors; translations to other formalisms disregard essential features of sequence diagrams such as guard conditions and critical regions.This paper proposes an alternative to sequence diagrams, a generalized model that provides further understanding of sequence diagrams to assimilate them into a new modeling language called thinging machine (TM).The sequence diagram is extended horizontally by removing the superficial vertical-only dimensional limitation of expansion to preserve the logical chronology of events.TM diagramming is spread nonlinearly in terms of actions.Events and their chronology are constructed on a second plane of description that is superimposed on the initial static description.The result is a more refined representation that would simplify the modeling process.This is demonstrated through remodeling sequence diagram cases from the literature.
HYBRID APPROACH OF GARBAGE CLASSIFICATION USING COMPUTER VISION AND DEEP LEARNING

As waste segregation becomes an important issue in our lives, with the use of technology like deep neural networks and computer vision, we can make the process efficient and robust by image segmentation and classification. These systems on the rise need accurate and efficient segmentation and recognition mechanisms and this demand coincides with the increase of computational capabilities of modern computer architectures and more effective algorithms for image recognition. This paper does a comparative analysis of various different approaches and methods like Simple CNN, ResNet50, VGG16, etc in brief. The comparative analysis and study explains the performance of every approach, this paper concludes that ResNet50 gives excellent performance. VGG16 network also provides good performance which meets the needs of daily use.
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation

The shortcomings of maximum likelihood estimation in the context of model-based reinforcement learning have been highlighted by an increasing number of papers. When the model class is misspecified or has a limited representational capacity, model parameters with high likelihood might not necessarily result in high performance of the agent on a downstream control task. To alleviate this problem, we propose an end-to-end approach for model learning which directly optimizes the expected returns using implicit differentiation. We treat a value function that satisfies the Bellman optimality operator induced by the model as an implicit function of model parameters and show how to differentiate the function. We provide theoretical and empirical evidence highlighting the benefits of our approach in the model misspecification regime compared to likelihood-based methods.
A Novel Framework Using Zero Shot Learning Technique for a Non-Factoid Question Answering System

Non-Factoid Question Answering (QA) is the next generation of textual QA systems, which gives passage level summaries for a natural language query, posted by the user. The main issue lies in the appropriateness of the generated summary. This paper proposes a framework for non-factoid QA system, which has three main components: (i) A deep neural network classifier, which produces sentence vector considering word correlation and context. (ii) Zero shot classifier that uses a multi-channel Convolutional Neural Network (CNN), to extract knowledge from multiple sources in the knowledge accumulator. This output acts as a knowledge enhancer that strengthens the passage level summary. (iii) Summary generator that uses Maximal Marginal Relevance (MMR) algorithm, which computes similarity among the query related answer and the sentences from zero shot classifier. This model is applied on the datasets WikiPassageQA and ANTIQUE. The experimental analysis shows that this model gives comparatively better results for WikiPassageQA dataset.
Automatic Classification of Neutralization Techniques in the Narrative of Climate Change Scepticism

Neutralisation techniques, e.g.denial of responsibility and denial of victim, are used in the narrative of climate change scepticism to justify lack of action or to promote an alternative view.We collect manual annotations of neutralised techniques used in these texts, and explore semi-supervised models to automatically classify them.1 https://www.wired.com/story/theanalogy-between-covid-19-and-climatechange-is-eerily-precise/ 2
Research on Blind Super-Resolution Technology for Infrared Images of Power Equipment Based on Compressed Sensing Theory

Infrared images of power equipment play an important role in power equipment status monitoring and fault identification. Aiming to resolve the problems of low resolution and insufficient clarity in the application of infrared images, we propose a blind super-resolution algorithm based on the theory of compressed sensing. It includes an improved blur kernel estimation method combined with compressed sensing theory and an improved infrared image super-resolution reconstruction algorithm based on block compressed sensing theory. In the blur kernel estimation method, we propose a blur kernel estimation algorithm under the compressed sensing framework to realize the estimation of the blur kernel from low-resolution images. In the estimation process, we define a new Lw norm to constrain the gradient image in the iterative process by analyzing the significant edge intensity changes before and after the image is blurred. With the Lw norm, the salient edges can be selected and enhanced, the intermediate latent image generated by the iteration can move closer to the clear image, and the accuracy of the blur kernel estimation can be improved. For the super-resolution reconstruction algorithm, we introduce a blur matrix and a regular total variation term into the traditional compressed sensing model and design a two-step total variation sparse iteration (TwTVSI) algorithm. Therefore, while ensuring the computational efficiency, the boundary effect caused by the block processing inside the image is removed. In addition, the design of the TwTVSI algorithm can effectively process the super-resolution model of compressed sensing with a sparse dictionary, thereby breaking through the reconstruction performance limitation of the traditional regularized super-resolution method of compressed sensing due to the lack of sparseness in the signal transform domain. The final experimental results also verify the effectiveness of our blind super-resolution algorithm.
CoRT: Complementary Rankings from Transformers

Many recent approaches towards neural information retrieval mitigate their computational costs by using a multi-stage ranking pipeline.In the first stage, a number of potentially relevant candidates are retrieved using an efficient retrieval model such as BM25.Although BM25 has proven decent performance as a first-stage ranker, it tends to miss relevant passages.In this context we propose CoRT, a simple neural first-stage ranking model that leverages contextual representations from pretrained language models such as BERT to complement term-based ranking functions while causing no significant delay at query time.Using the MS MARCO dataset, we show that CoRT significantly increases the candidate recall by complementing BM25 with missing candidates.Consequently, we find subsequent re-rankers achieve superior results with less candidates.We further demonstrate that passage retrieval using CoRT can be realized with surprisingly low latencies.
American Economic Association 2020–2021 Universal Academic Questionnaire Summary Statistics

American Economic Association 2020–2021 Universal Academic Questionnaire Summary Statistics by Charles E. Scott and John J. Siegfried. Published in volume 111, pages 647-49 of AEA Papers and Proceedings, May 2021