Reviewerly/exHarmony huggingface dataset:
Robustness of Accuracy Metric and its Inspirations in Learning with Noisy Labels

For multi-class classification under class-conditional label noise, we prove that the accuracy metric itself can be robust. We concretize this finding's inspiration in two essential aspects: training and validation, with which we address critical issues in learning with noisy labels. For training, we show that maximizing training accuracy on sufficiently many noisy samples yields an approximately optimal classifier. For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping. Previously, model selection using noisy validation samples has not been theoretically justified. We verify our theoretical results and additional claims with extensive experiments. We show characterizations of models trained with noisy labels, motivated by our theoretical results, and verify the utility of a noisy validation set by showing the impressive performance of a framework termed noisy best teacher and student (NTS). Our code is released.
TIRA in Baidu Image Advertising

Since an image can be perceived by customers in few seconds, it is an effective medium for advertising and adored by advertisers. Baidu, as one of the lead search companies in the world, receives billions of text queries per day. How to feed attractive images to capture the customers' attentions is the core task of Baidu image advertising. Traditionally, the query-to-image search is tackled by matching the text query with the image title. Nevertheless, title-based image search relies on high-quality image titles, which are not easy to be obtained or unavailable in some cases. A more reliable solution is to understand the image content and conduct content-based query-to-image retrieval. In this paper, we introduce a text-image cross-modal retrieval for advertising (TIRA) model, which has been launched in Baidu image advertising. The proposed TIRA is built upon the popularly used image classification model, ResNet and the recent state-of-the-art NLP model, BERT. It targets to bridge the modal gap by mapping the images and texts into the same feature space. Meanwhile, we propose to use contrast loss to train the TIRA model, which consistently outperforms existing methods based on pairwise loss or triplet loss. Since the proposed TIRA model directly conducts the content-based query-to-image and image-to-query retrieval, and does not rely on high-quality labeled titles, it significantly enhances the search flexibility. The TIRA model has been deployed in image2X and query2X frameworks of Baidu image advertising. After the launch of TIRA, it has achieved considerable improvement in click-through-rate (CTR) and cost per mille (CPM), which brings considerable revenue increase for advertisers.
Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion

We present InferWiki, a Knowledge Graph Completion (KGC) dataset that improves upon existing benchmarks in inferential ability, assumptions, and patterns.First, each testing sample is predictable with supportive data in the training set.To ensure it, we propose to utilize rule-guided train/test generation, instead of conventional random split.Second, InferWiki initiates the evaluation following the open-world assumption and improves the inferential difficulty of the closed-world assumption, by providing manually annotated negative and unknown triples.Third, we include various inference patterns (e.g., reasoning path length and types) for comprehensive evaluation.In experiments, we curate two settings of InferWiki varying in sizes and structures, and apply the construction process on CoDEx as comparative datasets.The results and empirical analyses demonstrate the necessity and high-quality of InferWiki.Nevertheless, the performance gap among various inferential assumptions and patterns presents the difficulty and inspires future research direction.Our datasets can be found in https://github.com/TaoMiner/inferwiki.
Combining Feature and Instance Attribution to Detect Artifacts

Training the deep neural networks that dominate NLP requires large datasets.These are often collected automatically or via crowdsourcing, and may exhibit systematic biases or annotation artifacts.By the latter we mean spurious correlations between inputs and outputs that do not represent a generally held causal relationship between features and classes; models that exploit such correlations may appear to perform a given task well, but fail on out of sample data.In this paper we evaluate use of different attribution methods for aiding identification of training data artifacts.We propose new hybrid approaches that combine saliency maps (which highlight "important" input features) with instance attribution methods (which retrieve training samples "influential" to a given prediction).We show that this proposed training-feature attribution can be used to efficiently uncover artifacts in training data when a challenging validation set is available.We also carry out a small user study to evaluate whether these methods are useful to NLP researchers in practice, with promising results.We make code for all methods and experiments in this paper available. 1
Implementation of a STEAM project in compulsory secondary education that creates connections with the environment (<i>Implementación de un proyecto STEAM en Educación Secundaria generando conexiones con el entorno</i>)

The teaching of STEAM has been gaining ground in the world of education for some years now.This article presents two fundamental contributions to STEAM literature.The first contribution is the detailed description of a STEAM work experience with secondary school students using Project Based Learning, in which the disciplines of science and art are connected and integrated.To achieve this, connections established among the disciplines corresponding to S, T, E, A and M are analysed.The second contribution presented in this paper is the study of the relationships established between the members of the educational community and the external educational agents (artists, researchers and research centres) involved in the project.These relationships
Understand My World: An Interactive App for Children Learning Arabic Vocabulary

COVID-19 imposed a new paradigm in education especially in elementary schools. Children are no longer able to go regularly to schools as normal. They should then rely mainly on themselves to learn and acquire knowledge. The Understand My World app is a new technological solution that allows children to learn new Arabic vocabularies interactively and independently using their smart phones or tablets (i.e., iPad, iPhone). They can use the devices’ camera and microphone to explore the world, understand spoken words, and read written language properly. Images captured by the camera are recognized, labeled, and defined in both speech and writing. Children can also speak into their device to record their speech and listen to it. They can address questions and receive answers. The app then presents the dictated words in both spoken and written form. Using the required Internet connection, the interface automatically provides exceedingly accurate image and speech recognition, all while preserving the participants’ anonymity. We use two AI platforms namely Clarifai and Houndify. The application is designed to be intuitive and seamless to use, which makes it very attractive to children to learn through multimedia.
I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting

3D object classification has attracted appealing attentions in academic researches and industrial applications. However, most existing methods need to access the training data of past 3D object classes when facing the common real-world scenario: new classes of 3D objects arrive in a sequence. Moreover, the performance of advanced approaches degrades dramatically for past learned classes (i.e., catastrophic forgetting), due to the irregular and redundant geometric structures of 3D point cloud data. To address these challenges, we propose a new Incremental 3D Object Learning (i.e., I3DOL) model, which is the first exploration to learn new classes of 3D object continually. Specifically, an adaptive-geometric centroid module is designed to construct discriminative local geometric structures, which can better characterize the irregular point cloud representation for 3D object. Afterwards, to prevent the catastrophic forgetting brought by redundant geometric information, a geometric-aware attention mechanism is developed to quantify the contributions of local geometric structures, and explore unique 3D geometric characteristics with high contributions for classes incremental learning. Meanwhile, a score fairness compensation strategy is proposed to further alleviate the catastrophic forgetting caused by unbalanced data between past and new classes of 3D object, by compensating biased prediction for new classes in the validation phase. Experiments on 3D representative datasets validate the superiority of our I3DOL framework.
Two-dimensional complex wavelet with directional selectivity used in fringe projection profilometry

The continuous wavelet transform (CWT) has been used in fringe projection profilometry (FPP) to recover a three-dimensional surface from a single-frame fringe. This Letter constructs a two-dimensional (2D) continuous complex wavelet employing a 2D real Mexican hat wavelet function, combined with the single-orthant analytical 2D Hilbert transform. The 2D complex wavelet has an asymmetric frequency envelope in the radial direction and the ability of direction selectivity, which results in better matching between the local fringe and daughter wavelets. We provide a complex wavelet construction model, present a theoretical analysis, and conduct experimental validation. The experiments demonstrate that the proposed method provides high phase accuracy in the single-frame FPP method.
Disk compression of k-mer sets

Abstract K-mer based methods have become prevalent in many areas of bioinformatics. In applications such as database search, they often work with large multi-terabyte-sized datasets. Storing such large datasets is a detriment to tool developers, tool users, and reproducibility efforts. General purpose compressors like gzip, or those designed for read data, are sub-optimal because they do not take into account the specific redundancy pattern in k-mer sets. In our earlier work (Rahman and Medvedev, RECOMB 2020), we presented an algorithm UST-Compress that uses a spectrum-preserving string set representation to compress a set of k-mers to disk. In this paper, we present two improved methods for disk compression of k-mer sets, called ESS-Compress and ESS-Tip-Compress. They use a more relaxed notion of string set representation to further remove redundancy from the representation of UST-Compress. We explore their behavior both theoretically and on real data. We show that they improve the compression sizes achieved by UST-Compress by up to 27 percent, across a breadth of datasets. We also derive lower bounds on how well this type of compression strategy can hope to do.
Continuous LWE

We introduce a continuous analogue of the Learning with Errors (LWE) problem, which we name CLWE. We give a polynomial-time quantum reduction from worst-case lattice problems to CLWE, showing that CLWE enjoys similar hardness guarantees to those of LWE. Alternatively, our result can also be seen as opening new avenues of (quantum) attacks on lattice problems. Our work resolves an open problem regarding the computational complexity of learning mixtures of Gaussians without separability assumptions (Diakonikolas 2016, Moitra 2018). As an additional motivation, (a slight variant of) CLWE was considered in the context of robust machine learning (Diakonikolas et al.~FOCS 2017), where hardness in the statistical query (SQ) model was shown; our work addresses the open question regarding its computational hardness (Bubeck et al.~ICML 2019).
AOMD: An analogy-aware approach to offensive meme detection on social media

This paper focuses on an important problem of detecting offensive analogy meme on online social media where the visual content and the texts/captions of the meme together make an analogy to convey the offensive information. Existing offensive meme detection solutions often ignore the implicit relation between the visual and textual contents of the meme and are insufficient to identify the offensive analogy memes. Two important challenges exist in accurately detecting the offensive analogy memes: i) it is not trivial to capture the analogy that is often implicitly conveyed by a meme; ii) it is also challenging to effectively align the complex analogy across different data modalities in a meme. To address the above challenges, we develop a deep learning based Analogy-aware Offensive Meme Detection (AOMD) framework to learn the implicit analogy from the multi-modal contents of the meme and effectively detect offensive analogy memes. We evaluate AOMD on two real-world datasets from online social media. Evaluation results show that AOMD achieves significant performance gains compared to state-of-the-art baselines by detecting offensive analogy memes more accurately.
Investigating label suggestions for opinion mining in German Covid-19 social media

This work investigates the use of interactively updated label suggestions to improve upon the efficiency of gathering annotations on the task of opinion mining in German Covid-19 social media data.We develop guidelines to conduct a controlled annotation study with social science students and find that suggestions from a model trained on a small, expert-annotated dataset already lead to a substantial improvement -in terms of inter-annotator agreement (+.14 Fleiss' κ) and annotation quality -compared to students that do not receive any label suggestions.We further find that label suggestions from interactively trained models do not lead to an improvement over suggestions from a static model.Nonetheless, our analysis of suggestion bias shows that annotators remain capable of reflecting upon the suggested label in general.Finally, we confirm the quality of the annotated data in transfer learning experiments between different annotator groups.To facilitate further research in opinion mining on social media data, we release our collected data consisting of 200 expert and 2,785 student annotations. 1
End-to-End Construction of NLP Knowledge Graph

This paper studies the end-to-end construction of an NLP Knowledge Graph (KG) from scientific papers.We focus on extracting four types of relations: evaluatedOn between tasks and datasets, evaluatedBy between tasks and evaluation metrics, as well as coreferent and related relations between the same type of entities.For instance, "F1 score" is coreferent with "F-measure".We introduce novel methods for each of these relation types and apply our final framework (SciNLP-KG) to 30,000 NLP papers from ACL Anthology to build a large-scale KG, which can facilitate automatically constructing scientific leaderboards for the NLP community.The results of our experiments indicate that the resulting KG contains high-quality information.
A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment

Word alignment and machine translation are two closely related tasks.Neural translation models, such as RNN-based and Transformer models, employ a target-to-source attention mechanism which can provide rough word alignments, but with a rather low accuracy.High-quality word alignment can help neural machine translation in many different ways, such as missing word detection, annotation transfer and lexicon injection.Existing methods for learning word alignment include statistical word aligners (e.g.GIZA++) and recently neural word alignment models.This paper presents a bidirectional Transformer based alignment (BTBA) model for unsupervised learning of the word alignment task.Our BTBA model predicts the current target word by attending the source context and both leftside and right-side target context to produce accurate target-to-source attention (alignment).We further fine-tune the target-to-source attention in the BTBA model to obtain better alignments using a full context based optimization method and self-supervised training.We test our method on three word alignment tasks and show that our method outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++.
Effects of using mobile augmented reality for simple interest computation in a financial mathematics course

Understanding the concept of simple interest is essential in financial mathematics because it establishes the basis to comprehend complex conceptualizations. Nevertheless, students often have problems learning about simple interest. This paper aims to introduce a prototype called “simple interest computation with mobile augmented reality” (SICMAR) and evaluate its effects on students in a financial mathematics course. The research design comprises four stages: (i) planning; (ii) hypotheses development; (iii) software development; and (iv) design of data collection instruments. The planning stage explains the problems that students confront to learn about simple interest. In the second stage, we present the twelve hypotheses tested in the study. The stage of software development discusses the logic implemented for SICMAR functionality. In the last stage, we design two surveys and two practice tests to assess students. The pre-test survey uses the attention, relevance, confidence, and satisfaction (ARCS) model to assess students’ motivation in a traditional learning setting. The post-test survey assesses motivation, technology usage with the technology acceptance model (TAM), and prototype quality when students use SICMAR. Also, students solve practice exercises to assess their achievement. One hundred three undergraduates participated in both sessions of the study. The findings revealed the direct positive impact of SICMAR on students’ achievement and motivation. Moreover, students expressed their interest in using the prototype because of its quality. In summary, students consider SICMAR as a valuable complementary tool to learn simple interest topics.
COVID-19 mitigation by digital contact tracing and contact prevention (app-based social exposure warnings)

A plethora of measures are being combined in the attempt to reduce SARS-CoV-2 spread. Due to its sustainability, contact tracing is one of the most frequently applied interventions worldwide, albeit with mixed results. We evaluate the performance of digital contact tracing for different infection detection rates and response time delays. We also introduce and analyze a novel strategy we call contact prevention, which emits high exposure warnings to smartphone users according to Bluetooth-based contact counting. We model the effect of both strategies on transmission dynamics in SERIA, an agent-based simulation platform that implements population-dependent statistical distributions. Results show that contact prevention remains effective in scenarios with high diagnostic/response time delays and low infection detection rates, which greatly impair the effect of traditional contact tracing strategies. Contact prevention could play a significant role in pandemic mitigation, especially in developing countries where diagnostic and tracing capabilities are inadequate. Contact prevention could thus sustainably reduce the propagation of respiratory viruses while relying on available technology, respecting data privacy, and most importantly, promoting community-based awareness and social responsibility. Depending on infection detection and app adoption rates, applying a combination of digital contact tracing and contact prevention could reduce pandemic-related mortality by 20-56%.
COVID-19 Apps as a Digital Intervention Policy: A Longitudinal Panel Data Analysis in South Korea

Many countries have developed COVID-19 tracking apps that help individuals trace and detect “people” who are likely to have come in contact with confirmed patients. However, their adoption rates remain low. This study, therefore, investigated South Koreans’ adoption and usage behaviors of COVID-19 apps that detect the “place” where infectious people are found and alert people within 100m in dangerous zones. Our focus was on such apps’ impact on various facets of human life . Specifically, we analyzed mobile app usage data from 5,940 panelists at the start of the pandemic in South Korea and after the first major wave (January 6 to August 2, 2020). Findings showed that higher-income and more educated individuals were more likely to adopt COVID-19 apps early, and male and low-income people tended to use the COVID-19 tracking apps more frequently. In addition, this study offered empirical evidence of health protective behaviors, such as driving, shopping online, ordering food online, and avoiding travel and public transportation, and supported social- and religious-coping for people using COVID-19 apps. The implications are valuable for policy makers to implement a digital policy to motivate people to voluntarily engage in self-protective and coping behaviors through COVID-19 apps.
Tripal, a community update after 10 years of supporting open source, standards-based genetic, genomic and breeding databases

Online, open access databases for biological knowledge serve as central repositories for research communities to store, find and analyze integrated, multi-disciplinary datasets. With increasing volumes, complexity and the need to integrate genomic, transcriptomic, metabolomic, proteomic, phenomic and environmental data, community databases face tremendous challenges in ongoing maintenance, expansion and upgrades. A common infrastructure framework using community standards shared by many databases can reduce development burden, provide interoperability, ensure use of common standards and support long-term sustainability. Tripal is a mature, open source platform built to meet this need. With ongoing improvement since its first release in 2009, Tripal provides full functionality for searching, browsing, loading and curating numerous types of data and is a primary technology powering at least 31 publicly available databases spanning plants, animals and human data, primarily storing genomics, genetics and breeding data. Tripal software development is managed by a shared, inclusive governance structure including both project management and advisory teams. Here, we report on the most important and innovative aspects of Tripal after 11 years development, including integration of diverse types of biological data, successful collaborative projects across member databases, and support for implementing FAIR principles.