Reviewerly/exHarmony huggingface dataset:
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding

Coarse-grained linguistic information, such as named entities or phrases, facilitates adequately representation learning in pre-training.Previous works mainly focus on extending the objective of BERT's Masked Language Modeling (MLM) from masking individual tokens to contiguous sequences of n tokens.We argue that such contiguously masking method neglects to model the intra-dependencies and inter-relation of coarse-grained linguistic information.As an alternative, we propose ERNIE-Gram, an explicitly n-gram masking method to enhance the integration of coarse-grained information into pre-training.In ERNIE-Gram, n-grams are masked and predicted directly using explicit n-gram identities rather than contiguous sequences of n tokens.Furthermore, ERNIE-Gram employs a generator model to sample plausible n-gram identities as optional n-gram masks and predict them in both coarsegrained and fine-grained manners to enable comprehensive n-gram prediction and relation modeling.We pre-train ERNIE-Gram on English and Chinese text corpora and finetune on 19 downstream tasks.Experimental results show that ERNIE-Gram outperforms previous pre-training models like XLNet and RoBERTa by a large margin, and achieves comparable results with state-of-the-art methods.The source codes and pre-trained models have been released at https://github.com/PaddlePaddle/ERNIE.
Video-aided Unsupervised Grammar Induction

We investigate video-aided grammar induction, which learns a constituency parser from both unlabeled text and its corresponding video.Existing methods of multi-modal grammar induction focus on learning syntactic grammars from text-image pairs, with promising results showing that the information from static images is useful in induction.However, videos provide even richer information, including not only static objects but also actions and state changes useful for inducing verb phrases.In this paper, we explore rich features (e.g.action, object, scene, audio, face, OCR and speech) from videos, taking the recent Compound PCFG model (Kim et al., 2019) as the baseline.We further propose a Multi-Modal Compound PCFG model (MMC-PCFG) to effectively aggregate these rich features from different modalities.Our proposed MMC-PCFG is trained end-to-end and outperforms each individual modality and previous state-of-the-art systems on three benchmarks, i.e.DiDeMo, YouCook2 and MSRVTT, confirming the effectiveness of leveraging video information for unsupervised grammar induction.* This work was done when Songyang Zhang
Sonar image recognition of underwater target based on convolutional neural network

Underwater target recognition is one core technology of underwater unmanned detection. To improve the accuracy of underwater automatic target recognition, a sonar image recognition method based on convolutional neural network was proposed and the underwater target recognition model was established according to the characteristics of sonar images. Firstly, the sonar image was segmented and clipped with a saliency detection method to reduce the dimension of input data, and to reduce the interference of image background to the feature extraction process. Secondly, by using stacked convolutional layers and pooling layers, the high-level semantic information of the target was automatically learned from the input sonar image, to avoid damaging the effective information caused by extracting image features manually. Finally, the spatial pyramid pooling method was used to extract the multi-scale information from the sonar feature maps, which was to make up for the lack of detailed information of sonar images and solve the problem caused by the inconsistent size of input images. On the collected sonar image dataset, the experimental results show that the target recognition accuracy of the present method can recognize underwater targets more accurately and efficiently than the conventional convolutional neural networks.
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers

The COVID-19 pandemic has spawned a diverse body of scientific literature that is challenging to navigate, stimulating interest in automated tools to help find useful knowledge.We pursue the construction of a knowledge base (KB) of mechanisms-a fundamental concept across the sciences, which encompasses activities, functions and causal relations, ranging from cellular processes to economic impacts.We extract this information from the natural language of scientific papers by developing a broad, unified schema that strikes a balance between relevance and breadth.We annotate a dataset of mechanisms with our schema and train a model to extract mechanism relations from papers.Our experiments demonstrate the utility of our KB in supporting interdisciplinary scientific search over COVID-19 literature, outperforming the prominent PubMed search in a study with clinical experts.Our search engine, dataset and code are publicly available.
Calibrate Before Use: Improving Few-Shot Performance of Language Models

GPT-3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g., those that are placed near the end of the prompt or are common in the pre-training data. To mitigate this, we first estimate the model's bias towards each answer by asking for its prediction when given the training prompt and a content-free test input such as "N/A". We then fit calibration parameters that cause the prediction for this input to be uniform across answers. On a diverse set of tasks, this contextual calibration procedure substantially improves GPT-3 and GPT-2's average accuracy (up to 30.0% absolute) and reduces variance across different choices of the prompt.
GhostBERT: Generate More Features with Cheap Operations for BERT

Transformer-based pre-trained language models like BERT, though powerful in many tasks, are expensive in both memory and computation, due to their large number of parameters.Previous works show that some parameters in these models can be pruned away without severe accuracy drop.However, these redundant features contribute to a comprehensive understanding of the training data and removing them weakens the model's representation ability.In this paper, we propose GhostBERT, which generates more features with very cheap operations from the remaining features.In this way, GhostBERT has similar memory and computational cost as the pruned model, but enjoys much larger representation power.The proposed ghost module can also be applied to unpruned BERT models to enhance their performance with negligible additional parameters and computation.Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of our proposed method.
What does AI mean for smallholder farmers?

research-article Open Access Share on What does AI mean for smallholder farmers?: a proposal for farmer-centered AI research Authors: Courtney Heldreth Google GoogleView Profile , Diana Akrong Google GoogleView Profile , Jess Holbrook Google GoogleView Profile , Norman Makoto Su Indiana University Bloomington Indiana University BloomingtonView Profile Authors Info & Claims InteractionsVolume 28Issue 4July - August 2021 pp 56–60https://doi.org/10.1145/3468004Online:30 June 2021Publication History 1citation11,532DownloadsMetricsTotal Citations1Total Downloads11,532Last 12 Months10,273Last 6 weeks100 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF
Information security management in ICT and non-ICT sector companies: A preventive innovation perspective

Despite the growing dependence of companies on information technology and the increasingly negative impact of security incidents worldwide, there is little research on the management of information security at the company level. This paper seeks to expand knowledge on the implementation of an information security management system based on the widely used international standard ISO/IEC 27001. We present motives, experienced impacts, and obstacles related to ISO/IEC 27001 implementation using data from a survey of 125 ISO/IEC 27001 certified companies in Germany. Since adoption rates vary between ICT and non-ICT sector companies, we highlight sector-related variations. We classify the adoption of this standard as a preventive organizational innovation and apply Structural Equation Modeling to unearth explanations for the comparatively low adoption of this management system standard among companies outside the ICT sector. We, therefore, derive recommendations for policymakers, standardization, and certification bodies to foster its diffusion.
CANVA SEBAGAI MEDIA PEMBELAJARAN PADA MATA PELAJARAN DASAR LISTRIK DAN ELEKTRONIKA

This research was conducted to find out the feasibility of using canva learning media in Basic Electrical and Electronics subjects in class X Cooling and Air Conditioning Engineering at SMK Negeri 3 Pariaman in the academic year 2018/2019. The research method used is the research development of the Instructional Development Institute. The research instrument used was a questionnaire. Respondents in the questionnaire were experts, teachers, and students. In canva learning media, the validity, reliability and practicality of the media are tested. Based on the results of validation by experts and teachers, the average value of validation is 0.83 which is at the validity value of ≥0.667, so it is in the valid category. In the media reliability test, the Cronbach's Alpha value was 0.731, so the criteria for media reliability were in high criteria. In the practicality test of the media by teachers and students, the average values were 90% and 86% respectively, so the practicality category of the media was in the very practical category. Based on the results of the three tests it can be seen that canva learning media is feasible to use in the learning processKeywords: Canva Learning Media, Media Validity, Media Reliability, Media Practicality.
Syntax-Enhanced Pre-trained Model

We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa.Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages.Such a problem would lead to the necessity of having human-annotated syntactic information, which limits the application of existing methods to broader scenarios.To address this, we present a model that utilizes the syntax of text in both pre-training and fine-tuning stages.Our model is based on Transformer with a syntax-aware attention layer that considers the dependency tree of the text.We further introduce a new pre-training task of predicting the syntactic distance among tokens in the dependency tree.We evaluate the model on three downstream tasks, including relation classification, entity typing, and question answering.Results show that our model achieves state-of-the-art performance on six public benchmark datasets.We have two major findings.First, we demonstrate that infusing automatically produced syntax of text improves pre-trained models.Second, global syntactic distances among tokens bring larger performance gains compared to local head relations between contiguous tokens.
Query-Memory Re-Aggregation for Weakly-supervised Video Object Segmentation

Weakly-supervised video object segmentation (WVOS) is an emerging video task that can track and segment the target given a simple bounding box label. However, existing WVOS methods are still unsatisfied in either speed or accuracy, since they only use the exemplar frame to guide the prediction while they neglect the reference from other frames. To solve the problem, we propose a novel Re-Aggregation based framework, which uses feature matching to efficiently find the target and capture the temporal dependencies from multiple frames to guide the segmentation. Based on a two-stage structure, our framework builds an information-symmetric matching process to achieve robust aggregation. In each stage, we design a Query-Memory Aggregation (QMA) module to gather features from the past frames and make bidirectional aggregation to adaptively weight the aggregated features, which relieves the latent misguidance in unidirectional aggregation. To further exploit the information from different aggregation stages, we propose a novel coarse-fine constraint by using the Cascaded Refinement Module (CRM) to combine the predictions from different stages and further boosts the performance. Experimental results on three benchmarks show that our method achieves the state-of-the-art performance in WVOS (e.g., an overall score of 84.7% on the DAVIS 2016 validation set).
Do Context-Aware Translation Models Pay the Right Attention?

Context-aware machine translation models are designed to leverage contextual information, but often fail to do so.As a result, they inaccurately disambiguate pronouns and polysemous words that require context for resolution.In this paper, we ask several questions: What contexts do human translators use to resolve ambiguous words?Are models paying large amounts of attention to the same context?What if we explicitly train them to do so?To answer these questions, we introduce SCAT (Supporting Context for Ambiguous Translations), a new English-French dataset comprising supporting context words for 14K translations that professional translators found useful for pronoun disambiguation.Using SCAT, we perform an in-depth analysis of the context used to disambiguate, examining positional and lexical characteristics of the supporting words.Furthermore, we measure the degree of alignment between the model's attention scores and the supporting context from SCAT, and apply a guided attention strategy to encourage agreement between the two. 1
Design patterns and framework for blockchain integration in supply chains

The design and implementation of a blockchain as an infrastructure for a supply chain with efficient tracking and tracing is a recurring software design problem. The design pattern "Blockchain Data Management (BCDM)" provides a meta-model, chaincode for data object management and a REST-API for integration in a supply chain infrastructure. The analysis is grounded in the experiences of the NutriSafe research project with its scenarios and its use of the Hyperledger Fabric framework as technical platform.
How to Increase the Customer Experience by the Usage of Remote Control Robot Concierge Solutions

Abstract The objective of this study was to understand how consumers react and respond to a service experience provided by the robot, Pepper. To form our predictions, we applied and tested an extended version of the technology acceptance model (TAM). For the method, pre-recorded gestures were used to attract people to interact with Pepper, and participants were asked to pose questions to the robot for 5 to 10 min. Then, we distributed a questionnaire to these participants to gather data on their impressions of the interaction. Based on a final sample of 180 respondents, we used structural equation modeling (SEM) to test our hypotheses. Our results showed the fundamental importance of Social Presence when it comes to the use of robots. Social presence had a direct positive effect on Usefulness , Trust , and Emotional Appeal, and significant indirect effects through all three paths on Intention to Use . Moreover, The Fear of Robots negatively impacted the Intention to Use . We also found that when people were in discussion with the robot, they completely forgot that a human concierge is controlling the robot from another place. This study provides important insights into how and why consumers react to a service-providing robot, which is an important stream of research as this technology becomes more feasibly implemented in the hospitality industry.
RLC-GNN: An Improved Deep Architecture for Spatial-Based Graph Neural Network with Application to Fraud Detection

Graph neural networks (GNNs) have been very successful at solving fraud detection tasks. The GNN-based detection algorithms learn node embeddings by aggregating neighboring information. Recently, CAmouflage-REsistant GNN (CARE-GNN) is proposed, and this algorithm achieves state-of-the-art results on fraud detection tasks by dealing with relation camouflages and feature camouflages. However, stacking multiple layers in a traditional way defined by hop leads to a rapid performance drop. As the single-layer CARE-GNN cannot extract more information to fix the potential mistakes, the performance heavily relies on the only one layer. In order to avoid the case of single-layer learning, in this paper, we consider a multi-layer architecture which can form a complementary relationship with residual structure. We propose an improved algorithm named Residual Layered CARE-GNN (RLC-GNN). The new algorithm learns layer by layer progressively and corrects mistakes continuously. We choose three metrics—recall, AUC, and F1-score—to evaluate proposed algorithm. Numerical experiments are conducted. We obtain up to 5.66%, 7.72%, and 9.09% improvements in recall, AUC, and F1-score, respectively, on Yelp dataset. Moreover, we also obtain up to 3.66%, 4.27%, and 3.25% improvements in the same three metrics on the Amazon dataset.
Unsupervised Energy-based Adversarial Domain Adaptation for Cross-domain Text Classification

Transferring knowledge from a label-rich domain (source domain) to a label-scarce domain (target domain) for pervasive cross-domain Text Classification (TC) is a non-trivial task.To overcome this issue, we propose EADA, a novel unsupervised energy-based adversarial domain adaptation framework.First, a deep pre-trained language model (e.g.RoBERTa) is leveraged as a shared feature extractor that maps the text sequences from both source and target domains to a feature space.Since the source features maintain good feature discriminability because of the full supervised training, we design a method that encourages target features towards the source ones via adversarial learning.An autoencoder is designed as an energy function that focuses on reconstructing source feature embeddings, while the feature extractor aims to generate source-like target feature embeddings to deceive the autoencoder.In this manner, the target feature embeddings become domain-invariant and inherit great discriminability.Extensive experiments on multidomain sentiment classification (Amazon review dataset) and Yes/No question-answering classification (BoolQ and MARCO dataset) are conducted.The experimental results validate that EADA largely alleviates the domain discrepancy while maintaining excellent discriminability and achieves state-of-the-art cross-domain TC performance.
Retrospective Reader for Machine Reading Comprehension

Machine reading comprehension (MRC) is an AI challenge that requires machine to determine the correct answers to questions based on a given passage. MRC systems must not only answer question when necessary but also distinguish when no answer is available according to the given passage and then tactfully abstain from answering. When unanswerable questions are involved in the MRC task, an essential verification module called verifier is especially required in addition to the encoder, though the latest practice on MRC modeling still most benefits from adopting well pre-trained language models as the encoder block by only focusing on the "reading". This paper devotes itself to exploring better verifier design for the MRC task with unanswerable questions. Inspired by how humans solve reading comprehension questions, we proposed a retrospective reader (Retro-Reader) that integrates two stages of reading and verification strategies: 1) sketchy reading that briefly investigates the overall interactions of passage and question, and yield an initial judgment; 2) intensive reading that verifies the answer and gives the final prediction. The proposed reader is evaluated on two benchmark MRC challenge datasets SQuAD2.0 and NewsQA, achieving new state-of-the-art results. Significance tests show that our model is significantly better than the strong ELECTRA and ALBERT baselines. A series of analysis is also conducted to interpret the effectiveness of the proposed reader.
Data Protection in the Era of Artificial Intelligence: Trends, Existing Solutions and Recommendations for Privacy-Preserving Technologies

Abstract This chapter addresses privacy challenges that stem particularly from working with big data. Several classification schemes of such challenges are discussed. The chapter continues by classifying the technological solutions as proposed by current state-of-the-art research projects. Three trends are distinguished: (1) putting the end user of data services back as the central focal point of Privacy-Preserving Technologies, (2) the digitisation and automation of privacy policies in and for big data services and (3) developing secure methods of multi-party computation and analytics, allowing both trusted and non-trusted partners to work together with big data while simultaneously preserving privacy. The chapter ends with three main recommendations: (1) the development of regulatory sandboxes; (2) continued support for research, innovation and deployment of Privacy-Preserving Technologies; and (3) support and contribution to the formation of technical standards for preserving privacy. The findings and recommendations of this chapter in particular demonstrate the role of Privacy-Preserving Technologies as an especially important case of data technologies towards data-driven AI. Privacy-Preserving Technologies constitute an essential element of the AI Innovation Ecosystem Enablers (Data for AI).
Cross-lingual Text Classification with Heterogeneous Graph Neural Network

Cross-lingual text classification aims at training a classifier on the source language and transferring the knowledge to target languages, which is very useful for low-resource languages.Recent multilingual pretrained language models (mPLM) achieve impressive results in cross-lingual classification tasks, but rarely consider factors beyond semantic similarity, causing performance degradation between some language pairs.In this paper we propose a simple yet effective method to incorporate heterogeneous information within and across languages for cross-lingual text classification using graph convolutional networks (GCN).In particular, we construct a heterogeneous graph by treating documents and words as nodes, and linking nodes with different relations, which include part-of-speech roles, semantic similarity, and document translations.Extensive experiments show that our graph-based method significantly outperforms state-of-the-art models on all tasks, and also achieves consistent performance gain over baselines in low-resource settings where external tools like translators are unavailable.