Reviewerly/exHarmony huggingface dataset:
Local Averaging Helps: Hierarchical Federated Learning and Convergence Analysis.

Federated learning is an effective approach to realize collaborative learning among edge devices without exchanging raw data. In practice, these devices may connect to local hubs instead of connecting to the global server (aggregator) directly. Due to the (possibly limited) computation capability of these local hubs, it is reasonable to assume that they can perform simple averaging operations. A natural question is whether such local averaging is beneficial under different system parameters and how much gain can be obtained compared to the case without such averaging. In this paper, we study hierarchical federated learning with stochastic gradient descent (HF-SGD) and conduct a thorough theoretical analysis to analyze its convergence behavior. In particular, we first consider the two-level HF-SGD (one level of local averaging) and then extend this result to arbitrary number of levels (multiple levels of local averaging). The analysis demonstrates the impact of local averaging precisely as a function of system parameters. Due to the higher communication cost of global averaging, a strategy of decreasing the global averaging frequency and increasing the local averaging frequency is proposed. Experiments validate the proposed theoretical analysis and the advantages of HF-SGD.
Leveraging technology for healthcare and retaining access to personal health data to enhance personal health and well-being

Health data are a sensitive category of personal data. They can result in a high risk to individuals and health information-handling rights and opportunities unless there is a sufficient defense. Reasonable security standards are needed to protect electronic health records (EHRs). All personal data handling needs adequate explanation. Maintaining access to medical data, even in the developing world, would favor health and well-being across the world. Unfortunately, there are still countries that hinder the portability of medical records. Numerous occurrences have shown that it still takes weeks for medical data to be ported from one general physician to another. Cross-border portability is almost impossible due to the lack of technical infrastructure and standardization. The authors demonstrate the difficulty of the portability of medical records with some example case studies as a collaborative engagement exercise through a data-mapping process to describe how different people and datapoints interact and evaluate EHR portability techniques. They then propose a blockchain-based EHR system that allows secure and cross-border sharing of medical data. The ethical and technical challenges around having such a system are also been discussed.
Glioblastomas brain Tumor Segmentation using Optimized U-Net based on Deep Fully Convolutional Networks (D-FCNs)

Manual segmentation during clinical diagnosis, is considered as time-consuming and depend to the neuroradiologists level of expertise, however due to the large spatial and structural variability of brain tumors in shapes and sizes besides to the tumor sub-region voxels'high in-homogeneity could make a reliable and accurate and automated segmentation a challenging task. We proposed in this paper, an efficient and fully automatic deep-learning approach for Gliomas `brain tumor segmentation in multi-sequences Magnetic Resonance imaging (MRI). The proposed method is an optimization on the U-Net based on Fully Convolutional Networks (FCNs) called `U-Net DFCN' in which we introduced the fusion of multiple MRI modalities to incorporate features from different scales, furthermore, to address the problem of data heterogeneity due to difference in acquisition algorithms and MRI scanner technologies, we proposed an intensity normalization followed by data augmentation techniques in the preprocessing step which though not conventional (usual) in deep FCN-based segmentation approaches. Our method was evaluated on the Multimodal Brain Tumor Image Segmentation (BRATS 2018) training and validation datasets, experimental resulted showed the good performance of the proposed approach outperforming several recent state-of-the-art segmentation methods, achieving a Dice score Coefficient (DSC) of 0.88, 0.87 and 0.81 for complete tumor, tumor-core and enhancing-tumor respectively.
Tibetan Text Classification Method Based on BiLSTM Model

Text classification is a key technology in the field of information retrieval and data mining. It can effectively solve the problem of information clutter and locate effective information. This paper proposes a method of merging Word2vec and TF-IDF Tibetan text representation based on class frequence variance. Based on the representation method, BiLSTM network model is used to classify Tibetan text. First of all, it proposes to perform pre-processing work such as word segmentation on the Tibetan classification text, construction of a basic stop word list, and calculation of word frequency. Then the text representation uses the method of merging Word2vec and the TF-IDF algorithm based on class frequence variance, which takes into account both the importance of words and the distribution of words. Finally, the word vector is transmitted to the classification model to train the Tibetan text classifier, and the trained classifier is used to classify the unclassified Tibetan text. The experimental results show that the text representation method combined with Word2vec and TF-IDF based on class frequency variance can effectively improve the effect of text classification. The accuracy of Tibetan text classifier based on BiLSTM can reach 89.03%, which is significantly better than RNN LSTM.
Improvement of sentiment analysis via re-evaluation of objective words in SenticNet for hotel reviews

In order to extract the correct sentiment polarity from word of mouth (WOM), a wide-scale and well-organized sentiment lexicon is generally beneficial. SenticNet is one such lexicon. However, it consists of a high proportion of objective words, which are generally considered to be of little use for sentiment classification due to their ambiguity and lack of sentiments. In the literature, there is a dearth of models that focus on this issue. An objective word appearing more frequently in positive sentences than in negative sentences implies a strong relationship in a positive sentiment orientation, and conversely, an objective word appearing more frequently in negative sentences implies a strong relationship in a negative sentiment orientation. Thus, the ratio of an objective word appearing in positive and negative sentences provides the sentiment orientation. Based on this concept, this paper re-assigns the sentiment values to the objective words in SenticNet and builds a revised SenticNet. Three classification techniques, the J48 decision tree, support vector machine, and multilayer perceptron neural network are used for classification. According to the experiments, the proposed models which extract sentiment values from the revised SenticNet, significantly outperform those models which extract sentiment values from the original non-revised SenticNet.
FastFormers: Highly Efficient Transformer Models for Natural Language Understanding

Transformer-based models are the state-of-the-art for Natural Language Understanding (NLU) applications. Models are getting bigger and better on various tasks. However, Transformer models remain computationally challenging since they are not efficient at inference-time compared to traditional approaches. In this paper, we present FastFormers, a set of recipes to achieve efficient inference-time performance for Transformer-based models on various NLU tasks. We show how carefully utilizing knowledge distillation, structured pruning and numerical optimization can lead to drastic improvements on inference efficiency. We provide effective recipes that can guide practitioners to choose the best settings for various NLU tasks and pretrained models. Applying the proposed recipes to the SuperGLUE benchmark, we achieve from 9.8x up to 233.9x speed-up compared to out-of-the-box models on CPU. On GPU, we also achieve up to 12.4x speed-up with the presented methods. We show that FastFormers can drastically reduce cost of serving 100 million requests from 4,223 USD to just 18 USD on an Azure F16s_v2 instance. This translates to a sustainable runtime by reducing energy consumption 6.9x - 125.8x according to the metrics used in the SustaiNLP 2020 shared task.
Mapping the “long tail” of research funding: A topic analysis of <scp>NSF</scp> grant proposals in the division of astronomical sciences

Abstract “Long tail” data are considered to be smaller, heterogeneous, researcher‐held data, which present unique data management and scholarly communication challenges. These data are presumably concentrated within relatively lower‐funded projects due to insufficient resources for curation. To better understand the nature and distribution of long tail data, we examine National Science Foundation (NSF) funding patterns using Latent Dirichlet Allocation (LDA) and bibliographic data. We also introduce the concept of “Topic Investment” to capture differences in topics across funding levels and to illuminate the distribution of funding across topics. This study uses the discipline of astronomy as a case study, overall exploring possible associations between topic, funding level and research output, with implications for research policy and practice. We find that while different topics demonstrate different funding levels and publication patterns, dynamics predicted by the “long tail” theoretical framework presented here can be observed within NSF‐funded topics in astronomy.
Cryptocurrencies and entrepreneurial finance

This chapter explores how cryptocurrencies and the underlying blockchain technology affect entrepreneurship, by providing an overview on the initial coin offerings (ICO) phenomenon and outlining its advantages and risks. ICOs are a new form of fund-raising from the public that leverages the power of cryptocurrencies and the underling blockchain technology. One of the key features of an ICO is that it works on a distributed ledger technology or blockchain. This enables new ventures to expand their funding opportunities compared to traditional crowdfunding. By relying on blockchain technology, ICOs can reduce the friction in fund-raising, ease access to capital and thereby promote entrepreneurship and innovation. Future studies may investigate the effectiveness of ICOs in terms of their returns and successes after the offerings, as well as their consequent ability to foster entrepreneurship. Research on the role played by the human and social capital of the founders is also needed in explaining ICOs outcomes.
Customizing Triggers with Concealed Data Poisoning.

Adversarial attacks alter NLP model predictions by perturbing test-time inputs. However, it is much less understood whether, and how, predictions can be manipulated with small, concealed changes to the training data. In this work, we develop a new data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input. For instance, we insert 50 poison examples into a sentiment model's training set that causes the model to frequently predict Positive whenever the input contains James Bond. Crucially, we craft these poison examples using a gradient-based procedure so that they do not mention the trigger phrase. We also apply our poison attack to language modeling (Apple iPhone triggers negative generations) and machine translation (iced coffee mistranslated as hot coffee). We conclude by proposing three defenses that can mitigate our attack at some cost in prediction accuracy or extra human annotation.
Investigating the mediating role of organizational trust in the relationship between perceived organizational support and Knowledge sharing

Knowledge sharing is vital for any organization, and it has an essential impact on developing ‎‎skills, increasing value, and maintaining a competitive advantage in the organization. In the knowledge management (KM) literature, ‎‎perceived organizational support and organizational trust have been pointed out as ‎‎primary factors for knowledge sharing. The present study investigated the role of ‎‎perceived organizational support in promoting nurses’ knowledge sharing, considering ‎‎organizational trust as a mediator‎. The cross-sectional study design was applied to examine the ‏hypothesized relationships.‏ ‎Data were ‎‎collected using three questionnaires from three hundred thirty ‎‎nurses working in five public hospitals in Shiraz city. The Partial Least Squares Structural ‎‎Equation Modeling (PLS-SEM) approach was used to analyze data. The results revealed a significant and positive association between perceived organizational support, organizational trust, and knowledge sharing. The findings also demonstrated that organizational trust plays a mediating role between perceived organizational support and knowledge sharing.
Life cycle support software components

Modern software development is based on a systems approach, in which a program or software complex is considered as a system of interacting software components. Models of software components are analogs of complex system subsystems. Therefore, a complex program is considered as a system of software components. The organization of the structure of software components affects the quality and result of the program. The organization of interaction between software components affects the efficiency of the program. An important factor in the system of software components is the life cycle, which determines the effectiveness and feasibility of using this program. Software differs from many complex systems and information systems in that it has the ability to increase its life cycle. Moreover, the need to increase the life cycle is characterized by two factors: external and internal. The internal factor arises due to the obsolescence of the program. In this case, it does not meet the new conditions, for example, a new operating system. The external factor arises from external influences in the form of interference or purposeful actions, such as computer viruses. The problem of creating the structure of software components of computing systems and information systems that ensure the duration of the life cycle in the presence of external influences is topical. The study of this problem contributes to the improvement of the technological base of computing systems and information systems that solve applied problems. The article presents a new life cycle model based on two models of growth and degradation. The article recommends a resource-based approach for life cycle assessment. As an analytical solution, it is proposed to use a logistic equation, which describes the mechanisms of the life cycle formation process quite well. The article discusses three types of resource in calculations: physical, technological and communicative. A general redundancy solution is proposed to create a network with the inclusion of a multigraph model.
RETRACTED: Research on College English Online Learning Platform Model Based on Big Data Technology

Abstract In this era of rapid development, the level of human science and technology is constantly improving. With the continuous development of the times and the continuous progress of society, big data technology has been widely used in our daily production and life. The research of College English online learning platform model is a hot topic in the teaching of many college English majors. In order to better study the model of College English online learning platform, many colleges and universities put forward the method of using big data technology to analyze the research of College English network learning platform model, through the big data technology to college English Based on the prediction of the development of e-learning platform model, combined with the new requirements for the research of College English online learning platform model in this era of rapid development, this paper provides guidance for the research of College English online learning platform model in China. Through the analysis, it is found that the method of using big data technology to analyze the model of College English online learning platform has a profound guiding role in the research of College English online learning platform model.
Attention-based Convolutional Neural Network for Answer Selection using BERT

Question answering is at the heart of natural language processing and is composed of two sections: Reading Comprehension and Answer Selection. Prior to deep learning, all natural language processing solutions including Question Answering were based on statistical methods and researchers generated set of features based on text input. Answer Selection is a fundamental task in Question Answering, also a tough one because of the complicated semantic relations between questions and answers. Attention is a mechanism that has revolutionized deep learning community. Leveraging pretrained language models have made a breakthrough in most natural language processing tasks. Bert is one of the top pretrained deep language models that has achieved state-of-the-art on an extensive area of nlp tasks. In this paper we utilize an attention-based convolutional neural network. First, we employ BERT, a state-of-the-art pre-trained contextual as the embedding layer. Second, we enhance the model by adding some more attentive features. We evaluate the performance of our model on WikiQA dataset. Our experiments show that our model is superior to many other answer-selection models.
Graph Domain Adaptation for Alignment-Invariant Brain Surface Segmentation

The varying cortical geometry of the brain creates numerous challenges for its analysis. Recent developments have enabled learning cortical data directly across multiple brain surfaces via graph convolutions. However, current graph learning algorithms fail when brain surface data are misaligned across subjects, thereby requiring to apply a costly alignment procedure in pre-processing. Adversarial training is widely used for unsupervised domain adaptation to improve segmentation performance on target data whose distribution differs from the training source data. In this paper, we exploit this technique to learn surface data across inconsistent graph alignments. This novel approach comprises a segmentator that uses graph convolution layers to enable parcellation across brain surfaces of varying geometry, and a discriminator that predicts the alignment-domain of surfaces from their segmentation. By trying to fool the discriminator, the adversarial training learns an alignment-invariant representation which yields consistent parcellations for differently-aligned surfaces. Using manually-labeled brain surface from MindBoggle, the largest publicly available dataset of this kind, we demonstrate a 2%–13% improvement in mean Dice over a non-adversarial training strategy, for test brain surfaces with no alignment or aligned on a different reference than source examples.
Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models

Studies on the Named Entity Recognition (NER) task have shown outstanding results that reach human parity on input texts with correct text formattings, such as with proper punctuation and capitalization. However, such conditions are not available in applications where the input is speech, because the text is generated from a speech recognition system (ASR), and that the system does not consider the text formatting. In this paper, we (1) presented the first Vietnamese speech dataset for NER task, and (2) the first pre-trained public large-scale monolingual language model for Vietnamese that achieved the new state-of-the-art for the Vietnamese NER task by 1.3% absolute F1 score comparing to the latest study. And finally, (3) we proposed a new pipeline for NER task from speech that overcomes the text formatting problem by introducing a text capitalization and punctuation recovery model (CaPu) into the pipeline. The model takes input text from an ASR system and performs two tasks at the same time, producing proper text formatting that helps to improve NER performance. Experimental results indicated that the CaPu model helps to improve by nearly 4% of F1-score.
Generating Compact and Relaxable Answers to Keyword Queries over Knowledge Graphs

Keyword search has been a prominent approach to querying knowledge graphs. For exploratory search tasks, existing methods commonly extract subgraphs that are group Steiner trees (GSTs) as answers. However, a GST that connects all the query keywords may not exist, or may inevitably have a large and unfocused graph structure in contrast to users’ favor to a compact answer. Therefore, in this paper, we aim at generating compact but relaxable subgraphs as answers, i.e., we require a computed subgraph to have a bounded diameter but allow it to only connect an incomplete subset of query keywords. We formulate it as a new combinatorial optimization problem of computing a minimally relaxed answer with a compactness guarantee, and we present a novel best-first search algorithm. Extensive experiments showed that our approach efficiently computed compact answers of high completeness.
Bone microarchitecture characterization based on fractal analysis in spatial frequency domain imaging

ABSTRACT This paper suggests a new technique for trabecular bone characterization using fractal analysis of X‐Ray and MRI texture images for osteoporosis diagnosis. Osteoporosis is a chronic disease characterized by a decrease in bone density that can lead to fracture and disability. In essence, the proposed fractal model makes use of the differential box‐counting method (DBCM) to estimate the fractal dimension ( FD ) after an appropriate image preprocessing stage that ensures a robust estimation process. In this study, we showed that within the frequency domain generated through discrete cosine transform (DCT), only a quarter of DCT coefficients are enough to characterize osteoporotic tissues. The algorithmic complexity of the developed approach is of the order of where N stands for the size of the image, which, in turn, likely yields important gain in terms of medication cost. We report a successful separation of healthy and pathological cases in term of both P − value (using statistical Wilcoxon rank sum test) and margin difference. A comparative statistical analysis has been performed using a publicly available database that contains a set of MRI and X‐Ray texture images of both healthy and osteoporotic bone tissues. The statistical results demonstrated the feasibility and accepted performance level of our fractal model‐based diagnosis to discriminate healthy and unhealthy trabecular bone tissues. The developed approach has been implemented on a medical device prototype.