Reviewerly/exHarmony huggingface dataset:
Federated Learning in Risk Prediction: A Primer and Application to COVID-19-Associated Acute Kidney Injury

Background: Modern machine learning and deep learning algorithms require large amounts of data; however, data sharing between multiple healthcare institutions is limited by privacy and security concerns. Summary: Federated learning provides a functional alternative to the single-institution approach while avoiding the pitfalls of data sharing. In cross-silo federated learning, the data do not leave a site. The raw data are stored at the site of collection. Models are created at the site of collection and are updated locally to achieve a learning objective. We demonstrate a use case with COVID-19-associated AKI. We showed that federated models outperformed their local counterparts, even when evaluated on local data in the test dataset, and performance was like those being used for pooled data. Increases in performance at a given hospital were inversely proportional to dataset size at a given hospital, which suggests that hospitals with smaller datasets have significant room for growth with federated learning approaches. Key Messages: This short article provides an overview of federated learning, gives a use case for COVID-19-associated acute kidney injury, and finally details the issues along with some potential solutions.
Teaching creative careers in the pandemic

During the COVID-19 pandemic, one of the major concerns at the Universidad de Ciencias y Artes de América Latina (UCAL; the University of Sciences and Arts of Latin America) has been to keep offering high-quality education with effective teaching methodologies and creativity at its core. This article aims to describe and understand the use of digital tools for class preparation, synchronous encounters, storage, interaction, collaborative work and assessment by UCAL’s instructors in creative careers. It finds that instructors were proactive about learning and using a variety of digital tools. Gamification apps and interactive boards were instructors’ favourites because they tended to motivate their students the most. No statistically significant associations were found between tool selection and course, sex, or age. Based on the evidence, this article will propose general guidelines for a training plan for instructors.
Vanilla Graph Neural Networks

In this section, we describe the vanilla GNNs proposed in Scarselli et al. [2009]. We also list the limitations of the vanilla GNN in representation capability and training efficiency. After this chapter we will talk about several variants ofthe vanilla GNN model.
Immersing learners in stories: A systematic literature review of educational narratives in virtual reality

The aim of this study, via a systematic review, was to investigate the addition of narratives in immersive virtual reality (IVR) and the associated impacts on learning. Narratives in IVR put the learner in the story, which, until recent developments in IVR head-mounted display technology, was out of reach in most classrooms. The review found that added immersion afforded by VR is particularly important in relation to learning where situational context is desirable. Importantly, IVR experiences with a narrative may have the potential to increase affective outcomes for learners, without reducing cognitive gains. Additionally, data concerning learning theories and design methodologies was extracted from the studies. The systematic review yielded 12 relevant and applied papers with this demonstrating there are still significant gaps in the research concerning the impact on learning in narrative IVR. This review highlights that the inclusion of narratives in educational IVR offers many potential benefits, however is yet to be fully explored. There are endless opportunities to tell stories in IVR. How they can be used and their impact on education is a rich ground for further research and development. Implications for practice or policy Educators and VR developers should consider including a narrative in an educational IVR experience because of the potential to increase affective outcomes such as motivation and engagement, without reducing cognitive gains. Learners can benefit from educational IVR experiences with narratives because they can provide situational context, elicit an emotional response and scaffold complex learning.
Market orientation and technological orientation in business services: The moderating role of organizational culture and human resources on performance

Contemporary conditions of the functioning of enterprises mean that they are increasingly looking for opportunities to improve organizational performance in strategic management. Scientists are looking for optimal solutions, an appropriate combination of assets and resources, so the debate in the field of strategic orientations is still valid and gaining in importance. Several studies have explored the construct of market orientation, but few include technological orientation with the moderating effects of company assets. In the era of the highly competitive technology market, the area of technological business service providers are particularly interesting, but still undiscovered. This paper examines the effects of market orientation and technological orientation on organizational performance with the inclusion of organizational culture and human resources as moderators. Using questionnaire responses from technological business service providers (n = 689), a regression analysis was conducted to confirm the hypotheses. The results established evidence of positive relationships between market orientation—organizational performance and technological orientation—organizational performance, although in technological firms, the market orientation had a stronger correlation with organizational performance than the technological orientation. Moreover, the organizational culture and human resources play a moderating role in the relationships of market orientation—organizational performance and technological orientation—organizational performance, while weak human resources management weakens relationships market orientation—organizational performance and technological orientation—organizational performance and strong organizational culture reduce the effect of market orientation on organizational performance, significantly reducing the effect of technological orientation on firm performance.
Design and Development of Array POS for Airborne Remote Sensing Motion Compensation

Multi-antenna airborne remote sensing systems have received more attention recently because they can realize high-resolution three-dimensional (3-D) imaging, such as array Synthetic Aperture Radar (SAR). Their high-precision imaging needs multi-antenna motion and relative motion between antennas. However, the existing facility and technology hardly meet the motion measurement precision demand of array SAR. To solve this problem, an array Position and Orientation System (POS) for airborne remote sensing motion compensation is designed and developed. It is composed of a high-precision POS, several small-size Inertial Measurement Units (IMU), and a 6-D deformation measurement system based on Fiber Bragg Grating (FBG) sensors. Firstly, the transfer alignment method based on 6-D deformation is used to measure the relative motion between array POS. Then, the motion conversion method from array POS to array SAR is presented to obtain the multi-antenna motion and relative motion between antennas. Finally, the ground experiment results identify that the accuracies of multi-antenna position, multi-antenna attitude, and flexible baseline length between antennas are superior to 3 cm, 0.01°, and 0.1 mm, respectively, which can meet the motion measurement precision demand of array SAR.
Leibniz Data Manager – A Research Data Management System

FAIR principles aim to enhance machine-actionability of research data management, and enable data consumers and providers to scale up to incoming data avalanches. This demo paper describes Leibniz Data Manager (LDM), a research data management repository that resorts to Semantic Web technologies to empower FAIR principles. During the demonstration, the attendees will create various digital objects, and observe the crucial role of metadata in efficient and effective management and analysis of research data management. LDM is publicly available: https://service.tib.eu/ldmservice/ .
The Dynamics of Governing Enterprise Blockchain Ecosystems

The aim of this paper is to analyze how the governance of an enterprise blockchain ecosystem changes as it matures and increases in size. A review of the literature serves to identify five behavioral drivers of governance, which appear to affect the long-term viability of a blockchain ecosystem: access rights, decision rights, incentives, accountability, and conflict resolution. We subsequently report the findings from a comparative case study of how three large blockchain ecosystems implemented various governance mechanisms to exploit and modify the five behavioral drivers over time. Based on twenty-six interviews and approximately 200 h of participant observations, we propose an analytical framework that consists of three distinctive stages in the life cycle of a blockchain ecosystem. Each stage is characterized by an intricate relationship between off-chain and on-chain governance mechanisms. Based on these findings, various recommendations are provided to increase the long-term viability of blockchain ecosystems.
Resourceful Thinking as a Key Competence of a Teacher in the Context of Digitalization of Educational Environment

The article considers resourceful thinking as a key competence of a teacher for the first time in scientific literature. Resourceful thinking is defined as metacognitive ability to become aware and utilize as many external and internal resources for solving a particular task as possible. This competence is viewed from the standpoint of a subject-action approach. The author gives the rationale for considering resourceful thinking as a key competence of a teacher. The correlation of resourceful thinking of a teacher and certain personality traits has been revealed with the help of the author’s technique. Based on a meta-systemic approach, for the first time resourceful thinking of a teacher has been viewed as a meta-resource allowing for managing pedagogical activities using one’s individual resources to the full, as well as opportunities of digitalization of educational environment.
Abnormal detection method of accounting data based on information extraction technology

In order to overcome the problems of low detection accuracy and efficiency of traditional abnormal detection methods for accounting data, this paper proposes a new abnormal detection method of accounting data based on information extraction technology. This method uses the information extraction technology based on XML to extract the accounting data, and eliminates the redundant data in the accounting data through the redundancy strategy of erasure code and full backup and the dynamic copy management optimisation model. Using the processed accounting, the principal component analysis method is used to realise the abnormal detection of accounting data. The experimental results show that the proposed method has high detection accuracy, the highest detection accuracy is 0.98, and the highest detection efficiency can reach 97%, which can meet the requirements of abnormal detection of accounting data.
Accelerating Training and Inference of Graph Neural Networks with Fast  Sampling and Pipelining

Improving the training and inference performance of graph neural networks (GNNs) is faced with a challenge uncommon in general neural networks: creating mini-batches requires a lot of computation and data movement due to the exponential growth of multi-hop graph neighborhoods along network layers. Such a unique challenge gives rise to a diverse set of system design choices. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment, under which we identify major performance bottlenecks hitherto under-explored by developers: mini-batch preparation and transfer. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler, a shared-memory parallelization strategy, and the pipelining of batch transfer with GPU computation. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised. Such an observation unifies training and inference, simplifying model implementation. We report comprehensive experimental results with several benchmark data sets and GNN architectures, including a demonstration that, for the ogbn-papers100M data set, our system SALIENT achieves a speedup of 3x over a standard PyTorch-Geometric implementation with a single GPU and a further 8x parallel speedup with 16 GPUs. Therein, training a 3-layer GraphSAGE model with sampling fanout (15, 10, 5) takes 2.0 seconds per epoch and inference with fanout (20, 20, 20) takes 2.4 seconds, attaining test accuracy 64.58%.
Collaboration Equilibrium in Federated Learning

Federated learning (FL) refers to the paradigm of learning models over a collaborative research network involving multiple clients without sacrificing privacy. Recently, there have been rising concerns on the distributional discrepancies across different clients, which could even cause counterproductive consequences when collaborating with others. While it is not necessarily that collaborating with all clients will achieve the best performance, in this paper, we study a rational collaboration called "collaboration equilibrium'' (CE), where smaller collaboration coalitions are formed. Each client collaborates with certain members who maximally improve the model learning and isolates the others who make little contribution. We propose the concept of benefit graph which describes how each client can benefit from collaborating with other clients and advance a Pareto optimization approach to identify the optimal collaborators. Then we theoretically prove that we can reach a CE from the benefit graph through an iterative graph operation. Our framework provides a new way of setting up collaborations in a research network. Experiments on both synthetic and real world data sets are provided to demonstrate the effectiveness of our method.
The emergence of language in the human mind and brain—Insights from the neurobiology of language, thought and action.

The capacity for language has evolved remarkably quickly in recent human history. Its advent likely coincided with a range of cognitive innovations not found elsewhere at this level of complexity in the rest of the animal kingdom. This late yet near-simultaneous florescence of higher language and cognition is difficult to account for in terms of strictly modular neurocognitive systems, each with its own dedicated function and evolutionary trajectory. Nor does it legitimize the neurocognitive study of language in isolation from other systems of human thought and action. In the wake of emergentist approaches to key human cognitive abilities, the present article considers language as the differentiated product of multiple neural networks dedicated to qualitatively distinct cognitive functions-including (at least) the cortical systems of general semantic cognition and control and the sensorimotor systems supporting language production. A model is proposed to account for how these systems congregate to produce language, featuring a dual-stream architecture of the semantic interface into item-based and item-independent semantic knowledge on the one hand, and a notion of the sensorimotor interface as a key component for the temporal tracking and verbal rehearsal of task-relevant information on the other. Avenues are also offered for enriching this architecture in future versions of the model. Finally, it is proposed that language is an "optimal" combination of these neurocognitive systems, enabling fast and cost-effective transfer of information at the systems level. This last point underpins evidence for the privileged status of language as a tool for adaptive thought and behavior, as well as some important features of brain evolution, development, and functional organization. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Adversarial Training for Large Neural Language Models

Generalization and robustness are both key desiderata for designing machine learning methods. Adversarial training can enhance robustness, but past work often finds it hurts generalization. In natural language processing (NLP), pre-training large neural language models such as BERT have demonstrated impressive gain in generalization for a variety of tasks, with further improvement from adversarial fine-tuning. However, these models are still vulnerable to adversarial attacks. In this paper, we show that adversarial pre-training can improve both generalization and robustness. We propose a general algorithm ALUM (Adversarial training for large neural LangUage Models), which regularizes the training objective by applying perturbations in the embedding space that maximizes the adversarial loss. We present the first comprehensive study of adversarial training in all stages, including pre-training from scratch, continual pre-training on a well-trained model, and task-specific fine-tuning. ALUM obtains substantial gains over BERT on a wide range of NLP tasks, in both regular and adversarial scenarios. Even for models that have been well trained on extremely large text corpora, such as RoBERTa, ALUM can still produce significant gains from continual pre-training, whereas conventional non-adversarial methods can not. ALUM can be further combined with task-specific fine-tuning to attain additional gains. The ALUM code is publicly available at https://github.com/namisan/mt-dnn.
Domain-Informed Probing of wav2vec 2.0 Embeddings for Phonetic Features

In recent years large transformer model architectures have become available which provide a novel means of generating high-quality vector representations of speech audio.These transformers make use of an attention mechanism to generate representations enhanced with contextual and positional information from the input sequence.Previous works have explored the capabilities of these models with regard to performance in tasks such as speech recognition and speaker verification, but there has not been a significant inquiry as to the manner in which the contextual information provided by the transformer architecture impacts the representation of phonetic information within these models.In this paper, we report the results of a number of probing experiments on the representations generated by the wav2vec 2.0 model's transformer component, with regard to the encoding of phonetic categorization information within the generated embeddings.We find that the contextual information generated by the transformer's operation results in enhanced capture of phonetic detail by the model, and allows for distinctions to emerge in acoustic data that are otherwise difficult to separate.
UniBO at SemEval-2022 Task 5: A Multimodal bi-Transformer Approach to the Binary and Fine-grained Identification of Misogyny in Memes

We present our submission to SemEval 2022 Task 5 on Multimedia Automatic Misogyny Identification.We address the two tasks: Task A consists of identifying whether a meme is misogynous.If so, Task B attempts to identify its kind among shaming, stereotyping, objectification, and violence.Our approach combines a BERT Transformer with CLIP for the textual and visual representations.Both textual and visual encoders are fused in an early-fusion fashion through a Multimodal Bidirectional Transformer with unimodally pretrained components.Our official submissions obtain macroaveraged F 1 =0.727 in Task A (4th position out of 69 participants) and weighted F 1 =0.710 in Task B (4th position out of 42 participants).
Why Knowledge Distillation Amplifies Gender Bias and How to Mitigate from the Perspective of DistilBERT

Knowledge distillation is widely used to transfer the language understanding of a large model to a smaller model.However, after knowledge distillation, it was found that the smaller model is more biased by gender compared to the source large model.This paper studies what causes gender bias to increase after the knowledge distillation process.Moreover, we suggest applying a variant of the mixup on knowledge distillation, which is used to increase generalizability during the distillation process, not for augmentation.By doing so, we can significantly reduce the gender bias amplification after knowledge distillation.We also conduct an experiment on the GLUE benchmark to demonstrate that even if the mixup is applied, it does not have a significant adverse effect on the model’s performance.
From spoken dialogue to formal summary: An utterance rewriting for dialogue summarization

Due to the dialogue characteristics of unstructured contexts and multi-parties with firstperson perspective, many successful text summarization works have failed when dealing with dialogue summarization.In dialogue summarization task, the input dialogue is usually spoken style with ellipsis and co-references but the output summaries are more formal and complete.Therefore, the dialogue summarization model should be able to complete the ellipsis content and co-reference information and then produce a suitable summary accordingly.However, the current state-of-the-art models pay more attention on the topic or structure of summary, rather than the consistency of dialogue summary with its input dialogue context, which may suffer from the personal and logical inconsistency problem.In this paper, we propose a new model, named ReWriteSum, to tackle this problem.Firstly, an utterance rewriter is conducted to complete the ellipsis content of dialogue content and then obtain the rewriting utterances.Then, the co-reference data augmentation mechanism is utilized to replace the referential person name with its specific name to enhance the personal information.Finally, the rewriting utterances and the co-reference replacement data are used in the standard BART model.Experimental results on both SAMSum and DialSum datasets show that our ReWrite-Sum significantly outperforms baseline models, in terms of both metric-based and human evaluations.Further analysis on multi-speakers also shows that ReWriteSum can obtain relatively higher improvement with more speakers, validating the correctness and property of ReWrite-Sum.
Learning Structural Information for Syntax-Controlled Paraphrase Generation

Syntax-controlled paraphrase generation aims to produce paraphrase conform to given syntactic patterns.To address this task, recent works have started to use parse trees (or syntactic templates) to guide generation.A constituency parse tree contains abundant structural information, such as parent-child relation, sibling relation, and the alignment relation between words and nodes.Previous works have only utilized parent-child and alignment relations, which may affect the generation quality.To address this limitation, we propose a Structural Information-augmented Syntax-Controlled Paraphrasing (SI-SCP) model.Particularly, we design a syntax encoder based on tree-transformer to capture parent-child and sibling relations.To model the alignment relation between words and nodes, we propose an attention regularization objective, which makes the decoder accurately select corresponding syntax nodes to guide the generation of words.Experiments show that SI-SCP achieves state-ofthe-art performances in terms of semantic and syntactic quality on two popular benchmark datasets.Additionally, we propose a Syntactic Template Retriever (STR) to retrieve compatible syntactic structures.We validate that STR is capable of retrieving compatible syntactic structures.We further demonstrate the effectiveness of SI-SCP to generate diverse paraphrases with retrieved syntactic structures.