Reviewerly/exHarmony huggingface dataset:
Developing the research potential of a social teacher in the context of digital technologies

In the system of modern domestic education, the problem of professional-pedagogical activity of teachers is one of the most popular and relevant, deserving separate empirical research and further practical study. This article aims to consider the role of a social teacher in society and reviews the research potential of a social teacher. Opportunities such as advanced training courses in research capacity development, ICT courses, speech at scientific conferences, participation in professional competitions, project works, master classes, and participation in network communities of social teachers are identified. The features of each opportunity noted in the development of research capacity are discussed in detail. Essential features of the notion's potential and research potential are disclosed in the article. Structural elements of the pedagogical model on three stages of teacher’s research potential development at the system of further professional training. Keywords: ICT courses; network community; research potential; social teacher.
Data-Driven Chance Constrained Control using Kernel Distribution Embeddings

We present a data-driven algorithm for efficiently computing stochastic control policies for general joint chance constrained optimal control problems. Our approach leverages the theory of kernel distribution embeddings, which allows representing expectation operators as inner products in a reproducing kernel Hilbert space. This framework enables approximately reformulating the original problem using a dataset of observed trajectories from the system without imposing prior assumptions on the parameterization of the system dynamics or the structure of the uncertainty. By optimizing over a finite subset of stochastic open-loop control trajectories, we relax the original problem to a linear program over the control parameters that can be efficiently solved using standard convex optimization techniques. We demonstrate our proposed approach in simulation on a system with nonlinear non-Markovian dynamics navigating in a cluttered environment.
Posterior temperature optimized Bayesian models for inverse problems in medical imaging

We present Posterior Temperature Optimized Bayesian Inverse Models (POTOBIM), an unsupervised Bayesian approach to inverse problems in medical imaging using mean-field variational inference with a fully tempered posterior. Bayesian methods exhibit useful properties for approaching inverse tasks, such as tomographic reconstruction or image denoising. A suitable prior distribution introduces regularization, which is needed to solve the ill-posed problem and reduces overfitting the data. In practice, however, this often results in a suboptimal posterior temperature, and the full potential of the Bayesian approach is not being exploited. In POTOBIM, we optimize both the parameters of the prior distribution and the posterior temperature with respect to reconstruction accuracy using Bayesian optimization with Gaussian process regression. Our method is extensively evaluated on four different inverse tasks on a variety of modalities with images from public data sets and we demonstrate that an optimized posterior temperature outperforms both non-Bayesian and Bayesian approaches without temperature optimization. The use of an optimized prior distribution and posterior temperature leads to improved accuracy and uncertainty estimation and we show that it is sufficient to find these hyperparameters per task domain. Well-tempered posteriors yield calibrated uncertainty, which increases the reliability in the predictions. Our source code is publicly available at github.com/Cardio-AI/mfvi-dip-mia.
A code-based hybrid signcryption scheme

A key encapsulation mechanism (KEM) that takes as input an arbitrary string, i.e., a tag, is known as tag-KEM, while a scheme that combines signature and encryption is called signcryption. In this paper, we present a code-based signcryption tag-KEM scheme. We utilize a code-based signature and a CCA2 (adaptive chosen ciphertext attack) secure version of McEliece's {encryption} scheme. The proposed scheme uses an equivalent subcode as a public code for the receiver, making the NP-completeness of the equivalent subcode problem be one of our main security assumptions. We then base the signcryption tag-KEM to design a code-based hybrid signcryption scheme. A hybrid scheme deploys an asymmetric- as well as a symmetric-key encryption. We give security analyses of both our schemes in the standard model and prove that they are secure against IND-CCA2 (indistinguishability under adaptive chosen ciphertext attack) and SUF-CMA (strong existential unforgeability under chosen message attack).
Medical Aegis: Robust adversarial protectors for medical images

Deep neural network based medical image systems are vulnerable to adversarial examples. Many defense mechanisms have been proposed in the literature, however, the existing defenses assume a passive attacker who knows little about the defense system and does not change the attack strategy according to the defense. Recent works have shown that a strong adaptive attack, where an attacker is assumed to have full knowledge about the defense system, can easily bypass the existing defenses. In this paper, we propose a novel adversarial example defense system called Medical Aegis. To the best of our knowledge, Medical Aegis is the first defense in the literature that successfully addresses the strong adaptive adversarial example attacks to medical images. Medical Aegis boasts two-tier protectors: The first tier of Cushion weakens the adversarial manipulation capability of an attack by removing its high-frequency components, yet posing a minimal effect on classification performance of the original image; the second tier of Shield learns a set of per-class DNN models to predict the logits of the protected model. Deviation from the Shield's prediction indicates adversarial examples. Shield is inspired by the observations in our stress tests that there exist robust trails in the shallow layers of a DNN model, which the adaptive attacks can hardly destruct. Experimental results show that the proposed defense accurately detects adaptive attacks, with negligible overhead for model inference.
Federated Learning with Superquantile Aggregation for Heterogeneous Data

We present a federated learning framework that is designed to robustly deliver good predictive performance across individual clients with heterogeneous data. The proposed approach hinges upon a superquantile-based learning objective that captures the tail statistics of the error distribution over heterogeneous clients. We present a stochastic training algorithm that interleaves differentially private client filtering with federated averaging steps. We prove finite time convergence guarantees for the algorithm: $O(1/\sqrt{T})$ in the nonconvex case in $T$ communication rounds and $O(\exp(-T/κ^{3/2}) + κ/T)$ in the strongly convex case with local condition number $κ$. Experimental results on benchmark datasets for federated learning demonstrate that our approach is competitive with classical ones in terms of average error and outperforms them in terms of tail statistics of the error.
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance

In this paper, we study contrastive learning from an optimization perspective, aiming to analyze and address a fundamental issue of existing contrastive learning methods that either rely on a large batch size or a large dictionary of feature vectors. We consider a global objective for contrastive learning, which contrasts each positive pair with all negative pairs for an anchor point. From the optimization perspective, we explain why existing methods such as SimCLR require a large batch size in order to achieve a satisfactory result. In order to remove such requirement, we propose a memory-efficient Stochastic Optimization algorithm for solving the Global objective of Contrastive Learning of Representations, named SogCLR. We show that its optimization error is negligible under a reasonable condition after a sufficient number of iterations or is diminishing for a slightly different global contrastive objective. Empirically, we demonstrate that SogCLR with small batch size (e.g., 256) can achieve similar performance as SimCLR with large batch size (e.g., 8192) on self-supervised learning task on ImageNet-1K. We also attempt to show that the proposed optimization technique is generic and can be applied to solving other contrastive losses, e.g., two-way contrastive losses for bimodal contrastive learning. The proposed method is implemented in our open-sourced library LibAUC (www.libauc.org).
KGR4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation

Generative commonsense reasoning requires machines to generate sentences describing an everyday scenario given several concepts, which has attracted much attention recently. However, existing models cannot perform as well as humans, since sentences they produce are often implausible and grammatically incorrect. In this paper, inspired by the process of humans creating sentences, we propose a novel Knowledge-enhanced Commonsense Generation framework, termed KGR4, consisting of four stages: Retrieval, Retrospect, Refine, Rethink. Under this framework, we first perform retrieval to search for relevant sentences from external corpus as the prototypes. Then, we train the generator that either edits or copies these prototypes to generate candidate sentences, of which potential errors will be fixed by an autoencoder-based refiner. Finally, we select the output sentence from candidate sentences produced by generators with different hyper-parameters. Experimental results and in-depth analysis on the CommonGen benchmark strongly demonstrate the effectiveness of our framework. Particularly, KGR4 obtains 33.56 SPICE in the official leaderboard, outperforming the previously-reported best result by 2.49 SPICE and achieving state-of-the-art performance. We release the code at https://github.com/DeepLearnXMU/KGR-4.
DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs

Denoising diffusion probabilistic models (DDPMs) are expressive generative models that have been used to solve a variety of speech synthesis problems. However, because of their high sampling costs, DDPMs are difficult to use in real-time speech processing applications. In this paper, we introduce DiffGAN-TTS, a novel DDPM-based text-to-speech (TTS) model achieving high-fidelity and efficient speech synthesis. DiffGAN-TTS is based on denoising diffusion generative adversarial networks (GANs), which adopt an adversarially-trained expressive model to approximate the denoising distribution. We show with multi-speaker TTS experiments that DiffGAN-TTS can generate high-fidelity speech samples within only 4 denoising steps. We present an active shallow diffusion mechanism to further speed up inference. A two-stage training scheme is proposed, with a basic TTS acoustic model trained at stage one providing valuable prior information for a DDPM trained at stage two. Our experiments show that DiffGAN-TTS can achieve high synthesis performance with only 1 denoising step.
Experiences in engineering design training at Samara University

This paper aims to share experience gained by Samara National Research University in training students in engineering degree programme “Manned and Unmanned Spacecraft and Space Systems”. Core items of engineering degree curricula are projects because they basically form practical experience of future design engineers. Main items of the course project “Calculation of main parameters and generation of land remote sensing satellites conceptual design based on regard to required efficiency indices” are discussed. These include acquisition and processing of statistical data on space systems and satellites; determination of satellite orbit parameters; determination of massdimensional characteristics of spacecraft on-board systems and construction; formation of on-board systems; spacecraft construction models and assembly model; choice of launcher and spacehead model development. The activity is supported by appropriate training materials and software.
Analysis of the Generation of a Synthetic Response to the Application of Contrast Agents in Breast Medical Images Using Generative Adversarial Networks

Since different studies alerted about deposits of contrast medium in the brain after conducting contrast-enhanced magnetic resonance examinations, developing strategies to reduce or avoid the use of those contrast agents has become a highly relevant research topic. In this work, we propose the use of a conditional generative adversarial network model (cGAN) for generating a synthetic response to the application of contrast agents in medical images, specifically magnetic resonance images of the breast. A large MRI data set with 30527 images corresponding to 163 examinations were used to train and validate the model. The results show a successful generation of the spatial structure of images; however, a large variability in the results is observed, especially when the breast tissue is high density. A detailed analysis shows an average error of 8% for regions of diagnostic interest, which is a promising result.
SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalization

Zero-shot learning is the problem of predicting instances over classes not seen during training. One approach to zero-shot learning is providing auxiliary class information to the model. Prior work along this vein have largely used expensive per-instance annotation or singular class-level descriptions, but per-instance descriptions are hard to scale and single class descriptions may not be rich enough. Furthermore, these works have used natural-language descriptions exclusively, simple bi-encoders models, and modality or task-specific methods. These approaches have several limitations: text supervision may not always be available or optimal and bi-encoders may only learn coarse relations between inputs and class descriptions. In this work, we present SemSup, a novel approach that uses (1) a scalable multiple description sampling method which improves performance over single descriptions, (2) alternative description formats such as JSON that are easy to generate and outperform text on certain settings, and (3) hybrid lexical-semantic similarity to leverage fine-grained information in class descriptions. We demonstrate the effectiveness of SemSup across four datasets, two modalities, and three generalization settings. For example, across text and image datasets, SemSup increases unseen class generalization accuracy by 15 points on average compared to the closest baseline.
Recommendation method for fusion of knowledge graph convolutional network

Abstract In the application of internet of vehicles system, it is particularly important to obtain real-time and effective vehicle information and provide personalized functional services for vehicle operation. This algorithm combines knowledge graph technology with convolutional network and presents a new algorithm model, that is, when calculating the representation of a given entity in the knowledge graph, the information of the neighboring entity is combined with the deviation. Through the integration of neighbor entity information, the local neighborhood structure can be better captured and stored in each entity, and the weight of different neighbor entities depends on the relationship between them and the specific user, which can better reflect the user's personalized interests, in order to fully demonstrate the characteristics of the entity. Compared with the traditional coordinated filtering technology SVD model, this model has improved accuracy and F 1 value.
Occluded Human Mesh Recovery

Top-down methods for monocular human mesh recovery have two stages: (1) detect human bounding boxes; (2) treat each bounding box as an independent single-human mesh recovery task. Unfortunately, the single-human assumption does not hold in images with multi-human occlusion and crowding. Consequently, top-down methods have difficulties in recovering accurate 3D human meshes under severe person-person occlusion. To address this, we present Occluded Human Mesh Recovery (OCHMR) - a novel top-down mesh recovery approach that incorporates image spatial context to overcome the limitations of the single-human assumption. The approach is conceptually simple and can be applied to any existing top-down architecture. Along with the input image, we condition the top-down model on spatial context from the image in the form of body-center heatmaps. To reason from the predicted body centermaps, we introduce Contextual Normalization (CoNorm) blocks to adaptively modulate intermediate features of the top-down model. The contextual conditioning helps our model disambiguate between two severely overlapping human bounding-boxes, making it robust to multi-person occlusion. Compared with state-of-the-art methods, OCHMR achieves superior performance on challenging multi-person benchmarks like 3DPW, CrowdPose and OCHuman. Specifically, our proposed contextual reasoning architecture applied to the SPIN model with ResNet-50 backbone results in 75.2 PMPJPE on 3DPW-PC, 23.6 AP on CrowdPose and 37.7 AP on OCHuman datasets, a significant improvement of 6.9 mm, 6.4 AP and 20.8 AP respectively over the baseline. Code and models will be released.
Towards a machine-readable literature: finding relevant papers based on an uploaded powder diffraction pattern

We investigate a prototype application for machine-readable literature. The program is called "pyDataRecognition" and serves as an example of a data-driven literature search, where the literature search query is an experimental data-set provided by the user. The user uploads a powder pattern together with the radiation wavelength. The program compares the user data to a database of existing powder patterns associated with published papers and produces a rank ordered according to their similarity score. The program returns the digital object identifier (doi) and full reference of top ranked papers together with a stack plot of the user data alongside the top five database entries. The paper describes the approach and explores successes and challenges.
Assessing Gender Bias in Predictive Algorithms using eXplainable AI

Predictive algorithms have a powerful potential to offer benefits in areas as varied as medicine or education. However, these algorithms and the data they use are built by humans, consequently, they can inherit the bias and prejudices present in humans. The outcomes can systematically repeat errors that create unfair results, which can even lead to situations of discrimination (e.g. gender, social or racial). In order to illustrate how important is to count with a diverse training dataset to avoid bias, we manipulate a well-known facial expression recognition dataset to explore gender bias and discuss its implications.
Digital Literacy as the New Dimension in Measuring Advertising Literacy: Towards a “Super Smart Society”

A 'super smart society' is a core concept from Society 5.0 inspired by the Japanese Government which aims to develop both information and humancentered societies.Since the society, in general, is becoming increasingly digitized and technology connected-this study aims to define digital literacy as a new dimension in measuring the advertising literacy towards to one of the visual pollutantsfly posters.To represent the low-income society distribution, a total of 574 respondents from five (5) Public Housing Program (PHP) were selected.The study was guided by the combination of both media literacy theory and an advertising literacy model.Findings revealed that the low-income society are comprised of highly advertising literate individuals due to digital technological factors.The discussions of this study add on to the debate on how digital literacy and collaborative efforts can be crucial in developing an intelligent society, the 'super smart society', the Society 5.0.
Hierarchical clustering: visualization, feature importance and model  selection

We propose methods for the analysis of hierarchical clustering that fully use the multi-resolution structure provided by a dendrogram. Specifically, we propose a loss for choosing between clustering methods, a feature importance score and a graphical tool for visualizing the segmentation of features in a dendrogram. Current approaches to these tasks lead to loss of information since they require the user to generate a single partition of the instances by cutting the dendrogram at a specified level. Our proposed methods, instead, use the full structure of the dendrogram. The key insight behind the proposed methods is to view a dendrogram as a phylogeny. This analogy permits the assignment of a feature value to each internal node of a tree through ancestral state reconstruction. Real and simulated datasets provide evidence that our proposed framework has desirable outcomes. We provide an R package that implements our methods.
DisCo: Effective Knowledge Distillation For Contrastive Learning of  Sentence Embeddings

Contrastive learning has been proven suitable for learning sentence embeddings and can significantly improve the semantic textual similarity (STS) tasks. Recently, large contrastive learning models, e.g., Sentence-T5, tend to be proposed to learn more powerful sentence embeddings. Though effective, such large models are hard to serve online due to computational resources or time cost limits. To tackle that, knowledge distillation (KD) is commonly adopted, which can compress a large "teacher" model into a small "student" model but generally suffer from some performance loss. Here we propose an enhanced KD framework termed Distill-Contrast (DisCo). The proposed DisCo framework firstly utilizes KD to transfer the capability of a large sentence embedding model to a small student model on large unlabelled data, and then finetunes the student model with contrastive learning on labelled training data. For the KD process in DisCo, we further propose Contrastive Knowledge Distillation (CKD) to enhance the consistencies among teacher model training, KD, and student model finetuning, which can probably improve performance like prompt learning. Extensive experiments on 7 STS benchmarks show that student models trained with the proposed DisCo and CKD suffer from little or even no performance loss and consistently outperform the corresponding counterparts of the same parameter size. Amazingly, our 110M student model can even outperform the latest state-of-the-art (SOTA) model, i.e., Sentence-T5(11B), with only 1% parameters.