Reviewerly/exHarmony huggingface dataset:
Сontradictions in the development of the scientific and pedagogical community as a problem of foreign and domestic research

The subject of sociological analysis in the article is one of the important resources of modern higher education – the scientific and pedagogical community (SPS), considered by the authors as a key social community of universities (along with students and managers). The focus of attention is on two main incarnations of the SPS, that determine the place of the university in domestic and international rankings of higher education institutions. This is a professional-pedagogical and scientificeducational activity of the members of the community. It is emphasised that it is the SPS that today constitutes the “core” of the university’s intellectual asset. That leads to the growing attention to the SPS as an object of theoretical and empirical study in foreign and domestic science. Hence, the purpose of the article was defined as the identification of trends in the SPS research, shown through the prism of its contradictions and problems common to Russian and foreign universities. Special attention is drawn to two paradoxical situations typical of the domestic SPS. Firstly, as its number decreases, the structure of professional activity becomes more complicated, the number of tasks and the degree of responsibility increase, the amount of academic workload and unnecessary bureaucratic burden systematically increase. Secondly, the growing dependence of the university development strategy on scientific and pedagogical staff is combined in a contradictory way with a decrease in real managerial attention to them. Two theories served as a general theoretical framework for the authors – theory of social community and theory of resource dependence. The article presents the results of a systematic analysis of the literature, that allowed to reveal the contradictions in the reproduction of the scientific and pedagogical staff of universities, the impact of the development strategies of higher education and universities on the functioning and development of the SPS, as well as the contradictions in the development of the resource capacity of the SPS.
Optimizing over an Ensemble of Trained Neural Networks

We study optimization problems where the objective function is modeled through feedforward neural networks with rectified linear unit (ReLU) activation. Recent literature has explored the use of a single neural network to model either uncertain or complex elements within an objective function. However, it is well known that ensembles of neural networks produce more stable predictions and have better generalizability than models with single neural networks, which motivates the investigation of ensembles of neural networks rather than single neural networks in decision-making pipelines. We study how to incorporate a neural network ensemble as the objective function of an optimization model and explore computational approaches for the ensuing problem. We present a mixed-integer linear program based on existing popular big-M formulations for optimizing over a single neural network. We develop a two-phase approach for our model that combines preprocessing procedures to tighten bounds for critical neurons in the neural networks with a Lagrangian relaxation-based branch-and-bound approach. Experimental evaluations of our solution methods suggest that using ensembles of neural networks yields more stable and higher quality solutions, compared with single neural networks, and that our optimization algorithm outperforms (the adaption of) a state-of-the-art approach in terms of computational time and optimality gaps. History: Andrea Lodi, Area Editor for Design &amp; Analysis of Algorithms—Discrete.
An Efficient Substitution Box design with a chaotic logistic map and Linear Congruential Generator for secure communication in Smart cities

The study provides a unique method for creating an efficient substitution box (S-box) for advanced encryption standards using a Chaotic Logistic Map (CLM) and a Linear Congruential Generator (LCG) (AES) for secure communications in a smart city. The Pseudo-Random Number Generator (PRNG), which is further examined, is constructed using an extensive search of reasonable possibilities for the initial seed and set parameters. Using statistical testing, the performance analysis of the new S-box is assessed. Additionally, the resilience of differential, as well as linear cryptanalysis, is shown. It is derived using other features, including nonlinearity, the Bit Independence Criterion (BIC), and the Strict Avalanche Criterion (SAC). The suggested S-box has good potential and is usable for symmetric key cryptography, according to the features of the new S-cryptographic box.
Timing Performance Benchmarking of Out-of-Distribution Detection Algorithms

Abstract In an open world with a long-tail distribution of input samples, Deep Neural Networks (DNNs) may make unpredictable mistakes for Out-of-Distribution (OOD) inputs at test time, despite high levels of accuracy obtained during model training. OOD detection can be an effective runtime assurance mechanism for safe deployment of machine learning algorithms in safety–critical applications such as medical imaging and autonomous driving. A large number of OOD detection algorithms have been proposed in recent years, with a wide range of performance metrics in terms of accuracy and execution time. For real-time safety–critical applications, e.g., autonomous driving, timing performance is of great importance in addition to accuracy. We perform a comprehensive and systematic benchmark study of multiple OOD detection algorithms in terms of both accuracy and execution time on different hardware platforms, including a powerful workstation and a resource-constrained embedded device, equipped with both CPU and GPU. We also profile and analyze the internal details of each algorithm to identify the performance bottlenecks and potential for GPU acceleration. This paper aims to provide a useful reference for the practical deployment of OOD detection algorithms for real-time safety–critical applications.
A Blockchain-Based Data-Driven Fault-Tolerant Control System for Smart Factories in Industry 4.0

Modern technologies and data-driven approaches have enabled fault-tolerant controllers in Industry 4.0 smart factories to detect, identify, and mitigate anomalies in real-time with a high level of accuracy. However, this has also presented new challenges and requirements for cybersecurity, data analytics, and computational complexity for Data-Driven Fault-Tolerant Controllers (DD-FTC) in smart factories. To address these issues, a Blockchain-Based Data-Driven Fault-Tolerant Control (BB-DD-FTC) framework for smart factories is proposed in this paper. Blockchain ensures the integrity of data logs via its immutable ledger and decentralized architecture. Moreover, the blockchain smart contract functionality, embedded with a Data-Driven Intrusion Detection System (DD-IDS), and reconfiguration conditions, realizes DD-FTC and undertakes the mitigation response in case of cyber-attacks. The DD-IDS mechanism utilizes the principal component analysis technique and observer models, trained via neural networks, to detect an attack and identify the compromised component. The Tennessee Eastman (TE) industrial benchmark process is considered a case study to investigate the performance of the proposed framework. Two kinds of integrity attacks are applied to the sensors of the TE process with simulation results demonstrating the effectiveness of the method in mitigating the adversarial effect of the applied attacks on the overall system performance. As feedback delays can negatively impact performance, a detailed delay analysis is performed using network calculus. The security advantages and limitations of the proposed method are finally highlighted in the performed security analysis. The results are encouraging for the wider adoption of the control-over-the-blockchain concept.
Compressing convolutional neural networks with cheap convolutions and online distillation

Visual impairment assistance systems play a vital role in improving the standard of living for visually impaired people (VIP). With the development of deep learning technologies and assistive devices, many assistive technologies for VIP have achieved remarkable success in environmental perception and navigation. In particular, convolutional neural network (CNN)-based models have surpassed the level of human recognition and achieved a strong generalization ability. However, the large memory and computation consumption in CNNs have been one of the main barriers to deploying them into resource-limited systems for visual impairment assistance applications. To this end, most cheap convolutions (e.g., group convolution, depth-wise convolution, and shift convolution) have recently been used for memory and computation reduction but with a specific architecture design. Furthermore, it results in a low discriminability of the compressed networks by directly replacing the standard convolution with these cheap ones. In this paper, we propose to use knowledge distillation to improve the performance of compact student networks with cheap convolutions. In our case, the teacher is a network with the standard convolution, while the student is a simple transformation of the teacher architecture without complicated redesigning. In particular, we introduce a novel online distillation method, which online constructs the teacher network without pre-training and conducts mutual learning between the teacher and student network, to improve the performance of the student model. Extensive experiments demonstrate that the proposed approach achieves superior performance to simultaneously reduce memory and computation overhead of cutting-edge CNNs on different datasets, including CIFAR-10/100 and ImageNet ILSVRC 2012, compared to the previous CNN compression and acceleration methods. The codes are publicly available at https://github.com/EthanZhangYC/OD-cheap-convolution.
Immersive competence and immersive literacy: Exploring how users learn about immersive experiences

While immersive experiences mediated through near-eye displays are still a relatively immature medium, there are millions of consumer devices in use. The level of awareness of the forms of the interface and media will vary enormously across the potential audience. Users might own personal devices or might encounter immersive systems in various venues. We introduce the term immersive competence to refer to the general practical knowledge and skills that users accumulate about how typical immersive interfaces work—the ways in which buttons are used, main locomotion techniques, etc. We then introduce the term immersive literacy to refer to awareness of how immersive interfaces are unique, when they might be appropriate, typical forms of media, etc. We sketch out how users develop competence and literacy with immersive media, and then highlight various open questions that are raised.
Effects of Digital Pedagogical Skills of Mathematics Teachers on Academic Performance

The study aimed to measure the effect of digital pedagogical skills of mathematics teachers and their awareness on student learning performance. The cross-sectional survey design was used among 462 mathematics teachers in Nepal. Structural equation modeling and machine learning were the main statistical techniques used in the study. The findings indicate that teachers’ digital pedagogical skills were significantly low in internet surfing, using communication tools, subject-related applications, and the development of audiovisual instructional materials. Internet surfing skills and collaboration tools are significant predictors of policy and ethical digital awareness.
Stragglers Are Not Disasters: A Hybrid Federated Learning Framework with Delayed Gradients

Federated learning (FL) is a new machine learning framework that trains a joint model across a large number of decentralized computing devices. Existing methods, e.g., Federated Averaging (FedAvg), are able to provide an optimization guarantee by synchronously training the joint model, but usually suffer from stragglers, i.e., IoT devices with low computing power or communication bandwidth, especially on optimization problems with non-i.i.d distributed training data. To mitigate the influence of stragglers, this paper presents a novel FL framework, namely Hybrid Federated Learning (HFL), which consists of two major components: synchronous kernel and asynchronous updater that correspondingly addresses the normal devices and stragglers. Unlike traditional synchronous FL frameworks, HFL introduces the asynchronous updater which actively pulls unsyn-chronized and delayed local updates from stragglers to improve the efficiency and effectiveness of FL. Particularly, we propose a novel algorithm, delayed federated-SGD (DF-SGD), to merge the delayed local updates into the joint model. The results of comprehensive experiments under the HFL framework show that DF-SGD outperforms current FL methods.
Attention-based Partial Decoupling of Policy and Value for Generalization in Reinforcement Learning

In this work, we introduce Attention-based Partially Decoupled Actor-Critic (APDAC), an actor-critic architecture for generalization in reinforcement learning, which partially separates the policy and the value functions. To learn directly from images, traditional actor-critic architectures use a shared network to represent the policy and value functions. While a shared representation allows parameter and feature sharing, it can also lead to overfitting that catastrophically damages generalization performance. On the other hand, two separate networks for policy and value can help to avoid overfitting and reduce the generalization gap, but at the cost of added complexity both in terms of architecture design and computation time. APDAC is a hybrid architecture that builds upon the combined strengths of both architectures by sharing initial layer blocks of the network and separating the later ones for policy and value. APDAC incorporates an attention mechanism to enable robust representation learning. We present meaningful visualization of the policy and value that explains the perception of the trained agent. Our empirical analysis, including an ablation study, shows that APDAC significantly outperforms the standard PPO baseline on the challenging RL generalization benchmark Procgen and achieves performance that is competitive with the recent state-of-the-art method (IDAAC) while using fewer convolutional layers and requiring less computational time. Our code is available at https://github.com/nasiknafi/apdac.
Decision Boundaries of Deep Neural Networks

Deep neural networks and in particular, deep neural classifiers have become an integral part of many modern applications. Despite their practical success, we still have limited knowledge of how they work and the demand for such an understanding is evergrowing. In this regard, one crucial aspect of deep neural network classifiers that can help us deepen our knowledge about their decision-making behavior is to investigate their decision boundaries. Nevertheless, this is contingent upon having access to samples populating the areas near the decision boundary. To achieve this, we propose a novel approach we call Deep Decision boundary Instance Generation (DeepDIG). DeepDIG utilizes a method based on adversarial example generation as an effective way of generating samples near the decision boundary of a neural network model. Then, we introduce a set of important characteristics that take advantage of the generated borderline instances and provide multifaceted understandings of deep neural networks. We have performed extensive experiments on multiple representative datasets across various deep neural network models and characterized their decision boundaries. The entire code is publicly available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Smooth Trajectory Collision Avoidance through Deep Reinforcement Learning

Collision avoidance is a crucial task in vision-guided autonomous navigation. Solutions based on deep reinforcement learning (DRL) has become increasingly popular. In this work, we proposed several novel agent state and reward function designs to tackle two critical issues in DRL-based navigation solutions: 1) smoothness of the trained flight trajectories; and 2) model generalization to handle unseen environments.Formulated under a DRL framework, our model relies on margin reward and smoothness constraints to ensure UAVs fly smoothly while greatly reducing the chance of collision. The proposed smoothness reward minimizes a combination of first-order and second-order derivatives of flight trajectories, which can also drive the points to be evenly distributed, leading to stable flight speed. To enhance the agent’s capability of handling new unseen environments, two practical setups are proposed to improve the invariance of both the state and reward function when deploying in different scenes. Experiments demonstrate the effectiveness of our overall design and individual components.
SECOE: Alleviating Sensors Failure in Machine Learning-Coupled IoT Systems

Internet of Things (IoT) domains are characterized by continuous streams of data originating from diverse, geographically distributed sensors. Sensor/network failures that result in data stream interruptions is a major challenge in applying ML techniques to IoT domains. Unfortunately, the performance of many ML applications quickly degrades when faced with data incompleteness. With the aim of building robust IoT-coupled ML applications, this paper proposes SECOE – a unique, proactive approach for alleviating potentially simultaneous sensor failures. The fundamental idea behind SECOE is to create a carefully chosen ensemble of ML models in which each model is trained assuming a set of failed sensors. SECOE includes a novel technique to minimize the number of models in the ensemble by harnessing the correlations among sensors. We demonstrate the efficacy of the SECOE approach through a series of experiments involving two distinct datasets.
Post-Training Quantization for Energy Efficient Realization of Deep Neural Networks

The biggest challenge for the deployment of Deep Neural Networks (DNNs) close to the generated data on edge devices is their size, i.e., memory footprint and computational complexity. Both are significantly reduced with quantization. With the resulting lower word-length, the energy efficiency of DNNs increases proportionally. However, lower word-length typically causes accuracy degradation. To counteract this effect, the quantized DNN is retrained. Unfortunately, training costs up to 5000× more energy than the inference of the quantized DNN. To address this issue, we propose a post-training quantization flow without the need for retraining. For this, we investigated different quantization options. Furthermore, our analysis systematically assesses the impact of reduced word-lengths of weights and activations revealing a clear trend for the choice of word-length. Both aspects have not been systematically investigated so far. Our results are independent of the depth of the DNNs and apply to uniform quantization, allowing fast quantization of a given pre-trained DNN. We excel state-of-the-art for 6 bit by 2.2% Top-1 accuracy for ImageNet. Without retraining, our quantization to 8 bit surpasses floating-point accuracy.
Dynamic Binary Cross Entropy: An effective and quick method for model convergence

The current deep learning era has huge models of over a million parameters. Training such huge models can take a significant amount of time. The recent advancements in hardware like CUDA-enabled GPUs to train the machine and deep learning models have become extremely popular. Claims suggest that the learning times can often are reduced from days to hours. However, the training time can further be reduced by optimizing the software component of the machine and deep learning models. This is the opportunity we take to introduce our loss function, the Dynamic Binary Cross Entropy (DBCE). Recent studies have shown that dynamizing the parameters of the loss functions facilitates the training of deep learning models. We aim to perform minimal calculations to update the parameters of the loss function with no extra hyper-parameters, yet not compromising on the performance. We claim that by the end of an epoch, the DBCE updates its weights such that the model’s performance is better than the model trained by its vanilla counterpart. This can help in the early stopping of the model’s training, as with DBCE, the model can achieve higher metric values earlier than the model trained with standard Binary Cross Entropy.
Equipping Recommender Systems with Individual Fairness via Second-order Proximity Embedding

Graph neural networks (GNNs) have been widely used for recommender systems over knowledge graphs. An important issue of GNN-based recommender systems is individual user fairness in recommendations (i.e., similar users should be treated similarly by the systems). In this paper, we make the following contributions to enable recommender systems to be equipped with individual user fairness. First, we define new similarity metrics for individual fairness, where these metrics take knowledge graphs into consideration by incorporating both first-order proximity in direct user-item interactions and second-order proximity in knowledge graphs. Second, we design a novel graph neural network (GNN) named SKIPHop for fair recommendations over knowledge graphs. By passing latent representations from both first-order and second-order neighbors at every message passing step, SKIPHop learns user embed dings that capture their latent interests present in the second-order networks. Furthermore, to realize individual user fairness, we add fairness as a regularization to the loss function of recommendation models. Finally, through experiments on two real-world datasets, we demonstrate the effectiveness of SKIPHop in terms of fairness and recommendation accuracy.
One-Shot Federated Group Collaborative Filtering

Non-negative matrix factorization (NMF) with missing-value completion is a well-known effective Collaborative Filtering (CF) method used to provide personalized user recommendations. However, traditional CF relies on a privacy-invasive collection of user data to build a central recommender model. One-shot federated learning has recently emerged as a method to mitigate the privacy problem while addressing the traditional communication bottleneck of federated learning. In this paper, we present the first one-shot federated CF implementation, named One-FedCF, for groups of users or collaborating organizations. In our solution, the clients first apply local CF in-parallel to build distinct, client-specific recommenders. Then, the privacy-preserving local item patterns and biases from each client are shared with the processor to perform joint factorization in order to extract the global item patterns. Extracted patterns are then aggregated to each client to build the local models via information retrieval transfer. In our experiments, we demonstrate our approach with two MovieLens datasets and show results competitive with the state-of-the-art federated recommender systems at a substantial decrease in the number of communications.