Reviewerly/exHarmony huggingface dataset:
Depth Estimation from Monocular Images and Sparse Radar Data

In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods developed for LiDAR data and images to the new fusion problem between Radar data and images. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions. Extensive experiments demonstrate that our method outperforms existing fusion methods. We also provide detailed ablation studies to show the effectiveness of each component in our method.
Augmented reality assisted learning achievement, motivation, and creativity for children of low‐grade in primary school

Abstract There is a large volume of published studies describing the positive role of Augmented Reality (AR) in educational settings. However, there is a paucity of empirical work that explores its role in supporting primary school pupil's motivation and creativity. Moreover, numerous studies have confirmed that pupils have difficulties in learning geometry and thinking geometrically. In this context, this paper explores the possibility of using AR software for low‐grade primary school pupils to enhance their creative thinking and increase their motivation during the informal geometry training course. A research‐based design process is adopted to replicate the primary school pupil's learning in this study, where pupils in the experimental group used mobile application to access AR virtual manipulatives, pupils in the control group used physical manipulatives. The applied AR technology allows pupils to handle 2D and 3D rounded corners and simple closed curves to find the most effective solutions for some learning challenges. Sixty‐two pupils from the first‐grade primary school in Egypt have participated in this study with the aim of investigating: (a) the primary school pupil's achievements and creative thinking in using AR techniques, and (b) the specific strategies that can increase their motivation and engagement. The most obvious finding to emerge from this study is that there was no statistically significant difference between the groups in geometry learning achievement, while statistically significant differences were found between the groups in the motivation and creative thinking skills in favour of the experimental group.
Learning programs by learning from failures

Abstract We describe an inductive logic programming (ILP) approach called learning from failures . In this approach, an ILP system (the learner) decomposes the learning problem into three separate stages: generate , test , and constrain . In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints (constraints on the syntactic form of hypotheses). In the test stage, the learner tests the hypothesis against training examples. A hypothesis fails when it does not entail all the positive examples or entails a negative example. If a hypothesis fails, then, in the constrain stage, the learner learns constraints from the failed hypothesis to prune the hypothesis space, i.e. to constrain subsequent hypothesis generation. For instance, if a hypothesis is too general (entails a negative example), the constraints prune generalisations of the hypothesis. If a hypothesis is too specific (does not entail all the positive examples), the constraints prune specialisations of the hypothesis. This loop repeats until either (i) the learner finds a hypothesis that entails all the positive and none of the negative examples, or (ii) there are no more hypotheses to test. We introduce Popper, an ILP system that implements this approach by combining answer set programming and Prolog. Popper supports infinite problem domains, reasoning about lists and numbers, learning textually minimal programs, and learning recursive programs. Our experimental results on three domains (toy game problems, robot strategies, and list transformations) show that (i) constraints drastically improve learning performance, and (ii) Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.
Neuromorphic Binarized Polariton Networks

The rapid development of artificial neural networks and applied artificial intelligence has led to many applications. However, current software implementation of neural networks is severely limited in terms of performance and energy efficiency. It is believed that further progress requires the development of neuromorphic systems, in which hardware directly mimics the neuronal network structure of a human brain. Here, we propose theoretically and realize experimentally an optical network of nodes performing binary operations. The nonlinearity required for efficient computation is provided by semiconductor microcavities in the strong quantum light-matter coupling regime, which exhibit exciton–polariton interactions. We demonstrate the system performance against a pattern recognition task, obtaining accuracy on a par with state-of-the-art hardware implementations. Our work opens the way to ultrafast and energy-efficient neuromorphic systems taking advantage of ultrastrong optical nonlinearity of polaritons.
Deep Cross-Modal Representation Learning and Distillation for Illumination-Invariant Pedestrian Detection

Integrating multispectral data has been demonstrated to be an effective solution for illumination-invariant pedestrian detection, in particular, RGB and thermal images can provide complementary information to handle light variations. However, most of the current multispectral detectors fuse the multimodal features by simple concatenation, without discovering their latent relationships. In this paper, we propose a cross-modal feature learning (CFL) module, based on a split-and-aggregation strategy, to explicitly explore both the shared and modality-specific representations between paired RGB and thermal images. We insert the proposed CFL module into multiple layers of a two-branch-based pedestrian detection network, to learn the cross-modal representations in diverse semantic levels. By introducing a segmentation-based auxiliary task, the multimodal network is trained end-to-end by jointly optimizing a multi-task loss. On the other hand, to alleviate the reliance of existing multispectral pedestrian detectors on thermal images, we propose a knowledge distillation framework to train a student detector, which only receives RGB images as input and distills the cross-modal representations guided by a well-trained multimodal teacher detector. In order to facilitate the cross-modal knowledge distillation, we design different distillation loss functions for the feature, detection and segmentation levels. Experimental results on the public KAIST multispectral pedestrian benchmark validate that the proposed cross-modal representation learning and distillation method achieves robust performance.
HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images

The majority of learning-based semantic segmentation methods are optimized for daytime scenarios and favorable lighting conditions. Real-world driving scenarios, however, entail adverse environmental conditions such as nighttime illumination or glare which remain a challenge for existing approaches. In this work, we propose a multimodal semantic segmentation model that can be applied during daytime and nighttime. To this end, besides RGB images, we leverage thermal images, making our network significantly more robust. We avoid the expensive annotation of nighttime images by leveraging an existing daytime RGB-dataset and propose a teacher-student training approach that transfers the dataset's knowledge to the nighttime domain. We further adopt a domain adaptation method to align the learned feature spaces across the domains and propose a novel two-stage training scheme. Furthermore, due to a lack of thermal data for autonomous driving, we present a new dataset comprising over 20,000 time-synchronized and aligned RGB-thermal image pairs. In this context, we also present a novel target-less calibration method that allows for automatic robust extrinsic and intrinsic thermal camera calibration. Among others, we use our new dataset to show state-of-the-art results for nighttime semantic segmentation.
Feed-Forward Neural Network Training by Hybrid Bat Algorithm

Artificial neural networks are very powerful machine learning techniques and they are capable to solve complex problems. In the artificial neural network, one of the most difficult challenges is to find the optimal values of the weights during the learning process. To address this issue, we propose a new hybridized metaheuristic method, called BAABC for weight connection optimization. The experiments are performed on two binary classification datasets. The obtained results are compared to other similar approaches where other metaheuristics are used. The obtained results show that the proposed algorithm can find the optimal weight connection values and achieve higher performance and the proposed BAABC outperformed the other methods.
A fast and yet efficient YOLOv3 for blood cell detection

These days, blood cell detection in microscopic images plays a vital role in cognition, the health of a patient. Since disease detection based on manual checking of blood cells is mostly time-consuming and full of errors, analysis of blood cells using object detectors can be considered as an effective tool. Hence, in this study, an object detector has been proposed which is used for detecting blood objects such as white blood cells, red blood cells, and platelets. This detector is called FED (Fast and Efficient YOLOv3) and it is a One-Stage detector, which is similar to YOLOv3, performs detection in three scales. For the purpose of increasing efficiency and flexibility, the proposed object detector utilizes the EfficientNet Convolutional Neural Network as the backbone effectiveness. Furthermore, the Dilated Convolution is indeed applied in order to increase receptive view of the backbone. In addition, the Depthwise Separable Convolution method is utilized to minimize the detector’s parametersand the Distance Intersection over Union is further used for bounding box regression. Besides, for increasing the performance, the Swish activation function is employed. The experiments are run on the BCCD dataset that the average precision of platelets, red blood cells, and white blood cells become 90.25%, 80.41%, and 98.92%, respectively. The results of experiments and comparisons demonstrate that the proposed FED detector is more efficient than other existing studies for blood cell detection.
Soft Tissue Simulation Environment to Learn Manipulation Tasks in Autonomous Robotic Surgery

Reinforcement Learning (RL) methods have demonstrated promising results for the automation of subtasks in surgical robotic systems. Since many trial and error attempts are required to learn the optimal control policy, RL agent training can be performed in simulation and the learned behavior can be then deployed in real environments. In this work, we introduce an open-source simulation environment providing support for position based dynamics soft bodies simulation and state-of-the-art RL methods. We demonstrate the capabilities of the proposed framework by training an RL agent based on Proximal Policy Optimization in fat tissue manipulation for tumor exposure during a nephrectomy procedure. Leveraging on a preliminary optimization of the simulation parameters, we show that our agent is able to learn the task on a virtual replica of the anatomical environment. The learned behavior is robust to changes in the initial end-effector position. Furthermore, we show that the learned policy can be directly deployed on the da Vinci Research Kit, which is able to execute the trajectories generated by the RL agent. The proposed simulation environment represents an essential component for the development of next-generation robotic systems, where the interaction with the deformable anatomical environment is involved.
Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation

Existing neural network-based autonomous systems are shown to be vulnerable against adversarial attacks, therefore sophisticated evaluation of their robustness is of great importance. However, evaluating the robustness under the worst-case scenarios based on known attacks is not comprehensive, not to mention that some of them even rarely occur in the real world. Also, the distribution of safety-critical data is usually multimodal, while most traditional attacks and evaluation methods focus on a single modality. To solve the above challenges, we propose a flow-based multimodal safety-critical scenario generator for evaluating decision-making algorithms. The proposed generative model is optimized with weighted likelihood maximization and a gradient-based sampling procedure is integrated to improve the sampling efficiency. The safety-critical scenarios are generated by efficiently querying the task algorithms and a simulator. Experiments on a self-driving task demonstrate our advantages in terms of testing efficiency and multimodal modeling capability. We evaluate six Reinforcement Learning algorithms with our generated traffic scenarios and provide empirical conclusions about their robustness.
Mimetics: Towards Understanding Human Actions Out of Context

Recent methods for video action recognition have reached outstanding performances on existing benchmarks. However, they tend to leverage context such as scenes or objects instead of focusing on understanding the human action itself. For instance, a tennis field leads to the prediction playing tennis irrespectively of the actions performed in the video. In contrast, humans have a more complete understanding of actions and can recognize them without context. The best example of out-of-context actions are mimes, that people can typically recognize despite missing relevant objects and scenes. In this paper, we propose to benchmark action recognition methods in such absence of context and introduce a novel dataset, Mimetics, consisting of mimed actions for a subset of 50 classes from the Kinetics benchmark. Our experiments show that (a) state-of-the-art 3D convolutional neural networks obtain disappointing results on such videos, highlighting the lack of true understanding of the human actions and (b) models leveraging body language via human pose are less prone to context biases. In particular, we show that applying a shallow neural network with a single temporal convolution over body pose features transferred to the action recognition problem performs surprisingly well compared to 3D action recognition methods.
DeepLocalize: Fault Localization for Deep Neural Networks

Deep Neural Networks (DNNs) are becoming an integral part of most software systems. Previous work has shown that DNNs have bugs. Unfortunately, existing debugging techniques don't support localizing DNN bugs because of the lack of understanding of model behaviors. The entire DNN model appears as a black box. To address these problems, we propose an approach and a tool that automatically determines whether the model is buggy or not, and identifies the root causes for DNN errors. Our key insight is that historic trends in values propagated between layers can be analyzed to identify faults, and also localize faults. To that end, we first enable dynamic analysis of deep learning applications: by converting it into an imperative representation and alternatively using a callback mechanism. Both mechanisms allows us to insert probes that enable dynamic analysis over the traces produced by the DNN while it is being trained on the training data. We then conduct dynamic analysis over the traces to identify the faulty layer or hyperparameter that causes the error. We propose an algorithm for identifying root causes by capturing any numerical error and monitoring the model during training and finding the relevance of every layer/parameter on the DNN outcome. We have collected a benchmark containing 40 buggy models and patches that contain real errors in deep learning applications from Stack Overflow and GitHub. Our benchmark can be used to evaluate automated debugging tools and repair techniques. We have evaluated our approach using this DNN bug-and-patch benchmark, and the results showed that our approach is much more effective than the existing debugging approach used in the state-of-the-practice Keras library. For 34/40 cases, our approach was able to detect faults whereas the best debugging approach provided by Keras detected 32/40 faults. Our approach was able to localize 21/40 bugs whereas Keras did not localize any faults.
RETRACTED ARTICLE: Current trends in the digital transformation of higher education institutions in Russia

This research aimed to analyze the current trends in the digital transformation of educational institutions. The investigation lasted from November 2019 to May 2020 and was based on three higher educational institutions (HEIs) in the Russian Federation: Belgorod State University, Moscow State Institute of International Relations, and Kutafin Moscow State Law University. The number of respondents enrolled was 420 people. In an attempt to bring together views and experiences of different actors of the educational sector, three individual online surveys were conducted among administrative staff, teachers, and students of mentioned educational institutions. All respondents were provided with questionnaires aimed at identifying current trends in educational paradigm transformation and studying the extent to which digital strategies are applied in the process of HEIs’ development. The detailed analysis of survey outcomes allowed indicating positive and negative trends in digitalization of educational processes from the perspective of university administrative staff, teachers, and students. Positive trends were: ensuring the availability of higher education through the implementation of full-fledged distance learning courses; enhancing the students’ experience through the introduction of innovative teaching methods; providing open access to educational resources and research results; opportunity to participate in global “open science” initiatives; and reduction of higher education cost to ensure its accessibility and mass scale. The barriers to the digital transformation of the educational sphere were: lack of funds for the implementation of a comprehensive digital transformation strategy; resistance to changes on the part of the staff; and a low level of confidence in technological solutions used in teaching practice. As a result, the conducted investigation uncovered the essence of digital transformation in the institutional structures of the higher education sector. However, given that digitalization is a complex process affecting most spheres of socio-economic interaction, consideration of only positive and negative digital transformation trends is insufficient to thoroughly analyze global digital shifts and the formation of national vectors for digital society development.
Cross Subspace Alignment Codes for Coded Distributed Batch Computation

The goal of coded distributed computation is to efficiently distribute a computation task, such as matrix multiplication, N-linear computation, or multivariate polynomial evaluation, across S servers through a coding scheme, such that the response from any R servers ( R is called the recovery threshold) is sufficient for the user to recover the desired computed value. Current state-of-art approaches are based on either exclusively matrix-partitioning (Entangled Polynomial (EP) Codes for matrix multiplication), or exclusively batch processing (Lagrange Coded Computing (LCC) for N-linear computations or multivariate polynomial evaluations). We present three related classes of codes, based on the idea of Cross-Subspace Alignment (CSA) which was introduced originally in the context of secure and private information retrieval. CSA codes are characterized by a Cauchy-Vandermonde matrix structure that facilitates interference alignment along Vandermonde terms, while the desired computations remain resolvable along the Cauchy terms. These codes are shown to unify, generalize and improve upon the state-of-art codes for distributed computing. First we introduce CSA codes for matrix multiplication, which yield LCC codes as a special case, and are shown to outperform LCC codes in general in download-limited settings. While matrix-partitioning approaches (EP codes) for distributed matrix multiplication have the advantage of flexible server computation latency, batch processing approaches (CSA, LCC) have significant advantages in communication costs as well as encoding and decoding complexity per matrix multiplication. In order to combine the benefits of these approaches, we introduce Generalized CSA (GCSA) codes for matrix multiplication that bridge the extremes of matrix-partitioning and batch processing approaches and demonstrate synergistic gains due to cross subspace alignment. Finally, we introduce N-CSA codes for N-linear distributed batch computations and multivariate batch polynomial evaluations. N-CSA codes include LCC codes as a special case, and are in general capable of outperforming LCC codes in download-constrained settings by upto a factor of N. Generalizations of N-CSA codes to include X-secure data and B-byzantine servers are also provided.
The Fusion of Deep Learning and Fuzzy Systems: A State-of-the-Art Survey

Deep learning presents excellent learning ability in constructing learning model and greatly promotes the development of artificial intelligence, but its conventional models cannot handle uncertain or imprecise circumstances. Fuzzy systems, can not only depict uncertain and vague concepts widely existing in the real world, but also improve the prediction accuracy in deep learning models. Thus, it is important and necessary to go through the recent contributions about the fusion of deep learning and fuzzy systems. At first, we introduce the deep learning into fuzzy community from two perspectives: statistical results of relevant publications and conventional deep learning algorithms. Then, the fusing framework and graphic form of deep learning and fuzzy systems are constructed. Followed by, are the current situations of several types of fuzzy techniques used in deep learning, some reasons why use fuzzy techniques in deep learning, and the application fields of the fusion, respectively. Finally, some discussions and future challenges are provided regarding the fusion technology of deep learning and fuzzy systems, the application scenarios of fusing deep learning and fuzzy systems, and some limitations of the current fusion, respectively. After summarizing the recent contributions, we have found that this field is an emerging research direction and it is increasingly paying much more attention. Especially, fuzzy systems make great effects on deep learning models in the aspect of classification, prediction, natural language processing, auto-control, etc., and the fusion is applied into different fields, like but not limited to computer science, natural language, medical system, smart energy management systems and manufacturing industry.