Reviewerly/exHarmony huggingface dataset:
Deep Reinforcement Learning for Synthesizing Functions in Higher-Order Logic

The paper describes a deep reinforcement learning framework based on self-supervised learning within the proof assistant HOL4. A close interaction between the machine learning modules and the HOL4 library is achieved by the choice of tree neural networks (TNNs) as machine learning models and the internal use of HOL4 terms to represent tree structures of TNNs. Recursive improvement is possible when a task is expressed as a search problem. In this case, a Monte Carlo Tree Search (MCTS) algorithm guided by a TNN can be used to explore the search space and produce better examples for training the next TNN. As an illustration, term synthesis tasks on combinators and Diophantine equations are specified and learned. We achieve a success rate of 65% on combinator synthesis problems outperforming state-of-the-art ATPs run with their best general set of strategies. We set a precedent for statistically guided synthesis of Diophantine equations by solving 78.5% of the generated test problems.
Evaluation of a Financial Portfolio Visualization using Computer Displays and Mixed Reality Devices with Domain Experts

With the advent of mixed reality devices such as the Microsoft HoloLens, developers have been faced with the challenge to utilize the third dimension in information visualization effectively. Research on stereoscopic devices has shown that three-dimensional representation can improve accuracy in specific tasks (e.g., network visualization). Yet, so far the field has remained mute on the underlying mechanism. Our study systematically investigates the differences in user perception between a regular monitor and a mixed reality device. In a real-life within-subject experiment in the field with twenty-eight investment bankers, we assessed subjective and objective task performance with two- and three-dimensional systems, respectively. We tested accuracy with regard to position, size, and color using single and combined tasks. Our results do not show a significant difference in accuracy between mixed-reality and standard 2D monitor visualizations.
Stateful Premise Selection by Recurrent Neural Networks

In this work we develop a new learning-based method for selecting facts (premises) when proving new goals over large formal libraries. Unlike previous methods that choose sets of facts independently of each other by their rank, the new method uses the notion of state that is updated each time a choice of a fact is made. Our stateful architecture is based on recurrent neural networks which have been recently very successful in stateful tasks such as language translation. The new method is combined with data augmentation techniques, evaluated in several ways on a standard large-theory benchmark and compared to state-of-the-art premise approach based on gradient boosted trees. It is shown to perform significantly better and to solve many new problems.
Stochastic region pooling: Make attention more expressive

Global Average Pooling (GAP) is used by default on the channel-wise attention mechanism to extract channel descriptors. However, the simple global aggregation method of GAP is easy to make the channel descriptors have homogeneity, which weakens the detail distinction between feature maps, thus affecting the performance of the attention mechanism. In this work, we propose a novel method for channel-wise attention network, called Stochastic Region Pooling (SRP), which makes the channel descriptors more representative and diversity by encouraging the feature map to have more or wider important feature responses. Also, SRP is the general method for the attention mechanisms without any additional parameters or computation. It can be widely applied to attention networks without modifying the network structure. Experimental results on image recognition datasets including CIAFR-10/100, ImageNet and three Fine-grained datasets (CUB-200-2011, Stanford Cars and Stanford Dogs) show that SRP brings the significant improvements of the performance over efficient CNNs and achieves the state-of-the-art results.
EvoPreprocess—Data Preprocessing Framework with Nature-Inspired Optimization Algorithms

The quality of machine learning models can suffer when inappropriate data is used, which is especially prevalent in high-dimensional and imbalanced data sets. Data preparation and preprocessing can mitigate some problems and can thus result in better models. The use of meta-heuristic and nature-inspired methods for data preprocessing has become common, but these approaches are still not readily available to practitioners with a simple and extendable application programming interface (API). In this paper the EvoPreprocess open-source Python framework, that preprocesses data with the use of evolutionary and nature-inspired optimization algorithms, is presented. The main problems addressed by the framework are data sampling (simultaneous over- and under-sampling data instances), feature selection and data weighting for supervised machine learning problems. EvoPreprocess framework provides a simple object-oriented and parallelized API of the preprocessing tasks and can be used with scikit-learn and imbalanced-learn Python machine learning libraries. The framework uses self-adaptive well-known nature-inspired meta-heuristic algorithms and can easily be extended with custom optimization and evaluation strategies. The paper presents the architecture of the framework, its use, experiment results and comparison to other common preprocessing approaches.
Smart Mobile Application for Public Transport Schedules - Logical Model

Logical model of a portal for smart mobile application for public transport schedules on the base of three classes Model/View/Controller (MVC) has been described in this article. This concept based on design templates, leads to greater flexibility and possibility for repeated use in various applications.
Visual Transfer For Reinforcement Learning Via Wasserstein Domain Confusion

We introduce Wasserstein Adversarial Proximal Policy Optimization (WAPPO), a novel algorithm for visual transfer in Reinforcement Learning that explicitly learns to align the distributions of extracted features between a source and target task. WAPPO approximates and minimizes the Wasserstein-1 distance between the distributions of features from source and target domains via a novel Wasserstein Confusion objective. WAPPO outperforms the prior state-of-the-art in visual transfer and successfully transfers policies across Visual Cartpole and both the easy and hard settings of of 16 OpenAI Procgen environments.
Revisiting the particle-size constraint of the 10-μm silicate feature

We experimentally measure the mid-IR spectra of half-millimeter-sized olivine particles with very compact morphology and smooth surfaces. We find that the 10-μm silicate feature is present in these large, irregularly shaped samples having surface roughness that is much smaller than optical wavelengths. Based on Mie theory, which assumes the particles are spherical, this feature should not exist for such large particles. As a consequence, its presence has been taken as an indicator that the particles or grains composing such particles were micron-sized or smaller. The measurement of this feature in real, irregularly shaped particles, suggests that the assumption of sphericity may severely limit our interpretations of remote-sensing data.
Neural Machine Translation with Error Correction

Neural machine translation (NMT) generates the next target token given as input the previous ground truth target tokens during training while the previous generated target tokens during inference, which causes discrepancy between training and inference as well as error propagation, and affects the translation accuracy. In this paper, we introduce an error correction mechanism into NMT, which corrects the error information in the previous generated tokens to better predict the next token. Specifically, we introduce two-stream self-attention from XLNet into NMT decoder, where the query stream is used to predict the next token, and meanwhile the content stream is used to correct the error information from the previous predicted tokens. We leverage scheduled sampling to simulate the prediction errors during training. Experiments on three IWSLT translation datasets and two WMT translation datasets demonstrate that our method achieves improvements over Transformer baseline and scheduled sampling. Further experimental analyses also verify the effectiveness of our proposed error correction mechanism to improve the translation quality.
Combining Cloud Computing and Artificial Intelligence Scene Recognition in Real-time Environment Image Planning Walkable Area

This study developed scene recognition and cloud computing technology for real-time environmental image-based regional planning using artificial intelligence. TensorFlow object detection functions were used for artificial intelligence technology. First, an image from the environment is transmitted to a cloud server for cloud computing, and all objects in the image are marked using a bounding box method. Obstacle detection is performed using object detection, and the associated technique algorithm is used to mark walkable areas and relative coordinates. The results of this study provide a machine vision application combined with cloud computing and artificial intelligence scene recognition that can be used to complete walking space activities planned by a cleaning robot or unmanned vehicle through real-time utilization of images from the environment.
A Study of Generalization and Fitness Landscapes for Neuroevolution

Fitness landscapes are a useful concept for studying the dynamics of meta-heuristics. In the last two decades, they have been successfully used for estimating the optimization capabilities of different flavors of evolutionary algorithms, including genetic algorithms and genetic programming. However, so far they have not been used for studying the performance of machine learning algorithms on unseen data, and they have not been applied to studying neuroevolution landscapes. This paper fills these gaps by applying fitness landscapes to neuroevolution, and using this concept to infer useful information about the learning and generalization ability of the machine learning method. For this task, we use a grammar-based approach to generate convolutional neural networks, and we study the dynamics of three different mutations used to evolve them. To characterize fitness landscapes, we study autocorrelation, entropic measure of ruggedness, and fitness clouds. Also, we propose the use of two additional evaluation measures: density clouds and overfitting measure. The results show that these measures are appropriate for estimating both the learning and the generalization ability of the considered neuroevolution configurations.
Optimizer Benchmarking Needs to Account for Hyperparameter Tuning

The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration. The efficacy of optimizers is often studied under near-optimal problem-specific hyperparameters, and finding these settings may be prohibitively costly for practitioners. In this work, we argue that a fair assessment of optimizers' performance must take the computational cost of hyperparameter tuning into account, i.e., how easy it is to find good hyperparameter configurations using an automatic hyperparameter search. Evaluating a variety of optimizers on an extensive set of standard datasets and architectures, our results indicate that Adam is the most practical solution, particularly in low-budget scenarios.
Joint Multi-view 2D Convolutional Neural Networks for 3D Object Classification

Three-dimensional (3D) object classification is widely involved in various computer vision applications, e.g., autonomous driving, simultaneous localization and mapping, which has attracted lots of attention in the committee. However, solving 3D object classification by directly employing the 3D convolutional neural networks (CNNs) generally suffers from high computational cost. Besides, existing view-based methods cannot better explore the content relationships between views. To this end, this work proposes a novel multi-view framework by jointly using multiple 2D-CNNs to capture discriminative information with relationships as well as a new multi-view loss fusion strategy, in an end-to-end manner. Specifically, we utilize multiple 2D views of a 3D object as input and integrate the intra-view and inter-view information of each view through the view-specific 2D-CNN and a series of modules (outer product, view pair pooling, 1D convolution, and fully connected transformation). Furthermore, we design a novel view ensemble mechanism that selects several discriminative and informative views to jointly infer the category of a 3D object. Extensive experiments demonstrate that the proposed method is able to outperform current state-of-the-art methods on 3D object classification. More importantly, this work provides a new way to improve 3D object classification from the perspective of fully utilizing well-established 2D-CNNs.
Design of Short-Term Load Forecasting based on ANN Using Bigdata

Short-term power load prediction is crucial for scheduling generators, securing transmission capacity, and determining economic market price. This paper proposes a Bigdata-based Artificial Neural Network (B-ANN) model for a short-term load forecasting by utilizing decision-tree method. The bigdata has been composed of 2018 time index, meteorological data, exchange rates, oil price, and past power demand in Seoul. The decision-tree method is applied to classify the data in accordance with the value of decreasing entropy. In addition, the resilient back propagation algorithm (RPROP) is applied for ANN that derives fast results and result in reduced error through variable learning rates. The applicability and effectiveness of the proposed model are verified by conducting simulations for forecasting 1-day hourly power load. The results confirm that the proposed model has decreased the mean absolute percentage error (MAPE) dramatically in comparison with other forecasting methods such as multiple regression analysis, time series analysis, and auto-regressive integrated moving average (ARIMA) model.
The Performance Analysis of Generalized Margin Maximizers on Separable Data

Logistic models are commonly used for binary classification tasks. The success of such models has often been attributed to their connection to maximum-likelihood estimators. It has been shown that gradient descent algorithm, when applied on the logistic loss, converges to the max-margin classifier (a.k.a. hard-margin SVM). The performance of the max-margin classifier has been recently analyzed. Inspired by these results, in this paper, we present and study a more general setting, where the underlying parameters of the logistic model possess certain structures (sparse, block-sparse, low-rank, etc.) and introduce a more general framework (which is referred to as Generalized Margin Maximizer, GMM). While classical max-margin classifiers minimize the 2-norm of the parameter vector subject to linearly separating the data, GMM minimizes any arbitrary convex function of the parameter vector. We provide a precise analysis of the performance of GMM via the solution of a system of nonlinear equations. We also provide a detailed study for three special cases: (1) l₂-GMM that is the max-margin classifier, (2) l₁-GMM which encourages sparsity, and (3) l_∞-GMM which is often used when the parameter vector has binary entries. Our theoretical results are validated by extensive simulation results across a range of parameter values, problem instances, and model structures.
The LOINC Content Model and Its Limitations of Usage in the Laboratory Domain.

To unambiguously encode the semantic meaning of laboratory tests, the LOINC terminology is widely used. With regard to the constantly changing and diverse requirements of the laboratory domain, LOINC's long-established content model and related publications are reviewed conjointly, revealing some obstacles for flexible adaptation in terms of new or varying application needs as well as issues regarding the comprehensive reusability of lab data. In a concise overview, four specific limitations are identified that require adaptation or the usage of other terminologies.
Continual Learning in Recurrent Neural Networks with Hypernetworks.

The last decade has seen a surge of interest in continual learning (CL), and a variety of methods have been developed to alleviate catastrophic forgetting. However, most prior work has focused on tasks with static data, while CL on sequential data has remained largely unexplored. Here we address this gap in two ways. First, we evaluate the performance of established CL methods when applied to recurrent neural networks (RNNs). We primarily focus on elastic weight consolidation, which is limited by a stability-plasticity trade-off, and explore the particularities of this trade-off when using sequential data. We show that high working memory requirements, but not necessarily sequence length, lead to an increased need for stability at the cost of decreased performance on subsequent tasks. Second, to overcome this limitation we employ a recent method based on hypernetworks and apply it to RNNs to address catastrophic forgetting on sequential data. By generating the weights of a main RNN in a task-dependent manner, our approach disentangles stability and plasticity, and outperforms alternative methods in a range of experiments. Overall, our work provides several key insights on the differences between CL in feedforward networks and in RNNs, while offering a novel solution to effectively tackle CL on sequential data.
CT Image Super Resolution Based On Improved SRGAN

CT images are commonly used in medical clinical diagnosis. However, due to factors such as hardware and scanning time, CT images in real scenes are limited by spatial resolution so that doctors cannot perform accurate disease analysis on tiny lesion areas and pathological features. An image super-resolution (SR) method based on deep learning is a good way to solve this problem. Although many excellent networks have been proposed, but they all pay more attention to image quality indicators than image visual perception quality. Unlike other networks that focus more on image evaluation metrics, the super resolution generative adversarial network (SRGAN) has achieved tremendous improvements in image perception quality. Based on the above, this paper proposes a CT image super-resolution algorithm based on improved SRGAN. In order to improve the visual quality of CT images, a dilated convolution module is introduced. At the same time, in order to improve the overall visual effect of the image, the mean structural similarity (MSSIM) loss is also introduced to improve the perceptual loss function. Experimental results on the public CT image dataset demonstrate that our model is better than the baseline method SRGAN not only in mean opinion score(MOS), but also in peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) values.
A Multi-Task Learning Approach to Sarcasm Detection (Student Abstract)

Sarcasm detection plays an important role in natural language processing as it has been considered one of the most challenging subtasks in sentiment analysis and opinion mining applications. Our work aims to detect sarcasm in social media sites and discussion forums, exploiting the potential of deep neural networks and multi-task learning. Specifically, relying on the strong correlation between sarcasm and (implied negative) sentiment, we explore a multi-task learning framework that uses sentiment classification as an auxiliary task to inform the main task of sarcasm detection. Our proposed model outperforms many previous baseline methods on an existing large dataset annotated with sarcasm.
Applying Qualitative Methods to Investigate Social Actions for Justice Using Social Media: Illustrations From Facebook

Social media is becoming a valuable resource for hosting activism as illustrated in the rise of the hashtag movements, such as #MeToo and #Endrapeculture, used to speak out against rape culture. In this article, we discuss the use of social media as the source and object of research, using the case of the 2016 South African #nakedprotest. We used naturalistic observation on Facebook comment threads and followed these up with online Facebook focus groups. Qualitative content analysis and thematic decomposition analysis were used, respectively, to explore online discourses of rape culture. We found that the use of social media as a medium for data collection is valuable for exploring trending social issues such as the rape culture #nakedprotest. We uncovered that social media offers researchers the opportunity to collect, analyze, and triangulate rich qualitative data for the exploration of social phenomena. This study illustrates the usefulness of social media as a pedagogical instrument.
Evaluating Performance and Interpretability of Machine Learning Methods for Predicting Delirium in Gerontopsychiatric Patients.

Delirium is an acute mental disturbance that particularly occurs during hospital stay. Current clinical assessment instruments include the Delirium Observation Screening Scale (DOSS) or the Confusion Assessment Method (CAM). The aim of this work is to analyze the performance of machine learning approaches to detect delirium based on DOSS and CAM information obtained from two geropsychiatric wards in Tyrol. From a machine learning perspective, the questions of these two assessment instruments represent the features and the ICD 10 diagnoses of delirium (yes/no) is the corresponding class variable. We compare seven popular classification methods and analyze the performance and interpretability of the learning models. As our dataset is highly imbalanced, we also evaluate the effect of common sampling methods including down- and up-sampling methods as well as hybrid methods. Our results indicate a high predictive ability of advanced methods such as Random Forest that can handle even unbalanced datasets. Overall, combining a good performance of a prediction model with the ability of users to understand the prediction is challenging. However, for clinical application in fully electronic settings, a good performance seems to be more important than an easy interpretation of the prediction by the user. On the other hand, explanations of decisions are often needed to assess other criteria such as safety.
FROM ONLINE HIGH SCHOOL TO ONLINE EDUCATION

The article discusses the advantages and disadvantages of online learning, identifies technical, technological, mental and methodological problems of organizing the educational process in the conditions of forced transition of higher education from contact-classroom learning to education in an online format. It is concluded that it is advisable to introduce hybrid forms of educational process organization in higher education, that combine the advantages of both contact-classroom and online education. At the same time, attention is drawn to the fact that the principles of online education require a rethinking of teaching methods of academic disciplines that have established in the contact-classroom space. The basis for the development of methods for teaching academic subjects in online education can be a cognitive-developing paradigm of education, based on the problem-interactive method of organizing the educational space using modern educational and information and communication technologies.