Miracle-dz/UniFAR-FTUs huggingface dataset:
Self-Attentional Credit Assignment for Transfer in Reinforcement Learning

The ability to transfer knowledge to novel environments and tasks is a sensible desiderata for general learning agents. Despite the apparent promises, transfer in RL is still an open and little exploited research area. In this paper, we take a brand-new perspective about transfer: we suggest that the ability to assign credit unveils structural invariants in the tasks that can be transferred to make RL more sample-efficient. Our main contribution is SECRET, a novel approach to transfer learning for RL that uses a backward-view credit assignment mechanism based on a self-attentive architecture. Two aspects are key to its generality: it learns to assign credit as a separate offline supervised process and exclusively modifies the reward function. Consequently, it can be supplemented by transfer methods that do not modify the reward function and it can be plugged on top of any RL algorithm.
Retinal Vessel Segmentation based on Fully Convolutional Networks

The morphological attributes of retinal vessels, such as length, width, tortuosity and branching pattern and angles, play an important role in diagnosis, screening, treatment, and evaluation of various cardiovascular and ophthalmologic diseases such as diabetes, hypertension and arteriosclerosis. The crucial step before extracting these morphological characteristics of retinal vessels from retinal fundus images is vessel segmentation. In this work, we propose a method for retinal vessel segmentation based on fully convolutional networks. Thousands of patches are extracted from each retinal image and then fed into the network, and data argumentation is applied by rotating extracted patches. Two architectures of fully convolutional networks, U-Net and LadderNet, are used for vessel segmentation. The performance of our method is evaluated on three public datasets: DRIVE, STARE, and CHASE\_DB1. Experimental results of our method show superior performance compared to recent state-of-the-art methods.
BlendedMVS: A Large-Scale Dataset for Generalized Multi-View Stereo Networks

While deep learning has recently achieved great success on multi-view stereo (MVS), limited training data makes the trained model hard to be generalized to unseen scenarios. Compared with other computer vision tasks, it is rather difficult to collect a large-scale MVS dataset as it requires expensive active scanners and labor-intensive process to obtain ground truth 3D structures. In this paper, we introduce BlendedMVS, a novel large-scale dataset, to provide sufficient training ground truth for learning-based MVS. To create the dataset, we apply a 3D reconstruction pipeline to recover high-quality textured meshes from images of well-selected scenes. Then, we render these mesh models to color images and depth maps. To introduce the ambient lighting information during training, the rendered color images are further blended with the input images to generate the training input. Our dataset contains over 17k high-resolution images covering a variety of scenes, including cities, architectures, sculptures and small objects. Extensive experiments demonstrate that BlendedMVS endows the trained model with significantly better generalization ability compared with other MVS datasets. The dataset and pretrained models are available at https://github.com/YoYo000/BlendedMVS.
Machine-learning-based Classification of Lower-grade gliomas and High-grade gliomas using Radiomic Features in Multi-parametric MRI

Objectives: Glioblastomas are the most aggressive brain and central nervous system (CNS) tumors with poor prognosis in adults. The purpose of this study is to develop a machine-learning based classification method using radio-mic features of multi-parametric MRI to classify high-grade gliomas (HGG) and low-grade gliomas (LGG). Methods: Multi-parametric MRI of 80 patients, 40 HGG and 40 LGG, with gliomas from the MICCAI BRATs 2015 training database were used in this study. Each patient's T1, contrast-enhanced T1, T2, and Fluid Attenuated Inversion Recovery (FLAIR) MRIs as well as the tumor contours were provided in the database. Using the given contours, radiomic features from all four multi-parametric MRIs were extracted. Of these features, a feature selection process using two-sample T-test and least absolute shrinkage, selection operator (LASSO), and a feature correlation threshold was applied to various combinations of T1, contrast-enhanced T1, T2, and FLAIR MRIs separately. These selected features were then used to train, test, and cross-validate a random forest to differentiate HGG and LGG. Finally, the classification accuracy and area under the curve (AUC) were used to evaluate the classification method. Results: Optimized parameters showed that on average, the overall accuracy of our classification method was 0.913 or 73 out of 80 correct classifications, 36/40 for HGG and 37/40 for LGG, with an AUC of 0.956 based on the combination with FLAIR, T1, T1c and T2 MRIs. Conclusion: This study shows that radio-mic features derived from multi-parametric MRI could be used to accurately classify high and lower grade gliomas. The radio-mic features from multi-parametric MRI in combination with even more advanced machine learning methods may further elucidate the underlying tumor biology and response to therapy.
Deep Reinforcement Learning for Trading

In this article, the authors adopt deep reinforcement learning algorithms to design trading strategies for continuous futures contracts. Both discrete and continuous action spaces are considered, and volatility scaling is incorporated to create reward functions that scale trade positions based on market volatility. They test their algorithms on 50 very liquid futures contracts from 2011 to 2019 and investigate how performance varies across different asset classes, including commodities, equity indexes, fixed income, and foreign exchange markets. They compare their algorithms against classical time-series momentum strategies and show that their method outperforms such baseline models, delivering positive profits despite heavy transaction costs. The experiments show that the proposed algorithms can follow large market trends without changing positions and can also scale down, or hold, through consolidation periods. TOPICS: Futures and forward contracts, exchanges/markets/clearinghouses, statistical methods, simulations Key Findings • In this article, the authors introduce reinforcement learning algorithms to design trading strategies for futures contracts. They investigate both discrete and continuous action spaces and improve reward functions by using volatility scaling to scale trade positions based on market volatility. • The authors discuss the connection between modern portfolio theory and the reinforcement learning reward hypothesis and show that they are equivalent if a linear utility function is used. • The authors back test their methods on 50 very liquid futures contracts from 2011 to 2019, and their algorithms deliver positive profits despite heavy transaction costs.
Neural Networks Learning and Memorization with (almost) no Over-Parameterization

Many results in recent years established polynomial time learnability of various models via neural networks algorithms. However, unless the model is linear separable, or the activation is a polynomial, these results require very large networks -- much more than what is needed for the mere existence of a good predictor. In this paper we prove that SGD on depth two neural networks can memorize samples, learn polynomials with bounded weights, and learn certain kernel spaces, with near optimal network size, sample complexity, and runtime. In particular, we show that SGD on depth two network with $\tilde{O}\left(\frac{m}{d}\right)$ hidden neurons (and hence $\tilde{O}(m)$ parameters) can memorize $m$ random labeled points in $\mathbb{S}^{d-1}$.
Identification of time-varying source term in time-fractional diffusion equations

This paper is concerned with the inverse problem of determining the time and space dependent source term of diffusion equations with constant-order time-fractional derivative in (0, 2). We examine two different cases. In the first one, the source is the product of two spatial and temporal terms, and we prove that both of them can be retrieved by knowledge of one arbitrary internal measurement of the solution for all times. In the second case, we assume that the first term of the product varies with one fixed space variable, while the second one is a function of all the remaining space variables and the time variable, and we show that both terms are uniquely determined by two arbitrary lateral measurements of the solution over the entire time span. These two source identification results boil down to a weak unique continuation principle in the first case and a unique continuation principle for Cauchy data in the second one, that are preliminarily established. Finally, numerical reconstruction of spatial term of source terms in the form of the product of two spatial and temporal terms, is carried out through an iterative algorithm based on the Tikhonov regularization method.
DLGAN: Disentangling Label-Specific Fine-Grained Features for Image Manipulation

Recent studies have shown how disentangling images into content and feature spaces can provide controllable image translation/ manipulation. In this paper, we propose a framework to enable utilizing discrete multi-labels to control which features to be disentangled, i.e., disentangling label-specific fine-grained features for image manipulation (dubbed DLGAN). By mapping the discrete label-specific attribute features into a continuous prior distribution, we leverage the advantages of both discrete labels and reference images to achieve image manipulation in a hybrid fashion. For example, given a face image dataset (e.g., CelebA) with multiple discrete fine-grained labels, we can learn to smoothly interpolate a face image between black hair and blond hair through reference images while immediately controlling the gender and age through discrete input labels. To the best of our knowledge, this is the first work that realizes such a hybrid manipulation within a single model. More importantly, it is the first work to achieve image interpolation between two different domains without requiring continuous labels as the supervision. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed method.
Interactive Text Ranking with Bayesian Optimization: A Case Study on Community QA and Summarization

Abstract For many NLP applications, such as question answering and summarization, the goal is to select the best solution from a large space of candidates to meet a particular user’s needs. To address the lack of user or task-specific training data, we propose an interactive text ranking approach that actively selects pairs of candidates, from which the user selects the best. Unlike previous strategies, which attempt to learn a ranking across the whole candidate space, our method uses Bayesian optimization to focus the user’s labeling effort on high quality candidates and integrate prior knowledge to cope better with small data scenarios. We apply our method to community question answering (cQA) and extractive multidocument summarization, finding that it significantly outperforms existing interactive approaches. We also show that the ranking function learned by our method is an effective reward function for reinforcement learning, which improves the state of the art for interactive summarization.
This Car is Mine!: Automobile Theft Countermeasure Leveraging Driver Identification with Generative Adversarial Networks

As a car becomes more connected, a countermeasure against automobile theft has become a significant task in the real world. To respond to automobile theft, data mining, biometrics, and additional authentication methods are proposed. Among current countermeasures, data mining method is one of the efficient ways to capture the owner driver's unique characteristics. To identify the owner driver from thieves, previous works applied various algorithms toward driving data. Such data mining methods utilized supervised learning, thus required labeled data set. However, it is unrealistic to gather and apply the thief's driving pattern. To overcome this problem, we propose driver identification method with GAN. GAN has merit to build identification model by learning the owner driver's data only. We trained GAN only with owner driver's data and used trained discriminator to identify the owner driver. From actual driving data, we evaluated our identification model recognizes the owner driver well. By ensembling various driver authentication methods with the proposed model, we expect industry can develop automobile theft countermeasures available in the real world.
Efficient Exploration through Intrinsic Motivation Learning for Unsupervised Subgoal Discovery in Model-Free Hierarchical Reinforcement Learning

Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method based on unsupervised learning over a limited memory of agent's experiences during intrinsic motivation. Additionally, we offer a unified approach to learning representations in model-free HRL.
Physiotherapy for patients with head and neck cancer | NOVA. The University of Newcastle's Digital Repository

The management of head and neck cancer is complex and challenging. Treatment frequently results in physical morbidity, such as shoulder pain and dysfunction, and facial lymphoedema. Neck dissection surgery, involving lymph node removal, is the most common treatment for head and neck cancer. Intra-operatively, the accessory nerve can become injured, even when it remains macroscopically intact, causing reduced trapezius muscle activity and weakness. Shoulder pain and dysfunction ensues, impacting on quality of life and function. Facial lymphoedema can result from the surgical removal of cervical lymph nodes and radiation therapy. The primary aim of this thesis was to investigate accessory nerve injury related to neck dissection surgery, and the effectiveness of a biomechanically specific physiotherapy intervention compared to a control group. The secondary aim was to explore the effects of facial lymphoedema on patients, and the understanding of and management of the condition by treating health professionals. The qualitative facial lymphoedema study consisted of interviews of both patients with secondary facial lymphoedema as a result of treatment of head and neck cancer, and treating health professionals. The main effect of the condition experienced by patients concerned appearance and body image. The understanding of interviewed health professionals regarding the condition was found to be similar to the patient experience, however patients experiencing facial lymphoedema were generally not referred to physiotherapy. Treating health professionals need to routinely screen for any potential negative psychological and physical effects of facial lymphoedema, and affected patients referred to physiotherapy if required. Our literature review found that little evidence exists pertaining to the effectiveness of physiotherapy on accessory nerve shoulder dysfunction after neck dissection surgery exists, with only one previous randomised controlled study published. We then undertook a case control electromyography study, to investigate any scapular muscle activity differences following neck dissection surgery in patients with clinical signs of
Fast Sparse ConvNets

Historically, the pursuit of efficient inference has been one of the driving forces behind the research into new deep learning architectures and building blocks. Some of the recent examples include: the squeeze-and-excitation module, depthwise separable convolutions in Xception, and the inverted bottleneck in MobileNet v2. Notably, in all of these cases, the resulting building blocks enabled not only higher efficiency, but also higher accuracy, and found wide adoption in the field. In this work, we further expand the arsenal of efficient building blocks for neural network architectures; but instead of combining standard primitives (such as convolution), we advocate for the replacement of these dense primitives with their sparse counterparts. While the idea of using sparsity to decrease the parameter count is not new, the conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains. We aim to correct this misconception by introducing a family of efficient sparse kernels for several hardware platforms, which we plan to open source for the benefit of the community. Equipped with our efficient implementation of sparse primitives, we show that sparse versions of MobileNet v1 and MobileNet v2 architectures substantially outperform strong dense baselines on the efficiency-accuracy curve. On Snapdragon 835 our sparse networks outperform their dense equivalents by 1.3 − 2.4× – equivalent to approximately one entire generation of improvement. We hope that our findings will facilitate wider adoption of sparsity as a tool for creating efficient and accurate deep learning architectures.
The Stanford Acuity Test: A Precise Vision Test Using Bayesian Techniques and a Discovery in Human Visual Response

Chart-based visual acuity measurements are used by billions of people to diagnose and guide treatment of vision impairment. However, the ubiquitous eye exam has no mechanism for reasoning about uncertainty and as such, suffers from a well-documented reproducibility problem. In this paper we make two core contributions. First, we uncover a new parametric probabilistic model of visual acuity response based on detailed measurements of patients with eye disease. Then, we present an adaptive, digital eye exam using modern artificial intelligence techniques which substantially reduces acuity exam error over existing approaches, while also introducing the novel ability to model its own uncertainty and incorporate prior beliefs. Using standard evaluation metrics, we estimate a 74% reduction in prediction error compared to the ubiquitous chart-based eye exam and up to 67% reduction compared to the previous best digital exam. For patients with eye disease, the novel ability to finely measure acuity from home could be a crucial part in early diagnosis. We provide a web implementation of our algorithm for anyone in the world to use. The insights in this paper also provide interesting implications for the field of psychometric Item Response Theory.
On the Robustness of Signal Characteristic-Based Sender Identification

Vehicles become more vulnerable to remote attackers in modern days due to their increasing connectivity and range of functionality. Such increased attack vectors enable adversaries to access a vehicle Electronic Control Unit (ECU). As of today in-vehicle access can cause drastic consequences, because the most commonly used in-vehicle bus technology, the Controller Area Network (CAN), lacks sender identification. With low limits on bandwidth and payload, as well as resource constrains on hardware, usage of cryptographic measures is limited. As an alternative, sender identification methods were presented, identifying the sending ECU on the basis of its analog message signal. While prior works showed promising results on the security and feasibility for those approaches, the potential changes in signals over a vehicle's lifetime have only been partly addressed. This paper closes this gap. We conduct a 4~months measurement campaign containing more than 80,000 frames from a real vehicle. The data reflects different driving situations, different seasons and weather conditions, a 19-week break, and a car repair altering the physical CAN properties. We demonstrate the impact of temperature dependencies, analyze the signal changes and define strategies for their handling. In the evaluation, the identification rate can be increased from 91.23% to 99.98% by a targeted updating of the system parameters. At the same time, the detection of intrusions can be improved from 76.83% to 99.74%, while no false positives occured during evaluation. Lastly, we show how to increase the overall performance of such systems by double monitoring the bus at different positions.
Strain-induced phase transition in CrI3 bilayers

A monolayer of CrI3 is a two-dimensional crystal in its equilibrium configuration is a ferromagnetic semiconductor. In contrast, two coupled layers can be ferromagnetic, or antiferromagnetic depending on the stacking. We study the magnetic phase diagram upon the strain of the antiferromagnetically coupled bilayer with C2/m symmetry. We found that strain can be an efficient tool to tune the magnetic phase of the structure. A tensile strain stabilizes the antiferromagnetic phase, while a compressive strain turns the system ferromagnetic. We associate that behavior to the relative displacement between layers induced by the strain. We also study the evolution of the magnetic anisotropy, the magnetic exchange coupling, and how the Curie temperature is affected by the strain.
Correction: A Breakthrough in R&D for Neglected Diseases: New Ways to Get the Drugs We Need

The third sentence of the Methods and Findings section in the Abstract should read as follows: “Incidence rates rise with age, particularly above the age of 75 y, from 6.7 (95% confi dence interval, 3.8–12.4) per 1,000 person years at age 65–69 y to 68.5 (95% confi dence interval, 52.5–88.1) per 1,000 person years at age 85 y and above. ” The fi fth sentence should read as follows: “Hence, it is estimated that approximately 163,000 new cases of dementia occur in England and Wales each year. ”
Unsupervised Features Learning for Sampled Vector Fields

In this paper we introduce a new approach to computing hidden features of sampled vector fields. The basic idea is to convert the vector field data to a graph structure and use tools designed for automatic, unsupervised analysis of graphs. Using a few data sets we show that the collected features of the vector fields are correlated with the dynamics known for analytic models which generates the data. In particular the method may be useful in analysis of data sets where the analytic model is poorly understood or not known.
Improving Conditioning in Context-Aware Sequence to Sequence Models

Neural sequence to sequence models are well established for applications which can be cast as mapping a single input sequence into a single output sequence. In this work, we focus on cases where generation is conditioned on both a short query and a long context, such as abstractive question answering or document-level translation. We modify the standard sequence-to-sequence approach to make better use of both the query and the context by expanding the conditioning mechanism to intertwine query and context attention. We also introduce a simple and efficient data augmentation method for the proposed model. Experiments on three different tasks show that both changes lead to consistent improvements.