pwc-archive/papers-with-abstracts huggingface dataset:
EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts
Authors: Christian Requena-Mesa, Vitus Benson, Joachim Denzler, Jakob Runge, Markus Reichstein
Date: 2020-12-11 00:00:00
Tasks: Crop Yield Prediction, Earth Observation, Earth Surface Forecasting, Management, Video Prediction

Climate change is global, yet its concrete impacts can strongly vary between different locations in the same region. Seasonal weather forecasts currently operate at the mesoscale (> 1 km). For more targeted mitigation and adaptation, modelling impacts to < 100 m is needed. Yet, the relationship between driving variables and Earth's surface at such local scales remains unresolved by current physical models. Large Earth observation datasets now enable us to create machine learning models capable of translating coarse weather information into high-resolution Earth surface forecasts. Here, we define high-resolution Earth surface forecasting as video prediction of satellite imagery conditional on mesoscale weather forecasts. Video prediction has been tackled with deep learning models. Developing such models requires analysis-ready datasets. We introduce EarthNet2021, a new, curated dataset containing target spatio-temporal Sentinel 2 satellite imagery at 20 m resolution, matched with high-resolution topography and mesoscale (1.28 km) weather variables. With over 32000 samples it is suitable for training deep neural networks. Comparing multiple Earth surface forecasts is not trivial. Hence, we define the EarthNetScore, a novel ranking criterion for models forecasting Earth surface reflectance. For model intercomparison we frame EarthNet2021 as a challenge with four tracks based on different test sets. These allow evaluation of model validity and robustness as well as model applicability to extreme events and the complete annual vegetation cycle. In addition to forecasting directly observable weather impacts through satellite-derived vegetation indices, capable Earth surface models will enable downstream applications such as crop yield prediction, forest health assessments, coastline management, or biodiversity monitoring. Find data, code, and how to participate at www.earthnet.tech .
ADD: Augmented Disentanglement Distillation Framework for Improving Stock Trend Forecasting
Authors: Hongshun Tang, Lijun Wu, Weiqing Liu, Jiang Bian
Date: 2020-12-11 00:00:00
Tasks: Decoder, Disentanglement

Stock trend forecasting has become a popular research direction that attracts widespread attention in the financial field. Though deep learning methods have achieved promising results, there are still many limitations, for example, how to extract clean features from the raw stock data. In this paper, we introduce an \emph{Augmented Disentanglement Distillation (ADD)} approach to remove interferential features from the noised raw data. Specifically, we present 1) a disentanglement structure to separate excess and market information from the stock data to avoid the two factors disturbing each other's own prediction. Besides, by applying 2) a dynamic self-distillation method over the disentanglement framework, other implicit interference factors can also be removed. Further, thanks to the decoder module in our framework, 3) a novel strategy is proposed to augment the training samples based on the different excess and market features to improve performance. We conduct experiments on the Chinese stock market data. Results show that our method significantly improves the stock trend forecasting performances, as well as the actual investment income through backtesting, which strongly demonstrates the effectiveness of our approach.
Artificial Intelligence for COVID-19 Detection -- A state-of-the-art review
Authors: Parsa Sarosh, Shabir A. Parah, Romany F Mansur, G. M. Bhat
Date: 2020-11-25 00:00:00
Tasks: Management, Survival Prediction, Transfer Learning

The emergence of COVID-19 has necessitated many efforts by the scientific community for its proper management. An urgent clinical reaction is required in the face of the unending devastation being caused by the pandemic. These efforts include technological innovations for improvement in screening, treatment, vaccine development, contact tracing and, survival prediction. The use of Deep Learning (DL) and Artificial Intelligence (AI) can be sought in all of the above-mentioned spheres. This paper aims to review the role of Deep Learning and Artificial intelligence in various aspects of the overall COVID-19 management and particularly for COVID-19 detection and classification. The DL models are developed to analyze clinical modalities like CT scans and X-Ray images of patients and predict their pathological condition. A DL model aims to detect the COVID-19 pneumonia, classify and distinguish between COVID-19, Community-Acquired Pneumonia (CAP), Viral and Bacterial pneumonia, and normal conditions. Furthermore, sophisticated models can be built to segment the affected area in the lungs and quantify the infection volume for a better understanding of the extent of damage. Many models have been developed either independently or with the help of pre-trained models like VGG19, ResNet50, and AlexNet leveraging the concept of transfer learning. Apart from model development, data preprocessing and augmentation are also performed to cope with the challenge of insufficient data samples often encountered in medical applications. It can be evaluated that DL and AI can be effectively implemented to withstand the challenges posed by the global emergency
Differentiable Histogram with Hard-Binning
Authors: Ibrahim Yusuf, George Igwegbe, Oluwafemi Azeez
Date: 2020-11-20 00:00:00

The simplicity and expressiveness of a histogram render it a useful feature in different contexts including deep learning. Although the process of computing a histogram is non-differentiable, researchers have proposed differentiable approximations, which have some limitations. A differentiable histogram that directly approximates the hard-binning operation in conventional histograms is proposed. It combines the strength of existing differentiable histograms and overcomes their individual challenges. In comparison to a histogram computed using Numpy, the proposed histogram has an absolute approximation error of 0.000158.
Deep Reinforcement Learning for Long Term Hydropower Production Scheduling
Authors: Signe Riemer-Sorensen, Gjert H. Rosenlund
Date: 2020-12-09 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

We explore the use of deep reinforcement learning to provide strategies for long term scheduling of hydropower production. We consider a use-case where the aim is to optimise the yearly revenue given week-by-week inflows to the reservoir and electricity prices. The challenge is to decide between immediate water release at the spot price of electricity and storing the water for later power production at an unknown price, given constraints on the system. We successfully train a soft actor-critic algorithm on a simplified scenario with historical data from the Nordic power market. The presented model is not ready to substitute traditional optimisation tools but demonstrates the complementary potential of reinforcement learning in the data-rich field of hydropower scheduling.
6-Layer Model for a Structured Description and Categorization of Urban Traffic and Environment
Authors: Maike Scholtes, Lukas Westhofen, Lara Ruth Turner, Katrin Lotto, Michael Schuldes, Hendrik Weber, Nicolas Wagener, Christian Neurohr, Martin Bollmann, Franziska Körtke, Johannes Hiller, Michael Hoss, Julian Bock, Lutz Eckstein
Date: 2020-12-09 00:00:00

Verification and validation of automated driving functions impose large challenges. Currently, scenario-based approaches are investigated in research and industry, aiming at a reduction of testing efforts by specifying safety relevant scenarios. To define those scenarios and operate in a complex real-world design domain, a structured description of the environment is needed. Within the PEGASUS research project, the 6-Layer Model (6LM) was introduced for the description of highway scenarios. This paper refines the 6LM and extends it to urban traffic and environment. As defined in PEGASUS, the 6LM provides the possibility to categorize the environment and, therefore, functions as a structured basis for subsequent scenario description. The model enables a structured description and categorization of the general environment, without incorporating any knowledge or anticipating any functions of actors. Beyond that, there is a variety of other applications of the 6LM, which are elaborated in this paper. The 6LM includes a description of the road network and traffic guidance objects, roadside structures, temporary modifications of the former, dynamic objects, environmental conditions and digital information. The work at hand specifies each layer by categorizing its items. Guidelines are formulated and explanatory examples are given to standardize the application of the model for an objective environment description. In contrast to previous publications, the model and its design are described in far more detail. Finally, the holistic description of the 6LM presented includes remarks on possible future work when expanding the concept to machine perception aspects.
Deep Reinforcement Learning for Stock Portfolio Optimization
Authors: Le Trung Hieu
Date: 2020-12-09 00:00:00
Tasks: Deep Reinforcement Learning, Portfolio Optimization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Stock portfolio optimization is the process of constant re-distribution of money to a pool of various stocks. In this paper, we will formulate the problem such that we can apply Reinforcement Learning for the task properly. To maintain a realistic assumption about the market, we will incorporate transaction cost and risk factor into the state as well. On top of that, we will apply various state-of-the-art Deep Reinforcement Learning algorithms for comparison. Since the action space is continuous, the realistic formulation were tested under a family of state-of-the-art continuous policy gradients algorithms: Deep Deterministic Policy Gradient (DDPG), Generalized Deterministic Policy Gradient (GDPG) and Proximal Policy Optimization (PPO), where the former two perform much better than the last one. Next, we will present the end-to-end solution for the task with Minimum Variance Portfolio Theory for stock subset selection, and Wavelet Transform for extracting multi-frequency data pattern. Observations and hypothesis were discussed about the results, as well as possible future research directions.1
exploRNN: Understanding Recurrent Neural Networks through Visual Exploration
Authors: Alex Bäuerle, Patrick Albus, Raphael Störk, Tina Seufert, Timo Ropinski
Date: 2020-12-09 00:00:00

Due to the success of deep learning (DL) and its growing job market, students and researchers from many areas are interested in learning about DL technologies. Visualization has proven to be of great help during this learning process. While most current educational visualizations are targeted towards one specific architecture or use case, recurrent neural networks (RNNs), which are capable of processing sequential data, are not covered yet. This is despite the fact that tasks on sequential data, such as text and function analysis, are at the forefront of DL research. Therefore, we propose exploRNN, the first interactively explorable educational visualization for RNNs. On the basis of making learning easier and more fun, we define educational objectives targeted towards understanding RNNs. We use these objectives to form guidelines for the visual design process. By means of exploRNN, which is accessible online, we provide an overview of the training process of RNNs at a coarse level, while also allowing a detailed inspection of the data flow within LSTM cells. In an empirical study, we assessed 37 subjects in a between-subjects design to investigate the learning outcomes and cognitive load of exploRNN compared to a classic text-based learning environment. While learners in the text group are ahead in superficial knowledge acquisition, exploRNN is particularly helpful for deeper understanding of the learning content. In addition, the complex content in exploRNN is perceived as significantly easier and causes less extraneous load than in the text group. The study shows that for difficult learning material such as recurrent networks, where deep understanding is important, interactive visualizations such as exploRNN can be helpful.
Privacy and Robustness in Federated Learning: Attacks and Defenses
Authors: Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, Philip S. Yu
Date: 2020-12-07 00:00:00
Tasks: Federated Learning, Privacy Preserving

As data are increasingly being stored in different silos and societies becoming more aware of data privacy issues, the traditional centralized training of artificial intelligence (AI) models is facing efficiency and privacy challenges. Recently, federated learning (FL) has emerged as an alternative solution and continue to thrive in this new reality. Existing FL protocol design has been shown to be vulnerable to adversaries within or outside of the system, compromising data privacy and system robustness. Besides training powerful global models, it is of paramount importance to design FL systems that have privacy guarantees and are resistant to different types of adversaries. In this paper, we conduct the first comprehensive survey on this topic. Through a concise introduction to the concept of FL, and a unique taxonomy covering: 1) threat models; 2) poisoning attacks and defenses against robustness; 3) inference attacks and defenses against privacy, we provide an accessible review of this important topic. We highlight the intuitions, key techniques as well as fundamental assumptions adopted by various attacks and defenses. Finally, we discuss promising future research directions towards robust and privacy-preserving federated learning.
The Why, What and How of Artificial General Intelligence Chip Development
Authors: Alex James
Date: 2020-12-08 00:00:00
Tasks: Edge-computing

The AI chips increasingly focus on implementing neural computing at low power and cost. The intelligent sensing, automation, and edge computing applications have been the market drivers for AI chips. Increasingly, the generalisation, performance, robustness, and scalability of the AI chip solutions are compared with human-like intelligence abilities. Such a requirement to transit from application-specific to general intelligence AI chip must consider several factors. This paper provides an overview of this cross-disciplinary field of study, elaborating on the generalisation of intelligence as understood in building artificial general intelligence (AGI) systems. This work presents a listing of emerging AI chip technologies, classification of edge AI implementations, and the funnel design flow for AGI chip development. Finally, the design consideration required for building an AGI chip is listed along with the methods for testing and validating it.
Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment
Authors: Julien Launay, Iacopo Poli, Kilian Müller, Gustave Pariente, Igor Carron, Laurent Daudet, Florent Krzakala, Sylvain Gigan
Date: 2020-12-11 00:00:00

The scaling hypothesis motivates the expansion of models past trillions of parameters as a path towards better performance. Recent significant developments, such as GPT-3, have been driven by this conjecture. However, as models scale-up, training them efficiently with backpropagation becomes difficult. Because model, pipeline, and data parallelism distribute parameters and gradients over compute nodes, communication is challenging to orchestrate: this is a bottleneck to further scaling. In this work, we argue that alternative training methods can mitigate these issues, and can inform the design of extreme-scale training hardware. Indeed, using a synaptically asymmetric method with a parallelizable backward pass, such as Direct Feedback Alignement, communication needs are drastically reduced. We present a photonic accelerator for Direct Feedback Alignment, able to compute random projections with trillions of parameters. We demonstrate our system on benchmark tasks, using both fully-connected and graph convolutional networks. Our hardware is the first architecture-agnostic photonic co-processor for training neural networks. This is a significant step towards building scalable hardware, able to go beyond backpropagation, and opening new avenues for deep learning.
Closeness and Uncertainty Aware Adversarial Examples Detection in Adversarial Machine Learning
Authors: Omer Faruk Tuna, Ferhat Ozgur Catak, M. Taner Eskil
Date: 2020-12-11 00:00:00
Tasks: BIG-bench Machine Learning

While state-of-the-art Deep Neural Network (DNN) models are considered to be robust to random perturbations, it was shown that these architectures are highly vulnerable to deliberately crafted perturbations, albeit being quasi-imperceptible. These vulnerabilities make it challenging to deploy DNN models in security-critical areas. In recent years, many research studies have been conducted to develop new attack methods and come up with new defense techniques that enable more robust and reliable models. In this work, we explore and assess the usage of different type of metrics for detecting adversarial samples. We first leverage the usage of moment-based predictive uncertainty estimates of a DNN classifier obtained using Monte-Carlo Dropout Sampling. And we also introduce a new method that operates in the subspace of deep features extracted by the model. We verified the effectiveness of our approach on a range of standard datasets like MNIST (Digit), MNIST (Fashion) and CIFAR-10. Our experiments show that these two different approaches complement each other, and the combined usage of all the proposed metrics yields up to 99 \% ROC-AUC scores regardless of the attack algorithm.
Attack Agnostic Detection of Adversarial Examples via Random Subspace Analysis
Authors: Nathan Drenkow, Neil Fendley, Philippe Burlina
Date: 2020-12-11 00:00:00
Tasks: Adversarial Attack, Adversarial Attack Detection

Whilst adversarial attack detection has received considerable attention, it remains a fundamentally challenging problem from two perspectives. First, while threat models can be well-defined, attacker strategies may still vary widely within those constraints. Therefore, detection should be considered as an open-set problem, standing in contrast to most current detection approaches. These methods take a closed-set view and train binary detectors, thus biasing detection toward attacks seen during detector training. Second, limited information is available at test time and typically confounded by nuisance factors including the label and underlying content of the image. We address these challenges via a novel strategy based on random subspace analysis. We present a technique that utilizes properties of random projections to characterize the behavior of clean and adversarial examples across a diverse set of subspaces. The self-consistency (or inconsistency) of model activations is leveraged to discern clean from adversarial examples. Performance evaluations demonstrate that our technique ($AUC\in[0.92, 0.98]$) outperforms competing detection strategies ($AUC\in[0.30,0.79]$), while remaining truly agnostic to the attack strategy (for both targeted/untargeted attacks). It also requires significantly less calibration data (composed only of clean examples) than competing approaches to achieve this performance.
Does the brain function as a quantum phase computer using phase ternary computation?
Authors: Andrew Simon Johnson, William Winlow
Date: 2020-12-04 00:00:00

Here we provide evidence that the fundamental basis of nervous communication is derived from a pressure pulse/soliton capable of computation with sufficient temporal precision to overcome any processing errors. Signalling and computing within the nervous system are complex and different phenomena. Action potentials are plastic and this makes the action potential peak an inappropriate fixed point for neural computation, but the action potential threshold is suitable for this purpose. Furthermore, neural models timed by spiking neurons operate below the rate necessary to overcome processing error. Using retinal processing as our example, we demonstrate that the contemporary theory of nerve conduction based on cable theory is inappropriate to account for the short computational time necessary for the full functioning of the retina and by implication the rest of the brain. Moreover, cable theory cannot be instrumental in the propagation of the action potential because at the activation-threshold there is insufficient charge at the activation site for successive ion channels to be electrostatically opened. Deconstruction of the brain neural network suggests that it is a member of a group of Quantum phase computers of which the Turing machine is the simplest: the brain is another based upon phase ternary computation. However, attempts to use Turing based mechanisms cannot resolve the coding of the retina or the computation of intelligence, as the technology of Turing based computers is fundamentally different. We demonstrate that that coding in the brain neural network is quantum based, where the quanta have a temporal variable and a phase-base variable enabling phase ternary computation as previously demonstrated in the retina.
Differential Evolution for Neural Architecture Search
Authors: Noor Awad, Neeratyoy Mallik, Frank Hutter
Date: 2020-12-11 00:00:00
Tasks: Bayesian Optimization, Neural Architecture Search

Neural architecture search (NAS) methods rely on a search strategy for deciding which architectures to evaluate next and a performance estimation strategy for assessing their performance (e.g., using full evaluations, multi-fidelity evaluations, or the one-shot model). In this paper, we focus on the search strategy. We introduce the simple yet powerful evolutionary algorithm of differential evolution to the NAS community. Using the simplest performance evaluation strategy of full evaluations, we comprehensively compare this search strategy to regularized evolution and Bayesian optimization and demonstrate that it yields improved and more robust results for 13 tabular NAS benchmarks based on NAS-Bench-101, NAS-Bench-1Shot1, NAS-Bench-201 and NAS-HPO bench.
Commutative Evolution Laws in Holographic Cellular Automata: AdS/CFT, Near-Extremal D3-Branes, and a Deep Learning Approach
Authors: Hyunju Go
Date: 2020-12-10 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning

According to 't Hooft, restoring Poincar\'e invariance in a holographic cellular automaton (CA) requires two distinct evolution laws that commute. We explore how this is realized in the AdS/CFT framework, assuming commutativity as a fundamental principle--much like general covariance once did--for encoding curvature. In our setup, physical processes in a given spacetime are encoded in a CA; to preserve Poincar\'e symmetry, the spacetime curvature must effectively vanish, so we consider a near-extremal black D3-brane solution, in which both the stretched horizon and the conformal boundary are approximated by Minkowski space. AdS/CFT implies a spatial evolution law connecting these hypersurfaces. Commutativity means the final state does not depend on the order of time evolution on each hypersurface and spatial evolution between them, forcing the time evolution law on the horizon and boundary to coincide. To satisfy all these conditions, we aim to demonstrate that the spatial evolution law inevitably encapsulates the curvature of the bulk, including quantum effects. For a computational model, we compactify the hyperplanes to tori, reducing the degrees of freedom to a finite number; taking these tori to infinite size then restores Poincar\'e symmetry. We propose a deep learning algorithm that, given a known time evolution law and commutativity, deduces the corresponding spatial evolution law.