Active Tuning

1 INTRODUCTION . Recurrent neural networks ( RNNs ) are inherently only robust against noise to a limited extent and they often generate unsuitable predictions when confronted with corrupted or missing data ( cf. , e.g. , Otte et al. , 2015 ) . To tackle noise , an explicit noise-aware training procedure can be employed , yielding denoising networks , which are targeted to handle particular noise types and levels . Recurrent oscillators , such as echo state networks ( ESNs ) ( Jaeger , 2001 ; Koryakin et al. , 2012 ; Otte et al. , 2016 ) , when initialized with teacher forcing , however , are highly dependent on a clean and accurate target signal . Given an overly noisy signal , the system is often not able to tune its neural activities into the desired target dynamics at all . Here , we present a method that can be seen as an alternative to regular teacher forcing and , moreover , as a general tool for more robustly tuning and thus synchronizing the dynamics of a generative differentiable temporal forward model—such as a standard RNN , ESN , or LSTM-like RNN ( Hochreiter & Schmidhuber , 1997 ; Otte et al. , 2014 ; Chung et al. , 2014 ; Otte et al. , 2016 ) —into the observed data stream . The proposed method , which we call Active Tuning , uses gradient back-propagation through time ( BPTT ) ( Werbos , 1990 ) , where the back-propagated gradient signal is used to tune the hidden activities of a neural network instead of adapting its weights . The way we utilize the temporal gradient signal is related to learning parametric biases ( Sugita et al. , 2011 ) and applying dynamic context inference ( Butz et al. , 2019 ) . With Active Tuning , two essential aspects apply : First , during signal inference , the model is not driven by the observations directly , but indirectly via prediction errorinducted temporal gradient information , which is used to infer the hidden state activation sequence that best explains the observed signal . Second , the general stabilization ability of propagating signal hypotheses through the network is exploited , effectively washing out activity components ( such as noise ) that can not be modeled with the learned temporal structures within the network . As a result , the vulnerable internal dynamics are kept within a system-consistent activity milieu , effectively decoupling it from noise or other unknown distortions that are present in the defective actual signal . In this work we show that Active Tuning elicits enhanced signal filtering abilities , without the need for explicitly training distinct models for exactly such purposes . Excitingly , this method allows for instance the successful application of an entirely noise-unaware RNN ( trained on clean , ideal data ) under highly noisy and unknown conditions . In the following , we first detail the Active Tuning algorithm . We then evaluate the RNN on three time series benchmarks—multiple superimposed sine waves , a chaotic pendulum , and spatiotemporal wave dynamics . The results confirm that Active Tuning enhances noise robustness in all cases . The mechanism mostly even beats the performance of networks that were explicitly trained to handle a particular noise level . It can also cope with missing data when tuning the predictor ’ s state into the observations . In conclusion , we recommend to employ Active Tuning in all time series prediction cases , when the data is known to be noisy , corrupted , or to contain missing values and the generative differentiable temporal forward model—typically a particular RNN architecture—knows about the potential underlying system dynamics . The resulting data processing system will be able to handle a larger range of noise and corrupted data , filtering the signal , generating more accurate predictions , and thus identifying the underlying data patterns more accurately and reliably . 2 ACTIVE TUNING . Starting point for the application of Active Tuning is a trained temporal forward model . This may be , as mentioned earlier , an RNN , but could also be another type of temporal model . The prerequisite is , however , a differentiable model that implements dependencies over time , such that BPTT can be used to reversely route gradient information through the computational forward chain . Without loss of generality , we assume that the model of interest , whose forward function may be referred to as fM , fulfills the following structure : ( σt , xt ) fM7−−−→ ( σt+1 , x̃t+1 ) , ( 1 ) where σt is the current latent hidden state of the model ( e.g . the hidden outputs of LSTM units , their cell states , or any other latent variable of interest ) and xt is the current signal observation . Based on this information fM generates a prediction for the next input x̃t+1 and updates its latent state σt+1 accordingly . Following the conventional inference scheme , we feed a given sequence time step by time step into the network and receive a one-time step ahead prediction after each particular step . Over time , this effectively synchronizes the network with the observed signal . Once the network dynamics are initialized , which is typically realized by teacher forcing , the network can generate prediction and its dynamics can be driven further into the future in a closed-loop manner , whereby the network feeds itself with its own predictions . To realize next time step- and closed-loop predictions , direct contact with the signal is inevitable to drive the teacher forcing process . In contrast , Active Tuning decouples the network from the direct influence of the signal . Instead , the model is permanently kept in closed-loop mode , which initially prevents the network from generating meaningful predictions . Over a certain time frame , Active Tuning keeps track of the recent signal history , the recent hidden states of the model , as well as its recent predictions . We call this time frame ( retrospective ) tuning horizon or tuning length ( denoted with R ) . The principle of Active Tuning can best be explained with the help of Figure 1 and Algorithm 1 . The latter gives a more formal perspective onto the principle . Note that for every invocation of the procedure a previously unrolled forward chain ( from the previous invocation or an initial unrolling ) is assumed . L refers to the prediction error of the entire unrolled prediction sequence and the respective observations , whereas Lt′ refers to the local prediction error just for a time step t′ . With every new perceived and potentially noise-affected signal observation xt , one or multiple tuning cycles are performed . Every tuning cycle hereby consists of the following stages : First , from the currently believed sequence of signal predictions ( which is in turn based on a sequence of hidden states ) and the actual observed recent inputs , a prediction error is calculated and propagated back into the past reversely along the unfolded forward computation sequence . The temporal gradient travels to the very left of the tuning horizon and is finally projected onto the seed hidden state σt−R , which is then adapted by applying the gradient signal in order to minimize the encountered prediction error . This adaption can be done using any gradient-based optimizer . Note that in this paper , we exclusively use Adam ( Kingma & Ba , 2015 ) , but other optimizers are possible as well . Second , after the adap- Algorithm 1 : Active Tuning procedure Input : Current observation xt Output : Prediction x̃t ( filtered output ) , predictive hidden state σt x̃t , σt← fM ( x̃t−1 , σt−1 ) / * Generate current prediction based on previous forward chain * / for c← 1 to C do / * Perform multiple tuning cycles * / for t′← t to t−R do / * Back-propagate prediction error * / gt ′ ← ∂L ∂σt ′ = ∂Lt ′ ∂σt ′ + g t′+1 ∂σ t′+1 ∂σt ′ if t ′ < t 0 otherwise end for σt−R← update ( σt−R , gt−R ) / * Perform tuning step ( e.g . with Adam update rule ) * / for t′← t−R+ 1 to t do / * Roll out forward chain again based on adapted hidden state * / x̃t ′ , σt ′ ← fM ( x̃t ′−1 , σt ′−1 ) end for end for return x̃t , σt tion of this seed state ( and maybe the seed input as well ) the prediction sequence is rolled out from the past into the present again , effectively refining the output sequence towards a better explanation of the recently observed signal . Each tuning cycle thus updates the current prediction x̃t and the current hidden state σt from which a closed-loop future prediction can be rolled out , if desired . To transition into the next world time step , one forward step has to be computed . The formerly leftmost seed states can be discarded and the recorded history is shifted by one time step , making σt−R+1 the new seed state that will be tuned within the next world time step . From then on , the procedure is repeated , yielding the continuous adaptive tuning process . As a result , the model is predominantly driven by its own imagination , that is , its own top down predictions . Meanwhile , the predictions themselves are adapted by means of the temporal gradients based on the accumulated prediction error , but not by the signal directly . In a nutshell , Active Tuning realizes a gradient-based minioptimization procedure on any of the model ’ s latent variables within one world time step . While it needs to be acknowledged that this process draws on additional computational resources , in this paper we investigate the resulting gain in signal processing robustness . Intuitively speaking , Active Tuning tries to fit known temporal patterns , as memorized within the forward model , to the concurrently observed data . Due to the strong pressure towards consistency maintenance , which is naturally enforced by means of the temporal gradient information in combination with the repeatedly performed forward passes of the hidden state activities , the network will generate adaptations and potential recombinations of patterns that it has learned during training . Occurrences that can not be generated from the repertoire of neural dynamics will therefore not appear ( or only in significantly suppressed form ) in the model ’ s output . As a consequence , there is a much smaller need to strive for noise robustness during training . Our results below indeed confirm that the model may be trained on clean , idealized target signals . However , imprinting a slight denoising tendency during training proves to be useful when facing more noisy data . Enhanced with our Active Tuning scheme , the model will be able to robustly produce high-quality outputs even under extremely adverse conditions—as long as ( some of ) the assumed target signals are actually present . Our scheme is thus a tool that can be highly useful in various application scenarios for signal reconstruction and flexible denoising . Nevertheless , it should be mentioned that with Active Tuning the computational overhead for inference scales with the number of tuning cycles and the tuning length . 3 EXPERIMENTS . In order to investigate the abilities of Active Tuning we studied its behavior at considering three different types of time series data , namely , one-dimensional linear dynamics , two-dimensional nonlinear dynamics , and distributed spatiotemporal dynamics . For all three problem domains we used a comparable setup except for the particular recurrent neural network architectures applied . We trained the networks as one time step ahead predictors whose task is to predict the next input given both the current input and the history of inputs aggregated in the latent hidden state of the models . The target sequences were generated directly from the clean input sequences by realizing a shift of one time step . Moreover , we trained networks under six different denoising conditions ( normally distributed ) per experiment , where we fed a potentially noisy signal into the network and provide the true signal ( one time step ahead ) as the target value ( Lu et al. , 2013 ; Otte et al. , 2015 ; Goodfellow et al. , 2016 ) . These conditions are determined by their relative noise ratios : 0.0 ( no noise ) , 0.05 , 0.1 , 0.2 , 0.5 , and 1.0 , where the ratios depend on the respective base signal statistics . For instance , a noise ratio of 0.1 means that the noise added to the input has a standard deviation of 0.1 times the standard deviation of the base signal . As a result we obtained predictive denoising experts for each of these conditions . All models were trained with Adam ( Kingma & Ba , 2015 ) using its default parameters ( learning rate η = 0.001 , β1 = 0.9 and β2 = 0.999 ) over 100 ( first two experiments ) or 200 ( third experiment ) epochs , respectively .

Summary: Paper proposes a way to adapt an autoregressive model (RNN in examples) to the incoming noisy signal to generate noise-free data output. The approach is interesting due to applying updates to the hidden state of the past observation. The proposed approached is named Active Tuning and evaluated on 3 toy tasks. The idea sounds interesting, however the lack of comparisons with other approaches and theoretical justification of why this approach is superior makes it hard to convince reader.