Extrapolative Continuous-time Bayesian Neural Network for Fast Training-free Test-time Adaptation
Authors: Hengguan Huang
Xiangming Gu
Hao Wang
Chang Xiao
Hongfu Liu
Ye Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Brain-informed AI
Test-time Adaptation
Real-time Domain Adaptation
Bayesian Deep Learning
Dynamical system
Neural Differential Equation
Particle Filtering

Abstract: Human intelligence has shown remarkably lower latency and higher precision than most AI systems when processing non-stationary streaming data in real-time. Numerous neuroscience studies suggest that such abilities may be driven by internal predictive modeling. In this paper, we explore the possibility of introducing such a mechanism in unsupervised domain adaptation (UDA) for handling non-stationary streaming data for real-time streaming applications. We propose to formulate internal predictive modeling as a continuous-time Bayesian filtering problem within a stochastic dynamical system context. Such a dynamical system describes the dynamics of model parameters of a UDA model evolving with non-stationary streaming data. Building on such a dynamical system, we then develop extrapolative continuous-time Bayesian neural networks (ECBNN), which generalize existing Bayesian neural networks to represent temporal dynamics and allow us to extrapolate the distribution of model parameters before observing the incoming data, therefore effectively reducing the latency. Remarkably, our empirical results show that ECBNN is capable of continuously generating better distributions of model parameters along the time axis given historical data only, thereby achieving (1) training-free test-time adaptation with low latency, (2) gradually improved alignment between the source and target features and (3) gradually improved model performance over time during the real-time testing stage.

Extrapolative Continuous-time Bayesian Neural Network for Fast Training-free -ree Test-time Adaptation
Hengguan Huang! Xiangming Gu' Hao Wang* National University of Singapore
Chang Xiao Rutgers University
Hongfu Liu' Ye Wang""
Abstract
Human intelligence has shown remarkably lower latency and higher precision than most Al systems when processing non-stationary streaming data in real-time. Nu- merous neuroscience studies suggest that such abilities may be driven by internal predictive modeling. In this paper, we explore the possibility of introducing such a mechanism unsupervised domain adaptation (UDA) for handling non-stationary streaming data for real-time streaming applications. We propose to formulate in- ternal predictive modeling as continuous- Bayesian filtering problem within a stochastic dynamical system context. Such dynamical system describes the dynamics of model parameters of a UDA model evolving with non-stationary streaming data. Building such dynamical system, then develop extrap- olative continuous-tinn Bayesian neural networks (ECBNN D. which generalize existing Bayesian neural networks represent temporal dynamics and allow to extrapolate the distribution of model parameters before observing the incoming data, therefore effectively reducing the latency. Remarkably, empirical results show that ECBNN is capable of continuously generating better distributions of model parameters along the time axis given historical data only, thereby achieving (1) training-fres test-time adaptation with low latency, (2) gradually improved alignment between the source and target features and (3) gradually improved model performance over time during the real-time testing stage.
Introduction
Human consciousness requires building an internal predictive model that describes the external world 212. Numerous neuroscience studies have suggested that humans continually update their internal predictive model to represent the statistics of the fluctuating environments accurately K 5. 6.
Internal predictive modeling is crucial for humans to perceive a continuous stream of sensory inputs (e.g. sound and sight) in real-time, due to constraints in order to catch a fast-moving object, the internal predictive model required to forecast its trajectory ahead to compensate for the response latency of neurons; in order to recognize words from lip movements at various view angles in real-time, again, the internal predictive model is needed to ensure that decision outcome is not delayed [8].
neural transmission delays 77] For example,
Such mechanism allows humans to process non-stationary streaming sensory data with remarkably low latency and high precision by continuously and instantly generating internal predictive models. In contrast, most existing domain adaptation algorithms either require collection of additional data from a large amount of increasingly more domains for training 19, or the use of testing streaming data for test-time fine-tuning [10] 12 [3] which is impractical for real-time streaming applications.
*Correspondence to: <wangyel comp. nus edu. sg> "Our code will soon he available online:
Hengguan
Huang
whuang..engguse hww.uuanug.gggggu.mommmmmmm mdduuu Ye
Wang
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

This leads us to rethink why existing approaches in unsupervised domain adaptation (UDA) fail to achieve what humans are capable of when handling streaming data.
This work aims to introduce "internal predictive modeling" to augment traditional UDA algorithms in handling non-stationary streaming data. A UDA, augmented with such mechanism, should be able to (1) instantly extrapolate the model parameters throughout the real-time testing stage instead of simply adapting the model with unacceptable latency and (2) align features from the source stream and the target stream globally rather than local data streams. However, the development of a mathematical framework that can encode such mechanism involves several challenges: (I) real-world non- -ta-ttaiionary streaming data usually are highly uncertain in their temporal dynamics and therefore require us to design model that can encode uncertainty of the temporal dynamics and produce multimodal distributions of model parameters. (2) Only partial observation of the time series or the local streams available, resulting poor alignment quality.
To tackle the first challenge, we propose formulating internal predictive modeling as a continuous- time Bayesian filtering problem within a stochastic dynamical system context. Such system describes the dynamics of UDA's model parameters that are evolving with non-stationary streaming data. To our knowledge, such Bayesian treatment of time-evolving neural network (NN) parameters is mostly unexplored. Existing continuous-time Bayesian filtering approaches, e.g. CTBN [14), assume discrete-state for the dynamical system: and thus cannot scale to NNs. Furthermore, most of the Bayesian neural networks [15] [6] are designed to reason about static distributions of NN parameters. In this paper, we propose an extrapolative continuous-time Bayesian neural network (ECBNN), which generalizes existing static BNNs to encode uncertainty of the temporal dynamics and allows inferring multi-step ahead model distributions before observing the incoming data, therefore effectively reducing the latency. Furthermore, ECBNN adopts weighted set of particles to represent the posterior distribution, which can represent multimodal distributions with arbitrary shapes using a sufficient number of particles.
To tackle the second challenge, we propose to perform UDA on the entire data generation mechanism, which is described by latent stochastic process conditioned on historical data. This leads to another challenge: aligning two latent stochastic processes tends to result in an intractable diseriminator loss. We further derive an analytical upper bound for the discriminator loss. Calculating such a bound requires evaluating posterior distributions of the UDA encoder continuous time; this then leads us to reformulate the problem by deriving a new particle filter differential equation (PFDE) rather than simply applying traditional Bayesian filtering methods, such as particle filter (PF) [7]. It is worth noting that we adopt efficient implementation for solving the PFDE. Specifically, we adopt surrogate neural networks to approximate the solution. Therefore, the time-evolving model distributions can be efficiently inferred by a single forward pass of the associated surrogate neural networks.
We list our major contributions as follows:
(i) To the best of our knowledge, this work is the first to explore how to incorporate the mechanism of "internal predictive modeling" into traditional UDA algorithms in handling non-stationary streaming data for real-time applications with low latency requirements.
(ii) We present the first Bayesian treatment of time-evolving NN parameters for UDA with streaming data and propose extrapolative continuous-time Bayesian neural network (ECBNN).
(iii) We derive a novel anticle-fiiter--asee differential equation, thereby providing an analytical upper bound for ECBNN achieve eemporal-domain-nnarriie representation learning.
(iv) We empirically demonstrate that our approach is able to achieve (1) training-free test-time adaptation with low latency, (2) gradually improved alignment between the source and target features and (3) gradually improved model performance over time during the real-time testing stage.
2 Related Work
Brain -rain-informed Artificial Intelligence. In recent years, artificial intelligence (AI) has seen tremendous advances and breakthroughs in building intelligent systems that display human perceptual and cognitive abilities, including dialogue system [18), automatic speech recognition [19 20 and

---

high-fidelity image generation [1], among others. However, mainstream Al systems have not yet reached the capabilities of highly sophisticated bio-intelligence. Human brains are the only evidence that general intelligence at the human level is possible. Brain-inspired Al (22), aiming to bridge the gap between artificial intelligence and brain science, play a vital role opening up the field of artificial neural network and continuing to provide the foundation for research on deep learning
and reinforcement learning [23]. However, such an Al framework only takes modern theories in brain science rough guidance for developing new AI algorithms, and it mostly ignores biological plausibility. 124 [25] proposed a brain-informed AI framework, which consists of a task-specific
component for achieving variety of AI tasks, and brain-informed component for encoding specific brain science mechanism. For instance, deep graph random process (DGP) (24] incorporates relational thinking, an assumption of cognitive neuroscience, to conversational AI, which enables the learning of the relational structure from the conversations without requiring any relational labels during training; stochastic boundary ordinary differential equation (STRODE) (25 links the time perception and postdiction mechanism with generative Al and conversational Al respectively, allowing directly learning event-times from time-series and creating neurally plausible model whose outputs match the postdiction assumption. This work can also be classified as a brain-informed AI, where the internal predictive modeling is integrated into unsupervised domain adaptation, enabling the model to learn continuously and immediately from the non-stationary streaming data during real-time testing.
Unsupervised Domain Adaptation with Streaming Data. There is vast body of prior work [6 (7) (2 29 BI exploring
representation learning for unsupervised domain adaptation (UDA). These works' key idea is harnessing the expressive power of neural networks to transform the input data into latent representations that are further aligned across static domains. However, in real-time streaming applications, streaming data usually follows statioatio distribu- tions. Consequently, these work and a few other test-time adaptation methods [22 32 13] mostly fail
to handle such data due to the assumption of static domains or the requirements of low latency.
The UDA for non-stationary streaming data has been studied before but in different problem settings with different focuses. For example, continuous manifold adaption (CMA) [9] addresses this problem by adapting the model with the additional unlabeled data from several pre-defined intermediate domains; predictive domain adaptations [33 [0] attempt to solve specific scenario where target data is unavailable but the metadata that characterizes the target domain is provided. Continuously indexed domain adaptation [0) addresses this problem by jointly adapting across continuously indexed domains with domain indices such as 'time" and other domain attributes. However, these methods cannot be directly applied to our problem settings due the unavailability of the meta-data, e.g- absolute time stamps of each data point. The closest work to ours is [I], in which an online test-time adaptation framework is introduced to handle streaming data with evolving domains. However, (II) requires using partial test data for test-time fine-tuning, greatly limiting the applicability to real-time streaming situations. In contrast, our ECBNN allows reasoning multi-step ahead posterior distribution of UDA's model parameters before observing the incoming data, thereby achieving training-free test-time adaptation with much lower latency.
Bayesian Neural Networks with Dynamical Systems. Bayesian neural networks (BNNs) 33
aim to infer the posterior distribution of the NN parameters given observations and prior imposed on these parameters. With a different aim, the early work on neural dynamical systems, e.g. neural ordinary differential equation (ODE) (35], is aimed approximating the dynamics of layer-wise representations of deep NNs by using a non-Bayesian NN. Bayesian Neural ODEs (36] 37 seek to improve neural ODE by replacing the non-Bayesian NN with a BNN. Unlike the above studies, our work aims to develop a generalized BNN that captures the time-evolving posterior distribution of the NN parameters over time. A different strand of research focuses on building a continuous-deple BNN using stochastic differential equations [15]. Although it is close to our work, their model assumes a static distribution for NN parameters; and thus cannot scale to capture the time-varying distribution of NN parameters. In general, our work belongs to the category of Bayesian deep learning
(BDL) 33 39 24 25 40, and is the first BDL method that handles time-evolving NN parameters. (Please refer to Appendix for related work on particle filtering with neural networks.)
3 Method
In this section, we first formalize the problem of test-time streaming domain adaptation for non- stationary streaming data, and then describe our methods for addressing the problem.
3

---

3.1 Problem Setting: Test-time Streaming Domain Adaptation
Problem. We consider the unsupervised test-time domain adaptation setting for non con-ttatonnary
streaming data. Specifically, given source stream {B; target stream {B; H- and a testing
stream (B: where denotes the number of data batches; the source data batches B =
(x) (1 are labeled, while the target and testing data batches B = {xf 1x-1+---11 (i----)
B = (xx ( (- are unlabeled, where N, the number of frames in each batch, is usually small for machine learning applications that require streaming inputs, e.g. streaming video processing systems. We first train UDA model based on the source and target streams {B: and [B:) with the trained model, we aim to perform adaptation as each testing stream data batch B arrives.
Low Latency and Timeline. Notably, during the real-time testing stage, our problem setting has strict requirements for the latency caused by adaptation. Therefore, existing offline adaptation methods or online adaptation methods that require training fine-tuning are not applicable our problem setting. Furthermore, the source stream, target stream, and testing stream may not necessarily share the same timeline, and the absolute time stamps of the data are unknown; thus, DA methods [30] that require absolute time stamps fail to handle our problem setting as well.
3.2 Overview
Below we provide an overview of our method (Fig. D
Training. Our approach adopts both source stream and target stream data for training: (i) it starts by adopting PFDE, taking in- put the historical streaming batches B1:-11 infer the posterior distributions of NN parameters of the encoder fe; based on such distributions, particles are sampled to update the encoder fe (updated before processing B, ): (ii) further takes the updated encoder to generate 2, the encoder output for B,, and Σ, the encoder output for the approximated data required for calcu- lating the proposed analytical upper bound (see Sec 3.6 for more details of the approx- imated data ); (ii) finally, we follow the typical adversarial UDA training procedure to evaluate the training loss given z and ī and update the whole framework (see Sec. 3,5 for more detailed deseription).
ECBNN
B11-1
PFDE
Source
or Target
Sampling
De
Predicted
Labels
Figure 1: General framework of real-time streaming do- main adaptation; the components marked with dotted line are removed during testing.
Real-time Testing During the real-time testing stage, the testing stream cannot be revisited and should be processed as each data batch arrives. The testing procedure is mostly similar to the training, except that discriminator are removed.
3.3 Particle Filter Differential Equation (PFDE)
Traditional PF methods (see Appendix on Background of Particle Filtering), c.g. bootstrap filters [17], require calculating likelihood for observations at each time point, which is infeasible in our problem setting since only historical data is available when performing multi-step ahead Bayesian inference during the real-time testing stage. We therefore develop, by the following theorem, type of dynamical system, dubbed Particle Filter Differential Equation (PFDE). that describes the evolution of the state in an input-driven continuous-time stochastic dynamical system.
Theorem Suppose we are given a time series {20 Assume {xx) are conditionally independent given a sequence of latent states { which is itself assumed to be Markovian.
Assume marginal distribution of the state p(f(t)) is stationary. Let E [0, e), where is small positive
constant. Assume x, with E [a - e, + E) stationary. Let p(0(1) - T)) be the transition distribution. Let wx))()) be the importance weight. Then we have:
((og(())))) ((00)))))""((00((())0000000"""1" (100)(((1)010000

---

where (-Y and (-)" denote the first-order and second-order temporal derivatives respectively.
The derivation of PFDE in Theorem [ (see Appendix A for the proof) is based on the bootstrap filter. By assuming the marginal distribution of the state p(((1)) as stationary distribution (which is mild assumption as conditional distribution can still be non-stationary), we successfully build the link between the transition distribution and the importance weight, whose calculation longer requires calculating the intractable likelihood term.
3.4 Extrapolative Continuous-tina Bayesian Neural Networks (ECBNN)
At ECBNN's core is a PFDE, which provides a Bayesian treatment of time-evolving NN parameters and inherits several advantages from both particle filters and differential equations. For instance, PFDE enables inferring posteriors of NN parameters along the time axis by solving itself at any given time point.
Specifically, due to the intractable parametric form of the transition distribution p(0((0,-1) in our setting, we use an approximate distribution @(((t)) = (m((t),,s()) as the importance distribution, where mtt) and s(t) are the mean and variance of the Gaussian. Let us denote w(t) as the importance weight and ((t)() as the particle. By continuous-time Markovian assumption and Theorem I the PFDE can be written as:
f(m(t), m' (t), m mmmmm"(())))) ((), (())),)(())))))))))) ((1)()) (log(w((1)(0))"
where f()) is nonlinear function, whose detailed form is described in Appendix G It follows that the solution of PFDE can be written as:
G(t) = (m st), (w(t)
(2)
Then ECBNN takes the following form to approximate the posterior of the NN parameters ((O())):
((((t)) ("00)===0000000000000 w(t))))((()))))) 0(1)()), 0()) 7((()) ((m()),s(()),
where 8 denotes the Dirac-delta function. We further demonstrate the inference and learning of ECBNN by applying it to address the problem of test-time streaming DA.
3.5 Application of ECBNN for Test-time Streaming Domain Adaptation
This section demonstrates how embed ECBNN into a typical UDA framework for test-time streaming DA.
Learning. UDA usually contains encoder fe to project the input to a domain-invariant space with the help of a discriminator fa, such that the target data can be accurately predicted by the predictor Jp- Denoting optimization with the value function V (fee fá, fp) as:
the parameters of the encoder fee UDA performs minimax
min). maxfa (f(((=))) -a
(4)
where Vp the expectation of the prediction loss (e.g., cross-entropy loss for classification tasks) over the source data batches; Xa is hyperparameter balancing both losses :V is the expectation of the discriminator loss over source and target data batches. For simplicity, we only emphasize the parameters of the encoder fe and omit other parameters.
We extend UDA with ECBNN handle test-time streaming DA by adopting ECBNN as the encoder of UDA. The architecture of our model is shown in Fig. [ Recall that ECBNN adopts an approximate distribution ((())) the importance distribution due the intractability of the transition distribution. This leads us to adopt variational inference to optimize our model, in which ((())) treated as the variational distribution of the transition distribution. Let us denote p(O(t)) as the prior and (-; 0, fp: fa) as the function inside the minimax optimization of 4 Since both terms in l, can treated as negative log- likeeioooooo, we directly combine their negation's expectation with KL terms as the evidence lower bound (ELBO), which can be written as:
4 Σ. JXN
Σ k 1 ((()))))()000))))
(-KL,((())))(
(5)
5

---

In the following subsections, we will demonstrate how to obtain the solution of PFDE and how we can jointly learn ECBNN and UDA under the constraints of the PFDE.
The key to reducing latency is to force the inference of variables involved in PFDE to depend historical batch data only. Specifically, suppose we given up to i 1-th batch B we aim to infer the encoder distribution for each frame of the incoming batch by solving the PFDE. However, existing ODE numerical solvers fail give the approximate solution as the initial or boundary values of the variables are computationally intractable. Inspired by [25] 41,, we adopt surrogate neural networks to approximate the solution of these variables, such that:
Inference.
(6) where fe. are surrogate neural networks (implementations detail in Appendix G To encourage such solutions to satisfy the governing differential equation of PFDE defined Eq. we further include in the ELBO the following additional loss term lde:
m(t) = Som (t, BI:-1), s(t) fo, (I,B log w(() fow (1, (())),
de -E-, 1=1
IXN (f(m(t), ml ml(),)))))))))))) (1), m² m²),,, s(1), (1), 8 (1), (1og(w(()0)))pp (7)
To process B1.-11. fe. contains a recurrent neural network (RNN), which maintains hidden states to represent the historical batches without the need to repeatedly process the sequence from the start. Notably, our solution doesn't 't require absolute time stamps, e.g. to solve for the encoder distribution for the incoming batch at the first frame, neural networks as a non-linear function of time t, allowing all temporal derivatives required by Eq. (7) to be computed by the chain rule and evaluated by means of automatic differentiation 42]
set = 1. Furthermore, we specifically design these
Prior. We use NNs to parameterize the mean mI( and variance (D( of the prior p(0(1)):
(t fm (B.), st) fe.. (B.),
where fo. are neural networks (implementations detail in Appendix G Recall that we force the posterior to depend historical data batches only. To encourage the posterior extrapolate the future, we take a further step and adopt the current batch to estimate the prior for "future parameters". It's worth noting that this will not affect the latency as the prior will be removed during testing.
3.6 Toward Temporal soteeralddommiiinvariiaaa for Streaming Data
Due to the drifting nature of the non-stationary streaming data, aligning features extracted across the source and target streams is critical yet challenging for the success of unsupervised domain adaptation (UDA). Simply aligning using partial observation of the time series leads to poor aligmment quality. To address this challenge, we proposed to perform UDA on the entire data generation mechanism. Similar to particle filters, PFDE is also capable of capturing the latent iime-varying distribution of the non- stationary streaming data. Therefore, the solution of PFDE G(t) = (mt)), st), {wot) (f 1 can be treated stochastic process that describes the generation mechanism of the non-stationary data. We, therefore, give a temporal-domen invariant loss that is the expectation of the discriminator loss Va over the stochastic process G(t):
4 4-
E &
w(t) Va(fa
However, the upper limit of integration in the above loss function approaching infinity results in an improper integral, which may not converge. To overcome the above challenges, we propose the following theorem, which provides an analytical upper bound for the temporal sooerall-ommi--iiii invariant loss.
Theorem 2. Suppose are given an arbitrary function, π (t) with t [0, +oo). Let B fa/ Let e be a positive real constant and let h. [-1, + R be continuous function. There exists an H. |- 1, 0) + [0, +00) that satisfies an initial value problem:
H1 = h((3), H(-1) = 0, where h(3)
a
t TTy log(- 3))
23 28 log-log(--0 (og(-9)
Such that as + 0, we have:
(t))dt < lim lim(H((e)) (-(2cc) - H(-)1l)
c

---

CINSS
Torget
-100 +55
Frams
Figure 2: (a) Initial data samples at 0-th frame, where red dots and blue crosses are data samples from class O and class 1 respectively; Green sector separates the data circle into the source and target domain.. (b) Visualization of motion dynamics of decision boundary and data, where Black solid line represents the decision boundary, the red hash line represents data for class 0 and blue hash line for class 1, time ranges of source and target domain are indicated by different colors. (c) The accuracy of different methods by varying the length of historical stream data Growing Circle testing set.
Let (L) be the integrand of the right-hand-side of the Eq. ( Theorem 2 (proof given in Appendix states that we can construct [-1,0) 0) + [0, +o0) that satisfies an initial value problem (IVP):
H' (8) h()),, H(-1) =0
where (33 can be written as:
h(3) 28V
Σ
(10)
logl B)
where o- -- ( is the approximated data. Note that we use Z to denote the encoder output for the approximated data. Then by Theorem |
the upper bound of the lemporal-domain-
invariant loss, 4, can be written as:
4 < 1imm c0
lim lim(H((e))) (| (6)) - H(-))l),
(11)
where (-cc) and (-2c) can be obtained by solving the IVP. For instance, these two terms can be approximated using the ODE numerical solvers such as the Euler method and the Runge- Kutta methods [35]. In this work, we adopt the Euler method to calculate these two terms, in which is set the step size of the Euler method. Notably, by setting suitable step size, x()), where E (0, N) required calculate the bound; and thus (Ø) can approximated by simply using linear interpolation over the hidden representation of the current batch. In summary, the final loss of our model can be written as a minimax optimization problem:
min maxfa (- Io(:; S0 Sps fa) )actael; So) XeTe(;
(12)
where Ade and X; are importance weights; fa (so. S0,, Sow and So. Wo Soot. (See Appendix for the detailed form training loss.)
Experiments
We evaluate ECBNN on both synthetic and real-world streaming data. Specifically, we conduct preliminary experiments on low dimensional synthetic toy data (Growing Circle), and then study the generalization ECBNN in handling high dimensional synthetic video dataset based on handwritten digits (Streaming Rotating MNIST->USPP We finally evaluate our model on Oulu realistic multi-view lip reading dataset. (See Appendix G for the detailed data configuration of datasets)
4.1 Baselines
We compare ECBNN with state-of-the-art UDA methods including DANN 27
ADDA 28 CIDA
30 and DSAN [2). Since these UDAs assume static domains, we extend them handle streaming data in our experiments by adopting recurrent neural network (RNN) in the encoder. Since

---

1-50
1-60
EDA
ADDA
ECBNN
Figure 3: Results on the Growing Circle testing set. Black lines represent the decision boundaries generated according model predictions.
absolute time stamps unknown, extend CIDA by using both the frame index and categorical domain label as the input to the encoder. Notice that all the above methods can only perform offline adaptation, while ECBNN can perform online test-time adaptation by continuously producing encoder distributions. We also compare our model with a typical online UDA method, (EDA) [T]] which able perform online test-time adaptation but requires collecting data batches for fine-tuning. For fair comparisons, all models have similar numbers of weight parameters, with a similar neural network architecture of encoder for predictor fy. and discriminator fa. Following (30], Ag chosen from {0.2, 0.5, 1.0, 2.0, 5.0} (see implementations details in Appendix G
4.2 Toy Dataset
Growing Circle. We conduct preliminary study on toy dataset, Growing Circle, for binary classification task; Growing Circle extends the Circle dataset in [30 to incorporate temporal changes. Fig. 2 shows the initial data samples at the O-th frame, in which green sector separates the data on circle into the source and target domain. Fig. (b) shows the motion dynamics of decision boundary and data, in which time ranges of the source and target domain are indicated by different colors. We include 0-th to 39-th frame to source domain and 50-th to 89-th to target domain.
ECBNN Can Capture Time-evolving Decision Boundaries. We visualize the predicted classifica- tion boundaries generated by different methods at frames t = 50 and t = 60, shown in Fig. [3 (more results in the Appendix HH. We can see that ECBNN almost captures the ground truth (GT) decision boundaries at different time stamps, while other methods perform poorly when handling data with time-varying distributions.
ECBNN Can Effectively 'Learn' from Testing data. To evaluate ECBNN's capability of continu- ously generating better distribution of model parameters from historical stream data real-time, we test our model by varying the length of the historical stream data ranging from 0 to 50 frames on the testing set. Fig. c) shows the results for ECBNN and baselines. ECBNN consistently outperforms baselines. Interestingly, ECBNN's performance grows increasingly better with the accumulation of observed history, demonstrating ECBNN's potential in continuously learning from streaming data during real-time testing.
4.3 Streaming Rotating
Streaming Rotating UNIST=UUSP We further evaluate ECBNN on high-dimensional streaming data. To do this, we construct synthetic video dataset based MNIST and USPS: Streaming Rotating MNIST-USSPS. The video sequences are generated from the rotating MNIST and USPS handwritten digits with sinusoidal angular velocity. We take the first 40 frames of the Streaming Rotationg MNIST video sequence as source data, whose rotating angles range from 0' to 80%; we take the 50-th to 90-th frames of the Streaming Rotationg USPS video sequence target data, whose rotating angles range from 110" to 190%; we also take the 90-th to 110-th frames of the Streaming Rotationg USPS video sequence construct additional Appendix G for the detailed data configuration)
(OOD) testing set. (See
ECBNN Can be Generalized to High-dimensional Data. We report the frame-level classi- fication accuracy on the source-testing set, target-testing set and the OOD-testing set from the source domain, the target domain and out-of-domain, respectively, as shown in Table [
8

---

We can see that Source-Only achieves an accuracy of 28.8%, which indicates this task is very challenging. ECBNN achieves the best accuracy of 60.9%. dramatically outperforms other methods, e.g. it outperforms the hest-performing baseline method (ADDA) by 8.9% absolute. For domain generalization, ECBNN outperforms the best-performing baselines by 4.2% absolute, demon- strating ECBNN's potential temporal-domain- invariant representation learning in the entire time- line.
Table I: Accuracy (% for various DA methods Streaming Rotating MNIST-+USPS Dataset. We report the ac- curacy the source domain, target domain and out-of-domain.
We further evaluate ECBNN's ability to continuously generate better distribution of model parameters from high-dimensional stream data in the target domain. We evaluate ECBNN and baselines on Rotating USPS testing set and report the accuracy at each frame Fig. Remarkably, while handling the real-time stream, ECBNN achieves gradually bet- ter performance over time. This demonstrates that ECBNN can continuously generate better encoder dis- tribution over time based on historical data, whereby it becomes better and better at handling the data from the target domain.
Method
MNIST Source
USPS
Target
OOD
Source- Only
97.8
28.8
24.9
DANN 27
97.7
45.9
33.0
ADDA
97.3
52.0
34.3
CIDA 30
97.5
46.5
31.1
DSAN 29
96.1
46.8
33.0
EDA [1
97.9
45.5
31.9
ECBNN (Ours)
97.1
60.9
38.5
Ablation Study. Our final loss has three parts, namely the ELBO, Is, PFDE loss, lde and the upper bound of the temporal- domain invariant loss, 4, To verify the effectiveness of each part, we conduct an ablation study the Streaming Rotating MNIST-UUSPS dataset. Specifically, we design two ablation experiments: (i) re- moving 4 from the final loss; (ii) removing both 1 and lde from the final loss. Table gives the frame-level classification accu- racy on the target-testing set and the out-of- domain-testing set (OOD-testing set), from the target domain and "out-of-domain" re- spectively. We see that by removing the upper bound of the temporal-domain- invariant loss, 4. the accuracy drops dra- matically for both the target domain and out-of-domain by 7.3% and 8.6%, respectively. The accuracy drop is more severe for "fut-of--dmmann.. verifying the effectiveness of the proposed upper bound of the remporal-domain-nneerl drops further for both target domain and out-of-domain after removing the PFDE loss, lde; this verifies the effectiveness of the PFDE loss.
60
Acturacy 196)
CIDA
DSAN
40
EDA
FCRNN
50
55
60
65
70
75
80
85
90
Frame
Figure 4: The frame-wise accuracy of different methods Streaming Rotating USPS testing set.
loss in generalizing to unseen streaming data. Moreover, the accuracy
Table 2: Ablation study results ( aaccuracy (%) on Streaming Rotating MNIST-USSP dataset
Method
Target Domain f
Out-of-Domain t
ECBNN (ours)
60.9
38.5
w/o 7, w/o 7, w/o late
53.6 (-7.3)
29.9 (-8.6)
43.2 (-17.5)
28.0 (-9.5)
4.4 Multi-veiw Lip Reading
OuluVS2.
OuluVS2 dataset was originally proposed by [43, which is, as far as we know,
only publicly available dataset that contains multi-view lip motion data. It includes lipreading data recorded by 52 speakers from different views. The train, valid and test sets include 5250, 750 and 1800 sequences, respectively. We adopt experiment settings similar to [4], e.g. the sequence-level

---

class label is voted by class labels of all frames. To formulate the streaming domain adaptation scenes, we consider the sequences from 0 view the source domain and sequences from [30% ,45%, 60" 909] as target domains. Besides, each data batch contains ten frames, and each frame is greyscale image of size 44 x 50.
ECBNN Can be Generalized to Real-world Streaming Data. Table 3 shows the sequence-level classification accuracy of the baseline models and our new ECBNN on the Test of OuluVS2. We can see that ECBNN performs the best among all models in terms of accuracy in both source and target domains. ECBNN outperforms EDA, the online adaptation baseline, by 5.0% absolute, while having remarkably lower latency in generating model parameters. We also report training time per epoch and the detailed accuracy of various DA methods on different views in Appendix G
Table
Run time (s) and Accu- racy ( (%) for various DA methods on OuluVS2 Dataset We report run time per batch necessary to generate NN pa- rameters and the sequence-level accu- racy at the target domain.
Method
Run time
Target
Source-Ome
46.0
DANN 27
71.5
ADDA 28
69.3
ECBNN Can Produce Gradually Improved Align- ment. The production of lip motion data is governed by highly nonlinear dynamics over time, e-g. the move- ments of various articulators. Therefore, aligning such data in terms of the highly nonlinear temporal domain dynamics is very challenging. Here, we aim to qualitatively evaluate whether ECBNN can produce encoder distributions that achieve temporal-domain -nain-iiinnn feature learning for real data. We randomly select data from 4 classes for both the source domain (0") and the target domain (30%), and use t-SNE to visualize the corresponding encoder output z at time = 9, 15, and 21. Fig. 5 shows the visualization. Surprisingly, the alignment quality increases over time, and features at f 21 align much better than features at l = 9. This suggests that ECBNN can gradually improve the alignment given sequentially arriving data streams, thereby gradually improving accuracy on the target streaming data over time.
CIDA 30
63.0
DSAN 29
66.4
EDA 11
0.23
70.3
ECBNN (Ours)
0.01
75.3
Conclusion
This paper extends the brain-informed artificial intelligence family with in- ternal predictive modeling. Building on the connection between Bayesian neural networks and continuous-time stochastic dynamical system, we de- rive novel particle-filter bisteer-based differ- ential equation, thereby proposing ex- trapolative continuous-time Bayesian neural network (ECBNN), which gen- eralizes existing Bayesian neural net- works to represent temporal dynamics. We then augment the unsupervised do- main adaptation framework by adopting ECBNN its encoder for handling non-stationary streaming data. We further provide an upper bound with a theoretical guarantee for ECBNN to achieve temporal-dome waweraa--dmmi--inviraaaa representation learning. We show that ECBNN gradually improves model performance over time with low latency during the real-time testing stage. Our current approach assumes continuous dynamics of NN parameters; leave for future work be able learn more complex discrete dynamics introduced by discrete events.
I+
br15
1-21
Figure 5: t-SNE visualizations of features at source domain (in light colors) and target domain (in dark colors) sampled at time 9, 15, and 21. We use different colors mark different classes.
Acknowledgments
The authors would like to thank the anonymous reviewers for their insightful comments and sugges- tions and Dr. Qin Ling for her assistance in proofreading the initial manuscript. This project was partially funded by research grant R-255-000--B8--11 HW is partially supported by NSF Grant IIS 127918.
10

---

References
[1] Michael SA Graziano and Sabine Kastner. Human consciousness and its relationship to social neuroscience: a novel hypothesis. Cognitive neuroscience, 2(2):98-113, 2011.
[2] Artur Luezak and Yoshimasa Kubo. Predictive Neuronal Adaptation as Basis for Conscious- ness. Frontiers in Systems Neuroscience, 15, 2022.
[3] Karl Friston. The free-energy principle: unified brain theory? Nature reviews neuroscience, 11(2):27--33,, 2010.
[4] Jill X O'reilly. Frontiers in neuroscience, 7:105, 2013.
Making predictions in changing world inference, uncertainty, and learning.
[5] Jill O'Reilly, Urs Schüffelgen, Steven Cuell, Timothy EJ Behrens, Rogier B Mars, and Matthew FS Rushworth. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proceedings of the National Academy of Sciences, 11038))33600EE366) 2013.
[6] Ezgi Kayhan, Marlene Meyer, Jill O' O'Reilly, Sabine Hunnius, and Harold Bekkering. Nine- month-old infants update their predictive models of a changing environment, Developmental cognitive neuroscience, 38: 100680, 2019.
[7] Hinze Hogendoorn. Perception in real-time: predicting the present, reconstructing the past. Trends in Cognitive Sciences, (282):228-141 2022.
[8] Prem C Pandey, Hans Kunov, and Sharon M Abel. Disruptive effects of auditory signal delay on speech perception with lipreading. Journal of Auditory Research, 1986.
[9] Judy Hoffman, Trevor Darrell, and Kate Saenko. Continuous manifold based adaptation for evolving visual domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 867-874, 2014.
[10] Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Adagraph: Uni- fying predictive and continuous domain adaptation through graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6568-6577, 2019.
[11] Hong Liu, Mingsheng Long, Jianmin Wang, and Yu Wang. Learning to adapt to evolving domains. In NeurlPS, 2020.
[12] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno A Olshausen, and Trevor Darrell. Fully test-time adaptation by entropy minimization. 2020.
[13] Aurick Zhou and Sergey Levine. Training on test data with bayesian adaptation for covariate shift. arXiv preprint arXiv:2109.12744 2021.
[14] Uri Nodelman, Christian R Shelton, and Daphne Koller. Continuous time bayesian networks. arXiv preprint arXiv: I 000000591,100 2012.
[15] Winnie Xu, Ricky TQ Chen, Xuechen Li, and David Duvenaud. Infinitely deep bayesian neural networks with stochastic differential equations. In International Conference on Artificial Intelligence and Statisties, pages 721-738. PMLR, 2022.
[16] David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, and Aaron 2017.
Courville. Bayesian hyperntworks.. arXiv preprint
[17] Neil J Gordon, David Salmond, and Adrian FM Smith. Novel approach to nonlinear/non- gaussian bayesian state estimation. In IEE Proceedings F-radar and signal processing, volume 140, pages 107-113, IET, 1993.
[18] Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. A survey on dialogue systems: Recent advances and new frontiers. Acm Sigkdd Explorations Newsletter, 19(2):25-35, 2017.
11

---

[19] Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6):82-97, 2012.
[20] Hengguan Huang, Hao Wang, and Brian Mak. Recurrent poisson process unit for speech recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6538, 2019.
[21] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint 1XXi:20011.13450 2020.
[22] Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. Neuroscience-inspired artificial intelligence. Neuron, 95(2):245-258, 2017.
[23] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press. 2018.
[24] Hengguan Huang, Fuzhao Xue, Hao Wang. and Ye Wang. Deep graph random process for relational wawpoonal-tiiiiin---aaaaaae speech recognition. In ICML, 2020.
[25] Hengguan Huang, Hongfu Liu, Hao Wang, Chang Xiao, and Ye Wang. Strode: Stochastic boundary ordinary differential equation. In International Conference on Machine Learning, pages 4435 4445. PMLR, 2021.
[26] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International conference machine learning, pages 97-105. PMLR, 2015.
[27] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franccois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):009-20300, 2016.
[28] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167-7176, 2017.
[29] Petar Stojanov, Zijian Li, Mingming Gong, Ruichu Cai, Jaime Carbonell, and Kun Zhang. Domain adaptation with invariant representation learning: What transformations to learn? Advances Neural Information Processing Systems, 34, 2021.
[30] Hao Wang. Hao He, and Dina Katabi. Continuously indexed domain adaptation. arXiv preprint arX1v:2007.01807 2020.
[31] Min-Hung Chen, Zsolt Kira, Ghassan AlRegib, Jackwon Yoo, Ruxin Chen, and Jian Zheng. Temporal attentive alignment for large-scale video domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6321-6330, 2019.
[32] Viraj Prabhu, Shivam Khare, Deeksha Kartik, and Judy Hoffman. Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 8558-8588 2021.
[33] Yongxin Yang and Timothy M Hospedales. Multivariate regression on the grassmannian for predicting novel domains. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5071-5088 2016.
[34] Jouko Lampinen and Aki Vehtari. Bayesian approach for neural networks-reviee and case studies. Neural networks, 14(3):257-274, 2001.
[35] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
12

---

[36] Cagatay Yildiz, Markus Heinonen, and Harri Lahdesmaki. Ode2vae: Deep generative second order odes with bayesian neural networks. Advances in Neural Information Processing Systems, 32, 2019.
[37] Raj Dandekar, Karen Chung, Vaibhav Dixit, Mohamed Tarek, Aslan Valade,,, Kr- ishna Vishal Vemula, and Chris Rackauckas. Bayesian neural ordinary differential equations. arXiv preprint arXiv:20/2.072244 2020.
[38] Hao Wang and Dit-Yat Yeung. Towards bayesian deep learning: A framework and some existing methods. TDKE, 28( 2812333395-3408 2016.
[39] Hao Wang and Yan Yeung. A survey on bayesian deep learning. CSUR, 53(5):1-37,
[40] Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International Conference on Machine Learning, pages 1704-1713. PMLR, 2018.
[41] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics- informed neural networks: deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
[42] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic diffcrentiation in pytorch. 2017.
[43] Iryna Anina, Ziheng Zhou, Guoying Zhao, and Matti Pietikäinen. Ouluvs2: A multi-view audiovisual database for non-rigid mouth motion analysis. In 2015 11th IEEE International Conference and Workshops Automatic Face and Gesture Recognition (FG), volume 1, pages 1-5. IEEE, 2015.
[44] Stavros Petridis, Yujiang Wang, Zuwei Li, and Maja Pantic. End-to-end multi-view lipreading. In British Machine Vision Conference, BMVC 2017, 2017.
[45] Hengguan Huang, Hongfu Liu, Hao Wang, Chang Xiao, and Ye Wang. Strode: Stochastic boundary ordinary differential equation. In International Conference on Machine Learning. pages 4435 4445. PMLR, 2021.
[46] Fernando Caballero, Luis Merino, Iván Maza, and Aníbal Ollero. A particle filtering method for wireless sensor network localization with an acrial robot beacon. In 2008 IEEE International Conference on Robotics and Automation, pages 596-601. IEEE, 2008
[47] Brenda Ng. Avi Pfeffer, and Richard Dearden. Continuous time particle filtering. In IJCAI, pages 1360-1366 Citeseer, 2005.
[48] Peter Karkus, David Hsu, and Wee Sun Lee, Particle filter networks with application visual localization. In Conference on robot learming, pages 169-178. PMLR, 2018.
[49] Petar M Djuric, Jayesh H Kotecha, Jianqui Zhang, Yufei Huang, Tadesse Ghirmai, Mónica F Bugallo, and Joaquin Miguez. Particle filtering. IEEE signal processing magazine, 19-5)1 2003.
[50] Jean-Mare Valin, Franecois Michaud, and Jean Rouat. Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering. Robotics and Autonomous Systems, 55(3):216-228, 2007.
[51] Authony E Brockwell, Alex L Rojas, and Robert Kass. Recursive bayesian decoding of motor cortical signals by particle filtering. Journal of neurophysiology, 1199::199-1997, 2004.
[52] Kyung-Sik Choi, Jac-Won Lee, and Suk-Gyu Lec. Navigation of mobile robot using reduced particles based on incorporating particle filter with neural networks. In 2010 International Conference Intelligent Control and Information Processing, pages 128-133. IEEE, 2010.
[53] Xiao Ma, Peter Karkus, David Hsu, and Wee Sun Lee. Partiele filter recurrent neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5101-5108, 2020.
13

---

[54] Fred Daum, Jim Huang, and Arjang Noushin. Exact particle flow for nonlinear filters. In Signal processing, sensor fusion, and target recognition XIX, volume 7697, page 769704, International society for optics and photonics, 2010.
[55] Xinshi Chen, Hanjun Dai, and Le Song. Particle flow bayes' rule. In International Conference on Machine Learning, pages 1022-10314 PMLR, 2019.
[56] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. Iclr, 2(5):6, 2017.
[57] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (6(11))2278-22) 1998.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] The Supplement contains detailed instructions to reproduce the main experimental results and the code
will be released when the paper is accepted.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please more details in the Supplement.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] Please see more details in the Supplement.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Please see more details in the Supplement.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you' 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "l5Rh5RVso0O", "reviewer": "Reviewer_dicg", "summary": "The paper describes a low-latency unsupervised domain adaptation (UDA) framework for non-stationary streaming data. To this end, the method includes a Bayesian neural network, which produces a temporally evolving parameterization to the feature extractor, acting as a meta network with temporally changing output. It includes furthermore a discriminator, allowing adversarial domain adaptation training. Thirdly a predictive module is fed by the invariant representation. The Bayesian inference is tackled via Variational Inference relying on a continuous time particle filtering idea.\n", "questions": "Please clarify what is happening during training. Figure 1 suggests that two embedding is generated $\\tilde{z}$ and $z$. Two interpretation is possible, either $z$ corresponding to the source domain, and $\\tilde{z}$  to the target domain. Therefore both go to the discriminator and only $z$ goes to the predictor.\nOther interpretation would be that for both domain, two embeddings are generated, resulting in $z^s$, $z^a$, $\\tilde{z}^s$ and $\\tilde{z}^a$. The statement in Line 135-136 calling $\\tilde{z}$ as \"the encoder output for the approximated data\" not helps this confusion. What is \"approximate data\"?\n\nIs $l_u(\\cdot ; \\theta)$ itself an extreme value (as described by Equation 4), which later enter to the loss, describing a multilevel optimization problem? Or $l_u(\\cdot ; \\theta)$  supposed to be the function inside the saddle point optimization? (I assume the second is true. If so please fix the equation)\n\nEquation 5 have two more opening brackets than closing brackets, it is possible to guess what this equation was intended to be, but please fix this. (Eq. 6 also missing a closing bracket)\n\nPlease give a more detailed motivation for using $l_i$ temporal-domain-invariance loss besides the $V_d$ term in $l_u$. Even better would be to execute an ablation study.\n\n\n\n\n\n", "limitations": "The authors describe that their method is only capable to work with continuously evolving systems at the moment, and they mention handling discrete dynamics as a future work.\n\nIt is not entirely clear how large datasets can be handled. The method is quite complex. The evaluation time computational complexity seems quite favorable (Table 2). The training time complexity can be interesting as well.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Domain adaption  on non-stationary stream in real time is unquestionably a problem with both theoretical and practical significance.\nThe described framework clearly novel and the description of the method is clear.  Despite the clarity of the description the paper is quite dense, as the method is quite complex. It takes 5 or 6 equations to describe its loss function alone. This level of complexity requires further justification. Ablation studies are entirely missing from the paper. The question inevitably appears in the reader, does this complexity really necessary?\n\nIt is not entirely clear why Bayesian approach is necessary. It is easy to imagine that it helps, it is just far from being trivial that it does. What are the weak points of an evolving point estimate approach versus an evolving Bayesian approach for estimating the encoder parameters?\n\nThe problem setup could be described with a bit more clarity, as it is not always entirely clear what is the main difference between the source and target domain. In the growing circles dataset for example, Fig 2A shows that the source corresponds to a green wedge in the input space, and the target is the other part. Fig 2B shows, that the Source and the Target is temporally separated (0 -39th frame and 50-89th frames respectively). Is it both?\n\nL210-212 describes an unusual solution, the prior is used more like a target, where the KL term in the variational lower bound is used at training time to force the estimated parameters to follow this target. This use seems against the meaning of prior in Bayesian framework.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "d_-hjjFh5kl", "reviewer": "Reviewer_jpUT", "summary": "The paper proposes an approach to unsupervised domain adaptation by formulate predictive modeling as a continuous-time Bayesian filtering problem. The works introduces extrapolative continuous-time bayesian neural networks with a particle filtering differential equation at its core that only rely on historical data for inference and models the change of importance weights. The proposed model is trained in a framework that combines the minmax optimization of unsupervised domain adaptation and the ELBO of VAE. To  apply the model to non-stationary streaming data, an additional temporal domain invariant loss term was used to encourage the model to generalize to unseen data. Experiment results show strong performance of the proposed ECBNN against other unsupervised domain adaptation models.", "questions": "An important component of many discrete-time particle filters is to resample the particles to remove particles of low weights based on effective sample size. Will the proposed filtering approach based on PFDE suffer from the similar problem where the weights are concentrated on a few particles?", "limitations": "The author address the limitations of the work but not discusses negative social impact. But I think it might not be required for this work which is a primarily methodology paper.", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \nTreating the time-evolving NN parameters as a continuous-time stochastic dynamic system and extending bayesian neural networks to continuous-time domains is a novel approach to unsupervised domain adaptation. I also appreciate the proposal of PFDE. It is an original contribution to the reviewer’s best knowledge with strong motivation for efficient multi-step ahead inference. The experiment settings considered by the paper are comprehensive and diverse\n\nWeakness:\nA minor concern I have about the paper is the organization of the method section. The traditional particle filtering method introduced in the background section is not part of the original contribution of the work and may be better incorporated in the related works section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "btlfBJGeWB", "reviewer": "Reviewer_ZNi1", "summary": "This paper proposes an adaptive unsupervised domain adaptation based on a variational particle filter Bayesian posterior on an encoder, which is then used to form predictions on a non-stationary (time changing) task. The particle filter is run with respect to a continuous-time differential equation and some theoretical results are proposed to associate the particle weights with the parameter-space transition distribution, used to form one of the terms of the training objective. Some comparative experimental results are presented.", "questions": "- why do we need a differential equation to model the trajectory of theta? what is the advantage of this burdensome choice?\n- clarify what '' means in Thm 1 before it is used (2nd derivative wrt time, not one page later)\n- I do not understand the last line 1 of page 1 (supp. mat.) of the proof of Theorem 1: p(x_t) may be an invariant distribution function wrt t but it is not a constant wrt the *value* of x_t.\n- how does the proposed algorithm scale up wrt things like the number of parameters of the encoder, the number of training examples, and other relevant quantities? How about the computational cost of solving the differential equation? \n- in the context of line 120, I would like to say that offline adaptation is biologically plausible, since we know that consolidation from hippocampus to cortex is taking place during sleep and may be a form of offline adaptation; if this is good enough for animals, why not for AI tasks?\n- sec. 3.3 did not make sense to me, needs more work to clarify\n- eq 8: does it make sense to have the current batch B_i as an input to compute a Bayesian prior?\n- l. 213 I did not understand why this should be true\n\n", "limitations": "- computational costs are not analyzed\n- results do not compare against the family of neural processes (which have similar properties)\n", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n- this paper attempts to address many difficult questions, including an efficient prediction of the Bayesian parameter posterior, do it in a non-stationary environment (albeit assuming it is stationary which is weird), and doing it with neural networks and solving differential equations to capture the trajectory of the posterior\n- comparative results (although not against all the relevant baselines) seem impressive\n\nCons:\n\n- The overall system seems very complicated and I cannot say that I was able to get an overall satisfying grasp of it\n- I have some concerns about the proof of Theorem 1.\n- I have some concerns that the posterior is a unimodal Gaussian over parameters\n- in terms of related work and baselines, it seems that this paper has a blind spot wrt the family of conditional neural processes architectures (which also propose an efficient way to directly generate samples from the parameter posterior, conditioned on the past training data).\n- there is a bit of a contradiction between the UDA setup where new tasks can appear at any time and the assumption of stationarity of p(theta(t)) assumed for Thm 1.\n- a much more transparent analysis of different computational costs is needed\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept