Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes an adaptive unsupervised domain adaptation based on a variational particle filter Bayesian posterior on an encoder, which is then used to form predictions on a non-stationary (time changing) task. The particle filter is run with respect to a continuous-time differential equation and some theoretical results are proposed to associate the particle weights with the parameter-space transition distribution, used to form one of the terms of the training objective. Some comparative experimental results are presented.
strengths_and_weaknesses: Pros:
- this paper attempts to address many difficult questions, including an efficient prediction of the Bayesian parameter posterior, do it in a non-stationary environment (albeit assuming it is stationary which is weird), and doing it with neural networks and solving differential equations to capture the trajectory of the posterior
- comparative results (although not against all the relevant baselines) seem impressive

Cons:

- The overall system seems very complicated and I cannot say that I was able to get an overall satisfying grasp of it
- I have some concerns about the proof of Theorem 1.
- I have some concerns that the posterior is a unimodal Gaussian over parameters
- in terms of related work and baselines, it seems that this paper has a blind spot wrt the family of conditional neural processes architectures (which also propose an efficient way to directly generate samples from the parameter posterior, conditioned on the past training data).
- there is a bit of a contradiction between the UDA setup where new tasks can appear at any time and the assumption of stationarity of p(theta(t)) assumed for Thm 1.
- a much more transparent analysis of different computational costs is needed

questions: - why do we need a differential equation to model the trajectory of theta? what is the advantage of this burdensome choice?
- clarify what '' means in Thm 1 before it is used (2nd derivative wrt time, not one page later)
- I do not understand the last line 1 of page 1 (supp. mat.) of the proof of Theorem 1: p(x_t) may be an invariant distribution function wrt t but it is not a constant wrt the *value* of x_t.
- how does the proposed algorithm scale up wrt things like the number of parameters of the encoder, the number of training examples, and other relevant quantities? How about the computational cost of solving the differential equation? 
- in the context of line 120, I would like to say that offline adaptation is biologically plausible, since we know that consolidation from hippocampus to cortex is taking place during sleep and may be a form of offline adaptation; if this is good enough for animals, why not for AI tasks?
- sec. 3.3 did not make sense to me, needs more work to clarify
- eq 8: does it make sense to have the current batch B_i as an input to compute a Bayesian prior?
- l. 213 I did not understand why this should be true


limitations: - computational costs are not analyzed
- results do not compare against the family of neural processes (which have similar properties)
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes an approach to unsupervised domain adaptation by formulate predictive modeling as a continuous-time Bayesian filtering problem. The works introduces extrapolative continuous-time bayesian neural networks with a particle filtering differential equation at its core that only rely on historical data for inference and models the change of importance weights. The proposed model is trained in a framework that combines the minmax optimization of unsupervised domain adaptation and the ELBO of VAE. To  apply the model to non-stationary streaming data, an additional temporal domain invariant loss term was used to encourage the model to generalize to unseen data. Experiment results show strong performance of the proposed ECBNN against other unsupervised domain adaptation models.
strengths_and_weaknesses: Strength: 
Treating the time-evolving NN parameters as a continuous-time stochastic dynamic system and extending bayesian neural networks to continuous-time domains is a novel approach to unsupervised domain adaptation. I also appreciate the proposal of PFDE. It is an original contribution to the reviewer’s best knowledge with strong motivation for efficient multi-step ahead inference. The experiment settings considered by the paper are comprehensive and diverse

Weakness:
A minor concern I have about the paper is the organization of the method section. The traditional particle filtering method introduced in the background section is not part of the original contribution of the work and may be better incorporated in the related works section.
questions: An important component of many discrete-time particle filters is to resample the particles to remove particles of low weights based on effective sample size. Will the proposed filtering approach based on PFDE suffer from the similar problem where the weights are concentrated on a few particles?
limitations: The author address the limitations of the work but not discusses negative social impact. But I think it might not be required for this work which is a primarily methodology paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper describes a low-latency unsupervised domain adaptation (UDA) framework for non-stationary streaming data. To this end, the method includes a Bayesian neural network, which produces a temporally evolving parameterization to the feature extractor, acting as a meta network with temporally changing output. It includes furthermore a discriminator, allowing adversarial domain adaptation training. Thirdly a predictive module is fed by the invariant representation. The Bayesian inference is tackled via Variational Inference relying on a continuous time particle filtering idea.

strengths_and_weaknesses: Domain adaption  on non-stationary stream in real time is unquestionably a problem with both theoretical and practical significance.
The described framework clearly novel and the description of the method is clear.  Despite the clarity of the description the paper is quite dense, as the method is quite complex. It takes 5 or 6 equations to describe its loss function alone. This level of complexity requires further justification. Ablation studies are entirely missing from the paper. The question inevitably appears in the reader, does this complexity really necessary?

It is not entirely clear why Bayesian approach is necessary. It is easy to imagine that it helps, it is just far from being trivial that it does. What are the weak points of an evolving point estimate approach versus an evolving Bayesian approach for estimating the encoder parameters?

The problem setup could be described with a bit more clarity, as it is not always entirely clear what is the main difference between the source and target domain. In the growing circles dataset for example, Fig 2A shows that the source corresponds to a green wedge in the input space, and the target is the other part. Fig 2B shows, that the Source and the Target is temporally separated (0 -39th frame and 50-89th frames respectively). Is it both?

L210-212 describes an unusual solution, the prior is used more like a target, where the KL term in the variational lower bound is used at training time to force the estimated parameters to follow this target. This use seems against the meaning of prior in Bayesian framework.


questions: Please clarify what is happening during training. Figure 1 suggests that two embedding is generated $\tilde{z}$ and $z$. Two interpretation is possible, either $z$ corresponding to the source domain, and $\tilde{z}$  to the target domain. Therefore both go to the discriminator and only $z$ goes to the predictor.
Other interpretation would be that for both domain, two embeddings are generated, resulting in $z^s$, $z^a$, $\tilde{z}^s$ and $\tilde{z}^a$. The statement in Line 135-136 calling $\tilde{z}$ as "the encoder output for the approximated data" not helps this confusion. What is "approximate data"?

Is $l_u(\cdot ; \theta)$ itself an extreme value (as described by Equation 4), which later enter to the loss, describing a multilevel optimization problem? Or $l_u(\cdot ; \theta)$  supposed to be the function inside the saddle point optimization? (I assume the second is true. If so please fix the equation)

Equation 5 have two more opening brackets than closing brackets, it is possible to guess what this equation was intended to be, but please fix this. (Eq. 6 also missing a closing bracket)

Please give a more detailed motivation for using $l_i$ temporal-domain-invariance loss besides the $V_d$ term in $l_u$. Even better would be to execute an ablation study.






limitations: The authors describe that their method is only capable to work with continuously evolving systems at the moment, and they mention handling discrete dynamics as a future work.

It is not entirely clear how large datasets can be handled. The method is quite complex. The evaluation time computational complexity seems quite favorable (Table 2). The training time complexity can be interesting as well.
review_title: null

Meta-review: metareview: Paper combines the use of a particle filtering differential equation (newly proposed in this paper) for sampling posterior parameters of a bayesian neural network, with unsupervised domain adaptation, and achieves strong results on tasks demonstrated. Reviewers found the method novel and the problem important. Several questions were raised about the motivation of using Bayesian neural networks, about the utility of combining several of the pieces of the loss function proposed, about the computational cost, but the authors provided satisfactory answers to the questions and provided ablations showing the utility of the different components of the loss function. One downside pointed out was that the method is quite dense and involves several parts that are intertwined. I hope the authors will revise their submission, taking into account the points raised by the reviewers. 
confidence: Certain
Decision: Accept