Task: Language Identification
Input (Spanish, English): This paper presents a variational inference based method for learning nonlinear dynamical systems. Unlike the deep Kalman filter, the proposed method learns a state space model, which forces the latent state to maintain all of the information relevant to predictions, rather than leaving it implicit in the observations. Experiments show the proposed method is better able to learn meaningful representations of sequence data.  The proposed DVBF is well motivated, and for the most part the presentation is clear. The experiments show interesting results on illustrative toy examples. I think the contribution is interesting and potentially useful, so I’d recommend acceptance.  The SVAE method of Johnson et al. (2016) deserves more discussion than the two sentences devoted to it, since the method seems pretty closely related. Like the DVBF, the SVAE imposes a Markovianity assumption, and it is able to handle similar kinds of problems. From what I understand, the most important algorithmic difference is that the SVAE q network predicts potentials, whereas the DVBF q network predicts innovations. What are the tradeoffs between the two?  Section 2.2 says they do the latter in the interest of solving control-related tasks, but I’m not clear why this follows.   Is there a reason SVAEs don’t meet all the desiderata mentioned at the end of the Introduction?  Since the SVAE code is publicly available, one could probably compare against it in the experiments.   I’m a bit confused about the role of uncertainty about v. In principle, one could estimate the transition parameters by maximum likelihood (i.e. fitting a point estimate of v), but this isn’t what’s done. Instead, v is integrated out as part of the marginal likelihood, which I interpret as giving the flexibility to model different dynamics for different sequences. But if this is the case, then shouldn’t the q distribution for v depend on the data, rather than being data-independent as in Eqn. (9)?
Output (English): en