Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a method that utilizes NFs to construct a more general distribution over observations.
The paper provides two different training methods based on the EM algorithm or the co-occurence method.
The paper demonstrates the efficacy of both methods on synethetic and real datasets.
strengths_and_weaknesses: Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited

Nothing presented is particularly novel in its own right but, to my knowledge, the combination of ideas is novel and the paper explores the combinations thoroughly (though additional baselines would have been nice wrt the real datasets).


Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?

The paper appears to be technically sound and the results are consistent with the claims. Current limitations and directions for future work are (briefly) discussed. These directions seem reasonable as a separate work and I do not think they need to be included in this effort.


Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)

The paper is well-written and well-cited. I have no major complaints. There are a few minor typos (e.g., line 180 "...due *ti* the fact..."). The figures and synthetic experiments do an excellent job of demonstrating and clarify the ideas discussed in the text.


General comments:

Overall, I found this paper very accessible and the proposed method intuitive. The paper presents its ideas clearly and demonstrates them reasonably well. It would have been nice to see additionaly baselines on the real datasets and might have been nice to see higher dimension examples.
questions: See above
limitations: N/A
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper presents the FlowHMM, a novel combination of a hidden Markov models (HMMs) and flow based deep network architectures, along with two methods of training the model. The first method is based on expectation-maximization (EM), and the second is based on matching co-occurrence matrices computed from data. Experiments are performed comparing both methods with a Gaussian mixture model baseline.
strengths_and_weaknesses: The main ideas of the paper are clearly presented, and the details of the model along with two different training schemes are treated with care. The background on how continuous HMM models are trained in the past and their pros and cons are also made very clear to the reader. The experiment section provides comprehensive results on fitting several low-dimensional datasets, including a total of 6 different datasets/variants, including 2 real datasets and 2 synthetic ones.

Despite the aforementioned strengths, however, the paper has a couple of weaknesses that are hard to overlook. One major weakness is the lack of comparison with similar continuous non-linear HMM models (e.g. structured variational autoencoders with HMM dynamics). Only comparing with Gaussian mixture HMMs doesn't seem to be sufficient and renders the results somewhat weak in my opinion. 

Furthermore, the $\mathcal{F}^Q$ model, which seems to be the more novel of the two presented, yields unsatisfactory results in the majority of runs in examples 5 and 6 (table 3), which featured 2 dimensional datasets. This huge drop of performance is not addressed or explained in the paper. 

To me, it seems like the drop of performance for $\mathcal{F}^Q$ in 2D is a consequence of the discretization of the observation space that is central to the approach. As dimensionality increases the discretization is going to incur an exponential cost which seems highly problematic. This point is breifly addressed on line 208 but I believe it deserves more attention and perhaps experimental verification. 

Lastly, it is claimed that the EM training method is inefficient for long sequences, but the computational costs are not reported or analyzed. A suggestion would be to talk about these computational tradeoffs to give more information about the practical value of these algorithms.

To conclude, the paper is pretty well written and the model proposed is somewhat novel, but the experimental results are quite weak and the difficultly of fitting to high-dimensional data is an important problem that is missing from this work.
questions: Some typos I found:
- Line 142: I believe it should be $\sum_{v_i,v_j\in\mathcal{V}}$
- Line 180: "due to the fact"

Questions:
- In equation (14) you treat the matrices $Q_\beta$ and $\hat{Q}$ as probabilistic distributions. Given how these matrices are defined, is there a way to make this more rigorous (i.e. give a probabilistic interpretation of $Q_\beta$ and $\hat{Q}$ in their definition)?
- In equation (16) random perturbations are added to the grid values for the computation of $P_\beta$ to prevent overfitting. Is there a reason why the same is not done for $\hat{Q}$ in equation (15)?
- Have you tried discretization for higher dimensional data? Are the results competitive with mixture models/other flow-based models (Ghosh 2020)?
- Could you clarify what you mean by "as a black box" (line 248) and "discover the hidden states of the data unobserved during training" (line 250)? Are you suggesting that it is hard to do inference in the model by Ghosh et. al.?
limitations: As I mentioned in the strengths and weaknesses section, the main limitation I see with this work is the difficultly of learning in higher dimensions, which the authors do not address in the main paper.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces FlowHMM for modeling time-series data. The authors explain that HMMs can either be trained using EM, which is slow and prone to being in poor local minima, or using a co-ocurrence matrix in the case of discrete HMMs. Additionally, HMMs model the observations via simple, tractable distributions like Mixtures of Gaussians. Instead, FlowHMM models these observations using normalizing flows. 

The authors propose two techniques to train their model. The first is with the standard EM approach, and the second approach involves discretizing the continuous HMM into a discrete one, and then using a co-ocurrence matrix based approach.

strengths_and_weaknesses: The authors very carefully describe how they can exploit the structure of their FlowHMM model (flow-based emission probabilities) to easily get a good co-occurence matrix based approach for training their model. I feel that this should be highlighted at the end of the introduction.

questions: Can you provide some insight into how to decide on which training technique to use? (EM, or discretization and co-occurrence matrix). Specifically, can you show empirically what happens when you use co-occurrence matrix when “the assumption that the Markov chain on a set of hidden states is stationary - a limitation not present in the EM-based approaches” (line 330) does not hold. 

limitations: I do not see any potential negative societal impact.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose to use continuous normalizing flows to extend Hidden Markov Models (HMM) by modeling the conditional distribution of the observables given the hidden state with a flow. They call their method FlowHMM and present two training algorithms: First, they point out that the Expectation-Maximization (EM) algorithm can be used for this purpose, which is a common procedure training HMMs. Second, the authors suggest training their FlowHMM model by finding parameters to match the empirically computed co-occurrence matrix of two subsequent observations. They applied their method to several synthetic and real world sequential datasets and thereby demonstrate the effectiveness of the two training algorithms and outperform the baselines given by Gaussians or mixture of Gaussians instead of a continuous normalizing flow modeling the distribution of the observables.
strengths_and_weaknesses: ## Strengths

I appreciate that the article is well organized and easy to read. The authors have an illustrative figure of their procedure, i.e. Figure 1, which ameliorates the understandability of the presented material even further. They also clearly point out their contributions both in the abstract and in the main paper.

Moreover, in their experiment section, the authors consider a variety of datasets, including synthetic once and real-world examples.

## Weaknesses

My main criticism concerns the motivation of the method and its comparison to competing approaches.

The authors chose a continuous normalizing flow to model the conditional distribution of the observables given the latent variables, but this density could have been modeled by any model designed for density estimation, such as a discrete normalizing flow or an autoregressive model. They argue that a continuous normalizing flow is more expressive than a Gaussian or a mixture of Gaussians, which I agree with, but they give no reason why they did not choose another of the advanced density estimation models I mentioned.

Moreover, Ghosh et al. (2020) developed a very similar approach and made a follow-up ([Ghosh et al, 2021](https://arxiv.org/abs/2107.00730)), which is not even mentioned in the paper. They claim that Ghosh's model cannot be applied in the cases they are considering since Real NVP cannot deal with one-dimensional data; however, instead of Real NVP they could just use a Neural Spline Flow or a Residual Flow and made a comparison. This should be done in my point of view to highlight the strength and novelty of the presented method.

## Conclusion

Given that there are essential things missing in the paper as I argued above, I cannot vote for acceptance of this paper at the current stage. I ask the authors to clarify these points and are willing to update my score if they do so.

**Edit**

Since the authors addressed some of my concerns, I'll increase my score slightly.
questions: * Why did you choose continuous normalizing flows as a density estimation method in your model and not another approach such as a discrete normalizing flow or an autoregressive model?
limitations: The authors highlight in the final section of their paper that the method relies on the co-occurrence matrix for training and is not validated sufficiently on multidimensional time series data.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors study the problem of fitting Hidden Markov Models (HMM) when the continuous observation model is non-Gaussian. Precisely, they consider a family (one for each state of the latent variable) of continuous normalising flow models as the observation model. To jointly fit the transition probabilities of the latent states as well as the continuous flow models, the authors propose two methodologies:
* FlowHMM \mathcal{F}^{EM} : maximising the log-likelihood, whilst parameterising the joint latent distribution through a soft-max. 
* FlowHMM \mathcal{F}^{Q} : fit the observation co-occurrence matrix, whilst: 
    - parameterising the emission probabilities through a soft-max of the normalizing flow
    - discretising the continuous observation, allowing the observation co-occurrence matrix to be substituted by the empirical variant. 

The second method being motivated by the fact that directly optimising the log-likelihood is more computationally costly as the length of the time series increases. 
The authors evaluate these two methods against Gaussian mixture model methods on both: 1 and 2 dimensional synthetic examples, as well as real world financial and air pressure datasets. 

Post Rebuttal Edit: Thank you to the authors for providing follow up experiments and discussion. I have updated my score.  
strengths_and_weaknesses: Strengths 
* Extending HMM to non-Gaussian observation domains is likely an important problem, where normalizing flows potentially offer a natural and flexible solution. 
* The manuscript is overall clearly written, with good introductions to the background and problem setting. 

Weaknesses 
* **Curse of dimensionality**: The approach of discretising the response tends to be limited to one (or low) dimensional observations for the co-occurrence method. 
* **Computational cost**: Experiments provide few details on how the two proposed methods are fairly compared from a computational cost perspective. That is, the Appendix states (where T is the time length of the sequence) that "... for T > 1000 we randomly [for the EM model] sample sub-sequences of a given size T = 1000 at each step". The impact of this decision is neither discussed or investigated within the main body. 

questions: Questions:
* **Computational Cost**: Following the weakness associated to the computational cost:
	- What is the impact of sub-sampling sequence length on the statistical performance of the EM method? Particularly, the air pressure data set where there are 40k time steps and only 1k is sampled at any given time. 
	- Would the EM method be more competitive with longer sampled sequences and fewer epochs? 

* For the real datasets, S&P 500 and Dow Jones, are the datasets prices? If so:
    - it is standard practice in financial applications to work with log returns \log(y_k/ y_{k-1}), which tend to be more Gaussian (albeit heavy tailed).
     - how do the Gaussian mixture models perform compare to the normalizing flow methods in this case?

* When Gaussian noise is added to the observations in equation (16) 
	- how is the Gaussian noise hyperparameter chosen? (does it depend on the grid resolution ?)
	- what is its impact on training performance ? 

Notations:

* Within background, the subscript notation for the parameters in x i.e. \alpha_{x,k}, \mu_{x,k}, \sigma^2_{x,k}, is overloaded when using \beta, as \beta_l = \alpha_{\ell,k}, \mu_{\ell,k}, \sigma^2_{\ell,k}. This should be made clearer. 
* Equation (4) z_{n-1} needs to be defined 
* Section 3.3 : "due ti the fact" ti -> to 


limitations: * **Curse of dimensionality**: The method used to discretize the observations for higher-dimensional observations seems ad-hoc, with the experiments suggesting the EM method is better when the observation model is more than 2 dimensional.
review_title: null

Meta-review: metareview: the paper proposes a method that combines a hidden Markov model with neural flow models for modelling sequences. The main contribution is a fitting method based on a combination of gradient-based learning and Expectation Maximization, with variants of the method proposed for different data scenarios. 

The reviewers found that the paper was very strong from a clarity and quality standpoint. There were some small questions about the novelty - the building blocks of FlowHMM are a HMM and a neural flow model, but the reviewers and I agree that his is an interesting combination and that the inference methods are interesting and novel. 

The work is well investigated. The lowest-scoring reviewer (ujjf ) requested more experimentation, which the authors provided wholeheartedly in the rebuttal. Reviewer ujjf upped their score from 3 to 4 based on this, which I'd consider still harsh given the extent of the experimentation in the paper and in the rebuttal. 

Overall, the reviewers agree that his is a compelling paper. 
confidence: Certain
Decision: Accept