Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper looks at transfer learning on sequence.

First individual Bayesian moment matched algorithms are used on each individual.

Then for a test setting, the probabilities for the new domain are a similarity weighted set of probabilities from the training setting. The similarity weighting is given by the (approximate) posterior probability of the observation (either complete or so far, depending on whether an online scheme is used) for each domain.

A few comments on the model. First there is a discrepancy between train and test: in the test domain there is an assumption in some sort of relationship between individuals, but at training all individuals are treated independently. This contrasts with models that try to analyze between-subject and within-subject variation inherently in the training data. A discussion of these points would be valuable. Does this assume the data about individuals is extensive, so such sharing is not necessary?

The Bayesian posterior on lambda and pi provides a means of model averaging. But weighted averaging of models is very different from weighted averaging of transition probabilities. Given this discrepancy, it would have been good to have a bit more discussion about this choice. Clearly it is pragmatic, but what do you lose and what do you gain? Does this fit into the moment matching interpretation? Of course parameterized sharing is common in multitask settings, but more could be said about this.

Experiments: This seems to be the real point of the paper: the authors have an actual problem to solve and developed this as a method to do this. Yet from the title it feels like the authors felt they had to bill it as a methodological paper for submission to ICLR. Personally I think it is unfortunate that this was a perceived need. A demonstration of what can be transferred in a domain like this is as important as how it is done. The experiments are on a valuable real world problem that people widely care about. This is the real strength of this paper, and a focus on this demonstrative aspect, and a corresponding conclusion would strengthen the paper.

review_title: Transfer learning of activity recognition models across subjects

final_score: rating: 7: Good paper, accept
review_content: I'm bumping up my score to a 7 to acknowledge that the authors responded satisfactorily to reviewer feedback, and to indicate that I think that the updated manuscript is a strong paper and that I do not object to its acceptance to ICLR.

However, I also would not fight for its acceptance. I still think that it is a better fit for a venue with an explicit interest in more traditional Bayesian latent variable models (ICML, UAI, NIPS).

-----

This manuscript describes a novel Bayesian approach to transfer learning focused on online sequence modeling settings where he primary concern is less distribution drift in an ongoing sequence and more variability between individual sequences (since each sequence can be thought of as defining its own conditional distribution over subsequent states). They provide the example of human gait classification, where each individual's gait may differ from others even while performing the same activity. In this setting we typically train the gait model on gait sequences from a set of "source" individuals but then apply it to previously unseen people. The core model is an HMM, which they give a full Bayesian treatment; the central problem this introduces is that each new observation that arrives introduces an additional product term to the posterior that is itself a product over M components (clusters or hidden states). This rapidly becomes intractable. This paper applies Bayesian moment matching (BMM) to this problem, in which the posterior is approximated via projection onto a more tractable distribution that is adjusted to match some moments of the posterior. The experimental results are quite promising

Strengths:
- The problem setting is very appropriately framed as online sequence prediction via Bayesian transfer learning. There is almost certainly individual variability between the training sequences (for which explanatory variables not be available in the inputs). The Bayesian approach gives a natural approach to performing transfer and handling low data regimes (common in all experiments).
- The Bayesian formulation creates a computational challenge (the posterior becomes intractable). The proposed BMM approximation is both reasonable and relatively novel (particularly given the popularity of MCMC and variational methods).
- With the caveat that I am not well-versed in recent work on Bayesian moment matching, a cursory literature search suggests this is a novel application of BMM. A lot of the related BMM work is roughly contemporaneous with this work, and none of it seems concerned with online transfer learning.
- The overall results look quite strong: in activity recognition and flow prediction, the transfer learning approach is in general superior to the one-size-fits-all HMM, even when trained using BMM (which in turn is generally superior to the EM-based HMM). The proposed approach appears to beat the LSTM across all tasks (including sleep stage classification), possibly due to the lack of training data.

Weaknesses:
- The description of the LSTM training and architecture search is vague (and in one instance, contradictory), strongly implying that it was not fully tuned and may be an artificially weak baseline. While it is plausible that the proposed approach might excel given the small data sets used in the experiments, there is not sufficient evidence and detail to support this claim. The authors should provide more detail about architecture search, hyperparameter tuning, and most important, attempts at regularization, given the limited training data. In particular, the authors should experiment with a sufficiently rich set of settings for # hidden layers, # hidden units, weight decay, and dropout.
- Given the emphasis on transfer learning and the use of a Bayesian framework, the decision to train source models independently is a little odd. Why not perform some kind of joint training?
- The authors provide no analysis, analytical or empirical, of the proposed framework's (storage and computational) complexity, especially at prediction time. At the top of page 10, they mention using only one EEG channel in order to "reduce complexity and processing time." This is an ominous hint that the proposed framework may not scale practically.
- Additionally, I have a meta-concern about this paper's fit for ICLR. "Representation learning" -- the general theme of ICLR -- is not featured prominently in this work. Given the competitive nature of ICLR, we should consider seriously whether this paper is of interest to the wider ICLR community or whether it might be a better fit for a meeting such as AISTATS, ICML, or UAI.

Comments:
- The plots in Figures 1-3 are difficult to interpret. Putting patients along the X-axis is unintuitive since their order is arbitrary. Why not just make scatter plots of one vs. the other model's accuracy. The shape of the scatter should hopefully make it clear if there is a general trend.
- In Experimental Setup, the authors make conflicting claims about the LSTM architecture. They first state the single hidden layer has number of cells equal to the number of inputs. They then say that the number of LSTM units is finetuned based on empirical performance.
- While generally well-written, the paper has several obscenely long paragraphs. The single paragraph "Experimental Setup" section, for example, takes up more than half of page 8. These should be broken up into shorter paragraphs to make them easier to read. A good rule of thumb is that no paragraph should take up more than ~1/6 of a page.

In general, I like this work, but the vague details around the LSTM training raise serious red flags about their experimental results, at least the comparison vs. LSTMs, and I have concerns about how well it meets ICLR's CFP. That said, my policy for interactive review is to carefully consider author responses with an open mind, so I will serious consider changing my score, if warranted.
review_title: Novel approach to online transfer learning for sequences but flaws in experiments, questions about fit for ICLR

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes an online inference algorithm by using online Bayesian moment matching for HMM-GMM. The method uses transfer learning by utilizing individual sequence estimators to predict a target sequence based on a weighted combination of individual HMM-GMM. Online Bayesian moment matching has a benefit of updating HMM-GMM parameters frame-by-frame, and fits to this problem. The authors compare the proposed method with the other sequential modeling methods including RNN and EM, and show the effectiveness of the proposed method. The paper is well written overall. 

Comments:
1) Could you provide the average performance in table? It is difficult to compare the performance only with individual performance. Also, it seems that the EM performance is sometimes good 
2) I’m curious how initialization and hyper-parameter settings affect the final performance. If you provide some information about it, that is great.
3) It would be better to provide a figure of describing the transfer-learning-based proposed methods, since this is a unique and a little bit complicated setup.

review_title: null

final_score: rating: null
review_content: In your introduction you say what you do, but you do not outline what you consider to be the fundamental contributions, in terms of what you introduce conceptually that goes beyond what people have done before. It may be a good idea, but it seems very close to many other ideas for transfer. What is your main claim?
review_title: Novelty

final_score: rating: null
review_content: Can you provide some comparisons with the other Bayesian inference methods?
You should at least discuss the comparison of MAP-EM, Variational Bayes, and MCMC approaches.
These are applied to GMM/HMM and also have some online extensions.
Is the EM used in the paper variational EM, MAP EM, or ML EM?
This also has to be clarified.
review_title: Are there any comparisons with other approximated Bayesian inference methods?

final_score: rating: null
review_content: I'd like the authors to comment on two questions:

(1) What are the the parameter storage and test time computation complexities of this approach, and how does it scale to "big" training data (i.e., lots of source domains) and large label spaces (perhaps unlikely in most applications)?

(2) What do you think the ICLR community's primary interest in this work would be (beyond "we can beat RNNs in a low data, transfer sequential learning setting")? Why did you decide to submit it to ICLR rather than, e.g., AISTATS or UAI, which prima facie seem like better fits?
review_title: Question about scalability and fit for ICLR

Meta-review: Though the method does not seem to really break new ground in transfer learning (see Reviewer 1), the reviewers do not question the validity of the approach. The online aspect of the approach as well as an application of Bayesian moment matching to HMMs with GM emissions seem novel. Though the topic may seem a bit peripheral within ICLR, I agree that it can be considered as a representation learning method and the divergence from the mainstream (within ICLR) may be as much a pro as a con. Also, on the positive side, the applications are interesting and real, and the methods seem well suited to the task. The paper is well written.
 
 + well-written
 + technically solid
 + interesting application
 
 - innovation is moderate 
 
 +/- not a typical ICLR paper
Decision: Accept (Poster)