Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: **Quality**
Overall, the technical content appears correct. The authors present results that show their proposed ARMCMC method outperforms RLS and conventional MCMC for a Hunt-Crossley problem. The results also show the robustness of the proposed algorithm to adapt to abrupt changes in the model parameters.  

**Clarity**
Overall, the paper is well written. Paper should be revised to improve clarity and correct typos (non-exhaustive):
•	Page 1: "Besides, inaccuracy, inaccessibility, and cost are the typical shortcomings that make them not ideal for solely use" What does "them" refer to. 
•	Page 2: appllicable, performance, t_m in definition of D^t
•	Page 4: \sigma_e or \sigma_v?
•	Define \theta in Equation (1)
•	Equation (6): e^t vs e^t_n?
•	Page 6: k_{min} on both sides of (17)
•	Page 7: “\theta_1 = K_e; \theta_1 = B_e; \theta_3 = p”
•	Define all acronyms when first used (e.g., RLS).
•	Consistency in notation (ARMCMC-APS vs. AR-APS)
•	Captions should be more explanatory to describe figure/table content. Define acronyms in figure/table captions so that the reader does not rely on information in the text to interpret the information in the figure. Increase the font size in figures. 
o	Figure 1: Scale on y-axis (fractional minimum number of samples?). Label x-axis.  
o	Figures 2 and 3: Labels based on the actual parameters, not generic \theta_1, \theta_2, \theta_3. A better layout is to time align probability distribution (top) with the corresponding parameter estimates (bottom).
o	Figure 4: Plot the results separately to allow easier comparison

**Originality**
Authors provide a summary of the prior work and outline the shortcomings that their proposed work addresses. The authors claim the proposed adaptive modifications to the MCMC provide the flexibility for exploitation or exploration based on the estimated model mismatch, updating the proposal distribution to better match the current data conditions, and selecting the number of iterations for parameter estimation based on a desired level of precision and reliability. 

**Significance**
This paper describes an adaptive recursive algorithm to address the restrictive requirements of MCMC in Bayesian parameter estimation (“systems Linear in Parameter (LIP), having Persistent Excitation (PE) requirements, and assuming Gaussian noise.”). The proposed algorithm can be applicable to other latent parameter inference problems.
---------------------------------
**Pros**
•	The authors outline the proposed modifications, which includes some assumptions and derivations - the differences between MCMC and ARMCMC are highlighted.
•	The proposed algorithm is compared with other algorithms. 
•	The authors provide code (however, should be better annotated).

**Cons**
•	It is difficult to assess the generality/utility of the proposed modifications (e.g., variable jump distribution) to other applications as the performance of the proposed method is only demonstrated in a specific case. A more comprehensive analysis is needed to assess the implications of the various approximations on model performance (e.g., model initialisation, substitution with a Gaussian distribution in (15), forgetting factor). 
•	It is not clear how some of the parameters are chosen, e.g., threshold (\zeta_{th}).
•	Authors should first describe the methods of the experiment before presenting results to facilitate reproducibility. 
•	Provide quantitative results for the claim: "For the algorithm to run realtime, MCMC requires more time to converge." (page 7).

review_title: ARMCMC: ONLINE MODEL PARAMETERS DENSITY ESTIMATION IN BAYESIAN PARADIGM 

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper introduces a new Markov chain Monte-Carlo (MCMC) algorithm to obtain and track the posterior distribution over unknown parameters in a non-linear system. Despite its simple elegance, i.e., the introduction of a data-driven _temporal forgetting factor_ into the usual Metropolis-Hastings algorithm, the approach is, to my knowledge, novel. Its discovery seems to be the result of the intersection between fields: system identification and Bayesian sampling techniques, leading to new bridges. 

In fact, novelty and interdisciplinarity are the strong points of this paper, but mostly everything else is compromised due to a severe lack of clarity. Although Section 1 is clear and fulfills its role of contextualizing the paper well, from Section 2 onward the a) organization, b) notation, c) technical inaccuracies in writing, and d) lack of detail on the application make the paper hard to read. To give just one example in each category: a) a major change in the usual MCMC scheme is explained in (and only in) _Remark 2_, Section 2.1, which is supposed to be part of the preliminaries on MCMC, b) the number of different notations for the collected data is ludicrous, and they are impossible to distinguish, c) $P_k$ in $(9)$ is not a probability density function (PDF), simply a sample of the MCMC algorithm,  and d) Section 2.4 is confusing because $p$ and $n$ are sometimes interchanged, no contextualizing figure is provided, assumptions are not interpreted, etc.   

I recommend to reject this paper, but only because I believe it should be rewritten to be exposed clearly. Although the contribution is worthwhile, the paper is confusing and misleading. I am open to changing my recommendation if the authors introduce major changes in the presentation, writing, and structure, while still presenting the same content.

Among writing inaccuracies, using the term "density estimation" in the title of the paper is among the worse. Any Bayesian approach obtains a posterior distribution (explicitly or implicitly), but this is not the same problem as estimating the probability distribution of some given data. 

The following questions point to some of the key problems I have found in the paper, without any specific order:  
* Have the authors evaluated the assumption that the Gaussian model for noise is sufficiently robust for the application even when other noise models are used for the empirical results?  
* Could the authors define the times at which the algorithm is run and the times at which data is collected? as well as the relation between them? A simple, intuitive, and correct notation for the data is needed. That could be the first step towards obtaining it. 
* Could the authors clarify in which sense the minimum number of data points is restricted (not confined) by the fact that the algorithm is aiming at real-time operation? 
* Could the authors clarify the role of $\rho$? It is mentioned in _Remark 2_ and does not seem to appear again. 
* Could the authors give some hints to the readers on how the expectation-maximization algorithm is used for delay compensation? This would greatly aid reproducibility. 
* Could the authors explain what is, and discuss the results of, AR-MAPS in Table 1? 
* Could the authors use clearer measures of improvement than percentages of improvement? These are ill-defined and it is hard to pin which percentages are being reported. 
* Is the resulting posterior as informative when the parameters are zero because the needle is not in contact with the soft surface? Could the vertical axes in Figure 2 be expanded to explore that? 
* Could the authors clarify in which sense their simulated data is realistic? The appendix is quite uninformative to non-experts in the application. 

Suggestions to improve the paper -  not specifically related to my recommendation: 
* Thoroughly revise the use of English: some sentences are missing key components.
* Emphasize and clarify throughout the paper the role of the _temporal forgetting factor_, and the fact that it is not simply a user-defined constant, but a function of the observed model mismatch: this can be confusing to those that are not used to the concept.
* Use the entire available horizontal space in Figures 2, 3 and 4: it will help visualize some of the key features and look better.
* Use transparency for the line plots in Figures 3 and 4: it will allow to see their individual behavior better.
* Resize the fonts in all plots to match the size of the Figure captions: this will make them much easier to read.
* Correct the references (capitalization, etc.), e.g., Hunt-Crossley vs hunt-crossley in (Haddadi, Hashtrudi-Zaad, 2012).



review_title: Nice, simple, novel contribution awfully explained

final_score: rating: 6: Marginally above acceptance threshold
review_content: **The changes of the review after rebuttal are  indicated in bold text.**

### Summary of the contribution
This article proposes an algorithm called ARMCMC (Adaptive Recursive Markov Chain Monte Carlo" in the context of contact dynamic, for example when detecting the presence/absence of contact over time between a needle and a soft material. 
ARMCMC aims at estimating the parameters of the Hunt-Crossley model, related to the elasticity and the viscosity of the material.
The algorithm handles both a real time estimation, and the fact that there are sudden changes in the dynamic (contact/no contact).
After reminding the basics of Bayesian estimation and MCMC, Section 2 finishes with the description of the contact dynamic model. Section 3 describes the ARMCMC algorithm and its main components, while Section 4 applies it to the example cited and compares it to Recursive Least-Squares (RLS).

### Strengths
The paper is rather well written and interesting. 
It tackles a problem that seems difficult, especially since it is discontinuous and it is treated in real time, hence the need for both accurate and fast estimation.

### Weaknesses and concerns
1. Organization of the paper and clarity 
- The title and introduction are a bit misleading as we expect to see a general method, while it appears later that it is designed specifically for the example given in the experiment.
If it is more general than that, I encouraged the authors to discuss other possible applications. 

**The authors added an experiment to show that the method is more general than it appeared in the first version, giving more strength to the paper.**

- For a submission to a conference in artificial intelligence, it would be better to have less details on the well known MCMC (or at least put the details in appendix), 
and more details about the specific application. 
- The application envisioned is only specified in Section 4, while it would improve the comprehension were it already described in Section 2. 
In particular, there is mention of a needle after Eq 10, the purpose of which we only understand 3 pages later. 
2. Related work
Only one other algorithm is compared with the one proposed (RLS) it seems to me that related work has been overlooked.
- please discuss how ARMCMC differs from RJMCMC (Reversible Jump MCMC), where we could jump between model 1 (no contact) and model 2 (contact) back and forth.

**The authors answered that comment a bit quickly, in my opinion, as RJMCMC has been applied to a change-point problem in the original paper, which seems to be similar to the one presented in the paper. However, their argument on realtime is valid.**

- also, how does the proposed algorithm relate to the following ones?
	o "Particle filters for non-Gaussian hunt-crossley model of environment in bilateral teleoperation", Agand et al (2016), which is cited for other reasons
	o "Neural Network Control of a Robot Interacting With an Uncertain Hunt-Crossley Viscoelastic Environment", Bhasin et al (2008)
	o "Exact restitution and generalizations for the Hunt–Crossley contact model", Carvalho and Martins (2019)

**The authors agree that the work mentioned propose different approaches to the same model, but do not compare these approaches to the one they proposed in their experiment, and do not give enough details as to why.**

3. Minimum number of evaluations
Figure 1 shows an evaluation of the minimum number of evaluations depending on the values of the parameters: 
- how come there are values lower than 1? 
- In practice, was $k$ round up from $k_{min}$?
- is it in purpose that it is called $k_{min}$ in Eq 17 and Figure 1, and $k_{max}$ in Algorithm 1?

**The authors answered all the questions related to $k_{min}$, and it is now clearer.**

4. Empirical results
- The setting of the experiment is not specified: which material was used? Or was it synthetic data?
- There is a mention of results without EM at the end of section 4.1, but are there any WITH EM, and if not why is it so?
- In Table 1, why use different metrics for the parameters and for $F_e$? Why not use RMS everywhere? Does that mean that there are some points that are strongly over- or under-estimated? Or is it because of the lag (see comment below)?
- In Figure 3: 
	o how come the lag for the absence of contact is so important with ARMCMC, while there is almost no lag for the beginning of contact? is it because of the temporal factor?
	o Can you comment on why the RLS looks so unstable?
	o also, which of the 2 versions of ARMCMC is presented here? APS or MAPS?
	o Why is there no comparison with MCMC in Figure 3, whereas it appears in all other results?
- Figure 4 is rather hard to read: please consider using doted/dashed lines and possibly markers to distinguish better between the estimators
- There is mention in the introduction and in Section 2 that RLS requires some restrictive conditions:
	o are they not realistic in practice?
	o it would be interesting to see a case where those conditions are not met, showing the strength of the proposed algorithm

### Question
Do the parameters for viscosity and elasticity of a material vary in time? Is it really important to keep estimating them in real time, or can't they just be updated once in a while?

### Minor comments
- Eq. 8 and 9, please replace the implication sign by a sentence for better readibility.
- In section 2.4, 2nd line: put the reference of Haddadi and Hashtrudi-Zaad in parenthesis to improve readibility.
- After Eq.10, the third parameter is designed by $n$ instead of $p$.
- After Eq 11, the acronym RLS is not defined: I had to go see the reference to see that the R stands for recursive, and not reweighted e.g.
- Eq. 19 mentions the variables $\delta_2$ and $\delta_0$, but there are no $\delta_1$ anywhere.
- Please increase the fontsize of the axis and legends in Figures 2 to 4.
- **Typo "apllicable" on page 2**

### Overall evaluation
While the topic and the approach are interesting, I believe the state of the art has not been discussed sufficiently. 
If some of the suggested work indeed relate to the proposed one, they should be included in the experiment for comparison.

**After rebuttal, I think the authors did a good job in clarifying some points and adding an example in the experiment, which is why I upgrade the score I gave. However, I don't understand why they don't compare in the experiment with some of the methods from the cited literature concerned with the same model, especially Diolaiti et al. (2005).**
review_title: Interesting paper with good results, but related work seems overlooked

Meta-review: Pros:
- The authors propose a novel method to perform MCMC in the condition where there is a distribution over models describing the data, rather than just a distribution over the parameters of a single consistent model (ie, in the theme of reversible jump MCMC). Sampling from the posterior of mixtures of parametric models is something current MCMC algorithms are very bad at, so this addresses an important need.
- Reviewers believed the proposed technique was novel and technically correct.
- The paper appears to build a bridge between the fields of system identification and Bayesian sampling techniques

Cons:
- Major concerns were raised about lack of clarity
- - From a *lightweight* read, I also had difficulty understanding what the paper was proposing, or even the precise problem it was tackling
- Experimental validation was limited, and missing baselines
- During discussion, the paper had no strong advocate
Decision: Reject