Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation

Abstract: Domain adaptation (DA) aims to alleviate the domain shift between source domain and target domain. Most DA methods require access to the source data, but often that is not possible (e.g. due to data privacy or intellectual property). In this paper, we address the challenging source-free domain adaptation (SFDA) problem, where the source pretrained model is adapted to the target domain in the absence of source data. Our method is based on the observation that target data, which might no longer align with the source domain classifier, still forms clear clusters. We capture this intrinsic structure by defining local affinity of the target data, and encourage label consistency among data with high local affinity. We observe that higher affinity should be assigned to reciprocal neighbors, and propose a self regularization loss to decrease the negative impact of noisy neighbors. Furthermore, to aggregate information with more context, we consider expanded neighborhoods with small affinity values. In the experimental results we verify that the inherent structure of the target features is an important source of information for domain adaptation. We demonstrate that this local structure can be efficiently captured by considering the local neighbors, the reciprocal neighbors, and the expanded neighborhood. Finally, we achieve state-of-the-art performance on several 2D image and 3D point cloud recognition datasets. Code is available in https://github.com/Albert0147/SFDA_neighbors.


Reviews and author responses:

 The score is corrected.

This paper proposes a method for source-free domain adaptation (SFDA). While methods for unsupervised domain adaptation access the labeled source domain data during adaptation, SFDA does not allow models to do so. The proposed method utilizes the neighborhood affinity of the target data. The experimental results show that the proposed method achieves state-of-the-art performance on image classification and point cloud classification.  Originality:
- In Section 5, the authors insist, "To the best of our knowledge, existing works on DA do not explicitly exploit this source of information, and they could be combined with our method." However, the use of local structure in the target domain is pretty well-known. [a] is one of the examples having a similar motivation.

[a] Pan et al., Exploring Category-Agnostic Clusters for Open-Set Domain Adaptation. CVPR, 2020

Quality: 
- The papers below have a similar problem definition to that in this paper. 

[b] Li et al., Source-free domain adaptation for semantic segmentation. CVPR, 2021.

[c] Kurmi et al., Domain Impression: A Source Data Free Domain Adaptation Method. WACV, 2021.

[d] Yeh et al., SoFA: Source-data-free Feature Alignment for Unsupervised Domain Adaptation. WACV, 2021.

Clarity: 
- Figure (c) shows that the samples misclassified once can be pushed toward a cluster for each class. However, it is unclear why the proposed method can form completely clean clusters.

Significance:
- In Section 5, the authors insist, "To the best of our knowledge, existing works on DA do not explicitly exploit this source of information, and they could be combined with our method." The former is already addressed in the originality part. If the authors insist that the proposed method can be combined with their method, they could have shown some results.
- The performance gain is sometimes very moderate. For example, results of Office-31 and Office-Home are very close to those by SHOT [21] and ModelAdaptation [20]. By the way, the authors of [20] have named their method not as ModelAdaptation but as Collaborative Class Conditional GAN (3C-GAN). There are no descriptions of limitations and societal impacts.


 To further show the stability of the hyperparameter selection, we have performed one additional experiment. We randomly split the whole target data (VisDA) into validation (20%) and test set (80%). We use the labeled validation set to find the optimal K and M. We report the accuracy on the test set (for the same number of iterations as the maximum was reached on the validation set). The results show that the maximum found on the validation set (K,M=5 which was also chosen in our paper) leads to high results on the test set. Note that here the adaptation to the target domain is only done with 80% of the data (excluding the validation set) and we therefore report lower results (in the paper we report 85.9%). SHOT obtains 82.9% when using 100% of data for adaptation which is still significantly lower than our method. 

| VisDA | Avg on val | Avg on test |
| ---------- | ---- |---- |
| K, M = 2       | 82.2 | 82.3 |
| K, M = 3       | 83.3 | 83.4 |
| K, M = 4       | 84.5 | 84.9 |
| K, M = 5       | 84.9 | 85.0 |
| K, M = 6       | 84.8 | 85.0 |

 We thank the reviewer for his/her feedback. We are happy to see that he/she 'leans towards acceptance'. We wonder if the reviewer would like to update his score to 6 which better reflects this qualification. 

 The reviewer has read the other reviews and all responses from the authors; the first score has been upgraded.

1. The fact that K-means is global or non-local is somewhat disconcerting since it is ultimately a matter of hyperparameter selection, but the reviewer agrees that the proposed method using KNN with small $K$ and $M$ makes use of a more local structure. The reviewer also agrees that the difference between these methods is meaningful because it has been shown that leveraging the reciprocity of features in the neighborhood achieves better performance than related methods, as described below.
1. If the experimental protocols in the existing papers are different, the right policy would be to re-implementing them according to the protocol in this paper and comparing their performances. Therefore, the comment about [d] is not very fundamental. As for the others [b, c], it would be okay if the descriptions in the rebuttal are also added to the camera-ready version.
1. The authors states "Our method does not obtain 'completely clean clusters' but improves over previous methods." If so, Figure 1 (c) still seems to be a misleading advertisement and should be modified.
1. On VisDA and PointDA-10, the reviewer agrees that the performance gains over the existing methods seem significant.

Mainly because of 1. and 4., the reviewer leans to acceptance.

 **[1]** Ideally parameter selection is based on a validation set. However, in UDA such a set is not available, the generally applied approach therefore is to select some parameter setting and show that the results of the method do not vary significantly (show a degree of robustness) with respect to the chosen parameters. We agree that this setting is not optimal and a defined validation set would be preferable, but have followed the same protocol as the other papers. We respectfully disagree with the reviewer that the results ‘vary a lot’; our results show that our method remains state-of-the-art for all the tested parameter settings for SFDA (our results vary from 83.4% ~ 85.9% (see Figure 4 right) versus SHOT 82.9%) on the VisDA-C.

**[2]** We have chosen the VisDA-C dataset for the memory usage and runtime analysis (including all operations) since memory usage is a problem for large datasets. On the office home dataset, remembering all examples (vectors) is of negligible overhead (2.4~4.3 M per domain) . We share the reviewer’s intuition that on this dataset (with 2427-4439 target samples per domain for 65 classes) storing only 10% of the data might be more detrimental, but since this is a small dataset we do not think this is a very interesting setting for memory usage analysis, and have therefore chosen to evaluate memory usage on the largest dataset VisDA-C.
 
 
**[3]** We outperform SHOT on the more challenging VisDA-C dataset by a large margin. Also, we have reproduced results of SHOT to the best of our ability on the PointDA-10 and consider these results to be comparable. We significantly outperform SHOT (by around 10%) on the PointDA-10. Note, that similar results for SHOT on the PointDA-10 dataset have been reported in another article [a].

**[4]** The DomainNet dataset is not used by standard UDA and SFDA methods  (it is used for multi-source DA and semi-supervised DA), we therefore did not consider it. Also note that Office-Home (also Office-31) has more categories and even less data, and our method works well.
 
*reference:*

[a] Tian, Jiayi, et al. "VDM-DA: Virtual Domain Modeling for Source Data-free Domain Adaptation." arXiv preprint arXiv:2103.14357 (2021).


 **1. In Section 5, the authors insist, "To the best of our knowledge, existing works on DA do not explicitly exploit this source of information, and they could be combined with our method." However, the use of local structure in the target domain is pretty well-known. [a] is one of the examples having a similar motivation. If the authors insist that the proposed method can be combined with their method, they could have shown some results.**

We will adapt the sentence and include a discussion of the mentioned paper. Indeed [a] uses the cluster structure, however, [a] is using K-means clustering, our method is using K-nearest neighbor. Technically [a] uses the cluster information (class-mean), thus they are considering **global/non-local** information, while we use neighborhood information (local). Our novelty: 1. We provide the insight that by simply considering neighbors, it is possible to achieve adaptation even without source data. To our knowledge there are no existing papers doing similar work. 2. More importantly, our major contribution is to consider reciprocity between the feature and its neighbors, which has not yet been investigated in the DA community. The cited sentence by the reviewer with respect to combination with other methods was referring to future work, we will rewrite that sentence to make this clear. 

[a] Pan et al., Exploring Category-Agnostic Clusters for Open-Set Domain Adaptation. CVPR, 2020

**2. The papers [b,c,d] below have a similar problem definition to that in this paper.**

We will include a discussion of these papers in any final version. The most relevant methods are SHOT and 3C-GAN which are extensively compared in our paper. [b] focuses on semantic segmentation, and note [b] is based on synthesizing fake source samples which is a much more complex process. [c] only reports results on Office-31, which is significantly lower  than ours (83.5% versus 89.4%). [d] does not follow the mainstream experimental setting, they adopt AlexNet as backbone instead of ResNet which is used by almost all recent methods. It is therefore difficult to compare with [d]. 

[b] Li et al., Source-free domain adaptation for semantic segmentation. CVPR, 2021.

[c] Kurmi et al., Domain Impression: A Source Data Free Domain Adaptation Method. WACV, 2021.

[d] Yeh et al., SoFA: Source-data-free Feature Alignment for Unsupervised Domain Adaptation. WACV, 2021.



**3. Figure (c) shows that the samples misclassified once can be pushed toward a cluster for each class. However, it is unclear why the proposed method can form completely clean clusters.**

Figure 1(c) is for illustration purpose, to show that reciprocal neighbors are considered more importantly. In addition, Fig 1b illustrated that reciprocal neighbors are more reliable. We propose a method to explicitly exploit the intrinsic neighborhood structure (and improve it by using reciprocal neighbors). The t-sne visualizations in Figure 1(a) and Figure 5 (right) show that our method performs effective feature clustering. Our method does not obtain ‘completely clean clusters’ but improves over previous methods.

**4. The performance gain is sometimes very moderate**

The reviewer is right that on two dataset our gains are moderate. However, on two datasets we obtain significant performance gains: 3% and 10.1% on VisDA and PointDA-10. And more importantly, our paper provides an insight that simply considering neighborhood relation along with reciprocity can achieve source-free domain adaptation.

| PointDA-10 | Avg  |
| ---------- | ---- |
| SHOT       | 42.5 |
| Ours       | 52.6 |





 [1]. Parameter analysis.  Figure 4 provided the sensitivity of two parameters M and K on the VisDA-C, the accuracy varies a lot for different values. However, the model selection part (how to determine the optimal parameters is not explained, which is a key problem for unsupervised domain adaptation.)

2. The complexity. The complexity provided here ignores the ranking operation and feature dimensionality. Also, since the number of samples for each class in VisDA-C is large, so reducing the size of the memory bank is feasible. Could the authors provide results on other datasets like Office-Home?

3. Overall, the advantages of the proposed method over SHOT are not significant on (digits, Office, Office-Home). Since the original paper of SHOT does not provide the results on PointDA-10, such a result can not fully verify the advantages of the proposed method.

4. Experiments on multi-class DomainNet are more persuasive. It would not be hard to conduct such an experiment during the rebuttal period. 

 Thanks for responding.

1. Thanks for indicating your assumption of homophily in low confidence regions. 
2. It is interesting to note the result with a negative $r$ value.
3. Please do cite and include all related work. 

I do not have any further questions on my end. Thanks.




 I would like to thank the authors especially for providing additional experimental results on runtime analysis. I think these results further strengthen this study.

I'm satisfied with the response and believe nothing to discuss with authors remains.


 **1. How to determine the parameters M and K, as shown in "Model Details", it seems that these two parameters are carefully tuned.**

It is always nontrivial to find an optimal $K$ for K-NN. Figure 4 shows some degree of robustness to these two hyperparameters, implying that we can simply set $K$ and $M$ to a small number such as 2 or 3 (only considering one neighbor is noisy). We will improve discussion in the paper.

**2. It is not clear how to calculate the affinity matrix A. do we need to calculate the $n_t*n_t$ distance matrix for each mini-batch?**

We do not compute the affinity matrix for all target features in every mini-batch. Since we are doing mini-batch training, the computation of neighbors will only contain $\mathcal{O}(bs \times n_t+K \times n_t)$ ($bs$ is the batch size), that is, in every mini-batch, we first do K-nearest neighbor retrieval for all target features in the mini-batch. Then for each of the K-nearest neighbors we further retrieve its M nearest neighbors (expanded neighbors). 

Nevertheless, our method can also be easily adapted to lower memory and computational requirements by using a smaller bank which is replenished cyclically (following the setting in [1], only storing a fixed number of features and prediction scores). That is, they are implemented as queues where every iteration the memory banks are updated with the features/predictions of the current minibatch, while those of the oldest remaining minibatch are discarded. Thus both memory and computational requirements decrease, with a small and graceful drop in performance, yet still remaining state-of-the-art. This shows that our method can be efficient and effective even in larger datasets.

| VisDA                         | **runtime (s/epoch with one TITAN Xp)** | Acc (%) |
| ----------------------------- | --------------------------------------- | ------- |
| SHOT                          | 618.82                                  | 82.9    |
| **Ours**                      | 540.89                                  | 85.9    |
| **Ours(20% for memory bank)** | 507.15                                  | 85.3    |
| **Ours(10% for memory bank)** | 499.49                                  | 85.2    |
| **Ours(5% for memory bank)**  | 492.28                                  | 85.1    |

[1] Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., & Zisserman, A. (2021). With a little help from my friends: Nearest-neighbor contrastive learning of visual representations. arXiv preprint arXiv:2104.14548.

**3. The advantages of the proposed method over SHOT are not significant.**

The reviewer is right that on two dataset our gains are moderate. However, on two datasets we obtain significant performance gains: 3% and 10.1% on VisDA and PointDA-10.

| PointDA-10 | Avg  |
| ---------- | ---- |
| SHOT       | 42.5 |
| Ours       | 52.6 |


**4. The results on a multi-class DomainNet (365 classes) would be more persuasive since the neighbors found may be noisy.**

We will consider the experiments on DomainNet since it is really time consuming containing 0.6 million images, and currently only multi-source DA and semi-supervised DA methods report results on this dataset


 **1. Runtime analysis** 

Our method is scalable to the amount of target data. First, note that our memory banks only store feature vectors and prediction vectors. Second, our memory banks can store all the features/predictions for VisDA (the largest dataset in our experiments) and remain computationally more efficient than SHOT (see runtime analysis below).

Nevertheless, our method can also be easily adapted to lower memory and computational requirements by using a smaller bank which is replenished cyclically (following the setting in [1], only storing a fixed number of features and prediction scores). That is, they are implemented as queues where every iteration the memory banks are updated with the features/predictions of the current minibatch, while those of the oldest remaining minibatch are discarded. Thus both memory and computational requirements decrease, with a small and graceful drop in performance, yet still remaining state-of-the-art. This shows that our method can be efficient and effective even in larger datasets.


| VisDA                         | **runtime (s/epoch with one TITAN Xp)** | Acc (%) |
| ----------------------------- | --------------------------------------- | ------- |
| SHOT                          | 618.82                                  | 82.9    |
| **Ours**                      | 540.89                                  | 85.9    |
| **Ours(20% for memory bank)** | 507.15                                  | 85.3    |
| **Ours(10% for memory bank)** | 499.49                                  | 85.2    |
| **Ours(5% for memory bank)**  | 492.28                                  | 85.1    |

[1] Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., & Zisserman, A. (2021). With a little help from my friends: Nearest-neighbor contrastive learning of visual representations. arXiv preprint arXiv:2104.14548.

**2. How to optimally set the number of nearest/expanded neighbors, it may heavily depend on both the dimensionality of the features and the number of target data.** 

It is always nontrivial to find an optimal $K$ for K-NN. Figure 4 shows some degree of robustness to these two hyperparameters, implying that we can simply set $K$ and $M$ to a small number such as 2 or 3 (only considering one neighbor is noisy) on all datasets. And we do not think it is quite sensitive to the feature dimensions since feature dimensions in our method are the same for all datasets. We will improve discussion in the paper.

**3. It is not clear to see whether the final objective function is minimized if and only if all p_i are one-hot vectors with ground-truth labels.** 

Please note that basically the $p_i$ will never one-hot.  Since for all those objective functions, the gradient comes only from the $p_i$ in the dot-product, maximizing the dot product is to make the $p_i$ similar to the predictions ($\mathcal{S}_j$ which is not one-hot) of their neighbors, which is the softmaxed output of the classifier. So these objectives aim to ensure consistency between the predictions of features and their neighbors.

**4. Something wrong in Eq. (7). I think both \bar{p}_c and q_c cannot be label distribution but scalar.** 

Thanks for pointing out this, only $\bar{p}$ and $q$ are category distribution. $\bar{p}_c$ is the $c$-th value of the empirical category distribution $\bar{p}$, and the same for $q_c$.




 **1. It is not clear how RNN could perform better than NN since the most confident samples are classified correctly anyway.** 

It should be noted that the RNN and nRNN in Figure 1(b) are inside the K-NN ($K$ nearest neighbors = $K_1$ RNNs + $K_2$ nRNNs), which shows that RNN in the K-NN are of high possibility to share the same correctly predicted label. Indeed the points in the low-confidence area (typically near the decision boundary) are more important. The assumption of our method is that target features with high degree of connectivity are possibly belonging to the same class (this assumption is known as homophily and is widely use in graph clustering methods). And this is expected to hold also for those low confidence points. In that sense, our method is also related to label propagation for semi-supervised learning, but in our case labels are actually propagated from those points with higher confidence to those with lower confidence. And in this process, the RNN are more important since they have higher likelihood of belonging to the same class.


**2. Trying different affinity values for nRNN and expanded neighbors** 

Since we can not really tell whether the nRNN belong to the different class or not. it is not safe to directly assign a negative affinity. We ran this experiment with r = -0.1 which gets 4.9% lower than r = 0 on VisDA. Thanks for the valuable suggestion and we will consider this improvement. 

**3. Related papers**

We will improve the paper and include the related works [1,2] into the paper, and also add the limitation or societal impact.


This paper introduces a new method from the viewpoint of neighborhood regularization for source-free domain adaptation where only the source model instead of the raw source data is provided to the target domain. The proposed method distinguishes reciprocal nearest neighbors (RNN) and non-reciprocal nearest neighbors (nRNN) among the k-nn neighborhood. By exploiting the memory bank and the relationship with RNN, the proposed method obtains better results than previous source-free methods like SHOT and Model Adaptation on 2D image datasets and a 3D point-cloud dataset.  Overall, this paper is well-written and easy to read. The main contribution lies in the usage of reciprocal nearest neighbors (RNN) to preserve the intrinsic neighborhood structure of the target data during feature adaptation.

Pros:
1. Different from previous methods using clustering or neighborhood consistency, the usage of RNN in this paper is interesting and effective for target feature adaptation;
2.  Obtaining promising results on several cross-domain recognition benchmarks including Office-Home.

Cons:
1. How to determine the parameters M and K, as shown in "Model Details", it seems that these two parameters are carefully tuned. 
2. During the proposed method, it is not clear how to calculate the affinity matrix A. Since we want to find the intrinsic neighbors, do we need to calculate the n_t*n_t distance matrix for each mini-batch? If so, the time cost would be quite large.
3. In table 4, the authors are encouraged to run the results of SHOT and Model Adaptation for a complete comparison. Since on Office and Office-Home in Table 1-2, the advantages of the proposed method over SHOT are not significant.


Besides, the results on a multi-class DomainNet (365 classes) would be more persuasive since the neighbors found may be noisy. Not applicable

This paper explores the effectiveness of reciprocal nearest neighbors for the challenging task of Source-Free domain adaptation where the target domain is used to train the model already pre-trained on the source domain. The paper exploits the clusterability of the target dataset in absence of the source domain and highlights the importance of a self regularization. The results demonstrate promising performance on 2D and 3D tasks.  Nearest Neighbour (NN) based techniques have been used extensively in DA literature to structural properties of the target domain. The paper finds that using Reciprocal NN (RNN) instead of NN provides an improvement in the performance of the model. 

Further, the paper identifies certain gaps with RNN and exploits extended neighborhoods for further improvements. The ideas described in this paper are novel and have been presented well. 

The results are significant and demonstrate promising accuracy for 2D and 3D computer vision tasks.

Concerns:
1. The top K/M RNN points are shown to have a higher correct ratio percentage compared to the top K/M NN points. However, the performance of the source-free model really depends on the points that lie in low confidence regions (which are typically not present in the top K) because they contribute to the majority of the samples. So even if the extended neighbourhood is considered, it is not clear how RNN could perform better than NN since the most confident samples are classified correctly anyway.

2. On Eq. 1 ($\mathcal{L}_{\mathcal{N}}$), each of the terms on the right-hand side ($A, \mathcal{S}, p$) are always positive. So the dot product of these vectors will be limited to a positive regime. Wouldn't it be ideal to choose a negative $A$ value for the nRNN data point (consider it as a highly dissimilar point) that needs to be "pushed away" instead of $r=0.1$ (L143)? Also, in Fig. 3, a value of $r$ that lies in between -1 and 0 has not been shown. For e.g., a $r= -0.1$.

Suggestions to authors:
1. Training the model in a curriculum fashion could boost the performance of your method since the model has the flexibility to learn the signal from beyond the data points present at the top-K. [Ref 2]

Minor typos:
1. L136: "non-reciprocal neighbors neighbors" .. repeated words.
2. L143: "If not specified otherwise r is set to 0.1." .. remove otherwise here?

Consider including the following references in your paper as they are closely related to your method.

[Ref 1] Yuang Liu, Wei Zhang, Jun Wang, "Source-Free Domain Adaptation for Semantic Segmentation', CVPR'21

[Ref 2] Venkat N, Kundu JN, Singh DK, Revanur A, Babu RV, "Your Classifier can Secretly Suffice Multi-Source Domain Adaptation", NeurIPS'20


 The authors have not included any limitation or societal impact in the current draft of the paper.
I encourage authors to consider how a source-trained model could inadvertently induce bias during source-free domain adaptation that might result in societal impact. For e.g., For a task that involves human facial images, choosing RNN to sample a set of points might automatically favor a certain group of people who are dominantly present in the source dataset.

Please consider including a broader impact statement as well, if you see fit.

The authors proposed a novel method for source-free domain adaptation. Based on empirical observation, they assume that the features of target data extracted by the pre-trained model form clear clusters. To explicitly capture this intrinsic structure in the feature space, the local affinity of the target data is defined, and the model is trained to predict consistent labels if there is high local affinity. The experimental results with several benchmark datasets show that the proposed method works well in the source-free setting.   <Review summary>

This study is well-motivated and tackles an important problem, which is unsupervised domain adaptation without access to the source data. The proposed method is carefully designed to solve this problem and works well in several benchmark datasets, while there are some concerns on hyper-parameter setting and scalability. I vote for "weak accept."


<Details>

Strength

- This paper is well-written and easy-to-follow.

- This study tackles an important and challenging problem, which is source-free UDA. The source-free UDA should be beneficial in many practical situations where domain adaptation is needed.

- The experimental results show that the proposed method works well in several benchmark datasets. In addition, rigorous ablation studies validate the design of the proposed method.



Weakness

- Scalability to the number of target data. Since we need to store and continuously update the extracted features of all target data, it should spend a lot of memory, when the number of target data is large. In addition, computational cost can be also prohibitively large due to the nearest-neighbor search process in each iteration. Although small-scale target data can be expected in some situations where domain adaptation is required, this point should be discussed in the paper.

- How to optimally set the number of nearest/expanded neighbors. Although Fig. 4 shows some robustness of the proposed method against suboptimal setting of K and M, this robustness may heavily depend on both the dimensionality of the features and the number of target data. Since it is difficult to tune hyper-parameters in the adaptation phase under the source-free setting, this point should be carefully analyzed/discussed.

- It is not clear to see whether the final objective function is minimized if and only if all p_i are one-hot vectors with ground-truth labels. Showing loss curve with error curve may help.

- Something wrong in Eq. (7). I think both \bar{p}_c and q_c cannot be label distribution but scalar. Please check it. There is no description on limitations. I strongly encourage the authors to describe some of them such as what I raised at <weakness> in Main Review.

There is no description on possible negative societal impact, but it is not so much problematic, because this study is on general machine learning techniques.

Meta-review: Following the rebuttal and discussion period, this paper received borderline scores with three leaning towards acceptance and one recommending rejection. All reviewers agreed that this paper proposes a novel algorithm that leverages local structure in a way different from prior work (using nearest neighbor consistency instead of global cluster structure through K-means). Most reviewers found the empirical results to be compelling, though eN4k did have doubts about whether the results would scale (both in terms of performance and complexity) to larger datasets such as DomainNet. However, after considering all reviews, discussion, and rebuttal comments, the AC is inclined to agree that the algorithmic novelty is interesting and sufficiently empirically justified with the existing datasets to warrant publication. The authors should be sure to include the discussion around K-NN vs K-means structure and clarify their claims and the other minor points brought up in their final version. Further both scalability and how to set the hyperparams K, M were brought up by multiple reviewers, please add a discussion around these points. 

Decision: accept