Paper: SOI: Scaling down computational complexity by estimating partial states of the model (ICLR-2024)

Reviewer comment (Experiments):
The paper could be strengthened by including a more comprehensive comparison with other state-of-the-art methods, which would further enhance the paper's quality.

Author response:
We believe that added experiments in Appendices E-G will strengthen our paper. Please note that there are not a lot of methods that we can compare to as the SOI will work alongside most (if not all) of state-of-the-art methods like any type of pruning.

Reviewer comment (Experiments):
The paper primarily discusses the application of SOI to convolutional neural networks (CNNs). While this is a significant contribution, the generality of the method could be questioned if it is not easily adaptable or applicable to other neural network architectures. For CNNs, the only baseline lies on STMC, which seems not generalizable.

Author response:
We believe that the contribution to just CNNs is still substantial. Please be informed that our method can work in RNN, in fact we have the RNN-based model that work with SOI but we are planning to release it separately as in this model there is much more going on than just SOI. Another reason to not include RNN is that this paper already has 19 pages and we would like keep it concise and easy to understand.

Reviewer comment (Experiments):
The paper provides examples and results primarily related to audio separation and acoustic scene classification tasks. Demonstrating the effectiveness of SOI across a wider variety of use cases and domains would strengthen its generality and appeal to a broader audience.

Author response:
We added video action recognition task in Appendix E.

Reviewer comment (Evaluation):
The 9.8% reduction in metrics for the speech separation task seems significant. Could you provide some context to justify why this reduction is considered minor? For example, you could claim that in another paper, their performance drop was much larger (e.g., xx.x%) when complexity was reduced by 50%, so that readers can gain a rough impression that 9.8% is minor.

Author response:
SOI is quite adaptable and can provide the range of complexity reductions at different cost. In our case those 9.8% of SI-SNRi do not contribute significantly to the output of a model and from our experience and experiments such difference is not detectible by a human listeners during blind tests (in this specific case).

Reviewer comment (Experiments):
I would suggest adding at least two more experiments to the experiment part to strengthen and demonstrate the advantages of the proposed method.

Author response:
We added 3 new sections to appendix. 

In Appendix E, we added experiment with action recognition task from video data to show that our method works well in other domains. In this section we also applied SOI to MoViNets which we used with their original ‘stream buffers’ method instead of STMC to show that it can also work with methods similar to STMC.

In Appendix F, we compare our method to resampling using common resampling methods.

In Appendix G, we show that our method works well with pruning which is commonly used in our scenario.

Reviewer comment (Experiments):
Although the current experiment is cool, to better support the authors' theory, it is always good to shoot some or more real application cases with sound results.

Author response:
We added 3 new sections to appendix. 

In Appendix E, we added experiment with action recognition task from video data to show that our method works well in other domains. In this section we also applied SOI to MoViNets which we used with their original ‘stream buffers’ method instead of STMC to show that it can also work with methods similar to STMC.

In Appendix F, we compare our method to resampling using common resampling methods.

In Appendix G, we show that our method works well with pruning which is commonly used in our scenario.

Reviewer comment (Experiments):
The empirical comparisons also seem very limited, including only STMC and some variants of their method.

Author response:
> The empirical comparisons also seem very limited, including only STMC and some variants of their method.

In appendix E, we added experiment with action recognition task from video data using ResNet-10 architecture and MoViNets in A0 and A1 variants. This experiment also proves that our method is not dependent on STMC as with MoViNets utlize ‘stream buffers’.

In appendix F we added comparison of our method to common resampling methods which can be used to reduce computational complexity as well.

In appendix G we show how our method works with pruning.

Reviewer comment (Presentation):
Fig. 1: C and E seem identical.

Author response:
> Fig. 1: C and E seem identical.

Fig. 1C and Fig. 1E are not identical as operation in Fig. 1E has an additional shift in time represented by relative position of time frames in time axis (time axis is also depicted). Position of frames in time is important as it defines how we interpret them throughout the network and at output. As this was not clear we added cropping to Fig. 1E to make it easier to follow.

Reviewer comment (Writing):
Sec. 2.2: X seems to have length N and N also seems to be the number of samples.

Author response:
> Sec. 2.2: X seems to have length N and N also seems to be the number of samples.

Yes, 1D vector X Is composed of N samples and that’s why it has length N.

Reviewer comment (Writing):
In Abstract, second to last sentence: Not clear what latter refers to.

Author response:
> in Abstract, second to last sentence: Not clear what latter refers to

Modified to clearly state that this sentence refers to FP variant of SOI

Reviewer comment (Presentation):
Fig. 2 caption has two B). Seems to be off by one.

Author response:
> Fig. 2 caption has two B). Seems to be off by one.

Fixed. Thank you!