Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes an architecture and gradient-based training for an encoder/decoder scheme to reduce communication of an exogenous signal to a remote optimization-based controller. Such scenarios are given, for example, when a remote location has forecast information (e.g., traffic) that is relevant for an optimization-based controller (e.g., optimal scheduling of taxis). The goal of the paper is to reduce communication of the exogenous, while maintaining reasonable performance. The main idea is to take the control objective into account when training the compression scheme, so that the encoder/decoder is optimized for the task at hand. The claimed contributions are (i) introduction of a novel problem, (ii) analytical results for the LQR case, and (iii) improved performance and compression on real-world data sets.
main_review: ## Originality

1) The components of the presented approach (encoder/decoder for reduced communication, gradient-based end-to-end training, remote control architecture) are not new. Also the idea to take a downstream task into account when designing reduced communication schemes is not entirely new (see next comment). Nonetheless, this reviewer is not aware of the proposed combination of techniques. Thus, I see the main contribution in that combination and the demonstration that this leads to reduced communication.

2) Related work: The discussion of related work is mostly adequate.  However, I was missing a *specific* discussion on the technical results for the LQR case with exogeneous input in Sec. 3.1.   The LQR problem with exogenous inputs has been studied before - so, how are the technical results derived in Sec. 3.1 related to results known in control?  Just to be clear - the combination of results is likely new, but still, it would be good to see a discussion on how the obtained equations, e.g. for the optimal controller, compare to existing works. 

I have further suggestions for the discussion of related work:

2.1) The idea to reduce amount of network traffic by transmitting only the data that is relevant for a certain task has also been investigated under the terms of "event-triggered communication/estimation/control" (see surveys Heemels et al, An introduction to event-triggered and self-triggered control; or Miskowicz, Event-based control and signal processing) and, more recently, "event-triggered learning".  For example, Schlueter et al. (Event-triggered Learning for Linear Quadratic Control) also use LQR control cost to trigger learning of a better model when needed. These ideas have also been applied to reduce network communication in wireless sensor systems with savings similar to those reported herein (e.g., Beuchert et al, Overcoming bandwidth limitations in wireless sensor networks by exploitation of cyclic signal patterns: An event-triggered learning approach) . While the mentioned event-triggered approaches emphasize sparsity in time, the manuscript at hand has a different perspective on reducing communication.  Thus, the works are not conflicting (maybe orthogonal?), yet are closely related on the level of motivation and application to reducing communication in control networks. In fact, a combination of the approaches might be interesting for further research.  
2.2) When reading the "Related Work" subsection at the beginning of page 2, I was surprised to see no discussion on teleoperation, e.g. in robotics.  The authors later add such a discussion (to some extent), so it might be worth pointing out already at the end of the "related work" subsection, that more discussion will follow after introducing the technical problem in detail.



## Quality

The paper seems technically sound.  Limitations are briefly discussed at the end of Sec. 5., and in the conclusion.  The numerical examples serve to illustrate main claims of the paper.  IMHO, the experimental results could, however, be improved by reporting on nonlinear examples:

3) The numerical examples presented in the main paper all include linear dynamics.  Obviously, these are less challenging then nonlinear problems.   For a nonlinear problem, the reader is deferred to the supplementary material.  I would find it much more convincing if the nonlinear example was part of the main paper. 



## Clarity

I found the submission clear.  I only have a minor comment on the abstract:

4) In the abstract, "forecasts are designed without knowledge of a downstream controller’s task objective, and thus simply optimize for *mean* prediction error."  -- I don't see this line of argumentation.  Why does the ignorance of a downstream task give rise to optimizing the MEAN only?  In fact, as one does not know what downstream task will be performed, one might want to optimize for predicting full distributions (even though this is hard, of course).



## Significance

5) As stated above, I see the main contribution in the combination of existing techniques.  To my knowledge, the concrete setting and architecture is novel.  Though, I did not find it too surprising that one can train an encoder/decoder in such a way to save communication. The empirical results demonstrate improvement over more standard ways to train encoder/decoder (without knowledge of the downstream task), yet they don't compare to alternative ways to save communication (e.g., see works mentioned under 2)), so it is hard to judge how significant the communication savings are.
limitations_and_societal_impact: Yes, IMO.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies sending compressed representations for cooperative control by creating task-driven representations that are co-designed with the control task. They present theoretical compression results (of LQR being low-rank compression in prop 1) and go on to study non-linear cellular, IoT, and electricity load data settings. The idea, summarized in Algorithm 1, is to parameterize encoders/decoders, use the decoded signal for optimization, and optimize the loss in Eq 8 that includes the control loss and a prediction component.
main_review: This is a well-motivated and timely study of end-to-end learning for cooperative control. They take the existing pieces of differentiable control and end-to-end model learning and theoretically and empirically study them in a new cooperative context. To the best of my knowledge, the theoretical results in the LQR setting are novel, and the experiments in Fig 2 validate the idea that including the downstream signal enables more efficient latent representations of the data to be learned. The battery storage optimization task from [3] is reasonable extended from the single to multiple battery setting. My largest experimental concerns is that the bottleneck dimensions used in Fig 2 are relatively small and only consider dimensions <10. Realistically, the bandwidth in these settings should allow for higher-dimensional data to be transmitted.
limitations_and_societal_impact: The paper does not significantly discuss these and it could be insightful to discuss scalability limitations on how scaling the bottleneck dimension and control problem sizes impact the overall tractability/learnability.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a solution to learn succinct, compressed time-series embeddings, which also considers downstream modular controller’s task objective.

Contributions:
1. novel problem: learning compressed time series representations which are also tailored to the downstream control tasks.
2. Analytic compression results on LQR control for illustrations on the problem.
3. Experiments on real-world datasets
main_review: Originality: This paper studies a novel task that learns time series representations for the downstream control tasks. This paper discussed clearly how this work differs from previous contributions on page 3,  with the most relevant paper cited.

Quality: The submission is technically sound and the claims are well supported by theoretical analysis on the  LQR problem. Then the paper expands to more general settings like DNN forecasters. 

Clarity: The submission is clearly written and well organized.

Significance: The results are important, not only from the perspective of improvements on baselines but also from the perspective of possible application domains. This paper discussed the diverse application scenarios like smart factory on IoT sensors, taxi dispatch, and battery storage. 
limitations_and_societal_impact: The authors pointed out that they do not automatically learn the optimal bottleneck size that minimizes control cost nor necessarily learns a human-interpretable latent representation.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper discusses an approach for deterministic optimal control under estimates of covariate states.
main_review: Overall, I found the paper a good read and easy to follow. However, I am not sure if there is a significant merit to the paper, as the theoretical work seems to me well known and pretty straightforward. I would have either hoped for a bigger focus in the experimental section, which I would have expected to include a real world control scenario or for more work in a theoretical direction for example by including stochastic dynamics. 

Originality: The task overall is not known to me and I am not sure if there are a lot of real world applications for the scenario. The methods presented in the work are pretty standard and there are not many new theoretical insights presented. To me it was not completely clear how the work positions itself in the field w.r.t. related work.

Quality: I did not find any technical mistakes and the mathematics are correct as far as I could check. 

Clarity: The submission is nicely written, overall easy to follow and well organized.

Significance: I am not completely sure if the results are very significant. 

Aspects I found which were missing from a theoretical standpoint are:
- The work only discusses a deterministic system and therefore no noisy dynamics. Additionally, a probabilistic model description especially for the estimation of the forecast state is probably very favorable in many scenarios.
-  For the forecast only point estimates are considered. I think a Bayesian estimate would be more appropriate. As this work does not include any uncertainty about the estimates. However, a controller should probably balance its control decisions if estimates are uncertain.
- Something I did not completely understand was why the controller for $\hat{u}$ should have the same parametric form as the one for $u$. What is the motivation for this design, similar to certainty equivalence?

Things I missed from the experimental section:
- Why is the work only evaluated for the very simple dynamical system? For a synthetic evaluation I would have expected also non-linear system dynamics. 
- I would have appreciated if the system would have been evaluated for a real world control problem.
- For the  closed form solutions in the linear case, I would have appreciated an evaluation.

limitations_and_societal_impact: N/A
review_title: null

Meta-review: This paper proposes an architecture and gradient-based training for an encoder/decoder scheme to reduce communication of an exogenous signal to a remote optimization-based controller. The proposed method takes the control objective into account when training the compression scheme, so that the encoder/decoder is optimized for the task at hand. 

This paper introduces a (to some extend) novel problem, presents theoretical results for the LQR control special case and  demonstrates improved performance on a set of benchmark tasks. However, the reviewers also find that the proposed method is a mix and patch of different techniques (though the proposed combination seems novel) and the numerical demonstrations could be strengthened by including additional baselines.

This paper tackles an problem that might become increasingly important in the future. The reviewers therefore advocate the acceptance of the work, but assume that the criticized details will be addressed in the revision, including
- a discussion of the stochastic problem variation
- clarification of technical contributions and differences to related work (citing (Singh, Pal, 2017) is not the crucial point here, rather that the discussion is _specifc_  (as drafted in the author's response))
Decision: Accept (Poster)