Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper proposes a system to predict the gain, in terms of observed space, of a push action on the plant foliage. To this end, SRPNet is proposed to regress the predicted space behind the occlusions as an image, thus providing spatial information. The full system PPG uses the outcome of SRPNet to evaluate sampled actions and decide for the best action that predicts most of the space to be discovered. The networks is trained and tested in two setups, synthetic vines and a real dracaena plant.
review: Strengths:

- The paper is mostly clearly written and easy to follow.
- This is an interesting application that has potential.
- SRPNet follows similar ideas as in other applications (depth completion, semantic scene completion, temporal forecasting, world models) where data-driven methods have shown good predictive results.

Weaknesses:
Three main concerns raised from this paper:
1. How to decide for an action it was not totally clear. From algorithm 1 it seems that every time there is a need to sample all possible action (used in training) and then evaluate SRPNet to then take the action that maximizes the information gain. This scales poorly with the action space and the complexity of the plant. Please comment.
2. The whole PPG idea reads like an ungrounded heuristic. There are plenty of methods for motion planning that could have been adapted to this application, giving the right observation and action spaces, for instance Next Best View (like to maximize discovering), of frontier based ones (where frontiers could be defined as occlusions). Please comment.
3. The claim of generalization is misleading. For generalization I'm expecting to train in one plant/scenario and test in a different one, maybe same species, in different scenario and environmental conditions. The need for training in the same plant that will be deployed speaks against the usefulness of the approach. Please comment.

Finally, although the problem seems interesting, I guess the real value would be in answering the question "What is behind this leaf/branch?" and not just moving everything at once.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Weakness overweight the interesting problem setup.

**After rebuttal**
I appreciate the effort put into the rebuttal to try answer the open points. 

However,  the weaknesses I highlighted are not well addressed in my opinion. The system can be reduced to a heuristic search, with no comparisons to a information gain driven (NBV or others) approach. From field robotics deployment, the generalization capabilities are not demonstrated ( it would have been more convincing to try out a different Dracaena in an outdoor environment to show some generalization). And finally, my last point was not about moving a selected leaf, it was about the reasoning to discover, or reveal more in a sequence of decisions given the actions and observations taken so far. But, I understand the confusion from my phrasing.

All in all, I keep my rating, but as already mentioned I am fine with acceptance if the rest of the reviewers go for it.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: The authors introduce a robotic method for investigating plants. A large part of that is moving foliage out of the way and the paper presents (a) a neural net that estimates how a certain manipulation would affect visibility and (b) a policy that successively explores the plant, trying to maximize viewing new, previously occluded areas over the course of several manipulations.
review: 
### Strengths

- **S.1** Robotics is already hard and adding plants into the mix is a lot harder. I have huge respect for the authors choosing to tackle such a difficult (and "messy") problem and providing real-world experiments.
- **S.2** The method is clean, easy to follow, easy to implement, yet very useful.
- **S.3** The paper is well-structured and well-written. All the figures are very illustrative and a lot of attention went into everything. This reads like a very polished paper.
- **S.4** The experiments illustrate the strength of the contribution well.

### Weaknesses

I honestly can't think of any major concern. My only issue is that the authors use a 2D model for view novelty, not a 3D model but (a) that's not in the scope of this work and (b) the authors clearly state this as future work. The current submission is great as it is.
I was about to write that I would've liked a video to go along with the submission until I remembered that you referenced a website at the beginning of your paper. I checked out the videos on the website and they're very good.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This is my favorite paper at this year's CoRL and I'll stand fight to get it accepted. It's very well-written, and presents a straight-forward solution and great experiments for a very difficult problem.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper focuses on an autonomous agriculture application. It proposed a learning-based method (SRPNet) to manipulate the plant foliage to look behind the leaves and the branches. The proposed SRPNet predicts revealed space for each action candidate and uses the cross-entropy method to achieve closed-loop control. Finally, the proposed method is evaluated on two plants in the real world.
review: ### strengths
1. Autonomous agriculture is an important but less-explored area. This paper proposed a data-driven method to solve an important challenge in this field.
2. The proposed method is straightforward and easy to understand. The cross-entropy method, simple yet effective, enables multi-step closed-loop control.
3. The proposed method and comprehensive ablations are evaluated on plants in the real world.

### weaknesses
1. Action space is not clearly described. For example, how many bins are there for one-hot orientation and z height. Related to this, what's the reason to use one-hot representation for orientation and z height, instead of absolute value?
2. The task solved in this paper is to maximize the cumulated revealed space. Why this task is important and how does the proposed method help other downstream tasks?
3. The proposed method is only evaluated on two specific plants. It would be better to conduct a systematic and slightly large-scale evaluation.
4. plants are deformable and elastoplastic objects. The proposed method doesn't explicitly model the physical properties and dynamics.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: New application for robot learning and comprehensive evaluation and ablation study on real-world platforms.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: This paper presents a self-supervision-based approach -- Push Past Green (PPG) -- for learning to de-occlude parts of a plant by sequentially moving plant parts with a (real) robot arm. It also introduces a novel dataset for two different plant types (vines and Dracaena) that consists of ~50 hours of plant interaction and associated ground truth "space revealed" metrics (data collected using random robot arm actions). This dataset is used to train the PPG system via self-supervision on the random actions in the dataset, and associated "space revealed" scores.

The model input is an RGB + height image of the plant, a history of space revealed so far, as well as an encoding of the suggested action. A subnetwork called "Space Revealed Prediction Network" (SRPNet) takes the action encoding and a crop around the start pixel, encodes this and decodes it into a probability map of space revealed.

Extensive experimental results (using a real robot) showcase the advantages of the method.
review: **STRENGTHS:**

* **Highly relevant from a robotics-in-agriculture perspective**, and this relevance is well motivated in the main text (e.g. the introduction).

* The suggested approach is well motivated, e.g. Line 29-38 in the introduction. In particular, it motivates the need for the collected dataset. Also, to the best of my knowledge (I'm not an expert on robotics-in-agriculture), the work is put into context of relevant related work (see the related works section) and the suggested approach is well-motivated given the various shortcomings / missed opportunities of past works. To the best of my knowledge, the approach and type of dataset are novel.

* A **relevant, real-data and large-scale dataset has been collected** and will be made **publicly available for future research** within the plant self-occlusion problem.

* **Great that the method can be (and is!) trained in a self-supervised manner**, so that expensive annotations can be avoided.

* **Strengths related to the experimental results:**
    - **Good set of ablations** that showcase the design choices (see e.g. Table 1). The associated analyses (see e.g. Line 199-211) are good too, and provide interesting insights (e.g. "There are no trivial solutions of the form 'space gets revealed where height is high'.").
    - Good that **multiple runs with different random seeds are done**, and that confidence intervals are included.
    - Good that **runtimes are reported** (albeit in the supplementary material; such an important metric may be best to add to the main paper).
    - The **proposed method outperforms reasonable baselines.**

* **Very impressive additional material**, including the video summary on the attached webpage.

**WEAKNESSES:**

* **Some things were not quite clear to me and would improve the paper if addressed:**
    - **Why the use of a CRM approach for action selection?** It would be good to see some experimental results that compare some form of pure argmax-style action selection during evaluation.
    - In Table 2, I interpreted the "Random Horizontal / Tangential Action" as equal to the "Tiling Baseline". If this is true, consider renaming "Random Horizontal / Tangential Action" to "Tiling Baseline". If not, consider clarifying.
    - It's sometimes hard to grasp the Area Revealed metric when it is reported e.g. in cm^2. Could one perhaps also include some metric which is in the [0,1]-range, i.e. normalized somehow?
    - It was a bit difficult for me to understand the difference / connection between "PPG w/ Other Dynamics Models" and "Hand-crafted Dynamics Model". It seems like the latter could be a subset of the former (?).
    - In Fig. 7, I was a bit confused why the methods require multiple steps to reveal a single plant part? It seems a single, deterministic step would do it, given that the targeted area is quite small (?).

* The greedy nature of the reveal actions imply that **the proposed methodology may be inherently ill-suited for certain types of inferences.** The greedy reveal actions are certainly good for the specific task of revealing things (assuming all reveals are considered equally important / informative -- see my next remark in the below item), but if the revealing is merely a step towards some form of 'global-level' inference related to the plant (e.g. its overall health status), then greedy reveals may not be the globally optimal approach for said global inference. An RL-based method with a sparse reward (obtained at the end) could be a better choice in such cases, but on the other hand it is well-argued why such an approach is difficult in these contexts (see e.g. the second paragraph of the intro).

* Oftentimes, **not all visible space is equally important for a given type of inference** (it may e.g. be that some parts of a plant typically does not hide anything 'interesting', and so more focus should be put on revealing those parts that are likely to hide something interesting (e.g. berries)). It would have been great to see some discussion (e.g. in the Limitations section) that mentions this fact, and ideas for addressing it. 

* In the same spirit as the last item above: **It would have been great to have some form of experimental results on downstream performance** (e.g. the task of counting things that are occluded by plant parts (e.g. berries)), and not only evaluate the "reveal" metric for which the model is explicitly trained. I however understand that the scope of a paper has its limits.

* While there is quite some degree of out-of-domain generalization being showcased, some quite simple tricks could have further assessed this. For example, the data collection could have been done in different lightning conditions, so that models trained in a certain lightning condition would be tested on another condition.

* Again, great with seed sensitivity analyses. But since the number of trials is less than 20 (it's 10 as I understood it), perhaps +- std would have been sufficient, rather than 95% intervals (since they are not statistically significant given the few trials (less than 20), if I don't misremember my statistics-101-class).

**SUGGESTIONS (not part of evaluation):**

* Might be nice with a last column in Table 1, that simply computes the average numbers across the three existing columns.

* Might be interesting future work to combine your proposed methodology with some form of methods that predict what lies beneath plant parts (similar to e.g. human pose estimation, where occluded body parts are often estimated, albeit at varying accuracy). The policy may then adapt is "reveal movements" based on how accurate it thinks that the model already is at predicting what is beneath the plant part even before the reveal. In other words, only those plant parts are revealed where the model cannot confidently guess what is beneath.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper is very well-written, proposes a solid approach and dataset for an important task within robo-for-agriculture, and extensively showcases the merits of the proposed method and its various design choices. I think the paper should definitely be accepted. It may however be further improved in various ways (see my comments above), but all-in-all it's a no-brainer for acceptance in my view.
review_title: null

Meta-review: This paper focuses on manipulating plants to reveal hidden areas (e.g. moving a leaf). The method predicts revealed space for a given action and uses CEM to move the plants to reveal as much space as possible. Evaluations are conducted on two real-world plants.

Strengths:
- The presentation is clear and readable
- The method is sound and makes sense for this application
- The results demonstrate that the method is effective at revealing space behind plant parts

Weaknesses:
- It is not clear that what the method does is useful for agricultural (or other applications). Simply revealing occluded space does not seem like the correct metric. Rather, revealing specific targets (e.g. fruit or side-shoots) is what such manipulation should seek to do. As the method does not have knowledge of the overall plant structure, it seems the only way to find specific targets is to reveal as much space as possible, which is very inefficient.
- The generalization claim is overstated. It is expected that such a method transfer across a species of plants, but instead the training is plant-specific, requiring that data is collected on the specific plant to be manipulated. This is clearly not going to scale to agricultural applications. The authors should evaluate how transferable the method is across different plants. The claims about the plant growing over the time the research was conducted are not quantified and difficult to evaluate for generalization.

Overall, the rebuttal did not significantly change the reviewers' opinions.
Decision: Accept (Poster)