Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose a new portfolio policy network architecture for DRl to exploit cross-asset dependency information, which is shown to be able to achieve better performance than existing ones. Such a scheme mainly introduces a permutation invariance property, and it is very interesting.
main_review: Strengths: 
1. The target issue of creating permutation invariant policy networks for DRL is fundamentally interesting and important.
2. The proposed design is well-grounded and has potential applications.
3. The theoretical characterizations are appreciated.

Weaknesses:
1. The portfolio management task is a good example but should be treated as an application, not a goal.
    Such a permutation invariant policy network may have more applications.
2. For comparison methods of the portfolio management task, beside equal weight (EW), it is also expected to compare with minimum variance or mean-variance strategy, which are also widely used.
3. In Table 2~4, the maximum drawdown seems to be relatively high, over 20%?  which is not normal. Please check carefully.


summary_of_the_review: Interesting design with potential applications.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a deep reinforcement learning model for portfolio optimization that harnesses both cross-asset dependencies as well as time dependencies. To achieve this goal, the paper presents a residue block that is consists of dilated convolution component for time dependency and Correlation component for cross-asset dependency.  Experiments are carried out on three datasets using price data and compared to three existing methods to demonstrate the effectiveness of the proposed method.
main_review: Strength:

1. The paper presents a clear formulation of the portfolio optimization problem in section 2. The illustration of the architecture and its specificity is also detailed.

2. The paper presents a reasonable solution to deal with permutation invariance across assets, which is a relevant problem in portfolio optimization with deep learning.

3. The paper presents experimental results on real-world data that demonstrate the effectiveness of the proposed method compared to the alternatives in consideration.

Weakness:
1. Presentation: The description of the architecture is not clear. The intention of some of the components, such as causal convolution, skip connections are not specified. Related work such as WaveNet is not well discussed. 

2. Contribution: While the paper identifies the asset permutation invariance property and proposed a solution to deal with this problem in portfolio optimization, I find this contribution somewhat limited because permutation invariance as a general property is not new.

3. Experiments: While the proposed method is validated on various datasets competing with various existing methods, it is not clear to me whether the compared methods are indeed state-of-the-art as claimed. It is also not clear to me whether the experiments indeed demonstrate that the proposed method can capture cross-asset information.

Question:
The paper considers permutation across assets m, is it necessary to also consider permutation across channels d?
summary_of_the_review: Correctness: I did not spot any obvious errors in the paper.
Novelty and significance: this is my major concern. It is unclear to me whether the proposed method is original and significant enough.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper applies deep reinforcement learning to address the problem of portfolio management where decisions need to be made over time and many factors such as market conditions need to be considered. 
This paper proposed a portfolio policy network that has the permutation invariance (equivariance) property when treating multiple assets’ information. The proposed approach is evaluated on three sets of data from the Canadian and US stock markets. The results show that the proposed approach outperformed baselines in terms of different metrics such as annual return and Sharpe ratio. 

main_review: Strength  
-  Applying deep reinforcement learning to address the portfolio management problem is an interesting yet less explored direction in the RL community. 
- The proposed permutation invariant (equivariance) architecture seems effective. It outperforms baselines in multiple datasets.   

Weakness
   - The reviewer has some questions / concerns about the clarity of some technical details. Please see the comments below:
	    - It is unclear how to obtain the random vector of asset returns. Is it estimated from the historical data? How do you construct the random vector?
	    - To achieve permutation invariance, graph neural networks and self-attention mechanisms are widely used in the literature. What is the motivation for using Corr layer instead of popular methods such as graph neural net or transformer? 
	    - The details for the RL algorithm are missing. What is the RL algorithm used to optimize the policy. Section 2.2 describes a vanilla policy gradient. Is vanilla policy gradient used in the experiments? In addition, the hyper-parameters used in the RL algorithms should be provided.   
	  - Remark 3.1 confuses the reviewer. Specifically, it reads “the architecture proposed in Figure 3 violates Cai et al.’s PI property”. What is Cai et al.’s PI property? What’s the difference between Cai et al.’s PI property and that defined in Definition 3.1?
- Experiments
   - Some metrics seem not defined. For instance, the definition of “Maximum drawdown”, “Daily hit rate”, and “turnover” are missing.
   - How many environment steps are used to train the policy? 
 

summary_of_the_review: In summary, the reviewer found applying deep RL to portfolio management interesting. However, the reviewer has some concern about the clarity of the presentation. Clarification for some technical contents are needed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper develops the first CNN based portfolio optimization network named **WaveCorr**, which is capable of capturing both temporal and cross-sectional correlation structure for the training data. The paper proposes a new type of the permutation invariant (PI) principle for a class of functions (e.g., neural network blocks), and it demonstrates that \texttt{WaveCorr} structure satisfies the PI principle. By conducting numerical studies using data from both Canadian (TSX) and American (S\&P 500) stock markets, the paper testifies the superior performance of **WaveCorr** in terms of Sharpe ratio and stability, compared with several benchmark models.
main_review: I have following major comments on the paper:

Pros:

- For financial data, the estimation of correlation structure is a notoriously hard problem prone to overfitting, due to low signal-to-noise ratio and high dimensional nature of covariance matrix. In my personal view, the PI principle can be viewed as another regularization structure to reduce the complexity of the function class, thus alleviate the over-fitting problem. The idea of adding the corr layer and the result that **WaveCorr** portfolio policy network architecture satisfies the PI property are interesting.

- The numerical section is well structured and the training procedures are reported in detail.  

Cons:
- The paper discusses in Remark 3.1 about the distinction of definition of PI between this paper and the previous literature, but it seems to me that the motivation for this part is still insufficient. Except for the fact that the architecture in this paper violates the PI in previous literature, is there any other motivation or intuition to define PI in the current setting?

- The current approach seems also applicable to long-short portfolio (i.e., $\sum w_i = 0$). Is it possible to include some simple test in the appendix to test the performance in this setting?

- In the numerical part, the standard deviations reported in tables solely reflect the randomness in selection of assets, but fail to capture the uncertainty from temporal variation. For example, one question to consider is: what would be the ex-ante standard deviation estimation, if the proposed portfolio is traded in the next year? If this part is not addressed, one can not rule out the possibility that the superior performance of the proposed method is due to a lucky test period.
summary_of_the_review: In general the paper is well-written, the motivation is clear, and the numerical result is convincing. I recommend acceptance for this paper.
review_title: null

Meta-review: This paper proposes an architecture of a policy network (WaveCorr) that is particularly effective for portfolio management tasks.  A key observation that leads to the design of WaveCorr is that the dependency across asset should be treated differently from the dependency across time.  The proposed WaveCorr has the property that it is "permutation invariant" with respect to assets, which means that the class of functions that can be represented by WaveCorr is invariant to permutation of assets.  WaveCorr is shown to achieve the state-of-the-art performance in a portfolio management task.

A major point of discussion was the definition of "permutation invariance".  The reviewers and AC understood the difference between the permutation invariance defined in this paper and that studied in the prior work (the output of a network is insensitive to the permutation of the particular values of the input).  With the definition in this paper, however, a fully connected layer is permutation invariant, but the Corr layer proposed in the paper appears to have more structure.  It is unclear exactly what properties of the Corr layer leads to the performance improvement.
Decision: Reject