reviews: 
Comments: This work approaches the problem of dealing with long horizon tasks in decision making systems (such as Robotics), particularly with the transformer architecture, whose memory costs grow non-linearly with the memory length. For this matter, it introduces three techniques to be applied on top of the TransformerXL architecture: Automatic Chunking, Forget Span, and SimilarityWeight. These techniques implement heuristics for retrieval and forgetting of non-parametric memories based on the similarity of state representations. The work presents experiments on a set of memory tasks, ranging from gridworld to multi-modal navigation tasks. It claims improvements in sample efficiency by using the proposed techniques.
Strengths: - The work brings an interesting set of tasks that focus on memory aspects and progressively increases the difficulty on the representation and environment dynamics, which is ideal to evaluate the claims in the work.
Weaknesses: - The work has a major flaw in terms of clarity. First, the structure can be greatly improved. In more details:
     - The Introduction could be more clear on the contributions of the paper and better describe the motivations; 
    - The Section 2 could be better modularized in two:
        - A Preliminaries Section, to introduce, for instance, the problem formulation, the RL algorithm, and the transformerXL architecture);
         - A proper Methodology section (describing what the work is proposing); 
    - Section 3 could also be better structured, by first bringing the experimental setup (with task description, training setup) and then raising results and discussions later on.
    - During most of the work, the sentences are very long and the reasoning is confusing; also, the description of the techniques is repeated several times in different parts of the work (for instance, the Automatic Chunking technique is described during Section 1 and 2, and Subsection 2.1).
    - The mathematical formulas should also be rewritten for better clarity. The paper should define mathematical symbols and use them to present equations.
    - Some part of Section 3.1 (above Figure 3) is simply cut off. There are also typos and missing full stops. Crucially, the paper must be proofread carefully before an official submission.


- In the Abstract and Introduction, the work starts with strong claims regarding the human cognition, and decision-making process, to serve as motivation for the work. Nevertheless, these claims are not backed-up with proper literature and state general assumptions on how the memory works, which is actually an active topic of research in many disciplines.

- In the same line, the work does not bring any literature review. The study of memory systems in RL is very active nowadays, as well as transformers for RL. Therefore, the work must discuss related work and contrast with prior similar techniques, to contextualize the readers and clarify the contributions of what is being proposed.
    - In the same topic, the Reference section does bring some works, which are not properly cited in the text. This suggests that something is wrong with the template files used to write this paper.

- There are many design choices that are not clear or unjustified. I will raise them as questions below. Furthermore, the choice of a transformer architecture is not motivated. Transformer optimization is often hard, especially in RL setting [1], so why is that required for the proposed formulation?

- Without a proper argument in the paper to contrast prior work, it seems to lack originality in the proposed techniques. Indeed, there are several works on non-parametric memory with forgetting heuristics ([2, 3] to name a few), which should be contrasted and compared against in order to evaluate the effectiveness.

- There are also flaws in the experimental execution:
    - No experiment brings confidence intervals, which raise questions about statistical significance. This is crucial on RL experiments given the intrinsic variance of methods and non-determinism of benchmarks [4, 5].
    - The work does bring some ablation on top of the transformerXL architecture, but there are no baselines on memory architectures to compare against. For instance, the paper could reproduce the methods in [2, 3] to serve as baselines.
    - In Figure 8, there is no baseline at all, which makes it hard to evaluate the effectiveness of the proposed method.
    - In Appendix A, some information seems confusing and even contradictory. For instance, please look at the chunk size information in Appendix A.3 in the list and text description.

**Further Suggestions/Minor Concerns:**

- It would be interesting to bring a Pseudocode for the proposed techniques. This would help to formalize the proposed heuristics and better situate the reader in the algorithmic flow.

- As a way to improve clarity, the environment descriptions could be moved to an appendix, in order to decouple from the experiment's results and discussion. Otherwise, it breaks the reading flow by switching the context from setup description to analysis.

- Another way to improve clarity is to present some illustrations of the proposed techniques. Figure 1 is confusing, as it is hard to know what is part of the transformerXL and what is the contribution.  A schematic illustration, clearly identifying which of the proposed modules, would be very helpful.


**References**

[1] Parisotto et. al. Stabilizing Transformers for Reinforcement Learning. ICML, 2020.

[2] Yalnizyan-Carson and Richards. Forgetting Enhances Episodic Control With Structured Memories. Frontiers of Computer Neuroscience, 2022.

[3] Coda-Forno et. al. Leveraging Episodic Memory to Improve World Models for Reinforcement Learning. Memory in Artificial and Real Intelligence at NeurIPS, 2022.

[4] Henderson et.al. Deep Reinforcement Learning that Matters. AAAI, 2018.

[5] Agarwal et.al. Deep Reinforcement Learning at the Edge of the Statistical Precipice. NeurIPS, 2021.

**Summary of the Review**

The work brings several major flaws and needs a careful revision with rewriting in order to clearly state its claims, motivations, methodology, and experimental results.
Recommendation: 3: reject, not good enough
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 1 poor
Clarity: 1 poor