Paper: Think Before You Act: Decision Transformers with Internal Memory (ICLR-2024)

Reviewer comment (Novelty):
The novelty is limited. The main idea is to integrate an external memory for Transformers to improve memorization and reasoning. There have been many works along this line of research. The proposed memory read and write mechanism is also straightforward and heavily based on the mechanism (content-based attention, add, erase, ...) of NTM and DNC.

Author response:
Thank you for your insightful feedback! 

## Q1

>The novelty is limited. The main idea is to integrate an external memory for Transformers to improve memorization and reasoning. There have been many works along this line of research. The proposed memory read and write mechanism is also straightforward and heavily based on the mechanism (content-based attention, add, erase, ...) of NTM and DNC.
    
A: **Internal Memory vs. NTM**: While both the internal memory module and NTM aim to introduce memory mechanisms to neural architectures, our internal memory is inspired by human cognitive processes and is specifically tailored to decision-making agents. The design and integration of this module into DT, and its subsequent application, is a **novel contribution** in the context of our work. Besides, there are other papers inspired by NTM: \[1\] Memory Augmented Neural Networks with Wormhole Connections. ArXiv:1701.08718 \[2\] Hybrid computing using a neural network with dynamic external memory. Nature 2016. Same as the above mentioned papers, despite that our work inspired by NTM, we design a new memory reading and writing techniques for DT.

**Use of LoRA**: While the concept of LoRA might not be new, its application in conjunction with our internal memory module and within the realm of LLM-based decision-making agents brings a novel perspective. The combination of these components, and the synergies they produce, offers a unique approach to addressing the generalization and adaptation in MDT.

The novelty of our work isn't merely in the individual components but in the holistic approach and the results achieved. By blending inspiration from human cognition with state-of-the-art machine learning techniques, our paper is the **first** paper that introduces internal memory in DT improved training efficiency and generalization in a variety of tasks, as evidenced in our experiments.

**Contextual Novelty**: It's crucial to recognize that novelty doesn't always stem from introducing entirely new concepts. Often, the innovative combination, adaptation, and application of existing ideas to new contexts or challenges can bring significant advancements to a field.

Reviewer comment (Experiments):
It is unclear why the proposed memory has advantages over other Transformers with internal/external memory or even just long-range attention [1,2,3,4,5]. More explanations are required in the method and more baselines need to be included in the experiments. In particular, the current baselines have only RMDT as a memory-based Transformer, which is not enough, especially when other memory-based Transformers can be adapted easily to offline RL in the same way as the Decision Transformer. Also, retrieval-based RL [6] can be a good baseline as well to highlight the benefit of internal memory.

Author response:
## Q2

>It is unclear why the proposed memory has advantages over other Transformers with internal/external memory or even just long-range attention [1,2,3,4,5]. More explanations are required in the method and more baselines need to be included in the experiments. In particular, the current baselines have only RMDT as a memory-based Transformer, which is not enough, especially when other memory-based Transformers can be adapted easily to offline RL in the same way as the Decision Transformer. Also, retrieval-based RL [6] can be a good baseline as well to highlight the benefit of internal memory.
    
A: Thank you for the valuable suggestions. However, RMDT is the most state-of-the-art method that related to our settings, memory module for DT.

**Compare to existing methods**:

- **Memory Writing**:
	1. **∞-former** represents memory as a continuous signal using radial basis functions (RBFs). When new information is encountered, it is integrated into this continuous representation. This process involves evaluating the continuous signal at specific locations and then concatenating these evaluations with new vectors coming from the short-term memory.
	2. **DT-Mem** involves sophisticated mechanisms using attention to manage the significance of new and existing data. This process would be based on calculating correlations between the input and memory and updating the memory with the attended weight of the input sequence.
	3. **LONGMEM** caches paired attention keys and values from the previous context into a non-differentiable memory bank using a frozen backbone Large Language Model (LLM) as the memory encoder.
	4. **KNN-transformer** uses (key, value) pairs from the local context are appended to the end of an external memory.
- **Memory Reading**:
	1. The reading process of **∞-former** utilizes a continuous-space attention framework.
	2. **DT-Mem** uses content-based addressing for memory retrieval. This process would involve using attention mechanisms to read from the updated memory, focusing on the content relevant to the current task or context.
	3. **LONGMEM** uses a decoupled memory module, specifically a residual side-network (SideNet), for memory retrieval and reading. The SideNet retrieves cached key-value pairs of previous contexts from memory using the attention query of the current input.
	4. **KNN-transformer**: features a kNN-augmented attention layer that combines standard dense self-attention with approximate k-nearest-neighbor search into the external memory. The kNN lookup retrieves the top-k (key, value) pairs for each query from the input subsequence, constructing an attention matrix that represents these memories differently for each query.

In summary:

- **DT-Mem**: Stands out for its efficient decision-making capabilities and balanced approach to memory retention and adaptation, making it particularly suitable for tasks requiring contextually relevant decisions.
- **∞-former**: Excels in handling unbounded contexts with precision and flexibility, ideal for extensive and complex datasets.
- **LONGMEM**: Its decoupled architecture and token-to-chunk retrieval make it adept at managing large contexts.
- **kNN-augmented Transformer**: Offers flexibility in context length and rapid adaptation to new data, enhancing the model's real-time applicability.

Linformer is a method trying to reduce the complexity of transformer model, which is not related to our paper. Thus, we don’t compare with this method.

Due to the limit time, we would love it if the reviewer could provide a bit more information about what are the experiments they would like us to prioritize, what they hope to get from the additional results they ask of us, and how that would affect their assessment of our work.

Reviewer comment (Writing):
The writing lacks sufficient background content. The paper should provide more details on Decision Transformer and offline RL setting.

Author response:
## Q3

>The writing lacks sufficient background content. The paper should provide more details on Decision Transformer and offline RL setting.
    
A: Thank you for the comment. We have added more details about DT and offline RL in the paper revision Section 3.1 lines 141-146.

Reviewer comment (Novelty):
The related work section should include more memory-based Transformer papers.

Author response:
## Q4

>The related work section should include more memory-based Transformer papers
    
A: Thank you for the suggestion. We have added these related works in the paper revision lines 121-125.

Reviewer comment (Writing):
Some explanations in the paper are unclear. For instance, the reference to "Large Language Model based decision making agents" needs clarification. It's unclear if this refers to Transformer-based agents or models within the Decision Transformer family.

Author response:
This phrase generally refers to all the large language model-based agents, which includes both decision transformer, trajectory transformer [1] and the following works. In this paper, we mainly focus on decision transformer family. We’ve revised this sentence in the paper revision line 1.

[1] Offline Reinforcement Learning as One Big Sequence Modeling Problem. [https://arxiv.org/abs/2106.02039v4](https://arxiv.org/abs/2106.02039v4)

Reviewer comment (Theory):
The motivation for this work is not entirely clear. While the paper argues that memorization through neural network parameters can lead to weaker performance in forgetting during training, it remains unclear how this relates to generalization performance for unseen tasks.

Author response:
**Memorization and Forgetting in Neural Networks**: When a neural network is trained on a series of tasks, it tends to "memorize" the specifics of those tasks through its parameters. This process can lead to a phenomenon known as "catastrophic forgetting," where learning new tasks impairs the network's performance on previously learned tasks [1]. This is a significant issue in the context of decision-making models, where adaptability and the ability to handle a variety of scenarios are crucial.

**Generalization to Unseen Tasks**: Generalization refers to a model's ability to perform well on new, unseen tasks, not just the ones it was trained on. The problem with the traditional approach of memorizing through parameters is that it often leads to overfitting on the training data, which can diminish the model's ability to generalize. When a model is overfitted, it becomes too tailored to the specifics of its training data and loses the flexibility needed to adapt to new situations or tasks [2].

**Relation Between Forgetting and Generalization**: The link between the forgetting phenomenon and generalization is that both are impacted by how a model learns and stores information. A model that is prone to forgetting may struggle with generalization because it cannot effectively retain and utilize the broad range of knowledge required to handle new tasks [3]. Conversely, a model designed to minimize forgetting (such as through an internal memory mechanism like DT-Mem) can potentially maintain a more diverse and generalizable knowledge base.

**Motivation of DT-Mem**: The motivation for introducing DT-Mem, as such, is to address these interrelated issues. By incorporating an internal memory module, DT-Mem aims to reduce the reliance on parameter-based memorization, thereby mitigating the effects of catastrophic forgetting. This is hypothesized to improve the model's ability to generalize to new tasks, as it can more effectively retain and apply knowledge from a broader range of experiences. Section 5.3 Table 1 the results of model generalization support this findings.

[1] Overcoming catastrophic forgetting in neural networks. National academy of sciences 2017.

[2] Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.

[3] Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521-3526. 2017

Reviewer comment (Reproducibility):
Is the memory not initialized throughout the entire training process? Clarifying this point could help readers better understand the novelty of this approach, as memory initialization is typically performed per episode (e.g., NTM).

Author response:
The memory is initialized only once during the entire training process. This approach ensures that the knowledge acquired from one task can be leveraged in subsequent tasks, leading to improved pre-training performance, as evidenced in Fig. 6.

Reviewer comment (Experiments):
Could you investigate whether MDT can be fine-tuned using the LoRA technique? As LoRA is applicable to the general Transformer architecture, it would be insightful to assess the potential for fine-tuning MDT using this approach.

Author response:
The results shown in Section 5.5 Figure 5 are all fine-tuned using the LoRA technique. As we concluded in lines 342-351. The results show that “the consistent superior performance of DT-Mem across most games suggests that this method might have a more adaptable approach.”

Reviewer comment (Experiments):
Have you considered testing an external memory-equipped DT? Many recent attempts to incorporate memory utilize a naive appending-style external memory, which lacks the sophistication of models like NTM. Comparing your memory architecture to this version could help highlight its strengths.

Author response:
Thank you for the valuable suggestions. However, we couldn’t find any papers related to external memory-equipped DT. We would love it if the reviewer could provide a bit more information about what are the experiments they would like us to prioritize, what they hope to get from the additional results they ask of us, and how that would affect their assessment of our work.

Reviewer comment (Experiments):
Expanding the tasks to include those with long-term dependencies, such as MemoryMaze, could be valuable. Given that DT-Mem has a memory module capable of retaining distant knowledge, it may excel in tasks where other DTs, except RMDT, struggle.

Author response:
We greatly appreciate your suggestion. We are actively conducting the recommended experiments and will promptly share the results as soon as they become available.

Reviewer comment (Writing):
In line 243, the phrase "the Transformer module to generate actions that can reduce this value as close to zero as possible" may benefit from clarification. The objective appears to be maximization rather than reduction, as it involves the sum of rewards.

Author response:
Thank you for highlighting the confusion regarding this section. We adhere to the conventional 'return-to-go' approach in Decision Transformers, where the return-to-go at time step $t$, denoted as $\hat{r}_t$, represents the total expected reward from that point to the episode's end. The training objective for models like Decision Transformers typically focuses on maximizing the expected return, or equivalently, minimizing the negative return. In our case, the goal is specifically to minimize the **negative return** as much as possible, aiming towards zero. This clarification has been incorporated in the revised manuscript at line 246.

Reviewer comment (Experiments):
Figure 5 raises questions about the absence of NFT results. Are all results in the plot derived from fine-tuned models?

Author response:
Yes, all results in the plot derived from fine-tuned models. The terms 'NFT' and 'top3' were included erroneously and are, as you correctly noted, typographical errors. Thank you for pointing out these issues, we have revised Figure 5 and Figure 6 captions in the paper revision.

Reviewer comment (Presentation):
In Figure 6, where you mention the top 3 rollout, could you provide more context or clarification about what this entails?

Author response:
Yes, all results in the plot derived from fine-tuned models. The terms 'NFT' and 'top3' were included erroneously and are, as you correctly noted, typographical errors. Thank you for pointing out these issues, we have revised Figure 5 and Figure 6 captions in the paper revision.

Reviewer comment (Evaluation):
DT-Mem demonstrates superior performance in pre-training, generalization, and fine-tuning, particularly evident in Figure 10. However, there seems to be a disconnect between the paper's motivation and the observed results. Clarifying the link between implicit memory's ability to mitigate the forgetting phenomenon and the improved generalization and fine-tuning performance would strengthen the paper's alignment and overall impact.

Author response:
**Memorization and Forgetting in Neural Networks**: When a neural network is trained on a series of tasks, it tends to "memorize" the specifics of those tasks through its parameters. This process can lead to a phenomenon known as "catastrophic forgetting," where learning new tasks impairs the network's performance on previously learned tasks [1]. This is a significant issue in the context of decision-making models, where adaptability and the ability to handle a variety of scenarios are crucial.

**Generalization to Unseen Tasks**: Generalization refers to a model's ability to perform well on new, unseen tasks, not just the ones it was trained on. The problem with the traditional approach of memorizing through parameters is that it often leads to overfitting on the training data, which can diminish the model's ability to generalize. When a model is overfitted, it becomes too tailored to the specifics of its training data and loses the flexibility needed to adapt to new situations or tasks [2].

**Relation Between Forgetting and Generalization**: The link between the forgetting phenomenon and generalization is that both are impacted by how a model learns and stores information. A model that is prone to forgetting may struggle with generalization because it cannot effectively retain and utilize the broad range of knowledge required to handle new tasks [3]. Conversely, a model designed to minimize forgetting (such as through an internal memory mechanism like DT-Mem) can potentially maintain a more diverse and generalizable knowledge base.

**Motivation of DT-Mem**: The motivation for introducing DT-Mem, as such, is to address these interrelated issues. By incorporating an internal memory module, DT-Mem aims to reduce the reliance on parameter-based memorization, thereby mitigating the effects of catastrophic forgetting. This is hypothesized to improve the model's ability to generalize to new tasks, as it can more effectively retain and apply knowledge from a broader range of experiences. Section 5.3 Table 1 the results of model generalization support this findings.

[1] Overcoming catastrophic forgetting in neural networks. National academy of sciences 2017.

[2] Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530.

[3] Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521-3526. 2017

Reviewer comment (Experiments):
In Section 3.1, it seems that the focus is on offline RL rather than RL. The two fields have differences, and a brief introduction of offline RL, as well as existing methods as baselines such as DT, MGDT, RMDT, and HDT, may be needed.

Author response:
>For Section 3.1. It seems that you focus on offline RL, rather than RL. The two fields have differences. You may need a brief introduction of offline RL, as well as existing methods as baselines, such as DT, MGDT, RMDT, HDT.  
    
A: We appreciate the insightful suggestion. Accordingly, a concise overview of offline RL has been incorporated into Section 3.1 (lines 141-146). Additionally, due to page constraints, we have briefly introduced the baselines in Section 5.2.

Reviewer comment (Presentation):
Figure 2 is somehow misleading. In Figure 2a, does the memory module constitute a layer of the transformer? The reviewer thinks they are separate, but the plot suggests that the memory is stacked with the transformer. In Figure 2b, the retrieved memory is referred to—this may more accurately be described as retrieved experiences or another term. The terminology should be unique and clear.

Author response:
> Figure 2 is somehow misleading. Figure 2a, does the memory module is a layer of the transformer? I think they are separated, however, the plot seems that the memory is stacked with the transformer. Figure 2b, the retrieved memory is another memory? I think should be the retrieved experiences, or other terms. Please make the terms unique and clear.
    
A: We appreciate the suggestion. To clarify, in Figure 2a, the memory module is depicted as an independent module, distinct from the transformer module. Conversely, in Figure 2b, the memory that is retrieved serves as an intermediate input to the transformer module, integrating with it more directly. We have updated Figure 2 in the revised version of the paper to better illustrate these configurations and their respective operational dynamics.

Reviewer comment (Writing):
The introduction uses large language models as the motivation; however, even when using the GPT-2 architecture, the embeddings and tokens are not about words but should be game-specific tokens. "Transformer" or "decoder-only transformer" should be used instead to avoid confusion.

Author response:
> The introduction uses large language model as the motivation, however, even using GPT-2 architecture, the embedding and the tokens I believe is not about words, it should be game-specific tokens. so please using transformer or decoder-only transformer to avoid any confusion.
    
A: Thank you for the comments. We have revised this term in line 16 and line 20.

Reviewer comment (Experiments):
The largest model used in the paper is 50M parameters. Compared with LLMs, this is still very small. The necessity of using LoRA is questionable, as LoRA can be used with any models and is not a technical contribution of this paper. Additionally, LoRA may harm the performance of DT-Mem. The suggestion is to avoid using LoRA and instead fine-tune the entire model to focus on the memory part, which would help evaluate the importance of the memory.

Author response:
>It seems that the largest model used in the paper is 50M. Compared with LLM, it still very small. Does LoRA is necessary? LoRA can be used to any models, which is not a technical contribution of this paper...
    
A: **Model Size:** The choice of a 50M model size was dictated by the computational resources available to us. Nonetheless, in Figure 3, we explore the scaling law of DT-Mem, which reveals an interesting trend: as the number of parameters increases, so does performance. Importantly, when compared to the 200M MDT model, our DT-Mem, with only a quarter of the parameter size, demonstrates superior performance. This not only underscores the efficiency of DT-Mem in generalizing across games but also empirically supports its effectiveness even with fewer parameters.
    
**Motivation of LoRA**: The motivations of using LoRA to fine-tune the model can be concluded in following two reasons:
    
1. According to a paper \[1\], compared to other adapter methods, LoRA method not only don't introduce inferences latency nor reduce input sequence length while retaining high model quality. It allows for quick task-switching when deployed as a service by sharing the vast majority of the model parameters
2. Parameter-efficient fine-tuning (PEFT) approaches (such as LoRA) only fine-tune a small number of (extra) model parameters while freezing most parameters of the pretrained LLMs, thereby greatly decreasing the computational and storage costs \[1\]. This also overcomes the issues of catastrophic forgetting \[2\]. PEFT approaches have also shown to be better than fine-tuning in the low-data regimes and generalize better to out-of-domain scenarios \[3\].
    
**Full Fine-tuning (FFT) vs. LoRA**: To assess whether the use of LoRA adversely affects performance, we conducted experiments contrasting Full Fine-Tuning (FFT) of memory parameters with LoRA. In this context, FFT-single refers to fine-tuning all parameters exclusively on a single game, whereas FFT-All represents fine-tuning on the entire set of games simultaneously. Results are DQN-normalized score.
    
|Game|LoRA|FFT-Single|FFT-All|
|---|---|---|---|
|Alien|127.4%|116.8%|113.9%|
|MsPacman|130.8%|122.8|77.1%|
|SpaceInvaders|100.8%|86.8%|73.4%|
|StarGunner|158.3%|55.7%|40.6%|

Based on above results, we conclude the following observations:

- LoRA appears to be the most consistently effective strategy across the games provided.
- While **FFT-Single** occasionally outperforms PEFT (like in Alien), **FFT-All** consistently trails behind the other two.

The reason full fine-tuning is not comparable to PEFT comes from the following parts: 1. Fine-tuning dataset size. Note that we only use 50k data in LoRA and full fine-tuning compares on 500k used in MDT paper 2. The benefits of LoRA listed above: "This approach also addresses catastrophic forgetting [2] and has outperformed standard fine-tuning in low-data and out-of-domain situations [3]”

[1] LoRA: Low-Rank Adaptation of Large Language Models, ICLR 2022
[2] Adapter-Fusion: Non-destructive task composition for transfer learning.EACL, 2021.
[3] Prefix-tuning: Optimizing continuous prompts for generation. ACL, 2021.

Reviewer comment (Novelty):
Regarding memory, there are related methods such as neural episodic control (https://arxiv.org/abs/1703.01988) for writing and lookup, which share many similarities with the proposed method. A detailed comparison between the proposed memory and all related methods should be added so that the contributions are fully understood.

Author response:
>About the memory. There are some related methods, neural episodic control ([https://arxiv.org/abs/1703.01988](https://arxiv.org/abs/1703.01988)) for the writing and lookup. The two methods share many similarities, so maybe add a detailed comparison of the proposed methods and all related methods, so we can fully understand the contributions.
    
A: Thank you for the comments. Due to the page limit, we have added these comparisons in the paper revision Appendix Section E

Reviewer comment (Writing):
There is a need for clarification on the exact difference between external memory and internal memory. An example of the two types of memories, as well as the advantages of the proposed memory, would be helpful.

Author response:
>Still about the memory. What is exactly the difference between the external memory and the internal memory? Could you provide an example about the two kinds of memories, as well as the advantages of the proposed memory.
    
### External Memory vs. Internal Memory

1. **External Memory**:
	- **Definition**: External memory in AI and machine learning models refers to a separate storage unit that is not part of the core neural network architecture. It's like an additional database or repository that the model can access.
	- **Example**: A neural Turing machine (NTM) or Differentiable Neural Computer (DNC), which uses an external memory matrix that it can read from and write to, separate from its neural network parameters.
	- **Advantages**:
		- **Scalability**: Can store large amounts of data beyond the capacity of the model's parameters.
		- **Flexibility**: Allows the model to store and retrieve information in a more structured way, similar to how a computer uses RAM and disk storage.
2. **Internal Memory**:
	- **Definition**: Internal memory refers to the capacity of a neural network to store information within its own architecture, such as in its weights and activations. It does not use a separate storage unit but relies on the network's inherent structure.
	- **Example**: LSTM (Long Short-Term Memory) networks that use their recurrent connections to store information over time within the network.
	- **Advantages**:
		- **Efficiency**: More efficient in terms of retrieval speed, as the information is stored within the network.
		- **Integration**: Better integrated with the model's learning process, enabling more seamless information flow and processing.

### Advantages of the Proposed Memory

1. **Contextual Retrieval**: Utilizing mechanisms like attention to retrieve and store information more contextually and relevantly.
2. **Enhanced Generalization**: By having a more flexible memory system, the model generalize better to new, unseen tasks.

Reviewer comment (Experiments):
In the experiments, while they are generally sufficient, the reviewer has suggestions: (i) Can prompting be used for DT-Mem, as prompting is easier than fine-tuning? (ii) What is the limit of the internal memory given the fixed size, i.e., parameters, of the memory?

Author response:
> Some issues about the experiments. I generally think the experiments are sufficient, but with some suggestions: i) can we use the prompting for the DT-Mem? as we know prompting is much easier than fine-tuning.
    
A: Thank you for the suggestion. The primary objective of the fine-tuning in Section 5.5 is to investigate whether modifications to the memory module can boost the performance of DT-Mem without altering other transformer parameters. This is intended to assess the role of the memory module's stored information in aiding decision-making for new tasks. Since the memory module engages in read and write operations through cross-attention, the use of prompt tuning, which does not alter DT-Mem's stored information, is not aligned with our approach.

Nevertheless, we have conducted a comparative analysis of prompt-tuning DT (PDT) and DT-Mem in Table 2. The results demonstrate the effectiveness of our proposed method and highlight the advantages of fine-tuning the memory module.
    

> And ii) what is the limit of the internal memory, given the fixed size, i.e., parameters, of the memory?
    
A: The major limitation of internal memory is **Capacity Limitation**: The capacity of internal memory is inherently limited by the size of the neural network, i.e., the number of its parameters and the architecture's complexity. Once these parameters are set, the capacity to store and recall information is fixed. The ability to store information depends on how densely information can be encoded within the network's parameters, which has practical limits.

Reviewer comment (Presentation):
Fig. 2(b) is too simple, making it difficult to correspond one-to-one with the steps in the method part. The authors should make the figure more comprehensive and understandable.

Author response:
Thank you for the comments. We have revised this figure in the paper revision Fig. 2

Reviewer comment (Novelty):
There are many papers demonstrating the ideas about internal memory, and the authors should explain the differences with similar methods in more detail in the method section.

Author response:
Thank you for pointing this out. To the best of my knowledge, the work most closely related to ours (DT with internal memory) is RMDT, which we have discussed extensively in the paper. I would greatly appreciate it if the reviewer could share specific papers they believe should be compared to our work. We are more than willing to discuss these differences in the revised version of the paper.

Reviewer comment (Experiments):
Add more analysis about different situations, such as the input misleading by the content stored in the memory (i.e., noise or dissimilar pattern), how does the method eliminates this type of impact.

Author response:
We thank the reviewer for their valuable suggestion. Following this, we conducted an experiment to assess the robustness of the proposed method against input distortion. This involved adding Gaussian noise to the input frames of Atari games. Specifically, we set the mean to 0 and experimented with various standard deviation values. The results are detailed in the table below:
    
|  | Alien | MsPacman | SpaceInvaders | StarGunner |
| --- | --- | --- | --- | --- |
| MDT | 3.8% | 13.2% | 8.6% | 2.3% |
| DT-Mem | 51.0% | 69.3% | 53.6% | 62.2% |
| DT-Mem (std=0.5) | 55.3% | 67.6% | 53.0% | 57.8% |
| DT-Mem (std=1) | 35.6% | 56.1% | 40.0% | 34.6% |
| DT-Mem (std=2) | 25.9% | 35.6% | 30.5% | 21.1% |
    
From the results above, we conclude that the proposed DT-Mem demonstrates greater robustness to noisy inputs compared to the MDT method. This is evident as the DT-Mem consistently outperforms MDT under various levels of Gaussian noise. Notably, the performance with a standard deviation of 0.5 shows minimal difference compared to the no-noise scenario, illustrating DT-Mem's effectiveness in mitigating the impact of varying input distortions.