I ran into a ai agent research paper that I thought was very interesting. I had some ideas similar to this and I would like to explore these concepts with you and potentially try building a system that replicates these techniques. The paper is titled "MSI-Agent: Incorporating Multi-Scale Insight into Embodied Agents for Superior Planning and Decision-Making" and it attempts to address the problem of long term memory and learning in embodied agents. every action the agent takes is considerred an experience (containing: task, insights, plan, observations and outcomes) after succeeding or failing at the task and finishing the experience. the information is processed and the system updates its insights. insights are rules that the system discovers and remembers over time through the observations and outcomes of experiences. Insights can be: general, environmental (just for a specific environment) and task (for a specific task). I am very interested in how this technique could be applied to non embodied agents.

PAPER METHOD:

Method
Figures 2 and 3 illustrate our approach. Initially,
utilizing historical task data (train set), we employ
the task execution module to collect a sufficient
number of experiences. (§3.1) These experiences
are then subjected to the experience selector, which
identifies experiences/experience pairs suitable for
generating insights. (§3.2) Subsequently, the multiscale insights will be generated and stored in the
insight database. (§3.3) When a new task arises, we
retrieve relevant sights from the database based on
predefined rules. (§3.4) These insights, along with
task background, and user queries, are provided to
the task execution module to facilitate execution.
We refer to the process from experience collection
to insight generation as insight summarization, and
the subsequent insight selection and task execution
as insight utilization.
3.1 Experience Generation
As shown in Figure 2, we regard training data as
history tasks. For each history task, the executor leverages LLM to generate a plan based on
task background and user queries. Subsequently,
the robot employs first-order logic to decompose
the plan into atomic actions (e.g., moving forward,
picking up objects) and execute them in an environment. In some tasks or cases, the executor may
replan based on the environment feedback. Upon
completion, task background, user queries, agent’s
plans, environmental feedback, and execution results are stored as experiences for summarization.
Detailed information can be found in Appendix A.
3.2 Experience Selection
The selection of experiences is crucial in summarizing insights, as it determines the quality of insights
the model consolidates. As shown in Figure 3, our
Experience Selection employs two modes:
Success mode: We select experiences with successful execution results as the success mode experiences.
Pair mode: For each successful experience ss,
we identify a corresponding experience sf from the
unsuccessful experience database Sf by:
sf = argmax
s∈Sf
emb(s) · emb(ss)
p
||emb(s)||2||emb(ss)||2
(1)
Where emb is the embedding of the experience’s
user query and the (ss, sf ) is the final experience
pair in the pair mode.
These two types of selected experience (pair)
collections are subsequently preserved and utilized
as seed experience for insight summarization.
3.3 Multi-Scale Insight Generation
Multi-Scale Insight We categorize the insights
into several scales. For all tasks, we will generate general scale and subtask scale insights. If
the task provides a specific environment category
(for example, kitchen), we will also generate environment scale insights. General insight refers to
the knowledge required for all tasks, which should
be high-level. Environment insight pertains to the
knowledge needed in a specific environment, and
subtask insight involves the understanding of executing particular subtasks. The overall pipeline can
be seen in Figure 3’s Insight Generation module.
Insight Generation We initialize the insight
database to be empty. Whenever a seed experience merges, we select all insights in the order of
general, subtask.2
as a pool of candidate experience
for updating.
Subsequently, we prompt the LLM with templates containing the candidate insight, all experience information, and descriptions of 5 atomic
actions: adding, removing, editing, agreeing on
an insight, and moving an insight between scales,
requesting the LLM to update the insight database
through these atomic actions (Zhao et al., 2023).
For subtask insight, we also require the LLM to additionally generate a subtask name corresponding
to the insights. 3
After the LLM generation is complete, we update the insight database in the order of general,
environment (if have), and subtask, according to
the atomic actions.
Align with Expel, we also employ a scoring
mechanism in insight generation. Specifically, each
2
If there is a specific environment category in the task, we
will select environment and subtask insight that is consistent
with the experience’s environment category, and the order is
general, environment, and subtask
3The prompt of Insight Generation can be seen in Appendix C
insight receives an initial score of 2 when an "add"
or "move" action is executed, the score increases by
1 for an "agree" action, remains unchanged for an
"edit" action, and decreases by 1 for a "remove" action. An insight is discarded when its score reaches
zero.
3.4 Multi-Scale Insight Selection
Similar to the generation process, we use general
and subtask insights2
as candidate insights. For
subtask insights, we adopt two modes for further
selection:
Hashmap indexing: We extract all subtask
names from the subtask insight database, combine
them with user queries, and provide them to the
LLM, requiring the LLM to return all task names
related to the user query. Subsequently, we consider all insights under returned subtask names as
the subtask insights for this user query. The prompt
of hashmap subtask selection can be seen in Appendix D
Vector indexing: We compute the cosine similarity between all subtask insights and the user
query, selecting insights with at most 2000 tokens.4
Ultimately, we provide the different scales of
insights, and the user query to the task execution
module to accomplish the task.