Original: C ONTRIBUTION OF INTERNAL REFLECTION IN LANGUAGE EMERGENCE WITH AN UNDERRESTRICTED SITUATION
Revised: C ONTRIBUTION OF INTERNAL REFLECTION IN LANGUAGE EMERGENCE WITH AN UNDERRESTRICTED SITUATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Owing to language emergence, human beings have been able to understand the intentions of others, generate common concepts, and extend new concepts.
Revised: Owing to language emergence, human beings have been able to understand the intentions of others, generate common concepts, and extend new concepts.

Original: Artiﬁcial intelligence researchers have not only predicted words and sentences statistically in machine learning, but also created a language system by communicating with the machine itself.
Revised: Artiﬁcial intelligence researchers have not only predicted words and sentences statistically in machine learning, but also created a language system by communicating with the machine itself.

Original: However, strong constraints are exhibited in current studies (supervisor signals and rewards exist, or the concepts were ﬁxed on only a point), thus hindering the emergence of real-world languages.
Revised: However, strong constraints are exhibited in current studies (supervisor signals and rewards exist, or the concepts were ﬁxed on only a point), thus hindering the emergence of real-world languages.

Original: In this study, we improved
Revised: In this study, we improved

Original: Batali (1998) and Choi et al.
Revised: Batali (1998) and Choi et al.

Original: (2018)’s research and attempted language emergence under conditions of low constraints such as human language generation.
Revised: (2018)’s research and attempted language emergence under conditions of low constraints such as human language generation.

Original: We included the bias that exists in humans as an “internal reﬂection function” into the system.
Revised: We included the bias that exists in humans as an “internal reﬂection function” into the system.

Original: Irrespective of function, messages corresponding to the label could be generated.
Revised: Irrespective of function, messages corresponding to the label could be generated.

Original: However, through qualitative and quantitative analysis, we conﬁrmed that the internal reﬂection function caused “overlearning” and different structuring of message patterns.
Revised: However, through qualitative and quantitative analysis, we conﬁrmed that the internal reﬂection function caused “overlearning” and different structuring of message patterns.

Original: This result suggested that the internal reﬂection function performed effectively in creating a grounding language from raw images with an under-restricted situation such as human language generation.
Revised: This result suggested that the internal reﬂection function performed effectively in creating a grounding language from raw images with an under-restricted situation such as human language generation.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: For artiﬁcial intelligence studies, it is important to build machines that can handle words and concepts (Bruner, 1981; Mikolov et al., 2016).
Revised: For artiﬁcial intelligence studies, it is important to build machines that can handle words and concepts (Bruner, 1981; Mikolov et al., 2016).

Original: Deep learning, especially the attention mechanism, has improved the success of various natural language processing tasks (Xu et al., 2018; Das et al., 2018;
Revised: Deep learning, especially the attention mechanism, has improved the success of various natural language processing tasks (Xu et al., 2018; Das et al., 2018;

Original: Mao et al., 2019; Das et al., 2017a;
Revised: Mao et al., 2019; Das et al., 2017a;

Original: Huang et al., 2016)
Revised: Huang et al., 2016)

Original: Furthermore, studies on multi-modal information are progressing.
Revised: Furthermore, studies on multi-modal information are progressing.

Original: Supervised learning based on large datasets regards languages as statistical pattern processing with discrete symbols.
Revised: Supervised learning based on large datasets regards languages as statistical pattern processing with discrete symbols.

Original: Humans, as well as animals, have emerged information and concepts through the exchange of languages (Lupyan & Bergen, 2016; Lupyan & Clark, 2015).
Revised: Humans, as well as animals, have emerged information and concepts through the exchange of languages (Lupyan & Bergen, 2016; Lupyan & Clark, 2015).

Original: Languages enable the understanding of the intentions of others and a socially common system to be built (Bloom, 2002; Frank et al., 2009; Tomasello, 2010).
Revised: Languages enable the understanding of the intentions of others and a socially common system to be built (Bloom, 2002; Frank et al., 2009; Tomasello, 2010).

Original: In addition, languages can be used to recursively construct concepts and create new ones (Berwick & Chomsky, 2016; Hauser et al., 2002).
Revised: In addition, languages can be used to recursively construct concepts and create new ones (Berwick & Chomsky, 2016; Hauser et al., 2002).

Original: Language intelligence involves more than statistically predicting words and sentences.
Revised: Language intelligence involves more than statistically predicting words and sentences.

Original: It is important for the intelligence to formulate and systematize a language from contexts and situations.
Revised: It is important for the intelligence to formulate and systematize a language from contexts and situations.

Original: Introducing the language system using the current datasets has been difﬁcult in machine learning (Lake et al., 2017; Niven & Kao, 2019; Dua et al., 2019).
Revised: Introducing the language system using the current datasets has been difﬁcult in machine learning (Lake et al., 2017; Niven & Kao, 2019; Dua et al., 2019).

Original: Researchers have attempted to build agents to create the language system by communicating with each agent instead of language processing using statistical information (Steels, 2003; Mikolov et al., 2016; Rahwan et al., 2019).
Revised: Researchers have attempted to build agents to create the language system by communicating with each agent instead of language processing using statistical information (Steels, 2003; Mikolov et al., 2016; Rahwan et al., 2019).

Original: Complex tasks could be solved using these generated messages.
Revised: Complex tasks could be solved using these generated messages.

Original: Formal rules and messages that emerge owing to this communication have a system that differs from that of a human.
Revised: Formal rules and messages that emerge owing to this communication have a system that differs from that of a human.

Original: However, current language emergence studies have not achieved the autonomous grounding of mental images as those of humans and animals (Kottur et al., 2017; Bouchacourt & Baroni, 2018).
Revised: However, current language emergence studies have not achieved the autonomous grounding of mental images as those of humans and animals (Kottur et al., 2017; Bouchacourt & Baroni, 2018).

Original: Two issues remain: (1) The correspondence of generated messages to concepts is mediated by the architecture.
Revised: Two issues remain: (1) The correspondence of generated messages to concepts is mediated by the architecture.

Original: Messages cannot be directly grounded to concepts.
Revised: Messages cannot be directly grounded to concepts.

Original: In some cases, a meaningless message was generated, and in other cases, a meaningful message was assigned to random noise images.
Revised: In some cases, a meaningless message was generated, and in other cases, a meaningful message was assigned to random noise images.

Original: In current studies, the “supervisor” role is essential because agents are rewarded for acquiring the language system when communication is established.
Revised: In current studies, the “ supervisor ” role is essential because agents are rewarded for acquiring the language system when communication is established.

Original: The supervisor method provided implicit answers such as “direction of correct communication for task resolutions” and “direction of a correct language system corresponding to a concept.”
Revised: The supervisor method provided implicit answers such as “ direction of correct communication for task resolutions ” and “ direction of a correct language system corresponding to a concept. ”

Original: Few studies have focused on language systems that emerge from prior knowledge regarding pure machines.
Revised: Few studies have focused on language systems that emerge from prior knowledge regarding pure machines.

Original: These issues appear in language emergence models, which are different from the emergence and learning of human languages.
Revised: These issues appear in language emergence models, which are different from the emergence and learning of human languages.

Original: Humans have prior knowledge of the concept and corresponding grounding messages (Imai & Masuda, 2013; Milligan et al., 2007).
Revised: Humans have prior knowledge of the concept and corresponding grounding messages (Imai & Masuda, 2013; Milligan et al., 2007).

Original: They can create language systems independently from the supervisor (Senghas et al., 2004).
Revised: They can create language systems independently from the supervisor (Senghas et al., 2004).

Original: Little has been reported on language emergence under these conditions, in which the agent possesses individual prior knowledge and is independent of the supervisor.
Revised: Little has been reported on language emergence under these conditions, in which the agent possesses individual prior knowledge and is independent of the supervisor.

Original: The purpose of this study is to conﬁrm a compositional language emergence method, in which machines autonomously ground concepts to each agent in an almost human-like manner.
Revised: The purpose of this study is to conﬁrm a compositional language emergence method, in which machines autonomously ground concepts to each agent in an almost human-like manner.

Original: We developed a method and architecture that can generate a language from discrete symbols.
Revised: We developed a method and architecture that can generate a language from discrete symbols.

Original: In a human-like manner, agents must autonomously generate messages corresponding to cluster images that are not ﬁxed at one point.
Revised: In a human-like manner, agents must autonomously generate messages corresponding to cluster images that are not ﬁxed at one point.

Original: To create formal rules within these under-restricted situations, we introduce the human cognitive bias (Brennan & Clark, 1996; Lake et al., 2019).
Revised: To create formal rules within these under-restricted situations, we introduce the human cognitive bias (Brennan & Clark, 1996; Lake et al., 2019).

Original: Using the generated messages and the reconstructed images, we compared the characteristics of the agreed formal rules by bias and discovered the emergent parameters necessary for the rules.
Revised: Using the generated messages and the reconstructed images, we compared the characteristics of the agreed formal rules by bias and discovered the emergent parameters necessary for the rules.

Original:  RELATED WORK
Revised:  RELATED WORK

Original: Artiﬁcial life studies of language emergence have discovered the dynamism of the generated language and the condition under which language systematization and formality has emerged (Steels, 2000; Martin & White, 2003; Nolﬁ & Mirolli, 2009).
Revised: Artiﬁcial life studies of language emergence have discovered the dynamism of the generated language and the condition under which language systematization and formality has emerged (Steels, 2000; Martin & White, 2003; Nolﬁ & Mirolli, 2009).

Original: Recently, a neural network model has been used to generate a complex language using reinforcement learning or supervised learning.
Revised: Recently, a neural network model has been used to generate a complex language using reinforcement learning or supervised learning.

Original: It is classiﬁed into (1) the task solution type and (2) the concept correspondence type.
Revised: It is classiﬁed into (1) the task solution type and (2) the concept correspondence type.

Original: In the task solution type, multiple agents are prepared, and the tasks are solved by communicating each other’s situations and actions.
Revised: In the task solution type, multiple agents are prepared, and the tasks are solved by communicating each other’s situations and actions.

Original: The agents materialize words (instructions) to generate appropriate actions (Mordatch & Abbeel, 2018;
Revised: The agents materialize words (instructions) to generate appropriate actions (Mordatch & Abbeel, 2018;

Original: Foerster et al., 2016).
Revised: Foerster et al., 2016).

Original: Additionally, the tasks are solved through combinations of agent communication by generating answers to speciﬁc questions, such as VQA methods (Das et al., 2017b; Evtimova et al., 2018; Sukhbaatar et al., 2016).
Revised: Additionally, the tasks are solved through combinations of agent communication by generating answers to speciﬁc questions, such as VQA methods (Das et al., 2017b; Evtimova et al., 2018; Sukhbaatar et al., 2016).

Original: The concept correspondence type generate messages corresponding to objects recognized through a reinforcement learning(RL)-based approach (Havrylov & Titov, 2017; Lazaridou et al., 2016; Cao et al., 2018; Lazaridou et al., 2018).
Revised: The concept correspondence type generate messages corresponding to objects recognized through a reinforcement learning(RL)-based approach (Havrylov & Titov, 2017; Lazaridou et al., 2016; Cao et al., 2018; Lazaridou et al., 2018).

Original: This approach comprises different modules for generating and learning messages.
Revised: This approach comprises different modules for generating and learning messages.

Original: It converges on a common language because the reward is transmitted to each module.
Revised: It converges on a common language because the reward is transmitted to each module.

Original: However, the RL-based approach suffers from internal consistency (Choi et al., 2018; Kottur et al., 2017; Bouchacourt & Baroni, 2018; Lowe et al., 2019).
Revised: However, the RL-based approach suffers from internal consistency (Choi et al., 2018; Kottur et al., 2017; Bouchacourt & Baroni, 2018; Lowe et al., 2019).

Original: Choi et al.
Revised: Choi et al.

Original: (2018) proposed the obverter technique to generate compositional languages.
Revised: (2018) proposed the obverter technique to generate compositional languages.

Original: The obverter technique, which was proposed by Batali (1998), can solve “the problem of internal consistency in languages” in the RL-based approach and enables language emergence based on “the theory of mind (Premack & Woodruff, 1978).”
Revised: The obverter technique, which was proposed by Batali (1998), can solve “the problem of internal consistency in languages” in the RL-based approach and enables language emergence based on “the theory of mind (Premack & Woodruff, 1978).”

Original: However, in Batali’s study, experiments were conducted under strong constraints that allowed the corresponding concepts to be uniquely determined.
Revised: However, in Batali’s study, experiments were conducted under strong constraints that allowed the corresponding concepts to be uniquely determined.

Original: Language emergence under conditions without disentanglement information has not been studied.
Revised: Language emergence under conditions without disentanglement information has not been studied.

Original: In Choi et al’s study, supervised signal was required for generation.
Revised: In Choi et al’s study, supervised signal was required for generation.

Original: Our study is inspired by the studies of Batali (1998) and Choi et al.
Revised: Our study is inspired by the studies of Batali (1998) and Choi et al.

Original: In this study, we extend the abovementioned studies to conceptually generate corresponding massages with formal rules without a human supervisor.
Revised: In this study, we extend the abovementioned studies to conceptually generate corresponding massages with formal rules without a human supervisor.

Original: The formation process and concept correspondence of human communication are useful as reference.
Revised: The formation process and concept correspondence of human communication are useful as reference.

Original: Human language acquisition does not involve a supervisor in generating new words .
Revised: Human language acquisition does not involve a supervisor in generating new words .

Original: Furthermore, it is not characterized by not learning concepts and words simultaneously (Imai & Masuda, 2013; Milligan et al., 2007).
Revised: Furthermore, it is not characterized by not learning concepts and words simultaneously (Imai & Masuda, 2013; Milligan et al., 2007).

Original: The emergence of sign languages in
Revised: The emergence of sign languages in

Original: Africa (Senghas et al., 2004) and the formation of languages such as the Pidgin and Creole languages demonstrate the unsupervised and asynchronous nature in language systems.
Revised: Africa (Senghas et al., 2004) and the formation of languages such as the Pidgin and Creole languages demonstrate the unsupervised and asynchronous nature in language systems.

Original: Additionally, it is known that animals acquire different formalities depending on the species (Spierings & ten Cate, 2016).
Revised: Additionally, it is known that animals acquire different formalities depending on the species (Spierings & ten Cate, 2016).

Original:  M ETHOD 3. MULTI-AGENTS DESCRIPTION GAME
Revised:  M ETHOD 3. MULTI-AGENTS DESCRIPTION GAME

Original: The description game consists of a teacher agent that generates messages and a student agent that receives and learns the messages (Lazaridou et al., 2016; Choi et al., 2018).
Revised: The description game consists of a teacher agent that generates messages and a student agent that receives and learns the messages (Lazaridou et al., 2016; Choi et al., 2018).

Original: The student receives the messages from the teacher and associates them with the presented image.
Revised: The student receives the messages from the teacher and associates them with the presented image.

Original: Repeating this process creates a language with concepts and message shared between agents.
Revised: Repeating this process creates a language with concepts and message shared between agents.

Original: We herein propose the multi-agent description game to establish a language system for multiple agents without a human supervisor.
Revised: We herein propose the multi-agent description game to establish a language system for multiple agents without a human supervisor.

Original: The game proceeds as follows for each epoch.
Revised: The game proceeds as follows for each epoch.

Original: (1) One teacher is randomly selected from all agents.
Revised: (1) One teacher is randomly selected from all agents.

Original: The teacher agent randomly selects one image from a list and generates the corresponding messages.
Revised: The teacher agent randomly selects one image from a list and generates the corresponding messages.

Original: The teacher agent sends the image and message to the student agent, and the student agent associated the message with the image.
Revised: The teacher agent sends the image and message to the student agent, and the student agent associated the message with the image.

Original: (4) (1) to (3) are performed a speciﬁed number of times.
Revised: (4) (1) to (3) are performed a speciﬁed number of times.

Original: One hundred agents are prepared.
Revised: One hundred agents are prepared.

Original: No image labels are presented between in-game agents.
Revised: No image labels are presented between in-game agents.

Original: Therefore, the agent must associate the message with the representation read from the image.
Revised: Therefore, the agent must associate the message with the representation read from the image.

Original: There is no guarantee that a message will completely correspond to a label 1 .
Revised: There is no guarantee that a message will completely correspond to a label 1 .

Original: The image object used in this study was the MNIST database.
Revised: The image object used in this study was the MNIST database.

Original: A training dataset was used during learning and generation, and a test dataset was used when evaluating the messages.
Revised: A training dataset was used during learning and generation, and a test dataset was used when evaluating the messages.

Original: 3. MODEL ARCHITECTURE AND OBVERTER TECHNIQUE
Revised: 3. MODEL ARCHITECTURE AND OBVERTER TECHNIQUE

Original: We herein propose an agent architecture for the multi-agent description game.
Revised: We herein propose an agent architecture for the multi-agent description game.

Original: An agent comprises a language module for reading and generating a message, and a visual module for processing an image and associating the image with the messages.
Revised: An agent comprises a language module for reading and generating a message, and a visual module for processing an image and associating the image with the messages.

Original: Each agent cannot exchange information other than messages and images.
Revised: Each agent cannot exchange information other than messages and images.

Original: Figure 1 shows the conﬁguration of the agent module.
Revised: Figure 1 shows the conﬁguration of the agent module.

Original: The visual module uses a variational autoencoder (VAE) (Kingma & Welling, 2013) and the language module uses a gated recurrent unit (GRU) (Cho et al., 2014).
Revised: The visual module uses a variational autoencoder (VAE) (Kingma & Welling, 2013) and the language module uses a gated recurrent unit (GRU) (Cho et al., 2014).

Original: The visual module has prior knowledge of the MNIST.
Revised: The visual module has prior knowledge of the MNIST.

Original: The language module outputs a numerical value corresponding to the latent space by inputting an arbitrary message that has been converted into a one-hot vector.
Revised: The language module outputs a numerical value corresponding to the latent space by inputting an arbitrary message that has been converted into a one-hot vector.

Original: This structure enables the generation of messages corresponding to the image and reconstruction of a
Revised: This structure enables the generation of messages corresponding to the image and reconstruction of a

Original: Dream(reconstructed image) (Ha & Schmidhuber, 2018) that corresponds to the messages. 
Revised: Dream (Ha & Schmidhuber, 2018) that corresponds to the messages. Dream is deﬁned as an output image reconstructed by a message It not an output image

Revised: VAE input

Revised: (the output image is expressed as Reconstructed Image).

Original: The details of the model architecture are described in Appendix A.
Revised: The details of the model architecture are described in Appendix A.

Original: This module architecture differs from the preexisting architecture in terms of three characteristics: (1) A language system can emerge by a pure machine, excluding the human viewpoint.
Revised: This module architecture differs from the preexisting architecture in terms of three characteristics: (1) The proposed architecture enables language emergence by agent interaction without using human-prepared supervised signals and rewards.

Original: The concept corresponding to the generated message can be directly conﬁrmed.
Revised: The concept corresponding to the generated message can be directly conﬁrmed.

Original: The structure, characteristics, and limitations ofthe messages generated by the machine can be determined.
Revised: (3) We can explore the structure and characteristics of messages organized by agent interaction.

Original: An agent learns and generates a language using the obverter technique (Batali, 1998).
Revised: An agent learns and generates a language using the obverter technique (Batali, 1998).

Original: In the obverter technique, at the time of generation, assuming that a teacher has a mind similar to that of a student, the teacher generates a message that maximizes his own understanding.
Revised: In the obverter technique, at the time of generation, assuming that a teacher has a mind similar to that of a student, the teacher generates a message that maximizes his own understanding.

Original: In the teacher agent, the model parameters are ﬁxed.
Revised: In the teacher agent, the model parameters are ﬁxed.

Original: When a target image is input, the visual module outputs a latent space value z corresponding to the image.
Revised: When a target image is input, the visual module outputs a latent space value z corresponding to the image.

Original: After initializing the GRU hidden layer, the language module evaluates the output of all possible symbols.
Revised: After initializing the GRU hidden layer, the language module evaluates the output of all possible symbols.

Original: During this evaluation, the symbol with the minimum value between latent space z and the output of the language module is selected.
Revised: During this evaluation, the symbol with the minimum value between latent space z and the output of the language module is selected.

Original: The GRU hidden vector derived by the selected symbol is used by the next GRU unit.
Revised: The GRU hidden vector derived by the selected symbol is used by the next GRU unit.

Original: This procedure is repeated until the minimal value between z and the output value of the language module is less than a predeﬁned threshold or until the max message length is attained.
Revised: This procedure is repeated until the minimal value between z and the output value of the language module is less than a predeﬁned threshold or until the max message length is attained.

Original: In the student agent, the GRU model parameters are updated.
Revised: In the student agent, the GRU model parameters are updated.

Original: The parameters are updated by backpropagating the mean square error loss between the output of the GRU unit and the latent space z.
Revised: The parameters are updated by backpropagating the mean square error loss between the output of the GRU unit and the latent space z.

Original: These algorithms are described in the
Revised: These algorithms are described in the

Original: Appendix B.
Revised: Appendix B.

Original: The abovementioned procedure corresponds to self-supervised learning from the viewpoint of one agent.
Revised: The abovementioned procedure corresponds to self-supervised learning from the viewpoint of one agent.

Original: However, the agents as awhole have a movement to create a system in which languages is unsupervised and one conceptual pact is formed.
Revised: An agent set takes an emergent behavior that forms a conceptual pact without supervised signals.

Original: 3. INTERNAL REFLECTION FUNCTION
Revised: 3. INTERNAL REFLECTION FUNCTION

Original: We introduced cognitive bias to the agents.
Revised: We introduced cognitive bias to the agents.

Original: In human conversations, not all words reach another human, and in some cases, they are rejected by composing them with one’s own concept (Brennan & Clark, 1996;
Revised: In human conversations, not all words reach another human, and in some cases, they are rejected by composing them with one’s own concept (Brennan & Clark, 1996;

Original: Lake et al., 2019) .
Revised: Lake et al., 2019) .

Original: This function means that the student compares the messages generated by the teacher.
Revised: This function means that the student compares the messages generated by the teacher.

Original: The student agent rejects the message if it is signiﬁcantly different from the self-concept.
Revised: The student agent rejects the message if it is signiﬁcantly different from the self-concept.

Original: In this study, we incorporated this function as the “internal reﬂection function” into the agents.
Revised: In this study, we incorporated this function as the “internal reﬂection function” into the agents.

Revised: The student agent performs the learning so that the received message corresponds to the image.

Original: Thealgorithm of the internal reﬂection functionwas incorporated in the learning algorithm of the student agent.
Revised: The internal reﬂection function algorithm was built into the student agent’s learning.

Original: (1) Receive an image and generate a message based on self-knowledge.
Revised: (1) The student agent receives an image that the teacher agent used to generate a message.

Original: (2) Compare the self-generated message with thereceived message  from the teacher agent.
Revised: Using the image, each student agent generates a message by the obverter technique shown in § 3.2. (2) Each student agent calculates a degree of comparison similarity between the message received from the teacher agent and the self-generated message using gestalt pattern matching (Ratcliff & Metzener, 1988). The degree of comparison similarity by gestalt pattern matching is output in the range of 0-1. The value of 1 means an exact match.

Original: (3-A)
Revised: (3-A)

Original: When the  comparison result is lower than the threshold value, the received message is regarded as an error and learning is not performed.(3-B) When the  comparison result is higher than the threshold value, the received message is regarded as correct and learning is performed.
Revised: When the degree of comparison similarity is lower than the threshold value, the received message is regarded as an error and learning is not performed. (3-B) When the degree of comparison similarity is higher than the threshold value, the received message is regarded as correct and learning is performed.

Original: The Gestalt approach was used to calculate the similarity of each message.

Original: In this experiment, three objects were prepared for comparison.
Revised: In this experiment, three objects were prepared for comparison.

Original: The ﬁrst object was NORMAL :no internal reﬂection function.
Revised: The ﬁrst object was NORMAL : internal reﬂection function.

Original: The second object was INTERNAL : an internal reﬂection function was incorporated from the beginning of learning.
Revised: The second object was INTERNAL : an internal reﬂection function was incorporated from the beginning of learning.

Original: Thesimilarity threshold  was set to 0.4.
Revised: The threshold degree of comparison similarity was set to 0.4.

Original: We tested each learning object ﬁve times and evaluated the results.
Revised: We tested each learning object ﬁve times and evaluated the results.

Original: 3. EVALUATION METHODS
Revised: 3. EVALUATION METHODS

Original: We analyzed the language generation characteristics of each learning method.
Revised: We analyzed the language generation characteristics of each learning method.

Original: In particular, we investigated the correct answer rate of Dream, which the agent reconstructed from the messages, and the systematic formal rules of the messages.
Revised: In particular, we investigated the correct answer rate of Dream, which the agent reconstructed from the messages, and the systematic formal rules of the messages.

Original: These points are necessary for the compositional properties of the emergence language.
Revised: These points are necessary for the compositional properties of the emergence language.

Original: The ﬁrst analysis was to conﬁrm that learning has converged.
Revised: The ﬁrst analysis was to conﬁrm that learning has converged and to examine the similarity of messages between labels.

Original: We veriﬁed the mean square error loss between the output of the GRU unit and the latent space 
Revised: We veriﬁed the mean square error loss between the output of the GRU unit and the latent space z.

Original: z. Jaccard coefﬁcient were applied to test the message similarity between labels. We compared the coefﬁcients of the same label and different labels.
Revised: We compared the similarity of the same inter-label and different inter-label by Jaccard similarity coefﬁcient.

Original: For each agent set, 100 images were presented andcorresponding messages were generated.
Revised: For each agent, 100 images were presented and messages were generated by obverter technique.

Original: Each agent set was tested thrice and the average was calculated.
Revised: The messages for each label are compared. We calculate the Jaccard similarity coefﬁcient between labels for all agent combinations. Since there are 100 agents in an agent set, a 100 x 100 Jaccard similarity coefﬁcient matrix is generated for each inter-label.

Original: The  value was compared for each learning object.
Revised: The matrix represents the similarity between labels. Then, to derive the similarity between labels of the agent set, We calculate the average of each matrix. By dividing each the value into the same inter-label group and different inter-label group and taking their average values, we conﬁrmed the similarity of messages between labels in each agent set. We tested three times using different images for each agent set.

Original: The second analysis was the evaluation of Dream.
Revised: The second analysis was the evaluation of Dream.

Original: We entered the message into an agent to evaluate the reconstructed Dream.
Revised: We entered the message into an agent to evaluate the reconstructed Dream.

Original: The message was generated as follows.
Revised: The message was generated as follows.

Original: Three arbitrary agents were selected from each agent set, and 200 symbol strings were generated.
Revised: Three arbitrary agents were selected from each agent set, and 200 symbol strings were generated.

Original: For each agent, 100 messages were generated using a common image among the agents.
Revised: For each agent, 100 messages were generated using a common image among the agents.

Original: Another 100 messages were generated using independent images for each agent.
Revised: Another 100 messages were generated using independent images for each agent.

Original: Validity was evaluated using ﬁve MNIST classiﬁers with over 98% accuracy, using Vedantam et al.
Revised: Validity was evaluated using ﬁve MNIST classiﬁers with over 98% accuracy, using Vedantam et al.

Original: (2018) as reference.
Revised: (2018) as reference.

Original: In the third analysis, the characteristics of lead symbol composition were compared with label and head symbol analyses.
Revised: In the third analysis, the characteristics of lead symbol composition were compared with label and head symbol analyses.

Original: The analyses comprised 600 messages generated in the 2nd evaluation.
Revised: The analyses comprised 600 messages generated in the 2nd evaluation.

Original: In the label analysis, the composition ratio of the lead symbol to one label was conﬁrmed.
Revised: In the label analysis, the composition ratio of the lead symbol to one label was conﬁrmed.

Original: If one label’s lead symbol consisted of only one or a few symbols, a clear owning relationship existed between the symbol and label.
Revised: If one label ’ s lead symbol consisted of only one or a few symbols, a clear owning relationship existed between the symbol and label.

Original: However, if it was composed of a plurality of lead symbols, no common rule based on the lead symbols existed.
Revised: However, if it was composed of a plurality of lead symbols, no common rule based on the lead symbols existed.

Original: This analysis indicated the stability of the generated symbol.
Revised: This analysis indicated the stability of the generated symbol.

Original: Speciﬁcally, the number of labels whose lead symbol constituted more than 70% of each label was counted.
Revised: Speciﬁcally, the number of labels whose lead symbol constituted more than 70% of each label was counted.

Original: In the head symbol analysis, whether the generated head symbol is biased to a speciﬁc symbol is evaluated as a variance.
Revised: In the head symbol analysis, whether the generated head symbol is biased to a speciﬁc symbol is evaluated as a variance.

Original: If the head symbol was generated without bias (i.e., using various symbols for the head), the variance was small.
Revised: If the head symbol was generated without bias (i.e., using various symbols for the head), the variance was small.

Original: However, if a speciﬁc head symbol was commonly (monopoly) used (i.e., using only a single symbol), the variance was large.
Revised: However, if a speciﬁc head symbol was commonly (monopoly) used (i.e., using only a single symbol), the variance was large.

Original: The variance formula is expressed as follows.
Revised: The variance formula is expressed as follows.

Original: var = 1 n (cid:80) ni ( x i − µ ) 2 , where x i represents the number of uses of a certain head symbol and n represents head symbol’s number.
Revised: var = 1 n ∑ ni ( x i − µ ) 2 , where x i represents the number of uses of a certain head symbol and n represents head symbol’s number.

Original: The equality and division properties of the generated symbols were evaluated in this analysis.
Revised: The equality and division properties of the generated symbols were evaluated in this analysis.

Original: Using these two indexes, the characteristics of the generated messages were evaluated quantitatively (Brighton & Kirby, 2006).
Revised: Using these two indexes, the characteristics of the generated messages were evaluated quantitatively (Brighton & Kirby, 2006).

Original: The fourth analysis conﬁrmed the formal rule for the message corresponding to Dream.
Revised: The fourth analysis conﬁrmed the formal rule for the message corresponding to Dream.

Original: We entered a message to each agent set and conﬁrmed the conditions to be reconstructed.
Revised: We entered a message to each agent set and conﬁrmed the conditions to be reconstructed.

Original: Language structure by recursion and multiple symbols is the basic element of “Compositionality” in language emergent research (Kirby & Hurford, 2002).
Revised: Language structure by recursion and multiple symbols is the basic element of “Compositionality” in language emergent research (Kirby & Hurford, 2002).

Original: Therefore,We attempted to describe repeated patterns that appeared in the message as a formal rule.
Revised: Therefore,We attempted to describe repeated patterns that appeared in the message as a formal rule.

Original: This formal rule was from the human perspective.
Revised: The analyzed formal rules such as pattern and key symbol were evaluated from a human criterion.

Original: Therefore, we evaluated whether the evaluation of human formalization conformed to the agent’s viewpoint.
Revised: Therefore, we evaluated whether the evaluation of human formalization conformed to the agent’s viewpoint.

Original: An index exists from the agent’s viewpoint: Similarity evaluation between generated messages and formalized messages.
Revised: An index exists from the agent’s viewpoint: Similarity evaluation between generated messages and formalized messages.

Original: We analyzed the degree to which the messages formalized from the human viewpoint were utilized among the agents.
Revised: We analyzed the degree to which the messages formalized from the human viewpoint were utilized among the agents.

Original: Three different messages based on the formal message were prepared as the criterion of similarity degree 1.
Revised: Three different messages based on the formal message were prepared as the criterion of similarity degree 1.

Original: Each agent set input an image and generated a message.
Revised: Each agent set input an image and generated a message.

Original: The similarity between the generated and formal messages was compared using a histogram.
Revised: The similarity between the generated and formal messages was compared using a histogram.

Original: A total of 10000 images were prepared.
Revised: A total of 10000 images were prepared.

Original: Next, we simpliﬁed the formal rules and analyzed whether a speciﬁc syntax appeared in the generated rules.
Revised: Next, we simpliﬁed the formal rules and analyzed whether a speciﬁc syntax appeared in the generated rules.

Original: Simpliﬁcation omits parentheses and repetitions.
Revised: Simpliﬁcation omits parentheses and repetitions.

Original: The formal rules were aligned for each leading symbol.
Revised: The formal rules were aligned for each leading symbol.

Original: The analysis was focused on the head symbol and the unique symbol.
Revised: The analysis was focused on the head symbol and the unique symbol.

Original: In the  unique symbol, symbol duplication was conﬁrmed by comparing the formal rules of the same head symbol.
Revised: In the [Table] unique symbol, symbol duplication was conﬁrmed by comparing the formal rules of the same head symbol.

Original: In the head symbol, it was conﬁrmed that the ﬁrst two-digit symbol did not overlap with other formal rules.
Revised: In the head symbol, it was conﬁrmed that the ﬁrst two-digit symbol did not overlap with other formal rules.

Original: Using these indices, the structure of the formal rules in each learning object was conﬁrmed.
Revised: Using these indices, the structure of the formal rules in each learning object was conﬁrmed.

Original:  R ESULT 4. BASIC INFORMATION EVALUATION
Revised:  R ESULT 4. BASIC INFORMATION EVALUATION

Original: We conﬁrmed the characteristics of the learning objects in the message generation and learning phases.
Revised: We conﬁrmed the characteristics of the learning objects in the message generation and learning phases.

Original: First, the distance between the message and target image was analyzed using the MSE.
Revised: First, the distance between the message and target image was analyzed using the MSE.

Original: The loss tended to decrease during advanced learning in all learning objects (Figure 2).
Revised: The loss tended to decrease during advanced learning in all learning objects (Figure 2).

Original: In INTERNAL, a gradual decrease was shown compared with NORMAL.
Revised: In INTERNAL, a gradual decrease was shown compared with NORMAL.

Original: NORMAL, which had no internal reﬂection[Table] function, showed an upward trend from epoch
Revised: NORMAL, which had no internal reﬂection function, showed an upward trend from epoch

Original: = 850.
Revised: = 850.

Original: In INTERNAL, this tendency was not observed until epoch = 1100 but turned slightly upward at epoch = 
Revised: In INTERNAL, this tendency was not observed until epoch = 1100 but turned slightly upward at epoch = 1200.

Original: Meanwhile, averages from epoch = 1100 to epoch
Revised: Meanwhile, averages from epoch = 1100 to epoch

Original: = 1200 were 0.023 and 0.017 for NORMAL and
Revised: = 1200 were 0.023 and 0.017 for NORMAL and

Original: INTERNAL, respectively.
Revised: INTERNAL, respectively.

Original: A clear increase in loss function  observed only in 
Revised: A clear increase in loss function was observed only in NORMAL.

Original: Thus, these data exhibited different loss function transitions depending on the learning objects.
Revised: Thus, these data exhibited different loss function transitions depending on the learning objects.

Original: Subsequently, we attempted to analyze epoch points that have a smaller loss function value for each learning objects.
Revised: Subsequently, we attempted to analyze epoch points that have a smaller loss function value for each learning objects.

Original: (NORMAL = 810, INTERNAL = 1200.
Revised: (NORMAL = 810, INTERNAL = 1200.

Original: For the analysis with NORMAL = 810, we created ﬁve new veriﬁcation agent sets at epoch = 810.
Revised: For the analysis with NORMAL = 810, we created ﬁve new veriﬁcation agent sets at epoch = 810.

Original: The decrease in loss function showed a similar trend.)
Revised: The decrease in loss function showed a similar trend.)

Original: Next, symbol identity by each label was analyzed using Jaccard coefﬁcients.
Revised: Next, symbol identity by each label was analyzed using Jaccard coefﬁcients.

Original: All learning objects exhibited tendencies toward symbol similarity for the same label and tendencies toward exclusiveness for different labels (Table 1).
Revised: All learning objects exhibited tendencies toward symbol similarity for the same label and tendencies toward exclusiveness for different labels (Table 1).

Original: 4. CONVERT SYMBOL TO DREAM
Revised: 4. CONVERT SYMBOL TO DREAM

Original: We conﬁrmed the accuracy of Dream reconstruction for the message.
Revised: We conﬁrmed the accuracy of Dream reconstruction for the message.

Original: In all the learning objects, the average accuracy was 0.47- 0.50 (Figure 3).
Revised: In all the learning objects, the average accuracy was 0.47- 0.50 (Figure 3).

Original: Statistically signiﬁcant differences in the accuracy were not observed.
Revised: Statistically signiﬁcant differences in the accuracy were not observed.

Original: The clustering accuracy of the VAE latent space was approximately 0.45 using the v-score.
Revised: The clustering accuracy of the VAE latent space was approximately 0.45 using the v-score.

Original: Therefore, Dream reconstruction by the message exhibited similar ability as that of latent space clustering.
Revised: Therefore, Dream reconstruction by the message exhibited similar ability as that of latent space clustering.

Original: Shareability was conﬁrmed as well.
Revised: Shareability was conﬁrmed as well.

Revised: [Table]

Original: In additional, the accuracy of the labels by the learning objects was compared.
Revised: In additional, the accuracy of the labels by the learning objects was compared.

Original: In INTERNAL, the difference in the maximum and minimum values of the accuracy among the labels was small.
Revised: In INTERNAL, the difference in the maximum and minimum values of the accuracy among the labels was small.

Original: However, NORMAL exhibited some degree of dispersion between the minimum and maximum values.
Revised: However, NORMAL exhibited some degree of dispersion between the minimum and maximum values.

Original: An example messages whose label Dream is to be reconstructed is shown in Appendix E.
Revised: An example messages whose label Dream is to be reconstructed is shown in Appendix E.

Original: [Table]

Original: [Table]

Original: 4. FORMAL EVALUATION OF LEADING SYMBOLS
Revised: 4. FORMAL EVALUATION OF LEADING SYMBOLS

Original: From the label and head symbol analyses, characteristics of lead symbol constructability are compared.
Revised: From the label and head symbol analyses, characteristics of lead symbol constructability are compared.

Original: Table 2 (the label analysis) showed almost the same value at the end of learning in the learning objects.
Revised: Table 2 (the label analysis) showed almost the same value at the end of learning in the learning objects.

Original: No statistically signiﬁcant difference was observed.
Revised: No statistically signiﬁcant difference was observed.

Original: Meanwhile, in the head symbol analysis, each learning object showed different trends.
Revised: Meanwhile, in the head symbol analysis, each learning object showed different trends.

Original: At epoch = 210, the variance was almost equal in all learning objects.
Revised: At epoch = 210, the variance was almost equal in all learning objects.

Original: In NORMAL, the variance remained almost constant.
Revised: In NORMAL, the variance remained almost constant.

Original: INTERNAL showed a downward trend at each epoch.
Revised: INTERNAL showed a downward trend at each epoch.

Original: Finally, a statistically signiﬁcant difference appeared in the value of the head symbol variance between NORMAL and an internal reﬂection function such as INTERNAL.The existence or nonexistence of the internal reﬂection function caused the lead symbol to exhibit a quantitatively different structure.
Revised: Finally, a statistically signiﬁcant difference appeared in the value of the head symbol variance between NORMAL and an internal reﬂection function such as INTERNAL.The existence or nonexistence of the internal reﬂection function caused the lead symbol to exhibit a quantitatively different structure.

Original: 4. ATTEMPTS TO FORMALIZE MESSAGES 4.4. DECODING FROM HUMAN VIEWPOINT OF SYMBOL FORMAL RULE
Revised: 4. ATTEMPTS TO FORMALIZE MESSAGES 4.4. DECODING FROM HUMAN VIEWPOINT OF SYMBOL FORMAL RULE

Original: One agent group was extracted from each learning object.
Revised: One agent group was extracted from each learning object.

Original: We tested the formal rule of the message.
Revised: We tested the formal rule of the message.

Original: The formal rules for each label are shown in Table 3.
Revised: The formal rules for each label are shown in Table 3.

Original: In all learning objects, each agent set formed its own formal rule.
Revised: In all learning objects, each agent set formed its own formal rule.

Original: A key symbol was created to generate each label’s Dream.
Revised: A key symbol was created to generate each label ’ s Dream.

Original: In NORMAL, for example, the formal rule to be reconstructed with label “1” required the symbol “d” that is at least ﬁve digits long.
Revised: In NORMAL, for example, the formal rule to be reconstructed with label “ 1 ” required the symbol “ d ” that is at least ﬁve digits long.

Original: In INTERNAL, labels “4,” “7,” and “9” began with the same symbol, and each label comprised a repeated pattern of different symbols.
Revised: In INTERNAL, labels “ 4, ” “ 7, ” and “ 9 ” began with the same symbol, and each label comprised a repeated pattern of different symbols.

Original: A formal rule having repeated patterns including the order and combination of symbols emerged in INTERNAL.
Revised: A formal rule having repeated patterns including the order and combination of symbols emerged in INTERNAL.

Original: Additionally, Internal identiﬁed the presence of exclusive symbols that prevented the label from being reconstructed.
Revised: Additionally, Internal identiﬁed the presence of exclusive symbols that prevented the label from being reconstructed.

Original: Meanwhile, NORMAL had few labels with such a formal rule.
Revised: Meanwhile, NORMAL had few labels with such a formal rule.

Original: NORMAL showed few clear repeated pattern and could reconstruct Dream by the minimum necessary key symbol.
Revised: NORMAL showed few clear repeated pattern and could reconstruct Dream by the minimum necessary key symbol.

Original: A sample of Dream reconstructed by the message with formal rules is presented in the Appendix G. When the similarity is low, we conﬁrmed in Appendix H that INTERNAL with a repeating pattern contributes effectively to the reconstruction of Dream.peaks formal with considered the higher the peak value existed [Table]
Revised: A sample of Dream reconstructed by the message with formal rules is presented in the Appendix G. When the similarity is low, we conﬁrmed in Appendix H that INTERNAL with a repeating pattern contributes effectively to the reconstruction of Dream.

Original: Next, we conﬁrm whether the formal rule message decoded from the human viewpoint is generated as the mainstream message in the agent set above.
Revised: Next, we conﬁrm whether the formal rule message decoded from the human viewpoint is generated as the mainstream message in the agent set above.

Original: The results are shown in Table 4.
Revised: The results are shown in Table 4.

Original: In all learning objects, half of the labels had similarity peaks of 0.55 or higher.
Revised: In all learning objects, half of the labels had similarity peaks of 0.55 or higher.

Original: The results showed that the similarity between a message with a formal rule decoded by a human and that generated by an agent could be conﬁrmed.
Revised: The results showed that the similarity between a message with a formal rule decoded by a human and that generated by an agent could be conﬁrmed.

Original: We compared the characteristics of each label in the learning objects.
Revised: We compared the characteristics of each label in the learning objects.

Original:  NORMAL tended to show a low similarity peak when of plural symbols are combined in the form rule.
Revised: [Table] [Table] NORMAL tended to show a low similarity peak when of plural symbols are combined in the form rule.

Original: However, In INTERNAL ( except label “4” in INTERNAL that was at a low formal rule peak of 0.4.), labels using multiple symbols showed high similarity peaks.
Revised: However, In INTERNAL ( except label “4” in INTERNAL that was at a low formal rule peak of 0.4.), labels using multiple symbols showed high similarity peaks.

Original: 4.4. PARSING OF FORMAL RULES
Revised: 4.4. PARSING OF FORMAL RULES

Original: After simplifying the formal rule shown in Section 4.4.1, the syntax of the generated rule was analyzed (Table 5).
Revised: After simplifying the formal rule shown in Section 4.4.1, the syntax of the generated rule was analyzed (Table 5).

Original: [Table]
Revised: [Table]

Original: was frequently used in the lead symbol.

Original: Three of the four were composed of the same head symbol.

Original: In addition, no unique symbol existed for each formal rule.

Original: The head symbol was the same in the “c” series.

Original: However, a unique symbol existed for each formal rule.

Original: In the “d” series, the unique symbol and the head symbol appeared on the same formal rule.

Original: INTERNAL :
Revised: INTERNAL :

Original: The lead symbol is allocated almost evenly.
Revised: The lead symbol is allocated almost evenly.

Original: In the “b” series, a distinctive head symbol appeared, while the unique symbol appeared in one formal rule.
Revised: In the “b” series, a distinctive head symbol appeared, while the unique symbol appeared in one formal rule.

Original: In the “c” series, a distinctive head symbol appeared in two formal rules, while the unique symbol appeared in two rules.
Revised: In the “c” series, a distinctive head symbol appeared in two formal rules, while the unique symbol appeared in two rules.

Original: In the “d” series, a distinctive head symbol appeared in two rules, while the unique symbol appeared in one rule.
Revised: In the “d” series, a distinctive head symbol appeared in two rules, while the unique symbol appeared in one rule.

Original: In INTERNAL, head symbol duplication was less than in NORMAL.
Revised: In INTERNAL, head symbol duplication was less than in NORMAL.

Original: The number of unique symbols appeared was almost the same in each learning object.
Revised: The number of unique symbols appeared was almost the same in each learning object.

Original: However, in the “c” series of NORMAL, the unique symbol appeared as parenthesized symbols in the formal rules of each other and are not completely independent symbols.
Revised: However, in the “c” series of NORMAL, the unique symbol appeared as parenthesized symbols in the formal rules of each other and are not completely independent symbols.

Original: Even in the “b” series, which requires a clear divisions, only one formal rule appeared clearly independent.
Revised: Even in the “b” series, which requires a clear divisions, only one formal rule appeared clearly independent.

Original: Meanwhile, in INTERNAL, distinctive head symbols and unique symbols appeared in a balanced manner in each rule.
Revised: Meanwhile, in INTERNAL, distinctive head symbols and unique symbols appeared in a balanced manner in each rule.

Original: Each rule had a proprietary tendency.
Revised: Each rule had a proprietary tendency.

Original: It was conﬁrmed that even a formal rule not having a unique symbol exhibited a different pattern.
Revised: It was conﬁrmed that even a formal rule not having a unique symbol exhibited a different pattern.

Original: Therefore, the tendency to form a unique symbol combination in INTERNAL was conﬁrmed.
Revised: Therefore, the tendency to form a unique symbol combination in INTERNAL was conﬁrmed.

Original: The spatial representation of the message is shown in Appendix I.
Revised: The spatial representation of the message is shown in Appendix I.

Original: It represented the representational difference for each message in the agent.
Revised: It represented the representational difference for each message in the agent.

Original:  DISCUSSION
Revised:  DISCUSSION

Original: In this study, we proposed a multi-agent description game with 100 agents and a model architecture.
Revised: In this study, we proposed a multi-agent description game with 100 agents and a model architecture.

Original: The obverter technique was used for message generation.
Revised: The obverter technique was used for message generation.

Original: In addition, the internal reﬂection function, which imitates human language acquisition, was introduced.
Revised: In addition, the internal reﬂection function, which imitates human language acquisition, was introduced.

Original: As described in Sections 4.1 and 4.2, messages corresponding to the label was successfully generated, and the reconstruction of Dream was conﬁrmed from the messages.
Revised: As described in Sections 4.1 and 4.2, messages corresponding to the label was successfully generated, and the reconstruction of Dream was conﬁrmed from the messages.

Original: It was found that by the proposed method, messages that correspond to concepts and allow an agreement between agents emerged without any supervisor.
Revised: It was found that by the proposed method, messages that correspond to concepts and allow an agreement between agents emerged without any supervisor.

Original: However, as reported in Section 4.1, the decreasing tendency of the loss function was different between learning objects.
Revised: However, as reported in Section 4.1, the decreasing tendency of the loss function was different between learning objects.

Original: As described in Sections 4.3 and 4.4, the formal rule between learning objects differed.
Revised: As described in Sections 4.3 and 4.4, the formal rule between learning objects differed.

Original: In NORMAL, a rule responding to a speciﬁc symbol element is generated, while in INTERNAL, a symbol combination pattern is generated.
Revised: In NORMAL, a rule responding to a speciﬁc symbol element is generated, while in INTERNAL, a symbol combination pattern is generated.

Original: Each label has its own combination pattern.
Revised: Each label has its own combination pattern.

Original: It was found that the formal rules of messages that could be conceptually agreed by internal reﬂections were structured differently.
Revised: It was found that the formal rules of messages that could be conceptually agreed by internal reﬂections were structured differently.

Original: This result suggests the existence of qualitatively different formal trends caused by internal reﬂection function.
Revised: This result suggests the existence of qualitatively different formal trends caused by internal reﬂection function.

Original: Therefore, we considered the features of the internal reﬂection function.
Revised: Therefore, we considered the features of the internal reﬂection function.

Original: NORMAL allowed students to learn all the messages generated by the teacher as a “positive example.”
Revised: NORMAL allowed students to learn all the messages generated by the teacher as a “positive example.”

Original: In this situation, the “this and that” symbol was learned; therefore, even a slight difference was absorbed as a “correct” message.
Revised: In this situation, the “this and that” symbol was learned; therefore, even a slight difference was absorbed as a “correct” message.

Original: The results of the loss function indicated that this situation occurred in “overlearning.”
Revised: The results of the loss function indicated that this situation occurred in “overlearning.”

Original: Meanwhile, in the internal reﬂection function, a learning constraint based on the agent’s similarity evaluation existed.
Revised: Meanwhile, in the internal reﬂection function, a learning constraint based on the agent’s similarity evaluation existed.

Original: Because of this constraint, it is presumed that each agent creates a “negative example” in a pseudo manner by comparing itself with others.
Revised: Because of this constraint, it is presumed that each agent creates a “negative example” in a pseudo manner by comparing itself with others.

Original: The positive and negative examples might have constructed a pseudo boundary among the labels.
Revised: The positive and negative examples might have constructed a pseudo boundary among the labels.

Original: Therefore, we assumed that a conservative message has been constructed.
Revised: Therefore, we assumed that a conservative message has been constructed.

Original: Consequently, a whole agent tends to a generate message having a common symbol for the same label.
Revised: Consequently, a whole agent tends to a generate message having a common symbol for the same label.

Original: This situation, in which only common-term messages are generated, could be regarded as a bottleneck in information transmission between agent groups.
Revised: This situation, in which only common-term messages are generated, could be regarded as a bottleneck in information transmission between agent groups.

Original: Vogt (2005) reported that bottlenecks (Kirby, 2001; Brighton, 2002; Smith et al., 2003) produced syntactic structures.
Revised: Vogt (2005) reported that bottlenecks (Kirby, 2001; Brighton, 2002; Smith et al., 2003) produced syntactic structures.

Original: The bottleneck was considered to have resulted in the generation of massages having a pattern in the formal rule.
Revised: The bottleneck was considered to have resulted in the generation of massages having a pattern in the formal rule.

Original: An additional experiment was conducted on “SWITCH” in Appendix C. For SWITCH, the learning start point was NORMAL, and after some learning steps, the internal reﬂection function was activated.
Revised: An additional experiment was conducted on “SWITCH” in Appendix C. For SWITCH, the learning start point was NORMAL, and after some learning steps, the internal reﬂection function was activated.

Original: The results are shown in the Appendix.
Revised: The results are shown in the Appendix.

Original: SWITCH showed the same results as INTERNAL.
Revised: SWITCH showed the same results as INTERNAL.

Original: SWITCH was assumed to have broken the language structure at the time of NORMAL; thus, a new formal rule emerged.
Revised: SWITCH was assumed to have broken the language structure at the time of NORMAL; thus, a new formal rule emerged.

Original: Therefore, the internal reﬂection function can be expressed as an emergent parameter of a conceptually agreed language system.
Revised: Therefore, the internal reﬂection function can be expressed as an emergent parameter of a conceptually agreed language system.

Original: In this study we established a method for the emergence of a language system, in which consensus could be achieved in a group without a human supervisor.
Revised: In this study, we established a method for the emergence of a language system, in which consensus could be achieved in a group without a human supervisor.

Original: We proposed an emergence parameter, i e., the “internal reﬂection function,” which acquired structuring formal rules; additionally, we conﬁrmed its effectiveness.
Revised: We proposed an emergence parameter, i.e., the “internal reﬂection function,” which acquired structuring formal rules; additionally, we con- ﬁrmed its effectiveness.

Original: This suggests that it performs effectively under-restricted situations similar to human language generation.
Revised: This suggests that it performs effectively under-restricted situations similar to human language generation.

Original: Further studies are required to generalize the relationship between formalization and concept.
Revised: Further studies are required to generalize the relationship between formalization and concept.

Original: We conﬁrmed the laws of systematization generated by machines using data other than those of MNIST and having “different concepts” in each agent.
Revised: We conﬁrmed the laws of systematization generated by machines using data other than those of MNIST and having “different concepts” in each agent

Original: A MODEL ARCHITECTURE
Revised: A MODEL ARCHITECTURE

Original: We used PyTorch for all of our implementations.
Revised: We used PyTorch for all of our implementations.

Original: The module is structured independently.
Revised: The module is structured independently.

Original: This was constructed with reference to conceptual correspondence research (Higgins et al., 2018; Vedantam et al., 2018) and translation method by Pivoting (Gella et al., 2017; Lee et al., 2018).
Revised: This was constructed with reference to conceptual correspondence research (Higgins et al., 2018; Vedantam et al., 2018) and translation method by Pivoting (Gella et al., 2017; Lee et al., 2018).

Original: Among them, SCAN (Vedantam et al., 2018) connected the concept of latent space disentangled by bata-VAE (Higgins et al., 2017) with the corresponding words using the KL divergence.
Revised: Among them, SCAN (Vedantam et al., 2018) connected the concept of latent space disentangled by bata-VAE (Higgins et al., 2017) with the corresponding words using the KL divergence.

Original: Consequently, image areas corresponding to human words (nouns, adjectives, and conjunctions) can be automatically estimated from raw pixel data without annotation.
Revised: Consequently, image areas corresponding to human words (nouns, adjectives, and conjunctions) can be automatically estimated from raw pixel data without annotation.

Original: Furthermore, SCAN has successfully reconstructed images that were not directly learned from supervisor data.
Revised: Furthermore, SCAN has successfully reconstructed images that were not directly learned from supervisor data.

Original: To create a language system without a supervisor, which is the purpose of this study, methods using concepts and symbols will be useful as reference.
Revised: To create a language system without a supervisor, which is the purpose of this study, methods using concepts and symbols will be useful as reference.

Original: A. VISUAL MODULE
Revised: A. VISUAL MODULE

Original: We used VAE (Kingma & Welling, 2013) to learn MNIST.
Revised: We used VAE (Kingma & Welling, 2013) to learn MNIST.

Original: (1) Encoder side.
Revised: (1) Encoder side.

Original: The input layer is composed of 782 dimensions.
Revised: The input layer is composed of 782 dimensions.

Original: The activation function used the ReLU.
Revised: The activation function used the ReLU.

Original: The next hidden layer consists of 512 dimensions.
Revised: The next hidden layer consists of 512 dimensions.

Original: To represent the latent space, the sampling layer that generates the latent space z is connected to the hidden layer.
Revised: To represent the latent space, the sampling layer that generates the latent space z is connected to the hidden layer.

Original: In this study, the latent space is set to four dimensions.
Revised: In this study, the latent space is set to four dimensions.

Original: (2) Decoder side.
Revised: (2) Decoder side.

Original: The decoder consists of a hidden layer that converts four-dimensional information in the latent space to 512 dimensions and an output layer of 782 dimensions.
Revised: The decoder consists of a hidden layer that converts four-dimensional information in the latent space to 512 dimensions and an output layer of 782 dimensions.

Original: The activation function of the output layer is a sigmoid function.
Revised: The activation function of the output layer is a sigmoid function.

Original: The optimization function is Adam (Lr = 1e-3).
Revised: The optimization function is Adam (Lr = 1e-3).

Original: For creating the agent, 100 visual modules were created by learning different initial values.
Revised: For creating the agent, 100 visual modules were created by learning different initial values.

Original: A. L ANGUAGE M ODULE
Revised: A. L ANGUAGE M ODULE

Original: We used a single layer GRU (Cho et al., 2014) to implement the language module.
Revised: We used a single layer GRU (Cho et al., 2014) to implement the language module.

Original: The size of the hidden layer was 128.
Revised: The size of the hidden layer was 128.

Original: The GRU uses the many-to-one output method.
Revised: The GRU uses the many-to-one output method.

Original: The GRU outputs a fourdimensional value to any input symbol string.
Revised: The GRU outputs a fourdimensional value to any input symbol string.

Original: The optimization method was SGD (Lr = 0.005).
Revised: The optimization method was SGD (Lr = 0.005).

Original: The initial value of the language module was different.
Revised: The initial value of the language module was different.

Original: B MESSAGE GENERATION BY MULTI-AGENT DESCRIPTION GAME AND OBVERTER TECHNIQUE
Revised: B MESSAGE GENERATION BY MULTI-AGENT DESCRIPTION GAME AND OBVERTER TECHNIQUE

Original: The obverter technique algorithm at the time of generation is as follows algorithm 1.
Revised: The obverter technique algorithm at the time of generation is as follows algorithm 1.

Original: We improved the original paper (Batali, 1998).
Revised: We improved the original paper (Batali, 1998).

Original: Algorithm 1: process used in our study while | s | < max message length do

Original: Module
Revised: Algorithm

Original: − MSE t v
Revised: < max length

Original: Control parameters consisted of the max message length and a threshold value.
Revised: Control parameters consisted of the max message length and a threshold value.

Original: In this experiment, the max message length was 10 and the similarity threshold was 0.005.
Revised: In this experiment, the max message length was 10 and the similarity threshold was 0.005.

Original: In learning, the role of a teacher is randomly selected for each epoch.
Revised: In learning, the role of a teacher is randomly selected for each epoch.

Original: One hundred images were presented for each epoch, and the teacher randomly selected and generated a corresponding message.
Revised: One hundred images were presented for each epoch, and the teacher randomly selected and generated a corresponding message.

Original: Students learned using the message.
Revised: Students learned using the message.

Original: When it reached the deﬁned number of times, the next epoch was tested.
Revised: When it reached the deﬁned number of times, the next epoch was tested.

Original: In this experiment, the deﬁned number of times was 100.
Revised: In this experiment, the deﬁned number of times was 100.

Original: C VALIDATION OF SWITCH
Revised: C VALIDATION OF SWITCH

Original: To verify the strength of the constraints of the internal reﬂection function, we activated the internal reﬂection function during learning.
Revised: To verify the strength of the constraints of the internal reﬂection function, we activated the internal reﬂection function during learning.

Original: This object was called SWITCH .
Revised: This object was called SWITCH .

Original: C. METHOD
Revised: C. METHOD

Original: SWITCH required parameters for the similarity threshold and a starting point for the internal reﬂection function.
Revised: SWITCH required parameters for the similarity threshold and a starting point for the internal reﬂection function.

Original: The parameter used is a threshold value of 0.6/start at epoch =
Revised: The parameter used is a threshold value of 0.6/start at epoch =

Original: Five agent sets were created.
Revised: Five agent sets were created.

Original: C. BASIC INFORMATION EVALUATION
Revised: C. BASIC INFORMATION EVALUATION

Original: First, the distance between the message and the target image was analyzed by the MSE (Figure 4).
Revised: First, the distance between the message and the target image was analyzed by the MSE (Figure 4).

Original: SWITCH and NORMAL showed the same decreasing trend because they had no internal reﬂection function at the beginning of learning.
Revised: SWITCH and NORMAL showed the same decreasing trend because they had no internal reﬂection function at the beginning of learning.

Original: In SWITCH, the loss function did not increase as in NORMAL after epoch = 800, but turned slightly upward at epoch = 1200.
Revised: In SWITCH, the loss function did not increase as in NORMAL after epoch = 800, but turned slightly upward at epoch = 1200.

Original: Owing to the internal reﬂection function, the trend differed from that of NORMAL.
Revised: Owing to the internal reﬂection function, the trend differed from that of NORMAL.

Original: The analysis was performed at epoch = 1200.
Revised: The analysis was performed at epoch = 1200.

Original: [Table]
Revised: [Table]

Original: In Jaccard analysis, the trend of SWITCH is the same as those of other learning methods (Table 6).
Revised: In Jaccard analysis, the trend of SWITCH is the same as those of other learning methods (Table 6).

Original: C. CONVERT SYMBOL TO DREAM
Revised: C. CONVERT SYMBOL TO DREAM

Original: We conﬁrmed the accuracy of Dream reconstruction for the message.
Revised: We conﬁrmed the accuracy of Dream reconstruction for the message.

Original: In SWITCH, the average accuracy was 0.47–0.50 (Figure 5).
Revised: In SWITCH, the average accuracy was 0.47–0.50 (Figure 5).

Original: No statistically signiﬁcant differences were observed in the accuracy.
Revised: No statistically signiﬁcant differences were observed in the accuracy.

Original: SWITCH tended to exhibit some degree of dispersion between the minimum and maximum values.
Revised: SWITCH tended to exhibit some degree of dispersion between the minimum and maximum values.

Original: C. FORMAL EVALUATION OF LEADING SYMBOLS
Revised: C. FORMAL EVALUATION OF LEADING SYMBOLS

Original: [Table]
Revised: [Table]

Original: The lead symbol in Table7 was analyzed.
Revised: The lead symbol in Table7 was analyzed.

Original: In label analysis, NORMAL and INTERNAL showed similar results.
Revised: In label analysis, NORMAL and INTERNAL showed similar results.

Original: No statistically signiﬁcant difference was observed.
Revised: No statistically signiﬁcant difference was observed.

Original: Meanwhile, the head symbol analysis showed a characteristic trend.
Revised: Meanwhile, the head symbol analysis showed a characteristic trend.

Original: SWITCH showed the same value as NORMAL before the internal reﬂection function was applied (before epoch = 600).
Revised: SWITCH showed the same value as NORMAL before the internal reﬂection function was applied (before epoch = 600).

Original: However, after the internal reﬂection function was applied, SWITCH showed a similar value to INTERNAL.
Revised: However, after the internal reﬂection function was applied, SWITCH showed a similar value to INTERNAL.

Original: A signiﬁcant difference was observed with NORMAL but not with INTERNAL.
Revised: A signiﬁcant difference was observed with NORMAL but not with INTERNAL.

Original: C. ATTEMPTS TO FORMALIZE MESSAGES C.5. DECODING FROM HUMAN VIEWPOINT OF SYMBOL FORMAL RULE
Revised: C. ATTEMPTS TO FORMALIZE MESSAGES C.5. DECODING FROM HUMAN VIEWPOINT OF SYMBOL FORMAL RULE

Original: [Table]

Original: One agent group was extracted from SEITCH agent set.
Revised: One agent group was extracted from SEITCH agent set.

Original: The formal rules for each label are shown in the Table 8.
Revised: The formal rules for each label are shown in the Table 8.

Original: In SWITCH, an agent set forms its own rule system.
Revised: In SWITCH, an agent set forms its own rule system.

Original: Labels “0 ”,“ 5”, and “6 ”  began with same symbol, and each label comprised a repeated pattern of different symbols.
Revised: Labels “ 0 ” , “ 5 ” , and “” began with same symbol, and each label comprised a repeated pattern of different symbols.

Original: The characteristics of the formal rule were similar to Internal.
Revised: The characteristics of the formal rule were similar to Internal.

Original: Next, we conﬁrm whether the formal rule message decoded from the human viewpoint is generated as the mainstream message in the agent set above (Table 9).
Revised: Next, we conﬁrm whether the formal rule message decoded from the human viewpoint is generated as the mainstream message in the agent set above (Table 9).

Original: As in INTERNAL and NORMAL,
Revised: As in INTERNAL and NORMAL,

Revised: [Table]

Original: SWITCH tended to exhibit a high peak value.
Revised: SWITCH tended to exhibit a high peak value.

Original: The prototype messages of SWITCH were composed of a combination pattern of multiple symbols.
Revised: The prototype messages of SWITCH were composed of a combination pattern of multiple symbols.

Original: SWITCH was particularly prone to exhibit bimodal peaks.
Revised: SWITCH was particularly prone to exhibit bimodal peaks.

Revised: [Table]

Original: C.5. PARSING OF FORMAL RULES
Revised: C.5. PARSING OF FORMAL RULES

Revised: [Table]

Original: After simplifying the formal rule shown in Section C.5.1, the syntax of the generated rule was analyzed (Table 10).
Revised: After simplifying the formal rule shown in Section C.5.1, the syntax of the generated rule was analyzed (Table 10).

Original: SWITCH has a symbol syntax trend similar to that of INTERNAL.
Revised: SWITCH has a symbol syntax trend similar to that of INTERNAL.

Original: In “a” series, unique symbols appeared in two formal rules.
Revised: In “a” series, unique symbols appeared in two formal rules.

Original: A distinctive head symbol appeared in one form rule.
Revised: A distinctive head symbol appeared in one form rule.

Original: In the “b” series, although the symbols were similar, different head symbols appeared.In
Revised: In the “b” series, although the symbols were similar, different head symbols appeared.

Original: [Table] [Table] series aaa bd cc a dad/dda aa d /aa dd bbdb c b dda c a cb ccd/ c d dda/ d b the “c” series, a unique symbol existed for every rule.
Revised: In the “c” series, a unique symbol existed for every rule.

Original: In the “d” series, a distinctive head symbol appeared in two rules.
Revised: In the “d” series, a distinctive head symbol appeared in two rules.

Original: D AGENT LEARNING PARTICIPATION TENDENCY BY INTERNAL REFLECTION FUNCTION
Revised: D AGENT LEARNING PARTICIPATION TENDENCY BY INTERNAL REFLECTION FUNCTION

Original: [Figure]

Original: Figure 6 shows the agent learning participation tendency by the internal reﬂection function.
Revised: Figure 6 shows the agent learning participation tendency by the internal reﬂection function.

Original: With
Revised: With

Original: INTERNAL, the number of agents participating in learning increases with the epoch number.
Revised: INTERNAL, the number of agents participating in learning increases with the epoch number.

Original: It has shifted to a plateau by epoch
Revised: It has shifted to a plateau by epoch

Original: = 400.
Revised: = 400.

Original: At the beginning of learning, the entire agent does not contribute to emergence.
Revised: At the beginning of learning, the entire agent does not contribute to emergence.

Original: The language system understood by the entire agent was generated as learning progressed step by step.
Revised: The language system understood by the entire agent was generated as learning progressed step by step.

Original: In SWITCH, the learning intermediate point was set as the internal reﬂection function start time.
Revised: In SWITCH, the learning intermediate point was set as the internal reﬂection function start time.

Original: Because messages that can be reconstructed as Dream have been generated, messages corresponding to labels can be generated even with 70% learning participation.
Revised: Because messages that can be reconstructed as Dream have been generated, messages corresponding to labels can be generated even with 70% learning participation.

Revised: [Figure]

Original: E EXAMPLE OF DREAM RECONSTRUCTED FROM MESSAGES
Revised: E EXAMPLE OF DREAM RECONSTRUCTED FROM MESSAGES

Original: The quantitative evaluation in Section 4.2 simultaneously evaluates the accuracy and agent sharing for the label.
Revised: The quantitative evaluation in Section 4.2 simultaneously evaluates the accuracy and agent sharing for the label.

Original: We conﬁrmed the subjective characteristics in the reconstructed Dream.
Revised: We conﬁrmed the subjective characteristics in the reconstructed Dream.

Original: Using an agent set that conﬁrmed the formal rule of Section 4.4, we demonstrated that Dream corresponded to the input messages.
Revised: Using an agent set that conﬁrmed the formal rule of Section 4.4, we demonstrated that Dream corresponded to the input messages.

Original: Figure 7 shows the results of reconstructing Dream for all agents.
Revised: Figure 7 shows the results of reconstructing Dream for all agents.

Original: As shown, all learning objects have the same ability to reconstruct Dream.
Revised: As shown, all learning objects have the same ability to reconstruct Dream.

Original: Dream was reconstructed on many labels.
Revised: Dream was reconstructed on many labels.

Original: Some labels have low agent sharing for the messages.
Revised: Some labels have low agent sharing for the messages.

Original: For example, labels “6,” “8” for NORMAL, label “6” for INTERNAL, and labels “2,” “6” for SWITCH showed this tendency.
Revised: For example, labels “6,” “8” for NORMAL, label “6” for INTERNAL, and labels “2,” “6” for SWITCH showed this tendency.

Original: [Figure]
Revised: However, we conﬁrmed that all the same messages reconstructed Dream corresponding to the label.

Original: F LEAD SYMBOL COMPOSITION ANALYSIS
Revised: F LEAD SYMBOL COMPOSITION ANALYSIS

Original: Lead symbol composition is a reference for creating a formal rule.
Revised: Lead symbol composition is a reference for creating a formal rule.

Original: Therefore, referring to the symbol correspondence to the concept of Brighton & Kirby (2006), we conﬁrmed the correspondence between the lead symbol and the label.
Revised: Therefore, referring to the symbol correspondence to the concept of Brighton & Kirby (2006), we conﬁrmed the correspondence between the lead symbol and the label.

Original: Figure 8 shows that the symbol sequence was randomly generated regardless of the label at epoch = 0.
Revised: Figure 8 shows that the symbol sequence was randomly generated regardless of the label at epoch = 0.

Original: Unlike that reported in the main text, the lead symbol compares only the ﬁrst symbol.
Revised: Unlike that reported in the main text, the lead symbol compares only the ﬁrst symbol.

Original: One agent set was extracted from each learning object.
Revised: One agent set was extracted from each learning object.

Original: The composition state of the lead symbol and the symbol composition ratio with respect to the label are shown.
Revised: The composition state of the lead symbol and the symbol composition ratio with respect to the label are shown.

Original: NORMAL :
Revised: NORMAL :

Original: Figure 9 (Top) shows characteristic results.
Revised: Figure 9 (Top) shows characteristic results.

Original: In label analysis, most labels have their own symbols.
Revised: In label analysis, most labels have their own symbols.

Original: However, it was found that the owning symbol originated from a speciﬁc head symbol and was biased.
Revised: However, it was found that the owning symbol originated from a speciﬁc head symbol and was biased.

Original: This indicates that various labels concentrate on the same head symbol.
Revised: This indicates that various labels concentrate on the same head symbol.

Original: This agent set shows that labels are concentrated on symbols “b” and “d.
Revised: This agent set shows that labels are concentrated on symbols “b” and “d.”

Original: INTERNAL :
Revised: INTERNAL :

Original: Figure 9 (middle) shows that the symbols are used equally.
Revised: Figure 9 (middle) shows that the symbols are used equally.

Original: In this agent set, “c” corresponds to “4,” “7,” “9” and “d” corresponds to “0,” “5,” “6.”
Revised: In this agent set, “c” corresponds to “4,” “7,” “9” and “d” corresponds to “0,” “5,” “6.”

Original: As symbols are used more equally, in the label analysis, many labels established owning relationships with particular symbols.
Revised: As symbols are used more equally, in the label analysis, many labels established owning relationships with particular symbols.

Original: SWITCH : Figure 9 (bottom) shows that the symbols are used equally as in INTERNAL.
Revised: SWITCH : Figure 9 (bottom) shows that the symbols are used equally as in INTERNAL.

Original: Additionally, owning relationships are shown from label analysis.
Revised: Additionally, owning relationships are shown from label analysis.

Original: However, it appeared that some labels were constructed with exactly half of the owning symbols (for example, labels “3” and “5” simultaneously used “a” and “d”, and label “7” simultaneously used “b” and “c”).
Revised: However, it appeared that some labels were constructed with exactly half of the owning symbols (for example, labels “3” and “5” simultaneously used “a” and “d”, and label “7” simultaneously used “b” and “c”).

Original: Labels with bimodality tended to appear in SWITCH.
Revised: Labels with bimodality tended to appear in SWITCH.

Original: G DREAM RECONSTRUCTION SAMPLE OF MESSAGES WITH FORMAL RULES
Revised: G DREAM RECONSTRUCTION SAMPLE OF MESSAGES WITH FORMAL RULES

Original: We conﬁrmed the Dream that is reconstructed from the message, and established the formal rule.
Revised: We conﬁrmed the Dream that is reconstructed from the message, and established the formal rule.

Original: In each learning object, the formal rule was constructed based on the generated messages.
Revised: In each learning object, the formal rule was constructed based on the generated messages.

Original: We entered the message individually and built a formal rule based on the reconstructed Dream’s relationship.
Revised: We entered the message individually and built a formal rule based on the reconstructed Dream ’ s relationship.

Revised: [Figure]

Original: Figure 10 shows a message for each label and the corresponding Dream.
Revised: Figure 10 shows a message for each label and the corresponding Dream.

Original: For each learning object, two characteristic formal rules are shown.
Revised: For each learning object, two characteristic formal rules are shown.

Original: [Figure] NORMAL : labels “0” and “7” are targeted (Figure 10, (a)).
Revised: NORMAL : labels “0” and “7” are targeted (Figure 10, (a)).

Original: For label 0: Dream was reconstructed by message [a]*.
Revised: For label 0: Dream was reconstructed by message [a]*.

Original: However, label 0 could not be reconstructed when the number of symbols in “a” was small.
Revised: However, label 0 could not be reconstructed when the number of symbols in “a” was small.

Original: A single “a” symbol was reconstructed for label “5.”
Revised: A single “a” symbol was reconstructed for label “5.”

Original: For label “7”: Dream was reconstructed by message [b]*.
Revised: For label “7”: Dream was reconstructed by message [b]*.

Original: However, if the number of “b” symbols was less than ﬁve digits, it was reconstructed to another label (“4” or “9”).
Revised: However, if the number of “b” symbols was less than ﬁve digits, it was reconstructed to another label (“4” or “9”).

Original: Hence, NORMAL has a formal rule by the key symbol.
Revised: Hence, NORMAL has a formal rule by the key symbol.

Original: This occurs when the reconstructed Dream is different owing to on the length of the symbol string.
Revised: This occurs when the reconstructed Dream is different owing to on the length of the symbol string.

Original: The formal rule includes the length of the symbol string.
Revised: The formal rule includes the length of the symbol string.

Original: INTERNAL : labels “5” and “8” are targeted (Figure 10, (b)).
Revised: INTERNAL : labels “5” and “8” are targeted (Figure 10, (b)).

Original: For label “5”: “d + [bd]*” ‘s formal rule appeared.
Revised: For label “5”: “d + [bd]*” ‘ s formal rule appeared.

Original: A clear pattern was structured on this label; e.g., “dbddbd” could not be reconstructed.
Revised: A clear pattern was structured on this label; e.g., “dbddbd” could not be reconstructed.

Original: For label “8”: “b,” “c” and “d” are required in the formal rule.
Revised: For label “8”: “b,” “c” and “d” are required in the formal rule.

Original: Since Dream was generated successfully, it was shown that it was patterned including the symbol “c”.
Revised: Since Dream was generated successfully, it was shown that it was patterned including the symbol “c”.

Original: Hence, strict and robust labels existed for the pattern of the message to be reconstructed in Dream.
Revised: Hence, strict and robust labels existed for the pattern of the message to be reconstructed in Dream.

Original: SWITCH : labels “2” and “9” are targeted (Figure 10, (c)).
Revised: SWITCH : labels “2” and “9” are targeted (Figure 10, (c)).

Original: For label “2”:
Revised: For label “2”:

Original: The formal rule was most difﬁcult to obtain.
Revised: The formal rule was most difﬁcult to obtain.

Original: However, two symbol strings, such as “bbdba” and “bdbab,” were successfully  reconstructed in Dream.
Revised: However, two symbol strings, such as “bbdba” and “bdbab,” were successfully [Figure] reconstructed in Dream.

Original: This was a characteristic formal rule of a ﬁve-digit symbol string as the key.
Revised: This was a characteristic formal rule of a ﬁve-digit symbol string as the key.

Original: For label “9”: Dream was reconstructed for the message of [cd]*.
Revised: For label “9”: Dream was reconstructed for the message of [cd]*.

Original: The “c” symbol was the reconstructed Dream of labels “4” and “7.”
Revised: The “c” symbol was the reconstructed Dream of labels “4” and “7.”

Original: However, the “d” symbol showed a direction to label “9.”
Revised: However, the “d” symbol showed a direction to label “9.”

Original: For label “9,” we conﬁrmed that the combination “c + d” was the key pattern.
Revised: For label “9,” we conﬁrmed that the combination “c + d” was the key pattern.

Original: H CHARACTERISTIC OF MASSAGES WHERE DREAM RECONSTRUCTION IS SHOWN AT THE SAME SIMILARITY
Revised: H CHARACTERISTIC OF MASSAGES WHERE DREAM RECONSTRUCTION IS SHOWN AT THE SAME SIMILARITY

Original: The condition of the message in Dream reconstruction was conﬁrmed from messages with similarity 0.5.
Revised: The condition of the message in Dream reconstruction was conﬁrmed from messages with similarity 0.5.

Original: Because a message with low similarity was distant from the messages to be reconstructed, the
Revised: Because a message with low similarity was distant from the messages to be reconstructed, the

Original: Dream reconstruction was considered difﬁcult.
Revised: Dream reconstruction was considered difﬁcult.

Original: Hence, we analyzed the pattern of the message to be reconstructed as Dream (Figure 11).[Figure]
Revised: Hence, we analyzed the pattern of the message to be reconstructed as Dream (Figure 11).

Original: In all the learning objects, even if the degree of similarity of the formal rule was approximately 0.5, messages capable of Dream reconstruction existed.
Revised: In all the learning objects, even if the degree of similarity of the formal rule was approximately 0.5, messages capable of Dream reconstruction existed.

Original: Dream reconstruction was observed in messages[Figure] comprising patterns or symbols according to the formal rule.
Revised: Dream reconstruction was observed in messages comprising patterns or symbols according to the formal rule.

Original: The Normal and Internal reﬂection functions showed every different feature in the messages when the similarity was low.
Revised: The Normal and Internal reﬂection functions showed every different feature in the messages when the similarity was low.

Original: In the Internal reﬂection function, Dream tended to be reconstructed when a repeated pattern of formal rules in the target message existed (Figure 12 showed labels “4” and “7” for INTERNAL, and labels “5” and “8” for SWITCH.)
Revised: In the Internal reﬂection function, Dream tended to be reconstructed when a repeated pattern of formal rules in the target message existed (Figure 12 showed labels “4” and “7” for INTERNAL, and labels “5” and “8” for SWITCH.)

Original: Meanwhile, in NORMAL, Dream reconstruction might not succeed even if formal rules exist repeatedly.
Revised: Meanwhile, in NORMAL, Dream reconstruction might not succeed even if formal rules exist repeatedly.

Original: A tendency to be reconstructed existed when the minimum necessary symbol appeared.
Revised: A tendency to be reconstructed existed when the minimum necessary symbol appeared.

Original: In the Internal reﬂection function, patterns in the formal rules were inviting for the rebuilding of DREAM even when the degree of similarity was low.
Revised: In the Internal reﬂection function, patterns in the formal rules were inviting for the rebuilding of DREAM even when the degree of similarity was low.

Original: Meanwhile, in NORMAL, the symbols presented in the formal rule were the minimum requirements for Dream reconstruction.
Revised: Meanwhile, in NORMAL, the symbols presented in the formal rule were the minimum requirements for Dream reconstruction.

Original: I VISUALIZATION OF LANGUAGE MODULE REPRESENTATION
Revised: I VISUALIZATION OF LANGUAGE MODULE REPRESENTATION

Original: We conﬁrmed the cluster structuring of the language module using the generated messages.
Revised: We conﬁrmed the cluster structuring of the language module using the generated messages.

Original: Using t-sne (Maaten & Hinton, 2008), the four-dimensional output was compressed to a two-dimensional output (Figure 12).
Revised: Using t-sne (Maaten & Hinton, 2008), the four-dimensional output was compressed to a two-dimensional output (Figure 12).

Original: For evaluation, Agent No.10 was used.
Revised: For evaluation, Agent No.10 was used.

Original: Messages with a similarity of 0.7 or higher were plotted.
Revised: Messages with a similarity of 0.7 or higher were plotted.

Original: The parameters used in t-SNE were iter =
Revised: The parameters used in t-SNE were iter =

Original: 2500 and perplexity = 100.
Revised: 2500 and perplexity = 100.

Original: [Figure]

Original: When symbols in the message have different tendencies in each learning object, they are represented in a space separated for each label.
Revised: When symbols in the message have different tendencies in each learning object, they are represented in a space separated for each label.

Original: However, when symbols in the message are similar, the tendency is different depending on the learning objects.
Revised: However, when symbols in the message are similar, the tendency is different depending on the learning objects.

Original: INTERNAL and SWITCH showed signs of spatial separation even in messages with the same tendency.
Revised: INTERNAL and SWITCH showed signs of spatial separation even in messages with the same tendency.

Original: For INTERNAL, in the formal rule of label 05-6, “d” is a common lead symbol.
Revised: For INTERNAL, in the formal rule of label 05-6, “d” is a common lead symbol.

Original: However, a cluster was conﬁrmed for each label.
Revised: However, a cluster was conﬁrmed for each label.

Original: In the SWITCH  4-7-9 formal rule, “c” is the lead symbol.
Revised: In the SWITCH [Figure] 4-7-9 formal rule, “c” is the lead symbol.

Original: Despite the overlap, clusters were created for each label.
Revised: Despite the overlap, clusters were created for each label.

Original: Similar trends were shown in INTERNAL label’s 1-3-8 and SWITCH label’s 1-2-7.
Revised: Similar trends were shown in INTERNAL label’s 1-3-8 and SWITCH label’s 1-2-7.

Original: The formal rule of those labels is constructed to repeat speciﬁc symbols.
Revised: The formal rule of those labels is constructed to repeat speciﬁc symbols.

Original: Meanwhile, in NORMAL, a speciﬁc label could not be independent although a cluster was generated in a formal rule containing the same symbol.
Revised: Meanwhile, in NORMAL, a speciﬁc label could not be independent although a cluster was generated in a formal rule containing the same symbol.

Original: For example, labels “1” and “2” have formal rules sandwiched between symbols “d.”
Revised: For example, labels “1” and “2” have formal rules sandwiched between symbols “d.”

Original: However, the cluster at label “2” was included on top of label “1.”
Revised: However, the cluster at label “2” was included on top of label “1.”

Original: Additionally, clusters were formed at the same positions as labels “5” and “8.” 
Revised: Additionally, clusters were formed at the same positions as labels “5” and “8.” [Figure]

Original: The internal reﬂection function shows that the generated messages not only appears as a formal rule, but also affects the representation space.
Revised: The internal reﬂection function shows that the generated messages not only appears as a formal rule, but also affects the representation space.

Original: We considered a new research topic for creating recursive relationships that disentangled the acquired representations according to the acquired language.
Revised: We considered a new research topic for creating recursive relationships that disentangled the acquired representations according to the acquired language.