Acquisition and Exchange of Knowledge From Real to Virtual Embodiment

Today’s computer power enables us to create software agents that can process large amounts of data in very short time. Higher level cognitive processes, however, still remain the domain of humans. The aim of our research is to combine the power of both sides, to realize virtual agents that provide capable assistance to their users. In our approach we do not attempt to mimic human cognition. Rather, we enable the agents to learn via self-experience, from positive and negative feedback by the user, and from communication with other agents of their kind using grounded and agreed upon symbols. The design of the agents is inspired by insights from embodied cognition in particular from affordance-based robotics that are transferred to a virtual context. The Rascalli Objective In the RASCALLI (Responsive Artificial Situated Agents that Live and Learn on the Internet) project, we aim at virtual agents, the Rascalli, that are capable of autonomously exploring the Internet, and of communicating with the user and with each other. The agents have to deal with an environment that is constituted of data including strings of written language, markup tags, audio and image files, log-files of user activities, etc. For an agent to explore its inherently dynamic environment according to the user’s interests it needs to gain a certain degree of autonomous and flexible behaviour. To achieve this, we transfer insights from affordance-based research initiated by Gibson (Gibson 1986) and from affordance-based robotics research (Kintzler et al. 2007) to the design of the virtual agents and their environmental framework. Thus we develop agents whose behaviour is biologically inspired and who gain knowledge clearly different from human knowledge, but grounded on the theory of human use of affordances which amongst others forms the basis to realize communication between agents, and in the long run between agent and user. Affordance-based Approach Following the affordance approach, the environment is not perceived in terms of object or structure descriptions, but in an action oriented way, based on previously made experiences by the agent. The focus lies on what can be done with the structures in the environment what they afford in relation to the acting agent. Thereby affordances describe the relation between the agent’s capabilities and its environment. For example, an action close fingers applied to a small object will result in an outcome gripped and therefore the affordance grippability exists between the agent and a small object of a certain width. Representing knowledge in that manner offers a basis for flexibility in performing tasks, e.g. if a human being wants to hammer a nail into a wall has therefore a desired outcome in mind and no hammer is currently present, she is flexible in finding objects that can be used to hammer a nail even if they do not have the body structure of a hammer. Solving such a task involves experience fragments about wall structures and materials, liftability of objects, how to control one’s own body, etc. that are combined and used for this particular situation. In humans this experience fragments are gained via a life-long interaction and learning process, rooted in infancy and childhood. To gain the mentioned flexibility in solving tasks requires that the represented interaction-based knowledge is at a level of abstraction that allows generalisation and dealing with novel situations. The size of this experience base of an agent correlates with the agent’s flexibility to adapt to different situations. Exploring the environment, the own interaction capabilities and outcomes of applying actions on the environment provide an experience base for further more purposeful interactions. In humans and also animals this is equivalent to play and exploration. Transferring this biologically inspired interaction-based approach to the Rascalli agents enables the agents to gain knowledge about what they can do based on their own interaction capabilities and their own perception. This provides the basis for the required autonomous und flexible behaviour. As a very simple example, flexibility is required if a search engine or a database is unreachable. In this situation the virtual agent can choose a different one by knowing that the functionality and outcome of an alternative resource is similar. Flexibility is also required to adapt to different users and their interests – to know what types of output based on given input are appreciated by an individual user and what tools can be applied to provide the desired output. Virtual Agents Inspired by Robotics To realize an affordance based architecture for virtual environments, a virtual embodiment for our agents is created. They are equipped with a collection of sensor channels geared towards the particular environment, and a set of specialized software tools (actions) through which they interact with the environment. The outcomes of the tools (of applying actions on the environment) are again treated as an input to the agent and perceived via the sensors channels in a way similar to a robot’s perception of the consequences of an action application. Depending on the developmental stage of the agent, tools are selected and executed arbitrarily, motivated by drives, or deliberately chosen based on the given input and previously made experience. Tools can be cascaded, whereby an output of a tool is part of the input of the succeeding tool. In this manner action chains are realized. After tool execution the environment, which amongst others contains the consequences of the tool application as well as user feedback and internal states of the agent, is again perceived. For each tool application the perceptions of the pre-application (I) and the post-application (O) phase are stored, over time leading to tool-specific (T) application spaces containing all the episodes experienced with the individual tools. By finding similarities and deriving representative descriptions from the individual episodes of an application space a generalization process takes place (Irran et al. 2006). These I-T-O (input-tooloutcome) triplets form the agent’s experience base which enables the agent to gain representations of general input situations and desired outcomes of tool applications. It allows the agent to act more purposefully on future input. Since the episodes related with each tool application include user input and feedback, the agent does not only know what it can do, but also what is appreciated by the user and what is not. This together with the agent’s drive to get positive feedback from the user forms the basis for learning successful interactions in its environment. The Role of Communication To enrich the capabilities of the Rascalli agents they are equipped with symbol acquiring and communication mechanisms. This allows agents to share their experiences obtained through interaction with the environment. Each agent has its own experience base comprising generalizations over the outcomes (O) of action applications, and generalizations of the input types (I) tools (T) can apply to. Due to the differences in their experience bases a negotiation process between Rascalli takes place establishing common labels for their individual knowledge of inputs and outcomes. Such an agreement process includes several cycles of exchanging prototypes and/or single episodes. For instance an agent A provides a prototype description d to another agent B, B tries to match d to its own experience base. Given a successful match, the agents choose a common label depending on the existence of labels from previous negotiations. In case the match at prototype level is not successful, the agents resort to instance level. This may lead to the creation of new prototypes in the agents which form a new basis for establishing a common label. If the negotiation at instance level fails, the agents’ current experience bases are too distant. This however may change with further acquisition of knowledge. Using a set of labels, agents can exchange information more efficiently than by exchanging prototype representations or instance data. Since the shared labels are grounded in each agent’s experience due to virtual embodiment and the affordance approach, knowledge exchange is possible even though the agents do not share the same internal representations. With the agreed labels, agents can exchange task solving strategies: which tools or tool chains to use in a given situation, how to react on a given input, or reach a desired outcome. For example, an agent may present user input to another agent and ask for a recommendation what tool to use. This reduces the search space of individual agents for finding applicable tool or tool chain usage and increases the probability of satisfying the user. With the presented affordance-inspired approach, we have established a basis which allows us to systematically explore how the design of an environment and the sensor channels of a virtual agent influence the acquisition of knowledge via self-experience, how grounded and negotiated labels can lead to shared knowledge among individual agents, how the distribution of knowledge in groups of agents is influenced by different strategies of label agreement, and which strategies of communication and learning best increase the agent’s capabilities in assisting the user.