Patent ID: 12242944

DETAILED DESCRIPTION

The following description contains specific information pertaining to implementations in the present disclosure. One skilled in the art will recognize that the present disclosure may be implemented in a manner different from that specifically discussed herein. The drawings in the present application and their accompanying detailed description are directed to merely exemplary implementations. Unless noted otherwise, like or corresponding elements among the figures may be indicated by like or corresponding reference numerals.

The present application is directed to automated systems and methods enabling a virtual agent to simulate human-like affect-driven behavior that address and overcome the deficiencies in the conventional art.

It is noted that, as used in the present application, the terms “automation,” “automated”, and “automating” refer to systems and processes that do not require the participation of a human interaction editor or guide. Although, in some implementations, a human editor may review or even modify a behavior determined by the automated systems and according to the automated methods described herein, that human involvement is optional. Thus, the methods described in the present application may be performed under the control of hardware processing components of the disclosed automated systems.

FIG.1shows a diagram of a system enabling a virtual agent to simulate human-like affect-driven behavior, according to one exemplary implementation. As shown inFIG.1, system100includes computing platform102having hardware processor104, system memory106implemented as a non-transitory storage device, input module130, and output module124. According to the present exemplary implementation, system memory106stores interaction history database108, story and character library112, and software code110providing virtual social agent150(hereinafter “virtual agent150”).

As further shown inFIG.1, system100is implemented within a use environment including communication network120, character146ain the form of a robot or other type of machine, one or more guest objects148(hereinafter “guest object(s)148”), and guest system140including display screen142and optional keyboard or other input device144. In addition,FIG.1shows network communication links122of communication network120interactively connecting guest system140and character146awith system100. Also shown inFIG.1is character146bin the form of a virtual character rendered on display screen142, human guest users126aand126b(hereinafter “guests126aand126b”) having respective interactions128aand128bwith respective characters146aand146b, and interaction128cbetween character146aand guest object(s)148.

It is noted that, although the present application refers to software code110providing virtual agent150as being stored in system memory106for conceptual clarity, more generally, system memory106may take the form of any computer-readable non-transitory storage medium.

The expression “computer-readable non-transitory storage medium,” as used in the present application, refers to any medium, excluding a carrier wave or other transitory signal that provides instructions to hardware processor104of computing platform102. Thus, a computer-readable non-transitory medium may correspond to various types of media, such as volatile media and non-volatile media, for example. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM), while non-volatile memory may include optical, magnetic, or electrostatic storage devices. Common forms of computer-readable non-transitory media include, for example, optical discs, RAM, programmable read-only memory (PROM), erasable PROM (EPROM), and FLASH memory.

It is further noted that althoughFIG.1depicts software code110, interaction history database108, and story and character library112as being co-located in system memory106, that representation is merely provided as an aid to conceptual clarity. More generally, system100may include one or more computing platforms102, such as computer servers for example, which may be co-located, or may form an interactively linked but distributed system, such as a cloud based system, for instance. As a result, hardware processor104and system memory106may correspond to distributed processor and memory resources within system100.

According to the implementation shown byFIG.1, guest system140and character146ainteract with system100over communication network120. In one such implementation, computing platform102may correspond to one or more web servers, accessible over a packet-switched network such as the Internet, for example. Alternatively, computing platform102may correspond to one or more computer servers supporting a local area network (LAN), or included in another type of limited distribution network, such as a private wide area network (WAN) or private cloud for example.

Although guest system140is shown as a desktop computer inFIG.1, that representation is also provided merely as an example. More generally, guest system140may be any suitable mobile or stationary device or system that implements data processing capabilities sufficient to support connections to communication network120, and implement the functionality ascribed to guest system140herein. For example, in other implementations, guest system140may take the form of a free standing or wall mounted display, laptop computer, tablet computer, smartphone, smart TV, or gaming console. It is noted that display screen142of guest system140may take the form of a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or another suitable display screen that performs a physical transformation of signals to light.

It is further noted that althoughFIG.1shows both input module130and output module124as residing on computing platform102, that representation is also merely exemplary. In other implementations, input module130and/or output module124may be integrated with character146ain the form of a robot or other type of machine. In other words, in some implementations, character146amay include input module130and/or output module124.

FIG.2shows a more detailed diagram of input module230suitable for use in system100, inFIG.1, according to one implementation. As shown inFIG.2, input module230includes multiple character sensors232, multiple environmental sensors234, multiple guest sensors236, and microphone or microphones (hereinafter “microphone(s)”)238. Input module230corresponds in general to input module130, inFIG.1. Thus, input module130may share any of the characteristics attributed to input module230by the present disclosure, and vice versa.

According to the exemplary implementation shown inFIG.2, character sensors232of input module130/230may include one or more cameras (hereinafter “camera(s)”)232aand inertial measurement unit (IMU)232b. Camera(s)232amay include one or more red-green-blue (RGB) still image cameras and/or one or more RGB video cameras, for example. IMU232bmay include one or more accelerometers, one or more gyroscopes, and in some implementations, a magnetometer. It is noted that the specific sensors shown to be included among character sensors232are merely exemplary, and in other implementations, character sensors232may include more, or fewer, sensors than camera(s)232aand IMU232b. Moreover, in other implementations, character sensors232may include a sensor or sensors other than camera(s)232aand IMU232b.

As further shown inFIG.2, environmental sensors234of input module130/230may include temperature sensor234a, light sensor234b, ambient humidity/rain sensor234c, and radio-frequency (RF) communications receiver234dcapable of receiving local weather and climate data. Analogously to character sensors232described above, the specific sensors shown to be included among environmental sensors234are merely exemplary, and in other implementations, environmental sensors234may include more, or fewer, sensors than temperature sensor234a, light sensor234b, ambient humidity/rain sensor234c, and RF communications receiver234d. Moreover, and also by analogy to character sensors232, in other implementations, environmental sensors234may include a sensor or sensors other than temperature sensor234a, light sensor234b, ambient humidity/rain sensor234c, and RF communications receiver234d.

In one implementation, guest sensors236of input module130/230may include radio-frequency identification (RFID) sensor236a, facial recognition (FR) sensor236b, automatic speech recognition (ASR) sensor236c, object recognition (OR) sensor236d, and guest response sensor236e. The specific sensors shown to be included among guest sensors236are merely exemplary, and in other implementations, guest sensors236may include more, or fewer, sensors than RFID sensor236a, FR sensor236b, ASR sensor236c, OR sensor236d, and guest response sensor236e. Moreover, in other implementations, guest sensors236may include a sensor or sensors other than one or more of RFID sensor236a, FR sensor236b, ASR sensor236c, OR sensor236d, and guest response sensor236e. It is noted that in some implementations, input module130/230may be configured to receive manual inputs from guest126bvia a computer mouse or track pad, optional keyboard144, or a touch screen display corresponding to display screen142.

FIG.3shows a diagram of exemplary virtual agent350provided by software code100, inFIG.1, according to one implementation. It is noted that virtual agent350corresponds in general to virtual agent150, inFIG.1. Thus, virtual agent150may share any of the characteristics attributed to virtual agent350by the present disclosure, and vice versa.

By way of overview, and referring back toFIG.1, it is noted that, in some exemplary implementations, guests126aand126bmay interact with respective characters146aand146bthat may inhabit a story world (W) of a story having a timeline and a narrative arc or plot. Story world W is a blend of a virtual world and the real world, and can be changed by guests126aand126bas well as characters146aand146b. Software code110providing virtual agent150/350, when executed by hardware processor104, may control virtual agent150/350to simulate human-like affect-driven behavior, such as a human-like social interaction by one or both of characters146aand146bwith respective guests126aand126b.

It is further noted that, in some implementations, characters146aand146bmay be different characters, while in other implementations, characters146aand146bmay be different versions or instantiations of the same character. It is also noted that, in some implementations, even in lieu of an interaction between a character and one or more guests or guest object(s), the affective state of the character, including for example a personality profile, a mood, a physical state, an emotional state, and a motivational state of the character, can continue to evolve with advancement of the story including the character.

According to the exemplary implementation shown inFIG.3, virtual agent150/350determines the behavior of a character, i.e., one of characters146aand146bbased on the affective parameters Emotion356(E), Motivation358(M), Personality360(P), and Mood362(B), as well as the State354(S) of that character. Each of the affective parameters Emotion356, Motivation358, Personality360, and Mood362is discussed in detail below.

With respect to the state354of a character, it is noted that state354combines a description of the story world (the world state) and a description of the character's physical state. The world state may be represented by an ever updating knowledge base (KB). For example, virtual agent150/350transforms the sensory signals received from input module130/230into domain knowledge including states corresponding to all objects in the story world W (including guests) that are perceivable to a character. The physical state may be represented by continuous variables that model the internal physical conditions of the character, e.g., such as hunger or tiredness.

The characters, e.g., characters146aand146bare under the control of system100and live only in the virtual world. The guests, e.g., guests126aand126bare not controlled by the system and interact with the characters from the real world. It is noted that guest object(s)148is something of a hybrid. For example, in some implementations, guest object(s)148may be one or more inanimate or non-autonomous object. However, in other implementations, like characters146aand146b, guest object(s)148may be one or more characters under the control of system100, such as one or more characters in the form of a robot of other machine and/or one or more virtual characters rendered on a display screen, that nevertheless assumes the role of a guest in an interaction with character146aor146b. It is emphasized that creating a believable control of the characters for the guests is the main objective of virtual agent150/350.

The KB of a character can change if a change in the world is perceived by the character, e.g., through a volitional act by guest126aor126bor the character, or through a non-volitional act by the environment. The KB of the character can also be changed directly by the character, e.g., by asserting something to be true, planning something, deriving a fact, and so forth.

Personality360(P): The personality360of a character may be modeled using the five-factor model (FFM) known in the art. The FFM applies the five factors: 1) Openness to new experience, 2) Conscientiousness, 3) Extraversion, 4) Agreeableness, and 5) Neuroticism to characterize a personality. Due to the five specific factors applied, the FFM is also referred to as the OCEAN model. Personality360can be modeled as a vector p∈P, where each trait piis assigned a value between [0, 1]. The personality360of a character influences its emotional state and behavior. It is noted that although the present implementation five-factor OCEAN model of personality, in other implementations, other personality models known in the art may be utilized.

Motivation358(M): The motivations of a character can be modeled based on the Reiss Motivation Profile (RMP™), known in the art. According to Reiss, there are sixteen fundamental motivations: Power, Curiosity, Independence, Status, Social contact, Vengeance, Honor, Idealism, Physical exercise, Romance, Family, Order, Eating, Acceptance, Tranquility, and Saving. Referring toFIG.3, the motivational state of a character can be modeled using two 16-dimensional vectors. One of the vectors, mc∈M, represents the current state366of motivational fulfillment of the character, and the other vector, md∈M, represents the target state368of motivational fulfillment of the character. Each dimension of these vectors corresponds to one of the Reiss fundamental motivations and is assigned a value between [0,1], and the character seeks to minimize the distance between mdand mc, i.e., the difference between its target state of motivational fulfillment and its current state of motivational fulfillment.

According to some implementations, forward planner376with an A* heuristic search algorithm is used to produce a plan πA*to move current state366of motivational fulfillment closer to target state368. The heuristic may be expressed as a weighted distance between the two motivational vectors, mcand md.

d⁡(mc,ma,w)=(∑11⁢6wi·(mid-mic)2)1/2,(Equation⁢1)
where motivation weight vector370, w, defines the importance given to each of the dimensional values, as determined by the personality and emotional state of the character. As noted above, the distance between mdand mcrepresents how far the character is from fulfilling its motivations, and the search algorithm attempts to choose behaviors that decrease this distance. Once the heuristic search has found a plan that adequately satisfies current motivations with respect to target motivations, the plan πA*is sent to behavior manager378.

It is noted that the heuristic expressed as Equation 1 is merely provided as an example. In other implementations, another heuristic could be utilized, provided that other heuristic captures the influence of personality and correlates with distance between target state368of motivational fulfillment and current state366of motivational fulfillment of the character.

Emotion356(E): Emotions reflect a short-term affect that arises as a result of stimuli from the environment of a character. In one implementation, a set of predetermined rules may be used to map State354of the character to an instance of emotion type ei∈E and its intensityI(ei). The twenty-one emotion types eiin the OCC theory of Ortony, Clore, and Collins (hereinafter “OCC emotions”) may be used and may be differentiated by i. After an emotion eiis generated at time t0and assigned an initial intensityIt0(ei), that intensity may decay as a function of time according to:
It(ei)=It0(ei)·e−β(t-t0),(Equation 2)
where the constant β determines how fast the intensity of the particular emotion eiwill decrease over time. Once an emotion is generated, it is stored in the list of active emotions364(ϵ) until its intensity falls below a predetermined threshold near zero. It is noted that, as discussed in greater detail below, the initial intensity of an emotion, It0(ei), is modulated by the personality traits p and current mood b of a character.

Virtual agent150/350monitors the intensity of active emotions (364) ϵ, and when the intensity of ei∈ϵ crosses an emotional threshold τe(i.e., I(ei)>τe), virtual agent150/350propagates a goal gei=(emotionalreactionei) to emotional reaction planner374to produce an appropriate plan πGP(gei). Compared to heuristic based forward planner376, emotional reaction planner374does not operate in the 16-dimensional motivation space, but rather has a fixed goal in the State354space of the character and operates in a conventional Stanford Research Institute Problem Solver (STRIPS) style backward search.

State354(S): State354includes both a representation of the current physical state of the character and a representation of the story world in the form of a knowledge base (KB). For example, a physical state manager may track variables that describe the physical condition of the character, such as tiredness, depth-of-sleep, hunger, and the like. Generally, these physical states are continuous variables that are modeled by an internal dynamical system. Additionally, the dynamical system may be influenced by external stimuli (e.g., hunger can be reduced by the behavior of eating, or increase when a nice meal is perceived, visually or through smell). Physical states are responsible for creating behavioral triggers, and may also be used to influence the KB. These triggers can directly result in producing hardcoded reactions355. The state of the KB is used in the planning process as it determines when an action can be applied, which may be modeled with a precondition, and it is used to describe the outcome of an action, which may be modeled with a postcondition. It is noted that this includes the planning process of heuristic based forward planner376and the planning process of emotional reaction planner374.

FIG.4presents pseudocode for exemplary deliberation algorithm400utilized by virtual150/350, according to one implementation, that more formally describes the deliberations associated with plans πA*and πGP. It is noted that at every time step of deliberation algorithm400, active emotions364(ϵ) are decayed and filtered, while newly generated emotions are added. If a sufficiently intense emotion is present, an emotional reaction πGPcould be planned for, or, alternatively, motivation weight vector370(w) could be modified based on the experienced emotion. If an emotional reaction is not underway, forward planner376(A*) is tasked with finding a motivationally driven plan πA*based on Equation 1.

Mood362(B): Mood362is distinguished from emotion356by its resolution and relative stability over time. The mood of a character b∈B can be described using three traits: Pleasure (P), Arousal (A), and Dominance (D) as a vector in Pleasure-Arousal-Dominance space (PAD space) where each dimension ranges from negative one to one (−1-1). In addition, a baseline or default mood (b0) of the character is identified based on a mapping between the FFM personality traits of the character and the PAD space. For example, the following mapping developed by Gebhard, known in the art, may be utilized:
Pleasure=0.21*Extraversion+0.59*Agreeableness+0.19*Neuroticism
Arousal=0.15*Openness+0.30*Agreeableness−0.57*Neuroticism
Dominance=0.25*Openness+0.17*Conscientiousness+0.60*Extroversion−0.32*Agreeableness

The mood of a character at a given time (i.e., current mood372(bt)) is determined by the baseline mood and active emotions of the character. Because mood362is modeled in PAD space and the OCC emotion types represent the emotions, the emotions can be mapped to the PAD space: ϕ(e): E→B, where B=PAD. Once the emotions are represented in PAD space, they can be combined to describe an effective mood (bteff), which is used at each time step t to “pull” on the current mood bt:
bt=(1−α)·bt-1+α·bteff.  (Equation 3)
where α parameterizes the strength of the pull mechanism. It is noted that in one implementation α is approximately 0.01. The effective mood is determined based on the default mood of the character, which incorporates the personality360of the character, and the set of all currently active emotions:
bteff=ω0·b0+Σi∈Eωieff(t)·ϕ(ei),  (Equation 4)
ωieff(t)=min(1,Σej∈ϵIt(ej)·(ei, ej)),  (Equation 5)
where ω0is a weighting of the baseline mood, j iterates over all active emotions ϵ, ϕ(ei) is the mapping of the i'th emotion to the PAD space, and(ei, ej) is an indicator function equal to one when i=j, and equal to zero when i≠j. Thus, mood362is pulled toward the point in PAD space that reflects the joint set of all currently active emotions and the default mood of the character. Consequently, when in isolation, the mood362of the character will converge to the baseline mood.

As defined above by Equation 2, the intensity of an active emotion exponentially decays over time It0. However, the initial strength of the emotion can vary widely between the different emotions ei. That variation can be captured based on the predetermined emotional trigger that generates the emotion, the current mood, and the personality:
It0(ei)=Ik·(Ib→ei+Ip→ei)/2,  (Equation 6)
where Ikdescribes the predetermined strength of the k'th emotional trigger, Ib→eiand Ip→eiare multiplicative influences based on the respective mood and personality of the character. While the emotional trigger typically holds the most influence over the intensity of an emotion, the influence of the character's mood and personality over the intensity offers an empirically-based way to incorporate differences between characters, as well as varying emotional context. To that end, the following equations may be used:
Ib→ei=1+|bt|·((ϕ(ēi),bt)−(ϕ(ēi),−bt))   (Equation 7)
Ip→ei=1+Σj∈Ppj·(pj, ei),  (Equation 8)
wherebencodes the discretization of the PAD position to the corresponding mood-octant (−bcorresponds to the inverse octant) pjis the j'th parameter of the FFM personality traits of the character, andis the mapping between personality and emotion shown by table500inFIG.5.

It is noted that while Equations 7 and 8 can be replaced with other formalisms, it is advantageous to utilize formalisms that, like Equations 7 and 8, are able to capture the main empirically-based influences of various affective components on emotion. That is to say: (a) the intensity of an experienced emotion that is close to current mood372is strengthened (or weakened if the experienced emotion is far from current mood372), and (b) the personality of a character can up or down regulate an emotion. Moreover, by modeling the influence of mood on emotion, the mood of the character influences behavior through the emotional state of the character. For example, if a character experiences a negative emotion while already in a bad mood, the mood might enhance the intensity of the emotion and drive the character to an emotional reaction. Alternatively, the character might behave by regulating the emotion if its mood is such that the intensity of the emotion is attenuated.

The emotions356experienced by a character can also affect the motivational state of the character by changing the motivation weight vector370, w, included in Equation 1. Formally, the motivation weight vector is modified, w→w′, to influence the motivational heuristic used by forward planner376. An emotional intensity range may be defined, [ILB, τe], within which w is influenced, where ILBand τe, respectively, are the lower and upper bounds of the range.

It is noted that, if the intensity of an emotion exceedsτe, an emotional reaction is triggered, as discussed above. As the intensity of the emotion decays, as shown for example by Equation 2, its weighting of the motivational dimensions of w′ weakens, i.e., w′ converges back to w. If an emotion, ei, has an intensity that is within the defined range (i.e., ILB<It(ei)<τe)), the modified weight vector may be defined as:

wi′={wi·(1+(It⁢(ei)-ILB))⁢if⁢i∈ζM,eiwi·(1-(It⁢(ei)-ILB))⁢if⁢i∉ζM,ei,(Equation⁢9)
where ζM,eirepresents the set of corresponding motivations of the emotion e, according to the correspondence between emotions and motivations shown by table600inFIG.6.

The foregoing model for simulating human-like affect-driven behavior will be further described by reference toFIG.7, which shows flowchart790presenting an exemplary method for enabling virtual agent150/350to simulate a human interaction, according to one implementation. With respect to the method outlined inFIG.7, it is noted that certain details and features have been left out of flowchart790in order not to obscure the discussion of the inventive features in the present application. It is noted that as used hereinafter in the present disclosure and the claims appended thereto, the term “guest” may refer to guest126a, guest126b, a group of human guests, e.g., guests126aand126bsharing a common location, or to guest object(s)148.

Referring toFIG.7in combination withFIGS.1,2, and3, flowchart790begins with identifying character146a/146bassumed by virtual agent150/350(action791). Identification of character146a/146bincludes identification of the affective qualities of character146a/146b, such as personality360, target state368of motivational fulfillment, baseline mood362, and emotions356of character146a/146b. Identification of character146a/146bmay be performed by software code110, executed by hardware processor104, by reference to story and character library112and based on character146a/146bassumed by virtual agent150/350. For example, story and character library112may include character profiles for each character assumable by virtual agent150/350and including information describing the personality360, target state368of motivational fulfillment, baseline mood362, and emotions356of each character.

Flowchart790continues with identifying current physical state354, current state366of motivational fulfillment, and currently active emotions364of character146a/146b(action792). Identification of current physical state354, current state366of motivational fulfillment, and currently active emotions364of character146a/146bmay be performed by virtual agent150/350, under the control of software code110executed by hardware processor104.

For example, and as noted above, identification of current physical state354can be performed by tracking variables that describe the physical condition of character146a/146b, such as tiredness, depth-of-sleep, hunger, and the like. Identification of current state366of motivational fulfillment of character146a/146bmay be performed based on Equation 1, described above, and may further based on the modification to the motivation weight vector370, w→w′, introduced by Equation 9, also described above. Identification of currently active emotions364of character146a/146bmay be performed based on Equation 2 and/or Equation 6 described above.

Flowchart790continues with determining current mood372of character146a/146bbased on baseline mood362and currently active emotions364of character146a/146b(action793). Determination of current mood372of character146a/146bmay be performed by virtual agent150/350, under the control of software code110executed by hardware processor104. For example, current mood372of character146a/146bmay be determined based on Equation 3, described above.

Flowchart790continues with receiving an input corresponding to an interaction or an event experienced by character146a/146b(action794). In some implementations, the input received in action794may be detection data indicating the presence of guest126a/126bor guest object(s)148. As noted above, input module130/230may include one or more guest sensors236, such as RFID sensor236a, FR sensor236b, ASR sensor236c, OR sensor236d, and/or guest response sensor236e. As a result, guest126a/126bor guest object(s)148may be detected based on detection data in the form of sensor data produced by one or more of guest sensors236. In addition, or alternatively, in some implementations input module130/230may include microphone(s)238. In those latter implementations, guest126a/126bor guest object(s)148may be detected based on speech of guest126a/126bor guest object(s)148received by microphone (238).

For example, in some implementations, guest object(s)148may be an inanimate or non-autonomous object, such as a coffee cup. In those implementations, guest object(s)148may be detected using RFID sensor236aor OR sensor236d, for example. In other implementations, guest object(s)148may be one or more other characters, such as one or more characters in the form of a robot of other machine and/or one or more virtual characters rendered on a display screen. In those implementations, guest object(s)148may be detected using one, some, or all of RFID sensor236a, FR sensor236b, ASR sensor236c, OR sensor236d, and/or guest response sensor236e. Moreover, in implementations in which guest object(s)148is/are capable of generating speech, guest object(s)148may be detected based on speech of guest object(s)148received by microphone(s)238.

In implementations in which virtual character146binteracts with guest126b, detection of guest126bmay be performed based on one or more inputs to guest system140. For example, guest126bmay be detected based on one or more inputs to keyboard144or display screen142by guest126b. Receiving the input in action794, may be performed by software code110, executed by hardware processor104, and using input module130/230.

It is noted that, in some implementations, action794may include identifying guest126a/126bor guest object(s)148. As discussed above, the presence of guest126a/126bor guest object(s)148can be detected based on sensor data received from input module130/230. That sensor data may also be used to reference interaction history database108to identify guest126a/126bor guest object(s)148. Thus, identification of guest126a/126bor guest object(s)148may be performed by software code110, executed by hardware processor104, and using input module130/230and interaction history database108.

It is noted that virtual agent150/350may make preliminary determinations regarding identification of human guest126a/126bbased on data retained from previous interactions, such as the day of the week, time of day, weather conditions, or other contextual cues, for example. In addition, or alternatively, human guest126a/126bmay carry a unique identifier, such as an RFID tag worn as a pin or bracelet and enabling virtual agent150/350to distinguish human guest126a/126bfrom other humans.

In some implementations, action794also includes obtaining the interaction history of character146a/146bwith guest126a/126bor guest object(s)148(action685). The interaction history of character146a/146bwith guest126a/126bor guest object(s)148may be obtained from interaction history database108by software code110, executed by hardware processor104. It is noted that although virtual agent150/350and/or interaction history database108may retain data enabling the virtual agent150/350to “identify” human guest126a/126bwith whom virtual agent150/350interacts, the data retained is exclusive of personally identifiable information (PII) of human guest126a/126b. Thus, although virtual agent150/350is typically able to distinguish one anonymous human guest with whom a previous character interaction has occurred from another, as well as from anonymous human guests having no previous interaction experience with the character, the present simulated human interaction solutions do not retain information describing the age, gender, race, ethnicity, or any other PII of human guests126a/126bwith whom virtual agent150/350interacts. In other words, virtual agent150/350may, in effect, be able to “identify” guest126a/126bas distinguishable from other guests, while the real-world identity or other PII of human guest126a/126bremains unknown to system100.

Flowchart790continues with planning multiple behaviors including at least a first behavior, a second behavior, and a third behavior for character146a/146b(action795). The first behavior may correspond to hardcoded reaction355, and may be based on the input received in action794and the current physical state of character146a/146b. The second behavior may correspond to an emotional reaction by character146a/146band may be based on the input received in action794, as well as personality360, current mood372, and active emotions364of character146a/146b. The second behavior may be expressed as emotional reaction plan πGP, described above, and generated by emotional reaction planner374.

By contrast, the third behavior may be a motivationally influenced behavior based on the difference between target state368of motivational fulfillment and current state366of motivational fulfillment. The third behavior may be expressed as motivationally inspired plan πA*, described above, and generated by heuristic-based forward planner376. Planning of the multiple behaviors for character146a/146bin action795may be performed by virtual agent150/350, under the control of software code110executed by hardware processor104.

The first, and/or second, and/or third behaviors for character146a/146bplanned in action795may include an interaction with human guests126a/126bor guest object148in the form of another character. In those implementations, the first, and/or second, and/or third behavior for character146a/146bmay be one or more of a language-based communication, such as speech or written text, a body movement such as a gesture, and a facial expression.

As noted above, in some implementations, character146amay be a machine, such as a robot, for example. In those implementations, the first, and/or second, and/or third behavior planned for character146amay be an interaction with a virtual object or other real object, modeled in the story world. As also noted above, in some implementations, character146bmay be a virtual character rendered on display screen142. In those implementations, the first, and/or second, and/or third behavior planned for character146bmay be an interaction with a virtual object. However, it is noted that when rendered as a virtual character on display screen142, an interaction by character146bwith a virtual object may affect the real world. For example, character146brendered as a virtual character on display screen142may press a virtual button that results in a real world machine, such as a coffee maker for instance, being turned on or off.

Flowchart790can conclude with rendering one of the multiple behaviors planned for character146a/146b(action796). For example, hardware processor104may execute software code110to render one of the behaviors planned in action795via action scheduler380of virtual agent150/350and output module124. It is noted that, in some implementations, the multiple behaviors planned in action795may be sent to behavior manager378configured to schedule the behaviors with respect to one or more of a priority associated respectively with the behaviors, conflicts amongst the behaviors, and/or conflicts with a behavior plan currently being executed. Alternatively, or in addition, behavior manager378may utilize another type of behavior selection strategy, such as optimizing progress towards the achievement of short term or long term goals of character146a/146b.

In some implementations, as noted above, the behaviors planned in action795may include a language-based communication by character146a/146b. In those implementations, output module124may provide data enabling the rendering of text on display screen142, or enabling speech by an audio output device integrated with character146ain the form of a robot or other type of machine. According to some implementations, the behaviors planned in action795may include a facial expression, gesture, or other movement by character146a/146b. In those implementations, hardware processor104may execute software code110to cause character146a/146bto perform the behavior.

Moreover, in some implementations, hardware processor104may execute software code110to learn from the behavior rendered in action796in order to improve the performance of virtual agent150/350. For example, in one implementation, hardware processor104may execute software code110to detect a response by guest126a/126bor guest object(s)148to the behavior rendered in action796, via guest response sensor236e, for example, and to generate an updated present status of character146a/146band an updated interaction history of character146a/146bwith guest126a/126bor guest object(s)148.

From the above description it is manifest that various techniques can be used for implementing the concepts described in the present application without departing from the scope of those concepts. Moreover, while the concepts have been described with specific reference to certain implementations, a person of ordinary skill in the art would recognize that changes can be made in form and detail without departing from the scope of those concepts. As such, the described implementations are to be considered in all respects as illustrative and not restrictive. It should also be understood that the present application is not limited to the particular implementations described herein, but many rearrangements, modifications, and substitutions are possible without departing from the scope of the present disclosure.